Satellite Data Brokerage

June 3, 2016

earthsat_hd_lrg

There are 64 operational satellites, managed by 32 agencies in 20 countries that are currently generating vast quantities of data relevant to CSST applications. In addition, there are many petabytes of data from previous satellite missions which are likely to have applications when data are not needed in real-time. Although data are made freely available by many international space agencies, entry-level barriers to accessing and using these data are high – data are not always provided through easily accessible means, there is a plethora of data formats that are often complex and/or obscure, data are provided in a range of coordinate systems and in different projections, and the utility of data for specific purposes is seldom articulated such that specialist knowledge is required.

To lower the bar of entry to using space-based measurements and resultant products and services, CSST will establish a national satellite-data brokerage. This facility will perform two primary services:

  1. Act as a broker of satellite data for all organisations in New Zealand with an interest in these data in much the same way as Pharmac acts as a broker for accessing pharmaceuticals on behalf of all of New Zealand. CSST will negotiate pricing and licensing for commercially acquired satellite data in an effort to lower costs for use of these data across New Zealand.
  2. Establish and maintain a data storage facility of space-based measurements for the greater New Zealand domain for the benefit of all New Zealand industries, educational institutions, research organisations, and national, regional and local bodies that could potentially make use of these data. The facility will host data sets obtained from international space agencies, relevant data sets and products obtained from collaborating New Zealand organisations, data from the CSST CubeSat programme, and the CSST-derived products built off those source data. Facilities for sharing data with regional industries that do not have access to fast internet would also be established (e.g. posting external hard drives).

 

The focus for this pillar of CSST operations is on facilitating access to data products that New Zealand’s regional industries, both with regard to necessary infrastructure and with regard to developing in-house skills and expertise in those regional industries capable of exploiting space-based measurements. In spite of the clear value that space-based measurements would bring to business-led R&D efforts, many organisations lack knowledge of that potential value.  Even when aware of that value, businesses often lack the necessary experience to access and use space-based measurements in their own activities or as a component in their own R&D. CSST will support business-led efforts by:

  • Educating relevant regional businesses around New Zealand on how to make use of space-based measurements (primarily imagery).
  • Working on behalf of potential users of these data to source data from the appropriate space agencies and provide the data in a format that meets their needs. This would likely include subsampling the data to the location or limited area of their interest and reformatting the data.
  • Creating a one-stop-shop for satellite data.

Pricing and licensing for data access from the CSST archive will allow businesses to experiment with the utility of space-based imagery without having to commit to the cost of purchasing large quantities of such imagery. Because the CSST satellite data archive will provide a one-stop shop for satellite data access in New Zealand, CSST will be able to form small user groups who share a common interest in a particular satellite data set. In addition to fostering the sharing of tools, data processing pipelines and experience with these data, such groups will also foster collaboration on the exploitation of these data for new purposes.

The data exchange will begin with a 2 petabyte data storage capability. The system will routinely assess the relevance of data to the archive, identifying which data are to be maintained and in what fashion e.g. the frequency of data use will determine the choice of storage mechanism between online storage and long-term archive.

Rather than developing significant in-house infrastructure to serve the data exchange, CSST will work with expert advisers to identify an appropriate industry partner to develop a solution that optimises value to New Zealand. The CSST data exchange will be connected to CRIs and Universities to allow rapid transfer of high data volumes around the country as well as connections to international research and education networks.