This page contains instructions on contributing “live” national data sets to the portal – called “data entity indexes”. The page is intended for national data providers taking part in the EMODnet Geology project (work package 9). Some EMODnet Geology data products are collected, harmonized and distributed manually and thus not described here.
If not part of the project, we encourage you to take a look at EMODnet Data Ingestion, that seeks to identify and to reach out to other potential providers in order to make their data sets also part of the total offer. It aims at streamlining the data ingestion process so that data holders from public and private sectors that are not yet connected to the existing marine data management infrastructures can easily release their data for safekeeping and subsequent distribution through EMODnet.
How to create a data entity index
We are creating three pan-European data entity indexes:
- Seismic surveys
- Multibeam backscatter surveys
Each data provider sharing these types of indexes must create local indexes for central harvesting via WFS.
First step involves gaining access to data. Index data is typically stored in a relational database like PostgreSQL or similar. In this case, a query is created to fetch the information necessary. Is your data stored in files, a manual process must be undertaken to reach the same result. Make sure it’s possible to programmatically repeat the data export on a regular basis. We plan to harvest updated data from data providers at least weekly, so being able to repeat the process is vital.
Next step involves harmonizing your data. Go to the EMODnet Geology portal’s intranet page and open the link “WP9 data index WFS registration form”. Study the table structures and associated vocabularies. Build formulas to convert your original data set into a valid EMODnet Entity Index. Since the data models are yet to be finalized, your first trial should contain just these four columns:
- A unique ID
The harvesting routines will evaluate each record for re-harvesting based on this ID.
- A geometry
We need a spatial geometry in order to display features on a map.
- A download link
The data provider must make additional data available for download at the link specified. For borehole data, this is lithology. For seismic and multibeam surveys, this is measurements – e.g. SEG-Y and TIFF. If the data provider requires registration prior to download, the link may point to a web page detailing the process to acquire the data – perhaps including a sample preview of the data. Zip files are allowed.
- A metadata link
This link points to the metadata web page describing the index created in the following step, so leave it blank for now.
We will later add more attributes to each index, but this suffice for data flow testing.
Next step involves setting up the service. Data providers must make each provided index accessible as layers in a WFS 2.0 service. Reusing existing service infrastructure is recommended whenever possible. Data providers are also allowed to create static files instead, if the file format follows the format of a WFS 2.0 GML3 GetFeature request. The file(s) must be available via HTTP or HTTPS (ordinary web page protocol). There are numerous free solutions available for setting up a WFS 2.0 service. We recommend GeoServer or Mapserver. If you need guidance, please contact WP9 lead (GEUS).
- Ressource title
- Ressource abstract
- Ressource type
- Ressource locator
- Unique ressource identifier
- Ressource language
- Topic category
- Geographic bounding box
- Temporal reference
- Spatial resolution
- Conditions for Access and use
- Limitations on public access
- Responsible organisation
- Metadata Point of contact
- Metadata date
- Metadata language
The data provider can choose to host index metadata on their own web server. The format should be either HTML or XML and a link to the metadata must be added to each index record. If the data provider would like to use an existing metadata database, consider using the EGDI Metadata Database.
The next step involves registering your data. Again, go to the EMODnet Geology portal‘s intranet page and open the link “WP9 data index WFS registration form”. Fill out your indexes on the first tab of the spreadsheet. Inform WP9 lead (GEUS) for QC.
When above is QC’ed by WP9 lead, await the final data model from WP9 lead and extend the attributes list accordingly.
This is the resulting data flow: