.. _harvesting-simpleurl-services:

Simple URL harvesting (opendata)

This harvester connects to a remote server via a simple URL to retrieve metadata records. This allows harvesting opendata catalogs such as opendatasoft, ESRI, DKAN and more.

Adding a simple URL harvester

- **Site** - Options about the remote site.

- *Name* - This is a short description of the remote site. It will be shown in the harvesting main page as the name for this instance of the harvester.

- *Service URL* - The URL of the server to be harvested. This can include pagination params like ``?start=0&rows=20``

- *loopElement* - Propery/element containing a list of the record entries. (Indicated as an absolute path from the document root.) eg. ``/datasets``

- *numberOfRecordPath* : Property indicating the total count of record entries. (Indicated as an absolute path from the document root.) eg. ``/nhits``

- *recordIdPath* : Property containing the record id. eg. ``datasetid``

- *pageFromParam* : Property indicating the first record item on the current "page" eg. ``start``

- *pageSizeParam* : Property indicating the number of records containned in the current "page" eg. ``rows``

- *toISOConversion* : Name of the conversion schema to use, which must be available as XSL on the GN instance. eg. ``OPENDATASOFT-to-ISO19115-3-2018``

Note: GN looks for schemas by name in https://github.com/geonetwork/core-geonetwork/tree/4.0.x/web/src/main/webapp/xsl/conversion/import. These schemas might internally include schemas from other locations like https://github.com/geonetwork/core-geonetwork/tree/4.0.x/schemas/iso19115-3.2018/src/main/plugin/iso19115-3.2018/convert. To indicate the ``fromJsonOpenDataSoft`` schema for example, from the latter location directly in the admin UI the following syntax can be used: ``schema:iso19115-3.2018:convert/fromJsonOpenDataSoft``.

**Sample configuration for opendatasoft**

- *loopElement* - ``/datasets``

- *numberOfRecordPath* : ``/nhits``

- *recordIdPath* : ``datasetid``

- *pageFromParam* : ``start``

- *pageSizeParam* : ``rows``

- *toISOConversion* : ``OPENDATASOFT-to-ISO19115-3-2018``

**Sample configuration for ESRI**

- *loopElement* - ``/dataset``

- *numberOfRecordPath* : ``/result/count``

- *recordIdPath* : ``landingPage``

- *pageFromParam* : ``start``

- *pageSizeParam* : ``rows``

- *toISOConversion* : ``ESRIDCAT-to-ISO19115-3-2018``

**Sample configuration for DKAN**

- *loopElement* - ``/result/0``

- *numberOfRecordPath* : ``/result/count``

- *recordIdPath* : ``id``

- *pageFromParam* : ``start``

- *pageSizeParam* : ``rows``

- *toISOConversion* : ``DKAN-to-ISO19115-3-2018``

- **Privileges** - Assign privileges to harvested metadata.