Harvesting metadata between PDH.stat indicator database and Pacific Data Hub

Update from Wednesday, October 30, 2019

A harvester has been developed to "pull" metadata from the PDH.stat indicator database across to the Pacific Data Hub dataset catalogue to improve discoverability of these datasets.

The harvester enables Pacific Data Hub users to easily "discover" indicators/datasets which are contained within the PDH.stat database.

Here is what a collection of datasets look like in PDH.stat:

 

And the same datasets being listed in the Pacific Data Hub:

And when a listing is opened up:

 

Metadata is harvested once every 24h to check for changes to the PDH.stat database.

 

About the project

PDH.stat

Pacific Data Hub .Stat Data Explorer
Status

PDH.stat is a re-branded version of “.Stat Suite”, the SDMX-based statistical indicator platform built by the Organisation for Economic Co-operation and Development (OECD) through the Statistical Information System Collaboration Community. This platform will replace, and expand on the very popular, yet outdated National Minimum Development Indicator (NMDI) database which was established to report on Pacific development indicators including the Millennium Development Goals (MDGs).

NMDI-image

PDH.stat will be hosted under the soon to be released Pacific Data Hub (PDH). The Pacific Data Hub architecture is designed to accommodate data or links to data from relevant governments, Non-governmental organisations, Intergovernmental organisations, academic institutes, think-tanks and other organisations with data and interests in the Pacific. New regional initiatives established by partners will have the option of publishing their data through the PDH instead of standing up their own bespoke portals, hubs or websites. The data will be made be accessible through either open data licensing agreements, or where appropriate confidential or restricted data agreements.

What is SDMX?

SDMX, which stands for Statistical Data and Metadata eXchange is an international initiative that aims at standardising and modernising (“industrialising”) the mechanisms and processes for the exchange of statistical data and metadata among international organisations and their member countries.

Features of PDH.stat architecture:

  • stores stuctured SDMX datasets,
  • thorough metadata on datasets and individual indicators,
  • powerful API allows machine-to-machine access to all datasets in the database enabling development partners, researchers and other, users to programatically extract information.

Features of PDH.stat explorer/browser interface:

  • find datasets with free-text search, fine-tune search results through context-specific filters (facets) for topics and relevant data dimensions, and download an entire dataset,
  • preview and download multi-dimensional tables and charts, and
  • share created tables and charts in blogs and social media, or embed them dynamically in web pages.