ADAGUC: a project to provide convenient access to the data treasures of KNMI

In its role as a National Data Centre the Royal Netherlands Meteorological Institute is hosting a true treasure in the form of many unique datasets.

This treasure grows every day but it is a common property of treasures that accessing them can be difficult. Historically the atmospheric, meteorological and seismological communities are separate worlds with their own data formats and tools for data handling making sharing of data difficult and cumbersome. On the other hand, KNMI data is becoming increasingly of interest to the outside world because of the continuously improving spatial and temporal resolution of e.g. model and satellite data and the interest in historical datasets.

Figure 1a. Monthly average of Sciamachy NO2 tropospheric columns over Europe visualized in Google Earth.
Figure 1a. Monthly average of Sciamachy NO2 tropospheric columns over Europe visualized in Google Earth.

New user communities have come into existence that use geographically based datasets from many different fields in a cross-fertilizing way. This development is supported by the progress made in Geographical Information System (GIS) software. Almost all KNMI datasets contain the geospatial dimension and could be offered to the outside world in a scientifically correct and GIS-friendly way making it less cumbersome to work with.

The ADAGUC project1) (Atmospheric Data Access for the Geospatial User Community) aims at reducing the need for users to invent their own converter and mapping tools. Selected space borne atmospheric datasets will be made accessible by GIS to allow easy data comparison, resampling, selection, manipulation and visualization. Representatives of the (inter)national user communities are strongly involved in the project to guarantee proper accommodation of the user requirements. Next to developing easy access to KNMI datasets for the outside world, the (international) operational meteorology and meteorological research can profit from the introduction of GIS technology (see e.g. COST 7192)). Within ADAGUC emphasis will be laid on interoperability and harmonization of data resources such that a ‘GIS -enabled’ user can work with these datasets. ADAGUC serves as a pilot for applying GIS-technology in the future to the seismological, meteorological and climatology datasets of KNMI.

The ADAGUC project will aim at delivering the following: Open Source conversion tools for conversion of selected atmospheric datasets into an Open Standard GIS format, publish atmospheric datasets in the previous mentioned GIS format, and a web service to demonstrate the usability of the above to the geospatial and atmospheric community. Dissemination of results is pursued by publications, workshops and international co-operation. The challenge is to create an environment that has the potential to become the next generation operational solution, dealing with international frameworks, standards, and cross domain end-users.

The world of GEO-ICT

In general (geo) data usage is accomplished by tools that are provided by third parties, for instance commercial companies or government institutes that see profit in or need for geospatial solutions. Other stakeholders are working on the standardization of data formats, metadata descriptions and services that can be used to interchange geospatial information. The most relevant developments in tooling, standards and services are discussed below.

Tools: Google Earth

Google Earth made the public aware of the importance of geospatial information. As a next step third parties, commercial and non-profit organizations, are exploring the possibilities to place their own products into the Google Earth system. A wealth of geo-information is becoming available via Google Earth, ranging from holiday pictures, hotels, forest fires, earthquakes and meteorological forecasts. An example (Figure 1a and 1b) with Sciamachy data can be found at the SRON website3). Google Earth uses the KML (Keyhole Markup Language) standard. This standard is under control of Google Earth and is turned into a propriety format thereby limiting the development of an open user community. Moreover, for scientific and professional applications Google Earth is of limited use despite its important role for GIS-awareness.

Figure 1b. Population density for the same area that clearly show a correlation with the NO2 distribution in figure 1a.
Figure 1b. Population density for the same area that clearly show a correlation with the NO2 distribution in figure 1a.

Tools: Geographical Information Systems

A well established set of GIS tools is a necessary framework to deal with different kinds of spatial data. Since the 1960’s GIS has evolved from digital cartography into geo-ICT and offers standardized ways to store, process, analyze and visualize spatial data. The current (commercial and free) GIS software cannot deal with the large data volumes and the temporal aspects of atmospheric data yet, although these issues are being addressed because of pressure from user interest groups. A good example of this is the support of NetCDF (Network Common Data Form) COARDS (Cooperative Ocean/atmosphere Research Data Service) and CF (Climate and Forecast Metadata Convention) in ArcGIS 9.2 (commercial GIS system from ESRI). NetCDF is a set of software libraries that enable the (technical) information interchange unambiguous across different computer platforms. COARDS and CF are metadata definitions (descriptions) to promote the exchange of information stored in NetCDF files by using a common standard. The incorporation of NetCDF in ArcGIS was promoted by the National Centre for Atmospheric Research4)(NCAR) and ESRIs Atmospheric Special Interest Group5).

Tools: ESRI ArcGIS

ESRI is the world leader in professional GIS software. ESRI provides solutions to deal with geospatial problems. Their software products serve the ‘earth bound’ communities like governments (e.g., cadastre like tasks), road and traffic organizations, petrochemical industries, soil and vegetation related institutes. ESRI’s next step is to fulfil the wishes of the atmospheric data users: importing of atmospheric data formats, satellite view transformations and time resolutions from centuries to seconds are all on the list of being investigated by ESRI for future releases (Figure 2).

Figure 2. Tropospheric NO2 distribution from Sciamachy combined with the orography of eastern China with major cities, in ESRI's ArcGIS.
Figure 2. Tropospheric NO2 distribution from Sciamachy combined with the orography of eastern China with major cities, in ESRI's ArcGIS.
Figure 3. Example of an OGC based web service displaying features of ozone measurements from GOME that are stored in a PostgreSQL spatial database.
Figure 3. Example of an OGC based web service displaying features of ozone measurements from GOME that are stored in a PostgreSQL spatial database.

Standards: The Open Geospatial Consortium (OGC)

The OGC6) deals with standardization of all geo related issues; their focus is on defining specifications by open working groups to create a common ground for implementations. In this way the interpretation of geographical information becomes identical for all OGC compliant software making it easier to work with datasets and exchange these. OGC is for the interoperability of GIS what W3C (World Wide Web consortium) is for internet technology.

Standards: Infrastructure for Spatial Information in the European Community (INSPIRE)

On 25 April 2007 the European Parliament adopted the INSPIRE directive that aims at establishing coordination between the users and providers of spatial information so that the information and knowledge from different sectors can be combined. The directive applies to spatial data held by or on behalf of public authorities and to the use of spatial data by public authorities in the performance of their public tasks. In the forthcoming years INSPIRE will define the standards on data, metadata and services to which certain KNMI datasets have to comply. Sometimes these standards can be conflicting with standards of other international frameworks like the WMO Information System (WIS). The guidelines of all these standardizing bodies requires an overall vision and adequate participation to evaluate the potential impacts for KNMI.

Standards: Metadata, ISO19100 series

Optimal search and order capabilities rely on the use of homogeneous and uniform descriptions of metadata. ISO7) has developed a series of standards (the 19100 series) for defining, describing and managing geographic information that includes a standard for metadata. This standard has been adapted by organisations like the WMO and the OGC. INPSPIRE recommends the use of this standard for metadata. In the Netherlands GeoNovem8), has defined the Dutch Metadata Standard for Geography based on the ISO standards. Within KNMI the ISO 19100 series has for the first time been used for the KNMI Operational Data Center9) (KODAC) datasets metadata descriptions (ISO 19115). Currently the KODAC metadata is also compliant with Dutch Metadata Standard for Geography and available in both Dutch and English.

Standards: GIS compliant (meta-) data formats

The exchange of information is more then moving data from A to B. One has to be sure that the data is interpreted in a correct manner by all users and that the interpretation is independent of the user community. Data formats like NetCDF and HDF (Hierarchical Data Format, developed by the National Center for Supercomputing Applications) are self-explanatory in terms of a file’s content, which is in sharp contrast with the older binary files. The latter suffer from the fact that unfamiliar (remote or new) users cannot easily relate content and description, rendering them useless unless additional information is available. A similar aspect applies to the meta-data of a data collection. When meta-data descriptions are not uniform in terms of content and format, searching and ordering of data is cumbersome and often results in incomplete and erroneous results.

Services: Service Oriented Architecture (SOA)

Service Oriented Architecture is a concept in which many ‘small’, loosely coupled, distributed services interact to create a larger infrastructural service that is better maintainable and scalable: A change in one service has low impact on the overall service / production chain. SOA focuses on the interfaces and not on the implementation inside a service which is the responsibility of the owner. SOA is the opposite of the traditional monolithic architectures in which scalability and additional functionality can have considerable (cost) impact. An example of SOA in practice is the KODAC service that has completed its second phase by disclosing extra KNMI datasets via additional services.

The ADAGUC User Communities

The first ADAGUC workshop was organized at KNMI on 3-4 October 2006. Four distinct user communities, based on their specific needs for reliable and flexible access to atmospheric data, were identified:

Policy Makers are interested in long term averaged/forecast products and maps that are archived for easy access. Many (inter)national stakeholders provide atmospheric data to their governments to develop policies with respect to the social and political impact of i.e. climate change.
Earth System Scientists and Atmospheric Scientists are interested in data of the highest possible quality and resolution, both archived and real-time. The atmospheric users are particularly interested in aerosols, clouds and trace gases like NO2 and SO2.
GIS Users are interested in high quality and averaged data, including archived datasets. GIS users are characterized as being non-experts in the field of atmospheric research; their focus is more on earthbound features and earth-atmosphere interactions.
Risk assessment community is interested in mapped data, preferably in (near-) real time. These users are interested in products at an urban scale that are focused on industrial calamities, like gas leakage or chemical fires. The combination of inter-comparable data sources provides vital input to crisis management teams.

These communities are reflected in the five use cases defined by users in the ADAGUC team: 1) Air quality information based on satellite images; 2) Distribution maps of trace gases; 3) Relating remote sensing data with land use and vegetation data; 4) Improving river basin simulation models, and 5) Weather related disaster management. All these use cases are documented and made available to the communities and will be used as a starting point in identifying the specific user requirements. Implementation of the required services is planned in the next phase.


Being able to exchange information according to international standards is an important scientific, social and economic benefit for different communities. In the forthcoming years new standards for data exchange and data services will be legally imposed by means of the INSPIRE directive. The ADAGUC project is a pilot that aims at preparing KNMI, amongst others, for these upcoming changes. The project has a set of well-defined deliverables. Moreover KNMI becomes visible as a stakeholder in the GIS community and knowledge-sharing networks. The use of GIS also introduces a number of science-related challenges: mapping, gridding and projecting data must be done in such a way that the scientific integrity of the data is preserved. The ADAGUC project will identify these problems and seek for solutions.

ADAGUC consortium consists of the following partners: Royal Netherlands Meteorological Institute (KNMI, lead), Netherlands Institute for Space Research (SRON), Wageningen University and Research Centre (WUR), Vrije Universiteit Amsterdam (VU), Institute for Marine and Atmospheric research Utrecht (IMAU), European Space Agency (ESA), National Center for Atmospheric Research (NCAR), Harvard University, Institute of Environmental Physics Heidelberg (IUP), UNIDATA programme, Institute of Methodologies for Environmental Analysis (IMAA)

ADAGUC is funded by the Space for Geo-Information Programme (RGI) 

Niet gevonden wat u zocht? Zoek meer achtergrond artikelen