From the foregoing chapters the very great need of better observational data clearly emerged. In this chapter we consider the observational problem in some more detail, with emphasis on the observations which are essential for the determination of temporal and spatial patterns of climate variability. Other observations, important for a better understanding of climate processes will not be discussed again.

The general strategy is simple. Existing data sources containing information about the climate of the past must be made (better) accessible. There is also a need to supplement these data with new palaeoclimatological observations. Furthermore, it is essential that monitoring of our climate system continues and is extended, in order to know what is going on, and to build up data sets for future analysis.

Basic variables are temperature, humidity, winds and clouds. All of these must be measured as a function of height or pressure. The first three of these variables are observed from balloon-lofted instrument packages (radiosondes) released at weather observing sites on land and from ships. Satellites provide vertical profiles of temperature and humidity and measurements of winds and clouds under special conditions.

Temperature, precipitation, pressure, humidity and winds are observed on the land surface with instruments located at manned and automatic stations, and at sea with instruments located on ships and moored platforms. Winds at sea are obtainable from satellite observations. Precipitation is observed on land using a variety of techniques including gauges and radar. At sea precipitation is observed on ships, but emerging satellite techniques are also showing promise. Land surface conditions including hydrological variables, such as snow cover, soil moisture, river runoff, etc., are observed routinely at many manned and automatic stations.

Additional essential variables for the study of climate change are related to the radiance balance and the composition of the atmosphere. Total solar irradiance and outgoing short- and long-wave radiation at the top of the atmosphere can be measured from satellites. Greenhouse gas concentrations (water vapour, carbon dioxide, methane, sulphur and nitrous oxide, ozone and CFCs) and the concentrations of different aerosols (industrial sulphur containing, volcanic, soot, etc) represent climate forcings and need to be monitored. Large-scale land-usage change, representing another class of climate forcings, can be monitored with satellites of the Landsat class.

It is useful to distinguish between surface, upper-ocean and deep-ocean observations. Important surface variables are sea surface temperature (SST), sea surface salinity (SSS), ocean waves, sea level, surface currents, sea ice, surface winds and surface fluxes of momentum, heat and fresh water. In the upper ocean temperature, salinity and currents need to be measured. In the deep ocean the main interest is in budgets and in transports of heat and mass.

An adequate measurement of all of these variables requires a network consisting of profiling floats, tide gauge stations, moored buoys, voluntary observing ships (VOS) and drifting buoys, supplemented with satellites and deep hydrographic sections from research vessels. The role of voluntary observing ships could be greatly enhanced by the introduction of new or higher accuracy equipment (flux measurement packages, acoustic Doppler current meters, thermosalinographs and XCTDs.)

Satellites are particularly suited to measure SST (AVHRR on board NOAA/TIROS and EUMETSAT/METOP), surface winds (ADEOS/NSCAT, QSCAT, ERS-1/2 and METOP), altimetry (TOPEX/POSEIDON, ERS, ENVISAT, JASON) to be hopefully complemented with a gravity mission (GOCE), and sea-ice (SSMI, AVHRR). Radiative and latent heat fluxes at the surface of the ocean are also key parameters which can to some extend be inferred from space observations.

The collection of observations of the atmosphere and the ocean is not an exclusive CLIVAR interest. Therefore, CLIVAR will build on and collaborate with operational meteorological services and other programmes, such as the World Weather Watch (WWW), the Global Climate Observing System (GCOS), the Global Ocean Observing System (GOOS) and the Global Terrestrial Observing System (GTOS). The CLIVAR Implementation Plan gives an overview of the basic system that is already in place and the enhancements that are required by CLIVAR. The following general recommendations can be made:

• Maintain the current level of sustained observations for the atmosphere.
• Develop and improve routine observations for oceans.
• For specific scientific issues, extend present observations in space and time.

Detailed recommendations for a European contribution to an observational network in the Atlantic have been made already in chapter 4. They comprise:

• An upper ocean observational network,
• Intermediate and deep ocean observations,
• Monitoring of the Arctic freshwater export.

The implementation of this observational network needs to be done in a multinational effort, jointly by groups from both sides of the Atlantic, and in collaboration with (Euro)GOOS. An overview panel needs to be established. We recommend that such a panel is set up by CLIVAR.

Teleconnection studies and studies of tropical variability benefit greatly from the observational system in the Pacific (tide gauges, ships, the TAO array, satellite observations of SST and sea level). These surface and upper ocean observations should be extended into the Indian and tropical Atlantic Ocean. PIRATA will play a crucial role here. Scale interaction studies will require high quality observations of clouds and convective activity. With regards to the Asian Summer Monsoon there is a need to identify key regions that influence the monsoon variability and to set up a monitoring system in these areas in order to enhance the capability of operational forecasting systems. The surface and upper air observation system of Africa needs to be improved.

Detection of climate change will depend heavily on GCOS. Several recommendations have been made, namely 1. the necessity of temporal overlap of satellite missions; 2. the need for space-borne solar irradiance measurements; 3. the addition of several new measurements, including the measurement of atmospheric sulphur.

Data assimilation in ocean models (ODA) is relatively new, but it has proven its usefulness for climate variability and predictability research. Therefore, some priority should be given to further developing, testing and implementing data assimilation methods for ocean models. The most immediate benefit is found in the study of interannual variability in the tropical ocean, but ocean data assimilation also has potential for the longer time scales and for the extratropics. From the outset it should be clear that there is no uniform approach to ocean data assimilation. It will be a challenge to find the best approach for each given research objective. Ocean data assimilation has many other applications than climate variability research. Therefore, where useful, collaboration with other interest groups should be sought.

There are ongoing activities on which a European ODA contribution to CLIVAR can build. Both the UK Met Office and ECMWF routinely assimilate upper-ocean observations into their ocean models which are then used for seasonal forecasting with a coupled ocean/atmosphere system. In France, the MERCATOR project aims at the assimilation of ocean observations in a 1/10 degree ocean model. Similar assimilations are planned by the UK Met Office in their FOAM model. On a European level the AGORA project will contribute to the development of several new methods and will carry out several ocean reanalyses. On the international scene, two important activities should be mentioned. One is WOCE-AIMS, which is set up in order to make a synthesis of all WOCE field data. A second initiative, the Global Ocean Data Assimilation Experiment (GODAE) is to demonstrate the practicality and feasibility of routine real-time global ocean data assimilation and prediction.

• It is important that assessments of the impact of the upper-ocean assimilation on the forecast quality be continued.
• Data assimilation systems should be made flexible so that in principle the wide variety of available data streams can be assimilated, such as XBT, CTD, buoys, autonomous profiling floats, drifters, current meters, SST, altimeters, tomography, etc, both in the tropics and in the extratropics. The impact of assimilation of additional observations should be investigated.
• Attention should be given to the problem of error estimates: both model and instrumental errors should be quantified. Special attention should go to the problem of biases. Users of ocean analyses should be made aware of the information content of the analysis products.
• The benefit of using advanced assimilation methods should be further investigated. Such methods (the adjoint approach or the ensemble Kalman filter approach, for example) take estimates of prior and predicted error statistics properly into account. They may be superior in providing a consistent analysis and posterior error estimate, and allow for testing assumptions about prior error statistics.
• Ocean reanalyses may be useful. However, in practice, groups make frequent reanalyses already while developing their systems. This is possible because of the large time scales and the limited number of data, as compared to atmospheric data assimilation.
• For oceanographic applications we suggest a high-resolution atmospheric reanalysis of the 90s when both altimeter and scatterometer data are available. This could be done with a high-resolution version of the 40-year reanalysis (section 7.4). Such a reanalysis should provide surface momentum fluxes with enough energy at small horizontal scales.
• A focused intercomparison of different assimilation systems based on common observational data sets is recommended.

The use of ocean data assimilation for DecCen and anthropogenic climate change studies is much less clear. A primary objective of WOCE-AIMS is the reconstruction of the mean state of the ocean. This in itself does not give information about variability. Nevertheless, as part of WOCE-AIMS and GODAE valuable information about the variability of the upper ocean on decadal time scales will become available, and the road towards decadal prediction will paved. The importance of resolution in model data assimilation remains an issue. The GODAE approach calls for both high-resolution for estimates of the high-frequency/wavenumber variability, and low resolution for longer-period climate estimates, for which the contribution of many different data types is important. For DecCen and longer time scales, the European focus on the Atlantic is logical. However, the possibility of global oceanic teleconnections at these time scales should not be ruled out. for the study of these teleconnections global ocean models and global ocean observations are essential. The long time and large space scales of DecCen, and the full coupling of the climate system, and the inherent advantages of the ocean (increased signal to noise) should encourage ODA activities in these areas, at least for analysing variability and patterns, though probably not trends. Assimilation in regional eddy-resolving models can be useful for the improvement of parametrisations in larger scale models.

In summary the following overall recommendation is made:

• Ocean state estimation, possibly including reanalyses, should be used as a basis for studying and understanding climate variability and predictability, exploiting the global CLIVAR sustained observing system. Increased focus is required on data quality, data set homogeneity and (model and data) bias in order for ocean analyses to be useful also at long time scales.

Global three-dimensional data for the atmosphere exist essentially only for the last 18 years as a high quality data set, and soon for another 20 years at lower quality. For the oceans, data are mainly restricted to the upper layer. Data for other aspects of the climate system: deep ocean, atmospheric chemistry, biochemical cycles, sea ice and land ice are still very patchy. Surface data for temperature and pressure can at best be broadly constructed for most regions back to the late 19th century. In some regions the data began only 40 years ago and in a few areas they have hardly ever existed at all. Before that time we have essentially only indirect or proxy observations, which can be interpreted into geophysical quantities by a variety of techniques. The use of proxies for climate research is a subject specifically addressed by IGBP/PAGES.

Long-period homogeneous global analyses of the three-dimensional structure of the atmosphere will contribute to a number of CLIVAR objectives. Important issues are: better analysis, understanding and prediction of climate processes, accelerated development of seasonal prediction systems, improved understanding of climate change, studies of seasonal, interannual and, if the analyses are sufficiently long, decadal variability. The practical way to produce such analyses is to use the global data assimilation systems that are employed to produce initial conditions for operational weather prediction. With current procedures such analyses could be produced for a 40-year period, starting from 1958, the International Geophysical Year, or perhaps somewhat earlier, to the present day. In addition to the basic observations that are contained in the European and North American archives, further data, which may be in other national archives or the DARE collection, are required. During this 40-year period the global observing system has evolved considerably. Observations made from surface-based systems have improved and remotely-sensed data from satellite-borne instruments became available from the early 1970s. So, in order for the resulting analyses to be useful for quantitative climate studies, considerable efforts should be made to assess the impact of the heterogeneity of the observing system over the 40-year period. Where possible, the observations should be adjusted to remove heterogeneity and bias. It is expected that extensive studies of the effects of progressive changes in the radiosonde network and satellite systems on the analyses will be required.

• We recommend that a 40-year reanalysis be carried out within the next 5 years by Europe to complement and improve existing efforts in the USA. The observational data for such a major undertaking are available in European, USA and some other archives. During this 40-year period, however, the global observing system has evolved considerably and considerable efforts should be made to assess the impact of the heterogeneity of the observing system.
• We also recommend that a limited-area high-resolution reanalysis be carried for much of Europe after the 40-year reanalysis has been completed. This is expected to provide dynamically consistent statistics of the distributions and extremes of parameters like rainfall on resolutions comparable with the topography of Europe.

Surface measurements of climate parameters are available for considerably longer periods than the full three-dimensional description of the climate system. The fastest progress in improving these data is likely to come through their widespread availability and analysis. Thus a close interaction between users and developers of the datasets is essential.

Although monthly fields are sufficient for most CLIVAR-related activities, a worldwide set of century-long daily time-series of temperature, precipitation and pressure would enable changing frequencies of extremes in day-to-day weather variability to be studied. The GCOS Surface Network would provide the basis for the most appropriate choice of stations. Europe has the longest instrumental climate records of any region of the world, so the natural variability of climate could be assessed for at least 100 years before any anthropogenic influences can be expected to have occurred. An EU project, soon to start, will develop series for seven stations from the 18th century.

• Global scale fields of surface temperature over land and ocean, sea-level pressure and precipitation can be developed back to about 1870 on a monthly grid, employing a number of recently developed techniques for filling poorly covered regions in both space and time. The degree to which such fields can be filled is both time- and region-dependent. The techniques can also be used to estimate the sampling errors of the analyses.
• Over Europe, gridded analyses could be extended back to about 1750 using the many longer records available from this continent. Thus, a gridded monthly sea level pressure data set for 35 -70 N and 30 W - 40 E back to 1780 has recently been developed using regression techniques. We recommend that gridded monthly analyses of temperature, precipitation and sea level pressure be created back to 1750, with the assistance of ECSN.
• A more directed search for additional data to extend some of the regularly used regional circulation indices (e.g. SOI, NAO, and the Southern Hemisphere Trans-Polar Index (TPI)) would enable recent unusual behavior to be placed in a long term context. In the European region there is potential to extend the NAO back to about 1780 using instrumental data. We also recommend that such indices be extended as far back through the last millennium as possible using palaeoclimatic data. It is also important to assess the variability of relationships between such circulation indices and surface climate over as long a period as possible.
• Many of the basic data required to achieve the above objectives are currently available but there are a number of additional data sources which could lead to significant improvements. Effort should be concentrated on times and regions where coverage is known to be poor. There are large numbers of ship based observations (SST, air temperature, pressure and wind) for periods before 1950 and especially before 1920, available in Europe and elsewhere. Their digitalisation would greatly improve analyses of these parameters. The benefits would be most apparent during the 1910s and 1940s globally and over the Pacific Ocean throughout the period. A joint USA/UK project is planned for 1998/99, but much more work is needed. A WMO funded project (DARE) has microfilmed much African climate data for the 1930-1980 period. The data are held at the Royal Meteorological Institute of Belgium but are currently inaccessible to CLIVAR. We recommend that these data be catalogued and selectively digitised. This would greatly improve CLIVAR activities in the region.
• To extend the above analyses, we recommend that ECSN make available appropriate daily data from national meteorological agencies, as far into the past as possible.

Useful as instrumental studies may be, they are limited by the available data record, which generally extends back less than 150 years. This duration is too short to reconstruct past climatic parameters and spatial patterns over the full range of variability likely to be present. Climates from before the recent instrumental period must be deduced from proxy (non-instrumental) records. These include tree rings, deep-sea sediments, corals and ice cores and involve isotopic, geochemical and micropalaeontological analysis.

In order to facilitate climate research it is essential to provide easy access to the relevant data. There is a need to seek out the relevant climate data sets and to encourage the deposition of data in the public domain. This will involve collation, digitisation, transcription, homogenisation, standardisation and quality control. This is usually best carried out by those experts who have collected or assembled the data set. In many cases, the data will also be archived and distributed by those centres of expertise. These experts will form a distributed network of data providers for climate research.

There will also be the need to provide support to the research community in locating and using the data through advertisement of data sets, provision of catalogue search facilities, documentation and software for data extraction and manipulation. Within CLIVAR, there is therefore an important role for "Gateway Data Centres" who have specific expertise in the area of cataloguing, distribution and data manipulation software, associated with each of the types of data required for climate research e.g. surface data, radiosonde data, marine data, palaeoclimatic data, three-dimensional analyses, climate model data, etc. They would act as a gateway to information and data held at other sites, providing the capability for users to identify the existence, location and accessibility of a relevant climate data set, with sufficient information to determine whether data held at a particular site meets their requirements in terms of content, coverage and quality. They should also provide archive, distribution and data manipulation facilities for data providers and users who do not have adequate capabilities themselves.

These Gateway Data Centres should be based at existing facilities which are already established and experienced in these activities. In Europe, such data centres already exist, for example the mirror site of the World Data Centre-A for palaeoclimatology at Toulouse, the British Atmospheric Data Centre (BADC) for instrumental data and the German Climate Computing Centre (DKRZ) for three-dimensional analyses and model data.

The CLIVAR programme will produce many data sets that are of lasting value beyond the lifetime of the programme. A permanent archive strategy will need to be developed in order to ensure long-term data survival. This is likely to involve lodging selected data sets not only at permanent data archives but also the possibility of publishing certain data sets on CD-ROMs or their future equivalent. There will also be an important role, therefore, for CLIVAR Gateway Data Centres in the formulation of the long-term data archive strategy.

In the establishment of this infrastructure of designated Gateway Data Centres, serving a distributed network of data providers, it will be essential to maintain close collaboration and cooperation with related international and European programmes so that there may be full inter-operability between the data systems. For example, GCOS has already developed a Data and Information Management Plan (GCOS-13, WMO/TD - No. 677) with detailed recommendations. More recent developments of the GCOS data plan have involved collaboration with GOOS and GTOS and coordination with WMO, WCRP, IGBP and CEOS. The recommendations outlined below for the CLIVAR programme are compatible with the GCOS plan and with current practices and plans of the European Climate Support Network (ECSN) and the European Environment Agency (EEA). The proposed distributed data network is also similar to other European data initiatives in related science areas such as the CEO (Centre for Earth Observation) and EWSE (European Wide Service Exchange).

• Access to climate data sets should be based upon a two-tier system with 1. a number of designated CLIVAR Gateway Data Centres serving 2. a distributed network of data providers who are affiliated with the CLIVAR programme.
• Data sets should preferably be made available by the experts who have collected or collated them. These are the data providers.
• Data providers should be encouraged to place their data sets in the public domain wherever possible, preferably through on-line access over the internet. Where restricted access to bona fide researchers is necessary, this can be achieved with appropriate interface software.
• Existing Data Centres should be strengthened to help meet the needs of CLIVAR and therefore perform the role of the Gateway Data Centres. This should be done in close collaboration with the International CLIVAR Project Office.
• The Gateway Data Centres will provide advice and assistance where appropriate on the archiving, cataloguing and distribution of data sets, for example, the development of guidelines for data exchange via WWW.
• The Gateway Data Centres should provide links to useful WWW climate sites and develop a catalogue of affiliated CLIVAR data. This should be available for searching through the CLIVAR WWW pages. It could be populated initially by entries describing data held at the Gateway Data Centres and, in the longer term, by entries from all CLIVAR-affiliated data providers.
• Data providers should not be required to use common formats, software or database systems. However, a set of recommended formats should be developed by the Gateway Data Centres in consultation with the data providers, who should be encouraged to adopt one of these wherever possible.
• Data of long-term value should be converted to one of the recommended formats wherever possible. Gateway Data Centres should provide assistance with this as required.
• Data providers should be required to provide an adequate level of documentation and catalogue information and should be encouraged to supply data extraction and manipulation software. This "metadata" should also be available over the internet e.g. through the WWW.
• There is also a role for the Gateway Data Centres in recommending and/or developing software for data providers to use with their climate data and other products, such as animations and real-time derivation of additional data products. For example, this could involve the development of a "toolkit" of re-usable client and server utilities including Common Gateway Interface (CGI) wrappers and Java applets to allow legacy software to be run from a web interface and also Harvest technology to search the web for specified topics.
• There is a need for a long-term data archiving strategy to ensure that data sets of long-term value are not lost. The designated Gateway Data Centres will have the relevant expertise to contribute to the formulation of this.

Start of main document