BBC2 SUPPORTED DATA FORMATS

The dataformat that will be used for the BBC2 campaign will be similar to the format used for the previous BBC-campaign. See the 4D-CLOUDS format page for more information.

However, we encourage the use of netCDF-format for any data but especially large volume data, like data from radars and lidars. Data stored as ASCII and netCDF files should be zipped (gzip). The (CLIWANET) data format is described below in two sections, one on the filenames and one on the file format. Data providers are encouraged to provide quicklooks together with the data. Quicklooks may be provided in any common graphics format like gif, jpg or png. However if you provide gif-formatted image files please use (preferably) only compressed gif files. Quicklooks to be displayed on the results page of this website should preferably be limited to 100 kB.

Presently we have not yet defined the format of the netCDF-files, but we try to stay as close as possible to the CloudNET-format which is mainly for radar and lidar data. For other datatypes you can use the Climate and Forecast metadata convention.

The proposed filename convention is: SS_IIIIIIII_YYMMDDNN.EXT
where:

SS	=	station/aircraft code (see table 1)
I..I	=	instrument, variable or model (see table 2 for an alphabetical list)
YY	=	year
MM	=	month
DD	=	day
NN	=	hour (UTC), number or alphanumerical code (NN may be extended to more than 2 characters)
EXT	=	extension, "DAT" for cliwanet formatted files and "NC" for netCDF formatted files resp.

An example: "CA_MRADMICC_03050100.DAT" is the MICCY microwave radiometer at Cabauw on May 1, 2003, starting at 0 hours UTC.

If you want to propose a new variable/abbreviation for this list, please send an email to Victor Venema and Henk Klein Baltink, so that all codes used in the BBC-campaign will be listed on the Bonn BBC2 pages and here.

A data file consists of three parts: a header, an optional line with y-axis or variables and a data segment. The header gives basic information on variables and some comments. A line with a y-axis or variables is included to make processing in some software packages easier. After the header there is the data section. The measurements are stored one row per time step. Columns contain time (in decimal of an hour), variable 1, variable 2, ...

At the moment there are two file formats in use: Format 2.3 and 3.0. Format 2.3 was made for 1 or 2 dimensional data. As for 4d-clouds, more dimensions were neccessary, so a new format was made: 3.0. If possible use the newest format, 3.0. Data providers, please adhere to the standard as close as possible, so that other people do not have to rewrite their reading routines when using your data. There is a lot of redundancy in the format, this can make the files bigger, but it helps to find errors and can make using the data easier for new users.

Explanations

'# HD LINES',x

number of header lines: total number of lines of the header (x=integer), including this line itself, but excluding the line with the y-axis or variables.

'FORMAT VERS',x.x

Header format version: version number of the file format, x.x is either 2.3 or 3.0.

'DATA VERS',x.x

Data version: number identifies updates of the data. The first version will be: 'x.x = 1.0'; minor updates of the data (e.g. new calibrations, small errors corrected, or time synchronized) update the last number: 1.1, 1.2. Significant changes (e.g. new processing methods) update the first number 2.0, 3.0.

'INS NAME','name'

Instrument name: in case of higher level products the instrument name can also be the tool(s) or the algorithm used, .

'LAT/LON',f1,f2

Latitude (f1=float) and longitude (f2) of the instrument, if not relevant or if changing (aircraft): 0, 0.

'ELEV',f,'unit'

Elevation of the instrument above the ground: f in "unit", if it is changing like for the aircraft: -9999 m. Please, give the height corresponding to the ground as height axis of the products or/and specify this relation in the comments.

'START',YYYY:MM:DD,HH:MM:SS

Starting date and time of the data, year (YYYY) in four numbers, month (MM), day (DD), hour (HH), minutes (MM) and seconds (SS) in two, time should always be in UTC.

'STOP',YYYY:MM:DD,HH:MM:SS

Ending date and time of the data, year (YYYY) in four numbers, month (MM), day (DD), hour (HH), minutes (MM) and seconds (SS) in two.

'DIM',x

Dimension of the data product, which also specifies the number of '* INFO' lines in the header. This is only used in Format version 3.0.

'DATA LINES',x

The number of lines (x) with data. This is only used in Format version 3.0.

'U INFO',f1,f2,f3,'name','unit',dt

These lines provide the axis information. The number of these lines is equal to the dimension given in DIM, and they are called U, V, W, X, Y, Z, A, B, ... INFO, respectively. f1, is the number of data bins, f2 the value of the starting bin, f3 the value of the last bin. If the bins are not equidistant, please, specify this in the comments. The first dimension should be time. Only this dimension has to specify a time shift (dt), which is the difference between the time the instrument indicated originally and the time used in this data synchronized with other instruments. The last dimension specifies the data bins of the y-axis. If no line with y-axis or variables is given after the header, then x,y,z,'name','unit' should be: 0.0,0.0,0.0,'null','null'. If there is a line with variables present the last dimension INFO should be 1.0,0.0,0.0,'null','null'.

'X INFO',x,y,z,'name','unit',dt
'Y INFO',x,y,z,'name','unit'

See 'U INFO for comments. these names are used in format version 2.2.'

'# VAR TYPES',x

The number of variables (x), which is also the number of headerlines that follows with variable information. Note, that this does not have to be the same as the number of columns, as the number of columns for a variable can be more than one.

variables
'variable','unit',x

Name of the variable ('variable'), its unit ('unit') and how many columns this value (x) has, this could for example be drop counts for each diameter bin of an FSSP or radiances at a number of frequencies. If x is larger than one, please make sure that the user knows exactly what they are, either in the y-axis information (which can only be done for one variable) or in the comments lines.

%% BEGIN COMMENT
%% END COMMENT

Between these two lines you can dump everything, you still wanted to say. It is a good idea to give e-mail and web-adresses for more information and cooperation.

y-axis or variables

This line is optional, although its presence should correspond with the information in the last dimension specified in the header, see 'U INFO'. This line can be used if there is one dimension that is not sampled equidistant. It also makes plotting 2D data easier in packages like Transform and 1D data in spreadsheets and many graph-packages. This line is not part of the header.

Data lines

The data lines should start with a time, then other data separated by tabs. Please do not include any non-numeric data in here, but specify values for NaN, +INF, -INF, etc. in the comments.