Impression of Schiphol airport during a fog event. Photo: Peter de Vries
Impression of Schiphol airport during a fog event. Photo: Peter de Vries
An update of the visibility forecast system is achieved in close collaboration with the main users of the airport, i.e. Air Traffic Control, the airport authorities and KLM airlines. Next to the improved forecast system, a methodology is provided to support the user’s challenge of making an economically well-founded decision based on a probability forecast.

Tailoring the visibility forecast
The existing automated system that is used to create visibility forecasts comprises HIRLAM (High Resolution Limited Area Model) output in combination with a statistical post-processing module, the TAFG (Terminal Aerodrome Forecast Guidance) (1). Output of both is supplied to the forecaster who issues the operational probability forecast, the Schiphol Probability Forecast (in Dutch: Schipholkansverwachting; SKV). Figure 1 gives a schematic overview of the forecast cascade that leads to the SKV. To tailor the visibility forecasts for the specific needs of the Schiphol airport community, we have extended the statistical post-processing module TAFG.
Table 1. Relation between categories of visibility/ceiling and capacity/flow restrictions at Amsterdam Airport Schiphol (May 2007). 1) VIS: Visibility,2) RVR: Runway Visual Range, 3) LVP: Low Visibility Procedure. Cloud base (Ceiling) gives the base height of the lowest clouds having at least a cloud cover of 5/8.
Table 1. Relation between categories of visibility/ceiling and capacity/flow restrictions at Amsterdam Airport Schiphol (May 2007). 1) VIS: Visibility, 2) RVR: Runway Visual Range, 3) LVP: Low Visibility Procedure. Cloud base (Ceiling) gives the base height of the lowest clouds having at least a cloud cover of 5/8.

Special Visibility Definitions
The statistical post processing module TAFG has been extended with special predictands for several combinations of visibility and ceiling, fully in line with the thresholds for LVP phases used by Air Traffic Control. These thresholds include special visibility definitions, such as Runway Visual Range (RVR). RVR values are usually derived from the regularly observed visibility, the Meteorological Optical Range (MOR), and observations of the background luminance. In order to be able to provide forecasts for the RVR, forecasts of the background luminance and a probabilistic MOR forecast have to be combined. Obviously, for the background luminance astronomical parameters play the dominant role (sunrise and sunset), but use of meteorological parameters like cloud amount and cloud base height can improve the statistical skill of the background luminance forecast. To give an example of a result of the procedure, a predictand denoting the probability of MOR visibility being less than 400 m can be transformed into a probability of RVR being less than 867 m.

For determining the LVP phase, we need the probabilities of RVR at the thresholds of 200, 350, 550 and 1500 m (Table 1). The probabilities of the LVP phases are interpolated from the derived RVR probabilities.

Verification
In this paragraph the improved SKV forecasts are verified against observations to examine their performance. In addition, verification results of the old and the new TAFG are compared.

In Figure 2 a reliability diagram is presented for LVP class A or worse. For the period July 2004 to April 2007 comparison is shown between the old TAFG, the new TAFG and the Schiphol probability forecast SKV as issued by the forecaster (and based on the old TAFG). This dataset was also used to develop the new TAFG. This means that the new TAFG verification results in this figure are dependent.

Ideally, all points should be on the diagonal in a reliability diagram (2). The comparison between the new TAFG and the old TAFG gives a strong indication of increased reliability. The old TAFG and the SKV show overforecasting, i.e. forecast probabilities are higher than observed frequencies (data points below the diagonal). The forecaster achieves for the short term about the same reliability as the old TAFG, but has added value by making a more distinct forecast by also forecasting high probabilities. The forecaster reached up t0 100% whereas both the old and new TAFG do hardly exceed 80%, in other words the forecaster adds resolution. In addition to the reliability, the skill of the forecasts is determined.

The Brier Skill Score (BSS) expresses the quality of the forecast by comparing to a defined reference forecast, in this case the climatological probabilities. A BSS of 100% indicates a perfect forecast, 0% indicates the same skill as climatology and a negative percentage means that the climatological forecast is better2). All forecasts show a positive BSS (Figure 2), the new TAFG provides the highest score followed by the forecaster.

From June 2007 and onwards the new TAFG could be verified on an independent data set. The verification results in Figure 3 show the results for that period. Both forecaster (SKV) and TAFG show a high reliability. Again the forecaster adds resolution, whereas the BSS is somewhat higher for the TAFG. These verification results demonstrate the usefulness of the new TAFG and the Schiphol probability forecast.
Figure 1. Schematic overview of the forecast cascade leading to the probabilistic forecast (SKV) for Schiphol. HIRLAM is the numerical weather prediction model, which feeds the TAFG and the forecaster. Observations are necessary input for HIRLAM, the TAFG and the forecaster. Note that the TAF (Terminal Aerodrome Forecast) in the lower right of the overview is an existing internationally agreed standard forecast which is not discussed in this highlight.
Figure 1. Schematic overview of the forecast cascade leading to the probabilistic forecast (SKV) for Schiphol. HIRLAM is the numerical weather prediction model, which feeds the TAFG and the forecaster. Observations are necessary input for HIRLAM, the TAFG and the forecaster. Note that the TAF (Terminal Aerodrome Forecast) in the lower right of the overview is an existing internationally agreed standard forecast which is not discussed in this highlight.



Figure 2. Left panel: Reliability diagram for short term forecast (3, 6 and 9 hours ahead) of LVP phase A or worse (see Table 1). In the reliability diagram the observed frequency is plotted against the forecast probability. Reliability is indicated by the proximity of the plotted curve to the diagonal. Verification is shown for the old TAFG, new TAFG and the SKV as issued by the forecaster (based on the old TAFG). The number of the various forecast values is depicted below the curves and also shown in the right panel on a logarithmic scale. Brier Skill Scores are also given for these three forecasts. Verification is performed against Schiphol (AUTO)SYNOP data for the period July 2004 until April2007. This dataset was also used to develop the new TAFG.
Figure 2. Left panel: Reliability diagram for short term forecast (3, 6 and 9 hours ahead) of LVP phase A or worse (see Table 1). In the reliability diagram the observed frequency is plotted against the forecast probability. Reliability is indicated by the proximity of the plotted curve to the diagonal. Verification is shown for the old TAFG, new TAFG and the SKV as issued by the forecaster (based on the old TAFG). The number of the various forecast values is depicted below the curves and also shown in the right panel on a logarithmic scale. Brier Skill Scores are also given for these three forecasts. Verification is performed against Schiphol (AUTO)SYNOP data for the period July 2004 until April2007. This dataset was also used to develop the new TAFG.


Optimizing the categorical decision
Once the probability forecast is available, it is the user’s challenge to use this probabilistic information to make an economically well-founded categorical decision. Ideally, the user chooses the category that corresponds to a fixed predetermined (optimal) percentile of the probability distribution.

In Fig 4 a conceptual methodology is presented which shows the potential benefit that can be gained by combining: a) the quality of the probabilistic forecasts and b) the user’s sensitivity to false alarms and misses in his categorical decision. The system performance over a recent period is given in the form of a contingency table of forecast versus observed categories. In a so-called expense matrix the extra ‘costs’ or damage due to wrong forecasts is specified. Multiplying the performance table by the expense matrix yields the user-specific costs due to wrong forecasts over the period involved. The ‘optimal’ percentile can be derived by calculating the extra costs for a large number of choices of the percentile and determine for which value these costs are lowest. Although based on past performance, it is plausible that also for actual forecasts this optimal percentile will be economically the best choice.


In Figure 5 an example of the above is presented; it shows the expenses as a function of the decision-percentile for a number of fictitious users with different ratio of sensitivity for false alarms and misses.
Figure 3. Reliability diagram, similar as Figure 2, for the new TAFG and the SKV as issued by the forecaster (based on the TAFG). Data are from the period June 2007 until February 2009; from May 2008 the new TAFG became available to the forecaster.
Figure 3. Reliability diagram, similar as Figure 2, for the new TAFG and the SKV as issued by the forecaster (based on the TAFG). Data are from the period June 2007 until February 2009; from May 2008 the new TAFG became available to the forecaster.


Figure 4. The system performance multiplied with the user’s cost estimation delivers the total expense. The system performance is based on a threshold percentage of 25%; the category that corresponds to the 25th percentile of the probability distribution is taken. The headings CD, B, A and MG refer to the LVP classes as described in Table 1.
Figure 4. The system performance multiplied with the user’s cost estimation delivers the total expense. The system performance is based on a threshold percentage of 25%; the category that corresponds to the 25th percentile of the probability distribution is taken. The headings CD, B, A and MG refer to the LVP classes as described in Table 1.


Figure 5. Expense as a function of threshold percentage for different false alarm miss ratios (indicated by Fa/Mi). Different ratios lead to different optimal threshold percentages (at minimum of the plotted curve). The forecasts are the +4 of the new TAFG system over the period 2003-2007.
Figure 5. Expense as a function of threshold percentage for different false alarm miss ratios (indicated by Fa/Mi). Different ratios lead to different optimal threshold percentages (at minimum of the plotted curve). The forecasts are the +4 of the new TAFG system over the period 2003-2007.

Conclusion
The new statistical post-processing yields a tailored and improved low visibility forecast for Amsterdam airport Schiphol. The newly developed forecast for background luminance in combination with - the already available - MOR forecast made the production of a RVR forecast possible. Verification of the RVR forecast showed a reliable and skilful product. Under low visibility conditions this runway related visibility is important in the decision making process.

This improved and tailored forecast can directly benefit the operations at the airport. By taking into account the user-specific operation and optimizing the decision threshold, the benefit can be even higher. In future, accurate forecasts will continue to be important for the airport. Future research is planned to further improve the low visibility forecasting and the understanding of the physical processes underlying fog.

The use of high-resolution 3D models, a 1 column model for the very local physical processes and the introduction of new sensor technology for optimizing the detection of low visibility, are possibilities to be considered. The practical and meteorological feasibility for differentiating the Schiphol probability forecast, by making specific forecasts for various parts of the airport, is another issue to be investigated.

Figure 6. Impression of Schiphol airport during a fog event. Photo: Peter de Vries
Figure 6. Impression of Schiphol airport during a fog event. Photo: Peter de Vries

References


  1. Jacobs, A.J.M. and N. Maat, 2005. Numerical Guidance Methods for Decision Support in Aviation Meteorological Forecasting. Weather and Forecasting, 20, 82-100.
  2. Wilks D. S, 1995. Statistical Methods in the Atmospheric Sciences. An introduction. Academic press, San Diego, US, 627pp.
  3. KDC-LVP Project team [Hove, R. ten and J.B. Wijngaard (Eds)], 2008. Improved Low visibility and Ceiling Forecasts at Schiphol Airport, final report, part 1. KNMI Publication 222.