Management and Exploitation of Solar Resource Knowledge  





Download the benchmarking report: Benchmarking of radiation products

Benchmarking is the largest activity within the MESoR project. The aim of the benchmarking exercise is to establish a coherent set of benchmarking rules and reference data sets to enable a transparent and comparable evaluation of the different solar radiation data sources. The rules are developed in conjunction with the IEA Task 36 on "Solar Resource Management" of the Solar Heating and Cooling Implementing Agreement and shall serve as a standard for benchmarking to make results comparable.

Reference data

This activity focuses on collection of high quality ground measurements which can be used as a reference in the benchmarking exercises. The measurements should be conducted with high accuracy, high frequency and traceable maintenance of the equipment. Data has been collected from the Baseline surface radiation network (BSRN), International Daylight Measurement programme (IDMP), the meteomedia network, the World Radiation Data Center (WRDC) and the Global Atmospheric Watch (GAW) programme. In addition further measurements were collected from scientific institutions, providing they fulfil the quality criteria above.

A common quality control procedure has been defined for all broadband time series data. The parameters for the quality assessment have been deducted from the Baseline Surface Radiation Network Operation Manual and operational experience of the partners involved.

Benchmarking measures and rules

Benchmarking of solar radiation products can be done in different ways. If a kind of reference data is available which is assumed to be the "truth", the modelled data sets can be compared and ranked how well they represent the reference data. But there is not always reference data available: e.g. for solar radiation spatial products (maps). Here benchmarking can assess the uncertainty of mapping products by their cross-comparison.

For site specific time series there are a number of different measures for benchmarking. A first set is based on first order statistics. These are the well known bias, root mean square deviations, standard deviations, their relative values to the average of the data set and the correlation coefficient. They compare how well data pairs at the same point of time compare with each other. They are important if one needs an exact representation of real data, e.g. for evaluations of real operating systems or forecasts of solar radiation parameters.

This exact match is not always important, e.g. for system design studies. Here the similarity of statistical properties as frequency distributions is more important than the exact match of data pairs. The MESoR project therefore suggests a number of parameters based on second order statistics.

Solar maps can be benchmarked in two ways, either point based or map based. The point based benchmarking is similar to the time series benchmarking. Data is extracted from the maps and compared to the measurements ("ground truth"). First and second order statistics can be applied. Map based cross comparison of solar radiation provides means for improved understanding of regional distribution of the uncertainty by combining all existing resources (calculating the average of all) and quantifying their mutual agreement by the means of standard deviation. A sample evaluation has been done with five spatial data bases: ESRA, PVGIS, Meteonorm, Satel-light and NASE SSE.

Yearly sum of global horizontal irradiation, average of all five data bases [kWh/m²] (left), uncertainty at 95% confidence interval from the comparison of the five data bases. [%] (right). - contact: carsten.hoyer-klick AT © DLR 2010