Research Papers

Proposed Metric for Evaluation of Solar Forecasting Models

[+] Author and Article Information
Ricardo Marquez

Mechanical Engineering and Applied Mechanics
School of Engineering,
University of California,
Merced, CA 95343

Carlos F. M. Coimbra

Department of Mechanical and
Aerospace Engineering,
Jacobs School of Engineering,
Center of Excellence in Renewable
Resource Integration,
University of California,
San Diego, La Jolla, CA 92093
e-mail: ccoimbra@ucsd.edu

1Corresponding author.

Contributed by the Solar Energy Division of ASME for publication in the JOURNAL OF SOLAR ENERGY ENGINEERING. Manuscript received April 25, 2011; final manuscript received July 2, 2012; published online October 23, 2012. Assoc. Editor: Carsten Hoyer-Klick.

J. Sol. Energy Eng 135(1), 011016 (Oct 23, 2012) (9 pages) Paper No: SOL-11-1098; doi: 10.1115/1.4007496 History: Received April 25, 2011; Revised July 02, 2012

This work presents an alternative metric for evaluating the quality of solar forecasting models. Some conventional approaches use quantities such as the root-mean-square-error (RMSE) and/or correlation coefficients to evaluate model quality. The direct use of statistical quantities to assign forecasting quality can be misleading because these metrics do not convey a measure of the variability of the time-series for the solar irradiance data. In contrast, the quality metric proposed here, which is defined as the ratio of solar uncertainty to solar variability, compares the forecasting error with the solar variability directly. By making the forecasting error to variability comparisons for different time windows, we show that this ratio is essentially a statistical invariant for each forecast model employed, i.e., the ratio is preserved for widely different time horizons when the same time averaging periods are used, and therefore provides a robust way to compare solar forecasting skills. We employ the proposed metric to evaluate two new forecasting models proposed here, and compare their performances with a persistence model.

Copyright © 2012 by ASME
Your Session has timed out. Please sign back in to continue.


Lew, D., and Piwko, R., 2010, “Western Wind and Solar Integration Study,” National Renewable Energy Laboratories, Technical Report No. NREL/SR-550-47781.
California Independent System Operator (CAISO), 2010, “Integration of Renewable Resources: Operational Requirements and Generation Fleet Capability at 20 Percent RPS,” available online at http://www.caiso.com/2804/2804d036401f0.pdf.
Rodriguez, G. D., 2010, “A Utility Perspective of the Role of Energy Storage in the Smart Grid,” Power and Energy Society General Meeting, IEEE, Minneapolis, MN, July 25–29, pp. 1–2. [CrossRef]
Marquez, R., and Coimbra, C. F. M., 2011, “Forecasting of Global and Direct Solar Irradiance Using Stochastic Learning Methods, Ground Experiments and the NWS Database,” Sol. Energy, 85(5), pp. 746–756. [CrossRef]
Pedro, H. T. C., and Coimbra, C. F. M., 2012, “Assessment of Forecasting Techniques for Solar Power Output With No Exogenous Inputs,” Sol. Energy, 86, pp. 2017–2028. [CrossRef]
Perez, R., Kivalov, S., Schlemmer, J., Hemker, K., Renne, D., and Hoff, T. E., 2010, “Validation of Short and Medium Term Operational Solar Radiation Forecasts in the US,” Sol. Energy, 84(5), pp. 2161–2172. [CrossRef]
Cao, J., and Lin, X., 2008, “Study of Hourly and Daily Solar Irradiation Forecast Using Diagonal Recurrent Wavelet Neural Networks,” Energy Convers. Manag., 49(6), pp. 1396–1406. [CrossRef]
Mellit, A., 2008, “Artificial Intelligence Technique for Modelling and Forecasting of Solar Radiation Data: A Review,” Int. J. Artif. Intell. Soft Comput., 1, pp. 52–76. [CrossRef]
Martin, L., Zarzalejo, L. F., Polo, J., Navarro, A., Marchante, R., and Cony, M., 2010, “Prediction of Global Solar Irradiance Based on Time Series Analysis: Application to Solar Thermal Power Plants Energy Production Planning,” Sol. Energy, 84(10), pp. 1772–1781. [CrossRef]
Mills, A., and Wiser, R., 2010, “Implications of Wide-Area Geographic Diversity for Short-Term Variability of Solar Power,” Lawrence Berkeley National Laboratory, Technical Report No. LBNL-3884E.
Hoff, T. E., and Perez, R., 2010, “Quantifying PV Power Output Variability,” Sol. Energy, 84(10), pp. 1782–1793. [CrossRef]
Ineichen, P., 2006, “Comparison of Eight Clear Sky Broadband Models Against 16 Independent Data Banks,” Sol. Energy, 80, pp. 468–478. [CrossRef]
Zarzalejo, L. F., Polo, J., and Ramirez, L., 2004, “Gc_model5_irradiance,” (Matlab computer program) CD-ROM accompanying Ref. [14].
Badescu, V., 2008, Modeling Solar Radiation at the Earth Surface, Springer-Verlag, Berlin/Heidelberg.
Hoff, T. E., and Perez, R., 2011, “Modeling PV Fleet Output Variability,” Sol. Energy, 86(8), pp. 2177–2189. [CrossRef]
Lave, M., and Kleissl, J., 2010, “Solar Variability of Four Sites Across the State of Colorado,” Renewable Energy, 35(12), pp. 2867–2873. [CrossRef]
Bishop, C., 1995, Neural Networks for Pattern Recognition, Oxford University, Great Clarendon Street, Oxford, UK.
Marquez, R., Gueorguiev, V. G., and Coimbra, C. F. M., 2012, “Forecasting Solar Irradiance Using Sky Cover Indices,” ASME J. Sol. Energy Eng. (in press).
Lorenz, E., Hurka, J., Heinemann, D., and Beyer, H. G., 2009, “Irradiance Forecasting for the Power Prediction of Grid-Connected Photovoltaic Systems,” IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., 2(1), pp. 2–10. [CrossRef]
Lave, M., Kleissl, J., and Arias-Castro, E., 2012, “High-Frequency Irradiance Fluctuations and Geographic Smoothing,” Sol. Energy, 86(8), pp. 2190–2199. [CrossRef]
Marcos, J., Marroyo, L., Lorenzo, E., Alvira, D., and Izco, E., 2011, “Power Output Fluctuations in Large Scale PV Plants: One Year Observations With One Second Resolution and a Derived Analytic Model,” Prog. Photovoltaics, 19(2), pp. 218–227. [CrossRef]


Grahic Jump Location
Fig. 1

Comparison of ESRA and polynomial-fit clear-sky models. The coefficient of determination between the two models is (R2) = 0.998 and the RMSE is RMSE = 14.7 W/m2.

Grahic Jump Location
Fig. 2

Example of persistent model performance for a clear and a partially cloudy day (Mar. 20–21, 2010). The clear day is approximated very well by persistent model, whereas a “time delay” is observed for the partially cloudy day.

Grahic Jump Location
Fig. 3

Measured, modeled, and forecasted clear sky days arbitrarily selected for 2010. This figure illustrates the improved accuracy of a clear sky persistence forecast model over original clear sky model. The RMSEs are, respectively, 20.7 W/m2 and 26.6 W/m2 for the clear sky persistence forecast model and the original model.

Grahic Jump Location
Fig. 4

Time series of global horizontal irradiance (I) values, estimated clear-sky I and calculated values of stochastic step changes, Δk (Data for May 8–10, 2010 in Merced, CA)

Grahic Jump Location
Fig. 5

Time series of solar irradiance and Δk. The figure illustrates the partition of the time series into window sizes of Nw=500 h. Each dashed vertical line represents the boundaries of the 500-h time windows.

Grahic Jump Location
Fig. 6

Scatter plot of U and V using various clear sky models including a polynomial-based, the ESRA-based, and the clearness index model which uses extraterrestrial irradiance for normalization

Grahic Jump Location
Fig. 7

Evaluation of 〈s〉=1-U/V versus Nw (time-window sizes) after modifying algorithm with different clear sky and persistence models

Grahic Jump Location
Fig. 8

Hourly forecasting comparisons for five consecutive days (Oct. 27–31, 2010) in the validation data set with night values removed

Grahic Jump Location
Fig. 9

Root mean square errors (RMSEs) for different forecast models versus RMSE of persistent model: (a) NAR and NARX model and (b) CMF model [6]. The outlier point within the dashed circle was ignored for calculating the regression line in (b).

Grahic Jump Location
Fig. 10

Empirical data compared with modeling predictions of uncertainty (forecast errors) reduction




Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In