HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Figures Only Full Text (PDF) Free Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in Web of Science Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Web of Science (17) Citing Articles via Google Scholar Google Scholar Articles by Fila, G. Articles by Acutis, M. Search for Related Content PubMed Articles by Fila, G. Articles by Acutis, M. Agricola Articles by Fila, G. Articles by Acutis, M. Related Collections Software

NOTES & UNIQUE PHENOMENA

IRENE_DLL

A CLASS LIBRARY FOR EVALUATING NUMERICAL ESTIMATES

^a Res. Inst. for Industrial Crops, via di Corticella 133, 40128 Bologna, Italy
^b Dep. of Crop Sci., via Celoria 2, 20133 Milan, Italy

^* Corresponding author (g.fila{at}isci.it).

Received for publication June 21, 2002.

	ABSTRACT

TOP ABSTRACT INTRODUCTION Description Structure and Basic Concepts Application and Availability Remarks Appendix: REFERENCES

 
IRENE_DLL (Integrated Resources for Evaluating Numerical Estimates–Dynamic Link Library) is a Microsoft (MS) COM (component object model) class library providing a set of routines designed to facilitate the implementation of model evaluation techniques. Statistical procedures (difference-based analysis, correlation–regression analysis, probability distributions, pattern analysis, statistics aggregation, and time mismatch analysis) are applied to allow comparing estimates against measurements, either individually taken or replicated. The dynamic link library (DLL) can be easily interfaced with applications developed under a MS Windows programming language. An essential description of the program is given along with the basic concepts of usage.

Abbreviations: DLL, dynamic link library • MS, Microsoft

	INTRODUCTION

TOP ABSTRACT INTRODUCTION Description Structure and Basic Concepts Application and Availability Remarks Appendix: REFERENCES

 
THE INCREASING COMPLEXITY of agroecological models has stimulated modular approaches in model development (Acock and Reynolds, 1997; Luo et al., 1997; Kage and Stützel, 1999; Jones et al., 2001). The concept of modularity gained strong momentum with the widespread of the object-oriented techniques in software development (Sequeira et al., 1991; Silvert, 1993), which made easier the realization, even in agroecological models, of modern software engineering principles such as components integration, code reusability, systematic model development, easy documentation and maintenance, and language neutrality (Van Evert and Campbell, 1994; Acock and Reddy, 1997; Reynolds and Acock, 1997; Pan et al., 2000). This trend is finding increasing consensus, and it is evolving toward the standardization of simulation environments based on standard classes and interfaces (Hillyer et al., 2003).

In this context, we approached the issue of evaluating model estimates, which has always been regarded as a crucial step in the complex task of modeling activity in the fields of agrometeorology, soil science, and cropping systems (e.g., Penning de Vries, 1977; Martorana and Bellocchi, 1999; Sinclair and Seligman, 2000). The literature gives a number of statistical techniques on model evaluation; however, a well-grounded standard theory is missing, and careful attention to the modeling concepts and terminology used is required (Oreskes et al., 1994). The availability of dedicated software tools providing extensive, integrated statistical capabilities (e.g., IRENE; Fila et al., 2003) is highly supportive for exploring various aspects of model evaluation. In general, however, crop growth and hydrologic modelers deal with model evaluation by means of simplified statistical procedures incorporated within their own software. The lack of specifically designed libraries makes necessary the implementation of these procedures from the ground every time a software requiring model evaluation is developed, to the detriment of complexity and reliability of routines. To overcome this limitation, we developed a DLL that incorporates the most relevant aspects of statistical evaluation of models and which can be plugged into existing application software running on a MS Windows–based operating system.

	Description

TOP ABSTRACT INTRODUCTION Description Structure and Basic Concepts Application and Availability Remarks Appendix: REFERENCES

 
IRENE_DLL is a collection of ready-to-use routines that evaluates models by comparing model estimates against measured data. The present description gives a synthetic overview of the program capabilities and some principles of usage. For the interested reader, a complete, detailed documentation accompanies the software with an extended set of code examples.

Six groups of analysis are implemented in IRENE_DLL and are listed in Table 1.

When replicates are available, confidence intervals and test statistics are computed (Smith et al., 1997). If estimates and measurements are replicated in pairs, model failure/goodness is evaluated according to two variation sources: experimental error and replication variability (Pitman–Morgan procedure).

Statistical Association–Based Analysis
Measures of statistical association such as correlation and regression coefficients provide quantitative estimates of the statistical covariation between estimated and measured values (Addiscott and Whitmore, 1987).

Regression parameters are calculated according to two methods: ordinary least squares and reduced major axis. Significance tests are applied to both correlation coefficients (Pearson and Spearman) and regression parameters (intercept and slope).

Pattern Analysis
Systematic biases of model residuals against external variables are assessed by means of pattern indices of different type (range-based, F-based; fixed, variable), developed by Donatelli et al. (2000)(unpublished, 2003).

Probability Distributions
Density functions and cumulative probability distributions (i.e., probability of exceedence; Weibull, 1961) are computed for evaluating outputs from models that include stochastic inputs.

Statistics Aggregation
The aggregation of more statistics by fuzzy-based rules was proposed (Bellocchi et al., 2002a, 2002b) to capture inaccuracies of different source into synthetic indices, giving a global score about model performance. The user is required to define the relative importance of each basic index and the limits above and under which it is certainly favorable or unfavorable. Statistics of different type may be aggregated into modules and more modules into indicators (second-level aggregation).

Time Mismatch Analysis
The presence of mismatches in time histories generated by dynamic models can be detected, according to Donatelli et al. (2002). The time mismatch is identified by the time shift (forward or backward) at which the best value of evaluation statistics are reached.

	Structure and Basic Concepts

TOP ABSTRACT INTRODUCTION Description Structure and Basic Concepts Application and Availability Remarks Appendix: REFERENCES

 
IRENE_DLL consists of 10 classes, including data services, mathematical routines, and some data analysis procedures. There is not a hierarchical structure, so all classes can be accessed individually by the client code (Fig. 1 ; Table 2). Some naming conventions have been adopted to designate the elements of the program. Classes are prefixed by IRENEobj_, constants and enumerations are prefixed by IRENEconst_, and variable types are marked with IRENEvar_.

View larger version (30K): [in this window] [in a new window]   Fig. 1. The 10 class modules of IRENE_DLL.

Math functions are grouped in five distinct classes: IRENEobj_Index, IRENEobj_Regression, IRENEobj_Tests, IRENEobj_DistrFunctions, and IRENEobj_Pattern. They accept data in the IRENEobj_DataObject format and return the outputs in specifically designed variable types. The functions for building the module and computing its value are contained in the class IRENEobj_Module. Multiple modules may be aggregated in one more complex index, the Indicator, by the class IRENEobj_Indicator.

All functions accept a shift integer-type parameter, which tells the function to displace the estimated data with respect to the measured data series of a given number of places, before computing the index. By iterating the function over a range of shifts, it is possible to detect time-related model biases.

	Application and Availability

TOP ABSTRACT INTRODUCTION Description Structure and Basic Concepts Application and Availability Remarks Appendix: REFERENCES

 
IRENE_DLL was developed within an agroecological modeling background. The intended users are MS Windows programmers (in Visual Basic, C++, Delphi, etc.) needing to incorporate model evaluation procedures into their applications. The Appendix shows some pseudocode examples of how to use IRENE_DLL procedures.

The DLL was developed in MS Visual Basic 6. It runs under MS Windows NT/2000/XP. Installation of the DLL requires about 1 MB free disk space for the DLL file and the help file. IRENE_DLL does not require any external math library to run because all of the algorithms were either developed or customized in house from free code collected over the years. Basic statistical distributions (normal, t, and F) were verified against Snedecor and Cochran's tables (Snedecor and Cochran, 1980).

IRENE_DLL is available free of charge for noncommercial purposes. Both the techniques implemented and the scientific background are fully documented by the help file. The installation package and the stand-alone help file can be downloaded from the Internet at http://www.isci.it/tools/IRENE/IRENE.htm (verified 25 Apr. 2003). Comments about IRENE_DLL may be sent to agronomy{at}isci.it.

	Remarks

TOP ABSTRACT INTRODUCTION Description Structure and Basic Concepts Application and Availability Remarks Appendix: REFERENCES

 
The DLL for the computation of model evaluation statistics is part of a large action toward a commonly agreed architecture for agronomic/agrometeorological modeling. It serves as a convenient means to support collaborative model development among a large, distributed network of scientists involved in creating object-oriented models in the agronomy and agrometeorology fields. It can be used in any model that is written in any object-oriented language under MS Windows. Code sharing significantly increases transparency of the underpinning science, and any improvement in the science can be easily transferred into the DLL routines. The developers are committed to enriching its offerings to keep up with evolving statistical methodology.

	Appendix:

TOP ABSTRACT INTRODUCTION Description Structure and Basic Concepts Application and Availability Remarks Appendix: REFERENCES

 
pseudocode examples of how to use irene_dll
IRENE_DLL is available with a detailed documentation, which includes a commented MS Visual Basic 6 sample code for each available procedure. The following pseudocode examples are intended to give a general outlook on how the DLL works.

Example 1—Load Input Data into an IRENEobj_DataObject
* Source data stored in two bidimensional arrays

Get array MyEstimates(1 to 365, 1 to 12)

Get array MyMeasures(1 to 365, 1 to 12)

* Start an instance of the IRENEobj_DataObject

Create object MySelection As IRENEobj_DataObject

* Fill the IRENEobj_DataObject object with the source data

For i = 1 to 365

For j = 1 to 12

MySelection.Estimated(i, j) MyEstimates (i, j)

MySelection.Measured(i, j) MyMeasures (i, j)

Loop j

Loop i

Example 2—Compute Linear Regression by Least-Squares Method
* Get an IRENEobj_DataObject already loaded with input data

Get IRENEobj_DataObject MySelection

* Declare a purposely defined variable for storing results

Declare Variable Regr_results As IRENEvar_ Regression

* Start an instance of the IRENEobj_Regression, which contains the desired function

Create object RegrCalculator As IRENEobj Regression

* Call the desired function from the parent object and write the outputs in the storing variable (other function specifications omitted). This single line does all the work. Regr_results RegrCalculator.Regression_ LS(MySelection,,,)

* Display results in an appropriate manner

Display Regr_results

Example 3—Display Results of Example 2
* Get an output variable from a previous computing process

Get Variable Regr_results

* Display all the outputs contained in the variable

Display Regr_results.Slope

Display Regr_results.Slope_StandError

Display Regr_results.Slope_Prob_Tvs0

Display Regr_results.Slope_Prob_Tvs1

Display Regr_results.Intercept

Display Regr_results.Intercept_StandError

Display Regr_results.Intercept_Prob_Tvs0

Display Regr_results.F

Display Regr_results.Prob_F

	ACKNOWLEDGMENTS

 
Research was done under the auspices of the Italian Ministry of Agricultural and Forestry Policies, project SIPEAA (http://www.sipeaa.it), paper no. 9.

	REFERENCES

TOP ABSTRACT INTRODUCTION Description Structure and Basic Concepts Application and Availability Remarks Appendix: REFERENCES

 

• Acock, B., and V.R. Reddy. 1997. Designing an object-oriented structure of crop model. Ecol. Modell. 94:33–44.
• Acock, B., and J.F. Reynolds. 1997. Introduction: Modularity in plant models. Ecol. Modell. 94:1–6.
• Addiscott, T.M., and A.P. Whitmore. 1987. Computer simulation of changes in soil mineral nitrogen and crop nitrogen during autumn, winter and spring. J. Agric. Sci. (Cambridge) 109:141–157.
• Bellocchi, G., M. Acutis, G. Fila, and M. Donatelli. 2002a. An indicator of solar radiation model performance based on a fuzzy expert system. Agron. J. 94:1222–1233.[Abstract/Free Full Text]
• Bellocchi, G., G. Fila, and M. Donatelli. 2002b. Integrated evaluation of cropping systems models by fuzzy-based procedure. p. 241–242. In Proc. Eur. Soc. for Agron. Congr., 7th, Cordoba, Spain. 15–18 July 2002. Junta de Andalucía, Consejería de Agricultura y Pesca, Cordoba, Spain.
• Donatelli, M., M. Acutis, and G. Bellocchi. 2000. Two statistical indices to quantify patterns of errors produced by models. p. 186. In Proc. Int. Crop Sci. Conf., 3rd, Hamburg, Germany. 17–22 Aug. 2000. Eur. Soc. of Agron., Hamburg, Germany.
• Donatelli, M., M. Acutis, G. Fila, and G. Bellocchi. 2002. A method to quantify time mismatch of model estimates. p. 269–270. In Proc. Eur. Soc. for Agron. Congr., 7th, Cordoba, Spain. 15–18 July 2002. Junta de Andalucía, Consejería de Agricultura y Pesca, Cordoba, Spain.
• Fila, G., G. Bellocchi, M. Acutis, and M. Donatelli. 2003. IRENE: A software to evaluate model performance. Eur. J. Agron. 18:369–372.
• Fox, D.G. 1981. Judging air quality model performance: A summary of the AMS workshop on dispersion models performance. Bull. Am. Meteorol. Soc. 62:599–609.
• Hillyer, C., J. Bolte, F. van Evert, and A. Lamaker. 2003. The ModCom modular simulation system. Eur. J. Agron. 18:333–343.
• Jones, J.W., B.A. Keating, and C.H. Porter. 2001. Approaches to modular model development. Agric. Syst. 70:421–443.
• Kage, H., and H. Stützel. 1999. HUME: An object oriented component library for generic modular modelling of dynamic systems. p. 299–300. In Proc. Int. Symp. on Modelling Cropping Syst., 1st, Lleida, Spain. 21–23 June 1999. Eur. Soc. of Agron., Lleida, Spain.
• Kobayashi, K., and M.U. Salam. 2000. Comparing simulated and measured values using mean squared deviation and its components. Agron. J. 92:345–352.[Abstract/Free Full Text]
• Loague, K., and R.E. Green. 1991. Statistical and graphical methods for evaluating solute transport models: Overview and application. J. Contam. Hydrol. 7:51–73.
• Luo, Y., C.B. Field, and H.A. Mooney. 1997. Adapting GePSI (Generic Plant Simulator) for modeling studies in the Jasper Ridge CO₂ project. Ecol. Modell. 94:81–88.
• Martorana, F., and G. Bellocchi. 1999. A review of methodologies to evaluate agroecosystem simulation models. Ital. J. Agron. 3:19–39.
• Oreskes, N., K. Schrader-Frechette, and K. Belitz. 1994. Verification, validation, and confirmation of numerical models in the earth sciences. Science 263:641–646.[Abstract/Free Full Text]
• Pan, X., J.D. Hesketh, and M.G. Huck. 2000. OWSimu: An object-oriented and web-based simulator for plant growth. Agric. Syst. 63:33–47.
• Penning de Vries, F.W.T. 1977. Evaluation of simulation models in agriculture and biology: Conclusions of a workshop. Agric. Syst. 2:99–107.
• Reynolds, J.F., and B. Acock. 1997. Modularity and genericness in plant and ecosystem models. Ecol. Modell. 94:7–16.
• Sequeira, R.A., P.J.H. Sharpe, N.D. Stone, K.M. El-Zik, and M.E. Makela. 1991. Object-printed simulation: Plant growth and discrete organ to organ interactions. Ecol. Modell. 58:55–89.
• Silvert, W. 1993. Object-oriented ecosystem modelling. Ecol. Modell. 68:91–118.
• Sinclair, T.R., and N. Seligman. 2000. Criteria for publishing papers on crop modeling. Field Crops Res. 68:165–172.
• Smith, P., J.U. Smith, D.S. Powlson, W.B. McGill, J.R.M. Arah, O.G. Chertov et al. 1997. A comparison of the performance of nine soil organic matter models using datasets from seven long-term experiments. Geoderma 81:153–225.[Web of Science]
• Snedecor, G., and W. Cochran. 1980. Statistical methods. 7th ed. The Iowa State Univ. Press, Ames.
• Van Evert, F.K., and G.S. Campbell. 1994. CropSyst: A collection of object-oriented simulation models of agricultural systems. Agron. J. 86:325–331.[Abstract/Free Full Text]
• Weibull, W. 1961. Fatigue testing and analysis of results. Pergamon Press, Oxford, UK.
• Willmott, C.J., and D.E. Wicks. 1980. An empirical method for the spatial interpolation of monthly precipitation within California. Phys. Geogr. 1:59–73.
• Yang, J., D.J. Greenwood, D.L. Rowell, G.A. Wadsworth, and I.G. Burns. 2000. Statistical methods for evaluating a crop nitrogen simulation model, N_ABLE. Agric. Syst. 64:37–53.

This article has been cited by other articles:

				D. W. Meek, T. A. Howell, and C. J. Phene Concordance Correlation for Model Performance Assessment: An Example with Reference Evapotranspiration Observations Agron. J., July 7, 2009; 101(4): 1012 - 1018. [Abstract] [Full Text] [PDF]

				L. Carlini, G. Bellocchi, and M. Donatelli A Library to Generate Synthetic Precipitation Data Agron. J., September 5, 2006; 98(5): 1312 - 1317. [Abstract] [Full Text] [PDF]

				M. Donatelli, M. Acutis, G. Bellocchi, and G. Fila New Indices to Quantify Patterns of Residuals Produced by Model Estimates Agron. J., May 1, 2004; 96(3): 631 - 645. [Abstract] [Full Text] [PDF]

This Article Abstract Figures Only Full Text (PDF) Free Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in Web of Science Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Web of Science (17) Citing Articles via Google Scholar Google Scholar Articles by Fila, G. Articles by Acutis, M. Search for Related Content PubMed Articles by Fila, G. Articles by Acutis, M. Agricola Articles by Fila, G. Articles by Acutis, M. Related Collections Software