HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Figures Only Full Text (PDF) Free Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in Web of Science Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Web of Science (21) Citing Articles via Google Scholar Google Scholar Articles by Bellocchi, G. Articles by Donatelli, M. Search for Related Content PubMed Articles by Bellocchi, G. Articles by Donatelli, M. Agricola Articles by Bellocchi, G. Articles by Donatelli, M. Related Collections Agroclimatology Other Models

MODELING

An Indicator of Solar Radiation Model Performance based on a Fuzzy Expert System

^a Res. Inst. for Ind. Crops, via di Corticella 133, 40128 Bologna, Italy
^b Dep. of Crop Sci., via Celoria 2, 20133 Milan, Italy

* Corresponding author (g.bellocchi{at}isci.it)

Received for publication October 1, 2001.

	ABSTRACT

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION CONCLUSIONS REFERENCES

 
When evaluating models, various indices or test statistics are computed, quantifying the magnitude of model residuals, the correlation between estimates and measurements, patterns of residuals over external variables, etc. Such indices are variously related to each other, thus making model comparison difficult. Problems of this type emerge when testing solar radiation models. This paper proposes a fuzzy expert system to calculate a modular indicator, I_rad, which reflects an expert perception about the quality of the performance of solar radiation models. Three modules were formulated reflecting the magnitude of residuals (Accuracy), the correlation estimates and measurements (Correlation), and the presence or absence of patterns in the residuals against independent variables (Pattern), respectively. The modules Accuracy and Pattern resulted from the aggregation of three (relative root mean square error, modeling efficiency, and t-Student probability) and two (pattern index vs. day of the year and pattern index vs. minimum air temperature) indices, respectively, while the module Correlation was identified by a single index (Pearson's correlation coefficient). For each index, two functions describing membership to the fuzzy subsets Favorable (F) and Unfavorable (U) have been defined. The expert system calculates the modules according to both the degree of membership of the indices to the subsets F and U and a set of decision rules. Then the modules are aggregated into the indicator I_rad. Sensitivity analysis is presented, along with module and I_rad scores for some application cases.

Abbreviations: BC, Bristow–Campbell (model) • CD, Campbell–Donatelli (model) • DB, Donatelli–Bellocchi (model) • EF, efficiency (index) • F, Favorable (subset) • I_rad, indicator of solar radiation model performance • PI, pattern index • PI_doy, pattern index day of year • PI_Tmin, pattern index daily minimum air temperature • RMSE, root mean square error • RRMSE, relative root mean square error • U, Unfavorable (subset)

	INTRODUCTION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION CONCLUSIONS REFERENCES

 
IN THE PROCESS of testing model performance, several indices and test statistics are commonly used (e.g., Smith et al., 1997; Martorana and Bellocchi, 1999; Metselaar, 1999; Yang et al., 2000). Some of them quantify the departure of the model response from experimental measurements (model residuals) while others focus on the correlation between model estimates and measurements. Other indices have been proposed recently to assess systematic behaviors of model residuals against model inputs or other independent variables (Donatelli et al., 2000; Donatelli et al., unpublished, 2002). The interpretation of these statistics is in itself essentially descriptive and primarily based on scientific background, rather than on statistical significance (Willmott, 1982). Some users may appraise the degree of reliability of model outputs simply from the model's graphical display while others may require more in-depth evaluation. Indeed, there is some evidence to suggest that the manner in which a study is performed (i.e., the extent to which the simulation study meets a user's expectations) is more important in forming a user's quality perception than the results of a statistical evaluation (Robinson, 1998).

Each statistic allows only a partial insight into the model performance. Therefore, giving a solid quality judgment about model results requires one to simultaneously consider several statistics. Balancing different aspects involved in model testing, such as departure of estimates from measurements, modeling efficiency (i.e., better fit than the average of measurements), correlation measures, presence or absence of systematic behavior in the residuals, etc., is often complicated by the fact that different statistics may provide contrasting results. Hence, combining several statistics into one aggregated measure is desirable to have a comprehensive assessment of the model response. This would be helpful when judging one model response, evaluating its performance in a variety of conditions, or requesting one to choose the best model out of a list of candidates.

Before aggregating statistics, the user must describe the constraints needed to evaluate the model (Bardossy et al., 1985). Generally, this is an internal process that is not typically well defined or thoroughly documented. Indeed, it varies from individual to individual according to a number of factors, which in fact, influence and characterize each individual (Dubois and Prade, 1980). Personal preferences and intentions may strongly influence a user's judgment. To capture the knowledge and preferences of the user, weights may be used to establish the relative importance of complementary statistics. The weights are determined by the user and may change for different users. These subjective aspects cannot be completely removed from the evaluation process, but they may be proficiently captured and described in mathematical terms (Jones and Barnes, 2000). This could be achieved by aggregating indices by summation, multiplication, or a combination of both. These approaches pose mathematical and conceptual problems (Keeney and Raiffa, 1993) because evaluation statistics differ in their nature, dimensions, and range of possible values.

Considering the inadequacy of such methods, a different approach to evaluate model performance can rely on setting up a fuzzy expert system (Hall and Kandel, 1991) using decision rules (fuzzification). This technique is robust on uncertain and imprecise data such as subjective judgments and allows the aggregation of dissimilar measures in a consistent and reproducible way (Bouchon-Meunier, 1993).

Daily solar radiation is an important meteorological measure for various analyses in many applied sciences, including agricultural engineering, crop physiology, ecology, hydrology, meteorology, physics, and soil science (Brutsaert, 1982; Iqbal, 1983; Tracy et al., 1983). In spite of that, solar radiation is measured infrequently, and the access to reliable radiation data is limited (e.g., Thornton and Running, 1999). As a consequence, the need for accurate estimates of solar radiation exists.

A model for estimating global radiation based on daily temperature extremes was proposed by Bristow and Campbell (1984) using data from three stations in the northwestern USA. Improved models were successively developed by Donatelli and Marletto (1994), Donatelli and Campbell (1998), Bechini et al. (2000), and, more recently, by Donatelli and Bellocchi (2000). Evaluating and comparing the models Bristow–Campbell (BC), Campbell–Donatelli (CD), and Donatelli–Bellocchi (DB) against data sets from a wide range of latitudes was a complex process (Donatelli and Bellocchi, 2000) because of the number of statistical indices used. To overcome such a problem, in this paper, we describe an expert system to calculate an indicator of model performance, I_rad, which reflects an expert perception of the performance of radiation models.

	METHODS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION CONCLUSIONS REFERENCES

 
The Structure of the Indicator
It is assumed here that a comprehensive assessment about radiation model performance should consider: (i) the ability of the model to produce small residuals, (ii) the extent to which the model estimates are correlated with measurements, and (iii) the ability of the model to produce residuals uniformly distributed over the range of two relevant independent variables (day of the year and minimum air temperature).

According to such premises, we defined three indicator modules, named Accuracy, Correlation, and Pattern. The value of each module depends on one or more indices (Table 1) and a set of decision rules. For each module, a dimensionless value between 0 (best model response) and 1 (worst model response) is calculated. The procedure, based on the multivalued fuzzy set theory introduced by Zadeh (1965), follows the so-called Sugeno or Takagi–Sugeno–Kang method of fuzzy inference (Sugeno, 1985). This approach is computationally efficient and well suited for mathematical analysis. It has been applied to a wide variety of problems, such as the design of an indicator for assessing environmental impact of pesticides (van der Werf and Zimmer, 1998) and the development of novel approaches to support decisions regarding sustainable development (Cornelissen et al., 2001). Three membership classes (or subsets) were basically defined for all indices given in Table 1, according to an expert judgment: F, U, and partial (or fuzzy) membership. Several indices were aggregated into modules, and the modules in the final indicator, using fuzzy-based logic rules. The procedure is explained in detail in Appendix A.

[1]

[2]

[3]

where D_i is the difference E_i - M_i, E_i is the ith estimated value, M_i is the ith measured value, n is the number of pairs E_i/M_i, is the average value of all measured values, is the average of all differences D_i, and s_D is the standard error of the differences D_i. The computed t is compared against the critical t with 2 x (n - 1) degrees of freedom.

In place of RMSE, we considered it more appropriate to use a relative measure, the relative root mean square error (RRMSE), where:

[4]

The index RRMSE may vary from 0 to positive infinity. The smaller the RRMSE is, the better the model performance. The RRMSE is a dimensionless index, allowing comparison among different model responses, regardless of units and range of values. Problems could arise because RRMSE tends to become unstable when is close to zero and is undefined when = 0. However, this problem does not occur when computations are made on daily solar radiation data. The limit to the fuzzy subset F for this index was set equal to 20 (RMSE 20 is F) while the limit to the subset U was established equal to 40 (RMSE 40 is U). Both limits come from our expert judgment, working on multiyear weather data sets from about 200 locations (Donatelli and Bellocchi, 2000).

The index EF is very informative because it allows the immediate identification of inefficient models. It is upper-bounded by 1 and can assume negative values (lower-bounded at negative infinity). Negative values of EF indicate that the average value of all measured values is a better predictor than the model used. When estimating daily solar radiation, the limit for the subset U, EF = 0.40, and the limit for the subset F, EF = 0.90, were chosen (EF 0.40 is U and EF 0.90 is F). Again, both limits are derived from our extensive experience, covering radiation estimates over a wide range of latitudes (Donatelli and Bellocchi, 2000).

In testing numerical estimates against measured data, the paired t statistic is used to test the null hypothesis average residual equal to zero at a given probability level (statistical significance). If significant, the t statistic shows that all of the differences between estimated and measured values cannot be attributed to experimental error. If not significant, the t test cannot prove that the estimated and measured values are identical, but it does indicate that there is no statistically significant reason for rejecting the hypothesis that the two outputs represent the same response. When the paired t statistic is used to test the difference between estimated and measured data, the data set of D_i should be normally distributed with a null hypothesis average D_i equal to zero. In our research, all D_i data sets used were characterized by a roughly normal distribution. Generally, low t values indicate a satisfactory response; however, under certain conditions, they may not adequately characterize significant departure of simulation estimates from measured data. These conditions occur with high values of standard error of D_i. In such cases, low t values may be obtained with large departures of estimates from measurements; consequently, the t test may be unsuitable for evaluation purposes. Although cases of this type can theoretically occur in the evaluation process, they were not experienced with the data sets used in the present work. The paired t test is handled here giving its significance level, i.e., P(t). Because P(t) represents the probability of t under random differences between pairs, then the best value for P(t) is 1, with 0 being the worst value. The selection of a particular statistical significance is problematic. Popular values are 0.01, 0.05, and 0.10. The probability level of 0.05 is almost universally accepted as the threshold to reject a null hypothesis; therefore, P(t) = 0.05 was set as limit for U. Some scientists have used a higher level of statistical significance, e.g., 0.10 (Kedzie, 1997; Kleijnen et al., 1998; CTAP, 2000) because they were concerned about the lack of power in their test. Therefore, we set P(t) = 0.10 as the limit for F, attributing a response in between P(t) = 0.05 and P(t) = 0.10 to a transition interval.

The value of the module Accuracy was calculated from the input indices according to eight decision rules, as summarized in Table 2. The expert reasoning runs as follows: If all indices are F, the value of the module is 0 (identity of estimates and measurements); if all indices are U, the value of the module is 1. In setting up the decision rules for the other combinations, we had to decide on the relative importance of each index. In our experience, RRMSE and EF assume a relevant importance in the evaluation process; thus, a fairly large weight (0.80) was attributed to the rule when both RRMSE and EF are U. The condition = 0 is a sound requisite but scarcely informative by itself, and it may hide model outputs drifting toward large biases. Thus, the weight is low (0.20) when P(t) only is U.

[5]

where is the average of estimates. The coefficient r may vary from -1 (full negative correlation) to 1 (full positive correlation). The closer r is to 1, the better the model. Besides the indices based on differences, the coefficient of correlation r between estimates and measurements is commonly computed. The use of this index is questioned (e.g., Willmott, 1982) because its value is not related to the accuracy of estimate. However, the index r is a universal measure with multiple interpretations. For instance, Cahan (1987) looks at r as a measure of identity between standardized values. Moreover, the value of r may help recognize the fluctuation of the estimates among the n measurements (Kobayashi and Salam, 2000). For these reasons, the index r is generally still regarded as a useful measure of model performance.

The membership limits attributed here to the correlation coefficient come from the general categorization made by Hinkle et al. (1994), who took correlation coefficients 0.90 as very high correlations (limit for F: r = 0.90) and coefficients 0.70 as moderate and little correlations (limit for U: r = 0.70). Such limits do conform to our expert judgment.

It must be pointed out that statistical significance for correlation coefficients does not always imply practical significance. The limits attributed here are mere descriptors for the practical interpretation of correlation coefficients and do not take into account statistical significance. The latter depends on the number of data points and can be verified by a t test, provided that both estimated and measured series do conform to the assumptions required for the appropriate application of the test.

Given that there is only one index in the module, the computation of Correlation is simplified to two decision rules: If r is F then 0, and if r is U then 1.

The Module Pattern
The module Pattern accounts for two relevant independent variables in radiation models, day of year and the daily minimum air temperature. For the computation of pattern indices (PIs), the range of such independent variables is divided into four subranges (quartiles), thus producing four groups of residuals. Pattern indices are based on the pairwise differences between average residuals of each quartile (Donatelli et al., 2000):

[6]

where R is the model residual, l and m indicate two groups being compared, q_l and q_m represent the number of residuals in the groups, i_l and i_m identify each value of residuals in the groups. The PI values have the same units as the variable under study (in this case, MJ m^-2 d^-1).

The PIs are targeted at pointing out macropatterns in the residuals (Donatelli et al., unpublished, 2002). The presence of patterns usually means that the residuals contain structure that is not accounted for in the model. When applied to different types of residual plots, PIs may provide meaningful information on the adequacy of different aspects of the model, such as lack of inputs, poor parameterization, etc.; therefore, they should integrate difference- and correlation-based indices when evaluating model performance.

We refer here to PIs computed against day of year and daily minimum air temperature because daily radiation model residuals often show nonrandom distribution of residuals over the range of such variables, and some of the models commonly used for estimating radiation include parameters specifically devoted to account for patterns of this type.

The limits attributed to both PIs reflect the authors' experience. Values of PI are considered F when 1.0 MJ m^-2 d^-1 and U when 2.5 MJ m^-2 d^-1. Examples of PIs computed with different models at tropical (Patos de Minas, Brazil) and temperate (Würzburg, Germany) sites in selected years are shown in Fig. 1 and Fig. 2 , respectively.

View larger version (35K): [in this window] [in a new window]   Fig. 1. Examples of pattern indices (PIs) vs. two independent variables [day of the year (doy) and minimum air temperature (Tmin)] computed on residuals generated at Patos de Minas (Brazil) in 1997 by different radiation models [Bristow–Campbell (BC), Campbell–Donatelli (CD), and Donatelli–Bellocchi (DB)].

View larger version (33K): [in this window] [in a new window]   Fig. 2. Examples of pattern indices (PIs) vs. two independent variables [day of the year (doy) and minimum air temperature (Tmin)] computed on residuals generated at Würzburg (Germany) in 1998 by different radiation models [Bristow–Campbell (BC), Campbell–Donatelli (CD), and Donatelli–Bellocchi (DB)].

The value of the indicator I_rad depends on the modules Accuracy, Correlation, and Pattern, according to a set of eight decision rules (Table 4). The definition of the limits of the transition interval is the same for the three modules: We assigned complete membership to F if the value of the module was 0 (i.e., identity of estimates and measurements, unit correlation, no pattern of residuals vs. independent variables) and complete membership to U if the value of the module was 1. In setting up the other decision rules, we had to establish the relative importance of each module. As a general rule in model evaluation, more emphasis is given to the amount of residuals, whereas the correlation of estimates vs. measurements carries less weight. Because of their recent development, PIs are rarely used in model evaluation. Based on our experience, decreasing importance was given to the modules Accuracy, Pattern, and Correlation, respectively. Thus, for instance, if Accuracy is U, then the weight is 0.55. If both Accuracy and Pattern are U, then the expert conclusion is 0.85. If both Accuracy and Correlation are U, then the conclusion is 0.70.

	RESULTS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION CONCLUSIONS REFERENCES

View larger version (22K): [in this window] [in a new window]   Fig. 3. Sensitivity analysis of the indicator Irad to variation of all input indices. Each input index is varied over its transition interval from 0.0 [completely Favorable (F)] to 1.0 (completely Unfavorable (U)] while the other input indices are kept at U (top graph), at the median value of their transition interval (middle graph), or at F (bottom graph). The traces of the pattern index day of year (PIdoy) and pattern index daily minimum air temperature (PITmin) are superimposed. RRMSE, relative root mean square error; EF, efficiency.

The RRMSE tends to be more influential than the PIs when all other indices are U (Fig. 3, top) as a consequence of the different weight attributed to these indices when they are F and the rest are U in the respective module (0.6 and 0.5, respectively). The opposite occurs when all other indices are kept F (Fig. 3, bottom).

The influence of r when other inputs are U is somewhat large (Fig. 3, top) due to the noticeable weight (0.85) attributed to the rule when the module Correlation is F and both other modules are U. Correlation coefficient and PIs exert the same incidence on the indicator (Table 5); thus, their curves are somewhat complementary (Fig. 3, top, middle, and bottom).

Example Applications
Another illustration of the functioning of the system is given by the computation of either the basic indices or the three modules and the I_rad indicator over yearly sets of radiation data. Selected locations and years with dissimilar response in the various evaluation indices were used for the computations (Table 6).

Calibrated model parameters for the locations and years selected are given in Table 7. The results are reported in Table 8 (evaluation indices) and Table 9 (modules and indicator). The outputs of Table 9 can be used to rank different radiation models with respect to the value of one or more modules or of the I_rad indicator.

The results obtained here are again the consequence of the choices we made regarding the selection of input variables, the definition of their transition intervals, and the values given to the conclusions of the decision rules. These are illustrative results only and are not meant as conclusive findings on the ability of each model at specific sites. The investigation should be extended over multiyear data sets before providing final results. With large, multiyear data sets, a statistically based investigation could be performed. In particular, a comprehensive assessment of the model performance would include the quantification of the variability associated to I_rad, the significance of the computed I_rad scores, and the correlation between I_rad scores and the respective modules. This would allow statistical separation of I_rad scores. Another option would involve cluster analysis to discriminate about model performance over a large amount of locations (or subsets of them), using both the I_rad values and those of the modules Accuracy, Correlation, and Pattern.

	DISCUSSION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION CONCLUSIONS REFERENCES

 
In the design of a system to assess radiation model performance, two major questions have to be answered: (i) Which input index should be taken into account and (ii) how should the basic indices be aggregated? The method presented in this paper proposes an answer to both questions, but its relevance lies primarily in the answer it provides to the second question. The approach contains two key elements: the use of a fuzzy set and the use of decision rules. The use of a fuzzy set provides a well-designed solution to the problem of deciding the cutoff values for input indices: e.g., the limits among a F, U, and transition response. The use of decision rules provides a rational aggregation of input indices in the related module: e.g., RRMSE, EF, and P(t) in the module Accuracy. The combination of these two concepts (limits in the response and mode of aggregation) in sets of fuzzy rules is attractive because although the combinations of values of input indices are infinite, a single set of fuzzy rules connects them all.

In this application, the system is based on a compromise between operational suitability (the evaluation of radiation model performance) and flexibility (hierarchization of objectives and aggregation of preferences). It requires extended corroboration, considering that the objective of an expert system is the simulation of a human expert. The expert system is corroborated if it displays, under a variety of conditions, the same responses that a human expert would display. Experts are therefore invited to comment on the setup and results of this system. If there is disagreement between expert perception of radiation model performance and the input of the system, the cause of this divergence will be examined in view of: (i) choice of input indices, (ii) choice of the limits of the transition interval, (iii) formulation of the decision rules, and (iv) formulation of the mode of aggregation of the modules. All of these points may be modified according to expert consensus, after an extensive testing of the methodology. This process may be relatively time demanding to perform on large data sets. For this purpose, the software IRENE (Fila et al., 2001; Fila et al., unpublished, 2002), which supplies provisions for fuzzy-based aggregation, may be of help (free download from the site: http://www.isci.it/tools). IRENE allows the creation of reusable modules and indicators (ASCII files), thus serving as a convenient means to support collaborative work among large, distributed network of scientists involved in creating and aggregating fuzzy-based components.

	CONCLUSIONS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION CONCLUSIONS REFERENCES

 
We propose a fuzzy expert system reflecting our expert judgment of the quality of radiation model performance. Providing usable values of basic evaluation indices, the method allowed for the building of an aggregated indicator with a modular structure. The system takes into account three types of input variables: magnitude of model residuals, correlation between estimated values and measured data, and patterns in the behavior of residuals vs. independent variables. The resulting output can be used as a support to rank or choose among alternative models under a variety of conditions.

Although a wide expert examination is still required for a general consensus about weights and limits applied to indices and aggregated modules, the fuzzy sets suggested here may represent a pragmatic approach toward a satisfactory solution.

This approach to model evaluation is useful for a number of reasons:

1. It allows users to express mathematically individual or collective values and preferences (uncertainty factors).
   
2. It highlights the degree of model failure or goodness associated with each information source (i.e., model accuracy, correlation, and patterns in the residuals).
   
3. It elucidates the degree of reliability of response associated with each alternative model.
   
4. It facilitates structuring of various components of the evaluation process.
   
5. It reduces several sources and levels of information into a single value.
   
6. It allows examination of operational equivalence between several indices and modules.
   

The modular structure also presents advantages. In the first place, users have access to both a synthetic indicator reflecting overall judgment and to each of the modules. This means a transparency of each step and a control opportunity exists for anybody concerned by the process itself. Second, the mode of module aggregation can be changed, and new modules can eventually be added. The multivalue nature of the issue we are dealing with is explicitly stated; the rules are easy to read; and the numerical scores used for their conclusion are easy to tune to match expert opinions.

The method illustrated here is flexible and can be extended to aggregating more indices of different types. The same method could be conveniently applied to state variables other than solar radiation, e.g., evapotranspiration. Further evolution may be the implementation of this system in exhaustive optimization algorithms and its use as a cost function in estimating model parameters.

APPENDIX A:

FUZZY EXPERT SYSTEMS
Zadeh (1965) proposed the use of fuzzy set theory to describe relationships that are best characterized by compliance to a collection of attributes. In setting up the set of decision rules in model evaluation, the attributes are the basic evaluation indices, and the user must decide on the relative importance of each one. At the same time, the user must give the limit values beyond which the index is certainly F or U. With this procedure, three membership classes are created for the index values: F, U, and partial (or fuzzy) membership.

Fuzzy-set theory addresses this type of problem by allowing one to define the degree of membership of an element in a set by means of membership functions (transition functions) that can take any value from the interval [0, 1]. The value 0 represents complete nonmembership; the value 1 represents complete membership; and values in between are used to represent partial membership. For classical or crisp sets, the membership function only takes two values: 0 (nonmembership) and 1 (membership).

The hierarchical structure of this technique is used to aggregate indices into first-level fuzzy indicators (modules) and, next, into a second-level fuzzy indicator. Each objective in the attribute hierarchy is given a weight. This process of aggregation may continue, hypothetically, until a final-level fuzzy indicator is achieved. The indicator developed here is a second-level indicator. For simplicity, in the example below, only first-level aggregation is developed.

The aggregation process is accomplished by combining weighted fuzzy values. According to this approach, we can characterize the shape of the membership function of each input index by the two limits of the transition interval. Fuzzy membership functions may have different shapes, depending on someone's experience or even preference. We used membership functions that are S shaped in the transition interval because they provide smoother variations of the input values than functions that are linear in the transition interval. If x is the value of the index, and lower and upper bound, respectively, the S function is flat at a value of 0 and 1 for x and for x , respectively. Between and , the S function is a quadratic function of x (Liao, 2002):

[A1]

where ß = ( + )/2. Two adjacent fuzzy terms with S-shaped membership functions have 0.5 overlap at the midpoint between the two extremes. Equation A1 can only represent the left-hand side. For the right-hand side, the complement of Eq. [A1] is needed.

For each module, we formulated a set of decision rules attributing values between 0 and 1 to an output variable according to the membership of its input indices to the fuzzy subsets F and U.

The linguistic description of these components is accomplished in the form of fuzzy rules with a relatively simple syntax. In fact, fuzzy-rules inference involves computation of fuzzy rules that are mostly if ... then ... statements. When two indices are aggregated, the principle of the method makes use of the conjunctive operator AND, as formalized by four rules:

[Rule 1]

[Rule 2]

[Rule 3]

[Rule 4]

where x_j (j = 1, 2) is an input index, A_ij is a fuzzy subset, y_i is an output variable, and B_i (i = 1, 2, 3, 4) is a number. x_j is A_ij is called a premise of the ith rule; y_i is B_i is called conclusion (or expert weight) of the ith rule. In this case, the decision rules consist of two premises (if... AND if...) linked by and followed by a conclusion (then...).

The process continues identifying the degree of truth in the premise portion of each rule and then aggregating the truth of linked conditions. Let ‘x₁ and ‘x₂ be the values taken by x₁ and x₂ and A_ij(‘x_j) be the membership value of ‘x_j to the fuzzy set A_ij (given by the membership function that defines A_ij). According to Sugeno’s inference method, when the premises are linked by a conclusion, the truth value of a decision rule is defined as the smallest of the truth values of its premises. Therefore, a fuzzy subset is assigned to each output variable for each rule using min aggregator where min means minimum value of. One can define w₁, w₂, w_3, and w_4, the truth values of the rules, as follows:

[truth value of Rule 1]

[truth value of Rule 2]

[truth value of Rule 3]

[truth value of Rule 4]

The first rule infers w₁ x B₁, the second one w₂ x B₂, and so on. The fuzzy sets that represent the outputs of each rule are combined by summation into a single fuzzy solution set (‘y₀):

A solution of this type is sometimes known as singleton output membership function, and it can be thought of as a predefuzzified fuzzy set. The global output ‘y is inferred by:

This last operation (the weighted average of a few data points) is a common method (centroid calculation) adopted to reduce the final fuzzy set to a crisp value (defuzzification) in the Sugeno-type systems.

The computation of the aggregated indicator is primarily influenced by the truth value of each rule (w_i), for which the expert weight (B_i) is a multiplication factor. As a result of this insight, it should be clear that weights on rules are not simply measures of the relative importance of each rule. They are essentially measures of the importance of the increase from the worst to the best level of performance on one rule. The B_i terms are subjective elements introduced in the process of aggregation, which the global output may be sensitive to. This means that the quality of outputs is influenced by the weights of the individual rules. This crucial issue should be investigated within the context of interest, which the truth values depend on.

To illustrate Sugeno's inference method, a numerical example will be given for the composition of the module Pattern, obtained by aggregating two PIs, PI_doy and PI_Tmin (see details in "The Structure of the Indicator" section). We gave three classes of model response with respect to the presence of patterns in the residuals against one independent variable (Fig. 4A) . We gave a response classified as pattern (PI_doy, PI_Tmin 2.5 MJ m^-2 d^-1) a membership value of 1 for the fuzzy subset U and a membership value of 0 for the fuzzy subset F. Model response classified as no pattern (PI_doy, PI_Tmin 1.0 MJ m^-2 d^-1) is given a membership value of 0 for the fuzzy subset U and a membership value of 1 for the fuzzy subset F (Fig. 4B). The class of borderline values (1.0 MJ m^-2 d^-1 < PI_doy, PI_Tmin < 2.5 MJ m^-2 d^-1) falls within a transition interval in which the membership value for U increases from 0 (at PI_doy, PI_Tmin = 1.0 MJ m^-2 d^-1) to 1 (at PI_doy, PI_Tmin = 2.5 MJ m^-2 d^-1) and the membership value for F decreases from 1 to 0 (thus the functions characterizing F and U are complementary). In our example, the reasoning for the four rules is:

[Rule 1]

[Rule 2]

[Rule 3]

[Rule 4]

View larger version (21K): [in this window] [in a new window]   Fig. 4. Graphical presentation of (A) crisp and (B) fuzzy sets for the pattern index.

Let's assume that PI_doy was computed equal to 2.00 MJ m^-2 d^-1 and PI_Tmin equal to 1.70 MJ m^-2 d^-1. For both indices, membership to fuzzy subsets F and U has to be defined. The membership functions of Fig. 5 allow the calculation of the truth value of the premises, i.e., the degree of membership to the fuzzy subset concerned for PI_doy (Fig. 5, top) and PI_Tmin (Fig. 5, bottom). Results are shown in Table 10. The value of Pattern is calculated as the sum of the conclusions of the decision rules, weighted by the sum of their truth values:

View larger version (24K): [in this window] [in a new window]   Fig. 5. Membership to the fuzzy sets Favorable (F) and Unfavorable (U) for a hypothetical model response in terms of pattern index day of year (PIdoy) and pattern index daily minimum air temperature (PITmin). PIdoy = 2.00 and PITmin = 1.70.

DAILY SOLAR RADIATION MODELS
The general approach followed by the models used in this study for estimating solar radiation at ground level (Rad, MJ m^-2 d^-1) is of the form:

[B1]

where tt_i is the atmospheric transmissivity and R_a is the extraterrestrial radiation (that is, the calculated solar radiation outside the earth's atmosphere).

Based purely on solar geometry and the solar constant, an ordinary routine for estimating daily R_a at given latitudes (e.g., Swift, 1976; Bristow and Campbell, 1984; Campbell and Norman, 1998) was used.

As regards tt_i, three models to estimate it are used:

Model BC:

[B2]

Model CD:

[B3]

Model DB:

[B4]

where all temperatures are in °C and

The parameters b and c display the physics involved in the relationship and determine the rate of increase of the exponential function as T increases. The parameter c is usually kept equal to 2 (Ndlovu, 1994; Donatelli, 1995; Weiss et al., 2001). The parameters Tnc, c1, and c2 are empirical parameters accounting for seasonal effects.

	ACKNOWLEDGMENTS

 
The structure of the indicator was basically inspired by the fuzzy-based indicator of pesticide environment impact by van der Werf and Zimmer (1998), adapted with novel ideas and concepts to conform to the purpose of the paper. The meteorological data sets used in this study are part of a large database provided from several partners and used in previous studies. The list of people and institutions who contributed by providing data is too large to be reported here, and it can be found in the Acknowledgments section of the manual of the software RadEst3.00. Research conducted under the auspices of the Italian Ministry of Agricultural and Forestry Policies, project SIPEAA (http://www.sipeaa.it), Paper no. 3.

	REFERENCES

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION CONCLUSIONS REFERENCES

 

• Addiscott, T.M., and A.P. Whitmore. 1987. Computer simulation of changes in soil mineral nitrogen and crop nitrogen during autumn, winter and spring. J. Agric. Sci. (Cambridge) 109:141–157.
• Bardossy, A., I. Bogardi, and L. Duckstein. 1985. Composite programming as an extension of compromise programming. p. 375–408. In P. Serfini (ed.) Mathematics of multiple objective optimisation. Springer-Verlag, Wien, Austria.
• Bechini, L., G. Ducco, M. Donatelli, and A. Stein. 2000. Modelling, interpolation and stochastic simulation in space and time of global solar radiation. Agric. Ecosyst. Environ. 81:29–42.
• Bouchon-Meunier, B. 1993. La logique floue. Presses Universitaires de France, Paris.
• Bristow, K.L., and G.S. Campbell. 1984. On the relationship between incoming solar radiation and daily maximum and minimum temperature. Agric. For. Meteorol. 31:159–166.
• Brutsaert, W.H. 1982. Evaporation into the atmosphere: Theory, history and applications. D. Rediel, Boston.
• Cahan, S. 1987. Stability and change in human characteristics. John Wiley & Sons, New York.
• Campbell, G.S., and J.M. Norman. 1998. An introduction to environmental biophysics. 2nd ed. Springer, New York.
• Cornelissen, A.M.G., J. van den Berg, W.J. Koops, M. Grossman, and H.M.J. Udo. 2001. Assessment of the contribution of sustainability indicators to sustainable development: A novel approach using fuzzy set theory. Agric. Ecosyst. Environ. 86:173–185.
• [CTAP] Critical Trends Assessment Project. 2000. Air quality trends in Illinois. p. 91–153. In CTAP (ed.) The changing Illinois environment: Critical trends. Vol. 2. Air resources. Illinois Nat. Resour. Inf., Springfield, IL.
• Donatelli, M. 1995. Sistemi nella gestione integrata delle colture. PANDA project, General Series, Paper 1. ISA, Modena, Italy.
• Donatelli, M., M. Acutis, and G. Bellocchi. 2000. Two statistical indices to quantify patterns of errors produced by models. p. 186. In Proc. Int. Crop Sci. Conf., 3rd, Hamburg, Germany. 17–22 Aug. 2000. Eur. Soc. for Agron., Hamburg.
• Donatelli, M., and G. Bellocchi. 2000. New methods to estimate global solar radiation. p. 186. In Proc. Int. Crop Sci. Conf., 3rd, Hamburg, Germany. 17–22 Aug. 2000. Eur. Soc. for Agron., Hamburg.
• Donatelli, M., and G. Bellocchi. 2001. Estimate of daily global solar radiation: New developments in the software RadEst3.00. p. 213–214. In Proc. Int. Symp. Modelling Cropping Syst., 2nd, Florence, Italy. 16–18 July 2001. Inst. for Biometeorol., CNR, Florence.
• Donatelli, M., and G.S. Campbell. 1998. A simple model to estimate global solar radiation. p. 133–134. In Proc. Eur. Soc. for Agron. Congr., 5th, Vol. 2, Nitra, Slovak Republic. 28 June–2 July 1998. The Slovak Agric. Univ., Nitra, Slovak Republic.
• Donatelli, M., and V. Marletto. 1994. Estimating surface solar radiation by means of air temperature. p. 352–353. In Proc. Eur. Soc. for Agron. Congr., 3rd, Abano-Padova, Italy. 18–22 Sept. 1994. The Padova Univ., Padova, Italy.
• Dubois, D., and H. Prade. 1980. Fuzzy sets and systems: Theory and applications. Academic Press, New York.
• Fila, G., G. Bellocchi, M. Acutis, and M. Donatelli. 2001. IRENE: A software to test model performance. p. 215–216. In Proc. Int. Symp. Modelling Cropping Syst., 2nd, Florence, Italy. 16–18 July 2001. Inst. for Biometeorol., CNR, Florence.
• Fox, D.G. 1981. Judging air quality model performance: A summary of the AMS workshop on dispersion models performance. Bull. Am. Meteorol. Soc. 62:599–609.
• Hall, L.O., and A. Kandel. 1991. The evolution from expert systems to fuzzy expert systems. p. 3–21. In A. Kandel (ed.) Fuzzy expert systems theory. CRC Press, Boca Raton, FL.
• Hinkle, D., W. Wiersma, and S. Jurs. 1994. Applied statistics for the behavioral sciences. 3rd ed. Houghton Mifflin Co., Boston.
• Iqbal, M. 1983. An introduction to solar radiation. Academic Press, New York.
• Jones, D., and E.M. Barnes. 2000. Fuzzy composite programming to combine remote sensing and crop models for decision support in precision crop management. Agric. Syst. 65:137–158.
• Kedzie, C.R. 1997. Quantitative analyses: The empty corner. In C.R. Kedzie (ed.) Communication and democracy: Coincident revolutions and the emergent dictator's dilemma [Online]. Available at: http://www.rand.org/publications/RGSD/RGSD127/sec4.html (accessed 1 Apr. 2002; verified 5 July 2002). RGSD-127. RAND, Santa Monica, CA.
• Keeney, R.L., and H. Raiffa. 1993. Decisions with multiple objectives: Preferences and value tradeoffs. Cambridge Univ. Press, New York.
• Kleijnen, J.P.C., B. Bettonvil, and W. van Groenendaal. 1996. Validation of trace-driven simulation models: A novel regression test. Manage. Sci. 44:812–819.
• Kobayashi, K., and M.U. Salam. 2000. Comparing simulated and measured values using mean squared deviation and its components. Agron. J. 92:345–352.[Abstract/Free Full Text]
• Liao, T.W. 2002. A fuzzy C-medians variant for the generation of fuzzy term sets. Int. J. Intell. Syst. 17:21–43.
• Loague, K., and R.E. Green. 1991. Statistical and graphical methods for evaluating solute transport models: Overview and application. J. Contam. Hydrol. 7:51–73.
• Martorana, F., and G. Bellocchi. 1999. A review of methodologies to evaluate agroecosystem simulation models. Ital. J. Agron. 3:19–39.
• Metselaar, K. 1999. Auditing predictive models: A case study in crop growth. Ph.D. thesis. Wageningen Agric. Univ., Wageningen, the Netherlands.
• Ndlovu, L.S. 1994. Generating weather data for crop simulation model. Ph.D. diss. Biol. Syst. Eng. Dep., Washington State Univ., Pullman.
• Robinson, S. 1998. Service quality in the management of simulation projects. Ph.D. thesis. Lancaster Univ., Lancaster, UK.
• Smith, P., J.U. Smith, D.S. Powlson, W.B. McGill, J.R.M. Arah, O.G. Chertov et al. 1997. A comparison of the performance of nine soil organic matter models using datasets from seven long-term experiments. Geoderma 81:153–225.[Web of Science]
• Sugeno, M. 1985. An introductory survey of fuzzy control. Inf. Sci. (NY) 36:59–83.
• Swift, L.W. 1976. Algorithm for solar radiation estimation on mountain slopes. Water Resour. Res. 12:108–112.
• Thornton, P.E., and S.W. Running. 1999. An improved algorithm for estimating incident daily solar radiation from measurements of temperature, humidity, and precipitation. Agric. For. Meteorol. 93:211–228.
• Tracy, C.R., K.A. Hammond, R.A. Lechleitner, W.J. Smith, II, D.B. Thompson, A.D. Whicker et al. 1983. Estimating clear-day solar radiation: An evaluation of three models. J. Therm. Biol. 8:247–251.
• Van der Werf, H.M.G., and C. Zimmer. 1998. An indicator of pesticide environmental impact based on a fuzzy expert system. Chemosphere 36:2225–2249.[Medline]
• Weiss, A., C.J. Hays, Q. Hu, and W.E. Easterling. 2001. Incorporating bias error in calculating solar irradiance: Implications for crop yield simulations. Agron. J. 93:1321–1326.[Abstract/Free Full Text]
• Willmott, C.J. 1982. Some comments on the evaluation of model performance. Bull. Am. Meteorol. Soc. 63:1309–1313.
• Yang, J., D.J. Greenwood, D.L. Rowell, G.A. Wadsworth, and I.G. Burns. 2000. Statistical methods for evaluating a crop nitrogen simulation model, N_ABLE. Agric. Syst. 64:37–53.
• Zadeh, L.A. 1965. Fuzzy sets. Inf. Control 8:338–353.

This article has been cited by other articles:

				G. Fragoulis, M. Trevisan, A. Di Guardo, A. Sorce, M. van der Meer, F. Weibel, and E. Capri Development of a Management Tool to Indicate the Environmental Impact of Organic Viticulture J. Environ. Qual., March 1, 2009; 38(2): 826 - 835. [Abstract] [Full Text] [PDF]

				M. Donatelli, M. Acutis, G. Bellocchi, and G. Fila New Indices to Quantify Patterns of Residuals Produced by Model Estimates Agron. J., May 1, 2004; 96(3): 631 - 645. [Abstract] [Full Text] [PDF]

				G. Fila, G. Bellocchi, M. Donatelli, and M. Acutis IRENE_DLL: A CLASS LIBRARY FOR EVALUATING NUMERICAL ESTIMATES Agron. J., September 1, 2003; 95(5): 1330 - 1333. [Abstract] [Full Text] [PDF]

This Article Abstract Figures Only Full Text (PDF) Free Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in Web of Science Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Web of Science (21) Citing Articles via Google Scholar Google Scholar Articles by Bellocchi, G. Articles by Donatelli, M. Search for Related Content PubMed Articles by Bellocchi, G. Articles by Donatelli, M. Agricola Articles by Bellocchi, G. Articles by Donatelli, M. Related Collections Agroclimatology Other Models