Agronomy Journal Grow Your Career With ASA
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
 QUICK SEARCH:   [advanced]


     


This Article
Right arrow Abstract Freely available
Right arrow Figures Only
Right arrow Full Text (PDF) Free
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in Web of Science
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Web of Science (17)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Carbone, G. J.
Right arrow Articles by Stooksbury, D.
Right arrow Search for Related Content
PubMed
Right arrow Articles by Carbone, G. J.
Right arrow Articles by Stooksbury, D.
Agricola
Right arrow Articles by Carbone, G. J.
Right arrow Articles by Stooksbury, D.
Related Collections
Right arrow Agroclimatology
Right arrow Crop Growth and Development
Right arrow Global Change
Right arrow Crop Models
Right arrow Plant and Environment Interactions
Right arrow Site-Specific Analysis
Agronomy Journal 95:537-544 (2003)
© 2003 American Society of Agronomy

MODELING

Evaluating CROPGRO-Soybean Performance for Use in Climate Impact Studies

Gregory J. Carbone*,a, Linda O. Mearnsb, Theodoros Mavromatisc, E. John Sadlerd and David Stooksburye

a Dep. of Geogr., Univ. of South Carolina, Columbia, SC 29208
b Natl. Cent. for Atmos. Res., P.O. Box 3000, Boulder, CO 80303
c Dep. of Agric. and Biol. Eng., Univ. of Florida, Gainesville, FL 32611
d USDA-ARS, Coastal Plains Soil, Water, and Plant Res. Cent., 2611 W. Lucas St., Florence, SC 29501
e Dep. of Biol. and Agric. Eng., Univ. of Georgia, Athens, GA 30602

* Corresponding author (greg.carbone{at}sc.edu)

Received for publication September 17, 2001.

    ABSTRACT
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
Researchers frequently use crop simulation models to estimate the impacts of climate change on agricultural production. While most models used for this purpose have been validated thoroughly at the research plot level, few studies have evaluated them for multiple years and sites and with inputs commonly used in climate impact studies. Here, we examine how well CROPGRO-Soybean performs across space and time using cultivar coefficients provided with the model, estimates of solar radiation, and soil inputs that were estimated from readily available soil surveys. Modeled yield for three maturity groups was compared with that observed from 8 to 23 yr at eight agricultural experiment stations in the southeastern United States. The model was evaluated with respect to its ability to replicate the mean and standard deviation of observed yield. The mean squared deviation (MSD), weighted according to the number of years at each station, was 0.42 (Mg ha-1)2. The model simulated mean yield and the magnitude of interannual yield variability very well. The component of MSD related to the pattern of interannual variability contributed most to MSD. Our results support the use of crop models in studies that require accurate simulation of the temporal mean and variance of yields.

Abbreviations: LCS, lack of correlation weighted by the standard deviations • MSD, mean squared deviation • SB, squared bias • SD, standard deviation • SDSD, squared difference between standard deviations


    INTRODUCTION
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
PROCESS-ORIENTED CROP MODELS have been used extensively in climate impact studies (Rosenzweig and Iglesias, 1994; Hoogenboom et al., 1995; Haskett et al., 1997; Mearns et al., 1999; Alexandrov and Hoogenboom, 2000; Guereña et al., 2001) because they simulate crop response to changing climatic conditions. This confidence results from model validation at the scale of research plots. For example, the model used in this study—CROPGRO-Soybean (Hoogenboom et al., 1994; Boote et al., 1998)—has been shown to accurately predict reproductive development and yield (Brisson et al., 1989; Colson et al., 1995; Sau et al., 1999) at various field locations. Because researchers often use these models to investigate climate impacts, the models also should be evaluated in ways that supplement this field-level validation and more closely resemble the means by which inputs are derived for impact studies. This includes simulation during multiple years, at different locations, and with inputs that are not measured directly in the field, but estimated from other published or unpublished data sources. Such inputs include solar radiation, cultivar coefficients, soil wilting point, field capacity, and saturation levels.

Most studies estimating how climate change influences agriculture are conducted at regional or continental scales. At these scales, practical considerations limit generation of the spatially detailed weather, cultivar, soil, and management inputs demanded by many crop models. Therefore, researchers often must characterize regional conditions without access to detailed physical samples (e.g., Curry et al., 1995; Alexandrov and Hoogenboom, 2000). There is a need to evaluate crop models using a methodology similar to that used in many climate impact studies (e.g., Mearns et al., 1999, 2001; Brown et al., 2000; Easterling et al., 2001).

Yield trials from state agricultural experiment stations provide an appropriate data source to examine general model performance during many consecutive years (Mavromatis et al., 2001, 2002). While data limitations preclude intensive model evaluation at the experiment stations, the relatively long time series of yield is particularly relevant to most research on agricultural impacts where climate change scenarios are developed by adjusting observed meteorological time series and preserving interannual variability (Acock and Acock, 1991). Evaluation of the magnitude and pattern of deviations between simulated and observed yield for multiple years and stations can measure, under relatively controlled conditions, how uncertainties inherent in model inputs affect simulated yield estimates (Bouman, 1994; Moen et al., 1994; Nonhebel, 1994; Hansen and Jones, 2000). Moreover, crop model simulation for multiple years allows evaluation of interannual variability, an important quality for climate impact research (Aggarwal, 1995; Mearns et al., 1996, 1997; Kiniry et al., 1997).


    MATERIALS AND METHODS
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
CROPGRO-Soybean and Data Inputs
CROPGRO-Soybean v. 3.5 is a process-oriented model that simulates C, water, and N balances for the soybean [Glycine max (L.) Merr.] plant and soil (Hoogenboom et al., 1994; Boote et al., 1998). Model equations express relationships between plant processes (including phenological development, photosynthesis, respiration, biomass growth and partitioning, and soil–plant water balance) and daily temperature, photoperiod, insolation, and water or N stress. Growth is integrated at a daily time step, ultimately predicting biomass, leaf area, seed dry weight, and yield. Cultivar coefficients determine duration within growth stages, response to daylength, maximum growth rates, and seed characteristics. Like many similar crop models, CROPGRO-Soybean does not explicitly treat weeds, diseases, pests, wind damage, or other extreme weather events although efforts have been made to address some of these issues (Batchelor et al., 1993).

The model requires daily weather inputs, soil parameters at various profile depths, and cultural or management details. In this study, direct field measurements from eight agricultural experiment stations in Alabama, Georgia, Louisiana, and South Carolina, USA (Fig. 1) , were used as inputs whenever possible. For example, daily maximum and minimum air temperatures and precipitation were obtained from the cooperative weather stations associated with each experiment station. In contrast, solar radiation was measured at only one station (Tifton, GA) and had to be generated stochastically with WXGEN (Richardson and Nicks, 1990) elsewhere.



View larger version (12K):
[in this window]
[in a new window]
 
Fig. 1. Locations of eight agricultural experiment stations used in analysis.

 
Soil type was published for each experiment station, and descriptions of each soil used in this study are available from USDA (Soil Survey Div., 2001). Detailed texture characteristics and organic matter and compaction data for each soil were derived from the STATSGO soils database (USDA-NRCS, 2001). These characteristics were used to estimate saturation point and drained upper and lower limit (Baumer and Rice, 1988; Nat. Resour. Conserv. Serv., 1988; Baumer et al., 1994). The method estimates volumetric soil water content for a range of matric potentials and then fits a soil water retention curve using the RETC program (van Genuchten and Nielsen, 1985) to describe hydraulic properties of unsaturated soils. While such estimation methods produce results that vary from field-measured values (Gijsman et al., 2002), they are used frequently because field-measured wilting point, field capacity, and saturation are not available across broad regions.

We used agricultural experiment station performance reports for crop and management inputs, including row spacing, plant population, planting date, irrigation and fertilizer application, and cultivar. CROPGRO-Soybean v. 3.5 includes 15 cultivar coefficients that characterize phenology and vegetative and reproductive growth. Published values of these coefficients are available for a variety of individual cultivars as well as generic maturity groups, 000 to X (Tsuji et al., 1994). These latter cultivar coefficients were derived from experiments with several varieties belonging to each particular maturity group. We used published cultivar coefficients for two specific cultivars—Forrest (maturity group V) and Ransom (maturity group VII)—at the three Alabama stations where these varieties were consistently grown during a 20-yr period. At the other stations, we used cultivar coefficients for generic maturity groups V, VI, and VII. We did not tune the coefficients for local conditions using observed data. While this could affect model performance, our goal was to measure how well the model performs using data typically available for a climate impact study.

For all simulations, the CROPGRO model was run with water balance, soil N balance, N fixation, and plant N balance options turned on. We ran the crop model for 4 yr before the first growing season to estimate initial soil water and N amounts. We ran the model in sequence mode through consecutive years to preserve soil moisture values from one season to the next. Simulation length varied according to data availability at each station (Table 1).


View this table:
[in this window]
[in a new window]
 
Table 1. General simulation features at the eight agricultural experiment stations. The National Weather Service Cooperative Network Station Number is provided for each site.

 
Evaluating Model Performance
Agricultural experiment station performance reports also provided observed crop yield data. We examined model performance using MSD, the index of agreement (d), and coefficient of determination (r2) (Willmott, 1982). These measures were applied to individual stations but were also aggregated by maturity group to summarize model performance across space and time. We also employed the methods of Kobayashi and Salam (2000) to decompose MSD for individual stations. In their method, MSD consists of three parts: MSD = SB + SDSD + LCS. These three components characterize (i) model squared bias (SB), or the squared difference between average simulated and average observed yield; (ii) the magnitude of variability (SDSD), measured as the squared difference between simulated and observed standard deviations (SDs); and (iii) the pattern of variability between observed and simulated yield (LCS), which measures the lack of correlation weighted by the SDs of observed and simulated yield.

These measures allowed us to examine the relative contribution to MSDs between modeled and observed yield (adjusted to account for moisture content). Our analysis included examination of overall model performance at individual stations and for specific varieties or maturity groups. When summarizing for a particular maturity group, we weighted results by the number of seasons of each contributing station. We compared differences between mean observed and mean simulated yield using t tests. The experiment station reports also provided qualitative records of seasonal conditions that help to describe disease and insect problems, lodging, and plant stress that results from nutrient deficiency, water stress, or extreme weather conditions.


    RESULTS
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
Maturity Group V
The weighted mean simulated yield for maturity group V across all stations and years was 19% higher than mean observed yield. Simulated yield was significantly higher ({alpha} = 0.05) overall and at two individual stations—Fairhope and Marion Junction. The SD was slightly higher for simulated yield than observed yield at each station while the coefficient of variation for simulated and observed yield was about the same (Table 2). The MSD ranged from 0.36 (Mg ha-1)2 at Tifton to 0.96 (Mg ha-1)2 at Marion Junction (Fig. 2) . The LCS component was the largest fraction of MSD at most stations, indicating that the model captured mean yield and the magnitude of interannual yield variation better than the pattern of interannual yield. Closer examination reveals that a few anomalous years disproportionately inflate the MSD. At Crossville, for example, heavy rains in 1972 delayed harvest until 16 Jan. 1973 (Thurlow, 1973). In 1975 and 1979, severe lodging reduced harvestable yield (Fig. 3a ; Thurlow, 1976; Thurlow and Thomason, 1980). At Marion Junction, the model also captured mean yield and the magnitude of variability well but did not reproduce the pattern of interannual variability as well. The results at this station were strongly influenced by large overestimation during 1977, 1986, and 1987 (Fig. 3b) when severe Fe chlorosis reduced yield (Thurlow, 1978; Thurlow and Johnson, 1987, 1988). Such nutrient deficiencies are not simulated by CROPGRO-Soybean; all factors not explicitly accounted for are considered nonlimiting. When these years are eliminated, MSD drops significantly (Table 3).


View this table:
[in this window]
[in a new window]
 
Table 2. Error statistics for maturity group V and Forrest cultivar (SD, standard deviation; CV, coefficient of variation; r2, coefficient of determination; d, index of agreement).

 


View larger version (29K):
[in this window]
[in a new window]
 
Fig. 2. Components of mean squared deviation for maturity group V and Forrest cultivar. LCS, lack of correlation weighted by the standard deviations; SDSD, squared difference between standard deviations; SB, squared bias.

 



View larger version (34K):
[in this window]
[in a new window]
 
Fig. 3. Time series of simulated and observed yield for Forrest cultivar at (a) Crossville, AL, and (b) Marion Junction, AL.

 

View this table:
[in this window]
[in a new window]
 
Table 3. The influence of anomalous years on mean squared deviation (MSD) and its components—squared bias (SB), lack of correlation weighted by the standard deviations (LCS), and squared difference between standard deviations standardized bias (SDSD).

 
Maturity Group VI
The weighted mean simulated yield when all maturity group VI varieties were pooled was 6% higher than observed; this difference was not significant at the {alpha} = 0.05 level. The difference between simulated and observed yield also was not significant at any individual station (Table 4). The SD of simulated yield was higher than that observed in Blackville, but at other stations, simulated and observed SD values were nearly identical; the coefficient of variability for simulated and observed yields was about the same at all stations. Overall model performance at the four stations was very good. The MSD values ranged from 0.16 (Mg ha-1)2 at Jeanerette, LA, to 0.33 (Mg ha-1)2 at Alexandria, LA (Fig. 4) . The mean and variance of 10 yr of simulated yield closely approximated observed values at Jeanerette. As with many stations, MSD at this site came largely from differences in the pattern of variation (LCS). At Jeanerette, this component was accentuated by overestimation of about 25% during 2 yr (1986 and 1989; Fig. 5) .


View this table:
[in this window]
[in a new window]
 
Table 4. Error statistics for maturity group VI (SD, standard deviation; CV, coefficient of variation; r2, coefficient of determination; d, index of agreement).

 


View larger version (23K):
[in this window]
[in a new window]
 
Fig. 4. Components of mean squared deviation for maturity group VI. LCS, lack of correlation weighted by the standard deviations; SDSD, squared difference between standard deviations; SB, squared bias.

 


View larger version (13K):
[in this window]
[in a new window]
 
Fig. 5. Time series of simulated and observed yield for maturity group VI at Jeanerette, LA.

 
Maturity Group VII
The weighted mean simulated yield for maturity group VII was about 13% higher than observed (Table 5). While this difference is significant at the 0.01 level, Fairhope was the only individual station with a significant difference ({alpha} = 0.01) between observed and simulated yield. The SD of simulated yield was consistently higher than observed yield, but the coefficient of variation of simulated and observed yield was nearly identical at five of the eight stations examined. The MSD ranged from 0.33 (Mg ha-1)2 at Tifton, GA, to 0.81 (Mg ha-1)2 at Marion Junction, AL (Fig. 6) . Two stations illustrate different contributions to such errors. At St. Joseph, LA, the model replicated the mean yield and the magnitude of yield SD well, but significant discrepancies in 1985 (overestimation) and 1986 (underestimation) influenced the pattern of interannual variability (Fig. 7a) . Consequently, the 0.44 (Mg ha-1)2 MSD came entirely from the LCS component and resulted from poor estimates in only 2 yr; during the other 7 yr, the model performed very well. At Fairhope, AL, the model captured the magnitude and pattern of interannual variability quite well, but high overestimation during individual years contributed to a large SB. The single highest difference between simulated and observed yield occurred in 1979 when Hurricane Frederic caused large yield reductions (Fig. 7b; Thurlow and Thomason, 1980).


View this table:
[in this window]
[in a new window]
 
Table 5. Error statistics for maturity group VII and Ransom cultivar (SD, standard deviation; CV, coefficient of variation; r2, coefficient of determination; d, index of agreement).

 


View larger version (33K):
[in this window]
[in a new window]
 
Fig. 6. Components of mean squared deviation for maturity group VII and Ransom cultivar. LCS, lack of correlation weighted by the standard deviations; SDSD, squared difference between standard deviations; SB, squared bias.

 



View larger version (30K):
[in this window]
[in a new window]
 
Fig. 7. Time series of simulated and observed yield for (a) maturity group VII at St. Joseph, LA, and (b) Ransom cultivar at Fairhope, AL.

 

    DISCUSSION
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
The overall weighted average MSD for all stations, cultivars, and years was 0.47 (Mg ha-1)2, the sum of the following components: SB = 0.09, SDSD = 0.03, and LCS = 0.35. The total MSD value should be viewed in the context of previous studies evaluating the performance of CROPGRO-Soybean. Jones and Ritchie (1990) reported relatively small model deviation from observed yield measurements [about 0.2 (Mg ha-1)2] using field-measured inputs. In their experiments evaluating model response to planting date, Egli and Bruening (1992) had an MSD value of about 0.25 (Mg ha-1)2. They too used inputs measured directly in the field, such as solar radiation and soil water-holding characteristics. Mavromatis et al. (2001) reported an MSD value of 0.59 (Mg ha-1)2 for the ‘Stonewall’, maturity group VII cultivar. They reduced the MSD for this cultivar to 0.163 (Mg ha-1)2 by systematically adjusting cultivar coefficients and soil parameters to minimize the squared difference between simulated and observed yield. In a subsequent paper, Mavromatis et al. (2002) used the same optimization scheme and reported average MSD values of 0.15 (Mg ha-1)2 and 0.24 (Mg ha-1)2 in North Carolina and Georgia, respectively. Colson et al. (1995) found higher MSD values, ranging from 0.79 to 1.28 (Mg ha-1)2. Their study did not tune cultivar coefficients but used other direct field measurements.

Our MSD values are slightly higher than those of previous work that used direct solar radiation or soil moisture measurement or adjusted soil parameters or genetic coefficients to minimize observed vs. simulated yield. Of course, these detailed validation studies are required for model improvement. So too is evaluation of crop models across broad regions and over multiple years using inputs that are not directly measured because this is the context within which many climate impact studies use these important tools.

The MSD is sensitive to large individual differences between observed and simulated yield, and our overall results reflect such sensitivity. In most of the years where we found large differences between observed and simulated yield, extreme weather events or other factors (e.g., lodging, pests, soil nutrient deficiencies, or disease) compromised model performance. CROPGRO-Soybean does not account for such factors that reduce observed yield. This leads to overprediction, which inflates the SB component of the MSD. Overestimation at Fairhope, where yield was reduced because of hurricane damage during several years, provides an obvious example. Extreme events also influence the LCS component by affecting the pattern of interannual variability. The example for maturity group VII at St. Joseph demonstrates that while mean yield and the magnitude of interannual variability of simulated and observed yield match well, poor model performance during only 1 or 2 yr leads to a higher LCS, inflating MSD (Fig. 7a).

CROPGRO-Soybean performed well under a wide range of environmental conditions, despite the use of several inputs that were not measured directly in the field. The pattern of interannual variability, which was largely influenced by poor model simulation during individual years, contributed most significantly to MSDs between simulated and observed yield. In many of the individual years when the model overestimated yield, observed values were influenced by pests, lodging, soil nutrient deficiencies, or extreme weather events. Because most crop models currently do not consider these effects completely, it is unreasonable to expect that the models will perform well in every individual year. When such years are eliminated, MSD drops significantly.

Model deviations also should be viewed within the context of specific applications. Changes in mean yield and the magnitude of yield variance often serve as inputs to agricultural economics models (Lambert et al., 1995; McCarl et al., 2000) used in long-term climate change studies. Some have argued that accurate prediction of the range of interannual yield variability is more important than estimating absolute yield values in an individual year because year-to-year variability has greater potential economic impact (Moen et al., 1994). While Haskett et al. (1995) found poor performance during individual years, they argued that an ability to successfully simulate soybean mean yield and long-term variability justifies model use for climate change studies. The results presented here suggest that CROPGRO-Soybean meets this criterion.


    ACKNOWLEDGMENTS
 
The authors gratefully acknowledge Larry McDaniel and Jennifer Rainman for data preparation and graphics assistance and Dr. Gerrit Hoogenboom, Dr. William Payne, Karen Beidel, and three anonymous reviewers for their comments on the first draft of this paper. This research was supported by the U.S. Environmental Protection Agency, NCERQA (Grant no. R824997-01-C); the National Aeronautics and Space Administration, MTPE (Grant no. OA99073, W-19, 080); and the U.S. Department of Agriculture, NRICGP (98-35106-6837).


    REFERENCES
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 




This article has been cited by other articles:


Home page
J Exp BotHome page
A. J. Challinor, F. Ewert, S. Arnold, E. Simelton, and E. Fraser
Crops and climate change: progress, trends, and challenges in simulating impacts and informing adaptation
J. Exp. Bot., July 1, 2009; 60(10): 2775 - 2789.
[Abstract] [Full Text] [PDF]


Home page
Agron. J.Home page
W. Xiong, D. Conway, I. Holman, and E. Lin
Evaluation of CERES-Wheat simulation of Wheat Production in China
Agron. J., November 7, 2008; 100(6): 1720 - 1728.
[Abstract] [Full Text] [PDF]


Home page
Agron. J.Home page
J. W. White, G. Hoogenboom, and L. A. Hunt
A Structured Procedure for Assessing How Crop Models Respond to Temperature
Agron. J., March 1, 2005; 97(2): 426 - 439.
[Abstract] [Full Text] [PDF]


This Article
Right arrow Abstract Freely available
Right arrow Figures Only
Right arrow Full Text (PDF) Free
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in Web of Science
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Web of Science (17)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Carbone, G. J.
Right arrow Articles by Stooksbury, D.
Right arrow Search for Related Content
PubMed
Right arrow Articles by Carbone, G. J.
Right arrow Articles by Stooksbury, D.
Agricola
Right arrow Articles by Carbone, G. J.
Right arrow Articles by Stooksbury, D.
Related Collections
Right arrow Agroclimatology
Right arrow Crop Growth and Development
Right arrow Global Change
Right arrow Crop Models
Right arrow Plant and Environment Interactions
Right arrow Site-Specific Analysis


HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
The SCI Journals Crop Science Vadose Zone Journal
Journal of Natural Resources
and Life Sciences Education
Soil Science Society of America Journal
Journal of Plant Registrations Journal of
Environmental Quality
The Plant Genome