Combining multiple statistical methods to evaluate the performance of process-based vegetation models across three forest stands

Open access


Process-based vegetation models are crucial tools to better understand biosphere-atmosphere exchanges and ecophysiological responses to climate change. In this contribution the performance of two global dynamic vegetation models, i.e. CARAIB and ISBACC, and one stand-scale forest model, i.e. 4C, was compared to long-term observed net ecosystem carbon exchange (NEE) time series from eddy covariance monitoring stations at three old-grown European beech (Fagus sylvatica L.) forest stands. Residual analysis, wavelet analysis and singular spectrum analysis were used beside conventional scalar statistical measures to assess model performance with the aim of defining future targets for model improvement. We found that the most important errors for all three models occurred at the edges of the observed NEE distribution and the model errors were correlated with environmental variables on a daily scale. These observations point to possible projection issues under more extreme future climate conditions. Recurrent patterns in the residuals over the course of the year were linked to the approach to simulate phenology and physiological evolution during leaf development and senescence. Substantial model errors occurred on the multi-annual time scale, possibly caused by the lack of inclusion of management actions and disturbances. Other crucial processes defined were the forest structure and the vertical light partitioning through the canopy. Further, model errors were shown not to be transmitted from one time scale to another. We proved that models should be evaluated across multiple sites, preferably using multiple evaluation methods, to identify processes that request reconsideration.

Anav, A., Frielingstein, P., Beer, C., Ciais, P., Harper, A., Jones, C. et al., 2015: Spatio-temporal patterns of terrestrial gross primary production. Reviews of Geophysics, 53:785-818.

Aubinet, M., Grelle, A., Ibrom, A., Rannik, U., Moncrieff, J., Foken, T. et al., 2000 : Estimates of the annual net carbon and water exchange of forests: The EUROFLUX methodology, Advances in Ecological Research, 30:113-175.

Aubinet, M., Heinesch, B., Perrin, D., Moureaux, C., 2005: Discriminating net ecosystem exchange between different vegetation plots in a heterogeneous forest. Agricultural and Forest Meteorology, 132: 315-328.

Aubinet, M., Vesala, T., Papale, D. (eds.), 2012: Eddy Covariance: A Practical Guide to Measurement and Data Analysis. Springer Atmospheric Sciences, Dordrecht, Netherlands, 424 p.

Botkin, D., 1993: Forest Dynamics: An Ecological Model. Oxford University Press, Oxford & New York, UK. 309 p.

Braswell, B. H., Sacks, W. J., Linder, E., Schimel, D. S., 2005: Estimating diurnal to annual ecosystem parameters by synthesis of a carbon flux model with eddy covariance net ecosystem exchange observations. Global Change Biology, 11:335-355.

Bugmann, H., Grote, R., Lasch, P., Lindner, M., Suckow, F., 1997: A new forest gap model to study the effects of environmental change on forest structure and functioning. In: Mohren, G. M. J., Kramer, K., Sabate, S. (eds.): Impacts of Global Change of Tree Physiology and Forest Ecosystem. Kluwer Academic Publisher, Dordrecht, The Netherlands, p. 255-261.

Calvet, J. C., Noilhan, J., Roujean, J. L., Bessemoulin, P., Cabelguenne, M., Olioso, A. et al., 1998: An interactive vegetation SVAT model tested against data from six contrasting sites. Agricultural and Forest Meteorology, 92:73-95.

Carrer, D., Roujean, J. L., Lafont, S., Calvet, J. C., Boone, A., Decharme, B. et al., 2013: A canopy radiative transfer scheme with explicit FAPAR for the interactive vegetation model ISBA-A-gs: impact on carbon fluxes. Journal of Geophysical Research: Biogeosciences, 118:888-903.

Chen, C. W., 1993: The response of plants to interacting stresses: PGSM Version 1.3 Model Documentation. EPRI TR-101880. Electric Power Research Institute, Palo Alto, CA, USA, 128 p.

Collatz, G. J., Ball, J. T., Grivet, C., Berry, J. A., 1991: Physiological and environmental regulation of stomatal conductance, photosynthesis and transpiration: a model that includes a laminar boundary layer. Agricultural and Forest Meteorology, 54:107-136.

Crowther, T. W., Todd-Brown, K. E. O., Rowe C. W., Wieder, W. R., Carey, J. C., Machmuller, M. B. et al., 2016: Quantifying global soil carbon losses in response to warming. Nature, 540:104-108.

Davidson, R. L., 1969: Effect of root/leaf temperature differentials on root/shoot ratios in some pasture grasses and clover. Annals of Botany, 33:561-569.

Decharme, B., Boone, A., Delire, C., Noihlan, J., 2011: Local evaluation of the interaction between soil biosphere atmosphere soil multilayer diffusion scheme using four pedotransfer functions. Journal of Geophysical Research: Biogeosciences, 116:1984-2012.

De Pury, D. G., Farquhar, G. D., 1997: Simple scaling of photosynthesis from leaves to canopies without the errors of big-leaf models. Plant, Cell and Environment, 20:537-557.

Dietze, M. C., Vargas, R., Richardson, A. D., Stoy, P. C., Barr, A. G., Anderson, R. S. et al., 2011: Characterizing the performance of ecosystem models across time scales: a spectral analysis of the North American Carbon Program site-level synthesis. Journal of Geophysical Research: Biogeosciences, 116: G04029.

Dury, M., Hambuckers, A., Warnant, P., Henrot, A.-J., Favre, E., Ouberdous, M. et al., 2011: Response of the European forests to climate change: a modelling approach for the 21st century. iForest - Biogeosciences and Forestry, 4:82-99.

Dyck, S., Peschke, G., 1995: Grundlagen der Hydrologie. Verlag für Bauwesen, Berlin, Germany, 536 p.

Farquhar, G. D., von Caemmerer, S., Berry, J. A., 1980: A biochemical model of photosynthetic CO2 assimilation in leaves of C3 species. Planta, 149:78-90.

Fischer, J. B., Huntzinger, D. N., Schwalm, C. R., Sitch, S., 2014: Modeling the terrestrial biosphere. Annual Review of Environment and Resources, 39:91-123.

François, L. M., Delire, C., Warnant, P., Munhoven, G., 1998: Modelling the glacial-interglacial changes in the continental biosphere. Global Planetary Change, 16-17:37-52.

François, L., Utescher, T., Favre, E., Henrot, A.-J., Warnant, P., Micheels, A. et al., 2011: Modelling late Miocene vegetation in Europe: results of the CARAIB model and comparison with palaeo-vegetation data. Palaeogeography, Palaeoclimatology, Palaeoecology, 304:359-378.

Franko, U., 1990: C- und N-Dynamik beim Umsatz organischer Substanz im Boden. Akademie der Landwirtschaftswissenschaften der DDR, Dissertation 2, Berlin, Germany, 140 p.

Gérard, J. C., Nemry, B., François, L. M., Warnant, P., 1999. The interannual change of atmospheric CO2: contribution of subtropical ecosystems? Geophysical Research Letters, 26:243-246.

Gibelin, A. L., Calvet, J. C., Viovy, N., 2008: Modelling energy and CO2 fluxes with an interactive vegetation land surface model, evaluation at high and middle latitudes. Agricultural and Forest Meteorology, 148:1611-1628.

Glugla, G., 1969: Berechnungsverfahren zut Ermittlung des aktuellen Wassergehaltes und Gravitationswasserabflusses im Boden. Albrech-Thaer-Archiv, 13:71-376.

Golyandina, N., Zhigljavsky, A., 2013 (eds.): Singular Spectrum Analysis for Time Series. Springer Briefs in Statistics, Springer, Heidelberg, Germany, 119 p.

Goudriaan, J., van Laar, H. H., van Keulen, H., Louwerse, W., 1985: Photosynthesis, CO2 and plant production. In: Day, W., Atkin, R. K. (eds.). Wheat Growth and Modelling. Plenum Press, New York, USA. NATO AS/ Series, Series A, 86:107-122.

Goudriaan, J., 1986: A simple and fast numerical method for the computation of daily totals of crop photosynthesis. Agricultural and Forest Meteorology, 38:249-254.

Grote, R., Suckow, F., Bellmann, K., 1998: Modelling of carbon-, nitrogen-, and water balances in pine stands under changing air pollution and deposition. In: Hüttl, R. F. and Bellmann, K. (eds.): Changes of Atmospheric Chemistry and Effects on Forest Ecosystems. A Roof Experiment Without Roof. Kluwer Publishers, Dordrecht, The Netherlands, p. 251-281.

Haxeltine, A., Prentice I. C., 1996: A general model for the light-use efficiency of primary production. Functional Ecology, 10:551-561.

Henrot, A. - J. , Utescher, T., Erdei,B., Dury, M., Hamon, N., Ramstein, G. et al., 2017: Middle Miocene climate and vegetation models and their validation with proxy data. Palaeogeography, Palaeoclimatology, Palaeoecology, 467:95-119.

Hickler, T., Vohland, K., Feehan, J., Miller, P. A., Smith, B., Costa, L. et al., 2012: Projecting the future distribution of European potential natural vegetation zones with a generalized, tree species-based dynamic vegetation model. Global Ecology and Biogeography, 21:50-63.

Hickler, T., Amming, A., Werner, C., 2015: Modelling CO2 impacts on forest productivity. Current Forestry Reports, 1:69-80.

Hollinger, D. Y., Richardson, A. D., 2005: Uncertainty in eddy covariance measurements and its application to physiological models. Tree Physiology, 25:873-885.

Jacobs, C. M. J., 1994: Direct impact of atmospheric CO2 enrichment on regional transpiration, PhD thesis, Agricultural University, Wageningen, The Netherlands, 192 p.

Jansson P. - E., 1991: Simulation model for soil water and heat conditions. Description of the SOIL model (vol. 165). Swedish University of Agricultural Sciences, Department of Soil Sciences, Division of Agricultural Hydrotechnics, Uppsala, Sweden, 72 p.

Joetzjer, E., Delire, C., Douville, H., Ciais, P., Decharme, B., Carrer, D. et al., 2015: Improving the ISBACC land surface model simulation of water and carbon fluxes and stocks over the Amazon forest. Geoscientific Model Development, 8:1709-1727.

Jorritsma, I. T. M., van Hees, A. F. M., Mohren G. M. J., 1999: Forest development in relation to ungulate grazing: A modeling approach. Forest Ecology and Management, 120:23-34.

Kartschall, T., Döring, P., Suckow, F., 1989: Simulation of nitrogen, water and temperature dynamics in soil. Systems Analysis Modelling Simulation Journal, 6:117-123.

Keane, R. E., Morgan, P., Running, S. W., 1996: FIRE-BGC - a mechanistic ecological process model for simulating fire succession on coniferous forest landscapes of the northern Rocky Mountains. Research Paper INT-RP-484, United States Department of Agriculture, Forest Service, Intermountain Research Station, Missoula, MT, USA, 122 p.

Keenan, T. F., Baker, I., Barr, A., Ciais, P., Davis, K., Dietze, M. et al., 2012: Terrestrial biosphere model performance for inter-annual variability of landatmoshpere CO2 exchange. Global Change Biology, 18:1971-1987.

Koitzsch, R., 1977: Schätzung der Bodenfeuchte aus meteorologischen Daten, Boden- und Pflanzenparametern mit einem Mehrschichtmodell. Zeitschrift fur Meteorologie, 27:302-306.

Krause, P., Boyle, D. P., Base, F., 2005: Comparison of different efficiency criteria for hydrological model assessment. Advances in Geosciences, 5:89-97.

Landsberg, J. J., Waring, R. H., 1997: A generalised model of forest productivity using simplified concepts of radiation-use efficiency, carbon balance and partitioning. Forest Ecology and Management, 95:209-228.

Le Quéré, C., Andrew, M., Canadell, J. G., Sitch, S., Korsbakken, J. I., Peters, G. P. et al., 2016: Global carbon budget 2016. Earth System Science Data, 8:605-649.

Larocque, G. R., Bhatti, J., Arsenault, A., 2014: Integrated modelling software platform development for effective use of ecosystem models. Ecological Modelling, 288:195-202.

Lasch, P., Badeck, F. - W., Suckow, F., 2005: Modelbased analysis of management alternatives at stand and regional level in Brandenburg (Germany). Forest Ecology and Management, 207:59-74.

Lasch, P., Badeck, F. - W., Suckow, F., Lindner, M., Mohr, P., 2005: Model-based analysis of management alternatives at stand and regional level in Brandenburg (Germany). Forest Ecolgy and Management, 207:59-74.

Lasch-Born, P., Suckow, F., Gutsch, M., Reyer, C., Hauf, Y., Murawski, A. L. et al., 2015: Forests under climate change: potential risks and opportunities. Meteorologische Zeischrift, 24:157-172.

Lau, K. M., Wang, H. - Y., 1995: Climate signal detection using wavelet transform: How to make a time series sing? Bulletin of the American Meteorological Society, 76:2391-2402.

Laurent, J. - M., François, L., Bar-Hen, A., Bel, L., Cheddadi, R., 2008: European bioclimatic affinity groups: data-model comparisons. Global and Planetary Change, 61:28-40.

Law, B. E., Falge, E., Gu, L., Baldocchi, D. D., Bakwin, P., Berbigier, P. et al., 2002: Environmental controls over carbon dioxide and water vapor exchange of terrestrial vegetation. Agricultural and Forest Meterology, 113:97-120.

Li, H., Zhang, F., Li, Y., Wang, J., Zhang, L., Zhao, L. et al., 2016: Seasonal and inter-annual variations in CO2 fluxes over 10 years in an alpine shrubland on the Qinghai-Tibetan plateau, China. Agricultural and Forest Meteorology, 228-229:95-103.

Li, X. R., Zhao, Z., 2006: Evaluation of estimation algorithms, part I: Incomprehensive measures of performances. IEEE Transactions on Aerospace and Electronic Systems, 42:1340-1358.

Lloyd, J., Taylor, J. A., 1994: On the temperature dependence of soil respiration. Functional Ecology, 8:315- 323

Loehle, C., LeBlanc, D., 1996: Model-based assessments of climate change effects on forests: a critical review. Ecological Modelling, 90:1-31.

Mahecha, M. D., Reichstein, M., Lange, H., Carvalhais, N., Bernhofer, C., Grünwald, T. et al., 2007: Characterizing ecosystem-atmosphere interactions from short to interannual time scales. Biogeosciences, 4:743-758.

Mahecha, M. D.,Reichstein, M., Jung, M., Seneviratne, S. I., Zaehle, S., Beer, C. et al., 2010: Comparing observations and process-based simulations of biosphere- atmosphere exchange on multiple time scales. Journal of Geophysical Research: Atmospheres, 101:7111-7125.

Mäkelä, A., 1990: Modeling structural-functional relationships in whole-tree growth: resource allocation. In: Dixon, R. K., Meldahl, R. S., Ruark, G. A., Warren, W. G (eds.): Process Modeling of Forest Growth Responses to Environmental Stress. Timber Press, Portland, Oregon, 81-95.

Masson, V., Le Moigne, P., Martin, E., Faroux, S., Alias, A., Alkama, R. et al., 2013: The SURFEXv7.2 land and ocean surface platform for coupled or offline simulation of Earth surface variables and fluxes. Geoscientific Model Development, 6:929-960.

Medlyn, B. E., Robinson, A. P., Clement, R., McMurtrie, E., 2005: On the validation of models of forest CO2 exchange using eddy covariance data: some perils and pitfalls. Tree Physiology, 25:839-857.

Mintz, Y., Walker, G., 1993 : Global fields of soil moisture and land surface evapotranspiration derived from observed precipitation and surface air temperature. Journal of Applied Meteorology, 32:1305-1334.

Nash, J. E., Sutcliffe, J. V., 1970: River flow forecasting through conceptual models part I - A discussion of principles. Journal of Hydrology, 10:282-290.

Nemry, B., François, L. M., Warnant, P., Robinet, F., Gérard, J. C., 1996 : The seasonality of the CO2 exchange between the atmosphere and the land biosphere: a study with a global mechanistic vegetation model. Journal of Geophysical Research-Atmospheres, 101:7111-7125.

Noilhan, J., Planton, S., 1989: A simple parameterization of land surface processes for meteorological models. Monthly Weather Review, 117:536-549.

Noilhan, J., Mahfouf, J. F., 1996: The ISBA land surface parameterization scheme. Global and Planetary Change, 13:145-159.

Otto, D., Rasse, D., Kaplan, J., Warnant, P., François, L., 2002 : Biospheric carbon stocks reconstructed and the Last Glacial Maximum: comparison between general circulation models using prescribed and computed sea surface temperature. Global Planetary Change, 33:117-138.

Pan S., Tian, H., Dangal, S. R. S., Ouyang, Z., Tao, B., Ren, W. et al., 2014: Modeling and monitoring terrestrial primary production in a changing global environment: toward a multiscale synthesis of observation and simulation. Advances in Meteorology, 2014:1-17.

Parton, W. J., Schimel, D. S., Cole, C. V., Ojima, D. S., 1987: Analysis of factors controlling soil organic matter levels in Great Plains grasslands. Soil Science Society of America Journal, 51:1173-1179.

Reichstein, M., Falge, E., Baldocchi, D., Papale, D., Valentini, R., Aubinet, M. et al., 2005: On the separation of net ecosystem exchange into assimilation and ecosystem respiration: review and improved algorithm. Global Change Biology, 11:1-16.

Reyer, C., 2015: Forest productivity under environmental change - a review of stand-scale modeling studies. Current Forestry Reports, 1:53-68.

Richardson, A. D., Mahecha, M. D., Falge, E., Kattge, J., Moffat, A. M., Papale, D. et al., 2008: Statistical properties of random CO2 flux measurement uncertainty inferred from model residuals. Agricultural and Forest Meteorology, 148:38-50.

Richardson, A. D., Anderson, R. S., Arian, M. A., Barr, A. G., Bohrer, G., Chen, G. et al., 2012: Terrestrial biosphere models need better representation of vegetation phenology: results from the North American Carbon Program Site synthesis. Global Change Biology, 18:566-584.

Running, S. W., Gower, S. T., 1991: FOREST-BGC, A general model of forest ecosystem processes for regional applications II. Dynamic carbon allocation and nitrogen budgets. Tree Physiology, 9:147-160.

Saxton, K., Rawls, W. J., Romberger, J., Papendick, R., 1986: Estimating generalized soil-water characteristics from texture. Soil Science Society of America Journal, 50:1031-1036.

Schaber, J., Badeck, F. - W., 2003: Physiology based phenology models for forest tree species in Germany. International Journal of Biometeorology, 47:193-201.

Schultz, M., Mudelsee, M., 2002: REDFIT: estimating red-noise spectra directly from unevenly spaced paleoclimatic time series. Computers & Geosciences, 28:421-426.

Shinozaki, K., Yoda, K., Hozumi, K., Kira, T., 1964: A quantitative analysis of plant form - the pipe model theory. I. Basic analysis. Japanese Journal of Ecology, 14:97-105.

Siqueira M. B., Katul, G. G., Sampson, D. A., Stoy, P. C., Juang, J. - Y., McCarthy, H. R. et al., 2006: Multiscale model intercomparisons of CO2 and H2O exchange rates in a maturing southeastern US pine forest. Global Change Biology, 12:1189-1207.

Stoy, P. C., Katul, G. G., Siqueira, M. B., Juang, J. - Y., McCarthy, H. R., Kim, H. - S. et al., 2005: Variability in net ecosystem exchange from hourly to interannual time scales at adjacent pine and hardwood forests: a wavelet analysis. Tree Physiology, 25:887-902.

Suckow, F., Badeck, F. - W. Lasch, P., Schaber, J., 2001: Nutzung von Level-II-Beobachtungen für Test und Anwendungen des Sukzessionsmodells FORESEE. Beiträge für Forstwirtschaft und Landschaftsökologie, 35:84-87.

Sykes, M. T., Prentice, I. C., 1996: Carbon storage and climate change in Swedish forests: A comparison of static and dynamic modelling approaches. In: Apps, M. J., Price, D. T. (eds.). Forest Ecosystems, Forest Management and the Global Carbon Cycle. Springer-Verlag, Berlin, Germany, NATO ASI Series, 40:69-78.

Torrence, C., Compo, G. P., 1998: A practical guide to wavelet analysis. Bulletin of the American Meteorological Society, 79:61-78.

Trudinger, C. M., Raupach, M. R., Rayner, P. J., Kattge, J., Liu, Q., Pak, B. et al., 2007: OptIC project: An intercomparison of optimization techniques for parameter estimation in terrestrial biogeochemical models. Journal of Geophysical Research, 112: G02027.

Uppala, S. M., Kallberg, P. W., Simmons, A. J., Andrea, U., Da Costa Bechtold, V., Fiorino, M. et al., 2005: The ERA-40 re-analysis. Quarterly Journal of the Royal Meteorological Society, 131:2961-3012.

Vinnichenko, N., 1970: The kinetic energy spectrum in the free atmosphere - 1 second to 5 years. Tellus, 22:158-166.

Walker, W. E., Harremoes, P. J., Rotmans, J., Van Der Sluis, J. P., Van Asselt, M. B. A., Janssen, P. et al., 2003: Defining uncertainty: a conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4:5-17.

Wang, T., Brender, P., Ciais, P., Piao, S., Mahecha, M., Chevalier, F. et al., 2012: State-dependent errors in a land surface model across biomes inferred from eddy covariance observations on multiple timescales. Ecological Modelling, 246:11-25.

Warnant, P., François, L. M., Strivay, D., Gérard, J.-C., 1994: CARAIB: a global model of terrestrial biological productivity. Global Biogeochemical Cycles, 8:255-270.

Warnant, P., 1999: Modélisation du cycle du carbone dans la biosphère continentale à l’échelle globale. PhD thesis, Université de Liège, Liège, 276 p.

Wythers, K. R., Reich, P. B., Tjoelker, M. G., Bolstad, P. B., 2005: Foliar respiration acclimation to temperature variable Q10 alter ecosystem carbon balance. Global Change Biology, 11:435-449.

Central European Forestry Journal

The Journal of National Forest Centre – Forest Research Institute Zvolen

Journal Information

CiteScore 2016: 0.56

SCImago Journal Rank (SJR) 2016: 0.230
Source Normalized Impact per Paper (SNIP) 2016: 0.454


All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 362 358 35
PDF Downloads 132 130 9