Are regional models ready for prime time?

The third Climate Dialogue is about the value of models on the regional scale. Do model simulations at this level have skill? Can regional models add value to the global models?

We have three excellent participants joining this discussion: Bart van den Hurk of KNMI in The Netherlands who is actively involved in the KNMI scenario’s, Jason Evans from the University of Newcastle, Australia, who is coordinator of Coordinated Regional Climate Downscaling Experiment (CORDEX) and Roger Pielke Sr. who through his research articles and his weblog Climate Science is well known for his outspoken views on climate modelling.

Climate Dialogue editorial staff
Rob van Dorland, KNMI
Marcel Crok, science writer
Bart Verheggen

Introduction regional modelling

Are climate models ready to make regional projections?

Climate models are vital tools for helping us understand long-term changes in the global climate system. These models allow us to make physically plausible projections of how the climate might evolve in the future under given greenhouse gas emission scenarios.

Global climate projections for 2050 and 2100 have, amongst other purposes, been used to inform potential mitigation policies, i.e. to get a sense of the challenge we are facing in terms of CO2 emission reductions. The next logical step is to use models for adaptation as well. Stakeholders have an almost insatiable demand for future regional climate projections. These demands are driven by practical considerations related to freshwater resources, especially ecosystems and water related infrastructure, which are vulnerable to climate change.

Global climate models (GCMs) though have grid scales that are quite coarse (>100 km). This hampers the reconstruction of climate change at smaller scales (regional to local). Regions (the size of e.g. the Netherlands) are usually covered by only a few grid points. A crucial question therefore is whether information from global climate models at this spatial scale is realistic and meaningful, in hind cast and/or for the future.

Hundreds of studies have been published in the literature [1] presenting regional projections of climate change for 2050 and 2100. The output of such model simulations is then used by the climate impacts community to investigate what potential future benefits or threats could be expected. However several recent studies cast doubt whether global model output is realistic on a regional scale, even in hind cast. [2-5]

So a legitimate question is whether global and/or regional climate models are ready to be used for regional projections? Is the information reliable enough to use for all kinds of medium to long term adaptation planning? Or should we adopt a different approach?

To improve the resolution of the models other techniques, such as regional climate models (RCMs), or downscaling methods, have been developed. Nesting a regional climate model (with higher spatial resolution) into an existing GCM is one way to downscale data. This is called dynamical downscaling. A second way of downscaling climate model data is through the use of statistical regression. Statistical downscaling is based on relationships linking large-scale atmospheric variables from either GCMs or RCMs (predictors)and local/regional climate variables (predictands) using observations. [6]

Both methods are widely used inside the regional modelling community. The higher spatial resolution allows a more detailed representation of relevant processes, which will hopefully, but not necessarily, result in a “better” prediction. However RCMs operate under a set of boundary conditions that are dependent on the parent GCM. Hence, if the GCM does not do an adequate job of reproducing the climate signal of a particular region, the RCM will simply mimic those inaccuracies and biases. A valid question therefore is if and how the coupling of a RCM to a GCM can provide more refined insights. [7,8]

Recently Kerr [9] caused quite a stir in the regional modelling community by raising doubts about the reliability of regional model output. A debate about the reliability of model simulations is quickly seen as one between proponents and sceptics of anthropogenic global warming. However as Kundzewicz [10] points out “these are pragmatic concerns, raised by hydrologists and water management practitioners, about how useful the GCMs are for the much more detailed level of analysis (and predictability) required for site-specific water management decisions (infrastructure planning, design and operations).”

Climate dialogue
The focus of this Climate Dialogue will be on the reliability of climate simulations for the regional scale. An important question will be if there is added value from regional climate downscaling.

More specific questions:

1) How realistic are simulations by GCM’s on the regional scale?

2) Do some parameters (e.g. temperature) perform better than others (e.g. precipitation)?

3) Do some regions perform better than others?

4) To what extent can regional climate models simulate the past?

5) What is the best way to determine the skill of the hind cast?

6) Is there added value of regional models in comparison with global models?

7) What are the relative merits of dynamical and statistical downscaling?

8) How should one judge projections of these regional models?

9) Should global/regional climate models be used for decisions concerning infrastructure development? If so how? If not, what should form a better scientific base for such decisions?

References:

[1] The CMIP3 and CMIP5 list of publications is a good starting point, see http://www-pcmdi.llnl.gov/ipcc/subproject_publications.php and http://cmip.llnl.gov/cmip5/publications/allpublications

[2] G.J. van Oldenborgh, F.J. Doblas Reyes, S.S. Drijfhout, and E. Hawkins, "Reliability of regional climate model trends", Environmental Research Letters, vol. 8, pp. 014055, 2013. http://dx.doi.org/10.1088/1748-9326/8/1/014055

[3] Anagnostopoulos, G. G., Koutsoyiannis, D., Christofides, A., Efstratiadis, A. &Mamassis, N. (2010) A comparison of local and aggregated climate model outputs with observed data. Hydrol. Sci. J. 55(7), 1094–1110

[4] Stephens, G. L., T. L’Ecuyer, R. Forbes, A. Gettlemen, J.‐C. Golaz, A. Bodas‐Salcedo, K. Suzuki, P. Gabriel, and J. Haynes (2010), Dreary state of precipitation in global models, J. Geophys. Res., 115, D24211, doi:10.1029/2010JD014532

[5] J. Bhend, and P. Whetton, "Consistency of simulated and observed regional changes in temperature, sea level pressure and precipitation", Climatic Change, 2013. http://dx.doi.org/10.1007/s10584-012-0691-2

[6] Wilby, R. L. (2010) Evaluating climate model outputs for hydrologicalapplications – Opinion. Hydrol. Sci. J. 55(7), 1090–1093

[7] Kundzewicz, Zbigniew W. and Stakhiv, Eugene Z.(2010) 'Are climate models “ready for prime time” inwater resources management applications, or is more research needed?', Hydrological Sciences Journal, 55: 7, 1085 —1089

[8] Pielke, R. A. S., and R. L. Wilby, 2012: Regional climate downscaling: What’s the point? Eos Trans.AGU, 93, PAGE 52, doi:201210.1029/2012EO050008

[9] R.A. Kerr, "Forecasting Regional Climate Change Flunks Its First Test", Science, vol. 339, pp. 638-638, 2013. http://dx.doi.org/10.1126/science.339.6120.638

[10] Kundzewicz, Zbigniew W. and Stakhiv, Eugene Z.(2010) 'Are climate models “ready for prime time” in water resources management applications, or is more research needed?', Hydrological Sciences Journal, 55: 7, 1085 —1089

Guest blog Bart van den Hurk

The added value of Regional Climate Models in climate change assessments

Regional downscaling of climate information is a popular activity in many applications addressing the assessment of possible effects of a systematic change of the climate characteristics at the local scale. Adding local information, not captured in the coarse scale climate model or observational archives, can provide an improved representation of the relevant processes at this scale, and thus yield additional information, for instance concerning topography, land use or small scale features such as sea breezes or organisation of convection. A necessary step in the application of tools used for this regional downscaling is a critical assessment of the quality of the tools: are regional climate models (RCMs), used for this climate information downscaling, good enough for this task?

It is important to distinguish the various types of analyses that are carried out with RCMs. And likewise to assess the ability of the RCM to perform the task that is assigned to them. And these types of analyses clearly cover a wider range than plain prediction of the local climate!

Regional climate prediction

Pielke and Wilby (2012) discuss the lack of potential of RCMs to increase the skill of climate predictions at the regional scale. Obviously, these RCM predictions heavily rely on the quality of the boundary conditions provided by global climate models, and fail to represent dynamically the spatial interaction between the region of interest and the rest of the world. However, various “big brother” type experiments (in which the ability of RCMs to reproduce a filtered signal provided by the boundary conditions (Denis et al, 2002), for instance carried out by colleagues at KNMI) do show that a high resolution regional model can add value to a coarse resolution boundary condition by improving the spatial structure of the projected mean temperatures. Also the spatial structure of changes in precipitation linked to altered surface temperature by convection can be improved by using higher resolution model experiments, although the relative gain here is generally small (Di Luca et al, 2012).

Van Oldenborgh et al (2013) point out that the spatial structure of the mean temperature trend in the recent CMIP5 model ensemble compares fairly well with observations, but anomalies from the mean temperature trend aren’t well captured. This uncertainty clearly limits the predictability of temperatures at the regional scale beyond the mean trend. Van Haren et al (2012) also nicely illustrate the dependence of regional skill on lateral boundary conditions: simulations of (historic) precipitation trends for Europe failed to match the observed trends when lateral boundary conditions were provided from an ensemble of CMIP3 global climate model simulations, while a much better correspondence with observations was obtained when reanalyses were used as boundary condition. Thus, a regional prediction of a trend can only be considered to be skilful when the boundary forcings represent the signal to be forecasted adequately. And this does apply to mean temperature trends for most places in the world, but not for anomalies from these mean trends, nor for precipitation projections.

For regional climate predictability, the added value of RCMs should come from better resolving the relationship between mean (temperature) trends and key indicators that are supposedly better represented in the high resolution projections utilizing additional local information, such as temperature or precipitation extremes. Also here, evidence of adding skill is not univocally demonstrated. Min et al (2013) evaluate the ability of RCMs driven by reanalysis data to reproduce observed trends in European annual maximum temperatures, and conclude that there is a clear tendency to underestimate the observed trends. For Southern Europe biases in maximum temperatures could be related to errors in the surface flux partitioning (Stegehuis et al, 2012), but no such relationship was found for NW Europe by Min et al (2013).

Thus indeed, the limitations to predictability or regional climate information by RCMs as discussed by Pielke and Wilby (2012) and others are valid, and care must be taken while interpreting RCM projections as predictive assessments. But is this the only application of RCMs? Not really. We will discuss two other applications, together with the degree to which limitations in RCM skill apply and are relevant.

Bottom up environmental assessments

A fair point of critique to exploring a cascade of model projections ranging from the global scale down to the local scale of a region of interest to developers of adaptation or mitigation policies is the virtually unmanageable increase of the range of degrees of freedom, also addressed as “uncertainty”. Uncertainty arises from imperfect models, inherent variability, and unknown evolution of external forcings. And in fact the process of (dynamical) downscaling adds another level of uncertainty, related to the choice of downscaling tools and methodologies. The reverse approach, starting from the vulnerability of a region or sector of interest to changes in environmental conditions (Pielke et al, 2012), does not eliminate all sources of uncertainty, but allows a focus on the relevant part of the spectrum, including those elements that are not related to greenhouse gas induced climate change.

But also here, RCMs can be of great help, not necessarily by providing reliable predictions, but also by supporting evidence about the salience of planned measures or policies (Berkhout et al, 2013). A nice example is a near flooding situation in Northern Netherlands (January 2012), caused by a combined occurrence of a saturated soil due to excessive antecedent precipitation, a heavy precipitation event in the coastal area and a storm surge with a duration of several days that hindered the discharge of excess water from the area. This is typically a “real weather” event that is not necessarily exceptional but does expose a local vulnerability to superfluous water. The question asked by the local water managers was whether the combination of the individual events (wet soil, heavy rain, storm surge) has a causal relationship, and whether the frequency of occurrence of compound events can be expected to change in the future. Observational analyses do suggest a link between heavy precipitation and storm surge, but the available dataset was too short to explore the statistical relationships in a relevant part of the frequency distribution. A large set of RCM simulations is now explored to increase the statistical sample, but – more importantly – to provide a physically comprehensive picture of the boundary conditions leading up to an event like this. By enabling the policy makers to communicate this physically comprehensive picture provides public support for measures undertaken to adapt to this kind of events. This exploration of model based – synthetic – future weather is a powerful method to assess the consequences of possible changes in regional climate variability for the local water management.

Process understanding

Apart from a tool to predict a system given its initial state and the boundary forcings on it, a model is a collection of our understanding of the system itself. Its usefulness is not limited to its ability to predict, but also to describe the dynamics of a system, governed by internal processes and interactions with its environment. Regional climate models should likewise be considered as “collections of our understanding of the regional climate system”. And can likewise be used to study this system, and learn about it. There are numerous studies where regional climate model studies have increased our understanding of the mechanism of the climate system acting on a regional scale. A couple of examples:

  • Strong trends in coastal precipitation, and particularly a series of extreme precipitation events in the Netherlands, could successfully be attributed to anomalies in sea surface temperature (SST) in the nearby North Sea (Lenderink et al, 2009). Strong SST gradients close to the coast needed to be imposed to the RCM simulations carried out to reveal this mechanism. The relationship between SSTs and spatial gradients in changes in (extreme) precipitation is an important finding for analysing necessary measures to anticipate future changes in the spatial and temporal distribution of rainfall in the country.
  • During the past century land use change has given rise to regional changes in the local surface climatology, particularly the mean and variability of near surface temperature (Pitman et al, 2012). A set of GCM simulations dedicated to quantify the effect of land use change relative to changes in the atmospheric greenhouse gas concentration over the past century revealed that the land use effect is largely limited to the area of land use change. Wramneby et al (2010) explored the regional interaction between climate and vegetation response using a RCM set-up, and highlighted the importance of this interaction for assessing the mean temperature response particularly at high latitudes (due to the role of vegetation in snow covered areas) and in water limited evaporation regimes (due to the role of vegetation in controlling surface evaporative cooling).
  • In many occasions the degree to which anomalies in the land surface affect the overlying atmosphere depends on the resolved spatial scale. As an example, Hohenegger et al (2009) investigated the triggering of precipitation in response to soil moisture anomalies with a set of regional models ranging in physical formulation and resolution. This issue that deserves a lot of attention in the literature due to the possible existence of (positive) feedbacks that may affect occurrence or intensity of hydrological extremes such as heatwaves. In her study, RCMs operating at the typical 25 – 50 km scale resolution tend to overestimate the positive soil moisture – precipitation feedback in the Alpine area, which is better represented by higher resolution models. It is a study that points at a possible mechanism that needs to be adequately represented for generating reliable projections.

Each of these examples (and many more that can be cited) generates additional insight in the processes controlling local climate variability by allowing to zoom in on these processes using RCMs. They thus contribute to setting the research agenda in order to improve our understanding of drivers of regional change.

Climate predictions versus climate scenarios

The notion that a tool – an RCM – may possess shortcomings in its predictive skill, but simultaneously prove to be a valuable tool to support narratives that are relevant to policy making and spatial planning can in fact be extended to highlighting the difference between “climate predictions” and “climate scenarios”. Scenarios are typically used when deterministic or probabilistic predictions show too little skill to be useful, either because of the complexity of the considered system, or because of the fundamental limitations to its predictability (Berkhout et al, 2013). A scenario is a “what if” construction, a tool to create a mental map of the possible future conditions assuming a set of driving boundary conditions. For a scenario to be valuable it does not necessarily need to have predictive skill, although a range of scenarios can be and are being interpreted as a probability range for future conditions. A (single) scenario is mainly intended to feed someone’s imagination with a plausible, comprehensible and internally consistent picture. Used this way, also RCMs with limited predictive skill can be useful tools for scenario development and providing supporting narratives that generate public awareness or support for preparatory actions. For this, the RCM should be trustworthy in producing realistic and consistent patterns of regional climate variability, and abundant application, verification and improving is a necessary practice. Further developments of RCMs as a Regional Earth System Exploration tool, by linking the traditional meteorological models to hydrological, biogeophysical and socio-economic components, can further develop their usefulness in practice.

Biosketch
Bart van den Hurk has a PhD on land surface modelling, obtained in Wageningen in 1996. Since then he has worked at the Royal Netherlands Meteorological Institute (KNMI) as researcher, involved in studies addressing modelling land surface processes in regional and global climate models, data assimilation of soil moisture, and constructing regional climate change scenarios. He is strongly involved with the KNMI global modelling project EC-Earth, and is co-author of the land surface modules of the European Centre for Medium Range Weather Forecasts (ECMWF). Since 2005 he is part-time professor “Regional Climate Analysis” at the Institute of Marine and Atmospheric Research (IMAU) at the Utrecht University. There he teaches masters students, supervises PhD-students and is involved in several research networks. Between 2007 and 2010 he was chair of the WCRP-endorsed Global Land-Atmosphere System Studies (GLASS) panel, since 2006 member of the council of the Netherlands Climate Changes Spatial Planning program, and since 2008 member of the board of the division “Earth and Life Sciences” of the Dutch Research Council (NWO-ALW). He is convenor at a range of incidental and periodic conferences, and editor for Hydrology and Earth System Science (HESS).

References

Berkhout, F., B. van den Hurk, J. Bessembinder, J. de Boer, B. Bregman en M. van Drunen (2013), Framing climate uncertainty: using socio-economic and climate scenarios in assessing climate vulnerability and adaptation; submitted Regional and Environmental Change.

Denis, B., R. Laprise, D. Caya and J. Côté, 2002: Downscaling ability of one-way-nested regional climate models: The Big-Brother experiment. Clim. Dyn. 18, 627-646.

Di Luca, A., Elía, R. & Laprise, R., 2012. Potential for small scale added value of RCM’s downscaled climate change signal. Climate Dynamics. Available at: http://www.springerlink.com/index/10.1007/s00382-012-1415-z

Hohenegger, C., P. Brockhaus, C. S. Bretherton, C. Schär (2009): The soil-moisture precipitation feedback in simulations with explicit and parameterized convection, in: Journal of Climate 22, pp. 5003–5020.

Lenderink, G., E. van Meijgaard en F. Selten (2009), Intense coastal rainfall in the Netherlands in response to high sea surface temperatures: analysis of the event of August 2006 from the perspective of a changing climate; Clim. Dyn., 32, 19-33, doi:10.1007/s00382-008-0366-x.

Min, E., W. Hazeleger, G.J. van Oldenborgh en A. Sterl (2013), Evaluation of trends in high temperature extremes in North-Western Europe in regional climate models; Environmental Research Letters, 8, 1, 014011, doi:10.1088/1748-9326/8/1/014011.

Pielke, R. A. S., and R. L. Wilby, 2012: Regional climate downscaling: What’s the point? Eos Trans.AGU, 93, PAGE 52, doi:201210.1029/2012EO050008

Pielke, R. A., Sr., R. Wilby, D. Niyogi, F. Hossain, K. Dairuku,J. Adegoke, G. Kallos, T. Seastedt, and K. Suding (2012), Dealing with complexity and extreme events using a bottom-up, resource-based vulnerability perspective, in Extreme Events and Natural Hazards: The Complexity Perspective, Geophys. Monogr. Ser., vol. 196, edited by A. S. Sharma et al. 345–359, AGU, Washington, D. C., doi:10.1029/2011GM001086.

Pitman, A., N. de Noblet, F. Avila, L. Alexander, J.P. Boissier, V. Brovkin, C. Delire, F. Cruz, M.G. Donat, V. Gayler, B.J.J.M. van den Hurk, C. Reick en A. Voldoire (2012): Effects of land cover change on temperature and rainfall extremes in multi-model ensemble simulations; Earth System Dynamics, 3, 213-231, doi:10.5194/esd-3-213-2012.

Stegehuis A., Vautard R., Teuling R., Ciais P. Jung M. Yiou, P. (2012) Summer temperatures in Europe and land heat fluxes in observation-based data and regional climate model simulations, in press by Climate Dynamics; doi 10.1007/s00382-012-1559-x

Van Haren, R. G.J. van Oldenborgh, G. Lenderink, M. Collins en W. Hazeleger (2012), SST and circulation trend biases cause an underestimation of European precipitation trends; Clim. Dyn., doi:10.1007/s00382-012-1401-5.

Van Oldenborgh, G.J. F.J. Doblas-Reyes, S.S. Drijfhout en E. Hawkins (2013), Reliability of regional climate model trends; Environmental Research Letters, 8, 1, 014055, doi:10.1088/1748-9326/8/1/014055.

Wramneby, A., B. Smith, and P. Samuelsson (2010), Hot spots of vegetation-climate feedbacks under future greenhouse forcing in Europe, J. Geophys. Res., 115, D21119, doi:10.1029/2010JD014307.

Guest blog Jason Evans

Are climate models ready to make regional projections?

Global Climate Models (GCMs) are designed to provide insight into the global climate system. They have been used to investigate the impacts of changes in various climate system forcings such as volcanoes, solar radiation, and greenhouse gases, and have proved themselves to be useful tools in this respect. The growing interest in GCM performance at regional scales, rather than global, has come from at least two different directions: the climate modelling community and the climate change adaptation community.

Due, in part, to the ever increasing computational power available, GCMs are being continually developed and applied at higher spatial resolutions. Many GCM modelling groups have been increasing the resolution from ~250km grid boxes 7 years ago to ~100km grid boxes today. This model resolution increase leads naturally to model development and evaluation exercises that pay closer attention to smaller scales, in this case, regional instead of global scales. The Fifth Coupled Model Intercomparison Experiment (CMIP5) provides a large ensemble of GCM simulations, many of which are at resolutions high enough to warrant evaluation at regional scales. Over the next few years these GCM simulations will be extensively evaluated, problems will be found (as seen in some early evaluations1,2), followed hopefully by solutions that lead to further model development and improved simulations. This step of finding a solution to an identified problem is the hardest in the model development cycle, and I applaud those who do it successfully.

Probably the stronger demand for regional scale information from climate models is coming from the climate change adaptation community. Given only modest progress in climate change mitigation, adaptation to future climate change is required. Some sectors, such as those involved in large water resource projects (e.g. building a new dam), are particularly vulnerable to climate change. They are planning to invest large amounts of money (millions) in infrastructure, with planned lifetimes of 50-100 years, that directly depend on climate to be successful. Over such long lifetimes, greenhouse gas driven climate change is expected to increase temperature by a few degrees, and may cause significant changes in precipitation, depending on the location. Many of the systems required to adapt are more sensitive to precipitation than temperature, and projections of precipitation often have considerably more uncertainty associated with them. The question for the climate change adaptation community is whether the uncertainty (including model errors) in the projected climate change is small enough to be useful in a decision making framework.

Regional projections
From a GCM perspective then, the answer to “Are climate models ready to make regional projections?” is two-fold. For the climate modelling community the answer is yes. GCMs are being run at high enough resolution to make regional scale (so long as your regions are many 100kms across) evaluations and projections useful to inform the model development and hopefully improve future simulations. For the climate change adaptation community, whose spatial scale of interest is often much lower than current high resolution GCMs can capture, the answer in general is no. The errors in the simulated regional climate and the inter-model uncertainty in regional climate projections from GCMs is often too large to be useful in decision making. These climate change adaptation decisions need to be made however, and in an effort to produce useful regional scale climate information that embodies the global climate change a number of “downscaling” techniques have been developed.

It is worth noting that some climate variables, such as temperature, tend to be simulated better by climate models than other variables, such as precipitation. This is at least partly due to the scales and non-linearity of physical processes which effect each variable. This is demonstrated in the fourth IPCC report which mapped the level of agreement in the sign of the change in precipitation projected by GCMs. This map showed large parts of the world where GCMs disagreed about the sign of the change in precipitation. However this vastly underestimated the agreement between the GCMs3. They showed that much of this area of disagreement is actually areas where the GCMs agree that the change will be small (or zero). That is, if the actual projected change is zero then by chance, some GCMs will project small increases and some small decreases. This does not indicate disagreement between the models, rather they all agree that the change is small.

Regional climate models
Before describing the downscaling techniques, it may be useful to consider this question: What climate processes, that are important at regional scales, may be missing in GCMs?

The first set of processes relates directly to how well resolved land surface features such as mountains and coastlines are. Mountains cause local deviations in low level air flow. When air is forced to rise to get over a mountain range it can trigger precipitation, and because of this, mountains are often a primary region for the supply of fresh water resources. At GCM resolution mountains are often under-represented in terms of height or spatial extent and so do not accurately capture this relationship with precipitation. In fact, some regionally important mountain ranges, such as the eastern Mediterranean coastal range or the Flinders Range in South Australia, are too small to be represented at all in some GCMs. Using higher spatial resolution to better resolve the mountains should improve the models ability to capture this mountain-precipitation relationship.

Similarly, higher resolution allows model coastlines to be closer to the location of actual coastlines, and improves the ability to capture climate processes such as sea breezes.

The second set of processes are slightly more indirect and often involve an increase in the vertical resolution as well. These processes include the daily evolution of the planetary boundary layer, and the development of low level and mountain barrier jets.

A simple rule of thumb is that one can expect downscaling to higher resolution to improve the simulation of regional climate in locations that include coastlines and/or mountain ranges (particularly where the range is too small to be well resolved by the GCM but large enough to be well resolved at the higher resolution) while not making much difference over large homogeneous, relatively flat regions (deserts, oceans,...).

So, there are physical reasons one might expect downscaling to higher resolution will improve the simulation of regional climate. How do we go about this downscaling?

Downscaling
Downscaling techniques can generally be divided into two types: statistical and dynamical. Statistical techniques generally use a mathematical method to form a relationship between the modelled climate and observed climate at an observation station. A wide variety of mathematical methods can be used but they all have two major limitations. First, they rely on long historical observational records to calculate the statistical relationship, effectively limiting the variables that can be downscaled to temperature and precipitation, and the locations to those stations where these long records were collected. Second, they assume that the derived statistical relationship will not change due to climate change.

Dynamical downscaling, or the use of Regional Climate Models (RCMs), does not share the limitations of statistical downscaling. The major limitation in dynamical downscaling is the computational cost of running the RCMs. This generally places a limit on both the spatial resolution (often 10s of kilometres) and the number of simulations that can be performed to characterise uncertainty. RCMs also contain biases, both inherited from the driving GCM and generated with the RCM itself. It is worth noting that statistical downscaling techniques can be applied to RCM simulations as easily as GCM simulations to obtain projections at station locations.

The RCM limitations are actively being addressed through current initiatives and research. Like GCMs, RCMs benefit from the continued increase in computational power, allowing more simulations to be run at higher spatial resolution. The need for more simulations to characterise uncertainty is being further addressed through international initiatives to have many modelling groups contribute simulations to the same ensembles (e.g. CORDEX - COordinated Regional climate Downscaling EXperiment http://wcrp-cordex.ipsl.jussieu.fr/). New research into model independence is also pointing toward ways to create more statistically robust ensembles4. Novel research to reduce (or eliminate) the bias inherited from the driving GCM is also showing promise5,6.

Above I used simple physical considerations to suggest there would be some added value from regional models compared to global models. Others have investigated this from an observational viewpoint7,8 as well as through direct evaluation of model results at different scales9,10,11. In each case the results agreed with the rule of thumb given earlier. That is, in the areas with strong enough regional climate influences we do see improved simulations of regional climate at higher resolutions. Of course we are yet to address the question of whether the regional climate projections from these models have low enough uncertainty to be useful in climate change adaptation decision making.

Performance
To date RCMs have been used in many studies12,13 and a wide variety of evaluations of RCM simulations against observations have been performed. In attempting to examine the fidelity of the regional climate simulated, a variety of variables (temperature, precipitation, wind, surface pressure,...) have been evaluated using a variety of metrics14,15, with all the derived metrics then being combined to produce an overall measure of performance. When comprehensive assessments such as these are performed it is often found that different models have different strengths and weaknesses as measured by the different metrics. If one has a specific purpose in mind, e.g. building a new dam, one may wish to focus on metrics directly relevant to that purpose. Often the projected climate change is of interest so the evaluation should include a measure of the models ability to simulate change, often given as a trend over a recent historical period16,17. In most cases the RCMs are found to do a reasonably good job of simulating the climate of the recent past. Though, there are usually places and/or times where the simulation is not very good. Not surprising for an active area of research and model development.

Given the evaluation results found to date it is advisable to carefully evaluate each RCMs simulations before using any climate projections they produce. Being aware of where and when they perform well (or poorly) is important when assessing the climate change that model projects. It is also preferable for the projections themselves to be examined with the aim of understanding the physical mechanisms causing the projected changes. Good process level understanding of the causes behind the changes provides another mechanism through which to judge their veracity.

Ready for adaptation decisions?
Finally we come to the question of whether regional climate projections should be used in climate change adaptation decisions concerning infrastructure development? In the past such decision were made assuming a stationary climate such that observations of the past were representative of the future climate. So the real question here is will the use of regional climate projections improve decisions made when compared to the use of historical climate observations?

If the projected regional change is large enough that it falls outside the historical record even when considering the associated model errors and uncertainties, then it may indeed impact the decision. Such decisions are made within a framework that must consider uncertainty in many factors other than climate including future economic, technological and demographic pathways. Within such a framework, risk assessments are performed to inform the decision making process, and the regional climate projections may introduce a risk to consider that is not present in the historical climate record. If this leads to decisions which are more robust to future climate changes (as well as demographic and economic changes) then it is worthwhile including the regional climate projections in the decision making process.

Of course, this relies on the uncertainty in the regional climate projection being small enough for the information to be useful in a risk assessment process. Based on current models, this is not the case everywhere, and continued model development and improvement is required to decrease the uncertainty and increase the utility of regional climate projections for adaptation decision making.

Biosketch
Jason Evans has a B. Science (Physics) and a B. Math (hons) from the University of Newcastle, Australia. He has a Ph.D. in hydrology and climatology from the Australian National University. He worked for several years as a research associate at Yale University, USA, before moving to the University of New South Wales, Sydney, Australia. He is currently an Associate Professor in the Climate Change Research Centre there. His research involves general issues of regional climate and water cycle processes over land. He focuses at the regional (or watershed) scale and studies processes including river flow, evaporation/transpiration, water vapour transport and precipitation. He is currently Co-Chair of the GEWEX Regional Hydroclimate Panel, and coordinator of the Coordinated Regional Climate Downscaling Experiment (CORDEX), both elements of the World Climate Research Programme.

References

[1] van Oldenborgh GJ, Doblas Reyes FJ, Drijfhout SS, Hawkins E (2013) Reliability of regional climate model trends. Environmental Research Letters 8

[2] Bhend J, Whetton P (2013) Consistency of simulated and observed regional changes in temperature, sea level pressure and precipitation.

[3] Power SB, Delage F, Colman R, Moise A (2012) Consensus on twenty-first-century rainfall projections in climate models more widespread than previously thought. Journal of Climate 25:3792–3809

[4] Bishop CH, Abramowitz G (2012) Climate model dependence and the replicate Earth paradigm. Clim Dyn:1–16

[5] Xu Z, Yang Z-L (2012) An Improved Dynamical Downscaling Method with GCM Bias Corrections and Its Validation with 30 Years of Climate Simulations. Journal of Climate 25:6271–6286

[6] Colette A, Vautard R, Vrac M (2012) Regional climate downscaling with prior statistical correction of the global climate forcing. Geophys Res Lett 39:L13707

[7] di Luca A, Elía R de, Laprise R (2012) Potential for added value in precipitation simulated by high-resolution nested Regional Climate Models and observations. Clim Dyn 38:1229–1247

[8] di Luca A, Elía R de, Laprise R (2013) Potential for small scale added value of RCM’s downscaled climate change signal. Climate Dynamics 40:601–618

[9] Christensen OB, Christensen JH, Machenhauer B, Botzet M (1998) Very High-Resolution Regional Climate Simulations over Scandinavia—Present Climate. Journal of Climate 11:3204–3229

[10] Önol B (2012) Effects of coastal topography on climate: High-resolution simulation with a regional climate model. Clim Res 52:159–174

[11] Evans J, McCabe M (2013) Effect of model resolution on a regional climate model simulation over southeast Australia. Climate Research 56:131–145

[12] Wang Y, Leung L, McGregor J, Lee D, Wang W, Ding Y, Kimura F (2004) Regional climate modeling: Progress, challenges, and prospects. Journal of the Meteorological Society of Japan 82:1599–1628

[13] Evans JP, McGregor JL, McGuffie K (2012) Future Regional Climates. In: Henderson-Sellers A, McGuffie K (eds) The future of the World’s climate. Elsevier, p 223–252

[14] Christensen J, Kjellstrom E, Giorgi F, Lenderink G, Rummukainen M (2010) Weight assignment in regional climate models. Climate Research 44:179–194

[15] Evans J, Ekström M, Ji F (2012) Evaluating the performance of a WRF physics ensemble over South-East Australia. Climate Dynamics 39:1241–1258

[16] Lorenz P, Jacob D (2010) Validation of temperature trends in the ENSEMBLES regional climate model runs driven by ERA40. Climate Research 44:167–177

[17] Bukovsky MS (2012) Temperature trends in the NARCCAP regional climate models. Journal of Climate 25:3985–3991

Guest blog Roger Pielke Sr.

Are climate models ready to make regional projections?

The question that is addressed in my post is, with respect to multi-decadal model simulations, are global and/or regional climate models ready to be used for skillful regional projections by the impacts and policymaker communities?

This could also be asked as

Are skillful (value-added) regional and local multi-decadal predictions of changes in climate statistics for use by the water resource, food, energy, human health and ecosystem impact communities available at present?

As summarized in this post, the answer is NO.

In fact, the output of these models are routinely being provided to the impact communities and policymakers as robust scientific results, when, in fact, they only provide an illusion of skill. Simply plotting high spatial resolution model results is not, by itself, a skillful product!

Skill is defined as accurately predicting changes in climate statistics on this time period. This skill must be assessed by predicting global, regional and local average climate, and any climate change that was observed over the last several decades (i.e. “hindcast model predictions”).

One issue that we need to make sure is clearly understood are the terms “prediction” and “projection”. The term “projection”, of course, is just another word for a “prediction” when specified forcings are prescribed; e.g. such as different CO2 emission scenarios - see Pielke (2002). Thus “projection” and “prediction” are synonyms.

Dynamic and statistical downscaling is widely used to refine predictions from global climate models to smaller spatial scales. In order to classify the types of dynamic and statistical downscaling, Castro et al (2005) defined four categories. These are summarized in Pielke and Wilby (2012) and in Table 1 from that paper.

In the current post, I am referring specifically to Type 4 downscaling. For completeness, I list below all types of downscaling. The intent of downscaling is to achieve accurate, higher spatial resolution of weather and other components of the climate system than is achievable with the coarser spatial resolution global model. Indeed, one test of a downscaled result is whether its results agree more with observations than does the global model results simply downscaled by interpolation to a finer terrain and landscape map.

Types of Downscaling

Type 1 downscaling is used for short-term, numerical weather prediction. In dynamic Type 1 downscaling, the regional model includes initial conditions from observations. In Type 1 statistical downscaling the regression relationships are developed from observed data and the Type 1 dynamic model predictions.

The Type 1 application of downscaling is operationally used by weather services worldwide. They provide very significant added regional and local prediction skill beyond what is available from the parent global model (e.g. see http://weather.rap.ucar.edu/model/). Millions of Type 1 forecasts are made every year and verified with real world weather, thus providing an opportunity for extensive quantitative testing of their predictive skill (Pielke Jr. 2010) .

Type 2 dynamic downscaling refers to regional weather (or climate) simulations in which the regional model’s initial atmospheric conditions are forgotten (i.e., the predictions do not depend on the specific initial conditions), but results still depend on the lateral boundary conditions from a global numerical weather prediction where initial observed atmospheric conditions are not yet forgotten, or are from a global reanalysis. Type 2 statistical downscaling uses the regression relationships developed for Type 1 statistical downscaling except that the input variables are from the Type 2 weather (or climate) simulation. Downscaling from reanalysis products (Type 2 downscaling) defines the maximum forecast skill that is achievable with Type 3 and Type 4 downscaling.

Type 2 downscaling provides an effective way to provide increased value-added information on regional and local spatial scales. It is important to recognize, however, that type 2 downscaling is not a prediction (projection) for the cases where the global data comes from a reanalysis (e.g. a reanalysis is a combination of real world observations, folded into a global model). An example of this type of application is reported in Feser et al., 2011, Pielke 2013 and Mearns et al 2012, 2013b).

When Type 2 results are presented to the impacts communities as a valid analysis of the skill of Type 4 downscaling, those communities are being misled on the actual robustness of the results in terms of multi-decadal projections. Type 2 results, even from global models used in a prediction mode, still retain real world information in the atmosphere (such as from long wave jet stream patterns), as well as sea surface temperatures, deep soil moisture, and other climate variables that have long term persistence.

Type 3 dynamic downscaling takes lateral boundary conditions from a global model prediction forced by specified real world surface boundary conditions, such as for seasonal weather predictions based on observed sea surface temperatures, but the initial observed atmospheric conditions in the global model are forgotten. Type 3 statistical downscaling uses the regression relationships developed for Type 1 statistical downscaling, except using the variables from the global model prediction forced by specified real-world surface boundary conditions.

Type 3 downscaling is applied, for example, for seasonal forecasts where slowly changing anomalies in the surface forcing (such as sea surface temperature) provide real-world information to constrain the downscaling results. Examples of the level of limited, but non-zero, skill achievable are given in Castro et al (2007) and Veljovic et al (2012).

Type 4 dynamic downscaling takes lateral boundary conditions from an Earth system model in which coupled interactions among the atmosphere, ocean, biosphere, and cryosphere are predicted [e.g., Solomon et al., 2007]. Other than terrain, all other components of the climate system are calculated by the model except for human forcings, including greenhouse gas emissions scenarios, which are prescribed. Type 4 dynamic downscaling is widely used to provide policy makers with impacts from climate decades into the future. Type 4 statistical downscaling uses transfer functions developed for the present climate, fed with large scale atmospheric information taken from Earth system models representing future climate conditions. It is assumed that statistical relationships between real-world surface observations and large scale weather patterns will not change.

The level of skill achievable deteriorates from Type 1 to Type 2 to Type downscaling, as fewer observations are used to constrain the model realism. For Type 4, except for the prescribed forcing (such as added CO2), there are no real world constraints.

It is also important to realize that the models, while including aspects of basic physics (such as the pressure gradient force, advection, gravity), are actually engineering code. All of the parameterizations of the physics, chemistry and biology include tunable parameters and functions. For the atmospheric part of climate models, this engineering aspect of weather models is discussed in depth in Pielke (2013a). Thus, such engineering (parameterized) components can result in the drift of the model results away from reality when observations are not there to constrain this divergence from reality. The climate models are not basic physics.

There are two critical tests for skillful Type 4 model runs:

1. The model must provide accurate replications of the current climatic conditions on the global, regional and local scale? This means the model must be able to accurately predict the statistics of the current climate. This test, of course, needs to be performed in a hindcast mode?

2. The model must also provide accurate predictions of changes in climatic conditions (i.e. the climatic statistics) on the regional and local scale? This means the model must be able to replicate the changes in climate statistics over this time period. This can also, of course, only be assessed by running the models in a hindcast mode.

For Type 4 runs [i.e. multidecadal projections], the models being used include the CMIP5 projections.

As reported in The CMIP5 - Coupled Model Intercomparison Project Phase 5

CMIP5 promotes a standard set of model simulations in order to:

· evaluate how realistic the models are in simulating the recent past,

· provide projections of future climate change on two time scales, near term (out to about 2035) and long term (out to 2100 and beyond), and

· understand some of the factors responsible for differences in model projections, including quantifying some key feedbacks such as those involving clouds and the carbon cycle

The CMIP5 runs, unfortunately, perform poorly with respect to the first bullet listed above, as documented below.

If the models’ are not sufficiently realistic in simulating the climate in the recent past, they are not ready to be used to provide projections for the coming decades!

A number of examples from the peer reviewed literature illustrate this inadequate performance.

Summary of a Subset of Peer-Reviewed Papers That Document the Limitations of the CMIP5 model projections with respect to Criteria #1.

· Taylor et al, 2012: Afternoon rain more likely over drier soils. Nature. doi:10.1038/nature11377. Received 19 March 2012 Accepted 29 June 2012 Published online 12 September 2012

“…the erroneous sensitivity of convection schemes demonstrated here is likely to contribute to a tendency for large-scale models to `lock-in’ dry conditions, extending droughts unrealistically, and potentially exaggerating the role of soil moisture feedbacks in the climate system.”

· Driscoll, S., A. Bozzo, L. J. Gray, A. Robock, and G. Stenchikov (2012), Coupled Model Intercomparison Project 5 (CMIP5) simulations of climate following volcanic eruptions, J. Geophys. Res., 117, D17105, doi:10.1029/2012JD017607. published 6 September 2012.

The study confirms previous similar evaluations and raises concern for the ability of current climate models to simulate the response of a major mode of global circulation variability to external forcings.

· Fyfe, J. C., W. J. Merryfield, V. Kharin, G. J. Boer, W.-S. Lee, and K. von Salzen (2011), Skillful predictions of decadal trends in global mean surface temperature, Geophys. Res. Lett.,38, L22801, doi:10.1029/2011GL049508

”….for longer term decadal hindcasts a linear trend correction may be required if the model does not reproduce long-term trends. For this reason, we correct for systematic long-term trend biases.”

· Xu, Zhongfeng and Zong-Liang Yang, 2012: An improved dynamical downscaling method with GCM bias corrections and its validation with 30 years of climate simulations. Journal of Climate 2012 doi: http://dx.doi.org/10.1175/JCLI-D-12-00005.1

”…the traditional dynamic downscaling (TDD) [i.e. without tuning) overestimates precipitation by 0.5-1.5 mm d-1.....The 2-year return level of summer daily maximum temperature simulated by the TDD is underestimated by 2-6°C over the central United States-Canada region".

· Anagnostopoulos, G. G., Koutsoyiannis, D., Christofides, A., Efstratiadis, A. & Mamassis, N. (2010) A comparison of local and aggregated climate model outputs with observed data. Hydrol. Sci. J. 55(7), 1094–1110

".... local projections do not correlate well with observed measurements. Furthermore, we found that the correlation at a large spatial scale, i.e. the contiguous USA, is worse than at the local scale."

· Stephens, G. L., T. L’Ecuyer, R. Forbes, A. Gettlemen, J.‐C. Golaz, A. Bodas‐Salcedo, K. Suzuki, P. Gabriel, and J. Haynes (2010), Dreary state of precipitation in global models, J. Geophys. Res., 115, D24211, doi:10.1029/2010JD014532.

"...models produce precipitation approximately twice as often as that observed and make rainfall far too lightly.....The differences in the character of model precipitation are systemic and have a number of important implications for modeling the coupled Earth system .......little skill in precipitation [is] calculated at individual grid points, and thus applications involving downscaling of grid point precipitation to yet even finer‐scale resolution has little foundation and relevance to the real Earth system.”

· Sun, Z., J. Liu, X. Zeng, and H. Liang (2012), Parameterization of instantaneous global horizontal irradiance at the surface. Part II: Cloudy-sky component, J. Geophys. Res., doi:10.1029/2012JD017557, in press.

“Radiation calculations in global numerical weather prediction (NWP) and climate models are usually performed in 3-hourly time intervals in order to reduce the computational cost. This treatment can lead to an incorrect Global Horizontal Irradiance (GHI) at the Earth’s surface, which could be one of the error sources in modelled convection and precipitation. …… An important application of the scheme is in global climate models….It is found that these errors are very large, exceeding 800 W m-2 at many non-radiation time steps due to ignoring the effects of clouds….”

· Ronald van Haren, Geert Jan van Oldenborgh, Geert Lenderink, Matthew Collins and Wilco Hazeleger, 2012: SST and circulation trend biases cause an underestimation of European precipitation trends Climate Dynamics 2012, DOI: 10.1007/s00382-012-1401-5

“To conclude, modeled atmospheric circulation and SST trends over the past century are significantly different from the observed ones. These mismatches are responsible for a large part of the misrepresentation of precipitation trends in climate models. The causes of the large trends in atmospheric circulation and summer SST are not known.”

· Driscoll, S., A. Bozzo, L. J. Gray, A. Robock, and G. Stenchikov (2012), Coupled Model Intercomparison Project 5 (CMIP5) simulations of climate following volcanic eruptions, J. Geophys. Res., 117, D17105, doi:10.1029/2012JD017607. published 6 September 2012.

The models generally fail to capture the NH dynamical response following eruptions. ……The study confirms previous similar evaluations and raises concern for the ability of current climate models to simulate the response of a major mode of global circulation variability to external forcings

· Mauritsen, T., et al. (2012), Tuning the climate of a global model, J. Adv. Model. Earth Syst., 4, M00A01, doi:10.1029/2012MS000154. published 7 August 2012 During a development stage global climate models have their properties adjusted or tuned in various ways to best match the known state of the Earth’s climate system…..The tuning is typically performed by adjusting uncertain, or even non-observable, parameters related to processes not explicitly represented at the model grid resolution.

· Jiang, J. H., et al. (2012), Evaluation of cloud and water vapor simulations in CMIP5 climate models using NASA “A-Train” satellite observations, J. Geophys. Res., 117, D14105, doi:10.1029/2011JD017237. published 18 July 2012. The modeled mean CWCs [cloud water] over tropical oceans range from ∼3% to ∼15× of the observations in the UT and 40% to 2× of the observations in the L/MT. For modeled H2Os, the mean values over tropical oceans range from ∼1% to 2× of the observations in the UT and within 10% of the observations in the L/MT….Tropopause layer water vapor is poorly simulated with respect to observations. This likely results from temperature biases

· Van Oldenborgh, G.J., F.J. Doblas-Reyes, B. Wouters, W. Hazeleger (2012): Decadal prediction skill in a multi-model ensemble. Clim.Dyn. doi:10.1007/s00382-012-1313-4 who report quite limited predictive skill in two regions of the oceans on the decadal time period, but no regional skill elsewhere, when they conclude that "A 4-model 12-member ensemble of 10-yr hindcasts has been analysed for skill in SST, 2m temperature and precipitation. The main source of skill in temperature is the trend, which is primarily forced by greenhouse gases and aerosols. This trend contributes almost everywhere to the skill. Variation in the global mean temperature around the trend do not have any skill beyond the first year. However, regionally there appears to be skill beyond the trend in the two areas of well-known low-frequency variability: SST in parts of the North Atlantic and Pacific Oceans is predicted better than persistence. A comparison with the CMIP3 ensemble shows that the skill in the northern North Atlantic and eastern Pacific is most likely due to the initialisation, whereas the skill in the subtropical North Atlantic and western North Pacific are probably due to the forcing."

· Sakaguchi, K., X. Zeng, and M. A. Brunke (2012), The hindcast skill of the CMIP ensembles for the surface air temperature trend, J. Geophys. Res., 117, D16113, doi:10.1029/2012JD017765. published 28 August 2012

the skill for the regional (5° × 5° – 20° × 20° grid) and decadal (10 – ∼30-year trends) scales is rather limited…. The mean bias and ensemble spread relative to the observed variability, which are crucial to the reliability of the ensemble distribution, are not necessarily improved with increasing scales and may impact probabilistic predictions more at longer temporal scales.

· Kundzewicz, Z. W., and E.Z. Stakhiv (2010) Are climate models “ready for prime time” in water resources managementapplications, or is more research needed? Editorial. Hydrol. Sci. J. 55(7), 1085–1089. who conclude that “Simply put, the current suite of climate models were not developed to provide the level of accuracy required for adaptation-type analysis.”

Even on the global average scale, the multi-decadal global climate models are performing poorly since 1998, as very effectively shown in an analysis by John Christy in the post by Roy Spencer which is reproduced below. As reported in Roy’s post, these plots by John are based upon data from the KNMI Climate Explorer with a comparison of 44 climate models versus the UAH and RSS satellite observations for global lower tropospheric temperature variations, for the period 1979-2012 from the satellites, and for 1975 – 2025 for the models.

Thus the necessary criteria #1 is not satisfied. Obviously, the first criteria must be satisfactorily addressed before one can have any confidence in the second criteria of claiming to skillfully predict changes in climate statistics.

Summary
To summarize the current state of modeling and the use of regional models to downscale for multi-decadal projections, as reported in Pielke and Wilby (2012):

1. The multi-decadal global climate model projection must include all first-order climate forcings and feedbacks, which, unfortunately, they do not.

2. Current global multi-decadal predictions are unable to skillfully simulate regional forcing by major atmospheric circulation features such as from El Niño and La Niña and the South Asian monsoon , much less changes in the statistics of these climate features. These features play a major role of climate impacts at the regional and local scales.

3. While regional climate downscaling yields higher spatial resolution, the downscaling is strongly dependent on the lateral boundary conditions and the methods used to constrain the regional climate model variables to the coarser spatial scale information from the parent global models. Large-scale climate errors in the global models are retained and could even be amplified by the higher-spatial- resolution regional models. If the global multi-decadal climate model predictions do not accurately predict large-scale circulation features, for instance, they cannot provide accurate lateral boundary conditions and interior nudging to regional climate models. The presence of higher spatial resolution information in the regional models, beyond what can be accomplished by interpolation of the global model output to a finer grid mesh, is only an illusion of added skill.

4. Apart from variable grid approaches, regional models do not have the domain scale (or two-way interaction between the regional and global models) to improve predictions of the larger-scale atmospheric features. This means that if the regional model significantly alters the atmospheric and/or ocean circulations, there is no way for this information to affect larger scale circulation features that are being fed into the regional model through the lateral boundary conditions and nudging.

5. The lateral boundary conditions for input to regional downscaling require regional-scale information from a global forecast model. However the global model does not have this regional-scale information due to its limited spatial resolution. This is, however, a logical paradox because the regional model needs something that can be acquired only by a regional model (or regional observations). Therefore, the acquisition of lateral boundary conditions with the needed spatial resolution becomes logically impossible. Thus, even with the higher resolution analyses of terrain and land use in the regional domain, the errors and uncertainty from the larger model still persist, rendering the added simulated spatial details inaccurate.

6. There is also an assumption that although global climate models cannot predict future climate change as an initial value problem, they can predict future climate statistics as a boundary value problem [Palmer et al., 2008]. However, for regional downscaling (and global) models to add value (beyond what is available to the impacts community via the historical, recent paleorecord and a worst case sequence of days), they must be able to skillfully predict changes in regional weather statistics in response to human climate forcings. This is a greater challenge than even skillfully simulating current weather statistics.

It is therefore inappropriate to present type 4 results to the impacts community as reflecting more than a subset of possible (plausible) future climate risks. As I wrote in Pielke (2011) with respect to providing multi-decadal climate predictions to the impacts and policy communities, there is a

“serious risk of overselling what [can be] provide[d] to policy makers. A significant fraction of the funds they are seeking for prediction could more effectively be used if they were spent on assessing risk and ways to reduce the vulnerability of local/regional resources to climate variability and change and other environmental issues using the bottom-up, resources-based perspective discussed in Pielke and Bravo de Guenni (2004), Pielke (2004), and Pielke et al. (2009). This bottom-up focus is “of critical interest to society.”

We wrote this recommendation also in Pielke and Wilby (2012):

As a more robust approach, we favor a bottom-up, resource-based vulnerability approach to assess the climate and other environmental and societal threats to critical assets [Wilby and Dessai, 2010; Kabat et al., 2004]. This framework considers the coping conditions and critical thresholds of natural and human environments beyond which external pressures (including climate change) cause harm to water resources, food, energy, human health, and ecosystem function. Such an approach could assist policy makers in developing more holistic mitigation and adaptation strategies that deal with the complex spectrum of social and environmental drivers over coming decades, beyond carbon dioxide and a few other greenhouse gases.

This is a more robust way of assessing risks, including from climate, than using the approach adopted by the Intergovernmental Panel on Climate Change (IPCC) which is primarily based on downscaling from multi-decadal global climate model projections. A vulnerability assessment using the bottom-up, resource-based framework is a more inclusive approach for policy makers to adopt effective mitigation and adaptation methodologies to deal with the complexity of the spectrum of social and environmental extreme events that will occur in the coming decades as the range of threats are assessed, beyond just the focus on CO2 and a few other greenhouse gases as emphasized in the IPCC assessments.

This need to develop a broader approach was even endorsed in the climate research assessment and recommendations in the “Report Of The 2004-2009 Research Review Of The Royal Netherlands Meteorological Institute”.

In this 2011 report, we wrote

The generation of climate scenarios for plausible future risk, should be significantly broadened in approach as the current approach assesses only a limited subset of possible future climate conditions. To broaden the approach of estimating plausible changes in climate conditions in the framing of future risk, we recommend a bottom-up, resource-based vulnerability assessment for the key resources of water, food, energy, human health and ecosystem function for the Netherlands. This contextual vulnerability concept requires the determination of the major threats to these resources from climate, but also from other social and environmental issues. After these threats are identified for each resource, then the relative risk from natural- and human-caused climate change (estimated from the global climate model projections, but also the historical, paleo-record and worst case sequences of events) can be compared with other risks in order to adopt the optimal mitigation/adaptation strategy.

Since the 2011 report (which I was a member of the Committee that wrote it), I now feel that using the global climate model projections, downscaled or not, to provide regional and local impact assessment on multi-decadal time scales is not an effective use of money and other resources. If the models cannot even accurately simulate current climate statistics when they are not constrained by real world data, the expense to run them to produce detailed spatial maps is not worthwhile. Indeed, it is counterproductive as it provides the impact community and policymakers with an erroneous impression on their value.

A robust approach is to use historical, paleo-record and worst case sequences of climate events. Added to this list can be perturbation scenarios that start with regional reanalysis (e.g. such as by arbitrarily adding a 1C increase in minimum temperature in the winter, a 10 day increase in the growing season, a doubling of major hurricane landfalls on the Florida coast, etc). There is no need to run the multi-decadal global and regional climate projections to achieve these realistic (plausible) scenarios.

Hopefully, our debate on this weblog will foster a movement away from the overselling of multi-decadal climate model projections to the impact and policy communities. I very much appreciate the opportunity to present my viewpoint in this venue.

Biosketch
Roger A. Pielke Sr. is currently a Senior Research Scientist in CIRES and a Senior Research Associate at the University of Colorado-Boulder in the Department of Atmospheric and Oceanic Sciences (ATOC) at the University of Colorado in Boulder (November 2005 -present). He is also an Emeritus Professor of Atmospheric Science at Colorado State University and has a five-year appointment (April 2007 - March 2012) on the Graduate Faculty of Purdue University in West Lafayette, Indiana.
Pielke has studied terrain-induced mesoscale systems, including the development of a three-dimensional mesoscale model of the sea breeze, for which he received the NOAA Distinguished Authorship Award for 1974. Dr. Pielke has worked for NOAA's Experimental Meteorology Lab (1971-1974), The University of Virginia (1974-1981), and Colorado State University (1981-2006). He served as Colorado State Climatologist from 1999-2006. He was an adjunct faculty member in the Department of Civil and Environmental Engineering at Duke University in Durham, North Carolina (July 2003-2006). He was a visiting Professor in the Department of Atmospheric Sciences at the University of Arizona from October to December 2004.
Roger Pielke Sr. was elected a Fellow of the AMS in 1982 and a Fellow of the American Geophysical Union in 2004. From 1993-1996, he served as Editor-in-Chief of the US National Science Report to the IUGG (1991-1994) for the American Geophysical Union. From January 1996 to December 2000, he served as Co-Chief Editor of the Journal of Atmospheric Science. In 1998, he received NOAA's ERL Outstanding Scientific Paper (with Conrad Ziegler and Tsengdar Lee) for a modeling study of the convective dryline. He was designated a Pennsylvania State Centennial Fellow in 1996, and named the Pennsylvania State College of Earth and Mineral Sciences Alumni of the year for 1999 (with Bill Cotton). He is currently serving on the AGU Focus Group on Natural Hazards (August 2009-present) and the AMS Committee on Planned and Inadvertent Weather Modification (October 2009-present). He is among one of three faculty and one of four members listed by ISI HighlyCited in Geosciences at Colorado State University and the University of Colorado at Boulder, respectively.
Dr. Pielke has published over 370 papers in peer-reviewed journals, 55 chapters in books, co-edited 9 books, and made over 700 presentations during his career to date. A listing of papers can be viewed at the project website: http://cires.colorado.edu/science/groups/pielke/pubs/. He also launched a science weblog in 2005 to discuss weather and climate issues. This weblog was named one of the 50 most popular Science blogs by Nature Magazine on July 5, 2006 and is located at http://pielkeclimatesci.wordpress.com/.

References

Castro, C.L., R.A. Pielke Sr., and G. Leoncini, 2005: Dynamical downscaling: Assessment of value retained and added using the Regional Atmospheric Modeling System (RAMS). J. Geophys. Res. - Atmospheres, 110, No. D5, D05108, doi:10.1029/2004JD004721. http://pielkeclimatesci.wordpress.com/files/2009/10/r-276.pdf

Castro, C.L., R.A. Pielke Sr., J. Adegoke, S.D. Schubert, and P.J. Pegion, 2007: Investigation of the summer climate of the contiguous U.S. and Mexico using the Regional Atmospheric Modeling System (RAMS). Part II: Model climate variability. J. Climate, 20, 3866-3887. http://pielkeclimatesci.wordpress.com/files/2009/10/r-307.pdf

Mearns, Linda O. , Ray Arritt, Sébastien Biner, Melissa S. Bukovsky, Seth McGinnis, Stephan Sain, Daniel Caya, James Correia, Jr., Dave Flory, William Gutowski, Eugene S. Takle, Richard Jones, Ruby Leung, Wilfran Moufouma-Okia, Larry McDaniel, Ana M. B. Nunes, Yun Qian, John Roads, Lisa Sloan, Mark Snyder, 2012: The North American Regional Climate Change Assessment Program: Overview of Phase I Results. Bull. Amer.Met Soc. September issue. pp 1337-1362.

Mearns, L.O. , R. Leung, R. Arritt, S. Biner, M. Bukovsky, D. Caya, J. Correia, W. Gutowski, R. Jones, Y. Qian, L. Sloan, M. Snyder, and G. Takle 2013: Reply to R. Pielke, Sr. Commentary on Mearns et al. 2012. Bull. Amer. Meteor. Soc., in press. doi: 10.1175/BAMS-D-13-00013.1. http://pielkeclimatesci.files.wordpress.com/2013/02/r-372a.pdf

Pielke, R.A. Jr., 2010: The Climate Fix: What Scientists and Politicians Won't Tell You About Global Warming. Basic Books. http://www.amazon.com/Climate-Fix-Scientists-Politicians-Warming/dp/B005CDTWBS#_

Pielke Sr., R.A., 2002: Overlooked issues in the U.S. National Climate and IPCC assessments. Climatic Change, 52, 1-11. http://pielkeclimatesci.wordpress.com/files/2009/10/r-225.pdf

Pielke Sr., R.A.,, 2004: A broader perspective on climate change is needed. Global Change Newsletter, No. 59, IGBP Secretariat, Stockholm, Sweden, 16–19. http://pielkeclimatesci.files.wordpress.com/2009/09/nr-139.pdf

Pielke Sr., R., K. Beven, G. Brasseur, J. Calvert, M. Chahine, R. Dickerson, D. Entekhabi, E. Foufoula-Georgiou, H. Gupta, V. Gupta, W. Krajewski, E. Philip Krider, W. K.M. Lau, J. McDonnell, W. Rossow, J. Schaake, J. Smith, S. Sorooshian, and E. Wood, 2009: Climate change: The need to consider human forcings besides greenhouse gases. Eos, Vol. 90, No. 45, 10 November 2009, 413. Copyright (2009) American Geophysical Union. http://pielkeclimatesci.wordpress.com/files/2009/12/r-354.pdf

Pielke Sr., R.A., 2010: Comments on “A Unified Modeling Approach to Climate System Prediction”. Bull. Amer. Meteor. Soc., 91, 1699–1701, DOI:10.1175/2010BAMS2975.1, http://pielkeclimatesci.files.wordpress.com/2011/03/r-360.pdf

Pielke Sr, R.A., 2013a: Mesoscale meteorological modeling. 3rd Edition, Academic Press, in press.

Pielke Sr., R.A. 2013b: Comment on “The North American Regional Climate Change Assessment Program: Overview of Phase I Results.” Bull. Amer. Meteor. Soc., in press. doi: 10.1175/BAMS-D-12-00205.1. http://pielkeclimatesci.files.wordpress.com/2013/02/r-372.pdf

Pielke, R. A. Sr.and L. Bravo de Guenni, 2004: Conclusions. Vegetation, Water, Humans and the Climate: A New Perspective on an Interactive System, P. Kabat et al., Eds., Springer, 537–538. http://pielkeclimatesci.files.wordpress.com/2010/01/cb-42.pdf

Pielke Sr., R.A., and R.L. Wilby, 2012: Regional climate downscaling – what’s the point? Eos Forum, 93, No. 5, 52-53, doi:10.1029/2012EO050008. http://pielkeclimatesci.files.wordpress.com/2012/02/r-361.pdf

Pielke Sr., R.A., R. Wilby, D. Niyogi, F. Hossain, K. Dairaku, J. Adegoke, G. Kallos, T. Seastedt, and K. Suding, 2012: Dealing with complexity and extreme events using a bottom-up, resource-based vulnerability perspective. Extreme Events and Natural Hazards: The Complexity Perspective Geophysical Monograph Series 196 © 2012. American Geophysical Union. All Rights Reserved. 10.1029/2011GM001086. http://pielkeclimatesci.files.wordpress.com/2012/10/r-3651.pdf

Pielke Sr, R.A., Editor in Chief., 2013: Climate Vulnerability, Understanding and Addressing Threats to Essential Resources, 1st Edition. J. Adegoke, F. Hossain, G. Kallos, D. Niyoki, T. Seastedt, K. Suding, C. Wright, Eds., Academic Press, 1570 pp. Available May 2013.

Pielke et al, 2013: Introduction. http://pielkeclimatesci.files.wordpress.com/2013/04/b-18intro.pdf

Veljovic, K., B. Rajkovic, M.J. Fennessy, E.L. Altshuler, and F. Mesinger, 2010: Regional climate modeling: Should one attempt improving on the large scales? Lateral boundary condition scheme: Any impact? Meteorologische Zeits., 19:3, 237-246. DOI 10.1127/0941-2948/2010/0460.

Leave a Reply

Expert comments to Are regional models ready for prime time?

Jump to public comments | Jump to off-topic comments
  • First comments on the guest blog of Bart van den Hurk:

    Bart begins by discussing the ability of regional climate models to add value (over global models) to a regional climate projection. Despite some confusion about whether an RCM is providing a prediction or a projection (I suggest in this context we are only talking about projections), I largely agree with his assessment. There are plenty of publications that demonstrate added value from RCMs. As he points out, this doesn’t guarantee a better regional climate simulation however, and some aspects, such as trends, can be quite sensitive to the lateral boundary conditions from global models (an inherent limitation for RCMs). So RCMs are not perfect but in many cases are good enough to be useful.

    Bart goes on to discuss some other applications of RCMs that come from the fact that they are “collections of our understanding of the regional climate system”. As a climate scientist, one of the main ways I use RCMs is to explore questions around various climate processes in a manner similar to that described. I would even take this one step further and suggest that even when using RCMs to provide regional climate projections, they can be used to identify the climate processes most responsible for any changes seen in the projection. Thus, they can help us better understand why some aspect of the climate may be changing, which may be more important than knowing the change itself.

  • First comments on the guest blog of Roger Pielke Sr.:

    Roger presents a case for abandoning climate model projections altogether – or at least not allowing the impact/adaptation community and policy makers to see them as they will gain “an erroneous impression on their value.” In my experience this is certainly the case if you talk about the simulations as predictions rather than projections – the climate models are not predicting what the weather will be on the 5th of May 2051 – they are providing projections of the climate based on emission scenarios and initial conditions. When dealing with those outside the climate science community I have certainly found the distinction to be important.

    The climate modelling community has gone to great lengths to explore, understand and quantify the uncertainty associated with climate model projections. For GCMs, the CMIP range of international collaborative experiments have produced large ensembles of simulations which are freely available for people to download and investigate. For regional models, similar ensembles of simulations have been assembled through projects like PRUDENCE, ENSEMBLES, NARCCAP, and CORDEX. In each case a significant amount of work has gone into quantifying and trying to understand the causes for the uncertainty within the ensembles.

    When Roger says “the output of these models are routinely being provided to the impact communities and policymakers as robust scientific results” … I don’t think he means the large model ensembles with their included uncertainty.

    I have worked with impacts/adaptation researchers for some time now. At first they were very resistant to dealing with an ensemble of future climate projections that embodied significant uncertainty. They wanted a single model projection that was the “truth” to apply in their own impacts models. In the early impacts literature you do indeed see many circumstances where a single climate projection is used as the “truth” to explore climate change impacts. My guess is that this is closer to the situation that Roger is referring to. The impacts research community is becoming much better at dealing with large ensembles that embody uncertainty however, and the one model “truth” is becoming a thing of the past.

    After a summary of his 2012 paper, Roger argues that climate models perform poorly in terms of modelling a realistic recent past and presents quotes from some papers to support this claim. While there are certainly problems with GCMs, and these papers point out a few of them, there were 100s of relevant papers published over the same period and many of them show good performance. In fact, the majority of published works will point out both when the models perform well and when they perform poorly. This is a necessary step in the model development and improvement process. In this process, pointing out the problems is the easy part, finding the solutions is where the hard work is done.

    He finishes by summarizing more of his 2012 paper and concluding “It is therefore inappropriate to present type 4 results to the impacts community as reflecting more than a subset of possible (plausible) future climate risks.” Indeed by producing large ensembles, for multiple different emission scenarios, we do present the climate projections as a set of possible future climate risks, with associated uncertainty. Providing such projections also does not preclude the use of a bottom-up vulnerability assessment either as shown by the inclusion of climate model projections in the Nederlands Meteorologisch Instituut report that Roger quotes.

    In the end, climate models are our best tools for understanding how the climate system works. As climate scientists, we will continue to use these tools to improve our understanding of the climate system, and use our understanding of the system to improve these tools. Part of this includes exploring the impact of changing levels of greenhouse gases on the climate by creating future climate projections.

    So long as climate projections that embody our best understanding of the climate system exist, it is not surprising that others involved in climate change impacts/adaptation research, and policy makers, would want to consider these projections in their work. Because of the open nature of climate research and the freely available model data, they can do this (and have done this) without directly working with climate scientists or understanding the uncertainties and limitations associated with the data. I think the real way forward is for closer collaboration between climate scientists and the impacts/adaptation and policy making communities – not less. In this way the climate scientists can ensure that the projections are understood with their associated uncertainties, and the impacts/adaptation researchers can ensure that the climate scientists are aware of the parts of the climate system they are most sensitive to, providing a focus for climate model development efforts.

    • Rasmus Benestad

      I have a remark to Jason and the limitations of downscaling,

      “…they assume that the derived statistical relationship will not change due to climate change.

      Dynamical downscaling, or the use of Regional Climate Models (RCMs), does not share the limitations of statistical downscaling.”

      I think that RCMs very much share the potential caveat of non-stationarity, as they too involve statistically fitted models, known as parameterisation schemes. The bulk effects of unresolved small-scale processes are essentially represented by statistical models, e.g. clouds and surface fluxes. What’s more, imperfections in these feed back into the calculations in RCMs, whereas such discrepancies merely cause a growing bias in the statistical schemes.

      It is also possible to evaluate the downscaling for these effects, e.g. by applying them to the past. Furthermore, since the RCMs and statistical downscaling draw information from independent sources, and one need to look at both (do they converge/diverge) before using the results for decision making.

  • Bart van den Hurk

    First comments on the guest blog of Jason Evans:

    In Jason Evans’ blog the methodological and conceptual components of downscaling and skill assessment are nicely illustrated. However, I tend to disagree with the underlying rationale that a continuous development of models and their frequent confrontation with observations is going to reduce the uncertainty in the projections. First of all, it is not uncertainty that is the driving principle behind depicting a plausible range of future conditions, it is the range of likelihoods that is of interest. For many decision making processes (certainly those that dominate in the public political agenda) uncertainty is not a bottleneck for decision making. Instead, this process aims at keeping all options open as long as possible (that’s why most “important” decisions are taken near or past a deadline: to ensure that as many as possible options can be considered).

    Second, I think climate projections do not only affect decisions when they are “out of range of current conditions”, but merely when they provide better insight in the processes that are responsible for the decision. Also a system component (climate, population, water requirement….) that does not change can affect the decision taken, if the underlying picture of (lack of) change is convincing enough.

    And third, development of models not rarely is associated with the realization that new interfaces or boundary conditions, that appear to play a role, need to be included. This drives the evolution from atmospheric models to physical climate models to earth system models. Adding new boundary conditions or feedbacks does generally increase the number of degrees of freedom, which is difficult to reconcile with the notion of “reduced uncertainty”. Indeed, the range of temperature projections of CMIP3 does not strongly differ from CMIP5, although the modellers do claim that their models have improved and the level of understanding has increased (and thus inherent uncertainty has decreased).

  • Bart van den Hurk

    First comments on the guest blog of Roger Pielke Sr.:

    Roger Pielke provides an impressive list of studies demonstrating the lack of skill of (global and regional) climate models to predict changes in climate characteristics at the regional scale. But, as I have illustrated in my blog, a systematic bias in current day climate models – which is indeed well documented and agreed – does not make them useless for future assessments, which I think is the crucial element in this discussion.

    Roger considers projections and predictions to be synonyms, but I disagree with this, and also feel that this assumption is an important element in the controversy under discussion. A prediction is a statement on the expected situation in the near future starting from a current state of the system, while a projection is a possible evolution of the system given an assumed (external) driver. A climate prediction focuses at the predictable time scale, which depends a lot on the spatial scope. Making a prediction of the future global mean temperature is a lot easier than predicting it at the regional scale. And even there, I agree with the statement of v Oldenborgh et al (2012) that the prediction of mean temperature at the regional scale can be done fairly well given the robust temperature trend.

    The targets set out by CMIP5, listed by Roger, do not speak of predictions, and that is not without reason. I agree that the ambition to make (regional) climate predictions even at decadal or longer time scales can not be supported by the current apparent feasibility, given the studies that demonstrate the lack of predictive skill. There are some source of predictability that are still not fully resolved (including those dealing with improving climate models, but also related to unexplored initial conditions or driving conditions), and a great benefit of these predictability studies is that they mimic the practice of weather prediction by confronting models with observations at the relevant time and spatial scales, leading to the necessary inspiration for this model improvement. But even with perfect models, climate predictability will always be constrained by unknown evolutions of climate forcings, both anthropogenic and non-anthropogenic.

    In his summary he claims that regional climate downscaling fails to provide additional scale. As is illustrated by Jason Evans’ blog (and also mine) this is not fully true: there where local information (mountains, coasts, land use) gives new information, a more reliable projection can be produced.

    The purpose of a projection is to depict the possible (plausible) evolution of the system. To my opinion, the process of decision making is not dependent on the (quantitative) predictions provided by climate models, but by the plausibility that the future will bring situations to which the current system is not well adapted. For that a set of imperfect projections is far more useful than a single optimized prediction (which, like a weather forecast with a perfect model, can still be wrong). Decision making does make use but does not entirely rely on this climate information.

    Maybe more important is the ability to feed the imagination, and to denote the relevance of the projection for the system under study. I fully embrace Pielke’s plea for a system analysis that takes the vulnerability of the system as a starting point. But from this kind of analyses, frequently the stakeholders are the participants that ask for support from (regional) climate models to illustrate the possible alternative future conditions. An approach where “real” weather events, in combination with their local impact on the sector or area at steak is increasingly adopted in climate research, including at KNMI, partly inspired by Rogers recommendations in his role as member of the recent review committee. But also this “future weather” events need to be framed, to avoid that the producers of these event based scenarios apply cherry picking to depict the situation that best suites their interests or assumptions. A framing of these individual events is needed, and a top-down scenario approach is well capable of providing this framework, certainly when complemented by scenarios of other drivers than climate, such as population, water requirements, land use etcetera.

    An increasing number of studies succeed in adequately combining the valuable pieces of information coming from both climate and non-climatic models and assessments. A strong degradation of (regional) climate models by pointing at their lack of skill does not help the intelligent application of the inherent level of understanding that is generally captured by these models.

  • Roger Pielke Sr.

    First comments on the guest blog of Jason Evans (I added my comments to Jason’s article):

    Are climate models ready to make regional projections?

    Global Climate Models (GCMs) are designed to provide insight into the global climate system.

    I agree

    They have been used to investigate the impacts of changes in various climate system forcings such as volcanoes, solar radiation, and greenhouse gases, and have proved themselves to be useful tools in this respect.

    They clearly have not “proved” skill at predicting in a hindcast mode, changes in climate statistics on the regional scale, and even in terms of the global average surface temperature trend, in recent years they have overstated the positive trend.

    The growing interest in GCM performance at regional scales, rather than global, has come from at least two different directions: the climate modelling community; and the climate change adaptation community.

    Due, in part, to the ever increasing computational power available, GCMs are being continually developed and applied at higher spatial resolutions. Many GCM modelling groups increasing the resolution from ~250km grid boxes 7 years ago to ~100km grid boxes today.

    As one note, resolution and grid length are not the same. One needs at least 4 grid increments to reasonably resolve an atmospheric (and other climate) feature. See the discussion of this issue in

    Pielke, R.A., 1991: A recommended specific definition of “resolution”, Bull. Amer. Meteor. Soc., 12, 1914 http://pielkeclimatesci.files.wordpress.com/2009/09/nt-27.pdf

    Pielke Sr., R.A., 2001: Further comments on “The differentiation between grid spacing and resolution and their application to numerical modeling”. Bull. Amer. Meteor. Soc., 82, 699. http://pielkeclimatesci.wordpress.com/files/2009/10/r-241.pdf

    Laprise, R., 1992: The resolution of global spectral models. Bull. Amer. Meteor. Soc., 9, 1453-1454 http://pielkeclimatesci.files.wordpress.com/2009/09/nt-27a.pdf

    This model resolution increase leads naturally to model development and evaluation exercises that pay closer attention to smaller scales, in this case, regional instead of global scales. The Fifth Coupled Model Intercomparison Experiment (CMIP5) provides a large ensemble of GCM simulations, many of which are at resolutions high enough to warrant evaluation at regional scales. Over the next few years these GCM simulations will be extensively evaluated, problems will be found (as seen in some early evaluations1,2), followed hopefully by solutions that lead to further model development and improved simulations. This step of finding a solution to an identified problem is the hardest in the model development cycle, and I applaud those who do it successfully.

    I agree; these tests are essential. In my guest post, I presented examples of such CMIP5 comparisons using hindcast runs, and they are shown, even with respect to the current climate, to have major deficiencies.

    Probably the stronger demand for regional scale information from climate models is coming from the climate change adaptation community. Given only modest progress in climate change mitigation, adaptation to future climate change is required.

    Adaption to climate is always required. Moreover, climate is always changing. The separation into a “change” part is actually a redundancy. Adaptation to climate is required at all times.

    Some sectors, such as those involved in large water resource projects (e.g. building a new dam), are particularly vulnerable to climate change. They are planning to invest large amounts of money (millions) in infrastructure, with planned lifetimes of 50-100 years, that directly depend on climate to be successful. Over such long lifetimes, greenhouse gas driven climate change is expected to increase temperature by a few degrees, and may cause significant changes in precipitation, depending on the location. Many of the systems required to adapt are more sensitive to precipitation than temperature, and projections of precipitation often have considerably more uncertainty associated with them. The question for the climate change adaptation community is whether the uncertainty (including model errors) in the projected climate change is small enough to be useful in a decision making framework.

    The global models, by themselves , or dynamically and statistically downscaled, have shown no skill at predicting changes in the climate statistics of temperature, precipitation or other climate variables when examined in hindcast runs. If they cannot skilfully predict regional and local changes in these statistics for the past several decades, they are not ready to be given to the impacts communities as robust results for the coming decades.

    From a GCM perspective then, the answer to “Are climate models ready to make regional projections?” is two-fold. For the climate modelling community the answer is yes.

    This is just an engineering answer. The mechanics of the models produce regional scale results, but, until the multi-decadal regional predictions of changes in climate statistics can be shown to be skilful, the added spatial resolution provides an erroneous illusion of skill.

    GCMs are being run at high enough resolution to make regional scale (so long as your regions are many 100kms across) evaluations and projections useful to inform the model development and hopefully improve future simulations.

    The use of the term “hopefully” illustrates the uncertainty. Science needs quantitative tests of skill, not a presentation of results based on “hope”.

    For the climate change adaptation community, whose spatial scale of interest is often much lower than current high resolution GCMs can capture, the answer in general is no.

    I assume you mean here “whose spatial scale of interest is often much SMALLER than current high resolution GCMs can capture”. If so, we agree on this issue.

    The errors in the simulated regional climate and the inter-model uncertainty in regional climate projections from GCMs is often too large to be useful in decision making. These climate change adaptation decisions need to be made however, and in an effort to produce useful regional scale climate information that embodies the global climate change a number of “downscaling” techniques have been developed.

    Adaptation decisions do not need to have robust predictions. They often make decisions without predictive skill. We discuss this concepts as “contextual vulnerability” in our article

    Pielke Sr., R.A., R. Wilby, D. Niyogi, F. Hossain, K. Dairaku, J. Adegoke, G. Kallos, T. Seastedt, and K. Suding, 2012: Dealing with complexity and extreme events using a bottom-up, resource-based vulnerability perspective. Extreme Events and Natural Hazards: The Complexity Perspective Geophysical Monograph Series 196 © 2012. American Geophysical Union. All Rights Reserved. 10.1029/2011GM001086. http://pielkeclimatesci.files.wordpress.com/2012/10/r-3651.pdf

    The approach that you are presenting is, in contrast, a top-down global climate model approach to assess vulnerability, which is referred to as “outcome vulnerability”. We show that this method is too limiting, and if it is based on model results that show no skill at predicting changes in the climate statistics that matter to the impacts communities, than it is actually misleading them.

    It is worth noting that some climate variables, such as temperature, tend to be simulated better by climate models than other variables, such as precipitation. This is at least partly due to the scales and non-linearity of physical processes which effect each variable. This is demonstrated in the fourth IPCC report which mapped the level of agreement in the sign of the change in precipitation projected by GCMs. This map showed large parts of the world where GCMs disagreed about the sign of the change in precipitation. However this vastly underestimated the agreement between the GCMs3. They showed that much of this area of disagreement is actually areas where the GCMs agree that the change will be small (or zero). That is, if the actual projected change is zero then by chance, some GCMs will project small increases and some small decreases. This does not indicate disagreement between the models, rather they all agree that the change is small.

    This is a flawed claim for model predictive robustness. The models can all agree and yet differ from reality. The only proper way to test the models (which are, after all, hypotheses), is to run them in hindcast on multi-decadal time periods. As I show in my guest post, the CMIP models not only have not shown skill at predicting (in hindcast) regional changes in climate statistics, but often not even the current average climate!

    Regional climate models

    Before describing the downscaling techniques, it may be useful to consider this question: What climate processes, that are important at regional scales, may be missing in GCMs?

    The first set of processes relates directly to how well resolved land surface features such as mountains and coastlines are. Mountains cause local deviations in low level air flow. When air is forced to rise to get over a mountain range it can trigger precipitation, and because of this, mountains are often a primary region for the supply of fresh water resources. At GCM resolution mountains are often under-represented in terms of height or spatial extent and so do not accurately capture this relationship with precipitation. In fact, some regionally important mountain ranges, such as the eastern Mediterranean coastal range or the Flinders Range in South Australia, are too small to be represented at all in some GCMs. Using higher spatial resolution to better resolve the mountains should improve the models ability to capture this mountain-precipitation relationship.

    Similarly, higher resolution allows model coastlines to be closer to the location of actual coastlines, and improves the ability to capture climate processes such as sea breezes.

    For Type 1 and 2 (and to a more limited extent in Type 3) downscaling, the additional terrain and landscape resolution adds significant skill. However, type 4 downscaling, while providing the illusion of higher skill because of the high spatial resolution climate fields, has never shown skill at prediction beyond what is already there in the parent global model. Indeed, since the only added information is the terrain and landscale (including land-water boundaries), how could this added information correct for and/or refine the predictions from the global model, unless the input from the global model is correct?

    The second set of processes are slightly more indirect and often involve an increase in the vertical resolution as well. These processes include the daily evolution of the planetary boundary layer, and the development of low level and mountain barrier jets.

    Vertical resolution is a necessary but not a sufficient requirement for more skilful model simulations.

    A simple rule of thumb is that one can expect downscaling to higher resolution to improve the simulation of regional climate in locations that include coastlines and/or mountain ranges (particularly where the range is too small to be well resolved by the GCM but large enough to be well resolved at the higher resolution) while not making much difference over large homogeneous, relatively flat regions (deserts, oceans,…).

    “A simple rule of thumb” is based on what scientific demonstration? This requires a quantitative comparison with real-world observations. These observations must be of changes in regional and local climate statistics. For current climate Type 2 downscaling from reanalyses (e.g. the NARR) is the best approach we have.

    So, there are physical reasons one might expect downscaling to higher resolution will improve the simulation of regional climate. How do we go about this downscaling?

    The “might expect” is a hypothesis. Where is your quantification of this improvement for Type 4 downscaling?

    Downscaling techniques can generally be divided into two types: statistical and dynamical. Statistical techniques generally use a mathematical method to form a relationship between the modelled climate and observed climate at an observation station. A wide variety of mathematical methods can be used but they all have two major limitations. First, they rely on long historical observational records to calculate the statistical relationship, effectively limiting the variables that can be downscaled to temperature and precipitation, and the locations to those stations where these long records were collected. Second, they assume that the derived statistical relationship will not change due to climate change.

    I agree. We discuss this is in our paper

    Pielke Sr., R.A., and R.L. Wilby, 2012: Regional climate downscaling – what’s the point? Eos Forum, 93, No. 5, 52-53, doi:10.1029/2012EO050008. http://pielkeclimatesci.files.wordpress.com/2012/02/r-361.pdf

    Rob Wilby is an expert on statistical downscaling, and its limitations.

    Dynamical downscaling, or the use of Regional Climate Models (RCMs), does not share the limitations of statistical downscaling. The major limitation in dynamical downscaling is the computational cost of running the RCMs.

    You have not differentiated between the types of downscaling. For the purpose of our guest posts, we are talking about Type 4 downscaling. The major limitations to adding value with Type 4 dynamic downscaling are much more than computational cost, as Rob and I summarized in our paper.

    This generally places a limit on both the spatial resolution (often 10s of kilometres) and the number of simulations that can be performed to characterise uncertainty. RCMs also contain biases, both inherited from the driving GCM and generated with the RCM itself. It is worth noting that statistical downscaling techniques can be applied to RCM simulations as easily as GCM simulations to obtain projections at station locations.

    The RCM limitations are actively being addressed through current initiatives and research.

    Please summarize papers that report on the added skill from Type 4 downscaling when run in hindcast. This is both for the current average climate statistics and the CHANGES of these statistics over the hindcast period of time.

    Like GCMs, RCMs benefit from the continued increase in computational power, allowing more simulations to be run at higher spatial resolution. The need for more simulations to characterise uncertainty is being further addressed through international initiatives to have many modelling groups contribute simulations to the same ensembles (e.g. CORDEX – COordinated Regional climate Downscaling EXperiment http://wcrp-cordex.ipsl.jussieu.fr/). New research into model independence is also pointing toward ways to create more statistically robust ensembles4. Novel research to reduce (or eliminate) the bias inherited from the driving GCM is also showing promise5,6.

    I summarize a number of papers in my post on the lack of skill of the CMIP5 runs when performed in hindcast. There remain major problems even with respect to current climate, much less with respect to changes in climate statistics.

    Above I used simple physical considerations to suggest there would be some added value from regional models compared to global models. Others have investigated this from an observational viewpoint7,8 as well as through direct evaluation of model results at different scales9,10,11. In each case the results agreed with the rule of thumb given earlier. That is, in the areas with strong enough regional climate influences we do see improved simulations of regional climate at higher resolutions.

    These studies are not Type 4 applications when run in a hindcast mode (which is the only scientific way (i.e. the comparison with actual real world observations) to evaluate their skill.

    Of course we are yet to address the question of whether the regional climate projections from these models have low enough uncertainty to be useful in climate change adaptation decision making.

    What level of uncertainty can the impacts community accept? We summarized these requirements in our article http://pielkeclimatesci.files.wordpress.com/2013/04/b-18intro.pdf

    in Pielke Sr, R.A., Editor in Chief., 2013: Climate Vulnerability, Understanding and Addressing Threats to Essential Resources, 1st Edition. J. Adegoke, F. Hossain, G. Kallos, D. Niyoki, T. Seastedt, K. Suding, C. Wright, Eds., Academic Press, 1570 pp. Available May 2013. http://store.elsevier.com/product.jsp?isbn=9780123847034

    The questions with respect to a specific resource are:

    1. Why is this resource important? How is it used? To what stakeholders is it valuable?

    2. What are the key environmental and social variables that influence this resource

    3. What is the sensitivity of this resource to changes in each of these key variables? This includes, but is not limited to, the sensitivity of the resource to climate variations and change on short (e.g., days); medium (e.g., seasons) and long (e.g., multi‐decadal) time scales. What changes (thresholds) in these key variables would have to occur to result in a negative (or positive) response to this resource?

    4. What are the best estimates of the probabilities for these changes to occur? What tools are available to quantify the effect of these changes? Can these estimates be skillfully predicted?

    5. What actions (adaptation/mitigation) can be undertaken in order to minimize or eliminate the negative consequences of these changes (or to optimize a positive response)?

    6. What are specific recommendations for policymakers and other stakeholders?

    To date RCMs have been used in many studies12,13 and a wide variety of evaluations of RCM simulations against observations have been performed. In attempting to examine the fidelity of the regional climate simulated, a variety of variables (temperature, precipitation, wind, surface pressure,…) have been evaluated using a variety of metrics14,15, with all the derived metrics then being combined to produce an overall measure of performance. When comprehensive assessments such as these are performed it is often found that different models have different strengths and weaknesses as measured by the different metrics. If one has a specific purpose in mind, e.g. building a new dam, one may wish to focus on metrics directly relevant to that purpose. Often the projected climate change is of interest so the evaluation should include a measure of the models ability to simulate change, often given as a trend over a recent historical period16,17. In most cases the RCMs are found to do a reasonably good job of simulating the climate of the recent past.

    Please provide specific demonstration of the skill of Type 4 downscaling. What has been done, for example, such as by Mearns et al 2012, is to report on Type 2 downscaling results (in which there is added skill) and then to infer this provides the measure for Type 4 downscaling. This is a flawed scientific approach.

    Mearns, Linda O. , Ray Arritt, Sébastien Biner, Melissa S. Bukovsky, Seth McGinnis, Stephan Sain, Daniel Caya, James Correia, Jr., Dave Flory, William Gutowski, Eugene S. Takle, Richard Jones, Ruby Leung, Wilfran Moufouma-Okia, Larry McDaniel, Ana M. B. Nunes, Yun Qian, John Roads, Lisa Sloan, Mark Snyder, 2012: The North American Regional Climate Change Assessment Program: Overview of Phase I Results. Bull.

    Amer.Met Soc. September issue. pp 1337-1362.

    I discuss their paper in my Comment

    Pielke Sr., R.A. 2013: Comment on “The North American Regional Climate Change Assessment Program: Overview of Phase I Results.” Bull. Amer. Meteor. Soc., in press. doi: 10.1175/BAMS-D-12-00205.1. http://pielkeclimatesci.files.wordpress.com/2013/02/r-372.pdf

    Their response is here

    Mearns, L.O. , R. Leung, R. Arritt, S. Biner, M. Bukovsky, D. Caya, J. Correia, W. Gutowski, R. Jones, Y. Qian, L. Sloan, M. Snyder, and G. Takle 2013: Reply to R. Pielke, Sr. Commentary on Mearns et al. 2012. Bull. Amer. Meteor. Soc., in press. doi: 10.1175/BAMS-D-13-00013.1. http://pielkeclimatesci.files.wordpress.com/2013/02/r-372a.pdf

    Though, there are usually places and/or times where the simulation is not very good. Not surprising for an active area of research and model development.

    Given the evaluation results found to date it is advisable to carefully evaluate each RCMs simulations before using any climate projections they produce. Being aware of where and when they perform well (or poorly) is important when assessing the climate change that model projects. It is also preferable for the projections themselves to be examined with the aim of understanding the physical mechanisms causing the projected changes. Good process level understanding of the causes behind the changes provides another mechanism through which to judge their veracity.

    I have already documented that the use of Type 2 downscaling (and Type 1 and 3) cannot be used with respect to Type 4, other than to provide an upper bound on possible skill with Type 4 downscaling or for sensitivity experiments. I also have provided examples of inadequate representation of current climate by the CMIP5 model runs in hindcast, which certainly must primarily be due to inadequate representation of climate physics.

    Finally we come to the question of whether regional climate projections should be used in climate change adaptation decisions concerning infrastructure development? In the past such decision were made assuming a stationary climate such that observations of the past were representative of the future climate. So the real question here is will the use of regional climate projections improve decisions made when compared to the use of historical climate observations?

    There has never been a “stationary climate”. Examples that document this includeRial, J., R.A. Pielke Sr., M. Beniston, M. Claussen, J. Canadell, P. Cox, H. Held, N. de Noblet-Ducoudre, R. Prinn, J. Reynolds, and J.D. Salas, 2004: Nonlinearities, feedbacks and critical thresholds within the Earth’s climate system. Climatic Change, 65, 11-38. http://pielkeclimatesci.wordpress.com/files/2009/10/r-260.pdf

    Lovejoy, S., 2013: Eos, Transactions American Geophysical Union Volume 94, Issue 1, 1 January 2013, Pages: 1–2, S. Lovejoy Article first published online : 2 JAN 2013, DOI: 10.1002/2013EO010001. http://onlinelibrary.wiley.com/doi/10.1002/2013EO010001/pdf

    If the projected regional change is large enough that it falls outside the historical record even when considering the associated model errors and uncertainties, then it may indeed impact the decision. Such decisions are made within a framework that must consider uncertainty in many factors other than climate including future economic, technological and demographic pathways. Within such a framework, risk assessments are performed to inform the decision making process, and the regional climate projections may introduce a risk to consider that is not present in the historical climate record. If this leads to decisions which are more robust to future climate changes (as well as demographic and economic changes) then it is worthwhile including the regional climate projections in the decision making process.

    Of course, this relies on the uncertainty in the regional climate projection being small enough for the information to be useful in a risk assessment process. Based on current models, this is not the case everywhere, and continued model development and improvement is required to decrease the uncertainty and increase the utility of regional climate projections for adaptation decision making.

    You write

    “Of course, this relies on the uncertainty in the regional climate projection being small enough for the information to be useful in a risk assessment process. Based on current models, this is not the case everywhere,…”

    More accurately, this should be written as

    Of course, this relies on the uncertainty in the regional TYPE 4 climate projection OF CHANGES IN CLIMATE STATISTICS IN THE COMING DECADES being small enough for the information to be useful in a risk assessment process. Based on current models, this is not the case ANYWHERE,…”

    Jason – thank you for this opportunity for constructive dialog on this subject. Roger

  • Roger Pielke Sr.

    First comments on the guest blog of Bart van den Hurk (I added my comments to Bart’s article):

    Regional downscaling of climate information is a popular activity in many applications addressing the assessment of possible effects of a systematic change of the climate characteristics at the local scale. Adding local information, not captured in the coarse scale climate model or observational archives, can provide an improved representation of the relevant processes at this scale, and thus yield additional information, for instance concerning topography, land use or small scale features such as sea breezes or organisation of convection. A necessary step in the application of tools used for this regional downscaling is a critical assessment of the quality of the tools: are regional climate models (RCMs), used for this climate information downscaling, good enough for this task?

    It is important to distinguish the various types of analyses that are carried out with RCMs. And likewise to assess the ability of the RCM to perform the task that is assigned to them. And these types of analyses clearly cover a wider range than plain prediction of the local climate!

    Regional climate prediction

    Pielke and Wilby (2012) discuss the lack of potential of RCMs to increase the skill of climate predictions at the regional scale. Obviously, these RCM predictions heavily rely on the quality of the boundary conditions provided by global climate models, and fail to represent dynamically the spatial interaction between the region of interest and the rest of the world. However, various “big brother” type experiments (in which the ability of RCMs to reproduce a filtered signal provided by the boundary conditions (Denis et al, 2002), for instance carried out by colleagues at KNMI) do show that a high resolution regional model can add value to a coarse resolution boundary condition by improving the spatial structure of the projected mean temperatures. Also the spatial structure of changes in precipitation linked to altered surface temperature by convection can be improved by using higher resolution model experiments, although the relative gain here is generally small (Di Luca et al, 2012).

    Higher spatial structure is not a sufficient demonstration of model skill. The benchmark with respect to downscaling should be:

    Does the downscaling (in this case Type 4 downscaling) provide a more accurate result of climate variables requested by the impacts communities than can be achieved by interpolating the global parent model prediction to the finer grid and landscape? Accuracy must be tested by comparing in hindcast runs of the models their ability to :

    1. Predict the current average climate statistics over this time period and
    1. Predict the CHANGES in these climate statistics during this time period.

    Van Oldenborgh et al (2013) point out that the spatial structure of the mean temperature trend in the recent CMIP5 model ensemble compares fairly well with observations, but anomalies from the mean temperature trend aren’t well captured. This uncertainty clearly limits the predictability of temperatures at the regional scale beyond the mean trend.

    You need to also report on what level of regional skill of the trends that they achieved. I present an extract from their paper in my guest post.

    Van Haren et al (2012) also nicely illustrate the dependence of regional skill on lateral boundary conditions: simulations of (historic) precipitation trends for Europe failed to match the observed trends when lateral boundary conditions were provided from an ensemble of CMIP3 global climate model simulations, while a much better correspondence with observations was obtained when reanalyses were used as boundary condition.

    This is an important result. The use of Reanalyses to drive downscaling is a Type 2 application. There is added skill for this type of downscaling. It does not, however, tell us the skill level with Type 4, other than the maximum level of skill possible from Type 4 runs.

    Thus, a regional prediction of a trend can only be considered to be skilful when the boundary forcings represent the signal to be forecasted adequately.

    Well said.

    And this does apply to mean temperature trends for most places in the world, but not for anomalies from these mean trends, nor for precipitation projections.

    I do not understand. This dependence on lateral boundary conditions must also apply to the skilful prediction of regional mean temperature trends.

    For regional climate predictability, the added value of RCMs should come from better resolving the relationship between mean (temperature) trends and key indicators that are supposedly better represented in the high resolution projections utilizing additional local information, such as temperature or precipitation extremes. Also here, evidence of adding skill is not univocally demonstrated. Min et al (2013) evaluate the ability of RCMs driven by reanalysis data to reproduce observed trends in European annual maximum temperatures, and conclude that there is a clear tendency to underestimate the observed trends.

    And this is just for Type 2 downscaling.

    For Southern Europe biases in maximum temperatures could be related to errors in the surface flux partitioning (Stegehuis et al, 2012), but no such relationship was found for NW Europe by Min et al (2013).

    Thus indeed, the limitations to predictability or regional climate information by RCMs as discussed by Pielke and Wilby (2012) and others are valid, and care must be taken while interpreting RCM projections as predictive assessments.

    We agree on this.

    But is this the only application of RCMs? Not really. We will discuss two other applications, together with the degree to which limitations in RCM skill apply and are relevant.

    Bottom up environmental assessments

    A fair point of critique to exploring a cascade of model projections ranging from the global scale down to the local scale of a region of interest to developers of adaptation or mitigation policies is the virtually unmanageable increase of the range of degrees of freedom, also addressed as “uncertainty”. Uncertainty arises from imperfect models, inherent variability, and unknown evolution of external forcings. And in fact the process of (dynamical) downscaling adds another level of uncertainty, related to the choice of downscaling tools and methodologies. The reverse approach, starting from the vulnerability of a region or sector of interest to changes in environmental conditions (Pielke et al, 2012), does not eliminate all sources of uncertainty, but allows a focus on the relevant part of the spectrum, including those elements that are not related to greenhouse gas induced climate change.

    We agree.

    But also here, RCMs can be of great help, not necessarily by providing reliable predictions, but also by supporting evidence about the salience of planned measures or policies (Berkhout et al, 2013). A nice example is a near flooding situation in Northern Netherlands (January 2012), caused by a combined occurrence of a saturated soil due to excessive antecedent precipitation, a heavy precipitation event in the coastal area and a storm surge with a duration of several days that hindered the discharge of excess water from the area. This is typically a “real weather” event that is not necessarily exceptional but does expose a local vulnerability to superfluous water. The question asked by the local water managers was whether the combination of the individual events (wet soil, heavy rain, storm surge) has a causal relationship, and whether the frequency of occurrence of compound events can be expected to change in the future. Observational analyses do suggest a link between heavy precipitation and storm surge, but the available dataset was too short to explore the statistical relationships in a relevant part of the frequency distribution. A large set of RCM simulations is now explored to increase the statistical sample, but – more importantly – to provide a physically comprehensive picture of the boundary conditions leading up to an event like this. By enabling the policy makers to communicate this physically comprehensive picture provides public support for measures undertaken to adapt to this kind of events. This exploration of model based – synthetic – future weather is a powerful method to assess the consequences of possible changes in regional climate variability for the local water management.

    I agree with the need to assess what is plausible, but if you agree with the Pielke and Wilby article, than the scenarios that you provide from the downscaled models may fall outside the range of what actually could occur. If one insists, they could be included, but there should be a disclaimer given to the policymakers that these regional forecasts have not shown skill when tested in a hindcast mode.

    Process understanding

    Apart from a tool to predict a system given its initial state and the boundary forcings on it, a model is a collection of our understanding of the system itself. Its usefulness is not limited to its ability to predict, but also to describe the dynamics of a system, governed by internal processes and interactions with its environment. Regional climate models should likewise be considered as “collections of our understanding of the regional climate system”. And can likewise be used to study this system, and learn about it. There are numerous studies where regional climate model studies have increased our understanding of the mechanism of the climate system acting on a regional scale. A couple of examples:

    • Strong trends in coastal precipitation, and particularly a series of extreme precipitation events in the Netherlands, could successfully be attributed to anomalies in sea surface temperature (SST) in the nearby North Sea (Lenderink et al, 2009). Strong SST gradients close to the coast needed to be imposed to the RCM simulations carried out to reveal this mechanism. The relationship between SSTs and spatial gradients in changes in (extreme) precipitation is an important finding for analysing necessary measures to anticipate future changes in the spatial and temporal distribution of rainfall in the country.
    • During the past century land use change has given rise to regional changes in the local surface climatology, particularly the mean and variability of near surface temperature (Pitman et al, 2012). A set of GCM simulations dedicated to quantify the effect of land use change relative to changes in the atmospheric greenhouse gas concentration over the past century revealed that the land use effect is largely limited to the area of land use change. Wramneby et al (2010) explored the regional interaction between climate and vegetation response using a RCM set-up, and highlighted the importance of this interaction for assessing the mean temperature response particularly at high latitudes (due to the role of vegetation in snow covered areas) and in water limited evaporation regimes (due to the role of vegetation in controlling surface evaporative cooling).
    • In many occasions the degree to which anomalies in the land surface affect the overlying atmosphere depends on the resolved spatial scale. As an example, Hohenegger et al (2009) investigated the triggering of precipitation in response to soil moisture anomalies with a set of regional models ranging in physical formulation and resolution. This issue that deserves a lot of attention in the literature due to the possible existence of (positive) feedbacks that may affect occurrence or intensity of hydrological extremes such as heatwaves. In her study, RCMs operating at the typical 25 – 50 km scale resolution tend to overestimate the positive soil moisture – precipitation feedback in the Alpine area, which is better represented by higher resolution models. It is a study that points at a possible mechanism that needs to be adequately represented for generating reliable projections.

    These are excellent studies. They are a type of sensitivity study. As I wrote in the 3rd Edition of my modelling book

    Pielke Sr, R.A., 2013: Mesoscale meteorological modeling. 3rd Edition, Academic Press, in press.

    There are three types of applications of these models: for process studies, for diagnosis and for forecasting.

    Process studies: The application of models to improve our understanding of how the system works is a valuable application of these tools. The term ”sensitivity study” can be used to characterize a process study. In a sensitivity study, a subset of the forcings and/or feedback of the system may be perturbed to examine its response. The model of the system might be incomplete and not include each of the important feedbacks and forcings. Marshall et al. (2004) is one example of this type of model application. This is also the type of modeling being used to make climate projections (predictions) in climate assessments such as the IPCC reports (e.g., see Pielke 2002a). This application of the models is made despite their inability to show multi-decadal regional and mesoscale skill in forecasting changes in climate statistics when run in a hindcast mode (e.g., see Pielke 2013, and also Section 13.5). These IPCC model runs are actually process model simulations.

    Diagnosis: The application of models, in which observed data is assimilated into the model, to produce an observational analysis that is consistent with our best understanding of the system as represented by the manner in which the fundamental concepts and parameterizations are represented. An example of this approach is the North American Regional Reanalysis [http://www.emc.ncep.noaa.gov/mmb/rreanl/].

    Forecasting: The application of models to predict the future state of the system. Forecasts can be made from a single realization, or from an ensemble of forecasts which are produced by slightly perturbing the initial conditions and/or other aspects of the model. An example of such predictions

    can be viewed at http://www.mmm.ucar.edu/wrf/users/forecasts.html.

    Each of these examples (and many more that can be cited) generates additional insight in the processes controlling local climate variability by allowing to zoom in on these processes using RCMs. They thus contribute to setting the research agenda in order to improve our understanding of drivers of regional change.

    I agree.

    Climate predictions versus climate scenarios

    The notion that a tool – an RCM – may possess shortcomings in its predictive skill, but simultaneously prove to be a valuable tool to support narratives that are relevant to policy making and spatial planning can in fact be extended to highlighting the difference between “climate predictions” and “climate scenarios”. Scenarios are typically used when deterministic or probabilistic predictions show too little skill to be useful, either because of the complexity of the considered system, or because of the fundamental limitations to its predictability (Berkhout et al, 2013). A scenario is a “what if” construction, a tool to create a mental map of the possible future conditions assuming a set of driving boundary conditions.

    To do these “what if” scenarios, the use of multi-decadal global climate model projections (downscaled or not) is an exorbitant way to achieve this goal. A much more robust approach is to use the impacts models themselves, and assess what are their critical thresholds. Then, if these thresholds potentially could be crossed in the real world (and we do not need a top-down model with no predictive skill to assess this), the use of the CMIP5 type modeling is overkill in terms of cost and peoples’ time. And even more importantly, it conveys a level of confidence in the scenarios that is not scientifically justified. We should make much more use of reanalysis as the basis to assess vulnerability.

    As an excellent example of how vulnerability could be assessed without using multi-decadal regional climate predictions (in this case for landfalling hurricanes), but using the historical record, I refer you to the paper

    Emanuel, K., S. Ravela, E. Vivant and C. Risi. 2006: A Statistical-Deterministic Approach to Hurricane Risk Assessment. BAMS. ftp://texmex.mit.edu/pub/emanuel/PAPERS/hurr_risk.pdf

    In this paper, they synthesized large numbers of storm tracks and then ran a deterministic hurricane intensity model along each track. They conclude

    “To the the extent that one has reliable characterizations of the interannual to interdecadel variation of atmospheric winds, for example, owing to El Niño…. the Atlantic Multidecadel Oscillation

    ….), or the North Atlantic Oscillation….and tropical cyclogenesis distributions,,,, [this approach] can be used to estimate …… hurricane wind risk.”

    While in a later paper

    Emanuel, K., R. Sundararajan, and J. Williams, 2008: Hurricanes and global warming: Results from downscaling IPCC AR4 simulations. Bull. Amer. Meteor. Soc. 89, 347-367. ftp://texmex.mit.edu/pub/emanuel/PAPERS/Emanuel_etal_2008.pdf

    they, unfortunately, apply the flawed Type 4 downscaling approach, their 2006 BAMS paper is a very important (and original) presentation of how we can much more accurately assess risk.

    For a scenario to be valuable it does not necessarily need to have predictive skill, although a range of scenarios can be and are being interpreted as a probability range for future conditions.

    They can then be interpreted as a “plausible range” (if they actually are possible). They do not provide a “probability”, however, unless one can show skill at predicting probabilities.

    A (single) scenario is mainly intended to feed someone’s imagination with a plausible, comprehensible and internally consistent picture.

    They are only consistent with the model they are run from. Unless they can show skill in terms of real world observations, they are not “consistent” with reality in terms of making a skillfil prediction.

    Used this way, also RCMs with limited predictive skill can be useful tools for scenario development and providing supporting narratives that generate public awareness or support for preparatory actions.

    But Type 4 regional downscaling has NO predictive skill.

    For this, the RCM should be trustworthy in producing realistic and consistent patterns of regional climate variability, and abundant application, verification and improving is a necessary practice.

    I agree. However, before they should be provided to the impacts communities as anything more than a model sensitivity study, they must be shown to have skill with respect to real world observations.

    Further developments of RCMs as a Regional Earth System Exploration tool, by linking the traditional meteorological models to hydrological, biogeophysical and socio-economic components, can further develop their usefulness in practice.

    As process studies that are closely linked to observational comparison, I agree with you.

    Bart – thank you for this opportunity for constructive dialog on this subject. Roger

  • Roger Pielke Sr.

    I appreciate Bart and Jason’s thoughtful replies to my post. I want to here discuss a few issues that they raise:

    First, the attempt to discriminate between a “prediction” and a “projection” is not appropriate. The use of the term “projection” with respect to added greenhouse gases, for example, is just a “what if” prediction (i.e. given that forcing). These models can be (and must be) tested in hindcast runs to assess their level of skill at predicting what actually occurred. This skill must be shown before the results of future runs on multi-decadal time scales (as “projections” can be trusted.

    I discuss the relationship (and misuse) of the terms “prediction” and “projection” im my paper

    Pielke Sr., R.A., 2002: Overlooked issues in the U.S. National Climate and IPCC assessments. Climatic Change, 52, 1-11. http://pielkeclimatesci.wordpress.com/files/2009/10/r-225.pdf

    See, for example, Figure 6 in that paper. In retrospect, I should have also emphasized that the small box” in the figure (i.e. a model result) can fall outside of the larger box (i.e. reality).

    Bart also writes

    “Roger considers projections and predictions to be synonyms”

    They are. The only difference is that a “projection” is a prediction if a certain forcing(s) is applied. These are more properly called “sensitivity” experiments. The only way we know they have skill is running them in hindcast where we know what the forcings are (within some level of uncertainty). This is where ensemble runs should be focused.

    Bart further writes

    “The purpose of a projection is to depict the possible (plausible) evolution of the system. To my opinion, the process of decision making is not dependent on the (quantitative) predictions provided by climate models, but by the plausibility that the future will bring situations to which the current system is not well adapted. For that a set of imperfect projections is far more useful than a single optimized prediction (which, like a weather forecast with a perfect model, can still be wrong).”

    Running the models to produce “plausbile” scenarios is reasonable, but it is very expensive in terms of time and other resources. Even more importantly, unless they can actually be shown to be “plausible”, it is not appropriate to present to the impacts community without the disclaimer that they have not shown skill at predicting the climate metrics of interest when the models are run in hindcast.

    My second comment is with respect to Jason’s post

    “After a summary of his 2012 paper, Roger argues that climate models perform poorly in terms of modelling a realistic recent past and presents quotes from some papers to support this claim. While there are certainly problems with GCMs, and these papers point out a few of them, there were 100s of relevant papers published over the same period and many of them show good performance.”

    Please list this models (those that show skill at predicting CHANGES in climate statistics on the globa, regional and local scales).

    With respect to Jason’s comment

    “When Roger says ‘the output of these models are routinely being provided to the impact communities and policymakers as robust scientific results’ … I don’t think he means the large model ensembles with their included uncertainty.”

    I am referring to these model runs. How can one quantify uncertainty without comparing against real world observations? This can only be done in hindcast runs. The impacts communities would like skillful changes in climate statistics of interest to them (e.g. changes in the tne year average precipitation for the Netherland, ect). Uncertainty cannot be properly assessed just by intermodel and intramodel comparisons.

    Jason/Bart – please list examples of the clmiate metrics that are required by the impacts communities (water resources, food, energy, human health, ecosystem function), and the level of skill that the models have shown in providing them when tested against real world data in hindcast runs.

    Bart writes

    “In his summary he claims that regional climate downscaling fails to provide additional scale. As is illustrated by Jason Evans’ blog (and also mine) this is not fully true: there where local information (mountains, coasts, land use) gives new information, a more reliable projection can be produced.”

    What is missed in the discussion is how just local information can improve the skill of Type 4 regional model simulations when they are so dependent on the lateral boundary conditions from the parent model? Skill can be added from Type 1 through 3 regional model runs because there is a real world constraint on the lateral boundaries. This is not the case for Type 4 runs. If the global model has significant errors in the larger scale field, such as the behavior of ENSO, the PDO, the NAO ect, there is no way a regional model can correct for these errors.

    Finally (for this post :-)), Jason writes

    “In the end, climate models are our best tools for understanding how the climate system works.”

    I disagree. Observations of the real world behavior of the climate system provide the best tool for understanding how the climate system works.

  • Roger Pielke Sr.

    Mr. Smith – With respect to your statement that this is all about semantics, you are missing the point. One of the three uses of models is to invesitgate “processes”. I have used models as such a tool thoughout my career. I agree an observation/model mix is required (e.g. such as with the reanalyses), but the absence of observational validation is what is a fundamental flaw in the use of Type 4 downscaling for multi-decadal impact assessments, as I have explained in detail in my posts.

    To present regional multi-decadal climate projections to the impact communities as part of their driving forces and boundary conditions (for their models and process studies), when there is NO skill on this time scale at predicting changes in climate statistics, is a serious misleading application of the scientific method.

  • Bart van den Hurk

    I would like to come back to the usefulness of (imperfect) models for depicting possible future climatic conditions, and illustrate my case around the concept of Future Weather.

    Roger states that one cannot consider climate model predictions (his type 4) at the regional scale when their predictive skill in hindcast mode is not demonstrated. The kind of (model supported) scenario construction that is deserving a lot of attention nowadays is the generation of synthetic weather events using a climate model, but cast in a future setting by adjusting the boundary conditions driving the climate system (greenhouse gas, aerosol, land use, …). It allows to imagine conditions to which society may be vulnerable (e.g., a combination of extreme precipitation and a storm surge that hinders discharging the abundant water) but have never occurred yet in the historical record. Running a large ensemble of atmospheric model simulations for present day climate conditions can help to build statistics of this kind of complex multivariate events, and these statistics can to some extend be validated by comparing the generated distribution to the climatological distribution. “To some extend”, because the limited length of the observational record does not allow any estimate of very rare events, which in contrast can be done by a long model integration. Highly relevant for impact and vulnerability assessment, since these statistics enter the norms and design criteria of protection measures, for instance. During this design process the question of the consequence of climate change frequently emerges, as one desires to avoid over-, re- or underinvestment. A rerun of the instrument to generate synthetic weather under future conditions may give insight to the consequence of changes in large scale forcings on the relevant statistics. The weather model used can and has been tested very frequently when applied in its weather forecasting application mode (type 1) and statistics of these in hindcasts (type 2), but are now extended to an application that is to my opinion a blend of type 2 and type 4 forecasts. It is not the pure prediction of the (regional) trends that is aimed for (type 4), but the occurrence of certain (relevant) weather types conditioned on an assumed change in the large scale forcing.

    This is an illustration of where predictive skill at the regional scale can be limited (in terms of predicting the accurate trends of the local climate conditions), but where still useful information can be extracted from models applied to these future conditions. The models are credible to the extend they are skillfull in making good weather forecasts, or rather, in representing the climate processes that play a dominant role in generating (regional) climate variability, and this credibility is clearly present and on the rise.

  • Bart van den Hurk

    “In the end climate models are our best tools for understanding how the climate system works”. This is disputed by Roger, who refers to observations that best describe the working of the real world. To my opinion, any observation is supported by a conceptual model. The notion of a spatial structure of temperatures, for instance, makes no sense without even a faint idea of the notion of latitude, seasonality, altitude, land, weather.

    I don’t think we build our phyical comprehension of the real world solely on observations, we build it on observations that feed a conceptual model of this real world. And in that sense, a climate model is nothing more or less than a conceptual model, necessary to give a frame of reference to the state of the real world, which can never be generated by observations alone (which are incomplete, may be inconsistent, contradicting each other, biased, non-representative, …).

  • Bart van den Hurk

    The theory of Makarieva has been proved wrong by Meesters et al (2009), HESS (see http://www.hydrol-earth-syst-sci.net/13/1299/2009/hess-13-1299-2009.html).

  • Roger Pielke Sr.

    Hi Bart – Thank you for your further comment. You write

    “I don’t think we build our physical comprehension of the real world solely on observations, we build it on observations that feed a conceptual model of this real world. And in that sense, a climate model is nothing more or less than a conceptual model, necessary to give a frame of reference to the state of the real world, which can never be generated by observations alone (which are incomplete, may be inconsistent, contradicting each other, biased, non-representative, …).”

    I agree. We need a modeling construction, based on the accepted principles of physics, to permit the understanding of the interelationship between variables. The climate models are “concepts”, as you wrote, but they are then, therefore, hypotheses. With hypotheses, the following requirements are needed as I wrote in my weblog post http://pielkeclimatesci.wordpress.com/2010/11/15/hypothesis-testing-a-failure-in-the-2007-ipcc-reports/.

    A useful summary of the scientific method is given on the website sciencebuddies.org.where they list six steps

    ■ Ask a Question

    ■ Do Background Research

    ■ Construct a Hypothesis

    ■ Test Your Hypothesis by Doing an Experiment

    ■ Analyze Your Data and Draw a Conclusion

    ■ Communicate Your Results

    While one cannot falsify models, one can falsify model predictions. With respect to climate, multi-decadal model predictions can be falsified as models themselves are hypotheses and can be tested. They are, after all, engineering code, as large parts of the physics, chemistry and biology are parameterized using tunable parameters. Only the dynamic core of these models (i.e. advection, the pressure gradient force, gravity) are expressed in terms of fundamental physics.

    As I document in my post in our current debate, the multi-decadal regional and local climate model predictions have been falsified in terms of lacking skill at predicting (in hindcast) important climate variables. Even in terms of the global annual average surface temperature trends, they are close to failing as recent data is examined (e.g see http://rankexploits.com/musings/2013/individual-model-tests-jan-2001-march-2013/)

    Finally, I reiterate my request for you and Jason to present papers that document a skill of the multi-decadal (Type 4) regional climate models to predict (in hindcast) the observed CHANGES in climate statistics over this time period.

    Until they can show skill (which they have not, in any study that I am aware of), their use to provide multi-decadal predictions (projections) of climate conditions in the future for the impacts and policy communities is inappropriate and misleading.

  • Roger Pielke Sr.

    Hi Bart

    You write

    “The kind of (model supported) scenario construction that is deserving a lot of attention nowadays is the generation of synthetic weather events using a climate model, but cast in a future setting by adjusting the boundary conditions driving the climate system (greenhouse gas, aerosol, land use, …). It allows to imagine conditions to which society may be vulnerable (e.g., a combination of extreme precipitation and a storm surge that hinders discharging the abundant water) but have never occurred yet in the historical record..”

    In my view you are making an assumption without a robust foundation. You have not demonstrated that these “synthetic weather events” are actually possible in the real world. They are accepted only because a model produces these events. Yet, if these models cannot skillfully predict changes in climate statistics in the past few decades, there is no basis to accept their prediction of changes in weather events (as synthetic weather) in the future as realistic.

    If the impact community insists on using them, they should have the clear disclaimer sent along with the results that states “these climate results are presented without any demonstration of skill at predicting (projecting) changes in climate statistics when run over the last few decades”.

    In terms of generating synthetic weather events, I presented an example by Kerry Emanuel -

    Emanuel, K., S. Ravela, E. Vivant and C. Risi. 2006: A Statistical-Deterministic Approach to Hurricane Risk Assessment. BAMS. ftp://texmex.mit.edu/pub/emanuel/PAPERS/hurr_risk.pdf

    This uses the historical record (such as reanalyses) to introduce many more plausible landfalling hurricanes than actually occurred, in order to assess better risk.

    The proper question we should ask, of course, should be:

    What are the thresholds of climate changes that must occur before a negative effect occurs for particular key resources?

    As one of the next step in our debate, please list what are some of the requirements needed in terms of changes in weather events (e.g length of growing season; days above 35C, etc) needed by specific impact communities? Then, what evidence is there that the models can skillfully predict plausible changes in these events? Generating synthetic weather events without showing a foundation of skill is not scientifically robust.

    I propose,as a robust method, using our bottom-up approach, where we seek to assess these thresholds, and then ask the question, could real-world weather events occur to cause such a threshold to be passed? These real-world events can be constructed not only from the historical record, but, for example, from the paleo-record and by sequencing different historical time periods together (e.g, the driest 10 years in the historical record, etc). Even arbitrary scenarios could be constructed, built on reanalyses; for instance, adding 10% more water vapor on the lateral boundaries of a Type 2 downscaling run.

    We do not need to spend all of the money and time running the climate models to create synthetic weather events, which may have no basis in reality, when we can create these scenarios much more efficiently using the robust approach I outlined above.

  • Marcel Crok

    Below is a ‘guest’ comment from Gerbrand Komen, who is sitting in the advisory board of Climate Dialogue. Gerbrand is the former research director of KNMI and has a special interest in the current discussion about RCM’s. He emailed it to us but with his permission we bring it into the dialogue because we think it can help to structure the discussion. Marcel

    Guest comment by Gerbrand Komen:

    Hi Marcel,

    I am following the discussion between Bart, Jason and Roger with great interest. This is a very important topic indeed, and I hope that this discussion will not only lead to better mutual understanding but also indeed to a better insight in the way in which model results should be interpreted.

    Perhaps you could benefit from some of my observations.

    First of all, Bart, Jason and Roger seem to agree that combining observations and models is the best way forward if you try to understand the climate system.

    But there is also some real disagreement.

    Roger repeatedly makes the following points:

    · There is no difference between predictions and projections.

    · Climate models have no skill in a hindcast mode and are therefore not useful for ‘type 4’ predictions.

    · Adaptation (always important, with and without climate change) should be done on the basis of a vulnerability analysis.

    Roger also feels that the use of RCMs for the development of adaptation strategies is cost-inefficient.

    Jason and Bart both maintain that RCMs can (or may?) provide useful additional information. As an example Bart mentions the case where excessive precipitation and an extreme storm surge coincide.

    I hope the discussion will clarify a few questions that came to my mind.

    Question 1a: What exactly is a prediction?

    I tend to agree with Roger that there is no sharp distinction between predictions and projections. They are both modelled constructions of the state of the climate system at a future time. The real question is about the belief in the possibility that these predictions may verify. This is not black or white, because the prediction may be approximately correct for some variables and wrong for some other variables, and because one can distinguish different degrees of beliefs about the likelihood that a prediction will (or might) come true [very uncertain, possible, very unlikely, . . very likely . .]. Roger seems to assume that skill in hindcasts is necessary for usefulness. I can see that skill strengthens our belief in the likelihood, but in my essay [http://bit.ly/cCb1n2 or http://home.kpn.nl/g.j.komen/Uncertainties.pdf] on Rogers website I have made two points that may be relevant for the present discussion: 1. One should realize that there is ALWAYS a chance that predictions do not come true, even if the model has shown skill in hindcast studies; 2.There are a number of tests (but more than just the skill in a hindcast) that feed our belief in the usefulness of a model, indeed in a sort of Bayesian manner.

    Question 1b: What exactly is meant by the word skill?

    There are technical definitions, such as

    http://www.ecmwf.int/products/forecasts/guide/Measure_of_skill_the_anomaly_correlation_coefficient.html, but, more loosely speaking, the word skill could also be used to indicate agreement between modelled and observed quantities. It seems important that Bart, Jason and Roger use the same definition.

    Question 2: How robust is the distinction Roger makes between 4 different downscaling types?

    I have my doubts. Bart also, apparently. He somewhere called his scenarios as ‘between type 2 and 4’. To me it seems that there are different ‘dimensions’ to the problem:

    1. how are we nesting, initializing and forcing?

    2. how are we validating?

    3. what prediction horizon is considered?

    4. how are we using the results?

    Roger seems to project this on a one-dimensional space (type 1 – 4), but perhaps a more subtle division could benefit the present discussion.

    Question 3: (for Bart and Jason): Would it be possible to be more specific about the added value of RCMs by giving a list of references and quotes?

    This is exactly what Roger has been asking for several times. Roger gave specific lists of model failures, but I suspect that he has been cherry picking. It would be very useful therefore to also have a list of model successes. From my own extensive modelling experience I know that there always are aspects that are not well-described by a model. By developing one’s belief in a model one has to weigh failures and successes. Roger – and many others – seem to have the outdatedPopperian idea that you can refute a model of a complex system. This is simply not true. For example, in our ocean wave prediction model the wave height was generally better computed than the wave period. So we knew that reality was more complex than our model, but this did not make our model useless. In fact, the results have been widely and successfully used. So for the present discussion, it seems essential to have lists of both successes and failures of RCMs

    Question 4: Would it be feasible to focus on one specific vulnerability case, and to explore what would be the difference between the approaches of Roger on the one hand, and of Bart and Jason on the other hand?

    An interesting case might be the precipitation/storm surge case mentioned by Bart, but any other specific case would also be helpful.

    So, these are my questions. It would be great if Bart, Jason and Roger could come up with answers.

    To end, it may be helpful when I indicate how I try to understand the different viewpoints in a simplified, daily life example. In daily life we make scenarios all the time. For example, I can imagine getting involved in a traffic accident at some future time. So this is a realistic scenario, and a statement about what might happen. As a result I take insurance. Fortunately, I do not need to run a sophisticated agent-based model of traffic in the Netherlands to generate this scenario. The observation that traffic accidents occur and the fact that I am not very different from other people suffices. This is more or less what Roger says about RCMs. However, I can imagine that an insurance company would want to run sophisticated models to investigate whether the accident rate goes up, if, for example, the traffic density increases. And this is the position of Bart and Jason. This suggests that the value of using models not only depends on the trust one has in the model but also on the particular question that is being asked. Of course, I am not sure whether this analogy stands, but perhaps reactions to question 4 can shed light on this.

    So these are my observations. I hope they are helpful, and, above all, I really hope that the discussion between Bart, Jason and Roger on climatedialogue will bring us forward.

    All the best,
    Gerbrand

  • Roger Pielke Sr.

    Hi Gerbrand
    Thank you for the opportunity to further clarify. I will answer each of your questions.

    In the context of our discussion of multi-decadal climate, it is the forecast of the evolution of climate variables over this time period of such climate variables as temperature, precipitation, soil moisture, sea ice, vegetation etc over local, regional and global average scales. Such a prediction is unconstrained in its evolution as observed climate variables are not used except as initial conditions.

    The prediction can be expressed as an ensemble of model runs (i.e. a set of realizations with perturbed initial conditions, forcings etc) or as one realization. The climate variables can be expressed as values with different time and space averages (e.g. such as a decadal average regional maximum 2m temperature for Amsterdam).

    You write that

    “1. One should realize that there is ALWAYS a chance that predictions do not come true, even if the model has shown skill in hindcast studies; 2.There are a number of tests (but more than just the skill in a hindcast) that feed our belief in the usefulness of a model, indeed in a sort of Bayesian manner.”

    If the predictions are inaccurate in hindcast studies, they certainly should not be claimed to be able to have skill in any projections for the coming decades. Of course, a prediction, even if it shows skill in a hindcast set of runs, can still be inaccurate in the future. But a NECESSARY condition to accept a model prediction as accurate is that they show skill.

    On your second comment, what tests are you referring to? Also, the use of the word ‘belief’ is not a robust scientific demonstration of added value. Please be more specific.

    Question 1b: What exactly is meant by the word skill?

    In the context of multi-decadal climate predictions of climate change, skill is defined as an ability to produce model results for climate variables that are at least as accurate as achieved from reanalyses (averaged to correspond to the time period of the model prediction). The skill needs to be tested using hindcast runs against i) the average climate over a multi-decadal time period and ii) CHANGES in the average climate over this time period.

    Moreover, a model cannot be said to be accurate if it deviates less from reality for one climate variable (e.g. temperature) than does another variable (e.g. precipitation). They are all interrelated. If precipitation is incorrect, clouds, near surface soil moisture, vegetation condition etc will be affected. This will affect the surface sensible and latent heat fluxes, and then must alter temperature. If the temperature were closer to the observed but clouds and precipitation were not, this clearly should be a red flag that the model is not obtaining its temperature results for the right reason.

    Question 2: How robust is the distinction Roger makes between 4 different downscaling types?

    The distinction is based on the degree of constraint provided by real-world observational data. With Type 2, real world data (either that retained from a global model in terms of long wave circulation patterns, sea surface temperature, etc, or from reanalyses) provides a control on how much the model results can deviate from reality. With Type 4, except for the prescribed external forcing, terrain, deep soil type, deep ocean temperature and salinity etc which vary over much longer time periods, all other climate variables have no real world constraint.

    The four types of downscaling provide a robust delineation of downscaling in that the time period of the integration and the degree of independence from real world data are clearly distinct.

    Question 3: (for Bart and Jason): Would it be possible to be more specific about the added value of RCMs by giving a list of references and quotes?

    I am also still waiting for examples where Type 4 runs in hindcast provide skill in predicting climate change. Even predicting current climate statistics is a daunting challenge as I illustrated with the examples from the literature.

    Question 4: Would it be feasible to focus on one specific vulnerability case, and to explore what would be the difference between the approaches of Roger on the one hand, and of Bart and Jason on the other hand?

    I agree.

    Let’s select the vulnerability of Amsterdam to future (multi-decadal) environmental conditions, including climate.

    On the example with respect to traffic accidents, this is an excellent example of the bottom-up approach to vulnerability. If an insurance company runs a sophisticated model to investigate whether the accident rate goes up, if, for example, the traffic density increases, they, of course, would use a model that was validated as having skill for this type of prediction. :-)

    We can use the same bottom-up approach with climate. For example, if sea level rose by 50cm, 1m etc over the next 50 year what would be the consequences for Amsterdam?

    Sea level rise is, of course, the climate variable that the global models should be able to project on multi-decadal time scales (if they can for any climate variable) since the sea level rise is based to a significant amount on large scale average heating of the oceans. Unfortunately, however, the multi-decadal global climate model have shown no skill at predicting changes in sea level beyond what is obtained by just extrapolating the observed sea level rise into the future.

    Even with the absence of skillful projections, however, we can address the question ‘what level of sea level rise is plausible’? We can do this by

    i) assuming a continuation of the observed sea level rise for the Netherlands over the last century;

    ii) assume a release of fresh water from glaciers in Antarctica and Greenland that is double, triple etc what it was estimated to be over the last several decades,

    iii) assume part of the ice in Antarctica falls into the sea, etc.

    Each of these would increase the threat to Amsterdam but with (probably) less likelihood. We would like to have quantitative estimates of the probability of these events occurring, but, unfortunately, our level of skill is not there yet.

    The global and regional climate models, without showing skill at prediction on multi-decadal time scales, is only providing an illusion of skill to the policymakers with respect to this climate variable (i.e. sea level change) as well as changes in other climate variables. They cannot provide us with verifiably robust estimates of future sea level rise beyond what we have observed in recent decades.

    By presenting the global, regional, and local climate projections as robust (skillful) to the impacts and policy communities we are misleading them on the actual level of our scientific capability. The large amount of funds that are being used to make these multi-decadal projections could be better spent on adaptation to climate and other environmental risks, regardless of our degree of confidence that we can predict changes in risk in the coming decades. Indeed, this is a much better insurance policy than relying on global climate models for this information.

    The recent tragic tornado in Oklahoma USA is a good example of how money that is being used for multi-decadal impact studies based on global climate models could have been reallocated to provide tornado shelters for schools in Oklahoma, with a resultant much more beneficial result (i.e. it could have saved lives). In my view, we need a reassessment of funding priorities with respect to the risks that society faces.

    Gerbrand – thank you again for your excellent questions and comments! Roger

  • Bart van den Hurk

    Thanks Gerbrand for this useful attempt to stratify the discussion. Let me, like Roger, go through your questions.

    What is a prediction? What is skill?

    I agree with Gerbrand that it is not the technical definition of a prediction that is of importance, but the credibility associated to it. For predictions at the decadal time scale, as Roger identifies in his Type 4 application, assessment of skill is actually barely possible. Even a perfect model can deviate significantly from past observed trends or changes, just because the physical system allows variability at decadal time scales; the climate and its trend that we’re experiencing is just one of the many climates that we could have had. I don’t know how to assess skill of decadal trends, and so do not require models to reproduce the past trends. A measure of skill of predictions thus should be that the observed climate trends fall within the range of an ensemble of hindcast predictions. Similarly, a “prediction” of a future state can be very uncertain, and is in fact useless when interpreted as a deterministic forecast. It is a projection of a possible range of future conditions that is of interest, and also is the guidance for adaptation measures. Measures that are able to adapt to the range of projected futures are considered robust. And again: I don’t think that real-world adaptation measures are solely based on model projections of future climate, nor that the only role of climate predictions is to support adaptation design. They help to get insight in the working of a complex system, and part of this knowledge is used for adaptation related planning.

    How robust is the type typology?

    I agree with Gerbrand that Roger’s typology is too strict, and that an application dimension should at least be added. Is it designed as a deterministic forecast? Or a probabilistic one? Or one that generates understanding underlying our decisions? Those applications give very different criteria for skill and consistence.

    Examples of skill

    In terms of a contribution to the understanding of the working of the climate system I have already given a few examples of skill in my initial post. But some examples where regionalized or higher resolution models do provide more skill in generating relevant (changes in) climate statistics are for instance

    * Changes in extreme wind conditions related to small scale hurricane-type storms can not be skillfully detected in models that have a resolution to coarse to resolve this storms. Haarsma et al (http://onlinelibrary.wiley.com/doi/10.1002/grl.50360/abstract;jsessionid=885F86BAD593643C939DBD1032EC4B01.d04t02) use a high resolution model to make projections of changes in hurricanes reaching the European coast. One can still argue that this projection is highly subjective, unreliable due to limited model skill etc. But one can use the model to generate a physical picture of the way elevated SSTs lead to alterations in the statistics of this kind of events, and also the underlying mechanisms can be studied and adopted as being physically plausible. To make this into a prediction (as is suggested by the rather alarmistic title of this paper) the study is not at all complete yet: checking the results with other models, detailed analysis of the cyclogenesis etc still need to be carried out. But as a placeholder to keep in mind when thinking about coastal defense strategies this high resolution model has skill. The progress in forecast quality of the ECMWF system can clearly be partly attributed to increases in spatial resolution (Jung et al, http://journals.ametsoc.org/doi/abs/10.1175/JCLI-D-11-00265.1?journalCode=clim).
    * I was just recently pointed at a study by Cretat et al (http://link.springer.com/article/10.1007/s00382-013-1796-7) demonstrating the added skill of RCMs over GCMs to capture the characteristics of extreme precipitation events over Africa

    I will continue to look for more examples.

    A test case
    The role of models to explore implications of the combined occurrence of surges and heavy precipitation events has already been discussed before. Roger suggests to start from the vulnerability perspective and take the example of Amsterdam, where sea level rise is an issue of concern.

    I agree that the application of higher resolution model equipment does not help to reduce the uncertainty range of the sea level in the Amsterdam harbours: assumptions about changes in the heat storage, icecap melting and changes in the gravity field dominate this uncertainty range (although some regiona features related to oceanic circulation and heat redistribution may be better resolved in higher resolution models). The assistence of higher resolution projections to the vulnerability assessment of Amsterdam may lie in the analysis of consequences of an assumed sea level change on the probability that a major storm or inland precipitation event (or a combination of these) lead to water levels that are disruptive for the city. One can take an historic event, reproduce this in a state-of-the-art weather forecasting model (which is shown to have skill), but alter the boundary conditions related to the sea level. Likewise, such a scenario study can be done with different boundary conditions related to population or building density, traffic, capital, … This kind of scenario studies is being used in planning the mitigation of disasters by localizing weak spots in evacuation routes, communication nodes etc. To my opinion a useful application of high resolution models set in the context of a future condition.

  • Roger Pielke Sr.

    Hi Bart
    Thank you for your follow on comment. I have a few responses:

    I agree with you that

    “a ‘prediction’ of a future state can be very uncertain, and is in fact useless when interpreted as a deterministic forecast.”

    However, you add

    “It is a projection of a possible range of future conditions that is of interest, and also is the guidance for adaptation measures.”

    This is not consistent.

    First, “probability” predictions (projections), as applied in multi-decadal climate simulations, are deterministic. They are being based on an ensemble run of model realizations (with different models, different initial conditions etc) and their spread is then used to estimate a “probability”. This spread results because the model equations provide a deterministic set of results that each can be different since the climate is a chaotic nonlinear system both in the model, and even more so in the real world.

    Thus these probability forecasts must also be validated in hindcast runs as a necessary condition to at all claim they should be used for input to impact studies for the coming decades. Multi-decadal predictions of climate probabilities, as well as all climate statistics based on the global and regional and global climate models are deterministic model exercises.

    On the typology of downscaling, please be more specific with examples in the literature. I know of no study that cannot be assigned to one of our four types.

    In terms of the examples of “skill” that you provide, none are with respect to type 4 downscaling which is the type used by the impact communities for their planning for the coming decades. Indeed, the title of the paper you list

    “Changes in extreme wind conditions related to small scale hurricane-type storms can not be skillfully detected in models that have a resolution to[o] coarse to resolve th[ese] storms”

    appears to contradict your statement!

    I agree that the ECMWF has provided outstanding improvements in forecast skill, but this is on the one or two weekly time periods, NOT on the multi-decadal time scale. ECMWF weather forecasts involve Type 1 downscaling.

    On the paper Cretat et al (http://link.springer.com/article/10.1007/s00382-013-1796-7
    they are not looking at CHANGES in climate statistics, so their statement that

    “The RCM performs at least as well as the most accurate global climate model, demonstrating a clear added value to general circulation model simulations and the usefulness of regional modeling for investigating the physics leading to intense events and their change under global warming.”

    overstates their findings. Moreover, if the RCMs performs

    “at least as well as the most accurate global climate model”

    is hardly an endorsement for the RCM adding substantial skill.

    They also write

    “The majority of the AGCMs and AOGCMs greatly overestimate the frequency of intense events, particularly in the tropics, generally fail at simulating the observed intensity, and systematically overestimate their spatial coverage.”

    and

    “Both RCM simulations provide similar results. They accurately capture the spatial and temporal characteristics of intense events, while they tend to overestimate their number and underestimate their intensity.”

    This example that you provide actually further makes my point that the regional projections of climate on multi-decadal time periods are challenged with

    1. even reproducing current climate when run in hindcast,

    and

    2. their remains NO skill at predicting in hindcast CHANGES in climate statistics

    On the test case, I agree. High resolution models can be applied to better understand the climate processes which could amplify a rise in sea level. For example, running a regional model with lateral boundaries from a reanalysis which has the jet stream wind associated with an oncoming historically observed storm increased by 10% to see to if the resulting winter storm could produce an even higher storm surge event.

    This is, however, quite a distinct approach from running a multi-decadal global climate model with added CO2 to present to the impacts community (such as the government of the Netherlands) as a robust projection.

    I still retain my request for specific detailed documentation when you write

    “To my opinion a useful application of high resolution models set in the context of a future condition.”

    You still have not presented examples of skillful predictions (in hindcast) of changes in climate statistics. Since this has not yet been provided, the answer to the question

    Are regional models [with respect to robust multi-decadal projections of changes of climate statistics] ready for prime time?

    is clearly still NO.

  • Marcel Crok

    Hi Roger,

    In an earlier comment you wrote:

    Question 1b: What exactly is meant by the word skill?
    In the context of multi-decadal climate predictions of climate change, skill is defined as an ability to produce model results for climate variables that are at least as accurate as achieved from reanalyses (averaged to correspond to the time period of the model prediction). The skill needs to be tested using hindcast runs against i) the average climate over a multi-decadal time period and ii) CHANGES in the average climate over this time period.

    In your blog post you mentioned a lot of recently published papers that show model simulations don’t do well compared to observations even in hindcast. However did any of these studies determine the skill in the way you defined above? Are you aware of any studies that followed your definition? If not then apparently you use a definition of skill that is not being used at all. What would be the reason?

    I ask these questions because I think different definitions are used for models being skillful, so we should get clearness about this issue.

    Thanks
    Marcel

  • Marcel Crok

    Hi Bart,

    In your last comment you wrote about “skill” (bold mine):

    For predictions at the decadal time scale, as Roger identifies in his Type 4 application, assessment of skill is actually barely possible. Even a perfect model can deviate significantly from past observed trends or changes, just because the physical system allows variability at decadal time scales; the climate and its trend that we’re experiencing is just one of the many climates that we could have had. I don’t know how to assess skill of decadal trends, and so do not require models to reproduce the past trends. A measure of skill of predictions thus should be that the observed climate trends fall within the range of an ensemble of hindcast predictions.

    In an earlier comment you wrote:

    Making a prediction of the future global mean temperature is a lot easier than predicting it at the regional scale. And even there, I agree with the statement of v Oldenborgh et al (2012) that the prediction of mean temperature at the regional scale can be done fairly well given the robust temperature trend.

    Together these two paragraphs are quite confusing. And I think it is relevant for the discussion whether the models have “skill”. On the one hand you say “I don’t know how to assess skill of decadal trends” and on the other hand you also claim that the “prediction of mean temperature at the regional scale can be done fairly well given the robust temperature trend”. Now the last claim sounds like “models have skill to me”. Could you clarify this?

    Marcel

  • Roger Pielke Sr.

    Hi Marcel

    The authors’ of the papers that I listed do indeed discuss skill at predicting (in hindcast runs) the ability of multi-decadal climate model runs to simulate the real world observed climate. A few of them also look at changes in climate over the past decades. Even with respect to current climate, however, as I document in the listed papers, the models are having serious issues with faithfully reproducing the climate over this time period.

    In terms of climate change, except perhaps for the global annual average surface temperature trend over the last century, the models’ are not showing evidence of an ability to accurately predict other changes. Even the observed trends in the global annual average surface temperature during the last decade are falling below most of the model predictions.

    The burden, of course, is for authors that present hindcast multi-decadal climate projections, to provide quantitative documentation of the ability of their model to predict changes in the climate metrics that are requested by the impact and policy communities. To present results for future decades without such an examination of skill is either naïve on the part of the modelers, or disingenuous.

    My challenge to Bart (and Jason if he comes back into the discussion) is to present their precise definition of what should be the formal tests of skill for multi-decadal climate projections. These tests must be performed using the framework of Type 4 downscaling.

    Roger Sr.

  • Marcel Crok

    Hi Roger and Bart,

    Bart, in your first reaction to Roger you wrote:

    The purpose of a projection is to depict the possible (plausible) evolution of the system. To my opinion, the process of decision making is not dependent on the (quantitative) predictions provided by climate models, but by the plausibility that the future will bring situations to which the current system is not well adapted.

    How could something “plausible” not being quantitative? I think Roger’s main point is that a first condition for making plausible projections is that the models have skill in hindcast. That would increase the plausibility of the model output. Roger is just saying that as long as there is no skill in hindcast why bother to spend so much money and research time on doing projections. Better spend the money on improving the hindcast skill or – if you really want to do something for the adaptation community – follow the bottom up approach.

    Let’s make it more specific and talk about the KNMI scenario’s for The Netherlands. The four scenario’s generated so far say that The Netherlands will warm 1 or 2 degrees in 2050. Now I would say this is a quantitative prediction/projection. I suppose you would say 1 or 2 degrees of warming is more plausible than 3 degrees of warming or 0 degrees of warming. Is that right? So there is some quantification?

    Now Roger, are such regional model scenario’s for 2050 meaningless to you? Would you say we cannot say anything meaningfull about the temperature in The Netherlands in 2050? If so does that mean that for you 1 or 2 degrees of warming in 2050 is not more plausible than other amounts of warming?

    Marcel

  • Bart van den Hurk

    Hi Roger and Marcel

    I do agree that the examples I gave do not demonstrate skill of changes in climate statistics. However I still feel uncomfortable with your definition of required skill. I think we should conclude that we agree on the fact that on shorter (decadal) time scales GCM/RCM have shown little regional skill to predict/hindcast observed changes. But that does not necessarily imply that they are useless or have no skill on longer time scales.

    Let’s try to define the information needs required to assess changes in risks that adverse events strike society and lead to the need to adapt to these. A high-impact event can be considered as an expression of natural variability, that acts randomly, on short time scales, and thus gives quite different criteria for defining skill in their prediction than the gradual change in the background climate from which these extreme events emerge. Even while one cannot exactly assess the risk for such events in present day conditions, one can assess that the risk for being exposed to particular events changes when background conditions change. Just like the fact that the chance for a temperature higher than 25 degrees is higher in summer than in winter, one can deduce that flood risk increases with sea level rise, or with enhanced precipitation in the upstream headwaters. Projections of these changes of risk using models in which changes in the background climate are incorporated, and applied using models that do a fair job at the short time scale (like high resolution weather prediction, or hydrological discharge, or…) is thus a viable procedure, and does yield added value.

    Another confusing component in the discussion is the time scale issue. Any extreme, any risk, any trend comes with the definition of a time scale. And natural variability plays a role at nearly any time scale as well. As I indicated in an earlier blog, the natural variability at decadal time scales hinders the validation of any projection against observations, as these observations reflect just one possible trend. At relatively short time scales (say, a couple of decades), natural variability will dominate over systematic climate change, and there is more added value to expect from ensuring that the projections reflect natural variability adequately than to assess the degree to which the background climate changes. This is what you would demand if you say the models in hindcast simulations should show skill; you primarily demand that we can predict the phase of natural variations. We agree that at the moment models have little skill to do so (although for some areas initialized GCM runs show skill in some areas).

    But at longer time scales the change of the probability of a very rare extreme event is dominated by the change in the background climate, where small changes in the mean can lead to large changes in the probability of extremes (see e.g. the scaling between surface temperature and 99 percentile precipitation by Lenderink et al (2011), doi:10.5194/hessd-8-4701-2011, or the non-linear relation between trends and extremes by Rahmstorf and Coumou (2011), http://www.pnas.org/content/early/2011/10/18/1101766108.abstract). Again, superposition of this changing background climate on projections with high resolution models resolving the relevant process does give valuable insights of the possible changes in risks and vulnerability.

    The assumed change of the background climate, as is embedded in for instance the Dutch climate change scenarios (+1 or +2 K increase by 2050), can be interpreted as a quantitative assessment of how much the climate system will warm. We derive these values from the spread of IPCC model responses, realizing that these models do have systematic errors, but reproduce historic variations well enough to have confidence about the approximate range of warming in 2050. We don’t make scenario’s for a 0 degree warming, both because these models (and the theory that is embedded in them) show that this is quite unlikely, and because making a scenario for a situation that is similar to the present condition does not pose an unforeseen challenge relative to present conditions. But we also do not give scenarios for a 10, or even 5 degrees warming in 2050, because also this is not supported by our understanding of the climate system response, and thus also not by the climate model integrations. Although they are quantitative numbers, these scenarios are not to be interpreted as (type 4) forecasts, but as a set of conditions that is relevant to consider when making future assessments.

  • Roger Pielke Sr.

    Hi Bart

    Thank you for your further comments. It appears we agree on the inability of the global models to simulate the natural variations on decadal and, presumably longer time scales, of large scale circulation patterns such as ENSO, the PDO, the NAO etc. It is these features, of course, which have so much influence on the weather in western Europe and elsewhere.

    Where we still disagree, however, is the claim that a long term background climate can be skillfully predicted. What is your basis for this claim? Over the last decade or so, the models have not shown an ability to predict the lack (or very muted) change in the annual average global surface temperature trend. The lower troposphere has similarly not warmed since ~2000 [e.g. see http://www.ssmi.com/msu/msu_time_series.html When do you assume the warming would restart?

    Moreover, if we agree the climate models cannot skillfully predict natural variations, why would you conclude that they can skillfully predict changes in climate statistics needed by the impacts and policy communities? The assumption of a global annual average increase in the coming decades +1C and +2C, is of little use in defining changes in climate impacts at the regional and local scale, which are so dependent in how large scale circulation features would change in the coming decades. The impacts communities care about such things as the growing season, warm season precipitation, snow coverage, soil moisture, etc, for their locations, not an annual average change in the global surface temperature. These climate metrics are much more dependent on the large scale circulation features than a global annual average surface temperature trend.

    If one wants to assess the risk due to +1C and +2C, the impact models themselves can be run with this increase prescribed directly into their models. There is no need to run a computationally expensive multi-decadal global climate model downscaled to a region to make such a prescription to the impact models. Indeed, any results that are provided of changes in regional and local climate statistics have no skill as we have been discussing in our dialogue, and we both seem to agree on.

    My bottom line is that while the global climate models, when run with added CO2 and other greenhouse gases, show that this is a warming effect, they are inadequate tools to assess the consequences of these human climate forcings on the regional and local scale. Even worse, the models inadequately include the diverse myraid effects of aerosols and land use/land cover change on the climate system, so they are already hindered in their ability to accurately represent the real world spectrum of human climate forcings.

    You wrote with respect to the results from the multi-decadal global climate model projections that

    “Although they are quantitative numbers, these scenarios are not to be interpreted as (type 4) forecasts, but as a set of conditions that is relevant to consider when making future assessments.”

    and

    “The assumed change of the background climate, as is embedded in for instance the Dutch climate change scenarios (+1 or +2 K increase by 2050), can be interpreted as a quantitative assessment of how much the climate system will warm.”

    However, you did not present evidence as to why these results (this “set of conditions”) should be considered as the primary framework to communicate regional and local climate conditions expected in the coming decades to the impacts and policy communities. At most, they are just sensitivity studies that provide documentation that the climate system is affected by adding CO2 and other greenhouse gases to the atmosphere. Beyond this, they have shown no skill at predicting changes in climate statistics, as we now both appear to agree on.

    Thus, to write that we know

    “how much the climate system will warm”

    overstates where we actually are in terms of climate science. This sentence should be written, in my view, as

    “The assumed change of the background climate, as is embedded in for instance the Dutch climate change scenarios (+1 or +2 K increase by 2050), can be interpreted as ONE assessment of how much the climate system COULD warm.”

    And, of course, we do not need to global climate models to run impact models with an annual average increase in the mean surface air temperature of +1C and +2C prescribed for the Netherlands. We can prescribe these directly into the impacts models, as I wrote above.

    Thank you again for the opportunity to discuss. We are making excellent progress in showing where we agree, and where we still disagree.

  • Bart van den Hurk

    Dear Roger

    Indeed, there is a number of issues that we can agree on, and a few where we remain to have a different point of view.

    You do conclude that we tend to agree that current models are unable to simulate the natural variations on decadal (and longer) time scales. However, I think there remains a disagreement here, and that refers to the difference between simulation and prediction. Models are increasingly good in simulating ENSO, NAO, but the predictability of these phenomena remains to be limited owing to unpredictable components in the forcings and dynamics.

    Indeed, we also disagree in the ability to predict regional patterns of changing background climate. Van Oldenborgh et al (2012), to which I referred before, clearly demonstrate that most models do a fair job in simulating the observed spatial patters of warming; what is lacking is a good predictability of the variability around these trends.

    Even with imperfect predictions of regional trends, there is credibility in assuming global warming to be somewhere between 1 and 2 K in 2050. Indeed, there will always be the possibility that – owing to known and unknown model deficiencies and to natural variability – the true warming will be well outside this range, similar to smaller or stronger deviations at the regional scale. But the confidence that this is the right order of magnitude comes from that combination of models, basic undestanding of the greenhouse mechanism and involved feedback, and observed trends. Again, we agree that the probability that the true trends will be outside this range will be larger than zero, but I do think that the phrase that these climate projections are not credible is not justified. With our scenarios we do not attach probabilities to any of these scenario values, but we do consider these values to be valuable benchmarks for assessing the impact at regional scale. For that reason, I fully embrace your proposed rephrasing, that the numbers embedded in the Dutch Climate Scenarios can be interpreted as ONE assessment of how the climate COULD warm. But an assessment that, given the evidence we see, does make sense.

    Your statement that complex climate models are not needed for assessing local impacts of changing climate statistics and can be replaced by simple permutation of the forcing conditions is sometimes true and sometimes it clearly isn’t. It really depends on the application at hand. We do have a wide experience with climate impact assessment people (see for instance my overview in the volume on Society Vulnerability you recently edited; Van den Hurk et al, 2013: Vulnerability Assessments in the Netherlands Using Climate Scenarios. Climate Vulnerability: Understanding and Addressing Threats to Essential Resources. Elsevier Inc., Academic Press, 257–266 pp.). And for some applications a simple scaling may be sufficient, as long as all relevant variables can be well scaled with a single temperature measure. However, precipitation intensity changes nonlinearly with temperature (Lenderink, and van Meijgaard, 2008; doi:10.1038/ngeo262), circulations and the frequency of dry/wet spells may change (Haarsma et al, 2009; doi:10.1029/2008GL036617), simultaneous occurrence of floods and storm surges changes nonlinearly (Kew et al, in review), and this is just the “meteorological” part of the spectrum. Even seemingly “straightforward” applications like the assessment of the impact of changes in the mean annual cycle of temperature on shifting butterfly populations did find more climate variables than just temperature to be of importance (see e.g. WallisDeVries et al, 2011; http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3172409/), and this generally applies to many sectors and applications. So, a simple scaling is ok, as long as the relevant covarying variables can be changed adequately with this scaling.

    We get repeated requests from users not to provide as realistic as possible predictions of the future, but to provide physically realistic pictures that can be understood and interpreted as a possible consequence of altering climate statistics. And to make this realistic pictures, a simple scaling is often not adequate to depict all consequences of an altered background climate, and models are useful tools for this.

    To my feeling this discussion is much more than a repeteated rephrasing of statements and opinions. It helped me to realize even stronger that it is the society stakeholder that should be put in the middle of our work, and with respect to this you and your colleagues have echoed a very convincing and true sound in the debate around climate science and its applications. Also we invest most of our research resources not in climate model development per se, but in assessing the information that can be extracted from the existing model archive, in novel applications that are at the heart of interaction with stakeholders, in producing pictures of how our future climate COULD look like, which help feeding the imagination of those that have to take the real decisions. And we fully understand and acknowledge that for most decisions climate is just one of the many entries that is affecting them. By this attitude we surely hope and expect to serve society optimally with our expertise and dedication.

  • Dear participants, thanks for the interesting discussion. I’d like to make some observations and pose some questions.

    Roger and BartH apparently use very different definitions of “model skill”

    BartH:

    “A measure of skill of predictions thus should be that the observed climate trends fall within the range of an ensemble of hindcast predictions.”

    Roger:

    “In the context of multi-decadal climate predictions of climate change, skill is defined as an ability to produce model results for climate variables that are at least as accurate as achieved from reanalyses (averaged to correspond to the time period of the model prediction).”

    The definition most common in meteorology (correct me if I’m wrong) is that skill is the ability of a model to simulate the observations better than a naïve forecast (e.g. of “no change”).

    BartH notes that
    “At relatively short time scales (say, a couple of decades), natural variability will dominate over systematic climate change, and there is more added value to expect from ensuring that the projections reflect natural variability adequately than to assess the degree to which the background climate changes.”

    Presumably this means that for timescales shorter than, say, two decades it is extremely difficult for a model to show skill in the ‘traditional’ meaning as I gave just above. However,
    BartH gave some examples in this discussion however that these sorts of projections can still be useful and also examples where higher resolution and/or RCM’s in comparison with GCM’s resulted in increased model skill.

    Roger’s definition is very strict: The model simulations should be better than the (model interpolated) observations themselves (which is what reanalyses could be described as). Would any model simulation (e.g. also a weather forecast) have skill according to this definition? Isn’t this an impossible expectation to have of model simulations?

  • Roger Pielke Sr.

    Hi Bart
    Thank you for your comments. My response is as follows.

    You wrote

    “The definition most common in meteorology (correct me if I’m wrong) is that skill is the ability of a model to simulate the observations better than a naïve forecast (e.g. of ‘no change’).”

    This definition of skill is not what is generally applied even in weather forecasting. While “no change” is certainly an option, more generally persistence or climatology are used. As one example of how skill has been defined, see

    Landsea,C.W., Knaff,J.A., 2000: “How much skill was there in forecasting the very strong 1997-98 El Niño?” Bulletin of the American Meteorological Society, 81. http://www.aoml.noaa.gov/hrd/Landsea/LandseaKnaff_bulletinAMSSept2000.pdf

    In that study, they used

    “A previously developed simple statistical tool—the El Niño–Southern Oscillation Climatology and Persistence (ENSO–CLIPER) model—is utilized as a baseline for determination of skill in forecasting this event.”

    For multi-decadal climate forecasts, we need to specify what is the measure of skill.

    I have proposed that, in order to provide value to the impacts communities, the projections must have skill in predicting changes in regional climate statistics in hindcast runs. Otherwise, we should use the reanalyeses for these impact studies, with, as one set of applications, perturbations prescribed in the impact models to assess at what level of change in climate statistics would result in a negative threshold being crossed in one or more key societal and/or environmental resources.

    If the projections have no skill at predicting changes in regional climate statstics, but we give these changed forecasts to the impact and policy communities implying they are robust, we are not being honest with them.

    My question to you is

    “What is your quantitative measure of skill (value) to justify the cost and expense of running multi-decadal climate projections and providing these results to the impact and policy communities with the claim they are robust?

  • The following questions may be central to the current discussion (as suggested by Gerbrand Komen). Hopefully the discussants could give their opinion on each:

    1. How do we develop belief in scientific results, in particular in the ability of models to generate information about relevant possible futures?

    2. Which global and regional climate model properties support our belief in the usefulness of models as a tool for the exploration of possible futures?

    3. What are the relative advantages of the methods proposed by Bart vdH and Roger PSr for advising policymakers about climate (and other) vulnerability risks?

  • Gerbrand Komen asked me to submit the following:

    Hi Roger,

    In my previous contribution to this dialogue, I wrote:

    “There are a number of tests (but more than just the skill in a hindcast) that feed our belief in the usefulness of a model, indeed in a sort of Bayesian manner.”

    This prompted you to formulate the following questions:

    A. “On your second comment, what tests are you referring to?”

    B. “Also, the use of the word ‘belief’ is not a robust scientific demonstration of added value. Please be more specific.”

    Re question A. An example of ‘other tests’ , would be to test to what extent a model is able to represent ENSOs. Even if it is not able to skillfully predict the exact timing of ENSOs it may or may not be able to simulate the (random) occurrence of ENSOs. Many other examples of this type of test can be found in chapter 8 (Climate Models and Their Evaluation) of IPCC/AR4, which assesses both model successes AND model failures.

    Re question B. The word ‘belief’ as used here should not be seen as ‘religious belief’, but rather as a technical term used by philosophers, psychologists, and social scientists. There is an extensive body of literature from such fields as the philosophy of science and a field called social epistemology, that study the role of belief in science. It is my guess that this is also relevant for the present discussion.

    I hope this clarifies my earlier statement, and I very much look forward to the continuation of your discussion with Bart van der Hurk.

    Gerbrand

  • Roger Pielke Sr.

    Hi Bart

    You have asked several questions -

    “1. How do we develop belief in scientific results, in particular in the ability of models to generate information about relevant possible futures?”

    The only robust approach, in my view, is to show that the models have skill at predicting CHANGES in climate statistics on multi-decadal time scales. This must be done, of course, using hindcast runs. Otherwise, the appropriate approach is to use the models within the framework of reanalyses which permits the incorporation of real world observed data.

    “2. Which global and regional climate model properties support our belief in the usefulness of models as a tool for the exploration of possible futures?”

    I have reported on this in my earlier posts. If the models show a lack of skill and need tuning with respect to predicting (in hindcast)even the current climate statistics on multi-decadal time scales (much less than CHANGES in climate statistics), they are not ready to be used as robust projection tools for the coming decades.

    “3. What are the relative advantages of the methods proposed by Bart vdH and Roger PSr for advising policymakers about climate (and other) vulnerability risks?”

    The bottom-up, resource-based (contextual) approach to assess vulnerability is an inclusive approach which can accommodate the top-down projections as one plausible set of future environmental conditions. However, it permits the broad assessment of the entire spectrum of risks to key resources, and does not presuppose that the climate part of the risk can be skillfully predicted by the climate models.

    Best Regards

    Roger Sr.

  • Roger Pielke Sr.

    Hi Gerbrand

    You wrote

    “There are a number of tests (but more than just the skill in a hindcast) that feed our belief in the usefulness of a model, indeed in a sort of Bayesian manner.”

    and, as you noted, I asked

    A. “On your second comment, what tests are you referring to?”

    B. “Also, the use of the word ‘belief’ is not a robust scientific demonstration of added value. Please be more specific.”

    Your first response is

    “Re question A. An example of ‘other tests’ , would be to test to what extent a model is able to represent ENSOs. Even if it is not able to skillfully predict the exact timing of ENSOs it may or may not be able to simulate the (random) occurrence of ENSOs. Many other examples of this type of test can be found in chapter 8 (Climate Models and Their Evaluation) of IPCC/AR4, which assesses both model successes AND model failures.”

    My Reply- Even IF the climate model could predict the current statistics of the occurrence of ENSO, the NAO, the PDO, etc [which is proving quite a daunting challenge], this, by itself is not sufficient to justify their use to provide projection of CHANGES in the statistics of ENSO, the NAO, the PDO, etc events in the coming decades. The IPCC/AR4 failed to discuss this requirement, and thus, misled the impacts and policy communities on what is the current state of multi-decadal climate prediction modeling.

    Your second response is

    “Re question B. The word ‘belief’ as used here should not be seen as ‘religious belief’, but rather as a technical term used by philosophers, psychologists, and social scientists. There is an extensive body of literature from such fields as the philosophy of science and a field called social epistemology, that study the role of belief in science. It is my guess that this is also relevant for the present discussion.”

    My Reply – I suggest we defer using the word “believe”. The model results should be accepted as robust ONLY when model predictions (i.e. the hypotheses) cannot be shown to be falsified with real world data. This is the scientific method. I have presented peer reviewed papers that do, in fact, falsify the models even with respect to their ability to predict (in hindcast) the current climate.

    Thank you for continuing to discuss!

    Best Regards

    Roger Sr.

  • Roger Pielke Sr.

    Arthur Smith – You wrote

    “Roger Pielke Sr. in his latest comment seems to be what I can only describe as trolling. Does he deny the standard WMO definition of climate?

    Climate in a narrow sense is usually defined as the “average weather,” or more rigorously, as the statistical description in terms of the mean and variability of relevant quantities over a period of time ranging from months to thousands or millions of years. The classical period is 30 years, as defined by the World Meteorological Organization (WMO). These quantities are most often surface variables such as temperature, precipitation, and wind. Climate in a wider sense is the state, including a statistical description, of the climate system.”

    My Comment:

    As you noted, this is a “narrow” definition. In the AMS Glossary, climate is defined as

    The system, consisting of the atmosphere, hydrosphere, lithosphere, and biosphere, determining the earth’s climate as the result of mutual interactions and responses to external influences (forcing). Physical, chemical, and biological processes are involved in the interactions among the components of the climate system.”

    The Glossary defines “climate change” as

    “Any systematic change in the long-term statistics of climate elements (such as temperature, pressure, or winds) sustained over several decades or longer. Climate change may be due to natural external forcings, such as changes in solar emission or slow changes in the earth’s orbital elements; natural internal processes of the climate system; or anthropogenic forcing.”

    I am unclear why you feel compelled to use the term “trolling” but hope you are now clearer on what is meant by the term “climate”. This broader view was what was adopted in the NRC report

    National Research Council, 2005: Radiative forcing of climate change: Expanding the concept and addressing uncertainties. Committee on Radiative Forcing Effects on Climate Change, Climate Research Committee, Board on Atmospheric Sciences and Climate, Division on Earth and Life Studies, The National Academies Press, Washington, D.C., 208 pp.http://www.nap.edu/openbook/0309095069/html/

    which I recommend you read.

  • Hi Roger,

    Thanks for your elaborate answers.

    In your answer to 1 (24-06 12:32) you state that unless models have proven skill in predicting changes in climate statistics on multi-decadal timescales, they should not be used for projections or predictions. Given your very strict definition of skill and your statements that models, in your view, do not possess given skill, the question arises what alternative sources of information there might be to inform society and stakeholders about the potential or likely (given a certain scenario) trajectories of future climate? E.g. I would argue that even amidst the very large uncertainty, some trajectories are more likely than others (e.g. given a continued positive radiative forcing, warming is more likely than cooling).

    Regarding tests of a model’s usefulness (24-06 12:50) you state that models are unable to project changes in e.g. ENSO. You seem focused on what models do poorly, while Gerbrand gave an example here of what models do half-decent: Models do produce the characteristic ENSO behavior at irregular time intervals, even though this is not entered into the model as an input. It arises as an emergent property of the model. Does that not count as a test of usefulness to simulate some of the system’s physical processes, which aids our physical understanding?

    Finally, in your reply to Arthur Smith you mentioned that climate change is defined over timescales “over several decades or longer”. These timescales, I believe, is the issue that Arthur wanted to draw attention to: 10-15 years is relatively short in climate terms, so the apparent hiatus in surface warming (while the oceans keep accumulating heat at a fast rate) is not necessarily proof of model failure. Moreover, as you rightly pointed out, the timing of ENSO can not be predicted, so the real-world predominance of La Nina conditions over the past 10 to 15 years is not reflected by the model simulations.

  • Bart van den Hurk

    Dear Roger and Andries

    The use of the word “belief” in the discussion tends to give it another twist: to what sense can we “believe” that altering climate statistics might give rise to reconsider some of our society decisions when we have nothing more than (imperfect) modls to feed this “belief”? Andries Rosema even finds it “frightening” that generating pictures of how climate statistics could change is feeding these considerations, as they introduce a large risk of alarmistic messages and overinvestments in mitigation or adaptation measures. I tend to turn the argument around: ignoring the possibility that the climate could look different in the future while making these society decisions, is given the evidence we get from a large source of scientific studies (including, indeed, imperfect models) a bit naive.

    Let’s try to bring some focus in the discussion around the extend to which “believe” supports decision making, and how the arguments for these decisions are justified. I participated in a writing team lead by Prof Frans Berkhout that analyses the extent to which one can use plausible (model-derived) storylines for decision making processes in society. After appreciating that there are many types of decisions that do have very different dependence on the notion of changing climate statistics, we formally discerned a number of mind frames that form the starting point for this decision making process. From the (draft) manuscript (in review by Regional Environmental Change) I cite

    “In their role as decision support tools, scenarios have to compete with other ways in which decisionmakers may reason about the future. According to Rumelhart (1989), there are three common processes for reasoning about novel situations. They are:
    * Reasoning by similarity: a problem is solved by seeing the current situation as similar to a previous one in which the solution is known. In this category fall intuition, reasoning by example or experience, generalization and analogical reasoning.
    * Reasoning by simulation: a problem is solved by imagining the consequence of an action and making explicit the knowledge that is implicit in our ability to imagine an event. This category includes story-building to mentally simulate the events leading up to a certain ending.
    * Formal reasoning: a formal symbolic system, such as logic or mathematics, is employed in the solution of a problem. Examples are formal mathematical models of biophysical or social systems, including climate scenarios.

    Rumelhart DE (1989) Toward a microstructural account of human reasoning. In Vosniadou S, Ortony A (Eds.) Similarity and analogical reasoning. New York, NY, Cambridge University Press, pp. 298-312.”

    The discussion about the implication of using imperfect models in this discussion making comes down to comparing the second and third ways of reasoning. Roger (and Andries) tend to stress the need for formal reasoning, while I (and Gerbrand) make a plea for the second way of reasoning. Roger could reply again by stating that models that don’t show skill in projecting changing statistics cannot be used for this reasoning by simulation, but I remain to disgree with him: the skill of climate models to project changing climate statistics at decadal time scales can formally not be established due to large role of natural variability, but is also not always required for generating useful information that enters the imagination process.

    My main point is: we should ask ourselves what kind of arguments decision makers use when deciding on large investments in climate change mitigation or adaptation. To my opinion it is not true that the only way of reasoning for these decisions is the formal way of reasoning: like any decision an individual or society takes, it is based on a blend of evidence, gut feeling, windows of opportunity, compounding consequences etc. The formal arguments that are used for justification of decisions come from more sources than imperfect models. Arguments coming from the simulations can help to decide on no-regret measures, compounding consequences, and focus at vulnerable sectors or areas, and – I repeat myself – are thus a useful source of information.

  • Roger Pielke Sr.

    Bart Verheggen – Thank you for your response. I have several comments.

    With respect to your statement that

    “Models do produce the characteristic ENSO behavior at irregular time intervals.”

    Please provide the literature citation for this finding, along with a summary of its observational validation.

    I do agree with you that running the climate models “aids our physical understanding”.

    Indeed, I have published quite a few of these process studies; e.g. see our papers cited in

    Pielke Sr., R.A., A. Pitman, D. Niyogi, R. Mahmood, C. McAlpine, F. Hossain, K. Goldewijk, U. Nair, R. Betts, S. Fall, M. Reichstein, P. Kabat, and N. de Noblet-Ducoudré, 2011: Land use/land cover changes and climate: Modeling analysis and observational evidence. WIREs Clim Change 2011, 2:828–850. doi: 10.1002/wcc.144. http://pielkeclimatesci.files.wordpress.com/2012/01/r-369.pdf

    Mahmood, R., R.A. Pielke Sr., K. Hubbard, D. Niyogi, P. Dirmeyer, C. McAlpine, A. Carleton, R. Hale, S. Gameda, A. Beltrán-Przekurat, B. Baker, R. McNider, D. Legates, J. Shepherd, J. Du, P. Blanken, O. Frauenfeld, U. Nair, S. Fall, 2013: Land cover changes and their biogeophysical effects on climate. Int. J. Climatol., http://pielkeclimatesci.files.wordpress.com/2013/02/r-374.pdf

    However, such applications of the models are distinct from using them to make predictions.

    You also write

    “….the apparent hiatus in surface warming (while the oceans keep accumulating heat at a fast rate) is not necessarily proof of model failure.”

    Actually, it is evidence of a failure of the models to skillfully predict the evolution of the climate system. This transfer of heat to deeper levels (if it in fact is really occurring) was not predicted by the models beforehand. Indeed, the upper ~750m of the oceans has not been warming in recent years as clearly shown in the figure at this url [http://oceans.pmel.noaa.gov/].

    To document the failure of the models with respect to this climate metric, see this response in 2005 from Jim Hansen with respect to the GISS climate model –
    http://pielkeclimatesci.files.wordpress.com/2009/09/1116592hansen.pdf

    “Our simulated 1993-2003 heat storage rate was 0.6 W/m2 in the upper 750 m of the ocean. The decadal mean planetary energy imbalance, 0.75 W/m2, includes heat storage in the deeper ocean and energy used to melt ice and warm the air and land. 0.85 W/m2 is the imbalance at the end of the decade.”

    The real-world heating in this layer is not now in agreement with the GISS model projections, even if you claim the heat is now at deeper levels. This discrepancy is only a little over a decade but the obvious question is how much more time would have to pass with such a disagreement between the models and the real world before you would be prepared to reject them as credible tools to create regional climate scenarios?

    With respect to other multi-decadal climate model failings, see also the informative analyses by Bob Tisdale in his posts

    http://bobtisdale.wordpress.com/2013/06/20/model-data-comparison-daily-maximum-and-minimum-temperatures-and-diurnal-temperature-range-dtr/

    http://bobtisdale.wordpress.com/2013/06/15/model-data-comparison-hemispheric-sea-ice-area-2/

    http://bobtisdale.wordpress.com/2012/12/27/model-data-precipitation-comparison-cmip5-ipcc-ar5-model-simulations-versus-satellite-era-observations/

    http://bobtisdale.wordpress.com/2013/02/28/cmip5-model-data-comparison-satellite-era-sea-surface-temperature-anomalies/

    http://bobtisdale.wordpress.com/2013/04/20/model-data-comparison-with-trend-maps-cmip5-ipcc-ar5-models-vs-new-giss-land-ocean-temperature-index/

    http://bobtisdale.wordpress.com/2013/06/27/model-data-comparison-alaska-land-surface-air-temperatures/

    Bart – Thank you for engaging in this constructive discussion.

  • Roger Pielke Sr.

    Bart van den Hurk – Thank you for your responses.
    You write

    “In their role as decision support tools, scenarios have to compete with other ways in which decision makers may reason about the future. According to Rumelhart (1989), there are three common processes for reasoning about novel situations. They are:

    * Reasoning by similarity: a problem is solved by seeing the current situation as similar to a previous one in which the solution is known. In this category fall intuition, reasoning by example or experience, generalization and analogical reasoning.

    * Reasoning by simulation: a problem is solved by imagining the consequence of an action and making explicit the knowledge that is implicit in our ability to imagine an event. This category includes story-building to mentally simulate the events leading up to a certain ending.

    * Formal reasoning: a formal symbolic system, such as logic or mathematics, is employed in the solution of a problem. Examples are formal mathematical models of biophysical or social systems, including climate scenarios.”

    However, you did not define what is to be the output of such scenarios. I suggest that what the impacts and policymakers need to start with is an identification of what are the consequences of specified changes in climate statistics.

    As just one specific example, if the average July maximum temperature in Amsterdam were to rise by 1C, 2C etc in the coming decades what would be the effect in terms of electric energy demand.
    Once the risk to demand from such a temperature increase is identified, can we assess the probability, or less stringently, the plausibility, of such an increase occurring over the next several decades?

    Then, what is the cost, for each scenario, to make the electric system resilient to such increases in temperature. Policymakers need to determine what they want to pay for each scenario methodology given the uncertainty associated with them. The multi-decadal global climate model predictions can certainly be used as one option for scenario generation.

    However, the question is whether the large time and human resources that are used to create these climate model runs could be more effectively used for other scenario methodologies [which would be much less costly] and, even more importantly, in developing responses to climate (and other environmental threats) so as to reduce the risks we face.

    The climate models lack of skill when run in multi-decadal hindcast runs makes them a very poor choice as the primary tool to assess how the climate will behave in the coming decades. Not only are they expensive but their skill is lacking what is required.

    Thank you for continuing our informative discussion!

  • Roger Pielke Sr.

    On the website http://klimazwiebel.blogspot.com/2013/07/prediction-or-projection-nomenclature.html, I have also been discussing the issue of what is meant by “prediction” and “projection” with respect to multi-decadal climate runs. I am presenting my latest comment below, in response to Hans von Storch’s comment as the size of my reply seems too large for their website.

    My comment follow –

    Hi Hans –

    The attempt to distinguish between the terms “projection” and “prediction”, whether by the IPCC or others, has introduced an unnecessary confusion to the impacts and policy communities regarding the skill of regional and local multi-decadal climate model runs.

    The 2007 IPCC defines “projection”, “climate projection” and “climate prediction” as [http://www.ipcc.ch/publications_and_data/ar4/wg1/en/annexes.html]

    “Projection – A projection is a potential future evolution of a quantity or set of quantities, often computed with the aid of a model. Projections are distinguished from predictions in order to emphasize that projections involve assumptions concerning, for example, future socioeconomic and technological developments that may or may not be realised, and are therefore subject to substantial uncertainty.

    Climate prediction – A climate prediction or climate forecast is the result of an attempt to produce an estimate of the actual evolution of the climate in the future, for example, at seasonal, interannual or long-term time scales. Since the future evolution of the climate system may be highly sensitive to initial conditions, such predictions are usually probabilistic in nature.

    Climate projection – A projection of the response of the climate system to emission or concentration scenarios of greenhouse gases and aerosols, or radiative forcing scenarios, often based upon simulations by climate models. Climate projections are distinguished from climate predictions in order to emphasize that climate projections depend upon the emission/concentration/radiative forcing scenario used, which are based on assumptions concerning, for example, future socioeconomic and technological developments that may or may not be realised and are therefore subject to substantial uncertainty”

    Clearly a “climate projection” IS a “climate prediction, using the IPCC definitions, when the emission/concentration/radiative forcing scenario actually has occurred. This can be done using hindcast model runs, and these model runs can be quantitatively tested.

    The source of much of this confusion appears to be in the assumption by the IPCC and others, that while weather prediction is an initial value problem, climate prediction (on multi-decadal time periods) is a boundary value problem; the term “projection” then being reserved for the later.

    However, as succinctly stated by F. Giorigi

    F. Giorgi, 2005 : Climate Change Prediction: Climatic Change (2005) 73: 239. DOI: 10.1007/s10584-005-6857-4

    “….because of the long time scales involved in ocean, cryosphere and biosphere processes a first kind predictability component also arises. The slower components of the climate system (e.g. the ocean and biosphere) affect the statistics of climate variables (e.g. precipitation) and since they may feel the influence of their initial state at multi decadal time scales, it is possible that climate changes also depend on the initial state of the climate system (e.g. Collins, 2002; Pielke, 1998). For example, the evolution of the THC in response to GHG forcing can depend on the THC initial state, and this evolution will in general affect the full climate system. As a result, the climate change prediction problem has components of both first and second kind which are deeply intertwined.”

    You clearly adopt the assumption that the multi-decadal climate runs represent a boundary value problem when you use the term “simulation” and write

    “Technically, most if not all projections are conditional/what-if forward simulations, as we run a GCM or RCM with assumed forcing and a random initial state consistent with climatology.”

    The 2007 IPCC Glossary has no definition for “simulation”.

    The AMS Glossary defines a “numerical simulation” as [http://glossary.ametsoc.org/wiki/Numerical_simulation]

    “A numerical integration in which the goal is typically to study the behavior of the solution without regard to the initial conditions (to distinguish it from a numerical forecast).

    Thus the integrations are usually for extended periods to allow the solution to become effectively independent of the initial conditions.”

    This is a view of this definition “simulation” that is being shown to be flawed with respect to the real world climate; e.g. see

    S. Lovejoy, 2013: What Is Climate? EOS 2 JAN 2013 DOI: 10.1002/2013EO010001. http://onlinelibrary.wiley.com/doi/10.1002/2013EO010001/pdf

    and

    Rial, J., R.A. Pielke Sr., M. Beniston, M. Claussen, J. Canadell, P. Cox, H. Held, N. de Noblet-Ducoudre, R. Prinn, J. Reynolds, and J.D. Salas, 2004: Nonlinearities, feedbacks and critical thresholds within the Earth’s climate system. Climatic Change, 65, 11-38. http://pielkeclimatesci.wordpress.com/files/2009/10/r-260.pdf

    Much more important, however, than the name we use is how do you test the quantitative accuracy of the multi-decadal climate model results, regardless of whether they are called “projections”, “predictions”, “simulations” or “forecasts”?

    I have presented evidence from the peer reviewed literature; see my post at

    http://www.climatedialogue.org/are-regional-models-ready-for-prime-time/

    that shows they are not yet robust tools to be used to make multi-decadal climate (predictions; projections) of changes in regional and local climate statistics.

    Do you have peer reviewed results that counter this conclusion? I also would value your response to summarize how you conclude we should assess the skill of multi-decadal climate predictions of changes in climate statistics.

    Best Regards

    Roger Sr.

  • Roger Pielke Sr.

    I posted this comment at http://klimazwiebel.blogspot.com/2013/07/prediction-or-projection-nomenclature.html

    Hi Hans

    I have another query for you to address.
    You write

    “But the definition [of a projection] is more of the sort “description of a possible outcome, independent of the method”. A prediction, also independent of the methodology, is an ‘effort for describing a probable outcome’.

    This distinction, possible vs. probable is an important difference for dealing with the objects, which may look rather similar, and the IPCC is to be applauded for working the difference out. The advantage is that we can use the same terminology; if you do not want to follow this terminology, this is fine, but if others do, they are not ‘false’.”

    What you are telling us, in my view, are that the impact and policy communities are basing their studies and decisions on model results that are “possible” based on the multi-decadal climate model runs. This means, of course, that any of the results I present in my papers on land use change, as just one example, are “possible” and should be used as part of the assessment of risk. I have stated, however, that such model runs are “sensitivity studies” only.

    This framework that the results are “possible” and thus are a subset, perhaps a small subset of what will occur in the coming decades, is not a view that is communicated to the impacts and policy communities. Indeed, a review of multi-decadal climate impact papers makes frequent use of verbs such as “will” (rather than “could”), etc when presenting the model results for the coming decades. Figures are usually presented as, for example, the decade 2050-2059, etc.

    If you agree that a “projection” is a “sensitivity study” than we are actually in closer agreement than I thought. :-)

    However, you also write that, with respect to the distinction between a “projection” and a “prediction”,

    “the IPCC is to be applauded for working the difference out. The advantage is that we can use the same terminology; if you do not want to follow this terminology, this is fine, but if others do, they are not “false”.

    It is fine, of course, to use a different word (i.e. “projection” instead of a “sensitivity study”) as long as the meaning is clear and they mean the same thing.

    The IPCC, unfortunately, has failed to properly communicate that the “projections” are only providing, at best, what is possible, and does not in any way, provide a demonstration of quantitative likelihood or evidence of any actual regional and local prediction skill on multi-decadal time periods.

    I appreciate you engaging in this discussion, as you have permitted a clear statement of what the IPCC is actually doing when they introduce the terminology “projection”.

    With Best Regards

    Roger Sr.

    • Demetris Koutsoyiannis

      Dear Roger,

      I applaud your comment re. projection vs. prediction and I offer the following paragraph from the paper Koutsoyiannis et al. (2011), which is related to this issue:

      In fact, it is the IPCC that uses climate model outputs as predictions. Calling these by another name, such as “credible quantitative estimates of future climate change” (Randall et al., 2007, p. 591) does not change the essence. For example, in IPCC (2007, Fourth Assessment Report—AR4; Summary for policymakers, p. 15), we read (our emphasis): “It is very likely that hot extremes, heat waves and heavy precipitation events will continue to become more frequent”. This is one of a total of six occurrences of the word “will” in a similar context (in the three next pages of the section “Projections of future changes in climate”), the last one being “…anthropogenic carbon dioxide emissions will continue to contribute to warming and sea level rise for more than a millennium”—not to mention the over 20 appearances of expressions such as “it is expected”, “it would”, etc. The same style is adopted in other IPCC documents, including the Freshwater Chapter (Kundzewicz et al. 2007). The conviction that climate model outputs are credible predictions for the future propagates beyond IPCC texts, often without mentioning their origin (for which we cannot imagine anything else but climate models). Taking as an example the most cited “climate change” document (as seen by a Google Scholar search), the so-called Stern Review, we may see that in a single page (Stern 2006, p. vi of the Executive Summary) the word “will” appears ten times. The same is also obvious in many papers and conference talks, where sometimes the “projections” are presented as facts.

      Best regards,

      Demetris

      Reference

      Koutsoyiannis, D., A. Christofides, A. Efstratiadis, G. G. Anagnostopoulos, and N. Mamassis (2011), Scientific dialogue on climate: is it giving black eyes or opening closed eyes?, Hydrological Sciences Journal, 56 (7), 1334–1339.

  • Roger Pielke Sr.

    The discussion on the issue of multi-decadal climate prediction skill, and the meanings given to the terms “prediction”,”projection” and “scenario” have continued also in the comments to the posts

    http://klimazwiebel.blogspot.com/2013/07/prediction-or-projection-nomenclature.html

    http://klimazwiebel.blogspot.com/2013/06/vortragsankundigung-nacht-des-wissens.html [note you can use googe translator to translate the post but most of the comments are in English].

    My response to the claim by Hans von Storch regarding his proposed 4th type of climate modeling application is

    “Hi Hans

    Your fourth type of model application is using the process model application type and just renaming them as scenarios. There is really no fourth type.

    I have already documented the inadequacies of this approach in my guest post and comments at
    http://www.climatedialogue.org/are-regional-models-ready-for-prime-time/

    as well as on your weblog post

    http://klimazwiebel.blogspot.com/2013/07/prediction-or-projection-nomenclature.html

    What are being provided as “scenarios” to the impacts and policy communities are just model results with no demonstrated skill with respect to the climate metrics of climate change on multi-decadal time scales that they are requesting.

    I have shown that the use of scenarios as you define, with any claim that they have skill, is misleading those communities.

    Best Regards

    Roger Sr.”

    Roger A. Pielke Sr.

  • Roger Pielke Sr.

    In the July 2013 issue of the Bulletin of the American Meteorological Society, there are two articles on the issue of regional multi-decadal climate predictions (projections). They are

    Pielke Sr., R.A. 2013: Comment on “The North American Regional Climate Change Assessment Program: Overview of Phase I Results.” Bull. Amer. Meteor. Soc., http://journals.ametsoc.org/doi/pdf/10.1175/BAMS-D-12-00205.1

    and

    Mearns, L.O. , R. Leung, R. Arritt, S. Biner, M. Bukovsky, D. Caya, J. Correia, W. Gutowski, R. Jones, Y. Qian, L. Sloan, M. Snyder, and G. Takle 2013: Reply to R. Pielke, Sr. Commentary on Mearns et al. 2012. Bull. Amer. Meteor. Soc.,. doi: 10.1175/BAMS-D-13-00013.1. http://journals.ametsoc.org/doi/pdf/10.1175/BAMS-D-13-00013.1

    In the Mearns et al article, there is a remarkable admission. They write

    “We do argue, however, that regional climate models can provide useful information about climate change as long as there is some value in the large-scale infor¬mation provided by the multimodel GCM ensembles. This statement is a logical extension of the fact that regional climate predictability can be derived from regional forcing as well as the large-scale conditions. Hence, one would expect a fraction of the model skill demonstrated by the numerical experiments described in Mearns et al. (2012) to be retained in future pro¬jections given the role of regional forcing remains, and there is some skill in the large-scale conditions derived from the multimodel ensemble of GCM projections.”

    These authors ignore that there are major deficiencies in the GCM model runs (and thus in the ensemble) which I provided examples of in my guest post. Adding regional forcing cannot correct errors that are obtained from the larger model through lateral boundary conditions and interior nudging. The regional climate models are slaves to the parent global model results.

    Mearns et al write that

    “…one would expect a fraction of the model skill demonstrated by the numerical experiments described in Mearns et al. (2012) to be retained in future pro¬jections.”

    but fail to show what is the magnitude of this “fraction”. Their use of the words “one would expect” is just an hypothesis, but they present as a matter of faith. This is not the scientific method. It is incumbent on them to quantity this “fraction” both for replicating the current regional climate statistics, but also the CHANGES in the statistics over time.

    Simply, stating that

    “Approximately 100 articles have now been published using the NARCCAP simulations, with most articles by researchers other than NARCCAP principal investigators (PIs).”

    just means that these investigators have been mislead on the actual skill of the NARCCAP results if they are being used in lieu of reanalyses for current climate, and for impact and policy studies of future climate.

    In summary, I see that the regional climate modeling community is at a crossroads. They can either ignore the failings of their downscaling approach when applied to multi-decadal regional climate change impact studies and continue to mislead those communities, or they can reassess and focus on the quantification of the predictability of regional climate statistics and their changes on different time and space scales.

    If they continue to mislead, however, without quantifying their level of skill at predicting changes in regional climate statistics on multi-decadal time scales, they are not being honest to the impact and policy communities.

  • Arthur Smith

    This all seems a bit silly. Roger Pielke Sr.’s arguments seem to focus on semantics rather than any substance. As far as using regional model outputs as input to decision-making, surely every useful piece of information, together with associated uncertainties *AND* any arguments for why they might be wrong, should be used in making decisions about handling possible changes in climate. As regional modeling gains skill it should be able to reduce uncertainty and direct resources in a more focused way on those assets at greatest risk of loss, but no matter what the level of skill, the information they provide should not be ignored!

    And I disagree with both Jason and Roger – neither climate models nor observations provide the best tool for understanding – rather, an understanding of the essential physical components and how they interact and respond is what should provide the greatest understanding. A recent example of this is the debate that has arisen about the interactions between the jet stream and blocking events and the degree of ice cover in the Arctic. The apparent increase in blocking behavior was (as far as I know) not predicted by models, and before 10 years ago or so there was no observational evidence at all on the matter. Theoretical physical understanding of the interacting components – informed by both models and observations, is essential to understanding what’s going on. And I would up that theoretical understanding – applied through modeling and corroborated by comparison to observations, is what we are striving for especially in regional applications. Increases in extreme events, both precipitation and drought, changes in tropical storm behavior, etc. are all crucial regional climate change effects that we need to better understand, and neither models nor observation are sufficient by themselves to tell us what to expect.

  • Jim Cripwell

    I am certain that Roger Peilke needs no support from me; I am sure Roger has forgotten more about how the earth’s climate works than I know. But I do have a dog in this fight; my Canadian taxpayer dollars and the Keystone XL pipeline. So let me state my opinion that Roger is absolutely correct when he writes “Observations of the real world behavior of the climate system provide the best tool for understanding how the climate system works.”

    Athur Smith writes “And I disagree with both Jason and Roger – neither climate models nor observations provide the best tool for understanding – rather, an understanding of the essential physical components and how they interact and respond is what should provide the greatest understanding.”

    It is hard to disagree with the sentiments of what Athur wites, but he is being idealistic, and totally impractical. It will be at least decades, and maybe even centuries, before we understand the “essedntial physical proceesses” of how our climate works. There are dozens of reasons for this. Let me take but one.

    Please visit
    http://www.atmos-chem-phys.net/13/1039/2013/acp-13-1039-2013.html
    and
    http://judithcurry.com/2013/01/31/condensation-driven-winds-an-update-new-version/

    Anastassia Makeireva has a highly controversial theory about what causes winds. Her paper took two years before it was published; a process which normally takes two months. In the end, the editor published the paper, not because Anastassia had proved that she was right, but because nobody could pove that she was wrong.

    Now if we do not understand what causes winds to blow; if the models have the wrong physics as to why winds blow, then there is a fundamental error in our whole understanding of how our climate works. I have seen no estimate of cost as to how anyone would go about trying to find out if Anastassia is correct, but it must be in the millions of dollars, and there is no sign that this sort of money will be available for decades.

    So while Arthur is right on principle, he is totally wrong from the practical point of view. Roger Pielke is completely and utterly correct.

  • Jim Cripwell

    Bart van den Hurk, you write “The theory of Makarieva has been proved wrong by Meesters et al (2009),”

    I cannot comment on the science of whether Makarieva has been proven wrong or not. I merely note that her paper was published in January 2013, so the editors of the journal did not consider your reference, published in 2009, to be sufficient grounds to warrant the paper not being published.

  • vukcevic

    I looked into the CET variability, as the most scrutinised and longest regional set of data, also having good correlation with N. Hemisphere and some with global variability. Graphic results and extrapolation into future (with very short comments) are shown here:
    http://www.vukcevic.talktalk.net/CET-NV.htm

  • Curious George

    “In most cases the RCMs are found to do a reasonably good job of simulating the climate of the recent past.” True. Why? There is a huge difference between past and future; the past is much easier to forecast. I was tempted to say “or to model”, but the past is an important constraint for models, so they carefully adjust (many) parameters to match the past. So an acceptable performance of most global climate models for the past is almost guaranteed. That regional climate models can take past data from global climate models and produce a “reasonably good simulation of regional climate” is quite an achievement – almost a miracle to me.

    I looked into workings of NCAR’s CAM 5 – Common Atmospheric Model 5. They use a simplifying assumption that the latent heat of vaporization of water is a constant, independent of temperature. That causes an error of about 3% in the calculation of heat transfer by evaporation from tropical seas – probably where most of water on this planet evaporates. Do they know how this affects their iterative model? No. Why should I – or you – trust Global Climate Models?

  • Arthur Smith

    Roger Pielke Sr. in his latest comment seems to be what I can only describe as trolling. Does he deny the standard WMO definition of climate?

    “Climate in a narrow sense is usually defined as the “average weather,” or more rigorously, as the statistical description in terms of the mean and variability of relevant quantities over a period of time ranging from months to thousands or millions of years. The classical period is 30 years, as defined by the World Meteorological Organization (WMO). These quantities are most often surface variables such as temperature, precipitation, and wind. Climate in a wider sense is the state, including a statistical description, of the climate system.”

    http://www.wmo.int/pages/prog/wcp/ccl/faqs.html

    Skill in describing the statistical distribution of surface variables is very different from skill in predicting specific values of those variables over a limited period of time (like a decade). This is very fundamental. Boundary-value vs initial-value problems. Pielke seems to be deliberately confusing the two here. Why?

  • Eli Rabett

    Makarieva et al. is falsified by looking at the behavior of cloud formation as shown in any number of videos on YouTube (this is not a joke). For example this one which shows that as clouds form they expand as would be expected based on simple thermodynamical arguments (condensation provides heat energy to the surrounding atmosphere, which expands carrying the aerosols in the cloud outward)

  • Jim Cripwell

    Bart, you ask “1. How do we develop belief in scientific results, in particular in the ability of models to generate information about relevant possible futures?”

    Simple. Go back to Physics 101, and rely ONLY on empirical data. Forget about hypotheses like CAGW, and the output of non-validated models, and rely solely on what can actually be measured.

  • Andries Rosema

    Bart van der Hurk wrote:

    “We get repeated requests from users not to provide as realistic as possible predictions of the future, but to provide physically realistic pictures that can be understood and interpreted as a possible consequence of altering climate statistics. And to make this realistic pictures, a simple scaling is often not adequate to depict all consequences of an altered background climate, and models are useful tools for this.

    To my feeling this discussion is much more than a repeteated rephrasing of statements and opinions. It helped me to realize even stronger that it is the society stakeholder that should be put in the middle of our work, and with respect to this you and your colleagues have echoed a very convincing and true sound in the debate around climate science and its applications. Also we invest most of our research resources not in climate model development per se, but in assessing the information that can be extracted from the existing model archive, in novel applications that are at the heart of interaction with stakeholders, in producing pictures of how our future climate COULD look like, which help feeding the imagination of those that have to take the real decisions. And we fully understand and acknowledge that for most decisions climate is just one of the many entries that is affecting them. By this attitude we surely hope and expect to serve society optimally with our expertise and dedication”

    This is frightening. Science should not feed the fear of their stakeholders by “producing pictures of how our future climate COULD look like”. This statement provides good insight in how a hype like “global warming” develops

    I agree with Arthur Smith that it is all about the physical understanding and this has not been adressed on this site yet.

Off-topic comments (click to expand)

Jump to expert comments | Jump to public comments | Jump to off-topic comments

Leave a Reply