DRAFT October 26, 1998
II. THE CONVENTIONAL APPROACH
There is a steadily increasing literature on monitoring of environmental resources. Many of the ideas and techniques advanced in this literature share a common conceptual framework that we refer to as the conventional approach. The first principle of the conventional approach is that ecological monitoring is designed to measure and understand change. Many basic statistical concepts that were developed in other natural resource fields such as agriculture and biology have been incorporated into the vernacular of ecological monitoring. In the following sections, we step through some of the basic monitoring concepts in order to develop a common understanding and vocabulary that will be used throughout this paper.
Underlying the conventional approach to monitoring is a simple conceptual model of how the world operates. Basically, this model divides ecosystem dynamics into two components--stressors and responses. Stressors are agents of change that operate through ecological processes to either directly or indirectly create a response. Both stressors and responses are measured through a set of surrogate indicators. For example, cattle grazing in riparian areas (a stressor) is believed to change the composition of streamside vegetation and reduce bank stability (responses). In a monitoring sense, cattle grazing could be indexed by tracking the number of animals grazing within a riparian area over a designated time period, or more simply by noting whether an area is grazed or not. Similarly, riparian vegetation could be tracked through various measurements of composition and structure, and bank stability might be indexed by looking at the frequency of slumping or the proportion of bank that is overhanging the stream channel. As this example illustrates, every stressor-response combination can be indexed in a variety of ways.
The conventional approach to monitoring next assumes that a statistical model can be constructed relating stressor indicators to response indicators in a manner that caricaturizes the relationships between ecological stressors and responses. The idea of caricature is important; no statistical model purports to perfectly describe the underlying ecological linkage. These statistical models can vary widely in their complexity, ranging from simple univariate tests between pre- and post-treatments in a given area, to complex multivariate models with nonlinear or interacting terms. The statistical models are vital to the monitoring effort: ultimately the power to measure change is wholly dependent upon the statistical properties of the models linking ecological indicators. Thus, the first rule of monitoring is to start with a sound statistical model. One should be extremely wary anytime that a monitoring plan is proposed without proper specification of and justification for an explicit statistical model of the elements in question.
Noss and Cooperrider (1994) divide monitoring into four basic types: baseline, implementation, effectiveness, and validation. This classification has gained favor as a useful typology within the Forest Service (e.g., ICBEMP DEIS 1996), and we review them here.
Baseline monitoring is used to establish baseline reference conditions that can be used to quantify a change that might be due to management activities. Often, areas are included in the baseline set that are deemed to be minimally affected by management (e.g., Overton et al. 1993). In such cases, the underlying assumption is that differences in measurements taken in reference areas versus managed areas can be attributed to management.
Implementation monitoring, sometimes called compliance monitoring, is simply tracking to see if management direction is accurately interpreted and followed. Conceptually, it should be straightforward to monitor implementation, assuming that the initial direction is clear. Where there is imprecision in the direction itself, it can be difficult to tell who is at fault for poor implementation success.
Effectiveness monitoring is intended to measure whether progress is being made towards an objective. Every management decision is intended to achieve a given set of future conditions. Effectiveness monitoring can be used to compare existing conditions to both past conditions and the desired future conditions to describe the overall progress or success of the management activities.
Validation monitoring seeks to verify the assumed causal linkages between cause and effect. Generally, validation monitoring requires a more intensive effort than the other forms, albeit at a more limited number of sampling sites. As its name implies, validation monitoring is intended to validate the basic assumptions under which the management direction was developed.
It is important to note that validation monitoring and effectiveness monitoring reach their full potential only when used in combination. That is, knowing that the overall goals are or are not being met (effectiveness monitoring) is rather pointless without some assurance that the observed effect is due to the management activities (validation monitoring). Similarly, while its nice to know that one’s understanding of the world is correct (validation monitoring), that is of little utility in determining whether overall goals are being met (effectiveness monitoring). These two types of monitoring can be highly complementary if fully integrated.
There also is a misconception by some that effectiveness monitoring is somehow easier or less rigorous than validation monitoring. This has led to suggestions that effectiveness monitoring can be readily handled by the National Forest System and the BLM districts, while validation monitoring is shifted to research institutions. The fallacy in this case is (1) assuming that effectiveness monitoring is necessarily easy, and (2) decoupling validation and effectiveness monitoring into separate and perhaps disconnected exercises.
One of the major topics in the monitoring literature is the difference between design-based and model-based inference. In simplest terms, design-based approaches derive their utility from the strength of the sampling design used to gather the information. The observed sample is randomly drawn from a specific target population; inferences from the sample to the population at large follow naturally. In contrast, model-based approaches do not rely on random sampling. Rather, an attempt is made to identify representative sites, sometimes called sentinel sites, that are intensively studied for the purposes of constructing a more detailed model of the ecological process in question. This model is then applied more widely to similar sites or locations.
As an example, consider a plan to monitor eutrophication in small, high-elevation lakes through time. In a design-based approach, a relatively large number of lakes would be randomly sampled across the landscape, with every lake having the same probability of being included in the sample. Because of their potential remoteness, each lake might be sampled periodically for a limited set of water quality indicators. Tracking these indicators through time would indicate water quality changes in the population of lakes in question. In contrast, a model-based approach would intensely study eutrophication in a small number of high-elevation lakes in order to develop a more thorough understanding of eutrophication and use the results of that study to make some predictions about the level of eutrophication expected in lakes throughout the region.
The advantages of design-based approaches are that there should be no question about the populations for which the findings apply, assuming that the designs are correctly applied. The drawback with design-based approaches is that they can require large sample sizes and the probabilistic sampling called for in the design can increase the cost of collecting data substantially. Careful attention to probabilistic sampling and the use of advanced sampling designs can help alleviate some of these problems.
Model-based approaches can be more efficient in terms of collecting and using a variety of information, but it can be very hard to find truly representative sites. Using a small number of sites as indicators for the population at large, the so-called sentinel site approach, is especially problematic. Jassby (1998) identifies three demanding conditions that must be fulfilled for a network of sites to serve as sentinel sites for a particular stressor: (1) some subset of the network must encounter the stressor; (2) at some sites must be responsive to the stressor; and (3) the background noise at the sites must not disguise the response to the stressor. Jassby cautions that, "reliable extrapolation from sentinel-site networks to regional trends appears to be beyond our ecological understanding at the present time." Most statisticians tend to agree that model-based approaches such as sentinel sites are best used for understanding processes rather than measuring regional trends (Olsen et al. 1997).
In general terms, trend refers to a unidirectional change in an indicator variable across a second dimensional variable (e.g., space or time). In a ecological monitoring context, trends are often expressed across time, where time is viewed discretely as in annual increments. The statistical literature is rich in methodologies for analyzing trends in individual time series. In the general case, these time series analyses require only the time trace of the indicator variable. The underlying assumption is that future observations can be modeled solely as a function of past observations; causality is not addressed. Unfortunately, robust results from time series analysis require series considerably longer series than are generally available for ecological indicators.
Short time series raise all kinds of problems for ecological inference. For example, we know that natural systems exhibit a variety of cyclic patterns with alternating periods of increases and decreases. Many times there are even multiple cyclic patterns with different frequencies and amplitudes superimposed on each other. This creates the obvious problem of interpreting a relatively short increase or decrease in an indicator. Is the observed increase/decrease a part of the natural cycling or is it due to some fundamental shift in ecological process?
The case of a regional trend analysis that simultaneously assesses multiple sites that may display contrasting trends is an especially complex and challenging case. Urquhart et al. (1998) provide an example of regional trend analysis using water quality data collected by EPA. Urquhart et al. simply their analysis by using strictly linear models as a first approximation. Their analysis suggests that as concordant variation--the variation of all sites together around a common trend--increases, the ability to detect regional trends decreases. The best sampling designs required 10-15 years of sampling data to detect moderate trends. Urquhart et al. used the arithmetic average of the trends observed across sites as an indicator of regional trend. This raises the question, however, of whether a situation where half the sites are increasing and half are decreasing is truly a "no change" scenario ecologically.
We summarize key points regarding the conventional approach to ecological monitoring as follows: