DRAFT October 26, 1998
III. COMPLICATIONS
While the conventional approach to monitoring has enjoyed moderate success (Spellerberg 1991), monitoring plan development for projects such as the NWFP and ICBEMP soon encounter difficulties. In large part, the potential disparity between expectations of monitoring and its successful execution derives from the fact that the spatial scales at which we desire certainty are much greater than before. Not only are we dealing with larger scales, but also with notoriously open systems where change can result from unexpected quarters. We have little to no control over unintended or unexpected factors such as flooding events or unknown influences on the landscape behavior from past disturbances. Finally, both management agencies and the public desire high levels of certainty despite the intrinsic complexity and non-linear behavior of the environment.
The emergence of ecological theories (i.e., hierarchy theory, patch dynamics, landscape ecology, metapopulations), all which constitute the theoretical foundations for the NWFP and ICBEMP, involve spatially and temporally explicit consideration of ecological processes and patterns. As a result, large-scale monitoring has shifted to include not only the description of populations distributed across large spatial extents, but also to the description of large-scale processes: those ecological phenomena that encompass and occupy large, contiguous areas (e.g., watershed functions, metapopulations). In contrast, while ecological knowledge and scientific confidence are confined mostly to fine scales, great uncertainties about large-scale ecological processes abound (Ricklefs and Schulter 1993). We examine how the shift to an ecosystem and large-scale perspective challenges implementation of the four basic monitoring types.
A fundamental assumption in baseline monitoring (namely that measurement differences in reference versus managed areas can be attributed to management) must be carefully reassessed in the process of monitoring plan formulation for large scale systems. The presence of multiple variables in an open system cause landscape ecologists to wrestle with such questions: What constitutes reference conditions and reference sites in the landscape context? What constitutes a replication at the watershed scale, and what constitutes a reasonable "population" of watersheds?
The concept of a reference or replicate landscape is difficult. First, very often it is difficult to distinguish the effects of either past human or natural disturbances on the present landscape pattern. While at first glance two landscapes may appear quite similar, their subsequent behavior may be quite different. Given the subtleties of ecological interactions over time and space, it is problematic to designate an unmanaged or control area and its supposed complement for treatment. Differences in the order of disturbance, or differing combinations of various processes may create the same landscape pattern relative to our levels of detection. Thus, if we observe a difference in the response between the control and treated watershed, can we conclude with certainty that the difference derives from the treatment? Perhaps the difference in observed responses is the result of differing histories and the management treatment under question is relatively incidental. These issues require careful planning and problem definition to achieve a statistically sound experiment.
In practice, implementation monitoring should be the most straightforward of the monitoring types, provided that management direction is sufficiently clear. While scaling to larger frames may create logistical challenges, other issues should be minimal. The main challenge is to provide clear direction, but at the same time maintain the flexibility to accommodate local conditions and needs. It is easy to tell if an agency has followed the letter of the law, but more difficult to tell if the intent was properly followed. Problems arise with in sufficient articulation, poor training, and a lack of the right combination of specificity and flexibility to address the spectrum of objectives across scales.
Judging management success (i.e. the goal of much of effectiveness monitoring) first requires a definition of success and specific criteria by which the plan may be assessed. For example, in the case of a single species such as the Northern Spotted Owl, this may mean that the population has reached some predetermined level or composition. The identification and establishment of success criteria implicitly assumes the use of a working hypothesis (i.e., model) which articulates the relationship between the stressor (e.g., management) and response (e.g., population status). In the case of regional level assessments then, a specific model needs to be identified to allows for scaling of information collected at the fine scale (e.g., district) to the large scale (i.e., region). In other words, we need to articulate what assumptions are made when we use data collected at one set of scales (and under one set of assumptions)and use them at another scale (under a different set of assumptions). Model development needs to address both conceptual and technical issues involved. For example, the scaling process requires addressing of such questions as: Does the simple sum of individual numbers collected in the districts constitute an adequate representation of ecological condition for the region scale? Since the spatial distribution of responses and stressors involved are likely to vary, should individual reports be weighted by area? Has management direction succeeded if there are increases in some areas and decreases in others, leading to a zero net change for the region? Should a mechanistically or statistically-based model be used for the region?
Formulation of effectiveness measures for the region and relating them to data throughout the region requires the development of a conceptual framework which allows not only for both scaling from the bottom up but also the top down; it is necessary to scale down from the basin and regional levels to local levels, in the process of stepping down policy directives to district procedure. This entails the ability to concretely relate broad-brush policies all the way down to specific sampling designs. For example, in the NWFP this type of challenge begins with translating the Presidential mandate to "maintain biological diversity", and the series of decisions which relate this directive to a given sampling design for the Northern Spotted Owl at the district level.
Large scales imply not only larger spatial scales but also larger temporal frames. General ecological theory suggests that as the spatial scale of processes increase, the temporal scales over which significant change occurs also increase. Climatic responses and ecological scales of variation occur over decades to centuries and even millennia. From the sampling perspective, this means that time scales of 20 to hundreds of years may be needed to detect meaningful change in the system. However, while many of the target organisms and processes have life histories of decades to centuries, monitoring and decisions based on these data are scheduled annually or at most a few years; the institutions which oversee monitoring long-term processes and organisms are governed at very different, and much shorter, time scales. The scientific process and time scales associated with many ecological phenomena are infrequently commensurate with the scales at which the public and politicians function (Bradshaw 1998; Bradshaw and Fortin 1998). Ecological theory demands consideration of large-scales; policy requires statistically rigorous information for decision-making tractable thus far only at much finer scales (Bradshaw and Fortin, 1998; Ludwig et al,1993). As a result, it is often not possible to statistically distinguish human-derived from natural change given such short periods for detection (Innes 1998). Inference strength used to judge management success depends on the appropriateness of both the spatial and temporal sampling frame used for a given variable or attribute. This entails not only extending the temporal extent (e.g., allowing longer periods of time for monitoring before a decision is made) but also allowing flexibility in institutional response times (see adaptive management: Walters 1986; Gunderson et al. 1995). Many of these issues are central to some of the limitations of trend analysis discussed above.
The type of change, the levels, magnitude and types of change which define management success or effectiveness should be specified prior to monitoring to inform how, when, where, and what variables need to be sampled; that is, it must be decided what constitutes significant change for the region. Ecologically important change can occur in many forms: changes in trend, frequency, timing, variability, etc. Subsequently, all criteria for effectiveness may not be expected to be explained or supported by trend analysis. Ecosystems are notoriously non-linear in behavior. Because we have incomplete knowledge of system history, including land-use and natural disturbances, and the nature of multi-scale interactions, the significance of a simple increase or decrease is difficult to interpret. Ascertaining cause and effect is difficult. A change in slope over time may actually reflect a longer period response to climatic shifts rather than a specific shift in management practices. Specific responses to management may be hidden by other environmental signals during the sampling period. Complex and multiple interactions across scales impede ready identification of cause and effect and attenuate our abilities to detect meaningful change. The existence of cumulative effects, non-linear, and threshold behaviors argue strongly for the coupling of validation and effectiveness monitoring.
Once models and success criteria are selected, it is necessary to design a sampling scheme. This task is often frustrating. As discussed above, , the identification of the appropriate sampling units, frames, and population for large-scale, multi-scale, and multi-variate systems is difficult. Additionally, once these attributes are identified and sampling implemented, the results are often disappointing. Statistics and ecology do not always seem to match. For example, in some cases the biologist has a firmly held belief, either from extant literature from studies elsewhere, experience etc., that a given stressor has an important effect on a given process (e.g., sedimentation on fish health). However, when data comes back and is analyzed, the statistics indicate differently. The apparent mismatch between what is perceived as ecologically important change and its detection using statistical criteria occurs because of the aforementioned spatial and temporal scaling problems, but also because it is sometimes difficult to accurately measure critical, discrete quantities which will capture the dynamic and connected behavior of a complex system.
To this end, the use of indicators or indices (e.g., watershed integrity) have been suggested to help address the problem of handling multiple stressors and responses for large-scale behaviors. These integrative measures are intended to capture the essence of the ecosystem behavior when single or clusters of variables seem to fail. However, this introduces another problem. Often these value-laden indices rely on a qualitative or non-rigorous way of quantification. As a result, they are only relative terms with limited scope of inference beyond a certain set of ecological conditions, or are too subjective and introduce significant observer error, which is difficult to estimate. Thus, using such indices introduces another source of uncertainty in the inference and interpretation of monitoring results. In part, some monitoring can be directed to understanding these scale-related issues; such monitoring lies at the interface of research and validation monitoring.
Understanding knowledge gaps and sources and types of uncertainty, and articulation of the conceptual underpinnings of the monitoring plan are essential for appropriate assessment of effectiveness.
Many of the questions raised for effectiveness monitoring hold for validation monitoring. One of the most difficult, unresolved questions is: given the uniqueness of each watershed and landscape, how valid is it to generalize intensive, sentinel experiments to other sites with distinct environmental and land-use histories, let alone extrapolate to the region? Coordination with extant long-term sites and inter-site comparison coupled with modeling efforts can help in some cases.
Generally, larger spatial extents imply increased in sample sizes. Increased sample size means more time and effort are required for on-the-ground data collection. However, finite resources oblige management and planning teams operating at the district level to be parsimonious in the design of monitoring plans. Adequate sampling in space, time, and number of variables generally exceed local resource capacities. Logistics and funding constrain many studies which seek to use landscapes or watersheds as sampling units. As a result, managers must decide what variables receive priority. Fiscal constraints may be such that even the way in which a variable is sampled has to be altered to be tractable. The combination of multiple goals, large scales, and fiscal constraints put a tremendous pressure on resources and leave the question of prioritization up to the local manager. A given district or Forest is subject to multiple pressures and requirements ranging from local issues to national policies and regulations. Without thoughtful anticipation, insufficient or inadequate data for regional assessment may result. Prioritization of management objectives becomes a critical step to optimize resources and information.