Summary of Research, July 1997 - June 1998.

The manuscript contrasting the "BACI" and "ANOVA" approaches to assessment of local environmental impacts was submitted August 29 to Ecological Applications. However, it was too long for this (80 pp. of text and 40 pp. of Appendices), so was sent on to Ecological Monographs. Reviews came back in early May. They are favorable, raising no issues of content as far as I can tell, but some of style. The editor involved has encouraged a revision. The main concerns are length, clarity, and a "tendency to personalize the issues." Judging from the editor's letter and the reviews, the length issue is less important than the others - they feel the paper is "an important contribution to an important topic" and are concerned whether it will be read. They seem to feel clarity is key to this - there is a large number of analyses and some "relatively complex notation." They suggest a table to help readers keep track of analyses, and the editor suggests using numerical examples. The "personalization" issue arises because much of our paper is a contrast between the methods we advocate and methods which seem similar but are in fact very different, proposed mainly but not only by A. J. Underwood. In trying to represent these methods fairly, we quoted extensively from Underwood's papers; we also used "UAA," for "Underwood's ANOVA Approach," as a shorthand for these methods. It seems hard to avoid the quotations without risking misrepresenting this approach (or at least being accused of it), but we should be able to find a less personal shorthand for it, and also to add some other authors to quote. Although the changes do not seem large, I expect to spend much of the summer on them. I do think this paper is important, with some real stakes involved - the money involved in designing studies and making decisions on the basis of them, and a better understanding of the meaning of statistical inference in investigations which do not consist of obvious "replications."

The paper aims to establish basic principles on which assessment and monitoring projects can be based, especially where uncertainty calculation (inference) is involved. It lays out the relevant "effects" and the sources of uncertainty in our estimates of them, and suggests general ways to design the study and analyze the data so as to improve the estimates and reduce the uncertainty. However, these suggestions are not rooted in real examples, and they make some assumptions which have never (as far as we know) been tested. The main one is that it will often be possible to choose one or more "comparison" sites which are similar in some sense to the "Impact" site, i.e., the site at which some human alteration is to be built and whose change or deterioration is to be measured. The idea is that abundances, or other values, at the comparison sites can be used to estimate Impact site abundances during the period before the alteration is built, and thus can be used after the alteration to predict what the Impact sites values would have been had the alteration not been built. Of course, Impact site abundance at any time can be estimated on the basis of past Impact site values, so the comparison sites are useful only if they lead to better estimates and predictions: specifically, we want estimates whose errors are smaller and less affected by long-term serial correlation, than those based on past Impact values alone. An obvious first guess at a "similar" site is a nearby site that seems to have similar physical features (like depth and aspect) and a similar community of species. But whether such sites can be used to predict each other's abundances, or just what features make sites "similar" with respect to a given species or species group, does not seem to have been studied.

The focus of our project now is on long-term data from neighboring sites. The best example we know of is the data collected annually (1982-present) at 16 sites off the Southern California coast by the Channel Islands National Park (CINP) service. There are several groups of these data, but so far we have worked only with the Kelp Forest group, consisting of about 70 species. We have begun two basic approaches to exploring these data.

1. Search for empirical patterns, without regard to specific mechanisms. The main tool here is tables of correlation: for each species we can construct the correlation over time between each pair of sites. Sites that appear to be highly correlated are picked first for further study, described below. Of course, with many sites and species, there will be many high correlation due to chance alone. We can't avoid this, but we can try to reduce it by calculating correlation in more than one way - e.g., using ranks or logs rather than raw numbers, or removing "outliers." The latter is motivated by the observation that some high correlation arise when both sites have low numbers in most years but very high numbers in a single year. To check this, we use a form of jackknifing: we calculate the correlation that would arise if one year's data were missing, and report the lowest of these, i.e., the one obtained by omitting the year with the biggest effect. This is not necessarily a more "valid" calculation, just a way to check for chance effects and determine the types of correlation or distribution that can arise: it may be that the most beneficial effect of a comparison site is to reduce the distorting effects of very rare years of high or low abundance.

From these correlation tables, we try to find site pairs that tend to behave alike. One way to do this is simply to choose site pairs which are highly correlated for a particular species. This can be useful if the species is important, consistently abundant and reasonably well estimated, but overall it gives too many cases to look at. The number can be reduced by seeking site pairs which are highly correlated for several species, preferably species with similar life history characteristics, implying that the correlation might be caused by some associated mechanism rather than chance. This can be done by listing, for each pair of sites, the species whose correlation is above some cutoff value, then assessing the similarity of these species with respect to life history. The number of pairs to be checked can be reduced by varying the cutoff, or the number of species required to exceed it, or by categorizing the species into possibly overlapping life history groups and requiring a minimum number of same-group species.

Once a promising site pair is determined, we make time plots of the species, using raw and transformed (logs, square roots and reciprocals) abundances and their ranks, and also attempt to fit each series to a standard autoregressive moving-average (ARMA) time series model. We also fit each time series to the other, at present just using simple linear regression, and carry out similar plots and model fits for the residuals of these fits. We are particularly interested in removing signs of very long-term correlation, indicating that "random" environmental perturbations can continue to affect abundances for several years. These kinds of chance errors can make the conclusions of statistical assessment work seriously and unexpectedly wrong, since they might persist through most of the study (or, often as bad, through most of either the Before or the After part of the study) and thus not be detected from the pattern of fluctuations and not included in the calculation of error. Even studies with Before and After periods of five or more years can be badly affected by such a problem. Our sixteen or so years of data may give us a better chance of determining whether and when problems like this arise, and how well comparison sites help deal with them. It is possible that the use of comparison sites might be justified by removing or reducing the chance of these problems even if the result is an apparent increase in variability estimated assuming that years are independent or that serial correlation is only short-term.

2. The second approach is to seek site pairs which are related by mechanisms known to affect abundance, and check how these predictions are borne out by the data. Population abundances are determined by recruitment and survival, so locations with similar abundance patterns will be those having similar physical effects on survival and open to the same sources of recruitment. Steve Schroeter has begun calculating distances between sites by straight lines and along isobaths (or around the coastline of the island) to indicate similar susceptibility to storms or upwelling patterns. However, these are more important for survival than for recruitment. We expect recruitment to be the more significant connection between sites, especially for most benthic invertebrates and plants. This is likely to be largely determined by current patterns. These are complicated for the Santa Barbara Channel, but have been studied by several authors. We are constructing distance metrics that take into account North vs. South sides of islands; East vs. Western end of the chain; and warm vs. cold (to indicate whether the main currents are from the North or the South), and hope to make metrics based on more detail of the currents, including seasonal patterns related to recruitment seasons of particular species. The Park Service also has a large data set on temperature which may help indicate when currents are bringing water to a site from the South and when from the North. A grad student helping us, Anna Valeva, has begun analyzing these data for spatial and seasonal patterns.

This approach can be used in ways similar to the first one, but also in other ways. We hope it will suggest site combinations that allow one site to be predicted from two or more others, not just one. Because of the enormous number of combinations, this might not be feasible by the more scattershot first approach. It can also suggest prediction models other than linear regression and time series models for the errors other than ARIMA models, e.g., derived from the mechanisms represented by the inter-site "distances," or just arising from more extensive data plotting, because we can reduce the number of site combinations we need to explore. Non-Normal time series may be particularly worth considering: non-Normality usually has only small effects in standard statistical problems using independent data, because sums and averages quickly become approximately Normal, but they may have bigger effects in time series since a single extreme value (or the environmental perturbation that caused it) can affect the values obtained at later times.

The approach may also suggest ways to determine good comparison sites in future problems. We will compare the correlation between pairs of sites with various measures of the "distance" between them. We expect different measures of distance to be appropriate for different species, usually corresponding to the mechanisms or environmental variations which most strongly affect their abundances. Thus species with high survival but variable recruitment might be most strongly correlated between sites sharing the same currents in the recruitment season, while those with steady recruitment but high juvenile mortality might be strongly correlated between sites sharing the same upwellings, storms or aspect.

We continue to seek other data sets, e.g., data kept by sanitation districts. Data that have been collected several times a year may be especially interesting. Seasonal variation is usual in abundances, but not easy to deal with: the patterns are not always smooth like the standard modeling functions (sine waves), and the "seasons" themselves may vary considerably from year to year. Comparison sites may be particularly useful for removing seasonal effects of these kinds. However, even if they are successful, there are other possible modeling problems. Standard time series models assume data are taken at equal time intervals, each equally subject to perturbation. Neither of these assumptions may hold for these data.

Problems Encountered:

Nothing major. We will be doing a great deal of computer analyses and pattern-seeking, so will begin the summer by getting faster computers with more memory.

Future Plans:

Our most immediate aim is to get the BACI vs. ANOVA ms revised and accepted. We are continuing with developing measures of distances between sites (based on maps at present but hopefully on currents and temperature data before long) and, as reported recently, with debugging and extending the programs for computing correlation between sites. Summer is also a good time for literature search and reading work, which have been lagging. I have also been asked to give the closing talk on "Proper interpretation of experimental results" at the annual meeting of the Wildlife Society on "The role of large-scale experiments in wildlife management: principles and practice," and expect this to take quite lengthy preparation.