Introduction

Voles and lemmings (Arvicolinae, Muridae, Rodentia) are sometimes said to have more dental variation than any other mammal (e.g., Yablokov 1974; Carleton 1985). Variation can manifest itself at different hierarchical levels, notably between species (or clades) and within species (or populations). High levels of variation between species are an advantage for the study of fossil taxa because it makes species easily distinguishable from one another. Such inter-specific variation, which is sometimes called disparity, is known to be an important phenomenon in arvicoline rodents. The distinctive, rapidly evolving dental morphology of voles and lemmings make them important index fossils for late Cenozoic terrestrial biostratigraphy (e.g., Fejfar 1976; Repenning 1987; Maul et al. 1998; Bell et al. 2004). Conversely, variation within species is generally a disadvantage for the study of fossil taxa because it makes species difficult to diagnose. Such intra-specific variation is also known to be common in arvicolines, especially in the anterior cap of the lower molars (e.g., Hinton 1926; Van der Muelen 1973; Carleton 1985; Jaarola et al. 2004). High intra-specific variation can be an especial problem if it is coupled with low inter-specific variation because that combination leads to morphological overlap between species, hindering accurate identification of fossils and potentially leading to erroneous biostratigraphic or biogeographic interpretations.

Variation within species can be divided into genetic and environmental components of variance (e.g., Lynch and Walsh 1998). Quantitative geneticists describe this division using the equation

P = G + E, (Equation 1)

where P is the phenotypic variance, or the variation we see in morphological structures such as teeth, G is the genetic variance, or the variation in a species that is passed from parent to offspring, and E is the environmental variance, or the variation that is directly determined by local environment regardless of parentage. G represents heritable variation, regardless of whether the heritability has a direct one-to-one correspondence to genes coded on the DNA or a less direct but equally heritable developmental system whose morphogenetic outcomes are determined by the effects directly heritable regulator genes. E represents variation from all non-genetic sources, regardless of whether the sources are internal or external developmental noise, norms of reaction, or epigenetic variation. E is a quantitative genetics formulation of what is often known as morphological or ecophenotypic plasticity in the palaeontological literature (e.g., Newell 1948; Hughes 1991).

Generally speaking, the palaeontological study of taxonomy, species relationships, rates of evolution, and evolutionary patterns depends on variation being primarily genetic in nature because non-genetic variation may obscure underlying genetic and, therefore, evolutionary patterns. Conclusions about taxonomy or evolution could be misguided if environmental variance is great enough to cause undetected similarity among distantly related species or difference among conspecific populations. These potentially pernicious environmental effects, which are also called ecophenotypic or plastic variation, manifest themselves over the lifetime of an individual and should not be confused with evolutionary adaptation to environment, which is a consequence of genetic variance and natural selection over tens, hundreds, thousands, or millions of generations (e.g., Kratochvíl 1983). For these reasons, palaeontologists have often distinguished between the genetic and environmental components of variance in skeletal traits as a tool for better understanding evolutionary and environmental change (e.g., Schopf 1976; Pachut 1987; Hadly 1997; Polly 2004; Kavanagh et al. 2007).

Because the teeth of voles and lemmings are noted for both their inter- and intra-specific variation and because their teeth are so important for biostratigraphic interpretation, it is of interest to know the extent to which this variation is genetic. The proportion of genetic and environmental components of phenotypic variance is usually determined by large breeding experiments that allow the similarity between parents and their offspring to be measured directly or indirectly through 'common garden' experiments that bring genetically diverse individuals into the same environmental conditions in order to measure directly the resulting similarities (e.g., Lynch and Walsh 1998). Such experiments are expensive and time consuming even for living species, and in any case they cannot be applied to fossil animals.

Non-genetic variance can be estimated less laboriously by measuring asymmetry between right and left teeth. The right and left sides of an animal have the same genetic underpinnings, so the difference between the sides must logically be due to non-genetic factors (Grüneberg 1935; Van Valen 1962; Palmer and Strobeck 1986). Exceptions to this logic include directional asymmetry, consistent asymmetry in organs like the heart, and antisymmetry, consistent differences between right and left parts like the disproportionate size of the claws of some crabs (Morgan 1923; Rosenberg 2002). These exceptions do not, however, pertain to differences between right and left teeth, except insofar as the two sides are normally mirror images of one another. Difference in shape between left and right teeth (apart from mirroring) is an example of fluctuating asymmetry, or inconsistent, randomly distributed difference that results from the inability of the underlying genetics to determine identical structures. Leamy and Klingenberg (2005) reviewed the genetics of fluctuating asymmetry. An estimate of the variance between right and left teeth relative to the variance between the teeth of different individuals can be thought of as a minimum estimate of E, the environmental component of variance within that species, because the right-left asymmetry is non-genetic. Analysis of variance (ANOVA) can be used to mathematically partition the within-species variance in tooth shape into between-individual and between-sides components, the latter being an estimate of E. This estimate of environmental variance is a minimum one, however, because some environmental effects, like quality of diet or ambient temperature, will affect both sides equally to the extent that they have an effect at all. But despite its shortcomings, an estimate of E based on asymmetry can be made on fossil specimens as easily as extant ones, presuming that the left and right teeth can be associated with the same individual.

We estimated the minimum environmental component of variance using asymmetry in the outline shape of lower first molars from four species of arvicoline rodents. We used those estimates to determine the likelihood that environmental effects will result in erroneous species determinations. We also used the data to explore whether right and left molars from the same individual can be associated based on their shape alone.