Problems in the Analysis of Genetic Differentiation Among Populations – a Case Study in Quercus robur

The conclusions drawn from studies of genetic differentiation among populations largely determine our understanding of ecological and population genetic processes. These conclusions basically depend on the applied type of genetic marker and the method of measuring and estimating genetic differentation. However, concerns have been raised about the conceptual appropriateness of common methods of measuring genetic differentiation. The present paper contributes to the clarification of the problems involved by recalling the conceptual characteristics of F_ST (= G_ST), by specifying basic tests of the major causal factors of genetic differentiation with the help of permutation analysis, by comparing F_ST and Hedrick’s new normalization F’ST with the basic index δ of differentiation for data on allozymes and microsatellites obtained from 6 oak stands. All three descriptors display small values, among which δ is largest and closely followed by F’ST, while F_ST is distinctly smaller than both across all loci. Degrees of covariation of δ with F_ST and F’ST differ distinctly between allozymes and microsatellites as a probable consequence of confounding aspects of differentiation with aspects of fixation in the F_ST descriptors. Permutation analysis reveals that the boundary conditions provided by the number of populations and their (sample) sizes as well as the overall genetic variation across population samples determine the order of magnitude of differentiation. This mathematical artefact undermines the widely held opinion that small degrees of differentiation at many loci are the result of extensive gene flow or recent joint history. Differentation patterns vary considerably among allozyme loci (indicating the action of homogenizing and diversifying selection). In contrast, microsatellite loci consistently display significant differentiation as can be explained by mechanisms of non-recurrent mutation. These observations apply to all three descriptors for the relatively high within population polymorphism observed in the studied stands. At least for low within population polymorphism close to fixation, however, it is shown theoretically that the predictions may diverge distinctly among the three descriptors.