The retrieval of ancient DNA from osteological material provides direct evidence of human genetic diversity in the past. Ancient DNA samples are often used to investigate whether there was population continuity in the settlement history of an area. Methods based on the serial coalescent algorithm have been developed to test whether the population continuity hypothesis can be statistically rejected by analysing DNA samples from the same region but of different ages. Rejection of this hypothesis is indicative of a large genetic shift, possibly due to immigration occurring between two sampling times. However, this approach is only able to reject a model of full continuity model (a total absence of genetic input from outside), but admixture between local and immigrant populations may lead to partial continuity. We have recently developed a method to test for population continuity that explicitly considers the spatial and temporal dynamics of populations. Here, we extended this approach to estimate the proportion of genetic continuity between two populations, using ancient genetic samples. We applied our original approach to the question of the Neolithic transition in Central Europe. Our results confirmed the rejection of full continuity, but our approach represents an important step forward by estimating the relative contribution of immigrant farmers and of local hunter-gatherers to the final Central European Neolithic genetic pool. Furthermore, we show that a substantial proportion of genes brought by the farmers in this region were assimilated from other hunter-gatherer populations along the way from Anatolia, which was not detectable by previous continuity tests. Our approach is also able to jointly estimate demographic parameters, as we show here by finding both low density and low migration rate for pre-Neolithic hunter-gatherers. It provides a useful tool for the analysis of the numerous ancient DNA data sets that are currently being produced for many different species.
Keywords: Neolithic transition in Europe; ancient DNA; genomewide autosomal data; mtDNA; partial population continuity; population genetics; serial coalescent; spatial explicit simulations.