A comparison of regression methods for model selection in individual-based landscape genetic analysis

Andrew J. Shirk, Erin L. Landguth, Samuel A. Cushman

Research output: Contribution to journalArticlepeer-review

92 Scopus citations


Anthropogenic migration barriers fragment many populations and limit the ability of species to respond to climate-induced biome shifts. Conservation actions designed to conserve habitat connectivity and mitigate barriers are needed to unite fragmented populations into larger, more viable metapopulations, and to allow species to track their climate envelope over time. Landscape genetic analysis provides an empirical means to infer landscape factors influencing gene flow and thereby inform such conservation actions. However, there are currently many methods available for model selection in landscape genetics, and considerable uncertainty as to which provide the greatest accuracy in identifying the true landscape model influencing gene flow among competing alternative hypotheses. In this study, we used population genetic simulations to evaluate the performance of seven regression-based model selection methods on a broad array of landscapes that varied by the number and type of variables contributing to resistance, the magnitude and cohesion of resistance, as well as the functional relationship between variables and resistance. We also assessed the effect of transformations designed to linearize the relationship between genetic and landscape distances. We found that linear mixed effects models had the highest accuracy in every way we evaluated model performance; however, other methods also performed well in many circumstances, particularly when landscape resistance was high and the correlation among competing hypotheses was limited. Our results provide guidance for which regression-based model selection methods provide the most accurate inferences in landscape genetic analysis and thereby best inform connectivity conservation actions.

Original languageEnglish
Pages (from-to)55-67
Number of pages13
JournalMolecular Ecology Resources
Issue number1
StatePublished - Jan 2018


  • Mantel test
  • landscape genetics
  • linear mixed effects model
  • model selection
  • regression on distance matrices
  • simulation


Dive into the research topics of 'A comparison of regression methods for model selection in individual-based landscape genetic analysis'. Together they form a unique fingerprint.

Cite this