Impact of long-term chromosomal shuffling on the multispecies coalescent analysis of two anthropoid primate lineages

Large ancestral effective population size explains the difficult phylogenetic placement of owl monkeys

The phylogenetic position of owl monkeys, grouped in the genus Aotus, has been a controversial issue for understanding Neotropical primate evolution. Explanations of the difficult phylogenetic assignment of owl monkeys have been elusive, frequently relying on insufficient data (stochastic error) or scenarios of rapid speciation (adaptive radiation) events. Using a coalescent-based approach, we explored the population-level mechanisms likely explaining these topological discrepancies. We examined the topological variance of 2,192 orthologous genes shared between representatives of the three major Cebidae lineages and the outgroup. By employing a methodological framework that allows for reticulated tree topologies, our analysis explicitly tested for non-dichotomous evolutionary processes impacting the finding of the position of owl monkeys in the cebid phylogeny. Our findings indicated that Aotus is a sister lineage of the callitrichines. Most gene trees (>50%) failed to recover the species tree topology, although the distribution of gene trees mismatching the true species topology followed the standard expectation of the multispecies coalescent without reticulation. We showed that the large effective population size of the common ancestor of Aotus and callitrichines was the most likely factor responsible for generating phylogenetic uncertainty. On the other hand, fast speciation scenarios or introgression played minor roles. We propose that the difficult phylogenetic placement of Aotus is explained by population-level processes associated with the large ancestral effective size. These results shed light on the biogeography of the early cebid diversification in the Miocene, highlighting the relevance of evaluating phylogenetic relationships employing population-aware approaches.

Impact of long-term chromosomal shuffling on the multispecies coalescent analysis of two anthropoid primate lineages

Multispecies coalescent (MSC) theory assumes that gene trees inferred from individual loci are independent trials of the MSC process. As genes might be physically close in syntenic associations spanning along chromosome regions, these assumptions might be flawed in evolutionary lineages with substantial karyotypic shuffling. Neotropical primates (NP) represent an ideal case for assessing the performance of MSC methods in such scenarios because chromosome diploid number varies significantly in this lineage. To this end, we investigated the effect of sequence length on the theoretical expectations of MSC model, as well as the results of coalescent‐based tree inference methods. This was carried out by comparing NP with hominids, a lineage in which chromosome macrostructure has been stable for at least 15 million years. We found that departure from the MSC model in Neotropical primates decreased with smaller sequence fragments, where sites sharing the same evolutionary history were more frequently found than in longer fragments. This scenario probably resulted from extensive karyotypic rearrangement occurring during the radiation of NP, contrary to the comparatively stable chromosome evolution in hominids.