Coalescent methods are robust to the simultaneous effects of long branches and incomplete lineage sorting

Abstract

It is well known that species with elevated substitution rates can give rise to disproportionately long branches in the species tree. This combination of long and short branches can contribute to long-branch artifacts (LBA). Despite efforts to remedy LBA via increased taxon sampling and methodological improvements in gene tree estimation, it remains unclear how long and short branches affect species tree estimation in the presence of incomplete lineage sorting (ILS). Here, we examine the combined influence of long external and short internal branches on concatenation and coalescent methods using both simulated and empirical data. Our results demonstrate that the presence of long and short branches alone does not obviously confound the consistency of concatenation and coalescent methods. However, when long external and short internal branches occur simultaneously with high ILS, concatenation methods can be misled, especially when two of these long branches are sister lineages. In contrast, coalescent methods are more robust under these circumstances. This is particularly relevant because this topological pattern also characterizes numerous ancient rapid radiations across the tree of life. Because short internal branches can increase the potential for ILS and gene tree discordance, our results collectively suggest that coalescent methods are more likely to infer the correct species tree in cases of ancient rapid radiations where long external and short internal branches are in close phylogenetic proximity.

Keywords: ancient rapid radiation; coalescent methods; concatenation methods; incomplete lineage sorting; long-branch artifacts.