The biological mechanisms that underpin primate social evolution remain poorly understood. Asian colobines display a range of social organizations, which makes them good models for investigating social evolution. By integrating ecological, geological, fossil, behavioral, and genomic analyses, we found that colobine primates that inhabit colder environments tend to live in larger, more complex groups. Specifically, glacial periods during the past 6 million years promoted the selection of genes involved in cold-related energy metabolism and neurohormonal regulation. More-efficient dopamine and oxytocin pathways developed in odd-nosed monkeys, which may have favored the prolongation of maternal care and lactation, increasing infant survival in cold environments. These adaptive changes appear to have strengthened interindividual affiliation, increased male-male tolerance, and facilitated the stepwise aggregation from independent one-male groups to large multilevel societies.