Communities are comprised of individual species that respond to changes in their environment depending in part on their niche requirements. These species comprise the biodiversity of any given community. Common biodiversity metrics such as richness, evenness, and the species abundance distribution are frequently used to describe biodiversity across ecosystems and taxonomic groups. While it is increasingly clear that researchers will need to forecast changes in biodiversity, ecology currently lacks a framework for understanding the natural background variability in biodiversity or how biodiversity patterns will respond to environmental change. We predict that while species populations depend on local ecological mechanisms (e.g., niche processes) and should respond strongly to disturbance, community-level properties that emerge from these species should generally be less sensitive to disturbance because they depend on regional mechanisms (e.g., compensatory dynamics). Using published data from terrestrial animal communities, we show that community-level properties were generally resilient under a suite of artificial and natural manipulations. In contrast, species responded readily to manipulation. Our results suggest that community-level measures are poor indicators of change, perhaps because many systems display strong compensatory dynamics maintaining community-level properties. We suggest that ecologists consider using multiple metrics that measure composition and structure in biodiversity response studies.