Organ-on-a-chip development is an application that will benefit from advances in cell heterogeneity characterization because these culture models are intended to mimic in vivo microenvironments, which are complex and dynamic. Due in no small part to advances in microfluidic single cell analysis methods, cell-to-cell variability is an increasingly understood feature of physiological tissues, with cell types from as common as 1 out of every 2 cells to as rare as 1 out of every 100 000 cells having important roles in the biochemical and biological makeup of tissues and organs. Variability between neighboring cells can be transient or maintained, and ordered or stochastic. This review covers three areas of well-studied cell heterogeneity that are informative for organ-on-a-chip development efforts: tumors, the lung, and the intestine. Then we look at how recent single cell analysis strategies have enabled better understanding of heterogeneity within in vitro and in vivo tissues. Finally, we provide a few work-arounds for adapting current on-chip culture methods to better mimic physiological cell heterogeneity including accounting for crucial rare cell types and events.