Metastasis is the principal cause of cancer death, yet we lack an understanding of metastatic cell states, their relationship to primary tumor states, and the mechanisms by which they transition. In a cohort of biospecimen trios from same-patient normal colon, primary and metastatic colorectal cancer, we show that while primary tumors largely adopt LGR5 + intestinal stem-like states, metastases display progressive plasticity. Loss of intestinal cell states is accompanied by reprogramming into a highly conserved fetal progenitor state, followed by non-canonical differentiation into divergent squamous and neuroendocrine-like states, which is exacerbated by chemotherapy and associated with poor patient survival. Using matched patient-derived organoids, we demonstrate that metastatic cancer cells exhibit greater cell-autonomous multilineage differentiation potential in response to microenvironment cues than their intestinal lineage-restricted primary tumor counterparts. We identify PROX1 as a stabilizer of intestinal lineage in the fetal progenitor state, whose downregulation licenses non-canonical reprogramming.