Embryonic stem cells can be incorporated into the developing embryo and its germ line, but, when cultured alone, their ability to generate embryonic structures is restricted. They can interact with trophoblast stem cells to generate structures that break symmetry and specify mesoderm, but their development is limited as the epithelial-mesenchymal transition of gastrulation cannot occur. Here, we describe a system that allows assembly of mouse embryonic, trophoblast and extra-embryonic endoderm stem cells into structures that acquire the embryo's architecture with all distinct embryonic and extra-embryonic compartments. Strikingly, such embryo-like structures develop to undertake the epithelial-mesenchymal transition, leading to mesoderm and then definitive endoderm specification. Spatial transcriptomic analyses demonstrate that these morphological transformations are underpinned by gene expression patterns characteristic of gastrulating embryos. This demonstrates the remarkable ability of three stem cell types to self-assemble in vitro into gastrulating embryo-like structures undertaking spatio-temporal events of the gastrulating mammalian embryo.