Asian cultivated rice is believed to have been domesticated from a wild progenitor, Oryza rufipogon, offering promising sources of alleles for world rice improvement. Here we first present a high-quality chromosome-scale genome of the typical O. rufipogon. Comparative genomic analyses of O. sativa and its two wild progenitors, O. nivara and O. rufipogon, identified many dispensable genes functionally enriched in the reproductive process. We detected millions of genomic variants, of which large-effect mutations could affect agronomically relevant traits. We demonstrate how lineage-specific expansion of gene families may have contributed to the formation of reproduction isolation. We document thousands of genes with signatures of positive selection that are mainly involved in the reproduction and response to biotic- and abiotic stresses. We show that selection pressures may serve as forces to govern substantial genomic alterations that form the genetic basis of rapid evolution of mating and reproductive systems under diverse habitats.