Little is known about levels and patterns of genetic diversity for the entire range of endangered orchids native to China, Korea, and Japan. In this study, we focus on Cypripedium japonicum and suggest three hypotheses: 1) that genetic drift has been a primary evolutionary force; 2) that populations in central and western China harbor higher levels of genetic variation relative to those from eastern China; and 3) that C. japonicum in China maintains the highest genetic variation among the three countries. Using ISSR and SCoT markers, we investigated genetic diversity in 17 populations to test the three hypotheses. As anticipated, we found low levels of genetic diversity at the species level with substantially high degree of genetic divergence, which can be mainly attributed to random genetic drift. Chinese populations harbor the highest within-population genetic variation, which tends to increase from east to west. We also found a close relationship between Korean populations and central/western Chinese populations. Historical rarity coupled with limited gene flow seems to be important factors for shaping genetic diversity and structure of C. japonicum. Our results indicate that the mountain areas in central and western China were likely refugia at the Last Glacial Maximum.