Natural bone is a mineralized biological material, which serves a supportive and protective framework for the body, stores minerals for metabolism, and produces blood cells nourishing the body. Normally, bone has an innate capacity to heal from damage. However, massive bone defects due to traumatic injury, tumor resection, or congenital diseases pose a great challenge to reconstructive surgery. Scaffold-based tissue engineering (TE) is a promising strategy for bone regenerative medicine, because biomaterial scaffolds show advanced mechanical properties and a good degradation profile, as well as the feasibility of controlled release of growth and differentiation factors or immobilizing them on the material surface. Additionally, the defined structure of biomaterial scaffolds, as a kind of mechanical cue, can influence cell behaviors, modulate local microenvironment and control key features at the molecular and cellular levels. Recently, nano/micro-assisted regenerative medicine becomes a promising application of TE for the reconstruction of bone defects. For this reason, it is necessary for us to have in-depth knowledge of the development of novel nano/micro-based biomaterial scaffolds. Thus, we herein review the hierarchical structure of bone, and the potential application of nano/micro technologies to guide the design of novel biomaterial structures for bone repair and regeneration.