We experimentally demonstrate backward emission of high-harmonics of a near-infrared laser from MgO and Si crystals in the direction of specular reflection. We show that the variation of the high-harmonic power with the angle of incidence can be predicted with nonlinear reflection coefficients derived originally for perturbative nonlinearities. A comparison of transmission and reflection geometries suggests that backward-propagating high-harmonics are an excellent reference to study nonlinear propagation of intense light in solids. Backward emission will enable phase matching of the high-harmonic beam and the integration of the functionalities of extended gas-phase high-harmonic beamlines into a single optical element. The potential to achieve phase matching paves the way to solid-state based high-harmonic sources with higher flux than the best transmission-based sources, where high-harmonics are strongly absorbed by the crystal itself.