Solar compasses are designed to accurately find true North on sunny days. However, no data on their performance are available when sunlight is partially blocked, e.g., by a cloud. We have measured, for the first time to the best of our knowledge, the performance of one of the most accurate electro-optical solar compasses (accuracy better than 0.01 deg) as a function of the solar disk obscuration during the Sun's eclipse on 20 March 2015. The measurements show that the accuracy level is mainly dependent on the asymmetry of the obscuration with respect to the main axis of the optical detection system and, to a lesser extent, on the percentage of the solar disk covered. In particular, azimuth measurement suffered a maximum deviation of 0.08 deg when 35% of the solar disk was asymmetrically obscured. The deviation was smaller when 46% of the solar disk was more symmetrically obscured. This experiment demonstrates that, even in the case of a partially obscured Sun, the electro-optical solar compass maintains an accuracy better than magnetic and electronic compasses.