Bipolar disorder and unipolar depressive disorder (UD) may be different in brain structure. In the present study, we performed voxel-based morphometry (VBM) to quantify the grey matter volumes in 23 patients with bipolar I depressive disorder (BP1) and 23 patients with UD, and 23 age-, gender-, and education-matched healthy controls (HCs) using magnetic resonance imaging. We found that compared with the HC and UD groups, the BP1 group showed reduced grey matter volumes in the right inferior frontal gyrus and middle cingulate gyrus, while the UD group showed reduced volume in the right inferior frontal gyrus compared to HCs. In addition, correlation analyses revealed that the grey matter volumes of these regions were negatively correlated with the Hamilton depression rating scores. Taken together, the results of our study suggest that decreased grey matter volume of the right inferior frontal gyrus is a common abnormality in BP1 and UD, and decreased grey matter volume in the right middle cingulate gyrus may be specific to BP1.