Septo-optic dysplasia (SOD) is a heterogeneous brain midline anomaly associated with ophthalmological, endocrinological, and/or neurodevelopmental symptoms. The clinical phenotype correlates with abnormal brain magnetic resonance imaging (MRI) findings. However, variations of the septum pellucidum (SP) appearance and their clinical impact have not been studied in depth. Sixty-eight patients with optic nerve hypoplasia (ONH) were investigated for the presence of associated SP anomalies and correlations between clinical findings and their MRI abnormalities established. Thirty patients had either complete (n = 22) or partial (n = 8) absence of the SP. Pituitary hormone deficiencies were present in 64% or 25% of the cases, respectively. Neurological symptoms did not occur in patients with SP remnants or unilateral ONH. Hippocampus abnormalities (43%) that have not been described before in SOD and falx abnormalities (17%) correlated significantly with neurological symptoms and developmental delay (p < 0.05 and p < 0.01, respectively). Maternal age at birth was low (21.2 years) and drug abuse during pregnancy was reported in 27% of the patients. Twelve patients with pituitary anomaly and ONH but normal SP showed similar clinical and MRI features, and were classified as SOD-like. The remaining 26 patients were not assigned to SOD. We conclude that unilateral ONH and SP remnants are associated with a milder SOD phenotype. Hippocampus abnormalities and falx abnormalities seem to constitute important features of severe clinical disease, irrespective of SP appearance. Our anamnestic data support the hypothesis of vascular disruption during embryogenesis.