Cannabis use during adolescence is associated with an increased risk of developing psychosis. According to a current hypothesis, this results from detrimental effects of early cannabis use on brain maturation during this vulnerable period. However, studies investigating the interaction between early cannabis use and brain structural alterations hitherto reported inconclusive findings. We investigated effects of age of cannabis initiation on psychosis using data from the multicentric Personalized Prognostic Tools for Early Psychosis Management (PRONIA) and the Cannabis Induced Psychosis (CIP) studies, yielding a total sample of 102 clinically-relevant cannabis users with recent onset psychosis. GM covariance underlies shared maturational processes. Therefore, we performed source-based morphometry analysis with spatial constraints on structural brain networks showing significant alterations in schizophrenia in a previous multisite study, thus testing associations of these networks with the age of cannabis initiation and with confounding factors. Earlier cannabis initiation was associated with more severe positive symptoms in our cohort. Greater gray matter volume (GMV) in the previously identified cerebellar schizophrenia-related network had a significant association with early cannabis use, independent of several possibly confounding factors. Moreover, GMV in the cerebellar network was associated with lower volume in another network previously associated with schizophrenia, comprising the insula, superior temporal, and inferior frontal gyrus. These findings are in line with previous investigations in healthy cannabis users, and suggest that early initiation of cannabis perturbs the developmental trajectory of certain structural brain networks in a manner imparting risk for psychosis later in life.