The impact of rare recurrent copy number variants (rCNVs) and polygenic background attributed to common variants, on the risk of psychiatric disorders is well-established in separate studies. However, it remains unclear how polygenic background modulates the effect of rCNVs. Using the population-representative iPSYCH2015 case-cohort sample (N=96,599), we investigated the association between absolute risk of psychiatric disorders and carriage of rCNVs and polygenic scores (PGS), as well as the interaction effect between the two on disease risk. Carriers of rCNVs with higher gene constraint scores had an increased absolute risk for autism, ADHD, and schizophrenia, but not depression, whereas an increase in PGS for each respective disorder was associated with higher absolute risk across all four disorders. Similarly, elevated absolute risks were observed with the increase of both PGS and gene constraints of rCNVs except in the case of depression. In contrast to some previous case-control studies, our joint analysis of rCNV groups and PGS revealed no indication of significant interactive effect between these two factors on disease risk. Also, we found no significant interactions of PGS with any of the most common individual rCNVs, except in the case of 16p13.11 duplication, which was found to attenuate the effect of ADHD-PGS on the absolute risk of ADHD. This study advances our understanding of the interplay between rare and common important genetic risk factors for major psychiatric disorders and sheds light on the importance of population-based samples in implementing precision medicine.