Paternity assignment and genome-wide association analyses for fertility were applied to a Thinopyrum intermedium breeding program. A lack of progeny between combinations of parents was associated with loci near self-incompatibility genes. In outcrossing species such as intermediate wheatgrass (IWG, Thinopyrum intermedium), polycrossing is often used to generate novel recombinants through each cycle of selection, but it cannot track pollen-parent pedigrees and it is unknown how self-incompatibility (SI) genes may limit the number of unique crosses obtained. This study investigated the potential of using next-generation sequencing to assign paternity and identify putative SI loci in IWG. Using a reference population of 380 individuals made from controlled crosses of 64 parents, paternity was assigned with 92% agreement using Cervus software. Using this approach, 80% of 4158 progeny (n = 3342) from a polycross of 89 parents were assigned paternity. Of the 89 pollen parents, 82 (92%) were represented with 1633 unique full-sib families representing 42% of all potential crosses. The number of progeny per successful pollen parent ranged from 1 to 123, with number of inflorescences per pollen parent significantly correlated to the number of progeny (r = 0.54, p < 0.001). Shannon's diversity index, assessing the total number and representation of families, was 7.33 compared to a theoretical maximum of 8.98. To test our hypothesis on the impact of SI genes, a genome-wide association study of the number of progeny observed from the 89 parents identified genetic effects related to non-random mating, including marker loci located near putative SI genes. Paternity testing of polycross progeny can impact future breeding gains by being incorporated in breeding programs to optimize polycross methodology, maintain genetic diversity, and reveal genetic architecture of mating patterns.