We investigate the excitation and propagation of surface plasmon polaritons (SPPs) at a geometrically flat metal-dielectric interface with a parity-time (PT) symmetric modulation on the permittivity ϵ(x) of the dielectric medium. We show that two striking effects can be simultaneously achieved thanks to the nonreciprocal nature of the Bloch modes in the system. First, SPPs can be unidirectionally excited when light is normally incident on the interface. Secondly, the backscattering of SPPs into the far field is suppressed, producing a radiative-loss-free effect on the unidirectional SPPs. As a result, the lifetime and propagation distance of SPPs can be significantly improved. These results show that PT symmetry can be employed as a new approach to designing transformative nanoscale optical devices, such as low-loss plasmonic routers and isolators for efficient optical computation, communication, and information processing.