Rosai-Dorfman disease (RDD) is an uncommon disorder, characterized by an atypical expansion of histiocytes which classically shows emperipolesis and immunoreactivity with S-100 protein. RDD affects the lymph nodes as well as extranodal sites; however, RDD of the breast is exceptionally rare. Herein, we describe the histopathologic features of 22 cases of RDD occurring in the breast, with an emphasis on the differential diagnosis. All cases were notable for an exuberant lymphocytic infiltrate with and without germinal center formation, and the majority (19/22) showed numerous plasma cells: 5 to 132/high-power field (HPF). IgG and IgG4 immunohistochemical stains were available for 13 cases; in no instance were criteria for IgG4-related sclerosing disease met, though in a single case the IgG4/IgG ratio was increased to 25%. Sclerosis was present in the majority of cases (18/22), and was frequently prominent. RDD cells showing emperipolesis were present in all cases (22/22), and ranged from rare (<1/50 HPF) to numerous (>50/50 HPF). Two of the cases in our series were initially misdiagnosed as inflammatory myofibroblastic tumor and plasma cell mastitis with granulomatous inflammation. As emperipolesis can be indistinct, the presence of stromal fibrosis and a prominent lymphoplasmacytic inflammatory infiltrate should prompt a careful search for the characteristic histiocytes, which can be aided by the use of S-100 immunohistochemistry.