This study examined the serum and synovial fluid concentrations of cartilage oligomeric matrix protein (COMP) in relation to the evolution of joint cartilage damage and the requirement for surgery in 125 patients with rheumatoid arthritis (RA). We compared the erythrocyte sedimentation rate (ESR), C-reactive protein (CRP) level, and matrix metalloproteinase-3 (MMP-3) levels with COMP levels determined by specific enzyme-linked immunosorbent assay (ELISA). Patients were divided into three groups: (1) patients with least erosive disease (LES); (2) patients with more erosive disease (MES); and (3) patients with mutilating disease (MUD). In addition, synovial fluid samples were collected from patients undergoing arthroscopic synovectomy of the knee joint (ASS) and total knee arthroplasty (TKA). Serum COMP levels correlated with the ESR (P < 0.0001, r = 0.374, n = 125) and the CRP level (P = 0.0014, r = 0.281, n = 125). COMP levels did not correlate with the MMP-3 level (P = 0.182, r = 0.114, n = 125). The COMP levels of the LES group were significantly lower than those of the MES or MUD groups. Lastly, synovial fluid COMP levels in the TKA group were higher than in the ASS group. Therefore, these findings suggest that serum and synovial fluid COMP levels in patients with RA may reflect cartilage destruction and are correlated with the ESR and the CRP level, which are indicators of the acute-phase response.