The World Health Organization has recently declared the ongoing outbreak of COVID-19, which is caused by a novel coronavirus SARS-CoV-2, as pandemic. There is currently a lack of knowledge in the antibody response elicited from SARS-CoV-2 infection. One major immunological question is concerning the antigenic differences between SARS-CoV-2 and SARS-CoV. We address this question by using plasma from patients infected by SARS-CoV-2 or SARS-CoV, and plasma obtained from infected or immunized mice. Our results show that while cross-reactivity in antibody binding to the spike protein is common, cross-neutralization of the live viruses is rare, indicating the presence of non-neutralizing antibody response to conserved epitopes in the spike. Whether these non-neutralizing antibody responses will lead to antibody-dependent disease enhancement needs to be addressed in the future. Overall, this study not only addresses a fundamental question regarding the antigenicity differences between SARS-CoV-2 and SARS-CoV, but also has important implications in vaccine.