Neurotransmission depends on the exo-endocytosis of synaptic vesicles at active zones. Synaptobrevin 2 [also known as vesicle-associated membrane protein 2 (VAMP2)], the most abundant synaptic vesicle protein and a major soluble NSF attachment protein receptor (SNARE) component, is required for fast calcium-triggered synaptic vesicle fusion. In contrast to the extensive knowledge about the mechanism of SNARE-mediated exocytosis, little is known about the endocytic sorting of synaptobrevin 2. Here we show that synaptobrevin 2 sorting involves determinants within its SNARE motif that are recognized by the ANTH domains of the endocytic adaptors AP180 and clathrin assembly lymphoid myeloid leukemia (CALM). Depletion of CALM or AP180 causes selective surface accumulation of synaptobrevin 2 but not vGLUT1 at the neuronal surface. Endocytic sorting of synaptobrevin 2 is mediated by direct interaction of the ANTH domain of the related endocytic adaptors CALM and AP180 with the N-terminal half of the SNARE motif centered around M46, as evidenced by NMR spectroscopy analysis and site-directed mutagenesis. Our data unravel a unique mechanism of SNARE motif-dependent endocytic sorting and identify the ANTH domain proteins AP180 and CALM as cargo-specific adaptors for synaptobrevin endocytosis. Defective SNARE endocytosis may also underlie the association of CALM and AP180 with neurodevelopmental and cognitive defects or neurodegenerative disorders.