High throughput neuron image processing is an important method for drug screening and quantitative neurobiological studies. The method usually includes detection of neurite structures, feature extraction, quantification, and statistical analysis. In this paper, we present a new algorithm for fast and automatic extraction of neurite structures in microscopy neuron images. The algorithm is based on novel methods for soma segmentation, seed point detection, recursive center-line detection, and 2D curve smoothing. The algorithm is fully automatic without any human interaction, and robust enough for usage on images with poor quality, such as those with low contrast or low signal-to-noise ratio. It is able to completely and accurately extract neurite segments in neuron images with highly complicated neurite structures. Robustness comes from the use of 2D smoothening techniques and the idea of center-line extraction by estimating the surrounding edges. Efficiency is achieved by processing only pixels that are close enough to the line structures, and by carefully chosen stopping conditions. These make the proposed approach suitable for demanding image processing tasks in high throughput screening of neuron-based assays. Detailed results on experimental validation of the proposed method and on its comparative performance with other proposed schemes are included.