We provide a microscopic view of the role of halides in controlling the anisotropic growth of gold nanorods through a combined computational and experimental study. Atomistic molecular dynamics simulations unveil that Br(-) adsorption is not only responsible for surface passivation, but also acts as the driving force for CTAB micelle adsorption and stabilization on the gold surface in a facet-dependent way. The partial replacement of Br(-) by Cl(-) decreases the difference between facets and the surfactant density. Finally, in the CTAC solution, no halides or micellar structures protect the gold surface and further gold reduction should be uniformly possible. Experimentally observed nanoparticle's growth in different CTAB/CTAC mixtures is more uniform and faster as the amount of Cl(-) increases, confirming the picture from the simulations. In addition, the surfactant layer thickness measured on nanorods exposed to CTAB and CTAC quantitatively agrees with the simulation results.