Vibrational dynamics of the T1u CO stretching mode of tungsten hexacarbonyl is explored when the molecule is embedded in a nitrogen matrix at low temperature. Experiments combined infrared (IR) absorption spectroscopy and IR stimulated photon echoes at the femtosecond time scale. W(CO)6 is found to be trapped in two main families of sites differing by their symmetry (called hereafter Oh and D2h sites). In Oh sites, the vibrational coherence is strongly temperature dependent, exhibiting a coupling with librational phonons of the nitrogen lattice. Perturbation in D2h sites results in the splitting of the T1u band in three components. Each component is inhomogeneously broadened, with dephasing times in the tens of picoseconds, and is weakly coupled to the lattice phonons. Experiments in solid krypton are performed to compare the effect of atomic and diatomic host lattices. Dephasing time in Kr does not depend on temperature and remains in the hundreds of picoseconds, highlighting the molecular origin of the dephasing process in N2. Additionally, nonlinear signals show oscillations due to quantum beats and polarization interferences between different frequency components of the induced third order polarization, giving information, in particular, on the overtone vibrational transition.