We study the influence of the generation conditions on the group delay of attosecond pulses in high-order harmonic generation in gases. The group delay relative to the fundamental field is found to decrease with increasing gas pressure in the generation cell, reflecting a temporal walk-off due to the dispersive properties of the nonlinear medium. This effect is well reproduced using an on-axis phase-matching model of high-order harmonic generation in an absorbing gas.