Ventilatory instability may play an important role in the pathogenesis of obstructive sleep apnea. We hypothesized that the influence of ventilatory instability in this disorder would vary depending on the underlying collapsibility of the upper airway. To test this hypothesis, we correlated loop gain with apnea-hypopnea index during supine, nonrapid eye movement sleep in three groups of patients with obstructive sleep apnea based on pharyngeal closing pressure: negative pressure group (pharyngeal closing pressure less than -1 cm H(2)O), atmospheric pressure group (between -1 and +1 cm H(2)O), and positive pressure group (greater than +1 cm H(2)O). Loop gain was measured by sequentially increasing proportional assist ventilation until periodic breathing developed, which occurred in 24 of 25 subjects. Mean loop gain for all three groups was 0.37 +/- 0.11. A significant correlation was found between loop gain and apnea-hypopnea index in the atmospheric group only (r = 0.88, p = 0.0016). We conclude that loop gain has a substantial impact on apnea severity in certain patients with sleep apnea, particularly those with a pharyngeal closing pressure near atmospheric.