We present the design and implementation of a new, modular gas target suitable for high-order harmonic generation using high average power lasers. To ensure thermal stability in this high heat load environment, we implement an appropriate liquid cooling system. The system can be used in multiple-cell configurations, allowing us to control the cell length and aperture size. The cell design was optimized with heat and flow simulations for thermal characteristics, vacuum compatibility, and generation medium properties. Finally, the cell system was experimentally validated by conducting high-order harmonic generation measurements using the 100 kHz high average power HR-1 laser system at the Extreme Light Infrastructure Attosecond Light Pulse Source (ELI ALPS) facility. Such a robust, versatile, and stackable gas cell arrangement can easily be adapted to different experimental geometries in both table-top laboratory systems and user-oriented facilities, such as ELI ALPS.