An intriguing region of human visual cortex (the fusiform face area; FFA) responds selectively to faces as a general higher-order stimulus category. However, the potential role of lower-order stimulus properties in FFA remains incompletely understood. To clarify those lower-level influences, we measured FFA responses to independent variation in 4 lower-level stimulus dimensions using standardized face stimuli and functional Magnetic Resonance Imaging (fMRI). These dimensions were size, position, contrast, and rotation in depth (viewpoint). We found that FFA responses were strongly influenced by variations in each of these image dimensions; that is, FFA responses were not "invariant" to any of them. Moreover, all FFA response functions were highly correlated with V1 responses (r = 0.95-0.99). As in V1, FFA responses could be accurately modeled as a combination of responses to 1) local contrast plus 2) the cortical magnification factor. In some measurements (e.g., face size or a combinations of multiple cues), the lower-level variations dominated the range of FFA responses. Manipulation of lower-level stimulus parameters could even change the category preference of FFA from "face selective" to "object selective." Altogether, these results emphasize that a significant portion of the FFA response reflects lower-level visual responses.