Structured illumination microscopy (SIM) provides an enhanced spatial resolution of up to twice the conventional capacity. Recently, many approaches have attempted to combine deep learning frameworks with SIM reconstruction for better capability. Nonetheless, the inadequacy of training samples highlights the challenge of limited credibility and low generalization ability of deep learning, thus significantly constraining the application in biology. To tackle this issue, we propose an object-to-image plane degradation network (OIDN) guided by the physical process of optical imaging. Specifically, the proposed OIDN embeds the object-to-image plane degradation process into the reconstruction network to provide explicit guidance. With a set of learnable point spread function (PSF) parameters constrained by physical prior, OIDN successfully converts the conventional image-to-image data pattern mapping into the object-to-image plane degradation mapping that highly aligns with the optical processes of SIM imaging. Comprehensive experiments demonstrate that the proposed method reliably yields high-quality images across signal-to-noise ratio conditions, exhibiting superior generalization ability across diverse datasets and sample types.