Although endoscopy is a standard diagnostic tool for detecting gastric cancer, challenges persist in identifying cancer and assessing tumor extent, particularly in stomachs with atrophy and intestinal metaplasia. To address this issue, we aimed to introduce a novel, compact hyperspectral imaging system with artificial intelligence (AI) that utilizes structured illumination and hyperspectral imaging to diagnose gastric cancer based on intrinsic tissue optical properties. The optical properties of three types of gastric tissue (normal, adenoma, and gastric cancer) obtained from nine patients collected via endoscopic submucosal dissection were analyzed. Our findings reveal that cancer tissue displays unique optical properties, such as low reduced scattering coefficients and distinct reflectance spectral profiles when compared to normal and adenoma tissues. However, it was challenging to diagnose gastric cancer accurately using optical properties with conventional analysis methods due to their heterogeneity. Therefore, we employed a Vision Transformer model with a supervised learning approach to accurately classify tissue types based on intrinsic optical properties. To accurately train the AI model, we devised a novel image processing method to obtain single-pixel level ground-truth labeling data by aligning pathology results and imaging data. The trained AI model successfully demonstrated its ability to diagnose gastric cancer accurately in the nine patients studied. Given its compact design and rapid imaging capabilities, the proposed optical system can be a versatile clinical tool for on-site endoscopic diagnosis and could potentially aid in complete endoscopic tumor removal.