Purpose: Heart toxicity, such as major acute coronary events (ACE), following breast radiation therapy (RT) is of utmost concern. Thus, many studies have been investigating the effect of mean heart dose (MHD) and dose received in heart sub-structures on toxicity. Most studies focused on the dose thresholds in the heart and its sub-structures, while few studies adopted such computational methods as deep neural networks (DNN) and radiomics. This work aims to construct a feature-driven predictive model for ACE after breast RT. Methods: A recently proposed two-step predictive model that extracts a number of features from a deep auto-segmentation network and processes the selected features for prediction was adopted. This work refined the auto-segmenting network and feature processing algorithms to enhance performance in cardiac toxicity prediction. In the predictive model, the deep convolutional neural network (CNN) extracted features from 3D computed tomography (CT) images and dose distributions in three automatically segmented heart sub-structures, including the left anterior descending artery (LAD), right coronary artery (RCA), and left ventricle (LV). The optimal feature processing workflow for the extracted features was explored to enhance the prediction accuracy. The regions associated with toxicity were visualized using a class activation map (CAM)-based technique. Our proposed model was validated against a conventional DNN (convolutional and fully connected layers) and radiomics with a patient cohort of 84 cases, including 29 and 55 patient cases with and without ACE. Of the entire 84 cases, 12 randomly chosen cases (5 toxicity and 7 non-toxicity cases) were set aside for independent test, and the remaining 72 cases were applied to 4-fold stratified cross-validation. Results: Our predictive model outperformed the conventional DNN by 38% and 10% and radiomics-based predictive models by 9% and 10% in AUC for 4-fold cross-validations and independent test, respectively. The degree of enhancement was greater when incorporating dose information and heart sub-structures into feature extraction. The model whose inputs were CT, dose, and three sub-structures (LV, LAD, and RCA) reached 96% prediction accuracy on average and 0.94 area under the curve (AUC) on average in the cross-validation, and also achieved prediction accuracy of 83% and AUC of 0.83 in the independent test. On 10 correctly predicted cases out of 12 for the independent test, the activation maps implied that for cases of ACE toxicity, the higher intensity was more likely to be observed inside the LV. Conclusions: The proposed model characterized by modifications in model input with dose distributions and cardiac sub-structures, and serial processing of feature extraction and feature selection techniques can improve the predictive performance in ACE following breast RT.