Breast cancer is one of the most common cancer types, and treatment largely depends on early detection. A person has a 13% (1 in 8 risks) of developing breast cancer at some time in their lives. Computer-aided diagnostic (CAD) systems powered by deep learning algorithms have enabled data analysis at high rates without compromising performance. Our work proposes an Enhanced EfficientNet (EEF-Net) for determining the severity of breast cancer through mammograms. EEF-Net is built on top of EfficientNet and has been fine-tuned to classify mammograms into three classes: benign, malignant, and healthy. The architecture was trained using the publicly accessible MIAS dataset, and a sophisticated image pre-processing pipeline was used to remove noise and other artifacts from the mammograms. The model achieved state-of-the-art results in the classification of breast cancer, achieving an accuracy of 97.14%, 98.67% sensitivity, 99.30% specificity, and 98.30% precision. EEF-Net will assist radiologists in mass screening patients with high precision and will minimize the radiologists’ workload.