Mammography is one of the most commonly used tools for early screening of breast cancer. Developing computer-aided diagnosis (CAD) based on mammographic images to assist doctors in making efficient and accurate diagnoses holds significant research value. Mass segmentation in mammograms is a core component of breast cancer CAD systems and an essential step in further qualitative analysis of breast cancer. However, significant challenges persist in the field of mass segmentation in whole mammograms, including model misalignment due to the small proportion of mass regions and difficulties in segmenting boundaries caused by blurred edges of mass areas. To solve these challenges, this paper proposes a local attention and detail-enhanced network (LADE-Net) for mass segmentation in whole mammograms. LADE-Net employs an asymmetric encoder-decoder architecture and introduces a lightweight local attention (LA) module aimed at early and precise localization of breast mass regions. Importantly, we design a new detail-enhanced fusion residual network (DEFRB) to refine and enhance the learning of edge features in breast masses. We evaluated the performance of LADE-Net on two publicly available datasets (INbreast, CBIS-DDSM). Compared to previous works, LADE-Net achieved superior performance.