The use of computer aided diagnosis (CAD) systems, which are computer based tools for the automatic analysis of medical images such as mammogram and prostate MRI, can assist in the early detection and diagnosis of developing cancer. In the process of CAD for mammogram, the task of image processing (IP) plays a fundamental role in providing promising diagnostic results, by exploiting high-quality features extracted from the mammographic images. Normally, an IP procedure for mammographic images involves three mechanisms: region of interest (ROI) extraction, image enhancement (IE) and feature extraction (FE). However, an improper utilisation of IE may lead to an inferior composition of the features due to unexpected enhancement of any irrelevant or useless information in ROI. In order to overcome this problem, a fuzzy-rough refined IP (FRIP) framework is presented in this paper to improve the quality of mammographic image features hierarchically. Following the proposed framework, the ROI of each mammographic image is segmented and enhanced locally in the area of the block which is of the highest value of fuzzy positive region (FPR). Here, FPR implies a positive dependency relationship between the block and the decision with regard to the given feature set. The higher a block’s FPR value the more certain its underlying image category. To attain a high quality of the image enhancement procedure, the winner block will be further improved by a multi-round strategy to create a pool of IE results. As such, for a mammographic image, after embedding the candidate enhanced blocks into the original ROI, the respectively extracted features from the locally enhanced ROI are compared against each other on the basis of the value of FPR. A given image is therefore represented by a set of features which are supported by the premier FPR among all of the resulting extracted features. The quality of the extracted features by FRIP is compared against that of those directly extracted from the original images, from the globally enhanced images or from the randomly locally enhanced images in performing classification tasks. The experimental results demonstrate that the mammographic risk assessment results based on the features achieved by the proposed framework are much improved over those by the alternatives.