An AI-based method to retrieve hematoxylin and eosin breast histology images using mammograms

Early diagnosis of breast cancer can increase survival rate. The assessment process for breast screening follows a triple assessment model: appropriate imaging, clinical assessment and biopsy. Retrieving prior cases with similar cancer symptoms could be used to circumvent incompatibilities in breast cancer grading. Abnormal mass lesions in breast are often co-located with normal tissue, which makes it difficult to describe the whole image with a single binary code. Therefore, we propose an AI-based method to describe mass lesions in semantic abstracts/codes. These codes are used in a searching based method to retrieve similar cases in the archive. This simple and effective network is used for unifying classification and retrieval in a single learning process, while enforcing similar lesion types to have similar semantic codes in a compact form. An advantage of this approach is its scalability to large-scale image retrievals.