In this paper, a novel approach to the self-organization of hierarchical prototype-based classifiers from data is proposed. The approach recursively partitions the data at multiple levels of granularity into shape-free clusters of different sizes, resembling Voronoi tessellation, and naturally aggregates the resulting cluster medoids into a multi-layered prototype-based structure according to their descriptive abilities. Different from conventional classification models, it is nonparametric and entirely data-driven, and the learned model can offer a high-level of transparency and interpretability thanks to the underlying prototype-based nature. The system identification process underpinning the approach is driven by the aim of separating data samples of different classes into nonoverlapping multi-granular clusters. Its associated decision-making process follows the “nearest prototype” principle and hence, the rationales of the subsequent decisions made can be explicitly explained. Experimental studies based on popular benchmark classification problems, as well as on a practical application to remote sensing image classification, demonstrate the efficacy of the proposed approach.