TY - JOUR
T1 - An easy-to-use evaluation framework for benchmarking entity recognition and disambiguation systems
AU - Chen, Hui
AU - Wei, Bao-gang
AU - Li, Yi-ming
AU - Liu, Yonghuai
AU - Zhu, Wen-hao
N1 - This is the author accepted manuscript. The final version is available from Springer via http://dx.doi.org/10.1631/FITEE.1500473
PY - 2017/2/18
Y1 - 2017/2/18
N2 - Entity recognition and disambiguation (ERD) is a crucial technique for knowledge base population and information extraction. In recent years, numerous papers have been published on this subject, and various ERD systems have been developed. However, there are still some confusions over the ERD field for a fair and complete comparison of these systems. Therefore, it is of emerging interest to develop a unified evaluation framework. In this paper, we present an easy-to-use evaluation framework (EUEF), which aims at facilitating the evaluation process and giving a fair comparison of ERD systems. EUEF is well designed and released to the public as an open source, and thus could be easily extended with novel ERD systems, datasets, and evaluation metrics. It is easy to discover the advantages and disadvantages of a specific ERD system and its components based on EUEF. We perform a comparison of several popular and publicly available ERD systems by using EUEF, and draw some interesting conclusions after a detailed analysis
AB - Entity recognition and disambiguation (ERD) is a crucial technique for knowledge base population and information extraction. In recent years, numerous papers have been published on this subject, and various ERD systems have been developed. However, there are still some confusions over the ERD field for a fair and complete comparison of these systems. Therefore, it is of emerging interest to develop a unified evaluation framework. In this paper, we present an easy-to-use evaluation framework (EUEF), which aims at facilitating the evaluation process and giving a fair comparison of ERD systems. EUEF is well designed and released to the public as an open source, and thus could be easily extended with novel ERD systems, datasets, and evaluation metrics. It is easy to discover the advantages and disadvantages of a specific ERD system and its components based on EUEF. We perform a comparison of several popular and publicly available ERD systems by using EUEF, and draw some interesting conclusions after a detailed analysis
KW - entity recognition and disambiguation (ERD)
KW - evaluation framework
KW - information extraction
UR - http://hdl.handle.net/2160/45055
U2 - 10.1631/FITEE.1500473
DO - 10.1631/FITEE.1500473
M3 - Article
SN - 2095-9184
VL - 18
SP - 195
EP - 205
JO - Frontiers of Information Technology & Electronic Engineering
JF - Frontiers of Information Technology & Electronic Engineering
IS - 2
ER -