AI Bibliography

WIKINDX Resources

Alt, C., H"ubner, M., & Hennig, L. (2019). Fine-tuning pre-trained transformer language models to distantly supervised relation extraction. arXiv preprint arXiv:1906.08646. 
Resource type: Journal Article
BibTeX citation key: Alt2019
View all bibliographic details
Categories: Artificial Intelligence, Cognitive Science, Computer Science, Data Sciences, Decision Theory, General
Subcategories: Autonomous systems, Big data, Decision making, Deep learning, Machine intelligence, Machine learning, Machine recognition, Neural nets, Synthetic intelligence
Creators: Alt, H"ubner, Hennig
Publisher:
Collection: arXiv preprint arXiv:1906.08646
Attachments  
Abstract
Distantly supervised relation extraction is widely used to extract relational facts from text, but suffers from noisy labels. Current relation extraction methods try to alleviate the noise by multi-instance learning and by providing supporting linguistic and contextual information to more efficiently guide the relation classification. While achieving state-of-the-art results, we observed these models to be biased towards recognizing a limited set of relations with high precision, while ignoring those in the long tail. To address this gap, we utilize a pre-trained language model, the OpenAI Generative Pre-trained Transformer (GPT) [Radford et al., 2018]. The GPT and similar models have been shown to capture semantic and syntactic features, and also a notable amount of "common-sense" knowledge, which we hypothesize are important features for recognizing a more diverse set of relations. By extending the GPT to the distantly supervised setting, and fine-tuning it on the NYT10 dataset, we show that it predicts a larger set of distinct relation types with high confidence. Manual and automated evaluation of our model shows that it achieves a state-of-the-art AUC score of 0.422 on the NYT10 dataset, and performs especially well at higher recall levels.
  
WIKINDX 6.7.0 | Total resources: 1610 | Username: -- | Bibliography: WIKINDX Master Bibliography | Style: American Psychological Association (APA)