Text2KGBench: A Benchmark for Ontology-Driven Knowledge Graph Generation from Text

Simple item page
AudienciaPúblico en generales_ES
CoberturaMéxico[ 143]es_ES
Fecha de ingreso2026-10-05T16:35:00Z
Fecha de publicación2023-01-01
ResumenThe recent advances in large language models (LLM) and foundation models with emergent capabilities have been shown to improve the performance of many NLP tasks. LLMs and Knowledge Graphs (KG) can complement each other such that LLMs can be used for KG construction or completion while existing KGs can be used for different tasks such as making LLM outputs explainable or fact-checking in Neuro-Symbolic manner. In this paper, we present Text2KGBench, a benchmark to evaluate the capabilities of language models to generate KGs from natural language text guided by an ontology. Given an input ontology and a set of sentences, the task is to extract facts from the text while complying with the given ontology (concepts, relations, domain/range constraints) and being faithful to the input sentences. We provide two datasets (i) Wikidata-TekGen with 10 ontologies and 13,474 sentences and (ii) DBpedia-WebNLG with 19 ontologies and 4,860 sentences. We define seven evaluation metrics to measure fact extraction performance, ontology conformance, and hallucinations by LLMs. Furthermore, we provide results for two baseline models, Vicuna-13B and Alpaca-LoRA-13B using automatic prompt generation from test cases. The baseline results show that there is room for improvement using both Semantic Web and Natural Language Processing techniques. Resource Type: Evaluation Benchmark Source Repo: https://github.com/cenguix/Text2KGBench DOI: https://doi.org/10.5281/zenodo.7916716 License: Creative Commons Attribution (CC BY 4.0)es_ES
Doihttps://doi.org/10.1007/978-3-031-47243-5_14es_ES
ISBN978-303147242-8es_ES
URIhttps://riuat.uat.edu.mx/handle/123456789/6463
IdiomaInglés[20]es_ES
EditorialSpringer Science and Business Media Deutschland GmbHes_ES
RelaciónLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)es_ES
URL relacionadohttps://doi.org/10.1007/978-3-031-47243-5_14es_ES
DerechosAcceso abierto (Metadatos de producción científica)es_ES
Licenciahttp://purl.org/coar/access_right/c_abf2es_ES
FuenteLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Palabra claveBenchmarkes_ES
Palabra claveKnowledge Graphes_ES
Palabra claveKnowledge Graph Generationes_ES
Palabra claveLarge Language Modelses_ES
Palabra claveRelation Extractiones_ES
TítuloText2KGBench: A Benchmark for Ontology-Driven Knowledge Graph Generation from Textes_ES
TipoPonenciaes_ES
ArbitradoHa sido Arbitradoes_ES
AutorMihindukulasooriya, Nandana
AutorTiwari, Sanju
AutorEnguix, Carlos F.
AutorLata, Kusum
AutorMihindukulasooriya, Nandanaes_ES
AutorTiwari, Sanjues_ES
AutorEnguix, Carlos F.es_ES
AutorLata, Kusumes_ES
InstituciónUniversidad Autónoma de Tamaulipas
InstituciónUniversidad Autónoma de Tamaulipases_ES
Rango de páginas247-265es_ES
URL relacionadahttps://doi.org/10.1007/978-3-031-47243-5_14
Tipo de artículoIndexado
Tipo de artículoIndexadoes_ES
Volumen14266 LNCSes_ES

Files

Collections

Documentación Cientifica