Researchers release Build2SPARQL benchmark for text-to-SPARQL in building KGs
Researchers introduced Build2SPARQL, a new benchmark dataset for converting natural language into SPARQL queries on building knowledge graphs. The dataset includes 6,136 executable SPARQL queries and 30,680 questions derived from 201 building KGs (180 Brick, 21 ASHRAE 223P). The queries cover six pattern families—linear chains, branching, UNION, aggregation, OPTIONAL, and…
Key points
- Build2SPARQL dataset includes 6,136 SPARQL queries and 30,680 natural-language questions for building KGs
- Queries generated via code, questions by LLMs, ensuring correctness independent of model behavior
- Three open-weight models achieved 56–65% accuracy with retrieval-augmented prompts, up from 0.2–20% zero-shot
Human validation rated 98.8% semantic fidelity and naturalness in 300 sampled questions, with 84% judged operationally plausible. Testing three open-weight language models showed zero-shot accuracy ranged from 0.2% to 20%, but retrieval-augmented prompts improved it to 56–65%. The dataset aims to advance AI-driven query generation for building automation systems.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- A Holistic Assessment of the Carbon Footprint of Noor, a Very Large Arabic Language Model · 1 src
- Korean legal study finds KLUE-BERT outperforms GPT models in sexual offense text classification · 1 src
- Researchers question human-derived bias measures for LLM evaluation · 1 src
- Researchers propose GAP-DPO for personalized LLM alignment · 1 src
- arXiv study finds PRM-Pruned Fragment Grafting shows no benefit in reasoning tasks · 1 src
Comments
via GitHub Discussions