What is a Well again? Why the Taxonomy’s Wilderness Years are over

For a long time, “taxonomy” wasn’t a word that got anyone excited.

It sat in the same mental drawer as filing systems and library science: a nice idea, probably necessary, but unglamorous, always down the bottom of the to-do list. In the last year, that’s changed. Just like in other parts of Information Management, there’s a dawning realisation that well organised concepts are at least as important as well organised information: AIs need both to get the job done, and done reliably. Here’s why.

It’s not really about reducing hallucinations…

… though that’s certainly helpful. There’s real research that if you ground LLM generation in a curated, well-structured ontology, hallucination rates are reduced, and factual accuracy improves.

However, “Ground it in a taxonomy” assumes the taxonomy itself is worth grounding in. And building a taxonomy that’s good enough to improve AI reliability is hard. So hard, in fact, that there’s a lot of fundamental research going on in automating taxonomy generation through – you guessed it – use of AI, and LLMs. The challenge, of course, is that you just move the problem one layer down, and then give it authority across your entire organisation. A fleet of AI agents grounded in an inaccurate taxonomy is a board-level risk, not a benefit.

Publication Volume — Ontology in AI

Publication Volume

Ontology in AI — OpenAlex, 2015–2025

“Ontology in AI” works indexed on OpenAlex.
Hover over any bar for the exact count.

CAGR, 2020 → 2025

0×

2021–25 volume vs. 2016–20

2023

inflection year — LLMs4OL lands at ISWC

How hard can building a taxonomy actually be?

The semantic web research community runs an annual benchmark for exactly this question: can large language models build usable ontologies? This is the LLMs4OL (Ontology Learning) challenge, and the second edition’s results were published late last year.

The challenge is to automate four stages of ontology construction:

Extract terms from raw text (e.g. well, field, chemical, etc.);
Arrange those terms into hierarchies (e.g. a field is in a basin is in a province; monoethylene glycol is a chemical, and also a hydrate inhibitor); and
Extract relationships from them (e.g. monoethylene glycol (also known as MEG) is used to prevent hydrate formation in gas pipelines)
Derive axioms that constrain the above (e.g. a wellbore must have a parent well; a wellbore that has not been cored cannot have core data associated with it).

The LLMs4OL challenge attempts this for some of the best-documented, most public-data-rich domains available for this kind of task, across biomedicine, materials science, chemistry, and agriculture, as well as general knowledge.

The results? For the best-performing system, a mean F1 score of 0.374 across all subtasks. 37% right; 63% not right. The LLM approach has a very long way to go; and that’s just for initial taxonomy creation. One thing organisations expect is that their mission-critical taxonomy doesn’t shift wildly from one week to the next; so being able to consistently generate, update, optimise, and refine a taxonomy is much more than an LLM solution can handle today.

The days of ‘Claude, please generate an oil and gas taxonomy for me’ are a very long way off.

But why are taxonomies suddenly popular?

In the last year, the problem has shifted. It’s no longer about dealing with hallucinations, and more about being a good line manager to one’s AI agent workforce. If the agents don’t agree on a basic vocabulary, there’s little hope they’ll be able to work together, and hand off work between themselves, in a way that is reliable and cost effective.

If your agent handling procurement doesn’t agree with your agent handling compliance checks about basic supply chain terminology, expect chaos: misrouted actions and inconsistent records across every handoff between them. Research on agent reliability now suggests that the core cause of agentic system failures often isn’t missing data at all, but the absence of an ontology working alongside the model, checked before every action the agent takes.

In 2025, Gartner projected that task-specific AI agents will be integrated into 40% of enterprise applications by the end of 2026, with agent to agent collaboration the next stage in AI development, subject to investment in shared communication and interoperability standards – a prerequisite for agent to agent communications. Organisational taxonomies suddenly shift from an optimistic notion in the IM community to being the fundamental enabler for reliable agentic delivery.

Five Key Papers

Ontologies and LLMs

Pan, Luo, Wang, Chen, Wang & Wu – IEEE TKDE, 2024

Gives the field its shared vocabulary: a three-way split between using LLMs to build and refine knowledge graphs, using knowledge graphs to ground LLMs, and running the two in tandem. That structure is genuinely useful for organising a fast-growing literature. What it doesn’t do is resolve the problems it names – hallucination, validating what a KG-building LLM produces, and keeping pace with knowledge that changes faster than any model’s training data. It provides a helpful map of the terrain, rather than a finished route through it.

https://arxiv.org/abs/2306.08302

Babaei Giglou, D’Souza & Auer – ISWC 2023

The first systematic benchmark for ontology learning with LLMs: zero-shot testing across nine model families on three sub-tasks: naming a concept’s type, discovering its place in a taxonomy, and identifying non-taxonomic relationships – across WordNet, GeoNames and UMLS. That decomposition is the real contribution; it gave the field a shared, measurable task definition, and has since become a recurring ISWC shared challenge (2024, 2025). The gap that continues to surface across every edition – that performance is highly domain-dependent – underlines the continued need for domain-specific, curated ontologies. Non-taxonomic relation extraction remains the hardest of the three tasks by a wide margin.

https://arxiv.org/abs/2307.16648

Caufield et al. – 2023

Shows that a well-specified schema plus structured prompting can turn free text directly into populated knowledge-base entries, with zero task-specific training, proven out on the relatively contained area of biomedical ontologies. It’s a useful drafting technique, but remains challenging against something the size and scale of a large industrial ontology.

https://arxiv.org/abs/2304.02711

Funk, Hosemann, Jung & Lutz – 2023

One of the first attempts to have a language model construct an ontology’s structure, and not just populate one, by iteratively querying it for subsumption relationships, targeting simple description-logic (EL-profile) ontologies. The advance is methodological: a concrete protocol for drawing hierarchy directly out of a model rather than from labelled training data. It’s explicitly proof-of-concept scale, though, a long way from the precision and consistency needed for an industrial taxonomy grounded in commonly understood terms and concepts.

https://arxiv.org/abs/2309.09898

Fabian Neuhaus – Applied Ontology, 2023

Reframes the question rather than answering it: draws on the argument that ontology development is a process of consensus creation among domain experts, not merely representation, and asks what a model that can only represent – not negotiate – can actually contribute to that process. The most conceptually useful of the five for scoping what “automation” could even mean here, precisely because it doesn’t propose a technical fix. The open question it leaves is about the field, not the model: can an automated drafting step ever be folded into a consensus-driven process without eroding the trust that process is built to produce?

https://journals.sagepub.com/doi/abs/10.3233/AO-230072

But doesn’t oil and gas already understand reliability?

Absolutely. The 2026 Princeton study, “Towards a Science of AI Agent Reliability” looks at agent reliability through a process safety lens. They note that overall reliability statistics aren’t helpful in understanding the impact of an agent’s bad decisions. Does the agent fail randomly? Or does it fail predictably on specific tasks? Can the agent distinguish between a harmless formatting error, and something more consequential, such as updating drilling data in the wrong sidetrack?

The paper builds on the Safety Integrity Level framework behind IEC 61508 and 61511 to define four dimensions of reliability that sit independent of raw capability:

Consistency (does the system behave the same way twice under identical conditions);
Robustness (does it degrade gracefully when inputs are reworded, reordered, or perturbed, rather than failing abruptly);
Predictability (can it recognize when it’s likely to be wrong and defer rather than guess); and
Safety (when it does fail, is the resulting harm bounded or unbounded).

Applying these across 15 models on real agentic tasks, the authors found that reliability gains have lagged well behind capability progress for two years running. Moreover, every frontier model provider clusters similarly on this gap, suggesting there’s an industry-wide plateau, rather than something that can be fixed by switching model provider.

There’s been plenty of recent examples of AI agents going rogue – it seems almost a badge of honour for AI creators to own up to this. Given this, introducing AI agents into an environment adjacent to safety-critical equipment or decision making requires careful thought regarding the controls that need to be in place.

So, does a taxonomy help?

We think so. Taxonomies and ontologies underpin the guard rails that help identify when an agent needs correction, prompting the agent to check its work before confidently handing something dangerously, or expensively incorrect to the next worker.

They provide the semantics for normalising inputs, and checking outputs. Structured output is all well and good, but the output needs checking to make sure it is semantically, as well as structurally correct. The same applies to inputs, whether they’re human or machine generated. Is “Tell me about Bruce’s performance today” referring to a person, a field, or a platform? If you’re after a drilling update for the A18 well, what guides the LLM to the right slot number in the right platform, with the right wellbore identifier to join all the data points needed together with sufficient reliability for you to trust the result?

The value of taxonomies in constraining LLMs is already being seen in other industries.

Grama Chethan of Siemens calls this out clearly as a “semantic training gap”, with an example that sits well in the oil and gas environment. He describes a single device, a temperature transmitter, that might be known as TT-4201 per the instrumentation standard, %IW64 as a PLC address, and ReflowZone4 in the manufacturing execution system. An LLM-based AI agent would “treat them as three independent data points, reason about each in isolation, and produce an answer that is linguistically precise and operationally wrong”. They quote a 43% hallucination rate in tool calls that reduces to 0% when the tool calls are ontology-grounded. The oil and gas industry has the exact same issues, in everything from the offshore operating environment to all the different names given to the same wellbore through its lifecycle from planning through to abandonment. Normalising all those names to the single underlying entity is the value of the ontological approach.

Why now?

In oil and gas we use very specific terms that carry very significant operational, safety, and regulatory weight. An agent that’s slightly wrong about what one of them refers to isn’t a minor bug, it’s at best a compliance exposure, and at worst an operational risk. The realization that agentic AI needs a governed backbone to work reliably is spreading fast, and the data now backs that up: both on the challenges in building that backbone from scratch, and in the risk of operating without one at all.

Today’s question isn’t whether you need a taxonomy. It’s how much longer your systems and AI programmes can run safely without one. Your Information Managers have understood the value of taxonomies for decades. If you’d like to know more, and how Flare’s decades of investment in the world’s largest oil and gas taxonomy can help your business, just give us a call.

References

Babaei Giglou, H., D’Souza, J., Mihindukulasooriya, N., & Auer, S., “LLMs4OL 2025 Overview: The 2nd Large Language Models for Ontology Learning Challenge,” Open Conference Proceedings 6 (2025), DOI: 10.52825/ocp.v6i.2913;
Lewis et al., “Retrieval-Augmented Generation” (2020), arXiv:2005.11401;
Gartner, “Gartner Predicts 40% of Enterprise Apps Will Feature Task-Specific AI Agents by 2026, Up from Less Than 5% in 2025,” press release, 26 Aug 2025 (updated 5 Sept 2025);
Rabanser, S., Kapoor, S., Kirgis, P., Liu, K., Utpala, S., & Narayanan, A., “Towards a Science of AI Agent Reliability,” ICML 2026, arXiv:2602.16666;
Chethan, G, “The Semantic Training Gap: Ontology-Grounded Tool Architectures for Industrial AI Agent Systems”, Siemens, arXiv:2605.11234.