Can LLMs reliably build enterprise models using knowledge graphs?
Applying Large Language Models in Knowledge Graph-based Enterprise Modeling: Challenges and Opportunities
January 8, 2025
https://arxiv.org/pdf/2501.03566This paper explores using Large Language Models (LLMs) to automate creating enterprise models, specifically knowledge graph-based models. It compares LLM performance with human experts in mapping domain-specific concepts (like "Electronic Court Filing") to elements within the ArchiMate enterprise modeling language.
Key points for LLM-based multi-agent systems:
- LLMs show promise for automating parts of enterprise modeling, exhibiting greater consistency than human experts in some tasks.
- However, LLMs can struggle to identify irrelevant elements, and their interpretation of relationships between concepts can differ from human understanding.
- Knowledge graphs are crucial for providing LLMs with curated, reliable knowledge and ensuring the results aren't influenced by LLM training biases.
- Combining LLM strengths (processing data, drafting models) with human expertise (semantic correctness, complex reasoning) is key for robust, reliable automated modeling. This suggests a multi-agent approach where LLMs and humans collaborate.
- Future research directions include investigating more complex modeling scenarios and developing hybrid modeling processes that leverage both LLM automation and human oversight.