Towards Trustworthy Llm-Driven Knowledge Management Systems: A Cross-Context Evaluation And Governance Framework For Smes And Large Enterprises

Abstract

Large Language Models (LLMs) are changing the Knowledge Management System (KMS) as organizations are able to process and query large amounts of unstructured data, such as emails, reports, and conversation records. Nonetheless, current KMS architectures do not have stringent mechanisms to assess the integration of LLM, control their implementation, and enable trust in the organizational setting. This paper fills these gaps by suggesting a comprehensive model of knowledge management (KM-LLM) with the help of LLM. The study produces (1) a multi-dimensional assessment framework evaluating performance in terms of technical, knowledge effectiveness, and organizational impact within small and medium enterprises (SMEs) and large enterprises; (2) a layered governance model (GOV-LLM-KM) combining data, model and organizational controls; and (3) a formal trust model linking system transparency, system accuracy and user control to adoption results. The methodology used is a multi-case study which incorporates the quantitative performance measures with the qualitative user assessment in the organizational settings. The findings show that, with the addition of retrieval mechanisms (which can be RAG and knowledge graphs), LLCM-based KM systems increase knowledge retrieval efficiency by 25-40 percent and decrease information search time by up to 30 percent, and factual errors by up to 37 percent. Moreover, companies also reported an increase in knowledge reuse and speed of decision-making of 2035%, and SMEs were found to have faster adoption cycles but less governance maturity. Conversely, big companies were more reliable and had better compliance rates (by up to 30 percent perceived trust scores), but with more complexity in implementation. This finding also shows that the explainability, source attribution, and governance controls more significantly impact trust than raw model accuracy, and systems with human-in-the-loop validation have much greater user acceptance. These findings underscore the importance of context-specific assessment and combined governance systems to implement efficient LLM-KM. The suggested methodology makes contributions to both the research and the practice as it offers a scalable and reliable base of next-generation knowledge management systems....