Skip to content

KnowledgeBase doesn't work with Llama-Sharp on local LLM #1086

Description

@HHADI80

I have configured Llama-Sharp and I am using the llama3.1-8b-instruct-q6_k.gguf model on my cpu. I have installed the knowledgebase plugin and I have also installed and configured qdrant. The knowledge that I have installed in qdrant is of the Q&A type. When I ask a question in the chat, Llama Sharp does not use knowledge and gives the user the answer through LLM. I have put the agent configuration below.
{
"id": "121a6dab-afa9-469f-bbe6-8df111b77700",
"name": "Q&A bot",
"description": "This is a Q&A bot",
"type": "task",
"mode": "eager",
"createdDateTime": "2025-05-10T12:45:57.3500389Z",
"updatedDateTime": "2025-07-02T09:52:29.17704Z",
"llmConfig": {
"is_inherit": false,
"provider": "llama-sharp",
"model": "llama3.1-8b-instruct-q6_k.gguf",
"max_recursion_depth": 16,
"max_output_tokens": 4096
},
"isPublic": true,
"disabled": false,
"iconUrl": null,
"profiles": [],
"labels": [],
"mergeUtility": false,
"utilities": [
{
"name": "kg.knowledge-base",
"disabled": false,
"functions": [
{
"name": "util-kg-knowledge_retrieval"
}
],
"templates": [
{
"name": "util-kg-knowledge_retrieval.fn"
}
]
}
],
"mcpTools": [],
"rules": [],
"knowledgeBases": [
{
"name": "BotSharp",
"type": "question-answer",
"disabled": false,
"confidence": 0.1
}
],
"inheritAgentId": null,
"maxMessageCount": 4096,
"routingRules": []
}

And the Instruct was as follows:
"
You are a helpful Q&A assistant. When the user asks a question, search the knowledge base using the 'util-kg-knowledge_retrieval' function to retrieve relevant information, then respond clearly and helpfully based on the retrieved knowledge. If no relevant knowledge is found, politely inform the user. Always keep answers accurate and easy to understand.

Use this function for retrieving knowledge "util-kg-knowledge_retrieval"
"

I ask the experts to help me solve this problem.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions