GenAI Reporting - Knowledge Base Suggestions

This guide outlines the new feature that streamlines knowledge base (KB) management by automatically identifying and suggesting new content based on user interaction patterns. 

The feature minimizes the need for manual investigation by automating the creation of suggestions to be added to the KB, and an interface for managing suggestions.

Challenge

Previously, improving the effectiveness of the KB over time was a manual and time-consuming process:

1. Reports only provided high-level details without actionable insights.

2. Identifying gaps in the KB required manual investigation of each failed interaction.

3. Supervisors would need to think through what potential correct answers would look like.

New Functionality 

The new setup introduces an automated process to identify and add potential KB items based on specific criteria.

Here's how it works:

First, navigate to your KB and click on "view new suggestions". You'll need at least 10 conversations to have occurred where the chatbot could not find an answer in the KB.

Suggestions contain:

• an Issue - a summary of an unanswered question based on customer conversations.

• a Solution - a suggested answer to be added to the knowledgebase.

• a link to the related interaction (if you want to view the full transcript).

The Suggestion is created by an LLM based on the transcript. The idea is, if an agent has taken over the chat to answer the question, that answer will be used to inform the suggested KB text.

The system uses GPT-4o to suggest answers.

You can edit the suggestion as required, before pressing:

• Do nothing (will leave the suggestion there)

• Include 

• Reject

After you have gone through all of the suggestions, click "Review" which takes you to this page:

These suggestions will now be added to the KB.

At any time, you can review changes to see what has been added:

Other information

• Licence: Currently the feature is available on all subscription tiers. It is not licensed, although it does consume AI fair use tokens.

• Daily Automation: At 4:00 AM every day, the system checks for frequent failed lookups.

• Criteria for Adding Suggestions:

  • Suggestions will only show after the first 10 failed lookups.

  • Failed lookups are questions the chatbot was unable to answer based on its KB.

  • For subsequent interactions beyond the first 10, items are added after every 100 failed lookups.

Benefits

1. Time Savings:

   • Reduces the manual effort required to investigate missed KB entries.

   • Automates the process of adding frequently missed topics.

2. Improved Accuracy:

   • Ensures that the KB is continuously updated with relevant information based on real user interactions.

   • Still relies on human supervisors to vet KB additions.