We have the data – the challenge is knowing how to leverage it

Can you still remember how things were before we had access to digital aids like computers, tablets, and smartphones? As a matter of fact, this wasn’t even that long ago, yet life as we know it already seems virtually inconceivable without them. Today, we use these devices every day – intuitively and without being particularly tech-savvy – as if it were the most natural thing in the world.

Whether we’re reserving a table at our favorite restaurant, making a purchase, booking a trip, or simply asking for information, thanks to modern language assistants and chatbots together with advances in user experience and CUI (conversational user interface), we’ve long since grown accustomed to communicating with machines in place of people.

Possible applications for this technology run the gamut from rigid text-based solutions to more intelligent voice and speech-enabled technologies. Regardless of how fresh and revolutionary the technology we use may be, its actual implementation can still fail because of something seemingly trivial – data that is available but not (efficiently) used. Because this data forms the foundation for the answers that an application delivers.

The more comprehensive the knowledge base on which these conversation apps are based, the more powerful they are. At the same time, however, this means that the more exhaustive the data, the greater the manual effort required for maintaining and processing it all. You may well recall the hand-built and manually managed intranet systems that served as a knowledge base and frequently contained untold redundant copies of the original data. Today, the notion of maintaining a knowledge base manually is unthinkable, especially in the corporate context, given the sheer volume of structured and unstructured data – and also in terms of managing access rights to sensitive data. As a result, a growing number of companies are turning to so-called insight engines, which provide an intelligent solution for conversational applications.

Insight engines: The intelligent information hub

Insight engines consolidate all the information from the many different corporate data sources (its knowledge base) and then prepare and present the information accordingly.

If you ask the conversation app a question, the insight engine uses deep learning and transformer models together with word processing methods to derive information – for instance, the intention behind the question – and searches the existing knowledge base (company data) to find a pertinent answer, all the while complying with the access permissions. Questions about sensitive issues such as salaries, bonuses, test results, and so on can also be excluded. If someone searches for these terms, the insight engine will only return results if the appropriate permissions are in place.

Illustration: Corporate data is the knowledge base

The content and metadata of both the search queries and the documents are semantically processed using a multi-level pipeline. That way, both structured, focused, and transactional data (tables), as well as unstructured, broadly diversified, and informative content (from documentation, articles, audio, and video) can be extracted. A question like “Where can I reset my password for application XY?” will thus take the user straight to the appropriate page (transactional). The question “How do I reset my password for application XY?” also delivers specific instructions on how to reset the password (informative).

Illustration: Semantic pipeline

With so many different approaches and variations for designing conversational applications, it can be tricky for companies to find the right strategy. Whatever the case, a comprehensive and well structured knowledge base is the prerequisite for generating maximum acceptance and a positive user experience.

If you want to create the ideal knowledge base for your conversation application or your chatbot, and in addition, to provide employees with quick and simple access to the information they need, we look forward to hearing from you.

You can find out how other businesses are already using Mindbreeze successfully in the Case Studies section.