/ Let's build an open source chatbot builder

Part 1: Chatbots and Current challenges

Hi! Thank you for taking part! This is the beginning of an article series that will guide the development of an open source conversation builder from the beginning. It's not about how existing bot builders and frameworks can be used to make a bot. It's much more about building your own open source botbuilder including a visual dialog flow editor and a webchat frontend.

This series will be about everything that is important for this topic i.e. theories of conversation, issues in current botbuilders, data models or hosting. Learning about the term “graph” or “conversation” and how to store it in a database will only be some of the topics. In addition, I'll show you what I mean by the term "logical graph traversal" and how this always allows your chatbot to preserve a complex context. Later I would also like to discuss the connection of further services such as NLP or Recast.ai.

What skills you should bring

You need not have any kind of deep technical knowledge to follow my articles. The series is not just about technology. It also deals with the question of how processes can be designed user-friendly and which current challenges bots have to face. Therefore, I would be happy if you can contribute your experiences so far, even if you may have just a little understanding of nerd topics.

However, if you are generally interested in the technology stack and want to follow the development, you should be familiar with the basics of web development. Above all, you should have the ability to familiarize yourself with new topics. I will not go into all the details because I want to focus on the project as a whole. I will possibly address some things only marginally. Therefore you have to be able to explore certain areas by yourself.

Generally, I would like to use modern technologies like, NUXT or Vue.js, ArangoDB, Foxx Microservices, Cytoscape.js, JSON Web Tokens , Git and Docker. If you've already been familiar with these terms, it's even better.

What this series is not about

At first, this series is not about implementing machine learning or deep learning by yourself. I'll save that topic for future posts. It's not about building your own NLP. It's all about how one question leads to the next in a guided conversation and why this topic is anything but trivial.

However, I would like to show you how you can pair already existing NLP systems or chat channels in later posts and how your system can learn independently.

Why do I do this?

Maybe I am fundamentally wrong. But I believe that the chatbot topic in general is unfavorable in some areas. In my opinion many botbuilders are too complex and too cumbersome to maintain. From experience I know that sometimes real problems are difficult to transfer to the digital world. This often requires rethinking and discarding of old structures. Frequently differences between a digital representation and a real problem lead to cumbersome frontends and complicated processes.

1. Need of complicated programming

When constructing guided conversations for example, one often has to deal with self-programming, contextual variables and conditions to teach the bot a more complex or contextual behavior. Creating conversations is often more complex programming than simple design. I want to show you that you needn't deal with this to reach your goal.

2. It is often not easy to keep track of guided conversations

This complexity often makes it difficult to keep track of the planned conversation. It is not always easy to see which events lead to which reactions of the chatbot as the conversation grows. Also in various tests of bot platforms this seems to be a major problem. Let's try to change that.

3. Many botbuilders are black boxes

You often do not know how existing services actually work or what technology stack they use. You also do not know what happens to the data collected and where it is stored. In Addition you often have no idea how many people actually use a platform, whether it is a short hype or an insider tip or whether the service will still exist tomorrow. You often dependent on opaque and uncomfortable pricing models that limit your messages or impose a branding on you. These are all different facets of a black box which you cannot influence.

4. Lack of transparency and privacy

Most chatbots are centralized systems. Their AI runs on a separate server. This servers receive inputs and send messages back. Completely intransparent to the user, however, is how these messages come about. With today's technologies it would be possible to develop systems that run on client side.

5. Increasing encapsulation of knowledge

Many chatbots have a solid expertise. They conduct consultations or accompany through complex processes. This expertise is often packaged and encoded in botbuilding platforms, program codes, and APIs. Due to this fact it's difficult to extract this knowledge or make it accessible to other processes. Knowledge buried in this way is no longer available to all people independently.

I would like to try an experiment that will face this issues. The goal is a system in which very complex conversations are very easy to set up, very easy to store and also very easy to retrieve in a chat. This simplicity is dedicated to this series.

Here you will find more articles on the topic

Part 1: Chatbots and Current challenges
Share this