Despite their label, chatbots aren’t very good at making small talk. In a recent report, Facebook’s researchers pointed to several key areas in which they need improvement. The first problem is that these A.I.s do not have any consistent personality. They don’t stick to the same set of facts about themselves throughout a conversation, which can make the experience feel unnatural.
Perhaps more frustrating is the fact that the A.I. can’t remember its own past responses or those of the person it is talking to, resulting in conversations that can easily go off the rails. Finally, when asked a question they don’t have an answer to, these bots will often make use of canned pre-programmed responses.
Many modern chatbots are trained with lines taking from movies. This, predictably, has some issues since even the best-written scripts are not natural conversations. Everything is written with the intent of informing the viewer about the film’s characters, world, or narrative. This can often result in strange or nonsensical responses.
In order to help remedy this problem, Facebook engineers have constructed their own datasets to help train the A.I. These datasets are taken from Amazon’s Mechanical Turk marketplace and consist of more than 160,000 lines of dialogue.
The interesting thing about this data is that it isn’t entirely random. The Verge reports that in an effort to create consistent personalities for their chatbots, Amazon’s team was instructed to create a short biography for their chatbots. For example, one of the chatbots is based on the following statements: “I am an artist. I have four children. I recently got a cat. I enjoy walking for exercise. I love watching Game of Thrones.”
It’s hardly an award-winning novel, but it does serve to provide a bit of structure and consistency to the chatbots’ conversations, though it does have some downsides. While these bots did score well on fluency and maintain a consistent personality, users found them less interesting than A.I. based on movie scripts.
For now, these chatbots have a long way to go before they can truly imitate human speech, but they are improving.
