CHAI: Improved Chatbot AI For Task-Oriented Dialogue
Chatbot AI, CHAI, uses a combination of a pre-trained language model and offline reinforcement learning training to create more believable task-focussed dialogue between human and machine.
Created on April 19|Last edited on April 20
Comment
Verma et. al have released a paper titled, "CHAI: A CHatbot AI for Task-Oriented Dialogue with Offline Reinforcement Learning." In it they focus on the production of a more human-like chat bot designed for task-oriented dialogue.
When it comes to adaptive task-oriented dialogue problems, reinforcement learning takes the win over supervised learning for a variety of reasons. However, collecting a dataset quality enough to train a reinforcement learning model is particularily tough for dialogue training.
CHAI is a recent chatbot AI that uses a pre-trained language model as a baseline, with additional offline reinforcement learning training using goal-focussed dialogue datasets to mold the model into performing well at the given task.
The dataset used in this paper is the "CraigslistBargain" dataset, a dataset of dialogue between buyers and sellers of Craigslist ads. The goal in this case is to create a model which can pose as the seller and provide reasonable answers and negotiate prices with a buyer, played by a human.
CHAI vs other explored methods
The paper compares their CHAI model's approach against a few different models including a retrieval-based agent and a fine-tuned language model.
The different models were tasked to communicate with a buyer in the goal of producing reasonable dialogue including adaptive and descriptive answers as well as how well they could handle pricing negotiations (particularily, the amount of revenue gained was tracked and compared). The CHAI model's method of combining a language model with offline RL training proved much more effective compared to the other methods, being the most comprehensible and most successful at price negotiation out of all the models.
Here's just one example comparing how the different seller chat bots handled this inquiry (from left to right: the CHAI model, the retrieval-based model, and the fine-tuned language model):
To be fair, it's not indicated how cherry-picked this example is in the paper, and CHAI is certainly not perfectly human, but it's leagues above the other two bots in this task nonetheless. I highly recommend reading the appendix A.4 for a good laugh (quick link here: https://arxiv.org/pdf/2204.08426.pdf#page=16).
Find out more
Add a comment
Tags: ML News
Iterate on AI agents and models faster. Try Weights & Biases today.