Using ChatGPT For AI Generated Product Support - Initial Experience

Thanks to the team at Crisp Chat, Lead Intuition has been experimenting since the end of December 2022 with ChatGPT connected by API to our live Chat. Lead Intuition supports the marketing and sales automation products ActiveDEMAND and FunnelFLARE in the UK. As a result, most of the chat traffic we receive is support related to these products.

While we have relatively low support traffic, we direct all support questions from our website, Twitter, Facebook and email to our Crisp Chat app.

Currently, ChatGPT is not on complete autopilot, corresponding directly with our customers. We click a button and ChatGPT composes an answer, and our chat operator can either accept and send that response or modify the bot-generated reply and click to send. Alternatively, the operator can always compose their own reply as usual.

The act of us modifying the ChatGPT-generated answer before sending is Training the bot.

Our big-picture positive observations to Date

ChatGPT handles the conversational initial greetings and sign offs perfectly. A consistent tone/voice, always polite, authoritative and no typos. It even includes the first name of the contact where it's known. While we already have canned responses set up in our Crisp Chat for this, the ChatGPT-generated answer is always slightly different and more personal to the greeting tone initially made by the customer. If their tone sounds frustrated, it will reply with "Oh, I'm sorry to hear that……." for example.

When people ask a broad, open question like, "I'm having trouble with a campaign", it diligently asks for more information, such as which campaign + specifics of the issue they are experiencing. Pretty cool!

In conversation threads, it has learnt (Our corrections/training) the correct syntax for the product we support as ActiveDEMAND, not active demand. It learns quickly!

Most answers it gives are 2 to 3 lines of text which are down to the ChatGPT language model used on our API.

We recently witnessed it decline to answer an off-topic question.

Even emails we receive into Crisp, ChatGPT will compose an email format response with a greeting, line spacing + body message and a sign off.

Here come the Buts…

If you ask, "How do I upload a CSV file of new contacts?" for example, ChatGPT will generate an excellent detailed step-by-step answer with very plausible authority and looks spot-on even when viewed by an experienced operator at first glance. Except when you read it thoroughly, it is only 90% correct. The 10% error would cause the recipient and us more problems and brings into question the operator's credibility. In short, it guessed the answer based on the users input … hallucination, I believe it's called in the AI world. You absolutely must thoroughly read the ChatGPT answer and correct it before sending it!

We often share one of our 400+ support articles to supplement our responses to most of our users' questions where a relevant article exists. ChatGPT now correctly generates the first part of the support article that of our Support Portal URL. However, the complete URL extension for the specific article is more hallucination. It's (Badly) guessing the full URL of the support article based on the vocabulary used by the customer in their question. It remains to be seen if ChatGPT will remember the corrected URL's for future conversations.

While it may decline to answer off-topic questions (Like," can you help me with my Keto diet?"), it will respond if you push it again for an answer.

Conclusions 1 Month into the Project

Like most people who have tried the online, public version of ChatGPT, we are very impressed with its abilities. Thanks to Crisp Chat, our dedicated ChatGPT bot is growing in its usefulness to us.

For just general conversational Chat, ChatGPT is good out of the box. However, for specific, complex applications like a product support Helpdesk, ChatGPT is not ready for online live use. You need to be prepared for Training Time (which is just correcting its response). If your Chat app allows you to have ChatGPT provide answers automatically and supplied directly to the customer, don't do it! You will get yourself into trouble.

Compared to the other AI bots we have looked at on the market, which need considerable time and money invested to just get them working, adding ChatGPT does not require a capital investment, no big project plan, nor a team of specialists to get it operational. Just the same people who operate your Chat daily and a modest fee each month for the API. It is very much pay-as-you-go, and you add the Bot Training time chat-by-chat.

More updates to follow and hoping the URL hallucinations for our support articles will improve with training!

ActiveDEMAND has a native integration with Crisp Chat. With this integration you get the conversations logged right on your prospect's timeline. You also get enhanced contact profiles in Crisp, so you know who you’re chatting with, integrate to FaceBook Messenger or Twitter DM's as well ... And the best part, if you’re an ActiveDEMAND user, all you need to do is add Crisp to your website. The ActiveDEMAND magic does the rest.

ActiveDEMAND is sold and supported in the UK by Lead Intuition. To find out how we can help you adopt conversational marketing tactics for your website, simply book an appointment here for an online demonstration of ActiveDEMAND and Crisp Live Chat and Chat Bots.