[00:00:00] Nathan Wrigley: Hello there, and welcome once again to the WP Builds podcast. You have reached episode number 378 entitled, exploring AI in customer support with Aaron Edwards. It was published on Thursday, the 27th of June, 2024.
My name's Nathan Wrigley, and I'll be joined by Aaron in a few short moments. But before that a few bits of housekeeping, the first thing to say is, if you enjoy listening to WP Builds, please go and share it in whatever way you feel necessary. That could be a like in apple podcasts or whatever podcast player you're using, some kind of review. You could go and share it on your socials, Facebook, Twitter, Instagram, TikTok, whatever you like. We'd really appreciate that.
And if you want to keep up to date with everything that we do, head to WP Builds.com forward slash subscribe and pick the different bits and pieces that you would like to subscribe to.
My suggestion would be sign up to our newsletter. And then we will email you twice a week. Firstly, with our this weekend WordPress show and also the podcast, which is what you're listening to now. We'll alert you when all of that happens.
The other thing to mention is if you have a product or service in the WordPress space and would like that product or service putting in front of a WordPress specific audience, we have options for you. We can help your endeavor to promote your project or product. Head to WP Builds.com forward slash advertise. A bit like these to find companies did.
The WP Builds podcast is brought to you today by GoDaddy Pro. GoDaddy Pro the home of managed WordPress hosting that includes free domain, SSL and 24 7 support. Bundle that with The Hub by GoDaddy Pro to unlock more free benefits to manage multiple sites in one place, invoice clients and get 30% off new purchases. You can find out more by heading to go.me/wpbuilds. That's go.me/wpbuilds.
We're also sponsored by Bluehost. Bluehost, redefine your web hosting experience with Bluehost Cloud. Managed WordPress hosting that comes with lightning fast websites, 100% network uptime, and 24 7 priority support with Bluehost Cloud, the possibilities are out of this world. Experience it today at bluehost.com/cloud. That's bluehost.com/cloud.
And sincere, honest, grateful things go to GoDaddy Pro, and Bluehost for their support of the WP Builds podcast.
Okay. What have we got for you today? Well, it's AI. We can't avoid this subject, can we? It's episode number 378. And today I'm chatting with Aaron Edwards. We've had him on the podcast before talking about, Imajinn an AI image generation plugin, which he created. But today we're talking about something different.
We're talking about DocsBot AI, which is a SaaS service, which enables you to feed in your documentation documents, be that PDFs or word documents or whatever it may be. Your knowledge base or YouTube videos. It will consume all of that, and then hopefully the chats that you put on your website will represent, and answer questions with your answers in mind, with your documentation in mind. It's really interesting.
I know that not everybody is a big fan of AI, but certainly applications like this do seem to be really interesting, largely because they're relying on the things that you feed it. Not by scraping the internet in general. So I hope that you enjoy this episode and find it useful.
I am joined on the podcast today by Aaron Edwards. Hello Aaron.
[00:04:12] Aaron Edwards: Hi. Thanks for having me.
[00:04:14] Nathan Wrigley: You are super welcome. We've had Aaron on the podcast before talking about something entirely different. Today is gonna be firmly talking all about AI and particularly about a product that Aaron has launched already.
It's out there in the wild. It's being used in the wild. So there's maybe some data that he can give us a bit later about how it's going. But, Aaron, just before we get into the knots and the bolts of that, do you wanna just tell us a little bit about yourself? I guess it would be important to dwell on the, stuff that you've done with ai, given that everybody's doing AI and what, so knowing that you've been playing with it for a lengthy period of time would be good.
So over to you.
[00:04:53] Aaron Edwards: Yeah, background for me, I. served as CTO at WPMU Dev for, gosh, the last 12 years, I think, and was a developer there before that. so I've been in the WordPress world a lot, and, more recently I've been, building some of my own projects. the, kind of the first dip into AI stuff was Imagine AI is called, and that was basically focused on, AI image generation.
So it was I, made it my, the point to be like the first one there as soon as it was technically possible to do it in WordPress, and made my first, Gutenberg block that generates images, for your posts and pages and things like that. And, that was a lot of fun. That was my, first experiment there.
And that's still going, but it's not really. A huge project and, now there's a million plugins that let you create AI images and things like that. but more recently in the last year, I've been, since Chet PT came out and, this whole, This whole kind of revolution and generative AI for text, these large language models.
I really got into that and experimenting with that. And I started out by building kind of a proof of concept, which was called Chat wp. And basically I, I took all the WordPress documentation from wordpress.org and I scraped it and cleaned it up and all this stuff and, Turned it into training for a chat bot so that you could ask it any questions about WordPress and it would be able to pull from the context of the official documentation and provide answers.
So I launched that for free. And that was just like both, like a proof of concept experiment and also I just put a, wait list sign up on there, do you. Are you interested in this for your business? 'cause I thought, hey, this is like one of the first things I've seen with these large language models that could really become like a practical use case for businesses, and being with WPMU Dev so long, I started out doing technical support there for my plugins and. We have massive support team, and, just seeing how many questions from users and things like that come in, that could easily be answered by documentation. and even if you're training up like support staff, they still gotta go and find it in the documentation and then translate that into an answer to provide to the users.
that's probably a very large, 70% of requests are simply that. if users took the time to actually read the documentation. so the fact that you could let AI automate that part of the process for you and potentially replace a huge number of your support tickets, I thought that was like really useful.
So I. I launched that chat WP and the wait list form and a lot of people signed up. A lot of people were interested. So I was like, alright, I think I got a business idea that might work, so I got to building for the next month or so, trying to turn it into an actual product, software as a service product.
And we launched in, very early March, I think it was like March 5th or something, and it just took off from there. like I had. Of course, a bunch of my community, the WordPress folks, using it and trying it out. But we also went viral in Japan. Somehow some Japanese tech influencer said, this is the best chat bot for, Japanese businesses.
And, it just went nuts there. So it ended up being my whole business. Even today, I think half of my customers are from Japan. So it's been, a really interesting.
[00:08:36] Nathan Wrigley: Yeah, that's great. Especially if you, there was no intention of, penetrating the Japanese market, but it, the Japanese market just came right to you. That's
The power of
[00:08:46] Aaron Edwards: And without, without this new AI technology, it'd be impossible to do that because, I got all these support tickets and Japanese and stuff like that, and half the questions that get asked of my own support. Chatbot, or in Japanese. So it's really great having AI in the middle that can do all that translation automatically for you.
[00:09:07] Nathan Wrigley: Yeah, that's amazing. I'll just drop the URL for the, WP variant. It's called Chat wp, and you can find it at WP Docs chat. I'll just say that again. WP Docs chat. And that's obviously the software that allowed you to cut your teeth, if you like, for what we are gonna spend the rest of the time talking about.
and this a different URL, this is, docs bot.ai, so DOC. SBO t.ai, as you can imagine, it's called Docs bot ai. do you just wanna tell us what it is now? I know that it's not strictly, it's not strictly related to WordPress, but I can imagine that a lot of the people listening to this are agencies.
They've probably got, websites that they're building for clients who themselves have support requests and all of that. So there's definitely a, real solid overlap here. But, just tell us what it does and, then we'll dig into the weeds of how it all works and all of that.
[00:10:05] Aaron Edwards: Yeah. from the start, I wasn't sure exactly like the best way to position and market it. Primarily we've been focused on, a solution for customer support. So the real idea is like, what if you had chat GBT, but you could train it on your own documentation and content from your website and things like that.
And before there wasn't really any easy way to do that. I know like chat, GBT has recently released some features that allow you to do some of that stuff, like upload documents and things like that. but we've really focused on the, business side and business use case. So I. Essentially you just, you create a bot that's really simple like inner interface, and then you can train it with content from, gosh, I think we have 18 different source types now.
you could point it at the site map of your website. Your WordPress website, and it'll just crawl all those links and just automatically clean and filter all that content and ingest it. you can upload document files, PDF, markdown, all those kinds of things, to learn from. you can even, we even have a way that you could export your entire WordPress site and, upload that export file and we'll pull it in from there.
And RSS feeds, you can have it like automatically monitored. And the cool thing is we automatically update the training too. So if you're using like a site map, of your documentation site, for example, we can go in there and every week or every day, do a new crawl of the site and pull in the latest, information.
So you're like chat bot knowledge always stays up to date.
[00:11:43] Nathan Wrigley: It's like having your own little Google. It sounds
[00:11:46] Aaron Edwards: Yeah, exactly.
[00:11:47] Nathan Wrigley: yeah,
[00:11:49] Aaron Edwards: you're building your own little Google index but it's AI specific.
[00:11:52] Nathan Wrigley: And it sounds from that, that it's not just limited though in the same way that Google is to scraping the web, although that's a totally useful use case, especially for this podcast where everybody's building WordPress websites. But you, you mentioned that, I guess there's whole teams of people who have, although forms of documentation, so they might have, I don't know, a Google drive or a.
Dropbox or something equivalent full of Word documents or PDFs or things like that. And it can scrape all of those as well. Can it? You can upload those. And can it, in the same way with public facing repositories like a Google Drive,
facing, can it do the same job of repeatedly going in and spotting for itself that something's changed?
you modify one item in one of the Google Docs and can it figure that out, that's happened and so on.
[00:12:43] Aaron Edwards: Yeah, exactly. We have a bunch of different. Cloud sources and basically like for example, notion, Google Drive, Dropbox, OneDrive, box, SharePoint, Zendesk, Intercom. And we're always adding new ones. And so essentially you just do that kind of one click connect, where you grant, As permission to be able to read from it.
And then you can grant, you can do a file picker to choose which files or folders or pages in the case of notion, that you wanna give permission. And then from then on, we can go and check that as, as often as daily, for updates to keep your bot refreshed with data.
[00:13:20] Nathan Wrigley: So it's things like lists of URLs, documentation as we've described. Office, PDFs, email, whatever. Sorry. HTML. I dunno why I said email. maybe it can do email, I dunno. and then it can obviously scrape. Websites, WordPress included. you can run it off a site map and there's a whole bunch of other things.
RSS feeds you YouTube channels. That's cool. so there's just a whole bunch of things that you can point it at. Now, you dropped in. You said two things. You said we clean and filter and I guess that's part of the secret source, right? Because I could throw any old nonsense up on the website, but what I want my customers to see is the best version of it, and I want the AI to paraphrase or whatever it's doing.
What I imagine there's a lot in the words clean and filter.
[00:14:08] Aaron Edwards: Yes. That's honestly that. honestly, that is the hardest part
[00:14:13] Nathan Wrigley: that's what I thought. Yeah.
[00:14:14] Aaron Edwards: building this product. And a lot of people don't see that. that's the, dirty inside secret, like figuring out, okay, how do you clean up all this data to get just what's important? How do you preserve things like links and images?
how do you make sure, that it can be found properly? So let me, back up and talk, just how this
[00:14:37] Nathan Wrigley: Leah, let's get into this 'cause it's fascinating. Yeah.
[00:14:39] Aaron Edwards: Yeah. so like when we say train your chat bot, it's like a misnomer 'cause more, more traditionally that would mean like, basically what, OpenAI did to train GPT originally.
Like in, just in the whole Internet. Internet. but that is not as good for these kind of things. Basically, we're using the large language model, which in this case is like GPT. 3.5 or four, and we're using that for its reasoning abilities, and it's like text generation abilities. And we're combining that with facts that we pull from your documentation.
I. So when a user asks a question, then we use that question to somehow search all that documentation that was indexed and find the most relevant parts that can hopefully answer the question. Then we provide that to Chad GPT, as context. So basically it says, okay, here's all the sources, here's the question.
generate an answer for the user. only using what's found in these sources because it's very important to avoid what's called hallucination, especially in these, business use cases. 'cause if you just use chat GBT and probably almost everyone's tried this, it'll just make things up very confidently.
[00:15:58] Nathan Wrigley: guess that literally can't happen in this case. That would be the death nail of your business if it just absolutely made stuff up all the time. okay. Sorry.
[00:16:07] Aaron Edwards: Yeah, so, we do a lot of work to try to keep it as strict as possible to only answer factually. And if it can't, then, to be honest, that I'm not sure, here's how you can talk to a human. and,
[00:16:21] Nathan Wrigley: Oh, okay. That's, yeah, we'll come back to that. Okay, great.
[00:16:24] Aaron Edwards: Yeah, we'll come back to that. But, this whole process is basically called, it's called RAG or retrieval, augmented Generation.
It's called, and this is like many studies have shown that this is the way to make chatbots accurate, essentially. and of course, you're not gonna get a hundred percent, you can get very high amounts, like 80, 90% if everything's tuned and if you. prompt it correctly, then you can, those other edge cases, it can re pilot.
It doesn't know, usually, to avoid, to minimize hallucination as much as possible. so that's called retrieval, augmented degeneration. So a lot of. A lot of our work is in that parsing and cleaning up of the data and the index and figuring out the best search algorithms and things like that to, identify the sources that could best answer the question.
because these chat bots, they have a limit to how much, text you can like, prompt them with. They call that the, context window. And so imagine if you just pasted your entire, all the HTML from your entire documentation website into chat GPT, and then ask a question. you can't do that 'cause that's way too much.
And also the more context that you provide it, the more it can get confused. About like the logic to, to be able to pull out answers and things. So we have to do a lot of work on the front end to figure out, okay, what's most likely to be able to answer this question. And we use, techniques like, vector search and beddings and, These sparse keywords and all this stuff to try to identify. And we even do things, that we're about to release. It's called, multi query attention.
[00:18:09] Nathan Wrigley: Hmm.
[00:18:10] Aaron Edwards: And, basically the user asks a question. We take that question. So for example, often, not often, but sometimes a question may contain multiple like pieces or parts.
and to be able to answer that, you couldn't just like Google and look up one thing, you have to like. Break that down into three or four different searches. Then pull in the answer from your documentation for each of those different searches and then combine that. So that's, something that we do on the back end is we actually use AI to take that easier question in the context of the whole conversation and turn that into multiple queries that we can use to search their content to, to identify what it needs to know to be able to answer the question.
[00:18:56] Nathan Wrigley: it all sounds fiendishly. Difficult to have achieved, and I'm guessing it probably has kept you awake at various points, but it, if I can paraphrase what I think you've just described. So your service is sitting between the data storage blob, if you like, and the, your service is storing the. The blob as well.
But, also you've got this part in the middle. So you've got the end user asking the question, which is filtered through your, your service. Your service makes sense of that. And then in some way has to intuit which bits of the data blob to go and fetch and retrieve, and then comes back, creates the perfect answer, or at least what it thinks is best or admits it can't do it.
And then services that all back to the user. There's a lot going on there and presumably it's fairly intensive in terms of CPU as well, I imagine you've got a lot of hardware spinning around in the background there.
[00:19:54] Aaron Edwards: Yeah, we have very large, it's called a vector database. It's like the new AI type database to where everything's indexed with, embeddings from OpenAI and things like that to allow us to do semantic searches and things like that.
[00:20:10] Nathan Wrigley: And do, you have an intuition that this, the kind of service that you can provide, I'm sure that you are satisfied with it 'cause you've released it out into the wild. But do you get the intuition that I. All of the different CPUs that are coming on the market and also the latest updates to the different AI LLMs and what have you.
Are they meaning that this is gonna get better rapidly? So at some point in the near future, you'd be just crazy not to look at this? Or is, I don't know. I really know intuition as to how the industry's moving. is it going at breakneck speed or is it slowing down? Are the platforms locking services away as they realize that they can monetize it?
What's the sort of general landscape of. The rate of change of speed and how effective it is.
[00:20:52] Aaron Edwards: No, the speed is nuts. Like you have one side, which is like these AI models, GPT-4, or now you have all these other competitors coming out, some of them open source, smaller models like Mytral and it's exciting and we've tested these things and honestly, there's only For us, the most important things are, the reasoning ability. So if you provide it this context and then you ask it a question, is it gonna be able to be smart enough to figure out an answer from the context we provided it, right? and we've found that the original GPT-3 0.5, it's actually very good at that.
[00:21:34] Nathan Wrigley: Hmm.
[00:21:35] Aaron Edwards: and very affordable. if you want to get real good, real professional, then GT four is the best out there, obviously. But, and there are some open models that have approached like the GPT-3 0.5 level. mixed drill is, one example, but the other part that we have to factor in is multilingual support.
one of the things that sets like the open ai, chat g PT models apart is that they're trained on. Basically the whole internet. So they have, they're huge in comparison to some of these smaller open models, which have stayed small by limiting their training to like just English or just English and French and Spanish, just the most common type languages.
And so if you try to ask it a question Japanese, that's not gonna work, or, any of the other 95. Languages, the opening eye supports very well. so that's also been like a big requirement for us. but we're always monitoring that side to see if switching to another model or enabling support for that is, would be good for our customers,
[00:22:44] Nathan Wrigley: Yeah, you get the intuition that they're really advancing very quickly in the background. we get the incremental releases from all of these ais, don't we? And everybody's fairly astonished by how amazing the. Much better. It's got in the tiny amount of time that seems to have passed between the last release, but I guess part of your job is to keep on top of that and shift over as, as quickly as possible to the one that is the most effective.
And on, on the question of effective, I'm gonna ask you to be, hand on heart honesty. How, effective? Are these, are these chatbots, consuming that data? is there a point at which okay, you've not got enough data, you've got too much data. is there some sort of sweet spot? Is there a, is there the, a kind of query that it's still not very good at?
Or is it just blanket amazing at solving all support requests?
[00:23:37] Aaron Edwards: in practice for people that are using it for customer support, we track these stats as best we can, for the users and we have like a little. Analytics thing in the dashboard that they can view. And, what we usually see is it's somewhere around between, depending on, it's really depending on the quality of the documentation in a large part.
[00:23:56] Nathan Wrigley: Okay.
[00:23:57] Aaron Edwards: but we see what we call, like a 60 to 80% deflection rate. so that means that people that don't need to go on to create a ticket or escalate to a human, because they're able to get their question answered by the ai,
[00:24:14] Nathan Wrigley: Wow.
[00:24:15] Aaron Edwards: so to me that's huge. if you, calculate how much your support people spend, just doing that simple questions that could be answered by documentation.
honestly, for those things, the AI does a better job than most support people could.
[00:24:31] Nathan Wrigley: And this is, yeah, but this is the enterprise, right? You're trying to, you're acting as a go-between, between your documentation and then the frontline reel. Life support, if is that the, kind of business model
[00:24:48] Aaron Edwards: Yeah.
[00:24:49] Nathan Wrigley: trying to step in between to save a bit of time, possibly to save a few resources so that if it can answer it, then it awaited the races.
That person then drops off, as you said, 60 to 80% of the time. But for the remaining, 40 to 20%, they then push forward and get the human involved who then presumably has to endeavor and do that work.
[00:25:11] Aaron Edwards: Yeah, a big part of it is we want it to be very smooth, like transition between the AI and then your current support system that you're using, whether it's live chat or tickets or things like that. so we've built that kind of into like our support widget, like pretty easily. So you can, basically you can have it.
suggest, okay, do you need, if it can't answer, it can suggest you need to escalate to the human, here's how, here's the link to click. you can even have it automatically open. on my site, I use Help Scout. Which I know for ticketing, a lot of, WordPress companies use that for their, help desk software.
And so I have my chat bot that's in front of it. They talk to that first. If it can't answer or if it didn't give a good enough answer, it shows them a link right there if they wanna talk to a human. And then it automatically, they click that it closes the AI widget and it opens the Help Scout widget and can even pre-fill it with, like a summary of the question.
For the user. So, we try to make it very, and it's, that's compatible with almost any support software that provides some kind of widget or a support form on your website, whatever it is. We try to make that transition as seamless as possible. 'cause you don't want, you don't want a user ever to get mad, oh, this stupid bot.
It won't. Help me. we want, if there's even the tiniest amount of frustration, we wanna make it super easy for them to transition to a human. But in most cases, what we've found is people would, if it gives them a good answer, answers a question, then they are more happy than talking to a human.
[00:26:49] Nathan Wrigley: Interesting. Yeah, that's fascinating.
[00:26:52] Aaron Edwards: yeah, all that frustration of talking to bots and call centers and automated things that I don't think that comes down to the fact that it's automated. It comes down to the fact that they've had a poor experience.
[00:27:05] Nathan Wrigley: Okay. Yeah, because I
[00:27:06] Aaron Edwards: so that's what we're optimizing for.
[00:27:08] Nathan Wrigley: I have this intuition that people are just generally getting fed up of hearing the words. ai, but they're probably not that fed up with the results of AI should it turn out to be good. if, an image generator throws out a spectacular image, you're highly unlikely to say, oh God, AI just gave me a really great image.
That's awful. Or, AI chat bot just gave me a really rapid and perfect response. That's dreadful. can I, have the human back? It's interesting that it works that way, but, okay, What do you integrate with? You said that, you mentioned Help Scout there and ticketing and what have you, because I would imagine most companies have got some kind of CRM software sitting in the background.
Do you integrate with most of those so that, for example, if the person approaches the website, it knows their name and things like that, so it could be handed over seamlessly? Does it keep a track of what they've asked before so there's some context there as well?
[00:28:06] Aaron Edwards: our main. A thing that people use is our, support widget. So we have a widget that you can embed in your website, either as a traditional like floating widget that you know, that pops open, that they can talk to the chat bot. and then you can also embed that like directly in a page via an iframe.
And we also have a very powerful API. So some people they'll just code it directly into their systems. just using our API directly. so if they have some kind of, ticketing thing already, they just tie it in there to automatically answer things. for example, I know a number of them that have it, connected to their Help Scout or ticketing software, and it, writes a draft response to tickets automatically.
So a customer ticket comes in and then it writes a draft response, and then a support staff can view that draft response and click send or maybe edit it before they send it, so that's, another way of doing it. And we also support Zap here
[00:29:11] Nathan Wrigley: Oh, nice. Yeah.
[00:29:12] Aaron Edwards: Pley Connect and a lot of things like that, which allow you to, really tie it into almost any other software.
[00:29:17] Nathan Wrigley: Okay. Yeah. And do you have mechanisms in the backend for, let's say that I've used, deployed your service, but I just wanna keep a managerial check on whether or not the things that it's doing are actually in line with what I want. maybe that 60 to 80% drop off rate is entire frustration as opposed to satisfaction.
is there a way that I can inspect what it's doing and. I guess report that to you or somehow interfere with the way it surfaces those results and how it perceives what the correct answer would be.
[00:29:53] Aaron Edwards: Yeah, we see that as a very important for these professional use cases, to have that feedback and to have a constant, iteration of improvement. Using that feedback and the tools that we provide, unlike if you like build like a custom GPT with Chad GPT or something like that, you have no access to user logs or things like that to improve it or to know what people are asking of it.
we have, a logging functional functionality built in. So when users ask questions, then we store that in your logs and you can see what the user question was. You can see what the answer was. And more importantly, you can see what. sources it used to pull the answer from, which can really help you debug.
And then we also have ways of feedback. So the user, they can flag a response as being inaccurate or a problem. And that shows up in, in your logs. Also, if they use the built in functionality to like escalate to human, then we track that in your logs so that way, you can get stats and see, okay, here's the questions that users complained about, or.
The answers that users complain about or that required going up to support. And you can also rate things, responses yourself within the logs. So you can thumb up, thumb down them and then do filters and you can export that as a CSV. so you can get like ideas and, all that kind of stuff. And we also have stats charts, which show you over time how things improve your, that kinds of things.
And
one of my favorite
[00:31:26] Nathan Wrigley: would, the monitoring of that be able, can I feed back into the way that the, that it works? So in other words, if I spot that, okay. Typically we've got this kind of query and it's, it always seems to be servicing. Just this slightly the wrong article. I want it to give that one instead of this one, even though they're both more or less on, on, on, message.
Can you, amend the way that it behaves, in other words.
[00:31:54] Aaron Edwards: Yeah, we have something we call it like fine tuning, and basically when you're in the question logs, if you see one, let's say you filtered to ones that were rated poorly or required human input, then you can filter to that. And then for the ones that were problematic, you can just click, revise, answer, and then that.
That pops up a little form where you can actually like write a custom answer for that kind of question, and that, that gets added to your sources as like a q and a source type, which is just like a question answer pair. And because that question much more closely matches like. that user question or a similar one to it, that will pull up as, one of your highest sources in the future, when they ask that question.
So it's a way to iteratively fine tune your bot's responses to improve it over time.
[00:32:45] Nathan Wrigley: Okay. Oh, that's nice to know. And, how does the pricing model work from, your business perspective? are we bringing our own a open a. Open ai, API key, or are you selling us credits? And if you're not selling us credits, you must have a different pricing structure. like a monthly subscription or something like that.
and how does that work? I guess it's maybe volume based or something like that.
[00:33:14] Aaron Edwards: yeah, we just tried to set our plans and we have like hobby power, pro business, and then enterprise above that. most, users who are using it, like for customer support. Are using the pro version, and, basically you have like limits to the number of bots you can create the number of, like source pages that you can train it from.
That's probably one of the biggest limits, the more content you have. and then we also have like how many questions a month that you can handle. And of course there's other things like removing like the logo, stuff like that you see on every support software. but usually like our pro plan is our most popular plan by far.
And that's, it's 99 a month and 83 a month if you pay annually.
[00:34:06] Nathan Wrigley: And then are, you binding an OpenAI key? So the actual interaction with OpenAI, that's, on our account, right? We, lodge a credit card with OpenAI and the credits are accrued over there, so you can keep monitoring that separately.
[00:34:22] Aaron Edwards: Yeah. Like a lot of our customers, they. And, we've been divided on that, whether like we make like credits or you pay for credits, or if we just let you use your own OpenAI account, which is what we do now, so that you only pay for what you actually use there. And, more importantly, you own your data a little bit more.
[00:34:44] Nathan Wrigley: Okay. Yeah, that's a good point. Yeah.
[00:34:46] Aaron Edwards: yeah. because it's under your own opening I account and, things like that.
[00:34:51] Nathan Wrigley: So there's a lot going on in the background. When I ask open AI something I, it's usually something banal and boring that takes moments. That gives me a three sentence answer. There's a, as you've described, there's a lot more going on in the background here. Do you have any intuition? obviously it's a silly question because each, thing will be different, but give us a rough estimate as to the cost.
By us sense, of a typical query so that we could work out for ourselves, what a thousand would look like or what 10,000 interactions with it would look like. again, sorry. I know that there's no science probably to get that perfect, but roughly speaking.
[00:35:32] Aaron Edwards: Yeah, so a lot of it will depend on like, the matching context we provide and then how long the conversation is. 'cause as a conversation stretches out, then you to keep the context of understanding. How the conversation is going. You have to like, provide a lot of that back to the model each
[00:35:52] Nathan Wrigley: Oh, interesting. Okay. Yeah. All right.
[00:35:56] Aaron Edwards: on average though, like we've calculated, like we track all these stats and things like that, it's about 0 cents per question for GPT-3 0.5. so that's, so what is that 3 cents? 3
[00:36:15] Nathan Wrigley: A th a thousand.
[00:36:16] Aaron Edwards: do public math.
[00:36:17] Nathan Wrigley: no, I was just trying to do that as well. would it be a hundred for 3 cents or something like that? Anyway, it doesn't matter. It doesn't matter how bad our mass is. It's a tiny amount.
[00:36:27] Aaron Edwards: yeah. $3 for a thousand
[00:36:29] Nathan Wrigley: and I I'm intrigued by the business model there as well, because it's nice from a pers from my perspective, maybe not your perspective, the credit thing, sometimes I'm a little bit.
Nonplus by that because I just wonder how much profit is being made from the credits, if So not having that as a business model and you keep your pricing based upon all of the data that you just described, it just makes it clearer for me. And also the, data retention that's nice as well.
'cause who knows what stuff is going backwards and forwards. I'm. Let's hope it's clean for a start. But it could be email addresses, it could be anything. they might be disclosing all sorts of information over there. And so there is that question about, GDPR and how your service as the middleware if you like, between OpenAI and the front end.
What's your posture on that? do you have a public statement about, the, data integrity and all of that?
[00:37:28] Aaron Edwards: yeah. Obviously in today's day and age, that's one of the biggest questions or. Potential customers ask, are you GDPR compatible? Like, how do you do privacy? All those kind of things. so I can try to walk through that. It's very complicated. But we do, we did architect everything as best we could to, maintain privacy, as much as we can.
so first of all, by default we don't collect any user data other than the question that they, Ask the bot. So even the IP addresses, we, hash those, so they're not stored in any way that we could reverse them. so they're, everyone's treated anonymous anonymously by default. so it's really depends on what the user submits in the question.
Obviously if they submit something private, then that will. Go to OpenAI and we'll go to their chat logs, which of course they can delete if they need to or, whatever. OpenAI, a lot of people ask the question, do you use like our data to train new models and things like that? OpenAI is very strict about when you're using their API, that they don't store that information, for what they say is no more than 30 days.
And that's only. For the purposes, if they need to go back and check, like compliance, if someone was using it, in a way against their terms of service, and they needed to review that. that's why they store it for that short amount of time. And then they delete it so they don't use it to train future models.
Your customer's data is not gonna show up like in GPT five or anything like that. and we, make that same commitment to, people as well. We use internally, we use, our own kind of data sets that, we've pulled together for testing. So one of the tricky things of building a product like this is if you make an improvement to the code or what you think is an improvement, how do you know it actually helps when there's no, there's no like right or wrong answer, like to know if your questions were more accurate or not.
So we've actually built some automated pipelines that are really. Advance that you use data sets that we've collected. and basically we just run this entire data set of a hundred questions through the app to see, okay, is it giving more accurate answers now, more comprehensive, better. And you actually have to use AI to rate that, which is interesting since it's not.
[00:40:06] Nathan Wrigley: That's
[00:40:07] Aaron Edwards: it's a whole different
[00:40:08] Nathan Wrigley: healed thyself.
Yeah,
[00:40:09] Aaron Edwards: yeah. Unit testing and things like that, so you actually, you have the same set of questions and answers, and then you compare the two and you, the scores that GPT-4 generates saying, okay, this is a more accurate answer. it's really crazy how that
[00:40:25] Nathan Wrigley: It won't be long. Aaron, before you are completely out of the loop here and the AI will, will run itself. You just have to off you go. It wouldn't, that'd be nice. how easy is this to, implement from the perspective of a, let's say a WordPress website developer, maybe a freelancer, some of them very technical, we've got real understanding of the code and all of that.
Some of them are just acting as, the agent and they would be. Positioning your product as, okay, this is a credible alternative to, support. How technical do you need to be to deploy this on behalf of a client? Do you need to have lots of technical expertise or is it fairly
[00:41:07] Aaron Edwards: Yeah, no, we try to make it super simple. just a few clicks, you create your bot, you name it, and then you. Point it at, say the URL of your documentation site. And then from there, takes a few minutes or whatever to crawl the site, index it, and you can start chatting with it in our dashboard, like to test it out.
And then you can copy the embed code and stick it in WordPress or wherever. and you have a chat widge at just like that. it's very simple, like just a few minutes to give it a try.
And we do have a free plan that limits like how much data you can train it on and things like that, just so you can give it a try without needing a credit card or anything.
[00:41:47] Nathan Wrigley: Oh, that's perfect. The, I, guess that's the target audience for this podcast is WordPress developers. And so I'm guessing that, them suggesting it to their clients is probably gonna be the, most likely outcome. In terms of that, do you, offer, I dunno, is there like an affiliate scheme or something?
Something that might be a, quid pro quo? If they were to suggest the software and get it deployed on their client's sites, do you offer anything to, make that deal a bit sweeter?
[00:42:19] Aaron Edwards: we do have a affiliate program. We give 25% of all revenue for the first year.
[00:42:24] Nathan Wrigley: Wow. Okay.
[00:42:25] Aaron Edwards: yeah. So that's pretty generous. It's recurring. So you, get someone to sign up and even the business plan that's, what is that, 1200, if they do a whole year and then you get 25% of that. So it's, not bad.
[00:42:40] Nathan Wrigley: Yeah. Okay. And that's linked to on the, website somewhere. We can
[00:42:43] Aaron Edwards: Yeah. On the footer. Yeah. you'll see it right
[00:42:46] Nathan Wrigley: Yeah, that's great. And obviously, if you were to deploy something like this, you'd wanna know that the business that you are relying on to do your support is gonna be around for a little while.
And so you may refuse to answer this question, it's entirely up to you, or you may wish to answer it. How's it going from a business point of view? I know that you've. Dabbled with all sorts of things in the AI space, and it's been a real passion of yours. How's this one landed? Has this been, a success?
is it taken off? Do you have confidence that you'll be around in a couple of years time?
[00:43:15] Aaron Edwards: Yeah, it's, it, really took off and we're excited. We, we got, part of it is lucky and they say you make your own luck, like being in the right place at the right time and being ready to, To, ride that wave when it comes, And so we are, really lucky in that timing and launching, right at the beginning of that AI hype.
And so that gave us our initial real big boost to growth, to the point where it's paying for my full-time salary now, and even a few, staff and contractors that are helping out with marketing and development. Yeah, and we're, very profitable and we have, even with that, it's, been like a 75% profit margin.
[00:43:55] Nathan Wrigley: Woo.
Hey,
[00:43:58] Aaron Edwards: it's good. It's,
[00:43:59] Nathan Wrigley: Yeah. Life is good. Yeah. The ai,
[00:44:01] Aaron Edwards: a blessing, but now is the time that the hype is, has definitely, died off some. so our growth rates are not what they were, but, now we're just at that state of steadily growing, pushing new features. Doing more traditional marketing, those kind of things to more like a normal business.
but it's exciting that it's not just a hype thing. Like the people who are customers now are actually using it
[00:44:28] Nathan Wrigley: That's the thing,
[00:44:29] Aaron Edwards: value to them.
[00:44:30] Nathan Wrigley: how much nonsense silliness did you see over that? I, this isn't actually a question. it's more rhetorical. How much nonsense silliness did you see over the last couple of years to do with ai? so much I. Interesting stuff, but purely interesting. It was just, 1,001 different ways to do things that you probably never really needed to do.
But it was fascinating because AI could suddenly do this thing, whereas this is a real thing. This is a real thing, solving a real problem that businesses actually have. And so yeah, this is one of the few actual, apart from ai, the open AI platform itself, which obviously is probably immensely profitable, it's, one of the few things that I can point to and say, yeah.
Great. It's got legs, it's, solving a real problem. So Bravo is all I can say. I wish I'd done it.
[00:45:18] Aaron Edwards: yeah. we're excited and I plan to stick with this and, keep it going. So it's, it was always kinda my dream to have my own business. And I launched, gosh, six products in the year before that, and this was the one that. It had legs,
[00:45:36] Nathan Wrigley: Yeah, it, that's a funny thing. I've heard that if you do seven at some point. That the, seventh, for some reason is the one. So
[00:45:43] Aaron Edwards: is the one
[00:45:44] Nathan Wrigley: got in early, you got in one early. I mentioned the URL at the beginning, but I'll mention it again. If you're interested in looking at the WP Chat, functionality, which is obviously how, Aaron cut his teeth for this technology.
That's called chat wp. You can Google it, but you can just go to WP docs.chat. However, the product that we've been talking about today is called. Docs bot, and that's at docs bot.ai. You can go and find out all of the different bits and pieces over on the website. So now that we've dropped the website, URL, do you hang out on social anywhere or do you have a preferred place to be contacted?
LinkedIn or whatever? We'll just drop that before we end.
[00:46:25] Aaron Edwards: yeah, I am on, Twitter, pretty active, so it's ugly robot dev, all one word, ugly robot dev. 'cause my parent business is called ugly robot. So yeah, that's me.
[00:46:39] Nathan Wrigley: Okay,
[00:46:40] Aaron Edwards: Yeah. Do you mind if I share another interesting use
[00:46:43] Nathan Wrigley: Oh yeah, of course. Yeah, we did actually mention before we hit record that there was gonna be a conversation about that, and I completely forgot. So yeah, go for it.
[00:46:50] Aaron Edwards: Yeah, so, part of my thesis with this is not just customer support. I think that, if anyone's tried like some of the chat based, interfaces for plugins or different things like that to where basically you describe, okay, I want to create a form that does this, in WordPress, or I want to create a chat bot that does this, and you do it with normal language and it turns that into.
Like instructions for the computer to actually do things for you. And I think that's the future of UX or user experience, ui and in a lot of products, all of the people are gonna expect that. I find I'm expecting that. And anything I use, I pull up Google Sheets. I'm like, why can't I just tell it what to do?
I have to figure out these formulas. I'm getting spoiled by AI and I think that's gonna be an important part of every business. So I'll just give you an example, what we did at Doc Spot, like we are dealing with, retention and churn. So for a software as a service business, it's a very common thing.
You gotta do a lot of stuff to help prevent churn. customers go to cancel. and you wanna, ideally you want to figure out why they want to cancel and make sure, address their objections. if there's some reason, maybe it's too expensive, maybe they ran into a bug. Maybe there's a feature that they wanted that you don't have.
Maybe there's a feature that they thought we didn't have, but we do have, So I was building like a traditional cancel wizard where you like click it, then it asks like a little survey before you cancel. That kind of things. And then I was like, why can't. We use our AI to do this for us.
And so we use Docs spot to build a AI retention agent. so essentially it works kinda like the normal process. You click, okay, I want to cancel, and it says, gives you a list of reasons. Okay, why? but then you get passed off to this chat bot that's especially trained on our documentation and everything.
And also. We've also given it like instructions in the custom prompt, like how we want it to respond to these common like situations. And it's amazing how it works because it's able to take all the context about the customer. So the name of the chat bot they created, how much they used it, the kinds of things it was trained on.
plus. like what they're paying, all that kind of contextual information. We provide that as part of the question in the backend, and then it provides a custom like, response to the user that like addresses their objections that, suggests like next steps for how they could fix it. for example, they say, oh, we're, canceling because we're switching to this competitor.
I have a. A page on my site that's an alternative comparison page to this competitor. So my chat bot is trained to know all the differences between us and that competitor. So they say they're switching this competitor. The chat bot is able to use that knowledge. Say, Hey, do you know that's fine that you're switching to there, but do you know that they don't support this, they don't support this, that we're more affordable in this way?
It's able to automatically understand from that context and provide a custom response to the user. that's been really fun to look at the chat logs for that and see like the, responses that it gives. And we've really seen it make a pretty big impact in churn. Like we've seen a 30, 30% drop in churn, since we.
Launch that, which is huge when you add up that, like churn is like, exponential. In a subscription business, they're paying you every month. if you drop your churn in half, that's like a hundred percent increase in revenue or more, it's amazing.
[00:50:41] Nathan Wrigley: That is amazing. What a really neat idea. So you are scratching your own itch. And deploying. Deploying that feels like it could be a product all by itself, not just embedded in Dock
[00:50:53] Aaron Edwards: I know that's the challenge. We, figure out how to market all these use cases and things like that. We also use it
[00:51:01] Nathan Wrigley: ways to buy different products online, aren't they?
So integrating with all of those. Yeah, that's really interesting. it's a big pain point, isn't it? And I can imagine it in the product workflow. Being really helpful, in a, in an e-commerce setup, for example,
[00:51:18] Aaron Edwards: Yeah.
[00:51:19] Nathan Wrigley: purchasing something online, asking intelligent questions to try to get you through the journey.
some things are straightforward, right? You go to Amazon, you know what you want, so you just buy it. But if you're curious and you're doing product search, I would imagine that the likes of Amazon are gonna be deploying things like this to, drill down exactly what you want really, quickly.
[00:51:39] Aaron Edwards: yeah, we're about to add. Agent functionality, which our definition of that meaning that you can have your bot, perform actions on behalf of a user. So for example, you could connect it to the WooCommerce rest API. And if a user asked a question about a order, you would be able to list, collect their order number and tell 'em status about that or, any kind of things like that, or maybe search or suggest products live, not just from its training data.
[00:52:09] Nathan Wrigley: Oh, I see.
[00:52:10] Aaron Edwards: of like our cancellation wizard actually do the cancellation for them after it makes sure it collects good reasons,
[00:52:16] Nathan Wrigley: Yeah, so it's acting on dynamic data, not just on static data, which you've supplied it in the blob. It's it's reacting to things that you've done in the past. Yeah, that's really interesting. Boy.
[00:52:28] Aaron Edwards: Yeah.
[00:52:29] Nathan Wrigley: I think you're onto something, Erin. I think this AI thing's got legs.
[00:52:32] Aaron Edwards: Yes. I think so too.
[00:52:34] Nathan Wrigley: Yeah, I think we, the year 2024 is the year of ai.
we'll probably knock it on the head there if that's all right. That's what we say in the uk when we want to end things, we'll knock it on the head. But Erin, really fascinating chatting to you all about the, the project that you're working on and some of the things that you're obviously, trying to get down the pipe as well.
That's brilliant. Thanks for chatting to me today
[00:52:54] Aaron Edwards: Thank you. Thank you for having me.
[00:52:56] Nathan Wrigley: Well, I hope that you enjoyed that. Fascinating chatting to Aaron today. All about DocsBot AI. Hopefully you'll have found some of that useful. And obviously if you have a use for that, if you've got a product or service, which requires some kind of chat bot on your website, why not reach out to Aaron and see what you can make of that together.
The WP Builds podcast is brought to you today by GoDaddy Pro. GoDaddy Pro the home of managed WordPress hosting that includes free domain, SSL and 24 7 support. Bundle that with The Hub by GoDaddy Pro to unlock more free benefits to manage multiple sites in one place, invoice clients and get 30% off new purchases. You can find out more by heading to go.me/wpbuilds. That's go.me/wpbuilds.
We're also sponsored by Bluehost. Bluehost, redefine your web hosting experience with Bluehost Cloud. Managed WordPress hosting that comes with lightning fast websites, 100% network uptime, and 24 7 priority support with Bluehost Cloud, the possibilities are out of this world. Experience it today at bluehost.com/cloud. That's bluehost.com/cloud.
And sincere, honest, grateful things go to GoDaddy Pro, and Bluehost for their support of the WP Builds podcast.
Okay, that's all I've got for you this week. Just to say that we'll be back on Monday for this week and WordPress show. It's wonderful having so many people joining us in the comments. It's always at 2:00 PM UK time, and it's always at wpbuilds.com forward slash live.
And we'll be back next week with an interview episode, quite possibly a no script show episode with David Waumsley, but we'll be there next week. Until then I'm going to fade in some cheesy music, and say, stay safe, have a good week. Bye-bye for now.
