Episode Summary
Ian sits down with Netfor CTO David Cady to launch the company's new AI Readiness Assessment. David breaks down the seven categories that actually determine whether AI will work in an organization (knowledge, data, process, intelligence, integrations, team, and governance) and explains why knowledge and data matter more than the rest combined. The two get into why most AI initiatives fail in year one, why free self-assessments are biased and expensive consulting engagements are built to sell you something, and why the most valuable thing an outside assessment can sometimes tell you is "you're not ready yet." A practical, no-hype look at what it actually takes to know before you spend.
Episode Summary
Ian Medley: Good morning, good afternoon, wherever you are. Listening from David. Great to have you back on. Third time's a charm. We are here to talk about the launch of our newest AI service. And that is our AI readiness assessment. Correct. So before we dive into that, want to just catch up with you. We haven't talked in a month or two, I think so. Wanted touch base with you, see how everything's moving in the BI and dev department and hear about your new hobby.
David Cady: My new hobby. Okay. So in the BI department, we actually have a pretty exciting new development in the BI department. For those that have heard of it, we use something called Amazon QuickSight.
Ian Medley: Yes.
David Cady: And actually, technically, I think it's just called QWIK now because they change names quite often.
Ian Medley: That's pretty quick for QuickSight to go to QWIK.
David Cady: Yeah. So anyway, I had to do an update to a dashboard, and I realized that QuickSight has a feature that you can download a JSON file of a dashboard for backup purposes.
Ian Medley: Okay.
David Cady: And then you can upload a JSON for a dashboard for deployment purposes. So you want to deploy into a new environment, you don't have to rebuild your entire dashboard. You can download it and redeploy it. I realized that in between downloading it and uploading it, I could have something like Claude make changes to it. So basically now, when we need to create a new dashboard, I don't do any of the work. I talk to Claude. I explained to him what I need.
Ian Medley: Him.
David Cady: Him. It. Sorry.
Ian Medley: Or personalizing.
David Cady: Yeah, quite a bit. So I talked to Claude and I have it build the JSON file from scratch. I built a project in Claude that has a couple of files in it that shows it how to build certain things. So, for instance, if you want a bar chart, that translates into a piece of code that goes into the JSON file and how you want it formatted, and the whole nine yards. So I have built several dashboards now where I never actually touched the dashboard at all. It was have Claude build it. I would upload it, I would check it to make sure it was working and it was what I wanted. And then we would do the next thing. And then we would do the next thing. Usually takes, you know, 20, 30 iterations.
Ian Medley: But that's a lot of iterations.
David Cady: It's also a lot faster. So went from when we first moved to QWIK, or QuickSight, whatever you call it, it would take us about six weeks to knock out a dashboard.
Ian Medley: One dashboard.
David Cady: Yeah. And I can do it in a day.
Ian Medley: That's insane.
David Cady: I can do it in one workday.
Ian Medley: Now, was it all code back when it would take that long?
David Cady: No, it was UI. So you had to drag this and then set this filter and manually create this calculated field. And there was a lot of manual effort involved in that. And I can just bypass all that and have Claude just do it.
Ian Medley: So you give it a template.
David Cady: Yeah. When you build a project in Claude, there's a section where you can upload documents or files as context. And so every time you're working within that project, and then you have a conversation with Claude, it will reference those documents before it answers you.
Ian Medley: Sweet.
David Cady: So I took one of the dashboards that we had previously built that I thought was our gold standard dashboard, downloaded the JSON file of it, and gave it to Claude and said, here's how things should be formatted, here's what they should look like, here's the color scheme to use, etc. And then from there it was just working with him. For instance, they just had a new feature come out for sparklines. I don't know if you know what sparklines are.
Ian Medley: I don't know what a sparkline is in a chart.
David Cady: A sparkline is like a line graph, but very small and kind of just centered on one specific thing. So if you have the number of visits to a particular store, you could create a little tiny sparkline. Imagine you have a number that says this is 100.
David Cady: And then you would have a sparkline next to it that shows it going up over time or something like that. Anyway, QWIK just now released it, not just now, but maybe a month ago, released the ability to have sparklines on things that you couldn't do before. And so it took Claude and me a couple of back and forths to figure out exactly what syntax AWS wanted us to use in that JSON file. So now it's just a matter of keeping up to date on those things. Once we figured it out, I added the context to the project so that all future iterations, if I want a sparkline, it already knows how to do it.
Ian Medley: Oh yeah.
David Cady: So the more I use it, the better it will get over time. And I've even got it trained to ask specific questions. What is the point of this dashboard? Who's going to be looking at it? How often is the data refreshed? There's a lot of things that go into a dashboard that typically you would have a data analyst do.
Ian Medley: Yeah.
David Cady: And I'm essentially training Claude to be our data analyst.
Ian Medley: Nice.
David Cady: And it's been very successful. It's cut down the time to produce dashboards by a huge amount.
Ian Medley: And every single one of our clients gets a dashboard.
David Cady: Everyone.
Ian Medley: Everyone.
David Cady: Well, we have some field services clients that don't necessarily get dashboards because they have their own internal stuff that they do and they don't care about our dashboards. But if you're a service desk client, you definitely get one. Everybody gets the same dashboard. We actually have five that are standard. Once you sign up with us and you start having data flow in, it will automatically populate those dashboards, and then it's just a matter of getting you access to them.
Ian Medley: Sweet.
David Cady: Yeah, dashboards. QWIK's a great tool. It's not our first dashboarding tool. We've had other dashboarding tools in the past, and we actually just switched dashboarding into QWIK a year and a half ago. Prior to that we were using a company called BrightGauge. I'm really pleased with QuickSight. I think it's one of Amazon's better products. It wasn't originally designed to be a dashboarding tool, it was originally designed to be an analytics tool, and then they added dashboarding features over time, and now they're working toward parity with something like a Microsoft Power BI or Tableau. They're not there yet, but they're getting there and they're doing a lot of interesting things.
Ian Medley: I've never met anybody more passionate about dashboards. Listen, I know everything about dashboards now. We talked about dashboards for seven minutes right there.
David Cady: I think it's so funny. I just had a conversation earlier with somebody else about this. My first job in the tech industry was in data, and I've worked on the IT side where it's like deploy Microsoft Office, and I despise that kind of stuff. I don't know how to fix laptops, I don't want to know how to fix laptops. But if you ask me anything about data, data storage, data management, or data analytics, the whole data science field is where my passion really is.
Ian Medley: Awesome.
David Cady: Yeah.
Ian Medley: Well, since your passion is on the data side, tell me a little bit about the AI readiness assessment and how data is related to that. You're coming in and giving people a consulting service to tell them if their structure and processes can handle an AI tool being strapped to the front end of whatever they're doing. How does data relate to that? I'm sure it's overwhelming, the amount of data you get in one single assessment. But tell me about what the AI readiness assessment is.
David Cady: Well, this idea came from a lot of our clients. When we first start our integrations with them, we realize exactly the state of what we're dealing with. Their knowledge is fragmented. There are multiple versions of the same thing. Maybe we're supposed to be integrating with some kind of back end database, and then we have to deal with whatever state their database is in. We also have to deal with all of their processes. There are a lot of things that go into whether an AI product is going to be successful. And we've gotten to the point in this journey that we can kind of tell in advance what kind of work it's going to take to get one of our clients up and running on any given AI product.
Ian Medley: Is there a specific AI product that people are trying to get into?
David Cady: Right now we're focused on voice products. This would be something along the lines of an interactive voice assistant.
Ian Medley: A virtual assistant, which is also something that we do.
David Cady: Yes, absolutely it is. Actually our primary thing that we do... we started thinking about this every time we integrate a client. We always do, for some reason the word is escaping me, but where you go back and review how you did over the course of a project, there's a sprint word for it that for some reason is escaping me. But essentially you're live now and we want to know internally, did we do a good job onboarding you? Were there mistakes? Were there things we could do better?
Ian Medley: So a post-launch review kind of assessment.
David Cady: Yeah, yeah. So we were doing this for one of our clients, and we realized that earlier on we should have probably made it a little more clear what kind of effort it was going to take to get this to where they wanted it to be. And that led into this idea of these assessments. Now there are currently, on the market, tons of assessments you can do. Most of them, especially the free ones, are self-assessments. Self-assessments are good if your only goal is to raise awareness within your organization. So, for instance, if I'm some kind of leader in the company and I want to deploy AI, but nobody in the company is thinking about it, and I want them to start thinking about what it's going to take to actually deploy AI, you can do a self-assessment and that will get the conversation going. Now, a self-assessment though is only as good as...
Ian Medley: It's very biased.
David Cady: Yeah, it is.
Ian Medley: I'm great at this. Oh yeah, I could be really good at this.
David Cady: Our processes are great, our knowledge is great. So that's one type of assessment. Another kind of assessment you can get in the market right now is a paid assessment from a consulting firm. Those tend to, A, be six figures, and B, tend to be designed to sell you a thing at the end of that consulting engagement.
Ian Medley: Because they're consultants.
David Cady: Correct.
Ian Medley: They're going to set you up to assess you in a way that kind of feeds them a little bit at the end when they refer you into business.
David Cady: And they're going to give you a 200-page document that no one's ever going to fully read or understand.
Ian Medley: And how long does that process take for one of those?
David Cady: It could take months, it could take weeks. It depends on the size and exactly what the end goal is. But they typically don't tend to be a very fast process.
Ian Medley: No.
David Cady: Because, to be frank, the longer it takes, the more the consulting hours build up.
Ian Medley: So how long does it take for us to do an AI readiness assessment?
David Cady: So our idea was different. We saw that somebody needs to come in, step in, and be kind of the adult in the room. That's kind of the thought process behind this.
Ian Medley: The unbiased adult.
David Cady: Correct. So what we do is, I don't have to learn every single aspect of your entire company to be able to tell you that there are some things you could be doing right now so that in the future, whenever you are ready for AI, it makes the thing easier. And thinking about that, we realized that we could actually break it down into some specific categories. So, for instance, the very first category is always going to be your knowledge base. Everything lives or dies by your knowledge base, because the AI is going to be referencing that knowledge base tons of times per day, and even small errors will compound over time. So the state of your knowledge base, and what I mean by that is, is it clear who's responsible for updating it?
David Cady: Is it actually getting updated when a knowledge article is no longer accurate? Is there somebody in the company designated to change what is in that knowledge article? We handle this at Netfor by having an entire knowledge management division. They have knowledge writers and they're fantastic. They've been doing it for thirty-some-odd years, they've got the process nailed. But knowledge is always going to be the very first one, and then it's a matter of your data schema. This doesn't always mean a database, but typically an easy way of thinking about it is basically your database. If we have to reference your database and there's no indexes built, or there's no clear schema built, or, in some cases, I've seen somebody still using SQL 2012...
Ian Medley: What's that?
David Cady: Which is a very old version of SQL. There are like four or five version updates, and every update, especially for something like SQL, is very important because they introduce completely new concepts that never existed before. So if you're running a SQL 2012 instance, I can't use some of the commands I would normally use if you're on a newer version. So the state of your data is very important, understanding where it is and how it sits. Then you have to think about the processes your company uses. So if somebody comes to you and says, "how do I do X," and your answer is, "oh, Bob in accounting knows how to do that," you are not ready for AI. Let's just put it that way. That's a really easy way to understand the concept of processes.
Ian Medley: And some processes may be broken by employees going rogue and using their own tools, without their bosses' or their bosses' bosses' knowledge.
David Cady: Yeah, yeah. Traditionally, before AI, we would call this shadow IT. And shadow IT has been around since the dawn of time. You're never going to completely get rid of it. Nowadays it's shadow AI. If I'm an analyst or an engineer or something and my company has not formally introduced any AI tools for me to use, there are tons of free tools out there that are excellent at these things. Claude, ChatGPT, Gemini, goes on and on. I'm going to start using those tools, but now I'm doing it without the company's knowledge, and sometimes that doesn't always work out great, especially if I'm an engineer or a software developer. The chance of me accidentally putting something into Claude that it really shouldn't have access to is very high.
David Cady: And we've also seen plenty of stories in the news lately where Claude has done things that were completely unintentional. So if I'm a software engineer working for your company and you have given me no AI tools, I could theoretically give Claude access to my workspace. That's not something I would recommend, but I could theoretically do it. And now, when Claude gets access to my workspace, it can execute commands that I never asked it to execute. We've seen that happen. So providing a way for your employees to safely access AI within nice, tight guardrails is becoming more and more important to try to eliminate as much of that shadow AI as you possibly can.
Ian Medley: But it'll never fully be gone.
David Cady: No, I don't think so. And the funny thing is, my boss uses ChatGPT.
Ian Medley: Yeah.
David Cady: And has for a while now. ChatGPT is great, there's nothing wrong with it. I prefer Claude, I like Claude's models better, I think its reasoning is better, etc. So I was trying to talk to him about switching over to Claude, and he said it's a lot like switching banks at this point. He has a lot of context built into his ChatGPT conversations, it has a lot of history it can reference when responding to him, just like he has a lot of auto bills on his bank account. Switching from one bank to another bank is a giant pain. Switching from one AI to another AI can be the same.
Ian Medley: I get it. I made the switch to Claude about a year ago, and I think it thinks a little more strategically and gives me a bit more in-depth data when I want it. If I want to use GPT for something like writing, creative work, or something quick or personal, I'll always ask GPT.
David Cady: Yeah, absolutely. That's kind of where I am nowadays too. Speaking of my other hobby...
Ian Medley: Yes, I forgot about the hobby.
David Cady: I picked up miniature painting, like Warhammer. I started with Dungeons and Dragons minis, actually, and then moved into Warhammer because Warhammer is much more expensive, and I wanted to get good before I started paying the big-boy dollars.
Ian Medley: Is Warhammer a game?
David Cady: Yeah, Warhammer 40,000. It's not just a game, it's books, a video game, movies, Henry Cavill is doing an Amazon live-action series and animated series. Warhammer is a big thing, and I've always liked the lore. It's also a tabletop game. I've never played the tabletop game and don't necessarily want to, but to play the game you have to have these little miniatures, and they come unassembled. You have to glue them together and then paint them. There's a whole subculture of painting them to the point where they look extremely realistic, and you can build little dioramas to put them in a scene. It's a whole big thing. I needed a new hobby, and I've always been interested in the Warhammer lore, movies, and video games, so I thought, why not, I'll pick it up.
David Cady: How many have you painted so far?
Ian Medley: I'm on my sixth Warhammer mini. I don't want to geek out too much, but there are a lot of different factions and alien races, and there's one race that's essentially space elves, and they're the ones I like the most because they have the most interesting background lore, in my opinion. So I went out and bought a bunch of their miniatures, and I'm going to paint them.
David Cady: How long does each one take?
Ian Medley: It depends on the total number of hours. I'm not very good and I just started, so total hours, I'd say six to eight.
David Cady: Dang. How big are they?
Ian Medley: 25 millimeters, 28 millimeters, 32 millimeters. I think the biggest one is maybe about that big, but they have really small, intricate details built into the models, so when you're painting them you want to go slow.
David Cady: Have you ever heard of paint by numbers?
Ian Medley: Yeah, absolutely.
David Cady: Have you ever done one?
Ian Medley: What's funny is my daughter bought me one. You can take a normal picture and send it to a company, and they'll turn it into a paint-by-numbers kit. My daughter sent me one a couple of years ago, a picture of me and my wife.
David Cady: Yeah.
Ian Medley: Yeah, it's a lot of fun.
David Cady: We did one too, and there were 36 different colors, like nine different shades of green. As soon as I'd finish the six, I'd move on to the next color and then see a ton of different sixes I'd totally missed. It took us probably a year to finish, there were so many small details.
Ian Medley: Oh yeah, 100%. And the funny thing is, for years my wife would have nine different shades of red lipstick or whatever, and I never understood why she wanted nine different shades of red. But now that I'm painting, I need nine different shades of red to achieve the effect. So I'm starting to understand the color part of it.
David Cady: The different hues, different shades of the same color. Blue is blue to me, still just that color.
Ian Medley: Okay, you can cut this if you want, I don't care. When you close your eyes, do you see a picture of things in your head?
David Cady: No.
Ian Medley: Okay.
David Cady: I see dark.
Ian Medley: Like, even if I tell you to picture a blue elephant, you can't conjure an image of a blue elephant?
David Cady: No. So people can do that? I feel like that's a magic trick.
Ian Medley: Apparently it's called...
David Cady: You can do that?
Ian Medley: I cannot. It's called aphantasia. Normal people are supposed to be able to do that. The majority of people, when they close their eyes, can conjure an actual image in their head. I'm not able to do that at all. If I get a figurine I want to paint, I have to have a picture of what it's supposed to look like.
David Cady: Yeah.
Ian Medley: I can't imagine, in my head, "I want to change it and make it this color" or "that color." I'm not able to.
David Cady: I'm not able to imagine what it's going to look like with a green hat versus a red hat.
Ian Medley: I can't. I have to paint it in order to see it.
David Cady: I don't know anybody who's been like that.
Ian Medley: My wife does.
David Cady: Is she just saying that though?
Ian Medley: No, she's amazing. Her very first mini was so much better than anything I've ever done. It was embarrassing. She's good. In six months she'll be a pro at this.
David Cady: Awesome.
Ian Medley: Yeah.
David Cady: Next Rembrandt.
Ian Medley: Apparently people can hear music in their head when they think of a song, or conjure a smell they're familiar with. I can't do any of those things.
David Cady: I'm zero for two on all these magic tricks.
Ian Medley: Yeah, they can. Well, they have the name, and we don't. It's called aphantasia when you can't do it, and it's just called being normal when you can.
David Cady: I'm going to look into this and we're going to figure out where we lie. We were on processes, we just got done with processes. So the fourth element of this AI readiness assessment would be intelligence. In this instance, a good way of thinking about this is, if a client ever comes to you, regardless of what you sell or what process or service you offer, if a client ever comes to you and says, "how many X of Y did I do last month," and you can't immediately answer that question, this is about that. If you think back to how important data is, this intelligence is about understanding that data and deciphering it.
Ian Medley: And being able to explain it to people, an easier way.
David Cady: A really good way of thinking about this is VoIP systems. We use internally at Netfor a VoIP system called NICE, N-I-C-E, or NICE inContact. In the past we've used 5,9, and we're looking at maybe going with AWS Contact. There are a lot of different VoIP services out there, and they all handle phone calls. You would think the data is very similar when you go from one VoIP to another, and that is absolutely 100% not correct. When we moved from 5,9 to NICE, it took us months to figure out what NICE was telling us, because they might name something slightly different, or on the back end, if they're giving us a calculated field, I need to know what calculation they use to derive that number.
David Cady: Yeah.
Ian Medley: Because it may not be the same as what we traditionally use internally. So think about it...
David Cady: It's like cells in Excel.
Ian Medley: So think about, if I say "average hold time," how long did our agents place a caller on hold, on average? There are two completely legitimate ways of calculating that that derive wildly different answers. One is you take the total number of hold minutes and divide it by the total number of minutes of all phone calls.
David Cady: Okay. Not the total number of calls received.
Ian Medley: Or, that's fine too. You take the total number of hold minutes and divide it by the total number of calls received, and that gives you a perfectly accurate, true-to-the-name average hold time. But another way is to take the total number of hold minutes and divide it only by the number of calls that were placed on hold. Also a completely legitimate way of calculating it, but it comes up with a wildly different number.
David Cady: Yeah.
Ian Medley: So it depends. There's a lot of ambiguity when you start looking at data, and you need to be very specific when dealing with AI, because it does not necessarily deal with ambiguity very well. It might decide to do it one way this time and a completely different way the next time, and you don't want that happening. So intelligence is about understanding your data, understanding what it is and how it works, not just at a surface level. You can't just tell me, "oh, that's the average hold time." If you can't explain the calculation behind it, you don't have the level of intelligence you need to deploy AI.
David Cady: Boom. Amen. Yes.
Ian Medley: The next one's integrations. Integrations are big because, when it comes right down to it, almost invariably it means you're going to be dealing with some kind of API. APIs are notoriously bad for some companies and just okay for others. It's very rare that APIs are well built and well documented, and documentation is generally the worst aspect of it. APIs tend to be as good as the person who built them. There are very few industry-wide standards. We claim to have standards, but then nobody follows them, and we've got a hundred different standards at this point.
Ian Medley: And even when you hear somebody talk about MCP, or multi-context protocol, that's really just a fancy way of saying an API. Everything drives back to an API. So integrations is about understanding that ahead of time, what you don't want is to be 30 or 40 percent into deploying an AI project and then discover you need to integrate into some other product, like your email system or ticketing system, and that's when you discover there's no documentation for the API. It's a black box you just bang at until you figure it out, and that can take a long time.
Ian Medley: So thinking about that ahead of time, understanding what systems you need to integrate into, and starting to build that documentation ahead of time, is going to make your AI deployment way more successful.
David Cady: Yeah, and way easier for when you come in and can say, thank you for the documentation, you just shaved thirty minutes to an hour off our time together, and now I have a clearer picture and so do you. Everybody should have that kind of thing at the ready, especially if they own that process. With APIs, when I connected my Outlook to my Claude a couple of weeks ago, it seemed really simple, I just clicked a couple of buttons.
Ian Medley: Well, that's because some engineer at Claude has already gone through all of the hard process for you, and it's boiled down to a few clicks.
David Cady: Thank you.
Ian Medley: That's essentially what we're trying to do. If we build an IVA for you and we need to create a ticket and put the summary of the conversation within it, we need to know what those fields are, what the different API endpoints are. We need all of that documented, because otherwise it's just trial and error, and a good portion of AI deployment failures are because the deployment is taking so much longer than budgeted, whether that's capacity, money, or time. This is a perfect example of what you can do ahead of time to narrow that deployment window.
Ian Medley: If you walk into an integration with me and hand me really clean documentation of your API, that's going to save me a massive amount of time and effort. And we can be more accurate too. Our philosophy is, if we can be deterministic, we want to be, because that way we can guarantee it happens the same way every time. So, going back to the example, if we need to open a ticket in your ticketing system through our IVA, we don't want the LLM trying to do that on its own. It might get it right, we can give it the documentation for your API and it can write and execute code, and it probably will get it right, but not all the time.
Ian Medley: What's better is I write the code and tell the LLM to execute my code, and now I know it's going to happen the same way every time.
David Cady: It's got to be perfect code though.
Ian Medley: Yes. And then once you get past integrations, the next thing to think about is your team, your people. My suggestion, generally speaking, is a lot like data governance. When data governance became a really big thing in the 90s, you almost always had to have somebody somewhere in your company championing it. Most of the time it wasn't an entire department, some companies have data governance departments, but most of the time it's just a couple of people championing the concept to raise awareness. That's usually the first thing you have to do when trying something like this.
David Cady: Herding cats. Yes.
Ian Medley: AI is the same way. Having somebody on your team who is the AI champion is generally my very first recommendation.
David Cady: That person's probably going to be the one spearheading this initiative and asking for the assessment too.
Ian Medley: Yeah, 100%. And what that person needs to understand is the benefits of what we're trying to do. It's hard to say exactly what the benefits are because it depends on what you're using this for, but as an example, you want to grow your company but can't afford to hire new people. You have a staff of ten people answering phone calls, and they're already missing 20% of the calls. Instead of hiring more people, you can deploy an IVA to take all of the easy phone calls away from the ten people you already have, which frees them up to handle the harder issues, so they can handle more volume.
David Cady: The human-escalated issues that require a human touch, something the IVA, the intelligent virtual agent...
Ian Medley: Yes, or assistant, either way.
David Cady: Or assistant, agent, assistant. And we sell those by the agent. We actively sell an IVA to clients, correct?
Ian Medley: Yeah, absolutely. That's actually the number one thing we're doing right now in the AI space, IVAs. It's such an obvious use case for LLMs. Even back when ChatGPT first came out, I set up a little demo using one of our clients' knowledge bases. This was before it was even called RAG, but all I did was copy and paste a couple of knowledge articles into ChatGPT and told it to use that information as context to answer any questions that followed. I asked it a bunch of questions and it correctly answered all of them. So the concept has been around a while, but it's maturing at a rate I don't think anybody really expected. And there are a ton of benefits to using it. I go back to me building the QuickSight dashboards, I can take a six-week project and get it done in a day.
David Cady: Amazing.
Ian Medley: And that is exactly the kind of efficiency we're looking to drive with AI.
David Cady: Yeah, and those tools that people are coming to us to assess if they're ready for, you want to do it right the first time. I have a stat here, I think it's 80% of AI initiatives fail in the first year, or fail to deliver any positive ROI, mostly because of failures that trace back to bad data, no defined objectives, and change management breakdowns. Those are the three big areas.
Ian Medley: And really, when you think about those three, change management breakdowns generally means something in your environment changed but your knowledge base wasn't updated.
David Cady: Knowledge base. It's never going to change in the sense that people can come and go, people leave companies all the time, the way processes are structured moves around, but the knowledge could stay the same. So absolutely.
Ian Medley: And I do want to go back to something else you said, about getting it right the first time. You have to be careful with statements like that, because what I'd tell everybody is there's going to be a lot of iterations. It would be nearly impossible for me to deploy something in a perfect state, because as soon as it hits production, somebody somewhere is going to do something I wasn't anticipating. That's just the way it goes. I think of it more like hiring a new employee: I do my best to train that employee before he starts doing his job, but once he starts, he's going to learn more from the real-world work than he ever did in training.
Ian Medley: AI is the exact same way. If you build an IVA and expect it to be perfect out of the box, you're going to be severely disappointed. The more testing we do, the better it's going to get. There's a lengthy testing process, but as soon as we push it out, especially if random people in the world are calling it, somebody somewhere is going to do something you weren't anticipating, and you're going to get a response you may not want. So tweaking it over time, those first 90 days, I'd expect quite a few iterations, quite a few tweaks, even down to the point where sometimes its voice isn't quite the culture you want.
Ian Medley: You start thinking about where we are in the market today, and it's really no longer about AI voices, it's about AI personas. It's not just the actual tone of the voice, it's also the pacing.
David Cady: It is personality.
Ian Medley: Yeah, they all have personalities, and you can build any personality you want. So figuring out your culture and finding the personality that matches it might take a couple of tries. Some things are obvious, we're not going to give it an Australian accent if you're based out of Tennessee, it doesn't make sense. But over time we're going to understand your processes better, understand the kinds of questions you get, and be able to tweak it and make it better and better.
David Cady: All because you built that awesome dashboard that surfaces these things.
Ian Medley: Yeah, we absolutely do. The way we approach dashboards is we figure out a way to display your outcomes. What that means is different for every client, unfortunately, which makes my job hard, but understanding that, for this client, their main concern is booking new appointments, I'm going to build a funnel that shows calls coming in, and at every stage, here's the number of callers you're losing. Sometimes that number is zero. It's going to collect their name, then their email address, then ask a question, do this, do that, and at every step some drop off.
David Cady: The funnel going down until the action we want to happen is executed.
Ian Medley: So if I'm booking appointments for doctors and I see that all the drop-off is at the stage where they're selecting the location, or the doctor, or the date and time, what we can derive for you is that you'd get more bookings if the doctor were available at these other times, or if you had another doctor. Sometimes we can show you the value of the missed appointments, and when that value rises high enough that it makes sense to hire a whole new doctor so you stop missing them. Those are the kinds of insights we can derive for you, beyond just things like average handle time and service level.
David Cady: Of course, we also do that crowdsourced information piece that your front desk staff isn't trained to do. Nobody's trying to collect all these different data points when people say those things.
Ian Medley: Yeah, and if I'm just having conversations all day and not taking a massive amount of notes, I'm going to forget that, say, twelve people called today and all wanted July 9th. That's information that gets lost to the ether. But since we record all of these, when we open a ticket we do a summary of the conversation and keep the conversation itself, so we can always trace back things like hallucinations or intents. We have this data available, and I'm a big data guy, so it's fun for me to walk people through and show them, hey, if you did this and that, it would mean this and that.
David Cady: Yes. Awesome.
Ian Medley: And we always back our statements with data.
David Cady: Yes, you always have to.
Ian Medley: Yes.
David Cady: And the last part of the assessment...
Ian Medley: Governance. Once I deploy this AI product for you, it's not a black box you walk away from forever.
David Cady: The champion has to own it.
Ian Medley: Yes. It's something that takes ongoing attention, just like any software you deploy in your environment, it's going to have updates, changes, new features, new needs. So you need to understand, walking into it, that somebody on your side is going to have to own this product.
David Cady: Yeah. And how long does this whole thing take? Give us the skinny.
Ian Medley: Typically three to four hours.
David Cady: Okay.
Ian Medley: It's not a huge engagement. It's not built to dive in and give you advice on how to run your company. We are strictly looking at the state of affairs: if I wanted to deploy AI within your organization today, where would it fail, why would it fail, and what can we do now so that when you do deploy AI in the future, it's successful.
David Cady: And what you mentioned, a lot of those things are provided after the assessment, when you give them their... it's not a scorecard, but you give them their assessment back.
Ian Medley: I mean, if you call it a scorecard I wouldn't be offended, because in the end I'm going to take each of these categories and outline them for you.
David Cady: Are some weighted more heavily than others?
Ian Medley: Oh, 100%. Knowledge and data are always going to be the two most important things. Everything else in your organization could be perfect, but if your knowledge is failing, you're not AI ready. It's as simple as that. We've learned that again and again. If you don't have good knowledge, you're not going to have a good time deploying AI.
David Cady: Yeah.
Ian Medley: You might as well just call up ChatGPT and have it do whatever, it's going to be just as frustrating. And the assessment you're delivered typically has a plan of action: here are the different categories, here's how you rate in them, here are your weaknesses and strengths, and if you want to deploy AI, here are the things you should do before you start. And, like I said earlier about being the adult in the room, a lot of the conversations I have are, "you're not ready for AI."
David Cady: Yeah.
Ian Medley: It's just not going to work, it's going to fail, don't spend the money. Instead, spend the money on these things, get them done, and then build your AI.
David Cady: Yeah, sure, that's not what people want to hear, but it's what they need to hear.
Ian Medley: Yeah, and a lot of times it's exactly what they need to hear. Having somebody like me come in and say "you're not ready for AI," to be perfectly honest, can take a lot of the pressure off leaders who are being pushed to deploy AI.
David Cady: Yeah, because they can go back to their boss and say, here's why we shouldn't, here's a professional assessment.
Ian Medley: Yeah, absolutely. And, you know, I'm also the CTO, so I'm very aware of the pressures we're under and what we have to do to drive the bottom line. I get it. But what we've seen, and all the statistics show, is that if you just dive in because your boss is telling you to, it's probably going to fail and you're going to waste a lot of money doing it.
David Cady: Yeah. That's a lot of information you've shared with us on the whole initiative, so I'm glad this has launched. It's something new we're doing, so I'm excited to see how it goes.
Ian Medley: Me too, me too. I've spent most of my career talking to people, somehow or another.
David Cady: That's why you're really good to have on here.
Ian Medley: Yeah, and it's fun to talk to people, learn their businesses, and learn how we can help them. Sometimes we can't help them, and I don't mind having that conversation. It doesn't bother me. I'm not here to criticize, I'm not here to judge, I'm here to state facts: here's where you are, here's where you need to be, and here's the gap.
David Cady: Is AI consulting a big thing right now?
Ian Medley: Oh, yes.
David Cady: Okay.
Ian Medley: And frankly, it's not big enough. If there were more of us out there doing consulting on this, those failure numbers would be much smaller. 80% of AI deployments failing means most of those people simply weren't ready for AI yet. Sometimes we also see that they were probably ready and just deployed it in a really poor way.
David Cady: Yeah.
Ian Medley: And that's also something we can help them with.
David Cady: Awesome. Well, I think this is going to close it out for us, man.
Ian Medley: It's a lot.
David Cady: Episode four. It is a lot.
Ian Medley: We're never going to hit 46 minutes.
David Cady: That's insane. I thought we were going to get it done in 30, but I'm glad we discussed all this. So thanks for coming on again.
Ian Medley: Yeah, absolutely. It's a blast, I love being here.
David Cady: I'm glad you do. Well, thank you very much again.
Full Episodes Available on YouTube and our Podcasts page