The State of Generative AI

Scott King:

Hey thanks everyone for joining this episode of the Union Podcast. We’re gonna talk about McKinsey’s latest report, The State of AI in 2023. This is the generative AI’s breakout year, right? This is when everybody at least had a couple of months to see it. People have actually started to use this and McKinsey was nice enough to survey 1600 people and do a really nice report. We will link it in the in the show notes so you can take a look. Chris, just first off we both reviewed the report. We’ve got some notes here. We’re going to talk about four or five different points. What’s what’s your general takeaway from what McKinsey told us?

Chris Kraus:

So actually, if you like skip to the middle of the report, they talked about that in 2022, so the survey was looking at like some of the questions, what did you do in 2022? So it’s not like, what have you done this year? It’s like, what did you do last year that you can measure? And that people are actually taking actions there. And the neat thing was, the people who are actually doing stuff, they’re literally doing stuff. Like we’ve talked about, AI must do something. You need to solve a problem, augment business, be more successful. They’re not using it for what people are afraid of. It’s like, I’m gonna lose my job to AI. I mean, there’s American Airlines is picketing now. They’re worried about AI taking over their jobs. And it’s like, these people aren’t finding that that’s where the value is. It’s actually augmenting people, augmenting processes. So I think that’s what’s really exciting. to accomplish things and augment their business and make more money and be more successful.

Scott King:

Yeah, I found the same thing. It was really interesting how they actually broke it down. I think there was a sub-segment, I can almost say that, respondents that said what they’re doing with it or kind of the ROI that they’re realizing. But about half of the people said that they call them AI high performers in the report, I think. They said that they’re actually innovating, they’re earning more money for their companies using AI versus the other side of the fence, the other half that respond, like 90% of the people that answered this question said, well, we’re just looking at saving some money. So it was a huge disparity, although it’s a small group, but I think what we find, especially with our customers, they’re actually doing something more. with the same workforce and the same people versus looking at cost reductions.

Scott King:

Thank you everyone for joining this episode of the Union Podcast. Today, we’re discussing McKinsey’s latest report, The State of AI in 2023. This is the breakout year for generative AI. People have started using it, and McKinsey has surveyed 1600 individuals, producing a comprehensive report. The report, linked in the show notes, will be our focus today. Chris, after reviewing the report and compiling some notes, let’s discuss your general takeaway.

Chris Kraus:

Interestingly, the report delves into the activities of 2022. The survey asked, “What did you do in 2022 that you can measure?” It’s not about current actions but about measurable past actions. The exciting part is that those who are actively using AI are truly leveraging it. As we’ve often discussed, AI must solve a problem or enhance a business to be successful. The surveyed individuals aren’t using AI in the ways that many fear, such as job replacement. For instance, American Airlines is currently picketing due to concerns about AI taking over jobs. However, the report shows that AI’s real value lies in augmenting human capabilities and processes to achieve goals, enhance business, and increase profits.

Scott King:

Yeah, I found the same thing. It was really interesting how they actually broke it down. I think there was a sub-segment, I can almost say that, respondents that said what they’re doing with it or kind of the ROI that they’re realizing. But about half of the people said that they call them AI high performers in the report, I think. They said that they’re actually innovating, they’re earning more money for their companies using AI versus the other side of the fence, the other half that respond, like 90% of the people that answered this question said, well, we’re just looking at saving some money. So it was a huge disparity, although it’s a small group, but I think what we find, especially with our customers, they’re actually doing something more. with the same workforce and the same people versus looking at cost reductions.

Chris Kraus:

Yeah, this is very different value prop than like outsourcing for cheap labor or RPA to take over human tasks and eliminate humans. This is this is business transformation. This isn’t cost reduction workflows. And so that’s what’s really neat. So the fear of AI taking over a job shouldn’t be there. This research is proving that those other things never really did a good job of that. But this is definitely about business transformation and being more successful and making more money.

Chris Kraus:

That’s earnings before. That’s a cool thing, right?

Scott King:

Yeah, yeah, so we’re talking about a chart. Does this chart have a name, like a figure?

Chris Kraus:

It’s on page nine.

Scott King:

So smaller shares of AI high performers see cost reductions as their top objective for generative AI efforts. Well, you know, just generative AI efforts, but yeah, the high performers, they see the cost reduction as a smaller portion of what they’re trying to achieve. So. Well, I’ll take a snapshot and put that in there. But I thought that was one of the biggest takeaways. It’s what more can you do versus save money? Because I think when we buy IT stuff and we develop applications, okay, how can we do this less expensively? Other than digital native, right? So digital native, they build more products. I would like to see this chart broken down by industry. and really understand who answered this way because if you go back up to the industry chart, and I’m scrolling through this, I mean, it is, they call it advanced industries, is a segment. Business, legal, professional is one. Consumer goods and retail, energy, financial services, healthcare, and technology, media, and telecom. I mean, obviously, the industries with more data going back and forth, these are your consumer goods, right? When everybody has an account and a login. utilities, telecoms, things like that. Those are the ones that are gonna be able to do more with current resources than let’s say energy, right? So they’ll probably look at cost savings things, I’m just guessing. But I would like them to combine these two charts to figure out who really answered that way.

Chris Kraus:

That’d be awesome.

Scott King:

Yeah. So what else do we have here? So the people that are doing something of all the respondents that said yeah I’m actually using this I mean they cite the same concerns as everybody else, right? The inaccuracy and cybersecurity, right? I mean, those are the two big main concerns. Depending on how they’re deploying the technology, right, you obviously can’t put your IP in the chat GPT website, right? Because then they own it.

What did you find about the concerns?

Chris Kraus:

Yeah, so the third one was the IP. So the fact they could have looked at some law firms, I can see that there, but then they’re using it all the time. But quite honestly, those are things that actually can be overcome. Like cybersecurity, it’s like in the modern world, we know how to write services that are secured. We know how to do two factor authentication. We know how to encrypt our data at rest. Like we have SOC 2 compliance. We know how to solve cybersecurity. 15 years ago, we’d use the term crunching outside soft on the inside. But now, people have gotten good rigor on that. There’s layers of security all across your hardware, your operating systems, your networks, your applications. So cybersecurity, you can cite it, but really, we know how to solve for that, right? That’s a well-known thing. And accuracy is one that’s very interesting, because that is specific to maybe making a prediction, like calculating a number, categorizing something, email and support. Is this a return or is this a request on the status of something? Right. So there’s inaccuracies. Are we predicting the right numbers? Are we predicting our categories? Or say with generative AI, is it actually doing what it’s supposed to do and create a well thought out sentence and a well thought out paragraph. And if it doesn’t have the answer, filling it in with nonsense, we’re calling it hallucinations now so that the sentence structures well formed in the paragraph reads correctly because That’s the goal of generative AI is you want to get well written sentences and well thought out paragraphs. So it’s, you know, kind of augmenting and filling in spaces. We’re now calling that hallucination, but it’s something very natural that is supposed to do. And I mean, of course, we’ve talked about this before. We take a different approach. We say, let’s go. Krista or our software go find the right answers and then ask the engine to do the summarization of the right answers So that it gives you the right information. We’re not looking at them to LLM. So actually go Look across the internet to find the answer the answers to the questions are usually in enterprise They’re in your FAQs or in your share points. Maybe they’re in your systems of record They’re your or your customers orders your customers returns your customer statuses So, you know we figured out and think a novel to solve that no one else is doing right now.

Scott King:

Yeah, that’s the rise of the prompt engineer, right? How do I correctly prompt the LLM to generate a good response? Well, you give it the actual data, right? I mean, you give it the answer.

Chris Kraus:

Yeah.

Scott King:

And then John posted a blog on this. And then we’re going to do a future podcast on it. But yeah, like people cite the inaccuracy. Oh, this thing, you know, it went haywire. Well, it’s just trying to tell a story. It’s trying to, you know, complete a thought which is sometimes hard for us to do on this podcast, right? But… But just give it the actual data, right? Connect it to the real systems. Then you solve your cybersecurity problem, you solve your inaccuracy problems, you solve your regulatory and compliance problems and explain, I mean, you go down the list of these risks that these people are stating that they’re working on, you gotta have a system and a platform to do this for you. You can’t just hire the problem away.

Chris Kraus:

Yeah, and I think it’s interesting you mentioned prompt engineering and That’s actually something that internally we are outrageously fascinated with. Because I come old school. I’m like, well, SQL had rules. Select from order by group by sum. Like, you just applied rules to how the data would join or Cartesian product and things like that. Prompt engineering is more art, right? Because the prompts that we build into the product, say, when we’re doing stuff around compliance when we prompt engines we will use one type of prompt for chat gpd three five one four one for watson ai because we figured out there’s an art along with the science and how you prompt to get the right answers and get the results you’re looking for. So that’s just a whole, we’ll have to do a blog on that. It’s fascinating that it’s actually more art than science right now, but it’s super cool and we love providing that as part of Krista for our customers.

Scott King:

Yeah, I actually, so when we do the podcast, the software does the machine translation, right? And transcription. But the prompt that I’m using to actually clean it up, I mean, it’s like this long.

Chris Kraus:

yeah, tell them what it is exactly Scott. You know what it is. When

Scott King:

I mean it’s like, when Kraus says yeah, if you say the world or if you think about It takes all that out, right? Because it’s learned.

Chris Kraus:

So Scott has learned to prompt engineer my safety words. We get it.

Scott King:

All right, so moving on to what this says. So of all the people, the respondents that said that they’ve tried this at least once, what are they doing with this, right? So surprising, what did I write here? I’m looking at my notes.

Chris Kraus:

40% of tech media, telecom respondents regularly used it followed by financial services. So the people who are using it are actually using it. It was shocking. Like we were both like, wow, that’s actually, like if you’ve actually, people are actually doing it. they’re using it quite a bit. And so that means they’re definitely getting value of it, which could align to why they say 20% of the EBITDA is up or contributed to that. So because the people who are adopting it are actually using it in earnest. This isn’t some fluff because it’s a hype curve or something. They’re actually getting value out of it.

Scott King:

Yeah, and this kind of goes back to my point earlier, the tech media telecom financial services, there’s a lot of data going back and forth. Obviously, those have gotta be the guys that have figured out the 20% increase in earnings by using AI. That’s just incredible. So if you’re listening and you’re one of those guys, we’d love to hear from you, because I think that’s really, really cool.

Chris Kraus:

Especially because they were talking about that like for the genitive AM 2022. So that means people are actually doing this and measuring it. This isn’t like I thought about it last week. Super cool.

Scott King:

I don’t know. Yeah, this is in use. The analysts that I talk to, they always say, oh, we want to hear about production uses. I’m like, there’s production uses everywhere, right? I mean, this is not vaporware. This is, people are really doing this. All right, so the three most commonly reported use cases, I’m not really sure what page, oh, this is on page. for the report. Marketing, obvious. I mean, you’re generating content from a prompt. The marketing use cases, you can actually use the internet trained models. I mean, you’re just regurgitating what people have already said, although they write crazy. There’s all kind of passive tense and appositive phrases. It sounds like, it’s not very good. You have to edit every single thing. But product development, what is that about, Chris? So product development, what kind of products?

Chris Kraus:

So people are actually talking about like low code development. Can you actually generate like JavaScript? Or can it generate HTML to do something? Or can it generate, say, Java code or a class or Python? And I’ve used this. It’s like, write me a JavaScript to do this. Or write me a reg accent inside of a JavaScript to accomplish y. So I mean, that’s in which makes sense you can do that. Because there’s very specific rules and design patterns in development and coding. And there’s examples of those on the internet. There’s examples of those in forums. So it makes sense that it can learn by observing the way people have explained how to do something. That’s something the engines can do. And so a lot of people see this as co-pilots. It can read your code and say there’s a known security flaw here, or they can help complete your code. Maybe you’re creating a memory leak, which I don’t know if you really do anymore. It’s not like the old C-days with malloc and D-alloc and then service operations, obviously, that’s such a sweet spot for us. We have customers who are saying, we want chat bots that actually can answer a question, not just point to an FAQ and make you frustrated. We want to triage your emails. I had a customer who said, well, what if we get faxes? I was like, well, it’s really the same thing as getting an email. We just turn it into text, and it’s just. you know, paragraphs, just like paragraphs of an email, will determine the intent, will determine the sentiment of the person, extract specific data, and then reply back and answer. And maybe the reply back is to, you know, send them an email with a response, or maybe reply back with the facts. It’s still automation, right? It’s just that how are you getting to that get information and replying? So.

Scott King:

So someone asked you about faxes, are you making that up?

Chris Kraus:

No, no, of course.

Scott King:

Well, of course, they wouldn’t expect a fast response time if they’re faxing something, right?

Chris Kraus:

Well, some things the Library of Congress still controls that certain things have to be done in facts and banks and hospitals and, you know, HIPAA compliance. You have to fax things versus email them.

Scott King:

Yeah, if you’re in the CIA or a hacker, right, you’re faxing stuff.

Chris Kraus:

Yeah, exactly.

Scott King:

Right, so service operations. Yeah, that is the, I mean, that’s one of the best places to start because in your service organization, it is your newest least trained employees, right? You can just have a machine learning model answer all those questions. And you don’t, you know, and then have the employees like do more valuable stuff, right? Cause everyone, like the complicated issues, everyone loves to talk to a person, but they wanna talk to a knowledgeable person. So just get the easy stuff out of the way, all right.

Chris Kraus:

Yeah, it’s interesting because in that very first thing we talked about is like, you know, reduction of people versus actually accomplishing a task. And quite honestly, every time I go through this ROI with a customer, I’m like, do you actually answer every inbound email you get every day? It’s like, no. It’s like we could quadruple number of people and we’ll still be behind. So if AI can actually handle 75% of that. there’s still enough work of the important things that are complex for people to handle. So they’re seeing it as we actually get our work accomplished during the day and it’s complimenting people. So it’s not what they were feared of with RPA and outsourcing. It’s literally complimenting so you can focus on the good stuff.

Scott King:

Yeah, it did talk about the workforces and how people will shift. It’s really good to see that the report and the respondents said that they’ll re-skill workers, they’ll have them do something else or more valuable, because if they can have software or generative AI do those tasks, right? I mean, they’re actually removing, I mean, it’s a… If software can do it and a person’s doing it, they’re already doing a robot’s job, right? So just have the robot do it and then move that person to something more attuned to human ingenuity and thinking about more complex problems, right? So it was really cool to see that everybody is, they’re not going to shrink the workforce, they’ll re-skill and reposition everybody. I was really encouraged by that. Because as you mentioned earlier, that’s everyone’s big concern. it’s gonna take my job. Well, you not really right I mean we haven’t really seen that. So Chris, anything else in here before we wrap up? Anything else that you want to point out?

Chris Kraus:

No, I mean, other than I wonder, it’s like when I read this, I was like the middle was the interesting part and I would have organized it differently. So now I’m wondering, it’s like, did they use AI to help write this?

Scott King:

Of course, they did. Yeah, of course they did. Why wouldn’t they? Although

Chris Kraus:

Yeah.

Scott King:

They didn’t use AI to make all the charts.

Chris Kraus:

No.

Scott King:

Whoever does all the charts and all the visualizations at McKinsey, they’re amazing. The McKinsey website is amazing. So if you guys don’t frequent this website, you should. All right, so thanks, Chris. That’s all I have for this episode of the union and the review of this McKinsey report. I encourage everyone to download it and give it a read and look forward to the updates next year.

Chris Kraus:

Cool.

Scott King:

Thanks, everybody.