Intentional AI: The most important operation in analytics - understanding your "why?"

VIRGIL 0:00

When thinking about AI, one of the strengths I think it really has is in the area of analytics, and I think that's such an interesting topic because analytics are big data, and that's the one thing AI is really good at. But is it great, and can it tell you the information you want? Well, today we're going to talk about that. First you need a story to really make it work, and that's something that interests you. Join us as we start discussing stupid.

VIRGIL 0:41

Hi everybody. Welcome back to the podcast. Cole, why don't you kick us off and tell us what we're going to be talking about today?

COLE 0:48

We're talking about data analytics. Seems like a pretty great usage of AI, honestly, given that AI is pattern recognition at the end of the day. And what good is data without recognizing the patterns in it? That's kind of been the problem with it, honestly.

VIRGIL 1:12

Big pattern, I mean, big data. There's a lot of information in analytics, and it definitely overwhelms most marketers and most people trying to understand it.

COLE 1:21

Yeah, I mean, me, honestly, for the last four years I've been with High Monkey, a big part of data analytics has seemed almost like a foreign language. Of course I've dabbled where needed, but actually understanding and making a big impact with that data, it's daunting. I get it. But I do think there is an opportunity with AI to kind of lower the barrier of entry to coming to conclusions with the data.

VIRGIL 1:51

Definitely. If there's anything that I think AI can do well, it's that it can look at data from a large data set or data sets and help you understand it. I always think back to a few years back. We had a customer that was having a big problem on their website. It was being very slow and they were wondering if there were some performance issues. And we couldn't find any performance issues. But what we found in the web logs was there were constant hits to the site, like just this massive amount. And it appeared that it was some bots. Now they use Cloudflare and they wanted to block those bots. So I had them send me the logs, or I grabbed the logs and I tried to open them in Excel. In Excel it was like 3 million rows or whatever. And my Excel just started choking, so I was not able to do sorts or filters, the normal stuff I would do in Excel. So I used its Power Pivot component, which is kind of the predecessor to what it has now with Copilot, but it was its AI component. And I just asked it a question. I said, can you tell me the most frequent IP addresses in this list? And it thought and it ground on that. And I didn't think it was going to work. And it came back and it gave me like six different IP addresses that were each hundreds of thousands of rows. And so I was able to send that to them. They added it to Cloudflare. It took care of that problem for the most part. But that's the thing, is it looked through this massive amount of data and it was able to come back with an answer. And I think that's kind of the same thing with analytics.

COLE 3:34

Yeah. So what you're saying is a bot helped you figure out what bots were attacking you.

VIRGIL 3:41

Exactly. Yep, yep. And sometimes it is actually funny how often I use AI to help me figure out how to use AI. You know, you ask it like, what is the best way to write this prompt? What is the best way to do this? How can I get you to do this instead of that? You're asking it. So it's that loop.

COLE 4:02

Yeah, it does help that you can kind of ask AI how to best interact with it. I mean, the entire series has just been us doing very minimal prompts and just kind of showing what happens when you use that type of logic. But yeah, so back on the topic of analytics. I was scrolling through LinkedIn the other day and I saw a post. In the post, it was talking about how data is cheap, insight is expensive, storytelling is priceless. It was by Tim Stoddart, and I thought he raised some very interesting points. He included a graphic in the post, and it was kind of comparing data analytics gathering and making sense of it to the process of building Legos.

VIRGIL 4:57

Oh, well, Legos. That's good. My son loves to build Legos. I love to build Legos when I was a kid. So that's a great analogy.

COLE 5:04

I love it for something as complex as data analytics. I think it helps to have an analogy like this, at least for me. So the post was talking about: you start off with your data, which is just a pile of an assortment of different Legos, right? You have like black ones, you have yellow ones, different sizes and shapes. And then you sort that giant pile. In this particular post, he sorted by colors, but then you kind of arrange the colors in different ways. Basically it ends up with the Legos explained as a story. And it's just a house, right? It's a house with like a car and trees and stuff like that. Well, while the post raises a lot of interesting points and good conversations about data, it also assumes that the pile of bricks to begin with is valuable, is your right set of data. And that kind of reminds me of how, I think you mentioned the other day, Google Analytics hasn't really changed their standard data sets that they've gathered for a long time.

VIRGIL 6:26

And I think pretty much back to when it launched, even when they moved from Universal Analytics to Google Analytics 4. Yeah, some of this stuff changed, but overall it still collects the same information. The information was always there. And you obviously can get fancy with Google Tag Manager and creating your own events and setting up your own campaigns and everything, but overall you're right, it's collected the same thing. And so I like your Lego analogy, you're kind of bringing those together to see what fits and how it fits together. And I'll even take that another step, because this is something I did when I was a kid and my kid does all the time, which is you take Legos from multiple sets and you bring them together to create something new. And I think that's one of the most challenging things we run into with analytics, is it's not only about looking at Google Analytics on our website, but also, we talked about our podcast. We have the analytics in Captivate, which is what hosts and produces our audio podcast. We have the analytics in YouTube, which hosts and produces our video podcast. We have our analytics in Apple Podcast because now you can do both audio and video podcasts there, and it tracks some separate information. We've got analytics in Spotify for the same thing - it can do both audio and video, and we have analytics there. We have analytics in SoundCloud, and we have analytics on our website for what people are visiting. We have analytics in our mailing list about what people are clicking and all that kind of stuff. And we have all this information and it's all in separate things. So that ridiculous spreadsheet I tried to build that drove me crazy, trying to bring it all in and make something meaningful about our podcast. But overall, this is an area where I say this is where AI can step in.

COLE 8:31

Yeah, honestly. Because the spreadsheet that Virgil mentioned is basically just a gigantic, colossal orchestra. Well, okay, you can't really say orchestra, because an orchestra is like coordinated and composed. A better analogy would probably be the pile of bricks from that post we mentioned earlier. No offense, Virgil, but.

VIRGIL 8:59

No, that's okay. I mean, I broke one of the cardinal rules that you just talked about, something that I preach a lot, which is you got to know what you're putting the analytics together for. Whether you want to call it a story, whether you want to call it understanding personas, a journey. When I first started the company a long time ago, I kind of used my background in instructional design from my education, 4,000 years ago, and kind of said, back when I started in the late 90s, early 2000s, companies that built websites were big development companies, and they would really work on the technical aspect of it and how it all worked. And then these were sites that nobody could find anything on because they were built to technical specifications, never talking about what the user actually needs. So the user experience. So I kind of started the approach of, what if you looked at this backwards? What are the outcomes you're trying to get to? What are the final goals here? And then you build the site backwards. And the argument was actually the last page you should build on a site is a homepage. Well, it's the same thing with analytics. There's a ton of data out there, and it's great because you can easily see pages visited, where they came from, that kind of stuff. But if you don't really have a way to interpret that or even understand what you want to get out of that, it's not very valuable. So you visit our site, what is the end result we want? Well, ideally, we want them to contact us or keep us in mind for future projects. Subscribe to our podcast, subscribe to our blog. There are outcomes that you want. Once you understand those outcomes, you can kind of start using the analytics to help that pattern. And it goes back to the podcast. The thing we want to know is, are we growing our podcast? Well, we can look at our listens and we can see that our listen numbers have went up. Where we were getting on average, during the two to four weeks after an episode went out, we were getting about 30 to 40 listens. Now we're getting 100 plus. So that's exponentially more, and it's going to get closer to that 200 range and keep going up. But what we don't know is anything about the quality. It could be somebody who's hitting listen, they hear the first two words out of my mouth and they're like, oh my God, that guy's annoying, and shuts it off. And we don't know any of that information, so we don't totally know the quality. But the benefit for AI is you can put all this stuff out there and say, here's what I want to learn, and you can see what it comes up with. And it's not always going to be great. I've run it across our Google Analytics several times, just kind of in that microcosm, and I've had mixed results. A good result was, since our podcast was put on our website and we moved our podcast to an area on our website, I said, what has it done to our traffic? And it gave me two pieces of information which were really good. I assumed it really greatly increased our traffic on our site, and actually to our monthly traffic, it only increased it 14%. Now anytime you get a 14% increase, that's great. But considering that we kind of thought it would go exponentially up from that, we're finding that where most people listen to our podcasts in their web browser, they're not listening to it on our site, which is okay. But the more interesting thing was that one thing it did do is it increased the engagement on our site. People are visiting other pages, people are staying on our site longer and engaging more when they're coming from our podcast. That is what we wanted to see. And AI was able to help me understand that without me having to dig through 500 reports or build a big Google dashboard. I was able to ask a simple one sentence question and get a massive amount of information.

COLE 13:24

Yeah, but again, we have the goals defined. And it's shaping up to be kind of a three piece equation in being successful with analytics and AI. It's knowing what you want and then clearly identifying what data you actually want to pull from. There's probably some data that kind of corrupts the big picture for you or just makes it massive.

VIRGIL 13:59

Oh, 100%. So one of the first prompts I ever did with AI was around what type of things were kind of happening on the site. One of the things it said is we're getting a lot of 404 errors. And for anybody who's not sure, 404 basically means a page is not found. And I'm thinking, okay, we moved the podcast, those URLs changed. Maybe somebody's visiting an old link from somewhere to an old customer case study or something that doesn't exist. So I said, can you help me figure out what these pages are they were trying to visit? Nope, can't do it. Don't have that information. So why do you tell me this stuff? This is a limitation of a system like this because it doesn't have that information or it doesn't understand that information. It can't help me, because ideally from an answer scenario, that's great, I know I'm having 404 errors, but if I can't figure out what's causing that, that doesn't help me. And so I dug into our Google Analytics and had a hard time finding it there. I went into Webflow, which is what we use for our website, and I looked at their analytics and couldn't find it there. They're all recording the 404 error. So just because you know the answer you want doesn't mean AI is going to be able to give it. But again, what would be best in that scenario is that it looked at both Google Analytics and Webflow and brought all that information together, compared and contrasted, and looked at everything. And I do think that really is the big future of AI and analytics. Where AI can really benefit is when you're not only looking at one data set, but potentially looking across multiple data sets and helping you draw conclusions from that. And you really do have to look at it as the 80/20 rule. Like most things, you're not necessarily going to get the perfect answer. But the one thing I like about the way it worked is it would give me an answer and give me the data behind it, which was really good. But more so, it said ask me this to get better information. And I've started to see AI tools doing that more. You look at from when we started this series at the beginning, late September, early October, whenever we started the season, to now. The tools have changed at least 50 times. Every single one of them have updated models, they've updated ways. But one of the things they're starting to do more that I've noticed is they sit there and say, well, I might not have given you the information you wanted, so ask me this and I can give you better. Or here are things that I could give you that are going to really help. From an end user perspective, that matters because a lot of times people don't really know what the next question is they need to ask to get a better answer. And now AI is starting to say, I may not have given you the answer you wanted, but here are some things you could get from me. That was actually significant on the podcast, because when I asked about what adding the podcast had done with traffic, it said, you may want to ask me about engagements, what has it done there? And I asked it about engagements, and that's where it said, well, it's really increased the engagement on your site. Not as much the traffic, but the engagement on your site. And that's actually a really significant thing that I want to dig more into.

COLE 17:40

It's massive because you're not going to your analytics for the numbers or the dashboard. I saw another post recently, or it might have been a YouTube video, but it was about the death of the dashboard. With the advent of AI, you're going for conclusions and you're going to move the needle forward. That's what analytics are for. It's understanding why the people interacting on your site are actually interacting the way they are, and then taking your goals and kind of synthesizing those two things into the future of your site.

VIRGIL 18:19

Yeah. And I think that analytics has kind of always been 90% magic and 10% actual strong data practices. I'm not saying that you can't do the strong data practices, but overall, people tend to, around SEO and all that stuff, just kind of say, let me try this and see what happens. Here's something where AI can actually help you build a baseline and understand that baseline of where you're at now, and then actually look at it. And I love that I can really ask it questions about trends. Like, let's say you're doing a new product launch or something like that. Instead of having to build all these events inside of Google Analytics and create these massive reports, now you could sit there and say, we just launched a new product last month and we launched one six months ago. What are you seeing happen? Because maybe internally you did different marketing efforts. Now you can kind of use it for pseudo AB testing inside your analytics and say, what do you see there? There's a lot of potential, but overall I think the biggest thing is people need to take that first step of really understanding their story and what they want to get out of it. What's going to be beneficial for you to understand from it. And if anybody is going, well, we just want to know how many people are visiting our site, then I'm going to say, yeah, that's not necessarily that helpful. Because we know like on our site, a ton of our traffic comes from one of our blog posts that somebody wrote like 15 years ago on Kanban boards. And we still get a ton of traffic on that on a daily basis because it comes up as the number one search result if you're looking for how to use OneNote with a Kanban board. And we've tried changing that story and tried to get people to push to other things and we've had kind of mixed success. But it's one of those things that if you just look at page visits, you think, oh, a lot of people are visiting our site. But the reality is when they visit that page, that does us nothing because it has nothing to do with what our company does. It just happened to be that our project manager at the time wrote that.

COLE 20:34

Yeah, go figure on that one.

VIRGIL 20:36

Yeah, you always have to laugh at it. So before we wrap up, the only thing I want to say is, a lot of people have been listening to these episodes for tools. The reality is there's not a lot of tools out there except analytic engines that have tools built into them. So you're mostly using the tool as part of the analytic package you're already using. Most people probably use Google Analytics, so you're going to probably use that. There are big data platforms and maybe someday we'll explore those a little bit, where you can bring in information from multiple systems and do that. But that is not quite where I think we want it to be. Because the reality is, you can bring in all this information, but then you have to map it out and you kind of have to be a database person to figure that all out. I have a feeling this is coming down the pipe. It's not quite there. You can't get a tool like Claude or something like that to access all these different systems and pull in the information. That's going to be more something that you're going to build your own custom agent for. Or hopefully there'll be some tools that come out in the market in the not so distant future. At the rate that everything's changing, it could happen tomorrow and we'll find a tool.

COLE 21:56

Imagine. I mean, honestly, it's changed so much the last half year.

VIRGIL 22:02

Yeah. It's crazy. Well, thank you everybody for joining us for this episode. We hope you enjoyed the topic and we will see you next time we start discussing stupid. Just a reminder, we'll be dropping new episodes every two weeks. If you enjoyed the discussion today, we would appreciate it if you hit the like button and leave us a review or comment below. And to listen to past episodes or be notified when future episodes are released, visit our website at www.discussingstupid.com and sign up for our email updates. Not only will we share when each new episode drops, but we'll also be including a ton of good content to help you in discussing stupid in your own organization. Of course, you can also follow us on YouTube, Apple Podcasts, Spotify or SoundCloud, or really any of the other favorite podcast platforms you might use. Thanks again for joining and we'll see you next time.

‍