Intentional AI: AI agents are only as good as your workflow

VIRGIL 0:00
In the world of marketing and IT, there's probably nothing we're more known for than taking a word that had some kind of meaning and making it into junk by using it for everything. So today we've got another guest with us, Sean from Kentico, who is going to talk to us about agentic AI and what that really means and how you can actually use it and how it's very purposeful - if I say it, intentional AI - and the way we can use it. So if this is something that interests you, go ahead and join me as we start discussing stupid.

VIRGIL 0:44
Hello, everybody. Welcome back to the podcast. We've got another special guest for us, if you can believe it. That's two special guests in a row. And this time we have Sean Wright joining us from Kentico, a CMS I am intimately familiar with, to help us talk about agentic AI. So, Sean, would you like to introduce yourself real quick?

SEAN 1:07
Sure. Thanks for having me on. So, yeah, I work for Kentico. I am Kentico's lead product evangelist. And what that means is I talk to our partner agencies and our customers about our product and make sure that everyone understands the value and the impact that Xperience by Kentico can have on their business.

VIRGIL 1:27
Perfect. And one of the big pieces of that experience right now has been a lot of the things you guys have been doing in the world of kind of AI enabling content management. So I'm going to throw it over to Cole and we're going to get started kind of talking about it from that side.

COLE 1:45
Yeah. So agentic AI, agents. It's funny, we were at a conference about a year, probably a little less than a year ago, M365 Twin Cities, and one of the hottest topics then was agentic AI. And it's been, I don't know, almost a year since then. I feel like we've gone down a lot of rabbit holes about agentic AI since then, but I feel like the more I learn about it, the more I understand that - I don't know, it seems like a pretty daunting topic for some people. But how would you kind of break down your perspective of agentic AI, Sean?

SEAN 2:16
I think I would start with something that most people would probably be familiar with, which is AI chat. So at this point, I think pretty much any white collar professional, at least in the US, has interacted with ChatGPT or Claude or Gemini or something like that. And you go to one of those chat experiences and you ask a question, you say to the AI, how do I bake sourdough bread? Or how do I improve this marketing campaign I'm working on? You know, whatever the question, and it comes back to you with a response and then it ends and nothing happens until you ask another question. And so that's the traditional chat experience. The difference between that and agents, or interacting with agents, is that it doesn't stop when you stop. So agents have the ability to make decisions, to take action, and then to iterate on those abilities until some goal has been accomplished.

VIRGIL 3:26
Yeah, I think that's a great way of putting it. I think probably if there's a problem, kind of going back to what Cole said, is it - you know, we love it in IT and marketing to take phrases that meant something and turn them into junk. And it's amazing how many times I talk to people and they talk about agentic AI and I'll immediately say, what does that mean? They honestly have no idea. It's just something that they heard out there because everybody's talking about it. And I think that's one of the big things, is people don't really understand. So when you talk about it continuing on, can you give a good example in the world of kind of content management about how you could use one of those to interact and kind of go on from there after you've done something?

SEAN 4:11
Yeah. So that continuing on really depends on the tools and abilities that the agent has access to. And so if you were to take that chat experience and just leave it at that, but somehow make the AI an agent, it really wouldn't change that much because it doesn't have any access to anything beyond the chat. Now, maybe it could take the role of driving the conversation instead of the person using the chat driving it, but it depends so much on the tools. And so in the context of a content management operations workflow that a marketer would be familiar with, the agent can then use the context of all the data and structured information and messaging in that content management system and take action based on its ability to access that. Maybe it creates content, maybe it modifies or transforms content, maybe it just goes out and uses some tools to analyze the content and then brings back some more information for the marketer using it that's beyond what is in its original training data.

COLE 5:21
Okay, a little bit of devil's advocate here. So if you provide, let's say, a good amount of data for whatever agent, is that kind of guaranteeing that you're going to get the same response every time or the same outcome every time? Or I guess just kind of introducing the side of different outputs.

VIRGIL 5:44
Which goes along with what we've seen, is that you can ask the same question twice and get a different answer.

SEAN 5:49
Yeah, absolutely. I think if you are looking for the same response or data every time, then use something like a static chart or graph or report that's going to give you deterministic output. You put some data in and you're going to get the same thing out. Now the only thing that's going to change in that report or that graph is the visualization or the data that's in the table, because that's what's changing on the way in. When you use AI, either an agent or not an agent, you should expect that you might get a different response. And it's not about perfect response or bad response, it's about kind of finding where the quality of the response is good enough for you to trust it and to take action on it.

VIRGIL 6:42
Yeah, I've been playing a lot with some AI image generators - and that probably is going to make Cole cringe - just to see how far I can push it. Not to create photos, but to create actual graphics, so drawings and, you know, based off it. And I agree with you, Sean. I think some of the challenges from a kind of user perspective is you ask a prompt and it gets you almost there. It's like, it's almost what I wanted. And you're like, okay, if you can just take this - and you'll even ask it, say, take this specifically right here and make this modification - and then you get back something completely different. And, you know, I know one of the AI image generators I'm playing with is called Mindful, and they just released a new V8 of their model. And the whole thing is about being able to actually do those kind of modifications, micro modifications. But I think that's one of the challenges. And you know, throughout this entire kind of podcast series, that's what we've seen when we've had it actually work with creative content and that kind of stuff, you get something different. And you can't just say, but I just want you to help me right here. So from Kentico's perspective, in integrating it into a content management system, what tasks have you kind of seen that it can work best in, especially with that thought that if you have it run it twice, it might decide to do it a little differently?

SEAN 8:11
Yeah. So for our product, Xperience by Kentico, AI is seamless throughout the product. It is just part of the experience that the marketer will have when working with the product. And there are areas where the AI technology, which we call AIRA - and it stands for AI Recommendations and Assistance - AIRA is working behind the scenes and the marketer really doesn't interact with it. They don't see it. And that's okay because we want AIRA, in that case, to take over some of the responsibility of the traditional marketing workflow and speed it up and improve it for the marketer. So in that case, an example would be AIRA's image transformation and optimization. So if you create, let's say, a web page or an email and you want to add some imagery to that, you can drag the image from wherever you got it - you know, maybe from a designer or maybe it's something you found in a stock photo library - you drag it over to Xperience by Kentico, it will upload, AIRA will optimize that image automatically for you. So this is something a marketer normally would have done with some other tool that they'd have to be trained on. You don't have to worry about that. The image gets optimized and you can define variants for images across your digital experiences. So again, websites, emails, headlessly driven applications, mobile apps, things like that - you can define these variants of images and the variants can be different shapes, they can have different crops or different resolutions, and AIRA will help generate all of those different variants of the image. And if you have crops of that image - so for example, if you have a very wide landscape image of some rolling hills in the Alps and there's a person on those hills off on the left side, if you have a cropped variant of your images that maybe is vertical and you take a crop in the middle of that image, you're going to cut out that person of interest, that focal point. Well, AIRA automatically detects the focal point and makes sure that the cropped variant of the image you just uploaded is focused on the right part of the image. And so all of this happens behind the scenes. The marketer just drags and drops, uploads the image, and it's taken care of. Additionally, AIRA has access to the content of the image to detect that focal point. It can also extract information about it, so it can fill in the alt text that would make your image accessible when delivered through a web experience. It can also apply taxonomy tags to it. So marketers in Xperience by Kentico define taxonomies. These can drive customer experiences, they can drive content discovery for marketers within the product. AIRA has access to those taxonomies, it detects the content of the image, and it automatically applies the right ones. And all of this is done without the marketer seeing AIRA pop up like Clippy, you know, years ago, and say, would you like me to help you with this? We just know that the marketer would like this to be part of their workflow and AIRA takes care of it automatically.

VIRGIL 11:17
And you talk about strengths in AI - and you know, our entire series has not been about bashing it, it's been about talking about where it works well. And definitely the places where it works well is when it needs to use logic and it needs to use math. And images, frankly, are all math. And it really excels at that. That's why coding, a couple episodes ago when we were talking about coding, it's such a good fit there. It's the creative side. And so I'm going to kind of put you on the spot a little bit about that and say, now one of the newer features you guys just released was a content strategist. And I have it, but I haven't got to play with it much yet. This is where we've seen AI really fall down, is when you start asking it to help mimic the human brain in the way we do things and our voices and styles, no matter how much you make it. So when you guys approach this, what were some of the opportunities and challenges you saw trying to do that?

SEAN 12:17
That's a great question because I have also experienced this myself as a marketer working with AI in other platforms and technologies. So the feature you're referring to is our content strategist agent. It is an agent, as we mentioned earlier, that AIRA can delegate work to, and specifically work to evaluate the content of a web page. The content strategist will take a look at that web page as your visitor or customer would see it, and then evaluates it against a rubric. And this is the key part. AI agents or otherwise can perform at different levels of quality of the results they deliver you, and that is often based on the context that they have available. So if you were to go to ChatGPT and you type in how do I bake some sourdough bread, well, it will give you a response. But if you somehow need gluten free sourdough bread - I don't even know if that exists, but let's say that's what you're looking for - and you don't provide that context that your diet is gluten free, it's not going to give you a result that is valuable to you. The same thing applies anywhere that you're using AI. And so with this content strategist agent, it will evaluate the page against a rubric, and that rubric is your content strategy for your organization. And it will grade that content against tone, style, and voice. And the guidelines for that tone, style, and voice come from that content strategy document. And you can co-craft that with the agent. It can help you to design a good content strategy, or you can provide it one. You can say, here's my content strategy, I already worked on it with my team, and I want you to use this from now on when evaluating the content across our website. And because you have provided that additional context, it makes sure that the results you get back from the agent are high quality. And of course, there is additional context within the product, within Xperience by Kentico. This additional context lives within the platform, within the product. It's the insights that we feed into AIRA and the agent from our understanding of how to make a marketer's experience within a CMS fantastic, and how to correctly evaluate content given a content strategy document. And so you combine all of this stuff - the context of the web page, the context of the content strategy document, and the context of how you orchestrate these things together that is baked into AIRA and Xperience by Kentico - and what you get out is an evaluation of that content that, in my experience using it as a marketer, has helped me out. So yeah, I use this content strategist to evaluate and give me kind of a double check before I publish something. Take a look at this. Does it match that tone, style, and voice that I established already that I'm using on the other articles and content that I'm writing? Does it match that? Because it's so easy to drift over time. Or, you know, maybe it's sunny out - right now it's snowy and cold, 17 degrees - but sometimes it's sunny out. I have the window open, I'm feeling great, and my mood is affecting how I write my content. So it might influence me to write something that's a bit more friendly, conversational. But if that doesn't match the style and tone of voice and the overall messaging that we want to deliver to our audience, then it's not going to match my other content that I've already written. And this content strategist agent can help me with that.

VIRGIL 16:03
I mean, what you say is what we've been preaching this entire series, which is it still takes planning, it still takes planning, it takes effort, it takes what you put into it. And I think that's one of the problems, is the way a lot of AI is marketed is just like, boom, it's going to go, it's magic, you don't really have to do a lot of work. And when I talk with people - you know, you talk to them about what kind of questions they're asking, like, I want you to create a document with this, and that's the context that they give - and they don't understand it. And yeah, I think that's a really good way of looking at it. Are you seeing other agents, especially in the world of content management - whether Kentico is going to develop them or not, but just things out there that you've seen that you think could be helpful for organizations to use as part of their workflows? When you start talking about a content lifecycle.

SEAN 16:58
Yeah, I think there are a lot of helpful tools out there and there are even some ways of using the standard tools that a marketer is familiar with. So I used ChatGPT for quite a while. I've now kind of switched over to Claude. But, you know, pick anyone that you would want as a marketer. You can take that tool without a whole bunch of other things, and if you understand how to provide that context to get a good result, it can impact your workflow in many different places without having to go and buy another tool. But the key is how much work is the marketer having to do to craft that context to get the high quality result. And so that's always going to be the case. If I go buy some best of breed AI content creation or content evaluation tool and I feed my content into it, it's very possible I could get a great result from that. But if that tool doesn't have the context, then either it will be the marketer's responsibility to provide it or they won't get as good of a result as they could have. So a lot of this comes down to a marketing team understanding what their workflow and process is today. They need to have a good grasp of that - who's involved, what are the checks, what are the goals, what are the KPIs. If they don't understand that, then AI is just going to help them move faster, but not in the right direction. They're going to maybe produce more content, but it's not going to be something that increases conversions or increases revenue. It'll just be more content for the sake of more content,

VIRGIL 18:41
Which strangely, a lot of times is the point of doing websites in the first place.

COLE 18:48
So it kind of seems to me like a big purpose of agents and AI in general is kind of acknowledging that humans have a very finite amount of cognitive load that they can take on while either operating in a CMS or whatever you're really doing in your job. And it kind of just allows you to acknowledge how you would do it, create a very structured model of how you would do it, and focus your attention elsewhere - like be it creativity or, you know, whatever it is that you want to focus on - but kind of just really acknowledging that, hey, I can't sit here and just go through a spreadsheet all day and expect to write a good blog. Would you kind of align with that perspective?

VIRGIL 19:33
Well, it's an 80/20 rule. I mean, let's be honest. I mean, I've been preaching that in workflow for 20-some odd years. Build a workflow that can help you with that 80% of the mundane work, then you can focus your brain and energy on the 20%. Again, I think the biggest problem with AI right now is it's sold as being able to do 100% of that. And, you know, every day we hear the news how it's replacing jobs - when, you know, we had that big thing about Amazon, how they replaced a bunch of jobs and then they had to hire everybody back because it ended up it didn't work - and it's this marketing. Because what did I just see? They need to spend another trillion dollars on infrastructure here by 2027 or something to be able to keep AI going. It's just different.

SEAN 20:16
I believe that AI will replace my job that I'm doing right now in the same way that if I was a marketer in 1990, over the course of time, technology would replace the parts of my job that I did then. If I was a top performing marketer, I would look for opportunities to do something different that I couldn't before, as technology commoditizes the work that I did previously. And I view it the same way right now. I am not afraid of AI taking my job today. I have a list of things to do that has been 10,000 items long for a very long time. And my job has been to prioritize those and pick the top three that are the most valuable over this quarter that I can actually tackle. The rest of them, they either get thrown away because they're now outdated or they just end up in the list, always kind of second best for priority. AI does not suddenly turn a list that's 10,000 items long into a list that's zero items long, and now there's nothing left for me to do. If anything, it just handles more of those pieces or lets me do them faster so that I can identify additional things that I've always looked kind of at the horizon to accomplish that I couldn't do in the past because I didn't have enough time.

COLE 21:45
So it's like one of those E scooters that you are controlling versus - I think we've talked in another episode about like the self-driving hover chairs from Wall-E - because that's how a lot of people kind of see AI maybe and how it's marketed. But really, it just seems like a big opportunity to excel in whatever field you're in. You know, especially for marketers where I also have quite the list of stuff to tackle here.

SEAN 22:21
I think AI might be the chair from Wall-E. It might be, but the question is, what does the person do with that chair? That's an individual's choice. I tell everyone if you aren't learning one new thing with the help of AI each day, you're using it wrong. Because you have the world's information at your fingertips. And before AI, it was Wikipedia or, you know, any craft or hobby. If you're not learning something new each day, you're kind of doing it wrong. And I think the same applies with AI. So if I get that Wall-E chair eventually because of AI and technology and it can fly me around everywhere, well, I'm going to use it to fly me places faster and then go and do something productive there, something useful, something inspiring, something that I want to do anyway but couldn't because I couldn't get from point A to point B fast enough in the past.

VIRGIL 23:19
Yeah, I can tell you over the last eight months of doing this series so far, I have learned exponentially from AI - mostly what AI can't do, or how I can ask the wrong question or not do it the right way. But I mean, you know, it's a learning experience. That's how you get better. And it is crazy. I mean, how many times have I seen people that write prompts that are like two pages long? And I just think that a typical person's patience is just not going to be there for that. But you're right, it's all about context and doing that kind of stuff in there. So I mean, it really does come down to it. And so in the end, when we start talking about agentic AI and what that really means, it's about AI with a purpose. Intentional AI. Wow, that sounds like a good name for a series. But I mean, that's what it is. It's about using it for a very intentional purpose versus using it as an all-catch-all from that side.

COLE 24:17
One thing I want to make sure we touch on in terms of agentic AI - we talked about this before the recording, Sean - you mentioned the concept of evals, making sure to do those to ensure you can keep track of how your agents are performing. Is that kind of what you're saying?

SEAN 24:35
Yeah, this is something that is a key part of vendors putting AI into their products. Because of AI's non-deterministic nature, you give it some context and you might get a different result each time. In fact, you probably will. And the question isn't can you achieve perfection, it's can that result be good enough to take action on and feel confident about? And so as we integrate AI and make it just part of the Xperience by Kentico marketing workflow, we need to evaluate - if we perhaps change models or provide additional or less context to the agent to fine tune the results - are we getting the same output, the same quality of output? And this is kind of like the content strategist that I mentioned, where the marketer crafts some content and then the agent evaluates it and says, here's the rubric and here's how you did, here's your scorecard. Evals are kind of the same thing, but it's an evaluation of the agent's output, not the marketer's. Now this is typically something that you see in a product where the product team is iterating on the AI implementation and they want to make sure they don't have a regression in the quality of the output. Or if a marketer puts in something kind of wild that's not expected, how is the AI going to respond? Evals can help to keep the implementation on track. Also, new models come out all the time. I'm sure everyone's aware of this. There's going to be something in the news tomorrow about a brand new model with all these capabilities. Do you adopt it? Well, that is a question that product teams - my team - has to think about. Do we adopt that new model? What impact will it have on the marketer's experience in the product? This can also apply though to the individual marketer. So if you're using, for example, Claude or ChatGPT, you can pick which model generation you use. And if you are used to getting a certain type of result with a specific set of context and then you change your model, that result might change. And so evals could be a way for a marketer or a marketing team to feel confident changing the context they provide or changing the model they're using over time, and know that they're going to continue to get a good result out of it.

VIRGIL 27:01
Yeah, no, that's such a big thing. And you know, I laughed immediately when you said about the upgrade models. I started thinking about hot fixes in the world of CMS and how every time a hotfix comes out we've got to evaluate it and decide if we're going to have to let our customers know so that they need to implement it. But it really is kind of the same thing. You're right. There's a new model every day. Everybody's trying to one up one another. I wish I could say that was going to slow down, but I don't really see it, because obviously this is the new multi-trillion dollar industry that is just going crazy and everybody's using it for almost anything. So, well, Sean, I can't tell you how much we really appreciate you joining us. This was a great conversation. I appreciate you sharing some of your thoughts on agentic AI and all that. And as we continue on here, I think this is a great area and it sure helped me understand a little bit more about it. Maybe that I get a little caught up in the hype as well.

SEAN 27:57
Yeah, happy to be here and thanks for inviting me on.

VIRGIL 28:02
Thanks everybody. That's it for the podcast.

‍