A Rare Conversation With 2 Firms Winning Enterprise AI Work

Transcript: Enterprise AI Strategy – Two Firms on Why Pilots Stall

Japheth Mast (00:14)

Hello everyone, welcome to this live panel with Management Consulted. My name is Japheth Mast, Vice President of Marketing, and I am very excited to have you with us today. We have folks tuning in from all across the globe, and we are joined by a great panel with Damien and Ugo from Adastra and Emerton Data, talking about what is coming for enterprise AI transformation, and how consulting firms are helping companies move from AI pilots into production.

When our editorial team was putting together our ranking of the top AI transformation consulting companies, we found a few notable insights. Mostly that there are a lot of consulting firms that claim to do AI transformation, but the number of firms that can actually get pilots into production is much smaller.

What separates the winners is AI programs that actually reach production. The number floating out there is that somewhere around 10% to 20% of AI pilots make it into production, which is crazy, but these two have a lot more success than that. The second piece of the recipe is governance systems that catch AI overspend and risk before those numbers get out of hand. And the third is scientific depth among the consultants, partners, and leaders at those firms, which really helps them gain an edge in the market.

With that, I am going to turn it over to my colleague Nare, who is part of the Management Consulted editorial team. She will be guiding today's conversation with Damien and Ugo.

Nare Israelyan (04:57)

Thanks, Japheth. Hi everyone. I am Nare Israelyan. I have been with the Management Consulted team for quite a few years now. Former BCG consultant, turned to entrepreneurship and entertainment, and I actually do some work in the AI space myself. So this will be a really interesting conversation.

---

Where These Firms Sit in the AI Consulting Market

Nare Israelyan (05:40)

Enterprise AI transformation consulting has become a crowded space, and most firms describe themselves in the same way. For those on the call who do not know your firms, where do you sit in the market, and what questions are clients bringing to you most often?

Damien Chan (05:59)

Thanks, Nare. Adastra is a global data and AI consultancy. We are owned by Carlyle, the PE firm, and we have clients across both North America and Europe today.

Part of our strength is that we do not just dabble in AI. We build the AI systems and the data infrastructure that our clients are running in production today. Our firm is organized by industry practice rather than just technology capability, so banking, insurance, CPG, retail, manufacturing, and energy are key sectors for us. And what is fairly unique to us is that over 75% of our AI pilots have now made it into production.

What a lot of clients are asking has pivoted. Over the last two years it has been show me that AI works, prove to me that the technology exists. In the last six months or so, that has shifted to what is the ROI behind this?

I have been at Adastra for seven months now, but I have been on both the client and the vendor side. Sitting in the client's shoes, it comes back to what are you turning off? What are you doing to add better value? How is the ROI being matched, and where is the business case behind it?

You can imagine where things started. The experimentation was around Copilot, adding efficiencies, making contact centers more productive. But at some point it is very hard to track those productivity gains in terms of value for money. Are people taking more calls a day in the contact center? Are you more productive day to day? Those are all very soft things to manage. So now they are looking at something more tangible.

Ugo Philippart (08:31)

Thanks for having me. In a nutshell, Emerton Data sits between strategy consulting on one end and AI solution development on the other.

What does that actually mean? About 80% of our workforce are people with a dual skill set, data scientists in their own right but also strategic consultants, both when we hire them and in the way we grow them in the company. We have a very strong scientific focus, and about a third of our consultants hold PhDs, most in computer science but some in adjacent fields like fundamental math or physics.

So you get the idea. Our clients tend to call us when they want something really challenging to work, and they want it fast. Both on the AI strategy and governance front, and on implementing custom use cases. When a nut is particularly hard to crack, whether because the business context is very specific or because the solution requires a really high level of technical or scientific sophistication, that is when we move in.

And sometimes, not in the majority of cases, we are the fire squad that gets called in to fix failed projects, often by companies you are all familiar with.

---

From "Where Do We Use AI" to "Where Is the ROI"

Nare Israelyan (10:21)

Damien, you touched on something I would love to go deeper on. The last two years, clients were asking where they should be using AI. In the last eighteen months it has shifted to, these pilots have not produced anything measurable. So what changed?

Ugo Philippart (10:49)

Where do I start? A lot has changed.

First, in most company boards this is now a topic. Before it was a CDO topic, a functional topic. Now it is a board-level topic. There is a sense of urgency, and for better or worse, there is a fear of missing out. So suddenly there is a lot of tension but also a lot of investment going into it.

At the same time the technology moved quite fast, and we have seen a lot of grassroots initiatives. Not just top-down projects or programs, but a lot more bottom up. People trying things out, being experimental, across all functions, across all seniority levels, and maybe more surprisingly across all levels of data and tech savviness.

That is a double-edged sword. On the one hand it is messy, costly, and risky, and we have all heard the stories. But at the same time there are virtues we do not talk about enough, around how it educates people and fosters ideation.

And finally, AI is now embedded in most of the software and most of the tools people use daily. It comes with illusions and disillusions, but that is a fact of life. Measurable results have been disappointing.

To be more optimistic, there are also important intangible benefits from what happened over the past couple of years. Expectations among the business, even among the least data-savvy people, and the general literacy around what data and AI can and cannot do, have improved a lot. There is also a realization that you cannot be too cheap about it. It is complex, and there is no shortcut.

---

Why AI Budgets Ballooned: CapEx, OpEx, and Token Costs

Nare Israelyan (13:15)

Is that something you have noticed, where clients are realizing they need to invest a lot more than they thought when everything started?

Ugo Philippart (13:27)

Yes. What they realize is the balance between CapEx and OpEx. They skimped on CapEx and said let us go with the flow, let us open some gates. Those gates turned into floodgates, and whether we are talking compute, tokens, or secondary expenditures, it ballooned out of control.

Now there is a realization, and I love that you mentioned ROI earlier, that there is definitely a return with a short breakeven point if you spend a bit more on CapEx, if you invest in proper governance, proper tooling, and proper training. Very quickly, what you gain in OpEx makes it all worth it.

Damien Chan (14:32)

I agree with everything Ugo mentioned. The barriers to entry a couple of years ago were low. The experimentation was easy.

From a budgetary perspective, as the Anthropics and OpenAIs of the world have matured, organizations are starting to see real token costs. What they have been paying over the last couple of years has tripled or quadrupled as people leverage AI more heavily in their environments.

Think about any large regulated organization. They have a limited budget on an annual basis, and their backlog of functionality they want to deliver to clients is huge. When I was on the client side, those were really hard trade-offs. When I ran IT and digital for one of the big banks, it was always: IT, why do you cost so much and take so long?

There are foundational elements required for AI to be successful. The experiments and pilots are easy. So why do most of these pilots get stuck in pilot mode and never make it to production? We see two main issues.

The first is governance. Security, the operational elements around MLOps, whether that has been modernized, incident management, change management, the whole nine yards.

The second, and probably the bigger one, is that the data pipelines are not ready. Grabbing data in an experimental setting is simple. But most organizations are on a path of building proper APIs and getting away from shortcuts, because their technology North Star is set to limit anything that adds to tech debt. When you are building APIs on old legacy mainframes and old platforms, in some cases it takes a year to eighteen months to do it properly.

What happens is that when you move from pilot into production, technology operations steps in and says here are all the pieces you need to consider to make this real.

So what we are helping organizations with right now is how to pull the business and the technology partners together. The business cannot wait eighteen months for technology to build the right North Star, and they cannot wait for the business value either. Boards are tasking their C-level executives with moving faster in the age of AI.

We are looking at new tools that can help, whether that is a CUA, a computer use agent, with something like OmniParser, getting accuracy rates up to 95% on data streams. But we see a lot of organizations getting stuck in that transition.

---

Is Your Organization Actually Ready for AI?

Nare Israelyan (18:24)

So would you say it comes down to being prepared to integrate AI into their ecosystems, and sometimes clients just are not ready? Is that something you audit before you even start work with them?

Damien Chan (18:50)

Definitely, and I think about it in two ways.

A lot of firms look at AI and think about how to get to the CIO and have those discussions. That was true for the big technology shifts of the past. Cloud transformation, software as a service, all of it was IT-led. This is a little different. AI is really business-led in terms of the outcomes and the use cases, and technology is more of an enabler.

The flip side is that some organizations think if we enable AI, we can pull back on some of the technology bureaucracy of the past. Those two things actually go hand in hand. It is not about pulling back, it is about embracing it end to end so that all parties are enabled.

When you get to production, performance testing matters. Clients have expectations about how they interact with organizations. All those aspects of running a business come together, as opposed to assuming that going fast means we can exclude other parts of the organization. So part of the fit is making sure you are bringing the business and IT closer together, and that everybody is educated through that discussion.

Nare Israelyan (20:24)

It is almost a speed rocket that you want to attach to the business. Like with everything, if you scale a gap, the gap gets larger. If there is an existing problem and you try to speed it up, you are just going to scale the problem if you do not patch it first.

Damien Chan (20:42)

A hundred percent.

---

What Happens to the 80% of Pilots That Never Ship

Nare Israelyan (20:44)

Only 10% to 20% of enterprise AI pilots end up making it into production, industry-wide, not for your firms specifically of course. That means they end up running in the business every day. What happens to the other 80%?

Damien Chan (21:05)

This is where we see a lot of pause. A lot of it goes back into restructuring the ROI business case and comes back through the budget cycle. But we do see a lot of it just stalling.

I opened with APIs and data pipelines. I have been amazed at how disconnected the business and IT can be. IT will come back and say, you said no more tech debt, you said build things properly, we do not want to go backwards. And the business says, why does IT cost so much and take so long?

The flip side is that when you embrace something like a CUA alongside other technologies, it does not take eighteen months. It takes about seven weeks. Educating the business to let IT go build their North Star, and then point things over when they are ready, becomes a compromise on both sides.

When we help clients understand that we can move faster, what we avoid is the outcome where the board is pushing the business to show results and IT gets painted in that negative light again. This is about taking everybody on the journey, because otherwise results just take longer to execute and the wins get smaller.

There is a limited amount the business can do by itself. When you think about major systems like underwriting, origination, AML and KYC capabilities, or back office, those are highly regulated. You need to be able to stand your ground and say, can you work through an audit tomorrow if the regulator comes in and asks the question. Those are thoughtful things that need to be worked through, but it really comes down to the communication structure.

Ugo Philippart (23:27)

Building on Damien's input. You have the ones that have not really been thought through and end up not being cost efficient, for a lot of good and bad reasons. And you have straight-up failures.

This hybridization, this communication between business and IT, that Damien described perfectly, sometimes one or the other side fails. The business dreams up something that on paper would be great, but it is built on sand. The foundations are not there. The prerequisites that were required ten years ago for a software development project are the same today. If you do not have good data and good infrastructure, it is not going to scale effectively.

Conversely, sometimes IT is doing great, it might be IT-led, and the business has not been involved enough. The design has not been thought through to scale, to account for all the complexity you end up facing across different markets and different functions.

At the end of the day, some might not even be total failures on either side's fault, but you end up without good adoption. Adoption is linked to all of that. Poor data, garbage in garbage out, design that is not future-proof.

But there is another factor: performance. You can build something well designed, built on solid foundations, but the way things have been modeled and plugged together, it is just underperforming. This is still a very experimental space, and sometimes you cannot predict performance. Sometimes it works, sometimes it does not, and that is fair game. But if you do not account for performance measurement and continuous improvement, the way you would with agile development, being very iterative and really focusing on improving the actual performance of whatever AI solution you are building, it is very likely to go wrong on pure performance as well.

So there are many reasons for failure. That is why we are sitting at 20% success and 80% failure. But there are solutions, and hopefully we will cover some of them today.

---

Can a Failed AI Project Be Salvaged?

Nare Israelyan (26:13)

You mentioned your firm is sometimes brought in to salvage a failed project. What is your experience with that, and how many projects do you think are salvageable? Are there small tweaks, or do you have to rewrite the entire system?

Ugo Philippart (26:39)

There really is not a one-size-fits-all answer. Sometimes it is not salvageable. In most cases it is. But the question is how far back you have to go. If you have to erase 90% of the work done and leverage 10%, it is still not a complete failure, but starting from a blank page would have been very different.

Generally speaking, and because of our specific positioning, when we go back and fix something that ended up a failure, it is because the scientific part of it was not thought through or sophisticated enough, not designed with the right level of expertise.

That is generally quite a big fix, in that you have to roll back a lot of the design, both functional and technical, even though most of the prerequisites are likely still valid. Potentially as well, most of the usage work, anything around change management and training, has already happened. It is very likely that the way the solution was designed to be consumed by the business and by the final users does not change fundamentally. But the build itself, the model, the tooling, the pipelining, most of it often has to go and be replaced.

I know that is not a very satisfactory answer. It is very much case by case. But we rarely face an unfixable situation.

---

What Enterprise AI Strategy Looks Like a Year Later

Nare Israelyan (28:28)

Let us flip to the other side. We talked about failures. Let us talk about real transformation. Take a company that has been through it. How does it run differently a year later, day to day, when it is a success?

Ugo Philippart (29:10)

I really do not think any large firm out there can claim a full-on successful AI transformation. But I have seen entire functions transform deeply thanks to AI.

What I have in mind is the research and innovation department of a large life sciences player we helped recently. The way things work today is that people are not trying to speed-run through their day or through their old processes. Instead they are managing a small support squadron of agents and other capabilities that allow them to accomplish greater things.

There is this popular belief that AI is going to replace us. In this particular firm, they ended up using their brains a lot more than before.

Nare Israelyan (30:15)

It almost becomes your intern, where you are delegating the labor-intensive tasks so you can do more strategic thinking. Would that be accurate?

Ugo Philippart (30:27)

Yes. Nobody is at the bottom of the food chain anymore, because you always have that layer of agents and capabilities below you. So everyone is a manager.

Damien Chan (30:48)

Maybe a little different for us. We are working with a couple of large regulated firms right now, one in insurance and one in banking.

We have pivoted from experimentation around efficiency toward what you can actually turn off and where you can make a difference. It is not new terminology. It is still straight-through processing and automation. But think about the big insurance companies and banks today. They have huge back offices. You want to do a mortgage, how many steps is that? Claims management is just as difficult. In the background you have people swivel-chairing between old legacy systems and new platforms. A lot of that can be done through straight-through processing with AI.

Ugo, you talked about process. Nobody wants to take an existing process from the 1980s and just AI-enable it for automation. So what we are doing is re-engineering the processes. We have tools that help organizations speed through that.

Maybe it is not about knowing all the intricacies of the current process. By the way, the documentation is horrible, the artifacts may not exist, and the people are not there anymore. So we leverage AI to ask, if this is what you wanted to do, how would you actually do it in today's world? In modernizing that process we ask what steps can come out, what can we jump through, and what is still manual after that. The manual piece is the really interesting part, where we get agents to emulate some of what that looks like.

Of course you have to hit a strong accuracy rate and certain thresholds where the human stays in the loop. But we are seeing organizations take that to the next level in terms of where they invest their time and energy. I do not think anybody has it at scale yet, but I am seeing small engagements start from that perspective.

On the application development side, we are seeing a big uptick in AI DLC methodology, the AI development lifecycle. Think about how a program is devised today. You have business analysis, development, and QA. In most organizations, when you price that out, business analysis is usually 10% to 15% of any program. AI can start doing that better and faster, and give you the artifacts and documentation out of the gate. You can probably shrink that to 5% or 6%.

On the development side, everybody has talked at length about low-code and what that looks like today. And then QA. In any program I have been through, QA has ranged between 30% and almost 40% of the project. Done properly, AI can process more test scripts and give better test coverage than you would get manually.

If you can link these things together and put the platform in place to transform your culture, your tools, and the organization, we are seeing better throughput. And you always have more work than you will ever get done. The backlog is huge. So how do you free up time to do other things? We are starting to see that take place. A lot of organizations are asking the right questions. I do not think it is at scale yet, but we are seeing it in pockets.

---

Who Should Get AI Tools, and How to Train Them

Nare Israelyan (35:11)

Previously organizations were giving AI tools to everybody. Now they are getting more specific about who should be using what, and what value they are getting for the money. What does a company that does this well do differently?

Damien Chan (35:41)

At a high level, they are segmenting. They look at who they give it to, they look at risk profiles.

But the companies that do it really well are also training their people. I am not talking about a one-time training or a lunch-and-learn. It is going through certifications, taking the time to invest in their people and understanding that it can be different.

Think about cost. Most people have AI running and they keep using the same chat over and over. I do not think they recognize that everything in that chat is being reprocessed every time they add to it. If you close it and start over, Claude, OpenAI, all of them have memory capabilities. You can save organizations a lot of money doing that.

So teach the fundamentals. What does a prompt look like, what is the right way to write one, what is the cost behind the scenes, and how do you leverage it properly in the right circumstances. Educate people on what agents look like, because everyone has repeatable functions that happen week over week and month over month. Get people queued up on that, because it can save a world of time.

The last thing I will say is that common sense prevails. I remember rolling out AI to a finance department and watching them get creative about manipulating their spreadsheets, with the little thinking circle spinning. I asked, just out of curiosity, in the old days when you did this with pivot tables, would you be done already? And they said, yes.

Come on. Let us use things where it makes sense and where you are going to be more productive. Let us not use it for the sake of using it.

Ugo Philippart (37:57)

The ultimate guiding principle where I have seen this transformation succeed, and you can call it governance, goes back to the fact that a company is a combination of human beings with their own interests. They do not really care about the company's interest, they care about themselves. So if your North Star is making life easier as opposed to making life more complicated, it is very likely to work.

When everybody else designs fancy AI governance with extra committees, workload, and responsibility because they are freaking out, they are making life more complicated for everyone. The ones who have real success build something extremely operational. Training, as Damien said, but also things like decision trees that make every decision people face on the ground every day around AI simple and straightforward, leaving no room for misinterpretation. Just making life easier. That is the quick recipe for success.

---

Getting Teams to Adopt AI When They Fear Replacement

Nare Israelyan (39:04)

The old consulting adage, do not boil the ocean. Speaking of people, there has been a lot of talk about AI coming to take your job. So how do you get teams to adopt something that looks like it is going to take their job?

Damien Chan (39:45)

Everyone will talk about being transparent, everyone will talk about communication. I think it is about trust within organizations.

I will give you an example from when I was on the client side. We were leveraging some of these tools in our QA environment. QA is one of the hardest jobs. At the last minute, when problems come up in a program and need to be retested, these individuals work all evening and all weekend trying to meet deadlines, and there are always more test cases than they can get to.

Making people feel comfortable and understanding that this is a tool that can help, versus it is taking our jobs away, is really hard. It came down to trust.

When we first rolled it out and talked about the capabilities, the amount of pushback was significant. As much as people hated working evenings and weekends, they would rather endure that 20% of hardship than embrace the tool. It was not until we built a safe environment for people to ask those questions, and to talk about whether we were really going to step into what we said we would step into or turn their jobs off tomorrow, that it changed.

It took a lot of effort. We are dealing with individuals and people at the end of the day. The technology will always grow and we will always have more capabilities, but there is that insecurity about what happens to me as this rolls out. We have had to change what change management looks like going forward, because I was shocked at how much people would endure the pain of that extra 20% rather than adopt.

Ugo Philippart (42:03)

Absolutely, and change management starts earlier now. But as soon as you introduce the paradigm that this will allow you to trade the most menial and repetitive aspects of your job for the most creative and rewarding ones, you get people on board quite quickly.

To be fair, it does not apply to everyone. But most people have an expertise of their own, a role to play in combination with the machine that is irreplaceable. The key is to help them figure out what that is. In very rare cases nobody can figure it out. Honestly, it has been easier than you might think to get people to stop freaking out and start embracing the change.

---

How to Measure Agentic AI Performance

Nare Israelyan (43:53)

Let us talk about a company that builds an agentic system, ships it, and months later it is underperforming. How should they be measuring performance so they catch this earlier?

Ugo Philippart (43:53)

To our viewers, if you have to take one concrete thing away from this panel, I think this is it.

It gets technical, but it is funny, because out of all the major technical leaps in computer science history, generative AI and agentic AI is the first where performance measurement was not really in the conversation from the outset. Which is strange, because it absolutely should have been.

Today there are basically two ways to assess performance, and neither is satisfactory on its own. That is probably also why the conversation has been avoided so much.

Option one: you ask knowledgeable people, your target users and experts, to run a series of manual tests and give feedback. But this is far more comprehensive than what you would usually do for regular software with clear functional specifications to test. So it is highly time consuming and very expensive. If you have to iterate over time to avoid drift, it gets completely out of control. Not always a realistic option in a corporate context.

Option two: you use an LLM as a judge, and you test a huge number of scenarios across different prompts and parameters. But by now you know that LLMs are overly optimistic, very biased animals. You will end up putting things into production too early that are dangerously underperforming, covering areas where they really should not.

So you can probably see where I am going. Your best bet is a combination of both. Use LLM-as-judge to retest at scale at an acceptable cost, while using a modest amount of human expert input to de-bias the LLM judge.

On that, I want to give a shout-out to an open-source library called Glide, which we developed to do exactly that. You can find it on GitHub, it is free. Go ahead and start using it and properly assess your AI performance early, before you put it into production and start doing crisis management.

Nare Israelyan (46:24)

How early would you say they should start?

Ugo Philippart (46:29)

As soon as you have the first output. Developing agentic systems is a lot more agile and iterative than what we used to do with software development. From the first few days after you start playing with some primary agents, you might as well get your hands on this, or at least build a simple suite of tools that will measure performance.

To me that is something you should build as a prerequisite. Your little functions, your little modules to assess the true performance of your system, before you even go ahead and start building, modeling, or parameterizing agents or generative AI models.

Damien Chan (47:28)

I would echo everything on performance, because performance also equals cost.

Think about how the LLMs are positioned. This goes back about eighteen months, when GPT-4 was out and giving accuracy rates in the 90% range, but the costs were two or three times what organizations wanted to spend. So the question became, can you do it with 3.5? At the time, 3.5 was giving accuracy rates in the 70% range.

Part of that goes back to Ugo's point about how you get teams to work together and agree on what good looks like. This is where business and technology need to work closer together. But even within technology, the application development group, the operations group, and the architecture group need to come together.

People sometimes forget that now we have these new tools, we can bypass some of this. MLOps needs to be brought in earlier. Part of the challenge is that in the past, tech ops has always been seen as the ones slowing us down, the ones with all the rules, the people who say no. Part of their job is to protect the systems and make sure they are always up and running. But that does not mean they cannot be helpful if you bring them in in a different light.

When we had that issue, we brought people together, we looked at it, and we got 3.5 performing at a 95% accuracy rate. People looked at us and said, what did you do? We have never seen that before, it is not known for that. But when you put the right people together and they want to solve a problem, those things are all achievable today.

Part of it again comes back to people. Anybody can do anything once, on their own. The question is how you make something sustainable and drive it for the long term.

---

Owning the IP: How It Changes the Engagement

Nare Israelyan (49:47)

Ugo, you mentioned a tool your firm has rolled out, and I know both of you have turned client work into products you own. How does owning that IP change the engagement?

Damien Chan (50:02)

Most organizations put a lot of trust in you when they want to do business with you. But if it is the first time you are building something, there is a lot of risk as well.

Similar to Ugo, we have built our own tools and platforms. The back office automation we talked about, the process re-engineering, supply chain optimization. We have built tools for our clients.

Every organization out there has talked to somebody about AI, and every board member has asked the C-level what they are doing this month about enabling more. There is a portion of AI fatigue happening, where people have had so much discussion about it.

It is one thing to talk about it and show the theoretical slide about where things need to go. The business will always have more use cases than are possible. But when you can show up and say we have actually done it, here is what we did, here is our experimentation, here is what is working, here is what other clients are doing with it, you are building a different level of trust.

Back to where we opened this discussion, less than 20% of pilots make it into production. When you can say here is what good looks like, here is how you get it, and here are all the people you need, that matters. It is not as simple as taking one or two individuals and saying we can do this ourselves. It is about how you pull the right foundation forward, and showing credibility. That has been a huge differentiator for us.

Ugo Philippart (52:05)

The ownership model really depends on the situation. Sometimes you need to own the IP to give trust.

But conversely there are also benefits to giving the IP away. In my experience, and we tend to do this a lot more these days than a couple of years back, transferring the intellectual property to the actual user of an AI-powered solution really impacts the scale and the sustainability of the impact over time. You get people's commitment to the success on the day of implementation, but also tomorrow, because it is going to be in someone's annual objectives and targets somewhere, not just a line in a P&L. They have something to lose if it fails, because they own it.

It also allows for continuous improvement. When they own something, they are incentivized to keep up with the latest technology and make sure it adapts to evolving processes. They learn from feedback, and it keeps improving as a company asset. I think that ensures impact over time. Autonomy and independence also lower the risk from this perspective. And generally speaking you are talking about slightly lower operating expenditure than if you have to pay licenses.

---

What Will Matter Most in Three Years

Nare Israelyan (53:47)

We are coming up on the hour, but I want to ask you both. Fast forward three years. What capability will matter more in enterprise AI consulting than it does today?

Ugo Philippart (54:13)

Honestly, I do not have a crystal ball, and three years is a really long time in this field.

But my bet would go on, and you might think I am obsessed with this, properly measuring the performance of complex agentic systems, tracking drift well, and better governing the scope of what goes into production, at what time, and what comes out of production, which is now a thing. Taking a model out of production is going to become an increasingly non-negotiable skill set for enterprise AI consultancies.

And as we have seen in other fields, expertise in highly specific sectors and use cases that cannot be cracked by generic agents, because there may be limited open intelligence for an LLM to feed on, is also going to become a lot more differentiating. Things like marketing mix modeling, and the list goes on.

Damien Chan (55:17)

I might add a slightly different perspective. I would say critical thinking is where things are really going to improve.

My fear, as I look at the market and at organizations that have made these changes, is that the people who built the systems are no longer there. The people who built the processes are no longer there. So unraveling that spaghetti is one thing. Putting your trust in a tool to do low-code application development going forward is another.

But what happens when things break is what we are going to need to look at. There was a lot of rigor in how we used to do things in the agile world. What I am noticing now is that it takes longer to understand the root cause of why some systems go down. The fact that they can bring them back up quickly is one thing, but they still cannot accurately pinpoint why they went down. I think that is going to be a gap moving forward.

So I continue to challenge everybody here. This is the cognitive era. Challenge yourself, stay on top of things. It is one thing to leverage the tool. Understanding the why behind it is going to be very valuable tomorrow.

A personal concern of mine is making sure somebody does not call me at three in the morning when things go down and we do not know how to bring it back up.

---

Why Build a Career in AI Consulting

Nare Israelyan (57:03)

Quick answer from both of you. What makes this work worth building a career in?

Ugo Philippart (57:16)

We are obviously going to be biased here. But it does not get more ever-changing than AI. It depends on your personality, but if you are inclined this way, you will keep learning every single day and you will keep helping people. Both feel pretty great.

And you will never get bored, because of that ever-changing set of expectations, technologies, operating models, and ways of working. Your colleagues and your clients are going to be different all the time. So you will never get bored, and I think you will never be boring at the dinner table either.

Damien Chan (57:58)

I hope I never get boring at the dinner table.

Think about what has been said about consulting. I met with a client the other day and they said, you are a consultant, what are you going to do now with AI? And I said, time out. Everything Ugo just said. We are at the cusp of everything that is new. It is so exciting. If you want to learn something different and not do the same thing over and over again, this is the place to be.

Two, it does change a little. Our junior consultants, our analysts, things like slide building, those will look really different tomorrow with AI. But think about what senior people start to look like. You are going to be deeper in the business, you are going to understand more about how things work and operate. Who is going to make the hard decisions about what to focus on and what the trade-offs look like? Who is going to help CEOs decide what they should focus on?

Our roles start to change, and if anything, as we get more senior, it gets more exciting. Every business is a little different. Everyone has a different starting point. Putting that thinking cap on continues to challenge you every day of the week. I think it is amazing.

---

Closing

Japheth Mast (59:30)

Could not agree more. Thank you both. My mind is swimming. So many good topics discussed. Thank you for sharing your experience, your perspectives, and your thoughts on where this is all heading. It is a very dynamic and exciting time in this space.

Be sure to check out what these two firms are doing, and connect with them on LinkedIn. They are certainly at the forefront of what is going on inside the world of enterprise AI transformation.

Thanks again, Ugo and Damien.