Your expertise isn't the moat

In my last piece, I argued that AI disruption of agencies and professional services firms is late, not cancelled. The companies that will do it are just getting started.

My advice for owners was to start encoding their knowledge and services for clients into agents. That would make them scalable and allow owners to serve many more clients.

Then Ethan Mollick published The Dot and the Swarm and I realized that advice was only half right. Mollick, a Wharton professor and leading applied AI expert, admits he got something fundamental about agents wrong. His assumption, like mine, was that getting them to work would require careful human design and for humans to embed their expertise into them. But he’d forgotten the Bitter Lesson.

I used the AI music generation service Suno.com and Claude to make a video demonstrating the key points of this post. Check it out. It’s pretty damn good:

In his 2019 essay, computer scientist Richard Sutton argues that decades of AI research show the same pattern: methods that scale with computation eventually beat systems built around specialist human knowledge.

Chess is one example. Researchers tried to make computers better by teaching them how experts thought about the game. But approaches that could search and learn at scale eventually overwhelmed those carefully encoded rules.

The game Go supplied an even more unsettling example. In 2016, Google DeepMind’s AlphaGo made its famous 37th move against Lee Sedol. Experts initially thought it made no sense. It helped win the game. AlphaGo had learned something no human had ever taught it.

That raises an uncomfortable question for anyone selling expertise: if machines can get past our rules for chess and organizing work, why not our knowledge of marketing, PR, law or medicine?

Mollick describes people building elaborate prompts that walk AI through each step of a process. That’s effectively encoding their expertise into the model. With newer models, he reports, some of those instructions make the output worse than simply asking for the result.

A playbook that helps an agent today can constrain it tomorrow.

Most of what an agency knows is general know-how: how to position a brand, write a pitch or build a campaign. Models have absorbed enormous amounts of writing about those subjects. That knowledge is becoming a commodity.

The smartest pushback I’ve read comes from this historian Yuval Levin and economist Arnold Kling. Neither is an AI engineer, and both say so. But they identify a limitation the AI crowd often skips.

The first wave of progress came from models absorbing human writing. The next wave increasingly depends on models trying things, getting graded and trying again. That process works best when something can check the answer.

Code is an ideal case. A program runs or it doesn’t. Tests pass or fail. Most professional work has no comparable test suite. Levin argues that models won’t improve as lawyers, teachers and doctors at the pace they’re improving as coders. There’s no automatic grader for a brand strategy.

Kling adds that much of our knowledge is tacit. He calls it “unknown knowns”: things we know how to do but have never written down. Models can’t learn from records that don’t exist.

If they’re right, agencies are safer than I’ve been saying. Branding and PR have some of the slowest, messiest scoreboards in business. No wonder agency owners on 21 Hats don’t feel the ground moving.

I think Levin and Kling are right about how AI improves. I think they’re wrong about what it means for services firms.

The bottleneck they identify is grading. A model can improve wherever something tells it whether it was right, and only as fast as that feedback arrives.

That explains the uneven pace. Ad creative, targeting, bidding and landing pages have relatively quick scoreboards. Try something and learn within days whether it worked. Automated tools can run more experiments than any human media buyer. Brand strategy and reputation take longer because the feedback is slower and harder to interpret.

But scoreboards aren’t fixed. They get built.

The labs don’t see your client’s pipeline. You do. Every time you connect a pitch to a placement to a deal, you make another piece of work measurable. Every client conversation you record and transcribe makes some tacit knowledge available to a model.

The next round of learning can happen inside whichever firm keeps score. Levin’s argument tells you where the moat could be.

What can’t a model brute-force? Results from experiments it can’t run or observe. It can read everything published about PR. It can’t see which of your pitches landed last Tuesday, which reporter picked up the angle, or whether the article helped move your client’s pipeline.

Those results exist where the work happens.

Three changes are transforming the services industry.

First, more can be measured. A PR placement used to be the end of the report. Now you can track whether a client appears in AI answers about its category and look for connections to inbound leads and closed deals.

Second, measurement is getting cheaper. Connecting pitches, placements and deals used to require an analyst nobody wanted to pay for. Agents can handle more of that work in the background.

Third (and this is the big one) recorded results can compound across a firm in ways individual expertise cannot.

For most of the agency business’s history, learning lived in senior people’s heads. Serve 10 clients a year for 10 years and you accumulate 100 clients’ worth of experience. Until recently, that was often better than the available data: sparse, messy and hard to interpret. Spend 15 minutes in HubSpot analytics and you’ll understand why a good gut could beat a dashboard.

Now a model can use structured results to find patterns across clients. Each outcome you log can improve future decisions without someone having to remember it. Expertise grows one career at a time and walks out the door when people leave. Recorded outcomes stay and scale.

Agencies have always reported stand-ins: placements, impressions, engagement and share of voice. The outcomes clients actually pay for like pipeline, win rate, price and churn usually sit in the client’s systems, beyond the agency’s view.

Getting access to those outcomes matters more now. Measurement becomes the fuel for a learning engine.

That changes what you should build. Stop trying to encode everything you know. Start recording what happened and let the model help find the patterns. Become the source of the evidence.

Here’s the math that changed my mind about who wins:

Same revenue. One firm learns from 30 clients; the other learns from a thousand. Assuming comparable work and usable feedback, that’s more than 33 times as many opportunities to learn.

That’s the part of my old thesis I’d sharpen. Low prices do more than undercut incumbents. They buy repetitions. Lower prices bring more clients, more clients produce more results, and those results can improve the service beyond what a competitor gets from the same model alone.

You’re racing everyone else who has access to that model. The labs keep raising the baseline. Your results are the potential edge above it. Stop collecting and using them, and you drift back toward what the cheap, off-the-shelf tool delivers.

The firm of the future may be a smaller group of people who decide, vouch, keep score and answer for results, supported by a learning engine fed by thousands of clients’ outcomes.

That’s a different business from the agency most owners run today.

If you own a firm, or are thinking about investing in one, here’s the test I’d use:

If the last answer is “because we know marketing,” the Bitter Lesson is coming for you.

If it’s “because we’ve seen what happened ten thousand times and they haven’t,” you might be the one doing the disrupting.

Sources