Last week, I started connecting the AI systems I’ve been building for video editing. Then I shared the results with our producer on Builders of Charlotte, and his feedback pretty much boiled down to:
“I like the thumbnails.”
“The clips lack complete thoughts.”
My defense was that I’d done that on purpose:
I want aggressive edits, open questions, and a reason to watch the full episode. I casually describe them as “half-baked thoughts or cliffhangers” in our conversation.
But his feedback raised a question I’m still working through:
How much does a teaser need to deliver before it earns the next click?
I’m not really trying to build one AI editor, so I keep going back to that question over and over.
I built several specialized agents and seeing what happens when their decisions start feeding into one another:
Story agent: studies the footage, finds the governing question, and decides what the edit is actually about.
Assembly agent: turns that story into a cut using existing footage, generated material, and documentary structure.
Visual agent: turns the premise into a visual promise through hooks, thumbnails, and supporting imagery.
Performance agent: studies what worked, what didn’t, and decides what the system should test next.
Distribution agent: packages the finished idea into something useful to publish and share.
Individually, none of those agents is especially interesting. But the behavior starts emerging when you connect them:
The story agent gives the visual agent something specific to promise.
The visual decisions constrain how the edit should be assembled.
The finished edit gives the performance agent something to evaluate.
The performance agent changes what the system tries on the next video.
Each agent leaves behind decisions and expectations that the next one has to carry forward.
That also opens an opportunity for human feedback to propagate through the system.
Our producer’s feedback then becomes a constraint I can hand back to the system:
Deliver one complete thought inside the short, then leave the larger question open.
Now the question is whether my original approach is performing better.
YouTube Studio shows an average view duration of 18 seconds across the channel, so I intentionally made a 17-second short.
My hypothesis is that a shorter, open-ended edit would get people through more of the video and make them curious enough to watch the full episode.
The AI-edited short reached roughly 1,100 views in Studio, ahead of the 934 reported for our previous most-viewed manual short.
Our AI edit averaged 61% of its 17-second runtime, compared with 29% of the 44-second manual short.
That looks like a huge improvement until you look at the seconds.
The AI edit averaged 10 seconds watched; the manual edit averaged 12.
Whether that’s better depends on the goal.
My current thinking is that if your business earns money from ads, longer visits can mean more opportunities to show them. Someone watching 90 seconds of a 6-minute video could be more valuable than someone watching 10 seconds of a 17-second video.
We don’t make money from ad impressions, or from anything, really.
So for us, keeping someone on the platform longer is, to some extent, irrelevant. I want to give the right person something useful and make them interested in the full conversation.
The thumbnail data is similarly messy.
Our AI thumbnail had a 4.76% click-through rate, but that represented 1 click from 21 impressions.
Another manual short had 4 clicks from 113 impressions.
Sure, the AI thumbnail has the better click-through rate, but the evidence is much thinner. The data analyst in me can’t call it better just because the percentage is higher.
And my original goal wasn’t retention or thumbnail CTR anyway.
It was to send people to the full episode.
So far, I’m NOT seeing that in YouTube.
For the next version, I’m keeping the tight edit but incorporating Seth’s constraint: give viewers one complete thought worth their time, then leave a larger question that makes continuing feel worthwhile.
You can judge the creative side yourself:
Here’s the manually crafted short.
And here’s the AI-edited, human-reviewed short.
And since I’ve spent this entire newsletter quoting his criticism, Seth deserves a proper shoutout.
Seth Wright, our producer on Builders of Charlotte, in his own words:
After more than a decade working internationally as a Director & DOP, I’ve recently moved back to North Carolina to reconnect with my roots and build my career closer to home.
I help businesses and brands communicate their story and grow through commercials, social media, branded content, and marketing productions.
Taking projects from ideation through final delivery.
My background spans commercial, documentary, social-first, and large-scale productions, with an approach that combines strong visual storytelling, strategic thinking, and an editor’s mindset.
I’m currently open to full-time opportunities and contracted work with brands, agencies, and businesses looking for a creative partner who can take an idea and turn it into effective content.
Back home. Ready to build.
Seth currently has capacity for more client work, so if you’re looking for an experienced director, cinematographer, or producer, do reach out to him!
And the full Builders of Charlotte episode is live.
Benyamin Holley, Head of GTM at Apero Advisors, joined us to talk about testing an offer, finding the right buyer, and asking for commitment before writing code.
There’s actually a nice connection between Benyamin’s argument and this entire editing experiment.
His point is that you shouldn’t spend weeks building something before figuring out whether anyone wants the outcome.
I’m discovering the same thing one layer down with the editing system.
Before I make the agents better at producing something, I need to be clear about what I actually want them to optimize for.
Retention?
Watch time?
Clicks?
Full-episode conversions?
Optimizing for one can easily make another worse.
If you’re building something and trying to figure out whether someone will actually pay for it, watch the full conversation with Benyamin here.
—Alan, Colin & the Torta Studios Team