We worked on a series of holiday films last year that were all shot in the middle of summer. The brief was simple: Make it look like winter.
No reshoots. No snow. About a week and a half to turn around 40+ VFX shots.
To a small studio like ours, this was basically an all-hands situation, and we didn’t have the benefit of pre-production chats or detailed notes as to the production itself.
Soooo, we figured it was time to roadtest an AI tool we’d been talking about, but hadn’t really put through its paces for a comprehensive workflow yet – Runway Aleph.
It’s a generative AI video tool that promises stuff like:
• Localized edits with precise input video preservation
• Make the ‘seasonal version’ of something (ie: Summer-to-Winter)
• Edit across multiple shots at once (this was our case, too)
• 1080p video output (still not 4k, though)
So this all sounds great, and was ideally what we needed, but could Runway actually deliver on these promises? We were about to find out.
We’re a modern content company with a backbone in post-production and VFX.
We know there are several ‘traditional’ ways to do a season swap, some in computer, some on-set, some a mix of both. Compositing, matte work, set extensions, practical overlays, all of these can produce strong results. However, those also take time. And budget. In this case, neither was on our side.
Yeah, we did have the discussion around the ethical and environmental impacts of using AI at this scale. As it turns out, this may have been the more environmentally friendly option when you do the math.
Runway provided an AI sustainability report, and that provided some interesting perspective.
The alternative on this project to computer-generated VFX would be the traditional ‘on-location pickups’ process, sending a camera crew to a location for extra video/winter shots, which requires gear/crew transport, flights to a cold environment, and diesel generators, which average 12 to 19 tons of CO₂e.
Not to mention, have you ever looked into the practical/on-set ‘fake’ snow used in films? Some of it is made with a chemical foam and if it doesn't get cleaned up properly, which is almost impossible to do 100%, certain synthetic agents can leach into the soil or affect water quality.
So when we boiled it down and re-visited the sustainability report, using AI here reduced this specific carbon footprint by over 99%, which felt like an acceptable trade-off.
With that in mind, we pressed ahead with Runway Aleph to generate a winter pass across the footage to give us a base to work from. It was the fastest path available. Not the cleanest. Not the most precise, but the one that made sense for the constraints.
“Turn summer into winter” sounds simple. In practice, it’s anything but.
AI got us into the ballpark quickly, but it introduced its own set of problems almost immediately.
• The output still needed cleanup and compositing.
• Resolution didn’t hold up. We were getting closer to 720
• Some passes would alter things we didn’t want touched at all. (Even though it promises NOT to do that.)
In one case, we added snow to a shot and ended up with completely different signage on a building, another time we got a different number of windows on a building than were in the source footage.
That’s the kind of thing AI does right now. It doesn’t understand intent. It interprets patterns.
Getting one shot to look right is manageable. Getting 40+ shots to feel like they belong in the same world is where things get difficult. Each AI pass behaves slightly differently. Snow density shifts. Textures change. Details drift.
So a lot of the real work became:
• Fixing inconsistencies
• Matching shots to each other
• Rebuilding continuity through comp work + color correction
At a certain point, you’re not just generating visuals, you’re stabilizing the generations.
To be fair, there were moments where AI was genuinely useful. Establishing shots with minimal camera movement worked great. Wide environments. Clean compositions. Little-to-no reflections.
In those cases, AI gave us a solid base that we could build from without fighting it too much.
That said...
We discovered the closer you get to anything important, the more fragile things become.
Faces. Hands. Foreground elements. Anything with detail or motion. That’s where AI starts to fall apart.
Small changes can create completely new problems, and fixing those problems often requires traditional compositing anyway. It becomes a balancing act. Push too far, and the shot breaks. Pull back, and you lose the effect.
In the end, AI wasn’t a magic solution. It was another tool in our post-production utility belt.
It made the work faster. It made the project feasible within the timeline and budget. It also made parts of the process more frustrating and unpredictable, and that’s the part that doesn’t show up in the super-cool demos you see on social media.
If you ask our team, the consensus is that AI is a useful tool in production, but it’s not replacing workflows. It’s becoming part of them.
For this project, it was the right call. Not because it was better, because it was more practical. AI was the right tool for the constraints, but not a magic tool. The assist from AI made the timeline possible, but it definitely didn’t make the work automatic.
Sooo TLDR (too late): There’s a gap between what AI demos promise and what real production requires. Or, as our founder, Andy Donovan, put it:
"Anyone can run a prompt, only an artist can finish the shot."
Well said, AD!
Do you need some VFX or want to talk more about post-production?
Hit us up on that 'LET'S CHAT' button right below! We promise it doesn’t have to involve AI at all (but if it does, at least you know we’ll be prepared).
