For a documentary interview piece, I was tasked with creating the illusion of a lively crowd in the background, despite the scene being filmed hours after the actual event. Budget constraints made it impossible to shoot custom plates, so the project relied on ActionVFX stock greenscreen plates of extras. Achieving seamless integration of these elements required solving a classic compositing challenge: matching the lighting between the stock footage and the filmed interview.
Traditionally, this process would involve painstaking manual adjustments to approximate the scene’s lighting and shadows. However, I employed locally run AI models to streamline the workflow and elevate the realism. Using these models, I extracted high-quality PBR material passes from the stock footage, including depth and normal maps. These passes provided stable and detailed data, which were critical for generating an accurate relight pass.
I began by recreating the rough geometry of the filming environment and matching the lighting conditions, including the color and position of light sources. The extracted depth and normal passes allowed me to precisely simulate how the scene’s lighting would interact with the stock footage, casting realistic shadows and highlights on the virtual extras.
With the relighting complete, I applied subtle color matching to ensure the crowd blended naturally into the background. The result was a convincingly populated scene that maintained the authenticity of the documentary while overcoming budgetary and logistical limitations.
This project demonstrated how AI tools can be seamlessly integrated into traditional workflows to solve complex problems efficiently. By leveraging these technologies, I was able to achieve results that would have been time-intensive or impractical using conventional methods alone. This approach not only saved time but also enhanced the creative possibilities, ensuring the final product met the highest standards of visual fidelity.