Transcript of QWEN + WAN = AI Compositing in Comfyui

Hi guys, my name is Ricky and welcome to Comfy Compositing. So, uh this week I thought we will talk about AI compositing in a video and uh I was playing around and using my previous two workflows I realized we can do a lot. Um that is something like replacing uh a car in a video. This is just an example but you can do a lot more. Um, you can relight it, you can add shadows, you can add moving reflections, um, you name it. And all of this is super simple and straightforward. Um, so let's jump in. So, as you can see on my screen, let's say we have this video and we want to replace this s this uh, sedan to the SUV car over here. And you can see the lighting is very different. The perspective is off, time of the day is off. And this is the output. We can see um the reflections are matching nicely with the environment. The relighting it did from day to night is also looking pretty good. Uh it turned on the lights of the car which also is looking amazing. Um so yeah let me show you how you can uh do AI compositing and replace anything. So we have to begin with uh a single frame uh uh workflow to make a reference frame and for this you can check out my first video about quen AI compositing. Um so in this is how it works. So we select our background. So in this case I made a clean plate of the um uh video on a single frame. Uh I just uh fed one single image in flux and asked it to remove the car and it did great. Um and over here we have the subject. Um it goes through remove background and removes everything except the subject. And over here we have a compositor um node in which we can do a rough composite. Um so something like this. Super rough. Nothing fancy. And even if it's slightly off, it's super smart to figure things out. And then this is our Quen V3L which we will need it later on. So we just need to cue it. So we have a very good prompt here which uh we will be using it in later in the video. And then over here this is just a save video if you need to but we don't need to. And then this is our preview bridge. we just cue it and it will fetch the rough comp um so that we can make paint selection. So, we go in here and make a very rough paint selection. And then moving forward, we check our preview image. And our preview image is looking good. And then yes, this is here. you copy and paste the prompt. Um, this is just a positive prompt for final image generation. And over here, let's cue this. So, as you can see, it uh warped the car a bit to match the perspective. You can see how the wheels and the car itself is completely repositioned to match the perspective in the scene and it did a pretty good job. It also turn on the lights and the reflections and yeah it looks very well done. Um moving forward um now you save the image out and then we go into the cleanup workflow which also works as the uh AI compositing workflow. So moving forward first thing we load the models and then we choose our reference image. Um next next thing is we uh choose our uh input video and over here again we run a detailed um prompt um using the reference image. So this is a reference image and I ask it to make it uh describe the image as if it's a 5-second long video of the SUV driving at night on the streets of Tokyo. Um and then it gives a very decent prompt and then over here we run Florence to run to segment the car. So it has uh done a good job here as we can see and then using the point editor we make a selection of the car. Um moving forward um this is our segmentation. So using the points editor and the segmentation tool it will make an alpha of the car and using the blockify mask and grow mask it will enlarge and uh change the shape of the car so it doesn't look like a car otherwise it will uh um the one ways to video tries to add a car if the alpha is um of the shape of the car. So when we cue this, you should get something straight out of the box working just like your reference image. And you'll see um the shadows are moving, the reflections are moving. And uh yeah, for the wheels, you can do an inpaint again of this workflow and make specify the speed of the wheels or ask it to make the wheels move. But for first attempt, I think it did an amazing job. So yeah, try this out and uh let me know what else you guys do. Thank you.