Streamline Your Video Editing Workflow with Runway AI: A Beginner’s Guide

How to Edit Video Clips With Runway AI Magic Tools

There are many different tools under the “edit videos” tab in the Runway dashboard. Let’s go through each one!

Runway AI Gen-1

Gen-1 is an artificial intelligence video generator that can animate and stylize videos with only a text prompt. This generator completely changes the input video and spits out something totally new. Gen-1 is a unique tool that can help you tell a more effective story. Although Gen-1 is extremely powerful, it has yet to showcase the temporal consistency needed to create coherent and understandable video narratives.

Runway AI Gen-2

Recently launched on June 7th, 2023, Gen-2 is the newest and most powerful AI tool in the Runway suite. Gen-2 has the power to create a video entirely from a text prompt. Currently, Gen-2 can only generate 4-second long clips, but the quality, temporal consistency, and prompt accuracy are impressive. There are a few small controls in the prompt generator. For the best results, we recommend interpolating frames, upscaling the existing image, and keeping the seed unlocked (the seed lock will generate all videos in a similar style). This is one of the most groundbreaking video tools on the market today. If you are a filmmaker or content creator of any kind, we highly recommend trying Runway Gen-2. To learn more check out our short video about Runway Gen-2.

@rescuemarketers Create videos in under a minute with @Runway! Simply type in a text prompt and wait for the magic to happen. How cool is that!! #runwayai #aivideo #aivideogenerator #aivideogeneratorfromtext ♬ original sound – Rescue Marketing

Remove Background

This tool does what the name entails: it removes the background from any video you upload. Otherwise known as masking, this tool speeds up the tedious task of frame-by-frame editing. To use this tool, drag any video into the “remove background” tool and it will automatically open in Runway’s in-browser video editor. All you have to do is click to place points on whatever you want to mask, then play the video and watch AI do the rest. You can refine the selection with the add and subtract tools and a feather feature. This effect works best with simple shots and slow movement. You can either export the video as an MP4 with the standard plan or upgrade to Pro to export as a ProRes or PNG file.

Blur Background

This tool is essentially the same as the Remove Background tool. You can even switch between the two once you isolate the subject. All you have to do is click on the areas of the video you want to isolate, and then select how much blur you want in the background. This feature is perfect for content creators.

Inpainting

This feature can remove anything from a video without the need for a clean plate. Well… anything small. To access this tool, all you have to do is click on the inpainting tool, upload a video, and paint over the object you’d like to remove. In a matter of seconds, the AI will remove the object. This tool works well with simple shots and removing small imperfections. It also does the best job when the object you’re removing is against a simple background and isn’t in frame for a long period of time.

Color Grade (LUT)

This tool allows you to use a text prompt input to describe the colors you want in the scene. When describing the colors, the more descriptive and creative, the better. As an example, the prompt I used for the image above was “gothic deep tones with desaturated blue shadows and golden hour highlights.” It’s not the most accurate, but it’s not bad either. The biggest upside is that once generated, you can download the colors as a .cube file, more commonly known as a LUT (Lookup Table). Exporting a color generation as a LUT preset is a great way to come up with new color and style ideas that you might not have thought of before!

Super-Slow Motion

This feature uses generative AI to create extra frames in between the already existing video frames. When we tested this feature, we discovered that you can slow down the frame rate from 24fps up to the equivalent of 240fps at 10% speed. When you ask the AI to create this many frames from scratch, things begin to break down, but at lower frame rates, it actually does a pretty good job. This effect doesn’t require any prompting, the only input required is a percentage slider that controls how slow the video clips become. This feature is useful for creating slow-motion b-roll from clips recorded at lower frame rates.

Blur Faces

The blur faces tool is as simple as it sounds, you upload your video to Runway and it tracks a blurred square across all the faces in a scene. Once detected, you have the option to remove any faces you want to unblur. This makes a very tedious job a lot faster with the use of AI. The only downside here is that you can’t change the shape of the blur.

Depth of Field

This tool is used to add bokeh (background blur) to your video, creating something that can typically only be achieved with a camera. Once you import your clip and generate the effect, changing the viewing mode to “editing” allows you to change the focus parameters. While this effect is visually pleasing, it doesn’t look the same as in-camera bokeh. It almost looks closer to a halation effect than anything else.

Scene Detection

This AI Magic Tool is a little different than all the rest. If you have a video that’s already edited and you want to re-edit it or separate it back into its original shots, you can use this tool to do that automatically. When you upload your video, the AI will automatically detect all the cuts in the scene and allow you to export each clip as a separate video.

Extract Depth

Extract Depth allows you to generate a depth map of the video you created. This can be used as a creative tool or a useful piece of information for any other digital media need. Like many of the videos, you can export this as an MP4 with the Standard plan and ProRes with the Pro plan.

Motion Tracking

The final video tool in Runway’s toolkit is Motion Tracking. This has the potential to save hours in the editing room with the use of AI tracking. To track a subject you just click on it and that’s it! If the tracking marker moves out of place, you just have to go to the point in the video where it moved and realign it, then AI will take care of the rest. After the marker is placed, you can link text or other media to the selection. This effect works best with small points on textured objects and minimal camera movement.