The Runway AI creative suite is cutting-edge software that is designed to help artists create stunning digital media in an efficient, fun, and streamlined way. Runway is the same company that helped create the Stable Diffusion AI model. The software is home to a large selection of video, text-to-image, and sound tools. This review will mostly focus on the video capabilities.
Table of Contents
Runway as an Applied AI Research Company
AI startup Runway is more than just an AI company, they’re also researchers building the systems that power other AI tools. Runway’s website has two main sections: research and Magic Tools. The Runway research page is a great resource if you’re interested in learning more about the AI technologies Runway is developing and how they work. The company’s research has enabled Runway to work with impressive clients like The Late Show and designers from New Balance, to name a few.
Intro to Your Dashboard
The first thing you’ll need to know is how to access your Runway dashboard and the AI tools inside. The Runway homepage has a lot of information about how to use the Magic Tools, but to actually get to the workspace, you’ll need to log in or create an account if you don’t already have one. Once you’re in, you will automatically be taken to all the AI tools. This is where you’ll be able to see all your AI “Magic Tools,” access your projects, and train AI models.
This is where you can create and manage workspaces, edit workspace settings, and invite other users or collaborators to sign on to your projects.
Under this tab, Runway has three options: generate videos, edit videos, and edit audio and subtitles. You can access all the video tools as well as the built-in browser video editor through this section.
Under images Runway has two AI tool categories: generate images and edit images. Generate images uses generative AI and edit images uses integrated AI tools. Both use text-to-image techniques.
This section has two categories: 3D and AI training. 3D is used to generate 3D textures, whereas AI training is a tool that allows you to train custom AI models.
Runway’s Two Types of AI Video Editing Tools
Across all their tools, Runway has two types of AI: generative ai and non-generative. Generative AI is used to create new visuals while non-generative helps speed up tasks. Their main generative AI video tools are called “Gen 1” and “Gen 2.” Gen 1 can only animate videos using an already existing video as a reference (the user can still use a text prompt to stylize the final generation). Gen 2 can can generate video clips from nothing using only a text prompt.
How to Edit Video Clips With Runway AI Magic Tools
There are many different tools under the “edit videos” tab in the Runway dashboard. Let’s go through each one!
Runway AI Gen-1
Gen-1 is an artificial intelligence video generator that can animate and stylize videos with only a text prompt. This generator completely changes the input video and spits out something totally new. Gen-1 is a unique tool that can help you tell a more effective story. Although Gen-1 is extremely powerful, it has yet to showcase the temporal consistency needed to create coherent and understandable video narratives.
Runway AI Gen-2
Recently launched on June 7th, 2023, Gen-2 is the newest and most powerful AI tool in the Runway suite. Gen-2 has the power to create a video entirely from a text prompt. Currently, Gen-2 can only generate 4-second long clips, but the quality, temporal consistency, and prompt accuracy are impressive. There are a few small controls in the prompt generator. For the best results, we recommend interpolating frames, upscaling the existing image, and keeping the seed unlocked (the seed lock will generate all videos in a similar style). This is one of the most groundbreaking video tools on the market today. If you are a filmmaker or content creator of any kind, we highly recommend trying Runway Gen-2. To learn more check out our short video about Runway Gen-2.
@rescuemarketers Create videos in under a minute with @Runway! Simply type in a text prompt and wait for the magic to happen. How cool is that!! #runwayai #aivideo #aivideogenerator #aivideogeneratorfromtext ♬ original sound – Rescue Marketing
This tool does what the name entails: it removes the background from any video you upload. Otherwise known as masking, this tool speeds up the tedious task of frame-by-frame editing. To use this tool, drag any video into the “remove background” tool and it will automatically open in Runway’s in-browser video editor. All you have to do is click to place points on whatever you want to mask, then play the video and watch AI do the rest. You can refine the selection with the add and subtract tools and a feather feature. This effect works best with simple shots and slow movement. You can either export the video as an MP4 with the standard plan or upgrade to Pro to export as a ProRes or PNG file.
This tool is essentially the same as the Remove Background tool. You can even switch between the two once you isolate the subject. All you have to do is click on the areas of the video you want to isolate, and then select how much blur you want in the background. This feature is perfect for content creators.
This feature can remove anything from a video without the need for a clean plate. Well… anything small. To access this tool, all you have to do is click on the inpainting tool, upload a video, and paint over the object you’d like to remove. In a matter of seconds, the AI will remove the object. This tool works well with simple shots and removing small imperfections. It also does the best job when the object you’re removing is against a simple background and isn’t in frame for a long period of time.
Color Grade (LUT)
This tool allows you to use a text prompt input to describe the colors you want in the scene. When describing the colors, the more descriptive and creative, the better. As an example, the prompt I used for the image above was “gothic deep tones with desaturated blue shadows and golden hour highlights.” It’s not the most accurate, but it’s not bad either. The biggest upside is that once generated, you can download the colors as a .cube file, more commonly known as a LUT (Lookup Table). Exporting a color generation as a LUT preset is a great way to come up with new color and style ideas that you might not have thought of before!
This feature uses generative AI to create extra frames in between the already existing video frames. When we tested this feature, we discovered that you can slow down the frame rate from 24fps up to the equivalent of 240fps at 10% speed. When you ask the AI to create this many frames from scratch, things begin to break down, but at lower frame rates, it actually does a pretty good job. This effect doesn’t require any prompting, the only input required is a percentage slider that controls how slow the video clips become. This feature is useful for creating slow-motion b-roll from clips recorded at lower frame rates.
The blur faces tool is as simple as it sounds, you upload your video to Runway and it tracks a blurred square across all the faces in a scene. Once detected, you have the option to remove any faces you want to unblur. This makes a very tedious job a lot faster with the use of AI. The only downside here is that you can’t change the shape of the blur.
Depth of Field
This tool is used to add bokeh (background blur) to your video, creating something that can typically only be achieved with a camera. Once you import your clip and generate the effect, changing the viewing mode to “editing” allows you to change the focus parameters. While this effect is visually pleasing, it doesn’t look the same as in-camera bokeh. It almost looks closer to a halation effect than anything else.
This AI Magic Tool is a little different than all the rest. If you have a video that’s already edited and you want to re-edit it or separate it back into its original shots, you can use this tool to do that automatically. When you upload your video, the AI will automatically detect all the cuts in the scene and allow you to export each clip as a separate video.
Extract Depth allows you to generate a depth map of the video you created. This can be used as a creative tool or a useful piece of information for any other digital media need. Like many of the videos, you can export this as an MP4 with the Standard plan and ProRes with the Pro plan.
The final video tool in Runway’s toolkit is Motion Tracking. This has the potential to save hours in the editing room with the use of AI tracking. To track a subject you just click on it and that’s it! If the tracking marker moves out of place, you just have to go to the point in the video where it moved and realign it, then AI will take care of the rest. After the marker is placed, you can link text or other media to the selection. This effect works best with small points on textured objects and minimal camera movement.
Runway AI Pricing
While Runway’s tools are cutting-edge, they do come with a price. Runway uses a monthly subscription (with the option to pay annually) alongside a credit-based system for their technology. There are two paid plans and one free plan:
- Free: 125 credits total, can’t purchase more
- Standard: $12 for 625 credits/month with the ability to buy more credits
- Pro: $20 for 2250 credits/month with the ability to buy more credits
- Option to buy extra credits with a minimum of 1,000 credits for $10 per purchase
- Enterprise: This is the third option for a paid Runway subscription, in order to use this you must contact Runway and work out a custom package
Should You Use AI Startup Runway to Edit Your Videos?
The answer is not a clear yes, it more depends on your needs and the types of videos you make. You can test out these creative tools with Runway’s free option or even their mobile app if the computer version is overwhelming. Overall, the average content creator can benefit from Runway’s tools, while serious filmmakers may need more. We suggest that you continue monitoring Runway while they develop their web tools and mobile app as they have the potential to open up a whole new world of AI. If you’re interested in learning more about the future of AI video content, check out these AI video tools.