When I was a little kid, I remember being asked whether I wanted a lot of little things or one big thing for Christmas. I wisely chose the latter, and found myself rewarded with a dream gaming systemthe Atari 2800. Adobe's After Effects team was clearly confronted with a similar dilemma. Deciding that we'd be better off with one big thing, Adobe converted the entire After Effects CS4 application pipeline to 64-bit compatibilitya change requiring most of the attention of most of the development team, in order to produce the fully 64-bit native After Effects CS5.
Other folks were also hard at work on behalf of the application. While trying to solve the problem of rotoscoping (the tedious process of making a frame-by-frame video selection), a couple of the original team members came up with the revolutionary Roto Brush. Some third-party developers also stepped up, creating valuable additions that ship with CS5, such as major upgrades to Mocha AE and Color Finesse, as well as the addition of the Freeform plug-in from Digieffects.
Choosing the top five features this time around wasn't so difficult, since the After Effects team focused on a few big new features instead of many smaller ones. It's more a question of how and why these add up to a big upgrade, which After Effects CS5 definitely is.
This feature has caused the greatest excitement (and the most questions) among users of previous versions of After Effects. To paraphrase Arthur C. Clarke, Roto Brush is "sufficiently advanced to be indistinguishable from magic." This tool won't magically erase your rotoscoping troubles, but it represents a quantum leap forward in moving image selection, opening up new possibilities that you might not have even considered for working with footage.
Here's how it works. Place any clip into a composition and double-click to open it in the Layer viewer. When the Roto Brush tool becomes available in the toolbar, click it to make it active. Now comes the strange part: You use this brush to paint a shape that simply suggests the form of the item you want to remove, working inside its boundaries (see Figure 1), and the tool attempts to understand on its own where the exact boundaries are (as in Figure 2).
In other words, you don't try to paint along the outline of the figure being rotoscopedyou sketch its basic shape with a brush that makes a kind of blotchy green paint stroke. Typically you may do a little touch-up by holding down the Option key and painting out (with a blotchy red stroke this time) anywhere that Roto Brush has guessed wrong. Then you preview forward and backward in time, making these same sorts of procedural additions and subtractions to the matte.
Figure 1 That pink outline was generated with just a couple of Roto Brush strokesno careful edge selection needed.
Figure 2 The result, with the automated Refine Matte also enabled, has picked up a startling amount of edge detail, with no need for fine adjustment by the artist.
What's happening here? Roto Brush isn't relying on something so simple as contrast (like a Luma or Extract Matte) or even color (like Linear Color Key or Keylight). Instead, it combines several methods of understanding what types of forms you might want to isolate and how they change over time. It's a bold first step at making the computer solve a tedious task that most of us often wish computers could do.
Does it work? Because of its name, the assumption is that Roto Brush is for rotoscopingthe process of isolating a moving element or figure completely from the frame. While it's certainly capable of doing that, particularly with an object that's well defined and doesn't have a lot of extraneous surface detail, Roto Brush tends to have more difficulty with fine detail such as hair and motion blur, as well as little gaps; for example, those triangular shapes under a slightly bent elbow.
The real power of Roto Brush comes from using it subtly, and in combination with other tools. For example, isolating a figure to color-correct just that single item is now a lot easierthere's no reason that the selection needs to be perfectbut the tool takes you far beyond what you can do with a quick hand-drawn garbage matte. Roto Brush plus a more sophisticated tool such as Timewarp, which prefers the element to be isolated, is a powerful combo. And even an ordinary green-screen key occasionally has elements that won't key properly, perhaps because they have too much green spill in themRoto Brush plus Keylight can save you a lot of work.
So even if Roto Brush doesn't eliminate 100% of my rotoscoping work, I'm not complaining if I see a 50[nd]70% reduction in the amount of roto I need to do by hand.