In order to facilitate a smoother workflow, many people isolate individual pieces of a composition to their own layer. It’s possible to place only what you want on an isolated layer. The empty space (often represented by a checkerboard pattern inside Photoshop) is called transparency.
Transparency is the foundation of compositing, and since there are many different situations, there are many approaches to extracting images. Some—such as using an eraser or selecting and deleting—are obvious. Intermediate techniques include layer masking.
Layer masking is the best way to extract an image. It’s extremely flexible in that it supports multiple levels of transparency. Also, you can continue to touch up your mask throughout the postproduction process. Change your mind, and you can restore any part or even all of the background.
To encapsulate the technology, a grayscale image is attached to the layer, which then masks part of the layer. The term mask is very appropriate because parts of the layer are hidden, not erased. You can add to or subtract from a layer mask simply by painting on the mask. Masks are stored with the document when you save it and can be modified at any point.
Create the layer mask
The best way to learn masks is to create one. This process is fairly complex, but the payoff is big. With practice, layer masking becomes significantly easier. Let’s explore layer masks by building one:
- Open Ch04_Layer_Mask.tiff from the chapter folder. This document contains no layers.
- You can’t mask the background of an image, because it isn’t a layer. You must convert the background to a layer by double-clicking and naming the layer. You can also Option+double-click (Alt+double- click) to float the layer without assigning a custom name.
- It’s best to start a layer mask with a rough selection. Once you have the Background layer unlocked (floating), make a rough selection with the Polygonal Lasso tool.
- With the selection active, click the Add Layer Mask button at the bottom of the Layers panel. A black-and-white matte is now added to the layer. Black areas are 100% transparent; white areas are 100% opaque. These layer masks can also be blurred for soft edges or contain gray values for partial transparency.
- When you’re ready to clean up the layer mask, select your Paintbrush tool and load the default colors by pressing the D key. White will be used for areas that are opaque (solid), and black will be used for areas that are transparent. You can now toggle between black and white by pressing the X key. (A good mnemonic for this shortcut is Devil’s Xylophone.) For large areas, use a large brush. You can quickly change brush size using the [ and ] keys. You can change brush softness with Shift+[ or Shift+].
- Zoom into the document so that you can easily see what you’re working on. Start to paint using a soft-edge brush. Make sure you’re working on the mask. Look for a thin border around the mask thumbnail, which is to the right of the layer thumbnail. If you’re working on a tough area, you may want to switch to a smaller brush or reduce the opacity of your working brush. Less opacity will require more strokes to build your mask. (Think of them as coats of paint.)
- You can use the Paintbrush tool to draw straight lines. Click for the starting point, hold down the Shift key, and click for the second point. Photoshop will draw the straight line.
Using the Masks panel
By using the Masks panel, you can quickly create and edit masks more easily. The panel allows you to create and modify both raster- and vector-based masks. Additionally, you can easily adjust mask density and feathering, as well as refine a mask’s selection.
- Open the file Ch04_ Bird_Mask.psd from the Chapter 4 folder.
- Click the Masks tab to select the Masks panel.
- Click the mask thumbnail for the layer named Bird to select the mask.
- In the Masks panel, adjust the Density and Feather properties to see their effect.
- Return the Density slider to 100% and Feather to 0 px.
- Click the eye icon to toggle the masks visibility off. Click again to restore the mask.
- Click the Mask Edge button to open the Refine Mask dialog. The controls are identical to the Refine Edge dialog, except here they’re used to modify the layer mask.
- Click the Smart Radius check box, and then set the Radius slider to 9 px to remove artifacts.
- Set the contrast slider to 25% to make softer edges crisp in the selection.
- Set Smooth to a low value like 5 to improve the curved edge, and then change feather to 1.0 px to get a softer edge.
- Drag the Shift Edge slider to the left to tighten the mask until the blue fringe is gone. A value of –15% works well for this image.
- Click the Decontaminate Colors check box, and set the Output to New Layer with Layer Mask.
- Click OK to store the changes to the mask. You can always return to adjust the mask by clicking the Mask Edge button again.
Touching up layer masks
Chances are, the layer mask is pretty good, but like most things, it can always be better. There are a couple of special tricks you can use to make a “perfect” layer mask:
Smudging: By using blend modes, the Smudge tool becomes a great way to touch up your layer mask. There are two additional modes available to the Smudge tool when in the Mask mode: Lighten and Darken. You can quickly shift between these modes by using the Shift++ and Shift+– key combo. Use the Darken mode to push the dark edges of the mask in; use the Lighten mode to push the white pixels out. Set the Smudge tool to a low-strength setting, and use short strokes to push the pixels around and fill up your layer mask. As always, the [ and ] keys change brush size, and the spacebar will give you the Hand tool to pan around.
Blurring: You can use the Blur tool to control the blurring directly or use a filter. Remember, you’re blurring the layer mask only, not the image. The more blur you use, the more the edges will feather. Experiment with blurring until you get a realistic edge. If you ever go too far, you can always paint detail back in by painting with white.
Once you have the perfect layer mask, you can use it in different ways. If you keep it inside of Photoshop, the transparency is preserved. To use it in applications that don’t recognize PSD layers, you must create an alpha channel (more on that later in the chapter). At any time, you can choose to enable, disable, or permanently apply the layer mask. You access these advanced options by right-clicking the Mask icon.
Saving Selections as Paths
An alternative to layer masks and alpha channels are paths. A path is a vector-based selection, which means that paths don’t allow soft edges. You’ll find paths used most frequently for print applications (where they’re often called clipping paths). However, video folks can use paths, too. Creating a path from an existing mask or channel is easy (as is converting a path back to an active selection).
- Open the image Ch04_Refine_Edge.tif from chapter folder.
- Switch to the Channels panel, and Cmd+click (Ctrl+click) the alpha channel thumbnail to load it as a selection.
- Switch to the Paths panel, which is usually docked with the Layers panel.
- Click the Make Work Path from Selection button at the bottom of the panel. You should see a path appear with the title “Work Path.”
- Italicized paths or masks are temporary. You need to double-click these temporary paths or masks and assign a name. Double-click
the Work Path, and name it “Airplane Path.”
The path can now be exported to Adobe Illustrator (File > Export > Paths to Illustrator) where it can be cleaned up or sized. This can be useful for creating a silhouette image. Vector graphics have the advantage of unlimited scaling, with no quality loss.
You may need to go the other direction when using stock images. Because paths are vector-based, they take up much less space in terms of file size. These savings make it much more likely to find clipping paths with stock photos collections. You can Cmd+click (Ctrl+click) the path’s icon and then switch over to the Channels panel. Click the Save Selection as Channel icon, and you’re set.