I’ve decided to start a new category on this blog called “Dear Adobe” for the purpose of writing open letters to you. I don’t really expect you to listen, I just want to put some ideas and open-ended questions out there for people to consider. This will never be meant to bash or criticize, just some friendly suggestions from someone who’s used your software for a while.

My first letter concerns one of my favorite features in Photoshop: the layer mask. I chose this feature mainly because it’s central to my process, but also because I feel it could be so much more powerful with only a little tweaking. So here goes, my suggestion for additional layer mask functionality in Photoshop CS5 (or 6, or 7, or just someday).

I propose a feature called “masking groups”. Why do layer masks have to be flat? Why can’t a layer mask be a composite of multiple layers in a group or even the result of a smart object which contains many layers? For instance, a masking group could contain many layers which are freely editable and independent of one another. The layer to be masked would see this group as a single, flattened mask and change it’s opacity according to the sum total of all layers in this masking group. At any time I would be able to go back into the masking group and move elements around, scale them, delete them, etc.

My workaround for this has been to nest the image to be masked within successive groups, each with their own mask attached. But due to the group depth limitation in Photoshop, this system only works for up to 5 layers. Also, it does not truly behave in the same way as the “masking group” I envision. With the current workaround, each successive group and mask can only be subtractive (black). If you try to put an additive layer mask (white) over the same space as a subtractive mask on a group nested anywhere below it, it will not reveal the underlying image. This is because the groups and their respective masks are not seen as a single, flattened image by the underlying layer to be masked. In the proposed masking groups, the sum total of all masks would be treated as one. The best analogy for this would be the Smart Object. While a smart object may have many layers within it, only the layers that are actually showing will be visible in the document in which they are placed.

I’ve included a mock-up of how these masking groups might work. Masking groups are just like any other group except that they are linked as a mask to another layer or normal group, sort of like a layered version of the clipping group. In fact, this concept could be accomplished simply by allowing multiple layers to be used as the base for a clipping group. Notice in the example below how the Square, Triangle, and Circle are all separate layers in the masking group, but that their flattened state is reflected in the composite thumbnail image at the top of the group. This is the true layer mask that will effect the “layer to be masked” below it. This composite is constantly updated as layers are edited within the masking group. Also note how the white (additive) square sits above the black (subtractive) triangle, effectively masking part of the triangle and subsequently revealing part of the underlying “layer to be masked”. This is the key to the whole concept, and it’s something that is not possible using the workaround I described earlier. A masking group could be applied to a single layer, or to another regular group containing many layers to be masked at once.

An example of how a masking group might work (click to enlarge)

I hope this all makes sense. I think a feature like this could unlock all sorts of new possibilities. If this is, in fact, already possible, please let me know how to accomplish it. When I worked for you I walked over to XD one day and asked a guy about this and he said it wasn’t possible, so for now, I’m going with that assumption.

Sincerely,
-Scott

P.S. The shower head upstairs has been leaking, could I get some help with that too?

If anyone else has any suggestions concerning masking functionality in Photoshop or just want to second the notion, sound off in the comments.