The Belgian VFX Guy: matte

This is part of a series on the basic elements of Visual Effects. Each post will talk about a certain element which is one of the basic bricks used for building VFX shots.

In this eighth and final post in the Back to Basics Series I will talk about compositing.

A CGI image with a split trough the middle which shows color channels in the top part and the alpha channel in the bottom part. Alpha channels are the key to compositing.

Where all the previous posts talked about generating elements, this one will talk about combining these elements into a final image. For people who don't know the word compositing I always compare it to "Photoshop with moving images". We layer up different images and combine them into one with the illusion that the final image looks as if everything was filmed at the same time.

Back in the old days

Although digital compositing is a very powerful tool in filmmaking, compositing itself did start out as an analog process with the use of the optical printer. The first optical printers were created in the 1920's and were improved up till the 1980's. At first the effects were quite simple like a fade in or fade out but the effects became increasingly complex with the addition of things like matte paintings and blue screen effects. All these elements were combined by exposing the film several times.

This also explains the necessity of using mattes. If the film would be exposed twice it would create a double exposure unless the area where the second element needs to come is masked out. For blue or green screen shots these mattes were made with high contrast film and the use of a color filter to separate out the background color. In some other cases mattes were painted by hand which could result in chattering edges.

Films like 2001: A Space Odyssey and Star Wars had a staggering amount of success with this optical workflow.

The digital era

At the end of the 1980's digital compositing started to take over. It has two major improvements over its optical counterpart:

In the optical process it is necessary to create the several layers like the mattes by copying it from one film strip to another. Each copy degrades the image and adds extra noise to the result. In the digital realm a copy is 100% the same as the original (unless you resample or recompress them but that is usually not done).
The second problem is that film going trough a machine can drift a bit which could result in chattering mattes or halos. If done correctly digital compositing does never suffer from this.

When we need to composite for film a digital intermediate is created by scanning the film. When shot on digital cameras the source is already digital and does not need scanning.

Popular compositing packages today are Nuke, Digital Fusion and After Effects. The first two are node based packages while After Effects is a layer based package and works in a similar way as Photoshop does. Node based systems are represented by a flow chart where every node applies an operator to tree. Life action and CGI compositing do benefit more from this method while motion graphics are usually done in layer based packages.

A Nuke node based network.

Layered based compositing in After Effects.

Color and Alpha channels

Let's have a look at how compositing works in the digital realm. A digital image consists out of pixels which are represented by a combination of the three primary colors red, green and blue. This combination can create a broad amount of other colors available in the visible spectrum. Every color is stored in a channel.

In order to combine different images, it is necessary to have a fourth channel which is called the alpha or matte channel. It contains the transparency information of each pixel and this dictates how the result looks like when one image is put atop another image. It works just like its analog counterpart.

There are two ways to show an image with an alpha channel. The premultiplied one where the value of the alpha channel is already multiplied by the color values or the unpremultiplied one where the color values have their original values. This is very important to understand as failing to grasp this concept will possibly make composites look horrible with weird matte lines as a result. Check what your software package expects you to work with.

CGI images are usually premultiplied automatically when rendered.

A magnified CGI image. Look at the anti-aliased edges. This image is premultiplied.

The corresponding alpha channel image of the above image which has the same anti-aliased edges.

When unpremultiplied the edges become aliased.

The Extra's

Composting packages do not only put one image over another but have a vast set of tools available:

Rotoscope tools: With these it is possible to create mattes to isolate parts of the image. These include also operations to modify edges or to paint over existing imagery.
Retiming tools: All kinds of operators to manipulate the speed of the sequence.
Color correction tools: Operators to analyze and change the color of the image. Very useful to match foreground and background elements.
Filters: All kind of filters like blur, motion blur, noise, denoise and many more.
Keying tools: These are useful for matte extraction from images with blue or green screen but also other methods are available like luma keying.
Layering tools: from simple operators to put one image over another to operators for premultiplication.
Transformation tools: Operators such as translate, rotate, scale, resize and distortions.
Tracking tools: Tools to track objects in a scene or to stabilize a shot.

Although compositing is a 2D process, nowadays most packages support 3D environments. This adds extra functionality as it is possible to have proper parallax, use projections and makes the integration of particle systems really useful.

Another rather recent development is stereo conversion tools. It allows regular images to be converted to stereographic images. Note that I use the term stereographic instead of more common and commercial term 3D. Stereographic images are not truly 3D as the audience can not look behind objects by moving their head. I therefor prefer the term stereographic.

This concludes the eighth part of the VFX Back to Basics series.

Make sure to subscribe (at the top) or follow me on Twitter (check the link on the right) if you want to stay informed on the release of new posts.

In this series:

1. What is rotoscoping?

2. What is 3D modeling?

3. What is tracking and match moving?

4. What is 3D animation?

5. What is shading and texturing?

6. What is lighting and rendering?
7. What is matte painting?

This is part of a series on the basic elements of Visual Effects. Each post will talk about a certain element which is one of the basic bricks used for building VFX shots.

In this seventh post I will talk about matte painting.

So far in this series I have used images and photos to illustrate what I am writing. These have been always made or photographed by myself. Although to me matte painting is one of the most amazing tools in film making I do not master it enough to give good examples. I am still a very novice painter. I do not like to steal other people's work to display them on my blog so for now this article will lack the proper visuals. There are some beautiful images out there by accomplished artists. Just google "matte painting" to see some examples.

If you are an artist and like to share your paintings to be featured here as examples please feel free to contact me. You will be properly credited. Mind you that these have to be of good quality. Now onto the article itself.

Introduction

From all the basic VFX elements we can conclude that matte painting is the oldest trick of the trade. It may be old but it is still alive and kicking although the way they are made has undergone a digital upgrade.

The first known matte painting was made as early as 1907 by Norman Dawn. Dawn painted buildings on a glass pane which was placed in front of the camera. The set behind the glass pane was visible trough the unpainted areas. This way it was possible to record the image in just one exposure. This technique is also known as glass painting.

Dawn also worked with the double exposure technique where the the transparent areas were painted black. This left the those areas unexposed and allowed them to be exposed a second time to add in the other elements. The technique was so much cheaper than building sets that it has been very successful over the last century of film making.

Matte painting today

In 1985 matte painting started to move into the digital realm. It allowed for more accuracy, realism and speed to produce the paintings. Today Photoshop is the number one application to create matte paintings but gets more and more support from 3D programs to create a basis for the painting.

Matte painting is still the most cost efficient way to make background shots which do not have extensive camera movements. Even with a simple pan or travel it can still work. Once the the camera starts moving around things it is better to consider to build everything in 3D.

Let's have a quick look at a couple of most used methods to make a matte painting.

1. Start a painting from scratch.
Starting a painting with an empty canvas is quite tricky and is usually done by trained artists. They start of with a sketch or a very rough drawing and keep on refining it till enough detail is present. They pay a lot of attention to color and light as this gives the mood of the painting. This method is mostly used to generate totally made up landscapes.

2. Starting from footage or a photograph.
Usually there is already some information available like some footage from the set. The matte painting just needs to extend or modify this base. This method can be used to modify a perfect building into a ruin or vice versa of course. Photographs can be used to add elements to all of this. The artist recolors these images so they fit in perfectly.

3. Starting from a 3D model.
The method to start from a 3D model is becoming more common. It allows to get the perspective right from the start as the 3D program calculates that for you. Instead of texturing the model inside the 3D package it is used to paint over it in the matte painting.

Matte Painting, a classical artistic job.

Although matte painting starts to move more into the 3D realm it is mostly a very artistic profession. Most of the matte painters have had a classical education in art. It therefor demands a lot of training to get it right. Just knowing photoshop doesn't do the trick. Knowing color and composition are essential skills.

This concludes the seventh part of the VFX Back to Basics series.