IBL and Environment Maps for CGI

Image Based Lighting, also known as IBL, has been around for a while now and is a great solution for lighting and integrating your VFX scene into shot footage. In this article I like to show you how I create the photograph needed for this technique.

Low Dynamic range example of an equirectangular image also known as a Lat/long environment map. Keep reading if you like to know how I make this.

The basics

It is perfectly doable to light your scene with CG lights. I have created hundreds of realistically lit shots this way but it can become very time consuming when you want to get the fine details completely right. Thanks to IBL we can gain some time and spend it on other parts of the project. The idea is that a photograph of the place where you shot the footage contains all the needed light information and can be used with your graphics software to simulate the lighting.

There are two problems we have to overcome to be able to use this technique.

• We need the light information of the whole scene. One photograph won't give us that unless you got a very expensive 360 degrees camera. This means we need to take multiple photographs and stitch those together till we got a complete view of our scene. Each photograph needs some overlap with the previous one. The field of view needs to be big enough otherwise we have to take too many photos which will become a nightmare to organise let alone all the time you spend by taking those pictures.
• We need to capture all light information, Including the little light there is in the shadows as well as the super bright highlights of the sun. The dynamic range of digital cameras isn't big enough to capture all this information into one photo. The shadows will be crushed and the highlights will be burned, especially when storing the photo into an 8 bit image type like jpeg. The subtlety will be lost and the lighting won't look realistic.

Important remark: Keep in mind that you should have as few as possible moving objects in your scene. You will be taking pictures at different shutter speeds and moving objects will become a blur at slow shutter speeds.

Today I am using a better but slightly more expensive method than a couple of years back. In the first section I will explain in short how I used to do it and in the second section I will explain how I do it today. 

The old way

A mirror ball. Note the scratches and blemishes. Although the map will work for lighting it might be a bit rough for good reflections.

You need at least a camera which can shoot in manual mode. You have to have full control on aperture, shutter speed and ISO. A regular DSLR will do the trick. It even doesn't have to be a very expensive one.

To grab a complete environment with a regular lens you need way too many pictures to stitch together which will be very time consuming. A neat solution is to photograph not your environment directly but to shoot a spherical mirror or better known as a mirror ball. You don't want those facetted disco balls but rather the smooth chrome like balls which give a perfect reflection without breakups. The great thing about a spherical mirror is that the reflection is more than 180 degrees. It is actually almost 360 degrees with the exception of the view directly behind the ball. The drawback is that the edges are extremely distorted and a lot of information will be squeezed into a few pixels. To counter this drawback it is good practice to photograph the sphere from three or more different angles and to stitch those together.

Pros:

• Cheap, available in garden shops unless you want a perfect chrome ball with no blemishes.
• Good enough for capturing the general lighting information of your scene.
• A regular DSLR camera with a regular lens will do the trick although I recommend a long one as you will be less visible in the reflection of the sphere.

Cons:

• Every blemish on the sphere makes your picture unsharp.
• You will always be in the picture as you are being reflected as well. You can paint yourself out but it consumes time.
• Low resolution. Might not be enough for perfect reflections in your CGI image.
• Measuring the distance between the camera and the sphere is critical. You want all the angles to be taken from the same distance.

Since regular jpegs have a low dynamic range we need to shoot different exposures and join those together into a High Dynamic Range Image or HDRI. This means the camera needs to be on a tripod as long exposures will be unavoidable. You also need to undistort the spherical distortion caused by the mirror ball. There are several programs available to do this for you. I used to use HDR Shop 1.0 but it is old and there is a newer version available.

Check steps 4 to 6 in the next section below to get an idea how to set exposure and how many pictures to take.

The new way

An low dynamic range tone mapped lat/long environment map. It wasn't stitched up properly. Notice the soft edges of the buildings.

Last year I invested in a whole new setup. It is superior to above method and get's better quality environment maps. Instead of working with a mirror ball I use a Fish Eye lens. You know, those funky super wide lenses which capture between 140 and 180 degrees field of view (depending on which lens you buy).

The aim is to take pictures with multiple exposures from six different angles plus a top and bottom shot. You always need a tripod for this.

Let's look at the kit.

• DSLR: I use a Nikon D7000. This is a good midlevel DSLR. Why Nikon? Because I have already invested quite a bit of money into Nikon lenses over the past decade. Any other brand will do as long as you can shoot in manual mode.
• The lens: This is the important part. I use a Samyang 8mm f/3.5 Fish Eye lens. This lens has a 180 degrees field when used with a DX camera like the D7000. You have to do a little research to know which lens will be the best choice for your camera. This Samyang is great quality but it is a fully manual lens including the focus. It has no proper chip for EXIF data (Although the new November 2011 model does). This is not a problem as such. You need fixed settings for your pictures anyway. I know this lens is also available for other camera's.
• Tripod: A regular stable tripod will do.
• 360 degree rig: Another very essential piece of equipment. This allows you to rotate the camera in fixed intervals around the central Y-axis. A good rig will measure the distances for you. I use the Nodal Ninja 4 for measuring the intervals with the EZ-Leveler II to get the camera perfectly horizontal.
• A laptop for tethered shooting: A laptop is not necessary but it is always great to save all those images straight to hard drive and to automate the whole process. I use Sofortbild, a free Mac application from Stefan Hafeneger which allows me to take multiple exposures with one mouse click. Sofortbild works only with Nikon camera's. I know Canon has its own software.
• The HDRI conversion software: I just use photoshop to join my multiple exposure brackets into one HDR image. Sofortbuild can do this on the fly while taking the pictures but I had some trouble with it lately and I didn't bother to figure out yet why.
• The stitching software: Although you could try to stitch all the images together in photoshop, it is far more convenient to use a HDR panoramic stitcher program. I use PTGui Pro for Mac. you can give indications on where pictures are overlapping and it will try to stitch them together for you. There is also a manual mode if it doesn't manage to stitch them automatically. It also transforms the whole image into a longitude/latitude image format and saves them in a 32bit image format of your choice. I always use the radiance file format which has the .hdr extension. They seem to work flawlessly in Maya.

Now we have to put all this kit into practice.

• Step 1: Put the whole rig on the spot where you want your light being captured. This is usual the location where you want your CG element to be in the scene.
• Step 2: Make sure your camera is level. Use a spirit bubble or the build in sensor to measure this. Be as accurate as possible. This becomes relatively easy when using an EZ-leveler II.
• Step 3: Put the nodal point of the lens right above the nodal point of the rig. If you skip this step your pictures won't align when stitching them. You can check this by taking pictures at different angles. You should get no parallax shift between the two pictures. If you do then you need to adjust the placement of the camera in comparison to the rigs nodal point accordingly.
• Step 4: We need to take pictures at different exposures. Use the shutter speed to control this. Fix all other settings. Put the ISO to 100 to get noise free images and put the aperture to f22 so you get as much depth of field as possible. It will make everything sharp in the picture and that is exactly what you want.
• Step 5: make a couple of stills to check the darks and the brights. Use the histogram function in your camera to see when the blacks aren't crushed anymore and the whites not clipped. It will show you what the minimum and the maximum shutter speed should be.
• Step 6: Start with the slowest shutter speed and take a picture every two stops till you reach the fastest shutter speed. This will usually be between 5 to 8 pictures. A program like Sofortbild will take them all in one go when using tethering.
• Step 7: Rotate the camera exactly 60 degrees (This can vary with other lenses but is a good benchmark) and repeat step 6. Make sure you take the same amount of pictures.
• Step 8: Keep doing this till you have a full rotation.
• Step 9: Take 2 extra sets for zenith and nadir. You could do without but it gives a better result. The cheap rigs won't allow you to do this though.
• Step 10: You should have 8 sets of images now. Convert each set to an HDR image in photoshop. (For CS6: File>Automate>Merge to HDR Pro).
• Step 11: Import the images into PTGui Pro and go trough the whole procedure to stitch them together into an equirectangular image.
• Step 12: Export the results as a new HDR and use it in your 3D software. 

Something extra:
If you don't want to have your rig in the photograph it is possible to paint it out. It will give a nicer result but it can be time consuming.

I realize this article doesn't explain all the intricate details of the process but it should be enough to get you on your way and to try it out for yourself.

Links

Sofortbild from Stefan Hafeneger

Playtime with the Sony NEX FS700

I get more and more filming jobs than real VFX work lately which in a way is a pitty but I also see it as an opportunity to learn new things. For one of my latest clients we needed a high speed camera. Luckily for us Sony released the NEX FS700, an affordable 1080p camera with amazing frame rates.

Not a Review

First of all I like to say this is not really a review but merely my thoughts after using the camera for two days. I didn't have time to do an in depth test or read the complete manual. We only used the camera for shooting at high frame rates as we wanted cool slow motion shots.

The Kit

The camera was a rental and came with the kit zoom lens which isn't the best glass around. It did do ok but having Nikon glass on my DSLR we are used to better. It has an apperture of 3.5 at 18mm and 5.6 at 200mm. There are lens adapters for sale so it is possible to use the glass you already own. We used a 32GB SDHC memory card which records the video in AVCHD. A great feuture is the build in ND filter. It also comes with a XLR boom mic. This is hardly important for a slow motion shoot but a great feature none the less. 

Handling the Camera and Shooting Slow Motion

If you ever used a video camera like a HVX200 then you will quickly get around using the FS700. It took us less than an hour to get used to handling the camera in manual mode.

We had the PAL version of the camera so the available frame rates for me were 200, 400 and 800 fps. The NTSC version does 240, 280  and 960 fps. At 200 fps you get full HD. The resolution goes down a bit at 400 fps but it still looks great. The resolution and quality at 800 fps really drops. So 800 fps is great fun to test but it is no longer usuable if quality is an issue. You can really see the difference, even on the LCD screen of the camera as the image gets cropped to record only a smaller section of the sensor.

Some remarks when shooting slow motion:

• You need extra light. The shutter speed goes up quite a bit which lowers the light input considerably. You could adjust the ISO to a higher value but that really brings out a lot of noise in the image.
• Don't use lights which are flickering like fluorecent tubes. We did the test and the flicker is horrible. Tungsten and sunlight are much safer here. We haven't checked HMI lights but as far as I know they flicker as well even with high frequency electronic ballasts. In a pinch you can get rid of bad flicker with the FurnaceCore tools from The Foundry but why fix it in post if you just avoid making the mistake while shooting.

The camera records the footage to a buffer which is only big enough for around ten seconds of footage (It depends on the frame rate you choose). After recording your scene, the camera needs time to write the images to the memory card which can take half a minute or so. It is stored as 25 fps footage so you can immediatly play it back on the camera to see if your shot is ok. We connected the camera with a HDMI cable to a monitor so the client was able to look as well without everybody having to bend over the little built in view screen.

The result of the Test

As a picture is worth a thousand words and a video even more I like to show you the video we made after an evening of test shooting. We used a 1K tungsten lamp to light the scene and shot various items at 400 fps. This is not the footage we shot for our client.

Review: Rode Videomic Pro and Stereo Videomic Pro

Sound is 50% of a movie. Most people stop watching your movie when the sound is badly recorded. With the whole revolution of shooting on DSLRs, new types of microphones have been designed. Rode has come up with the Videomic Pro and the Stereo Videomic Pro.

We have those microphones available to us so what is better than making a blog post about it? That's right, a video on our bbrevisited channel! Watch it and discover how these Rode microphones perform under different circumstances.

Some afterthoughts

I must say that it is rather annoying that the Nikon D7000 has no sound metering. This caused some of the recordings to be distorted as the signal was clipped. I did put the internal amplification on medium. The distortion could have been avoided by leaving the setting on automatic. That aside, the rest of the test went rather well.

The cardioid patterns of both microphones are very useful to pick up less sound behind the camera and more in front. They still pick up sound from a quite big environment though and do not filter as much out as a shotgun or a lavalier mic. I can agree that for the Stereo Videomic Pro this is rather an advantage than a drawback as it is designed for grabbing that environment. The big advantage for both is that they can record straight into the camera so no separate sound recorder is needed and they do beat the built in microphones of DSLRs hands down.

I found that the Videomic Pro picked up a bit more bass than the Stereo version but both performed well. The Stereo Videomic Pro records two channels which no built in mic can do.

Battery life is really not an issue. More than 70 hours on the Videomic Pro and even more than 100 hours on its stereo brother is really good performance. It will last you many days if you turn off the microphone every time you are not recording. Both microphones are also quite compact so they won't take up too much space in your bag.

If you are serious about recording sound properly I really recommend these two models from Rode.

Review: GoPro HD HERO2

The GoPro camera's are immensely popular. These low cost, tiny and robust camera's are quite unique and since I use one of them for my hobby projects I have decided to give it a review. 

There are three editions available for the GoPro HD HERO2 which are the Outdoor, motorsports and surf edition. I purchased the motorsports one as it comes with a suction cup to fix it to windows and other smooth surfaces like cars.

What's in the box of the motorsports edition

The content of the motorsports edition.

• 1 GoPro HD HERO2 camera
• 1 Waterproof housing
• 1 HD skeleton back door
• 1 Lithium Ion battery
• 1 Suction cup mount
• J-Hook buckle
• 2 Curved adhesive mounts
• 3 Flat adhesive mounts
• 1 three way pivot arm
• 1 USB cable
• 1 Booklet with instructions
• 3 GoPro stickers (for the real fans)

GoPro does not deliver an SD card with the package so make sure you order one yourself. I got myself a SanDisk Extreme HD Video (30 MB/s) 16GB which does a fine job at recording HD video and images.

The camera

The camera compared to the size of a 1 Euro coin, 1 Pound coin and a Quarter coin. Also check out the informative display.

The back of the camera with the Lithium Ion battery popped out. (I blurred out the serial number).

Most important part of the whole kit is the camera. It is small and lightweight and could easily fit in your pocket although I do not recommend so as it might damage the lens. The housing is made from sturdy plastic which feels soft to the touch. There are two buttons to control it, an informational LCD screen, a built in microphone, an HDMI out, a TV out, a USB connector and a jack for an external mic. At the back there is also an interface to connect an extra battery pack or LCD view screen which can be purchased as options. There are also four red LEDs which inform you if the camera is filming.

Some specs for the techies (yeah, I always like specs):

• 11 megapixel sensor
• f/2.8 fixed focus
• 170, 127 and 90 degrees field of view
• PAL and NTSC frame rates.
• 1080p at 25fps (PAL) and 30 fps (NTSC)
• 960p at 25fps, 50fps (PAL) and 30fps, 48fps (NTSC)
• 720p at 25fps, 50fps (PAL) and 30fps, 60fps (NTSC)
• WVGA (848x480) at 50fps, 100fps (PAL) and 60fps, 120fps (NTSC)

The camera records in the H.264 codec which is similar to a DSLR.

You might have noticed the different frame rates with PAL and NTSC. Although in the old days PAL and NTSC used to be incompatible because of different resolutions and frame rates it does not matter too much anymore. As long as you are aware at which frame rate you you are shooting all will be ok. So if shooting slow motion and the end result needs to be 25fps you might as well pick 120 fps as it might give a nicer result.

The battery can be charged trough the USB cable but you need the camera to do so. You need to get the optional battery back pack if you want to charge the battery without the camera.

The USB cable is also used to unload the movies and images or you can always take the memory card out and put it in a card reader.
The image quality is quite crisp and clear in daylight situations. It is only at low light that imagery becomes noisy but then again most camera's have that problem. It is impossible to change the ISO value, shutter speed and aperture on this camera.

The interface

The interface has had quite an upgrade from the old HERO camera. There is some true readable information now instead of just seeing a digit. This increases the user friendliness as you do not have to carry a manual with you at all times.

It also displays useful information such as how much remaining space there is on your memory card, at which resolution and speed you are filming, the battery status and if the camera is in filming mode or picture mode.

The front button moves you trough the menu while the top button is used for selections or starting/stopping the camera.

Not only can you film but there are also several options to capture still images:

• Single
• 10 frames burst
• Time lapse
• Self timer

Accessories in the box

The accessories give abundant options on how to attach and point your camera. The pivot arm has big enough handles to tighten and loosening the joints so no screw driver needed to change the configuration.

I haven't tried the self adhesive mounts yet so I can't tell you if they stick well or not.

The waterproof housing is very sturdy and keeps the camera clean in dirty or wet situations. The two buttons are easily accessible with big buttons from the outside so no need to take it out every time you start or stop it. 

My experience

Again, the new interface is a huge improvement to its usability. I worked with the old one before and it was always a problem if you didn't have the manual with you unless your brain is good at memorizing these things. With the HERO2 there is no more memorizing what those digits mean but clear information on how the camera is set up. The display also shows a timer when shooting and how many clips are already on the memory card.

The interface sound is on by default but I find the bleeps rather annoying so it is possible to turn it off in the settings menu. You have to dig deep into the menus to find the option but then again it is impossible to have an extensive accessibility with only two buttons and it is not something you have to change all the time.

The recorded sound is really muffled when the camera is in the waterproof housing. Make sure to use the skeleton back door if your sound quality needs to be better but keep in mind that it is no longer waterproof when you do so.

The menu's on the display are clear and readable although the LCD is not back lit so make sure to have a light ready in low light situations if you need to change some settings.

The suction cup works great. Do remember to clean your surfaces before attaching it. Dust can decrease the suction strength over time as air might seep in. And although the waterproof housing is incredibly strong and protects the camera really well, it is never fun when it falls off while driving a car.

We also have great fun with the high frame rate option. Unfortunately not at full HD but still a great option to have when doing crazy stuff like jumps. I think there are almost no cameras at this price point which can shoot at 100 fps at this moment in time.

A drawback is that you can't see what you are filming exactly so framing can be tricky. GoPro has thought of this and provides a LCD backpack which makes that possible. Keep in mind that the LCD screen lowers the battery performance.

The battery lasts about two to three hours which is not that long and the camera does get a bit hot while in operation so do not be alarmed when that happens.

Price/Value

The camera is priced at $299 when ordering from their site. I bought mine in the local shop at  309 Euro. I think the camera is very good value for money considering what you can do with it. Exposure is completely automatic so not much control there but the electronics inside do a pretty good job at getting it right.

The camera itself is also quite sturdy as I had a colleague dropping his camera into the water without the waterproof housing and after extensive drying it did still work (maybe he was just plain lucky).