Adventures in Digital Time Lapse Photography

I had time lapse photography in mind from the very beginning. Surely it would be a piece of cake to marry a digital camera to a computer, allowing the computer to control every aspect of the process; at least, every aspect beyond composition and focusing!

Five years later I have arrived at the point where I can make the time lapse movies I had been envisioning. Not all of that time (thankfully!) was spent working on this project. But it has not been as simple as it should have been. This page describes what I've been able to accomplish, and how I did it.

But I have to warn you first that I'm a Linux user, so to some of you everything here will have the look of doing it the hard way. Of course, I maintain that's the only way to learn anything; and I'm sure that you can do most of this stuff with other systems...

My Canon G1

My wife gave me my first digital camera, a Canon G1, in June of 2001. I had researched the cameras available at the time, and with the expert help of Digital Photography Review, had decided that the G1 had the flexibility and the picture quality I was looking for. After using it for a number of years, I do not regret that decision. I am particularly fond of the ISO 50 noise levels, the flip-out-and-tilt LCD screen, the Lithium Ion battery life and the infrared remote. I have even had some fun with taking short AVI movies. I have had some trouble with purple fringing around intensely back-lit subjects, and I made an unfortunate choice in the Speedlight 220EX external flash; never again will I buy a flash which only faces forward! I also would have liked finer manual focus control and a minimum shutter speed of at least 30 seconds for photographing the night sky. But, time lapse issues aside, if camera resolutions had not steadily increased over the years, I would not have found myself looking for a new digital camera at all.

Unfortunately, time lapse is not the G1's forte. The G1 supports a proprietary USB protocol as well as a subset of PTP, but the nub of the problem is this: when remotely tripping the shutter via USB, you have no control over the parameters of the shot. Every shot is taken in Auto mode, and for a while I thought I would not even be able to override the flash!

But thanks to the fact that I was using gPhoto2, I was able to delve into the source code and get at least part of my job done. Besides turning off the flash, I had one other problem to overcome: the G1 only supports USB 1.0 and in many cases, time is of the essence. It was too limiting to have to perform a remote capture, download the captured image to the PC and then delete the image (so that I didn't have to buy huge Compact Flash cards). My solution was to modify canon.c in libgphoto2 2.1.6 to

set the payload parameter to turn the flash off, and make
gphoto2 --capture-preview use "REMOTE_CAPTURE_FULL_TO_PC" as the transfer mode instead of "REMOTE_CAPTURE_THUMB_TO_PC"

These modifications are available as canon.c.diff. With them in place, I wrote a shell script (tlapse.g1) to control the time lapse process. The script expects two parameters: the number of images and the number of seconds between each image. In addition to using gphoto2 to take each image and sleeping the correct amount of time between each image, the script also:

names the images sequentially as tlapse.nnnnnn.jpg (nnnnnn running from 000001 to 999999);
uses /proc/acpi/battery/BAT0/state to keep track of the laptop battery state;
uses gphoto2 --summary to keep track of the camera battery state, and
sets /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor to powersave to maximize laptop battery life (you should dim the LCD screen as well).

The laptop battery considerations require a Linux kernel with the following options turned on:

Power management (CONFIG_PM=y)
ACPI (CONFIG_ACPI=y)
ACPI Battery (CONFIG_ACPI_BATTERY=y)
CPUFreq processor drivers (X86_ACPI_CPUFREQ=y)
CPU Frequency scaling (CONFIG_CPU_FREQ=y)

And of course you need a system with /proc and /sys support which has been correctly configured for your hardware (I used 2.6.12.3).

With this script I have been able to get about 5 hours out of my Dell Inspiron 1100 laptop (the 94 WHr battery), which coincidentally is about what I get from my G1 battery with the LCD closed. Of course, I can really only do this with outdoor subjects, since the flash dramatically decreases the G1 battery lifetime. But what can I do with my time lapse images? Enter the Gimp!

Preparing the images with the Gimp

The GNU Image Manipulation Program is one fantastic piece of software. With it, I typically crop my shots, scale them to the correct size, sharpen them up and adjust the level curve to maximize saturation. For individual photographs, you typically want to make these adjustments on a case by case basis. But for time lapse images destined to become a movie, you want to do this in batch mode, and the GIMP is there to help. The GIMP script crop-size-sharp-curve does it all (assuming you have Gimp 2.2 or better). In a directory full of images named *.jpg, executing this at a command prompt:

gimp -i -b '(crop-size-sharp-curve "*.jpg" \
         x-offset y-offset width height \
         new-width new-height \
	 x-at-y0 x-at-y255)' '(gimp-quit 0)'

will crop all of the images with parameters x-offset, y-offset, width and height; scale all the images to new-width and new-height; sharpen them with a parameter of 36, and adjust the level curve as in the following example:

where x-at-y0 is the x coordinate at the left side of the slanted line (28 in this example) and x-at-y255 is the x coordinate at the right side of the slanted line (204 in this example). This is a simple-minded but effective way to maximize contrast and saturation without losing data.

The aspect ratios width/height and new-width/new-height should be the same. If you intend to eventually view your time lapse movie on a PC, typical values for new-width and new-height would be 640 and 480. If you intend to eventually burn the movie onto a DVD to watch on a regular television set, the values should be 704 and 480.

The Gimp script produces a new set of images with the prefix "ed." appended to the original file names, saving the original images in case you want to re-edit them.

So now we have a set of clean images; how do we make an mpeg movie?

Using MJPEGtools to create the movie

I currently use MJPEGtools version 1.8.0 in a script called tlapse2mpg to make a movie out of my edited time lapse images. It takes one mandatory parameter: the directory containing the images; and one optional parameter: the number of frames to create from each image (defaulting to 1). The script is set up to create an NTSC silent movie whose name is the directory name with a suffix of .mpg. If you want to add a sound track it shouldn't be too hard; just change

dd if=/dev/zero bs=48000 count=$sec | 
    sox -t raw -r 48000 -u -b -c1 - -t wav -s -w -c2 -

sox soundtrack.wav -

but make sure the sound track is the correct length:

(number of images * number of frames per image / 29.97) seconds

The mpeg output should be suitable for burning onto a DVD.

The script requires not only the MJPEGtools package and sox, but mplayer and a patched version of yuvscaler (which is part of MJPEGtools) as well. The patches are available from http://parallel.vub.ac.be/~johan/MpegStills/; I made a new mjpegtools source directory, applied the patches to yuvscaler, did a make and then copied the yuvscaler binary into my home binary directory as ssyuvscaler ("ss" for slide show). My script uses that binary to make multiple frames from each image.

My Sony DSC-R1

So here we are, four and a half years later; I can make time lapse movies during the daytime. But I'm getting jealous of the 7 and 8 megapixels cameras that are now available, and I really want USB 2.0 support for speedier transfer (which will allow shorter time lapse intervals). What I want is a Canon G7 (the G6 still only supports USB 1.0). Only problem is: it doesn't exist! Then I discovered the Sony DSC-R1.

The R1 is a wonderful camera. The electronic view finder and the zoom and manual focus rings make me feel like I'm using an SLR again after all these years away from my old Nikon FM. I love the zebra exposure display and the fact that auto-focus is available at the touch of a button even in manual focus mode. The 10 megapixel 3:2 aspect ratio sensor allows me to crop usable images of subjects which are too far away for the 120 mm zoom, and provides me with a more usable aspect ratio with no discernible purple fringing.

But it doesn't support time lapse! Sony only supports a minimal subset of PTP's capabilities, which don't include time lapse. After a little research, I found a web site which described how to hack into the Sony RM-VD1 remote control. Armed with an idea and a little courage, I entered the world of hardware design. Sort of...

Hacking the RM-VD1

The idea is to bridge the shutter release switch on the VD1 with an electronic switch controlled from a USB port. First you have to pry the on-off switch from the VD1. Then pry the VD1 case open and remove the circuit board. Now comes the hardest part of the whole project: you have to solder two or three wires to the appropriate connections on the shutter release switch.

The orange wire is common; the yellow wire controls "partial depression" of the release (which activates auto-focus and exposure setting) and the green wire controls shutter release. But the tabs on the switches which connect them to the circuit board are tiny, and it takes far more manual dexterity than I have to solder the wires without destroying the switch! Luckily, my old friend Gene Terrell came to my rescue and did the job for me. If you try it on your own, be very careful not to short the wire to the switch case, and clean up any solder threads that may fall onto the board.

But that isn't the end of the story. After some extensive testing, I found that I had about a 33% miss rate; the computer would do it's job but the shutter wouldn't release. I clearly had an intermittent short on one of the connections. Problem was, in trying to figure out which wire was shorting, the yellow wire broke off! Not wanting to bother Gene again, I made the valiant attempt and failed. So I went to plan B: remove the switch altogether, which gives you far more room in which to work. I learned the hard way that removing the switch is easy; you just pry up with a small knife blade; but don't do it until you remove all the solder from the switch tabs! It is possible (the voice of experience is speaking here) to tear the tabs off the board. In that event you have to scrape enough of the surface off to expose a run to solder to. I had to do that to connect the orange wire to the right side of C3:

Luckily, the tab for the green wire remained in place, and it turns out that for manual focus you don't need the yellow wire at all. And in time lapse you should always manual focus... Finally, I coated the connections with rubber cement to guard against future shorting.

Computer control of the VD1

Now for the circuit. I chose a Ravar USBMod3 for an easy USB-to-serial interface. I got mine from Hobby Engineering. Since an idle serial data line is at logical high voltage, I needed a normally-closed electronic switch which would be open during idle. I could then close it by dropping the serial data line to logical low voltage by sending a string of NULs to the interface. I chose a Maxim MAX4679 electronic switch in case I needed options, but a 4677 would obviously be fine. Before soldering a permanent version, I prototyped the following circuit on a Bredblox for initial testing:

On the USBmod3:

pin 11 is tied to ground (pin 2) with a .033 mF capacitor (Radio Shack was out of them so I used a .022 and a .011 in parallel);
pin 9 is tied to pin 10 for bus-powered operation;
pin 3 (USB Bus Power) is tied to the 4679 V+ and VL (logic) pins (12 and 13);
pin 13 is tied to pin 18 for self-powering
pin 16 (ground) is tied to 4679 V- and ground (pins 4 and 5), and
pin 28 (transmit data) is tied to 4679 input for the normally-closed switch NC3 (IN3, pin 9).

In addition, on the 4679:

pin 10 (COM3: NC3 common) is tied to the VD1 common, and
pin 11 (NC3) is tied to VD1 (shutter) release.

The USBmod3 uses an FTDI FT232BM USB UART and so your Linux kernel will require support for the ftdi_sio module (USB_SERIAL_FTDI_SIO=m), which in turn requires USB Serial Converter support (USB_SERIAL=m) and "Prompt for development and/or incomplete code/drivers" (EXPERIMENTAL=y).

Because the transmit data is logically high voltage at idle, you must plug the box into the USB port of an operating PC before turning the camera on. Otherwise, the normally closed switch will be closed and the camera will take a picture as soon as the VD1 is plugged into the camera!

After a series of very successful tests, I soldered a 32 pin (machine pin) DIP socket (for the USBmod3) and a 16 pin socket (for the 4679) onto a 4.9" x 2.6" snap prototyping board, along with the capacitors, and completed the final circuit by soldering jumper wires as appropriate. I mounted everything in a plastic box, using grommets to secure the USB and VD1 cables in the holes I drilled for them:

I then modified the Canon time lapse script to

not worry about camera battery status (which we now have no access to);
modprobe for ftdi_sio;
set the baud rate to 110 with two start bits for /dev/ttyUSB0 (which you'll have to create by hand using mknod /dev/ttyUSB0 c 188 0), and
echo 10 NUL characters to /dev/ttyUSB0 to trip the shutter.

The new script is tlapse.r1. It obviously does nothing with the image files, as they are now stored on the camera until the time lapse series is finished. Then you can download them to your PC for processing using gphoto2 -R -P.

So how does it work? Like a charm!

Please send comments or suggestions to the author.

Visitors since 1/5/08: