SoFoBoMo where are you?

Based on the idea over on Sam's blog, please add yourself to the map if you've been in on the whole SoFoBoMo experience. You can also link to the map with this url : http://www.frappr.com/sofobomo

cow creek

A break from the constant barrage of SoFoBoMo posts. I spent a bit of time this evening looking through the final day of shooting that I did in Death Valley. I'd woken up early that morning, with a vague plan to go back to the sand dunes near Stove Pipe Wells for a final sand shoot. I'd met a few people that I was going to shoot with and we all got together at 4:30am. The wind was howling. But they were all feeling hardy and still wanted to head to the dunes. So I wished them luck!

Instead I went with another group down towards the salt flats near Cow creek. I'd scouted that area on the Saturday when I'd arrived and found some really beautiful fan shapes in the salt. I'd walked out without my camera, towards the center of the valley on that first day, just to get away from everything and listen to the wind. The water and wind had blown rivulets into the salt pan and the shapes were achingly delicate. I'd hoped to come back when the light was good and marked some GPS co-ordinates so I could find it easily.

So on the last morning, I got my chance to go back and it was breathtaking all over again. Such graceful sweeping patterns leading off to the far mountains. The sand from the Stove Pipe Wells dune fields was filling the air in the distance, blowing down the valley and obscuring the hills but maybe adding some interesting mystery to the scene, too. The wind was whipping around my ears and it was a joy just to be up and out there to experience it.

flickr set parser for GPSVisualizer

Geekier than normal post today. If you aren't interested in python programming, flickr's API and GPS display of images in Google maps or Google Earth then I'd stop reading now. Here's another picture from Death Valley. See you again tomorrow.

Right. Still here ? I've been using GPSVisualizer to combine a GPS tracklog, with geotagged images on flickr. Images are entered into GPSVisualizer as a series of CSV values, in a fairly flexible format. The first line provides the layout:

latitude, longitude, name, url, thumbnail, desc

Each line after that is one entry for an image, with the location in decimal degrees, followed by various text fields containing the description, title and links to the actual image. For example:

36.366953, -117.391867, "shot up car", "http://flickr.com/photos/mcgregorphoto/2355449153/", "http://farm3.static.flickr.com/2323/2355449153_c733f5ee1e.jpg", "a long way from nowhere, burnt out and shot up, just off the access road"
36.442947, -117.435447, "half way point", http://flickr.com/photos/mcgregorphoto/2356283640/, http://farm3.static.flickr.com/2388/2356283640_be78036888.jpg, "lunch about half way into the hike"

Initially I generated this file by hand, extracting the EXIF location from the image files using exiftool. I then went through each image, entered a title, found the URL for the image on flickr, extracted the URL for a thumbnail image and added a longer description. This was painful to say the least. It was only 16 images but it was a pain. Particularly as all the information was already there, in a flickr set. So I looked up the information on the flickr API, found a python library to access it and wrote the script below in half an hour. Given the URL or set id for a flickr set, it iterates over all of the photos and produces a CSV formatted list that's suitable to load straight into GPSVisualizer. This can then be linked along with the original GPS track log to generate a map with images and also the path traveled. It can be used for both Google Earth and Google maps path generation and runs pretty quickly. I'm posting it here in case it proves useful to anyone. To run, you'll need an up to date python install and additionally will need to download and install the flickrAPI for python. The final step is to obtain an API key for flickr. This key has to be added to the script, in the marked location. Once installed the program is run with:

python flickr2gpsv.py -s set_id > results.csv

or, alternatively,

python flickr2gpsv.py -s http://www.flickr.com/set_url/ > results.csv

If you find this useful, please let me know. You can download the script here. (probably better than cutting and pasting from below, because the download will keep the correct indentation). The archived version also includes a patch from Brad Crittenden to add unicode support.

# Author : Gordon McGregor
# Contact: http://gordonmcgregor.blogspot.com
#
# License : public domain
#
# Purpose: parses a flickr set to extract information to generate a map overlay, via http://gpsvisualizer.com
#
# Usage: python flickr2gpsv.py -s 72157604221838137
# or
#        python flickr2gpsv.py -s http://flickr.com/photos/mcgregorphoto/sets/72157604221838137/
#
#   If the URL is given, the set_id is automatically extracted

# Typically, you'll want to  redirect the output to a file, as errors & comments will appear on stderr (not in the file)
#
# e.g., python flickr2gpsv.py -s 72157604221838137 > output_list.csv
#
#
# only generates entries for photos with geographic information attached
#
# Required libraries:
# the flickrapi python libraries, from http://flickrapi.sourceforge.net/
# installation info here http://flickrapi.sourceforge.net/installation.html
#

import flickrapi
import time

import sys
from optparse import OptionParser
from urlparse import urlparse

# enter your api_key here to connect to flickr
# obtain one from http://www.flickr.com/services/api/keys/apply/
#

api_key = 'your_key_goes_here'

def getURL(sizes, size):

for element in sizes.sizes[0].size:
if element['label'] == size:
    return element['source']

raise flickrapi.exceptions.FlickrError, "No " + size+ " URL found."

# the main routine
# pass in a flickr set id
# outputs the appropriate data for GPSVisualizer to stdout
def parseSet(set_id):

cnt = 0

flickr = flickrapi.FlickrAPI(api_key)

photoset = flickr.photosets_getPhotos(photoset_id=set_id)

# iterate over list of photos in list

print 'latitude, longitude, name, thumbnail, url, desc'
for photo in photoset.photoset[0].photo:

try:
# get the various bits of data

    sizes = flickr.photos_getSizes(photo_id = photo['id'])
    info  = flickr.photos_getInfo(photo_id =  photo['id'])

# extract the required fields
    try:
        lat = info.photo[0].location[0]['latitude']
        lon = info.photo[0].location[0]['longitude']
    except AttributeError:
        raise flickrapi.exceptions.FlickrError, "No Geographical data found."

    name = photo['title']
    thumbnail =  getURL(sizes, 'Small')
    url = info.photo[0].urls[0].url[0].text
    desc = info.photo[0].description[0].text.strip()  # strip to remove extra newlines

    if(len(desc)):
        print '%s, %s, "%s", "%s", "%s", "%s"' % (lat, lon, name, thumbnail, url, desc)
    else:
        print '%s, %s, "%s", "%s", "%s",' % (lat, lon, name, thumbnail, url)
    sys.stderr.write('.')
    cnt = cnt + 1

except flickrapi.exceptions.FlickrError, e:
    sys.stderr.write( '\n'+photo['title'] +' : ' + e.__str__() + ' Skipping.\n')

return cnt



def main(argv=None):
if argv is None:
argv = sys.argv

usage = "usage: %prog [options]\nPass in a flickr set to produce output suitable for GPSVisualizer's google maps overlay"
opt_parser = OptionParser(usage=usage)

opt_parser.add_option('-s', '--set', dest='set', help="flickr set to process (full url or set number)")

(options, args) = opt_parser.parse_args()

if(options.set == None):
opt_parser.print_help()
return 1

# treat the command line value as a url
url = urlparse(options.set)

# even if it is just a set id, the value ends up in field 3 after urlparse (url[2])
# this removes any trailing /, explodes around any remaining /'s and takes the last value [-1]
# works for a full URL or simple set id
set_id = url[2].strip('/').split('/')[-1]

if (not set_id.isdigit()):
sys.stderr.write("set_id" + set_id + "is not a number. This is not expected.")
return 1

sys.stderr.write("Processing set id "+set_id+"\n")
start_time = time.time()
num = parseSet(set_id)
end_time = time.time()

total_time = end_time - start_time

average = 0
if(num):
average = total_time/num

results = "\nProcessed %d valid photos in %0.1f seconds (%0.2f seconds/photo). Finished.\n" % (num, total_time, average)
sys.stderr.write(results)

if __name__ == "__main__":
sys.exit(main())

birthday in the valley

One of the reasons I went to Death Valley at the beginning of the month was to spend my birthday out there. Amanda was away for work and I wanted do something, rather than sitting around the house! I spent the first few days doing a bit of short hiking and taking a lot of pictures. I wanted to try something a bit more adventurous on my actual birthday. I got up bright and early on the 4th, at 4:30am and headed out to the Badwater salt flats to shoot the pre-dawn and sunrise. A quiet, soft start to the day and almost nobody out there. I walked out further than I've been before, about a mile and a half, to where the salt really gets flat and white. Peaceful, beautiful and not a sound. Amanda recorded herself singing 'Happy Birthday' on my birthday present, a digital audio recorder, so I listened to that a few times out there!

After breakfast and a bit of packing, I drove over to the Panamint Valley. Both times I'd been to the park before I'd looked longingly at the Panamint dunes. They are a bit tough to get to - about 6 miles of dirt roads then another 5 mile cross country hike, over a rocky alluvial fan then up through sand dunes. You can see the road in the distance on the far left of this shot and then the dunes on the hillside to the far right. This time I was going to get there.

The hike was great - I took about a gallon and a half of water, my camera and some food and set off up to the dunes. It was a warm 85F and sunny at the start and got pretty hot as the afternoon wore on. Click here for a flickr slideshow of the hike. The soft dirt and sand made for quite tough going and I averaged a slow 2.5 mph on the way up. Just before the dunes, I found a big creosote bush and slept under it for an hour while the shadows lengthened and the light got better for photos. My Tilley hat made for a great shade, hung up in the branches of the bush. That was the first respite from the sun I'd found all afternoon and it was really welcome. So much cooler out there out of the sun with the really low humidity. Lovely. About 3 pm I left the pack behind and walked up into the dunes with my camera. It was a beautiful experience to have that whole expanse of dunes to myself, no foot prints, no people, no noise. Nobody for miles around. The dunes there are steeper than those near Stove Pipe Wells, the hillside and wind have combined to really pile them high and sharp. A couple of times I wasn't sure I'd be able to make it to the top of the highest dune, I almost rolled off down one slope. Eventually though I clambered to the top - the view was stunning - all the valley opened out below me. The narrow peak of the dune actually split into the 3 spines - a star shaped dune! Shooting right at the top was a bit disorientating, looking through the camera and trying not to step off into space. I sat for a while and drank it all in.

Heading back down was a whole lot easier, once I found my pack. Creosote bushes all look the same from a distance! A GPS waypoint made life easier, though I did worry about my batteries dying. It was all downhill to the car and not so much weight to carry either. I think I need to find some hikes where I start off at the top, hike down drinking the water then have a lighter load to carry up the hill! I got back to the car just as the sun dipped behind the mountains. All in all, a great way to spend my birthday. A really beautiful desert hike.

Lots more pictures in the map below - click on the labels on the right or zoom in and pan around.

I've been trying out various GPS tools and GPS Visualizer has been really useful. Massaging the data into a form to create the map above took a bit of work, some use of GPSBabel and a few custom python scripts that I wrote to convert co-ordinates into the right forms. The profile came straight from the GPS track log.

titus canyon

One of the fun things about visiting a National Park on my own was being able to set my own schedule. I could go where I wanted, when I wanted. Rather than on a workshop, where locations are dictated by the group or instructor, or with friends where photography is a reasonable last place. On this trip I could take my time, stop when I was interested in the landscape, take some pictures, scout out potential new areas for future shoots. There was time to just slow down and enjoy where I was. Workshops are a hugely creative, energetic experience but there isn't a whole lot of time to think or explore. The mid-day hours are given over the classroom work or critique and the rest of the time is a rush between shoots, sleep and food. On this trip to Death Valley I had a chance to go to some places I hadn't been before, including doing the 26 mile drive down through Titus Canyon. I left the park and made a trip to Rhyolite to shoot around there and had lunch, then set off down the 4WD only access road to the canyon. The sun beat down from a mostly cloudless sky but it was a still a pleasant 80F in the shade. The road wasn't particularly rough going and the scenery was breathtaking. I was amazed how much it changed over the drive, as I twisted and turned towards the small ghost town of Leadfield. Along the way I saw a red tailed hawk of some kind catching and circling above me with a snake in its talons. It soared on the strong breeze and just hung in the air over head for several minutes. I'd had enough of ghost towns with Rhyolite, so didn't stop in Leadfield. There were a few tin shacks and I'm sure some interesting things to explore, but I pressed on. After Leadfield the landscape changed quite quickly as the road dropped down into the real canyon. The walls got steeper and steeper and closer and closer. The early afternoon sun started to dip behind the canyon walls, giving some relief from the constant sun. The bouncing light between the canyon walls softened and picked up the orange glow of the rocks. Still the road wasn't particularly bad - certainly required high clearance in some places but nothing particularly testing, as it was dry. I found some petroglyphs by the roadside and stopped for a look. Unfortunately they are well marked and as a result have been added to by plenty of visitors.

Nearing the end of the canyon it got particularly narrow, finally birthing out onto an alluvial fan on the side of Death Valley. There's a small car park at the start of the two-way access road. I parked and walked back into the canyon a bit. The narrow canyon walls loomed over me and I was listening for cars all the time - there wasn't much room to get out of their way!

Titus Canyon was a really enjoyable drive, easy on the car and easier on the eye. Dramatic mountains, steep cliffs, red earth and black rock really add to the whole experience. I saw a few other people in there but mostly I was on my own and had a great time, traveling at my own pace. As I uploaded the images to Flickr, decked out with the appropriate GPS co-ordinates, I took a couple of screen captures of how the images appear in Google Earth and also on the Flickr mapping tool. Still exploring what is possible with the geotagged images. Being able to find them in Google Earth is interesting, seeing what other people have shot in the same area is particularly useful.

GPS tagging

50mm, 1/40sec @f1.4, ISO 1600

I have looked in to some options of combining GPS data with the photos I take. For a few hours I thought about writing my own software, I also found the open source Happy Camel project and considered using that. Eventually I realised that the download software that I already use, Downloader Pro, has a GPS option in the latest version. I tried it out briefly prior to traveling to Death Valley but didn't really spend much time working out how to use it. Turned out that it was exceptionally easy to use. I'm really impressed. There's a 30 day evaluation available if you were interested in trying it out. The GPS isn't connected to the camera at all. There's no wires or wireless connection. I don't have anything new mounted on the camera hot shoe, unlike some other GPS tagging solutions. All I have to do is switch on the GPS unit and put it in my bag. It is creating a tracklog of where I go and when. The time on the GPS and the time on the camera need to be synchronised, or you at least need to be aware how different the time is (you can enter the difference in the downloader software). Once you've got that worked out, the GPS knows where you are at a given time and the camera knows when you take a picture (it is recorded in the photo's EXIF data). That's all you need to know to pretty accurately tag where the picture was taken. All that is left is to combine the two and fill out the GPS data field in the photo's EXIF data. That's where Downloader Pro comes in. I have a Garmin GPSmap 60CSx GPS unit. It has a 10,000 point automatic track log, so when I switch it on, it starts tracking. I'm sure this sort of track log can be generated by any GPS unit that can connect to a computer. (GPSBabel might be useful to convert to a format that Downloader Pro can understand). When I get back to my computer, I plug the camera's compact flash card into a card reader and hook up the GPS unit via USB. I switch the Garmin into the USB mass storage mode so that it appears as a hard drive on the computer. In that mode the latest track logs are available in the root of the drive. Downloader Pro is configured to scan removable drives for track logs. (Under GPS Settings, enable Geo-Tagging, set the camera and GPS clock times and for Track Log Settings select 'Search removable drives'). Once Downloader Pro is configured, I simply plug in the GPS, plug in the CF card, set a job reference name and hit download. Everything else happens automatically. I don't have to convert track formats, it already understands the Garmin GPX log format. Simple. You can also produce Google Earth and Google Maps viewable versions of the track log. Next thing I want to try is integrating the images into Google earth to have the shots hanging in space. For now though, when I load up the images in Lightroom there is a new field filled out in the EXIF - the GPS co-ordinates where the image was taken. The 'location' set of metadata browsing in Lightroom shows all this in a concise form. You can then click on the arrow to the right of the GPS data and a web page is opened to Google maps, showing the satellite view of where the image was taken. I also found out that in Flickr, you can change your account permissions, so that it will automatically extract the location information when you upload images. This then places them on the map and indicates where the image was taken automatically. The option is off by default and is under You->Your Account->Privacy & Permissions in the Import EXIF location data option. You can set different permissions on each image for who can see the location data and who can see the image. So you could let anyone see the pictures but only friends and family know exactly where it was taken, for example.

If you click on the shot of Hamilton pool, above, you can use the show on map option in the lower right to see where it was taken. I am interested in any good examples of using geo-tagging in photos. It seems like a neat technology and fun to be able to see exactly where you were when you took a shot. I did use it quite a bit while scouting locations too, to mark a spot that I wanted to return to at a better time of day. Really then all I could have done was hit the mark button on the GPS and put a suitable name in. So far I've found this useful for sharing the location with others, more than anything else. When talking through the images, I could click on the GPS link and pull up the satellite view - handy to show just how far away from anything I was. In some other shots, it has been useful to be able to describe the route and thought process taken to finding a particular location. I have considered using it when out shooting in an urban area, to tag locations with great backgrounds for portraits - there are plenty around Austin and I occasionally forget where they are. Again there the information would be more useful for sharing those spots with others that might want to visit. I can already see my shots appearing automagically in Google Earth, when I switch on the flickr layers, which is quite neat. I'd actually like to work out if it is possible to include my images as floating billboards in Google Earth, when you fly past the location where they were shot. That might require directional information though, to really line up the scene. Useful Links

• Geotagging on Wikipedia
• Geotagging on Flickr
• Geocoding photos on Wikipedia
• Imageingester pro (another tagging program that I hear works well on the Mac)