Fork me on GitHub

heatmap.py

My Photos

heatmap.py -p photo-points -o photos.png --width 1000 --osm -B 0.8 --osm_base http://b.tile.stamen.com/toner

(I've been geocoding my photos since 2004.)

Map tiles by Stamen Design, under CC BY 3.0. Data by OpenStreetMap, under CC BY SA.

Graffiti

Graffiti complaints in San Francisco over 90 days.

There's a trade-off between seeing the precise location of each data point, and seeing the aggregate regional blobs that make trends easier to see. Here I did both and composited one on top of the other.

heatmap.py -b black -p graffiti.coords -r 30 -W 1200 -o g1.png -P equirectangular ;

heatmap.py -p graffiti.coords -r 5 -W 1200 -o g2.png -P equirectangular --decay 0.3 --margin 25 ;

composite g2.png -dissolve 50 g1.png graffiti.png

Data from datasf.org. I don't know what system the coordinates are in, so I just pretended they're square and mapped them as-is.

My Summer Vacation

heatmap.py -g orkney.gpx -o orkney.png --height 800 --osm

Background © OpenStreetMap & contributors; this image is available under CC-BY-SA

Burning Man 2006

heatmap.py --decay 0.8 -g bman.gpx -o bman.png --height 800 --margin 10 -b black --gradient gradient-1.2.png

(I carried a GPS around all week.)

There are a few kinds of heat maps. This program is for when you have data points, each with a lat-lon coordinate, and you want to plot them on a map such that they blob together and get bright where they pile up, showing off the data density.

If you've divided your map into regions and want each region colored a certain way (a choropleth map), or you have a formula that gives you a value for every pixel on the map, this is probably not the right tool.

Features include:

Features it probably should have, but doesn't yet:

Quick start guide

It's just one file: heatmap.py. I've put it on github in case anyone wants to contribute. If you want the --osm option to work, you'll also need osmviz and should read the bit about OSM licensing issues.

Now get yourself some data (or grab my list of photo coordinates) and just try it out. See the examples on this page, read the command line help for a very brief rundown on the options, and maybe eventually read the rest of this page. In general, the program will tell you if needs more information.

Why?

I wanted to make some data maps and I couldn't find any free tools that did a good job. I found proprietary tools (ick) that produced gorgeous output, and free tools that I felt took the wrong approach. Had I discovered gheat sooner, I probably would have started with that, but this is a bit different, better for some things, worse for others. There are some obvious next steps, but it's time to set it free.

It's a Python script, which makes it most suitable for server-side image generation or preparing images in the comfort of your own home. That's great for my needs, but doing the work in a browser would be a better choice for many applications these days.

Things you wish you didn't have to know

Speed

How long it takes is roughly proportional to num_points * radius2, so if you want to try things out quickly, start with a small radius, and/or less data.

Memory

The memory footprint depends on the settings. With --decay set between 0 and 1, the memory use will be dominated by num_points * radius2. With decay set to 0 or 1, the rendering only needs to hold data proportional to the size of the output image, which is usually negligible. At that point, the major factor will be whether it needs to read in all the data at once. Currently it needs to know the geographic extent of the data at the outset, so if you don't specify one using --extent, all the data will be read first, and then held in memory for the rest of the processing. So memory use will be proportional to the size of your data set. If you use --decay of 0 or 1 and use --extent and give either the height or width (or --osm and --zoom), then the memory footprint won't depend on the input significantly and you should be able to push any amount of data through. (I tried 10 million points, worked great.)

What's that --decay option for?

It's to make it prettier and to make it convey your information better. If you leave it at 1, it works like most people's heatmaps do, just adding points on top of each other linearly. Depending on your data distribution, this may mean areas with relatively low density are hard to see. The biggest trends are clear, but much of the data can be lost.

If you set it to 0, every data point will be visible, turning the plot into "how far is it to the nearest point" and conveying nothing at all about distibution density. It will look more like a scatter plot (or a line drawing, with GPS tracks).

All the other heatmap software I've looked at works at one of these extremes (usually the former), but I find it looks better and conveys more information somewhere in between. I was unable to find a magic value that works well for everything, so you may want to fiddle with this. There's no right answer. You just have to decide what makes sense for your application and your data.

The catch is that 1 and 0 are special cases of the algorthim and can be computed using much less memory. if you set it to anything in between, the memory footprint will go up considerably.

Changing the colors

You can specify the color gradient on the command line by providing hue, saturation, value, and alpha for the two extremes. The program will use linear interpolation to render the in-between values. Note that hue is specified with three digits in case you want to go around the other way on the color wheel. For example, 000ffff00 and 100ffff00 specify the same color, but pass through different hues on the way to 02affffff.

The other option is to supply an image. The pixels down the left edge are used to specify the gradient. (For example.)

Licensing OSM background tiles

If you use the --osm option to overlay your data onto OpenStreetMap tiles, you are bound by their terms of service.

The options

Here's what you get when you use --help.

Usage: heatmap.py [options]

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -g FILE, --gpx=FILE   
  -p FILE, --points=FILE
                        File containing one space-separated coordinate pair
                        per line, with optional point value as third term.
  --csv=FILE            File containing one comma-separated coordinate pair
                        per line, the rest of the line is ignored.
  --ignore_csv_header   Ignore first line of CSV input file.
  --shp_file=FILE       ESRI Shapefile containing the points.
  -s FLOAT, --scale=FLOAT
                        meters per pixel, approximate
  -W INT, --width=INT   width of output image
  -H INT, --height=INT  height of output image
  -P NAME, --projection=NAME
                        choices: mercator, equirectangular; default: mercator
  -e RANGE, --extent=RANGE
                        Clip results to RANGE, which is specified as
                        lat1,lon1,lat2,lon2; (for square mercator:
                        -85.0511,-180,85.0511,180)
  -R INT, --margin=INT  Try to keep data at least this many pixels away from
                        image border.
  -r INT, --radius=INT  pixel radius of point blobs; default: 5
  -d FLOAT, --decay=FLOAT
                        float in [0,1]; Larger values give more weight to data
                        magnitude.  Smaller values are more democratic.
                        default:0.95
  -S FILE, --save=FILE  save processed data to FILE
  -L FILE, --load=FILE  load processed data from FILE
  -o FILE, --output=FILE
                        name of output file (image or video)
  -a, --animate         Make an animation instead of a static image
  --frequency=FREQUENCY
                        input points per animation frame; default: 1
  --straggler_threshold=STRAGGLER_THRESHOLD
                        add one more animation frame if >= this many inputs
                        remain
  -F STR, --ffmpegopts=STR
                        extra options to pass to ffmpeg when making an
                        animation
  -K, --keepframes      keep intermediate images after creating an animation
  -b COLOR, --background=COLOR
                        composite onto this background (color name or #rrggbb)
  -I FILE, --background_image=FILE
                        composite onto this image
  -B NUM, --background_brightness=NUM
                        Multiply each pixel in background image by this.
  -m HEX, --hsva_min=HEX
                        hhhssvvaa hex for minimum data values; default:
                        000ffff00
  -M HEX, --hsva_max=HEX
                        hhhssvvaa hex for maximum data values; default:
                        02affffff
  -G FILE, --gradient=FILE
                        Take color gradient from this the first column of
                        pixels in this image.  Overrides -m and -M.
  -k KERNEL, --kernel=KERNEL
                        Kernel to use for the falling-off function; choices:
                        linear, gaussian; default: linear
  --osm                 Composite onto OpenStreetMap tiles
  --osm_base=URL        Base URL for map tiles; default
                        http://tile.openstreetmap.org
  -z ZOOM, --zoom=ZOOM  Zoom level for OSM; 0 (the default) means autozoom
  -v, --verbose         
  --debug