Secular Products

by Shawn Allen

Reporting for Duty

I’m excited to announce that next month I will be joining 18F, a new team within the United States General Services Administration tasked with improving digital services for the Federal Government and the American people.

Open Data

I’ve spent my career practicing and advocating for open data. At Stamen my colleagues and I urged our clients to provide us with easy-to-use web services (APIs) into their data, and when they couldn’t find the resources to we built them ourselves. These services were often purpose-made to power “special projects” such as Digg Labs and Trulia Hindsight, so they weren’t necessarily intended to be open. But in at least one notable case (Digg), the services that we designed outlasted the projects for which they were created and became an integral part of the product.

Mike Migurski (then one of my partners at Stamen, now CTO of Code for America) and I brought our technical perspective on open data to Carl Malamud’s Open Government Working Group in 2007. I felt out of my element, and mostly out of shyness I contributed little except for the unofficial Open Government gang sign. But I was inspired by the passion of the other attendees, including Lawrence Lessig, Adrian Holovaty and Dan X. O’Neil from the original Everyblock, Tom Steinberg, and folks from MapLight and the Sunlight Foundation. Until then I had thought of “open data” as a primarily technological problem. This meeting—and Carl Malamud’s tireless crusade to digitize government documents that might as well have vanished from the face of the earth without his efforts—helped me to understand it as a social problem.

Civic Technology

My first direct experience with “civic technology” was adapting Oakland Crimespotting for San Francisco in 2009. My colleagues Mike Migurski and Tom Carden had done all of the hard work building the site, API and map over the previous couple of years. All I had to do was fork the Oakland code and make it work with never-before-seen crime report data from the City’s new DataSF portal. Getting crime reports out of Oakland had been a thankless exercise in freeing data that should have been open (and eventually would be), but San Francisco just handed it out for free. About two weeks later, San Francisco Crimespotting went live, and I became its de facto maintainer.

Crimespotting became a calling card for Stamen, and the timing coincided with a particular civic interest of my own. I had been bicycling to work for a couple of years, and my curiosity with how communities could develop better transportation infrastructure led me down the urban planning rabbit hole. I read Streetsblog and The Life and Death of Great American Cities, and I became deeply interested in the often tenuous relationship between citizens and their governments. I attended the first Open Cities conference in DC and was once again inspired by Nick Grossman, Ben Berkowitz, the conference’s organizer and Next City founder Diana Lind, and many of the other passionate speakers and attendees.

But I was also skeptical of technology’s role in civic matters. Crimespotting, while an important stake in the ground for open data and interactive mapping, was problematic because it led many viewers to reach unfortunate conclusions about Oakland and San Francisco, and it reinforced negative perceptions of high-crime areas. Mike put it nicely (emphasis mine):

Just showing crime is relentlessly negative, and seems to really draw out the kind of graffiti-squad neighborhood busybodies who focus solely on little problems. A near-universal reaction from non-residents to this particular project has been relief that they don’t live in Oakland, but it’s really not that bad here. It just looks bad when all you show is crime. We’d like to map other things: city services (police, fire, emergency), tax parcels, effects of policy, other administrative information that’s hugely important.

I, too, was dissatisfied with just—cue Eric, with “just” in air quotes—plotting crime reports on a map and letting people draw their own conclusions. At Stamen we talked for years about augmenting Crimespotting with report statistics over time and other data, such as calls for service, but we simply never found the time (or the grant money) to do it.

Beauty and Utility

In my own spare time, though, I was experimenting with graphical treatments to show crime with other data, and when I spoke at Open Cities in 2010 I showed this slide:

I made the image above, Trees, Cabs and Crime in San Francisco, in 2009 from three seemingly unrelated data sources: the locations of street trees under the care of Friends of the Urban Forest in cyan, Yellow Cab taxi GPS pings from Cabspotting in yellow; and crime reports from SFPD via Crimespotting in magenta. The three colors were blended subtractively (crudely mimicking the CMYK color process) to create red, green, blue and black wherever they coincided. The phrase “beautiful and useful” was a response to a frequent critique of Stamen projects, “beautiful but useless.”

Even though the image was of dubious utility, I thought that it illustrated my point that beauty can serve a valuable role in making data relevant to society. The folks at the Institute for Urban Design seemed to know exactly what I meant when they chose Trees, Cabs and Crime to be featured in the Spontaneous Interventions exhibit at the 2012 Venice Biennale of Architecture, because they didn’t ascribe any higher purpose to it than “sussing out patterns.” I was genuinely intersted in seeing whether the three data sets that I’d chosen—admittedly, arbitrarily—matched up in interesting ways and, more generally, in experimenting with a novel way to combine multiple geographic data sets without obscuring any of them. It failed on the first point but succeeded on the second, and also in another way that I hadn’t anticipated: the unexpected beauty of the image attracted attention, and it raised questions.

Question-Making Interfaces

Just as information wants to be free, I believe that data wants to be seen. Visualization is a powerful tool for communicating and understanding complex data, and when it’s successful as a process it not only answers important questions but prompts new ones.

For instance, people often asked me why there aren’t any trees in the long swath of Golden Gate Park in the image above, and the reason is that the data came from a non-profit that plants and maintains street trees. (The Urban Forest Map now houses this database, and the Department of Public Works maintains their own list.) Similarly, when discussing Crimespotting I always took care to explain that it was not a map of crime, but of crime reports, which must pass through numerous institutional filters (in some instances, to protect the victim’s privacy) before being released to the public. Homicides, I discovered when I sought out to map them on Dotspotting, were initially included in San Francisco’s report data but mysteriously disappeared some time in 2011. Visually exploring data was an integral first step in nearly every single project at Stamen.

Seeing and interacting with the data helps us perceive its boundaries and better understand its limitations, its vagaries and subtleties. And understanding those aspects of data allows us to ask the really tough questions: not just about what’s in the data, but what’s not. Questions about timeliness and provenance, and the often bureaucratic processes by which data becomes open. Purveyors of open data (most notably, government) need to raise the bar above links to spreadsheets and learn to deal also in the interactive interfaces that help translate data into useful information.

Onward and Upward

It’s been a wild nine years since I started working at Stamen, and it was a wonderful and unique environment for me to develop these thoughts and skills. I simply wouldn’t be where I am today if it weren’t for all of my thoughtful and inspiring colleagues—in particular, my brilliant business partners and friends, Eric Rodenbeck and Mike Migurski—and clients with whom I’ve had the pleasure of working. I have so much more to learn, and I can’t wait to put it all to work for the American people.

At the end of last year I wrote my first post on this blog about my departure from Stamen. I’ve enjoyed my much-needed downtime, but I’ve also been keeping myself busy making things and plotting my next move. Here’s what I’ve been up to so far in 2014.

Pain Management

This spring I started seeing a craniosacral specialist who appears to have pinpointed at least one source of my mysterious pain. He thinks the primary culprit is a sprained sacroiliac (SI) joint, and the healing has involved lots of walking and icing my butt. I’m not 100% healed yet, but it’s definitely getting better. It may be many more months before I’m back on a bike, but I’m confident that I will ride again one day, and I look forward to being pain-free some day soon.

Leaving Stamen

In August I signed a piece of paper that ended my membership in Stamen, LLC. Eric has since brought on a new partner, Jon Christensen, and I’m very excited to see where they take the company together in the years to come. It has been my honor and pleasure to work alongside this talented team of wonderful people for the last eight years, and I wish them godspeed and good luck.

Leaving Stamen has been bittersweet, and made all the more so by the tragic death of former colleague Zach Watson, who was struck by a car thief while walking in the Tenderloin on July 28. He was recussitated on the scene and taken to SF General, where he survived on life support until August 16th. I worked with Zach for over two years, during which I came to know him as a gentleman and a scholar: a man of intensity and process who, given the time and resources, could have accomplished anything that he put his mind to. Zach will be missed by many; if you can find it in your heart, please make a donation in his memory.

Contract Work

In May I worked with my old Stamen colleague Tomas on an interactive map and listing of candidates in the 2014 national Indonesian elections paid for by the Asia Foundation. The code is all on GitHub.

In June, July and August I worked with Sam Borgeson on an interface for visually analyzing and exploring PG&E Smart Meter data for Ram Rajagopal’s lab at Stanford Energy. The work is all private for now, as the interface is intended for utilities and the data is highly sensitive, but I’ll add it to my portfolio as soon as it can be shown.


On August 21st my wife gave birth to our son, Milo Claiborne Nelson. His birth was the most intense and truly awesome experience of my life, and I will never forget grabbing him under the armpits and pulling him out, all blue and wrinkled and crying for the first time, and bringing him to rest on his mom’s chest. Andi was incredible: she went into labor on Tuesday evening, worked through contractions at home until 2:30 in the morning, labored at the hospital for another 20 hours straight, then pushed him out at 2am on Thursday—all without drugs. I am forever indebted to her strength and resilience.

Milo spent his first night in the newborn intensive care unit at UCSF, but was quickly downgraded to “high observation” status and moved to the nursery for monitoring. Andi was discharged 48 hours after giving birth, but a social worker found us a tiny room to stay in for the next day so that she could continue breastfeeding every three hours without having to leave. On Saturday evening he was disconnected from his tangle of monitoring leads and spent the night with us in what we affectionately referred to as “the closet”:

The closet at UCSF

A 6.0 earthquake struck at about 3:30 that morning, and the 15th floor of UCSF shuddered and swayed. Andi and I jumped out of our beds, but Milo slept right through it. That afternoon we all went home together and started the next chapter of our lives as a family of three. The first two weeks have been everything all at once: stressful, tiring, emotional, hilarious at times and, overall, wonderful. Parenthood promises to be the most challenging undertaking of my life, and the most rewarding by far.

Planning my Next Move

I was more than happy to have spent the first five months of 2014 without a job. It was my longest period of unemployment since I was 18, and I relished in oversleeping, going for long walks around the city, and meeting up with friends for meals in the middle of the day. With a baby on the way, though, I eventually decided to pick up the search for gainful employment.

I spent the better part of this year talking to a very new media organization about becoming their director of information design. This is indefinitely on hold while they grow and better define their structure and process, and I wish them lots of luck in doing so.

More recently, though, I’ve been plotting something completely different. It would be premature to announce while I’m still in the paperwork-filing stage, but suffice it to say that I’m very excited. More on that soon.

New Beginnings

Today was my last day as Interaction Director at Stamen. I’ve worked with Eric (and Mike, until he left last year) for over 8 years, and I’m proud of the the work that we’ve done and the business that we’ve built together. I made up my most recent title a couple of years ago to distinguish my role as the director of our interactive development practice, and facilitator of the interaction between our design and technology teams. Stamen is, after all, a design and technology studio, and the nexus of these disciplines is where our most interesting work happens.

A couple of years ago something else happened, though: a mysterious health issue surfaced, manifesting for me first as mild discomfort during long bike rides and eventually evolving into an icy pain that shot down the sides of my legs whenever I sat or stood still for even short periods of time. I saw a rotating cast of doctors, physical therapists, chiropractors, Pilates instructors, acupuncturists and pain specialists without a conclusive diagnosis until this November, when an osteopath finally discovered that that I have scoliosis. My spine is curved, and he believes that the pain is muscular fatigue produced by my body trying to account for its skeletal imbalance.

So I’m taking a break to heal the wounds from half a lifetime of sitting and standing in front of a computer screen, and when I’m healed I will return to Stamen with a new title: Partner and Advisor. It’s been my honor to work with so many wonderful people over the years, and I can’t wait to see what this amazing team does in 2014 and beyond. I’m very much invested in this company, and I look forward to discovering a role in which I can continue to help it kick ass and grow while pursuing my own passions.

It’s exciting to admit that I’m not quite sure what’s next for me. Whatever I end up doing, though, I hope to continue teaching the practice of interactive data visualization that we’ve pioneered at Stamen. Years ago we came up with a slogan for ourselves, to make data RAD: Relevant, Accessible and Desirable. We’ve come a long way as a society in figuring out how to visualize and communicate data, but we’re still at the beginning of infinity. It’s my life-long goal to advance this important discipline so that we can better understand both ourselves and the world around us. And I’m just getting started.