Danyel Fisher: Analyzing and Visualizing Big Data at Microsoft

What is it like doing academic research as part of a company?

Similarly, I've been doing a number of projects on the question of how people interact with ‘big data.’ It’s a fascinating complex of questions, with projects that have varied from building prototypes, to building a working system, to running ethnographic studies. I try to choose the research tool that will teach me the most. A more product-oriented work I've done was a project that started off looking at how to embed more kinds of charts in Excel. We built a prototype that the product team picked up, expanded, and modified heavily—today, it’s called “Apps for Office;” four years later, it was shipped as a major part of Office 2013.

Hotmap shows spots brighter where more people have looked at imagery, so this looks at where people are looking when they look at the map. As you can see, it follows population, coastlines and highways. But there is one other aberration: a bright spot below Africa, on a cross. That point turns out to be latitude 0, longitude 0. A bug in the search code would sometimes send users to that point—and they would begin to scroll around, trying to find out where they are. They were able to come to a number of useful conclusions—both finding bugs, and understanding user behavior better. This wasn't really a product—it was a demonstration, and a prototype. The team didn’t need to commercialize or productize the tool—once they’d seen the data, they knew most of what they would ever need to learn.

I picked my undergrad, Washington University in St. Louis, because it seemed to appeal to flexible people: there was no process of applying to the School of Engineering versus the School of Arts & Sciences. You applied to the university, and then made a decision of what to do. Me? I came in with a few AP credits, which let me skip early classes, and I was able to dual-major in Computer Science and Philosophy. (As you can tell, I was still a little commitment-phobic). The pairing was actually remarkably good—both are fields that teach disciplined, precise thinking, in which you go through arguments in some detail to figure out their respective strengths. (One semester, I got taught formal logic twice, from two complimentary perspectives!) However, CS grad students seemed much happier about their future prospects.

I went to graduate school still indecisive. I blame the Magnet for teaching me that many areas of thinking can be exciting, and that a well-rounded person knows many different things. I tried machine learning, and computer graphics, and neither fit quite right. It was human-computer interaction in general, and information visualization in particular, that won me over.

Tell us about what you do and what a typical day or week is like.

Microsoft Research is basically an academic organization—we look a lot like the research part of a university. My job: I produce prototypes, which demonstrate interesting ideas. I run those prototypes past users—whether by deploying on the web, or bringing them in for lab studies, or running them on the users’ desktops—and write papers that explain how users interacted with the systems. So the outcome of much of my work is a paper about a system, rather than a system itself. During the summer, I mentor interns, usually doctoral students.

A typical day? I’ll probably meet with a project team to talk about how we’re doing on one or another project. I’ll write some code during the design phase, or run a user study, or work on a paper, or run a data analysis.

How much of what you do is product development versus evaluating how people use those products?

That’s hard to say. I do very few things that are product-as-such, but lots of things that eventually speak to products. Let me give you an example. Back in 2007, Microsoft was rolling out its first mapping service—“Microsoft Virtual Earth,” they called it. (It’s now “Bing Maps.”) I talked with the product team about the way that they thought about their data. I learned that they were looking at the overall number of hits on their website, but didn't really think about how people were using the service. If they knew, for example, that people were more likely to look at city than country, or at beach rather than mountain, they might make different decisions about the interface and about the tool. I suggested that they consider seeing their data as a map, and built a prototype. [To the right], I have a screenshot of our system, “Hotmap.”

I don’t think I had a solid idea of what I wanted to do. It was a pretty generic “something involved in science.” For my senior project, I had a summer job out at NIST, where I worked on some interesting projects connected with spectroscopy and diffraction patterns. I spent another summer at the Naval Medical Center working on a project around antibody cloning. In other words, I broadly knew that I liked opening new frontiers and learning new things, but didn’t know what they would be—or even what field. While I enjoyed the Magnet computer science projects, I can’t say that I was more excited about them than the physics or chemistry classes, but by the time I left, I was pretty sure that I wanted to spend more time in computer science.

What were your aspirations then and how have they evolved to your current ones?

I was on a triumphant Odyssey of the Mind team, where (I believe) we wrote a play based on Ernest Hemingway’s “The Old Man & the Sea,” and not-so-successfully tried to build a Rube Goldberg device. I was consistently involved with the theater program—usually as stage crew, but once or twice as the back row of the chorus. I was on Silver Quill once, and helped organize the Magnet talent show.

What electives and clubs were you part of?

I was, I think, a proto-geek from early on. It wasn't specific to math and science: I liked reading and writing and learning anything I could get my hands on. (Except history. Junior High School me never could get the hang of history.) I’d been to the Junior High Magnet at Takoma Park, and loved it—it was a great level of challenge, and I was surrounded by my own kind. The Magnet seemed like a great match, despite the commute.