Making is Decision-Making: Maneesh Agrawala '90

by Hanna Mesfin for Silver Quest

Every January, the Blair Magnet community prepares for the highlight of the season: the SRP convention. Or—that was, until an unexpected snowstorm cancelled classes for an entire week. The program was rescheduled to March 26, but students did not let that stop them from showing out. Seniors presented research from their summer internships as a culmination of their four years in the Magnet program, with topics spanning from the biology of turtle brains to the complexities of Markov theory and the cosmic scale of interstellar dust. Attendees watching these presenters walked away with a plethora of new and illuminating facts.

While current Magnet seniors are the convention’s highlight, they aren’t the only ones involved. Every year, Blair also invites a guest speaker—often an alumnus—to address the community. For the 34th annual event, the program welcomed back Dr. Maneesh Agrawala (‘90) this year, presenting him with the Distinguished Alumnus Award. 

After graduating in 1990, he attended Stanford intending to major in mathematics, inspired in part by the Magnet’s rigorous curriculum. However, when Agrawala realized that it did not fully capture his interest, he fell back on computer science. He had been programming for a long time, and his father was a computer scientist. “I happened to take a computer graphics class first,” he said, “and just locked in. From that point, I was pretty much hooked into computer graphics.”

He eventually returned to Stanford as a professor, where he now teaches computer science and works closely with students on complex, creative projects. In fact, it was a student of his who played a key role in developing an idea behind the research he presented at the SRP convention.

In recent years, Agrawala’s work has focused on artificial intelligence and its role in creative processes. A type of machine learning model called Generative Adversarial Networks (GANs) that learned from existing datasets worked fairly well for image generation, but Agrawala pointed out that it was difficult to control. A more recent technique, called diffusion models—which, unlike GANs, does more than guessing by adding noise to the data until the recognizable structure disappears and trains the model to reverse the process and reconstruct the original image—has improved image generation quality. To address the persisting gap between human communication and AI, Agrawala and his student explored ways to make AI systems more controllable, particularly by improving how users guide spatial composition.

At the convention, Dr. Agrawala illustrated this challenge through the work of photographer Ansel Adams and his assistant Alan Ross, whose work relied on a shared understanding of artistic intent. Ross worked to understand the concept that Adams tried to communicate, and his own style ended up being heavily influenced by the man he worked for. Unlike human collaborators, however, AI lacks this shared foundation.

To test how grounded the AI’s understanding is, Agrawala prompted ChatGPT to generate an image of Stanford’s main quad after a snowstorm. The results required numerous iterations; some resulted in black and white images, and one even produced a nighttime scene. Even after 27 attempts, the output was only somewhat satisfactory. This was, as Agrawala called it, “AI slop.” This demonstration begged the question: what’s the difference between content made by computers and humans?

Agrawala explained that the core of this issue is a lack of “shared conceptual grounding.” While humans can directly communicate style choices to an assistant and build understanding over time, AI does not have shared prior experience. Instead, it requires plenty of trial and error. In 2023, Agrawala worked with a PhD student to address this, seeking a way to teach generative AI the concepts that authors think about when they are creating content. Spatial composition is a technique that human artists use, but AI looks for natural language descriptions. Agrawala and the student developed a block-and-detail approach that allowed artists to guide AI more precisely by combining spatial structure with the person’s sketches. The goal was to make AI a more effective partner in creation rather than a tool that necessitates constant correction.

Through years of experience mentoring students as a professor, Dr. Agrawala emphasized the lasting influence of education and mentorship. “One of the most important things that we learn [as students] from our teachers is how to form our own opinions and how to make choices,” he remarked toward the end of his speech. He credited a few of the most influential teachers he had, including the late Dr. Kleppner, an English teacher who strongly influenced his writing. “Don’t be like me—don’t wait 36 years to thank your teachers,” he added.

When asked what advice he would give current Magnet students, Agrawala offered a simple but powerful message: “Don’t worry about what you’re going to do—figure out what you care about.”

For students navigating their own paths, his words are a reminder that you don’t need everything planned to be successful. The people who influence you, your experiences, and pure curiosity are what ultimately shape the journey.