Can We Really Teach Machines to Smell?

Statistics Prof. Ambuj Tewari reviews computer code. Photo by Leisa Thompson Photography

The scene: An ancient city surrounded by the calming, fragrant scent of pink hybrid Damask roses. The soil is particularly well suited for their survival. It’s so rich a smell that those who walk through the streets are greeted with the aroma when passing the storm drains. For more than 400 years, those who have lived in India’s perfume capital of Kannauj have bottled their “liquid gold,” using traditional methods across more than 200 of the country’s perfume distilleries.

LSA’s Ambuj Tewari, professor of statistics, fondly remembers the sensory wonder of growing up near Kannauj. He recalls visiting family members who worked in the perfume business there and the certain attars, or essential oils, like spicy shamama, earthy khus, and sweet kewda, that take him back.

“As humans, we construct our reality primarily through sight and sound. But olfaction does play a major role in the human experience. The sense of smell is intimately connected with memories. We enjoy fragrances and perfumes, and the flavor of foods and beverages is a complex experience mediated by both smell and taste, but predominantly smell,” he says. 

Tewari combined his interests—smell, the first sense to develop as part of human evolution; chemistry, inspired by his father’s career; and machine learning, a branch of artificial intelligence that seeks to give machines the ability to learn from experience—to embark on an atypical career path. 

“I developed an unusual interest, I guess,” he laughs. “Over the years, I found myself drawn more and more to machine olfaction, or the automated simulation of the sense of smell.”

The automated simulation of the sense of smell? Yes, he’s talking about giving a robot a nose, in a way. This could potentially help with testing the quality of food, detecting diseases, finding illicit drugs, and monitoring the environment, among other uses.

If you’re picturing R2-D2 with a prosthetic nose, that’s not exactly right. Tewari’s robot is an ordinary computer, but instead of programming it to spit out binary data of ones and zeros, which would be much more typical in a statistics course, he creates a learning algorithm, consisting of common scent categories and descriptors created by the students, that allows his robot to learn and then assess the chemical composition of a molecule. The computer can then tell you, for example, if it would produce a woody and warm or fruity and fresh scent.

Statistics Prof. Ambuj Tewari meets with Yikun Han (center), a master’s student in statistics, and Rui Nie (right), a doctoral student in biostatistics, who have both been inspired by his passion for machine olfaction. Photo by Leisa Thompson Photography

Machine olfaction is a field spanning statistics, chemistry, computer science, and engineering. It only started to see major advancements in 2015 at the DREAM Olfaction Prediction Challenge, where one research team shared its work on creating predictive modeling with machine learning to determine the smell of a molecule based on its chemical structure. Their findings were published two years after that.

Tewari read the publication with excitement, believing he could replicate the team’s work in one of his statistics classes where he teaches students about deep learning. Deep learning is a branch of machine learning that uses algorithms modeled after the human brain structure, called multilayer neural networks, to solve problems like object recognition or playing chess.

He envisioned a reconstructed class where students would smell a variety of molecules and record their perceptions. Then, later in the semester, the undergraduate students would use a machine learning model to predict a human’s odor perception from a molecule’s chemical structure, just like in the paper. 

In Tewari’s lectures, students learn about the different branches of AI and ethical issues dominating the public discourse, listen to guest speakers who work in the data science field, and become familiar with popular software tools like Python and TensorFlow that make it easy to create deep learning models. During lab sessions, students line up at benches to interact with smell blotters and document what they think they’re smelling, or sit at a computer and practice creating neural networks with smell description tags, with the goal of identifying a chemical composition’s projected scent.

To support Tewari’s new Stats 315 class, LSA provided funding through the New Initiatives/New Instruction grant. The grant is available to faculty and lecturers who seek to foster student success in innovative ways with inclusive teaching practices.

Michelle Krell Kydd, the “Nose of Ann Arbor,” makes smell kits for the students in Prof. Tewari’s class. Kydd was inspired to become an educator and share her passion for smell with others. Photo by Scott Soderberg/Michigan Photography

You might be asking, why is any of this important in the first place? Why should anyone care about a robot being able to tell if something smells like mint or rotten eggs or a family member’s headache-inducing perfume? Actually, there are a lot of reasons that being able to detect smell, naturally or artificially, is important.

Smell isn’t only our first sense in terms of human evolution, but “it’s the first sense we develop in the womb,” according to Michelle Krell Kydd, an Ann Arbor native who was trained as a professional nose at Givaudan, a Swiss manufacturer of flavors and fragrances. “Smell is memory, and memory is identity. It’s sacred.”

Kydd, who has been referred to as the “Nose of Ann Arbor” and a “walking smell-o-pedia,” left a career in tech after the September 11 attacks, realizing she needed to start anew. She decided to pursue an area of interest that was, and still is, a grand passion: the art and science of perfumery.

After following the scent trail, she spent a few years working at a trade publication and meeting perfumers from around the world, after which she worked as a consultant for publicly traded food and fragrance brands. In 2011, Kydd decided on a new adventure: She gathered her library of flavor and fragrance books and moved to Ann Arbor. 

Inspired to become an educator and share her passion for smell, Kydd began creating opportunities for children and adults to learn with their noses, such as “Smell and Tell” events at the Ann Arbor District Library, 826Michigan, and U-M.

“Last year, I found Michelle from a TEDxUofM talk she gave in 2015, and I reached out to her to see if we could work together,” Tewari recalls. Kydd, intrigued by the prospect of creating a space for experiential learning in Tewari’s department, eagerly signed on and has helped create smell kits for his class.

“Students face pressure to get the best grades, and working in a space that isn’t restricted to visual or auditory learning kind of takes that pressure away. It’s hard to come up with words to describe smells, but sensory evaluation is subjective and students aren’t judged on their answers, because our perceptions of smell are tied to our own unique realities, and it’s OK that our realities coexist,” says Kydd. “As a matter of fact, it’s imperative in life and the classroom.”

To Tewari’s surprise, more than 100 students from various majors across the college enrolled in the course. While it’s true that there has been a growing interest in learning about AI, some of Tewari’s former students would say the excitement generated by the class could also be because Tewari is the professor.

Rui Nie (left), now a doctoral student at the School of Public Health, decided to write about machine olfaction for her honors thesis project after meeting Prof. Tewari. Yikun Han (right) became interested in conducting research in the field of machine olfaction after attending one of Michelle Krell Kydd’s “Smell and Tell” presentations. Photo by Leisa Thompson Photography

“I joined Professor Tewari’s lab in my junior year when I was studying statistics and math,” says Rui Nie, now pursuing her Ph.D. in biostatistics in the School of Public Health. She joined his lab because of the extensive interdisciplinary research opportunities Tewari provided. When she heard about his machine olfaction project, she asked to join his team so she could practice deep learning techniques.

Over the summer break before her senior year, she expected to continue studying statistical techniques following her professor’s guidelines, but instead, Tewari mailed her a book about how the sense of smell is formed in the brain.

“It was so intriguing to me how chemicals interact with receptors in the nose and stimulate neurons, eventually reaching the brain. It was helpful in clarifying biological concepts for me,” she says.

Rui Nie (below) says that learning how the sense of smell is formed in the brain “was helpful in clarifying biological concepts for me.” Photo by Leisa Thompson Photography

Nie can relate to Tewari in terms of her connection to scents that remind her of home, reflecting on her experience growing up in China and being exposed to pungent medicinal scents such as tangerine peel and mugwort leaves. Tewari’s passion for smell stuck with Nie, who decided to write about machine olfaction for her honors thesis project, for which she was awarded high honors. 

“Initially, I was only curious about what Professor Tewari was doing. When I think more deeply about it, though, I think I’ve realized how the sense of smell is as important as our vision or hearing,” she says. “Suppose you’re a chemistry student trying to produce a novel chemical through a reaction. You may or may not leak a gas that’s harmful to health. How would you know? The human capacity for smell is limited.”

Yikun Han, an assistant of Tewari and master’s student in the Department of Statistics, was also inspired by the professor’s excitement around the developing field of study when considering the next step he wanted to take in his education. 

“Professor Tewari’s passion for research was the biggest thing that stuck with me this semester. Many times when he shared a new article with us, I could feel the excitement on his face and in his words. Not only in the sense that we could learn from the experience of previous researchers in our subsequent research, but also in his own joy of learning something new,” Han says.

Yikun Han records his perceptions of an unidentified scent. Photo by Leisa Thompson Photography

After attending a “Smell and Tell,” Han expressed interest in continuing to work with Tewari but in a different way: conducting research in the field of machine olfaction. Now doing an independent study with Tewari, Han has helped develop materials for the deep learning class that he hopes will promote outside-the-box thinking, and is excited at the prospect of this becoming a research opportunity when he pursues his Ph.D. in the future.

“We’re both learning together right now. Compared to his familiarity with machine learning theory, maybe Professor Tewari isn’t as sure about which method will work best in this field, looking at machine learning applications. But I think that’s the beauty of scientific research: trying to solve problems that you don’t know yet can be solved or not,” Han says.

“The full picture of olfactory processing in the brain still remains a mystery. On the one hand, we want to understand how olfaction works in the natural world, and on the other hand, we want to build machines that can smell,” Tewari says. “One hopes that progress in one area will lead to advances in the other.”

Alex Wiltschko (B.S. 2009) comes from a long line of U-M alumni. When he realized around age 16 that he wanted to be a neuroscientist, he says there was no question he would apply for admission to LSA. From East Hall, he could attend his psychology and math classes without having to traverse too far across campus, and it was in those classes where his interests in olfaction and neuroscience came together. 

After continuing his studies at Harvard and joining Google as a researcher, he now runs his own company, Osmo, which seeks to digitize smells, like how we generate images and sounds today.

“It is clear now that the subset of machine learning that is deep learning has had a major impact on our society, and it is pervasive. It used to be that there were very few places where you could learn the basics in the field, extend the ideas, and apply them to the real world. But now, for folks that will be working with data or tools that make forecasts or predictions, becoming well versed in this technology is not an option anymore; it is required,” he says.

He also says Professor Ambuj Tewari, who is teaching the foundation of deep learning to students who are early in their journey of figuring out their profession or research career, is taking a brave but important bet as an educator on a field that is so early in its development.

“You say, ‘this is not really formed yet, but it is special. There is something about this that I think is going to be transformative.’ That is the way the study of color was in the 1910s and ’20s, and the way that molecular biology was in the ’50s and ’60s,” Wiltschko says. “There was a moment where there was barely a thing there and some people took a bet on it to teach people about it. Those are the people that continue to grow what we know as a species.”

Learn about supporting the Department of Statistics