In Bedford–Stuyvesant, Brooklyn (BedStuy), Yuri Fukuda regularly walks by a mural that showcases prominent female leaders. Since October 2005, a stunning 3,300 square foot mural, When Women Pursue Justice by ArtMakers-NYC, has had a central place in the BedStuy art scene. The mural celebrates 90 women who have been key in causes like advancing voting rights, health and environmental standards, and more over the last 150 years in the United States.
Yuri Fukuda is an R&D Lead Senior Software Engineer, technologist, and mom living in Brooklyn, New York. Yuri is invested in using new technology to easily empower others to communicate their ideas. Her day job at EachScape focuses on enabling media companies to create their own mobile applications without writing code.
To Yuri, the importance of When Women Pursue Justice has only grown with time. Yuri noted that the four-story mural, which took 17 artists and five interns a full summer to compete under lead artist Janet Braun-Reinitz, has once been slated for demolition in 2014 before community organizing prevented said destruction.
Given recent advances in image recognition and a passion for using technology to create new experiences, Yuri wondered how she could use computer vision to enable those in her community to more effectively connect with the world around them – especially When Women Pursue Justice.
Building a Computer Vision Enabled Street Mural Application
Yuri set her sights on creating a mobile application that would allow the user to point their phone at the mural and better understand who it profiles. Through exposing the mural's subjects, Yuri intends to bring greater awareness to the women whom have helped shape our society.
So, Yuri set out to capture dozens of photos of the mural from various perspectives. She labeled the images with LabelImg, uploaded them to Roboflow, and created a much larger dataset of her source images – like applying image augmentation that would help her model learn what the mural looked like in various lightings, at different angles (based on how the user was oriented to the mural), and more. (New: You can annotate your images directly in Roboflow!)
From here, she closely followed a tutorial on how to create an object detection application so she could deploy the solution to her Android device.
Jim Su
Yuri then used a test set of images to try to out her application.
To her delight, the application successfully identifies the first three figures she labeled in her training data: Dorothy Day, Audre Lorde, and Shirley Chisholm.
What's Next
Yuri notes that this is only the beginning for her work:
"I wanted to create an initial prototype to show what is possible with computer vision and augmented reality on street murals. Based on the feedback I receive, I plan to adapt my development."
Yuri notes that her initial training dataset is incredibly limited and will benefit from labeling more of the figures highlighted in the mural as well as capturing photos in different lighting conditions. "I tried the application when it was raining, and I realized I need more images from gray days like this in my training data!"
Considering building your own augmented reality mobile application? Read how to design an augmented reality app and how we built a computer vision app for board games.
Yuri is developing a hub where neighbors can participate in community-based technology projects and share ideas, learn, and build. She strongly encourages others to take up creating their own mobile apps that can enrich art in their communities as well. "I'm a beginner to computer vision, but with resources like Roboflow's excellent documentation, I was able to create my app in one week."
We elated to support work like this. As always, we cannot wait to see what else you build.
Cite this Post
Use the following entry to cite this post in your research:
Joseph Nelson. (Nov 15, 2020). Bringing Street Murals to Life with Computer Vision. Roboflow Blog: https://blog.roboflow.com/street-mural-computer-vision/
Discuss this Post
If you have any questions about this blog post, start a discussion on the Roboflow Forum.