Using Computer Vision to Clean Up San Francisco Streets

“Data is like garbage. You’d better know what you are going to do with it before you collect it.” - Mark Twain

Introduction

4.5 trillion cigarettes are at risk of entering our waterways and polluting up to 50 trillion liters of water, according to the WHO.

Every year, millions of tons of litter end up on streets and waterways worldwide, with urban areas facing the brunt of this environmental challenge. San Francisco, known for its iconic landmarks and tech innovation, unfortunately, is no stranger to this issue. Some streets in the city tell a disheartening tale of litter and hazardous waste accumulation, highlighting an urgent need for innovative solutions.

To help cities tackle this problem, Emin Israfil founded Rubbish. Rubbish is making strides in San Francisco and beyond, gathering crucial environmental data to address the litter predicament. Since 2017, Rubbish has been on a mission to create clean, healthy, and sustainable neighborhoods for everyone. With recent advancements in AI, the Rubbish team saw an opportunity to build computer vision-powered features to unlock new ways of capturing litter data.

At the Cerebral Valley: AI in Climate Tech Hackathon, the Rubbish team introduced LitterBug, an AI-powered tool that employs real-time vision models to detect, map, and spotlight litter hotspots in urban areas. Their efforts didn’t go unnoticed:

• 🥈2nd place overall
• 🏆 Winner community vote
• 🛴 Winner transportation prize

In this post, you’ll learn how capturing and understanding litter data is helping organizations make the world a cleaner place.

Why Litter Data is Important

“You can't improve what you don't measure.” - Peter Druker

In the realm of urban development, this adage rings especially true. Litter and environmental data serve multiple critical functions. It tracks year-over-year progress, informs where funding is most needed, optimizes the placement of infrastructure like trash cans, and sheds light on the silent struggles of underserved communities. Moreover, this data holds significance beyond local boundaries; agencies like the EPA and NOAA utilize it at the federal level to implement national clean water policy.

Case Study: SOMA West, San Francisco

Before 2019, SOMA West had a reputation as one of San Francisco's most littered areas. That year, the newly established SOMA West Community District teamed up with Rubbish, mapping concerns ranging from litter to hazardous waste. In two weeks, residents and staff mapped over 27,000 street issues they wanted to see addressed. With this data, the SOMA West team engaged city officials, targeted hot spots, secured additional resources via need-based grants, and kept themselves accountable to their community.

The outcome? A follow-up 2020 survey showed a significant reduction in litter and related hazards, including a 58% reduction in needles and 85% in human/pet waste, enhancing the district's environment for its residents. Read more about the SOMA West case study.

Rubbish and LitterBug: A Perfect Match

LitterBug is powered by dash camera footage, and employs real-time AI to detect and map litter across the city. More than just identifying litter, LitterBug crafts a comprehensive heat map, spotlighting litter hotspots.

This isn't just data; it's a tool, a guide to understanding and addressing the most affected areas. If you're curious to see LitterBug in action, here's a glimpse with this 1-minute demo.

Harnessing Roboflow for Real-time Litter Detection

Building an AI model from scratch, especially under the time constraints of a hackathon, can be a daunting task - especially after crashing your bicycle camera rig in the middle of SF traffic (see video above!). This is where Roboflow played a pivotal role.

With Roboflow, the Rubbish team was able to quickly preprocess and augment image datasets and train vision models. They managed to do this in the wee hours of the morning, and still had models up and running in time for the hackathon presentation.

The power of Roboflow's tools allowed the team to focus on optimizing LitterBug's capabilities without getting bogged down by the intricacies of model training.  

Impact and Future of Using AI to Create Cleaner Cities

LitterBug, initially conceived as a proof of concept for the Cerebral Valley: AI in Climate Tech Hackathon, showcased the transformative potential of AI in addressing real-world challenges. Since the hackathon, the team has been hard at work refining the litter detection models, fine-tuning the implementation, and gearing up for a production-ready launch as a feature in the Rubbish iOS App, expected by the end of the year.

And while San Francisco was the starting point, the potential of LitterBug extends far beyond. Partnerships around California are gearing up, particularly as part of the state’s Clean California initiative. With the ever-evolving landscape of AI and edge computing, tools like LitterBug can be deployed across cities globally, turning the tide in the fight against urban litter.

Get Involved

Contribute Datasets: Not all trash is created equal. If you have specialized models you'd like to contribute, whether for cigarette detection, brand identification, or any other specific litter type, contribute them to be integrated into the Rubbish application. Your expertise can help categorize litter even more effectively.

Join Engineering Projects: Passionate about making a difference and have the tech skills to back it? Rubbish is looking for volunteers to help with open-source engineering projects. Reach out directly to emin@rubbish.love and be a part of the change!

Support Open Data Initiatives: Partnering with NoTraceTrails & the Moore Institute for Plastic Pollution Research, Rubbish is embarking on the largest overland litter and micro-plastic survey ever conducted. Covering a whopping 2,650 miles along the Pacific Crest Trail and beyond, the team will be hiking and surveying along way. Want to be a part of this ambitious project? Join the Trash Team here.

Conclusion

In the intersection of technology and environmental advocacy lies the potential for transformative change. LitterBug, born out of a hackathon and nurtured by collaborations like the one with Roboflow, stands as a testament to this.

As developers and tech enthusiasts, it's upon us to leverage our skills, tools, and platforms to address real-world challenges. After all, every line of code can be a step towards a cleaner, greener future.