Build backstories: The smart device that aids in school safety

We recently attended PennApps XXIV— a vibrant, student-led hackathon where young minds came together to craft solutions aimed at addressing real-world challenges. 

Our focus was on submissions that exemplified the “Best use of Viam software in robotics.” From smart machines built to ensure school safety to those focused on recycling, these students delivered!

In the second blog of this series, we’re spotlighting the talented group behind Project LearnSafe: the winner of our award. This smart machine leverages machine learning (ML) to identify potential school threats and quickly alert the authorities. 

Dive into this Q+A and gain insights directly from the visionary creators about their inspiration, journey, and pivotal lessons from this endeavor.

Tell us about yourself and your team!

We're all sophomores at the University of Pennsylvania in the School of Engineering & Applied Science. We're a team, but also roommates, and decided to sign up for PennApps together to have a new experience and combine our diverse skillset to create something unique. 

Can you walk us through the initial concept of your build? 

Students of all ages should be able to learn comfortably and safely within the walls of their classroom.

According to the Washington Post (June 2023), since Columbine in 1999, more than 357,000 students in the U.S. have experienced gun violence at school. 

The intention of our ML model is to contribute a proactive approach that requires only a few pieces of technology but is capable of an immediate response to severe events. 

Our model is trained to recognize active threats with displayed weapons. When the camera senses that a person has a knife, it automatically calls 911. 

We also created a ML model that uses CCTV camera footage of perpetrators with guns. Specifically, this model was meant to be catered towards guns to address the rising safety issues in education. 

How did you use Viam for your build?

The majority of our project was built using Viam’s app and ML Model Service. 

First, we obtained an SD card with the IOS for Raspberry Pi and then downloaded viam-server onto the single-board computer.

To create a model that could positively identify people carrying dangerous weapons, including knives and guns, we:

• Searched the web and imported CCTV images of people with and without guns into Viam’s Data Management Service.
• Configured a camera within the ‘Config’ tab of Viam’s app and used this to take additional images of ourselves holding knives.
• Trained two models—one for CCTV footage, the other for images we captured—from the same pool of data within Viam by labeling images using a border bounding box functionality, filtering them, and clicking “train.”‍
• Deployed the models on our smart machine by configuring Viam’s Vision Service and ML Model Service.

After testing for recognition, we grabbed our starter code from the Code Sample tab directly in Viam’s app and dropped this code into Visual Studio; our favorite IDE.

We then integrated Twilio into our project, which allowed for an automated call feature once the ML model detected the dangerous weapons.

What was the learning curve like for Viam's platform?

The Viam platform was ideal for our goal and made the process much more efficient. No one in our team had experience with using Viam or with using ML, but their documentation had comprehensive tutorials that we were able to use to get a start on the project. 

The platform itself was incredibly convenient as it processed our images into the data tab immediately, which allowed us to prepare the data and train it all from the same place.

Additionally, the Viam team was incredibly supportive throughout the entire process.

They provided us with several Viam tutorials to help us get started, explained the ideal way to capture our data for accurate results, and gave us all the tools we would need. 

How will you upgrade your current hackathon build?

We hope to improve our machine learning model in a multifaceted manner. 

First, we’d incorporate a camera with better quality and composition for faster image processing to make detection in our model more efficient and effective. 

We’d also add more images, taken in different locations with different lighting, to our model to amplify our database in order to make our model more accurate. This would ultimately improve pattern recognition and expand the scope of detection. 

Finally, we’d test our machine learning model for guns with CCTV, and modify both models to include more weaponry.

Any final thoughts you would like to share with aspiring builders and innovators?

‍

Sonali: You don’t have to know everything there is to know in order to start building. Share your strengths with others, build off of theirs, and know that there are always people willing to help you.

Anika: Don’t be afraid to try new things! The struggle through a new concept makes the learning curve and final product even more rewarding.   

Nandini: Everything is learnable. Don’t ever look at someone else and be intimidated by their skills or knowledge. Just start. That can be you if you just put in the time and effort. 

‍
Feeling motivated to construct your own smart machine with Viam? Dive into our documentation to kickstart your journey and immerse yourself in our community to explore other innovative creations that will fuel your inspiration!