Computer vision applications are all around us, from unlocking your iPhone with your face to autonomous vehicles maintaining a safe distance from cars and pedestrians. With the rapid advances of AI, it’s a quickly growing field of engineering with applications in wide-ranging industries.
If you’re just dipping your feet in this space, it helps to get some computer vision projects under your belt to build your skills while developing a deeper understanding of the technology. These projects are even more fun when you can connect the machine learning (ML) models used for computer vision projects to real-world devices that can perform different tasks in your physical environment.
Combining online open-source platforms and Viam’s custom vision service, you can add ML to many devices you likely already have around your home. We’ve gathered 10 computer vision project ideas from the Viam community, complete with step-by-step instructions, so you can get your hands dirty and start building.
Let’s take a look.
Beginner computer vision projects
1. Hugging Face model deployment for enhanced vision systems
Skills targeted:
Model deployment, Hugging Face integration, and practical AI skills
Hardware you’ll need:
- Computer (macOS, Linux, or Windows)
- Webcam or compatible camera
Start experimenting with ML applications using Hugging Face’s pre-trained models integrated with Viam's vision services through our registry. By embedding these ML models into your system, you’ll be ready for advanced image analysis and pattern recognition with minimal setup.
After loading and configuring a pre-trained Hugging Face model in Viam, you can apply it to analyze specific visual patterns for your desired use case—like categorizing shoes or detecting hard hats based on trained ML data.
Steps to build:
1. Install ‘viam-server’ and add a camera component in the Viam app.
2. Add a YOLO vision service module (choose from YOLOv5 or YOLOv8) from our registry and configure it in your machine’s setup. Use our vision API to start writing script. See the full setup tutorial from Hugging Face.
3. Now comes the fun part: choosing a model based on your use case. Select and integrate any compatible Hugging Face YOLO model with the vision service—from categorizing your shoe collection to tracking machines on a construction site.
4. With your system set up, you can now test and visualize detections. A good place to start is using Viam's transform camera to display bounding boxes and then verify real-time object detection in the Viam app.
Take your computer vision project a step further:
Once you’ve created your vision system, customize it further using Viam’s APIs and flexible SDKs.
Jump to the full project tutorial for deploying Hugging Face models on Viam
2. Object tracking with a mobile rover (like SCUTTLE)
Skills targeted:
Color detection, robotics integration, and control systems
Hardware you’ll need:
- SCUTTLE robot kit or similar wheeled robot
- Webcam camera
Let’s build a robot with color detection capabilities! Your robot will be able to follow an object of a specific color—whether it’s a red soccer ball like in the video or your chihuahua sporting a bright yellow sweater. This computer vision project is an entertaining, hands-on way to learn object tracking with a robot.
Steps to build:
1. Assemble the SCUTTLE robot kit (or the Viam rover or another mobile rover) and attach and configure the webcam. Make sure the camera is connected to the rover’s board and then turn on power to the rover.
2. Set up the Viam platform on your machine and configure color detection. This example uses #a13b4c, a reddish color, but you can specify any color you like. Here’s a quick overview:
3. The detector will now draw boxes around objects of that color. Now test the detector from the CONTROL tab.
4. It’s time to program your rover for color detection. Set up your code environment using Virtualenv. Now, connect Python code and follow the steps in the tutorial to process the camera feed and detect the specified color.
5. Integrate the color detection results with the robot’s movement controls. Now, run the code again and test whether the rover follows an object of that color. Have fun with it!
Take your computer vision project a step further:
Next, try adding different colors for your rover to detect. You can even use the Viam SDK to train your rover to move in a square.
Jump to the full tutorial for following a colored object.
Intermediate computer vision projects
3. Light automation with object detection
Skills targeted:
Object detection, Python coding, and basic automation
Hardware you’ll need:
- Computer (running macOS or 64-bit Linux) with a webcam
- Viam-compatible hardware like a Kasa smart bulb/plug
- Table lamp
- Mobile phone to connect to smart plug
Have you ever wanted to turn a light on and off when you enter and leave a room, without using a smart lightswitch? This project uses object detection to control a lamp based on whether someone is nearby or not. Using a smart bulb, you’ll set up the light to respond automatically after detecting a person or movement.
This computer vision project involves using a Python object detection library like OpenCV or YOLO to capture live video data and detect objects in a specific area.
Steps to build:
1. Install ‘viam-server’ on your computer and add a new machine in the Viam app. Configure your webcam as an object detection camera. Then test the camera feed in the Control tab.
2. Set up computer vision services and configure the ML model on your machine.
3. Configure a transform camera to test bounding boxes for detected objects.
4. Set up the Kasa smart plug and install the smart plug Python API.
5. Write Python code to control your object detection device using the files in the GitHub repo for this project.
6. It’s time to test it out! Run the code using a command line from your computer.
Now, see how well this computer vision project works by moving in and out of the webcam’s line of sight to see if it detects people accurately. If everything’s configured properly, the system will detect you and trigger the smart plug to switch the lamp on and off.
Take your computer vision project a step further:
Modify the code to detect different types of objects listed in the ‘labels.txt’ file, like your cat or even a vacuum cleaner!
Jump to the full tutorial for automating your lights.
4. Home energy optimization with AI
Skills targeted:
Real-time monitoring, data analysis, and sustainability-focused engineering
Hardware you’ll need:
- ESP32-WROVER microcontroller with a camera (less than $10)
- Cloud server or personal computer to run ‘viam-server’
- SolarEdge inverter (or other compatible energy monitoring system)
If you’ve ever wondered how energy usage in your house fluctuates based on how many people are home, this computer vision project is for you. Sure, you may already have some smart home technology in place—like thermostats or batteries—but this project takes it to another level by correlating your energy usage data to the presence of people in your house.
With this computer vision project idea, you’ll be able to track when people enter and leave your home so that you can analyze how much energy is consumed based on how many people are in the house.
Steps to build:
1. Set up the ESP32 camera with ‘viam-micro-server.’
2. Configure the cloud server with ‘viam-server.’ Link the ESP32 as a remote part.
3. Set up AI services on ‘viam-server.’ Configure the ML model and computer vision services and then integrate OpenAI's ChatGPT vision module.
4. Implement person detection and energy monitoring. Use Viam Python SDK for detection and then integrate SolarEdge Monitoring API to track energy usage.
5. Now you’re ready to collect and analyze the data! The project creator used ChatGPT to analyze the raw data to extract insights into energy usage patterns, but feel free to use whatever tool you prefer. For data storage, use Viam’s Data Management or a similar service.
Take your computer vision project a step further:
To get a better sense of energy usage in other areas of the house, consider adding more sensors for appliances like lights, your HVAC system, or hot water heater. This data could help you keep track of where lights or heat/AC are being left on, or how much energy is being consumed by long showers.
Jump to the full tutorial for using AI to reduce home energy usage.
5. Helmet detection for job site safety
Skills targeted:
Safety-focused AI, real-time detection, and public safety applications
Hardware you’ll need:
- A computer (running macOS or Linux) or single-board computer (SBC)
- A webcam
This computer vision project takes us to the field of job site safety by automating the process of monitoring workers’ use of personal protective equipment (PPE)—in this case, wearing a hard hat or helmet. By detecting the presence or absence of helmets on people’s heads, this system helps companies enforce safety regulations and prevent accidents on job sites.
For this project, you’ll train object detection models to recognize helmets on workers’ heads. The system will scan live video feeds and then alert supervisors if a worker isn’t wearing a helmet.
Steps to build:
1. Connect your webcam to your computer and install the Viam server on your device. Then test your camera to make sure the video feed displays correctly.
2. Set up Viam's vision service and add the YOLOv8 module, then configure the hard hat detection model from Hugging Face.
3. Configure the ‘objectfilter’ module to process the camera feed and detect people with and without hard hats. Now test the detector to make sure it can detect people with hard hats and without hard hats.
4. Now, set up Viam’s data management features for data capture to upload images of people without hard hats to the cloud so you can see which workers aren’t wearing helmets.
5. The final step is to create a notification system. Write a serverless function to send email notifications each time the system detects a worker who isn’t wearing a hard hat. Make sure to test the whole system once all triggers are set up.
Take your computer vision project a step further:
Once you’re comfortable with this system, you can integrate it with mobile robots to more effectively monitor helmet usage throughout the job site.
Jump to the full tutorial for monitoring job site helmet usage.
6. Custom facial verification system
Skills targeted:
Facial recognition, security protocols, and model training
Hardware you’ll need:
- A computer
- Webcam
Learn how to create a facial verification system and create a simple and practical security solution for your home or office. You’ll learn two ML models for this project: facial detection and person recognition, and then train the facial detection models on facial images of actual people.
By comparing live facial data with stored images, your system will authenticate individuals and grant or deny “access” through an alarm system.
Steps to build:
1. Connect your webcam to your computer and install ‘viamserver.’ Configure the camera component in the Viam app.
2. Configure the ML model service. Then set up two vision detectors—a people detection model and a facial recognition model. Use the Viam app to add these services and configure their attributes.
3. Now you’ll add the module that can detect specific faces and arm or disarm the alarm based on that data. Add the verification system module to your machine and configure it to use the two vision detectors.
4. Finally, create the visual overlay by configuring a transform camera to display the current state of the verification system on your live camera feed. Now, you can see the system in action as it detects people.
Take your computer vision project a step further:
The project creator recommends using this system as a starting point to create your own DIY home security system.
Jump to the full tutorial for creating a facial verification system.
7. Pet photography and data filtering module
Skills targeted:
Image filtering, quality control, and AI-based image capture
Hardware you’ll need:
- Computer
- Webcam or other camera
- Colored item for your pet to wear
If you have a furry friend at home, this computer vision project will help you filter large amounts of data from a video feed to identify only images where your pet is present. It’s a great use case for keeping tabs on your hyperactive pup when you’re away for the day.
You’ll create an automated system that captures photos of your pet when they approach a designated area, like a food bowl, window, or door.
Steps to build:
1. Connect your camera to your computer and install ‘viamserver’ on your device.
2. Add a camera component using the Viam app. Then, add the ‘colorfilter’ module from Viam and configure it as a local module on your machine.
3. Configure both data management and vision services from Viam to detect the colored item and then save the captured pet photos to the cloud.
4. Now, configure your camera to filter by color and then set up data capture. Now, you’re all set to test your pet photographer system and make any adjustments.
Take your computer vision project a step further:
To filter the images with even more granularity, customize the code you’ve written and configure a sensor to eliminate blurry images or detect specific thresholds from your camera’s captures.
Jump to the full tutorial for Pet photographer: Create a data filtering module
8. Interactive bedtime songs bot
Skills targeted:
Human-centered design, sound integration, and real-time responsiveness
Hardware you’ll need:
- Computer with a webcam and speakers (or separate components that are already connected)
Have you ever wished you had a clone who could sing that same bedtime song over and over again to your kid? This project is for you—whether you’re a parent, aunt, uncle, grandma, or grandpa, or other doting relative. By using computer vision to detect a specific toy, the bot will play a specific, prerecorded bedtime song. You’ll give your vocal chords a rest while still offering the kiddos the comfort they need.
Steps to build:
1. If you’re using external cameras or speakers, connect them to your computer. Then install ‘viamserver’ and the Go Client SDK.
2. Configure your webcam to capture data, and then set up Viam’s data management services to save that data for analysis. Now, train the ML model to recognize specific toys by holding them up one by one in front of the webcam. Tag the data, filter it, and then train the model.
3. Now, set up your webcam to act as a shape classifier so it can recognize your toys.
4. Record the songs you want your bot to play for the kids. You can also download the project creator’s recorded songs on GitHub.
5. Program your robot to play a specific song when it sees a particular toy, using Viam’s Go SDK. Now test it out and see how it works.
Jump to the full project tutorial for building a bedtime songs bot.
9. Plant health monitoring with local LLM and ML
Skills targeted:
Local LLM deployment, plant care analysis, and sustainable design
Hardware you’ll need:
- A computer (MacBook, Windows with WSL, or edge devices like Raspberry Pi or OrangePi 5B)
- A camera (built-in webcam or external camera)
If plant care isn’t in your DNA, why not build an AI Assistant to handle it for you? This project combines computer vision and large language models (LLMs) to create an intelligent system for plant identification and care advice.
You can speak to the assistant, ask it questions about specific plants that you point to, and it will speak back to you with the requested information. No green thumb required!
Steps to build:
1. Install ‘viamserver’ on your computer and create a new machine in the Viam app following the installation instructions. Now set up the different components, including the camera and LLM. Make sure you also have the Viam Python SDK installed, then install the chat API.
2. Add a computer vision component to your machine and connect it to a Roboflow model for plant detection. Set up the necessary configurations in the Viam app.
3. Add a speech interface component so you can interact with the AI assistant through speech.
4. Connect computer vision to your LLM assistant by modifying some code. And now you can test it out by showing it plants and asking for advice on how to care for them.
Jump to the full tutorial for using a local LLM and ML for plant care.
Advanced computer vision projects
10. DIY hydroponic gardening system with vision technology
Skills targeted:
Smart agriculture and environmental sensors
Hardware you’ll need:
- PVC pipes for hydroponic setup
- Raspberry Pi 5
- BME280 and BME680 environmental sensors
- pH sensor
- Webcam
If you have a green thumb, or you’re an aspiring indoor gardener, this might be the solution you never knew you needed. By combining hydroponics (specifically the Kratky method), computer vision, and automation, this project creates an indoor smart garden that grows herbs and vegetables and uses ML for optimal plant care.
The system works by analyzing camera images and sensor data over time to monitor plant growth. By notifying you of any anomalies in the data—like spiking temperatures or dips in humidity—you can adjust the lighting and water levels to help foster the best growing conditions.
Steps to build:
1. Build the hydroponic system by constructing a PVC pipe system and then setting up the germination station.
2. Set up ‘viam-server’ on Raspberry Pi and connect ESP32, sensors, and webcam.
3. Configure the camera, set up time lapse imaging, and set up sensors for temperature, humidity, and pH monitoring.
4. Develop a data collection system, using Viam's Data Management for storage. Monitor your sensor data and make adjustments for optimal growing conditions.
Take your computer vision project a step further:
The next step will be to train and implement ML models based on the plant growth data you’ve collected. This will allow the system to predict growth and recommend adjustments for optimal conditions. You can also create a custom monitoring dashboard to visualize and interact with your hydroponic system's data in real-time.
Jump to the full tutorial for building a DIY hydroponic gardening system.
Build your own computer vision projects
From automating lights, to monitoring plant health, or building a facial verification tool, each project we’ve covered gives you hands-on experience with real-world applications of this technology. But these computer vision project ideas are just the tip of the iceberg. Keep building your skills with these Raspberry Pi AI project ideas.
With all of the open-source tools available today, including the Viam Registry, whatever computer vision project you can dream, you can build. Once you’ve sharpened your skills with these beginner-friendly projects, add your own twist to them and see what you come up with.
Join the Viam community
If you want to share your project outcomes or other computer vision project ideas and learn from others, join our Discord.
