%20(1600%20x%201000%20px).png)
Have you ever dreamed of transforming everyday devices—like a fridge, coffee machine, light fixture, or office appliance—into smarter, artificial intelligence (AI) powered tools?
We just wrapped up our inaugural Viam Challenge, where engineers, developers, and AI enthusiasts did exactly that—and now it's your turn!
Participants leveraged the Viam platform to bring their AI project ideas to life, from configuring new devices and deploying machine learning (ML) models to customizing applications that revolutionized everyday items like lights, laptops, and microcontrollers.
Here are the top five projects from the challenge that will inspire you to start your own AI-powered endeavor:
- DEB-Forge: A Viam-powered smart desk with a contextual lighting system
- Betsy: Your new personal office assistant
- AI Assisted Smart Trash System: Revolutionizing waste management
- Robo DeMo 1000: A companion robot for distance runners
- A Personal Assistant Using Raspberry Pi 4 and LLM Using Gemini API
The projects on AI were judged on their innovative use of the Viam platform, the completeness and functionality of their design, their applicability to real-world scenarios, and the quality of their documentation.
Let’s dive into these amazing AI projects for beginners and experts alike.
1. DEB-Forge: Viam Powered Smart Desk With Contextual Lighting System
Ever dreamed of a desk that understands your workspace needs? Enter the state-of-the-art smart desk created by Bevis Halsey-Perry, which not only won first place in the Viam Challenge but also revolutionizes how you work.
The desk (also known as DEB-Forge) integrates machine learning and robotics to feature a contextual lighting system. By combining vision detection with addressable LED technology, it illuminates exactly where you need it, significantly enhancing both productivity and comfort.
Experience the future of workspaces with Bevis's award-winning smart desk.
Project Summary
Difficulty level
Medium to advanced
Viam components & services
- Camera component
- Data Management Service
- Vision Service
- ML Model Service
- Python SDK
Shopping list
- Raspberry Pi 4 with 'viam-server' installed
- GoPro Hero 3 with Camlink 4k or a webcam
- WS281 RGBW LED Light Strip, aka Neopixels
- 5v 10a PSU
- Wiring supplies for soldering the LED strip
- A desk for the demonstration
Step-by-Step Instructions
- Set up the vision system by attaching the physical components and configure the smart machine in the app.
- Take photos of different hand gestures, capture the data, and train a ML model using these captured photos.
- Deploy and test the model by adding a detection camera, as well as Vision and ML Model services.
- Set up the lighting system by connecting Neopixels to the Raspberry Pi.
- Link the vision system to the lighting system by writing the main script logic using the Viam SDK.
See the full tutorial for this project
Tips & insights from the creator
Around their inspiration for the project:
No more having to get up every time you want to adjust the degree of illumination in your home. Recognizing the significance of lighting in a workshop area, especially for someone with light sensitivity, Bevis envisioned a solution that would light only the area where they were working.
Around using Viam:
"Once the dataset was sanitized, segmented, and labeled, I began training a model using it on the Viam platform. After just 5-10 minutes, the model was trained. Easily one of the most accessible online ML systems I've used!"
Regarding future upgrades:
Bevis has a list of upgrades he’s looking to make. Some include:
- Contextual Lighting Features: Training models to recognize hand shapes for controlling light brightness and color, with added settings like party mode.
- Desk Height Adjustments: Currently using an electric motorized IKEA frame for adjustable desk heights, with potential future enhancements to enable automatic adjustments via object detection or sensors.
- Videography Control: Integrating cameras and lights into the desk to support content creation, with capabilities for controlling camera angles, automating lighting for ideal exposure, and incorporating subject tracking.
A tip to help other build DEB-Forge:
To effectively track the location of a hand on the desk, Bevis suggests you train an object detection model rather than a simple classification model. This approach involved selecting a bounding box labeled 'hand' over each hand in every image to pinpoint its exact location.
2. Betsy: Personal Office Assistant
Ever find yourself swamped with tasks at work? Meet Betsy, your new office ally. Developed by Yogendran Govikrishnan and their team, this personal assistant robot streamlines your daily tasks and boosts your workspace experience.
Whether it’s ferrying tools, delivering messages, or providing a dose of entertainment, Betsy has you covered—and the robot even was awarded second place in the Viam Challenge. Now, you can tackle your busy schedule with Betsy by your side!
Project Summary
Difficulty level
Medium to Advanced
Viam components & services
- Components: board, motor, base, camera, input controller, sensor
- Base Remote Control Service
- ML Model Service
- Vision Service
- Speech module
- Python SDK
Shopping list
- Raspberry Pi 4 with 'viam-server' installed
- Assembled SCUTTLE rover
- T-slotted framing and the brackets for the robot
- Ultrasonic sensor - HC-SR04 (generic) x 3
- USB webcam 1080P HD camera
- Wireless game controller, EasySMX
- Computer speaker, Pro-Sound 2000
- Geared DC motor, 12 V x 2
- Motor driver
- DC-DC converter
- 12v lithium battery x 3
- AMS-AS5048 encoder x 2
- HDMI capacitive touch monitor
- A box to hold tools
- Emergency stop switch
Step-by-Step Instructions
- Attach the physical components to assemble Betsy.
- Configure your components and services in the app.
- Write the main script logic using the Viam Python SDK.
- Use the Vision Service and Sensor Service to detect people and avoid obstacles.
- Implement logic using Viam’s Motion Service and Navigation Service to move the robot between different locations based on user commands.
- Create Retrieval-Augmented Generation (RAG) for LLMs by optimizing the output of the LLM.
See the full tutorial for this project
Tips & insights from the creator
Around their inspiration for the project:
Inspired by previous creations—Bella, their in-house chatbot, and Tipsy, Viam’s drink-serving robot—the team developed Betsy, a versatile personal assistant robot. Initially designed to transport tools and items within the lab, Betsy’s scope quickly expanded. Now, she not only efficiently delivers messages and documents but also serves as an entertainment hub, offering lively interaction and companionship in the office environment.
Around using Viam:
Yogendran and their team said, “The intuitive UI for configuration streamlined our development process, making it easy to set up and manage the robot's functionalities even by an amateur. Additionally, the active Viam community on Discord provided invaluable support, allowing us to quickly address any challenges we encountered along the way.”
Regarding future upgrades:
- Security Patrol: Using Viam’s motion detection and facial recognition modules, Betsy could autonomously monitor spaces and identify personnel.
- Self-Docking Capability: Inspired by our Principal Developer Advocate Matt Vella’s YouTube Video, Betsy could autonomously recharge, ensuring continuous operation.
- Music Player Integration: Betsy could play music from platforms, like Spotify, and be enhanced with a custom speaker developed through Viam’s tools.
Some tips to help others build Betsy:
The team created a comprehensive personal assistant robot for under $300, proving that innovations in robotics, automation, and AI don't have to break the bank. Like them, focus on leveraging creative solutions and accessible platforms to efficiently reach your project goals.
3. AI Assisted Smart Trash System
If you've ever hesitated at the recycling station, unsure whether to toss items into paper, plastic, or general waste, this solution is tailor-made for you. Shadi Naguib's third-place winning project, the AI Assisted Smart Trash System, uses your laptop's camera and a bit of AI magic on the Viam platform to sort waste accurately and light up the right bin for each type of trash.
Project Summary
Difficulty level
Medium to Advanced
Viam components & services
- Camera Component
- ML Model Service
- Vision Service
- YOLOv5 Module
- Python SDK
Shopping list
- A computer running viam-server
- ESP32 microcontroller
- LED strips (various colors)
- Breadboard and connecting wires
- Power supply for ESP32 and LEDs
- Hugging Face account for accessing AI models
- Arduino IDE
- Jumper wires
Step-by-Step Instructions
- Set up the object detection model by configuring the camera, Vision Service and the object filter module.
- Set up ESP32 WebServer by connecting hardware and implementing code.
- Write the main automation script logic using the Viam SDK.
See the full tutorial for this project
Tips & insights from the creator
Around their inspiration for the project:
“Every year, billions of tonnes of waste are generated worldwide, and the vast majority ends up in landfills due to poor sorting and recycling practices. This project tackles this issue by using technology to make waste sorting efficient and accurate,” said Shadi.
Regarding future upgrades:
Shadi said, “Feel free to add new features or improve the system's accuracy by fine-tuning the AI model.”
Some ideas from Hazal, a Developer Advocate at Viam, include:
- Advanced Material Detection: Improve AI to identify and sort waste by material composition, utilizing sophisticated sensors and enhanced ML models.
- Voice Command Integration: Enable voice interaction, allowing users to inquire about proper disposal methods and receive vocal or visual guidance directly from the system.
- IoT Connectivity: Equip the system with Internet of Things capabilities to remotely monitor bin status, fullness levels, and system alerts, optimizing waste management operations.
4. Robo DeMo 1000 - Companion Robot for Distance Runners
If you’re a long distance runner, you know that sometimes self-motivation can be tough. That’s why Sir Walter Richardson, a runner themself, created Robo DeMo 1000: a companion robot specifically designed to be your personal cheerleader during the most challenging runs.
With the integration of Computer Vision and Text-to-Speech technology, Robo DeMo 1000 offers personalized motivational—or, if you need it, demotivational—messages tailored to your pace and performance.
Project Summary
Difficulty level
Medium to advanced
Viam components & services
- Components: board, camera, motor, input-controller
- ML Model Service
- Vision Service
- Chat LLM Module
- Python SDK
Shopping list
- Raspberry Pi 5 running viam-server
- Raspberry Pi RPI5 active cooler
- 16-Channel PWM Driver (PCA9685)
- DC motor (generic) x 4
- Motor controller DBH-12V x 2
- Webcam, Logitech® HD Pro
- Development Board, Class D Audio Amplifier Module
- USB soundcard
- XBOX controller
Step-by-Step Instructions
- Connecting hardware components using the wiring diagram found in the tutorial, optionally 3D printing the enclosure.
- Configure your machine in the app with all the components and services, such as ML Model Service and Vision Service.
- Integrate the text-to-speech and ChatGPT modules into the machine
- Write the main automation script logic using the Viam SDK.
See the full tutorial for this project
Tips & insights from the creator
Around their inspiration for the project:
“I'm an avid distance runner who loves to encourage fellow runners during my training sessions. However, with the increasing number of people taking up running, it's become challenging to personally motivate everyone you encounter on your routes.”
Around using Viam:
“Today, AI services are cool and accessible. Access to technologies, such as Viam and other open source software helps bring all those components [AI, speech, and sports performance] together.
Regarding future upgrades:
While Sir. Walter has many plans for this robot, here are a few:
- Advanced Facial Recognition: Enhance Computer Vision models to detect runner's expressions, triggering motivational or humorous messages based on their current emotional state during runs.
- Interactive Speech Response: Integrate ChatGPT technology to enable the robot to respond in real-time to commands or remarks shouted by runners, enhancing interactive communication.
- Smart Navigation: Utilize Viam’s built-in CV capabilities to recognize path markings and avoid obstacles, ensuring smoother navigation and safer training environments for runners.
5. A Personal Assistant Using Raspberry 4 and LLM Using Gemini API
Ever dreamed of a device that doesn't just listen but truly understands and responds? Meet Saikat Mukherjee's creation: a smart assistant powered by Raspberry Pi 4 and advanced LLM models via the Gemini API.
Speak up—it answers questions, analyzes images, and more, providing a dynamic and interactive experience on command.
Project Summary
Difficulty level
Medium
Viam components & services
- Components: board, camera
- ML Model Service
- Vision Service
- SpeechIO Module
- Roboflow Vision Module
- Python SDK
Shopping list
- Raspberry Pi 4
- Pi camera
- USB speaker
- USB microphone
- MicroSD card
- Power supply for board
Step-by-Step Instructions
1. Connect to Raspberry Pi and install viam-server onto the device.
2. Configure your machine’s board and camera in the app.
3. Add computer vision to the device via Viam’s Vision Service.
4. Set up the speech module, allowing text to speech from an LLM.
5. Integrate the Gemini API by including GenerativeModel for chat purposes and Vision Service for imagine input.
See the full tutorial for this project
Tips & insights from the creator
Around using Viam:
“Office hours were very helpful. Received lots of guidance from Nick.”
Like Saikat, take advantage of our Discord community to receive direct support from the engineers who are actively developing Viam.
A tip to help others build this device:
“Unlike the chat model, the vision model from Gemini doesn't provide support for follow-up questions regarding the image input inherently. To facilitate the follow-up question for a given image input, pass the previous response from the model in the prompt along with the follow-question from the user.”
Get started on building your own robot
The projects showcased in the Viam Challenge truly exemplify the innovative potential of AI integration into everyday devices. Each project offers unique solutions to common problems, demonstrating the versatility and power of AI-driven applications.
Now’s your chance—follow the “Hack a device with AI” prompt in our past challenge, create one of the community projects above, or try one of our AI-powered tutorials. By doing so, you’ll may just learn how to:
- Master hardware setup
- Make your hardware smarter with software and AI technology
- Design intuitive interfaces and interactive flows
- Problem-solve effectively
Join the Viam community
If you want to stay tuned for new Challenge updates, network with other passionate individuals, and get direct help from the team of engineers, join our Discord.
Together, let's push the boundaries of AI innovation and create a smarter, more connected world.
About the Author: Hazal Mestci is an artist and technologist with a focus on hardware and software interaction, and a Developer Advocate at Viam. Her day-to-day involves coding, designing, wiring hardware, and leading initiatives to engage, educate, and inspire other developers. She holds a masters degree in Computer Science and spends her time building robots and smart machines that highlight autonomous navigation, AI, IoT, sustainability practices and climate tech.
