Adding interoperable AI at every stage of the food supply chain

‍No-code configuration and standardized APIs = interoperability and faster iteration

As industries evolve, AI technology offers transformative solutions for a wide range of challenges. Whether you are managing a chain of grocery stores, improving equipment fleet management, or automating production lines in food manufacturing, AI can make a significant impact.

There are many options when it comes to adding AI to your projects. From computer vision (CV) models to large language models (LLMs), the menu grows each day. If you’ve ever been overwhelmed by a large diner menu, choosing AI for a project can sometimes feel like being faced with every diner menu you’ve ever seen, all at once. Beyond choosing the AI technology itself, you’re also faced with decisions and learning on how to run it, where to run it, and how to integrate it into your project.

Let’s examine the completely fictional grocery chain FoodChoice, which has realized that they are losing hundreds of thousands of dollars per year on damaged fruit and vegetables due to consumers squeezing them to test for ripeness. FoodChoice thinks that they can use AI to monitor customer behavior, aggregate learnings, and devise alternate ways for customers to know if the produce is ripe.

Managing AI models with Viam

The software engineers at FoodChoice first attempt to use a pre-trained ONNX-based computer vision model running on their local machines.They decide to use the Viam platform to manage their models, as Viam offers solutions to centrally manage versioning, deployment, and monitoring of AI models and code. 

Using the ONNX model with Viam requires only that they upload the model through the Viam app, click to deploy it to their machines, and visually configure an ONNX vision service from the Viam Registry. 

‍

Unfortunately, they find that the particular model does not accurately detect the consumer behavior they are looking for, so they begin to collect image data using Viam Data Management to train their own Tensorflow Lite model. 

When they are ready to test the new model, despite the model being a different format requiring a different machine learning (ML) framework, the Viam Registry handles infrastructure setup and Viam SDKs allow them to use it without changing their code:

detector = VisionClient.from_robot(robot, "produce-vision")
response = await detector.get_detections_from_camera("avo-cam")

Incorporating vision language models (VLMs)

Despite the new model being a bit more accurate, the folks at FoodChoice are still getting a number of false positives and false negatives. They learn that vision language models (VLMs) are a newer technology that might complement traditional computer vision models to improve their results. While VLMs are slower, they can use more context from both images and language to more accurately understand the task (in this case, if someone is squeezing an avocado).

They first try the multi-modal vision capabilities of the popular ChatGPT platform using the chatgpt-vision module from the Viam Registry. They add it to their machine configuration with a few clicks and add their OpenAI credentials.

The FoodChoice engineers continue to use the CV model to detect if a person is standing near the avocado display, but when this is detected, they then leverage the VLM, asking “Is someone squeezing the avocados?”

To do this, they add two lines of code—one to instantiate the VLM as a second vision service, and another to “classify” the image, passing in the question as a parameter.

By implementing the VLM as a second AI layer, accuracy is greatly improved. 

However, they realize that there might be some legal complications in sending images of shoppers to a shared cloud service. They notice that there are some open-source VLMs on the Viam Registry like Moondream and Uform that can be run locally within the store. As before, they find that the Viam Registry handles the setup required to run these models, and they can be used and tested without code modification.

vlm = VisionClient.from_robot(robot, "moondream-vision")
response = await vlm.get_classifications_from_camera("avo-cam", 1, extra={"question": "is someone sqeezing an avocado?"})

FoodChoice is using a fairly inexpensive local CPU, so VLM performance is slower than what they witnessed with ChatGPT. The local speed may be tolerable for now, but they plan to consider using VLMs with the Viam platform in conjunction with isolated cloud serverless solutions like Modal, which can be integrated with Viam Registry modules. 

They also plan on building a custom web portal for their data scientists, and find it beneficial that Viam’s gRPC-based API provides SDK interfaces that remain consistent and familiar from language-to-language:

const vlm = new VisionClient(client, "moondream-vision");
let response = await vlm.getClassificationsFromCamera("avo-cam', 1, {"question": speechText});

Companies like FoodChoice can leverage the Viam platform to accelerate their progress by utilizing AI technologies across various projects—ultimately spending less time and money. 

Other use cases

AI offers powerful solutions for overcoming the challenges faced at every stage of the food supply chain. 

Beyond point of sale at a grocery store, we’ve seen innovative companies adopting AI technologies across a range of use cases: from  predictive maintenance and smart equipment management to robotic automation and enhanced QA processes, AI can drive significant improvements in efficiency and quality.

Platforms that facilitate easy integration and management of AI models, like Viam, can further accelerate progress, allowing companies to adapt quickly and effectively.

Viam dramatically reduces iteration time by making ML, vision, and language models hot-swappable, enhancing the accessibility and usability of AI technologies. How will you use AI to make your machines smarter? Learn more about Viam and get started today.