Identify Human and Objects using Webcam

Tools and approach

• MODEL: DeepSeek Flash (text and vision) by default; Gemini, OpenRouter, HuggingFace and OpenAI are selectable
• TOOLS: Google AI Studio initially finally Hermes AI Agent.
• BUILT WITH: a response schema — heldObject, isLogo, personDescription, backgroundDescription, confidenceScore — so the UI lays out fields instead of parsing prose
• NOTE: no API key needed — the analysis runs on the club’s own server, which holds the DeepSeek key

Point the webcam at something and the app reports what it sees: the object being held, whether it is a brand logo, a description of any person, a description of the background, and a confidence score. You can switch between image-classification models and read what each one returns.

This one took several iterations. We finally settled on this simplified, more reliable code running on DeepSeek Flash, and used Hermes as the agent harness to build it — the earlier versions are still on the site, and the difference in how reliably they run is the point.

Review

THE GOOD: Forcing structured output was the right first decision. The same frame can be described by three different models and the answer still lands in the same fields, so swapping models became a config change instead of a rewrite — which is exactly what saved this project when its original models went offline.

THE BAD: A visitor must grant camera access before anything happens, so it cannot be judged from a link preview or a screenshot. It also shipped as source that transpiled in the browser rather than as a build, which made it start slowly and fail quietly.

TO IMPROVE: Add a recorded fallback clip for people who will not grant camera access, and let two models analyse the same frame side by side — because comparing models is what this app is actually for.

What we learned

This was the first project where the AI did the building instead of writing snippets we pasted in — and the first one where the models aged out from under the code. The vision model it was originally written against is retired, and an earlier iteration needed every visitor to supply their own API key. Several iterations later, the version that finally works reliably is the simplest: one model (DeepSeek Flash), one response schema, and the key held on the club’s server so a visitor just presses the button. Planning the app in Google AI Studio and letting an agent execute the plan worked, but it made the hand-off points obvious — choosing the model changes what you get back, camera access has to be granted before anything runs, and a demo that depends on a visitor’s key is a demo most visitors will not run.