How calorie counting from a photo works, and how accurate it is

By Robert Robinson, founder of Neato Health · September 15, 2026 · 4 min read

A photo of your plate becomes a calorie count in a few seconds. Here is what the model is actually doing, what the research says about how close photo methods get, and the situations where you should trust the number less.

Pointing a camera at a plate and getting a calorie count feels like magic, and the honest description is less magical and more useful: it is three ordinary steps done quickly, each with known strengths and known failure modes. Understanding them tells you when to trust the number and when to tap it and fix it.

Step one: what is on the plate

A vision-language model looks at the image and names what it sees: grilled chicken breast, white rice, steamed broccoli, a sauce. It does this the way it reads any image, from patterns learned across millions of labeled photographs, and it is good at it for common foods in ordinary lighting. It is less reliable for foods that look alike, a stew whose ingredients are hidden, or a dish from a cuisine it has seen less of. When it is unsure it tends to guess a generic version, “rice with meat and vegetables,” which is usually close in calories even when the name is vague.

Neato shows you the name it settled on. That is deliberate. If it says “chicken” and it was pork, or “sauce” and it was a quarter cup of oil, the fix is one tap and the calories follow.

Step two: how much of it

This is the hard step, for the model and for people. The image gives the model the plate, the food’s footprint on it, the utensils for scale, and its knowledge of typical servings. From those it estimates grams or a household measure for each item. Errors here are the main source of error in the final number, and the research on humans doing the same task, described below, shows that it is hard for everyone.

Three things help the estimate. Shooting from slightly above with the whole plate in frame gives the model the geometry it needs. A fork or a hand in the shot gives it scale. And a label, when there is one, removes the guesswork entirely: for packaged foods Neato reads the nutrition panel and uses the label’s own numbers, checked against the USDA’s FoodData Central database, so a packaged snack is as accurate as the label is.

Step three: the nutrition

With foods and portions in hand, the calories, protein, carbohydrate and fat come from nutrition data for each food. For whole and generic foods that is USDA data; for packaged goods it is the label. The arithmetic at this step is exact. The uncertainty is all upstream, in what and how much.

How accurate is that, really?

The fair comparison is not against a lab. It is against the alternative, which is you, estimating from memory at the end of the day. On that comparison the research is clear.

Lichtman and colleagues, in the 1992 study that every discussion of dietary reporting cites, measured actual intake with doubly labeled water in people who believed they ate very little, and found their reports were 47 percent below the truth. Across dietary surveys in general, written self-reports underestimate by 20 to 40 percent, more for snacks and for people trying to lose weight.

Photo methods do much better. Martin and colleagues validated the Remote Food Photography Method, in which trained analysts estimate intake from meal photos, against doubly labeled water and found it underestimated by about 3 to 6 percent, with the errors small enough for research use. Boushey’s 2017 review of image-based dietary assessment reached the same conclusion across studies: photos reduce portion-size error, reduce the burden of recording, and hold up over long periods better than any written method.

Neato’s model is doing the analyst’s job automatically, and automated estimation has its own error on top of the method’s. Individual meals can be off by a fifth in either direction. But the errors are not systematically low the way memory is, and they average out over a week of meals. For the purpose calorie counting serves, steering intake toward a target over weeks, a photo record that is unbiased and roughly right beats a written one that is precisely wrong by a third.

When to trust it less

• Hidden calories. Oil in a stir-fry, butter on vegetables, dressing under salad leaves, cream in a soup. The model cannot see what is absorbed or submerged. If you cooked it, you know; add it.
• Drinks. A glass of orange juice and a glass of water look the same. Say what it is, or log the drink separately.
• Deep bowls and stacked plates. Volume is hard to read when the food’s depth is hidden. A second photo from the side helps.
• Very large or very small portions. The model’s prior is a typical serving, so extreme portions pull toward the middle. A restaurant plate is more likely to be underestimated than a home one, as the restaurant guide explains.
• Foods you did not finish. The estimate describes what arrived. Cut the servings on the entry to what you ate.

Why every number stays editable

Neato treats the estimate as a draft. The description, the servings and the macros are all one tap from an edit, and the app remembers your corrections for foods you log repeatedly. The point is not to produce a perfect number from every photo; it is to produce a good number in three seconds, every time, for months, because the logging research says the record you actually keep is what changes the outcome. A perfect method nobody uses past week two loses to a good method used on day 200.

Neato’s guides are general wellness information built from the published research cited above. They are not a diagnosis, treatment, or medical advice, and they don’t know your history. For decisions about your health, talk to your doctor.