On slides, computer vision is clean. High-res images. Green checkmarks for "good," red crosses for "bad." A neural net somewhere in the cloud doing magic. The caption says "zero-defect manufacturing." On an actual line, the camera housing is smeared with oil, the lens is vibrating on a rusty bracket, the lighting changes) every time someone opens a maintenance hatch, and operators are fighting a jammed conveyor while your "smart" model quietly misclassifies half the parts. Most failures come from the same root error: treating the factory floor as a static dataset problem instead of a moving, dirty mechanical system. If you want computer vision that actually reduces scrap and downtime, you have to start by accepting a blunt fact: Sometimes a thirty-dollar sensor plus a decent bracket beats whatever model you were planning to train. ## The slide versus the line In the office, the problem is framed like this: We have defective parts.
We have cameras.
We have AI. Therefore, we will train a model to detect defects and solve quality. On the line, the actual constraints are: The part is moving.
The position is not perfectly repeatable.
The surface may be covered in dust, coolant, or scale.
The line layout was built fifteen years ago by someone who retired.
Operators have thirty seconds per hour to deal with your system, if that. That gap shows up in predictable ways: – POCs with perfect, hand-picked images that crumble the first day the coolant splashes differently.
– Models see-hear-understand tuned for "average" parts that panic when a new supplier's material changes texture.
– Latencies that are fine in a lab and disastrous when the reject gate has 90 milliseconds to make a decision. You cannot fix these with better convolutional layers. You fix them with better mounts, better lighting, cleaner triggers, and simpler problem definitions. ## Why "just add a camera" usually breaks There are three enemies: environment, variation, and timing. ### Environment Factory air is not clean. You have: – Dust and fibers in textiles.
– Oil mist and coolant spray in machining.
– Steam and condensation near wash stations.
– Scale and heat shimmer in steel and glass. Every one of those will: – Foul lenses and enclosures.
– Change apparent contrast and color.
– Introduce glare or bloom in ways your carefully collected training training models without centralizing data images never saw. A classic pattern: week one, model performance looks good. Week three, a thin film on the lens reduces contrast just enough that fine defects disappear. Nobody notices until a customer complaint arrives. ### Variation No two shifts run the line quite the same way. – One operator bumps the camera while clearing a jam.
– A maintenance tech replaces a belt tensioner, changing the part position by a few millimeters.
– A different batch of material has a slightly different finish. Your model, which was exquisitely tuned to a particular pose and texture, starts throwing false rejects. The operators, who are judged on throughput, quietly switch it to bypass. ### Timing On a moving line, you have a fixed budget between "part under camera" and "reject or accept." An example shape: – Encoder or sensor trigger fires.
– Strobe or lighting fires.
– Camera exposes and transfers the image.
– Edge box or PLC runs the model.
– Result must arrive before the part reaches the reject actuator. If your CV pipeline occasionally spikes from 20 ms to 120 ms because the edge device is doing other work, no one in IT cares. On the floor, that means good parts get kicked and bad parts slip through. Factory control systems)-reliability engineering grew up around deterministic timing. Most ML pipelines did not. ## Start with physics, not pixels Before you design anything with a camera, you need to sit with the maintenance lead and the line engineers and do something almost no one bothers with: a physical failure-mode analysis. For each problem: – What exactly is going wrong?
– Where in the process does the defect or failure become visible?
– What physical signals change when it happens: vibration, temperature, current, pressure, position, sound, geometry?
– How fast do those signals evolve relative to line speed? Only then ask: Is vision actually the cheapest, most robust signal we can use? Often the answer is no. You can catch a misaligned belt by watching power consumption and speed. You can detect a missing fastener with a simple proximity sensor. You can see a jam by looking at back-pressure or photoeyes. Vision is a last resort when you cannot instrument the thing more simply, or when the defect is intrinsically visual asset-models patterns tropes and backlash: surface scratches, label misprints, cracks, wrong color, missing components in a dense assembly. ## Where classical sensors beat cameras There are whole classes of problems where computer vision is the wrong hammer. ### Presence and counting If you just need to know "is something here or not," or "how many passed this point," a pair of photoeyes, an encoder, and a small PLC program are usually better than an entire imaging system. They are: – Boring.
– Deterministic.
– Easy to troubleshoot with a multimeter. ### Alignment and position If you need to know whether something is in the correct mechanical position, a limit switch, LVDT, or rotary encoder will give you a clean, thresholdable signal with no lighting drama. ### Process health Bearing going bad? Vibration and temperature see it earlier and more reliably than image wobble. Pump cavitating? Listen to noise and pressure. These signals are already part of most industrial control stacks. Even in vision-heavy lines like packaging, adding a one-shot sensor that detects "flap not folded" or "carton not present" often saves more downtime than trying to infer everything from images. ## When vision is the right weapon Vision makes sense when: – The defect is genuinely visual: scratches, dents, discoloration, contaminant particles, cracks, missing print.
– The range of acceptable variation is wide enough that hard-coded rules would explode in complexity.
– The economics justify it: the cost of a missed defect or recall dwarfs the cost of cameras, compute, and engineering. Examples that actually hold up: – Printed circuit board inspection where tiny solder bridges or missing components matter and rules-based systems are brittle.
– Bottle or can inspection where chips, cracks, or fill-level deviations must be caught at high speeds.
– Assembly verification in automotive interiors, where mix-ups on trim or color are expensive and frequent.
– Safety zones around robots or autonomous vehicles, where you need to detect humans and unexpected obstacles reliably. Even then, the systems that work share a pattern: they narrow the problem to a tight slice and treat everything around the model as part of the solution. ## Designing computer vision for dirty reality Almost every successful factory-floor CV system looks "over-engineered" to people who only write models. It has four layers that matter as much as the neural net. ### 1. Mechanical and optical discipline You build a stable world for the camera: – Rigid mounts tied into the same structure as the machine, not a vibrating handrail.
– Enclosures rated for dust and washdown, with accessible windows and air-knives or positive pressure to keep them clean.
– Fixed working distance and part presentation: guides, rails, or nests that ensure the region of interest is where you think it is. For more insights on this topic, see our analysis in Mixture-of-Experts at Scale: Sparse Compute, Routing, and Failure Modes. You spend more time with brackets and shims than with hyperparameters. ### 2. Lighting you control, not lighting you inherit A factory has "lighting" from overhead fixtures, skylights, forklift beams. You ignore that. You build your own. – Enclosed light tunnels for small parts.
– Dark-field or backlighting when it highlights the defect better than front lighting.
– Strobed LEDs slaved to the trigger, so every image sees the same illumination regardless of ambient flicker. The rule is simple: if you cannot make the image look nearly identical across shifts and seasons, your model is already compromised. ### 3. Edge compute that respects PLC reality You put the model where the timing is predictable. – As close to the sensor as practical, usually on an industrial PC or embedded device in the control cabinet.
– With dedicated CPU/GPU resources for inference, not shared with everything else IT thought was convenient.
– Talking fieldbus or industrial Ethernet to the PLC, with clear state machines: part present, image captured, decision done, actuator commanded. You design for worst-case latency and verify it under load. "Average 20 ms" is irrelevant if one frame in a hundred takes 200 ms and sends a bad part to a customer. ### 4. A minimal, stable model On the modeling side, the best systems are usually conservative. – Modest-sized architectures that run in bounded time.
– Clear strategies for handling unknowns: reject as uncertain rather than guessing.
– Training sets built from the actual plant, across product variants, shifts, and material suppliers, not just the golden path. You do not ship a model that breaks every time the upstream machine leaves a bit more oil on the surface. ## Keeping it alive: operations, not research The day you turn the system on is the day the real work starts. Operators and techs will: – Bump cameras clearing jams.
– Wipe lenses with whatever rag they have.
– Override alarms when they are behind on throughput.
– Request changes in thresholds because "yesterday it was fine." If you ignore this, your system will silently drift into irrelevance. What you actually need: – Simple, locked-down HMI screens that let operators see decisions, basic stats, and a handful of safe adjustments, but not model internals.
– Clear escalation paths: when the system starts misbehaving, who do they call, and how fast do you respond.
– Maintenance routines: lens cleaning, enclosure checks, lighting checks folded into standard PMs, with simple go/no-go criteria.
– Periodic model reviews with production data: what is being rejected, what sneaks through, how many manual overrides are happening. You treat the CV system like any other critical piece of equipment: it has owners, KPIs, and a maintenance schedule. ## A short decision framework: camera or sensor When you are tempted to propose computer vision on a line, force yourself through a blunt checklist. 1. What is the business event we care about? Scrap rate on a specific defect, missed label, jam, safety near-miss, wrong assembly. 2. Where and when can we see it first? Physically: at what station, under what conditions, with which signals naturally available. 3. What happens today when this event occurs? How is it detected, who responds, how long does it take, what downtime or customer impact follows. 4. Is there a non-vision sensor that would give us a robust early signal? If yes, why aren't we using it. If no, does the defect have a visual fingerprint that survives dirt, lighting variation, and normal process noise. 5. Can we control the imaging environment enough? If you cannot guarantee stable part presentation and lighting, do not pretend the model will improvise. Fix mechanics first. 6. Can we integrate decisions into the existing control and maintenance stack? If your design ends in "and then someone will watch this dashboard," you have already failed. If you make it through all of that and vision still looks like the right tool, then you spend money on cameras and models. Not before. ## When sensors, dust, and vibration win The factories that quietly outperform their peers are often the ones that look boring in photos. You see: – Clean wiring, sane sensor placement, and redundant interlocks.
– Few mystery alarms and even fewer abandoned screens.
– Maintenance technicians who know why a given limit switch is there and how to test it. These plants still use computer vision, but they use it where it is the only sensible way to see a problem, not as decoration. They accept that dust, vibration, and oil are not "edge cases" but the default setting. They accept that a simple current sensor on a motor may give them more reliable insight into failure than a camera watching the belt. If you approach the factory floor as a place where models must dominate messy physics, you will burn time and goodwill and still end up bypassed. If you let the physics lead, you will discover that sometimes the most intelligent thing you can do with a camera is not to install it.