Real-Time Fall Detection
A missed fall in an industrial or healthcare setting can cost minutes that determine the outcome.
Computer Vision • System Architecture • Real-Time Data Systems • ML Pipeline Design • Systems Integration
Background
The client was building a safety product for environments where fall detection has real consequences. Most existing systems require dedicated cameras or specialized hardware. That blocks adoption for organizations already running large CCTV networks.
The goal was to detect falls from existing camera streams without additional hardware.
The Solution
I designed a detection pipeline that layers on existing camera streams. It analyzes how people move over time to separate real falls from normal activity like crouching or sitting. Only confirmed events and a snapshot reach the monitoring interface.
The key challenge was that falls look like normal movement in a single frame. The system tracks how body position changes over time to separate the two.
Deep Dive
Architecture
The pipeline layers on top of a stream controller that handles RTSP ingestion via OpenCV, GStreamer, or FFmpeg depending on the source. Above it sit three layers: inference, tracking (Kalman filter), and heuristics (kinematic analysis to classify falls). Only the fall signal and a snapshot pass forward to a FastAPI backend and React/TypeScript interface. Everything else stays inside the pipeline.
Detection Approach
A single frame is not enough to detect a fall. The system builds confidence across three signals.
Bounding box position and size provide initial context. A box shrinking or dropping in frame is weak on its own but narrows the search space.
Joint positions from pose estimation give a richer picture. Raw positions alone are unreliable. Joints converge during normal activity and overlap in certain poses.
The real signal is joint kinematics: angular velocity and acceleration between key joint pairs. A fall has a characteristic kinematic signature that normal movement does not. How joint angles change over time separates a fall from bending, sitting, or stumbling.
I plotted joint angles and angular velocities to visualize how signatures differ between falls and normal movement. Heuristic thresholds come from that analysis, not intuition.
Pipeline Design and Async Processing
A key decision: let the pipeline run async, decoupled from the live stream. A single-frame delay is acceptable for fall detection. Processing every frame the instant it arrives is unnecessary and creates bottlenecks.
Async means inference, tracking, and heuristics pipeline without blocking each other. Inference runs on the latest available frames. Tracking and heuristics consume output without waiting on the next capture. Better throughput and consistent performance under load, with no meaningful latency impact.
Past heuristics, compute drops sharply. Tracking is just keypoint math. Inference is the expensive work, and async keeps it from becoming a ceiling.
Related Advisory Work
The client was also working on two related detection problems. My involvement was consultative.
Fire detection. The client's model had high false positive rates. Orange high-vis jackets kept triggering detections. The issue wasn't model quality but training data that didn't reflect the actual problem. Industrial fires have behavioral properties beyond color. I advised classical CV pre-filters: temporal behavior, flicker, and spatial context evaluated before the model runs. More training data doesn't help if it doesn't reflect what you're detecting.
PPE detection. More tractable because PPE labeled data is widely available. I outlined person detection, per-item PPE detection, and persistent tracking to associate PPE state with workers over time. The hard part isn't detecting a hard hat. It's attributing it to the right person across a shift and flagging when that breaks.
Same message in both cases. Collecting data and training a model is not a detection strategy. You have to understand what you're detecting, structure collection around that, and layer heuristics on the model to make output reliable. Without that surrounding system, the model output isn't actionable.