Real-Time Fall Detection
A missed fall in an industrial or healthcare setting can cost critical minutes that determine the outcome.
Computer Vision • System Architecture • Real-Time Data Systems • ML Pipeline Design • Systems Integration
Background
The client was building a safety product for environments where fall detection is time-critical. Most existing systems require dedicated cameras or specialized hardware, which limits adoption for organizations already running large CCTV networks.
The goal was to detect falls directly from existing camera streams without requiring additional camera installations.
The Solution
I designed a detection pipeline that runs on existing camera streams. It tracks changes in body position over time to separate real falls from normal activity like crouching or sitting. Only confirmed events and a snapshot are sent to the monitoring interface.
Falls look similar to normal movement in a single frame. The system tracks how body position changes over time to reliably distinguish between them.
Deep Dive
Architecture
The pipeline runs on top of a stream controller that handles RTSP ingestion via OpenCV, GStreamer, or FFmpeg depending on the source. Above this are three layers: inference, tracking (Kalman filter), and heuristics (kinematic analysis to classify falls).
Only the fall signal and a snapshot pass forward to a FastAPI backend and React/TypeScript interface. Everything else stays inside the pipeline.
Detection Approach
A single frame is not enough to detect a fall. The system builds confidence across three signals.
Bounding box position and size provide initial context. A box shrinking or dropping in frame is weak on its own but helps narrow the search space.
Joint positions from pose estimation add more detail, but raw positions are still unreliable. Joints can converge during normal activity and overlap in certain poses.
The strongest signal is joint kinematics: angular velocity and acceleration between key joint pairs. Falls have a consistent kinematic pattern that normal movement does not. How joint angles change over time separates a fall from bending, sitting, or stumbling.
I plotted joint angles and angular velocities to compare fall signatures against normal movement. Heuristic thresholds come from that analysis, not intuition.
Pipeline Design and Async Processing
A key decision was to run the pipeline asynchronously, decoupled from the live stream. A single-frame delay is acceptable for fall detection, so processing every frame immediately is unnecessary and creates bottlenecks.
Inference, tracking, and heuristics run independently without blocking each other. Inference operates on the latest available frames, while downstream stages consume results as they become available. This improves throughput and maintains consistent performance under load without adding meaningful latency.
After heuristics, compute requirements drop sharply. Tracking is lightweight keypoint math, while inference is the dominant cost. Running asynchronously prevents inference from becoming a system bottleneck.
Related Advisory Work
The client was also working on two related detection problems. My involvement was consultative.
Fire detection. The model had high false positive rates, with orange high-visibility jackets triggering detections. The issue was not model quality but training data that did not reflect the real problem. Industrial fires have behavioral properties beyond color. I advised adding classical CV pre-filters based on temporal behavior, flicker, and spatial context before running the model. More data does not help if it does not represent the target signal.
PPE detection. This problem is more tractable due to the availability of labeled data. I outlined a pipeline combining person detection, per-item PPE detection, and persistent tracking to associate PPE state with individuals over time. The challenge is not detecting a hard hat, but maintaining identity and detecting when compliance breaks.
In both cases, the core issue was the same: collecting data and training a model is not a complete detection strategy. Effective systems require understanding the signal, structuring data collection around it, and layering heuristics to make outputs reliable and actionable.