How on-device face detection works on a Mac.

Face presence versus face recognition

When people hear "face detection" in the context of a security app, they often picture a system that identifies you by name or compares your face against a database. The reality on modern macOS is more nuanced, and understanding this distinction is crucial to comprehending both what's possible and what's private.

Face presence detection answers a single binary question: is a human face visible in the camera frame right now? This is a lightweight operation. It does not attempt to identify you, match you against a known face, or remember what you look like. The system simply checks whether pixels in the frame contain features that resemble a face (eyes, a nose, a mouth in roughly the right spatial arrangement).

Face recognition, by contrast, goes much further. It extracts a unique mathematical representation of a particular face and compares it against a reference set. The system asks: is this the face of person A, person B, or someone unknown? This is computationally heavier and requires enrolment: storing a reference representation beforehand.

Apple's Vision framework supports both operations, and they run locally on your Mac. But they are fundamentally different in scope, cost, and privacy implications. An app that uses only presence detection is not building a record of who you are; it is simply monitoring whether someone is in front of the camera.

How the Vision framework detects presence

The Vision framework is Apple's foundational computer vision library, available in macOS 10.13 and later. It exposes machine learning models that run entirely on your device. When you ask it to detect faces, the framework loads a trained neural network that has learned to recognise face-like structures in pixel data.

On each frame from the camera, the Vision framework applies this model. It produces a list of detected faces, with coordinates and confidence scores. For a presence check, the app needs only one piece of information: did the model return at least one face above a certain confidence threshold? If yes, someone is present. If no faces are detected, the person is absent.

This happens in real time. Modern Macs with Apple Silicon process each camera frame in milliseconds. The system is efficient enough to run continuously in the background without draining battery or disrupting other work.

Critically, the raw camera frame is not saved to disk. Each frame is processed by the Vision model and then discarded. The only data that persists is the presence decision (whether a face was detected or not), and this stays on your Mac, available only to the app itself.

Feature prints and optional known-face matching

When an app wants to add a second layer (detecting not just whether a face is present, but whether it is your face in particular), it uses a feature print. A feature print is a compressed mathematical vector that represents the unique characteristics of a face: not a photo, not an image, but a string of numbers (typically 128 or 256 dimensions) computed by a deep neural network from the face visible in a frame.

Peeku can optionally use feature prints to distinguish between your face and someone else's. During enrolment, you position your face in front of the camera for a few seconds. The Vision framework extracts a feature print from multiple angles and stores it in a local file on your Mac (~/.config/proximity-lock/known-face-enrollment.json, readable only by your user account). No image is saved. No enrolment data leaves the Mac.

Later, when the camera detects a face, the app compares its feature print against your stored enrolment print. If they match above a threshold, the system recognises you. If they do not match, it treats the face as unknown. This is how Peeku can optionally enable a feature where the lock does not engage if someone else's face is detected in front of the Mac.

Feature prints are specific to the Vision framework implementation and are not transferable between apps or systems. They are mathematical artifacts of the enrolment moment and the particular neural network Apple chose. If you delete your enrolment, the prints are gone, and no cloud backup or recovery exists: by design.

Why on-device processing matters

Some face detection systems, particularly in cloud-based security or surveillance contexts, send camera frames to a server for processing. The server analyses the image, extracts relevant data (a face encoding, a person's identity, a timestamp), and returns a result to the client. The frame itself may be temporarily stored, logged, or retained for compliance.

On-device processing inverts this model. The machine learning model lives on your Mac. The frame is processed locally, discarded locally, and no frame data ever traverses your network or reaches a company's server. This means several things follow automatically:

• No network dependency. Face detection works even when your internet connection is down. The system does not need to "phone home" to work.
• No centralised database. Because no one else sees your frames, there is no risk of a company database being breached, subpoenaed, or sold.
• Lower latency. Processing happens at the speed of your local hardware, not the speed of your internet connection plus server processing time.
• Reduced attack surface. The attack surface is limited to your Mac itself, not a distributed system of servers, databases, and APIs.

This does not make on-device processing immune to privacy concerns. A malicious app with access to your camera could still misuse that access. But the absence of network transmission and server storage eliminates an entire category of exposure.

Efficiency and background operation

A natural question arises: if the Vision framework is powerful enough to run real-time face detection, how does it run continuously without consuming all available CPU, draining battery, or making your Mac sluggish?

Modern Macs, particularly those with Apple Silicon processors (M1 and later), include a Neural Engine: a dedicated hardware circuit for machine learning inference. The Vision framework is optimised to delegate face detection to this hardware when available. The CPU can continue serving your applications while the Neural Engine handles the face analysis in parallel.

Additionally, face detection is not as expensive as you might expect. The Vision models used for presence detection are relatively small and have been optimised for on-device use. They are not the massive, general-purpose vision models used in cloud systems; they are focused, efficient, and purpose-built for this task.

Apps like Peeku can further optimise by throttling the frame rate. Rather than processing every single camera frame in real time, an app might process one frame per second or one frame every 500 milliseconds. This is still responsive enough to catch when you walk away from your desk without consuming unnecessary resources.

Permissions and transparency

Because camera access is a sensitive permission in macOS, any app that wants to use the camera must declare its intention in advance. When you first launch an app that requests camera access, macOS prompts you to approve. This happens once per app, per macOS install. The OS also shows a visible indicator whenever the camera is in use: a green light beside the built-in camera, wired to the camera hardware, and on recent versions of macOS a camera indicator in the menu bar as well.

Peeku displays a prominent message during setup explaining that it uses your built-in camera for on-device face presence detection. Before you enable the auto-lock feature, the app shows you a live preview of what the camera sees and whether a face is detected. You can watch the system work and decide whether to trust it before arming the lock.

This transparency is not a technical guarantee, but it is an important practical control, and it helps separate the real risks from the common myths about Mac camera use. You can verify that the camera is being used only for presence detection and not for something hidden.

A contrast with other approaches

Some approaches to walkaway detection avoid cameras altogether, relying instead on Bluetooth proximity or keyboard/mouse idleness. These have their own merits: no camera permission, lower privacy footprint, straightforward logic.

Bluetooth proximity alone can be reliable if you always carry your phone or wear a watch, but it fails if your device runs out of battery or if you leave it on your desk while walking away. Keyboard and mouse idleness is easy to implement but triggers false locks when you are on a phone call or reading something on your screen.

Face detection solves a different part of the problem: it detects actual presence, not absence of interaction. If you walk away from your desk but leave your watch nearby, a face-detection system will lock the Mac. If you are staring at your screen during a video call and your hands are still, it will not lock.

Cloud-based face detection systems (familiar from remote surveillance or identification services) operate at the opposite end of the spectrum: they can run powerful models and store rich data about faces across many devices, but at the cost of sending your camera stream to a server and, implicitly, trusting that server and the company behind it.

Practical implications

Understanding how on-device face detection works has practical consequences for how you think about security and privacy. It means:

• When you enable face detection in Peeku, no one else sees your camera feed. Your ISP does not, your cloud storage provider does not, and the Peeku developer does not. Only your Mac sees it, processes it, and forgets it.
• Feature prints are a privacy-preserving way to distinguish between multiple users. An app can know "a face is present" and "it is probably your face" without storing a photograph or sending anything to the cloud.
• On-device processing makes this kind of system viable as a standalone app. You do not need to pay for server infrastructure to run face detection, and you do not need to maintain a subscription to keep your Mac locked.
• The system is only as trustworthy as the app itself. On-device processing is a strong privacy property, but it assumes the app is not malicious. For Peeku, the evidence of trustworthiness comes from the fact that it is a standalone paid app, not ad-supported, not cloud-dependent, and made by a solo developer with no incentive to monetise your camera data.

Next steps

If you want to see this in practice, you can install Peeku and watch the live presence panel in the menu bar before enabling auto-lock. You will see face presence detected in real time, with confidence scores, all on your Mac.

For deeper technical details on the Vision framework itself, Apple publishes extensive documentation. The Peeku codebase is private, but the underlying Vision APIs are open and well-documented. You can build your own face detection tools using the same framework.