Element 13.14
Vision and sensing at the edge
- Sheet
- 13.14
- Chapter
- 13 Digital infrastructure
- State
- Drafted
- Release
- 16 November 2026
- Editor
- Contributed by Praecise as open research
- Reviewers
- To be named at review
- Safety
- S1, as the chapter
- Licence
- CC BY-SA 4.0
- Resources
- 10
The element
- Requirement
Where the community uses cameras or microphones to watch its own sites, wildlife or water, the analysis runs on its own hardware, only the results needed leave the device, and every camera's purpose and field of view is agreed.
- How to tell
Each camera or recorder has a written purpose, an agreed field of view and a retention period. The detection runs locally: with the outside link unplugged, events are still detected and recorded. Detection is fast enough for the job: Frigate's documentation states, for example, about 10 ms per frame on a Coral, which caps it at about 100 frames a second (Frigate recommended hardware).
- Options
- Inference on the camera. The Raspberry Pi AI Camera runs the network on its Sony IMX500 sensor and sends only results to the Pi; object detection with MobileNet SSD runs wholly on the camera (AI Camera (Raspberry Pi documentation)). Fits single posts with little power and no accelerator.
- A small board with an accelerator. Raspberry Pi 5 with the AI HAT+ (Hailo-8L, 13 TOPS, or Hailo-8, 26 TOPS) for object detection and camera post-processing (AI HAT+ and AI HAT+ 2 (Raspberry Pi documentation)); Hailo states a typical 2.5 W for the Hailo-8 (Hailo-8 AI accelerator). The AI HAT+ 2 adds vision-language models for captioning, event logging and free-text search of video (When and why you might need the Raspberry Pi AI HAT+ 2).
- A board with a built-in NPU. The RK3588 has a 6 TOPS NPU and 8K decoding (RK3588 (Rockchip)); Frigate lists about 20 ms for YOLO-NAS small on three RK3588 cores (Frigate recommended hardware).
- A GPU module for many cameras. Jetson modules (Jetson Modules (NVIDIA)); Frigate lists about 20 to 40 ms on most Jetson models (Frigate recommended hardware).
- A local video recorder. Frigate records network cameras and runs detection on the detectors above; it recommends cameras with H.264 and several substreams and advises against Wi-Fi cameras for reliability (Frigate recommended hardware).
- Wildlife and sound. MegaDetector finds animals, people and vehicles in camera-trap images (Microsoft CameraTraps: MegaDetector and SPARROW Playground); SpeciesNet classifies more than 2,000 labels (SpeciesNet); BirdNET names birds in field recordings (BirdNET-Analyzer). These serve monitoring in domain 22.
- Settled, open, contested
Settled. Detection can run on the camera or a small board without an outside service (AI Camera (Raspberry Pi documentation), AI HAT+ and AI HAT+ 2 (Raspberry Pi documentation)). Open. Accuracy of each model at a given site, in its light and weather, must be measured locally; makers' figures are speeds, not accuracy. Contested. Cameras in shared spaces at all. One position holds that local detection protects property and people without sending video away; the other that any recording of members changes how a community lives. The purpose, view and retention rules are for the community to set (15.2).
The four stages
Learn: where inference runs and what leaves the device (AI Camera (Raspberry Pi documentation)). Design: write each camera's purpose and view; choose the detector from stated inference times (Frigate recommended hardware). Build: fit the camera or accelerator (AI HAT+ and AI HAT+ 2 (Raspberry Pi documentation)) and set up the recorder (Frigate recommended hardware). Deploy: test detection with the link unplugged, review false alarms, delete on the agreed schedule.
Resources at each stage
1 Learn
Understand
2 Design
Size and choose
- Frigate recommended hardware Guide, S1
- Hailo-8 AI accelerator Datasheet, S0
- Jetson Modules (NVIDIA) Datasheet, S1
- When and why you might need the Raspberry Pi AI HAT+ 2 Guide, S0
- RK3588 (Rockchip) Datasheet, S0
3 Build
Build or obtain
- BirdNET-Analyzer Tool, S0
- Frigate recommended hardware Guide, S1
- Microsoft CameraTraps: MegaDetector and SPARROW Playground Repository, S0
- AI Camera (Raspberry Pi documentation) Guide, S1
- AI HAT+ and AI HAT+ 2 (Raspberry Pi documentation) Guide, S0
- SpeciesNet Tool, S0
4 Deploy
Install, run and measure
- BirdNET-Analyzer Tool, S0
- Microsoft CameraTraps: MegaDetector and SPARROW Playground Repository, S0
- SpeciesNet Tool, S0
What changes between 30 adults and 300 people
At 30 adults one or two posts with an AI Camera or a Pi with an accelerator. At 300 people a recorder with a dedicated detector serves many cameras, recordings are on an SSD with a backup policy for what must be kept, and a named group reviews placement yearly.
Reference designs
Open for contributions Designs with drawings, specifications and a bill of materials. Add to it
Parts list
Every resource that supports this element. Those marked External carry the label “External resource. Not made or verified by Public Protocol”.
| Resource | Type | Origin | Maturity | Safety | Licence, as stated at its source |
|---|---|---|---|---|---|
| BirdNET-Analyzer | Tool | External | Established | S0 | Source code: MIT License. Models: Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License |
| Frigate recommended hardware | Guide | External | Established | S1 | Not stated on the page opened |
| Hailo-8 AI accelerator | Datasheet | External | Established | S0 | Not stated on the page opened |
| Microsoft CameraTraps: MegaDetector and SPARROW Playground | Repository | External | Established | S0 | MIT license |
| Jetson Modules (NVIDIA) | Datasheet | External | Established | S1 | Not stated on the page opened |
| AI Camera (Raspberry Pi documentation) | Guide | External | Established | S1 | Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0); some content from the eLinux wiki under CC BY-SA 3.0 Unported |
| When and why you might need the Raspberry Pi AI HAT+ 2 | Guide | External | Working | S0 | Not stated on the page opened |
| AI HAT+ and AI HAT+ 2 (Raspberry Pi documentation) | Guide | External | Established | S0 | Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0); some content from the eLinux wiki under CC BY-SA 3.0 Unported |
| RK3588 (Rockchip) | Datasheet | External | Established | S0 | Not stated on the page opened |
| SpeciesNet | Tool | External | Established | S0 | Apache-2.0 license |
Depends on, and completes
It is one of 25 elements that complete Digital infrastructure.
Environments
- Hot deserts: the AI HAT+ documentation recommends active cooling to prevent slowdown from overheating (AI HAT+ and AI HAT+ 2 (Raspberry Pi documentation)).
- Arctic cold: Hailo states an industrial grade of -40 to 85 degrees for the Hailo-8 (Hailo-8 AI accelerator); the host board's own range still applies.
- Islands and coasts, tropical humid forest, temperate forest, high altitude, flood plains: no source found on edge vision hardware in these settings.
Safety
State and evidence
Drafted
An editor has written the element's text from the resources listed under it, citing them. It is checked when it is reviewed.
Evidence from a real place or system is recorded here, with what did not work. What the states mean
History
- Drafted in the release of 16 November 2026.