Skip to content

Infinity3: working HAL, verified on an Imou LOOC 2 (MSC316DC, OV2735) - #64

Open
HeytalePazguato wants to merge 22 commits into
OpenIPC:masterfrom
HeytalePazguato:infinity3-looc2
Open

HeytalePazguato wants to merge 22 commits into
OpenIPC:masterfrom
HeytalePazguato:infinity3-looc2

Conversation

@HeytalePazguato

Copy link
Copy Markdown

The Infinity3 HAL was marked work in progress; this makes it work end to end on real hardware: an Imou LOOC 2 (IPC-C26EP-V2, MSC316DC, OV2735 MIPI, 128 MB), running OpenIPC with the vendor libmi.

What works on the camera:

  • H.264 main stream 1920x1080 at 25 fps, plus a 640x360 sub-stream on its own RTSP port
  • Audio capture (MP3 for RTSP, Opus for the browser MP4 stream) and playback (talk-back)
  • JPEG snapshots, MJPEG
  • Day/night without a light sensor, decided from the ISP exposure
  • OSD with region anchors
  • MP4 recording to SD card with retention

Commits (each builds on its own with the armhf glibc toolchain):

  1. Infinity3: HAL on the vendor MI library: call sequence and structure layouts checked against the vendor libmi.so on the camera, not taken from headers
  2. MP4: separate audio and video clocks, one audio frame per sample, Opus
  3. RTSP: a second server instance (sub-stream); optional login against the camera's own accounts (auth_pass: "@system", src/sysauth.c)
  4. Night mode: switch on ISP exposure when there is no light sensor (exposure_night / exposure_day)
  5. OSD: region anchors (top-left ... bottom-right) with offsets
  6. Recorder: refuses to write to the root filesystem, loop recording with a free-space reserve, fragments from the live muxer
  7. Server, media and page: sub-stream, talk-back, lamp, time zone, clip list and download with byte ranges, settings for the above
  8. Infinity3 on OpenIPC's musl base (glibc-compat): open libmi_isp.so before libmi.so, pass the mi.* settings to OpenIPC's libat.so, and with GLIBC_COMPAT leave out the empty stand-ins that would shadow the shim's working ones
  9. Zero the JPEG, MJPEG and MP4 encoder configs before filling them: frame rate, GOP, profile and quality limits held stack garbage. On Infinity3 the snapshot encoder was created at 198-230 fps and its port never came up
  10. Infinity3 night grayscale through the ISP (MI_ISP_SetColorToGray with the full attribute from I3_isp_api.xml): the encoder's MI_VENC_SetColor2Grey left the bottom of a 1080p frame in colour and the MJPEG encoder rejects it. Streams and snapshots now all turn gray; the encoder call stays as a fallback
  11. Infinity3 rate control: MI_VENC_CreateChn hands mode 1 to VBR and 2 to CBR (0 = none). The HAL passed 0 for CBR, so H.264 ignored the bitrate and sat at QP 48 (about 600 kbit/s at 1080p, blocky). Now CBR 2048 gives 2.1 Mbit/s at QP 33-36, CBR 4096 gives 4.2 Mbit/s, VBR follows its maximum. Also back off 10 ms when a restarting channel fails at once, instead of spinning (65000 vendor log lines per live settings change before)
  12. Night mode without a light sensor: fewer daylight probes. Each probe switches the IR lamps off for a moment and blanks the picture. Daylight is now caught from the exposure falling below the darkest level seen under the lamps (or the picture getting brighter than the AE target); the timed probes start after ten minutes and back off to about once an hour
  13. Recorder: /api/record?start&duration=N records for N seconds and repeating it moves the end, so an NVR or home automation event can trigger clips on the card without sending the stop; stop_in in the response

Tested on the LOOC 2 on top of 251d329, with the camera's glibc image and booted from OpenIPC firmware master (musl, kernel and rootfs built from HeytalePazguato/firmware branch infinity3-imou-looc2): live MP4 1080p + Opus, snapshots, RTSP 554/555 over TCP and UDP, continuous recording (15 s segments, H.264 + Opus) and recording to a real exFAT card, day/night switching with the IR lamp (gray streams and snapshots at night, colour back in day mode), 401 without login on web and RTSP.

Not tested: other Infinity3 chips (MSC313E, MSC316DM) and sensors other than OV2735.

Video, audio capture and playback, ISP exposure read-out, software OSD
widgets and the property store the vendor library reads its scaler
settings from. Verified on an Imou LOOC 2 (MSC316DC, OV2735).
Chrome 153 on Windows rejects several MPEG audio frames in one sample and
the Shine MP3 bitstream altogether; browsers get Opus (libopus loaded at
run time). Stale audio is dropped while no client is draining it.
The password '@System' checks Basic credentials against /etc/shadow
(src/sysauth.c), so the camera's own login is used and no password is kept
in the config.
Night when the exposure index is high and the picture dark; back to day
by briefly switching the IR LEDs off and measuring again. Adds a white
lamp pin.
Refuses to record onto the root filesystem, removes its own oldest
segments below a free-space reserve, cuts segments on keyframes, takes
fragments from the live muxer (so clips carry audio), retries continuous
mode when the card shows up later.
Sub-stream on a second RTSP port, talk-back (/api/speak), lamp, time
zone, clip list and download with byte ranges, Media Source live player,
sub-stream and recording settings, system-account web login, start/stop
fixes for /api/record, web port bind retry.
OpenIPC now builds infinity3 on musl, with glibc-compat for the vendor
libraries:
- open libmi_isp.so before libmi.so (musl binds their mutual symbols that
  way; glibc needed both preloaded)
- hand the mi.* settings to OpenIPC's libat.so as well, which keeps them
  in the process instead of the bionic property area
- with GLIBC_COMPAT defined, leave out the empty backtrace, __ctype_b_loc
  and pthread cancel stand-ins: the shim and musl have working ones, and
  the executable's own definitions would be found first

Verified on the Imou LOOC 2 on OpenIPC firmware master (musl): 1080p
H.264 + Opus, snapshot, sub-stream, RTSP on both ports.
The JPEG, MJPEG and MP4 channel configs were built on the stack and only
partly filled, so frame rate, GOP, profile and quality limits held
whatever was there. HALs that read those fields got garbage: on
Infinity3 the snapshot encoder was created at a random frame rate
(198-230 fps) and its port never came up, so every snapshot failed.
The sub-stream config was already cleared this way.
MI_VENC_SetColor2Grey() leaves the bottom of a 1080p frame in colour and
the MJPEG encoder rejects it, so night snapshots stayed pink. Use
MI_ISP_SetColorToGray() with the full attribute from I3_isp_api.xml
(manual mode), which turns every channel gray; keep the encoder call as
a fallback.
MI_VENC_CreateChn hands rate control mode 1 to _MI_VENC_SetVbr and 2 to
_MI_VENC_SetCbr; 0 means none. The HAL passed 0 for CBR, so the H.264
encoder ignored the bitrate and sat at QP 48 (about 600 kbps at 1080p,
visibly blocky). Measured on the LOOC 2 after the fix: CBR 2048 -> 2.1
Mbit/s at QP 33-36, VBR 2048 -> 2.1 Mbit/s, CBR 4096 -> 4.2 Mbit/s.

While a channel is being restarted, MI_VENC_GetStreamTimeOut fails at
once instead of waiting; back off 10 ms so the stream thread does not
spin (65000 vendor log lines per restart before, about 100 after).
Each probe switches the IR lamps off for a moment, which blanks the
picture. Daylight is caught from the exposure falling below the darkest
level seen under the lamps, or the picture getting brighter than the
auto-exposure aims for; the timed probes now start after ten minutes
and back off to about once an hour.
/api/record?start&duration=N records for N seconds; repeating the call
moves the end, so an NVR or home automation event can keep a clip going
without having to send the stop. A plain start still records until
stop, and the response reports the seconds left in stop_in.
@qodo-free-for-open-source-projects

Copy link
Copy Markdown

PR Summary by Qodo

Enable end-to-end Infinity3 camera streaming, audio, and recording

✨ Enhancement 🐞 Bug fix 🕐 40+ Minutes

Grey Divider

AI Description

• Make the Infinity3 HAL operational with vendor MI libraries and ALSA on the Imou LOOC 2.
• Add a sub-stream, browser-compatible MP4 audio, talk-back, system-account login, and
 exposure-based night mode.
• Improve SD-card recording safety, clip access, OSD placement, and web controls.
Diagram

graph TD
  Config["Camera settings"] --> HAL["Infinity3 HAL"] --> Media["Media pipeline"] --> Muxer["MP4 muxer"] --> Recorder["SD recorder"]
  Media --> RTSP["RTSP servers"]
  Muxer --> Server["HTTP API"] --> UI["Web page"]
  Config --> Server
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Use a separate recording muxer
  • ➕ Would isolate recording state from live HTTP delivery.
  • ➖ Would duplicate NAL and audio processing and risk producing clips with different timing or missing audio.
2. Use a standard authentication service
  • ➕ Could centralize account policy and avoid direct shadow-file parsing.
  • ➖ Would introduce a runtime dependency that may not exist on the camera firmware.

Recommendation: Keep the shared MP4 muxer and platform-scoped Infinity3 integration: they reuse working media paths and suit the constrained device. Review the direct system-account authentication and vendor ABI assumptions carefully, and validate on additional chips before treating Infinity3 support as general.

Files changed (24) +2445 / -921

Enhancement (19) +2300 / -898
index.htmlAdd live playback and camera controls +362/-17

Add live playback and camera controls

• Adds low-latency Media Source playback, microphone talk-back, lamp controls, and clip listing and download links. Exposes sub-stream, recording reserve, OSD anchors, and time-zone settings; applied settings are saved.

res/index.html

moov.cDescribe Opus tracks in MP4 headers +25/-4

Describe Opus tracks in MP4 headers

• Emits an Opus sample entry and dOps metadata instead of MPEG audio metadata when the audio codec is Opus.

src/fmt/moov.c

mp4.hExpose independent audio timing and frame ingestion +3/-0

Expose independent audio timing and frame ingestion

• Adds an audio decode-time field to MP4 state and declares frame-based audio ingestion.

src/fmt/mp4.h

i3_hal.cReplace the Infinity3 HAL with hardware-verified integration +755/-689

Replace the Infinity3 HAL with hardware-verified integration

• Implements the vendor MI video pipeline, encoder rate control, snapshots, ISP exposure and grayscale, and software YUV OSD widgets. Adds OpenIPC property handling plus tinyalsa microphone capture and speaker talk-back.

src/hal/star/i3_hal.c

i3_hal.hDeclare the operational Infinity3 HAL interface +26/-19

Declare the operational Infinity3 HAL interface

• Replaces obsolete component-header dependencies with declarations for property setup, video, audio, speaker, ISP, and region operations.

src/hal/star/i3_hal.h

main.cRun the sub-stream server and recorder timer +27/-2

Run the sub-stream server and recorder timer

• Starts and ticks a second RTSP instance, ticks timed and continuous recording, and reads the system time zone at startup.

src/main.c

media.cConnect Infinity3 media, Opus, and sub-stream routing +181/-8

Connect Infinity3 media, Opus, and sub-stream routing

• Wires Infinity3 capture and encoders into the media lifecycle, adds optional runtime-loaded Opus for MP4, and sends sub-stream video to its own RTSP server. Zero-initializes encoder configurations and prevents the Infinity3 sub-stream from sharing its scaler output with MJPEG.

src/media.c

media.hExpose the sub-stream RTSP handle and frame rate +2/-1

Expose the sub-stream RTSP handle and frame rate

• Declares the second RTSP handle and measured sub-stream rate for media consumers.

src/media.h

night.cAdd exposure-based day/night switching +85/-7

Add exposure-based day/night switching

• Uses ISP exposure to switch modes when no light sensor exists, with less frequent IR-off daylight probes. Adds independent white-lamp control and bounds GPIO pins before writing.

src/night.c

night.hExpose lamp and ISP exposure controls +4/-0

Expose lamp and ISP exposure controls

• Declares white-lamp state and control functions and the platform exposure query.

src/night.h

record.cProtect SD-card recording and support timed clips +229/-94

Protect SD-card recording and support timed clips

• Refuses root-filesystem recording, reserves free space by removing old recorder segments, and retries continuous recording when storage returns. Writes keyframe-aligned fragments from the live muxer and adds extendable timed recording.

src/record.c

record.hDeclare recorder timing, storage, and fragment APIs +8/-1

Declare recorder timing, storage, and fragment APIs

• Exposes timed-stop state, periodic retry, storage status, and shared-muxer fragment ingestion.

src/record.h

region.cPosition OSD regions relative to frame anchors +41/-4

Position OSD regions relative to frame anchors

• Calculates placements from nine anchors and offsets, and connects text and image regions to Infinity3 software OSD widgets.

src/region.c

region.hRepresent OSD anchor placement +4/-0

Represent OSD anchor placement

• Adds an anchor to each OSD region and declares anchor parsing and placement helpers.

src/region.h

rtsp.cSupport two RTSP instances and device-account login +41/-25

Support two RTSP instances and device-account login

• Separates connection and transfer pools for main and sub-stream servers. Adds optional Basic authentication against system accounts and retries port binding during restarts.

src/rtsp/rtsp.c

server.cExpose talk-back, clips, sub-stream settings, and login +438/-27

Expose talk-back, clips, sub-stream settings, and login

• Adds system-account web authentication, lamp and speaker APIs, sub-stream settings, exposure and time-zone controls, and recording status and duration controls. Lists clips and serves byte-range downloads from bounded worker threads while forwarding muxed fragments to the recorder.

src/server.c

server.hInclude filesystem definitions for clip serving +2/-0

Include filesystem definitions for clip serving

• Adds directory and file-stat includes needed by the HTTP clip-list and download implementation.

src/server.h

sysauth.cAuthenticate Basic credentials against system accounts +63/-0

Authenticate Basic credentials against system accounts

• Introduces optional @system password verification using /etc/shadow and runtime-loaded crypt, with a successful-credential cache and delay after failed checks.

src/sysauth.c

sysauth.hDeclare the shared system-login check +4/-0

Declare the shared system-login check

• Exposes Basic-credential verification to the HTTP and RTSP servers.

src/sysauth.h

Bug fix (3) +82 / -21
compat.cAvoid shadowing glibc-compat implementations +7/-2

Avoid shadowing glibc-compat implementations

• Excludes empty compatibility stand-ins when GLIBC_COMPAT is enabled so OpenIPC's working shim implementations remain available.

src/compat.c

mp4.cCorrect fragmented MP4 audio samples and timing +71/-19

Correct fragmented MP4 audio samples and timing

• Tracks audio and video decode time independently and supports one sample per supplied audio frame, including Opus. Splits buffered MP3 into frame-sized samples and bounds stale audio accumulation.

src/fmt/mp4.c

jpeg.cEnable Infinity3 JPEG snapshots +4/-0

Enable Infinity3 JPEG snapshots

• Routes snapshot lifecycle and capture through the Infinity3 HAL and zero-initializes encoder settings before use.

src/jpeg.c

Other (2) +63 / -2
app_config.cPersist new stream, night, audio, and storage settings +54/-2

Persist new stream, night, audio, and storage settings

• Adds defaults, parsing, and serialization for sub-streams, MP4 audio, white lamp, exposure thresholds, recording reserve, and OSD anchors. Expands the permitted GPIO range for Infinity3.

src/app_config.c

app_config.hExtend the application settings model +9/-0

Extend the application settings model

• Declares the new sub-stream, MP4 audio, lamp, exposure, and recording-reserve fields.

src/app_config.h

@qodo-free-for-open-source-projects

qodo-free-for-open-source-projects Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

Grey Divider


Remediation recommended

1. Recording downloads can expose other files ✓ Resolved
Description
send_clip_thread() follows a requested recording-directory symlink and checks only that its
resolved target is a regular file. If a symlink named with an .mp4 suffix is placed on writable
recording storage, a web client can download any target readable by the server, even outside that
directory.

2. The Stop button cannot stop continuous clips ✓ Resolved
Description
/api/record calls record_stop() for wantStop only when record_continuous is false. With
continuous recording enabled, the page's ?stop request instead enters the start branch and leaves
the current recording active.

3. Failed recordings leave the card writable ✓ Resolved
Description
record_start() remounts the card read-write before opening a file, but its open-failure path does
not restore read-only mode; fragment-write and rotation failures likewise mark recording inactive
without restoring it. A missing or unusable destination, failed write, or failed rotation therefore
leaves the card writable after recording has ended.

View review recommended (11)
4. A restarted recording can become read-only ✓ Resolved
Description
record_stop() releases recordMtx before its read-only remount. Another thread can start a new
recording and remount read-write in that interval, after which the old stop call remounts its newly
opened file read-only.

5. Low storage can erase unrelated clips ✓ Resolved
Description
record_make_room_locked() treats every name beginning with the configured stem and an underscore
and ending with the configured extension as one of its own segments. If the recording directory also
contains a user-supplied file with that pattern, low-space cleanup can select and unlink it without
checking the recorder's timestamped filename format.

6. Large timed requests never end ✓ Resolved
Description
/api/record parses duration as a nonnegative long but passes it to `record_stop_after(unsigned
int) without an upper bound. On a platform where long` can hold a larger value,
duration=4294967296 becomes zero and cancels the timed stop instead of ending the clip after the
requested interval.

7. JPEG quality settings have no effect ✓ Resolved
Description
i3_video_create() assigns fixed JPEG quality values, and i3_video_snapshot_grab() does not apply
its quality argument. On Infinity3, changing the configured snapshot quality or requesting a
different snapshot quality still produces images using the hard-coded encoder setting.

8. Clip lists can contain invalid JSON ✓ Resolved
Description
send_clip_list() excludes quotes and backslashes from filenames but inserts the remaining bytes
directly into JSON strings. A recording-directory filename containing a newline, tab, or other JSON
control character makes /api/recordings return a malformed list.

9. Failed logins freeze the web and RTSP servers ✓ Resolved
Description
sysauth_check_basic calls usleep(500000) on every failed credential, inside the caller's thread.
The HTTP server handles one request at a time in a single poll loop, and the RTSP thread checks auth
while holding its instance lock. So an unauthenticated client sending a bad Authorization header
about twice a second keeps both the page/API and new RTSP sessions unresponsive.

10. Login check can race between web and RTSP ✓ Resolved
Description
sysauth_check_basic reads and writes the static okCred cache with strcmp/strcpy and calls
the non-reentrant crypt(), all without a lock. The HTTP server thread and the RTSP server threads
(main and sub-stream) call it concurrently, so one thread's crypt() result or half-written
okCred can be compared by another, giving wrong accept/deny results.

11. Old password keeps working after a change ✓ Resolved
Description
sysauth_check_basic stores a successful raw Base64 credential in the process-global okCred and
accepts matching tokens on later calls without checking /etc/shadow again. If the password changes
or the account is locked, the old user:password continues to pass web UI and RTSP authentication
until the streamer restarts.

12. Camera can stay in night mode forever ✓ Resolved
Description
In the exposure-based night loop, if (idx > app_config.exposure_day) { ... continue; } skips the
daylight probe whenever the exposure index is above exposure_day, and exposure_day defaults to
0. Configuring only exposure_night (enough to turn this mode on) means any reading with idx >= 1
under the IR lamps blocks every probe, so the camera never goes back to day mode.

13. Sub-stream setup reports success after failing ✓ Resolved
Description
media_sub_enable logs failures from create_channel, i3_video_create, and bind_channel but
continues through later setup, assigns subIndex = index, and returns EXIT_SUCCESS. If any step
fails, startup treats the sub-stream as available, later vendor calls can run on partially built
channels, and subIndex can point to a released slot used for frame routing.

14. Low-storage cleanup can stall the live video ✓ Resolved
Description
record_flush_locked calls record_make_room_locked every 30 s. It scans the whole recording
directory and may unlink up to 256 files, all while holding recordMtx, inside record_fragment on
the encoder callback path that also holds mp4Mtx. With loop recording of 15 s segments the SD-card
directory grows to thousands of entries, so these slow scans and unlinks block the video thread and
stall MP4, RTSP and stream delivery.


Informational

15. A probe can leave the IR lamps off ✓ Resolved
Description
The daylight probe turns the IR LED off with night_irled(false), waits, and then on `if (manual)
continue;` skips the branch that turns the lamps back on. If manual mode is switched on through
/api/night during the roughly 2.4 s probe, the lamps stay off while IR-cut and grayscale remain in
night state.

16. Debug request tracing is left in the server ✓ Resolved
Description
free_client, respond_request and /api/speak each call access("/tmp/http.log") and append
request lines (URIs, queries, User-Agent, audio peaks) to that file whenever it exists. The DROP
line in free_client prints a stale errno that has nothing to do with the drop. This is ad-hoc
debug code: it adds a filesystem check to every request, and anyone with shell access can turn on
query logging by touching the file.

Grey Divider

Tip of the day
💡 Did you know, you can ask Qodo to dismiss a finding you disagree with, with your reason on record

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread src/server.c Resolved
Comment thread src/server.c Resolved
Comment thread src/record.c Resolved
Comment thread src/record.c Resolved
Comment thread src/record.c Resolved
Comment thread src/night.c Resolved
Comment thread src/media.c Resolved
Comment thread src/record.c Resolved
Comment thread src/night.c Resolved
Comment thread src/server.c Resolved
The /tmp/http.log request trace was debugging left over from the
Infinity3 bring-up. Clips are opened with O_NOFOLLOW so a link in the
recording directory cannot serve a file from elsewhere, and the clip list
skips links and names that would need JSON escaping. An explicit stop
now also ends continuous mode, which otherwise restarted the recording,
and a timed recording is limited to a day.
…o thread

Start and stop are serialized with their remounts, so a late stop cannot
remount the card read-only under a new recording. A failed start
remounts it read-only at once, and after a write or rotation failure on
the encoder thread the next tick does. The free-space cleanup runs from
the tick instead of under the recorder and muxer locks, and only removes
names in the exact <stem>_YYYYMMDD-HHMMSS[-N]<ext> form it creates.
The HTTP and RTSP threads both check logins: the cache and crypt() are
now under a lock. The last good credential is kept for five minutes and
only while /etc/shadow is unchanged, so a password change takes effect.
Instead of sleeping half a second in the caller's server loop after a
failure, new credentials are refused for a second without checking.
…upted probe

With exposure_day left at 0 the daylight probe never ran. A probe cut
short by manual mode or a settings change now turns the IR lamps back on
instead of leaving them off in night mode.
…t failure

The still and MJPEG encoders use the configured qfactor (fixed for the
channel's life). media_sub_enable returned success after a failed
channel, encoder or bind step.
The reports it sent to the camera's request trace went with the trace.
The snapshot channel is an MJPEG encoder: MI_VENC_SetMjpegParam changes
its quality, and the vendor only accepts that while the channel is
stopped, so it is set between snapshots, before the bind. Tested at
qfactor 20/50/80/95 (17/25/64/170 KB at 1280x720).
Since a903df9 the exported mmap takes a 32-bit offset. On infinity3 the
vendor libraries' mmap calls are renamed to the glibc-compat shim's mm32,
which calls mmap with musl's 64-bit off_t. With -rdynamic the exported
mmap overrides musl's for the shim as well, reads the offset from the
wrong argument slot, and the first vendor mapping faults with SIGBUS
during MI_VI init.

GLIBC_COMPAT builds already bridge the 32-bit callers through mm32, so
leave mmap to musl there.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant