# Pixel, soundtrack, and frame transport experiments

These codecs began as experimental transports for small binary payloads. PNG low-bit pixels, four-tone PCM WAV, and individual sparse-pixel PNG frames can now be selected as `bundle.json` website carriers and read by the browser runtime; see [FORMAT.md](FORMAT.md). The standalone exchange commands below transfer files or folders as sets of WAV or MP4 files. They use a separate media key and remain separate from `webApp.resources[].parts`. The standalone `frame-encode-set ... luma` still produces the older visible-tile experiment; website `frame-luma` writes an `AWF2` sparse-pixel packet. The amoosed.com demo places a fingerprint of its alternate film into three MP4 picture frames while carrying the full film in the MP4 UUID box. General muxed-video frame decoding is not a `bundle.json` carrier mode, and the Chrome extension does not capture YouTube playback. Encoding writes new files; it does not alter the inputs.

Generate a private key outside the published files with `amw media-key .amw/keys/media.hex`. Set `AMW_MEDIA_KEY_HEX` to its 64 hexadecimal characters before running any `pixel-*`, `audio-*`, or `frame-*` encode/decode command. The CLI encrypts the **payload bytes first** with AES-256-GCM and a random nonce, then converts the resulting ciphertext into pixels, tones, or frame symbols. Keys never appear in output media or `index.json`. The same key is required for recovery. The transport checksum and packet hashes cover ciphertext; GCM authenticates plaintext after recovery. The standalone media key is separate from website keys. Selecting `png-pixels` or `wav-tones` in `bundle.json` uses the existing website encoding and optional member encryption instead.

For a new local folder, create `.amw/keys/` first and keep `.amw/` in `.gitignore`. In PowerShell, load the saved key without typing it into command history: `$env:AMW_MEDIA_KEY_HEX = (Get-Content .amw/keys/media.hex -Raw).Trim()`. In a POSIX shell use `export AMW_MEDIA_KEY_HEX="$(cat .amw/keys/media.hex)"`. Store a private backup of this file; without it the ciphertext cannot be recovered.

## Lossless PNG pixel transport

`amw pixel-capacity image.png` reports the raw envelope capacity. `amw pixel-encode image.png payload.bin encoded.png` writes a new PNG; `amw pixel-decode encoded.png recovered.bin` verifies, decrypts, and recovers it. The input remains untouched. Only fully opaque pixels are used. The encoder changes at most one 8-bit level in each RGB channel and preserves alpha. It walks RGB channels with a deterministic coprime stride to spread the bits over the image. The first 44 decoded bytes are `AMWPX1\0\0`, a little-endian 32-bit ciphertext-envelope length, and SHA-256 of the envelope. The envelope adds 28 bytes (nonce and GCM tag). The maximum is 16 million input pixels; actual envelope capacity is `floor(opaque_pixels * 3 / 8) - 44` bytes.

The PNG is re-encoded, so ancillary metadata and exact source bytes are not preserved. This mode needs PNG byte-preserving hosting. JPEG conversion, image optimizers, resizing, and video encoding can destroy it. The checksum detects changed data but does not repair it. The CLI requires a fresh output path and refuses an existing one.

## Standalone file and folder exchange

The sender and recipient share a private `AMW_MEDIA_KEY_HEX` through a separate channel. The key is a transferable bearer secret; anyone with both the complete media set and the key can extract the files. The media contains no loose archive, index, filename list, or key file. Each encrypted part identifies its set and part number; SHA-256 checks the assembled archive and every extracted file. Keep all numbered parts together. A missing part fails extraction rather than producing a partial folder. Outputs are created in new directories and existing files are never overwritten.

```text
amw audio-encode my-folder tones --dry-run
amw audio-encode my-folder tones
amw audio-decode tones restored-folder

amw video-encode my-folder cover.mp4 videos --dry-run
amw video-encode my-folder cover.mp4 videos
amw video-decode videos restored-folder
```

Each command also accepts one input file. The decoder restores that file by its original name inside the output directory. `--dry-run` counts files and parts and estimates output bytes without writing files; its size estimate excludes filesystem overhead. WAV exchange divides the archive into roughly 4 KiB encrypted parts and writes numbered, audible 16 kHz PCM tracks. Each packet has three copies inside its WAV. The WAV output is much larger than the input and can take substantial time to encode or decode; use the dry run first. MP4 exchange divides the archive into 8 MiB encrypted parts, clones the supplied playable MP4 cover for each part, and writes ciphertext in a dedicated UUID box. This makes a set of playable videos; it does **not** encode the complete payload into picture pixels. The recipient needs the exact MP4 files. YouTube or another transcoding service may remove the UUID boxes, so this mode is for byte-preserving file transfer. These v1 commands process the archive in memory; available RAM and disk are practical limits, although the CLI imposes no small-payload cap.

Exchange v1 archive bytes begin with ASCII `AMXA1`, a big-endian 32-bit JSON-header length, JSON entries with relative name, byte count, and SHA-256, then file bytes in entry order. Paths are checked before extraction; symlinks are rejected on input and existing outputs are not overwritten. Each archive slice begins with `AMXS1`, a random 16-byte set ID, big-endian 32-bit part index and count, big-endian 64-bit total archive length, and the 32-byte archive SHA-256. WAV parts pass this shard through the existing AES-256-GCM media envelope and three-copy four-tone packet codec. MP4 parts put the same encrypted shard envelope in a UUID box with identifier `416d6f6f7365645472616e7366657231`. This box is appended to a byte-for-byte copy of the cover. The decoder checks the GCM tag, set identity, completeness, archive digest, and per-file digests before writing recovered files. No sidecar manifest is needed. This transfer format is separate from the website's AMW document header, website MP4 UUID mode, and member access keys.

## Audio packets for video soundtracks

`amw audio-encode-set payload.bin output-dir [clip-count] [cover-wav-dir]` creates one or more 16 kHz mono, 8-bit PCM WAV tracks and `index.json`. That compact WAV format is playable as an audio file. With no cover directory, the tones themselves are the piece: an audible data soundtrack can be intentional, such as a continuously playing museum installation, puzzle, or album companion. With a cover directory, provide `clip-001.wav`, `clip-002.wav`, and so on as 16, 44.1, or 48 kHz PCM WAV soundtracks, each long enough for its encoded packet track. The encoder mixes quieter tones into each cover, keeps the full cover duration, and writes new files. The artist or publisher chooses whether the signal should be heard; concealment is not a requirement of the format. Mixing has been tested with a synthetic music-like cover, but no perceptual listening test or real music corpus has been completed. Put each result into a separate video with a video editor, preserving timing; each `clip-NNN.wav` corresponds to one video. The index records each ciphertext packet's clip, copy number, start time in seconds, duration, full SHA-256, and overall ciphertext SHA-256. After extracting each video soundtrack as 16/44.1/48 kHz 8/16-bit PCM WAV, replace the local clip files and run `amw audio-decode-set output-dir/index.json recovered.bin`.

The codec uses four tones (1, 1.5, 2, and 2.5 kHz), with one 10 ms symbol for two bits. Ciphertext is divided into numbered 128-byte packets. Every packet is sent three times; with multiple clips, copies are distributed across clips. Each copy has a truncated SHA-256 check in its audio header, and the index has a full SHA-256 for each ciphertext packet and the complete ciphertext. A missing or corrupted copy can be ignored when another copy survives. This is repetition, not general forward error correction. The 4 KiB ciphertext ceiling and low bitrate make this useful for a key, index, or small clue, not a game archive. Audio data can in principle be recorded and played through cassette, radio, CD, or other sound paths, but each path needs its own recovery test and may introduce noise, filtering, speed drift, or timing changes.

## Video frame packets

`amw frame-encode-set payload.bin cover.png output-dir [fps] [clip-count] [pixel|luma]` generates numbered replacement PNG frames and an index. Pixel mode is the default: it reuses the RGB low-bit image codec, changes each channel by at most one level, and requires lossless video handling. The optional `luma` mode requires a 256×128 or larger opaque cover and uses paired brightness tiles. The index lists each target clip ID, frame index, second, packet number, copy number, full ciphertext packet SHA-256, and full ciphertext SHA-256. `amw frame-decode-set output-dir/index.json recovered.bin` verifies, decrypts, and reconstructs the payload from those PNG frames. Each packet is repeated three times, distributed across clips when requested. Tests recover after one missing replacement frame; the luminance mode also survives a coarse brightness-quantization test.

The luminance mode's tiles in the first 256×128 pixels are visibly altered; they are a diagnostic codec, not a finished unobtrusive watermark. The pixel mode looks close to the cover but its exact low bits are fragile under lossy video encoding. The tool creates replacement frames, not MP4/WebM files, and does not splice them into existing videos. The 1 KiB ciphertext ceiling is suitable for a clue or index. Before calling either mode a YouTube carrier, we need video assembly, upload, playback capture, recovered-frame timing, and actual transcode tests. A one-frame-per-second replacement may look like flicker, and YouTube may remove either signal.

The generated `index.json` is the dataset map. A future video profile can replace local WAV filenames with HTTPS video URLs or video IDs and use the same packet numbers plus time offsets. Unlike static media byte ranges, a hosted video has no stable file offset: streaming platforms may encode it into several formats. A consumer would need user-authorized playback capture and a demodulator. Chrome's `tabCapture` requires an extension invocation and its audio capture affects playback unless audio is explicitly routed back to the user. The ordinary YouTube iframe API controls playback but does not expose decoded audio samples. No YouTube round-trip test has been run; the prototype does not claim that YouTube's transcoding preserves these tones.

## MP4 picture-frame demonstration

`AMW_FFMPEG=<path-to-ffmpeg> node tools/create-demo-video.js` regenerates `demo/site/motion.mp4` and `demo/app/night-route.mp4`. The latter is a separate short film built into the Amoosed app. The compiler pins all of its chunks to the ordinary `motion.mp4` UUID box with `bundle.json` resource placement. The generator also writes an `AWV1` fingerprint of the film into frames 0, 24, and 48 of the ordinary 24 fps video. Each packet bit has three coordinate-selected brightness-pair copies. A public AES-CTR test key masks the first 21 bytes of the film's SHA-256 hash before embedding; this key provides no secrecy. The generator uses H.264 `yuv420p` with `-qp 0` for the cover, checks all three frames with FFmpeg, and the app checks one decoded Canvas frame against the reconstructed second film before enabling its player. The alternate film is in the UUID box, not wholly inside picture pixels. Other browser decoders and media transcodes remain unverified.

The MP4 frame experiment is limited to a tiny repeated fingerprint and is not a general video-frame website carrier. Lossy encoding, scaling, pixel-format conversion, and platform transcodes can erase the picture-frame data; transcoding also strips UUID boxes. No YouTube round trip is verified. The Amoosed app shows a resource-first table from its verified favicon index: each app file, its byte size and purpose, and the ordinary carrier files holding its chunks. This is an index view, not a fresh per-chunk integrity scan.

Teletext in analogue television used reserved vertical blanking lines. A digital video upload has no equivalent guaranteed top line. The prototype's top strip is ordinary picture data subject to scaling, cropping, chroma subsampling, and compression. Future work should distribute lower-contrast marks across frames, test perceptual impact, and validate after actual upload/playback. Reusing the PNG LSB profile in video frames will not survive ordinary lossy encoding reliably.

The media key is a transferable bearer secret. It protects a standalone payload but does not impose membership or prevent a recipient from sharing it. The website compiler has a separate member-key model. SHA-256 detects corruption relative to the index; AES-GCM fails if the ciphertext or key is wrong. A public index and observable signal can reveal that a transport exists even when its plaintext is encrypted.