WebWish

What it is

WebWish serves any SDL2 / Tk desktop application as a web app, with no changes to the app. It runs the program headless on a server and renders and controls it in a browser tab: the app's framebuffer is streamed to an HTML <canvas> over a WebSocket, and the viewer's mouse and keyboard travel back the same way. To the app, WebWish is another SDL video backend.

The driver works for anything that renders through SDL's software framebuffer path. The original target is undroidwish, AndroWish's Tcl/Tk-on-SDL runtime, run inside a browser.

The driver's internal name is wstiles (it appears in the environment variables and the on-the-wire handshake magic wtil). The project name is WebWish.

How it works

WebWish is an SDL2 video driver. When an SDL2 app selects it (SDL_VIDEODRIVER=wstiles), the app draws into an in-memory framebuffer the driver hands it, and the driver moves pixels to the browser and events back.

• Pixel format. The framebuffer is advertised as SDL_PIXELFORMAT_ABGR8888, whose little-endian byte order is R, G, B, A — what the browser's putImageData expects, so no per-pixel conversion is needed.
• Dirty-tile transport (lossless). Each frame is diffed against a shadow copy on a 64×64-pixel tile grid; only changed tiles are sent, each zlib-deflated. On first connect the whole screen goes out as a single deflated frame rather than hundreds of raw tiles — a flat 1024×768 UI's first paint drops from about 3 MB to roughly 19 KB (~160×), lossless. The browser inflates each tile with the native DecompressionStream("deflate") and paints it independently.
• Whole-screen AV1 (optional). With SDL_VIDEO_WSTILES_CODEC=av1, the driver hands each changed frame to a realtime AV1 encoder (libaom, tune=screen, 4:4:4) and the browser decodes it with the WebCodecs VideoDecoder. There is a live constant-quality knob and a bandwidth meter. From benchmarking: lossless tiles are smaller on flat UI (typically ~0.1 ms per tile, and smaller than AV1 for a static screen); AV1 is smaller on large animated or photographic change, e.g. a live shot graph.
• Browser-driven resize. The session sizes itself to the viewport — on connect and on window resize the client requests the space it can show, and the driver reallocates the framebuffer and tells the app. Opt out per page with window.WSTILES_AUTORESIZE = false, or cap it with SDL_VIDEO_WSTILES_SIZE.
• Input. Browser mouse and keyboard events are packed in a binary wire format and unpacked into SDL events. Control keys work, and multi-command round-trips have been verified end to end.

Three transports are available, chosen by environment variable:

1. Built-in server — the driver runs its own libwebsockets server and serves the client page directly.
2. stdio framing — length-prefixed [u32-be len][payload] frames on stdin/stdout, so an external proxy can bridge it. This is what the NaviServer reference bridge uses, and it means a container's stdin/stdout is the wire — no per-session port is opened.
3. oneshot — the process self-terminates when its last client disconnects, giving the per-session model.

A reference NaviServer bridge multiplexes many independent sessions through a single public port: each browser tab gets its own private process, spawned on connect and reaped on disconnect. The bridge is event-driven and runs off the connection-thread pool; an idle server measures ~0% CPU, and it handles many concurrent viewers.

The complete byte-level wire format is documented in the repository's docs/WIRE-PROTOCOL.md.

Status

Working alpha. As of mid-2026 the following are all verified live:

• Display, mouse (motion, click, drag), keyboard including control keys, and menus.
• Browser-driven resize.
• Lossless zlib tiles and whole-screen AV1 with a live quality slider.
• Per-session process spawn and reap (many users on one URL), plus the single-port, off-connection-thread NaviServer bridge.
• Runs on arm64 macOS (the primary development target) and in a hardened Linux container (x86-64 and arm64), built against AndroWish's SDL2 fork.

Prebuilt Docker images are published as a GitHub release: x86-64 and arm64, in lossless-tiles and AV1 variants.

Limitations: no touch input (mouse events only, so phones/tablets cannot drive a session); clipboard is one-way (server→browser frames are dropped by the client); WarpMouse is unimplemented; there is no shared multi-viewer stream (each viewer drives its own process); and it is a patch into an SDL2 build tree, not a drop-in shared library. Neither reference bridge authenticates or caps sessions on its own — see the security note below.

Security

Exposing a GUI app to untrusted users — especially undroidwish's default Tcl console — is remote code execution as a service. WebWish uses defense-in-depth; read the repository's SECURITY.md before exposing it to anyone:

1. Never expose a console. Ship a locked-down app inside a Tcl safe interpreter with resource limits.
2. Isolate every session. Run each one in a hardened, ephemeral container: --network none, read-only rootfs, non-root user, --cap-drop ALL, no-new-privileges, and pid/memory/CPU caps. Because the stdio transport makes the container's stdin/stdout the wire, no per-session port is even opened. For anonymous exposure, escalate to a stronger sandbox (e.g. gVisor or a microVM).
3. Authenticate and rate-limit the bridge. Per-session hardening bounds what one session can do, but does nothing about someone opening a thousand of them. Add auth and a session cap before any public deployment. (The driver's own built-in server does support HTTP Basic auth.)

Building & running

WebWish is a driver compiled into an SDL2 build and linked against. Constraint: it targets AndroWish's SDL2 fork (SDL 2.0.6), not stock upstream SDL2 — several internal APIs differ enough that stock SDL will not compile. The repository's docs/BUILDING.md states the requirement and the source layout; patches/ documents the edits that wire the driver into an SDL2 tree.

The prebuilt Docker image plus the bundled bridge requires no build:

# load the image for your architecture
gunzip -c webwish-undroidwish-amd64-tiles.tar.gz | docker load

# one hardened session; stdio is the wire, so no port is opened
docker run --rm -i --network none --read-only --tmpfs /tmp \
  --user 65534:65534 --cap-drop ALL --security-opt no-new-privileges \
  --pids-limit 128 --memory 256m --cpus 1 \
  webwish/undroidwish:latest-amd64

Given a binary, the driver's own server serves it directly:

SDL_VIDEODRIVER=wstiles SDL_VIDEO_WSTILES_PORT=8090 \
    ./undroidwish-wstiles yourapp.tcl
# then open http://localhost:8090/

For many users on one port, the self-contained server/ bridge drops into any NaviServer docroot subdirectory with nothing to edit — stream.adp locates its siblings relative to itself, and dropping an app.tcl beside it makes that the app each session runs. For an existing NaviServer, docs/DEPLOY-NAVISERVER.md documents a no-build deploy (load an image, copy one directory).

Key environment variables:

Implementation notes

• Zero-copy pixel format. Choosing ABGR8888 so the framebuffer's byte layout matches putImageData eliminates a conversion pass on every frame.
• One deflated frame, not hundreds of tiles, on connect. Sending the first paint as a single zlib stream turns a 3 MB cold start into ~19 KB while staying lossless.
• Independent per-tile decode. The client inflates and paints each tile on its own rather than chaining them; a single rejected promise in a chain would poison every later repaint.
• Codec chosen for the workload. Lossless tiles for static UI; AV1 with tune=screen (which fixes the text-blur of the older MPEG-1 path) for animated/photographic content — with a measured keyframe often ~2× smaller than a full zlib frame and clock-tick deltas of ~100 bytes.
• Bridge concurrency. A stdio child pipe and a WebSocket must cooperate inside NaviServer's event reactor: the socket side is event-driven only, and the child pipe needs a Tcl event loop. Each session runs on its own detached thread with event-driven output, off the connection-thread pool. An idle server measures 0% CPU with one detached channel per client and no leaks.
• Upstream bug found and fixed. After a root-window resize, SDL-Tk recreated the screen texture with the wrong pixel format at 32bpp, swapping red and blue on every subsequent frame (a blue desktop turned brown). The fix affects only backends that report ABGR8888 — which includes iOS (uikit) as well as wstiles — and is carried as a documented patch.

Repository

Source, build recipes, wire-protocol spec, security model, and prebuilt images: github.com/johnbuckman/WebWish. Released under the zlib license (the same as SDL); driver/SDL_wstiles.c is derived from SDL's SDL_jsmpeg.c.

docs/AGENT-BOOTSTRAP.md documents current state, build recipes for both architectures, the bugs already solved (with the wrong theories that were ruled out), and the traps to avoid.