funnel

Gathers many HTTP senders into one MoQT track. Senders POST messages to a track’s URL; the funnel orders them into the track, one group each, names each one’s sender from its credential, and publishes the track as MoQT. Each record is committed to a qumo-ledger track before it is published. That ledger is in memory by default, or durable with LEDGER_URI, and it also serves the track’s history. The funnel publishes through a relay it dials as a client, so subscribers reach its tracks on the relay they already use; it can also serve them on a listener of its own. It does not join the relay mesh as a peer.

It suits small, frequent messages from many senders that subscribers want as one stream, such as chat, reactions or presence.

Usage

qumo funnel

Takes no flags or arguments — configured entirely through environment variables.

Example

$ RELAY_URL=moqt://relay:4433 RELAY_SIGNING_KEY=funnel-key.jwk RELAY_PUBLISH=room \
  LEDGER_URI=file:///var/lib/qumo QUMO_AUTH_KEYS=keys.json qumo funnel
	Ingest  : :8090
	Relay   : moqt://relay:4433
	Serve   : none
	Ledger  : file:///var/lib/qumo
	Auth    : keys.json

Without LEDGER_URI, records are kept in memory and are lost when the process exits. Without QUMO_AUTH_KEYS, every sender is accepted. Without RELAY_URL, the funnel serves subscribers itself on :4433.

A sender posts to the track’s URL, /tracks/ followed by the broadcast path and the track name:

$ curl -X POST localhost:8090/tracks/room/123/chat \
    -H "Authorization: Bearer $ALICE" \
    -H "Idempotency-Key: 7f9c2b" \
    -d '{"text": "hello"}'
{"group":"e000001-g00000000","wallclock":1791370000000000000}

$ curl localhost:8090/tracks/room/123/chat?limit=50 -H "Authorization: Bearer $VIEWER"
{"records":[{"group":"e000001-g00000000","wallclock":1791370000000000000,"sender":"alice","payload":{"text":"hello"}}]}

Subscribers consume the track chat of the broadcast /room/123. Any number of senders post into the same track, and a subscriber receives all of them on one subscription.

Requests

The last segment of the URL is the track name; the segments before it are the broadcast path.

Each record is stored as {"sender": …, "payload": …}: the payload as it was posted, and the sender its credential names (below). A record whose credential names no sender has no sender.

A record sent with an Idempotency-Key header is stored once per track: a retry with the same key is answered as the first was and stores nothing. A track remembers its 1024 most recent keys.

With FUNNEL_SENDER_LIMIT and FUNNEL_TRACK_LIMIT, records are limited per sender and per track; past either, a record is answered 429 with a Retry-After.

Credentials

With a key set (QUMO_AUTH_KEYS), every request carries a qumo credential as Authorization: Bearer, the same kind the relay admits sessions with and verified against the same key set. Its scopes (auth) name the tracks it reaches:

• Posting (POST, PUT) needs a post scope matching the broadcast path and the track name. The record’s sender is the credential’s sub, whatever the payload claims; a credential without sub posts with no sender. A publish scope does not post: it sends through a relay, so a credential meant for posting can’t publish into the relay directly, and a publisher’s can’t post.
• Reading history (GET) needs a fetch scope matching the broadcast path and the track name.
• Redacting (DELETE) needs a redact scope matching the broadcast path and the track name. The redaction carries the credential’s sub as its sender. No other scope redacts, so a sender’s post credential can’t take anyone’s record out; an app that lets senders redact their own records checks the record and redacts with a credential it keeps to itself. Redactions are published live and kept in history, so a sub there is visible to every subscriber; sign redact credentials without sub to keep moderators anonymous.

{"sub": "alice",
 "scopes": [{"actions": ["post"], "broadcast": {"exact": "room/123"}, "track": {"exact": "chat"}},
            {"actions": ["fetch"], "broadcast": {"prefix": "room/123"}}],
 "iat": 1791370000, "nbf": 1791370000, "exp": 1791370600}

posts into chat of /room/123 as alice, and reads any track of /room/123 and the broadcasts beneath it. It posts into no other track.

A path_auth credential is read as the scopes it amounts to: its sub reads any track at or beneath its path. Its pub is publishing, so it posts nothing.

The credential is checked on every request: once it expires or its key leaves the key set, it is refused. A missing or invalid credential answers 401 with WWW-Authenticate: Bearer, and one that does not cover the broadcast or the track 403.

Publishing through a relay

With RELAY_URL, the funnel dials the relay as a client and announces its broadcasts on that session; the relay subscribes to them when its subscribers do. The session needs a credential that may publish there. The funnel either uses the ?jwt= in RELAY_URL, or, with RELAY_SIGNING_KEY and RELAY_PUBLISH, signs a fresh one for every session, granting publish on RELAY_PUBLISH and every path beneath it, from a key the relay’s key set trusts (qumo auth keygen, ideally confined with -prefix). A credential lasts at most an hour, so one in RELAY_URL admits sessions only until then; a long-running funnel signs its own.

When a session ends, the funnel dials again after two seconds and announces its broadcasts anew.

What a subscriber receives

Each record is one group holding one frame, the record as it was stored, {"sender": …, "payload": …}. A record is sent only after it is committed, in commit order. A group’s sequence is one more than the sequence of the ledger group that stores the record. A new subscriber starts at the track’s latest record.

A subscriber reaches a track once it has been created, by its first record or by a PUT. A subscription to any other track name of the broadcast is refused as not found.

At startup the funnel lists its store once and publishes every track it recorded before, with its latest record, so subscribers reach them across a restart before anyone records again.

Configuration

Limits

• Plain HTTP. The HTTP listener does not terminate TLS; put it behind something that does, since credentials travel in its requests.
• Memory is not bounded. With no LEDGER_URI, every record stays in memory until the process exits. Use it for development.
• One process per track. Records of a track are ordered within one process; two processes recording into the same track are not coordinated.
• Idempotency keys do not survive a restart. A retry that reaches a restarted funnel is stored again.
• A relay session ends when its credential does. Subscribers through the relay see the track end and subscribe again when the funnel redials, at most hourly with a signed credential.
• A quiet track answers late. A subscription to a track with no record yet is answered when the first record arrives.
• Recorded tracks are not served over HLS. Records carry no duration, which the hls renderers need.

See also

• relay — the relay the funnel publishes through.
• rtmp — ingest media over RTMP.
• hls — the other command built on qumo-ledger.