Support for Media over QUIC

[longnguyen2004]

Media over QUIC is supposed to be the "spiritual successor" of WebRTC, and comes with easier configuration (no more ICE troubleshooting), easier routing (it's just QUIC, so you can put it behind reverse proxies), browser support (moq-player), among other things. MediaMTX has already supported MoQ, there's an OBS plugin and a GStreamer plugin, and FFmpeg support is in progress. Will this server support MoQ as well?

[Thulinma]

Sort-of! We've actually had support for QUIC as a media transport since 2023, but that code was never made public for several reasons (it used pure QUIC rather than HTTP3, it was under NDA for a few years because it was built for a specific customer, and it required linking against not one but two separate encryption libraries).

We've been keeping a close eye on the development of the MoQ spec. It should be noted that as of right now, the spec isn't finalized yet - so anyone claiming they support it, merely supports the draft (which could definitely still change quite a bit!). I don't think we'll be implementing the MoQ spec as it's currently shaping up to be. We are looking into adding support for moq-lite though. (moq-lite is also what the OBS and GStreamer plugins implement, by the way.)

The main bottleneck remains the requirement of an extra encryption library. MistServer uses mbedtls, and mbedtls does not support QUIC yet and it doesn't look like it will anytime soon. Of course compiling with two encryption libraries is possible (albeit wasteful), and it's what we did for our 2023 QUIC implementation.
But QUIC is only half the story: you also need HTTP3 to make MoQ work (though the spec supports both with and without HTTP3, all practical implementations have been HTTP3-only, using it's WebTransport feature as the main transport). Adding HTTP3 support to MistServer is practicularly tricky, since we currently use a separate binary for every connection, and HTTP3 is inherently connectionless. We did recently solve this problem for WebRTC when using WHIP/WHEP, which has the same issue, so we're currently looking into using the same method we used there to also support HTTP3.

Once all that work is done, we'll likely support three separate QUIC-powered media transports:

• DTSC over QUIC (what we originally built in 2023), which does not work with browsers but has interesting advantages between servers
• MP4 and WebCodecs over WebTransport (an iterative improvement over our current MP4 over WebSocket and the WebCodecs over WebSocket protocol we plan to include in the upcoming 3.11 release)
• MoQ-lite with fMP4 packaging (the version of the MoQ spec most implementations support) and potentially 1 or 2 other packagings.

That said, there's currently no clear timeline on how long this work will take. Development of MistServer is effectively steered by two forces: what our paying customers request, and whatever we as developers think is interesting technology to include. None of our current customers have expressed interest in MoQ so far, which means that MoQ and HTTP3 fall into the second category. These are things that we build out of principle, rather than because of a direct market-driven force. And unfortunately we can't afford to put deadlines on our principles. 🙃

So, tl;dr: it's coming, but we don't know exactly when yet. I'll likely reply here again when we have more news, though!

[longnguyen2004]

About TLS library, can I ask why mbedtls was chosen instead of other more established libraries such as OpenSSL, LibreSSL, gnutls or wolfSSL? Was it due to size limitations, static linking requirements or licensing concerns?

[Thulinma]

A combination of those:

• OpenSSL was considered too big (we also have customers running MistServer on embedded platforms)
• libressl didn't exist yet (it was created in 2014 and we started in 2009)
• gnutls is LGPL-licensed and that made it tricky for our then-current aGPL licensing (now that we have Unlicense this should be an option)
• woflSSL seemed to have less active development than mbedtls back in 2009

We're currently evaluating whether it would be possible to make the encryption libraries a compile-time option and allow picking between mbedtls, wolfssl, gnutls and/or openssl. That way we're no longer tied to any specific library and users can select the one that is most compatible with their intended usage and/or licensing. Ideally we support them all (automatically disabling incompatible functions as necessary) but we may end up only supporting 2 or 3 from the list depending on how things go. 🙃