A new Media Source Extensions implementation in GStreamer for cross-platform apps

GStreamer Conference 2026

10 October 2026

Sebastian Dröge <sebastian@centricular.com>

The Goal

  • Play live video over a custom streaming protocol
    • fragmented MP4 over WebSockets
    • Control over buffering, seeking, reverse playback
    • A "play this file" API is not sufficient
  • As much shared implementation between platforms as possible
    • Android, iOS and Web

The Solution: Media Source Extensions

  • Standard API available on the Web via JavaScript

  • Same JavaScript code on Android/iOS via a thin React Native
    wrapper around GStreamer

  • GStreamer already had an MSE implementation

  • Spoiler: that did not work!

Problems!

  • Has no actual users after 3 years and now we know why!

  • Extracted from the WebKitGTK platform

    • Not designed as a standalone implementation

    • Thread-safety as an afterthought

    • Deadlocks, memory leaks, errors, inconsistent behaviour

  • Tried fixing but rabbit holes everywhere

  • Decided to write a new implementation from scratch

Let's rewrite it in Rust

  • Similar interface to the existing C implementation

    • msesrc + pipeline ≈ HTML video element
    • source buffer, source buffer list, media source ≈ MSE spec

  • Designed as a standalone component outside a browser

  • Message passing architecture (GstBus) for thread-safety

The result: Onions!

center

Status

  • Used in production
    • Limited codec/container support for now (→ easy to add but needs testing!)
    • Some minor APIs still unimplemented

  • Upstreaming: should ideally replace the gst-plugins-bad C version

    • Needs docs, examples and possibly a playbin-style convenience pipeline

  • Servo: Calvaris is looking into implementing MSE support based on this

    • We'll try adding whatever is needed there

Questions? Comments?

If you have similar requirements: consider giving it a try!

Talk to me later!

Code available at