This talk will take the usual bird's eye look at what's been happening in and around GStreamer in the last release cycle(s), current challenges and look forward at what's next in the pipeline.
This talk gives an overview of the synchronization primitives used in GStreamer 1.0 pipelines: pad tasks, refcounting, push and pull call chains, serialized flow...
Then it uses these basic concepts to explain the special case of flushes: how they're supposed to work and why.
As the GStreamer framework continues to be a leading tool for building complex video analytic pipelines, the need for seamless interoperability across vendor components is more critical than ever. While the GstAnalytics library, introduced in GStreamer 1.24, provides a flexible foundation for representing AI-driven metadata, its very flexibility has resulted in slight differences between...
WirePlumber is the default session and policy manager for PipeWire, responsible for deciding which audio and video streams get connected to which devices or applications, and under what rules. In 2026 its internals got a significant update, and this talk will cover the biggest changes.
First, a new permission manager lets WirePlumber grant or restrict what each client is allowed to see and...
Modern devices come with multiple different hardware components that can be used for inference workflows: CPU, NPU, iGPU, dGPU, etc. The APIs to access them differ not only between the individual components and hardware vendors, but also between different platforms. To complicate the situation further, multiple hardware vendors provide extension libraries on top of GStreamer for accessing...
The purpose of this talk to show and demonstrate the use of Gst-Audit on a running Gstreamer Pipeline and explain its architecture, from the binary injection to the React frontend passing through the REST endpoint and JS bindings.
Abstract:
Live video streaming has become part of our daily lives. Apart from entertainment and surveillance, it is used across industries as part of perception systems for both human and non-human operators. Most commercial camera products are not designed for safety critical environments and may pose hazards that make them unfit for use in railway. We present our way to the world's first...
This talk presents the implementation of Digitally Signed Content (DSC) within GStreamer to ensure trustworthy video authenticity. We'll demonstrate our new dscsigner element, which generates cryptographic signatures for coded bitstreams, alongside dscverifier, which verifies video authenticity in real-time. The presentation covers the integration of JVET supplemental enhancement information...
The webrtcbin element was first proposed upstream a little less than a decode ago and presented 9 years ago at the GStreamer conference. In that past decade, every* web browser has grown support for WebRTC and thus has the functionality for real-time audio/video streaming.
We will look at some of the existing challenges the current webrtcbin design has faced and discuss some potential...
A personal retrospective on two decades of building with GStreamer — from the first tool I ever wrote, through the toolbox of utilities that grew around it, to the debugging tricks learned the hard way. Ends with a look at recent work, including GstPipelineStudio (GPS).
In this talk, we will share our experience enabling production-grade Edge AI solutions on top of GStreamer. The discussion goes beyond building working video analytics pipelines and focuses on what it takes to turn them into enterprise-ready systems that can be deployed, operated, monitored, and extended reliably at the edge. We will address this through two complementary dimensions:...
This presentation will explore Fluendo's ongoing efforts to provide support for the Dolby audio ecosystem within GStreamer. We will start with an overview of Fluendo's new proprietary Dolby decoders (AC-4 and EAC3). Next, we will delve into the open-source aspect of the work, detailing Fluendo's recent OSS contributions. Finally, the session will conclude with an outlook on the future,...
In video conferencing, network conditions and how they are handled can directly impact the perceived quality at the receiving end. To measure this degradation, the correspondence between reference and presented frames must be tracked across hosts in various configurations. Rather than relying on in-band visual markers, which can be fragile under poor channel conditions, we present a...
At Pexip, we run a custom GstTracer in production as an always-on profiler that measures per-element CPU to drive our capacity planning. This talk covers why we chose the Tracing API over pad probes, and how we made the hooks light enough to leave on in production.
At GStreamer Conference 2024 I proposed GStreamer as the ideal nervous system for feeding AI brains with data. As Python is the dominant language in AI, a new Python framework, gst-python-ml, was introduced to easily connect PyTorch models with GStreamer pipelines, with support for object detection, tracking, local speech to text and local text to speech.
This talk covers the many new...
Many devices today ship with hardware-protected Digital Rights Management (DRM) pipelines built on GStreamer. Too often, developers assume that DRM is destined to remain proprietary and that the only way forward is with hacks and downstream workarounds. In practice, this leads to heavily diverging forks, making upgrades difficult, pinning products to ancient GStreamer versions, and exposing...
In this talk we'll go over the struggles of maintaining a "complicated" GStreamer application and dealing with the hangs and crashes that come with large deployments. This year we built an automated pipeline that captures crash and hang dumps from our clients' machines, resolves them against the exact GStreamer build they were running, and uses commercially available AI models to point our...
GStreamer has been able to produce MPEG-TS for decades, but the MPEG-TS it produces isn't always great. Why is that? What would it take to fix?
GStreamer has had a Media Source Extensions (MSE) implementation for 3 years now but has seen basically no adoption. For a project we recently needed an API that allowed more control than just "play this file", and ideally would be able to share one implementation of a custom streaming protocol across platforms: iOS and tvOS, Android, and the web. MSE appeared as an ideal candidate for...
Did you know GStreamer now has an official Python wheel?
GStreamer already has all kinds of elements to use proprietary, cloud-based services for transcription and translation. Cloud services are not always an option for privacy and cost reasons.
In the last months, Mathieu and I have been working on adding a local option for this: a whisper.cpp-based speech-to-text element and a llama.cpp-based LLM text transformation and translation...
For my hobby photobooth, I built a GStreamer source plugin for digital cameras. It can capture live previews and trigger photo exposures. I'll show how little effort this took by using libgphoto2 and its Rust wrapper.
A quick introduction to Vulkan Video Samples, the Khronos reference encoder and decoder.
As a continuation of my talk last year on Threadshare, this presentation will sketch a few improvements implemented to facilitate Threadshare integration with WebRTC Bin 2.
Threadshare is a set of elements that leverage an asynchronous executor to reduce thread overhead and context switching costs.
A short story about GStreamer and machine vision on the road.
Servo is a web browser engine that uses GStreamer for playback. In this lightning talk we'll quickly go through the current status and plans for the near future.
I will briefly present the new whisper transcriber, GStreamer's first transcriber that does not rely on a third party service.
Many applications using GStreamer opt for spawning multiple pipelines, potentially inter-connected . Using large log levels to investigate an issue that happens in a single pipeline has unwelcome consequences (performance, disk usage when stored).
In this talk I will present a new tracer that sorts logs per pipeline, and can be configured with varying, per-pipeline log levels.
A follow up to last year's mpeg-ts talk, explaining the investigation and implementation of writing a mpeg-ts crate and demuxer in rust
This presentation covers some of the different ways in which GStreamer is used inside WebKitGTK and WPE to implement the multimedia features of the web engine. Common design patterns in WebKit will be introduced, as a basis to understand the design of the modules to be presented next: Regular and Media Source Extensions video playback, Encrypted Media Extensions using the Thunder/OpenCDM...
- New decoders & test suites
- The new sample comparison method that allows tolerance with the audio reference decoder.
- New Docker image with a hardware support or libva-fake-driver.
- Meson to build reference decoders
- Fluendo custom test suite for H.264-High444
The challenges and lessons learnt from using GStreamer in a highly-scalable streaming application.
The usual talk about the recent developments on both GstVA and GstVulkan
The goal of real-time media processing is to minimise latency. It's the enemy, and hides in many places.
Where? Well, let's investigate, and see what tools we have to fight it with. We'll kill it with fire, or just maybe... learn to be friends?
We developed streaming infrastructure for Remote Train Operation (RTO) based on our SIL2 camera system. Cameras are installed in the cabin of a train and safely stream to a wayside driver desk where trains are controlled remotely. In the process, we encountered challenges in providing reliable streaming over lossy local and wireless (5G) networks. Finally, we overcame several challenges of...
At Spiideo, we work with sports clubs and leagues to help them analyze and broadcast games. We install cameras at their venues and process the video to add object detection and autofollow capabilities, ingesting 4K H.264/H.265 video over SRT.
We do not control the network infrastructure at these sites, and we're not going to ask sports clubs to open all "ephemeral outgoing UDP ports" on...
ICE (Internet Connectivity Establishment) is a widely used standard for NAT (Network Address Translation) hole punching used extensively within the realm of WebRTC and in some cases SIP.
librice is a sans-IO library that handles the intricacies of ICE and has some new tricks up its sleeve. We will discuss the current state of librice, the work in progress and design decisions that have been...
This talk will give an update on the work done over the last year in GStreamer and GStreamer Editing Services (GES), most of it now upstream or on its way there.
We'll start with the work done to give GStreamer a proper color correction story: the new float video formats, what is needed to composite and grade in scene linear, and the OpenColorIO integration we are working on.
This talk...
In the last months, there has been a lot of reporting on LLM-based tooling easily finding dozens of security vulnerabilities in open source software. Projects like the Linux kernel or curl are well known to have been hit by this heavily, with many discussions about the usefulness and validity of these reports and how this flood of reports can be handled. As GStreamer is used in many safety...
A short high-level overview of what we know and don't know about the shape of GStreamer 2.0 and how we might get there.
Plus a panel discussion to bounce around ideas and identify priorities, and also collect input from the audience.
Let's explore ways to have a "new" GStreamer without breaking API for existing users/applications.
GStreamer's negotiation system is a powerful and unique mechanism, with some shortcomings that have been identified over the years.
In this talk I will present various experiments I have been implementing for testing new, non-backwards compatible design ideas, with the goal of retaining the capabilities of the existing system and addressing its shortcomings.
