This talk will take the usual bird's eye look at what's been happening in and around GStreamer in the last release cycle(s) and look forward at what's next in the pipeline.
Laerdal Medical is dedicated to helping save lives by being a leading provider of medical training resources around the world, from hardware like CPR manikins to software, such as SimCapture. Underpinning SimCapture is a media recording and streaming service called CaptureNode, which, over time, has migrated from Flash to WebRTC and continues to grow to better serve the medical community. In...
The Threadshare framework is an asynchronous runtime and a set of elements which allows reducing resource usage when handling many streams.
It was introduced in 2018 and presented at the GStreamer conference in Edinburgh the same year.
After a reminder of the core principles of the framework, this talk will present the changes which occured since 2018.
This talk presents the implementation of a video encoder analysis within GStreamer. We'll demonstrate our video-encoder-stats element that collects real-time encoding performance metrics including bitrate, processing time, CPU usage, and VMAF quality scores, attaching this data as metadata to video buffers throughout the pipeline. The presentation covers our video-compare-mixer element that...
A common question from people who are interested in using GStreamer and who are planning to use C++ is, which C++ bindings to use. Most of the available choices so far are essentially unmaintained or not widely used, so the recommendation was usually to use the C API directly. I hope to change this recommendation now.
This year a new C++ bindings generator for GObject-based libraries was...
Developing and debugging applications leveraging hardware-accelerated video decoding often requires access to specific hardware, creating a significant barrier to early development and continuous integration.
This talk introduces a new V4L2 decoder implementation that provides a software backend, effectively emulating a hardware decoder. This allows developers to write and test their...
Brief history of GStreamer use at Twilio - starting from Video rooms in 2016 to modern day media processing on Gstreamer for voice, media recordings, media streams and integration with conversational AI providers.
Alpha Channel is an essential tool in modern video workflows, enabling transparency and visual effects as required in telepresence and Virtual Reality applications, or modern websites. The Versatile Video Coding (VVC/H.266) standard supports Alpha Channel natively by encoding transparency data as an independent auxiliary layer signaled via SDI and ACI SEI messages (ITU H.274).
This...
GstAnalytics has evolved into an important part of GStreamer, providing powerful elements and metadata APIs that streamline the creation of sophisticated analytics pipelines. This presentation showcases the significant advancements made to GstAnalytics this year, including:
- Tensor negotiation capabilities for seamless ML model integration
- More Pythonic bindings that facilitate...
Over a year ago, we introduced WirePlumber 0.5, bringing major advancements such as the event stack for fine-grained control of PipeWire events, runtime settings to dynamically adjust behavior, and smart filters for automatic audio and video filter handling. These features marked a big step forward. However, WirePlumber is still evolving, and many use cases remain unmet.
In this talk, we’ll...
A new version of the Enhanced RTMP(v2) specification was announced earlier this year and we have one of the features, Multitrack Capabilities, implemented (!9682) in GStreamer recently (audio only for now though).
In this talk, I will briefly introduce the...
ICE (Internet Connectivity Establishment) is a widely used standard for NAT (Network Address Translation) hole punching. If NAT hole punching fails, then TURN (Traversal Using Relays around NAT) can be used to relay data between peers.
librice is a sans-IO library that handles the intricacies of ICE and has recently gained support for communicating with TURN servers. We will...
HTTP Live Streaming (HLS), a widely adopted protocol for live video streaming, and has been supported by GStreamer for a long time. HLS enables streaming of multiple formats and bit rates, allowing players to dynamically adjust their streaming quality based on network conditions for ensuring optimal viewer experience.
HLS specification has support for I-frame only playlist where each media...
Go is a modern systems programming language that offers awesome concurrency. This talk will show you why you should consider Go for your next GStreamer project.
Basic support for VVC/H.266 was added in GStreamer 1.26. This talk will give an overview of the VVC codec and ecosystem, along with the building blocks and contributions in GStreamer for supporting this codec.
This is a continuation from my 2023 presentation (https://indico.freedesktop.org/event/5/contributions/232/).
In this iteration I will present the improvements that have been made to existing speech to text and translation elements, new transcription backends (deepgram, ..), and a new family of text to speech elements.
A demo might even happen!
In my master’s thesis, I explored region-based compression for sports broadcasting. We used FFmpeg because it provided a generic addroi filter for attaching ROI metadata that we could use for multiple encoders, but GStreamer currently lacks a simple way to define regions and pass them to encoders downstream, and I wanted to...
A short story about what's wrong with osxaudiosrc/sink, what the new elements will try to do better, and what possibilities will that unlock.
This talk will show how the buffer lateness, queue levels & pad push timings tracers can assist GStreamer developers in analysing buffer flows.
In WebRTC-based applications, video frames are often delivered at irregular intervals due to the nature of real-time communication. This irregularity can cause issues in waylandsink, which commits frames based on the arrival of frame_redraw_cb signals.
As a result, even when all frames are correctly delivered—for example, in a 60fps video where all 60 frames reach waylandsink—frames may...
https://github.com/Igalia/soothe
Soothe is a testing framework written in Python for encoder quality. It's a command line interface application that runs a number of test suites with the supported encoders. Its purpose is to compare different encoder implementations and configurations. It uses VMAF binary to measure the transcoded videos.
Audio source separation is the process of separating the individual sources from a mixed audio stream. This can be used, for example, to remove the vocals from a song in a karaoke application, to extract voice from a movie to allow for better transcription without background noise, or to remove one specific instrument from your favorite song and playing that instrument yourself.
In this...
A story of how we pushed the limits of LGPL compliance and created a monster dynamic library to rule them all.
Nicolas will share the latest updates on Video4Linux support in GStreamer, highlighting recent changes, ongoing integration work, and developments from the broader Linux Media community. This lightning talk continues the annual tradition of keeping the GStreamer community up to date with the fast-moving V4L2 ecosystem.
We have removed GStreamer-VAAPI subproject. It's mostly replaced by GstVA in gst-plugins-bad. We will talk about what's missing and how does the roadmap look.
Raven is an AI engine we are developing at Fluendo, designed for multimedia AI workflows. It combines AI inference with GPU-accelerated processing, giving full control over the entire GPU pipeline, from memory allocation to execution, allowing for deep customization across hardware and environments.
In this lightning talk, I will showcase a set of GStreamer plugins for background removal,...
Almost all existing code using the GstAnalytics API is in C or Python: inference elements, tensor decoders and all kinds of infrastructure elements.
In this lightning talk I will talk about my experience writing an inference element around the Rust burn deep-learning / machine-learning framework, writing a tensor decoder for YOLOX in Rust, and how it integrates with the remaining...
State of the art of the present and coming Vulkan video elements. We'll talk about architecture, codecs and challenges to achieve vulkan support ...
This talk will present skiareshape(gl), GStreamer elements that bring geometric transformations to your video pipelines using the Skia graphics library.
When your device's v4l2 implementation is so particular that it turns out to be easier to write a new element than to patch v4l2src
This talk will present the primary timeline registration feature we are working on in GES which enables live updates of Subtimelines. An important feature to enable true collaborative editing for complex projects.
In GStreamer, the media pipeline operation is defined by threads that are continuously pushing or pulling data through the pads of the linked elements. This is an elegant and very flexible way of operating the pipeline, but there are certain drawbacks.
PipeWire follows a different approach, using a single thread that wakes up elements in a dependency-based sequence to consume and produce...
We will guide you through our journey from a proprietary software and hardware solution to an open and Gstreamer-based platform for our IP based audio distribution at the Swedish public radio broadcaster, Sveriges Radio (SR). Sharing experiences, challenges, opportunities and crossroads we have faced when building, deploying and operating this hybrid cloud solution. Our platform supports both...
In WebRTC, data channels are used to exchange arbitrary data.
Data channels makes the perfect companion to the live video and audio features of WebRTC. Unfortunately, the throughput of the data channels is far from satisfactory in many network environments.
This presentation will summarize what Axis have been doing, are doing, and possibly will be doing to improve the performance in the...
Debugging and hardening software that relies on GStreamer often comes down to experience and the ability to pinpoint where problems really come from. Since GStreamer is Open Source, anyone can build that expertise, provided they can break problems down and trace them to their origin.
In this talk, Nicolas will describe the methods he uses in practice such as: splitting bitstreams into...
GStreamer’s playbin3 element provides a convenient and simple interface for basic media playback. Like earlier playbin elements, it has some support for external subtitles, but with some limitations. Extending it to handle multiple auxiliary streams - such as multiple audio or subtitle tracks - reveals a surprising number of complexities.
This talk delves into the challenges faced when...
The WebKit WPE and GTK ports are using GstWebRTC and webrtcbin as their WebRTC
backend. As the first Web engine to rely on GstWebRTC, improving spec
compatiblity is an important goal for us. During this talk we will present the
current integration status of GstWebRTC in WebKit and the achievements
accomplished since GStreamer Conference 2024.
In this talk, we introduce how AMD's HIP has been integrated into GStreamer. We will look at the motivation behind supporting HIP, the integration approach, and what it means for building GPU-accelerated media pipelines that run efficiently on both AMD and NVIDIA hardware.
In Pexip we’ve implemented a new base class, GstBaseIdleSrc, designed for elements that don’t need a dedicated streaming thread but only push buffers occasionally; either when signaled from the outside or on specific events. This avoids the overhead of an always-running thread and simplifies writing event-driven sources.
In this talk, we’ll introduce the motivation behind GstBaseIdleSrc,...
Abstract: Transforming Our Video Management System with GStreamer
In our journey to modernize and optimize our Video Management System (VMS), which currently manages more than 100000 cameras from various manufacturers, we've transitioned the foundation of our live RTSP camera streaming from a custom FFmpeg-based solution to GStreamer pipelines. Additionally, we have reengineered our offline...
Starting in the spring of 2018, our company has rebuilt the core technology of our video server platform on top of GStreamer. What started as a proof-of-concept to solve a cost of goods problem grew into a complete re-architecture of our application, enabling new products and solutions we never thought possible before.
In this talk, I’ll share our journey with GStreamer: how we first...
The purpose of this talk is to present the changes made after last year’s presentation. Initially, we will show again the goal behind this project, along with the following project's changes:
- WebTransport support in GStreamer
- Optimizing GStreamer with ORC for WASM
- Upstreaming process
- Real-life usage
The presenter has had the privilege of working on an app that requires low latency audio capture and render on macOS, Windows, and Linux. This has allowed him to make a direct comparison between the audio servers on these operating systems.
In this talk, he will talk about some of the challenges involved in shipping a production-ready desktop app that runs on all three OSes.
This talk will discuss the methods used to reduce latency in a bidirectional WebRTC pipeline, i.e., video but primarily audio in both directions. The primary topics are
- Splitting the pipeline using intersrc/intersink, discussing the global pipeline latency and how to do avoid unnecessary clock syncing. This discusses pipeline latency and what is needed to allow media streaming in two...
How we ported and brought dcsctp into GStreamer to replace the current stack (usrsctp).
This session will introduce newly added Direct3D12-based elements and features, and highlight the latest improvements. Building on last year’s presentation, we will showcase the progress made and share practical updates for creating efficient media pipelines on Windows
Earlier this year, I began writing a native PipeWire client library in Rust. The aim is to provide a safer alternative to the bindings around the C library, while also reducing the amount of boilerplate in both the library implementation and the user-facing API.
Achieving parity with the C API is no small task. In this talk, I will go over the overall approach to solving the problem, review...
