The AMD Alveo MA35D is a powerful media accelerator that also includes a dedicated AI processor. Upcoming release of MA35 GStreamer SDK will bring its AI features also to our favourite multimedia library. On a series of example pipelines, I will show how to use the new Alveo GStreamer elements to detect regions of interest, like faces or text, in video streams, leverage the object detection...
Mixing NVIDIA Jetson memory and the raw CUDA one is not straightforward. In this talk, we present the steps we follow to develop GStreamer plugins based on CUDA frame process. In order to work with this kind of memory, several data conversions and mappings are required. These mappings are expensive, so we based our filters on a custom CUDA mapped memory pool to get a noticeable speed-up.
Streaming from a PC a decent quality 3D video resulting from capturing a simple scene in a small room using a couple of cameras, though challenging, is getting possible. Using V3C standard, V-PCC extension for point cloud coding and GStreamer is compelling enough for adaptation. Because the standard’s reference implementation is not practically usable for real-time streaming, a more adequate...
GPAC is an open-source multimedia framework focused on modularity and standards compliance. GPAC provides tools to process, inspect, package, stream, playback, and interact with media content. GPAC is best known for its wide MP4/ISOBMFF/CMAF/DASH/HLS/CENC capabilities and is popular among video enthusiasts, academic researchers, standardization bodies, and professional broadcasters.
In this...
With the growing success of machine learning (ML) language and speech models over the past four years, ML systems are behaving increasingly like human brains. These brains must be be fed with data, and GStreamer is the perfect framework to do it. But how do we remove obstacles to rapid adoption ?
ML research and commercial development takes place almost exclusively in the Python world, and...
rtspsrc2 was merged into gst-plugins-rs early this year. In this presentation I'll talk about its architecture, its purpose, what it can do, and what it is supposed to do that the existing rtspsrc can never do.
You'll also hear about my experience writing a network source element in Rust.
This is an update after a year from the GstWASM: GStreamer for the web introductory talk. We will present the advances made, the challenges we faced, the ported elements, and finally, samples! - something we missed last time.
QUIC is a general purpose and connection-oriented transport protocol like TCP, however, unlike TCP it’s an application level protocol that’s built on top of UDP. It is built with the aim to improve latency, bandwidth estimation, congestion and solve the head-of-line blocking that has been typical of TCP-based connections.
Being quite new, there are some interesting challenges in realizing...
Raven AI Engine is a new cross-platform framework for developing AI-powered computer vision projects with full hardware acceleration across a wide range of edge and desktop devices. Raven provides ready-to-use solutions with pre-trained models for common vision tasks like tracking, background removal, or face anonymization, enabling developers to integrate these solutions into their...
Over the past 2-3 releases, a number of improvements have been done to plugins that use Apple-specific APIs, such as AVFoundation, CoreAudio, VideoToolbox, etc. There have also been build-system improvements, including continuous integration.
The out-of-the-box experience on macOS is much improved now, although there is still work to be done.
Attend to get an overview of the improvements...
A long time ago I joined a software agency working on a bunch of multimedia projects. Working for startups, we needed to iterate fast and deliver POCs validating their ideas. That's when the concept was born: a tool that would be powerful and customizable, yet written in a high-level language and possibly forgiving errors. That's how the [Membrane Framework][1] came to life, with API inspired...
Building robust audio policies on embedded systems that integrate well with upstream APIs and components such as PipeWire and PulseAudio has been a challenging task for a long time. In many implementations, custom hardware DSP management components are built, bypassing existing upstream functionality, making integration and maintenance harder than they need to be. In this talk, George will...
GStreamer's cross-platform support and robust RTP stack make it an excellent choice for building audio streaming applications. We leveraged these strengths to develop an AES67-compliant streaming application on Windows, designed to interoperate seamlessly with Dante devices.
We came across a number of interesting challenges in the process, some specific to being able to send packets every...
Urban Air Mobility (UAM) is a transportation system that uses electric vertical takeoff and landing (eVTOL) aircraft to transport people or cargo within urban areas, helping to reduce congestion on the ground. Aircraft operate in a busy urban environment, and may be semi or fully autonomous. In this use case there is a need to automate communication between aircraft and ground to reduce...
During different consultancy services we had some tasks to achieve high level of stability for some Windows GStreamer applications. We had to resolve race conditions, memory leaks and D3D11-specific issues, such as crashes in the driver. This talk aims to summarize what we have learned from those experiences.
Spoiler: The tests were helpful here.
ICE underpins all WebRTC connections and provides peers the ability to discover the most efficient path for transferring data. We will take a brief look into the steps involved in the ICE process and a more in depth look into how a sans-IO implementation might be implemented using the example of librice.
Sometimes, when I'm not working on GStreamer, I work on open source VR projects, and sometimes when I work on open source VR projects - I use GStreamer!
This presentation will talk a bit about the state of open source VR support, how GStreamer relates to it, some interesting use cases for GStreamer in that space, and things that can be improved.
Since Direct3D11 support was added in 1.18, GStreamer’s Windows graphics and hardware acceleration features have been greatly improved up to a level of GStreamer's Linux support, or even better than other OS supports in some aspect.
Then Direct3D12 based features were introduced in GStreamer 1.24 which gives another chance to improve performance and expand feature set.
In this...
