24-26 September 2023
Palexco
Europe/Madrid timezone

Deep Upstream : Hardware Agnostic GStreamer Analytics

25 Sep 2023, 12:40
40m
Room 2 (Palexco)

Room 2

Palexco

Speaker

Daniel Morin (Collabora Inc.)

Description

With the growing power of machine-learning, the time has come for GStreamer to support complex, platform-independent analytic pipelines for tracking, super-resolution, noise filtering, speech recognition and more general analysis of timed data streams. We discuss a new flexible and efficient design to address these problems, without vendor or framework lock-in, which can easily interoperate with existing downstream approaches.

To achieve this goal, we have designed new framework-independent graph-based infrastructure using the existing GstMeta structure to store complex metadata and their relationships. We have also generalized the existing ONNX-based object detector to easily support many new inference models targeting a variety of hardware backends, and have built a new OSD to visualize the generated analytics metadata. Care has been taken to ensure efficient pipelines with support for batch processing and zero-copy. Finally, we have built a bridge to non-GStreamer land with a new cloud metadata sink that can send analytics results to cloud servers.

We will also present a demo at the end of the talk showcasing a complex two-phase video analysis pipeline.

Speaker bio Aaron has been building open-source systems since 2009, and enjoys the simple pleasure of squeezing every last ounce of performance from both hardware and software. He currently works for Collabora and is based in Toronto, Canada. Daniel Morin is software developer, based in Ottawa, Canada, he's been working in the video surveillance industry for 10 years, where he worked on multimedia systems and video analytics. In 2022, he joined the multimedia team at Collabora as a senior software developer working on the framework he loves: GStreamer.

Primary author

Daniel Morin (Collabora Inc.)

Co-author

Aaron Boxer (Collabora Inc.)

Presentation Materials

There are no materials yet.
2024 Platinum Sponsor
Collabora
2024 Gold Sponsors
Arm
Google
Microsoft
NVIDIA
2024 Silver Sponsors
FEX-Emu
Igalia
Qualcomm
The Linux Foundation
2024 Bronze Sponsors
CodeWeavers
Khronos Group
Libre Computer