Speaker
Description
Modern devices come with multiple different hardware components that can be used for inference workflows: CPU, NPU, iGPU, dGPU, etc. The APIs to access them differ not only between the individual components and hardware vendors, but also between different platforms. To complicate the situation further, multiple hardware vendors provide extension libraries on top of GStreamer for accessing their specific hardware as efficiently as possible.
When writing applications, developers either have to implement a custom abstraction layer or, more often, lock into a single hardware/platform which increases maintenance costs and limits deployment.
Since the Analytics API was introduced to GStreamer in 1.24, this situation has considerably improved. Many inference framework-agnostic tensor decoders for different models were introduced, as well as new model-agnostic metadata for inference results and overlay and tracking elements. Together with the onnxinference and tfliteinference elements and the newly added whisper.cpp and llama.cpp elements, GStreamer can be used to develop write once, run anywhere inference pipelines.
This talk will provide an overview of the current state of inference in GStreamer, discuss possible future developments, and show how GStreamer can be used to develop cross-platform inference pipelines.
Speaker Bio
Sebastian works at Centricular on GStreamer and other projects, and has been working on GStreamer for about 20 years.
| Duration of the talk |
|---|
