It is expected that the transition from 0.10 to 0.11/1.0 will be much easier on application and plugin developers than the transition from 0.8 to 0.10 was.

GStreamer 1.0 "summit"

There was a meeting discussing GStreamer 1.0 at the Gran Canaria Desktop Summit in July 2009. Thomas took some notes which can be found here: ?GStreamer summit notes

GStreamer Conference

The first ever GStreamer conference took part in Cambridge (UK) on 25th October 2010. Wim Taymans announced the plan for GStreamer 1.0 in his keynote, with a retrospective of some of the things that will be improved (slides here).

List of issues to fix/look at before 1.0

This looks worse than it is, most things should be fairly easy!

?GstPadTemplates are currently full-fledged objects! (with unused pad-created signal..) Make them light-weight, we're creating LOADS of them when loading the registry [really? no, we're creating static pad templates..].. ideally make it so that we can embed them in element/factory instance structs and/or so that they're cachable and we can maybe even share the template between the factory and the element?) - basically: why not make all ?GstPadTemplates effectively ?GstStaticPadTemplates, ie. just simple structs? - slomo: maybe just get rid of static pad templates (and static caps?!) and only use ?GstPadTemplate objects?

MUST-FIX (Edward will work on that, to add annotations to g-i to express hierarchy for structs/boxed types)

caps/structures:

WANT-FIX: put a ?GstCaps pointer into ?GstStaticCaps instead of "deriving" from ?GstCaps? (what's the point? does it matter? would make atomic stuff much easier)

WANT-FIX: instead of having a private pointer in the public ?GstCaps struct, hide that, and have a 'private' internal extended ?GstCaps struct with the private details, since we don't allow subclassing from it anyway, do we?

have we solved the problem for bindings that writability of things like buffers, caps, queries etc. depends on refcounts?

define what codec_data should mean for each format, try to be as consistent as possible
* WANT-FIX, TRIVIAL:
* rename "codec_data" to "codec-data" everywhere
* rename "stream-header" to "stream-headers" everywhere and only allow an array of buffers (not a single buffer)
* document codec-data vs. stream-headers
* codec-data is just out-of-band data for the codec
* stream-headers is stuff that e.g. a tcp/http sink should send to a client before sending whatever other data it's got pending
* stream-headers should always also be pushed as buffers at the beginning

typefinder API for typefinding functions:

MUST-FIX: at least look at it

we should be able to come up with a better way for typefinders to get to the data

can we do the horrible ?DataScanCtx better somehow and integrate with API?

MUST-FIX: should drop requirement that the result of peek stays around for entire time the typefinder is active

typefinder registration API:

MUST-FIX: at least look at it

would like to be able to pass more information to typefind system, such as
* flags (is this a scanning typefinder or not, aware of multiple passes, does it look at the end/middle, if random access is wanted/needed)
* max rank this will ever return

typefind function signature:

MUST-FIX: at least allow for optional ?GstTypeFindContext argument (with gpointer typedef for now), so we can pass a context with more info to typefinders later
* with info such as: first pass/second pass, max rank found so far, if random access is available

new events:

MUST-FIX (can't be just added later, 1-N/N-N elements like demuxers need to know about that) (not absolute must-fix, but would be good to start)

GST_EVENT_STREAM_ACTIVATE:
* slomo has started this

-define GST_FLOW_RESEND in connection with RECONFIGURE event:-

-MUST-FIX (remove or define, can't really be added later)- Removed

-just makes things nicer if downstream sends a reconfigure event for some reason, then the chain func upstream gets to know about it directly and can possibly re-send the buffer, so we don't necessarily lose a buffer when reconfiguring-

use cases: streamsynchronizer in playbin2 to align different stream segments, hlssink, multifilesink, etc.

more functions should have a GError argument

MAYBE-FIX, TRIVIAL

e.g. gst_preset_{load|save} (+delete/rename?)

add GError * arguments to all synchronous API that may fail, so you get automatic exceptions for bindings
* Tim: imho metadata annotation to throw an exception on FALSE/NULL returns would be sufficient, but it was categorically stated that this is the wrong approach and GError is the expected way to report exceptions

don't abuse buffer offset/offset_end for granulepos, put in buffer metadata if still needed (ideally muxer should do that based on timestamps)

playbin/decodebin

CAN-BE-DONE-LATER

should reconfigure themselves on reconfigure event

if caps change upstream (e.g. HLS) rest downstream needs to be reconfigured (e.g. mpegtsdemux would not create new pads, but change AC-3 stream to MP3)
* in MXF, the format can change within the stream (internally implements pitivi-style timeline)

port streamsynchronizer to 0.11 and make it work as is
* and CAN-BE-DONE-LATER: make sure we can do all the hackish stuff properly in 0.11 (e.g. force preroll in a sink somehow without sending 0-size buffers)

check "bitrate" property of all encoders, should be in bits per second, not kbits/sec or so.

MAYBE-FIX: clean up the ?GstBus API (add_watch can only be called once), maybe deprecate add_watch in favour of "message" ?

rename adwatch

why do we export the core debug categories? (GST_CAT_*) - tidy up in general maybe?

WANT-FIX, TRIVIAL

ensonic: we export core categories for baseclasses or generic topics, whats the question here anyway?
* tpm: only CAT_QOS + CAT_PERFORMANCE are ever used really (and CAT_SCHEDULING twice in libgstbase). And of core, base, good, ugly, bad there are only a handful of usages of those at all anyway. Oh, and PERFORMANCE is not in the public headers (you did that), so has to be retrieved by name anyway. The question is if we shouldn't just hide them all from private headers given that they're essentially not used anywhere but a handful of places (where they could easily be replaced by the CATEGORY_GET thing) (however, this is about the least important item ever, so don't really care)
* ensonic: +1 for removing them from the headers.

check all elements to make sure they don't have properties of long/unsigned long type

MUST-FIX, TRIVIAL: gst-inspect-0.11 -a >/dev/null

check all elements for whether they have sometimes srcpads that should be converted to always src pads (with caps just set once known)

WANT-FIX, EASY

we didn't do that originally because decodebin couldn't handle that

e.g. wavpackparse, aiffparse, etc.

make sure we don't use long/unsinged long/glong/gulong in the API anywhere

MUST-FIX, TRIVIAL

-e.g. gst_pad_add_probe, gst_pad_remove_probe-

get rid of the boolean return in plugin_init() or element_register()?

WANT FIX, TRIVIAL

review interfaces and elements

WANT FIX: review all elements

whether signals + properties are useful

whether they should send messages instead of signals

bit/bytereader: check if it's not faster to increment data pointer as we go along, instead of doing reader->data[reader->pos] for every read

Sebastian has implemented this, partially at least, and says it's not actually faster

hide gstvalue stuff from caps by adding dedicated API for lists/arrays/etc.? Add ?GstCapsStructure which only takes certain allowed types; could get rid off all the ?GstValue registration and table indirection for compare/subtraction/intersection/etc., just do switch/case
* conclusion: hide GValue for now from API, add explicit per-type setters + convenience functions for arrays, lists etc.

get rid of libgstriff, or at least the data driving. Just parse headers and let caller do the rest

WANT-FIX: clean-up aggressively

get rid of funcs that drive data reading (pull etc.) (are those still used anywhere?)

should just parse header structure really, like codec parser lib

merge with some other lib? (containerutils lib?)

filesrc/filesink

use gio: portability

maybe just move gio plugin to core?

WORK IN PROGRESS

controller: Stefan is working on this

this is work-in-progress and/or mostly done. Stefan is working on it (28 Dec 2011)

MUST-FIX: make bindings friendly (if you touch multiple things you just get a gpointer and must interpret it differently depending on the type of the property)
* (ensonic) who is 'you' in the above comment?
* (ensonic) ?GstValueArray is now gone, still we pass the values as a gpointer, using a GValueArray would cause quite an overhead, for elements that want to apply sample-by-sample changes this is good, it apps (using bindings) want to get a preview of a control curve, they better loop over the time themself

-MUST-FIX: remove deprecated and unimplemented interpolation modes-

WANT-FIX: clean-up in general (see FIXMEs in code)
* suggest_next_sync would need playback_direction (two FIXMEs), we could also pass direction to gst_object_sync_values (the place of the other FIXME)

-getchild_by_name() is very inefficient in some cases, should be an interface vmethod that could (optionally) be implement, e.g. in cases where the bin knows about the names-

-maybe make add_child/remove_child also return the name of the object added/removed-

thread-safety needs fixing, there should also be an iterator-based API, and/or add API to get cookie
* ensonic: for which functions? thread safety regarding the collection of children while set/get?

DONE

Core + Libs

buffer timestamping

-MUST-FIX: how to express DTS / PTS (just two ts fields on buffer?)- FIXED: added pts and dts to buffers

-device probing, get rid of ?GstPropertyProbe (we need a proper device discovery/listing API by type that makes properties and caps available and creates a suitable element given a discovered device object)-

rationale: this fixes e.g. timing problems when tags change mid-stream like with icydemux - if there's buffering between icydemux and the sink the player will get the new tags while the old song is still playing

-MUST-FIX, TRIVIAL: pass GstURIHandler * handler to gettype() and get_protocol() methods. No, it needs to be the GType, we don't have an instance in the registry - -get rid of full methods in interface-

-v4l2src should set framerate=0/1 for webcams and optionally add a "max-framerate" field to signal this downstream (so v4l2src ! videorate ! ... doesn't negotiate to 25fps by default if 15fps was set on the driver)-

-this means a codec can express that it requires a fixed framerate by adding framerate = [ 1/MAX, MAX/1 ] to template caps-

-gst_filter_*() should be removed or renamed or be made private-

-MUST-FIX, TRIVIAL-

-pad templates:-

-MUST-FIX, TRIVIAL: fix pad template names, e.g. foo_%d doesn't make sense since we never want a pad called foo_-1-

-especially fix those that use foo_%2d (avi)-

-should all be %u-

-especially: rtpmanager-

-also consistently use underscores in pad template names, i.e. sink_%d and not sink%d-

-?GstTagDemux: convert src pad to always pad and just set caps once known-

-check new functions, must be gstfoo_bar_new_xyz, not gst_foo_bar_xyz_new()-

can we put pointers into ?GstMeta? If yes, use GIO's address object instead!
* or use GIO in API and provide conversion to/from meta (wtay: we can add pointers but we don't want to allocate a new GInetAddress for each udp buffers)