In this scenario _set_caps() will get called earlier than _buffer_alloc()
so we need to not override the number of buffers in the case that the
upstream element answers the query about number of requested buffers.