There is a simple workaround for the reference passing problem:
Since the reflection api has to handle all parameters in a generic way it has no chance to guess if you wish to pass data per value or reference.

But it seems that you can also decide to pass a reference from where you call the function or method (not just only by the ampersand prefix in its declaration).

So just do the following; which worked for me:

<?php
//...
$method->invoke($object, $inputValue, &$outputValue);
?>

Since this will only be necessary with arrays and primitive data types it should be acceptable in most cases to know in advance if you need to pass per reference. But it is probably although necessary to keep the ampersand always in the declaration (because of the at least two layers between the actual function and your invoke call).

If this is the expected behavior it will maybe make sense to mention it in the documentation for invoke and invokeArgs.