There are no way to pass given blocks to other methods without using block parameters.

One problem of this technique is performance. Proc creation is one of heavyweight operation because we need to store all of local variables (represented by Env objects in MRI internal) to heap. If block parameter is declared as one of method parameter, we need to make a new Proc object for the block parameter.

To avoid this overhead, I propose lazy Proc creation for block parameters.

Ideas:

At the beginning of method, a block parameter is nil

If block parameter is accessed, then create a Proc object by given block.

If we pass the block parameter to other methods like block_yield(&b) then don't make a Proc, but pass given block information.

We don't optimize b.call type block invocations. If we call block with b.call, then create Proc object.We need to hack more because Proc#call is different from yield statement (especially they can change $SAFE).

To delegate the given block to other methods, Single & block parameter had been proposed (https://bugs.ruby-lang.org/issues/3447#note-18) (using like: def foo(&); bar(&); end). This idea is straightforward to represent block passing. Also we don't need to name a block parameter.

The advantage of this ticket proposal is we don't change any syntax. We can write compatible code for past versions.

Associated revisions

insns.def (getblockparam, setblockparam): add special access
instructions for block parameters.
getblockparam checks VM_FRAME_FLAG_MODIFIED_BLOCK_PARAM and
if it is not set this instruction creates a Proc object from
a given blcok and set VM_FRAME_FLAG_MODIFIED_BLOCK_PARAM.
setblockparam is similar to setlocal, but set
VM_FRAME_FLAG_MODIFIED_BLOCK_PARAM.

compile.c: use get/setblockparm instead get/setlocal instructions.
Note that they are used for method local block parameters (def m(&b)),
not for block local method parameters (iter{|&b|).

insns.def (getblockparam, setblockparam): add special access
instructions for block parameters.
getblockparam checks VM_FRAME_FLAG_MODIFIED_BLOCK_PARAM and
if it is not set this instruction creates a Proc object from
a given blcok and set VM_FRAME_FLAG_MODIFIED_BLOCK_PARAM.
setblockparam is similar to setlocal, but set
VM_FRAME_FLAG_MODIFIED_BLOCK_PARAM.

compile.c: use get/setblockparm instead get/setlocal instructions.
Note that they are used for method local block parameters (def m(&b)),
not for block local method parameters (iter{|&b|).

insns.def (getblockparam, setblockparam): add special access
instructions for block parameters.
getblockparam checks VM_FRAME_FLAG_MODIFIED_BLOCK_PARAM and
if it is not set this instruction creates a Proc object from
a given blcok and set VM_FRAME_FLAG_MODIFIED_BLOCK_PARAM.
setblockparam is similar to setlocal, but set
VM_FRAME_FLAG_MODIFIED_BLOCK_PARAM.

compile.c: use get/setblockparm instead get/setlocal instructions.
Note that they are used for method local block parameters (def m(&b)),
not for block local method parameters (iter{|&b|).

History

I very much support this proposal. I have discussed ideas in this direction with Koichi earlier, but at that time was told that it may be too difficult to implement. I'm very glad to see that Koichi managed to implement it!

insns.def (getblockparam, setblockparam): add special access
instructions for block parameters.
getblockparam checks VM_FRAME_FLAG_MODIFIED_BLOCK_PARAM and
if it is not set this instruction creates a Proc object from
a given blcok and set VM_FRAME_FLAG_MODIFIED_BLOCK_PARAM.
setblockparam is similar to setlocal, but set
VM_FRAME_FLAG_MODIFIED_BLOCK_PARAM.

compile.c: use get/setblockparm instead get/setlocal instructions.
Note that they are used for method local block parameters (def m(&b)),
not for block local method parameters (iter{|&b|).

This change introduces a bug in RSpec. I'm working on a work around for RSpec (and hope to cut a release with a fix soon) but users running Ruby 2.5 with an older RSpec version will be affected, and the slight change in semantics introduced by this change might create bugs in other libraries and applications as well.

In RSpec, we've defined a Hook struct (which stores a block plus some associated metadata), and depend upon == working properly to compare two Hook instances. The fact that the proc is now initialized lazily is causing problems because what is fundamentally the same block can wind up with two different proc instances, whereas it had only one before. This causes two Hook instances which were equal before to no longer be considered equal. Here's a script that demonstrates the regression:

As the output shows, the two procs were equal on 2.4 but are no longer equal on 2.5. However, if we call a method on the block (such as inspecting it), it defeats the lazy initialization and allows them to still be equal.

As I said, I'm working on addressing this change in RSpec, and while I can fairly easily fix the tests that fail as a result of this, I'm concerned that there might be other bugs this introduces that are not caught by our test suite.

Is there a way to keep this feature w/o introducing this regression? If not, it might be worth considering reverting it since it does introduce a regression, and a very subtle one at that.

This change introduces a bug in RSpec. I'm working on a work around for RSpec (and hope to cut a release with a fix soon) but users running Ruby 2.5 with an older RSpec version will be affected, and the slight change in semantics introduced by this change might create bugs in other libraries and applications as well.

Four comments:

Don't report a bug on a closed feature, report it as a new bug (and link to the feature), thanks.

Comparing blocks/procs is always a somewhat dangerous thing.

Lazy allocation shouldn't mean multiple allocations. I haven't looked at the implementation, but I hope this can be fixed.

For a core library such as RSpec, please test the betas and release candidates (yes we know they were late this time). Things will be easier for you if you find problems earlier.

Don't report a bug on a closed feature, report it as a new bug (and link to the feature), thanks.

Thanks, I wasn't aware which way was preferred. I've opened #14267 to report this as a new bug.

Comparing blocks/procs is always a somewhat dangerous thing.

Agreed, but when dealing with RSpec hooks (which are fundamentally block + some metadata) this was the simplest way to make the features work. I'd have to put some more through into if we could refactor to avoid the need.

Lazy allocation shouldn't mean multiple allocations. I haven't looked at the implementation, but I hope this can be fixed.

I sure hope so!

For a core library such as RSpec, please test the betas and release candidates (yes we know they were late this time). Things will be easier for you if you find problems earlier.

I agree that would be a great idea, but to be completely honest, I'm very unlikely to make the time to do that. My open source time is very limited these days. If I have move time to spend on open source around future Ruby releases I will try to contribute in this way.