Add String.new(fixnum) to preallocate large buffer

=begin
Because Strings are used in ruby as arbitrary byte buffers, and because the cost of growing a String increases as it gets larger (especially when it starts small), String.new should support a form that takes a fixnum and ensures the backing store will have at least that much room. This is analogous to Array.new(fixnum) which does the same thing.

The simple implementation of this would just add a Fixnum check to the String.new method, and the result would be an empty string with that size buffer. This would allow heavy string-appending algorithms and libraries (like ERb) to avoid doing so many memory copies while they run.
=end

History

=begin
In Rubinius, we have found it useful to have String.pattern(size, value) where value can be a fixnum or a string. The string created will be size characters where value is repeated. For example:

|Because Strings are used in ruby as arbitrary byte buffers, and because the cost of growing a String increases as it gets larger (especially when it starts small), String.new should support a form that takes a fixnum and ensures the backing store will have at least that much room. This is analogous to Array.new(fixnum) which does the same thing.

I like the idea.

But I'd prefer adding a new class methodfor the purpose,
say. String#buffer(n), to adding new role to an argument by type,
or there may be a better name.

But I'd prefer adding a new class methodfor the purpose,
say. String#buffer(n), to adding new role to an argument by type,
or there may be a better name.

I thought String.new(1000) would be a nice equivalent to
Array.new(1000), since they both do essentially the same thing. But I'm
not opposed to a separate method. I would vote for something active and
descriptive like "allocate", but that's obviously not available.
"buffer" isn't bad.

I like "new" best. And of course Array.new changes behavior depending on
argument types too.

Yeah, seems like a reasonably good idea. The string formats seem a
little weird to me though...not what I'd expect. The analog with an
array would be to accept another array as the fill value, which seems
equally weird.

Or maybe just "size". IMHO the idea was that you do not need to have
a String beforehand so all solution which are instance methods receive
a String as argument are probably suboptimal because they will
typically be invoked like show above, i.e. with a String constructor
which needs one object allocation (including GC bookkeeping overhead)
and a bit of memory.

A typical use case would be this:

File.open "foo" do |io|
buffer = String.size(1024)
while io.read(1024, buffer)
# or even io.read(buffer.capacity, buffer)
$defout.write(buffer)
end
end

There is a slight difference. Array.new(1000) creates an array with a
thousand elements. The proposed String.new(1000) would create a string
with zero characters, but the ability to grow to 1000 characters (umm,
bytes) without internal reallocation.

I think this difference is enough to warrant different names.

Yes, that's a good point; and the resulting array would << to the 1001st
element.

I suppose then coming up with a common name that works for both would be
a good idea. I'm back to liking "buffer" in both cases.

String.buffer(1000) produces an empty string that can grow to 1000 bytes
without needing to resize/copy.

Array.buffer(1000) produces an empty array that can grow to 1000
elements without needing to resize/copy.

Whatever is decided, I think it's going to be something people want
(need) on 1.8, so I'll probably submit a backport request as well (and
perhaps write up a simple extension people can use until then).

I think the reason I dislike this is that you're creating methods that
are polymorphic on the types of their arguments, and yet we generally
don't do that in Ruby-level code. So by creating these methods, you're
giving them a different flavor from methods that would be written in
straight Ruby.

Neither of those methods are polymorphic on anything. They're both new
methods that accept a Fixnum.

Neither of those methods are polymorphic on anything. They're both new
methods that accept a Fixnum.

.new is

And already has multiple forms in Array, so there's precedent. Also,
adding multiple forms with different named arguments doesn't reduce the
complexity of that single method any.

String.buffer is not a meaningful name for a constructor, in my opinion,
whereas String.new has a pedigree. By adding a initial_size: n optional
argument, you exactly express the meaning—you're asking for an initial
allocation when String.new executes. Similarly,

Array.new([1,2,3], initial_size: 100)

But Array.new(initial_size: 100).size would == 0. That's confusing...I
think buffer better expresses that it's the backing store being sized
than the outward expression of the String or Array itself, which is what
initial_size means.

I would also expect that the cost of allocating and populating an
arguments hash for this would negate some of the gain from adding the
new form. Array.buffer(100) adds almost no overhead on top of the
physical creation of the backing store and object to wrap it, where
Array.buffer(initial_size: 100) creates both a new Array and a new Hash.
An implementation detail, sure, but we I think we just need something
simple here. Perhaps buffer just doesn't express it clearly enough?

Right now, we have File.open("fred", "w"), rather than
File.open_write("fred"). It seems like a good idea, particularly for an
interface that's likely to grow over time (I can forsee

It seems to me this is making the semantics of String.new much more
complicated, rather than simpler and more uniform. And at least encoding
is already available outside of "new", so this is little more than a
shortcut. But there's absolutely no way at present to allocate a string
with a guaranteed backing store size, and that's the sole intention of
this RFE.

I don't think the cost of a hash is going to be significant--if it is,
then I'd hope that implementors find a way of optimizing these styles of
keyword hashes, because they're used more and more (cough*Railscough*).

Hard to do, since it has to be a hash on the callee side. Constructing
the hash could perhaps be delayed, in case the callee was a C function,
but it still has to be something. In comparison, new(1000) is almost free.

Hard to do, since it has to be a hash on the callee side. Constructing the
hash could perhaps be delayed, in case the callee was a C function, but it
still has to be something. In comparison, new(1000) is almost free.

I'm at a total loss to see why
String.pattern("ab".size * 5, "ab")
should be in any way better than
"ab" * 5
but maybe that's just me. Can somebody explain?

It's not better. It's not intended that you see it. Behind the scenes
it has been useful for us to have this method. This is one example of
its usage. The string is allocated in one step and filled in one loop.

There are other places it has been useful. The most useful aspect is
requesting a particular size. The initial contents is a lesser, but
still useful aspect.

=begin
I guess the relative silence on this issue means there's not much more
to discuss. Here's a summary up to now:

Everyone seems to agree it's a good idea to add, so we should add it.
And I would like to see it backported to 1.8.6/7.

Everyone likes the flat fixnum form except Dave Thomas, who would like
it to be a keyword argument. But that would not support backporting and
no core methods currently accept keyword arguments, plus it would create
a throw-away hash in all current implementations.

Several names have been suggested: overload 'new', buffer,
preallocate, capacity, sized, reserve. I prefer 'buffer' and 'reserve',
with a strong lean toward 'buffer' because it mimics a well-known idiom
in the Java world: "String.buffer(1000)" == "new StringBuffer(1000)".

Other forms have been suggested that accept a fill fixnum or fill
string; however I believe we should skip these cases for now since we're
not actually creating a string of a certain size (and content), we're
creating an empty string with a backing store of a certain size. The
expectation is that the contents of that backing store are unimportant
(perhaps \000s), and so fill params are meaningless.

So for me, the solution is String.buffer(1000). I rest my case, your honor.
=end

=begin
I guess the relative silence on this issue means there's not much more
to discuss. Here's a summary up to now:

Everyone seems to agree it's a good idea to add, so we should add it.
And I would like to see it backported to 1.8.6/7.

Everyone likes the flat fixnum form except Dave Thomas, who would like
it to be a keyword argument. But that would not support backporting and
no core methods currently accept keyword arguments, plus it would create
a throw-away hash in all current implementations.

Several names have been suggested: overload 'new', buffer,
preallocate, capacity, sized, reserve. I prefer 'buffer' and 'reserve',
with a strong lean toward 'buffer' because it mimics a well-known idiom
in the Java world: "String.buffer(1000)" == "new StringBuffer(1000)".

Other forms have been suggested that accept a fill fixnum or fill
string; however I believe we should skip these cases for now since we're
not actually creating a string of a certain size (and content), we're
creating an empty string with a backing store of a certain size. The
expectation is that the contents of that backing store are unimportant
(perhaps \000s), and so fill params are meaningless.

So for me, the solution is String.buffer(1000). I rest my case, your honor.

=begin
Doesn't Ruby allocate already using a "double memory if you run out"
rule? That makes string concatenation (amortized) linear, even if the
string must be moved in the memory.

I doubt that there are real-world use cases that would be much faster
with preallocation. As Yusuke said, ERb is more of a counter-example.

Even with this API extension, we wouldn't have control over the
generation of the string buffer in many use cases, as in Array#join,
String#% or in literals using #{}. Its use would be limited to String#<<.

Doesn't Ruby allocate already using a "double memory if you run out"
rule? That makes string concatenation (amortized) linear, even if the
string must be moved in the memory.

Yes, it does. This is why I think experiment is needed.

Because the suggested feature can be used to omit first some
expansions, it will actually reduce time. But I guess if the
reduced time is not so much.

Even with this API extension, we wouldn't have control over the
generation of the string buffer in many use cases, as in Array#join,
String#% or in literals using #{}. Its use would be limited to String#<<.

Absolutely. The feature is hard to use.
Even if we pre-allocated a string, calling some method on the
string may shrink it.

I think we should call the feature just "optimization hint"
rather than API. It is better to think the hint may be even
ignored.

At first glance, the document explains the difference of destructive
and non-destructive concatenations, like String#+ and #<<.

It is absolutely different topic from pre-allocation.

It is related: the algorithm constructs large strings from smaller
ones in an elegant way using a "tower of Hanoi", and if the top
string concatenation gets bigger than the one below it, only then
are they joined together. Result is less copying and merging.
Admittedly, it is less applicable with mutable strings, but while
only the top of the tower is modified, there'd be less churn in
memory.

If String#<< is really O(1), there would seem to be little reason to
change anything. I still want to investigate this myself when I get a
chance.
O(n), where n is the size of the appended string.

But I think it's always worth to look into speedups even if we can't
expect to change the complexity class. The O-factor may not interesting
to theorists, but it matters greatly to programmers. JRuby, for example,
concats strings almost twice as fast in this benchmark:

Preallocation of String would be immensely useful in large ERB templates.

So much so, I was looking to patching into rb_str_resize(str, len) with a method, to get around related performance issues. Ruby Strings already support the difference between the string length and the allocated buffer size -- we need to expose it and ensure that Strings do not automatically "shrink" the internal String buffers. There should probably be a method to explicitly shrink the internal buffer, if needed.

From what I can tell string growth is roughly O(log2 N) because of the power-of-2 buffer resizing. For large buffers making this O(1) for large strings helps performance and reduces malloc() memory fragmentation.

Preallocation of String would be immensely useful in large ERB
templates.
How big would the buffer size have to be for this template?

<%= link_to @record.name, @record %>

So much so, I was looking to patching into rb_str_resize(str, len)
with a method, to get around related performance issues. Ruby
Strings already support the difference between the string length and
the allocated buffer size -- we need to expose it and ensure that
Strings do not automatically "shrink" the internal String buffers.
There should probably be a method to explicitly shrink the internal
buffer, if needed.
This sounds like C to me.

From what I can tell string growth is roughly O(log2 N) because of
the power-of-2 buffer resizing.
You probably mean O(N * log2 N). But even in the worst case (smallest
possible steps, string data must be relocated for each buffer
extension), it's still just O(N) where N is the length of the final
string. Example:

So it's exactly n bytes for the writes, and O(n) bytes must be relocated
in total (about 2*n since sum[i=0..k] 2i < 2k+1). Allocation itself
is O(1) for each step.

But I don't say it can't be further optimized in the real world.

For large buffers making this O(1)
for large strings helps performance and reduces malloc() memory
fragmentation.
Ropes have been mentioned, they provide constant time concatenation, but
have slower iteration and indexing. They also use more memory.

Is Array#join optimized for the case where all entries are strings? As in:

At first glance, the document explains the difference of destructive
and non-destructive concatenations, like String#+ and #<<.

It is absolutely different topic from pre-allocation.

It is related: the algorithm constructs large strings from smaller
ones in an elegant way using a "tower of Hanoi", and if the top
string concatenation gets bigger than the one below it, only then
are they joined together. Result is less copying and merging.

Ah, sorry. I had to read all more carefully.

The algorithm itself is interesting, but I understand it is
just workaround to implement efficient string buffer by usingimmutable strings (because Lua String seems always immutable).

But Ruby String is mutable. Is it also more efficient withmutable string than current direct concatenation? I wonder
if the algorithm needs more memcpy than the current.

Preallocation of String would be immensely useful in large ERB
templates.
How big would the buffer size have to be for this template?

<%= link_to @record.name, @record %>

Yes, it is generally difficult to determine the size.

We may be able to estimate it by using domain knowledge in some cases.
(e.g., certain page size is empirically known as about 10KB, etc.)
But if the expectation is disappointed, it will cause wasteful memory
allocation or no speed up.

So much so, I was looking to patching into rb_str_resize(str, len)
with a method, to get around related performance issues. Ruby
Strings already support the difference between the string length and
the allocated buffer size -- we need to expose it and ensure that
Strings do not automatically "shrink" the internal String buffers.
There should probably be a method to explicitly shrink the internal
buffer, if needed.
This sounds like C to me.

Agreed. It is too easy to waste memory.

But I don't say it can't be further optimized in the real world.

Agreed. So, we need a benchmark to discuss this.

For large buffers making this O(1)
for large strings helps performance and reduces malloc() memory
fragmentation.
Ropes have been mentioned, they provide constant time concatenation, but
have slower iteration and indexing. They also use more memory.

At first glance, the document explains the difference of destructive
and non-destructive concatenations, like String#+ and #<<.

It is absolutely different topic from pre-allocation.

It is related: the algorithm constructs large strings from smaller
ones in an elegant way using a "tower of Hanoi", and if the top
string concatenation gets bigger than the one below it, only then
are they joined together. Result is less copying and merging.

Ah, sorry. I had to read all more carefully.

The algorithm itself is interesting, but I understand it is
just workaround to implement efficient string buffer by usingimmutable strings (because Lua String seems always immutable).

But Ruby String is mutable. Is it also more efficient withmutable string than current direct concatenation? I wonder
if the algorithm needs more memcpy than the current.

Possibly. I've not gone into this in much depth. I thought it
might be helpful to raise it in case this would give significant
help to garbage collection. I'm thinking that as the strings get
longer they fill up space in the heap so need to be moved to the
newly allocated space. Dealing with only the top of the "tower of
Hanoi" would be handling smaller chunks. I think this would need to
be tested, but could be worth exploring. Lua is rather quick, and
the article talks about a big speed increase.

On the other hand, it is difficult to decide when to invoke this
algorithm. It is probably too heavy for just joining two strings,
but for reading in lots of chunks and appending them, it could be a
big help. I don't know how to detect that distinction in user code.
It might be too much work.

We may be able to estimate it by using domain knowledge in some cases.
(e.g., certain page size is empirically known as about 10KB, etc.)
But if the expectation is disappointed, it will cause wasteful memory
allocation or no speed up.

Generally, a given template should expand to about the same size every
time.

I’m getting the feeling thath the only real use case that we’ve got
for this so far is ERb. Wouldn’t it make more sense to change the way
ERb (and similar “string concatenators”) creates its result?

I’m getting the feeling thath the only real use case that we’ve got
for this so far is ERb. Wouldn’t it make more sense to change the way
ERb (and similar “string concatenators”) creates its result?
How about an optimized StringBuffer class in stdlib that's optimized for
this kind of stuff? But only if we really find a way to speed it up.

ERB template rendering is one of my greatest performance issues right now.
Have you really identified String concatenation as the primary issue?
There's so much more going on when building a template (especially in
Rails).

Somehow, my feeling is that the actual concatenation of a small string
takes even less time than the calling overhead of String#<< (accessing
self, method lookup, checking arguments, returning the recipient, ...)
We could be talking about, say, 2% of the time your template needs to
compile.

Here's a patch that doesn't work. I don't know what I'm doing wrong here: RESIZE_CAPA seemed just right.
Thank you for your writing a patch!
It seems to work on my environment. What made you think it does
not work?
The fact that the memory taken by the Ruby process didn't change in top.
I requested a 200MB buffer, and the process was still at 2.8MB.

Anyway, the overhead of concatenation seems not so big. I doubt
if it is the bottleneck.
That's my conclusion, too. But the JRuby team seems to have seen some
10% speedup:

Here's a patch that doesn't work. I don't know what I'm doing wrong here: RESIZE_CAPA seemed just right.
Thank you for your writing a patch!
It seems to work on my environment. What made you think it does
not work?
The fact that the memory taken by the Ruby process didn't change in top.
I requested a 200MB buffer, and the process was still at 2.8MB.

Hmm, I guess you saw physical memory size allocated.
On many platform, physical memory is not allocated until
writing into the page actually occurs.

If you use Linux, see virtual memory size (VSZ column of
ps command), instead of %MEM. It would reflect your huge
allocation.

The performance may be improved by using madvise, but I
don't think it should be supported by ruby core.

Ah, yes. "x" * 1000 is not so big string. then, its realloc() doesn't use mremap.
It mean string concat(i.e. "<<" operator) cause string copy on each time. but is
this real issue? Does small string copy makes big peformance issue? when? So, I
think we need good realistic benchmark.

Of course, this 10-15% improvement could simply be because the JVM
does not provide a "realloc" for its arrays (for various reasons, some
of them presumably because it moves objects around in memory a lot).
In order to grow a string, we have to allocate a new array and copy
its contents. Under those circumstances, String.buffer makes a lot of
sense, since the copying can get expensive at large sizes.

I don't know enough about MRI internals to implement an equivalent
String.buffer, but here's the patch to JRuby:

Of course, this 10-15% improvement could simply be because the JVM
does not provide a "realloc" for its arrays (for various reasons, some
of them presumably because it moves objects around in memory a lot).
In order to grow a string, we have to allocate a new array and copy
its contents. Under those circumstances, String.buffer makes a lot of
sense, since the copying can get expensive at large sizes.

Ok, we finally grasped the situation. To sum up:

This feature is meaningless with MRI, at least, on Linux.

But it serves as a workaround for slow string concatenation of JRuby
that cannot be optimized due to JVM.

mame: I don't think we can say it would not help MRI without testing an implementation, can we? I misunderstood realloc in my comment from two years (!!!) ago According to realloc docs:

The realloc() function tries to change the size of the allocation pointed to by ptr to size, and returns ptr. If there is not enough room to enlarge the memory allocation pointed
to by ptr, realloc() creates a new allocation, copies as much of the old data pointed to by ptr as will fit to the new allocation, frees the old allocation, and returns a pointer to
the allocated memory.

This seems to indicate that except under rare circumstances where the memory after the pointer is known to be free, realloc will behave exactly like the JVM, creating a new pointer, copying data, and freeing the old pointer.

To me, this means that a pre-allocated String construction method is most definitely useful.

It also occurred to me recently that String.new does not accept an integer argument. Perhaps all we need to do is add a String.new form that takes Integer, and possibly an optional fill byte/codepoint/single-char string?

to by ptr, realloc() creates a new allocation, copies as much of the old data

This "copy" is done by mremap(2) system call, which just reassembles OS's process-private virtual memory map to move a region of memory to another, in O(1). That is what mame said in "This feature is meaningless with MRI, at least, on Linux."

I do not believe for a moment that realloc or mremap can in all cases perform the operation in O(1) time, and the docs seem to agree with me...first based on the doc above for realloc, and then for this doc on mremap:

MREMAP_MAYMOVE
By default, if there is not sufficient space to expand a mapping at its current location, then mremap() fails. If this flag
is specified, then the kernel is permitted to relocate the mapping to a new virtual address, if necessary. If the mapping is
relocated, then absolute pointers into the old mapping location become invalid (offsets relative to the starting address of
the mapping should be employed).

It seems to me that preallocation is most definitely useful, even in the presence of realloc and mremap. I would like to see it added.

Perhaps you saw os x's realloc?
I wonder this issue is valid on os x.
Anyone can conduct a quantitative investigation?

headius (Charles Nutter) wrote:

I do not believe for a moment that realloc or mremap can in all cases perform the operation in O(1) time, and the docs seem to agree with me...first based on the doc above for realloc, and then for this doc on mremap:

mame: I do not understand how there's any way Linux would be different from any other platform. If there's no room in contiguous memory to expand a pointer, the data must be moved elsewhere in memory. Am I missing something?

mame: I do not understand how there's any way Linux would be different from any other platform. If there's no room in contiguous memory to expand a pointer, the data must be moved elsewhere in memory. Am I missing something?

Almost all recent practical operating systems are using the virtual memory mechanism.

In the OS based on the mechanism, there is a mapping from virtual memory addresses to physical ones.
By changing the map, contiguous virtual memory addresses can be (re)assigned without moving physical memory data.
(This is why the system call in question is named "remap", I think)

It would help JRuby and all runtimes that run on non-efficient-realloc platforms.

It does no harm and matches Array.new behavior.

For folks doing crypto stuff that want to know exactly how big the buffer is right away, this provides a way to do so.

I won't try to argue whether realloc is consistently efficient across platforms or not. It seems like it's not guaranteed to be on any platform.

It's also such a tiny addition...why not?

I don't imagine a lot of people take a string.buffer game for
optimization if it doesn't
have big benefit. now, this feature is unclear how much useful out of
jvm and how
much useful on jvm. afaik, nobody show realistic benchmark result nor
encompassing
affect platform lists. I'm not incline to agree guess game.

If the benefit is not so much, the feature will be dead and forgotten quickly.

For folks doing crypto stuff that want to know exactly how big the
buffer is right away, this provides a way to do so.

I'm not sure exactly what you mean. Do you mean to avoid leaving
sensitive data in the heap from realloc()? Yes it would help, but
I think this is a poor API for that purpose.

For security, you don't want strings to be growing and copying stuff
around in memory, so being able to allocate a specific size ahead of
time is useful.

Perhaps special methods like String#secure_cat and String#secure_wipe
is more obvious for security-concious users.

And if secure_cat didn't use realloc (because it could leave sensitive
data on the heap) you'd still have a need to preallocate what you
need. That doesn't solve anyhting.

I won't try to argue whether realloc is consistently efficient across
platforms or not. It seems like it's not guaranteed to be on any
platform.

I absolutely agree this can help performance regardless of platform,
however...

It's also such a tiny addition...why not?

I'm not a VM expert, but shouldn't it be possible for the VM to track
the growth of strings allocated at different call sites and
automatically optimize preallocations as time goes on?

A sufficiently smart compiler can do anything, of course. However I
know of no VMs that track the eventual size of objects allocated at a
given call site and eagerly allocate that memory, and such an
optimization would be very tricky to do right.