7 Answers
7

Actually, glibc's implementation of strlen is an interesting example of the vectorization approach. It is peculiar in that it doesn't use vector instructions, but finds a way to use only ordinary instructions on 32 or 64 bits words from the buffer.

On the other hand, at least on x86/x86_64 and gcc, you'll just get the compiler's builtin anyway.
–
LnxPrgr3Nov 21 '09 at 21:59

Yes, you have to give it another name if you want to make sure the version used is yours. If you are going to do this, you might as well somehow ensure that all strings your version will be passed are word-aligned (if possible) and remove the initial char-by-char loop completely.
–
Pascal CuoqNov 23 '09 at 12:55

+1: Hooray Fortran (and don't allow to change it in any way after the declaration)
–
Stefano BoriniNov 21 '09 at 7:42

i did large improvements on strcat using this technique
–
MandrakeNov 21 '09 at 13:00

Bob, most of the time, its possible to maintain explicit length when we are writing string. Sometimes, its not possible :( For exa: reading stream from file or network. I might have got 500 characters but only first 200 character makes valid string and remaining 300 are not useful
–
JackNov 23 '09 at 3:29

That... doesn't sound right to me. Most situations involving file/network IO give you back the number of bytes read.
–
Bob AmanNov 23 '09 at 4:20

Obviously, if your string has a known minimum length, you can begin your search at that position.

Beyond that, there's not really anything you can do; if you try to do something clever and find a \0 byte, you still need to check every byte between the start of the string and that point to make sure there was no earlier \0.

That's not to say that strlen can't be optimized. It can be pipelined, and it can be made to process word-size or vector chunks with each comparison. On most architectures, some combination of these and other approaches will yield a substantial constant-factor speedup over a naive byte-comparison loop. Of course, on most mature platforms, the system strlen is already implemented using these techniques.

The longer answer: do you really think that if there were a faster way to check string length for barebones C strings, something as commonly used as the C string library wouldn't have already incorporated it?

Without some kind of additional knowledge about a string, you have to check each character. If you're willing to maintain that additional information, you could create a struct that stores the length as a field in the struct (in addition to the actual character array/pointer for the string), in which case you could then make the length lookup constant time, but would have to update that field each time you modified the string.

Now, consider that you know the length is about 200 characters, as you said. Say you start at 200 and loop up and down for a '\0'. You've found one at 204, what does it mean? That the string is 204 chars long? NO! It could end before that with another '\0' and all you did was look out of bounds.

Thanks for answer. As I said, length is vaguely predicted and may not end after character 200. Also, if we start reading after 200th character, we might be reading invalid string (if string has ended around 100 character...)
–
JackNov 21 '09 at 7:31

How would vectorization help? Even if the buffer was, say, 4 KB, there's no guarantee about the contents of the string after the first null, so if the vectorization started 4 operations (threads?) on the 1 KB boundaries, there's no telling what the three starting from a non-zero offset would see. I suppose the result would have to be the minimum of the 4 returned values - where the non-zero offset threads would have to add their starting position to the returned length.
–
Jonathan LefflerNov 21 '09 at 7:28

I think Elalfer is proposing to assign each consecutive byte to a vector to be checked as a whole, and then scrolling the string scan of the length of the vector. That would definitely work, assuming you have a vector-based architecture.
–
Stefano BoriniNov 21 '09 at 7:46

2

@Jonathan Vectorization is not using threads! Vectorization means using the SIMD programming model to check consecutive bytes for zero simultaneously. en.wikipedia.org/wiki/SIMD It helps that vector alignment always make them fit in a single page. This implementation typically overflows the buffer but that is not caught by the MMU. We find these benign buffer overflows in the analyzer I work on. See also tsunanet.net/~tsuna/strlen.c.html for an impressive C implementation without special vector instructions.
–
Pascal CuoqNov 21 '09 at 9:32

@Jonathan: There is for example the "pcmpeqb" instructions, which compares 16 bytes at once. Additionally SSE 4.2 contains specific vector extensions for strings, like "pcmpistri". This has nothing to to with threading.
–
hirschhornsalzNov 21 '09 at 10:42