Merge mainstream 'master' branch into refactoring During merge I had to revert: Nikita's patch for php_splice() (it probably needs to be applyed again) Bob Weinand's patches related to constant expression handling (we need to review them carefully) I also reverted all our attempts to support sapi/phpdbg (we didn't test it anyway)

Constify streams API and a few other calls down the rabbit hole. (`char *` to `const char *` for parameters and few return values) In a few places int len moved to size_t len./php-src/main/streams/streams.c

stream_get_line() could contain the delimiter string if that string had more than one character. The bug manifested itself when a read on the stream ended with part of the delimiter string and the read after would start with the rest of the delimiter string; provided that the data of first read did not complete the max length result of the call to stream_get_line() with the partial delimiter used in that max length return. In that case, the delimiter will still appear in the result, divided in two subsequent return values. That is not a bug.

- Further fix for bug #60455 (stream_get_line misbehaves if EOF is not detected together with the last read). - Fixed bug #60817 (stream_get_line() reads from stream even when there is already sufficient data buffered). stream_get_line() now behaves more like fgets(), as is documented./php-src/main/streams/streams.c

- Fixed bug #61371 (resource leak). This bug had two parts, a long standing leak already fixed in trunk/5.3 and now merged onto 5.4 and a leak introduced in fixing bug #61115. This better fix for #61115 fixes the leak (the inhibition for deleting the context was too broad) and so prevents segfaults in new circumstances (where the inhibition was not broad enough)./php-src/main/streams/streams.c

- Fixed bug #61371 (resource leak). This bug had two parts, a long standing leak already fixed in trunk/5.3 and now merged onto 5.4 and a leak introduced in fixing bug #61115. This better fix for #61115 fixes the leak (the inhibition for deleting the context was too broad) and so prevents segfaults in new circumstances (where the inhibition was not broad enough)./php-src/main/streams/streams.c

- Fixed bug #61371 (resource leak). This bug had two parts, a long standing leak already fixed in trunk/5.3 and now merged onto 5.4 and a leak introduced in fixing bug #61115. This better fix for #61115 fixes the leak (the inhibition for deleting the context was too broad) and so prevents segfaults in new circumstances (where the inhibition was not broad enough)./php-src/main/streams/streams.c

- Fixed bug #61253: Wrappers opened with errors concurrency problem #NOTE: There is a very small possibility that this will further break #extensions that access wrapper->{err_stack, err_count}. On PECL SVN, rar is the #only one and it may leak memory after this. I say "further break" because #extensions that do that are already broken (will segfault) under ZTS, which is #why this patch is necessary. #There was what I deem as tacit acceptance from 5.3/5.4 RMs on this./php-src/main/streams/streams.c

- Fixed bug #61253: Wrappers opened with errors concurrency problem #NOTE: There is a very small possibility that this will further break #extensions that access wrapper->{err_stack, err_count}. On PECL SVN, rar is the #only one and it may leak memory after this. I say "further break" because #extensions that do that are already broken (will segfault) under ZTS, which is #why this patch is necessary. #There was what I deem as tacit acceptance from 5.3/5.4 RMs on this./php-src/main/streams/streams.c

- Fixed bug #61253: Wrappers opened with errors concurrency problem #NOTE: There is a very small possibility that this will further break #extensions that access wrapper->{err_stack, err_count}. On PECL SVN, rar is the #only one and it may leak memory after this. I say "further break" because #extensions that do that are already broken (will segfault) under ZTS, which is #why this patch is necessary. #There was what I deem as tacit acceptance from 5.3/5.4 RMs on this./php-src/main/streams/streams.c

- Fixed bug #61115 (stream related segfault on fatal error in php_stream_context_link). #run-tests.php is not currently detecting the segfault in the test #Missing 5.4 merge/php-src/main/streams/streams.c

- Fixed bug #61115 (stream related segfault on fatal error in php_stream_context_link). #run-tests.php is not currently detecting the segfault in the test #Missing 5.4 merge/php-src/main/streams/streams.c

- Further fix for bug #60455 (stream_get_line misbehaves if EOF is not detected together with the last read). - Fixed bug #60817 (stream_get_line() reads from stream even when there is already sufficient data buffered). stream_get_line() now behaves more like fgets(), as is documented. #withheld commit to 5.4/php-src/main/streams/streams.c

- Further fix for bug #60455 (stream_get_line misbehaves if EOF is not detected together with the last read). - Fixed bug #60817 (stream_get_line() reads from stream even when there is already sufficient data buffered). stream_get_line() now behaves more like fgets(), as is documented. #withheld commit to 5.4/php-src/main/streams/streams.c

- Merged to 5.3 change in trunk in _php_stream_free. This prevents the segfault in the testcase for bug #53624. The testcase still has to be fixed though, because it only works as intended on Windows./php-src/main/streams/streams.c

- Make fclose() actually close stream, even when the resource refcount is > 1. This reverts the fix for bug #24557. - Make php_stream_free delete the stream from the resources list, not merely decrease its refcount, as a single call to zend_list_delete does. #Not worth the risk merging to 5.3. While change #2 may prevent some segfaults, #a quick and dirty survey to the codebase only showed calls to php_stream_close #or php_stream_free on streams allocated in the same function, which would have #refcount == 1. May be reconsidered./php-src/main/streams/streams.c

- Fixed several comparisons that always result in true of false due to signedness of one of the operands, either by removing dead code or fixing it. - Thrown some comments around in php_stream_get_record. - See http://www.mail-archive.com/internals@lists.php.net/msg49525.html/php-src/main/streams/streams.c

- Fixed several comparisons that always result in true of false due to signedness of one of the operands, either by removing dead code or fixing it. - Thrown some comments around in php_stream_get_record. - See http://www.mail-archive.com/internals@lists.php.net/msg49525.html/php-src/main/streams/streams.c

- Fixed forward stream seeking emulation in streams that don't support seeking in situations where the read operation gives back less data than requested and when there was data in the buffer before the emulation started. Also made more consistent its behavior -- should return failure every time less data than was requested was skipped. - Small performance improvement by correcting off-by-one error that generate an invalid call to the seek handler or read handler. in _php_stream_seek./php-src/main/streams/streams.c

- Fixed forward stream seeking emulation in streams that don't support seeking in situations where the read operation gives back less data than requested and when there was data in the buffer before the emulation started. Also made more consistent its behavior -- should return failure every time less data than was requested was skipped. - Small performance improvement by correcting off-by-one error that generate an invalid call to the seek handler or read handler. in _php_stream_seek./php-src/main/streams/streams.c

Added ability to create local or remote (URL) user streams Local user streams must not be able to open(), URLs if allow_url_include is off Implemented new function stream_is_local() [ - stream_wrapper_register() extended with additional optional argument "flags" of type long. This time only one flag is implemented - STREAM_IS_URL, that means that userstream wrapper is remote (URL). By default stream is local. - stream_is_local() is a new function that accepts stream and tell if this stream is local or remote (URL) ]/php-src/main/streams/streams.c

This could have been done in stream_wrapper_register() without introducing the slight performance hit on wrapper registration since anyone registering a wrapper in an extension should know better.

The important thing is that since locate_wrapper makes the assumption that all schemes will be /^[a-z0-9+.-]+$/i Anything which registers them should make the same assumption as well./php-src/main/streams/streams.c

Register filters as resources when instantiated by stream_filter_(ap|pre)pend().

Export php_stream_filter_flush() internal function to wind buffered data out of a particular filter until consumed by a later filter or sent to stream->readbuffer or stream->ops->write()/php-src/main/streams/streams.c

Userdefined wrappers were being registered into a global wrapper hash which can cross threads. Termination of once instance then has the potential to leave an active stream in another instance with no wrapper leading to segfault./php-src/main/streams/streams.c

Fix a bug in the persistent socket liveness checks and feof(); they were using the default socket timeout of 60 seconds before returning the socket to the calling script. The reason they were using that value is that the same code is used for feof(), so the fix is allowing the caller to indicate the timeout value for liveness checks.

A possible remaining issue now is that 0 second timeout[1] for pfsockopen is possibly too short; it's impossible to specify a sane value for all possible uses, so maybe we need a stream context or an .ini option to control this, or maybe use the timeout value that was passed to pfsockopen().

# [1] by timeout, I mean the time that PHP will wait for data on a # persistent socket before deciding if a new connection should be made; # NOT the timeout while waiting for a new connection to be established./php-src/main/streams/streams.c

Plug leak (context options not freed) Make contexts auto-registered, ensures userland contexts and C API contexts are both dealt with on request shutdown. Also brings contexts in keeping with streams which are already auto-registered./php-src/main/streams/streams.c

Main Changes: - Implement a socket transport layer for use by all code that needs to open some kind of "special" socket for network or IPC. - Extensions can register (and override) transports. - Implement ftruncate() on streams via the ioctl-alike option interface. - Implement mmap() on streams via the ioctl-alike option interface. - Implement generic crypto API via the ioctl-alike option interface. (currently only supports OpenSSL, but could support other SSL toolkits, and other crypto transport protocols).

Impact: - tcp sockets can be overloaded by the openssl capable sockets at runtime, removing the link-time requirement for ssl:// and https:// sockets and streams. - checking stream types using PHP_STREAM_IS_SOCKET is deprecated, since there are now a range of possible socket-type streams.

Working towards: - socket servers using the new transport layer - mmap support under win32 - Cleaner code.