For a "graceful" close you're supposed to ensure there is no data
pending. To do that you:
shutdown(SD_SEND); // send only, not recv
<enter a loop reading all data remaining on the socket>
close();
The loop should read until recv returns 0. If recv returns -1 and the
socket is blocking it should error/exit. If recv returns -1 and the
socket is non-blocking it should check for [WSA]EWOULDBLOCK (and
select/sleep + loop) or error/exit.
The reason to do this is to flush all the data from the socket buffers on
the remote and local ends, otherwise a close can cause remote buffered
data to cause a "connection broken" error on the remote end, and/or (I am
guessing a little here) may cause the socket to close while negotiating a
graceful close, and/or remain in a TIME_WAIT state due to buffered data or
data "in flight".
.. are you setting any close options/timeouts i.e. LINGER?
--
Using Opera's revolutionary email client: http://www.opera.com/mail/

For a "graceful" close you're supposed to ensure there is no data
pending. To do that you:
shutdown(SD_SEND); // send only, not recv
<enter a loop reading all data remaining on the socket>
close();
The loop should read until recv returns 0. If recv returns -1 and the
socket is blocking it should error/exit. If recv returns -1 and the
socket is non-blocking it should check for [WSA]EWOULDBLOCK (and
select/sleep + loop) or error/exit.
The reason to do this is to flush all the data from the socket buffers
on the remote and local ends, otherwise a close can cause remote
buffered data to cause a "connection broken" error on the remote end,
and/or (I am guessing a little here) may cause the socket to close while
negotiating a graceful close, and/or remain in a TIME_WAIT state due to
buffered data or data "in flight".
... are you setting any close options/timeouts i.e. LINGER?

Thanks.
recv returns -1 for many requests. The errors are only WSAECONNABORTED
and WSAECONNRESET as described here:
http://msdn.microsoft.com/en-us/library/ms740668.aspx
I'm doing socket.shutdown(SocketShutdown.SEND) now after sending all my
data and reading until I receive 0 or -1. (doesn't really matter as
sending the FastCGI EndRequest makes the server shut it down as it
doesn't handle multiplexing)
I have tried with linger too, but it doesn't help:
socket.setOption(SocketOptionLevel.SOCKET, SocketOption.LINGER,
std.socket.linger(1, 30));
Could this be caused by some bad settings on the webserver?
PS: Seems my computer can handle about 16000 TIME_WAIT before it starts
"hanging".

To help me understand (I know nothing about fastcgi or nginx) can you
clarify...
1. Your D code is the client side, connecting to the web server and
sending GET/POST style requests?
2. You get these ABORTED and RESET errors on the client side?
3. As #3 even after doing as I described, shutdown(SEND), recv, then close?
If yes to all the above, then it sounds like the web server/fastcgi is
closing the socket without reading all the data you're sending, which
probably means you're sending something it's not expecting. I would
start by verifying exactly what data you're sending, and that it's all
expected by the remote end.

Yes. I've coded the client as follows:
1) start listening socket
2) wait for incoming connections or incoming data
3) receive(). If a socket returns 0 or -1, close it and process next
with data
4) read fastcgi request from server
5) write fastcgi response
6) write fastcgi EndRequest (the server should now end the request)
7) if the application should close the request, send shutdown(send)
8) accept incoming connection
9) back to 2)
FastCGI connections works in one of two ways: the server is responsible
for closing the connections (supports mulitplexing) or the application
should close the connection after a request has been sent.
For the latter I send SocketShutdown.SEND after writing EndRequest in
step 6), but it doesn't really matter as nginx doesn't support
multiplexing. It closes the connection after each request anyway. I see
the same result no matter what option I use.
I'm running the exact same request and writing the exact same response
for all queries, so there shouldn't be any unknown fields.
I also only get an error on <1/5 of the requests, and even when the
error occurs, the response has been written completely to the browser.

I'm doing socket.shutdown(SocketShutdown.SEND) now after sending all
my data and reading until I receive 0 or -1. (doesn't really matter as
sending the FastCGI EndRequest makes the server shut it down as it
doesn't handle multiplexing)

So, the socket closure is initiated by fastcgi/the web server. This
supports the theory that it's not reading some of your data, because
it's not expecting it, and this is likely the cause of the ABORT/RESET
errors you're seeing.

I have tried with linger too, but it doesn't help:
socket.setOption(SocketOptionLevel.SOCKET, SocketOption.LINGER,
std.socket.linger(1, 30));

The default LINGER options should be fine, as-is. But, double check the
D socket code just in case it is setting different LINGER options by
default (I haven't used it, or looked myself, sorry).

Linger is default off, but it doesn't help to turn it on. RCV/SNDTIMO is
also set to 0.

Could this be caused by some bad settings on the webserver?

It is possible, but I would double check your requests first. There may
be a setting, or settings for aborting connections which take too long,
or fail to send certain data, or connect from the wrong IP, or... If
your requests are otherwise working, then I suspect you're sending some
'extra' data which is not being read.

The requests are handled in ~1msec, so there shouldn't be any timeouts.
The default timeout on nginx for fastcgi is 60 seconds too.
I can easily process ~200 requests per second (and nginx and my server
doesn't break a sweat, it's my curl spammers that's using all the cpu)

PS: Seems my computer can handle about 16000 TIME_WAIT before it
starts "hanging".

You'll be running out of operating system handles or similar at that
point :p

Yup. I'll probably never have that problem in a production environment
though :)

Yes. I've coded the client as follows:
1) start listening socket
2) wait for incoming connections or incoming data
3) receive(). If a socket returns 0 or -1, close it and process next
with data
4) read fastcgi request from server
5) write fastcgi response
6) write fastcgi EndRequest (the server should now end the request)
7) if the application should close the request, send shutdown(send)
8) accept incoming connection
9) back to 2)
FastCGI connections works in one of two ways: the server is
responsible for closing the connections (supports mulitplexing) or the
application should close the connection after a request has been sent.
For the latter I send SocketShutdown.SEND after writing EndRequest in
step 6), but it doesn't really matter as nginx doesn't support
multiplexing. It closes the connection after each request anyway. I
see the same result no matter what option I use.

Ok, so your "client" (that you have coded) is also the "application" you
refer to in the bit about FastCGI above? Or are there 2 components here,
and are both written in D?

It's just one component to handle FastCGI requests. I didn't want to
rely on the external libfcgi.

Does the fastcgi "EndRequest" close the socket/connection? If so, doing
a socket.Shutdown /after/ this is not going to work as the socket has
already been closed (which implicitly does a shutdown(BOTH)). In that
case, try doing the shutdown /before/ the EndRequest, and make sure you
also read any/all data remaining on the socket before doing the
EndRequest/close.

EndRequest doesn't really close the socket, it's just a message to the
server telling that the full response is written (request handled). If
the server (nginx) is responsible, it can reuse the connection to give
other requests. If the server says that the application is responsible,
shutdown(send) is called. This is part of the specification.

The key question seems to be, at which point does nginx close the
connection? and therefore, is there any unread data on the socket (at
either end) when it does. If, for example, it flushes the response to
the other end, but does not wait for it to be read, and closes the
socket, you will get CONNRESET/ABORTED errors on the other end.

I'm running the exact same request and writing the exact same response
for all queries, so there shouldn't be any unknown fields.

I didn't mean unknown "field" I mean extra data of any kind, but I
suspect you're using an API to form the requests etc so this is probably
not the case.

I also only get an error on <1/5 of the requests, and even when the
error occurs, the response has been written completely to the browser.

Ahh, ok, I believe the problem is simply the timing of the
'close/EndRequest'. Sometimes it happens /before/ the data has been
completely read (1/5), other times after (4/5).
R

It seems nginx is to blame here, and not me. I tried Lighttp and it
works. It gives several EWOULDBLOCK, but I can just handle these again
with no problem. I should have tried this sooner... I've used a lot of
time trying to track down these problems :|
Thanks for all your help - I'll update this thread if I find a solution
to the nginx issue.

It seems nginx is to blame here, and not me. I tried Lighttp and it
works. It gives several EWOULDBLOCK, but I can just handle these again
with no problem. I should have tried this sooner... I've used a lot of
time trying to track down these problems :|
Thanks for all your help - I'll update this thread if I find a solution
to the nginx issue.

Well, that was quick... Seems I was running a development version of
nginx. I downloaded the stable version, and things work as expected - I
can finally try to get some actual coding done :)

For a "graceful" close you're supposed to ensure there is no data
pending. To do that you:
shutdown(SD_SEND); // send only, not recv
<enter a loop reading all data remaining on the socket>
close();
The loop should read until recv returns 0. If recv returns -1 and the
socket is blocking it should error/exit. If recv returns -1 and the
socket is non-blocking it should check for [WSA]EWOULDBLOCK (and
select/sleep + loop) or error/exit.
The reason to do this is to flush all the data from the socket buffers
on the remote and local ends, otherwise a close can cause remote
buffered data to cause a "connection broken" error on the remote end,
and/or (I am guessing a little here) may cause the socket to close while
negotiating a graceful close, and/or remain in a TIME_WAIT state due to
buffered data or data "in flight".
... are you setting any close options/timeouts i.e. LINGER?

To help me understand (I know nothing about fastcgi or nginx) can you
clarify...
1. Your D code is the client side, connecting to the web server and
sending GET/POST style requests?
2. You get these ABORTED and RESET errors on the client side?
3. As #3 even after doing as I described, shutdown(SEND), recv, then close?
If yes to all the above, then it sounds like the web server/fastcgi is
closing the socket without reading all the data you're sending, which
probably means you're sending something it's not expecting. I would start
by verifying exactly what data you're sending, and that it's all expected
by the remote end.

I'm doing socket.shutdown(SocketShutdown.SEND) now after sending all my
data and reading until I receive 0 or -1. (doesn't really matter as
sending the FastCGI EndRequest makes the server shut it down as it
doesn't handle multiplexing)

So, the socket closure is initiated by fastcgi/the web server. This
supports the theory that it's not reading some of your data, because it's
not expecting it, and this is likely the cause of the ABORT/RESET errors
you're seeing.

I have tried with linger too, but it doesn't help:
socket.setOption(SocketOptionLevel.SOCKET, SocketOption.LINGER,
std.socket.linger(1, 30));

The default LINGER options should be fine, as-is. But, double check the D
socket code just in case it is setting different LINGER options by default
(I haven't used it, or looked myself, sorry).

Could this be caused by some bad settings on the webserver?

It is possible, but I would double check your requests first. There may
be a setting, or settings for aborting connections which take too long, or
fail to send certain data, or connect from the wrong IP, or... If your
requests are otherwise working, then I suspect you're sending some 'extra'
data which is not being read.

PS: Seems my computer can handle about 16000 TIME_WAIT before it starts
"hanging".

You'll be running out of operating system handles or similar at that point
:p
--
Using Opera's revolutionary email client: http://www.opera.com/mail/

Yes. I've coded the client as follows:
1) start listening socket
2) wait for incoming connections or incoming data
3) receive(). If a socket returns 0 or -1, close it and process next
with data
4) read fastcgi request from server
5) write fastcgi response
6) write fastcgi EndRequest (the server should now end the request)
7) if the application should close the request, send shutdown(send)
8) accept incoming connection
9) back to 2)
FastCGI connections works in one of two ways: the server is responsible
for closing the connections (supports mulitplexing) or the application
should close the connection after a request has been sent.
For the latter I send SocketShutdown.SEND after writing EndRequest in
step 6), but it doesn't really matter as nginx doesn't support
multiplexing. It closes the connection after each request anyway. I see
the same result no matter what option I use.

Ok, so your "client" (that you have coded) is also the "application" you
refer to in the bit about FastCGI above? Or are there 2 components here,
and are both written in D?
Does the fastcgi "EndRequest" close the socket/connection? If so, doing a
socket.Shutdown /after/ this is not going to work as the socket has
already been closed (which implicitly does a shutdown(BOTH)). In that
case, try doing the shutdown /before/ the EndRequest, and make sure you
also read any/all data remaining on the socket before doing the
EndRequest/close.
The key question seems to be, at which point does nginx close the
connection? and therefore, is there any unread data on the socket (at
either end) when it does. If, for example, it flushes the response to the
other end, but does not wait for it to be read, and closes the socket, you
will get CONNRESET/ABORTED errors on the other end.

I'm running the exact same request and writing the exact same response
for all queries, so there shouldn't be any unknown fields.

I didn't mean unknown "field" I mean extra data of any kind, but I suspect
you're using an API to form the requests etc so this is probably not the
case.

I also only get an error on <1/5 of the requests, and even when the
error occurs, the response has been written completely to the browser.

Ahh, ok, I believe the problem is simply the timing of the
'close/EndRequest'. Sometimes it happens /before/ the data has been
completely read (1/5), other times after (4/5).
R

It seems nginx is to blame here, and not me. I tried Lighttp and it
works. It gives several EWOULDBLOCK, but I can just handle these again
with no problem. I should have tried this sooner... I've used a lot of
time trying to track down these problems :|

EWOULDBLOCK is to be expected, it simply means you've tried to read when
there is no data available, before the close/shutdown(SEND) from the other
end. :)
--
Using Opera's revolutionary email client: http://www.opera.com/mail/