I use IOCP for a socket server. When a new connection is created, the
connection's session is created in memory dynamically and its address
is passed to CreateIoCompletionPort as the completion key when the
connection's socket is associated with a completion port. My question
is - when it is 100% safe to remove the connection's session (address
of which GetQueuedCompletionStatus returns as the completion key when
an operation on the socket completes) from memory? In other words,
what combination of both the return value and the output parameters of
GetQueuedCompletionStatus is a 100% indication of that
GetQueuedCompletionStatus will never return this same completion key
again?

On Jun 21, 10:05*pm, Dennis Mikhailitsky <mikhailit...@gmail.com>
wrote:[color=blue]
> Hello,
>
> I use IOCP for a socket server. When a new connection is created, the
> connection's session is created in memory dynamically and its address
> is passed to CreateIoCompletionPort as the completion key when the
> connection's socket is associated with a completion port. My question
> is - when it is 100% safe to remove the connection's session (address
> of which GetQueuedCompletionStatus returns as the completion key when
> an operation on the socket completes) from memory? In other words,
> what combination of both the return value and the output parameters of
> GetQueuedCompletionStatus is a 100% indication of that
> GetQueuedCompletionStatus will never return this same completion key
> again?
>
> Thanks in advance[/color]

Usually, these are two ways to safe release resource associated with
IOCP.

The first is using refcounting. When issuing a WSARecv/WSASend,
increase
refcount. When GetQueuedCompletionStatus return, decrease refcount.
It is safe to release resouece when refcount is zero.

The second is using a hashtable. For example, pass SocketHandle to
CreateIoCompletionPort as the completion key, and push the pair --
(SocketHandle, Session) -- into hashtable. When
GetQueuedCompletionStatus
return, get the session from the hashtable by the SocketHandle.
When it need to release the session, remove the pair from the
hashtable first.
After that, when GetQueuedCompletionStatus return, it cannot get the
session
from the hashtable any more.

On Jun 21, 7:05*am, Dennis Mikhailitsky <mikhailit...@gmail.com>
wrote:
[color=blue]
> I use IOCP for a socket server. When a new connection is created, the
> connection's session is created in memory dynamically and its address
> is passed to CreateIoCompletionPort as the completion key when the
> connection's socket is associated with a completion port. My question
> is - when it is 100% safe to remove the connection's session (address
> of which GetQueuedCompletionStatus returns as the completion key when
> an operation on the socket completes) from memory? In other words,
> what combination of both the return value and the output parameters of
> GetQueuedCompletionStatus is a 100% indication of that
> GetQueuedCompletionStatus will never return this same completion key
> again?[/color]

Why could the following happen: a socket is closed in abortive manner
by calling closesocket, in about 15 mins the associated overlapped
sturcture is removed from memory, yet at some point after that
GetQueuedCompletionStatus returns non-zero as the return value, size
== 0, and pointer to the overlapped structure that has already been
removed from memory? This happens very rarely, therefore I cannot
explore the nature of the issue as I cannot reproduce it consistently,
and this drives me crazy.

I can restructure my original question a little bit - is there a way
to remove association between a socket and a completion port
completely and forever? In other words, is there a way to do an action
opposite to what CreateIoCompletionPort does - to make
GetQueuedCompletionStatus immediately stop returning a given
completion key once and forever, no matter what?

06-26-2008, 09:15 PM

unix

Re: GetQueuedCompletionStatus: when it's safe to delete the completion key

no - the only way to remove an IO completion port association is to close
the handle.

if this hapens in your code, it means that your reference counts are wrong
or you are frering the OVERLAPPED somwhere that is not the IO completion
handler

"Dennis Mikhailitsky" <mikhailitsky@gmail.com> wrote in message
news:4a877f5a-5212-4470-98da-818c32740015@m45g2000hsb.googlegroups.com...[color=blue]
>I can restructure my original question a little bit - is there a way
> to remove association between a socket and a completion port
> completely and forever? In other words, is there a way to do an action
> opposite to what CreateIoCompletionPort does - to make
> GetQueuedCompletionStatus immediately stop returning a given
> completion key once and forever, no matter what?[/color]

On Jun 26, 8:17*am, Dennis Mikhailitsky <mikhailit...@gmail.com>
wrote:
[color=blue][color=green]
> > Every operation will, unless canceled, complete exactly once. If that
> > doesn't answer your question, then I don't understand your question.[/color][/color]
[color=blue]
> Why could the following happen: a socket is closed in abortive manner
> by calling closesocket, in about 15 mins the associated overlapped
> sturcture is removed from memory, yet at some point after that
> GetQueuedCompletionStatus returns non-zero as the return value, size
> == 0, and pointer to the overlapped structure that has already been
> removed from memory?[/color]

For the reason I stated above -- every operation will, unless
canceled, complete exactly once. Some operation had been started and
hadn't complete yet, so it must complete later. There is no guaranteed
time frame for operation completion.
[color=blue]
> This happens very rarely, therefore I cannot
> explore the nature of the issue as I cannot reproduce it consistently,
> and this drives me crazy.[/color]

I cannot imagine why you would want to call 'closesocket' while an
operation is in progress on the socket. I don't know for sure whether
there's any safe way to do so -- it's not immediately obvious to me
that it's impossible, but it seems really difficult.

Consider:

1) A thread calls 'GetQueuedCompletionStatus' and gets a completion
indication for this socket, but it blocks on a page fault.

2) Some other thread closes the socket.

3) Lots of other stuff happens.

4) The thread in step 1 finally pages in the page it was waiting for
and continues, processing the completion indication it already got for
the socket you closed in step 2.

I would recommend not calling 'closesocket' until you are completely
finished using the socket because it seems like an awful lot of extra
work to do it safely. It will cancel all pending operations, they will
complete with an error. Microsoft says:

"Any pending overlapped send and receive operations ( WSASend/
WSASendTo/ WSARecv/ WSARecvFrom with an overlapped socket) issued by
any thread in this process are also canceled. Any event, completion
routine, or completion port action specified for these overlapped
operations is performed. The pending overlapped operations fail with
the error status WSA_OPERATION_ABORTED."

"An application should not assume that any outstanding I/O operations
on a socket will all be guaranteed to completed when closesocket
returns. The closesocket function will initiate cancellation on the
outstanding I/O operations, but that does not mean that an application
will receive I/O completion for these I/O operations by the time the
closesocket function returns. Thus, an application should not cleanup
any resources (WSAOVERLAPPED structures, for example) referenced by
the outstanding I/O requests until the I/O requests are indeed
completed."

On Jun 26, 9:26*am, Dennis Mikhailitsky <mikhailit...@gmail.com>
wrote:
[color=blue]
> I can restructure my original question a little bit - is there a way
> to remove association between a socket and a completion port
> completely and forever? In other words, is there a way to do an action
> opposite to what CreateIoCompletionPort does - to make
> GetQueuedCompletionStatus immediately stop returning a given
> completion key once and forever, no matter what?[/color]

I don't see how that will help you, since a thread may already be in
the process of returning from GetQueuedCompletionStatus.

> Why could the following happen: a socket is closed in abortive manner[color=blue]
> by calling closesocket, in about 15 mins the associated overlapped
> sturcture is removed from memory, yet at some point after that
> GetQueuedCompletionStatus returns non-zero as the return value, size
> == 0, and pointer to the overlapped structure that has already been
> removed from memory? This happens very rarely, therefore I cannot
> explore the nature of the issue as I cannot reproduce it consistently,
> and this drives me crazy.[/color]

If you are using reference counts on the overlapped structure then you
have a bug. Each operation completes once. Closing the socket doesnt
change that. The operations will still complete once. Increment the
reference count when you issue the request, decrement it when the
operation completes.

In your example above, why is the overlapped structure removed from
memory 15 mins after the socket is closed, what causes that removal?

Looks like the problem is gone. I wasnt sure exactly which sends and
receives get queued to the completion port and which dont, therefore I
didnt not use reference counters at all and was relying on the GQCS's
return code and combination of the out parameters when making a
decision to destroy the overlapped structure, which obviously was a
bad idea. Now I have implemented reference counters and the problem
seems to be gone. Thanks a lot to all of you for helping to solve this
problem.

Re: GetQueuedCompletionStatus: when it's safe to delete the completionkey

Hi..
As you say in the end of this thread your problem is solved.
OK its solved but how you can managed for time out of the operation.

I mean to say if we send data to the client and client not responding than
GetQueuedCompletionStatus will not come out from waiting state untill WsaSend
Completed.In this case how to remove the resources of the client at serverside.
Means how can we confirm that client is dad so remove it from CompletionPort.
And how to remove that client resource.

Actualy i have done some exersice i had make a thread that checking for time out
with one variable.This variable set with gettickcount when any event occur on socket
GetQueuedCompletionStatus come out from the loop with client completion key which is
a class pointer and with that pointer i reset the variable with new tick count.

In Timeout thread it will check for that variable goes above 100 sec if yes than it will close that socket and remove that class pointer but in this case some event comes after i have deleted the pointer of the class so its crash.

So please tell me how can i detect timeout for the socket and how to remove client resources from server.