theLizard wrote:Stepping through the code all seems to be fine BUT Quota Peak Non Paged Pool Usage and Quota Non Paged Pool Usage increases rapidly, if there is not connection to my router, Any thoughts?

I don't know if those particular values are indicative of a real memory leak. Things like the Working Set would, though. Try looking at that instead.

theLizard wrote:Is there any check I can do to determine if there is an actual connection.

The only reliable method is to connect and handle an error if it occurs.

Unless the OS knows for sure that a connection has been closed, anything else is guessing based on incomplete information. A dead socket closed incorrectly might not be invalidated by the OS for awhile, so any reads (including TIdIOHandler.Connected()) will simply act like 0 bytes are currently available, and any sends will be cached in the socket's internal buffer until it fills up.

While your code is technically OK, I would suggest not calling WriteDirect() directly, and not calling Readable() at all. Use the ReadTimeout property instead (500ms is a pretty small timeout, though).

rlebeau wrote:I don't know if those particular values are indicative of a real memory leak. Things like the Working Set would, though. Try looking at that instead.

There is a relationship between Quota Non Page Pool Usage and my issues.

Working Set memory is being managed by periodically by calling EmptyWorkingSet(GetCurrentProcess());

rlebeau wrote:The only reliable method is to connect and handle an error if it occurs.

I certainly thought the same, however, errors are not always triggered as my tests have concluded.

rlebeau wrote:Unless the OS knows for sure that a connection has been closed, anything else is guessing based on incomplete information. A dead socket closed incorrectly might not be invalidated by the OS for awhile, so any reads (including TIdIOHandler.Connected())

Agreed, but, when there is no physical connection to the endpoint, the OS knows this but TIdTcpClient does not in all cases seem to know and proceeds as if it has made a connection to an endpoint that does not exist.

rlebeau wrote:I would suggest not calling WriteDirect()

Unfortunately, (not that I mean it is unfortunate) but using WriteLn or other similar cannot be used, because the number of bytes sent are 2 more than what I want sent (cr/lf) in WriteLn for instance.

rlebeau wrote:and not calling Readable() at all

Agreed, this should not be necessary and was not, only put in to see what was going on, it to says buffers are readable where in fact there is NO physical connection to an endpoint because there is no connection to the outside world from the system the tests are being carried out on.

I can say for certain that the issues I am having is network related and TIdTcpClient appears to be the culprit in this instance.

theLizard wrote:Working Set memory is being managed by periodically by calling EmptyWorkingSet(GetCurrentProcess());

That is absolutely the wrong thing to be doing. It means you are mismanaging memory and just putting a bandaid on it instead of fixing the root problem.

theLizard wrote:Agreed, but, when there is no physical connection to the endpoint, the OS knows this

Yes, but not necessarily in a timely manner. Remember, TCP is actually designed to survive network outages. In the absence of signals to indicate the end of the connection (like FIN packets), it can take time for the OS to time out and invalidate an open socket. Yanking out the cable is not always enough. In fact, once upon a time, Windows did not react to yanked cables. You could yank a cable and put it back on, and Windows happily kept the connection alive. Modern Windows are better about reacting to yanked cables, but it is still not always guaranteed.

theLizard wrote:but TIdTcpClient does not in all cases seem to know and proceeds as if it has made a connection to an endpoint that does not exist.

If Indy is not the one closing the connection, it does not know the connection is gone until socket API calls report it. Either when FIN is received, or the OS times out and stops accepting new operations on behalf of the connection.

theLizard wrote:Unfortunately, (not that I mean it is unfortunate) but using WriteLn or other similar cannot be used, because the number of bytes sent are 2 more than what I want sent (cr/lf) in WriteLn for instance.

You can use `Write()` instead of `WriteLn()`. Just don't use `WriteDirect()`, it is not meant to be used directly, it is an internal call.

theLizard wrote:I can say for certain that the issues I am having is network related and TIdTcpClient appears to be the culprit in this instance.

I don't deny that you might be having network related problems, but I can say for certain that it is not Indy's fault, and it certainly does not leak memory the way you have described. Something else is going on.

rlebeau wrote:That is absolutely the wrong thing to be doing. It means you are mismanaging memory and just putting a bandaid on it instead of fixing the root problem.

Apart from the fact I had unexplained memory usage that ONLY occurred with TIdTcpClient behaving in a way that I could NOT trace into (including unexplained random lock up of the app) and that I wnet through every bit of code that may have been a source of memory leak and not finding anything, I put this extream measuer in an effort to find may be going on, I absolutely agree that needing to do this (and by the way, working set memory was not the problem anyway) is not the way to go about fixing a problem.

rlebeau wrote:Yes, but not necessarily in a timely manner. Remember, TCP is actually designed to survive network outages.

Again, I will agree on this however, stepping through the code showed a problem, whether it is TIdTcpClient or something else, including the device which is being polled is still a mystery.

I can report however that I appear to have solved the problem in an unusual way which I am sure that you would not agree with so I wont go into what I did but I appear to have stability, very few errors and NO (apparent) memory leaks so far.

I can also tell you that outside of the Berlin IDE, the app is running very smoothly, NO lock ups and doing exactly as it was designed to do, thread management is working like a dream,

rlebeau wrote:f Indy is not the one closing the connection,

Problem here is that I could not identify who or what was causing the problem, part of the Page Pool Usage was found within windows itself.

At the end of the day, the problem "seems" to be solved with the work around (fingers crossed) and I can get back to moving on with other things.

rlebeau wrote:If Indy is not the one closing the connection, it does not know the connection is gone until socket API calls report it. Either when FIN is received, or the OS times out and stops accepting new operations on behalf of the connection.

This slipped by me so will address this now.

Points to remember 1 - Network was physically fully disconnected before the app was started. 2 - Pinging the ip address of the device resulted in "Request Timed out" or "Destination host unreachable" 3 - 8 out of 10 times TidTcpClient threw a EIdConnectTimeout the other 2 times it assumed a connection was made and continued on as if there were no problems connecting.

Question is, is this normal behavior for TIdTcpClient, I am guessing not.

If the network was disconnected in the manner described, how can anything disconnect TIdTcpClient when there was no way it could have been connected in the first place.

I may be missing something and I agree that TIdTcpClient can not know if the connection is gone until the the OS says otherwise but should TIdTcpClient know from the OS that a connection had been made!!

I am just trying to get my head around this with some sort of basic understanding of how TIdTcpClient interacts with the OS to connect and disconnect it's services.

theLizard wrote: 3 - 8 out of 10 times TidTcpClient threw a EIdConnectTimeout the other 2 times it assumed a connection was made and continued on as if there were no problems connecting.

That should be physically impossible if the cable was not plugged in. The lower level socket API connect() function would never succeed, it would report an error, causing Indy's higher-level TIdTCPClient.Connect() method to raise an exception. There is no way for TIdTCPClient.Connect() to exit without error if the socket API connect() failed. See the implementation in TIdIOHandlerStack.ConnectClient().

theLizard wrote:I may be missing something and I agree that TIdTcpClient can not know if the connection is gone until the the OS says otherwise but should TIdTcpClient know from the OS that a connection had been made!!

Yes, and it does, by virtue of the fact that the socket API connect() function either succeeds or fails.

rlebeau wrote:That should be physically impossible if the cable was not plugged in.

Exactly, yet, what I have described is 100% correct, what I cannot determine is what maybe the cause of this.

The tests were carried out numerous times with the same result, I have looked at (almost) everything relating to TIdTcpClient, it's IOHandler and possible alternatives, I have NO doubt that my code is correct in the way it assigns port and host properties, checks for errors and ensuring that NO UI is accessed in the reading thread outside of Synchronize.

No, I created a test app using WinSock API iResult = WSAStartup(MAKEWORD(2,2), &wsaData);

When not connected I get invalid handles using the following example code from Microsoft , when connected I get data back from the device sometimes in full other times it gets stuck on iResult = recv(ConnectSocket, recvbuf, recvbuflen, 0); in the do .. while loop

This is something that i found in another forum.. Hope this helps.....-----------------------------------------------It looks like TIdSMTP is leaking memory is created at run time in C++.

There are no known leaks in TIdSMTP (or any other Indy component, for that matter).

Each click on the button will increase memory usage (as reportedby task manager).

You cannot use TaskManager to test for memory leaks. TM has no concept of HOW your app uses memory, only HOW MUCH memory has been allocated to the app. Keep in mind that Delphi/C++Builder's memory manager caches freed memory for later reuse, the memory is not returned to the OS. If TM reports memory usage is growing, that is not always an indication of a real leak. More likely you are just fragmenting memory instead, causing the memory manager to be unable to reuse memory it already has so it asks the OS for more memory. To properly test for leaks, you have to let the memory manager report the leaks to you, since only it knows what constitutes a real leak. Set the global System::ReportMemoryLeaksOnShutdown variable (http://docwiki.embarcadero.com/Librarie ... OnShutdown) to true, or else install the full version of FastMM (http://fastmm.sourceforge.net) into your project and enable its leak reporting capabilities.

In addition, madExcept also report a memory leak in line 645 ofIdMessageClient.pas:FMsgLineFold := TAB;

FMsgLineFold is a String member of the TIdMessageClient class.

In VCL, the only way that member can leak is if the TIdMessageClient destructor is not being called. TIdSMTP derives from TIdMessageClient, but you are destroying your TIdSMTP objects, so that could not be a leak. And every place where Indy internally creates a TIdMessageClient object is protected by a try/finally block to ensure the object is destroyed. So those cannot be a leak, either. If MadExcept is claiming that FMsgLineFold is leaking then it has to also be reporting that the owning TIdMessageClient is also leaking, and it should be showing you the call stack where that object is being created.

By chance, are you compiling this code in a FireMonkey mobile app? The code you showed WOULD "leak" the objects (at least until the Form itself is destroyed) because of ARC handling. The 'delete' operator does not destroy a TObject-derived object under ARC, it merely decrements the object's reference count. You are assigning an Owner to the TIdSMTP objects, and the Owner would have active references to the TIdSMTP objects, thus your 'delete' would not have an effect. To remove the active references, you would have to either: