If this is your first visit, be sure to
check out the FAQ by clicking the
link above. You may have to register or Login
before you can post: click the register link above to proceed. To start viewing messages,
select the forum that you want to visit from the selection below.

Re: Multiple thread question

Check out the QueueUserWorkItem api which will manage the threads for you. Try it, see if the performance is the same as the what you are seeing with manually managed threads, and if it is, never look back.

Re: Multiple thread question

Originally Posted by Arjay

Check out the QueueUserWorkItem api which will manage the threads for you. Try it, see if the performance is the same as the what you are seeing with manually managed threads, and if it is, never look back.

Thanks.
I'll experiment and report results.

All advice is offered in good faith only. You are ultimately responsible for effects of your programs and the integrity of the machines they run on.
C, C++ Compiler: Microsoft VS2015

Re: Multiple thread question

Originally Posted by Arjay

Check out the QueueUserWorkItem api which will manage the threads for you. Try it, see if the performance is the same as the what you are seeing with manually managed threads, and if it is, never look back.

Yes the performance is the same! The only slight complication is now knowing when all the files have been processed (XP).

PS The maximum thread count according to Task Manager is 6!

Last edited by 2kaud; January 16th, 2014 at 07:01 PM.

All advice is offered in good faith only. You are ultimately responsible for effects of your programs and the integrity of the machines they run on.
C, C++ Compiler: Microsoft VS2015

Re: Multiple thread question

With regard to determining when all the files have been processed..

If the number of files doesn't change during the processing, simply count the files before processing and decrement the count as each file is processed by QueueUserWorkItem. Use the interlockxxx api to safely change the count variable. When it reaches 0, you are finished.

Re: Multiple thread question

since this is over network...

I suspect the many-threads solution is benefitting from the fact that the IO requests to the server (and possibly the return route) get packaged/merged into single network packets. Sending stuff faster/more often may also largely override the inate network throttling (see "nagle's algorithm" and "delayed (network) acknowledgement. With stuff piling up on the sending side of the network stack all the delays get canceled and the nagle is forced out.

I'm not really seeing a 30fold increase in this though. Maybe 2x or 3x. Even combining this with command queueing, and multiple IO requests over raid. 30x is still a far way off.

If you have an app that is FILE I/O bound over network. Then the real down to earth issue is "why are you running thos over the network in the first place". THis is asking for either a service on the server, command remoting or RDP or some other means to run the actual processing at the server side literally avoiding the network issues entirely.

But again, if what you have now works for you "fine". You may however find out later that changes in hardware/software either on the client side or on the server side, or somewhere on the network may have drastic effects on run time. that many simultaneous threads is not a reliable model to build on.

As I said before, for an "in-house" solution it may be good enough (for now), but I wouldn't expect too much of it if you wanted this in a commercial product that needs to run on a wide variety of clients/servers/network hardware.

Re: Multiple thread question

Originally Posted by Arjay

With regard to determining when all the files have been processed..

If the number of files doesn't change during the processing, simply count the files before processing and decrement the count as each file is processed by QueueUserWorkItem. Use the interlockxxx api to safely change the count variable. When it reaches 0, you are finished.

Yes, that's what I've done. In the processing thread I use InterlockDecrement() and if the result is 0 I signal an event. In main() I use WaitForSingleObject() on that event. This works OK. The only downside is that now I've got a dependency between the thread function and main() which I didn't have before. As I said, a slight complication - but overall probably a more stable solution as it will probably give optimum throughput even if the data is stored on local disks rather than networked disks. I doubt my original experimental solution would give the same level of performance improvement on local disks.

Interesting results though, demonstrating the power of using multi-threading for i/o bound programs.

All advice is offered in good faith only. You are ultimately responsible for effects of your programs and the integrity of the machines they run on.
C, C++ Compiler: Microsoft VS2015

Re: Multiple thread question

If you have an app that is FILE I/O bound over network. Then the real down to earth issue is "why are you running thos over the network in the first place". THis is asking for either a service on the server, command remoting or RDP or some other means to run the actual processing at the server side literally avoiding the network issues entirely.

The data is stored on a RAID5 NAS device attached to the LAN. Irrespective of from where/how the program is run, the data still comes over the LAN.

I have, however, asked the network/systems people to look at the NAS device configuration and its performance to make sure we haven't got an undiagnosed problem there.

All advice is offered in good faith only. You are ultimately responsible for effects of your programs and the integrity of the machines they run on.
C, C++ Compiler: Microsoft VS2015

Re: Multiple thread question

Originally Posted by 2kaud

I suspect the answer to the vast performance increase is due to the explanation given by razzle in post #13.

I have another crazy idea. Could it be that your IO requests to that NAS were so sparse that it was getting to the stand by/sleep mode and had to wake up for the next IO? That could explain your 30-fold bump, as spinning up the drives sure takes much more time than reading a cluster or two.

Re: Multiple thread question

Originally Posted by VladimirF

I have another crazy idea. Could it be that your IO requests to that NAS were so sparse that it was getting to the stand by/sleep mode and had to wake up for the next IO? That could explain your 30-fold bump, as spinning up the drives sure takes much more time than reading a cluster or two.

?? Don't know. Not my area. Another question for the network/systems people. But I'm suspicious that with the new program running the network utilisation is only 5%.

Last edited by 2kaud; January 17th, 2014 at 11:46 AM.

All advice is offered in good faith only. You are ultimately responsible for effects of your programs and the integrity of the machines they run on.
C, C++ Compiler: Microsoft VS2015

Re: Multiple thread question

for curiosity sake, did you tried using the WT_SET_MAX_THREADPOOL_THREADS(Flags,Limit) macro to set Limit to just one thread to see if the problem was in the original code ? or simply measure the speedup for 1<=Limit<=6 ?

Re: Multiple thread question

Originally Posted by superbonzo

for curiosity sake, did you tried using the WT_SET_MAX_THREADPOOL_THREADS(Flags,Limit) macro to set Limit to just one thread to see if the problem was in the original code ? or simply measure the speedup for 1<=Limit<=6 ?

No. Good point. I'll investigate.

All advice is offered in good faith only. You are ultimately responsible for effects of your programs and the integrity of the machines they run on.
C, C++ Compiler: Microsoft VS2015

Re: Multiple thread question

Well, that was interesting. I used the macro to set the Limit to 1 thread and ....... the program completed in just over 10 minutes. Not believing this, I rebuilt the solution and tried again with the same results. Doing it again for the third time I found that the program is still using multiple threads according to Task Manager. So setting the Limit to 1 using the macro to set the Flags parameter basically has no effect.

Reading the API documention, it says
"By default, the thread pool has a maximum of 512 threads per process. To raise the queue limit, use the WT_SET_MAX_THREADPOOL_THREAD macro defined in Winnt.h."

So it looks like you can't use this to reduce the number of theads used, just increase them.

All advice is offered in good faith only. You are ultimately responsible for effects of your programs and the integrity of the machines they run on.
C, C++ Compiler: Microsoft VS2015

Re: Multiple thread question

Update to my post #27. I replaced the call to QueueUserWorkItem() with a direct call to the thread function - effectively making the program single thread again (confirmed by Task Manager). The time taken is back up to several hours again - so the issue is not a problem in the original code as expected.

All advice is offered in good faith only. You are ultimately responsible for effects of your programs and the integrity of the machines they run on.
C, C++ Compiler: Microsoft VS2015

Re: Multiple thread question

Reading the API documention, it says
"By default, the thread pool has a maximum of 512 threads per process. To raise the queue limit, use the WT_SET_MAX_THREADPOOL_THREAD macro defined in Winnt.h."

... uhm, but the very next sentence reads "Note that your application can improve its performance by keeping the number of worker threads low"... God knows

Originally Posted by 2kaud

[...] so the issue is not a problem in the original code as expected.

unless each thread decides on his own which files to process, it could still be a problem with the original code; a thread pool with a single thread and a single thread are not the same thing because the sequence of io operations is different; supposedly, in the former main() fills a queue of file paths and concurrently the worker processes them in order; in the latter file paths are gathered and processed serially. I'm not network savvy, but I vaguely recall a similar problem while accessing from windows a (badly configured) samba share ... ( yes, I know this is not very helpful )

moreover, it would be interesting to see how the speedup scales with the number of pool threads.