example:
I have a big list of items and I need to execute a function on every item.
So if I set to run X thread at the same time and I will execute the function on the on the X items etc..
Like scrapebox do when check pagerank or other things.
I tried everything, I read a lot of documentation but nothing
please help me with some examples
I use visual basic .net

How i do my threading, is create a class. For example i have a httpwebrequest class. I create properties in the class that hold values for example, a url, proxy ect. The most important property in the class would be a property called index. The index property would keep track of what thread is using the class, by the index.

The class would also have public events. Say i have an event in my class called DataArrival(Data as string,Index as integer)
we could raise that event when we get data back from a webrequest with the content we got back from the website, and the index that requested it, Witch would be the thread number that requested it. This way we know what thread requested what and each thread has its own information. How would you use the class? Example below.

Imports System.Threading

Public Class frmMain

Private objRequest(99) As Request '-- Request is the name of our class that holds the properties, events ect..

Here we fired of 100 threads, with each thread having its own cookies, content ect. In each sub you can do what you want next with the thread preserving its own thread data. That's just an example if you got vb.net down good enough you will be able to go from there.

One thing you must not forget is that you can never have more threads actively running then you have cores in your computer. Double it if it has HT technology. Now you may think I talk bullshit because you can run scrapebox with 400 threads if you like. That is true but the fact is that scrapebox and other internet software uses mechanisms that goes to sleep while it Waits for network to respond and so your computer moves on to the next thread meanwhile. In .net such classes can be webrequest operations and other network stream operations.
So what was the point of this. You can never load a bunch of threads to do mathematical operations or similar since they have no sleep state that lets other threads take over. If you don't put a sleep manually in them

One thing you must not forget is that you can never have more threads actively running then you have cores in your computer. Double it if it has HT technology. Now you may think I talk bullshit because you can run scrapebox with 400 threads if you like. That is true but the fact is that scrapebox and other internet software uses mechanisms that goes to sleep while it Waits for network to respond and so your computer moves on to the next thread meanwhile. In .net such classes can be webrequest operations and other network stream operations.
So what was the point of this. You can never load a bunch of threads to do mathematical operations or similar since they have no sleep state that lets other threads take over. If you don't put a sleep manually in them

Click to expand...

OS already have the capability to schedule threads
it doesn't depend on number of cores
you can create as many threads as you like and the OS will be able to run them
cmiiw

One thing you must not forget is that you can never have more threads actively running then you have cores in your computer. Double it if it has HT technology. Now you may think I talk bullshit because you can run scrapebox with 400 threads if you like. That is true but the fact is that scrapebox and other internet software uses mechanisms that goes to sleep while it Waits for network to respond and so your computer moves on to the next thread meanwhile. In .net such classes can be webrequest operations and other network stream operations.
So what was the point of this. You can never load a bunch of threads to do mathematical operations or similar since they have no sleep state that lets other threads take over. If you don't put a sleep manually in them

Click to expand...

So much wrong in one post. So, much, wrong.

Using the ThreadPool you can start already spawned threads, which improves performance.

I give up. Not so mutch wrong. most of it is correct but OS have abilliy to put threads in sleep by itself.
I made a test because of dummydecoy's post and i stand corrected.

However if you do not put sleep in the threads your computer will not be usable very quick. I made a test on a dualcore machine and it started up about 30threads in about 2-3min. Every thread gets slower to start.
I would still follow the rule that you should not exceed the threads then there is cores unless you have sleep events or use classes that makes your thread sleep. If you do not fire up more threads then you have cores your will loose performance on non sleeping threads.

Here is a test code for 400 threads with no sleep and endless loops. Add a textbox and a button to make it run. This will still prove my previous post to have a good point. Even if "you can not" part was wrong.

I give up. Not so mutch wrong. most of it is correct but OS have abilliy to put threads in sleep by itself.
I made a test because of dummydecoy's post and i stand corrected.

However if you do not put sleep in the threads your computer will not be usable very quick. I made a test on a dualcore machine and it started up about 30threads in about 2-3min. Every thread gets slower to start.
I would still follow the rule that you should not exceed the threads then there is cores unless you have sleep events or use classes that makes your thread sleep. If you do not fire up more threads then you have cores your will loose performance on non sleeping threads.

Here is a test code for 400 threads with no sleep and endless loops. Add a textbox and a button to make it run. This will still prove my previous post to have a good point. Even if "you can not" part was wrong.

Creating threads is an expensive job, thus the ThreadPool should be used. Sleeping a thread is bad and not needed, unless you want to delay some action. Context switches are also expensive, so spawning many threads is not suggestable, more is not always better. But if you need 2 mins to start 30 threads, then your code is really, really bad. I can start hundred of threads within seconds.

Creating threads is an expensive job, thus the ThreadPool should be used. Sleeping a thread is bad and not needed, unless you want to delay some action. Context switches are also expensive, so spawning many threads is not suggestable, more is not always better. But if you need 2 mins to start 30 threads, then your code is really, really bad. I can start hundred of threads within seconds.

Click to expand...

Doevents should not be used. I know. but instead of fire a thread away from the gui i choose doevents in this example to prove a Point. Now you can fire 100s of threads that has no sleep in it like mine. Do it and prove me wrong. I can also optimze the code above to be much much better. but the code i wrote is very simple and proves a Point.

Doevents should not be used. I know. but instead of fire a thread away from the gui i choose doevents in this example to prove a Point. Now you can fire 100s of threads that has no sleep in it like mine. Do it and prove me wrong. I can also optimze the code above to be much much better. but the code i wrote is very simple and proves a Point.

Click to expand...

Not exactly sure what you're asking of me, and not even sure what you are trying to prove. But starting 30 threads should not take 2 minutes.

What i mean is. if i start a thread that runs and have no sleep event in it it will consume ALOT of cpu. sure i can use threadpool and fire away 1000 threads or skip the wait event in my code above to start all 400 in a go. but all Those threads will not acutally start unless other threads go to sleep. Either with help from os or from some sleep system within the thread. They will only fall into a queue that is processed by os when there it time to work with them. My code above was slow because i waited for each and one thread to start Before i started Another. maybe it was not 2 minutes but it feelt like it.. i did not really Clock it and it will be different from computer to computer. i have written software that makes use of several hundreds of threads at the same time and work well and fast. however they do have classes in them that will put them in wait mode. Like when you use httpwebrequest and such to get / post data. So what i asked of you is to prove me wrong with some code that can run fast and simultainious without sleep in it.

What i mean is. if i start a thread that runs and have no sleep event in it it will consume ALOT of cpu. sure i can use threadpool and fire away 1000 threads or skip the wait event in my code above to start all 400 in a go. but all Those threads will not acutally start unless other threads go to sleep. Either with help from os or from some sleep system within the thread. They will only fall into a queue that is processed by os when there it time to work with them. My code above was slow because i waited for each and one thread to start Before i started Another. maybe it was not 2 minutes but it feelt like it.. i did not really Clock it and it will be different from computer to computer. i have written software that makes use of several hundreds of threads at the same time and work well and fast. however they do have classes in them that will put them in wait mode. Like when you use httpwebrequest and such to get / post data. So what i asked of you is to prove me wrong with some code that can run fast and simultainious without sleep in it.

Click to expand...

It will do that if you are doing it wrong. Multithreading is hard to get right.

Here is my latest app running 100 concurrent registrations(100 threads + 16 threads assigned by the .NET framework on startup ) with 2% CPU usage. Surely that's not A LOT of CPU, not a single time Sleep is being called in it. Maybe you should look up async web requests, should help you out.

asyc in theory fireing threads in itslef and are in sleep mode while waiting for data. but you will never see it so that is a bad example. I clearly see that we never will come to an understanding. hardware can not be tricked. OS have ability to force sleep threads so it can mange more but perfomance will suffer. In my first post i was talking about mathematical operations and threads without sleep. If you do async you fire away code FROM your thread.(or where do you Think it will be executed?). Try make some calcualtions for a 3d landscape. fire up some calcualtions and you can never do better with more threads then there is cores.

asyc in theory fireing threads in itslef and are in sleep mode while waiting for data. but you will never see it so that is a bad example. I clearly see that we never will come to an understanding. hardware can not be tricked. OS have ability to force sleep threads so it can mange more but perfomance will suffer. In my first post i was talking about mathematical operations and threads without sleep. If you do async you fire away code FROM your thread.(or where do you Think it will be executed?). Try make some calcualtions for a 3d landscape. fire up some calcualtions and you can never do better with more threads then there is cores.

Click to expand...

Indeed. Let's just agree to disagree. But one last thing, sure it's different when doing calculations, but not when it comes to automating websites.

When you are working with heavy processor usage functionality (e.g. compiling), the maximum number of threads that will optimize your speed is equal to the number of cores (real or if you have HT, virtual).

The reason is obvious - if you have 8 workers (cores), and you assign only 2 jobs in total, you have 6 of them sitting. If you assign 8 jobs total, they are all working on 1 job at the same time. If you assign 16 jobs in total, they still can only work on 1 job at a time, so the rest simple get queued. No extra gain above 8 but rather a small loss sine you have to manage the queue.

What hatemachine has in mind is that he gets better performance when he uses say 100 threads in his program that 8. That 's true also, but the difference is that his program is not heavy processor usage. When you use the network, it 's may orders of magnitude slower that your cores. Which means that a task gets to sleep anyway while waiting the network IO. So in that case, it 's optimal to employ more threads than your cores.

When you are working with heavy processor usage functionality (e.g. compiling), the maximum number of threads that will optimize your speed is equal to the number of cores (real or if you have HT, virtual).

The reason is obvious - if you have 8 workers (cores), and you assign only 2 jobs in total, you have 6 of them sitting. If you assign 8 jobs total, they are all working on 1 job at the same time. If you assign 16 jobs in total, they still can only work on 1 job at a time, so the rest simple get queued. No extra gain above 8 but rather a small loss sine you have to manage the queue.

What hatemachine has in mind is that he gets better performance when he uses say 100 threads in his program that 8. That 's true also, but the difference is that his program is not heavy processor usage. When you use the network, it 's may orders of magnitude slower that your cores. Which means that a task gets to sleep anyway while waiting the network IO. So in that case, it 's optimal to employ more threads than your cores.

Click to expand...

Indeed. I never said that he is not, my point was that it's different when doing heavy I/O work as you just confirmed.

Note that adblockers might block our captcha, and other functionality on BHW so if you don't see the captcha or see reduced functionality please disable adblockers to ensure full functionality, note we only allow relevant management verified ads on BHW.