I have left this code running for 14 hours now and it still has not completed getting a list of all the folders when passing calling it like

SetRoot( @"c:\" );

. The code is working, it is adding to the tree, but this is just ridiculous.

Essentially, I want to popular the treeview with all folders on my drive (so the treeview is searchable) AND use the actual folder icons (which I am using

SHGetFileInfo

p/invoke with the requisite params to do so). Even without getting the icon (which there is another problem I am having is that getting the icon of the folder is unique on a folder by folder basis even though the icon images themselves may be the same. I can not see a way to determine - quickly - if I already have that image saved in my TreeView's

ImageList

- aka, the folder icon for 'c:\windows' is the same as 'c:\windows\system32', however the handle, etc all return different information so there doesn't seem to be anything to go on to uniquely index them.).

What can be adjusted in my code to make this process much faster, while still retaining the folder icons from the system ? Keep in mind, that I also want ALL folders without skipping empty ones as

(I can't even show a picture of the TreeView as I stopped the loop from running after 14 hours before it finished displaying).

To test the speed of the TreeView control, I wrote the following code :

As you can see, to populate the TreeView with 100,000 items happens very quickly at approximately 2 minutes provided you use

BeginUpdate

and

EndUpdate

to prevent the control from drawing or refreshing on every item. This also shows that it is definitely not the TreeView control which is holding me back -- 14 hours is excessive - even with a drive from 1996, 14 hours to enumerate 100,000 folders, is way too long.

Upon further research I found that the issue is that the methods available for enumerating files and folders is very slow, and that when a folder is not accessible an UnauthorizedAccessException is thrown which has an inherant delay of around 200ms per incident. These exceptions stack and cause a large delay.

Additionally, Davids statement with regards to a quadratic exponent in adding items to the TreeView causes further delay is also true, however in this case, the additional delay is disproportionate to the scaling of TreeView node addition only.

To solve this problem, I was able to narrow it down to 3 issues, two of which I have completely solved so those portions of the control function within a reasonable time-frame. Breaking it down, here are the 3 issues causing delay with the OP question:

TreeView node addition is exponentially slow the deeper the tree, and the more nodes are added.

File system access does not access the native Journaling system available for NTFS and thus each file or directory is individually sourced per call. Further, if a folder is marked as restricted, the UnauthorizedAccessException imposes an artificial delay of approximately 200ms per encounter.

Retrieval of custom folder icons requests several IO actions (with their respective delays), and further the method above for storing every icon pair is inefficient, causing it's own additional delays even though minor in this scope, the delay is also relevant.

After narrowing the scope of the delay, I was able to target these factors to reduce them one by one.

This change allowed me to retrieve all folders recursively on the C drive, in under 4 seconds

NOTE: I have thus-far been unable to figure out a way to access the Journal without requiring the elevated (Administrator) privilege for the application. All attempts to access the Journal without elevation resulted in denied access exceptions.

Improving TreeView performance for large sets of nested Nodes

Next, I needed to improve the TreeView's performance to be able to add over 100,000 nested nodes as the current folder structure loaded. To do this, took a bit of Google-fu, and some tinkering to adjust the code to function for the above class's Usn format.

The result is the following addition to a customer usercontrol extending TreeView :

Testing this, it now only takes 6 seconds to fully load all 100,000+ folders into the TreeView, and user experience is instant expanding

Folders with custom icons

This is the last place which I am currently hung up on, and am still searching for a way to fully improve this.

What I have done so far, is to do a check to see if desktop.ini exists within the folder, and if it does, then call the SHGetFileInfo pinvoke to get the custom folder icon. I then add the folder that was being expanded to an internal list signaling that I already checked that folder and got any associated icons, which all happens inside the OnBeforeExpand event. While these calls are inexpensive, it still adds a significant delay (expanding c:\windows takes 12 seconds) to the process.

This is the last major set-back which I believe there is a way to access much faster than the conventional File.Exists() method and SHGetFileInfo pinvoke for obtaining custom folder icons in an inexpensive manner

Update: After more testing, I was able to narrow down this last problem to when adding an Icon to the ImageList. Each Time an image is added to an ImageList that is connected to the TreeView, the entire TreeView of nodes is refreshed. If anyone has an idea on how to make all of this work together while keeping the performance high at this magnitude of images, please let me know. Or if this internal refresh could somehow be pushed to the background in a way that doesn't lock the UI.