Is there any tips for programatically adding and managing the huge number of folders in document library?

In my scenario, a document library has 3 hierarchies and each hierarchies contains 50 - 100 folders. The number of folders has possible to be 1,000,000 (=100*100*100).
Additionally, the terminal folders should have unique role assignments.

In straightforward way, it takes more time to add folder and set role assignments to it as the number of created folders increases. The index fragmentation of SharePoint database may cause this problem. I have only idea to execute SQL command to rebuild index for solving this problem, but this is not good method.

+1 for Wictor advise. I don't have the whole picture, but what you describe looks like a "old-school network share structuration". SharePoint goes far more beyond this philosophy.
–
Steve BJun 5 '12 at 7:21

2

SharePoint is also not built to scale unique permissions on items either, so when you re-design your solution keep that in mind. Especially when role permissions are really unique (different unique permissions on each item) scaling is pretty horrible as the "limited access" permissions assigned will grow exponentially. Adding and deleting files will suffer, at the end it could literally take hours to add a single file, also when using the UI since it copy permissions down. Read more on performance and unique item level permissions here bit.ly/Mwk6Ox
–
Anders RaskJun 5 '12 at 9:02

I do not recommend the design considering limitations with Fine grained permissions in a list\library.

You must organize documents and folders with no more items than the list view threshold, which is 5000, in the root of any particular folder. So, You can create 1000000 folders provided you retrieve only 5000 items(folders+ documents) in a view or query.

There is a technical limit yes, but given a design of this nature it's a bad design regardless if you can do it technically. Just because you can do something doesn't mean you should.
–
Bil SimserJun 5 '12 at 17:18

@BilSimser , I just presented the technical facts provided from MS but did not recommend the design. There are two requirements here : Have the folder heirarchy and have fine grained permissions on them. The first one is ok as per MS guidelines, only second is not feasible.
–
Amit KumawatJun 6 '12 at 0:31

If you keep 100 folder at each level\view in a single document library, I don't think its a problem as you are not exceeding the list view threshold which is 5000.
–
Amit KumawatJun 6 '12 at 0:44