I have a SharePoint farm with many site collections. I wish to do some cleanup processing to a subset of those site collections; let's say 10 site collections. The cleanup functionality is exactly the same for each site collection; the only variance is the URL of site collection on which the process should run.

Is there a way to have a timer job which determines the 10 site collection targets and then starts 10 instances of a worker job which actually performs the required cleanup work? Each worker job would run independently and possibly in parallel.

I'm trying to find a way that I can define just one instance of the cleanup processing (an unscheduled timer job) but be able to run it 10 times with varying input parameters (the target site collection URL), hopefully at some level of concurrency. The optimal solution would be 1 timer job that dispatches 10 instances of cleanup timer jobs.

1 Answer
1

A single timer job could read from a data source (e.g. SharePoint list) which marks what Site Collections need to be processed, as well as what Site Collections have been processed (e.g. a column with the URL, and a column with a Boolean 'completed'). The timer job could then take 10 at a time and process only those. You could use multi-threading in order to spin up a single timer job that works in parallel on the 10 Site Collections or runs them sequentially. You may want a third column in your data source that denotes "high priority", allowing you to make certain sets of Site Collections bubble up to the top in order to be processed first, before other non-priority Site Collections.