I have just started to learn how to run 3b lately. Here are my questions.

I was following the example in the tutorial. The command is./multispot5_headless --save_spots test_data/results.txt --log_ratios test_data/markup1.bmp test_data/test.tif

results.txt should be the output. However what does markup1.bmp do and how is it generated?

Also, I have access to a big cluster and would like to take advantage of the parallel capability. My understanding is threeB itself is not parallelized. I have to split the image and run 3b on each subarea and combine them at the end. Is there a simple way to split the image? And how do I combine the results after running multispots5_headless in "parallel"?

oscar wrote:results.txt should be the output. However what does markup1.bmp do and how is it generated?

The markup1.bmp file is a binary image which indicates the area of the data to be analysed. The markup must be the same 2D size as the data. The area to be analysed is given by the non-black pixels. The image can be created in any image manipulation tool, such as ImageJ or the GIMP.

Also, I have access to a big cluster and would like to take advantage of the parallel capability. My understanding is threeB itself is not parallelized. I have to split the image and run 3b on each subarea and combine them at the end. Is there a simple way to split the image? And how do I combine the results after running multispots5_headless in "parallel"?

You are correct: 3B is not an inherently parallel algorithm. In order to run in parallel, you need to split the markup up into chunks, and analyse each chunk in parallel, then recombine them. We've recently added features to 3B to make this easier. First, you'll need one of the beta versions newer than the released version 1.1. You can get most of it here precompiled:

However, you'll also need the program make_grid_markup. This is not currently available precompiled, so you will need to compile it from the source. You'll need to get the latest version from the git repository. The compiling instructions as the same as in the tutorial except you need to type

Note that the squares abut but do not overlap. In order to get a good 3B run, you need to expand the squares a bit and then remove the overlapping areas. The very recent versions of 3B can do this automatically. To do this, run 3B with the following extra options:

this will dilate the areas to be run by 3.2 pixels, but will remember the original square, so during reconstruction, it can be used for filtering purposes. Run 3B on all the small markup images for an appropriate time.

You will now have a number of output files such as results-0000.txt. These can simply be aggregated into a single large file from the terminal:

Now, load the aggregated results file using the plugin. If you check the "filter" box in the load dialog, then for each run, it will filter out anything outside the original square. This will cleanly combine the data.

Hi Ed,I followed your instruction to create markup files and got a better understanding of the process. Still have a few questions:

1. So the idea is to divide up the whole image into subareas. This way 3B analysis of subareas can be run at the same time in parallel. To do that, the smaller jobs still have to be initiated manually, Am I correct?

2. If there are a lot of tiff files to be processed, with the command "multispot5_headless --save_spots results-0000.txt --log_ratios markup_small_0000.png --dilate_mask_as_filter_radius 3.2data/image-*.tiff", how are the files processed? Are the subareas of each frame being processed simultaneously? The next tiff image will be processed once all the subareas of first image is done processing? So if there are 100 60x60 images with markup size of 15x15, the program will generate 16x100=1600 result files?

3. My understanding is that, the image quality will be improved (signal noise ratio enhanced) after running 3B which will result in a new set of images with same dimensions as the inputs. Is that right?

oscar wrote:Hi Ed,I followed your instruction to create markup files and got a better understanding of the process. Still have a few questions:

1. So the idea is to divide up the whole image into subareas. This way 3B analysis of subareas can be run at the same time in parallel. To do that, the smaller jobs still have to be initiated manually, Am I correct?

That's correct. It can be automated, but the precise mechanism will depend on if you're using a cluster and what clustering software you're using.

2. If there are a lot of tiff files to be processed, with the command "multispot5_headless --save_spots results-0000.txt --log_ratios markup_small_0000.png --dilate_mask_as_filter_radius 3.2data/image-*.tiff", how are the files processed? Are the subareas of each frame being processed simultaneously? The next tiff image will be processed once all the subareas of first image is done processing? So if there are 100 60x60 images with markup size of 15x15, the program will generate 16x100=1600 result files?

Once you've used the make_grid_markup program, each file like markup_small_0000.png, make_markup_small_0001.png etc will be treated separately. This means if it gets split up into 50 separate images, you will need to issue 50 separate commands, and that will generate 50 separate results files. The program won't automatically move on when it's finished, it will simply stop. Therefore, you need to generate those 50 commands to run.

In general, knowing when to stop MCMC is an open research problem. What we generally do is to let each individual run go for 4 hours. At the end of the run, we automatically reschedule a continuation run with a slightly lower priority. In practice this means each area gets 4 hours of CPU time, then when all areas have had the 4 hours, it will add on another 4 hours etc. We let it keep going until it's reached enough iterations. This ensures fair queueing on the cluster. The precise details of this will vary depending on the system you're using.

3. My understanding is that, the image quality will be improved (signal noise ratio enhanced) after running 3B which will result in a new set of images with same dimensions as the inputs. Is that right?

Not quite:3B takes in an image stack (200 images or so), and in the marked area it computes a super resolution reconstruction using that stack. The output is a list of (x,y) pairs indicating possible fluorophore locations. This list must then be turned into an image for viewing. The ImageJ 3B plugin can do this. Typically the image created from these (x,y) pairs is just big enough to hold all the data, however, you can arrange it so that it covers the entire image area easily enough. The output is typically considerably larger, however. Input images are typically in the range 100-200nm/pixels, but an output image will generally be generated at 10nm per pixel to give a smooth reconstruction.

We're comparing the results imagej plugin and the results of the compiled program, we're using the same configurations on the mustilspot5.cfg but the results aren't the same, the later reconstruction seems to have less quality. The version of 3B is 1.1 in the two cases. Could you please advise us about it?

Another question, I'm just wondering if the 'iterations' shown the result file of imagej plugin are the same as the 'iterations' that are shown in the result file of the compiled program, i.e. does the algorithm needs to be stopped by the "Iteration: 50(200)"?