For cryo images (neural network denoised with JANNI)

The performance of the general model based on JANNI denoised data compared to low-pass filtered data did not improve. The average AUC on the validation data was in both cases the same (0.85). But this might be because of the data selected for the general model. I assume that especially on very noisy micrographs JANNI will improve the results.

During the installation of crYOLO you will see the following error message:
ERROR: imagecodecs-lite 2019.2.22 has requirement numpy>=1.15.4, but you'll have numpy 1.14.5 which is incompatible.
However, you can ignore it. It is actually also working with numpy==1.14.5

Issue 0: Training on multiple GPUs sometimes lead to worse performance (higher loss). We currently recommend to train on single gpus.

Issue 17: On the fly filtering (--otf) is slower than using it not, as the filtering is not parallelized in this case.

Closed issues

Closed issues

Issue 1: crYOLO sometimes not exit properly after training finished. Has to be killed manually.

Issue 2: If you use automatic filtering with .tif files, you get an error like “OSError: cannot identify image file 'filtered_folder/another_folder/my_image.tif'”. It will be fixed in the next release.

Issue 3: (Boxmanager) The visualization only shows the first filament when loading eman1 helical box files (start end coordinates). Will be fixed in the next release.

Issue 4: The filament mode will crash if crYOLO cannot identify a single particle in the image. Will be fixed in 1.2.2

Issue 5: If movies were aligned with cisTEM and picked with crYOLO, the box position are vertically flipped. Will be fixed in 1.2.2

Issue 6: crYOLO does overwrite the environmental variable “CUDA_VISIBLE_DEVICES” with 0 if no gpu is specified by the -g parameter. This leads to the behavior that crYOLO ignores previous settings in CUDA_VISIBLE_DEVICES. Will be fixed in 1.2.2

Issue 7: On K3 images crYOLO seems to add a offset toward the longer axis of the input image.

Issue 8: There is a logical error in filament tracing, which sometimes connects two parallel filaments.

Issue 9: Some people report an error when running cryolo prediction/training: “ImportError: numpy.core.multiarray failed to import”. It will be fixed in 1.2.3.

Issue 11: If the -g parameter is not provided, crYOLO will use the memory of all GPUs. Will be fixed in 1.2.3.

Issue 12: The LineEnhancer depdenceny of crYOLO is still dependent from opencv. Workaround: In the crYOLO environment: conda install opencv

Issue 13: After picking it can happen that some of the boxes are not fully immersed in the image. Will be fixed in 1.2.4.

Issue 14: Parallelization in filament mode is broken. Will be fixed in 1.2.4.

Issue 15: If the --gpu_fraction is used, crYOLO always uses GPU 0. Will be fixed in 1.3.1.

Issue 16: --gpu_fraction only works for prediction, not for training. Will be fixed in 1.3.2.

Issue 18: Prediction is broken in 1.3.2. It removes all particles as it claim they are not fully immersed in the image.

Issue 19: Filtering does not work if target image directory is absolute path.

Issue 20: crYOLO 1.3.4 has a normalization bug. During training the images are normalized seperately, but during prediction is done batch wise. Workaround: Use -pbs 1 during prediction. It will be fixed in 1.3.5.

Issue 21: The search range for filament tracing is too low for many datasets. To check if you are affected: Use your trained model and pick without the filament options. Check if your filaments a nicely picked (many consecutive boxes on a filament). In the next version, the search range will be increased and added as an optional parameter.

Issue 22: If absolute paths are used in the field “train_image” in your configuration file, filtering is skipped.

Issue 23: Since crYOLO 1.4.0 it sometimes take long until it starts picking. The reason seems to be the tensorflow update.<del>
* <del>Issue 24: Fine-tune mode does not start (cannot find layer model_3). Will be fixed in 1.4.1.<del>
* <del>Issue 25: When using GUI, prediction behaves differently than using command line. The reason is, that it uses a different multiprocessing start method. Will be fixed with 1.5.1

Issue 26: If you select filtering “None” crYOLO does not train properly.

Issue 27: Filament mode is not working with micrographs motion corrected by unblur. Will be fixed in the next release.

Issue 28: Minimum distance filter is not applied to cbox files.

2020/02/13 13:59
· twagner

That's it!

You might want to check if everything is running as expected. Here is a reference example:

Run it on the CPU

There is also a way to run crYOLO on CPU. To use it, just follow the instruction in the install section. This is especially useful when you would like to apply the generalized model and don't have a NVIDIA GPU.

Picking with crYOLO is also quite fast on the CPU. On my local machine (Intel i9) it takes roughly 1 second per micrograph and on our low-performance notebooks (Intel i3) 4 seconds.

Training crYOLO is much more computational expensive. Training a model with 14 micrographs from scratch on my local machine take 34 minutes per epoch on the CPU. Given that you often need 25 epochs until convergence it is a task to do overnight (~ 12 hours). However, you might want to try refining the general model, which takes 12 minutes per epoch (~ 5 hours).

Tutorials

Change log

crYOLO

Fixed a bug that was introduced with 1.5.5: Scaling of the anchor boxes was wrong. This leads to longer and unstable training and heavily affects the fine-tune mode. (Thanks to Jorge Jimenez de la Morena and Pablo Conesa)

Fixed a bug that leads to an exception (_tkinter.TclError: couldn't connect to display) at the end of the training on cluster machines. (Thanks to Wolfgang Lugmayr)

crYOLO 1.6.0:

In case of the general model, you can specify with --minsize MIN --maxsize MAX a minimum and maximum size. This will filter the particles according to their estimated size.

The estimated size and confidence distribution are now written in a new subfolder DISTR in your output folder. It will also write .csv files with a summary of the distributions.

In case of the general model, you don't need to specify the anchor size anymore.

With every run, crYOLO now writes the command used into the central log directory.

All log files (runfiles, commands, tensorflow) are now saved in the central log directory.

During training, the intermediate models now get a suffix “_tmp”. After training is finished they are renamed to the specified name in the configuration file (field: “save_weights_name”).

The boxmanager can now be started through the crYOLO GUI.

Fixed issue that the filament mode does not work with micrographs that were motion-corrected by unblur.

Fixed issue that the flaq --write_empty did not work for the filament mode.

Fixed issue that the minimum distance filter was not applied on particles in .cbox files.

Fixed issue with the evaluation tool that crashed if no particle can be found for a specific threshold.

Old crYOLO change logs

Old crYOLO change logs

crYOLO 1.5.6:

Fix installation issues

crYOLO 1.5.5:

Added a the option --use_multithreading for training. If python multiprocessing leads to problems during training (e.g. freezing, dying workers) use multithreading instead of multiprocessing.

New monitor mode for prediction: When this option is activated, crYOLO will monitor your input folder. This especially useful for automation purposes. You can stop the monitor mode by writing an empty file with the name “stop.cryolo” in the input directory. Just add --monitor in the command line or check the monitor box in GUI.

Training is now merged into one command instead of two

Add rotation as additional data augmentation

Number of layers for fine tuning are now changeable (-lft)

cryolo_evaluation.py will now output a html file with the results.

Set patch argument as deprecated

crYOLO will not allocate the complete GPU memory anymore.

Remove warmup as config file option. Please specify it with -w.

crYOLO 1.4.1:

Downgrade the dependencies to tensorflow 1.10.1 and numpy 1.14.5 as some users reported long initialization times. (Thanks to Shaun Rawson)

The initialization weights are not longer shipped with the package and downloaded on-the-fly (because they are big and pypi does not allow such big packages)

crYOLO is installed through pypi

crYOLO box manager is installed through pypi and automatically shipped with the crYOLO package

Fixed issue 22: If absolute paths are used in the field “train_image” in your configuration file, filtering is skipped.

crYOLO 1.3.5:

Fixed issue 20: During training the images are normalized separately, but during prediction is done batch wise. The lead to confusing results: some micrographs were perfectly picked, some totally unreasonable, even with the same defocus. This bug only affects the picking, already trained models can still be used.

Remove unnecessary dependencies

Add __version__ to __init__.py for easy access to package version.

crYOLO 1.3.4:

Support for SPHIRE 1.2

Changed the minimum threshold for cbox files from 0.01 to 0.1. Much faster in many cases but still low enough. If -t is lower than 0.1, the new threshold is used as minimum.

Installation now checks if python 3 is used.

Fix issue 19: Filtering does not work if target image directory is absolute path.

Fix crash when --otf was specified but filtering was not specified in the config file

crYOLO 1.3.3:

Fix issue 18: Prediction is broken in 1.3.2. It removes all particles as it claim they are not fully immersed in the image.

crYOLO 1.3.2:

Speedup prediction: Vectorized some parts of the code and optimized the creation of the cbox files. 30% speed up picking / 15% faster training compared to 1.3.1/1.3.0.

Bug fix in merging of filaments that sometimes throw “IndexError: list index out of range”. (Thanks to Alexander Belyy)

Fix in cryolo_evaluation: If the validation data is specified with -b instead of runfiles, all datasets with only one box file were ignored.

Change library requirement to PILLOW version 6.0.0

Fix issue 16: --gpu_fraction only works for prediction, not for training.

crYOLO 1.3.1:

Fix Issue 15: -g was ignored when –gpu_fraction was used.

crYOLO 1.3.0:

Fine tune the general network to your data using the new fine tune option with --fine_tune (https://1n.pm/x8rUH)

Fix Issue 8: crYOLO sometimes connects two parallel filaments. The filament tracing was optimized and seems now working properly.

Fix a severe bug in filament tracing. Curved filaments splitted by crYOLO in more straight sub pieces. However, during the division, one half of the splitted filament was lost. (Thanks to Sabrina Pospich)