Python + Blender + import sklearn

Recently I upgraded my Ubuntu 16.04 to 18.04. I should admit it was a charm to update without any fuss or muss. However, there was one peculiar problem, it was about setting up the python plugins that I wrote/writing for my ongoing Ph.D. research projects. Everything was working fine until reinstalling Blender 2.79 and using the good old python plugins of my research.

My research is oriented towards geometry-processing and there is no other suitable candidate other than Blender, which caters to my itchy research needs. Blender supports a beautiful rendering system using cycles and can also allow one to write python code such that its prowess can be extended with only creativity as the limits. As said earlier my research is towards geometry-processing and demands a lot from linear algebra packages such as numpy, and scipy. Also, scikit-learn is an important package that I am dealing with lately. It was while trying to use scikit-learn I bumped into issues. The weird issue was when using the below statement from the Blender console,from sklearn import preprocessing

Boom, I see another window with an error in the terminal, which says

from multiprocessing.semaphore_tracker import main;main(43)

After pulling out my hair over a few weeks I stumbled upon this link where this problem was being discussed. Following the read-up on this link lead me to this link. There the user mentions that the whole issue is a bug from multiprocessing python library.

The text from the last link states,

When SemaphoreTracker.ensure_running() from blender-2.79b-linux-glibc219-x86_64/2.79/python/lib/python3.5/multiprocessing/semaphore_tracker.py is executed it runs another Blender instance with arguments -c from multiprocessing.semaphore_tracker import main;main(9) by mistake.

It all became as clear as a day. A line (#55) in semaphore_tracker.py it tries to keep the python instance alive when importingsklearn. As we are running python as an instance inside the Blender software, the lineexe=spawn.get_executable() will return the path of the blender executable instead of the python interpreter inside Blender. Obviously, Blender is not a python instance. To fix this we have to manually point the variable to an actual python interpreter path used for Blender. This depends if you are using the system python or the bundled python. I would recommend that it is better to download and use the standalone Blender version instead of installable. Again the whole game behind this logic changes if you are a windows user. In this writing context, it is Ubuntu, so I chose to download the standalone version of Blender 2.79 and used the bundled python. So opening the file at /home/myusername/blender-2.79/2.79/python/lib/python3.5/multiprocessing/semaphore_tracker.py and changing the line #55 in that python file toexe = /home/myusername/blender-2.79/2.79/python/bin/python
solve the problem. As such I am not sure how good is this fix, however, it serves the purpose for now.