https://www.pugetsystems.com

Intel Python Preview

Yes, Intel is doing their own Python build! It is still in beta but I think it's a great idea. Python is a pretty important programming language. It has a large and growing number of useful libraries for mathematical/scientific computing and machine learning, NumPy, SciPy, pandas, Scikit-learn, PySpark, theano, ...etc... Python is often used as a frontend or "glue code" language so most important API's get a nice Python interface at some point. You can find Python modules for just about everything. Yeah, it's important!

Python is not my favorite scripting language for mathematical programming. It is "C" like which makes it awkward for mathematical computing in my opinion. Yes, I like Fortran! To me a vector acts like a "column", n x 1 matrix, and its "first" element is v[1] not v[0]! Math is hard enough without having to fight your programming language to get indexing and logic correct in your algorithms. Languages that get things "right" are Julia, Lua, Mathematica, Matlab and Fortran. At least that's my opinion/rant! I do actually like Python but I just have to give that rant when I get a chance :-)

People are using Python for serious computing work so Intel's effort to do an optimized Python is significant. You can build Python with Intel compilers yourself and link stuff like NumPy to the Intel MKL. However, Intel has some really great compiler guys and I expect that they will get a highly optimized Python put together.

Asside: It makes good sense for Intel to work on a Python build but really it would be great if they were looking at Julia! Julia is the most promising and, possibly, the most important new language to come out in over a decade! http://julialang.org/

Intel Python

You will need to register for the preview release and you will need to login to your Intel dev account, i.e. if you don't have an account at Intel you will need to create one. There are Python 2.7 and 3.5 builds for Linux and Windows. The email sent after registration contains a serial number and as says "The software license expires on April 29, 2016". The note in the email says you will need the serial number but the installer never asked for it. Be aware that this is "preview" stuff! The version I downloaded is tagged 1.0.0tp1 but the release notes are tagged v0.1. Be sure to read the release notes!

The version I downloaded untared to pythoni-2.7.20150803_184913. I ran the install script and installed in my home directory instead of doing a root install. There is an environment setup script bin/pythonvars.sh that you can source to set paths and such.

Lets try it

I did a simple matrix multiply and computed the norm of the result using loops and with numpy functions. The code at the end of this post was run using the CentOS 7.2 default python and the Intel python preview.

Here's the results:

Note: Using loops in python is ridiculously slow so I used a small matrix size for that and a much larger size for numpy. Keep in mind that matrix multiplication complexity increases as n cubed!

Note: I expected Intel's numpy to be fast but it is significant that plain old python code is much faster with the Intel version too.

Tagged

Subscribe

Don - how does the optimized Intel python stack up to Anaconda with the MKL optimizations? I have to imagine that would be fairly close since Anaconda is using MKL already. Wait, I have a great Puget Systems box you guys built me with CentOS and a Xeon 2650, and I have Anaconda 2.4.1 with MKL Optimizations installed. Here's your code on Anaconda:

Looks to me like the Intel preview is better on smaller problems, and Anaconda has a bit of an edge on the larger size. It will be interesting to see how this improves towards release! Cheers!

Posted on 2016-01-30 03:19:58

Donald Kinghorn

Hi Mark, Thanks for posting your results, I was wondering the same thing! I was figuring an MKL linked numpy would be about the same as the Intel built version. The thing that surprised me the most in my (short) test was that the plain loops code was faster, I take that as a good sign of things to come. I'll be sure to post any updates from Intel. ... should have something new before the end of Feb.. Best regards --Don

Posted on 2016-02-01 17:54:59

Ramos

This is exactly the kinda tests, I'd like to see! Thanks a lot Donald, keep up the good work.

Speaking of Anaconda Python btw, (I'd like to stress I don't work for Cloudera/Continuum) Cloudera just announced a Python parcel (package) for cluster use so a Cloudera Hadoop cluster can run Python on a cluster with full sklearn libraries and distributed.

HPC and BigData(in the Hadoop) variant is getting closer and closer. In some areas.

Posted on 2016-02-26 20:31:44

Donald Kinghorn

Hey Ramos, I've been looking at this all this too and, yes, I agree it's pretty impressive. I should be doing some posts on Anaconda and I'll try to do them as Jupyter notebooks. The tools coming together for data analysis and machine learning are looking really good. Travis Oliphant and Peter Wang ( and everyone involved!) have done some brilliant work!

programming. It is "C" like which makes it awkward for mathematical computing in my opinion. Yes, I like Fortran! To me a vector acts like a â€œcolumnâ€, n x 1 matrix, and its â€œfirstâ€ element is v[1] not v[0]! Math is hard enough without having to fight your programming language to get indexing and logic correct in your algorithms. Languages that get things â€œrightâ€ are Julia, Lua, Mathematica, Matlab and Fortran.

jdrch A mathematical mind should catch up with indexing... Its just a number and not a cause to blame a programming language.

Posted on 2016-03-21 07:41:16

Mark Madsen

Intel is now inviting folks (presumably those with an active compiler license) to preview the 2017 compiler products, which includes the Python beta. I'm installing it and will be able to see some of my simulation and analysis codes head to head. I'll try to post a link to some results and Jupyter notebooks for folks to replicate in the next week or two....

Posted on 2016-04-06 04:22:33

MIchal

Mark, your approach is great -- would you, kindly, include also as a bottom line test the plain python standard instruction-mix contained in pystone.py? That may seem as an obscure idea, however, there could hardly be a better reproducible plain-pythonesque piece of code, so for assumptions on how [ Intel Python ] may increase the execution-performance, this seems to me as a good base for any non-numpy / non-MKL et al part of the python projects we live with day by day. Thanks for you kind consideration.

Posted on 2016-06-12 18:52:23

Jack Spratt

Julia? Yet another compiler to end all compilers? I wish people who advocated this kind of nonsense would be forced to do all of the maintenance coding for a shop with a turnover of 10 people a year, each of which was allowed to use their own pet language to solve whatever problem was thrown their way. FORCED. Like you don't get to go home EVER until everything in your wheelhouse is working, has been vetted, gone through change control, and passed every litmus test in the shop. If just ONCE idiots who promote this kind of lunacy understood what such a Tower of Babel is like to try to support all such gibberish would grind to a halt post-haste.