Jonathans Website and Blog

Latest Posts

So I decided to ditch THREE.js and rewrite my old Waterfall-Plot project in pure WebGL. The only external Javascript library used is gl-matrix for matrix transformations.

All the line offsetting (z-direction in time, y-direction for frequency magnitude) is now done directly in the shader. This makes the animation much smoother, as no THREE.js points/lines have to be created and destroyed anymore. It also allows more flexibility in color effects.

To hide the lines behind each other, a gl.LINE_STRIP of triangles is rendered for each line. DEPTH_TEST is not enabled, the lines and triangles are just rendered in the right order.

This website used to be hosted on https://people.ee.ethz.ch/~muejonat/.

My studies at ETH will end soon and I will consequently lose the home directory web hosting there. To not lose the audience (and Google ranking), I decided to act early enough (1 year) and redirect everything to the new address https://jo-m.ch/ via 301.

So, you wanna play around a bit with machine learning. Or run a crazy particle physics simulation. But you only have a lame Macbook. Your gaming rig would do the job, but its fans are spinning so loud your neighbors complained. Solution? Just use ETHs Euler general purpose super computer! Even though they explicitely write that Euler is “not a supercomputer”, supercomputer just sounds cooler than “general purpose” computer.

So called shareholders (lab groups and ETH departments) invested money to own a reserved percentage of Eulers computing power. However, there is also a slice reserved for us students. The best thing about it? It works without need for any bureaucracy. Just log in and start.

Python example

We will run an example from the scikit-learn website for demonstration. There will be some some small changes, because we have no GUI. But first, let’s log in. Important: you can only log in to Euler from within the ETH network or when connected via VPN.

ssh <your nethz-name>@euler.ethz.ch

You will be greeted with a disclaimer you have to accept by typing Yes the first time. Then, we can start by loading the python module.

Now you’ll have to modify the script to make it run without X11. For this, we have to tell matplotlib to write to disk instead trying to display the images directly. Modify the downloaded file at the top and the bottom to look like this:

# at the top of the file,# after the long introductionary commentprint(__doc__)fromtimeimporttime# those two lines must be inserted hereimportmatplotlibmatplotlib.use('Agg')importmatplotlib.pyplotaspltimportnumpyasnp...# bottom# replaced plt.show() by the following lineplt.savefig('plot.png')

When you submit a job for the batch processing system, it will inherit the current environment. As we already loaded the python module, we are now ready to go:

After some time, in the directory a file called lsf.o<jobid> will appear, besides the plot.png our script generated. That’s it, we’re done!

Remarks:

Your job will be killed after 4 hours. You can use an option -W hh:mm with bsub let it run longer, but it will wait longer in the queue.

Same is valid for CPU cores (-n 2 uses 2 cores instead of 1) and memory (-R "rusage[mem=2048]" uses 2GB per core). You can use up to 48 cores at the same time (check using busers, MAX). If you submit several jobs, requiring more than 48 jobs in total, they will be run sequentially. If you submit a single job requiring more than 48 cores, it will probably be stuck in the queue forever.

GPU

Only the not publicly-accessible Leonhard cluster has GPU nodes. So if you need that, you will have to ask Cluster Support.

Support

I am a student who does not know anything about scientific computing, let alone ETHs HPC infrasctructure. This blog post just tries to provide some guidance for students. I do not provide any support. Also, I am not responsible if the admins get angry at you because you ran your stuff on a login node.

For D-ITET students

There is an additional resource for D-ITET students. You can run jobs on the idling tardis machines (in the ETZ computer rooms). If you ever wondered, that is the reason for the “do not turn off” labels. The batch system is very similar to the one on Euler. Find more infos in the D-ITET Computing Wiki.

For a long time, I wanted to try out WebGL. After some initial research, it became clear that it is not that simple to use WebGL without having prior experience with OpenGL. Thats why I settled with THREE.js. This library abstracts away all the WebGL things and gives you a nice 3D interface.

I have for a long time liked Joy Divisions Unknown Pleasures album cover. You can find something about its history here.

The idea now was to recreate a live version of this cover, using audio data from the PC microphone. So I also could learn the basics of webRTC!

Functionality in short: It captures WiFi packets from the air, and counts them. This then gets mapped to a scale from 0 to 10, and displayed on a rainbow-colored LED chart.

Hardware

This runs on a Raspberry Pi Model B, although all the other models should work too.

I am using a $8 WiFi dongle from Dealextreme. It just works™ after plugging into the RasPi. lsusb reports it as Ralink Technology, Corp. RT5370 Wireless Adapter with ID 148f:5370.

Be sure to have an ethernet connection to your Raspberry Pi! During setup we will change the WiFi device to monitor mode and thus lose the wifi connection. So you could lock out yourself if you SSH into it using WiFi (although a restart will fix it).

Install our script

scp -r raspi-mon pi@<ip_of_your_pi>

On your Pi, add the following line to /etc/rc.local (to enable autostart of the script):

sudo python /home/pi/raspi-mon/server.py &

LEDs

Now, you have to wire up some LEDs from the Raspberry Pi GPIO pins. The RasPi has no problem outputting 10x 20mA, so you can connect them directly. Don’t forget to add a resistor to each LED although, except you want to destroy them in short time. I used ports 2, 3, 4, 17, 27, 22, 10, 9, 11, 7. If you use different ports, you should change this line.

Software explanation

I decided to use SQLite. It serves for multiple things simultaneously:

Thread-safe communication between LED and capturing thread

Easy data analytics (just SQL)

Cheap storage on disk (well… SD-card)

There are some things running in parallel.

One thread runs Scapy and sniffs. It logs all the beacon and data packets to the database.

Another thread (ChannelHopper) changes the wifi channel every 0.3 seconds. This is because we want to listen on all available channels, not just one..

The third and last thread (main thread) pulls the data from the SQLite database and maps the captured activity to the LEDs. It then sleeps for one second, and repeats.

Conclusion

It works, although only on the 2.4GHZ band (the cheap adapter does not provide 5GHZ access). When I download something on my phone using a 2.4GHZ channel, I can see the LED bar going up! The whole thing is deployed in a cardboard box. It just runs after powering on (plugging in) the RasPi.