Tag Archives: ubuntu

Post navigation

The screen of the phone broke and you want to retrieve your contacts / files… Quite typical story. While getting your photos / files is quite trivial, plugging your phone to computer and copying necessary files would be enough.
The situation with getting out your contacts (if you happened not to sync them with Google) is slightly more complicated. Here is what I did in the case of Samsung S4 mini with broken screen. Note, the digitizer (touch screen) worked, but the USB debugging was OFF. Also, S4 mini has no video output. If your phone happens to have HDMI or MDL, just get the cable and plug it to your TV / monitor 😉

Enable USB debugging

This is hard part and can be done only manually! But it’s quite complicated with broken screen. You need to repeat 3 steps until you reach what you want: make a screenshot (HOME + POWER button in S4 mini), see what’s on the screen (navigate to you Phone storage > Pictures > Screenshots), do some action and repeat… This is extremely tedious, but proved to work with me.
In Android 4.4 which my phone had, you need to enter Settings > About, scroll down and press many times (~7 should work) Build Number. This will enable `Developer options` in Settings. You need to enter it and tick `USB Debugging` and press OK (here I needed to rotate the screen, as the right side of my digitizer didn’t work…).
I recommend clicking `Revoke access` and OK, as my computer couldn’t connect till I pressed it.
Then unplug the mobile phone and plug it again. New dialog asking for permission to access for your computer will appear on the screen. You should tick `Always allow access` and OK. From now on, the access through ADB is possible. You can check it with:

My first impressions?sudo apt is veeery slow. At first, I thought it’s due to old SD card I’ve been using, but it’s also true for newer SD card.
Some packages are missing (ie. git-lfs), but you can get them using some workarounds.

But everything just works!
You can check the mirror of https://ngschool.eu/ running on RPi2 here.
Maybe it’s not speed devil, but it stable and uses almost no energy 🙂

Initially, I had problem with streaming sound along with video. Adding, `:input-slave=alsa:// :v4l2-standard=1` solved this. You can try another values for `:v4l2-standard` ie. 0, 1 or 2, depending which microphone you want to use.

Above command will stream HD video (1280×720) in .ogg format (natively suported by most browsers) @ ~2Mbps (2000kbps). If you have slower connection, you can change `vb=2000` to `vb=1000` (~1Mbps) and play with lower resolutions. You can check available resolutions of your camera by:

lsusb -v | egrep -B10 'Width|Height'

This stream, however, is available to everyone. To limit it only to localhost, you can use iptables:

For a while, I’ve been thinking about encryption domains, like this one. But cost & complications associated with enabling SSL encryption prohibited me to do so…
Today, I’ve realised, Let’s encrypt, new certificate authority, that is completely free, automated and open, makes SSL encryption super easy!
Try it yourself (this if for Ubuntu 14.04 & Apache, for another system configuration check https://certbot.eff.org/):

Git is great, there is no doubt about that. Being able to revert any changes and recover lost data is simply priceless. But recently, I have started to be concerned about the size of some of my repositories. Some, especially those containing changing binary files, were really large!!!
You can check the size of your repository by simple command:

git count-objects -vH

Here, git Large File Storage (LSF) comes into action. Below, I’ll describe how to install and mark large binary files, so they are not uploaded as a whole, but only relevant chunks of changed binary file is uploaded.

First, I have check what can be installed from package manager.
Only Python and Perl.

Then I have checked if any of binaries are working.
For example, GapCloser is provided as binary. It took me some time to find source code…
Anyway, none of the binaries worked out of the box. It was expected, as Alpine Linux is super stripped…

I have installed build-base in order to be able to build things.
Additionally, BWA need zlib-dev.

After building & pushing, I have noticed that Alpine-based image is slightly smaller (99Mb), than the one based on Ubuntu (127Mb). Surprisingly, Alpine-based image is larger (273Mb) than Ubuntu-based (244Mb) after downloading. So, I’m afraid all of these hours didn’t really bring any substantial reduction in the image size.

Conclusion?
I was very motivated to build my application on Alpine Linux and expected substantial size reduction. But I’d say that relying on Alpine Linux image doesn’t always pay off in terms of smaller image size, forget about production time… And this I know from my own experience.
But maybe I didn’t something wrong? I’d be really glad for some advices/comments!

Nevertheless, stripping a few dependencies from my application (namely Biopython, numpy & scipy), resulted in much more compact image even using Ubuntu-based image (127Mb vs 191Mb; and 244Mb vs 440Mb after downloading). So I think this is the way to go 🙂

Lately, to make Redundans more user friendly, I have simplified it’s dependencies, by replacing Biopython, numpy, scipy and SQLite with some (relatively) simple functions or modules.

Here, I will just focus on replacing Biopython, particularly SeqIO.index_db with FastaIndex. You may ask yourself, why I have invested time in reinventing the wheel. I’m big fan of Biopython, yet it’s huge project and some solutions are not optimal or require problematic dependencies. This is the case with SeqIO.db_index, that relies on SQLite3. Here again, I’m a big fan of SQLite, yet building Biopython with SQLite enabled proved not to be very straightforward for non-standard systems or less experience users. Beside, on some NFS settings, the SQLite3 db cannot be created at all.

Ok, let’s start from the basics. SeqIO.index_db allows random access to sequence files, so for example you can rapidly retrieve any entry from very large file. This is achieved by storing the ID and position of each entry from particular file in database, SQLite3 db. Then, if you want to retrieve particular record, SeqIO.index_db looks up if this record is present in SQLite3 db, retrieves record position in the file and reads only small chunk of this file instead of parsing entire file every time you want to get some record(s).
Similar feature is offered by samtools faidx, but in this case, the coordinates of each entry are stored in tab-delimited file .fai (more info about .fai). This format can be easily read & write by any programme, so I have decided to use it. In addition, I have realised, that samtools faidx is flexible enough, so you can add additional columns to the .fai without interrupting its functionality, but about that later…

In Redundans, I’ve been using SeqIO.index_db during assembly reduction (fasta2homozygous.py). Additionally, beside storing index, I’ve been also generating statistics for every FastA file, like number of contigs, cumulative size, N50, N90, GC and so on. I have realised, that these two can be easily combined, by extending .fai with four additional columns, storing number of occurencies for A, C, G & T in every sequence. Such .fai is compatible with samtools faidx and provides very easy way of calculating bunch of statistics about this file.
All of these, I’ve implemented in FastaIndex. Beside being dependency-free & very handy indexer, it can be used also as alternative to samtools faidx to retrieve sequences from large FastA files.