Tag: Ubuntu

awscli (similar to the Azure az command) is packaged for Ubuntu, but the version which is in the Ubuntu 18.04 repositories is “out of date” and won’t work with AWS. You *actually* need to run the following:

At work, we share tips and tricks, and one of my colleagues recently called me out on the following stanza I posted:

I like this [ansible] one for Debian based systems: - name: "Apt update, Full-upgrade, autoremove, autoclean"
become: yes
apt:
upgrade: full
update_cache: yes
autoremove: yes
autoclean: yes
And if you’re trying to figure out how to do that in Shell:apt-get update && apt-get full-update -y && apt-get autoremove -y && apt-get autoclean -y

His response was “Surely you’re not logging into bash as root”. I said “I normally sudo -i as soon as I’ve logged in. I can’t recall offhand how one does a sudo for a string of command && command statements”

Well, as a result of this, I looked into it. Here’s one comment from the first Stack Overflow page I found:

You can’t run multiple commands from sudo – you always need to trick it into executing a shell which may accept multiple commands to run as parameters

I use Ansible… quite a bit :) and one of the things I do with Ansible is to have a standard build desktop that I can create using Vagrant. Recently I upgraded the base to Ubuntu 18.04, and it annoyed me that I still didn’t have a working keyboard combination, so I kept getting US keyboards. I spent 20 minutes sorting it out, and here’s how to do it.

Essentially, inspec is a way of explaining “this is what my server must look like”, so you can then test these statements against a built machine… effectively letting you unit test your provisioning scripts.

I’ve already built a fair bit of my current personal project using Ansible, so I wasn’t exactly keen to re-write everything from scratch, but it did make me think that maybe I should have a common set of tests to see how close my server was to the hardening “Benchmark” guides from CIS… and that’s pretty easy to script in inspec, particularly as the tests in those documents list the “how to test” and “how to remediate” commands to execute.

These are in the process of being drawn up (so far, all I have is an inspec test saying “confirm you’re running on Ubuntu 16.04″… not very complex!!) but, from the looks of things, the following playbook would work relatively well!

At home, I have a server on which I run my VMs and store my content (MP3/OGG/FLAC files I have ripped from my CDs, Photos I’ve taken, etc.) and I want to record material from FreeSat to play back at home, except the server lives in my garage, and the satellite dish feeds into my Living Room. I bought a TeVii S660 USB FreeSat decoder, and tried to figure out what to do with it.

I previously stored the server near where the feed comes in, but the running fan was a bit annoying, so it got moved… but then I started thinking – what if I ran a Raspberry Pi to consume the media there.

I tried running OpenElec, and then LibreElec, and while both would see the device, and I could even occasionally get *content* out of it, I couldn’t write quick enough to the media devices attached to the RPi to actually record what I wanted to get from it. So, I resigned myself to the fact I wouldn’t be recording any of the Christmas Films… until I stumbled over usbip.

USBIP is a service which binds USB ports to a TCP port, and then lets you consume that USB port on another machine. I’ll discuss consuming the S660’s streams in another post, but the below DOES work :)

There are some caveats here. Because I’m using a Raspberry Pi, I can’t just bung on any old distribution, so I’m a bit limited here. I prefer Debian based images, so I’m going to artificially limit myself to these for now, but if I have any significant issues with these images, then I’ll have to bail on Debian based, and use something else.

If I put on stock Raspbian Jessie, I can’t use usbip, because while ships its own kernel that has the right tools built-in (the usbip_host, usbip_core etc.), it doesn’t ship the right userland tools to manipulate it.

If I’m using a Raspberry Pi 3, there’s no supported version of Ubuntu Server which ships for it. I can use a flavour (e.g. Ubuntu Mate), but that uses the Raspbian kernel, which, as I mentioned before, is not shipping the right userland tools.

If I use a Raspberry Pi 2, then I can use Stock Ubuntu, which ships the right tooling. Now all I need to do is find a CAT5 cable, and some way to patch it through to my network…

Now I can consume the service from that device in tvheadend on my server. However, again, I need to make this persistent. So, let’s make sure the module is loaded on boot.

echo 'vhci-hcd' >> /etc/modules

And, finally, we need to attach the port on boot. Again, I’m using crontab, but should probably wrap this into a systemd service.

@reboot /usr/bin/usbip attach -r 192.0.2.1 -b 1-1.3

And then I had an attached USB device across my network!

Unfortuately, the throughput was a bit too low (due to silly ethernet-over-power adaptors) to make it work the way I wanted… but theoretically, if I had proper patching done in this house, it’d be perfect! :)

Interestingly, the day I finished this post off (after it’d sat in drafts since December), I spotted that one of the articles in Linux Magazine is “USB over the network with USB/IP”. Just typical! :D

Of course, if you try to put this into your Github repo, it’s going to get pillaged and you’ll be spending lots of money on monero mining very quickly… so instead, I spotted this which you can do to separate out your credentials:

A few months ago I was a guest on The Ubuntu Podcast, where I mentioned that I use Streisand to terminate my VPN connections. I waffled and blathered a bit about how I set it up, but in the end it comes down to this:

Add the line “config.vm.boot_timeout = 65535” after the one starting “config.vm.box”.

Change the streisand.vm.hostname line to be an appropriate hostname for my network, and add on the following line (replace “eth0” with the attached interface on your network and “192.0.2.1” with an unallocated static IP address from your network):

Run “vagrant up” in that directory to start the virtual machine. Once it’s finished starting, there will be a folder called “Generated Docs” – open the .html file to see what credentials you must use to access the server. Follow it’s instructions.

Once it’s completed, you should open ports on your router to the IP address you’ve specified. Typically, at least, UDP/500 and UDP/4500 for the IPsec service, UDP/636 for the OpenVPN service and TCP/4443 for the OpenConnect service.

I’ve previously made mention of my addiction to Google Play Music… but I was called out recently, and asked about the script I used at the time. I’m sorry to say that I have had some issues with it, and instead, have resorted to using X forwarding. Here’s how I do it.

I create a user account for that other person (note, GMM will only let you upload to 3 accounts using this method. For any more, you’ll need a virtual machine!).

Recently, I have used K5, which is an instance of OpenStack, run by Fujitsu (my employer). To do some of the automation tasks I have played with both python-openstackclient and Ansible. This post is going to cover how to get those tools to work with K5.

I have access to a Linux virtual machine (Ubuntu 16.04) and the Windows Subsystem for Linux in Windows 10 to run “Bash on Ubuntu on Windows”, and both accept the same set of commands.

In order to run these commands, you need a couple of dependencies. Your mileage might vary with other Linux distributions, but, for Ubuntu based distributions, run this command:

But, if you’re working with a few projects, it’s probably worth separating these out into clouds.yml files. This would be stored in ~/.config/openstack/clouds.yml with the credentials for the environment you’re using:

Optionally, you can separate out the password, username or any other “sensitive” information into a secure.yml file stored in the same location (removing those lines from the clouds.yml file), like this:

---
clouds:
root:
auth:
password: MySuperSecretPasswordIsHere

Now, you can use the Python based Openstack Client, using this invocation:

openstack --os-cloud root server list

Alternatively you can use the Ansible Openstack (and K5) modules, like this:

This post has been revised since it was initially published on 31st March due to errors found in the resulting build. It was also missing details on the shared data drive between the two machines, so has been amended to include that.

** WARNING ** This works for me – it might not for you!

The outcome of this build will leave you with the following:

Boot up, go through the VeraCrypt bootloader, enter a password for Windows, or press escape to load the Grub bootloader where you will boot (K|L|X|)Ubuntu(| Mate| Gnome).

The Windows environment will be encrypted with VeraCrypt, an open source Full Disk Encryption technology, while the Linux environment will be encrypted using Luks. The shared volume (between Windows and Linux) will be encrypted with VeraCrypt.

PLEASE BE AWARE THAT ANY WINDOWS 10 UPGRADES WILL FAIL TO APPLY AS IT WILL NOT RECOGNISE THE VERACRYPT FILE SYSTEM! To resolve this, decrypt the Windows volume, perform the upgrade, re-encrypt it, then transfer the new recovery ISO image to the boot volume, following the method below. Yes, this will take some time. No, you don’t need to decrypt the data volume. Yes, you can use that data volume to shunt the ISO image around.

Step 4d: Update the boot volume to use these changes

Step 4e: Ensure Grub is also installed to the MBR for testing

The first part installs grub to the boot position, even though it doesn’t like it, and the second forces the core file to be unchangeable… I’m not exactly sure of the impact of this, but it’s the only way to do the next part of this process. The last bit makes sure that you’ve got the latest grub config files installed.

Step 5: Reboot and test

Just check to make sure the machine boots OK!

You should have a booting Ubuntu derivative with an encrypted file system. Awesome.

Now let’s install Windows!

Step 6: Install Windows and Veracrypt.

You should boot from your install media, when you get to the partition selector, there should only be a single NTFS partition for it to use. Use it.

Install the latest version of Veracrypt from https://veracrypt.codeplex.com/

Once it’s installed, go to System, Encrypt System Partition/Drive, “Normal” system encryption, Windows System Partition, Multi-Boot (accept the warning), Boot Drive “Yes”, Single Disk, “Non-Windows Boot Loader” – No, and then… let it go through all the rest of the steps. There will be one stage where it asks you to create a rescue disk. Just save it for later. Once the encryption settings are collected, it will do a test (which is basically just rebooting to the boot loader, having you put in your password and going back into Windows), and then let it start performing the encryption.

Once the encryption finishes, reboot the computer, enter the decryption password and test it boots to Windows OK. Then reboot it again and press escape instead of putting in the password. It will boot to your Ubuntu system.

So, there you have it. One Dual-Boot system with encrypted disks everywhere!

Step 7: Setting up the shared volume.

After you’ve got the Ubuntu and Windows volumes sorted out, next we need the shared data volume to be organised. You’ll need Veracrypt for Ubuntu. Use the following to install the Veracrypt package for Ubuntu:

Once that’s installed, boot back into Windows and create a new volume – perhaps V: for Veracrypt, or E: for Encrypted – your choice, but make sure you create it using the same password that you used for the Windows partition.

Format this new volume with either NTFS or FAT32 so that you can mount it under either operating system. I chose NTFS.

Now, you need to go into Veracrypt’s Settings menu, and select “System Encryption Settings”. Tick “Cache pre-boot authentication password in driver memory” (be aware, this means that if your machine is compromised when powered up, the password could be recovered), then OK. This may prompt you to accept the UAC at this point.

Next, with the mounted volume selected, go to the “Favorites” menu, and choose “Add to System Favorites”. In the screen which comes up, select the box under “Global Settings” which says “Mount system favorite volumes when Windows starts (in the initial phase of the startup procedure)”. There will be a warning about passwords that appears. Click OK.

You may, at this point, want to move certain aspects of your Windows desktop (e.g. the “My Documents” location) to the new mounted drive.

On the Linux OS, become root, with sudo, and then add the following lines to your crontab:

These assume that your login user’s ID is 1000 (you can check that by running the command “id” as your logged in user), that you want to use “/shared_storage” as the mount point (it stops Ubuntu treating it as a “Mountable Volume” if it’s not in your home directory and not in /mnt or /media). These options also mean that only that user (and root) can access any of the files in that partition (although, it is only you on this laptop… right?), which means you can safely use it for any files which check user permissions before allowing you to access them (e.g. SSH keys). I then set up a symbolic link to /home/MYUSERACCOUNT/Documents into the /shared_storage/Documents directory, and /home/MYUSERACCOUNT/.ssh into the /shared_storage/SSH_Keys directory.

Citataions

The following list of resources helped me out when I was struggling with what to do next! They may not be canonical sources, but they helped.

https://www.youtube.com/watch?v=Z1yWbBIqh1o – Walk through of installing Veracrypt to Windows 10. I used this to see some of the terms after I’d already installed Veracrypt. I don’t quite follow the same route as him though.