ProxyJump

... from the black magic voodoo ssh box of the tech priests ...

[Scroll down for the TL;DR section]

I'm writing this post as an ode to ProxyJump of ssh, one of the little helpers that make your day awesome.
If you are working on multiple computers in different companies/networks at some point you encounter the scenario, where you want to access a computer, that is only reachable via another computer. Let's say, you need to access your office computer names datenhalde from home, but datenhalde is only reachable via the company network.
Luckily your company provides a public ssh gateway named gateway), where you can connect from your home computer (named zuhause). On a Friday you decide that it's a day where you want to work without interrupts from home. Perhaps you just brewed a nice cup of coffee and start your work

At some point, you might find it unnecessary boring to always type in ssh gateway and then ssh datenhalde and you wonder, if there is not a more convenient way, to directly access datenhalde from zuhause via gateway, but without the fuzz of redundant ssh typing.

This is where ProxyJump comes into play. Use -J

Here ssh connects first to gateway and then to datenhalde. Awesome!

ssh config for even more convenience

Even better, you can put the ProxyJump into your ssh config, so every time you access a host, if first jumps to the given gateway host and then to the destination. Too complicated formulated? Just look at the following example

Now, if you connect to datenhalde via ssh, it automatically and transparently first jumps to gateway and then to datenhalte. This configuration then applies to all protocols that are building atop ssh, like scp, rsync or libvirt.

TL;DR

Want to connect to your working computer datenhalde via a ssh gateway in one single command?

Want to configure your ssh-configuration to always jump to gateway before connecting to datenhalde?

Then your ssh connections will transparently always jump over gateway

ProxyJump for the glory!

ProxyJump is a tool for the tech priests, and it's imperative that every adept of the Adeptus Mechanicus shall be able to handle it.
... in the (unlikely?) case in Warhammer 40k they also use ssh ...

Lightning Node Revision 2

After my last post about the Lightning Node using a MOD-1016 (Embedded Adventures), some tests and time has passed by. Especially I noticed, the disturber and noise rate is high. High as in, it disturbs the normal workflow.
I was occasionally able to measure some lightnings, but with far than expected and anticipated precision and number. So back to the design, and start to think about pitfalls and design flaws.

My working hypothesis is, that because the small little antenna of the MOD-1016 is heading towards the Arduino Nano, it gets much noise from the microcontroller itself. So I decided to de-solder the current header and resolder it on the other side, so the antenna points to the outside. This is how the final product looks like

Arduino Nano with BME280 and MOD-1016 as Lightning station

Preliminary tests look pretty good, as I do not detect any noise and distrubers anymore, while still being able to communicate with the MOD-1016.

Now, back into the field with the box. Let's hope for some nice thunderstorms this week 🙂

Linux: Encrypt external HDD

#PrivacyByDefault #MassEncryption

There are two kinds of users: The ones, who encrypt their stuff and the ones, who never had lost something or never got something stolen.
Imagine your Laptop being stolen on the train. Not only, you probably lost all of your data (Backups!), but also there is now a stranger that has access to potentially very private data - Pictures of your last birthday, company records you need to keep secret or the new piece of code that is awesome and the capital of your startup you just wanted to create.
Plenty of reasons for investing a little bit of time in your digital self-defence, and a sane full-disk encryption is a major part of it.

I personally have two different approaches: VeraCrypt (successor of TrueCrypt after they surprisingly closed, leaving lots of speculations about their non-commitment to build in a secret backdoor for a large state agency, but that is based on pure speculation and is on the list of conspiracy theorems that probably turn out to be at least somehow concise enough to be taken seriously.) seems to be sane enough to be used or LUKS. In this article I'm gonna cover LUKS, as I consider this the canonical way for every mature Linux environment.

Recipe

Assuming the HDD is /dev/sdb and you want to call it Cryptodisk

  1. Reformat the external HDD. Create a single partition with any filesystem (we will overwrite this in the next step)
  2. Make sure all filesystems for that disk are unmounted.
  3. cryptsetup -y -v luksFormat /dev/sdb1
    1. Enter passphrase.
    2. Write the passphrase on a sheet of paper and store it on a safe place.
    3. REALLY do it. You probably will forget the passphrase and then you can cry your data goodbye
  4. Open the crypdevice: cryptsetup luksOpen /dev/sdb1 Cryptodisk
  5. mkfs.xfs /dev/sdb1
  6. cryptsetup luksClose Cryptodisk
  7. Plug the disk out and re-plug it in. In caja/nemo it should appear as encrypted device.
    If you click on it, it asks for the passphrase and it will be mounted.
    Alternatively, user cryptsetup cryptsetup luksOpen /dev/sdb1 Cryptodisk
    mount /dev/mapper/Cryptodisk /mnt/Cryptodisk

Note: It's possible to compartmentalize multiple partitions by putting a LVM volume atop cryptsetup.
This is more advanced but pretty much straightforward.

Step by Step guide

I plug in my HDD and assume it's gonna be recognised as /dev/sdb.

First, We need to make sure that it's unmounted

Next, format the HDD. I normally use parted, but gparted seems to be the nicer way, as it's graphical and pretty easy. So, start gparted on the disk

MAKE SURE IT's THE RIGHT DISK. Do the partitions look like the ones expected? Is there anything fishy? Once you clear your partition table or (even worse) wrote a new filesystem, it's unlikely you can fetch your data without any losses. Take a breath and double-check before doing anything.

OK, Then create the partition you want to encrypt. Select a random filesystem, as we are anyways going to delete the filesystem. It's only important to create the layout correctly. In my case it looks like the following: one partition that takes the full space (Little bit of empty space at the end is needed by GPT for the Backup table)

gparted with the newly created layout

Close gparted and encrypt the partition using cryptsetup

Congratulations, you created your first encrypted parition! Now we are gonna put a filesystem on that one, so you can actually use it 🙂

So, we are gonna "open" the cryptdevice. This means, we are putting an encryption/decryption layer, atop which we can run our filesystem

Cryptsetup asks for the Passphrase. After successfully opening the device, it will be listed as /dev/mapper/Cryptodisk

Now we create a filesystem. I chose xfs because it's a nice working horse, that runs everywhere, but you can choose whatever you want.

Great, now you've created the filesystem. Close the disk with cryptsetup

Wait until everything on the disk has been written (it stops flashing, depending on your disk) and unplug the disk.

The next time you plug your disk in, it will be recognised by caja/nemo as Encrypted Device, you type in your Passphrase and it will be automatically mounted (or with cryptstetup luksOpen and mount but the purpose was to create a convenient way to work with your external disks).

The encrypted disk appears conveniently and can be mounted with a single click

Congratulations, you just created your first fully encrypted external HDD!

The headline picture was created btw. by using the amazing dekryptize tool - a really cool ncurses animation to show how decrypting is definitely NOT working 😉

Raspbian and btrfs

I've just got a brand new Raspberry Pi 4. For now I'm just playing around a bit with it. Until openSuSE Leap will be available, I'm using Raspbian Buster which comes by default with ext4. Since I want to have snapshots, the first thing I want to do is to convert the existing root partition into btrfs. So let's do this.

0. Get Raspbian

First, flash Raspbian to a SD card and boot it. I also recommend to run a system update after booting into Raspbian. There are plenty of tutorials on the internet, that are probably far better than what I can write.

1. Prepare initramfs

In Raspbian btrfs is included as module. In order to make the kernel mount a btrfs root filesystem, we need to build the corresponding initramfs. First install the necessary tools

Now we add the btrfs module to /etc/initramfs-tools/modules

Next is to build the initramfs

And tell the bootloader to load the initramfs, by editing /boot/config.txt

And then reboot the device, to check if everything is set up properly.
If the boot succeeds, shutdown the Raspberry and take the SD-Card to another computer. If you run at this stage into trouble, probably a filename is wrong and you should be still able to recover. Otherwise: Just start from scratch - at this point really nothing is lost.

2. Convert ext4 rootfs to btrfs

In my case I insert the SD card into my laptop. The SD card gets recognised as /dev/mmcblk0 and contains the following partitions:

To convert the filesystem to btrfs, we are now doing the following steps:

  1. Optional: Make sure, the rootfs is clean (run fstck)
  2. Convert ext4 to btrfs using btrfs-convert
  3. Mount new btrfs root
  4. Edit /etc/fstab
  5. Edit /boot/cmdline.txt

On my system, I have to do the following steps

Now we edit /etc/fstab and change ext4 to btrfs. We also need to disable the filesystem-check by setting the last two digits in the btrfs line to 0

IMPORTANT: Set the last two settings in /etc/fstab to 0 and 0. The last 0 is especially important for btrfs root, since fsck and btrfs do not go so well together.

Lastly we edit /boot/cmdline.txt. We neet to replace rootfstype=ext4 to rootfstype=btrfs and set fsck.repair=no

IMPORTANT: It is crucial to set fsck.repair=no. I was stuck at some weird "mounting failed: Invalid argument" errors, because the system wanted to perform a fsck and failed.

3. Now the fun starts

This is only the kickoff. Now the funny things, like subvolumes, snapshots ecc. start

Have a lot of fun! 🙂

Caveats

  • After a kernel update, you will need to run mkinitramfs again. Probably it's the best to only do manual kernel updates (even security updates) as otherwise your Raspi might not be able to boot again.

Additional notes

Check those notes, in case something went wrong. Those emphasis the steps I had to to to make this work

  • Fsck had cause me a lot of trouble. In case you run into mount invalid errors, check if you have disable fsck in /etc/fstab (the last zero) and in /boot/cmdline.txt
  • Apperently btrfs-convert doesn't change the UUID. If you find yourself with "device not found" or similar errors, this might has changed and you will need to change the UUIDs
  • After a Kernel update you will need to run mkinitramfs again. Keep that in mind (and maybe disable auto-updates)

Common pitfalls

Crappy image of the console output with the "mounting ... failed: invalid argument" error

I got this error message when I forgot to edit cmdline.txt. Make sure, you have configured /boot/cmdline.txt correctly (especially the rootfstype=btrfs and fsck.repair=no)

Backup KVM machines using btrfs snapshots

I just wrote a small Bash script for creating offline-backups of a bunch of virtual machines on a server using btrfs snapshots.

The script shutsdown all running KVM machines, waits until they are down, creates a (readonly) btrfs snapshot and spins the machines back up. All together takes less than a minute. After the process I have an image of all KVM machines in the state, when the machines are shut down. This is then suitable for storing the machine image files on a different machine to have a complete working state of the machines. This is part of my backup (more hardware failure) strategy for one of our general purpose servers at work.

The KVM instances need to be in a btrfs subvolume, otherwise it doesn't work

See the script as gist on GitHub. You will need to do some adjustments and probably test it a couple of times, until it will work nicely.

London

Something old, something new, something red

When I first arrived here, I got the impression of a good old, yet modernized highly industrialized place. Maybe it was because we have been travelling from Heathrow to the inner city via the Tube, and there is something industrial about the old train tracks. Also, you get an immediate insight in how the city's infrastructure is organised and is maintained - alongside the tracks you constantly see different kind of cables, that sometimes are going a bit chaotic on the walls next to the train rails. Clearly the city grow went a bit out of hands already long time ago ... This impression holds throughout my whole experience in this amazing historical city where all kinds of people come together and live together.

I'm not gonna cover the default London things like Big Ben, Tower Bridge, ecc. Those have been exhaustively covered, I'm not gonna compete with much more talented people, who can do those things just much better than me. I'm gonna write through the humble eyes of a somewhat confused traveller discovering a huge city.

The tube (London Underground)

London's tube is old. I mean it was the world's first underground passenger railway, starting in 1863. In contrast to other subways I cannot shake the feeling of, that it would need some kind of modernization: You struggle more than once, if you try to get your 20+kg suitcase up a stupid staircase ... Even more when you are used to have escalators everywhere, like it is pretty much the default in plenty of other places ... At least my fitness-tracker is super happy about the number of floors you do throughout a day. I guess it's because the tube has already been there for quiet some time and it's very difficult to change or modernize the fundamental structure, as it would be required to put escalators everywhere.
Apart from that, it's also sometimes pretty much slippery, when it's raining. And, because it's England, that occurs not just occasionally.

Well, at least, they take everything with the right amount of British Charme 🙂

What I'm a bit missing in the tube, are the always present food and coffee stands. I guess for some reason they have banned them, and it's something that I really miss. There was always something nice about the bakeries and the coffee-to-go stands in underground/subway/metro stations, because it was a pretty reliable supply of breakfast and coffee for the tired traveller. I really miss them already, and it was day one, when I wrote this down!

All in all, the tube provides very good transportation, and the Oyster-Card is a super convenient method of having an anonymous pre-paid card for your inner-city travels.

Something old, something new, something red

Wherever you are, you will always find something old, like a industrial brick-stone building, a old sign or a old-looking bridge or passage and something new and modern, like a skyscraper a very modern bridge (Millennium bridge) or a new startup forming somewhere.
And this throughout whole London.

It's apparently a very busy and fast moving city, where lots of new start-ups are founded that try to somehow merge into a historic place, that was the foundation place of the industrial revolution.

And then, of course, you will always find the typical red telephone boots and the very typical red double-decker busses. They are just everywhere.

That's for now, I hope to have the time to write some follow-up posts. For now, I'm in the Peaks and don't want to waste the whole day messing around with the bazillions of pictures I made during the last days

Transfer network-manager connections to new computer

I just recently got a new Laptop (a T440p) to test it, if it suits my needs. The old x220 is still a nice companion, but I kind of need a bit more horsepower on my daily companion.

Let me tell you something about the magic of Linux - I just removed the old SSD from my x220 and put it into the T440p. It booted out of the box, with all my configurations and everything in place. No need to reconfigure or even reinstall anything. I could be productive in a couple of minutes. Also transferring the SSD was a matter of some screws, so no problem at all. That's how it should be. That's one of the reasons, why soldered-in SSD suck so badly.

So, everything was working nicely, except for some reason NetworkManager seemed to have forgotten all the Wifi connections. Except, they were still there and configured, just for the Wifi interface of the other laptop. And reconfiguring all of them is kind of boring. There has to be a better way

NetworkManager & System connection

NetworkManager stores all the connections in /etc/NetworkManager/system-connections. They are there as plaintext files (restricted to root through)

Turns out, the line we need to change in every file is the following

This is the mac-address of my old laptop, and I just need to replace it with the MAC of my new laptop. Easy as goo pie. The following one-liner does the job. Remember to replace YY:YY:YY:YY:YY:YY with the MAC address of your laptop

After that, I restarted NetworkManager, and it was nicely connecting to the available Wifi.

TL;DR

  • NetworkManager Wifi-Connections are MAC-Sensitive
  • To bring your connections to a new Wifi adapter you need to change the mac-address line of every connection with the following script

Island - CampingCard Locations 2019

Download: CampingCard-Locations-2019.gpx

Two years ago I wrote a simple python script in order to extract the locations of CampingCard.is and create a handy GPX file for my navi and mobile phone. I'm kind of surprised, that the same script still runs.

Here are the locations for this year. Enjoy! 🙂

Just for completeness, here are also the links to the files of the previous years