For fuck sake

(Sorry, it's in German) (Sorry again, ich bin wütend)

In Anbetracht des aktuellen politischen Klimas: Woher kommt denn nochmals die ganze Politikverdrossenheit? Warum nochmals fühlen sich vor allem jüngere Wähler nicht oder nicht mehr repräsentiert?

PS. Das Outcome der Entscheidung zum Uploadfilter ist hier vollkommen belanglos. Wir können einen Kampf führen und wir können auch verlieren. No hard feelings about that.

Aber WIE in den letzten Wochen mit Kritik umgegangen, wie von seitens einiger EU Parlamentarier bewusst diffamiert und beschwichtigt wurde, ist inakzeptabel.

Das werden wir nicht vergessen. Wir sehen uns bei den nächsten Wahlen.

Landbrot

I just found a new book to work with: "Brotbackbuch Nr. 1" from Lutz Geißler (in German) and love it from the beginning. Every recipe is listed with the effective and absolute time it needs, and apart from the recipies it also incorporates a lot of background informations and craftsmanship knowledge. I came to this book via the (also German) podcast Chaosradio Express, or more specifically, episode 213 - "Brot".

But enough waffle.

My latest creation is the most basic recipe from the book: The "Landbrot"

The recipe is very basic but because I cannot find it publicly at brotbackbuch.de I have to assume it's under copyright. Still some notes about making this bread

  • Plan it well ahead, on the baking day you will need up to 4 hours. Don't start at 8, otherwise you will bring your bread out of the oven at 12. Accelerating bread is not possible or at least not recommendable.
  • I didn't spend the recommended 1 and half hours of the "Stückgare". The result is still a very good bread, but could be a bit more fluffy. Again: don't try to haste bread. That will not work
  • The result is amazing. Very easy to make bread with potential for everyday usage. But I'm still at the beginning
  • I ate a full quarter of it in one morning. It was really good!

Linux 5.0

Yeah, I am running a recent self-made build of Linux 5.0 🙂

Despite the major version number change, there's nothing more special about this version, that with other releases. Still, I find this pretty cool!

Now, back to work ...

ssh config for IPv6

This is just a short note to remind me, how to configure a link-local IPv6 address in the ssh-config

Remember to put two precent signs, otherwise you might get errors similar to the following

IPv6 for the win!

Sourdough bread

Another weekend, another bread.

This time I was trying myself on a very simple sourdough recipe, for a tasty bread without any yeast

The recipe is really simple, and split into two parts: The sourdough and the actual bread:

Sourdough

100g flour, 100g water, 10g sourdough starter

I fed an took care of the sourdough started for about 4 days, until it was active enough to get to the bread. Then take out 10g and mix all ingredients together. Let them stand for about 12-16h at 25 degree.

Bread

Sourdough, 400g flour, 200g water, 10g salt

Mix together, and let stand for about 30 Minutes. Then knead it by hand for some minutes. Let stand for about 6 hours at at 25 degrees.

Bake at 200 degrees for about 40 minutes.

The result. The form is not so round because it was to gluey for the bread form, resulting in half of the bread sticking there and consecutively ending up in the sink 🙁

Initially I declared the other half on the baking tray as failure, but because the oven was hot and I wanted to try the taste I was anyways going to bake it. Turns out, that the bread was kind-of self-healing. Lesson learned: Bread can excuse small (and big) mistakes pretty well. At least this one 🙂

The inner part of the bread has very nice pores and tastes fluffy. Pretty much impressive for a bread without any baker's yeast! Nature gives you all you need, you only have to know how to use it. And it takes it's time. You can't hurry a good bread.

Taste-wise I am a bit disappointment. The bread has many aroma, especially in the crust, but the interior is a bit too sour. The reason is probably, that the sourdough started had too many sour-bacteria and too less yeast cultures, but something that I can make better the next time by selectively feeding the yeasts before starting the sourdough (so called "Hefeführung"). And let the bread-dough stand for not so long, not even to peak height.

Well, it was a try, and for now I am happy with the result. Especially after I wanted already to declare it failed after half of the dough sticking to the bread basket.

And for now: Happy baking!

Gridengine and CentOS 7

... there's life in the old dog yet!

We are still using the Gridengine on some of our high performance clusters and getting that thing running isn't really a piece of cake. Since Oracle bought Sun, things have changed a little bit: First of all, the good old (TM) Sun Grid engine doesn't exist anymore. There are some clones of it, with the most promising candidate being probably the Son of Grid Engine project. This is also what I will refer as gridengine henceforth. Noticeworthy, but not covered is the OpenGrid Scheduler and the commercial Univa Grid Engine (I'm not linking there), which is just the old Sun Grid engine, but sold to Univa and commercially distributed

In the Debian world, there is a gridengine deb packet, which just works nicely as it should do. There was a el6 port for CentOS 6, but there is nothing official for CentOS 7 (yet?). I've build them myself and everyone is free to use them as they are. They are provided as-they-are, so no support or warranty of any kind are provided. Also, they should work just fine as they are

Building the Son of Grid Engine

The process was difficult enough to make me fork the repository and setup my own GitHub project. My fork contains two bugfixes, which prevented the original source from building.
The project contains also build instructions in the README.md for OpenSuSE 15 and CentOS 7 and pre-compiled rpms in the releases section.

Short notes about building

The Gridengine comes with it's own build tool, called aimk. One can say a lot about it, but if treated correctly it works okayish. The list of requirements is long and listed in the README.md for CentOS 7 and OpenSuSE 15. It hopefully also works for any other versions.

SGE uses a lot of different libraries. Mixing architectures for a single cluster environment is in general a bad idea and SGE might work, but you really don't want to hassle with the inevitable white hairs that come with all of the unpredictable and sometimes not-easy-to-understand voodoo errors that occur. Just ... Don't do that!

I never used the Hadoop build, so all binaries and everything is tested with -no-herd.

For the impatient (not commented)

Notes about installing the Gridengine

I've tried to automate the install with ansible, but the install_execd -auto script proves to be quiet unreliable. After several failed attempts, I decided to install the Gridengine manually from a shared NFS directory.

This is in general a good idea, as the spool directory anyways needs to be in a NFS share. To prevent trouble I have separated the binaries (read-only NFS) from the spool directory (read-write access to all nodes).

I've tried to mix CentOS and OpenSuSE. The Gridengine works with each other, but you will run into other problems as the execution environment is different. Don't do that!

Running the SGE over NFS is the way I recommend. Be aware of the hassle, when the master node becomes unresponsive. In that case, don't do magic tricks, just reboot the nodes. Everything else is fishy.

Known problems with Son of Grid engine

This section is dedicated to document two bugs and make them appear on google, so that other unfortunate beings, who encounter the same problems can find a solution. I've encountered two errors, when trying to build the original 8.1.9 version

This problem was the reason for me to fork it. Comment out line 51 in sge-8.1.9/source/3rdparty/qtcsh/sh.proc.c

I encountered this error when building as root. Try building as unprivileged user (which you should do anyways!)

Mirrors

I am mirroring the current version of Son of Grid engine on my ftp-server. My own fork is in the GitHub repository gridengine.

iphex

I wrote a small bash script, that transforms IP addresses into HEX format. The tool consists of 10 lines of bash script

I needed the tool to match IP-addresses to HEX files for PXE boot. Normally PXE boot fetches first the MAC-address, and then iteratively for the HEX representation of the IP address, with reducing the number of matching characters. Oracle documents the behavior very nicely for the IP address "192.0.2.91"which matches "C000025B" and the imaginary MAC-address "88:99:AA:BB:CC:DD". Then the PXE client probes for the following files (in the given order)

Now, with iphex I can easily convert the more used numerical representation of IP-addresses like 192.168.2.91 into the IMHO not directly visible HEX representation.

zfs - CKSUM errors (and a bad SATA cable)

Just some days ago, I run a zpool scrub on my ZFS RAID-z array on my home NAS. This is actually a piece of beauty - Powered by a low-energy Celeron and using FreeBSD with ZFS, this is our main storage system for photos, pictures, data, disk images, ecc. It just powers the whole home infrastructure. So during a normal routine zpool scrub, I noticed CKSUM errors popping up.

zfs CKSUM errors

Now, this is not good as CKSUM indicates the number of uncorrectable checksum errors. So, rapid action is required!
As the affected disk is a rather old disk, my first guess was, that this is an indicator of a disk going bad. So, first I run smartctl to check the smart status. So, first I checked the glabel to find the underlying physical device

use glabel status to map the output of zpool status to the underlying hardware
Output of smartctl -a /dev/ada4

The smart status looked good, but I still decided to replace the disk, as the CKSUM errors made me nervous. I put into the new disk and replaced it with the default zfs tools via

The zpool replace command immediately exploded into my face. In dmesg I could trace back some weird messages with something like "No such pool or dataset"
Now, that's weird. So apparently the issue was not the disk itself, but what else? Probably it is rather just a faulty connection. Could be a failing SATA controller or just a bad SATA cable. The cable was working just fine for years, I didn't changed anything so I was wondering. Normally I always suspect moving or mechanical parts first to fail, so a plain cable is way below a failing HDD when CKSUM errors appear in my priority list.
Yeah, I was wrong. Just replaced the SATA cable (because I had one and it was orders of magnitude easier than replacing a SATA controller) healed the whole system. Now zpool scrub runs happily through for the second time (the first time it had still to recover some errors, probably from faulty writes in the first place) and now the NAS is running smoothly as it was doing always.

No errors, everything is happy again 🙂

So, in a nutshell

CKSUM errors in ZFS without any READ or WRITE errors are sometimes also just triggered by a faulty connection (a bad SATA cable) and do not necessarily indicate a failing hard disk.

Lucky me, now I have a sparse disk on stock in case one day one disk really goes bad 🙂

Zsh and Home/End/Delete buttons

I've notices that in Zsh under Mate, the HOME/END/DELETE buttons are for some reasons not working as I expected them to work. I use vim keybindings, and am still accustomed to sometimes hit the end button to reach the end of the line. So far it has never been a problem, but zsh just reacts weirdly here. Before triggering a rage quit, I found a solution of how to deal with it. Put the following lines in your .zshrc and you're good

I found this solution here and mirror it on my blog, in case the original solution gets lost or something.

I also link oh-my-zsh here, in case someone just hopped on zsh as well and wants to make it as fancy as possible 🙂

Iceland: Reynisdrangar and Jökulsárlón

The track goes from Vic to Höfn, a but further than Jökulsárlón because it's the next bigger town on the Ringroad. This was actually the longest route because the number of camping spots available to the Camping Card was the lowest here.
After our rainy trip to the Seljalandsfoss and Vestmannaeyjar, like typically Icelandic weather, the next day was clear and sunny. There's a saying "if you don't like the weather in Iceland, just wait for 5 minutes". On that day we had a very good view at the Reynisdrangar - a spectacular volcanic rock formation in the South of Iceland.
View from Vik to Reynisdrangar
After a nice breakfast we headed to our next goal, but before, I want to notice something peculiar for the Icelandic roads: Most bridges (would say almost all outside of the bigger towns) are single lane bridges. Although it sounds dangerous, in practice it works very well. Somehow as a human you seem to be able to communicate who goes first, if you encounter another car on the road. Probably it works so well because there aren't so many cars.
Most of the bridges in Iceland are single-lane
So, expect those bridges and don't be afraid. It works pretty well!

Jökulsárlón

The next goal of our trip was the Jökulsárlón - the blue glacier lagoon located directly at the Ringroad in the South. The place and the lake is absolutely brilliant. Funny part - Everything in Iceland is breeding while summer. Very close to the touristic areas there are lots of bird nests who are frequently attacking people who come too close. As a human you are tolerated here, but not more 😉 Some birds are more clever and breed on ice isles in the middle of the lake
Bird breeding on a floating iceberg
The lake is somehow an Oxymoron, because the sweet taste of the beauty also carries a little bit of a bitter taste: Fed by an abnormal high melting of the nearby glacier, the blue glacier lake is a witness of human made climate change. So, when you walk to the beach, and see all the glacier ice head into the ocean, keep in mind that they are causing the sea levels to rise. And that this phenomenon, as clear as the ice might be, should not occur in nature. But apart from that, the blue ice bergs are actually really beautiful. If you are very lucky you might also encounter a seal. They are rather shy, but sometimes you see them hunting in the icy waters. Swimming is absolutely not possible - In those waters only the experts of nature are home. A human would instantly freeze, the dangers of colliding ice bergs not to mention. We did a guided tour on the lake with a pretty coll amphibian vehicle. It was not cheap, but if you are interested in nice landscaped and a little bit of glaciology you might enjoy it. Doing some day-hikes near the like should be possible, but since we didn't do that I won't comment on that one. And be prepared that this place is crowded with tourists. Sill: The place is absolutely brilliant! Bring your camera 🙂