Resource control with systemd

I’m receiving more requests for upload accounts to the Deb-o-Matic servers lately (yay!), but that means the resources need to be monitored and shared between the build daemons to prevent server lockups.

My servers are running systemd, so I decided to give systemd.resource-control a try. My goal was to assign lower CPU shares to the build processes (debomatic itself, sbuild, and all the related tools), in order to avoid blocking other important system services from being spawned when necessary.

I created a new slice, and set a lower CPU share weight:
$ cat /etc/systemd/system/debomatic.slice
[Slice]
CPUAccounting=true
CPUShares=512
$


Then, I assigned the slice to the service unit file controlling debomatic daemons by adding the Slice=debomatic.slice option under the Service directive.

That was not enough, though, as some processes were assigned to the user slice instead, which groups all the processes spawned by users:

This is probably because schroot spawns a login shell, and systemd considers it belonging to a different process group. So, I had to launch the command systemctl set-property user.slice CPUShares=512, so all processes belonging to the user.slice will receive the same share of the debomatic ones. I consider this a workaround, I’m open to suggestions how to properly solve this issue 🙂

I’ll try to explore more options in the coming days, so I can improve my knowledge of systemd a little bit more 🙂

Advertisement

Adventures in cross-chroot creations

I’ve been playing with qemu-user-static a bit to create a set of porterboxes for my Deb-o-Matic build farm. After reading gregoa’s post on how to create cross-chroot with qemu-debootstrap, I was immediately able to create armel, armhf and powerpc boxes with very little efforts.

I tried to extend the number of porterboxes available by adding mips* and s390x, in order to have all the Linux-based architectures supported in Jessie, but with no luck. Here’s a summary of my attempts.

MIPS*
Chroot creation fails under both mips and mipsel trying to configure libuuid1. The problem is due to the fact libuuid1’s postinst script calls groupadd and useradd. Those two utilities rely on NETLINK sockets, which apparently are not handled by QEMU at the moment. I raised the question upstream to see whether it is possible to solve this problem.

s390x
Chroot creation used to fail with a SIGSEGV. This particular bug has been fixed recently, but it seems it’s not enough to have a working chroot. It fails with some gzip errors, probably because some portions of dpkg-deb are not fully covered by qemu-s390x-static.

Preparing to unpack .../base-files_7.3_s390x.deb ...
Unpacking base-files (7.3) ...
dpkg-deb (subprocess): decompressing archive member: internal gzip read error: '<fd:5>: incorrect data check'
/bin/tar: Unexpected EOF in archive
/bin/tar: Unexpected EOF in archive
/bin/tar: Error is not recoverable: exiting now
dpkg-deb: error: subprocess tar returned error exit status 2
dpkg: error processing archive /var/cache/apt/archives/base-passwd_3.5.33_s390x.deb (--install):
subprocess dpkg-deb --control returned error exit status 2

So long, and thanks for all the packages!

I’ve been running several Deb-o-Matic instances on several servers since ages. This service, meant to provide a way to automate the build of Debian source packages, has been the starting point for a lot of Italian developers to start packaging, and eventually become Debian or Ubuntu developers. I can’t exactly tell how many packages Deb-o-Matic servers have built so far, but it has been the primary build and test environment for at least three Debian Developers since four years.

This service was hosted for free since the beginning, but the good things always come to an end. My provider decided they cannot keep our servers running (its core business is totally unrelated to hosting servers), and asked me to shut them down. I started with the two kfreebsd-* machines, and soon I will shut down i386 and amd64 as well. When all services will be down, I’ll take care of removing the debian.net domains I created for them.

Deb-o-Matic development won’t stop, but developers who used Deb-o-Matic daily will have to find a different solution to build and test their packages.

Spare Parts TV Set

I don’t like watching videos sitting in front of a computer, I usually store them into a USB key to plug into my Samsung LE37 TV set. It’s not an easy task, though, as I have to fight with other components of my family who usually like watching something completely different than DebConf videos 😉 I could have invested 200 € to buy a new one, but I tried a different approach first.

Recycling a netbook

I received a Tweety device as Christmas gift, and I tried to connect it to my netbook to see how it worked. I was quite impressed by the good quality of the sound (if compared to the really poor quality of the netbook speakers), and I realized I could have recycled the netbook as a brand new TV set with some little adjustments. I plugged my SyncMaster 920n PC monitor into the netbook VGA port, switched Totem to fullscreen mode, and here it is my TV set!

Remote control

OK, that was quite straightforward to accomplish. A real TV set has a remote control too, but my netbook is not equipped with an IrDA port, so I had to think about a different solution. My BlackBerry is equipped with a cool app called BBSSH, which allows you to connect to any SSH server, this could be a good starting point to implement a remote control device. Then, I had to write an interface to Totem using Python and DBus binding to allow my “SSH remote control” to issue commands to start and pause the player. I published a first working implementation: at the moment the only supported operations are play, pause, stop and volume adjustments, but there are a lot of opportunities by implementing full MPRIS specification. Just to avoid typing a long command each time, I finally created a set of bash aliases, one for each command, so I just have to type one letter to have the desired function, as much as a real remote control 🙂

Turning 28

Getting older^W wiser, and definitely fatter than yesterday 😉

[ Photo of the cake is missing, have I already told you I became fatter? ]

Deb-o-Matic updates

I recently invested some time to work on several cool features in Deb-o-Matic.
I plan to release 0.9 version soon, so I wanted something new for its users 😉

I merged some patches by Alessio Treglia, who implemented a couple of nice things:

• Mailer support, to send build status notifications to the uploader
• Blacklist support, to temporarily limit builds for a given distribution

Then, I worked on some changes I really wanted to have:

• Rebuild support, to redo build of a package already in distribution archive
• Improved lock system, to avoid random build hangs which affect older code
• Fix pyinotify loop, which prevented Deb-o-Matic to quit in interactive mode

Now that the big part of the planned features is committed, I am going to test them to spot major issues, and add some minor improvements to have a shiny release 🙂

Last but not least, your help is appreciated to complete translations! There are really a few strings to process, and several languages are already half-completed, so if you can spend five minutes (or even less!), I will be grateful 😉

My first GNOME patch

Yesterday I was sponsored my first GNOME patch! OK, it’s small, very small… but I’m happy anyway 🙂
Kudos to Scott Balneaves for making that possible!

Deb-o-Matic, translators needed

… and when you’ve finished with GTG, you can give a go to Deb-o-Matic translations too, 19 strings are quick to process, so you will have a lot of time to try the software 😉

Getting Things GNOME! 0.1.9, call for testing and translations

GTG developers recently released first beta of Getting Things GNOME!, you can see release notes here.

Final 0.2.0 release is due very soon. If you’re a GTG user already, or if you are looking for a task manager for GNOME desktop environment, you can contribute to the process by testing it (0.1.9 packages are already available in Sid and Lucid), reporting bugs or helping with translations.

Enjoy it! 🙂

What does “DD” mean?

Dungeons & Dragons? No time to play right now… no!
Dolby Digital? My PC doesn’t support it… no!
Due Diligence? We already did them multiple times last year… no!
Donald Duck? I like comics, but… no
Debian Developer? YAY!

I think I owe many people a beer:

• Cédric Delfosse, who originally advocated me
• Bernd Zeimetz, who assigned me an AM and reviewed my application
• Martin Meredith, my brave AM
• Christoph Berg, who approved my application
• Jonathan McDowell, who added my key to the Debian keyring
• Peter Palfrader, who created my account

Thank to the several Debian Developers who assisted me so far.
A special thank goes to Piotr Ożarowski for bearing with me 🙂

And now… stop celebrating, there’s a lot of work to do!