I’m receiving more requests for upload accounts to the Deb-o-Matic servers lately (yay!), but that means the resources need to be monitored and shared between the build daemons to prevent server lockups.
My servers are running systemd, so I decided to give systemd.resource-control a try. My goal was to assign lower CPU shares to the build processes (debomatic itself, sbuild, and all the related tools), in order to avoid blocking other important system services from being spawned when necessary.
I created a new slice, and set a lower CPU share weight:
$ cat /etc/systemd/system/debomatic.slice
Then, I assigned the slice to the service unit file controlling debomatic daemons by adding the Slice=debomatic.slice option under the Service directive.
That was not enough, though, as some processes were assigned to the user slice instead, which groups all the processes spawned by users:
This is probably because schroot spawns a login shell, and systemd considers it belonging to a different process group. So, I had to launch the command systemctl set-property user.slice CPUShares=512, so all processes belonging to the user.slice will receive the same share of the debomatic ones. I consider this a workaround, I’m open to suggestions how to properly solve this issue 🙂
I’ll try to explore more options in the coming days, so I can improve my knowledge of systemd a little bit more 🙂
I’ve been playing with qemu-user-static a bit to create a set of porterboxes for my Deb-o-Matic build farm. After reading gregoa’s post on how to create cross-chroot with qemu-debootstrap, I was immediately able to create armel, armhf and powerpc boxes with very little efforts.
I tried to extend the number of porterboxes available by adding mips* and s390x, in order to have all the Linux-based architectures supported in Jessie, but with no luck. Here’s a summary of my attempts.
Chroot creation fails under both mips and mipsel trying to configure libuuid1. The problem is due to the fact libuuid1’s postinst script calls groupadd and useradd. Those two utilities rely on NETLINK sockets, which apparently are not handled by QEMU at the moment. I raised the question upstream to see whether it is possible to solve this problem.
Chroot creation used to fail with a SIGSEGV. This particular bug has been fixed recently, but it seems it’s not enough to have a working chroot. It fails with some gzip errors, probably because some portions of dpkg-deb are not fully covered by qemu-s390x-static.
Preparing to unpack .../base-files_7.3_s390x.deb ...
Unpacking base-files (7.3) ...
dpkg-deb (subprocess): decompressing archive member: internal gzip read error: '<fd:5>: incorrect data check'
/bin/tar: Unexpected EOF in archive
/bin/tar: Unexpected EOF in archive
/bin/tar: Error is not recoverable: exiting now
dpkg-deb: error: subprocess tar returned error exit status 2
dpkg: error processing archive /var/cache/apt/archives/base-passwd_3.5.33_s390x.deb (--install):
subprocess dpkg-deb --control returned error exit status 2
I wanted to create a Deb-o-Matic environment to testbuild packages for a different architecture. Taking inspiration on Stéphane’s excellent blog post, I tried to replicate the creation of a cross-architecture Linux container in Debian. Here are the steps I made:
Load binfmt_misc module:
# modprobe binfmt_misc
Install the required packages:
# apt-get install lxc debootstrap rsync qemu-user-static binfmt-support
Mount the cgroup virtual filesystem:
# mkdir /cgroup
# echo "none /cgroup cgroup defaults 0 0" >> /etc/fstab
# mount -a
I needed a specific template to create a cross-architecture container. I used the excellent one by Laurent Vivier. Download it, rename it as lxc-cross-debian, mark it executable, and store it under /usr/share/lxc/templates.
Create the cross-architecture container, an armhf one in this case:
# lxc-create -t cross-debian -n debian-armhf -- --arch armhf --suite sid --interpreter-path /usr/bin/qemu-arm-static
After a while, the container was created and I enjoyed my brand new armhf test machine 🙂
I’ve been running several Deb-o-Matic instances on several servers since ages. This service, meant to provide a way to automate the build of Debian source packages, has been the starting point for a lot of Italian developers to start packaging, and eventually become Debian or Ubuntu developers. I can’t exactly tell how many packages Deb-o-Matic servers have built so far, but it has been the primary build and test environment for at least three Debian Developers since four years.
This service was hosted for free since the beginning, but the good things always come to an end. My provider decided they cannot keep our servers running (its core business is totally unrelated to hosting servers), and asked me to shut them down. I started with the two kfreebsd-* machines, and soon I will shut down i386 and amd64 as well. When all services will be down, I’ll take care of removing the debian.net domains I created for them.
Deb-o-Matic development won’t stop, but developers who used Deb-o-Matic daily will have to find a different solution to build and test their packages.
While reading the platforms for the upcoming DPL election, I read some statements about hearing from Project members about their experience within the Debian Project, so I took the chance to write some thoughts about my experience in one of our core teams, in which I have the privilege to be part of: the FTP Team.
During NM process, my Application Manager asked me about some activities I could have involved into once I became a Debian Developer. I was fascinated by the hard work of the FTP folks when processing NEW packages (and I guess everybody reading these lines agree with me, remembering the happines of reading that wonderful ACCEPTED mail for their very first contribution to Debian, I still have it in my mailbox), so I decided I wanted to help out those folks to share a portion of their hard work.
I committed errors, sometimes they were so silly I’d be better to hide, but the team helped me to solve the mess I created. The “thank you” and “kudos” messages the FTP Team receives greatly overtake the frustration for my mistakes, and give the strength to keep up the good work 🙂
FTP Team is just one of the dozens teams which do an awesome work for the Debian sake, every day. Don’t be shy, or worried of being unsuitable or not skilled enough for the role. You’ll never know until you try. And it could happen you really like the job, and you could feel sorry for have waited so long 😉
I don’t like watching videos sitting in front of a computer, I usually store them into a USB key to plug into my Samsung LE37 TV set. It’s not an easy task, though, as I have to fight with other components of my family who usually like watching something completely different than DebConf videos 😉 I could have invested 200 € to buy a new one, but I tried a different approach first.
Recycling a netbook
I received a Tweety device as Christmas gift, and I tried to connect it to my netbook to see how it worked. I was quite impressed by the good quality of the sound (if compared to the really poor quality of the netbook speakers), and I realized I could have recycled the netbook as a brand new TV set with some little adjustments. I plugged my SyncMaster 920n PC monitor into the netbook VGA port, switched Totem to fullscreen mode, and here it is my TV set!
OK, that was quite straightforward to accomplish. A real TV set has a remote control too, but my netbook is not equipped with an IrDA port, so I had to think about a different solution. My BlackBerry is equipped with a cool app called BBSSH, which allows you to connect to any SSH server, this could be a good starting point to implement a remote control device. Then, I had to write an interface to Totem using Python and DBus binding to allow my “SSH remote control” to issue commands to start and pause the player. I published a first working implementation: at the moment the only supported operations are play, pause, stop and volume adjustments, but there are a lot of opportunities by implementing full MPRIS specification. Just to avoid typing a long command each time, I finally created a set of bash aliases, one for each command, so I just have to type one letter to have the desired function, as much as a real remote control 🙂