Our Journey to Xenial

nine Team Jun 13, 2016

Ubuntu 16.04 LTS, with the code name Xenial Xerus, is the latest version of Ubuntu LTS. It was released, as already indicated in the version number, in April 2016. LTS stands for “Long Term Support” here and means that this version of Ubuntu is supported/receives updates for 5 years.

As it is the LTS versions of Ubuntu that nine.ch use, we are of course anxious to be able to offer our customers the latest version as soon as possible. However, as a new LTS version is only released every two years, larger version leaps with applications are mostly unavoidable, which has provided a challenge for us in the introduction of the new Ubuntu version.

#First attempts with Xenial

To be ready to install Ubuntu Xenial on customer servers as soon as it was released, we began with initial preparations way back in February. This included in particular preparing a net install image, in order to be able to install Xenial directly via the network. As soon as this worked, Xenial was installed on the first servers.

The first noticeable change was the naming of the network interfaces, which are no longer called “eth0” and “eth1” etc., but for example “enp5s0”. However, we are still dependent on the “old” names for everything to work correctly for us. Therefore, we had to first of all find a solution to get these running on the Xenial servers. After several attempts, we were finally successful in installing a new server with the right network settings so that we could continue using the names.

So that the installation could then ultimately function error-free using Razor (our service for the automated installation of servers), some adjustments still needed to be made to the packages being installed. After these were implemented, we were then able to set up any server (whether virtual or physical) in a fully automated way.

#Preparing the basic configuration for Xenial

After we were able to install systems using automation, next up was our configuration management system “Puppet”. Here we first of all needed to prepare the basic configurations that we install on all of our Managed (V)Servers, for Xenial.

Lots of things worked inherently under Xenial, but some individual settings were outdated and therefore had to be adapted. This was important to ensure that these settings were only changed on Xenial systems, but not on older systems (e.g. Ubuntu Trusty).

Furthermore, lots of configurations were set up so that, for example, they did something different on Trusty (the previous LTS version) than on the previous distribution versions. The majority of these configurations, however, could be completely adopted by Xenial, or with just a few amendments.

These included, for example:

  • Installing Postfix on Ubuntu 12.04 & 14.04
  • Installing different versions of Ruby
  • Configuration of metrics collected from Ubuntu 12.04 upwards

For most of these settings we only had to change from version comparing with the name (if $::lsbdistcodename == 'trusty') to comparing with the version number (if versioncmp($::lsbdistrelease, '14.04') >= 0).

#Preparing services for Xenial

As our server had now been installed, and furnished with the basic configuration, it was time to make our services “Xenial-ready”.

Here we went from service to service and tried installing them without adaptations. With some services, this was no problem and these could therefore be used under Xenial straight away. Many of our managed servers needed a bit more attention to get them up and running. There were different reasons for this. On the one hand the version updates - some of which were extensive - had a big influence, and on the other hand it was necessary to convert our configuration management so that all necessary settings could be made for functioning with systemd. The switch from Upstart to systemd was one of the biggest changes.

In addition, the build process of our own software packages had to be adapted so that these could also be used under Ubuntu Xenial. These adaptations presented us with new challenges. One example of this is Ruby. Some of our packages were rebuilt to support multiple versions of Ruby, so that these could be installed on the “old” servers and on Xenial servers. We used this as an opportunity to standardise all Ruby versions across our servers.

#Canonical releases Ubuntu Xenial

After a few tests and adaptations in our infrastructure, the majority of our services were ready for Xenial. Thanks to starting early and making careful preparations, we awaited the release on 21 April 2016 with eager anticipation. Our task before making Xenial available to our customers was to repeat the concluding tests on the final version so that we could then deliver Ubuntu Xenial on time. Unfortunately, at the last moment Canonical made some significant changes which meant that we couldn’t keep to our planned release date. For example, the decision was made to completely remove PHP 5.6 and only continue to officially offer PHP 7.0.

#The finishing touches

In order to now be able to offer Xenial as quickly as possible in spite of the changes, we began straight away making amendments to the remaining areas where this was needed. There then followed test and quality controls, which showed us the tasks that still needed to be carried out. Gradually we identified errors and problems and then processed and eliminated them. This included, for example, problems with Monit we are using or errors in individual services.

After the last quality control, we were finally done and were able to share the good news with our customers: Ubuntu 16.04 LTS (Xenial Xerus) for managed (V)servers is available from nine.ch. Since 6 Jun 2016, we have now been installing Xenial on all new servers as standard.

Martin Wittwer works as Linux System Engineer in the Systems Team at nine.ch. Together with his colleagues from the Systems Team, he was responsible for the introduction of Xenial.