Search Results

Keyword: ‘etckeeper chronicles’

The etckeeper chronicles, part 4

March 5, 2009 12 comments

In the previous episodes of the etckeeper chronicles, we saw how useful etckeeper can (and could) be in its current form. For this last episode, let’s project ourselves in the future and imagine new use cases etckeeper could cover. While reading this braindump, please keep in mind some of these ideas are pretty close, some others would certainly require lots of work, and some others are probably plain stupid.

Pristine configuration branch

It’s interesting to be able to see what the original configuration files for a package looks like and be able to compare them to what you’re currently running. This is especially true once you start updating the package and keep using the same configuration file. This could be done by maintaining a specific branch of /etc containing the pristine configuration files as they would be installed by the package. All configuration merges could then be handled using branch merging features…

Configuration file integrity

Since we are tracking file differences, we are not that far away from what Tripwire does. We could have etckeeper “lock” and “unlock” commands that would prevent commits and notify you if files were modified while the lock was in place. That would certainly involve some bzr repository encryption. Being limited to /etc, this wouldn’t make sense as an intrusion detection system, but it might make sense as a policy compliance tool (no configuration change between 6am and 8pm ?). All in all, that’s an interesting area I want to investigate.

Showing installed package changes

When autocommitting APT changes, etckeeper currently shows (in the commit message) what packages were changed. This works as long as there is some /etc file change in the process. But it could maintain, in the same way it maintains the .etckeeper file, a list of packages installed. That way, any package changes would appear in the history. Comparing etckeeper branches would then also clearly show differences in package installed between two systems, making it more relevant.

One step farther, imagine if etckeeper (through more bzr integration) would also adjust packages installed whenever a revision gets restored to /etc. You could roll back your system (seen as packages installed + configuration applied) to a previous point in time. You could “branch” the configuration from an existing server and get everything but raw data in place.

Pushing your etckeeper branches on a management server

Once your complete system configuration is represented as a bzr branch, why not take advantage of more DVCS capabilities ? Pushing all your etckeeper bzr branches to a central location allows to keep a global picture of your systems configuration and to compare how they diverge in time. You could apply the same change to several servers. Imagine testing by branching your servers, and rolling back to production by merging the changes…

Other ideas ?

If you have another interesting, innovative way we could use etckeeper for (even if it implies some development), please add a comment to this article ! I’d love to hear from you and get new ideas from the community. I’ll keep you posted on this blog about any new etckeeper development in the future. Thank you for following these chronicles !

Categories: Ubuntu, Ubuntu Server

The etckeeper chronicles, part 3

March 2, 2009 6 comments

In the previous episodes of the etckeeper chronicles, we have covered basic usage and features of etckeeper in Jaunty, which hopefully showed how great it can be. In today’s episode though, we’ll have a look on its shortcomings, and how we can improve the solution to solve them in the future.

File permissions and ownership

One of the features etckeeper provides is tracking changes to file permissions and user/group ownership metadata. Revision control systems usually only consider changes in file contents. bzr additionally tracks the executable bit. So etckeeper covers the rest by maintaining a /etc/.etckeeper text file that lists files with unusual values. That way, you can tell the changes in metadata by looking at the changes in .etckeeper. This is, however, not really integrated in the output of the tools.

A confusing diff, an incomplete revert

Let’s take an example. If you change permissions to a file, the diff will not tell you that this precise file has changed, but that the .etckeeper file changed:

$ ls -l /etc/default/tomcat6
-rw-r--r-- 1 root root 1269 2009-02-27 18:50 /etc/default/tomcat6
$ sudo chgrp tomcat6 /etc/default/tomcat6
$ sudo chmod g+w /etc/default/tomcat6
$ sudo etckeeper commit "Make tomcat6 init config group-writeable"
$ sudo bzr diff -c-1 /etc | lsdiff

This can be confusing. And if you revert the change, you won’t really come back to the previous state…

$ sudo bzr revert -r-2 /etc/default/tomcat6
$ ls -l /etc/default/tomcat6
-rw-rw-r-- 1 root tomcat6 1269 2009-02-27 18:50 /etc/default/tomcat6

This is incomplete. You restore the file contents alright, but you need to manually check the permissions in .etckeeper and restore them appropriately…

bzr integration to the rescue

So the problem is that the VCS used basically ignores the .etckeeper file contents. We need to make it aware of it. That’s where the bzr etckeeper plugin, which currently only ensures that the .etckeeper file contents is refreshed before every commit, can be improved. It could also extend the “bzr diff” and “bzr revert” commands so that they integrate metadata changes by taking into account .etckeeper changes. That’s the next step, something I plan to work on in the next months and that should be ready for Karmic Koala !

Want to help ?

Please test, file bugs, ask for new interesting features ! Blog about it ! If you want to participate in development, join the etckeeper hackers group on Launchpad, follow the bugs filed on etckeeper

In next episode

In the last part of the etckeeper chronicles, we’ll imagine new features and uses for etckeeper in the future…

Categories: Ubuntu, Ubuntu Server

The etckeeper chronicles, part 2

February 26, 2009 5 comments

In the previous episode, we’ve covered installation and basic usage of etckeeper. One common argument against having /etc under revision control is that it’s rather tedious to commit changes every time you change the configuration. While this is indeed desirable (in order to get a change history that makes sense), etckeeper on Jaunty has a few features that really help in that area.

Autocommitting changes due to package installation/upgrade

Whenever you install, remove or upgrade packages, /etc contents may change. Rather than forcing you to commit those changes manually, etckeeper has a cool APT integration feature that does it for you. It even automatically commits any uncommitted change before the apt run so that you can really tell what changed. Let’s try that:

$ sudo apt-get upgrade
Committing to: /etc/                          
Committed revision 3.
$ sudo bzr log /etc
revno: 3
committer: thc 
branch nick: jaunty-test /etc repository
timestamp: Sun 2009-02-22 11:36:29 +0000
  committing changes in /etc after apt run
  Package changes:
  -lsb-base 3.2-20ubuntu2
  -lsb-release 3.2-20ubuntu2
  +lsb-base 3.2-20ubuntu3
  +lsb-release 3.2-20ubuntu3

No more cold sweat during upgrades !

Daily autocommits

Some lazy folks forget to commit their changes. Or they just don’t want to care about it. How about being able to get a day-by-day history of configuration file changes without ever manually committing anything ? That’s where the recently-added daily autocommit saves the day. It will catch uncommitted changes every night and autocommit them for you:

$ sudo bzr log --line /etc
4: root 2009-02-23 daily autocommit
3: thc 2009-02-22 committing changes in /etc after apt run
2: thc 2009-02-22 Change domain name to
1: thc 2009-02-22 Initial commit

Don’t want those autocommits to potentially mess up your well-maintained commit history ? You can easily disable them in /etc/etckeeper/etckeeper.conf.

etckeeper by default ?

With these two options enabled by default, without any specific user interaction, you can automatically build a nice history of the changes made by package installs or day-by-day by the user. You don’t really need to care about it, and this history will be ready for you the day you need it.

That makes etckeeper something that could be installed by default, for example on server installs. The user could completely ignore it until the day he needs it, and discover that during all that time etckeeper automatically logged everything.

In next episode

In part 3 of the etckeeper chronicles we’ll expose the current limitations of etckeeper and the future work planned to overcome them. Stay tuned !

Categories: Ubuntu, Ubuntu Server

The etckeeper chronicles, part 1

February 23, 2009 16 comments

One thing I have been working on in this cycle is beginning to integrate an easy solution to put /etc under revision control. I have found etckeeper, by Joey Hess, to be the right tool for the job. The version in Jaunty will feature better support of bzr as the underlying VCS, and a couple of improvements I pushed upstream. In this series of articles, I’ll present an overview of how this solution works in Jaunty and the future planned (post 9.04 release) improvements.

Why you want it

Putting /etc under revision control allows you to keep a precise history of the changes made to configuration files. This is a well-known best practice on server environments with multiple sysadmins, where being able to tell who changed what files, and when, is very useful. But this is also nice to have on home servers or on desktops ! And keeping a log is only the tip of the iceberg of possibilities that are available once you have those files under a DVCS. Think reversion to a previous revision, branching…


To install the etckeeper/bzr combination, you will simply run:

$ sudo apt-get install etckeeper bzr

Then to initialize the repository:

$ sudo etckeeper init
$ sudo etckeeper commit "Initial commit"
Committed revision 1

This will create a bzr repository for /etc, add all files below /etc (except etckeeper ignore list) to that repository and commit (save) the current contents as revision 1.

Use it

Using etckeeper is quite simple. Whenever you make a coherent change to your configuration files, you can commit it by calling etckeeper commit:

$ sudo vi /etc/hosts
$ sudo vi /etc/resolv.conf
$ sudo etckeeper commit "Change domain name"
Committing to: /etc/
modified hosts
modified resolv.conf
Committed revision 2.

Who did what when ?

To be able to get a nice history of changes, you’ll have to call the underlying VCS system, which in our Jaunty setup is bzr. For example, this will give you a nice one-revision-per-line history of the revisions present in the repository:

$ sudo bzr log --line /etc
2: thc 2009-02-22 Change domain name to
1: thc 2009-02-22 Initial commit

Here you go, a nice captain’s log of changes made to configuration files. One feature I recently pushed back upstream is sudo integration: the username shown (thc) is the one of the user running the ‘sudo etckeeper’ command, not an undetermined ‘root’.

Now suppose you want to see what changed in revision 2 ? Try:

$ sudo bzr diff -c2 /etc
=== modified file 'hosts'
--- hosts    2009-02-22 11:26:09 +0000
+++ hosts    2009-02-22 11:27:34 +0000
@@ -1,5 +1,5 @@ localhost
- jaunty-test
+ jaunty-test

# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback

=== modified file 'resolv.conf'
--- resolv.conf    2009-02-22 11:26:09 +0000
+++ resolv.conf    2009-02-22 11:27:34 +0000
@@ -1,2 +1,2 @@

Neat, uh ? You can also check the differences between the current configuration and the last-committed revision using:

$ sudo bzr diff /etc

In next episode

In part 2 of the etckeeper chronicles, we’ll see how etckeeper autocommits help in getting the right picture with minimal effort. Stay tuned !

Categories: Ubuntu, Ubuntu Server