2011 is almost finished, and what a year it has been. We started it with two core projects and one release behind us. During 2011, we got three releases out of the door, grew from 60 code contributors to about 200, added three new core projects, and met for two design summits.
The Essex-2 milestone was released last week. Here is our now-regular overview of the work that made it to OpenStack core projects since the previous milestone.
Nova was the busiest project. Apart from my work on a new secure root wrapper (detailed on previous articles of this blog), we added a pair of OpenStack API extensions to support the creation of snapshots and backups of volumes, the metadata service can now run separately from the API node, network limits can now be set using a per-network base and a per-flavor multiplier, and a small usability feature lets you retrieve the last error that occurred using nova-manage. But Essex is not about new features, it’s more about consistency and stability. On the consistency front, the HA network mode was extended to support XenServer, KVM compute nodes now report capabilities to zones like Xen ones, and the Quantum network manager now supports NAT. Under the hood, VM state transitions have been strengthened, the network data model has been overhauled, internal interfaces now support UUID instance references, and unused callbacks have been removed from the virt driver.
The other projects were all busy starting larger transitions (Keystone’s RBAC, Horizon new user experience, and Glance 2.0 API), leaving less room for essex-2 features. Glance still saw the addition of a custom directory for data buffering. Keystone introduced global endpoints templates and swauth-like ACL enforcement. Horizon added UI support for downloading RC files, while migrating under the hood from jquery-ui to bootstrap, and adding a versioning scheme for environment/dependencies.
The next milestone is in a bit more than a month: January 26th, 2012. Happy new year and holidays to all !
Last week saw the delivery of the first milestone of the Essex development cycle for Keystone, Glance, Horizon and Nova. This early milestone collected about two months of post-Diablo work… but it’s not as busy in new features as most would think, since a big part of those last two months was spent releasing OpenStack 2011.3 and brainstorming Essex features.
Keystone delivered their first milestone as a core project, with a few new features like support for additional credentials, service registration and using certificate-based SSL client authentication to authenticate services. It should be easier to upgrade from now on, with support for database migrations.
Glance developers were busy preparing significant changes that will land in the next milestone. Several bugfixes and a few features made it to essex-1 though, including the long-awaited SSL client connections. It also moved to UUID image identifiers.
The Nova essex-1 effort was mostly spent on bugfixing, with 129 bugs fixed. New features include a new XenAPI SM volume driver, DHCP support in the Quantum network manager, and optional deferred deletion of instances. Under the hood, the volume code was significantly cleaned up and XML templates were added to simplify serialization in extensions.
Essex-1 was also the first official OpenStack milestone for Horizon, also known as the Dashboard. New features include a instance details page, support for managing Nova volumes and a new extensible modular architecture. The rest of the effort was spent on catching up with the best of core projects in internationalization, developer documentation, and QA (frontend testing and JS unit tests).
Now, keep your seatbelt fastened, as we are one month away from essex-2, where lots of new development work is expected to land !
The 200 open seats for the Essex Design Summit were all registered in less than 9 days ! If you missed the boat, you can still register on the waiting list at http://summit.openstack.org.
For the last seats we need to give priority to existing OpenStack developers and upstream/downstream community members, so the waiting list will be reviewed manually. You will receive an email if you get cleared and get one of the very last seats for the summit.
Sometime next week, the website should allow registered attendees (as well as attendees on the waiting list) to propose sessions for the summit, so stay tuned !
August was very busy for OpenStack Nova and Glance developers, and the culmination of those efforts is the delivery of the final feature milestone of the Diablo development cycle: diablo-4.
Glance gained final integration with the Keystone common authentication system, support for sharing images between groups of tenants, a new notification system and i18n. Twelve feature blueprints were completed in Nova, including final Keystone integration, the long-awaited capacity to boot from volumes, a configuration drive to pass information to instances, integration points for Quantum, KVM block migration support, as well as several improvements to the OpenStack API.
Diablo-4 is mostly feature-complete: a few blueprints for standalone features were granted an exception and will land post-diablo-4, like volume types or virtual storage arrays in Nova, or like SSL support in Glance.
Now we race towards the release branch point (September 8th) which is when the Diablo release branch will start to diverge from a newly-open Essex development branch. The focus is on testing, bug fixing and consistency… up until September 22, the Diablo release day.
No rest for the OpenStack developers, today saw the release of the July development efforts for Nova and Glance: the Diablo-3 milestone.
With a bit more than 100 trunk commits over the month, Nova gained support for multiple NICs, FlatDHCP network mode now support a high-availability option (read more about it here), instances can be migrated and system usage notifications were added to the notification framework. Network code was also refactored in order to facilitate integration with the new networking projects, and countless fixes were made in OpenStack API 1.1 support.
We have one more milestone left (diablo-4) before the final 2011.3 release… still a lot to do !
About a month ago I commented on the features delivered in the diablo-1 milestone. Last week we released the diablo-2 milestone for your testing and feature evaluation pleasure.
Most of the changes to Glance were made under the hood. In particular the new WSGI code from Nova was ported to Glance, and images collections can now be sorted by a subset of the image model attributes. Most of the groundwork to support Keystone authentication was done, but that should only be available in diablo-3 !
Those same initial Keystone integration steps were also done for Nova, along with plenty of other features. We now support distributed scheduling for complex deployments, together with a new instance referencing model. Was also added during this timeframe: support for floating IPs (in OpenStack API), a basic mechanism for pushing notifications out to interested parties, global firewall rules, and an instance type extra specs table that can be used in a capabilities-aware scheduler. More invisible to the user, we completed efforts to standardize error codes and refactored the OpenStack API serialization mechanism.
And there is plenty more coming up in diablo-3… scheduled for release on July 28th.
There are multiple available delivery channels available to install OpenStack packages on Ubuntu.
First of all, starting with 11.04 (Natty), packages for OpenStack Nova, Swift and Glance are available directly in Ubuntu’s official universe repository. This contains the latest release available at the time of Ubuntu release: 2011.2 “Cactus” in 11.04. If you don’t use Ubuntu 11.04, or if you want a more recent version, you’ll need to enable one of our specific PPAs, please read on.
OpenStack release PPA
If you want to run 2011.2 on 10.04 LTS (Lucid) or 10.10 (Maverick) you can use the ppa:openstack-release/2011.2 PPA. Enabling it is as simple as running:
$ sudo apt-get install python-software-properties $ sudo add-apt-repository ppa:openstack-release/2011.2 $ sudo apt-get update
Starting with the Diablo cycle, we do a coordinated OpenStack release every 6 months. That said, Swift releases stable versions more often. You can get the latest Swift release through the ppa:swift-core/release PPA. Just replace the PPA name in the above example.
Also starting with the Diablo cycle, Nova and Glance deliver a development milestone every 4 weeks. If you want to test the latest features, you can enable those PPAs: ppa:nova-core/milestone or ppa:glance-core/milestone.
PPAs for testers and developers
Just before we deliver one of these intermediary releases or milestones, we use a specific “milestone-proposed” PPA to do final QA on release candidates. Enabling this one and reporting issues will help us in delivering high-quality milestones. Just enable ppa:nova-core/milestone-proposed, ppa:glance-core/milestone-proposed or ppa:swift-core/milestone-proposed.
Finally, for all projects and for every code commit, we generate a package in the trunk PPA. If you’re a developer, or like living on the bleeding edge, you should enable those: ppa:nova-core/trunk, ppa:glance-core/trunk or ppa:swift-core/trunk.
I hope this will help you select the best delivery channel for your use case, depending on whether you’re deploying, evaluating, helping in QA or actively developing. For future reference, the list of PPAs is maintained on the wiki.