Recovering Ubuntu CoreAug 1, 2015
A note of caution, most of this is an experiment and lacks finess.
Ubuntu Core was released over half a year ago using this nice thing called snappy, the design allows for transactional updates among others, these updates keep on rolling though their streams and can be kicked out (rolled back) if something was fishy, guaranteeing a certain level of confidence that the system should not break.
Now introduce the concept of storage, that thing that will always limit you no matter the amount; with this consider that a popular method to avoid this is to garbage collect, old things you forgot about will just go away.
To add a twist to the story, imagine you want to wipe your system. Given the fact that we have a clear separation of what is writable and what is intrinsic to what makes up the core, this is rather trivial. This would indeed reset any customization done to the system, but…
The OS parts that compose Ubuntu Core are also garbage collected, better said it is like a round robin of size 2, these parts of the system are implicitly garbage collected so if I want to do a real factory reset, there is no way to do that today because you don’t have the core part of the system that the device came with.
There’s a couple more questions:
- do we want to update the boot loader of the running system?
- can we recovery from a completely broken system in an autonomous way?
- how do we make this generic?
There are more…
Booting Ubuntu Core
We use two boot loaders to power this snappy system, one is
grub, the other
u-boot. We default to the former for x86 based systems while the latter we
use for arm.
Both are similar, using an A/B model to boot with some try variables that the boot loader in question reads to determine which system to boot and where to rollback to in case of an issue.
The OS part of the system lives in either a partition labeled
system-b, this is similar to the Ubuntu
rootfs with the booting parts
stripped out to a
platform specific part that lives in the
partition with an A/B scheme. Take note that the
platform name and full
functionality is under (re) design, and also currently known as
All snappy packages are intended to be real snaps, today these are snappy package types:
oem(to be repurposed as
As mentioned, two more package types are arriving,
today are driven by system image
pending a migration to actual snappy packages.
There is only one boot loader core that takes care of booting into the right system, updating this boot loader logic adds risk as breaking it would render a system useless. As long as it’s not updated everything should be fine.
When business is as usual, the boot loader will load its environment, read the
snappy_ab variable, which would contain a value of either
snappy_mode variable which would contain the value of
snappy_ab were to have the value
a, the kernel
cmdline would contain an
the kernel part (for
grub) would start with something like
linux /a/vmlinuz…, the
initrd line for
grub would be rather similar
Booting into an upgrade
When the system updates the
platform parts of the system, if the
system was currently running from
system-a it would drop the update onto
system-b and the
b part of the
system-boot partition for the kernel,
initrd and related files. The
snappy_mode variable would be set to
and after the system finished booting it would set the mode to
life would move on.
In this experiment we want to have a
recovery partition with its own boot
loader and the original image that was put on the system by the manufacturer.
This would allow:
- for potential updates to the running systems.
- a mechanism for a real factory wipe.
- a tentative installer.
For this, two new components are needed:
- a better
Additionally, and this is not final or has been discussed, we created some stub
os snappy packages. The
os snap is an ubuntu rootfs put into
squashfs while the
platform snap provides the kernel, a modified
that knows how to go into recovery or a running system and some boot loader
recovery component lives in the ubuntu rootfs.
For simplicity the focus was on
Creating an image.
To create an image in this experiment one would do:
sudo ./uflash \ --platform platform_rolling1_all.snap \ --os ubuntu_rolling1_amd64.snap \ --gadget generic-amd64_1.4_all.snap \ --snaps webdm.canonical_0.9_multi.snap \ core.img
This would create an image called
core.img, with 2 partitions:
grub, for grub’s
recovery, with all those snappy packages passed in the command line and a grub’s
I want to take into account again that this
uflash thing is just for play
and its cli will likely be different if it comes to fruition.
grub.cfg put into
recovery would boot into
recovery using the
os snaps to drive it. The
recovery logic would create two
It will then set up
boot to have an A/B schema and insert the
os snappy packages used in the
All the snappy packages passed in with
uflash will be installed
onto the system (depending on restraints defined in the
gadget snap which
are ignored here as it’s not part of the current experiment).
It will also install a
core.img which the boot loader in
jump to providing boot loader independence for the running system.
- Make sure the checksum is correct.
- Extract into
- Run it
kvm -m 1500 core.img
There is no indication of this merging into the main branch or product and the code is not production quality (yet).
A side benefit is that the
recovery logic could potentially serve as an
There are some things you just can’t do with this image, which if paying attention would be easy to spot but just in case, system image updates won’t work.