TL:DR - If you have an Ubuntu 22.04 full VM deployed from our official image, it’s liable to stop working the next time it’s rebooted (including if the node it’s running on loses power), regardless of whether you trigger a system update. Please follow the steps in the fix section below asap.
Additional notes
Having done some research on this, there’s a couple more important points to note:
- Ubuntu has automatic updates enabled by default. This means that affected VMs can be broken with just a reboot and avoiding a manual update is not enough to protect from this issue
- This seems to only affect Ubuntu 22.04 full VMs, by default (20.04 can be upgraded manually to the 5.15.x kernel versions, but it won’t happen automatically)
- Our official image for Ubuntu 22.04 full VM ships with the
linux-kvm
kernel and the latest version, linux-image-5.15.0-1046-kvm
, seems to have the same problem
Fix based on freezing kernel version
Rather than update to an early release build as shown above, I’ll show instead how to freeze the kernel version in our official Ubuntu 22.04 full VM images.
Friendly reminder: this would be a great time to backup any valuable data you might have on your VMs.
Undo the fix for generic kernels
If you already did the steps from the original post above, just undo them as follows:
add-apt-repository --remove ppa:canonical-kernel-team/ppa
apt remove linux-image-5.15.0-90-generic
Remove broken kernel version
Now remove the bad kernel version if it’s already installed and also remove the kernel meta packages:
apt remove linux-image-5.15.0-1047-kvm linux-headers-5.15.0-1047-kvm linux-modules-5.15.0-1047-kvm linux-kvm-headers-5.15.0-1047 linux-kvm linux-image-kvm linux-headers-kvm
Since I originally wrote this post, the kernel version was updated from 1046
to 1047
but the issue is still there in the new version. If the command above produces messages saying that the packages weren’t found, you might want to check the installed kernel version like this:
apt list --installed | grep linux
If you’re seeing version numbers including 1046
, you should instead remove them like this:
apt remove linux-image-5.15.0-1046-kvm linux-headers-5.15.0-1046-kvm linux-modules-5.15.0-1046-kvm linux-kvm-headers-5.15.0-1046 linux-kvm linux-image-kvm linux-headers-kvm
The version that ships in the image is 1002
. There’s no need to remove that version, it will automatically be removed during the next step.
Install known good kernel
Next, we can install the latest kernel that’s available from the more conservative “release (main)” line. This makes sure that a working kernel is used on the next boot:
apt update
apt install linux-image-5.15.0-1004-kvm linux-headers-5.15.0-1004-kvm linux-modules-5.15.0-1004-kvm linux-kvm-headers-5.15.0-1004
You can then go ahead and reboot it to make sure everything is working properly, but that’s not required.
Resume updates at a later time
When you’re ready to resume updates to the kernel (after a new version without the issue has been released), just reinstall the kernel meta packages:
apt install linux-kvm linux-image-kvm linux-headers-kvm
This will pull in the latest kernel and headers, and will also cause automatic updates to resume on the kernel.