← Back to Home

Ask the Argus

Booting Sandbox Part II

Experimenting With initramfs

Synopsis

In this lab I proceed to the second task of Shevek's boot fault lab: Broken initramfs. The lab can be found at the bottom of this page for download or in the sandbox archive listed on the homepage. As is most always the case, I start completely blind and clueless as to what to do, or as to what initramfs really is for that matter. I have Shevek make me a lesson plan and follow along. Along the way we encounter limitations to the lab created by my virtual machine using a LVM, I discover how to repair a broken system in three different ways as a consequence, and a means of activating dracut for the simulation is eventually arrived at.

Questions & Objectives
  1. I want to understand what initramfs is and it's role in booting.
  2. I will to deliberately miscofigure initramfs to see it's effects.
  3. I will then reconfigure the system so that it is returned to normal operation.
Process
    1. As is always the case, since I didn't request a lesson with my lab, I don't get one. There is no context or guide stage in the lab. Just a few obscure reference to file-paths. I have Shevek make me a lesson plan so I have a shot at learning in this sandbox.

      What Did You Hand Me, Shevek?

      Sheveks Shitty Lab
      Sheveks Shitty Lab

      initramfs Lab 1

      initramfs Lab 1
      initramfs Lab 1

      initramfs Lab 2

      intramfs Lab 2
      intramfs Lab 2

      initramfs Lab 3

      initramfs Lab 3
      initramfs Lab 3

      initramfs Lab 4

      initramfs Lab 4
      initramfs Lab 4

      initramfs Lab 5

      initramfs Lab 5
      initramfs Lab 5

      initramfs Lab 6

      initramfs Lab 6
      initramfs Lab 6

      initramfs Lab 7

      initramfs Lab 7
      initramfs Lab 7

      initramfs Lab 8

      initramfs Lab 8
      initramfs Lab 8

      initramfs Lab 9

      initramfs Lab 9
      initramfs Lab 9
    2. In order to do the lab, I need to damage or remove the initramfs image, per what Shevek states. Unfortunately at no point in time does he make it apparent how I am supposed to do this. I consult with Shevek. Shevek, how do we destroy my virtual machine? He provides me three options for doing so.

      Damaging initramfs 1

      Damagin initramfs 1
      Damaging initramfs 1

      Damaging initramfs 2

      Damaging initramfs 2
      Damaging initramfs 2

      Damaging initramfs 3

      Damaging initramfs 3
      Damaging initramfs 3

      Damaging initramfs 4

      Damaging initramfs 4
      Damaging initramfs 4

      Damaging initramfs 5

      Damaging initramfs 5
      Damaging initramfs 5

      Damaging initramfs 6

      Damaging initramfs 6
      Damaging initramfs 6
    3. I proceed to set up the conditions for the fault. I follow Sheveks instructions to the detail. But, yet again, I encounter a limitation to the lab. Despite having followed instructions and simply changed the name of initramfs, I am not yielding an emergency shell. The screen sits idly. I attempted a force restart to no avail. Shevek goes on to explain we created an even more catastrophic failure than just initramfs. The Kernel can not even start the initramfs environment. Not to worry, there is a solution.

      Changing the Name of initramfs

      Name Change initramfs
      Name Change initramfs

      Rebooting 1

      Rebooting 1
      Rebooting 1

      Rebooting 2

      Rebooting 2
      Rebooting 2

      Rebooting 3

      Rebooting 3
      Rebooting 3

      Booting Fault Explained 1

      Booting Fault 1
      Booting Fault 1

      Booting Fault Explained 2

      Booting Fault 2
      Booting Fault 2

      Booting Fault Explained 3

      Booting Fault 2
      Booting Fault 2

      Booting Fault Explained 4

      Booting Fault 3
      Booting Fault 3

      Booting Fault Explained 5

      Booting Fault 4
      Booting Fault 4

      Booting Fault Explained 6

      Booting Fault 5
      Booting Fault 5
    4. I follow the steps Shevek lays out for me and it works. When prompted in the GRUB menu to select a Kernel, I select the one whose initramfs has not been corrupted. I'm not certain HOW this gives me access to my directory, but it works. I change the name of initramfs back to it's original state. I reboot, select the original Kernel and it works as it did before.

      Using the "Rescue" Kernel 1

      Rescue Kernel 1
      Rescue Kernel 1

      Using the "Rescue" Kernel 2

      Rescue Kernel 2
      Rescue Kernel 2

      Using the "Rescue" Kernel 3

      Rescue Kernel 3
      Rescue Kernel 3

      Rebooting With the Original Kernel

      Original Kernel
      Original Kernel

      Booting Success

      Booting Success
      Booting Success
    5. I feel a little shaky on what a driver is and, seeing as to how they appear to play such a vital role in booting, I ask Shevek to elaborate on what they are. I appear to have the correct general understanding, but as always, Shevek refines it.

      Drivers Explained 1

      Drivers 1
      Drivers 1

      Drivers Explained 2

      Drivers 2
      Drivers 2

      Drivers Explained 3

      Drivers 3
      Drivers 3

      Drivers Explained 4

      Drivers 4
      Drivers 4
    6. I return to the initramfs lab, prompt Shevek that I have reconfigured my VM to boot correctly, and proceed. I employ the command sudo dracut --omit lvm --force. This creates a new initramfs, leaves out the LVM module, and will cause boot to fail into dracut. I will ask for more clarification on how that command works in the next section.

      Forcing dracut 1

      Forcing dracut 1
      Forcing dracut 1

      Forcing dracut 2

      Forcing dracut 2
      Forcing dracut 2

      Forcing dracut 3

      Forcing dracut 3
      Forcing dracut 3

      Forcing dracut 4

      Forcing dracut 4
      Forcing dracut 4

      Forcing dracut 5

      Forcing dracut 5
      Forcing dracut 5

      Forcing dracut 6

      Forcing dracut 6
      Forcing dracut 6

      Forcing dracut 7

      Forcing dracut 7
      Forcing dracut 7

      Forcing dracut 8

      Forcing dracut 8
      Forcing dracut 8
    7. The command sudo dracut --omit lvm --force leaves a thorn in my side. I am not the type of person to blindly type commands I have no comprehension of. I also know philosophy majors that would struggle to comprehend a man page. I consult Shevek to break down the command, piece by piece.

      Rebuilding initramfs 1

      initramfs Rebuild 1
      initramfs Rebuild 1

      Rebuilding initramfs 2

      initramfs Rebuild 2
      initramfs Rebuild 2

      Rebuilding initramfs 3

      initramfs Rebuild 3
      initramfs Rebuild 3

      Rebuilding initramfs 4

      initramfs Rebuild 4
      initramfs Rebuild 4

      Rebuilding initramfs 5

      initramfs Rebuild 5
      initramfs Rebuild 5
    8. I return to Sheveks new lab and proceed with his instructions from dracut. However, I encounter yet antoher speed bump. None of the commands I'm required to use work. What's going on here?

      New Lab Process 1

      New Lab 1
      New Lab 1

      New Lab Process 2

      New Lab 2
      New Lab 2

      New Lab Process 3

      New Lab 3
      New Lab 3

      New Lab Process 4

      New Lab 4
      New Lab 4

      New Lab Process 5

      New Lab 5
      New Lab 5

      New Lab Process 6

      New Lab 6
      New Lab 6

      Minor Command Issues

      No Commands
      No Commands
    9. Shevek comes up with a way around this, or so I think. He instrcuts me to call commands directly by their PATH. I try and do this, but that doesn't even work. Something is seriously wrong here. Shevek keeps overlooking details. This lab only has experiential and exposure value. Nothing about it is working the way it's supposed to.

      Commands by PATH 1

      Command by PATH 1
      Command by PATH 1

      Commands by PATH 2

      Command by PATH 2
      Command by PATH 2

      Commands by PATH 3

      Command by PATH 3
      Command by PATH 3

      Still Not Working, Shevek

      No Commands
      No Commands
    10. Not to worry, Shevek has another solution that sure to at least bring my system back to it's original state AND he has amother method for simulating failure to dracut. I could just use snapshot, but I would like to learn how to recover the system in such a situation. We disabled the LVM. It contains all the commands. How did Shevek not see this coming? Or is he deliberately doing this? I wonder. I follow his lead and boot from CD-ROM.

      System Rescue Via CDROM 1

      System Rescue 1
      System Rescue 1

      System Rescue Via CDROM 2

      System Rescue 2
      System Rescue 2

      System Rescue Via CDROM 3

      System Rescue 3
      System Rescue 3

      System Rescue Via CDROM 4

      System Rescue 4
      System Rescue 4

      System Rescue Via CDROM 5

      System Rescue 5
      System Rescue 5

      System Rescue Via CDROM 6

      System Rescue 6
      System Rescue 6

      System Rescue Via CDROM 7

      System Rescue 7
      System Rescue 7

      System Rescue Via CDROM 8

      System Rescue 8
      System Rescue 8

      System Rescue Via CDROM 9

      System Rescue 9
      System Rescue 9

      System Rescue Via CDROM 10

      System Rescue 10
      System Rescue 10
    11. Before I proceed with using chroot in order to reapir initramfs, I would like to better understand what it is exactly it does, and what it is used for. Shevek explains chroot exists for just such a purpose.

      chroot Explained 1

      chroot 1
      chroot 1

      chroot Explained 2

      chroot 2
      chroot 2

      chroot Explained 3

      chroot 3
      chroot 3

      chroot Explained 4

      chroot 4
      chroot 4

      chroot Explained 5

      chroot 5
      chroot 5

      chroot Explained 6

      chroot 6
      chroot 6
    12. I attempt Sheveks final lab with tremendous success. dracut loads as required. I am then able to activate LVM manually to access the tools I need. I am then able to mount root, prep chroot, chroot and rebuild initrafms. It took a long time to get here. I still have questions.

      initramfs Recovery 1

      initrafms Recovery 1
      initrafms Recovery 1

      initramfs Recovery 2

      initrafms Recovery 2
      initrafms Recovery 2

      initramfs Recovery 3

      initrafms Recovery 3
      initrafms Recovery 3

      initramfs Recovery 4

      initrafms Recovery 4
      initrafms Recovery 4

      initramfs Recovery 5

      initrafms Recovery 5
      initrafms Recovery 5

      initramfs Recovery 6

      initrafms Recovery 6
      initrafms Recovery 6

      initramfs Recovery 7

      initrafms Recovery 7
      initrafms Recovery 7

      initramfs Recovery 8

      initrafms Recovery 8
      initrafms Recovery 8

      initramfs Recovery 9

      initrafms Recovery 9
      initrafms Recovery 9

      initramfs Recovery 10

      initrafms Recovery 10
      initrafms Recovery 10

      initramfs Recovery 11

      initrafms Recovery 11
      initrafms Recovery 11

      initramfs Recovery 12

      initrafms Recovery 12
      initrafms Recovery 12

      initramfs Recovery 13

      initrafms Recovery 13
      initrafms Recovery 13

      initramfs Recovery 14

      initrafms Recovery 14
      initrafms Recovery 14

      initramfs Recovery 15

      initrafms Recovery 15
      initrafms Recovery 15
    13. I'd love to say I know what I just did, but I simply don't understand a lot of it. I consult Shevek regarding what's going on with the mount commands. He explains why they are neccessary before we can employ chroot.

      mount Commands Explained 1

      mount explained 1
      mount Explained 1

      mount Commands Explained 2

      mount explained 2
      mount Explained 2

      mount Commands Explained 3

      mount explained 3
      mount Explained 3

      mount Commands Explained 4

      mount explained 4
      mount Explained 4

      mount Commands Explained 5

      mount explained 5
      mount Explained 5

      mount Commands Explained 6

      mount explained 6
      mount Explained 6
    14. As I will be making a habit of doing to make certain I'm actually learning something, I repeat back to Shevek my new found understanding of to the objectives stated at the beginning of the lab. For a change, I am mostly right.

      I still appreciate how Shevek refines what I've learned so it's more comprehensible.

      Objectives Learned 1

      Objectives Learned 1
      Objectives Learned 1

      Objectives Learned 2

      Objectives Learned 2
      Objectives Learned 2

      Objectives Learned 3

      Objectives Learned 3
      Objectives Learned 3

      Objectives Learned 4

      Objectives Learned 4
      Objectives Learned 4

      Objectives Learned 5

      Objectives Learned 5
      Objectives Learned 5
Conclusions

Corrupting initrafms is no simple task when an LVM is involved. After three tries, Shevek and I managed to get the behavior of dracut we were looking for. Renaming initramfs only resulted in the Kernel being unable to locate initrafms. This resulted in no emergency shell and required the use of a rescue Kernel to salvage the system. Next, we tried muting the LVM at boot. This, however, resulted in having tools that existed somewhere on disc, but were unreachable nonetheless. This much more severe fault required booting from CDROM. Lastly, the effective method employed to get the desired results was modifying GRUB at boot.

Shevek's Haiku

Tiny root in RAM
Drivers waiting for first breath
Kernel finds its way

Sandbox