Power & Source of Big Ideas

Boot with root on NVMe?

Moderators: chensy, FATechsupport

Any thoughts on how to boot with root on NVMe? I'm guessing NMVe isn't supported by RockChip bootloader for native NVMe boot - but has anyone figured out the magic to set the "root=/dev/nvme0p3" in the kernel command line?

I've tried systemd/switch-root booting, which would be an OK compromise solution and it "works" to get root onto the NVMe drive - but none of the network devices seem to work.

Really looking for a solution here as my application does a lot of logging and boot from eMMC isn't a good outcome. I know the NVMe is only 1 lane of PCIe v2, but speed isn't the issue I am trying to solve for. Endurance/reliability is. Especially important r5s has soldered eMMC and killing the eMMC on this board really means killing the whole board.
Found the method and am now happily running FriendlyCore, upgraded to Jammy with root on NVMe.

Performance is OK, much better than eMMC (~165MBps writes to NVMe vs ~45MBps to eMMC). I wrote before - it isn't about performance. Even low-end NVMe drives have much better endurance than eMMC or SD.

Makes this a very functional little box for me.
And what's the magic? Don't let other users die stupid
Frank_M wrote:
And what's the magic? Don't let other users die stupid

There are others here? Strange - none of them ever speak up and answer anything...
More seriously. An outline of the steps as I don't have time right now to spell out the detail.

It is a hack...major league hacky hack...

1. Use e-flashed and load FriendlyCore to the eMMC. DO NOT BOOT FROM THIS!
2. Flash FrendlyCore to a uSD card and boot from this.
3. Use your favorite flavor partition tool and delete partition 9 from the eMMC (this is the overlayFS partition - it will get in your way)
3. Power down, remove the uSD and reboot to the eMMC
4. Partition your NVMe drive and and format an ext4 filesystem I'll assume you use /dev/nvme0n1p1
5. Copy the rootfs to this new filesystem on the NVMe drive
6. Download a hex editor
7. Use the hex editor on partition 3 of the emmc card - find the boot command line and replace /dev/mmcblk2p8 with /dev/nvme0n1p1
8. reboot and be happy.
How did you partition the NVMe? I notice the SD Card and the eMMC have like 8 partitions? Are they all necessary? I'm hoping I can skip the eMMC step and go straight from SD Card to M.2. Can you just clone the SD onto the M.2?

I haven't entirely decided what dist I'll use yet. I just want to run Docker. I'm hoping I can plug in some USB drives and use bind mounts with Jellyfin. I want to run some webservers too.

I'm a total noob. I got my pi last night. So far I've just tested running Debian from an SD Card.
So, I installed my NVMe today and I cloned my SD Card to it like so...

# check out your drives first with
fdisk -l
# or

# copy away... it's best if you've used the smallest SD Card possible in this step.
dd if=/dev/mmcblk0 of=/dev/nvme0n1 bs=1M status=progress

# then expand the last partition... well, delete it and make a new big one
fdisk /dev/nvme0n1

# you can mount it and it still has data
mkdir /mnt/nvme0n1p9
mount /dev/nvme0n1p9 /mnt/nvme0n1p9
cd /mnt/nvme0n1p9

So, my file system looks good, I think. I have 464GB partition, but how to edit the bootloader/boot process? Right now it's booting to SD Card. As I understand, the NanoPi boots to SD Card if there is one or to eMMC if there is not. So somewhere on where it boots to, I have to tell it ok, now go to the NVMe and do your magic.

I can't mount p3 of the SD Card which is where I think I have to tell it to go to the NVMe to finish booting.

fdisk - says nvme0n1p9 is 464GB but LuCI says it's 1GB
wonderful. I tried the same technique with r6c. root has become nvme0n1p1, and the worry about capacity has disappeared.

thank you.

$ df
Filesystem 1K-blocks Used Available Use% Mounted on
tmpfs 810984 2168 808816 1% /run
/dev/nvme0n1p1 1983464612 3875348 1879553464 1% /
tmpfs 4057160 0 4057160 0% /dev/shm
tmpfs 5120 4 5116 1% /run/lock
tmpfs 811432 80 811352 1% /run/user/1000
I can't mount the partition:

root@FriendlyWrt:/mnt# mount /dev/mmcblk0p3 /mnt/mmcblk0p3
mount: /mnt/mmcblk0p3: wrong fs type, bad option, bad superblock on /dev/mmcblk0p3, missing codepage or helper program, or other error.

But I can extract rk3568-sd-friendlywrt-23.05-docker-20240606.img with 7-zip then edit 2.dtbo.img with ImHex.

I change:
root=/dev/mmcblk0p8 rw rootfstype=ext4

root=/dev/nvme0n1p8 rw rootfstype=ext4

# look for the 6D 6D...

m: 6D
m: 6D
c: 63
b: 62
l: 6C
k: 6B
0: 30

n: 6E
v: 76
m: 6D
e: 65
0: 30
n: 6E
1: 31

I have these files with the modifed 2.dtbo.img to boot from NVMe:


But now how do I remake the rk3568-sd-friendlywrt-23.05-docker-20240606.img ??
I made a new img with dd

sudo losetup -fP rk3568-sd-friendlywrt-23.05-docker-20240606.img
losetup -a
# check your loop number here and use it instead of 39
sudo dd if=2.dtbo.img of=/dev/loop39p3 bs=1M
ls -al

This new image is just to point the SD Card to the NVMe. We've just replaced partition 3 with the modified 2.dtbo.img

I think it has really worked.

    If the Pi isn't booting after installing NVMe try a better power supply.
    If your SD Card gets corrupted, stick it in a digital camera to format it.

Who is online

In total there is 1 user online :: 0 registered, 0 hidden and 1 guest (based on users active over the past 5 minutes)
Most users ever online was 5185 on Wed Jan 22, 2020 1:44 pm

Users browsing this forum: No registered users and 1 guest