If you'd like to add an M.2 NVMe SSD to your RPi5, you can order one here:
RASPBERRY-PI SC1166 RPI 5 M.2 HAT
0 In stock
Non-Cancelable / Non Returnable
Price for each: $37.63
-----
Sneak Peek at Booting Raspberry Pi 5 From an NVMe SSD
By Ash Hill published 2 days ago
Place your bets for the Pi bootup race.
Raspberry Pi
(Image credit: Jeff Geerling)
One spec on the Raspberry Pi 5 that the community is eagerly buzzing about
is the new PCIe port. This enables compatibility with a variety of devices,
most notably SSD support. Today, we¡¯re taking a look at maker Jeff Geerling
and his experience with booting the Pi 5 using an NVMe SSD. Geerling was
given access to a PCIe breakout board by the Raspberry Pi engineering team,
but there will be an M.2 HAT addon coming in 2024. The addon will bring an
M.2 connection for a 2242 NVMe SSD.
There are multiple benefits to using an SSD with the Pi 5. One of the most
obvious benefits is the performance boost you get using an SSD over a
microSD card. Being able to access data quickly is a game changer if you¡¯re
using your Pi to access media on the regular. But if you want to take things
further, booting the Pi off an SSD allows the entire operating system to
perform with the speed of an SSD rather than just the SD card.
Setting up the Pi to boot off an NVMe SSD is simple and Jeff provides lots
of details explaining how to make it work. The only way you can use the SSD
to specifically boot is to enable the function first. This can be done by
adjusting the boot order configuration file on the Pi. Once it¡¯s set to
point to the SSD, the Pi will check it first when booting and run the OS if
it¡¯s installed.
Raspberry Pi
(Image credit: Jeff Geerling)
The external PCIe port also needs to be manually enabled. With it enabled
and the boot device order set properly, it¡¯s possible to boot from the SSD.
You will need a valid partition, though, and the Raspberry Pi Imager is only
set up to flash microSD cards.
To set up the SSD with Raspberry Pi OS, you will need to flash the OS onto a
microSD card. At this point, you can clone the boot volume off microSD card
and copy the data over to the SSD. As long as the SSD is connected directly
to the PCIe port, it will boot. That said, Geerling was unable to boot the
Pi from an SSD if it was connected behind a bridge or switch.
It's also worth keeping in mind that you won't get the full potential speed
of most SSDs when connecting to the Pi 5, as there is just a single PCIe 2.0
lane available. In theory, that gives you about 500MBps of PCIe bandwidth.
So you're going to top out at roughly the performance of a fast SATA SSD.
To get a closer look at this Raspberry Pi project in action, check out the
full post shared by Jeff to his blog. If this project is up your alley, you
should be sure to follow Jeff Geerling for more cool Pi projects as well as
any future creations.
------
NVMe SSD boot with the Raspberry Pi 5
October 21, 2023
Pi 5 PCIe NVMe Kioxia XG8 SSD
In my video about the brand new Raspberry Pi 5, I mentioned the new external
PCIe port makes it possible to boot the standard Pi 5 model B directly off
NVMe storage¡ªan option which is much faster and more reliable than standard
microSD storage (even with industrial-rated cards!).
Enabling NVMe boot is pretty easy, you add a line to /boot/config.txt,
modify the BOOT_ORDER in the bootloader configuration, and reboot!
Of course, you'll also need to get Pi OS onto the NVMe, and there are a few
ways to do that¡ªI'll walk you through my favorite method below.
Note: Raspberry Pi announced they are developing an official Raspberry Pi
M.2 HAT, but there is no word on a launch date for it yet. I am also
tracking other PCIe HATs for Raspberry Pi 5 here.
Enable the external PCI Express port
First, enable the external PCIe port on the Raspberry Pi 5. Edit
/boot/config.txt and add the following at the bottom:
# Add to bottom of /boot/config.txt
dtparam=pciex1
# Note: You could also just add the following (it is an alias to the above
line)
# dtparam=nvme
# Optionally, you can control the PCIe lane speed using this parameter
# dtparam=pciex1_gen=3
I have the pciex1_gen=3 part commented out above because Raspberry Pi allows
you to tweak the bus speed (you can choose Gen 1 for 2.5 GS/s, Gen 2 for 5
GS/s, and Gen 3 for 8 GS/s), but the port is only rated for up to PCIe Gen 2
speeds.
In practice, I have been able to run multiple NVMe SSDs at Gen 3.0 speed
(getting up to 900 MB/sec) on my alpha Pi 5, but YMMV¡ªPCIe can be very
fickle, depending on the quality of the FFC cable and connections on your
own setup.
Set NVMe early in the boot order
The PCIe connection should work after a reboot, but your Pi won't try
booting off an NVMe SSD yet. For that, you need to change the BOOT_ORDER in
the Raspberry Pi's bootloader configuration:
# Edit the EEPROM on the Raspberry Pi 5.
sudo rpi-eeprom-config --edit pieeprom.bin
# Change the BOOT_ORDER line to the following:
BOOT_ORDER=0xf416
# Press Ctrl-O, then enter, to write the change to the file.
# Press Ctrl-X to exit nano (the editor).
Read Raspberry Pi's documentation on BOOT_ORDER for all the details. For
now, the pertinent bit is the 6 at the end: that is what tells the Pi to
attempt NVMe boot first!
Reboot your Raspberry Pi 5 to make the change take effect.
NVMe boot won't work unless you have the external PCI Express port enabled,
and there's a working NVMe drive with a valid boot partition! If you don't
have that (e.g. you used Raspberry Pi Imager with an external USB NVMe
adapter to flash Pi OS to an NVMe drive from another computer), then follow
the steps in the next section to clone your existing Pi OS install to an
NVMe SSD.
Clone your microSD boot volume to an NVMe SSD
Assuming you already have Raspberry Pi OS on a microSD card that is booting
your Raspberry Pi 5 internally, and the NVMe SSD is connected and visible
(check if you see a device /dev/nvme0n1 after running lsblk), you can use
rpi-clone to clone the internal microSD boot volumes to your NVMe SSD:
# Install rpi-clone.
git clone -b 123-nvme
cd rpi-clone
sudo cp rpi-clone rpi-clone-setup /usr/local/sbin
# Clone to the NVMe drive (usually nvme0n1, but check with `lsblk`).
sudo rpi-clone nvme0n1
Note: I'm using my fork of rpi-clone, because the official version has not
merged NVMe support yet.
NVMe behind a PCIe bridge / switch
Currently the Raspberry Pi 5 only exposes one PCIe lane externally¡ªthough
there are four more lanes taken up by the RP1 chip. Typical PC motherboards
have a number of lanes to play with, so you often find two, three, or even
four M.2 NVMe slots on high-end motherboards.
Even there, some motherboards have PCI Express switches (or 'bridges') which
allow multiple PCIe devices to share the same lane or lanes, in a similar
way an Ethernet switch can allow multiple computers to share a single
network connection.
NVMe boot behind PCIe switch on Raspberry Pi
On the Compute Module 4, bootloader space constraints prevented NVMe boot if
you used a switch, but I wonder if that restriction is lifted on the
Raspberry Pi 5¡ªand if so, is it already implemented?
As of now, no. I can see and use an NVMe SSD through a PCIe switch, but I am
not able to boot the Raspberry Pi 5 from it, unless it is directly connected
(as the lone PCIe device on the bus).
I've opened an issue to ask about this feature in the Raspberry Pi firmware
repo: Can't boot Pi 5 via NVMe behind PCIe switch / bridge.
Further reading
The Raspberry Pi can boot off NVMe SSDs now
Testing PCIe on the Raspberry Pi 5
Forcing PCI Express Gen 3.0 speeds on the Pi 5
raspberry pi pi 5 nvme sbc boot eeprom performance
Add new comment
Comments
Steve ¨C 3 days ago
Well done Jeff, I don't know where you find the time! Hopefully we will see
some decent Pi5 nvme NAS hardware if PCIe switching is reliable.
My Pi5 4GB arrived yesterday, so retail has started shipping - at least in
the UK.
I am seeing some spurious nvme boot disk issues (the same nvme to USB
adapter and SSD from my Pi4). I'm investigating root cause and will open an
issue if I can't fix it.
reply
Andy W ¨C 2 days ago
In reply to Well done Jeff, I don't know¡ by Steve
Nice to hear your Pi5 has turned up. My 8GB is still sitting at ¡°waiting to
be picked¡± at Pimoroni. I was hoping it would at least ship on the 23rd.
reply
Karl Miller ¨C 2 days ago
I wonder if it is possible to bridge two raspberry pi 5s together via their
pcie buses?
The purpose would be to have one running Android TV and the other running
Hyperbian? That might avoid having to connect an external video
decoder/encoder for ambient style lighting for the LED strip around the rim
of the TV.
reply
anon ¨C 2 days ago
nice work bro
reply
Massimiliano ¨C 2 days ago
Hi Jeff,
can you suggest me, please, what I have to buy to attach an SSD NVMe to the
Raspberry Pi (and boot the OS from the NVMe)? What kind of device are using
you?
Regards
Massimiliano
reply
Anonymous ¨C 20 hours ago
Nice one!
I'm currently using an SSK M.2 NVME SATA Adapter plugged into the USB 3.0
port with a WD Blue SA510 2TB M.2 SATA SSD (Prime day sale purchase). It was
not happy at first, but after a little trial and error it is working
perfectly. The USB speeds are amazing, copying files of my Pi 4 takes hours
to a USB memory stick, then literally seconds or minutes to copy from the
drive to my Pi 5.
I can't wait for an official PCIe M.2 adapter, hopefully they are wallet
friendly. Currently transferring some of my old Docker containers from the
Pi 4 to the Pi 5 (mainly for the extra storage and speed for Jellyfin and
NextCloud, my Pi 4 has a 500GB M.2 SSD in the Argon ONE M.2).
reply