Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Image | Raspberry Pi 5: Testing and firmware migration script #6676

Open
TDuffinNTU opened this issue Oct 9, 2023 · 354 comments · Fixed by #6795 or #6892
Open

Image | Raspberry Pi 5: Testing and firmware migration script #6676

TDuffinNTU opened this issue Oct 9, 2023 · 354 comments · Fixed by #6795 or #6892

Comments

@TDuffinNTU
Copy link
Contributor

TDuffinNTU commented Oct 9, 2023

ADMIN EDIT

First Raspberry Pi 5 testing images are available now on our download page: https://dietpi.com/#download
You can also find images with the new kernel/firmware package set for other RPi models here: https://dietpi.com/downloads/images/testing/
The ones with the new firmware have "RPi1", "RPi2" and "RPi234" (64-bit) in their names. To migrate an existing system, use the migration script:

bash <(curl -sSf 'https://github.com/raw/MichaIng/DietPi/dev/.meta/dietpi-rpi-firmware-migration')

WARNING: This is currently a one-way ticket. dietpi-backup cannot be used to restore the old system, since the boot partition mount point has changed. If you want to be able to revert, create an image of the whole SD card/drive. dietpi-imager can be used from another DietPi (or Debian/Ubuntu) system, to minimise partition and filesystem automatically, to have a small compressed backup image.


Is the SBC officially supported by the Debian installer?

  • Yes(?) Looks like the board is inter-compatible with the same distro images as the 4B. That's pure speculation, mind you, but I did see a video of a reviewer transferring a Raspbian installation from a Pi4B to a Pi5 with no configuration necessary.

Notes

  • Not sure if this needs any specific developer legwork as the previous Pis seem to be inter-compatible with OS images for the most part (though I'm sure testing and drivers will be necessary!). Thought I'd make this issue now since it might be good to mark the Pi5 as provisionally supported on the website if it at least works out of the box.
@MichaIng MichaIng changed the title Raspberry Pi 5 Image | Raspberry Pi 5 Oct 10, 2023
@MichaIng
Copy link
Owner

MichaIng commented Oct 19, 2023

Okay this requires more work, sadly. With Bookworm and hence for RPi 5, the packages as well as the filesystem layout have changed dramatically:

  • For RPi 4 and earlier, the package is called linux-image-rpi-v8, which is a meta package and current pulls linux-image-6.1.0-rpi4-rpi-v8.
  • For RPi 5, the package is called linux-image-rpi-2712, which is a meta package and current pulls linux-image-6.1.0-rpi4-rpi-2712.
  • These packages are more streamlined with mainline Linux packages, installing kernel image, config, map, device trees and overlays like common Debian Linux packages:
    /boot/System.map-6.1.0-rpi4-rpi-2712
    /boot/config-6.1.0-rpi4-rpi-2712
    /boot/vmlinuz-6.1.0-rpi4-rpi-2712
    ...
    /usr/lib/linux-image-6.1.0-rpi4-rpi-2712/broadcom/bcm2712-rpi-5-b.dtb
    ...
    /usr/lib/linux-image-6.1.0-rpi4-rpi-2712/overlays/README
    
  • For the bootloader, there is now a raspi-firmware package, which installs the files as well more streamlined into /usr/lib/raspi-firmware/.
  • libraspberrypi0, libraspberrypi-bin and raspberrypi-sys-mods remain the same.
  • The RPi ROM bootloader stage still requires bootloader, config and kernel on a partition 1 with FAT filesystem, which was mounted to /boot. This cause the issue that, since dpkg does not support FAT, this rpikernelhack back and forth copying of all files in /boot was required, to bypass the need to dpkg to replace a file (which would fail). This procedure has now been replaced with another, IMO not less ugly workaround: The FAT filesystem of partition 1 is now mounted to /boot/firmware. The raspi-firmware package strictly requires this directory to be a mount point (now allowing a dir within a mount point) and then copies all kernel, dtb and bootloader files from /boot and /usr/lib/linux-image- over to /boot/firmware, applying the known naming schemes. So a copy back and forth of all those files has been replaced with a single copy, but files remaining permanently on both partitions. And of course, for the 64-bit kernel, it is now actually two kernel images and all related modules, hence disk space requirements have increased.
  • Furthermore, the raspi-firmware package does now strictly require an initramfs. I have to check whether this means that the kernel has essential modules no builtin anymore, or whether it is actually not required, but just installed and handled for in case. As far as I can see, there is a config flag to disable this copy to /boot/firmware stuff, which then would need to be done manually. So we could write our own script to do this, skipping the initramfs. But at least at the beginning, I am not keen to maintain such, as it will likely change a lot within the next months. Probably a tiny-initramfs is possible, like the moreless dummy one we do now use for Sparky SBC, one which does not include any kernel module, hence servers nearly no purpose (aside of supporting UUIDs on top of PARTUUIDs) aside of satisfying bootloader and/or package script expectations.
  • Same for the 32-bit image, with the same kernel packages names and suffixes: v6, v7, v7l and v8. But one quirk is that the linux-image-rpi-v8 packages are not contained within the armhf package list, only in the arm64 ones, which is not enabled by default on 32-bit userland.

@MichaIng MichaIng added this to the v8.24 milestone Oct 19, 2023
@Joulinar
Copy link
Collaborator

uhh does not sounds like a quick win. 🙄

@onurcoskun14
Copy link

Any updates?

@axxis-creator
Copy link

Same Question. :-) I buy a RPI-5 and want to install DietPi for LoxBerry.
When can we expect a version?

@gaurishhs
Copy link

any updates on this?

@MichaIng
Copy link
Owner

We will post any updates here when we have some. I was looking further into it. The migration of the partition and installation of the new kernel packages is not too hard. But there is another big problem: Our scripts, and probably RPi's own scripts/programs expect config.txt and cmdline.txt in /boot, while for booting, both must be located in /boot/firmware (with the new package set). This is solved in RPi OS via symlinks. The problem is now that sed -i naturally replaces symlinks with actual files, and we use it a lot to edit their contents. Hence we must replace every occurrence across our whole code with sed -i --follow-symlinks to avoid this. Not understandable why this is not the default since ages, since I cannot imagine any circumstance where one would want to have a symlink replaced with its target file, when using sed on a symlink. However, I can do this quickly with a single command for the whole repo with the use of ... sed 😃. But this needs to be carefully reviewed, since there are variances like sed -Ei and probably some cases or code comments where the replacement must not be applied.

Another problem or better question is what we do with /boot/dietpi and /boot/dietpi.txt. The first, I think is fine to stay on the ext4 partition in the first place, resp. being moved there as part of the migration. The dietpi.txt is however nice to be on the FAT partition, at least before first boot, so one can apply automation and pre-configure the image from Windows and macOS easily. We just added a trailing FAT partition to all other images to make it as easy there as on RPi, so it would be horrible if we now removed this possibility for RPi. Theoretically we could do it on RPi the same way we do on other SBCs: As part of the first boot rootfs expansion script, mount the FAT partition and copy over dietpi.txt, dietpi-wifi.txt and all other optional pre-config files to the rootfs, then unmount it again. This must of course only happen on those RPi images which have the FAT partition mounted to /boot/firmware already.

Btw, does someone of you guys have an RPi 5 already and can tell me the revision code? That way we can add support to our hardware detection script already and provide RPi 5 compatible images between releases, if I am not able to finish this until this Saturday (v8.24 release):

mawk '/^Revision/{print $3;exit}' /proc/cpuinfo

The 3rd and 2nd last characters define the model. "14" was CM4, so it is probably "15" or "16".

@jboots07
Copy link

C04170

@adrianog91
Copy link

adrianog91 commented Nov 16, 2023 via email

@MichaIng
Copy link
Owner

Thanks. Found it now here as well: https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#new-style-revision-codes

So "15" is the CM4S, which is a special industry customer variant, not publicly sold, and "16" for "internal use". Let's see whether "16" is a surprise, not announced yet 🙂.

@MichaIng
Copy link
Owner

First step: 6e848a1

@jboots07
Copy link

Thanks for your hard work and update! Let me know if I can help in any way, I have a pi 5.

@sorriso93
Copy link

I just received my PI5 and can't wait to install on it Dietpi to replace my PI4 :-)

@BugDave
Copy link

BugDave commented Nov 19, 2023

I just received mine too and am excited to put dietpi on it :-)

@MichaIng MichaIng modified the milestones: v8.24, v8.25 Nov 19, 2023
@m0jek
Copy link

m0jek commented Nov 21, 2023

@MichaIng when you are ready for testing in the wild on RaspberryPi 5, I am happy to test.

@FabioEight
Copy link

Available for testing on Pi 5 as well! Thanks for your efforts!

@elisenlebkuch
Copy link

Where can I get it?

@gaurishhs
Copy link

gaurishhs commented Nov 22, 2023

I don't think so there has been any image supporting Pi 5, You'll have to wait

@elisenlebkuch
Copy link

elisenlebkuch commented Nov 22, 2023

@gaurishhs
Copy link

I think that one is for Risc-V #6212

@LittleFreak
Copy link

LittleFreak commented Nov 26, 2023

Is it possible to add the dietpi later on a running raspbian?
I'll plan to docker all my programs anyway, so the host will stay pretty much raspbian lite until then.

If so, I'd just give my raspbian a diet later, when its compatible with raspi 5.

@MichaIng
Copy link
Owner

Is it possible to add the dietpi later on a running raspbian?

You mean on a Raspberry Pi OS? Currently does not work as dietpi-installer installs the old kernel/bootloader/firmware packages and our scripts use sed in various places a way that the /boot/config.txt => /boot/firmware/config.txt (same with cmdline.txt) symlinks are replaced with the actual files, so that the intended changes will not actually apply. See my longer comment above about this issue. Once we added compatibility, dietpi-installer can run on a new RPi OS image, but it will remove all installed software, including Docker in your case. What you could do is backup your /var/lib/docker. Then once dietpi-installer ran through and you boot DietPi, do not install Docker at first, but restore /var/lib/docker to /mnt/dietpi_userdata/docker-data, then install Docker via dietpi-software.

@cgmcfall
Copy link

Have a Pi5, happy to test :)

@alfredoanton82
Copy link

Also have a RPI5, happy to help testing.

@sergio-ingrao
Copy link

Here, I have another one ready to test.

@josh3003
Copy link

bash <(curl -sSf 'https://github.com/raw/MichaIng/Diet Pi/dev/.meta/dietpi-rpi-firmware-migration')
[FAILED] dietpi-rpi-firmware-migration | You can run this script on Debian Bookw orm or Trixie systems only!

@Joulinar
Copy link
Collaborator

As stated on the error. You would need to upgrade to Debian Bookworm as this would be required for running on RPi5.

@josh3003
Copy link

As stated on the error. You would need to upgrade to Debian Bookworm as this would be required for running on RPi5.

Thanks, in the end I was able to import/export my configs and loaded a fresh image. Probably for the best anyway and got it up and running. :)

@bwims
Copy link

bwims commented May 9, 2024

So if I get this right, I should be able to install this on a new RPi5 without any issues, correct?

Does anyone know if it works with the Pimoroni NVMe Base ?

Thanks!
B.

@MichaIng
Copy link
Owner

MichaIng commented May 9, 2024

With "this", you mean our dedicated RPi 5 image? Else after migrating a current image to the new firmware stack with our script, and selecting an RPi 5 compatible kernel to be installed, yes.

The NVMe board should work on that image just like it works on RPi OS, as both share the same bootlaader/kernel/firmware.

@bwims
Copy link

bwims commented May 9, 2024

It's a new RPi5 so doesn't need to migrate. I was assuming that a straight download + install should work, but I saw the caveats on the download page :-
Not all hardware features, especially overclocking, display and camera settings, can be controlled with our scripts yet.
so I wanted to check that the NVMe base would work ok.

Many thanks!

B.

@MichaIng
Copy link
Owner

MichaIng commented May 9, 2024

Yes, the RPi 5 image from our download page of course works on RPi 5 🙂.

NVMe support is not something that is/can be controlled by our script. It either boots/is detected, or not, based on bootloader and kernel. And everything that can currently not be controlled with out scripts, can instead be done manually via config.txt edits, just like on RPi OS. AFAIK, also raspi-config does not support any of these changes (display resolution/rotation and overclocking) for RPi 5 and KMS graphics stack, i.e. the RPi documentation applies the same for both.

@bwims
Copy link

bwims commented May 9, 2024

Many thanks for your help!

B.

@NiklasWallin
Copy link

Hello,
I just got a Raspberry pi 5, I tried to install the dietpi (test) image for the RP5 from the dietpi website.
When powering it up, the leds start to blink, but nothing happened on the screen.

I then tried to install dietpi on a raspberry 3b+ on the same SD-card, which succeeded.
When trying to run the migration script:
bash <(curl -sSf 'https://github.com/raw/MichaIng/DietPi/dev/.meta/dietpi-rpi-firmware-migration')

I got the following output on a fresh dietpi install on the Raspberry pi 3b+ :
dietpi@DietPi:$ sudo bash <(curl -sSf 'https://github.com/raw/MichaIng/DietPi/dev/.meta/dietpi-rpi-firmware-migration')
bash: /dev/fd/63: No such file or directory
dietpi@DietPi:
$ curl: (23) Failed writing body

What may be the issue here?
Should the RP5 test image be working at this moment, or is the migration script required to get dietpi on a RP5?
Thanks!

@MichaIng
Copy link
Owner

The test image should work. Do you have a UART adapter to check serial console output? And does RPi OS boot?

@NiklasWallin
Copy link

NiklasWallin commented May 11, 2024

Thanks for the quick answer.
Yes, I tried to flash raspbian on the SD-card and the RP5 started.
Currently the SD-card is in the RP3 (with dietpi), I will try to flash Dietpi again on for the RP5 and also try to get the uart log.
I will try it later today/tomorrow.

@NiklasWallin
Copy link

I did some additional investigation today. It seems that I had issues with the HDMI port. When testing Raspbian the monitor was working fine. However, when Dietpi was running, I could not manage to get the monitor to work.
I got the initial installation done with the root user through SSH, all good.
Cheers!

@MichaIng
Copy link
Owner

Probably KMS vs non-KMS makes a difference. Try to add dtoverlay=vc4-kms-v3d to config.txt and see whether this changes something.

In my case, it works well without KMS enabled, but it might depend on the monitor, and this is a significant difference between DietPi and RPi OS defaults.

@kris711
Copy link

kris711 commented May 24, 2024

grafik
After running the script on RPI3 I put SD Card to RPI 5 still I get the message
["Device-tree file "bcm2712-rpi-5-b.dtb" not found"] maybe I missed sth?
I am not sure if I understood it correctly but I do not have boot/firmware folder at all:
grafik

Thanks

@MichaIng
Copy link
Owner

@kris711 Next time (repeat), do not abort, but press ENTER to continue. Then you have a Debian Bookworm system. Next you need to run the migration script from first post, to actually migrate to the RPi kernel/firmware stack and install the RPi 5 kernel.

@Just2Confuse
Copy link

Im pretty sure this is not a Pi5 Problem? Or? DietPi (latest) was unexpected slower on a SSD, that all "writes".
#7092

@Joulinar
Copy link
Collaborator

@Just2Confuse pls have a look to your linked topic. There you got some questions.

@lukaszsobala
Copy link

I've tested the in-place upgrade script on rpi zero 2w and on a 4B. In both cases it went smoothly and with no issues.

BTW, you can also create an image of the boot drive straight from the running system. Just stop any services and use dd to copy the drive to a file (you can also pipe the dd output to xz to have a smaller image).

@MichaIng MichaIng modified the milestones: v9.5, v9.6 Jun 10, 2024
@MichaIng
Copy link
Owner

BTW, you can also create an image of the boot drive straight from the running system. Just stop any services and use dd to copy the drive to a file (you can also pipe the dd output to xz to have a smaller image).

At best, remount the filesystems R/O:

dietpi-services stop
swapoff -a
mount -o remount,ro /boot
mount -o remount,ro /

It sometimes keep complaining "drive is busy", in which case one would need to further check which process is keeping a file open in writable mode, but this is safest to avoid inconsistencies.

Or use dietpi-imager from another Debian-based Linux system 🧐:

bash <(curl -sSf 'https://github.com/raw/MichaIng/DietPi/master/.build/images/dietpi-imager')

This has the benefit, that the resulting image file does not have the size of the root drive, but filesystem and partition are minimised at first (and automatically expanded again on reboot), and the image further compressed. Would be actually an idea to allow running this on the origin system itself, where it could attempt the R/O remounts automatically. Of course, the image would then need to be stored to another drive.

@MichaIng MichaIng modified the milestones: v9.6, v9.7 Jul 10, 2024
@Trainax
Copy link

Trainax commented Jul 11, 2024

Hi everyone, sorry if this is a duplicate comment however I couldn't find what I am looking for. With summer approaching my Pi 5 with an official active cooler fan is running hotter than usual. When I log in using SSH the banner says something like:

CPU temp : 56 °C / 132 °F : Running warm, but safe

Is there any way to change the PWM fan profile to make it spin faster and keep the CPU temperature lower? I know it says "safe", however I don't see any reason why it shouldn't be allowed to run cooler if possibile. Inspired by this GitHub Gist I was able to make the fan spin faster by writing a higher number to /sys/class/thermal/cooling_device0/cur_state however it appears to be overwritten every few seconds so the change isn't permanent. I would really like to be able to configure the PWM profile to have something like this:

CPU temp (°C) cur_state Fan speed
> 65 4 FULL
55 - 65 3 HIGH
45 - 54 2 MEDIUM
35 - 44 1 LOW
< 35 0 OFF

Thank you and sorry again if this is duplicate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment