Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to the kernel8.img by default when the page size is 16k bytes #4346

Closed
openoms opened this issue Jan 1, 2024 · 7 comments
Closed
Labels
bug - confirmed Something isn't working and can be reproduced final testing was fixed - needs testing RPi5

Comments

@openoms
Copy link
Collaborator

openoms commented Jan 1, 2024

Jan 01 18:21:03 raspberrypi systemd[1]: Started fulcrum.service - Fulcrum.
Jan 01 18:21:03 raspberrypi Fulcrum[1301304]: <jemalloc>: Unsupported system page size
Jan 01 18:21:03 raspberrypi Fulcrum[1301304]: <jemalloc>: Unsupported system page size
Jan 01 18:21:03 raspberrypi Fulcrum[1301304]: <jemalloc>: Unsupported system page size
Jan 01 18:21:03 raspberrypi Fulcrum[1301304]: terminate called without an active exception
Jan 01 18:21:03 raspberrypi systemd[1]: fulcrum.service: Main process exited, code=killed, status=6/ABRT
Jan 01 18:21:03 raspberrypi systemd[1]: fulcrum.service: Failed with result 'signal'.

Solution:
https://t.me/raspibolt/23047

Fix for Fulcrum on Pi5:
Add the following to /boot/config.txt
kernel=kernel8.img
This will change page size to 4k from 16k.

The kernel change cannot be applied without a reboot.

Not sure if it is the right solution yet as not solved the issue even after reboot.
See:
https://www.raspberrypi.com/documentation/computers/config_txt.html#kernel

The Raspberry Pi 5 firmware defaults to loading kernel_2712.img because this image contains optimisations specific to Raspberry Pi 5 (e.g. 16K page-size). If this file is not present then the common 64-bit kernel (kernel8.img) will be loaded instead.

Check the current page size with:

₿ getconf PAGE_SIZE
16384
@openoms openoms added bug - unconfirmed Something isn't working - not (yet) reproduced bug - confirmed Something isn't working and can be reproduced RPi5 and removed bug - unconfirmed Something isn't working - not (yet) reproduced labels Jan 1, 2024
@neversleep404
Copy link

neversleep404 commented Jan 2, 2024

I have a Pi 5, but I'm not familiar with the problem. Which module is encountering the issue? What module use Fulcrum?
(I'm running raspiblitz-arm64-rpi-image-2023-12-15-40587e4 with Kernel 6.1.0-rpi7-rpi-2712 and had to start the compilation myself for some modules like CLBOSS. Both Blitz API and WebUI aren't functioning and were generating a very high load (load of 5), but everything runs fine without them (load of 1.5).)

@openoms
Copy link
Collaborator Author

openoms commented Jan 3, 2024

This exposes a more widespread issue with a growing list of software not compatible with the 16k byte PAGE_SIZE: raspberrypi/bookworm-feedback#107

I suggest we should use the kernel8.img by default.

As it is a kernel change it should be part of the build_sdcard.sh.

@openoms openoms changed the title Fulcrum error on RPI5 Switch to the kernel8.img by default when the page size is 16k bytes Jan 3, 2024
@neversleep404
Copy link

I suggest we should use the kernel8.img by default.

As it is a kernel change it should be part of the build_sdcard.sh.

II also recommend defaulting to 'kernel8.img'. Switching from a 4K to a 16K page size kernel on a Raspberry Pi 5 yields a 7% performance boost in random memory access, but be aware of potential issues, especially for non-standard page sizes. This performance gain is more pronounced in intensive computing tasks, though it may not be noticeable in everyday use.

https://forums.raspberrypi.com/viewtopic.php?p=2172765#:~:text=The%20default%20kernel%20for%20a,increase%20in%20random%20memory%20accesses

@neversleep404
Copy link

neversleep404 commented Jan 3, 2024

Interestingly, I have noticed that the load on Kernel 6.1.0-rpi7-rpi-v8 under raspiblitz-arm64-rpi-image-2024-01-03-35d0c22 is 10-20% lower than on raspiblitz-arm64-rpi-image-2023-12-15-40587e4 with Kernel 6.1.0-rpi7-rpi-2712.
(Running CORE Lightning, CL BOSS, BTC Mempool Space, RTL. Debug for TOR and I2P off and maxuploadtarget=500000)

Possible explanation:
In less busy systems, it's possible that the average kernel load with 4k paging might be lower than with 16k paging. This difference can be attributed to the overhead associated with managing larger page sizes. With 16k pages, the kernel might handle fewer, larger pages, but the overhead of managing these pages might not always be efficient for workloads that don't fully utilize the larger page's capacity. In contrast, 4k pages are smaller and might be more efficiently used by the system, leading to potentially lower kernel load in scenarios with lighter workloads. It's important to note that the actual impact varies based on specific workloads, system configuration, and how the operating system manages memory. For precise effects, benchmarking in the specific environment is often necessary.

@openoms openoms added the final testing was fixed - needs testing label Jan 6, 2024
@openoms
Copy link
Collaborator Author

openoms commented Jan 6, 2024

agree that it could be interesting to do some benchmarking, but until the list of incompatible applications is so long it does not worth it to have the nonstandard 16K page size as default.

@neversleep404
Copy link

agree. The selection of 4K paging for operations brings many benefits, and for now, it's not beneficial to consider 16K paging.

@openoms openoms added this to the 1.11.0 Release milestone Jan 23, 2024
@rootzoll
Copy link
Collaborator

Closing for release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug - confirmed Something isn't working and can be reproduced final testing was fixed - needs testing RPi5
Projects
None yet
Development

No branches or pull requests

3 participants