Gentoo on Radxa Rock5B (aarch64)

Waiting for 9 months since ordering pre-order discount for Radxa’s ROCK 5B order of mine finally came. I had some hopes for this as on paper this looked as something that can be “good enough(tm)” for day-to-day desktop replacement for me. So as soon as it arrived the first order of business was to get Gentoo on it running. Following is guide for getting Gentoo installed on eMMC of ROCK5b while running other system from SD card.

Requirements

Radxa ROCK 5B board - I’m using the version with 16GB of RAM but other should work too
Linux system running on Radxa ROCK 5B board - you can download either Debian or Ubuntu image for SD card from Radxa’s website - both should fit 4GB or larger SD card
Internet connection - for getting files to install Gentoo and packages
Approximately 1 hours of time :) - there are 2 steps where you have to wait for compilation to finish (first ~10 minutes for packages and later ~23 minutes for first kernel compilation)
eMMC card (tested with 64GB version, smaller should be good too)

Installation

Boot into system installed on SD card or some other live Linux distribution.
- SD card boot has priority over eMMC.
- I will use Debian image that was copied onto SD card (16GB) using dd.

Partition the eMMC (this manual will assume the eMMC is /dev/mmcblk1) - GPT will be used.

p1 - /boot partion with 16MB offset from start of eMMC (u-boot and idbloader will be there)
p2 - Gentoo /

#lsblk /dev/mmcblk1
NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
mmcblk1 179:0    0 57.6G  0 disk
# parted /dev/mmcblk1 mklabel gpt
# parted /dev/mmcblk1 mkpart primary fat32 32768s 300M
# parted /dev/mmcblk1 name 1 boot
# parted /dev/mmcblk1 toggle 1 boot
# parted /dev/mmcblk1 mkpart primary 300M 100%
# parted /dev/mmcblk1 print
...
Partition Table: gpt
Disk Flags:
  
Number  Start   End     Size    File system  Name     Flags
 1      16.8MB  300MB   283MB                boot     boot, esp
 2      300MB   61.9GB  61.6GB               primary

# lsblk /dev/mmcblk1
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
mmcblk1     179:0    0 57.6G  0 disk 
|-mmcblk1p1 179:1    0  270M  0 part 
`-mmcblk1p2 179:2    0 57.3G  0 part 

Create file systems
- To be on safe side use FAT32 for /boot
- Using EXT4 for Gentoo / for simplicity
```
# mkfs.vfat -F32 -n boot /dev/mmcblk1p1
# mkfs.ext4 /dev/mmcblk1p2
```

Mount (future) Gentoo /, download stage3 and unpack it there - I will use systemd variant of stage3 from nearby mirror.

# mount /dev/mmcblk1p2 /mnt
# cd /mnt
# wget http://gentoo.osuosl.org/releases/arm64/autobuilds/20221030T233144Z/stage3-arm64-systemd-20221030T233144Z.tar.xz
# tar xvf stage3-arm64-systemd-20221030T233144Z.tar.xz

Mount /boot and other virtual file systems.

# mount /dev/mmcblk1p1 /mnt/boot
# mount -t proc none /mnt/proc
# mount --rbind /sys /mnt/sys
# mount --rbind /dev /mnt/dev

# df -h /mnt /mnt/boot /mnt/proc /mnt/sys /mnt/dev
Filesystem      Size  Used Avail Use% Mounted on
/dev/mmcblk1p2   57G  1.2G   53G   3% /mnt
/dev/mmcblk1p1  270M  4.0K  270M   1% /mnt/boot
none               0     0     0    - /mnt/proc
sysfs              0     0     0    - /mnt/sys
udev            7.6G  8.0K  7.6G   1% /mnt/dev

Chroot into future Gentoo /.

# cp -L /etc/resolv.conf /mnt/etc
# chroot /mnt /bin/bash
# env-update
# source /etc/profile

Configure /etc/fstab - use the UUIDs from your outputs (NOTE: UUID is not same thing as PARTUUID).

# blkid /dev/mmcblk1p1 /dev/mmcblk1p2
/dev/mmcblk1p1: LABEL_FATBOOT="boot" LABEL="boot" UUID="aaaa-aaaa" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="boot" PARTUUID="aaaaaaaa-1111-1111-1111-111111111111"
/dev/mmcblk1p2: UUID="bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb" BLOCK_SIZE="4096" TYPE="ext4" PARTLABEL="primary" PARTUUID="bbbbbbbb-0000-0000-0000-000000000000"
# echo 'UUID=bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb / ext4 defaults 0 0' >> /etc/fstab
# echo 'UUID=aaaa-aaaa /boot vfat noauto,noatime 0 0' >> /etc/fstab

Get portage tree.
```
# emerge-webrsync
```

Configure timezone.

# echo "Asia/Seoul" > /etc/timezone
# emerge --config sys-libs/timezone-data

Configure CPU flags for CPU (you can get these flags from app-portage/cpuid2cpuflags).

# echo '*/* CPU_FLAGS_ARM: edsp neon thumb vfp vfpv3 vfpv4 vfp-d32 aes sha1 sha2 crc32 v4 v5 v6 v7 v8 thumb2' > /etc/portage/package.use/00cpuflags

Configure /etc/portage/make.conf.
- set architecture matching the CPU (you can also use -march=native)
- set concurrency so we utilize all (8) CPU cores
```
# nano /etc/portage/make.conf
...
COMMON_FLAGS="-O2 -pipe -march=armv8.2-a+crypto+fp16+rcpc+dotprod"
...
MAKEOPTS="-j9
```
Emerge tools needed for building kernel (19pkgs at time of writing, compilation time ~10 minutes).
- to boot Gentoo sys-kernel/linux-firmware is not needed therefore lets disable firmware USE flag to save some space
- to generate kernel and initramfs I will use sys-kernel/genkernel
- bc is typically needed for kernel compilation
- dev-vcs/git will be used later to obtain kernel sources
- dev-embedded/u-boot-tools needed during dtbs generation (it contains mkimage command)
```
# echo 'sys-kernel/genkernel -firmware' > /etc/portage/package.use/01_basic_os
# emerge -av dev-vcs/git genkernel bc dev-embedded/u-boot-tools
```

Get Radxa’s Linux kernel source (5.10) - Note that it contains significant changes compared to mainline Linux kernel (plain gentoo-sources will NOT work)

# git clone --depth 1  https://github.com/radxa/kernel.git -b stable-5.10-rock5 /usr/src/linux-5.10-radxa-rock5
# eselect kernel list
Available kernel symlink targets:
  [1]   linux-5.10-radxa-rock5
# eselect kernel set 1
# du -sh /usr/src/linux-5.10-radxa-rock5
1.7G	/usr/src/linux-5.10-radxa-rock5

Get some reasonable .config to start with for kernel compilation. For this guide I will use my minimized kernel config.

# curl -o /root/2022-11-rock-5b-config.cfg https://www.famera.cz/blog/assets/files/gentoo-radxa-rock5b/2022-11-rock-5b-config.cfg

Configure genkernel.
- disable microcode loading as there is no support for it on ARM CPUs
- add callback that will compile DTB file for Radxa ROCK 5B board that is needed to boot system and that will be copied into /boot automatically same way as kernel and initramfs.
```
# nano /etc/genkernel.conf
...
MICROCODE="no"
CMD_CALLBACK="make dtbs; make dtbs_install"
```
Compile the kernel for the first time (~23 minutes).
- you can customize kernel config by adding --menuconfig option
- subsequent compilations are faster (~14 minutes) as genkernel caches built binaries for initramfs
```
# genkernel --kernel-config=/root/2022-11-rock-5b-config.cfg all
# du -sh /usr/src/linux-5.10-radxa-rock5
2.3G	/usr/src/linux-5.10-radxa-rock5
```

Create /boot/extlinux/extlinux.conf that acts as menu for bootloader.

kernel version name will be determined from file /usr/src/linux/include/config/kernel.release
edit UUID of Gentoo / to match your system

# mkdir /boot/extlinux/
# cat > /boot/extlinux/extlinux.conf <<EOF
label kernel-$(cat /usr/src/linux/include/config/kernel.release)
    kernel /vmlinuz-$(cat /usr/src/linux/include/config/kernel.release)
    initrd /initramfs-$(cat /usr/src/linux/include/config/kernel.release).img
    devicetreedir /dtbs/$(cat /usr/src/linux/include/config/kernel.release)
    fdtoverlays  /dtbs/$(cat /usr/src/linux/include/config/kernel.release)/rockchip/overlay/rk3588-uart7-m2.dtbo 
    append   root=UUID=bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb earlycon=uart8250,mmio32,0xfeb50000 console=ttyFIQ0 console=tty1 consoleblank=0 loglevel=0 panic=10 rootwait rw init=/sbin/init rootfstype=ext4 cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory swapaccount=1 irqchip.gicv3_pseudo_nmi=0 switolb=1 coherent_pool=2M
EOF

Verify content of /boot.

# cat /usr/src/linux/include/config/kernel.release
5.10.66-arm64-gc428536281d6
# cat /boot/extlinux/extlinux.conf
label kernel-5.10.66-arm64-gc428536281d6
    kernel /vmlinuz-5.10.66-arm64-gc428536281d6
    initrd /initramfs-5.10.66-arm64-gc428536281d6.img
    devicetreedir /dtbs/5.10.66-arm64-gc428536281d6
    fdtoverlays  /dtbs/5.10.66-arm64-gc428536281d6/rockchip/overlay/rk3588-uart7-m2.dtbo
    append   root=UUID=bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb earlycon=uart8250,mmio32,0xfeb50000 console=ttyFIQ0 console=tty1 consoleblank=0 loglevel=0 panic=10 rootwait rw init=/sbin/init rootfstype=ext4 cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory swapaccount=1 irqchip.gicv3_pseudo_nmi=0 switolb=1 coherent_pool=2M
# ls -l /boot/*
-rwxr-xr-x 1 root root  6436352 Nov  5 15:19 /boot/System.map-5.10.66-arm64-gc428536281d6
-rwxr-xr-x 1 root root  3516112 Nov  5 15:31 /boot/initramfs-5.10.66-arm64-gc428536281d6.img
-rwxr-xr-x 1 root root 27291656 Nov  5 15:19 /boot/vmlinuz-5.10.66-arm64-gc428536281d6
  
/boot/dtbs:
total 4
drwxr-xr-x 3 root root 4096 Nov  5 15:23 5.10.66-arm64-gc428536281d6
  
/boot/extlinux:
total 4
-rwxr-xr-x 1 root root 614 Nov  5 18:35 extlinux.conf

Download idbloader and u-boot image for ROCK 5B. These files can be found in Radxa’s ROCK 5B dl page, at the time of writing they are:

# curl -o /root/rock5b-idbloader.img https://dl.radxa.com/rock5/sw/images/loader/rock-5b/release/rock-5b-idbloader-g49da44e116d.img
# curl -o /root/rock5b-u-boot.itb https://dl.radxa.com/rock5/sw/images/loader/rock-5b/release/rock-5b-u-boot-g49da44e116d.itb

dd downloaded rock5b-idbloader.img and rock5b-u-boot.itb into space before first partition (offset are based on information from WIKI > ROCK 5 > Installation > Install the image to microSD).

# dd if=/root/rock5b-idbloader.img of=/dev/mmcblk1 seek=64
552+0 records in
552+0 records out
282624 bytes (283 kB, 276 KiB) copied, 0.0533153 s, 5.3 MB/s
# dd if=/root/rock5b-u-boot.itb of=/dev/mmcblk1 seek=16384
2638+0 records in
2638+0 records out
1350656 bytes (1.4 MB, 1.3 MiB) copied, 0.0658143 s, 20.5 MB/s

Verify the beginning of the /dev/mmcblk1 with hexdump.

just after partition table you should see the idbloader
after some space you should see the u-boot image
and after that first /boot partition with FAT file system starts

# hexdump -C /dev/mmcblk1|less
...
=== BEGIN IDB loader
00008000  52 4b 4e 53 00 00 00 00  80 01 02 00 01 00 00 00  |RKNS............|
00008010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
...
0004cf20  6f 6f 74 2d 6f 72 64 65  72 00 00 00 00 00 00 00  |oot-order.......|
0004cf30  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
=== END IDB loader
...
=== BEGIN u-boot
00800000  d0 0d fe ed 00 00 08 cb  00 00 00 38 00 00 08 08  |...........8....|
00800010  00 00 00 28 00 00 00 11  00 00 00 10 00 00 00 00  |...(............|
...
00949ba0  6f 6c 64 2d 6d 69 63 72  6f 76 6f 6c 74 00 00 00  |old-microvolt...|
00949bb0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
=== END u-boot
...
=== BEGIN 1st partition with FAT filesystem
01000000  eb 58 90 6d 6b 66 73 2e  66 61 74 00 02 01 20 00  |.X.mkfs.fat... .|
...

Configure root password, exit chroot and REBOOT
```
# passwd root
# exit
# reboot
```
Once system boots you should be able to login via console. From this point you can configure the Gentoo systems the usual way.

Lessons learned

there are fixed offsets for rockchip “bootloaders” that system tries to boot from and they are not visible as partitions (but maybe they could be exposed as partitions for better visibility)
Device-tree for the Rock5b seems to be automatically loaded by kernel without need to specify which exact file is DTB

Additional resources and future plans

Radxa Rock5B wiki: https://wiki.radxa.com/Rock5/5b
my future plan is to make this into usable desktop machine for day-to-day work as the performance of RK3588 seems to be enough for that
getting the UART console migt be beneficial for this board - when system boots there is nothing showing on monitor Through HDMI port until OS starts to boot

Final thoughts

After my previous attempt with Odroid’s M1 getting Gentoo running on this board was easier. However the hidden location of u-boot and idbloader was a major slowdown in figuring out why system doesn’t boot from eMMC. Also the lack of UART console makes boot troubleshooting harder.

In case you got stuck in the above procedure at some point feel free to drop me an email.