Hi everybody, does anyone happen to have a gist or some public docs on how to get system extensions working with Talos on RK1? I'm running

v1.7.5

of [this patched fork of Talos](https://github.com/nberlee/talos) and spent 2-3 hours today trying to get the

siderolabs/iscsi-tools

and

siderolabs/util-linux-tools

running, but I don't really understand the sequence of steps required to build the needed custom image. I tried following the [official Talos docs](https://www.talos.dev/v1.7/talos-guides/configuration/system-extensions/), but I'm either missing something or just straight up don't understand what's written there 😄 Any tip would be great!

I am running Longhorn on my Talos cluster which seems to have similar requirements. First thing is building the image:

docker run --rm -t \
                                            -v $PWD/_out:/out \
                                            --network=host \
                                            -v /dev:/dev \
                                            --privileged \
                                            ghcr.io/nberlee/imager:v1.7.5 metal \
                                            --arch arm64 \
                                            --overlay-name turingrk1 \
                                            --overlay-image ghcr.io/nberlee/sbc-turingrk1:v1.7.5 \
                                            --base-installer-image ghcr.io/nberlee/installer:v1.7.5-rk3588 \
                                            --system-extension-image ghcr.io/nberlee/rk3588:v1.7.5@sha256:ba2b580d472c3a3fdaadf13943425b557d834b5d713ef6fe12cb36b691660487 \
                                            --system-extension-image ghcr.io/siderolabs/iscsi-tools:v0.1.4@sha256:717337c7437f0cf22124ecd86e89602b54aeae9938e8656d18cefb6c60d2410a \
                                            --system-extension-image ghcr.io/siderolabs/util-linux-tools:2.40.1@sha256:0f9a492b52ee161c192a7be5f99c7efe0b6d3fa8455e516149eaf76c48cb39c7

thanks a ton, on first glance it looks like I omitted one or two needed parameters to the builder command 🙂 I'll try that tomorrow!

Sorry, got a 50:50 chance copying this from my terminal history and I guess I picked the wrong one. Can you try this instead? I can't verify because my Lima VM doesn't come up properly right now, but this should work...

docker run --rm -t \
  -v $PWD/_out:/out \
  ghcr.io/nberlee/imager:v1.7.5 installer \
  --arch arm64 \
  --board turing_rk1 \
  --platform metal \
  --base-installer-image ghcr.io/nberlee/installer:v1.7.5-rk3588 \
  --system-extension-image ghcr.io/nberlee/rk3588:v1.7.5@sha256:ba2b580d472c3a3fdaadf13943425b557d834b5d713ef6fe12cb36b691660487 \
  --system-extension-image ghcr.io/siderolabs/iscsi-tools:v0.1.4@sha256:717337c7437f0cf22124ecd86e89602b54aeae9938e8656d18cefb6c60d2410a \
  --system-extension-image ghcr.io/siderolabs/util-linux-tools:2.40.1@sha256:0f9a492b52ee161c192a7be5f99c7efe0b6d3fa8455e516149eaf76c48cb39c7

You can use the deprecated way which is the easiest by adding iscsi to the extensions in the machine config. You can mix and match extensions from sidebar as long it doesn't have a kernel module and you add the rk3588 of my repo to the list Kernel Modules of siderolabs will not work as its a different kernel signing key and you need to add the rk3588 from my repo as it is removed when you manually add any extension in the machine configuration.

Thanks to this thread! I managed to get additional extensions set up on my RK1 Talos Cluster. Are there any plans to get the RK1 into the list of supported SBC's on https://factory.talos.dev ?

I only realized this recently when I actually tried using the Drbd module and went down a long rabbit hole of debugging... I hope there'll eventually be a method for signing the modules, Although, I get that making the key available is a security issue. So I guess it would have to be part of the custom kernel build process 😔

+1, I also would like to know how to push for getting the RK1 to the supported list @ https://factory.talos.dev/ or if there are plans already? Any obstacles? I have managed to get a cluster up and running using https://github.com/bguijt/turingpi2/tree/main/projects/talos/shell , but of course now when 1.8.0 soon is coming the factory way is sooo much easier and better (and supported).

Thanks for the repo link. Debating about moving off Ubuntu + k3s to talos. Did you get talos 1.8 running bychance? Also did you have any issues with Cilium?

No so far I have not upgraded to 1.8. Yes I had some issues with Cilium as I’ve not been using BPF before and did not really need it. Moving off it to use Metallb instead, but I’m not fully migrated to it yet.

I faced a few issues mostly related to the fact hairpinning is an issue with their ingress/gateway controllers and had to disable some features.

Great of you to share that. I’ll see if I might use your findings too, as I have had issues with ingress/gateway too even though I got it working good enough for my use cases.

Nope

I see. That is a shame!

@Nico if I wanna use your sbc overlay but build a 6.10 kernel would it just work? I did that with the RPi overlay and worked just fine but you mentioned your overlay patched the kernel so I am not 100% sure. Just wanna have an idea before I spend too much time on this 🙂 Also a mega thank you for the work you are doing here!

@User The overlay needs to have a patched DTB which I copy from the compiled kernel image. So if you build a new kernel, you should at least either: clone my repo and change https://github.com/nberlee/sbc-turingrk1/blob/main/installers/pkg.yaml#L8 or: build an image like:

FROM ghcr.io/nberlee/sbc-turingrk1:v<TalosVersion>
COPY --FROM=your_kernel_image /dtb/rockchip/rk3588-turing-rk1.dtb /arm64/dtb/rockchip/

But if you build your own kernel, please think about modules. As modules needs to be in /lib/modules/ therefore, you need to build an extension, or just make everything you need build in

Thanks Nico! This seems a good side project to experiment and learn new things, I might do this over the Xmas break perhaps taking inspiration from your work for sure! The CM4 was an easy enough work but I just used the mainline kernel as is (I think) and that was a big advantage

@CamSab for Cilium netkit?

My CM4 is in slot3 (wanted to use the SATA ports with that) and needed a newer kernel to boot (there is an HW issue that will not make it boot with the edfault talos kernel verison and needed 6.8 if I recall correctly) it so I just re-compiled the whole thing

Sorry, was out sick for a while. I don’t think I followed those instructions since I had installed Longhorn already a few times on other clusters. That said, I briefly looked at the docs and they don’t look much different than what I did.

nw i was able to install it

How?

followed this guide. https://longhorn.io/docs/1.7.2/advanced-resources/os-distro-specific/talos-linux-support/#requirements