Jetson Nano Module - Flashing, booting from NVME and more Turing Pi #│forum

Jetson Nano Module - Flashing, booting from NVME a...

mjptree

08/26/2023, 2:01 PM

In its default configuration the Jetson Nano module is perpetually deprived of sufficient storage. The full default install with all SDK components and software packages clocks in at just under 15 GB, leavig little to no space for maneouvering, let alone running arbitrary docker images. I have 2 Jetson Nano Modules: One 4 GB devkit module with SD card storage (P3448-0000), and then the non-devkit version with 16 GB embedded MMC storage (P3448-0002). I purchased two 1 TB NVME SSD cards early on, as I assumed from [this overview](https://docs.turingpi.com/docs/turing-pi2-specs-and-io-ports#mpcie-m2-nvme-sata-usb-ports-mapping) on the TP 2 docs that it is supported and would be straightforward to set up. But unless one is comfortable either just mounting 1 TB in random locations and using it only for personal storage or cleanly migrating an entire directory tree to the NVME device, it was inherently unsatisfying that the rest of the system was still squeezing itself onto the embedded limited storage. It took me almost 2 weeks to figure this one out, but I finally managed to get the Jetson Nano modules to mount the root filesystem onto the NVME drives during boot and use it as the main storage device. They still boot from the eMMC/SD card, but I am 100% fine with that. The caveat in this setup is that boot and kernel files are duplicated on the MMC storage and the NVME drive, but people cleverer than me can figure this one out.

mjptree

08/26/2023, 2:18 PM

I found the NVIDIA Jetson Linux Developer Guide (r32.7.3) and specifically the section about [setting up an NVME drive for booting](https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-3273/Tegra%20Linux%20Driver%20Package%20Development%20Guide/flashing.html#wwpID0E0PM0HA) an invaluable guide. TL;DR 1. Flash module normally, as you would without any other storage attached. 2. Ensure that you can boot normally from eMMC/SD card. 3. Power the module off, attach the nvme drive, then boot the module normally. 4. Ensure that you can see the nvme drive (e.g.

lsblk | grep nvme

). 5. Parition the drive as GPT. 6. Create the

APP

parition for the root filesystem (naming the partition

APP

correctly is important!). 7. Format partition as ext4. 8. Generate the rootfs on the host. 9. Mount the generate system image on the host, and the newly created nvme partition on the Jetson device. 10. Rsync the the system image with to the nvme drive.* 11. Unmount both the image on the host and the nvme partition on the device. 12. Reboot the device and open the uboot shell (requires a UART connection during boot). 13. Verify you can boot from the nvme drive and/or change the order of boot targets to move nvme drive to the top of the list. \ In the docs the setup of the nvme drive is performed on the host. This is not necessary. It is enough to generate the system image on the host and then rsync the image and the device over network. However, this requires ssh root login to be enabled for the duration of the setup.*

mjptree

08/26/2023, 2:28 PM

Also, for those wanting to setup a fully compliant k8s cluster, and wonder how to disable swapping: Swapping on (all) Jetsons should be configured via the script

/etc/systemd/nvzramconfig.sh

. To disable swapping and have this setting persist across restarts, verify that swapping is currently active with

Copy code

sudo systemctl status nvzramconfig.service

then disable this service via

Copy code

sudo systemctl disable nvzramconfig.service

As you can verify with

lsblk

and

free -h

this setting will not become effective until you reboot the device. To undo this change and reenable swapping, simple enable the service again via

Copy code

sudo systemctl enable nvzramconfig.service

and reboot.

DhanOS

08/26/2023, 2:39 PM

I'm using a similar setup but with the USB SSD drive. Indeed the

/boot

partition must reside on the internal storage (eMMC or SD) and you have to remember to sync the changes there (like when you install kernel modules, recompile the kernel, set the settings, etc.). What I haven't tried, but might work, is to mount the /boot partition from the eMMC.SD as

/root

(I do not recall why it is not)

CND

08/26/2023, 2:53 PM

I found this script to be most useful to boot and run from a NVME attached to my Nanos (with eMMC) and it was simple to use. https://github.com/hcfman/sbts-boot-from-SSD It leaves the eMMC untouched and creates an 'undo' script in your home dir on the Nano if you wanted to go back.

DhanOS

08/26/2023, 2:54 PM

I will later find how I did this and share here, but it was as simple as a few commands iirc

36 Views

Previous Next