Jetson Nano Module - Flashing, booting from NVME and more

Question

In its default configuration the Jetson Nano module is perpetually deprived of sufficient storage The full default install with all SDK components and software packages clocks in at just under 15 GB leavig little to no space for maneouvering let alone running arbitrary docker images I have 2 Jetson Nano Modules One 4 GB devkit module with SD card storage P3448 0000 and then the non devkit version with 16 GB embedded MMC storage P3448 0002 I purchased two 1 TB NVME SSD cards early on as I assumed from this overview https docs turingpi com docs turing pi2 specs and io ports mpcie m2 nvme sata usb ports mapping on the TP 2 docs that it is supported and would be straightforward to set up But unless one is comfortable either just mounting 1 TB in random locations and using it only for personal storage or cleanly migrating an entire directory tree to the NVME device it was inherently unsatisfying that the rest of the system was still squeezing itself onto the embedded limited storage It took me almost 2 weeks to figure this one out but I finally managed to get the Jetson Nano modules to mount the root filesystem onto the NVME drives during boot and use it as the main storage device They still boot from the eMMC SD card but I am 100 fine with that The caveat in this setup is that boot and kernel files are duplicated on the MMC storage and the NVME drive but people cleverer than me can figure this one out

mjptree · Answer

I found the NVIDIA Jetson Linux Developer Guide r32 7 3 and specifically the section about setting up an NVME drive for booting https docs nvidia com jetson archives l4t archived l4t 3273 Tegra 20Linux 20Driver 20Package 20Development 20Guide flashing html wwpID0E0PM0HA an invaluable guide TL DR 1 Flash module normally as you would without any other storage attached 2 Ensure that you can boot normally from eMMC SD card 3 Power the module off attach the nvme drive then boot the module normally 4 Ensure that you can see the nvme drive e g `lsblk | grep nvme` 5 Parition the drive as GPT 6 Create the `APP` parition for the root filesystem naming the partition `APP` correctly is important 7 Format partition as ext4 8 Generate the rootfs on the host 9 Mount the generate system image on the host and the newly created nvme partition on the Jetson device 10 Rsync the the system image with to the nvme drive 11 Unmount both the image on the host and the nvme partition on the device 12 Reboot the device and open the uboot shell requires a UART connection during boot 13 Verify you can boot from the nvme drive and or change the order of boot targets to move nvme drive to the top of the list In the docs the setup of the nvme drive is performed on the host This is not necessary It is enough to generate the system image on the host and then rsync the image and the device over network However this requires ssh root login to be enabled for the duration of the setup

mjptree · Answer

Also for those wanting to setup a fully compliant k8s cluster and wonder how to disable swapping Swapping on all Jetsons should be configured via the script ` etc systemd nvzramconfig sh` To disable swapping and have this setting persist across restarts verify that swapping is currently active with ```sudo systemctl status nvzramconfig service``` then disable this service via ```sudo systemctl disable nvzramconfig service``` As you can verify with `lsblk` and `free h` this setting will not become effective until you reboot the device To undo this change and reenable swapping simple enable the service again via ```sudo systemctl enable nvzramconfig service``` and reboot

_dhanos_ · Answer

I m using a similar setup but with the USB SSD drive Indeed the ` boot` partition must reside on the internal storage eMMC or SD and you have to remember to sync the changes there like when you install kernel modules recompile the kernel set the settings etc What I haven t tried but might work is to mount the boot partition from the eMMC SD as ` root` I do not recall why it is not

cnd747 · Answer

I found this script to be most useful to boot and run from a NVME attached to my Nanos with eMMC and it was simple to use https github com hcfman sbts boot from SSD It leaves the eMMC untouched and creates an undo script in your home dir on the Nano if you wanted to go back

_dhanos_ · Answer

I will later find how I did this and share here but it was as simple as a few commands iirc