https://turingpi.com logo
Join Discord
Powered by
Jetson Nano Module - Flashing, booting from NVME a...
# │forum
m
In its default configuration the Jetson Nano module is perpetually deprived of sufficient storage. The full default install with all SDK components and software packages clocks in at just under 15 GB, leavig little to no space for maneouvering, let alone running arbitrary docker images. I have 2 Jetson Nano Modules: One 4 GB devkit module with SD card storage (P3448-0000), and then the non-devkit version with 16 GB embedded MMC storage (P3448-0002). I purchased two 1 TB NVME SSD cards early on, as I assumed from [this overview](https://docs.turingpi.com/docs/turing-pi2-specs-and-io-ports#mpcie-m2-nvme-sata-usb-ports-mapping) on the TP 2 docs that it is supported and would be straightforward to set up. But unless one is comfortable either just mounting 1 TB in random locations and using it only for personal storage or cleanly migrating an entire directory tree to the NVME device, it was inherently unsatisfying that the rest of the system was still squeezing itself onto the embedded limited storage. It took me almost 2 weeks to figure this one out, but I finally managed to get the Jetson Nano modules to mount the root filesystem onto the NVME drives during boot and use it as the main storage device. They still boot from the eMMC/SD card, but I am 100% fine with that. The caveat in this setup is that boot and kernel files are duplicated on the MMC storage and the NVME drive, but people cleverer than me can figure this one out.
I found the NVIDIA Jetson Linux Developer Guide (r32.7.3) and specifically the section about [setting up an NVME drive for booting](https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-3273/Tegra%20Linux%20Driver%20Package%20Development%20Guide/flashing.html#wwpID0E0PM0HA) an invaluable guide. TL;DR 1. Flash module normally, as you would without any other storage attached. 2. Ensure that you can boot normally from eMMC/SD card. 3. Power the module off, attach the nvme drive, then boot the module normally. 4. Ensure that you can see the nvme drive (e.g. 
lsblk | grep nvme
). 5. Parition the drive as GPT. 6. Create the 
APP
parition for the root filesystem (naming the partition 
APP
correctly is important!). 7. Format partition as ext4. 8. Generate the rootfs on the host. 9. Mount the generate system image on the host, and the newly created nvme partition on the Jetson device. 10. Rsync the the system image with to the nvme drive.* 11. Unmount both the image on the host and the nvme partition on the device. 12. Reboot the device and open the uboot shell (requires a UART connection during boot). 13. Verify you can boot from the nvme drive and/or change the order of boot targets to move nvme drive to the top of the list. \ In the docs the setup of the nvme drive is performed on the host. This is not necessary. It is enough to generate the system image on the host and then rsync the image and the device over network. However, this requires ssh root login to be enabled for the duration of the setup.*
Also, for those wanting to setup a fully compliant k8s cluster, and wonder how to disable swapping: Swapping on (all) Jetsons should be configured via the script 
/etc/systemd/nvzramconfig.sh
. To disable swapping and have this setting persist across restarts, verify that swapping is currently active with
Copy code
sudo systemctl status nvzramconfig.service
then disable this service via
Copy code
sudo systemctl disable nvzramconfig.service
As you can verify with 
lsblk
and 
free -h
this setting will not become effective until you reboot the device. To undo this change and reenable swapping, simple enable the service again via
Copy code
sudo systemctl enable nvzramconfig.service
and reboot.
d
I'm using a similar setup but with the USB SSD drive. Indeed the 
/boot
partition must reside on the internal storage (eMMC or SD) and you have to remember to sync the changes there (like when you install kernel modules, recompile the kernel, set the settings, etc.). What I haven't tried, but might work, is to mount the /boot partition from the eMMC.SD as 
/root
(I do not recall why it is not)
c
I found this script to be most useful to boot and run from a NVME attached to my Nanos (with eMMC) and it was simple to use. https://github.com/hcfman/sbts-boot-from-SSD It leaves the eMMC untouched and creates an 'undo' script in your home dir on the Nano if you wanted to go back.
d
I will later find how I did this and share here, but it was as simple as a few commands iirc
15 Views
PreviousNext
Powered by