https://turingpi.com logo
#│forum
Mini PCIe comaptible cards
# │forum
d

daboone72

04/14/2023, 9:24 AM
Found one myself thought I'd start a forum to find others that work
"Tosuny Mini 2port PCI-e PCI to SATA 3.0 6Gbps Converter Hard Drive Expansion Card for for Windows(R) XP/Server2003/Vista/7/8 (32/64 bit) / Iinux"
Appears in lspci as: 01:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01) (prog-if 01 [AHCI 1.0]) Subsystem: ASMedia Technology Inc. ASM1062 Serial ATA Controller Flags: bus master, fast devsel, latency 0, IRQ 38 I/O ports at [disabled] I/O ports at [disabled] I/O ports at [disabled] I/O ports at [disabled] I/O ports at [disabled] Memory at 600010000 (32-bit, non-prefetchable) [size=512] Expansion ROM at 600000000 [virtual] [disabled] [size=64K] Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit- Capabilities: [78] Power Management version 3 Capabilities: [80] Express Legacy Endpoint, MSI 00 Capabilities: [100] Virtual Channel Kernel driver in use: ahci Kernel modules: ahci
And tested with a Samsung 850 which appears with all the goodies. [ 3.733068] ata2: SATA link up 6.0 Gbps (SStatus 133 SControl 300) [ 3.733313] ata2.00: supports DRM functions and may not be fully accessible [ 3.733320] ata2.00: ATA-9: Samsung SSD 850 EVO 500GB, EMT02B6Q, max UDMA/133 [ 3.734581] ata2.00: 976773168 sectors, multi 1: LBA48 NCQ (depth 32), AA [ 3.736450] ata2.00: Features: Trust Dev-Sleep NCQ-sndrcv [ 3.736744] ata2.00: supports DRM functions and may not be fully accessible [ 3.739233] ata2.00: configured for UDMA/133 [ 3.739507] scsi 1:0:0:0: Direct-Access ATA Samsung SSD 850 2B6Q PQ: 0 ANSI: 5 [ 3.740437] sd 1:0:0:0: [sda] 976773168 512-byte logical blocks: (500 GB/466 GiB) [ 3.740486] sd 1:0:0:0: [sda] Write Protect is off [ 3.740494] sd 1:0:0:0: [sda] Mode Sense: 00 3a 00 00 [ 3.740557] sd 1:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA [ 3.740693] sd 1:0:0:0: [sda] Preferred minimum I/O size 512 bytes [ 3.743653] sd 1:0:0:0: [sda] Attached SCSI disk [ 3.760871] sd 1:0:0:0: Attached scsi generic sg0 type 0
Jeff Geerlings page maybe most helpful for this thread too. https://pipci.jeffgeerling.com/
u

_dhanos_

04/14/2023, 9:34 AM
All ASM1061-based card will work out of the box with CM4s with Raspberry Pi OS and Ubuntu (most likely with others too, but these are the 2 I tested) and with Jetson Orin devices also out of the box
This is the one I have
d

daboone72

04/14/2023, 9:36 AM
Ahh great so it seems ASM1062 as well
u

_dhanos_

04/14/2023, 9:37 AM
ASM1062 can be problematic. This is a RAID chip
d

daboone72

04/14/2023, 9:38 AM
Ohh? Thought raid would mean more serious driver testing
u

_dhanos_

04/14/2023, 9:38 AM
In general, avoid hardware RAID (this is not really even a real hardware RAID)
Possibly too much troubles in the future
d

daboone72

04/14/2023, 9:39 AM
Well found a bug bug it also affects AM1061 too https://bugzilla.kernel.org/show_bug.cgi?id=212695
u

_dhanos_

04/14/2023, 9:40 AM
I also have and tested USB 3.0 controller based on
Renesas D720201
- works out-of-the-box on CM4 with raspberry Pi OS and Ubuntu (possibly more) and Jetson Orin modules
d

daboone72

04/14/2023, 9:41 AM
Have you tested any LORA modules? Am thinking for home serve fun
Zigbee the other I hope I can find a combination module
u

_dhanos_

04/14/2023, 9:41 AM
I'm not even sure if this is something that affects CM4s
I don't have any of these
Speaking of ASm1061 - they support SATA port multiplexing, so I'm working on this:
d

daboone72

04/14/2023, 9:45 AM
After a bit of googling I think the main issues with AM1061/2 have been in the 5 series kernels we're on 6 now.
https://www.reddit.com/r/linuxquestions/comments/p8amiu/asm1062_sata_controller_doesnt_work_under_kernel/ I must read more though. SOme issues can be fixed with a few boot options even in 5
u

_dhanos_

04/14/2023, 9:47 AM
Either way, I'd suggest to avoid it. Better to set a RAID usung MDADM or ZFS
d

daboone72

04/14/2023, 9:49 AM
You say send it back then? I think I'll persist with it. I only first planned to run a DVD drive off it. They don't say the chipset when you buy them.
u

_dhanos_

04/14/2023, 9:50 AM
I mean if you bought it already, go ahead and experiment with it. I'm not even sure if this is a hardware raid, or uses a CPU still. Then, if you set a array of disks and it breaks, you'll need another and maybe even some luck to make the array working again
Yes, this is ASM1061, the one that's on most of these cards, and on TPi2 board
d

daboone72

04/14/2023, 9:51 AM
I bought it because of Jeffs comments on how to create a persistence store.
u

_dhanos_

04/14/2023, 9:51 AM
The one I'm talking about 🙂
Where?
p

peter.duckett

04/14/2023, 9:53 AM
i want to look into this but i think you can only use the USB versions as i dont thinkg the mPCI have SPI lanes
d

daboone72

04/14/2023, 10:27 AM
@_dhanos_ ActuallY I tell a lie it was Turing Pi docs that suggested to use Longhorn storage it requires a minimum of 3 nodes with hard drives. https://help.turingpi.com/hc/en-us/articles/8942873470877-The-plan
My thought was though to check if Jeff had an ansible playbook for it. He does for Ceph not Longhorn though https://github.com/geerlingguy/pi-cluster
u

_dhanos_

04/14/2023, 10:30 AM
3 nodes means 3 devices, like 3 CM4s, each with a drive
He actually used ZFS here
I'm a little bit confused 😄
I mean how did you get to the ASM1062
d

daboone72

04/14/2023, 10:31 AM
Yes primiarly zfs though he does have a starter ceph playbook.
Well your own docs say "Using the SD card on Raspberry Pi is not recommended for this purpose (although possible)." . WHich leads to 3 real drives with 2 SATA sockets? I'd better get one more.
u

_dhanos_

04/14/2023, 10:32 AM
No, this means each of 3 nodes has to have a separate drive
These 2 SATA ports are connected to Node 3
You'll need 2 Mini PCIe SATA controllers, one for each Node 1 and Node 2
But a regular SATA controllers, not RAID controllers
This way Node 1, Node 2 and Node 3, each have a single drive. This is for Longhorn, of course
d

daboone72

04/14/2023, 10:34 AM
My controller certainly isn't a raid controller
u

_dhanos_

04/14/2023, 10:36 AM
Oh, I might have confused it with ARM1061R, in which case it's my bad
d

daboone72

04/14/2023, 10:36 AM
Maybe I'll wait then to RK1 I have two boards.
u

_dhanos_

04/14/2023, 10:37 AM
ASM1062 seems to be, indeed, an AHCI controller, and the difference is it can use 2 PCie lanes, which will never be used on TPi2
d

daboone72

04/14/2023, 10:38 AM
Well it didn't cost much anyway. I just figured with the keyword Linux in the title at least it has a shot of working
u

_dhanos_

04/14/2023, 10:40 AM
The most common is, however, ASM1061. ASM1062 is much harder to find
I believe the same drivers should work for both, though
s

srcshelton

04/15/2023, 10:20 AM
Just to check, is it possible to boot entirely from (onboard or Mini-PCIe?) ASM1061-connected drives, or does VideoCore still require its FAT partition to be on eMMC/SD or USB (with a reflash)? https://github.com/raspberrypi/usbboot says "The CM4 EEPROM supports MMC, USB-MSD, USB 2.0, Network and NVMe boot" - so the ASM1061 option is NVMe (or somehow overrides one of the others)?
(Off-topic for this forum but related to the query above: does switching the USB2.0 port in Device mode to a node also assert GPIO 40 (EMMC-DISABLE/nRPIBOOT) for that node automatically?)
u

_dhanos_

04/15/2023, 10:29 AM
I haven't heard anyone attempting this. I'm not sure if you can boot CM off of SATA disk entirely
Correct, this is why you need to set it back into the host mode after flashing
s

srcshelton

04/15/2023, 10:32 AM
Ah - so you have to explicitly change the recently-flashed node to Host mode, or moving the USB device to a different node will automatically reset this? Suggestion for the Community FW: If the Host/Device mode is actually switching GPIO 40, then node this in the BMC UI! (Also, if it is persistent per-node, then consider including a table of the current setting for each node?)
u

_dhanos_

04/15/2023, 10:34 AM
> so you have to explicitly change the recently-flashed node to Host mode, or moving the USB device to a different node will automatically reset this? Both will do > Suggestion for the Community FW: If the Host/Device mode is actually switching GPIO 40, then node this in the BMC UI! I'm not sure what do you mean by this. To note? The UI shows which node is being set and to which mode (the other nodes stay in a host mode)
s

srcshelton

04/15/2023, 10:40 AM
No, that makes sense - I was wondering whether if you set Node 1 to Device mode then moved USB2.0 to Node 2, whether GPIO 40 would still be asserted for Node 1. If not, there's nothing more to add!
u

_dhanos_

04/15/2023, 10:41 AM
Yes, only a single node will be put into the flashing mode at a time
c

cyliew

04/17/2023, 3:07 PM
I finally managed to get this one to work with a Jetson TX2 NX module booting from an NVME SSD drive: https://coral.ai/products/pcie-accelerator/ OS: Jetson Linux R32.7.3 (included as part of JetPack 4.6.3) Required Change before installing the Edge TPU runtime: add the following string to the APPEND parameter in /boot/extlinux/extlinux.conf: " pci=noaer pcie_aspm=off" and reboot the node. Without this change the system will boot up, but becomes near unresponsive.
I have this one, which works, but gets disconnected if I reboot the CM4. (power cycling the CM4 reconnects it). I think the eeprom on my CM4 is old, which might be causing this. So I will try updating the eeprom firmware on the CM4 at some point.
u

_dhanos_

04/17/2023, 8:45 PM
I have a USB controller based on the same chip, but haven't seen your issue (but maybe I haven't created such scenario either). This sounds more like a driver issue than the CM4 firmware, but who knows.
b

batz_10k

04/17/2023, 8:48 PM
It could indeed be a driver issue, I'm running Ubuntu 22.10 on this, and I might try testing with raspbian to see if the same issue happens there. (during a reboot it simply shows the pci link down in dmesg as if nothing was connected)
m

maxsharx

04/18/2023, 8:45 PM
Do you have any experience with the Renesas D720202 (2 port). Because it won't work here. Neither with node 1 nor 2. Have also powered up the floppy connector.
u

_dhanos_

04/18/2023, 8:47 PM
I do not have this one in hand to test
Did you connect the power it it?
m

maxsharx

04/18/2023, 8:49 PM
Yes, but it should be at least visible to the CM4 without. Only devices won't work without extra power.
u

_dhanos_

04/18/2023, 8:50 PM
Yes, but I'm not sur how did you check so I figured I'd ask
Check
dmesg
, maybe you can find something there
I'm guessing
lspci
does not list it?
m

maxsharx

04/18/2023, 8:55 PM
lspci and lsusb shows nothing
u

_dhanos_

04/18/2023, 8:59 PM
Hmm, some search shows that it should at least be visible in the
lspci
You are using CM4s and not other compatible modules, right?
Which OS are you running?
m

maxsharx

04/18/2023, 9:00 PM
Yes, will setup up CM4 from scratch with a clean Raspberry OS
u

_dhanos_

04/18/2023, 9:00 PM
I was about to suggest trying this
m

maxsharx

04/19/2023, 7:54 AM
Fresh install with Raspberry OS has helped. lsusb shows the card/controller. But, no devices - even not with extra power (5V/12V proved). 😞
u

_dhanos_

04/19/2023, 11:06 AM
``` sudo lspci -v ```what does it show?
m

maxsharx

04/19/2023, 4:26 PM
Already tried it and you can give lspci one or three "v" it still remains with (NULL), i.e. nothing. After updating the R-OS lsusb also only shows the internal USB2 bus. dmesg says: [ 1.796384] xhci-hcd fe9c0000.xhci: USB3 root hub has no ports
USB3 card is properly powered and can supply an Raspberry Pi 1
u

_dhanos_

04/19/2023, 5:13 PM
You said that
lsusb
shows the controller so
lspci
should show it as well. I think I'm a bit confused
m

maxsharx

04/19/2023, 6:07 PM
lspci has never shown the controller. Only lsusb.
u

_dhanos_

04/19/2023, 6:10 PM
This is super odd because the controller is attached to the PCI. Did you run it with sudo (might matter)?
m

maxsharx

04/19/2023, 6:25 PM
Yes of course. Found an old Apple Airport Express card. That's running and shown by lspci. CM4 seems to be OK and TuringV2, too. I'm going to send USB controller back.
j

jpconstantineau

04/21/2023, 5:45 PM
@cyliew were you able to get the edge TPU running with a CM4 or just with the Jetson? I got a pair of them and wasn't able to get the CM4 to detect them. Didn't investigate much beyond "lspci"...
u

_dhanos_

04/21/2023, 5:48 PM
From my (small) experience with Mini PCIe, try them with Raspberry Pi OS. Ubuntu, for example, works with USB controller connected to Mini PCIe, but not SATA
If it works under RPi OS, then try other OSes
Also, I'm assuming you have the Mini PCIe ones, right?
j

jpconstantineau

04/21/2023, 5:52 PM
Yeah. I have the minipcie ones. I did read something a long time ago that there were issues but I can't recall what and I assume that by now, it would be clear that they are compatible or not ... Considering that they were backordered for almost a year, perhaps it's not surprising that little information on them is available, beyond what Google/coral has on their site.
c

cyliew

04/21/2023, 6:10 PM
Only with the Jetson TX2 NX. I haven't managed to get hold of CM4 modules myself. Still waiting for better availability.
j

jpconstantineau

04/21/2023, 6:12 PM
I ordered mine about a year ago at digikey and they were back ordered for that long. Then someday, I got a package on my doorstep...
d

displague

05/18/2023, 10:16 AM
Did you make that change before running the flash script or did you run this from the booted node? I've been finding that my changes to /boot/extlinux are ignored. (For example, changing root= to nvme)
c

cyliew

05/18/2023, 11:19 AM
This is the order how I did it: 1. Flash the TX2 NX module 2. Ran the JetsonHack script to copy from EMMC to NVME and install the script to boot from NVME. 3. Changed the extlinux.conf file for the Coral Mini PCIe Accelerator. 4. Plugged the Coral Mini PCIe Accelerator into the TPI 2 Clusterboard 5. Followed the Google Coral Setup instructions
k

killmas

05/21/2023, 1:33 AM
@cyliew is this a jetson thing only or would this be applicable to the Orin NX as well? also just to confirm, the /boot/extlinux change is after you've installed the jetson stuff and are up and running with jetson off the NVME drive, correct?
@cyliew also how specifically did your extlinux.conf file look once you updated it?
c

cyliew

05/22/2023, 4:27 PM
Sorry, I have no idea if it will also work with the Orin NX. I have only tested this with the Jetson TX2 NX myself. I have indeed changed the extlinux.conf afger finalizing the Jetson stuff and moving the boot drive to NVME, but before plugging in the Coral mini PCIe Accelerator.
I am currently not at home because I am at a conference for work. I will share my extlinux.conf file when I am back home again coming Thursday.
I've attached my /boot/extlinux/extlinux.conf file. The root=/dev/mmcblk0p1 parameter is not relevant as the setssdroot.sh script that I installed using the rootOnNVME project from jetsonhacks switches the rootfs to the NVME drive during startup.