https://turingpi.com logo
#│forum
Turing-Pi 2 Evolution (feature request)
# │forum
p

peter.duckett

03/28/2023, 8:40 AM
Given that we should be starting to see the TPi2 in our hands soon what would people like to see changed in the original design IF WE EVER got a TPi V2.1 (Please no comments about the RTC issue that goes without saying) Please be positive in the community spirit some ideas I would like to see would be, The movement of header ports (Power Connector, SATA, USB3 header) away from the front of the broad preferably to the side to aid airflow though the cards installed Dedicated USB header for the CM4 Mode port (I could see this of more use on the front of a case) Move the reset BMC buttons onto the Back plate at right angle to the broad for easier access from the rear of the case Maybe an ATX version with 6 or 8 nodes? PWM fan for system fan (Maybe two instead of 1) Maybe also add PWM for the nodes also (Forgot that thanks @segabor ) Upgraded Ethernet 2.5gb maybe useful depending on the use case, I would expect that would almost allow Max throughput of all connected broads currently Maybe mPCIe that supports SPI for LORWAN gateways (though not sure how this might be achieved perhaps some small jumper switch) POE class 4
w

wvaske

03/28/2023, 1:16 PM
Agree with a lot of these. Multi-gig up link on the switch. It's become common enough they should be able to find a well priced switch chip. More nodes. The TPI1 had seven and is the same for factor with DIMM connectors. More PIs please. A custom heatsink. They know the dimensions very exactly, I'd like to see a custom heatsink that uses all the space, similar to what the Compute Blade Kickstarter is doing.
s

segabor

03/28/2023, 1:34 PM
Well, I love the current hardware but what I feel can be improved a lot is the BIOS. Everytime I want to shut down the whole stuff I need to make sure each OS is halted before pressing the Power button. Thermal management: is it working anyway? My Thermaltake case has a big front fan and it has never been rotated a bit. Also, UART pins could have an attachable PCI cover having an USB female port. More nodes would be a big win!
s

srcshelton

04/13/2023, 12:01 PM
Ability to update the routing of the on-board switch, for example to route Nodes 1 & 2 only to NIC1 and Nodes 3 & 4 only to NIC2, for example. Plus a dedicated BMC management port.
t

terarex

04/13/2023, 12:32 PM
The current switch chip supports tagged and protocol VLANs. That might be a way to get what you want. TMs hasn't disclosed the feature set they might implement in the new firmware.
u

unjeh

04/13/2023, 4:28 PM
Maybe remove the fixed mounting nuts (make em non fixed?) on the m2 x 60 spacing to allow for dual sided nvme 22x80 drives without desoldering said nuts.
t

terarex

04/13/2023, 4:49 PM
+1 for making the mPCIe and M.2 anchor points removable/relocatable.
I'd like for Turing Machines to fully debug and fix node1 USB flashing and HDMI. I'd also like to see the BMC chip updated to the T113-S4 (256MB vs 128MB) and increase the amount of flash available to the BMC to host dual, more full-featued, boot images.
j

jonathanbennett

04/14/2023, 12:19 AM
HDMI duplexing of all 4 slots would be really handy for troubleshooting and maintenance.
w

wilsonrighetti

04/19/2023, 9:00 PM
KVM switch?
u

_dhanos_

04/19/2023, 9:06 PM
It kind of has one - the USB port. But do you mean for video output?
w

wilsonrighetti

04/19/2023, 9:08 PM
well, many people on the chat here ask about video and keyboad/mouse. We kind of keep saying, video only Node 1, but not keyboard with Jetson and blah blah. If we have this KVM switch we could just configure which ones will get the inputs/outputs. But, maybe this would impact the lanes used by specific nodes, as example, node 3 has SATA and others not. Maybe new versions of compute modules would also help on this matter
u

_dhanos_

04/19/2023, 9:09 PM
Ok, I understand what do you mean now, but also can see the challenge here 🙂
t

terarex

04/19/2023, 9:35 PM
I'd be happy with a switchable 5 channel TTL to USB-A male adapter. Basically a serial KVM. Cabling would be a PITA. Having 5 USB-to-TTL/UART cable is overkill.
u

_dhanos_

04/19/2023, 9:36 PM
The question is if you can use the BMC for this, which already has UART connections to all of the nodes
p

phearzero

04/19/2023, 9:38 PM
Having the BMC as a node would be useful for upgrading/different use cases.
t

terarex

04/19/2023, 9:39 PM
I guess that works. I want a console connection if the BMC uses the tty ports. That way maybe they don't step on each other.
t

the_boom_boom

04/21/2023, 3:35 AM
moar nodes!
u

_dhanos_

04/21/2023, 3:35 AM
Oh, you never have too many 😄
a

alastair.q8

04/22/2023, 6:59 AM
I'd remove one sata port seeing that each node can have an nvme drive. Two sata ports seems like overkill to me.
u

_dhanos_

04/22/2023, 12:33 PM
There's never too many 😄
Also, CM4 cannot use M.2, so SATA will be useful 🙂
a

alastair.q8

04/22/2023, 12:36 PM
Well Daniel you're special. 😂 The removal of the one sata port could open space for something else is what I meant to say.
u

_dhanos_

04/22/2023, 12:41 PM
Hehehe 😉
When I think of it, there's still a lot of space. If anything, the ports could be stacked the other way opening some space anyway if the team needed this space
People also use 2 ports for data safety (drive mirror called RAID1)
I only mean 2 SATA oirts are useful 🙂
a

alastair.q8

04/22/2023, 12:47 PM
I see your point. In my use case I'll have one Pi installed and hopefully two RK1s and an AI accelerator. All the nodes except for the Pi will have nvme drives. The Pi will be attached to the SATA port. I don't need data redundancy. I am using this as a learning platform.
u

_dhanos_

04/22/2023, 12:50 PM
Yeah, I understand. But then if you put the board to actual work and you have actual data you care about, disk redundancy is rather a must have. In my PC I keep file with redundancy for many many years now (previously RAO1 off of 2 disks, now RAID5 off of 3 disks). This is why I would sat the second SATA port is actually useful 🙂
a

alastair.q8

04/22/2023, 1:00 PM
True about how important your data is to you. Well this is the first iteration of the retail TPi2 and the team might release more SKUs that are more tailored to specific use cases. The issue is that the more features are added on this size board the more complicated the construction of it gets. Not saying it will definitely lead to more issues but, chances are they will. At the end of the day I want something reliable which it is easy to forget is an intrinsic feature we all want.
s

srcshelton

04/25/2023, 8:08 AM
There’s much to learn from the TP2, but given the time it takes to make major design changes and the sheer cost, I’d expect the next revision will be TP3/RPi CM5 (… on the assumption that the RKx boards don’t become the primary use-case)
c

cguijt

04/26/2023, 9:50 PM
My 2 cents: if the earlier mentioned KVM is unavailable, move the USB3 controller to node 1 so that at least one node has HDMI + USB access 🙂
u

_dhanos_

04/26/2023, 10:36 PM
While this is a valid idea, if you want a USB right now, you can get a Mini PCIe USB controller - having a Mini PCIe for node 1 give you options what do you want to have connected to the Node 1
c

cguijt

04/26/2023, 10:38 PM
I'm aware, the idea I mentioned was purely to prevent the need of such controller if you only need it occasionally, like installing a Jetson module 😉
u

_dhanos_

04/26/2023, 10:39 PM
Sure, just making sure you are aware about this possibility 🙂
d

dethtungue

04/28/2023, 1:12 PM
Yes please. If HDMI Duplexing is not an option, having connected USB ports on the same node as the HDMI interface is a must-have. Otherwise, we all have to source and then swap in/out mini PCIe USB adapters.
t

the_boom_boom

04/28/2023, 5:51 PM
if you've populated node4 then you could use use usbip to give node1 access to the usb ports
s

scienceman.

05/01/2023, 10:47 PM
An Aspeed or other server-class BMC chip, please! It will make it much easier to develop code, implement useful features, etc.
g

goatherder.

05/03/2023, 2:35 AM
Whilst the UART connectors and the sd card connections are really important it would be cool if there was a header block with cable that you attach so you expose these interfaces on the outside of a case. So you do not have to crack the case everytime you need to interface to it
s

srcshelton

05/03/2023, 8:32 AM
I've been trying to find a PCI bracket with Female-Female pass-through USB connectors, so that a UART/USB cable could be used internally and a USB-A/A cable externally... although I've not found anything suitable yet. I have found UART/DB-9 PCI brackets, which could be used for an old-skool serial setup? (... I guess the other option would be a UART/RJ-45 face-plate?) I've not looked for PCI brackets with an SD slot, but I have an SD extension ribbon-cable installed in my system, meaning I have to crack open the case but not unmount the mainboard to get to the (BMC) SD card, which feels good enough for now.
c

cfsworks

05/03/2023, 8:18 PM
This one's a low-hanging-fruit idea for a future minor revision: if the node control GPIOs (
{EN, RST, USB_VBUS, RPIBOOT} * 4
) can be moved out to a GPIO expander (e.g. PCA9555 on I²C), that would free up 16 pins for other purposes... ...including running RGMII to the RTL8201F a GbE PHY, so we get 1Gbps to the BMC instead of 100Mbps. 😀
s

schraubersimmie

05/05/2023, 11:47 PM
For future IO-shields: If you don't put the cutouts in the "normal mounting position", but raise them by 20 mm. They still fit in the slot cover and it would allow to mount the TP2 board raised in every case to make space for NVMe SSD with coolers.
m

maarten morinsights.nl

05/14/2023, 2:16 PM
BMC ability to somehow send a "soft shutdown via power button" command to the nodes. Instead of a hard Node Power Off via the power control only being available. Please
r

rexter0

05/29/2023, 6:13 AM
Give a lot more focus to the ability to flash nodes. There should be no requirement to swap cards in and out just to do flashing or direct node management. 1. BMC should have access to nvme cards so one could flash from bms to all for nvme drives. 2. KVM switch to switch keyboard mouse monitor between each node, again controlled via bms 3. Put the display type switch in BMC config file so one can switch in software
a

acelink.io

05/30/2023, 1:09 AM
Move all USB connectors to USB type C! Death to microUSB! TypeA should be EoL
s

srcshelton

05/31/2023, 1:17 PM
It'd be great on a per-Node basis to expose at least GPIO pins (5V, GND, and something like GPIO18?) to drive a 5V PWM fan under software/OS control. Due to the positioning of my case-fan, I've got passive heatsinks on Nodes 1-3, but then a PWM fan on Node 4 which is controlled via software and Node 1's GPIO PWM output (based on a daemon process running on Node 1 and a simple REST interface on Node 4 which reports the current temperature!). This does mean that Node 4 only receives temperature-dependent cooling when Node 1 is running, of course...
t

teslamax

06/09/2023, 11:24 PM
perhaps an I²C PWM fan controller?
I think someone mentioned the possibility of a new CM4 adapter design that also included such an option, but I have no idea if it might ever become reality
s

scienceman.

06/09/2023, 11:45 PM
I think in addition to the excellent suggestions by @rexter0 above that making the existing 2-pin main board fan connector into a 4-pin one with speed signals controlled by a temperature sensor on the switch heat sink would make sense, since that is the hottest part of the current board.
d

dkryptlive

07/03/2023, 3:32 PM
I agree with this… don’t take away our sata, more sata please! Or at least figure out a way to get the RPi to work with one of these https://a.co/d/7b78WgM
t

terarex

07/03/2023, 5:18 PM
This appears to be the same device: https://a.co/d/93wWl58. It's based on the ASM1166 controller. Best case, if it works on a generic CM4 carrier, would be to use it for HDD connectivity. Performance with the RPi 4's single PCIe Gen 2 lane wouldn't take proper advantage of SSDs. If you need SATA ports for node1 and node2. Look for this mPCIe card. It's based on the ASM1064 and works with CM4 and Jetson modules. I assume it will work with the RK1. Cablecc Mini PCI-E PCI Express to... https://www.amazon.com/dp/B0C3CMH6B8?ref=ppx_pop_mob_ap_share
d

dkryptlive

07/04/2023, 1:15 AM
Thanks for the reply! For my project, don’t need full ssd speed. Looking to use one of the CM4’s to run a NAS. That adapter looks promising, but I need more than four SATA ports… I could always use a multiplier for the existing two sata on Node 3, but looking for something more elegant… another option I’ve considered is using a mini pcie to pcie riser, then a proper pcie sata card Mini PCIe Riser: https://a.co/d/iqGD7Le PCIe SATA: https://a.co/d/ikGQ2aI
u

_dhanos_

07/04/2023, 1:21 AM
A port multiplier is going to work as well as the module you initially mentioned you wanted to use
The reason for this is a speed limit of the PCIe lane that the CM4 can use
This is exactly what I'm using. I'm yet to test it well, but the initial test shows I can easily saturate the 1 Gb/s of the network speed: https://discord.com/channels/754950670175436841/1030129990211219486/1091312860304511107
c

cbmdk

07/07/2023, 8:20 AM
did you get this to work... just got 2 boards as well 😄
u

.mvcs

07/16/2023, 2:45 PM
Hi there, I'm considering connecting an old 12v FAN (Molex 2510 4PIN) I have to the TP2 FAN (see the image). Plugging the FAN using the 2 middle pins turns the FAN on full blast. The thing is, I want to be able to control the FAN using the PWM wire. Does anybody knows if it's possible to connect the blue wire (PWM I suppose) to the GPIO 18 on the Node 1 to control the FAN from there? I don't mind relying only on the NODE 1 cpu temp. I was considering using the python script I saw on this blob post (https://blog.driftking.tw/en/2019/11/Using-Raspberry-Pi-to-Control-a-PWM-Fan-and-Monitor-its-Speed/#Use-PWM-to-Control-Fan-Speed) as a starting point. Have anybody tried something like that? I'm mostly a software guy and I don't know much about the hardware intricacies 😆 Is it a stupid thing to do?

https://cdn.discordapp.com/attachments/1090193720294522900/1130148310825586801/Screenshot_2023-07-16_at_16.20.47.png

https://cdn.discordapp.com/attachments/1090193720294522900/1130148311232413736/IMG_8318.jpg

https://cdn.discordapp.com/attachments/1090193720294522900/1130148311651848244/IMG_8319.jpg

well, I managed to not fry anything it seems 😂 it's working as intended. Next step is to buy a proper jump wire to replace this work around
t

terarex

07/20/2023, 5:33 PM
If a revised TP2 is made available, I have a couple of suggestions regarding physical characteristics: - The mPCI and M.2 standoffs should not be surface mount/soldered. They should be removable. The current board design has sufficient keep out space around the soldered standoffs to accommodate wafer head screw attachment (with Loctite threadlocker blue). To save on manufacturing cost, the standoffs and screws do not need to be factory installed. Including them in a separate bag, like the SoM JST fan cables, is satisfactory. - Increase the height of the M.2 connectors on the rear. The current connectors appear to be 2mm. Why? High capacity M.2, NVMe SSDs are double sided, and PCIe Gen 4/Gen 5 SSDs require heat sinks with thermal pads. The current M.2 connectors and standoffs do not adequately support the underside SSD thickness required for airflow. I'm going to install 4mm (5mm would be better), removable standoffs, which should provide sufficient clearance, but the SSDs will be angled. I don't think I want to tackle the job of reworking a board to desolder the existing and resolder taller M.2 sockets.
d

dkryptlive

07/26/2023, 2:29 AM
My dream TP#: Node 1 (Main) - HDMI, mini pcie, 2x USB, ability to flash other 3 nodes Node 2 (Router) - mini pcie w/ sim, 4 external ports (1 WAN/3 LAN… 2.5G or 10G would be nice too!) Node 3 (NAS) - 8 SATA ports, 2x USB Node 4 (AIML/robotics) - 2x USB, 40pin GPIO, 2x MIPI CSI-2 Edit: all with NVMe
s

silopolis

08/03/2023, 12:22 PM
Indeed, and I'd rather have one SATA per module than none!
m

mvdk85

08/05/2023, 6:53 PM
I would want to replace the 1 Gbps switch part with a 2.5Gbps or 10Gbps capable part.
u

_dhanos_

08/05/2023, 6:54 PM
Each available module has a network link speed of 1 Gb/s, so a "faster" switch is not going to change the max speed you can access the module with (but would let you utilize the max speed link of multiple modules at the same time, of course).
m

mvdk85

08/05/2023, 6:55 PM
I thought that the RK part had a 2.5Gbps capable output?
(as an *GMII thing, but still)
Or is this one of the compromises that the *S part makes?
u

_dhanos_

08/05/2023, 6:57 PM
RK3588 supports up to 1GB/s with RGMII
We're not using the S version of RK3588
m

mvdk85

08/05/2023, 7:06 PM
I suppose you could theoretically hook up both GMACs to the same chip as long as it supports LAGs?
I mean, I think there's a RTL chip that would theoretically support that, and give you a 10 Gbps uplink - I don't know how much more expensive that part would be, but I imagine the traces would be hard to find?
u

_dhanos_

08/05/2023, 7:08 PM
Possibly, but then how LACP works is it won't utilize both links for a single connection, just one of them. Then how TPi2 is designed is it utilized only a single LAN connection. I'm not sure if RK1 exposes both
Where did you get the 10Gb/s from?
m

mvdk85

08/05/2023, 7:09 PM
LAGs allow you to balance flows across them
RTL8396L is one part that would theoretically work
u

_dhanos_

08/05/2023, 7:10 PM
Meaning connections, You'd need multiple (at least 2 in this case and hope they both get routed through different interfaces)
They still will be 1Gb/s links (so a total of possible 2Gb/s)
m

mvdk85

08/05/2023, 7:10 PM
Yeah, which is twice the existing capability 😉
c

cfsworks

08/05/2023, 7:11 PM
A fun theoretical problem I want to solve is what happens when there's 2 flows, 1Gbps each, but the switch is hashing them onto the same LAG member
m

mvdk85

08/05/2023, 7:11 PM
From https://svanheule.net/switches/rtl93xx, I see that the 9302A might work.
Yeah, it
*it's an imperfect solution.
u

_dhanos_

08/05/2023, 7:12 PM
Yeah, this is the "hope" part I mentioned 😄
LACP is not "intelligent enough" to make it the way you'd like all the time
m

mvdk85

08/05/2023, 7:13 PM
In the case of large numbers of flows, this should go away, but it's only in the limiting case.
In practice, one link is always more utilised than the other, and elephant flows are the common case.
(Unless you're a hyperscaler 🙂 )
But it's definitely better than 1 link. I wonder why I thought it had a better ethernet link?
c

cfsworks

08/05/2023, 7:16 PM
The RTL8370MB has a 16-way hash-to-link table, so the fun problem to tackle would be to make a daemon that can automatically balance out this table, which should solve this problem nicely.
At least for outgoing flows 🤔
m

mvdk85

08/05/2023, 7:17 PM
elephant flows in practice make balancing it impossible.
You can only ever hope to minimise the imbalance 😉
c

cfsworks

08/05/2023, 7:18 PM
Well, yes, that
It doesn't have to be perfectly 50:50, you just don't want high congestion
If you want true 50:50, you have to go round-robin, and either use only protocols that tolerate reordering, or do what Brocade does and balance the link latency in hardware.
m

mvdk85

08/05/2023, 7:24 PM
Huh, well, there is a promising way to add capability to it, and you only have to use one of the PCIe2.1 lanes.
PCIe2.1 supports 5Gbps - overheads (I think about 10%), so if you used a chip on the board that did PCIe to gigabit, you could connect them internally at 5Gbps - overheads.
From the "right compromise" perspective, I think the Intel i225-v is available in quantity for very cheap, so it might make sense to use it instead.
c

cfsworks

08/05/2023, 7:44 PM
So the idea is 2.5GBASE-T from the module to the board?
m

mvdk85

08/05/2023, 7:45 PM
No, it would be PCIe2.1 from the module to the board, and the board hosts the 2.5Gbe controller
c

cfsworks

08/05/2023, 7:46 PM
mm, in the M.2 or mPCIe slot?
m

mvdk85

08/05/2023, 7:46 PM
Or more accurately, 4 2.5GBe controllers
Potentially.
I don't know where we are currently extracting the SATA controllers to, apparently the pins are multiplexed with those.
c

cfsworks

08/05/2023, 7:49 PM
Multiplexed or switched?
m

mvdk85

08/05/2023, 7:50 PM
Multiplexed. It looks like you configure it somehow (it doesn't specify, but I'm guessing there'll be a magic fuse or something?)
We already use this mechanism for exposing USB3.1 for one of the things.
And mPCIe slot for another node
But there are 3 things to configure, so it feels like we might not have used one.
If so, we can use one of these as a victim for a 2.5GBe controller.
c

cfsworks

08/05/2023, 7:53 PM
But if the controller is on the TP2, not the module, which Jetson pins are the victims?
m

mvdk85

08/05/2023, 7:53 PM
Oh, I see.
Yeah, that makes sense - so I suppose it would have to be on the module.
Do we export it as a *MII interface for the Jetson right now?
c

cfsworks

08/05/2023, 7:54 PM
1000BASE-T
m

mvdk85

08/05/2023, 7:54 PM
So we'd export it as a 2.5GBASE-T
Not the most pleasant of things, but I suppose it would do.
c

cfsworks

08/05/2023, 7:55 PM
Which I like since it means RK1s for people who only want 1G don't pay the extra for 2.5G
m

mvdk85

08/05/2023, 7:55 PM
Yeah, that's true.
c

cfsworks

08/05/2023, 7:55 PM
The only thing on the TP2 that has to be upgraded is the switch
m

mvdk85

08/05/2023, 7:55 PM
And it's compatible both ways
It just autonegotiates the best capability.
How do we export the second interface right now, by the way?
Is the second just unconnected?
c

cfsworks

08/05/2023, 7:58 PM
Second interface of which interconnect technology?
m

mvdk85

08/05/2023, 7:58 PM
RK1
c

cfsworks

08/05/2023, 7:59 PM
I mean fill in the blank: second _______ interface? PCIe?
m

mvdk85

08/05/2023, 7:59 PM
Ethernet
c

cfsworks

08/05/2023, 8:00 PM
Didn't know there was a second on the RK3588, but I'd guess there's no PHY on it, so the pins only go as far as the RK1 PCB.
m

mvdk85

08/05/2023, 8:00 PM
I figured that was the probably answer 😉
There was one moderately surprising choice that was made - Most of the RTL "managed switch" chips have a MIPS chip that I think would be sufficiently capable of being a BMC. Was it just that it was MIPS, or was there another problem?
Like, they certainly have more than enough GPIO pins 😉
c

cfsworks

08/05/2023, 8:10 PM
The BMC was selected relatively late in the TP2 design process, after the Ethernet switch was already chosen. I don't know what criteria went into BMC selection though. I'm surprised that there's a separate Ethernet PHY when there's extension interfaces on the RTL8370MB though.
It might have been motivated in part by UART quantity and availability of a SD/MMC interface, which the switching SoCs might not have offered.
m

mvdk85

08/05/2023, 8:13 PM
That's true, generally flash parts are chosen for switch SoC instead.
Oh my, the 8370MB is cheap - even in tiny quantities, it's just over $10.
The equivalent RTL9301 part is US$25 in 200+ quantities, and $22 for 1200+
c

cfsworks

08/05/2023, 8:18 PM
I'm also suspecting a lot of answers to design trade-off questions will be "it was the first thing we found that solved the problem"
Realtek's demo schematic for the RTL8370MB appears almost verbatim in the finished TP2, including an unused SPI-NOR out beyond node4.
m

mvdk85

08/05/2023, 8:20 PM
Wait, why include an unused part?
Oh, I think I know
The default memory map for the processor will map the SPI-NOR straight into its address space.
Anything else would require config.
c

cfsworks

08/05/2023, 8:22 PM
Because it's in the demo schematic and they didn't delete it.
m

mvdk85

08/05/2023, 8:23 PM
Yeah, but I would think that the designers would go to some trouble to save the 50c or so per SPI-NOR, right?
c

cfsworks

08/05/2023, 8:23 PM
I'd think so too. Yet it's there.
m

mvdk85

08/05/2023, 8:26 PM
You know, from a forward compatibility perspective, I like the idea of a 2.5GBaseT interface more and more - a future designer wanting a 10G or higher part, assuming that future hardware were good enough, would be able to do just that.
c

cfsworks

08/05/2023, 8:30 PM
Or 5GBASE-T-minus-overhead if it's not much more expensive 🫰
m

mvdk85

08/05/2023, 8:31 PM
Yeah, indeed
I mentioned the Intel part because it's $2 😉
But it is only 2.5GBe.
c

cfsworks

08/05/2023, 8:50 PM
The only SATA is a PCIe adapter on the TP2; no SATA goes through the socket.
m

mvdk85

08/05/2023, 8:54 PM
Oh! We don't use that bit of the RK's stuff at all 🙂
u

_dhanos_

08/05/2023, 8:54 PM
I'm trying to catch up and you got me lost here. What do you mean by this?
m

mvdk85

08/05/2023, 8:55 PM
Oh, just that we'd use one of the PCIe IFs of the RK3588 for an i225v or similar, and replace the exported 1gbase-t interface with a 2.5gbase-t interface.
u

_dhanos_

08/05/2023, 8:55 PM
With TPi2 the thing is all modules are treated more or less the same, I guess. So this is why there is ASM1061 (SATA controller) connected to Node 3 and some chip I forgot the model of connected to the Node 4 for USB 3.0
You can put 10GBASE M.2 modules on the back, if you wish 🙂
And have 10Gb/s network between the modules 😛
m

mvdk85

08/05/2023, 8:57 PM
The idea is what substitutions could be made to make it possible in a TPI2.1
u

_dhanos_

08/05/2023, 8:57 PM
But yes, the only way to get more than 1Gb/s would be to connect something to the PCie
m

mvdk85

08/05/2023, 8:58 PM
So the idea would be to replace the RTL8370MB with RTL9301 or something like that
m

mvdk85

08/05/2023, 8:59 PM
and then add an i225v, or perhaps a marvell thing to the RK1.1, and then send that over the pins used for 1GBASE-T interface.
Nah, nothing here calls for a PCIe switch.
u

_dhanos_

08/05/2023, 9:00 PM
This would mean no Mini PCIe /SATA / USB 3.0 unless you also add a PCIe switch
Well, do you need the ethernet overhead instead of direct host-to-host communication? 🙂
m

mvdk85

08/05/2023, 9:01 PM
Forgive me, I thought there were 4 PCIe interfaces on the RK3588?
u

_dhanos_

08/05/2023, 9:01 PM
There are 2 - 1x and 4x
m

mvdk85

08/05/2023, 9:02 PM
I thought there were 3 PCIe2.1 and 1 PCIe3?
c

cfsworks

08/05/2023, 9:02 PM
Time to check the datasheet
u

_dhanos_

08/05/2023, 9:03 PM

https://cdn.discordapp.com/attachments/1090193720294522900/1137491087670390804/image.png

c

cfsworks

08/05/2023, 9:03 PM

https://cdn.discordapp.com/attachments/1090193720294522900/1137491165332131880/Screenshot_20230805-150328.png

So 1x2 + 2x1 might allow the i225v upgrade
m

mvdk85

08/05/2023, 9:04 PM
What about the Combo PIPE PHY interface?
u

_dhanos_

08/05/2023, 9:05 PM
I'm reading about it right now
m

mvdk85

08/05/2023, 9:08 PM
I was guessing that we were using one of these lanes for the M.2 feature 😉
u

_dhanos_

08/05/2023, 9:08 PM
M.2 uses PCIe3 x4
m

mvdk85

08/05/2023, 9:09 PM
Oh, so we're using these lanes for the other features, then
u

_dhanos_

08/05/2023, 9:09 PM
I guess there may be indeed 2 more PCIe 2.1 that I wasn unaware of
m

mvdk85

08/05/2023, 9:09 PM
Well, maybe just 1 of them for each, because it doesn't look like they know about bonding these.
u

_dhanos_

08/05/2023, 9:09 PM
Just one of the PCIe 2.1 would be in use then, unless there's some RK1 design choice I'm not yet aware of
m

mvdk85

08/05/2023, 9:10 PM
And so there'd be room to use another on the board. You'd just have to find room on it 😉
u

_dhanos_

08/05/2023, 9:10 PM
loks like there is indeed 1x PCIe 3.0 x4 and 3x PCIe 2.1 x1
m

mvdk85

08/05/2023, 9:11 PM
It has a lot of connectivity for a tiny SoC 🙂
u

_dhanos_

08/05/2023, 9:11 PM
The room is going to be a problem - as far as I know it was a challenge to fit everything on the module of this size
I would not be surprised if the additional PCie lanes might be exposed on the connector for future use
c

cfsworks

08/05/2023, 9:12 PM
What are the i225v's cooling requirements?
m

mvdk85

08/05/2023, 9:12 PM
It's got 1.8W TDP, I think?
1.3, sorry
i226v is the norm these days
c

cfsworks

08/05/2023, 9:13 PM
Might be able to squeeze it on the back of the module somehow, and just go bald (no heatsink), but still that might be hard to fit.
m

mvdk85

08/05/2023, 9:13 PM
7x7mm
And expected discontinuance in 2032.
c

cfsworks

08/05/2023, 9:15 PM
Integrated PHY too? It goes straight from PCIe to 2.5GBASE-T?
m

mvdk85

08/05/2023, 9:15 PM
Yep
It says "NBASE-T" as the iface supported, so yeah, direct to PHY.
I'll just check the BOM of a card that uses it, but I'm pretty sure there's no external PHY
There's also the RTL8125BG, which seems to have fewer reported issues.
Looking at the card, it definitely has an integrated PHY, but it's harder to tell what the TDP is.
Yeah, it seems most of the bugs in the Intel thing are related to "EEE", that is, the energy saving part of the spec, so I'd say that the realtek chip is actually the better of the two, probably, assuming that the TDP is reasonable.
Oh, found it - https://datasheet.lcsc.com/lcsc/2205121200_Realtek-Semicon-RTL8125BG-CG_C3013605.pdf - p44, maximum operating supply current. That gives a TDP of about 1W, probably less under the specific conditions that we're running.
b

blackphoenixx85

08/15/2023, 8:52 AM
For a future TPI I would like to have the LED's of the nodes not be on the board, or they have to at least have some FRONT IO for the Node LED's so it would be possible to show the Node LEDS externally on a case. Instead of having LED's next the board, would be more efficient to simple have a 8 pin 4x2 size connector, STATUS+GND for each node where you have to connect your own LED's, maybe saving on some production costs even.
s

srcshelton

08/30/2023, 5:39 PM
Blinkenlights!

https://cdn.discordapp.com/attachments/1090193720294522900/1146499562148724757/image.png

t

thegoldbug

09/01/2023, 5:04 AM
😀
b

blackphoenixx85

09/05/2023, 12:05 PM
Support for PWM Fan connector instead of the current 2-pin 12v connctor with temp sensor, so you can hook a Noctua Fan Controller directly to the motherboard
d

dirtywretch

10/03/2023, 2:36 AM
What he said. I ordered a Noctua fan for my case while waiting for the board and now trying to figure out how I'm going to make that work.
t

teslamax

10/03/2023, 2:37 AM
Got the whole "Thinking Machines" thing going.
c

cfsworks

10/03/2023, 2:40 AM
https://github.com/turing-machines/BMC-Firmware/pull/125 may be of interest to you and @blackphoenixx85
b

blackphoenixx85

10/03/2023, 2:41 AM
@cfsworks thanks. Will look into it. Right now I have a noctua FC1 controller so I can control it manually
d

dirtywretch

10/03/2023, 2:46 AM
Thanks for that update. I don't know where that leaves me since I don't understand the gestalt of that, but I'll take it as SOL and that I should look at the FC1 controller. Thank you!
c

cfsworks

10/03/2023, 2:48 AM
If you're comfortable soldering, you can order an EMC2301 chip and Molex 4-pin fan connector and get PWM fan control from the BMC with your current TP2 board. If you want to go that route (I did, works well for me).
d

dirtywretch

10/03/2023, 3:10 AM
I've never soldered anything before, so, not excited with that idea. In Geerling's "Racking" video from last year, he made it sound like the four pin connector he shows would have pwm control when the board went into production. "I know they're working on it." He was using a Noctua fan.
b

blackphoenixx85

10/05/2023, 8:43 AM
I went with Noctua Chromax 140mm Fan with a Noctua FC1 Fan Controller
j

j.lec

10/06/2023, 9:12 PM
Can you elaborate more on this setup?
d

dirtywretch

10/07/2023, 4:34 AM
I ordered the Noctua FC1. I have two Noctua 120x25 fans, one of which I'm going to return to get a thinner one (25mm > 15mm) to put at the top of the case after I order a fan bracket. I'd like to put a fan over the motherboard, but would need standoffs and a larger fan. Maybe with the top and bottom fans and the heat sink fans the thermals will be fine.
p

pink6

10/08/2023, 3:07 PM
Awesome. What expansion card are you using?
t

terarex

10/08/2023, 3:26 PM
For nodes 1 and 2, install a Mini PCIe card containing either a ASM1061 (2 SATA ports) or ASM1064 (4 SATA ports). Stay away from the ASM RAID cards. They don't work. ASM1061-based mPCIe cards are 2230. ASM1064-based mPCIe cards are 2242. With the latter you'll need to insulate or remove the standoff at the 2230 position to prevent it from shorting solder points on the bottom of the card. node 3 has access to 2 SATA ports from an on-TPi2 ASM1061. Note: you may need to install the AHCI/SATA driver package. Symptom is Linux doesn't see the SATA drives, but "lspci" shows the presence of the ASMedia controller
p

pink6

10/08/2023, 4:25 PM
Thanks Dan! When I get the RKs I want to set this up in a similar way. I want the ability to backup/RAID my drives too.
s

srcshelton

10/14/2023, 10:21 PM
I need to finalise the software and push it to GitHub - but it's basically the same as using using the GPIO pins to drive a directly-connected fan, just that the fan's actually connected to the furthest node...
p

phantom79

11/04/2023, 7:14 PM
Would love to see the SATA / PCI HIGH-speed USB-3 etc... shareable / assignable to any of the 4 slots ( thru firmware - just like we assign the usb2.0 port to any of the slots) - dunno how feasible this is - but right now having the SATA stuck to 1 node is limiting....
t

the_1st_bodybagger

11/04/2023, 8:07 PM
I'd like to see a method to connect from the BMC serial port to those of the nodes. I have an OrangePi One sitting on my M-iTX case, connected to a USB serial cable. I can SSH into my OrangePi (Debian) and use screen to connect to the BMC so it's acting as a serial terminal server. It would be nice to see a similar method to connect to the node serial ports while attached to the BMC serial port.
s

scienceman.

11/04/2023, 8:40 PM
You mean like a built-in KVM switch? I don't think that's within the hardware capabilities of the current TPi board, though I could be wrong. (Don't think so, though.)
t

the_1st_bodybagger

11/06/2023, 8:27 PM
Not quite a KVM, a serial switch, connects multiple serial ports to ethernet. I used to use them to connect to VAX servers. Can still purchase a dedicated one, but very $$$$! Perle still do them. They are also used today for out of band communication to physical network equipment in some datacentres https://www.perle.com/products/iolan-stg-terminal-server.shtml I could theoretically connect the GPIO to all 4 of the other UART ports on the board and as long as that device was functional I could SSH into it and use screen to connect to the other devices "debug" ports. Startech do a 4 port to USB one which I suppose I could connect to another raspberry pi device https://www.startech.com/en-ie/cards-adapters/icusb2324 Or I could just get 4 uart to USB cables and connect them direct to a raspberry pi... It would be nice if that functionality could go into the next revision TPi 3 or whatever
t

terarex

11/26/2023, 5:45 PM
I'm hoping Turing Machines retains and populates the 4-pin, PWM fan header and the EMC2301 (at U109) fan controller on the v2.5 board. Retaining the existing 2-pin fan header would also be useful. Adding default PWM support would eliminate the need to acquire a separate and extra cost variable speed fan controller.
u

_dhanos_

11/26/2023, 6:07 PM
TPi2 v2.5 is going to contain a PWM-controllable fan 🙂
t

terarex

11/26/2023, 9:01 PM
Excellent! Thanks for the info.
d

dethtungue

11/27/2023, 9:34 PM
If it hasn't been requested already, please put the board's SD card on top of the board. It is a real pain to access once the board is installed in a case.
p

phantom79

11/28/2023, 12:22 AM
+1 on this - particularly if we plan to make firmware install via sd card - it’s gonna be accessed more frequently- unless of course the official case makes it a breeze to access this via back plane along with the m.2 slots
s

srcshelton

11/28/2023, 8:34 PM
I used an SD extension cable so that I've got a tethered SD card socket elsewhere in my case!
g

gperdomor

12/19/2023, 3:07 PM
@_dhanos_ There is any plan for a 2.6 version, i don't know for include more bandwidth for Web UI flashing and improve times, or more capacity for the BMC, or fix any know issue that exists right now?... I want to decide if i should buy now or is better to wait a little more
t

terarex

12/19/2023, 4:48 PM
Not that we have much data, but Turing Machines seems to have a roughly 14 month cadence for new TPiv2 board revs. The v2.5 version will double off-chip BMC NAND storage from 128MB to 256MB. It will also address the known "issues" with the v2.4 board: RTC CR2032 battery; USB generally; system PWM fan control; and Node 3 SATA controller #clkreq connection. At one time, doubling BMC (AllWinner T113-S3) RAM from 128MB to 256MB seemed possible with the T113-S4 chip. Unfortunately, that product is either in very short supply and/or has been sold to a third party.
s

senkins

12/19/2023, 5:13 PM
Would be good to see replaceable modules for eMMC / NAND storage - should they fail for any reason, its not a simple job for someone to replace them at home, when soldered on. Something like IMO would work great for the RK1 ( Or i guess a "Lite" version with SD card ). Not sure what is out there for NAND though
k

kevinstock

12/31/2023, 12:27 AM
I'd love to see an alternative adapter board release for the CM4 that connected to the M.2 slots. Having to deal with Mini PCIe to SATA connectors to connect with storage for all nodes is a pain and M.2 would be MUCH nicer.
l

lamawithonel

12/31/2023, 7:23 AM
Please remove the M.2 2260 standoffs. I want to mount some NVMe storage with heatsinks for my Orin NX modules, but to do so I need to remove the standoffs. I've seen others mention they cannot use double-sided NVMe for the same reason. Removing them is currently very difficult, requiring some very hot desoldering work. In their place you could add through-holes with removable nut and bolt-style standoffs, like on this PCIe adapter. https://www.amazon.com/dp/B09VGTMX7W
u

_dhanos_

12/31/2023, 9:11 AM
We may make such an adapter in the future, but for now, there are no plans to do so.
This is one of the changes made with Turing Pi v2.5 board: https://docs.turingpi.com/changelog/turing-pi2-v25-list-of-improvements
r

rogue357.

01/01/2024, 2:29 AM
So when is v2.5 coming out? I still havent recieved mine yet. Would i get a 2.5 version?
u

_dhanos_

01/01/2024, 9:49 AM
The current estimate is February. And yes - you'll get the updated board revision.
a

ajtzak

01/05/2024, 1:03 AM
le
s

srcshelton

01/05/2024, 2:09 PM
... any news yet on offering a (cost-price?) replacement (or at least a solid discount?) to existing backers/owners who would like to upgrade to the v2.5 product?
s

scienceman.

01/05/2024, 3:15 PM
As a customer with a 2.4 board, I'm not sure how that would work economically. The new board would still have to be built at the same cost, and there aren't any good mechanisms to recycle or reuse components between the boards that I can think of. I don't know the extent of the changes but undoubtedly they involve alteratiosn to the internal traces of the board, and if upgrading functionality in place with the earlier boards were possible, that would have been tried on these boards by the community, no doubt. So apart from the desire to have a good-will discount for some reason, I don't think there is a good path for the project to offer such an upgrade path. Am I wrong? Are there any parts to be recaptured by returning a previous-generation board that are worth the effort to remove them?
s

srcshelton

01/06/2024, 1:40 AM
Yeah, I was thinking of a good-will discount - I doubt that remanufacturing v2.4 boards would be at all economic, if even feasible...
r

rogue357.

01/23/2024, 4:43 AM
Is it possible on a future revision duplicate functionality for nodes 3 and 4... ie they both have 2 sata and 4 usb ports. Or each node has 2 usb ports and 1 sata?
I am sure hoping the CM5 will fix the M.2 NVME port issue.
u

_dhanos_

01/23/2024, 10:53 AM
There is only a single PCIe lane available. Node 3 connects a SATA controller to it (that exposes 2 SATA ports) and Node 4 connects a USB controller (with 4 USB ports). To add multiple devices a PCIe switch would be necessary and more PCIe controllers. This is, of course, possible, but currently, there are no plans for such features.
t

terarex

01/27/2024, 8:56 PM
Just a random idea for some future TPiv(?) would be replacing the built-in SATA and USB 3.0 ports on node 3 and node 4 with mini PCIe slots. I don't know whether there is enough board real estate to pull off, but, with the v2.5 board improvements, this change would make for extremely flexible I/O configurations. There are tested mPCIe boards offering 4x USB 3.0, 4x SATA or, hopefully soon, 2x 2.5Gb Ethernet. I have this tonight: https://a.co/d/fERX2jw @_dhanos_ we, in the US can order it from Amazon. Apparently, Amazon EU is out of stock. If a contribution for QA/support would be of value, DM me.
u

_dhanos_

01/28/2024, 7:21 PM
Interesting idea and I thought of this too. We'll keep it in mind, but I have also thought of using PCIe switches. As for this network card - yes, I might need to buy it off Aliexpress. I would like to test one myself too. But yes, I'm open to seeing your test results too.
I also tested myself multiple USB 3.0 and SATA MiniPCIe cards, so indeed they could be used
u

_dr.smart

01/29/2024, 2:46 AM
considering the board real estate, will this require significant redesign?
u

_dhanos_

01/29/2024, 2:47 AM
What do you mean by "this" exactly?
u

_dr.smart

01/29/2024, 2:49 AM
I mean the requirements in order to fit each mPCIe slot for node 3 and 4
u

_dhanos_

01/29/2024, 2:49 AM
There's no really space for them currently
u

_dr.smart

01/29/2024, 3:29 AM
As far as I can tell with the current board layout, trying to add 2 mPCIe slot for node 3 and 4 would result in a completely new board that doesn't fit the mini-ITX form factor
At that point might as well call it Turing Pi 3
u

_dhanos_

01/29/2024, 3:30 AM
It still may be possible to fit them in the Mini ITX form factor, I have some ideas and the engineers will have some too
Other than that it's hard to say anything else. I personally like the idea
u

_dr.smart

01/29/2024, 3:36 AM
If the current form factor doesn't have space, a new form factor like microATX (244 × 244 mm) wouldn't be so bad
The LicheeCluster board is still using the mini-ITX form factor though
u

_dhanos_

01/29/2024, 3:48 AM
People have been asking about a board that can carry more modules.
I, of course, do not have any information about the possibility of such a board from Turing Machines existing in the future, but if this happened, most likely won't be a Mini ITX if we want to have some more functionality than just packing 7 modules on a MiniITX board
u

_dr.smart

01/29/2024, 3:53 AM
I think so as well, if one would willing to use such a massive form factor like ATX, might as well pack some additional features other than increasing the number of compute modules
u

_dhanos_

01/29/2024, 3:53 AM
Yes, I mean storage, PCIe/NVMe, SATA, USB, MiniPCIe, like on the TPi2 right now. And the BMC chip, of course
u

_dr.smart

01/29/2024, 4:01 AM
With how the current compute module are set up (perpendicular to the board), I wonder if it's possible reach the amount of expansion slot the Compute Blade have when using ATX form factor though?
Especially the Zymbit in the expansion module slot
u

_dhanos_

01/29/2024, 4:03 AM
Compute Blades, if nothing changed, are boards for CM4s and there's really not this much expansion because of a single PCIe lane
u

_dr.smart

01/29/2024, 4:05 AM
Yeah, the single PCIe lane of RPi is really limiting it's full potential, but with different compute modules in the Turing Pi, there's still space for improvement
u

_dhanos_

01/29/2024, 4:05 AM
Of course, there's a lot that can be done still
u

_dr.smart

01/29/2024, 4:06 AM
If it's possible to add different PCIe cards like NICs, DPUs and GPUs, this ATX super board would definitely be the attention
u

_dhanos_

01/29/2024, 4:07 AM
You can do that with both MiniPCIe and M.2 - the former exposes a single PCIe 2.0 lane and the latter up to 4 PCIe 4.0 lanes (depending on the module)
u

_dr.smart

01/29/2024, 4:09 AM
But it would need additional accessories the the breakout board & cable, the ATX super board might as well add a PCIe slot even if it means one less compute modules right? Who knows 🤔
u

_dhanos_

01/29/2024, 4:10 AM
Well, if you pack multiple modules on a board, you won't have space for full ATX cards IMO anyway. MiniPCIe is more practical and can be adapter to the full PCIe is you need that\
u

_dr.smart

01/29/2024, 4:13 AM
I agree, I wonder if this board would ever get released, will it change how the ecosystem of compute modules work, I mean will it encourage manufacturer like Radxa to try to match the specification to fit into the Turing Pi ecosystem?
As far as I know, the current compute module ecosystem is very fractured, with each manufacturer have their own carrier board and different OS
u

_dhanos_

01/29/2024, 4:15 AM
The OS is a module thing. And the boards will obviously differ since the goals might differ
The above are my own thoughts, I, obviously, don't have any info I can share publicly about the future TPi boards yet
But a brainstorm like this can yield interesting ideas and creates many thoughts of things to think on later
u

_dr.smart

01/29/2024, 4:17 AM
But ultimately, it would depend on whether someone willing to fund this ATX super cluster board
@_dhanos_ Btw, do you guys have any plan on experimenting with RISC-V?Rockchip has some impressive CPUs but on RISC-V side, there's SophGo as well.
Although they haven't have any impressive SoCs for international market yet
u

_dhanos_

01/29/2024, 4:23 AM
No information on that. And yes, I haven't seen any impressive RISC-V chips yet, but there are some really nice ones, like these that Jeff covered in his videos (the massive-multicore ones)
u

_dr.smart

01/29/2024, 4:26 AM
As far as I know, SophGo is about to release their 2.5 GHz 16-core RISC-V CPU named SG2380
Whether they have LTS Linux OS is another story, they're still designing the SG2380 architecture though
Most of the RISC-V CPUs are using the CPUs/cores from SiFive, they've just announced their P870/P870A core (expected clock speed to 3GHz), haven't seen any CPUs that uses this core yet
t

teslamax

01/29/2024, 4:44 PM
t

terarex

01/29/2024, 7:01 PM
I did see that forum thread. For my purposes, routing through a node adds complexity and creates a single point of failure. I'd rather use a separate device, but "YMMV". Regardless, I now have a CM4 node running the latest Raspberry Pi-distributed Ubuntu Server 23.10 (64-bit) image. The IO Crest 2x 2.5Gb mPCIe card seems to be working fine. I'll try the latest Raspberry Pi OS (64-bit, Bookworm) as well. I also have I have NVIDIA Jetson Xavier NX and Orin NX modules that I'll try. There, I'm more limited by what NVIDIA supports. Any RK1 testing will have to wait until Turing Machines ships the 32GB modules I have on order (April-ish, I hope). Since I only have 1Gb switches on hand, someone else will need to do performance and stability-under-load testing. Update: I've installed and compatibility checked Raspberry Pi OS Full (64-bit) (Bookwork) with the IO Crest Dual 2.5Gb mPCIe card. A required driver was not available. This was evidenced by "lspci" returning nothing. However, after doing "sudo apt upgrade" and rebooting, the appropriate drivers, including r8169, were present. The two interfaces appear in "ifconfig -a" as eth1 and eth2.
t

teslamax

01/29/2024, 10:15 PM
When I get my RK1’s I will have two spare CM4 so that’s also an option for me.
s

srcshelton

01/31/2024, 12:45 AM
Perhaps add some/all of the mPCIe sockets to the rear of the board (which would also help to keep the traces short?) - potentially in place of the M.2 slots (since IIRC mPCIe to NVMe adapters are passive components)?
u

_dr.smart

01/31/2024, 2:09 AM
you mean on the other side of the board?
I think there might be issues with mPCIe cards with antenna connectors though, it might add unwanted annoyance during installations of those cards
I made a mock-up of the microATX equivalent of Turing Pi 2 based on the image on the store and @terarex's idea, capable of 6 nodes and each node have its own mPCIe port. what do guys think? https://cdn.discordapp.com/attachments/1090193720294522900/1202625636070330450/mATX-scheme.png?ex=65ce2367&is=65bbae67&hm=dcc1c84aa145f06106fb3c4b2365f5ab7a9c42b306624dafb7d5046a3f03e4a8&
It's still missing a lot of stuff but I still planning to add more
t

terarex

02/01/2024, 4:06 PM
Glad to see it's possible. Considering the form factor change, I believe it would be helpful to put all USB ports and the microSD slot on the rear to ease access. The M.2 slots are fine on the bottom because they should only need to be accessed during installation or replacement. If Turing Machines is considering this type of product, the board and enclosure(s) should be designed and prototyped at the same time.
u

_dr.smart

02/01/2024, 4:12 PM
I'm thinking of moving the last 2 mPCIe slot to the same area as the other 4s, I wonder if this form factor have enough real estate to pull it off though?
I'm also designing a mock-up for the ATX equivalent as well, any idea contributions are welcomed
I also have that idea in mind, but I haven't add it in the mock-up yet, in case someone have additional features/ideas
Similar to the microATX, this form factor can have up to 8 nodes, in theory
@terarex If you have any ideas, just tell me more and I'll see if I can add it in the mock-up
But this is just a mock-up, I think there'll be significant changes in the layout if Turing Pi decided to work on this board
t

terarex

02/01/2024, 6:02 PM
Oh yeah. 😉 Definitely not a 2024 product.
u

_dr.smart

02/01/2024, 6:09 PM
Any other ideas for the mock-up beside the moving the usb ports to the rear?
The side where the SSDs are installed have a lot of free space, I wonder if are there anything can be added there?
t

terarex

02/01/2024, 6:57 PM
Yes, free surface space, but interior trace space may be a different matter. I don't know how many layers there are in the TPiv2 board. More board layers increases manufacturing cost and complexity.
n

neggles

02/01/2024, 10:42 PM
can i request that the resistor values for the power/link activity LEDs (leds in general tbh) be bumped up to more like 1K than the ~470R i assume they currently are? they're eye-searingly bright :/ gonna have to swap them out for higher resistance ones on my board
t

thegoldbug

02/02/2024, 1:15 AM
make sure the mounting holes are in the correct, standard locations.
u

_dr.smart

02/02/2024, 1:25 AM
Do you have any idea on where can the ethernet switch can be moved on the mock-up
I thinking the mock-up should have a 2.5Gbps switch chip and it requires space and mounting holes for a proper heatsink
n

neggles

02/02/2024, 1:37 AM
skip mPCIe imo and just go right to key-E or key-B m.2 slots for wifi/5G modules, gives you room for one per module on the right put the switch chip where you currently have the BMC with its over-the-top mPCIe
t

terarex

02/02/2024, 1:37 AM
In theory, that would be nice. Do currently available SoMs support 2.5Gb Ethernet without the addition of a separate controller? More interesting to me would be a PCIe switch.
n

neggles

02/02/2024, 1:38 AM
do you mean a pcie switch or a pcie matrix multiplexer (ie route different lanes to different modules)
and no, current modules have 1G PHYs on board and while Rockchip originally expected the RK3588 to handle 2.5G on one of its MACs, that didn't pan out in realityh
Orin Nano/NX are the same
t

terarex

02/02/2024, 1:42 AM
A PCIe switch is what @_dhanos_ was thinking. I don't think any available SoM has RDMA capability, but having a switch could make NUMA an option.
n

neggles

02/02/2024, 1:43 AM
no currently available PCIe switch chip that's remotely close to cheap enough to put in a board like this has multi-host/NTB support you could potentially use EP mode on three RK1s and host mode on one, or daisy-chain them with 2 lanes up and 2 lanes down like Mixtile do, but a generic switch doesn't help you here (they only support having one host)
and afaik the best you can do with the RK3588 PCIe endpoint driver right now is a pretend-o-ethernet
t

terarex

02/02/2024, 1:46 AM
Yeah. It's complex. In a past life, I worked with both SGI UV and Cray XC systems, AKA supercomputers.
n

neggles

02/02/2024, 1:48 AM
the problem is that pcie is a host-device link, host-to-host links require Shenanigans™️ like non-transparent bridges
and chips which can handle four x4 links to hosts are few and far between
RK3588 is capable of being a PCIe endpoint (device) on the 3.0x4/x2x2/x1x1x1x1 controller so you can build a sort of ring bus
that's what mixtile are doing here https://www.mixtile.com/blade-3/ - two module link works by just putting one into device mode, the 4-module stack does two counter-rotating 2-lane rings basically
t

terarex

02/02/2024, 1:51 AM
Yeah. I'm thinking RDMAoCE. Infiniband involves yet another fabric.
n

neggles

02/02/2024, 1:51 AM
technically if you're using PCIe EP mode you already have RDMA
but yeah one of the most straightforward and common ways to use PCIe endpoint mode, or a PCIe NTB chip, is to present a virtual ethernet interface with RoCEv2 support
but its all down to what you do with those links
and in any case what mixtile did is not the sort of thing you could put on a general-purpose cluster board because it's highly specific to the SoC used on the module 😦
you could look at doing something like a scaled-down version of https://www.dolphinics.com/products/PXH810.html and an external pcie fabric switch, but small chips for that role just don't exist and the pricing is extremely prohibitive
looks like of currently available chips the smallest is the Microchip PM8561
which is US$100 in 1ku quantities
ah nope sorry only has two NTBs, would need PM8531B/PM8571 which is $110-130 which is probably about the whole BOM cost of a tpi
t

terarex

02/02/2024, 2:15 AM
Yup. NVMeoF could be more interesting from a storage perspective. Before I retired, I worked in the storage systems group at Western Digital. Was hoping they would make major advances with the Kazan Networks technology, but WDC has OEM HDD DNA. (sigh)
n

neggles

02/02/2024, 2:18 AM
yeah WDC are making some interesting choices lately... anyway this is a bit off topic for the thread
t

thegoldbug

02/02/2024, 5:28 AM
Dan, I can hook you up with some SGI gear if you need/want a "fix". Challenge XL, Desksides, Octanes, Indigo 2s, Indigos, Indys, O2s or how about a Fuel.
t

terarex

02/02/2024, 5:31 AM
Thank you, no. While rewarding monetarily at the time, I have other challenges to expend my time on. Not the least of which is this tiny, very loosely coupled, four node cluster. 😉
u

_dr.smart

02/02/2024, 5:09 PM
as much as I like the M.2 form factor for its higher speed, more PCIe lane but the fact that you can plug any mPCIe card in without worrying about which socket are compatible with which type of card just make replacing the mPCIe with M.2 strange IMO
I wonder B-key or E-key have enough types of expansion card to warrant this replacement
I'm thinking of adding an B+M M.2 slot on the left, for SSDs, full size (80mm) AI accelerators
I wonder if it's possible to add since there are physical buttons and UART pins on the current board
s

scienceman.

02/03/2024, 4:37 AM
Do you know anyone who has done this? How does the cabled setup compare with the Mixtile Cluster Box and its custom backplane with respect to PCIe connectivity?
n

neggles

02/03/2024, 11:59 AM
B key is every M.2 LTE or 5G card and most 2-lane NVMe drives E-key is wifi
that is the mixtile cluster box
it's using the 4-lane PCIe PHY in 2x2 mode with one pair of lanes in PCIe endpoint mode and the other pair in host mode
two lanes up, two lanes down
if you want it to only support NVME drives you do key-M since all 2-lane M.2 NVME drives are key-B+M or key-M
also keep in mind that on these boards you have one 3.0x4 PHY and two 2.0x1 PHYs to play with
less if you want SATA or more USB3
i had a better one somewhere
u

_dr.smart

02/03/2024, 12:06 PM
Yeah, but since this is just a mock-up for a board that might exist in the future if Turing Pi decided to work on it
Hopefully at that point the SoMs and compute modules have improved even better
n

neggles

02/03/2024, 12:07 PM
they kinda already have but if you want better you have to spend $400-600/module on Orins
and since Rockchip still haven't even announced the RK3899 or whatever they end up calling it it'll be a While™️
u

_dr.smart

02/03/2024, 12:22 PM
The only reason I was thinking about when I added this slot was the potential uses of AI accelerators like Hailo-8 M.2 AI Acceleration Module or Axelera M.2 AI Edge Accelerator Module, the SSDs is just an afterthought https://www.axelera.ai/wp-content/uploads/2023/09/M.2-AI-Edge-accelerator-Module.pdf https://hailo.ai/products/ai-accelerators/hailo-8-m2-ai-acceleration-module/
As much as I like Orins, I'd like to wait for more potential releases from Rockchip or RISC-V manufacturers
n

neggles

02/04/2024, 1:46 AM
these will also not work with RK3588 fwiw and the internal NPU is about as powerful in reality (hailo has mem bw issues)
u

_dr.smart

02/04/2024, 1:54 AM
Where did you read about Hailo's memory bw issue? Can you send me a link?
n

neggles

02/04/2024, 9:18 AM
I'm not aware of anything public but it's a limitation of any AI accelerator without local high-speed memory; Hailo-8 only has a small amount of local on-die SRAM to work with
anyway -> #754950670645329920
u

_dr.smart

02/05/2024, 10:26 AM
An additional I want to see in the future is a Type C PD Pico PSU
t

teslamax

02/05/2024, 8:06 PM
I was looking for such a beast earlier…
I don’t know if it’s an idea that’s been vetted by EE best practices, but I did think about it myself some time ago
u

_dr.smart

02/06/2024, 1:24 AM
On SBCs, it's quite common to use USB-C to power the device, what's the difference in using this for the Pico PSU?
As far as I know, the board doesn't draw that much power either?
Is it possible for each node to have 2 SATA ports in mATX/ATX form factor?
I got this idea from Ambedded's Mars 400PRO
Each node in this server have an eMMC, SSD and SATA or SSD as an OSD disk
t

teslamax

02/06/2024, 9:18 PM
That depends upon the CPU/architecture of the node in question; usually the limitation is based upon how many PCIe interfaces there are
This can be alleviated by a PCIe switch chip per node… but I’m not experienced as to how complicated that’d be or how much overhead there is in using such a switch
The board doesn’t consume much as do the CM4 nodes. You are talking about using four nodes in the current TP2 however. If you use nVidia nodes or RK1’s along with all NVMe slots filled, mPCIe slots used, and SATA storage devices, the combined current draw could be a problem. Newer USB PD specs cover the larger wattage ranges, but are more expensive. I still think it’s a good idea!
tl;dr higher current/wattage draw is the reason I’ve read for USB PD/pico PSU’s not being a thing for now.
u

_dr.smart

02/07/2024, 2:00 AM
I think it's almost the same as node 3? Node 3 already have access to an M.2 and 2 SATA ports (depends on the compute module)
The main issue here is whether is it possible to fit additional mPCIe expansion cards for other nodes or not
Or the simple solution is just use a mPCIe to SATA card and each nodes have its own mPCIe slot like @terarex's idea
But doing that without adding SATA would be wasting wasting the storage interface on RK3588
Trying to cram 8 nodes x 2 SATA ports/node = 16 SATA ports into an ATX form factor would end up in taking up all the space for additional I/O for any other purpose
t

teslamax

02/07/2024, 9:49 AM
Using a compute module with more PCIe interfaces or PCIe switches would work
u

_dr.smart

02/07/2024, 10:37 AM
There's also the problem of available space on the board to fit the mPCIe slot as well
From what I've seen so far, longer mPCIe card might not fit if the rear of the board have I/O ports like SATA etc.
j

joesroom

02/07/2024, 12:55 PM
could there be a way to slot mpcie vertically, like full size pci?
u

_dr.smart

02/07/2024, 4:41 PM
I highly doubt it, and it introduces a bunch of problems about card stability for connection
t

teslamax

02/08/2024, 1:20 AM
some sort of riser card perhaps.... but overall I think it's asking for trouble to mess with all of this 😉
a lot of redesign work would be involved
Hypothetically, you could connect something similar to this to a picoPSU DC input, but this example is limited to 100W; You’d need an IC that supports the full USB PD 3.1 spec including 36/48V to support a higher wattage. (Turing’s Pico-PSU is 160W rated.) https://learn.adafruit.com/adafruit-husb238-usb-type-c-power-delivery-breakout/overview
You’d likely need a high rated USB-PD power supply and a high current rated USB-C cable
u

_dr.smart

02/09/2024, 6:43 PM
I know the DIY solution will definitely work if I have the skills to pull it off
But I would like to see such product in the future as well, it might be able to use in other motherboards as well
That's definitely a requirement if you don't want to burn your house down
t

teslamax

02/09/2024, 6:45 PM
there are folks in the SFF PC community that have been asking for such an animal for years now… myself included
I have a Flex ATX or similar inside my 2U enclosure for my TP2 atm
u

_dr.smart

02/09/2024, 6:47 PM
Any plan to upgrade to redunant?
t

teslamax

02/09/2024, 6:49 PM
I’m a hobbyist so not at the moment, my Dell R530 only has one PSU…
u

_dr.smart

02/09/2024, 6:57 PM
PowerEdge R530?
t

teslamax

02/09/2024, 11:22 PM
Yup
u

_dr.smart

02/10/2024, 1:34 AM
Do you use all the HDD slot of the case with TPi?
t

teslamax

02/10/2024, 2:53 AM
None atm
u

_dr.smart

02/13/2024, 7:21 PM
And also the possibility of cramming 2 SATA ports for each node while retaining the expansion card slot for each node
t

terarex

02/14/2024, 4:19 AM
My original suggestion was intended to preserve compatibility and improve I/O expansion flexibility, while cost-reducing and simplifying the board by removing built-in SATA and USB from nodes 3 and 4. Let's try to KISS.
u

_dr.smart

02/14/2024, 5:00 AM
I'll agree on the KISS aspect of the future versions of the board
According to the specification, the RK3588 has three PCIe 2.0/SATA 3.0 ports
Unless Turing Pi decided to ultilize the alternative PCIe interfaces of SoCs' SATA interfaces (If those SoCs have them)
Removing the SATA ports on the ATX version would be wasting the high speed interfaces of the SoC
t

terarex

02/14/2024, 10:07 PM
@_dr.smart After doing a little research, I found that the Jetson Xavier NX pin mux supports 1x + 4x PCIe lanes. While the RK3588 and the Orin NX support more PCIe lanes, the SoM edge connector does not.
t

teslamax

02/14/2024, 11:22 PM
How much overhead is there when using a PCIe switch anyway?
(Besides the usual “it depends” 😉 )
u

_dr.smart

02/15/2024, 3:33 AM
In short, the number of available PCIe lanes is limited by the SoM's physical connection?
t

terarex

02/15/2024, 3:36 AM
Yes. I suppose signals not used on the TPiv2 board could be reassigned, but that would break compatibility. Turing Machines needs to maintain compatibility with the Jetson Xavier NX pin mux in order to sell RK1s beyond their captive motherboard community. I don't work for and am not compensated in any way by Turing Machines.
u

_dr.smart

02/15/2024, 6:18 AM
Is there any way to overcome the 260-pin edge connector limitation?
t

terarex

02/15/2024, 6:29 AM
I wouldn't call it a limitation. It is "as-designed" to be compatible with NVIDIA Jetson SoMs. The Mixtile Core 3588E SoM claims to be Jetson TX2 NX plug compatible, so it probably uses the same or a similar pin mux. The FireFly Core-3588J SoM uses a different, SO-DIMM edge connector format (https://en.t-firefly.com/product/core/core3588j), which supports all the RK3588 features. It is essentially proprietary at this point in time. Frankly, the RPi CM3, CM4 and NVIDIA Jetson NX are the only SoMs with enough market presence to be considered du jour standards. Turing Machines chose the Jetson NX connector because they could build a CM4 adapter for it. There are various, other, CM4-style modules that can (somewhat or fully) work on the RPiv2 CM4 adapter.
u

_dr.smart

02/15/2024, 6:47 AM
Both RK1 and Jetson use 260-pin SO-DIMM connector
But the Firefly Core-3588J uses MXM3.0 314 pin connector though?
t

terarex

02/15/2024, 7:05 AM
Yes, a completely different interface. Mixtile is doing their own 4-nide cluster employing a PCIe switch and U.2 form-factoe NVMe SSDs. NVIDIA preserved some degree of compatibility with pre-Orin carrier boards because there is a large installed base that they wanted to sell into. The post-Orin NVIDIA SoM may break that compatibility.
u

_dr.smart

02/15/2024, 1:12 PM
@User But does more pin on physical connector on both SoMs and carrier boards mean more SoC features ultilized?
@User Since as far as I know and from what I understand from your comments, the reasons why PCIe lane was limited on RK1 and Jetson Orin is for 2 reason: 1.Backwards compatibility with older Jetson products in Orin's case and compatibility with Jetson ecosystem in RK1's case 2.The number of physical pinout is limited so some high speed interfaces have to be sacrificed?
w

wvaske

02/21/2024, 8:49 PM
With my RK1 delivery I finally got to setting up my TPis and a few things I'd like to see changed (may be listed in this thread, I didn't read all the posts) 1. The BMC SD card should be accessible on the top of the board or via the rear panel. I only realized I'll need to remove and reinstall the SD card a couple times to update the BMC after installing in the rackmount chassis. Being rear-accessible would help quite a bit. 2. Support dual-sided NVMe devices. The glued in screw terminal for the shorter size effectively prevents installation of a long dual sided NVMe device. Use flush mount screw recepticles with a movable standoff to use for the correct size. 3. Support 110mm M.2 drives. There are few consumer drives but quite a few enterprise M.2 drives in this size. Lean in to edge/SMB use cases and support the enterprise variants (the bigger size is frequently required for power protection capacitors) 4. It'd be interesting to have POE++ input option. I expect it'd be difficult to fit on the available space and may require a "hat" similar to how PIs themselves get POE power. POE++ supports up to 60 or 100 watts. Definitely not a need-to-have.
t

terarex

02/21/2024, 10:03 PM
#1) agreed. #2) I don't know whether the v2.5 board utilizes fixed or removable standoffs at the 2280 locations. Hoping every standoff, including the mini PCIe ones, are removable. That way, alternate lengths can be deployed to easily support dual-sided SSDs with heat sinks. I desoldered all the standoffs on my v2.4 board because I have dual-sided M.2 NVMe SSDs with 3rd-party, active heat sinks. 4TB, TLC M.2 SSD variants seem to be dual-sided. QLC would fit on a single surface, but they have relatively low endurance and limited sustained write capability. #3) Probably not in the v2.X board series. #4) Good idea.
u

_dr.smart

02/22/2024, 4:22 AM
Did you use this board without a case? I think that if you're using TPi + SSDs + heat sinks, they wouldn't fit in the case.
t

terarex

02/22/2024, 4:25 AM
No. I'm using a Mini-ITX case that has been modified to use 25mm board standoffs. That is just about as long as you can go and still use a custom I/O shield.
u

_dr.smart

02/22/2024, 4:26 AM
POE++ would be a good idea for a board with 4 nodes (6 nodes at best), but beyond that, like an 8-node ATX board, I don't think that's the best idea
How resilient is 110mm M.2 SSDs compare to shorter ones?
t

terarex

02/22/2024, 4:46 AM
The discussion was related to a next generation TPiv2. Probably beyond v2.5, except for the mini PCIe and M.2 NVMe standoffs. A microSD extender can be used to address BMC microSD card accessibility.
At one time (pre-3D NAND), 22110 M.2 NVMe SSDs were envisioned as an enterprise product. They never took off and generally have been replaced by other, denser, form factors for enterprise storage applications.
u

_dr.smart

02/24/2024, 3:41 AM
With all the news about consumer motherboards with Intel's ATX12VO connector, I'm wondering whether this is a good idea for a board like TPi.
Additional space is required for SATA power connectors since the motherboards are going to do the power conversion instead of the PSUs, thus increase the manufacture cost
w

wvaske

02/24/2024, 4:40 AM
I work for Micron. 110mm M.2 is still used in enterprise for boot drives and is expected to be used for quite a while. E1.S isn't expected to replace boot drives for 3+ years
You can get 80mm enterprise drives (by enterprise I mean drives that have capacitors to ensure the cache is written in the event of a power failure). 110 is still "prevalant".
u

_dr.smart

02/24/2024, 5:34 AM
The only drawback of this form factor is it's bigger than the M.2, not ideal for surface mount on motherboard with limited space But is it possible for E1.S drives to replace M.2 drives in the consumer market, or this is just a thing in the enterprise/server market? As far as I know, there are already PCIe adapters for 5.9 and 8mm thick E1.S drives
w

wvaske

02/26/2024, 2:20 PM
My point is really that 110mm M.2 will be around for at least a few more years due to the enterprise use cases. E1.S is unlikely to replace M.2 consumer drives and consumer drives are unlikely to use the 110mm form factor. I think the only "consumer" use cases for E1.S will be in things like Icy Dock carriers. That actually raises the point, it might be nice to have pcie ports on the top of the board similar to the new pcie connector on the Pi 5 to support backplanes or other add in card use cases.
u

_dr.smart

03/01/2024, 3:57 AM
@terarex is also suggesting each node to have its own mPCIe slot for expansion cards
3. One minor issue, with the current SSD layout, in future boards with larger form factor like mATX/ATX, using 110mm would block some mounting holes on those boards or causing issues with standoffs
m

machinchose

03/04/2024, 9:28 PM
not sure if someone mentionned it yet, but you should upgrade the switch internal bandwidth to 2.5 gbps or 10gbps
t

terarex

03/04/2024, 10:40 PM
That's unlikely. Getting the documented changes for v2.5 to work reliably have already required one board respin (https://discord.com/channels/754950670175436841/754950670175436848/1213465607492734977). Hoping everything on the second version (v2.5.2?) works properly. Otherwise, there will be another ~2 month delay.
u

.flokli

03/10/2024, 3:17 PM
I'd love if the BMC would get more than 100mbit outside connectivity. It's a pretty big bottleneck when flashing. And yeah, either [LACP](https://github.com/turing-machines/BMC-Firmware/issues/173) on the switch port, or maybe some of these sweet 2.5 Gbit ports 😆
t

terarex

03/10/2024, 4:30 PM
The onboard switch chip, RTL8370MB-CG+, supports 1Gbit connections. The external connection speed is automatically negotiated with an external, upstream switch. If you're only getting 100Mbit, check the cable and switch.
r

rogue357.

03/11/2024, 7:22 AM
You know one this that would be helpful, would be to include a QiiC (i2c) and a default SPI connector (maybe 5 pins jst) for each slot. There is no way to get to minor peripheral parts pf these boards that can serve very useful purposes. Maybe a CSI/DSI connector also.
You know I have an even better idea... make another slot with same pinout that provides specialization, either PCIe. NVMe (addons), USB,, SATA, (onboard connectors). Maybe even of the same card. This would allow configuring as all PCIe or a massive raid video processor.
u

.flokli

03/11/2024, 10:39 PM
The board is negotiating 1Gbit currently, yes, but the BMC only does do 100mbit
t

terarex

03/12/2024, 12:12 AM
Okay. That's surprising since the Allwinner T113-S3 datasheet (https://bbs.aw-ol.com/assets/uploads/files/1648883311844-t113-s3_datasheet_v1.2.pdf) states the chip supports 1Gbe operation.
u

.flokli

03/12/2024, 3:56 PM
> [ 13.289919] sunxi-gmac 4500000.eth eth0: Link is Up - 100Mbps/Full - flow control off
t

terarex

03/14/2024, 11:24 PM
Yup. As stated previously, the T113-S3 datasheet indicates the chip is capable of 1Gbps. However, the TPiv2 BMC firmware buildroot includes support for the older A20 GMAC. There is some discussion of the A20 GMAC implementation here: https://linux-sunxi.org/Ethernet. Since the T113-S3 includes the sun8i EMAC, the BMC firmware could use the Allwinner sun8i EMAC driver (DWMAC_SUN8I). Not certain I have the skills to integrate it, but maybe someone here could help with pointers. I also found this: https://lore.kernel.org/all/20230721134606.4505-9-andre.przywara@arm.com/T/. @User could you comment?
c

cfsworks

03/15/2024, 12:17 AM
Sure, so, here's the situation as it exists today: There's the RTL8370MB-CG Ethernet switch, which has 8 gigabit UTP ports (with integrated PHYs, thus suitable for connecting directly to the two physical jacks, the Ethernet lines of each node, and the BMC's PHY). It also has 2 "extension" ports, which implement a \*MII interface, which is useful for PCB designers that want to provide their own PHY or if they want a direct PHYCPU connection. The TPi2 doesn't use these extension ports. Connected to port 4 of the switch is a RTL8201F 100Mbps PHY. It's the little chip between slot 1 and the BMC with the crab on it. This PHY only does 100Mbps operation, and implements the [RMII]() interface. This interface requires 11 pins, which are connected to pins PE0-10 of the T113-s3. To get a gigabit BMCswitch link going, it is necessary to use a gigabit-capable MII, most likely RGMII directly to one of the extension ports rather than to a PHY (which saves a chip). The caveat is that RGMII requires 4 additional pins (the tx+rx lanes are widened from 2-bit to 4-bit each), and the PE11+ pins are in use for I²C, so some stuff would have to be shuffled around. The other wild thing is that, if you look at the Allwinner D1 (same die, different package/blocks enabled) datasheet, the RXD3 signal is supposed to be on pin PE14, but the T113-s3 package only goes up to PE13. So to get RGMII going, alternate pins PG6-9 would most likely have to be claimed for that function... which means more moving stuff around. I'm unsure if I can say more without violating NDA, however all of the above is public knowledge (pulled from the relevant part documentation and easily learned from studying the board itself). Still, if you have any more questions, ask away and I'll answer if I can. 🙂
t

terarex

03/15/2024, 12:25 AM
@cfsworks thanks for that info. So, nothing we can do with the existing board or, likely, the v2.5 board.
r

rogue357.

03/15/2024, 12:26 AM
blah blah blah... can only do 100 mbs just because. wait till next version
t

terarex

03/15/2024, 12:31 AM
I've been thinking that some, future, Turing Pi cluster board should implement a fifth module socket for the BMC with the same form factor and pinmux as the other modules. Conceptually similar to what Sipeed is doing with the RISC-V Lichee Pi 4A Cluster, where they're using an Allwinner D1 for the BMC. Turing Machines already has the IP for the CM4 adapter.
r

rogue357.

03/15/2024, 4:11 AM
I was thinking the same thing for expansion slots next to compute comdules that have either m.2 or PCIe connectors (or both) and support hardware. All the components would be on opposite side of pcb so that compute module sits one direction and expansion board face the opposite direction.
a

adrians2087

03/30/2024, 7:45 AM
Interesting ideas here, and I am just starting a build so not really in a position to comment in an informed way. However my point of view is a bit contrarian. For me it would be amazing to have a simpler board. Why not just have, for each node, M2, usb3xn, ethernet. BMC should have usb and perhaps ethernet. And, optional, a fancy programmable usb hub to configure a node USB to a external USB physical port? But my real point is with each node having M2, ethernet and usb3 we can connect any "weird" devices - like a screen and keyboard 🙂 - to any node we like, make design easier (less options) and perhaps makes the board itself easier to optimise(?). Just my 2 pence (cents)!
r

rogue357.

03/30/2024, 8:00 AM
Another idea to expand on that... built in USB KVM (selectable node) ...1 keyboard and screen to rule them all
m

mvdk85

04/08/2024, 4:11 PM
This would be awesome, using something like the rtl8372 - but as said, the tolerances are tighter for 2.5g. The modules would need to be respun to include an rtl8125 (or rtl8126 - providing 5gb support for the future!), but that is doable if you choose to sacrifice either the usb2 or sata (those are multiplexed on the same pins). The other big downside is cost - rtl8370 is about $10-$15 in quantity, whereas the rtl8372 is about $50-$60, I think, and has a bigger power budget. If you went for full 10gbe support, due to backward compatibility, you'd need to include something like the Marvell AQC design, which is about $25-$40 in quantity, I think, and use a 10gbase-T transceiver - which all have a large power budget. The alternative would be using some kind of direct attach cable like thing embedded onto each daughter board with a corresponding place to attach on the carrier board. This is well beyond my knowledge of the theory, and I'm sure it would almost certainly require a fairly large amount of engineering effort, and would also cost considerably more (I'd guess about $100 more for the carrier board's BOM, and of course, the engineering cost needs to be defrayed as well, plus the per daughter board cost). So I'd say the best engineering compromise would probably be the rtl8126 on the daughter boards, since it's available in quantity for relatively little? But of course, 2.5gbe for the main board, because the rtl8372 is so much cheaper than an rtl9303 plus 5 10gbase-T PHYs.
u

.jensc

04/10/2024, 5:20 PM
I wonder when there is the CM-5, will the carrier board work with it and will the nvme be available for those modules? Maybe not yet to answer since the CM-5 are not even available!
s

scienceman.

04/10/2024, 8:13 PM
Radxa has announced their CM5 and stated that their should work with CM4 carrier cards. I've ordered a few and will try them out.
5 Views