Firmware updates aka Turing Pi 2 Community Edition...
# │forum
d
I started a thing:
Copy code
_____ _   _ ____  ___ _   _  ____   ____  ___   ___  
|_   _| | | |  _ \|_ _| \ | |/ ___| |  _ \|_ _| |_  \ 
  | | | | | | |_) || ||  \| | |  _  | |_) || |    ) | 
  | | | |_| |  _ < | || |\  | |_| | |  _ / | |   / /  
  |_|  \___/|_| \_|___|_| \_|\____| |_|   |___| |___| 
Community Updates by DhanOS
I took the firmware and started performing updates and I'm just releasing the first version with more to come. The first version,
v0.1.0
contains: - Added SSH root logins (an ability to log in via SSH without a need to do any changes using UART or adb shell) - Set static MAC address (
12:34:56:78:9A:BC
- later you'll be able to pick own MAC, especially useful with multiple boards) - Added
ntp
and
ntpd
(automatic time synchronization from the internet) - A few more smaller changes Full changelog and initial TODO: https://github.com/daniel-kukiela/turing-pi2-community-firmware/blob/master/changelog.md Project GitHub: https://github.com/daniel-kukiela/turing-pi2-community-firmware Feel free to propose what updates would you like to see.
b
I've not looked at the code (still waiting for TPI2 to arrive), but I assume somewhere in there is routine that powers up each node in succession (defaulting to 1,2,3,4). My I suggest the ability to set the order that each node gets powered. So for example, If I have a NFS server running on node 3 or 4, I'd like to power one of those nodes before nodes 1 | 2 | 3 so that the NFS server is up and running by the time node 1 & 2 join the crowd? Just an Idea
d
Sure, can be done and could even be configurable with the delays between when each node is powered. Adding to the list
b
👍 🙂 -- Oh and just for fun, I'm trying to compile your code under WSL:debian (so far so good)
d
WSL is not a bad idea, might be actually more convenient than using a VM for this
b
...this could take a while though... I'm using a Celeron J4125 😉
d
Oh it's going to take time anyway 🙂
w
Yes, the code powers up 1,2,3,4 at 1 second intervals. Looks like it could be changed very easily and the delay increased to give the NFS server time to get itself sorted out.
Doh, @DhanOS (Daniel Kukiela) beat me to it.
t
I suggest using AllWinner's registered OUI of DC:44:6D in your code. That way those of us who maintain DHCP servers should, by default, be able to find our TPiV2's BMC. It's unclear whether AllWinner doesn't program an address somewhere on the chip or u-boot doesn't pick it up.
Please consider releasing your changes to the "official" GitHub repo as one or more diff patch file(s). That way we can clone the official code and apply patches we want. Backing a patch out is a simple action.
d
Hmmm, patches are going to be hard to maintain since one might be dependent on another, or later updates might change things, but I'm already working on a config file that you can put on the SD card (or even upload to the BMC's internal memory) that will let you enable, disable or set things. I'm not going to make it "my firmware", but more like get the requests and suggestions and let people enable and configure features the way they want
I'll think on this solution, maybe even make it so you can enable things on the compilation time, but I don't know yet
t
Yeah. I started looking at it. Just ran diff with the proper options against both trees. Might be more work than it's worth.
d
And it'll get more complicated with more changes and a way for people to configure things
t
I guess it depends on the changes. The Rust code will supercede what's under "app".
d
The new firmware will make my efforts obsolete, but I like the challenge and I'll provide the updates until then. The new firmware is most likely going to be entirely new/different
s
Feature request: system event log to record board-level hardware errors (common feature in enterprise BMCs, code may be available in OpenBMC)
g
fix spelling mistakes
usb mode:                           Host                           Device                         Host conneCt node:
y
Why don’t you use a codespace for it?
d
Why, do you think, I would need it?
u
Codespace is nice, you can provide a prebuilt image for people who don't want to run all the compilation on their own computers, GitHub is very generous with their free compute hours per month
I tried compiling in a VirtualBox on my few years older laptop but when it started glowing red I used Codespace instead 😄
d
I'm providing firmware images, no one needs to compile nothing
I'm not denying Codespace can be useful, I'm just not sure how would it be useful here and now 🙂
u
It was useful for me, since 0.2.0 isn't up yet 😄
d
0.2.0 does not work as expected, yet 🙂
u
In any case, the problem it solves is less friction for anyone interested in doing their own hacking on the firmware, trying out things, compiling their own images
Also nice to run on GitHub's fast servers instead of a slow laptop
d
I might look into it, but I also have to set priorities, and I think they're currently somewhere else
u
It's not much work, I think you just have to start a codespace, install the things listed in the readme, publish that as a "prebuilt" codespace and then the community can improve on that
Ah, anyway, I thought the SD card workaround in the new commits would solve the "no sdcard" error message I'm getting when trying to flash the firmware via HTTP but I guess I need to get a TTL cable to fix that
d
That'll be fixed in v0.2 (I mean already is)
It's not in the original firmware
Also the comment is a bit misleading
The change mainly allows for bigger rootfs, but also requires an SD card to make subsequent updates
u
yeah, I understand
I just need to workaround the error to be able to flash the firmware
d
Copy code
cd /mnt
mkdir sdcard
reboot
u
no ssh/serial access yet
d
Because the original firmware does not provide it
My v0.1 does
You'll need an USB/UART cable now or
adb shell
with the MicroSD cable
u
Yep, gonna go shopping tomorrow
d
Only if you want the USB/UART, otherwise you can use MIcroUSB cable and adb
u
I'm not touching adb unless it's on a separate computer 😄
d
Well, ok 🙂
r
I would love to build locally on my Mac but am having trouble as the build does not support an ARM architecture - funny considering that the target is ARM... Tried running it in docker and emulated i386 but in the end I gave up... Long story short - great that you provide prebuilt images!
Once you tick off the static IP configured on an SD card I might switch as I don't connect the TP2 to my local network with DHCP.
Love the NTP support btw.
d
And I will keep providing prebuilt images - every time I'll have something I consider a stable version 🙂 If there is a demand I could also provide nightly-ish builds (once a commit is being made to the repository).
Thank you 🙂
r
Maybe it is too much of a niche feature, but if you add dnsmasq to the BMC image and ideally with configuration and the tftp root on the SD card, I would start experimenting with netbooting the nodes.
I can help with a few notes on how that would/could work, if you want, but as I can't compile (see above) can't do much else than that and testing.
d
I'm trying to sort out the available space on the eMMC, Turns out we might have less space than I though. The chip is 128MB in size (there's a mistake in the docs) and it is using doubled filesystem. This means we might only have 32 MB for the root partition. I'm talking with the dev of the original firmware to sort this out. I might need to move some binaries on the SD card and require the SD card to run. I don't know yet. But once I sort this and add a few other features I totally might do this too.
I'll definitely ask once I have the binaries added to the firmware image (one way or another)
b
HELL Yes! Looks like you and I are on the same wavelength here. I picked up a 128GB pcie card that I'm hoping to use on node 1 | 2 as a netboot/pixie server for the other nodes to boot from and avoid sdcard's in at least 3 of my 4 CM4's
h
Maybe use the eMMC to install a boot loader and install the system on the SD, this will reduce the wear (over time) of the eMMC.
y
Well, I thought to overcome the local limitations, it could make sense to use a cloudbased service that was created for this… integrated into the repo…. And can fire on release/push…. Whatever…
d
This is one of the potential ways to solve this. I'm still finding out the options and I haven't made any decisions yet. Maybe I'll provide 2 firmware versions - slim that fits into the internal memory and full that will require the SD card. I just don't know yet 🙂
Maybe, I'm not denying. I don't think I'm limited, though, with 18 cores and 128 GBs of RAM 🙂
But in this case that's independent from the BMC, right?
b
Don't know yet... still waiting for my board to ship. I'm just fantasizing of how I want to set it up
d
Ok, but so far I don't see how the BMC could be involved here 🙂 The NVMe PCIe card - you could connect it indeed to Node 1 or Node 2 via Mini PCIe or M.2, but then this node will become responsible for hosting the images, not the BMC like in the rubberduck's idea 🙂
b
Isn't the BMC involved in communicating with each of the nodes?
d
It is not, it does not provide any method for the communication. You might try to use UART which won't make much sense, though. The way to communicate between the nodes would be the ethernet network
rubberduck's idea is to use BMC and it's SD card to host the images. The BMC is connected to the same network switch as the nodes, thus could serve the images to the nodes this way
b
Oh, ok I misunderstood what was being discussed.
d
So, the way you think of it would not need any changes to the BMC 🙂
b
Well certainly not DNSMasq, but still hoping to be able to set the order & delay each node starts
d
And this is actually something I'm going to add to the firmware. 🙂 Currently, I need to solve the available space issue and learn about the possibilities, so the new firmware version will be released later than I wanted
b
🙂 I know, we talked about it a few days back. I have a PCIE 128GB card and a PCIE WIFI card (same card Jeff, used in his video PI networking faster than his MAC). Hypothetically the WIFI card will be used to allow external connections into the box and the 128GB card will contain 1 running debian OS and 3 network boot images for the other 3 nodes (all nodes are CM4's (8GB/Lite)
d
Yeah, I think I remember this talk now, thank you 🙂
b
👍
d
What was your reasoning for using the WiFi card again, please? The communication between nodes would be up to 1 Gb/s anyway
b
The WiFi will be used to ssh in the cluster from remote. i won't using the ethernet ports (unless the Wifi fails)
d
Ah, ok, thank you
b
2 weeks, woo woo!
d
What do you mean? Or maybe wrong channel?
b
Haha, yes, sorry!
d
I'm running a pool in the #754950670175436848 (so more people can see it): https://discord.com/channels/754950670175436841/754950670175436848/1082662801874616352
s
This is the chip being used, right? https://linux-sunxi.org/T113-s3
d
Yes, correct
s
There are tools and info listed on that page that may help.
d
Yeah, but for now I want to keep the process simple so the most people can use the firmware. The pool seems to be in line. This means we need no tools for flashing 🙂 As for the firmware itself - I just started this project, I don't know yet where this is going to get us, if anywhere 🙂 If people like this work and we figure out we have ideas and we want o re-work it partially or entirely - we will.
s
Okay; just trying to find some tools and links. This may also be relevant though I'm getting out of my depth, having never built for this platform: https://github.com/linux-sunxi/sunxi-tools
d
Sure, thank you, this is always appreciated 🙂
t
It might be best to leave the T113-S3s internal partitioning alone and install minimally viable "firmware" on it. Just enough functionality to know whether a microSD card is installed and prompt for one if none is found, then retry the boot process. Move the wear and tear of multiple erase cycles onto something that can be replaced. It would require use of the BMC TTL or adb connection unless some sort in BMC LED error code pattern could be implemented. Wouldn't be the most user friendly solution. More like expert friendly. 😉
d
This is kind of what I am thinking of - a minimal firmware, like v0.1.0 (that adds SSH root logins, NTP and static MAC) as a failover and the full OS on the SD card. The idea is if your SD card break and you replace it, or the system cannot be booted for whatever reason, the failover firmware boots and lets you debug the problem and/or flash a new SD card. The whole root files ystem might be moved onto the SD card.
t
Works for me.
m
notes: not sure what's exactly is the purpose of this private key, but it generally not a good idea to have it in clear in a repo https://github.com/daniel-kukiela/turing-pi2-community-firmware/blob/master/br2t113pro/board/100ask/rootfs_overlay/mnt/self.key
i think this certificate should be autogenerated on first run instead, otherwise this means tpicli is not tls secure
d
It's also not being used for anything 🙂 I'm going to add SSL and will definitely not use this certificate 🙂
t
A self-signed key is suboptimal. A cert does need to be present for HTTPS support, but I don't like having to manually trust them. It's typical for low-cost routers to use this technique. Ideally Turing Machines would invest in a root signing cert, but I don't expect it.
d
I there any way to create a certificate that matches any domain name (also special-use domains like .local) and any IP that I am missing?
Like how you make a certificate for 192.168.0.123 and turingpi.local that's signed
t
Yeah. I know certificate trust chains are complex and inflexible. I wouldn't expose a Turing Pi V2 to the open Internet. I guess the BMC should default to a locally self-signed (not canned) one and have an interface to install one for those that need it.
d
The thing is more that .local is not TLD, you cannot have a certificate for such domain. This will yield some risk (because many networks could have turing.local and a single certificate will match them all). Same way you cannot have a certificate for an IP (technically you could, but not from the private IP pool, unless I'm missing something). So it's not possible to have such certificate, unless I'm missing something. So it's not a cheap router thing to have self-signed certificates
Without a public domain (that's also a TLD) your only choice is to use any certificate since it'll never match the domain (or IP) part in the browser, thus self-signed work the same way as any other certificates
The solution to this might be to have a public domain, like clients.turingpi.com and then a DNS server that contains private IPs of the boards (which would not be the best because of the security) and then the certificate for each clients subdomain, for example sometpiname.clients.turingpi.com and you'll have to use this exact domain and a private/public key generated for this exact subdomain on your TPi to have a signed certificate for your board
I haven;t seen any device in a local network with a valid certificate - no matter if it's a cheap router or expensive other device
Again, please let me know if I'm missing something. I might not know about something
So, anyway, a self-signed certificate should be generated on the first boot indeed, or there also could be added an ability to use own given private/puiblic key pair
With own private/public key part you can have a domain like somemydomain.com that's not really public but your network, and then you can have turing.somemydomain.com that has your private IP on your private DNS. Then you can use any cert provider, even free Let's Encrypt to add own, signed certificate (own, since it;s in your domain and under host name of your choice). Or get a *.somemydomain.com (so-called wildcard/star certificate)
r
What do you think about trying to have the minimal firmware check for the SD card and if finds one, mount it as an OverlayFS? That way you could customise and configure the firmware installation persistently without touching the emmc storage on the TPI2. Not sure how much sense this makes compared to the option of using a dual stage boot where the firmware just boots up whatever OS it finds on the SD card... Still learning about linux boot loading processes and embedded devices in particular, so... please bear with me. 🙂
d
Oh I'm still learning a lot too 🙂 I'm not sure about the OverlayFS, I might check it. Then booting a full OS from the SD card also sounds like a good idea. I'm still considering options.
r
In both cases you would have to switch root to the SD card, methinks.
d
So, the idea I have is to have a root file system on the SD card and the minimal one (with SSH root logins, etc) in the internal storage (it'll boot in case the SD-card one would not boot giving you an option to debug and flash new firmware if necessary)
r
Something like that. The dual stage approach is how I understand any PC's boot loader to work (might be wrong or overly simplified). The difference here is that we may want a real OS as a fall back and not just a bootstrap to get the real deal going.
In my previous setup (prior to getting the TP2 board), I used one node with emmc and WIFI as the master that would run a dnsmasq (dns/tftp) server for the other CM4 lite nodes without WIFI to netboot from. The WIFI on the master node was used as a gateway for the local network used by all the nodes of the cluster. The idea with adding dnsmasq to the BMC/SD card is simply building on this and would be an improvement as we would not waste node resources on hosting the dnsmasq server and could make all nodes use the same base image. More importantly, we can isolate the control plane of the nodes from the dataplane using VLAN tags. One ethernet port, the BMC and the nodes would be tagged as the control plane, and the other port and the nodes would be tagged as the data plane. This (so far imaginary) setup would provide a good isolation and separation of concerns, where BMC is responsible for management and the nodes, well they are responsible for whatever workload you run.
s
suggestion: modify uboot.conf during build to enable CONFIG_AUTOBOOT_STOP_STR, see https://github.com/lentinj/u-boot/blob/master/doc/README.autoboot
with that enabled, it allows you to stop the boot process and drop into a u-boot shell. Without it, it will autoboot every time no matter what keys are pressed. And if you have a bad board or need to troubleshoot (ie your BMC kernel panics like mine), not having that option enabled limits troubleshooting ability
d
m
in my homelab I have created a CA on my opnsense and added the authority for most devices so i can created "valid" certificates
d
Correct, but this still makes it self-signed and requires manual work to add the authority to the every device in the network that is going to validate the certificate, right? I agree, though, this can be useful since you only need to add the authority.
Note for myself - power on via command for separate nodes: https://discord.com/channels/754950670175436841/754950670175436848/1083045348613820467
m
yeah i need to inject it into CAs for every devices, my CA cert has a 10 years duration so i hope i won't have to do it twice... still, on containers (docker etc) sometimes it require more fidgetting
d
Just want to say that the new firmware versions are going to be slightly delayed. I'm figuring out hot to compile the firmware for the SD card, this is all quite new for me. There are some custom scripts and modifications for the NAND Flash version that I need to find out if and how to replicate into the SD card flash. Once this is done I'll have to find out how to make a firmware that the failover OS and the first stage bootloader for the normally running one are on the NAND Flash and the root filesystem resides in the SD card I also not necessarily want to do all of this on my TPi2 boards, so I ordered myself san SBC with the same CPU as the BMC on TPi2: https://www.aliexpress.us/item/1005004448277970.html To have the same setup, I also ordered the same NAND Flash chips: https://www.aliexpress.us/item/1005005284945182.html And to put this all together: https://www.aliexpress.us/item/1005005265080663.html This will give me a standalone BMC with an ability to replace NAND Flash if I wear it out. So I'm waiting for this to continue the development
g
👍
t
👍🏼
b
👍
To: DhanOS memory 🙂 consider having the firmware also shut down the BMC and turn off the power LED when the power button is pressed, so that people know they can remove the power completely. Or do people expect a BMC to stay powered on when the "system" power is off? 🤔
k
Yes, the BMC stays on when you "shutdown" all systems. How else would you turn it back on remotely? The whole point of BMCs is that you don't have to go into a server room to turn on servers after a power cut or accidental OS shutdown.
d
The original question, asked over #754950670175436848 was if we shutdown the BMC system (since it's running Linux) before depowering the board. And this is the point - to be able to shut it down peacefully before disconnecting the power. Otherwise it should stay running, of course
I then asked to post the thoughts here so they do not get missed
t
In my experience, every data center rack should have remotely-manageable power control. This way, if a piece of equipment hangs, it can be power cycled first to attempt remote recovery before going on-site. That might seem like overkill for an edge device, but the sheer potential volume of edge devices can make it cost effective.
d
I mean I agree, but if you answering to what I wrote - the reason for that would be only when you want to remove the power from the board, so also the BMC shuts down cleanly
t
I was (maybe tangentially) agreeing with you. If I shutdown the BMC, I expect to power cycle the board. Understand the potentially unintended consequences of "your" feature request. 😉
d
Well, you put the power button for 1 second to turn on the modules, and for 3 seconds to turn them off. So, maybe if you press it for 6 or 10 seconds you can power off the BMC too. Hard to miss the moment when the nodes are shutting down. Also that'd be (as everything) configurable, so you'll be able to to disable this feature
b
yeah the original question was just about whether the BMC needs a way of shutting it down at all. If it does then holding down the power button for an extra long time as a way of doing that is a good idea. Otherwise you need to separately SSH into it or use its API before removing the power from the board.
d
TODO: let the
tpi
command to turn modules on and off separately
s
Honestly Redfish would be a better starting point than rolling your own for things like selecting submodules (e.g., blades) to control, handling provision of boot images, power cycling, etc. Perhaps you could start with the OCP mockup? https://redfish.dmtf.org/redfish/v1 That's why this was invented.
d
Honestly, I don't know. This firmware updates, so far, were just simple updates that did not take long to make. Starting over with firmware from scratch is a bigger time investment. We know that the Team is hiring firmware developers, so some new firmware is going to be written. This does not mean we cannot have a community firmware, but at this stage I'm not thinking into making a completely new thing, but rather porovide missing pieces that people often ask about until a new firmware is made by the Team. We'll see, though.
g
@DhanOS (Daniel Kukiela) I can't seem to get your v0.1.0 firmware to flash via the OTA method. I follow the instructions to upload the image, a response page loads, and then the bmc reboots. After it reboots I check to see what the MAC address is and it's still random and I can't ssh in with root. Got any advice on what's going wrong?
d
This means the flashing does not succeed. Do you potentially have UART? UART will let us see the logs and the messages about flashing and let us know what's going on.
Also, when you upload the firmware, some JSON object is being printed in the browser. What does it say?
Can you tell me which file (the extension of the file) you are using?
g
I don't have UART yet, I was hoping to avoid having to order the converter but may just have to now. I am using the file turing_pi2_ce-0.1.0.swu and this is what is being printed to the browser (ip address redacted): Content-Type: text/plain VARS: HTTP_CONNECTION=keep-alive HTTP_CONTENT_LENGTH=25036499 HTTP_CACHE_CONTROL=max-age=0 FILE_FILENAME_file=/tmp/tmp-0.tmp UPLOAD_DIR=/tmp HTTP_ACCEPT_ENCODING=gzip, deflate HTTP_ORIGIN=http://xxx.xxx.x.xxx FILE_SIZE_file=25036288 HTTP_CONTENT_TYPE=multipart/form-data; boundary=----WebKitFormBoundaryBayjokm3KqqgmF6c HTTP_REFERER=http://xxx.xxx.x.xxx/index.asp HTTP_ACCEPT=text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7 type=firmware HTTP_ACCEPT_LANGUAGE=en-US,en;q=0.9 HTTP_UPGRADE_INSECURE_REQUESTS=1 FILE_CLIENT_FILENAME_file=turing_pi2_ce-0.1.0.swu HTTP_HOST=xxx.xxx.x.xxx HTTP_USER_AGENT=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36 FILE_CONTENT_TYPE_file=application/octet-stream opt=set {"response":[{"result":"ok"}]}
d
Yeah, I understand you should not these cable
the file is correct and what's printed is also fine
Hmmm
I'm trying to think how to see what's going on
May I encourage you to try to flash the
turing-pi_1.0.1.swu
from here: https://github.com/daniel-kukiela/turing-pi-2-community-edition-firmware/releases/tag/v0.0.0 ? This is the latest official firmware. Maybe your board has some other version and this causes issues. And after this try to install mine again?
g
I've also tried connecting with adb but my computer never detects the device
d
Which port are you using for ADB?
Did you install ADB?
g
I followed the instructions on Turing's page. I didn't see anything about as port for the adb. https://help.turingpi.com/hc/en-us/articles/8686945524893-Baseboard-Management-Controller-BMC-
d
From these instructions:
g
Yup
d
Oh, you meant the port:
This one. Just making sure
g
Yeah
You asked what port I was using
d
Yes, and if you installed ADP
So you installed ADB and are using this port, right?
g
Yes
d
Hmmm, this is interesting because people usually don't have issues with ADB
Is this a Windows PC, Linux PC or Mac?
And additional question - do you possibly have an Arduino (or eventually a Raspberry Pi or similar device)?
g
Windows and I've got a few pi's lying around
d
You should not have any issues on Windows. Did you, however try to restart it? Does
adb
command work?
As for RPis - may I propose using one for UART to see the boot and update logs? You'll need 3 wires to connect it to the TPi2 board
g
when I have my phone connected to my pc the adb devices command finds my phone.
d
Is your TPi2 powered for this? I think it needs to have power
g
it is
d
Ok, i connected my board to my PC (also a Windows PC), downloaded adb tools and ran `adb devices`:
Copy code
>adb devices
List of devices attached
0402101560      device
I can clearly see my board instantly
I'm just checking, I did not use ADB with this board yet
When you connect your board, do you hear the sound that Windows emits when you connect a new device?
I'm trying to make sure the cable is ok and the board is actually connected
g
I don't hear any sounds and I've tried several cables and different usb's on my pc too.
d
But you do hear this sound when you connect, for example, a pendrive, or the phone you mentioned. Right?
g
when I plug in a flash drive yes, phone no.
d
Interesting, because it should emit the sound when you connect a phone (it, for example, does this for me)
I'm trying to think how to help you with adb
Like what the reason might be you don't see it
g
I'll try to track down my canakit box that has wires tomorrow so I can connect a Pi to the UART.
d
It mentions you have to do some steps on both RPis - you need to do them on the RPI only in our case, the BMC is all set for UART communication
g
gotcha, I'll take a look tomorrow. thanks for the help so far, hopefully we can get this figured out.
d
I'm sure we can
As for the
ADB
, head to the Device Manager and see if you have any unknown device listed (like with missing drivers)
You might also try to restart TPi with a cable connected, wait for 30 seconds and try again
g
nothing even after a restart of TPi
d
This is odd, to be honest
What about this?
g
tried that, still getting the random mac address
d
Yeah, ok, at this point best we can do is to see the serial console output
g
sounds good. talk to you tomorrow once I find the wires.
d
👍
g
I finally got the BMC updated after reformatting the sd card but I'm still not able to connect via the adb...
d
So the issue was an SD card that you insterted? Interesting. Did you flash the board with the official firmware linked from the TPi help pages?
g
Yeah, when I tried to flash it last week I got a message that said there was no sd card installed for the bmc. I formatted one I had lying around but it would never mount so I bought a new one and was able to get that to mount. Last night I reformatted it to ext3 but that wouldn't mount either so I reformatted it to exfat and that seemed to do the trick. I flashed your firmware.
d
Oh, so this was actually an important bit of information. If I would know it, I'll know what's wrong. The question, though, is how you got the firmware that requires an SD card on your board. Not by the factory probably.
g
I was trying to flash v1.0.0 from their github page not realizing that it wasn't a new version.
d
So they uploaded this modified version. Good to know.
I still don't know what's wrong with your
adb
, though
g
Yeah, me neither. I was wondering if it might be because I have autorun disabled on my pc, but I'm able to see my phone when I plug it in...
d
g
How difficult would it be to have a BMC terminal embedded in the web interface?
d
Should be relatively easy to implement.
w
Feature request: simple way to send "magic sysrq keys" to nodes. It seems like C-c or maybe C-@ is the way with microcom but would be way easier to have a tiny tool for it. Maybe I can just write it myself and compile
d
Yes, my idea is to make every node to have a special login and password and use them for things like power down, obtaining an IP (to show node IPS, grab fam PWM speed, system load, memory/cpu stats, etc). So the plan is a bit more than just power off 🙂
s
Honest if we had just started from OCP with Redfish rather than using a mobile phone chip as a BMC this would all be easy
d
I've heard this already. A new firmware is being developed by the Team. This is a temporary solution, a great base for future updates but to not dig a rabbit hole immediately and let people wait weeks for something usable 🙂
w
What compiler does one use to compile programs for the BMC? I don't see some other toolchain being installed in the instructions you give...
hmm
Linux turing 5.4.61 #31 SMP PREEMPT Thu Oct 20 00:11:14 CST 2022 armv7l GNU/Linux
I suppose just armv7 then
f
For the BMC, what size Micro SD card would do?
t
that's a good question; I have a few sitting around... was thinking about throwing a 128GB on there just so I didn't need to uninstall my board to swap cards in the future
I'm curious what it can support, some controllers for microSD have limits
f
indeed, there seems to be nothing detailing it whatsoever in the docs website
t
the BMC (Allwinner T113-S3) supports the SD 3.0 specification; I'm trying to determine what all that means
depending on whether that means SD 3.0 or SD 3.01 it probably means a max of 32GB to 2TB
the max for the BMC is somewhere between those two capacities...
f
yeah
t
unfortunately, this doesn't narrow down the possibilities
it definitely supports Class 10, but may or may not support UHS-I
definitely doesn't support UHS-II
f
Hmm
any idea @DhanOS (Daniel Kukiela) about the SD card spec for the BMC, if it's max spec 3.0 or 3.01?
From what i can gather, even 3.00 supports 2TB and Class 10 - https://www.taterli.com/wp-content/uploads/2017/05/Physical-Layer-Simplified-SpecificationV6.0.pdf Page 5 The Part 1 Physical Layer Specification Version 3.00 or later and Part 2 File System Specification Version 3.00 or later allow Standard Capacity SD Memory Cards to have capacity up to and including 2 GB, High Capacity SD Memory Cards to have capacity up to and including 32 GB and Extended Capacity SD Memory Card to have capacity up to 2 TB
Also Page 176 Optional conditions to indicate Version 3.00 Card A card supports any of following functions shall satisfy essential conditions of Version 3.00 Card (1) Speed Class supported under the conditions defined in Version 3.00 (2) UHS-I supported card (3) CMD23 supported card
UHS-II is 4.00 onwards
i'll probably go by that for now, cheers
b
I have a 985GB card in my bmc, it sees the card fine and reports the correct capacity but I haven’t tried writing or reading on it yet.
t
thx
f
I saw some of those weird GB sized ones on Amazon like 985, not singling you out specifically, but does anyone know why they exist and not just the usual 1TB's?
never seen them before up until now
t
Likely a no-name manufacturer. NAND manufacturers "bin" their die just like CPU manufacturers do. Not all NAND die pass the manufacturer's criteria for integration into their name brand products. The rejects, which are spooled for SMD equipment, get sold in bulk. The weird capacity basically says that some NAND pages were marked bad.
d
Well, looks like this question is already answered. But what would you use the card for there?
f
Doesn't the BMC require one? I tried to update it and it whined about requiring one
Content-Type: text/plain {"response":[{"result":"err:no sdcard"}]}
d
It does not require. I mean if you updated firmware to the official 1.0.1 one, then it indeed require a card for the firmware upgrade process, but only for that (the firmware is not being flashed to the SD card). The CE version of the firmware does not have this requirement
f
I even get the same error with turing_pi2_ce-0.1.0.swu
d
Oh, and it'll require about any card, the firmware is a few 10s of MBs only
Because you flashed 1.0.1 on your board. Once you flash
turing_pi2_ce-0.1.0.swu
onto the board, it won't require an SD card for flashing anymore
f
Ahh ok, and only because it's on 1.0.1 even to flash to the CE it'll require it, and just for that only?
d
Yes, 1.0.1 require an SD card to flash any firmware
f
okee 🙂
Can you shunt a (temporary) USB device to it instead, when the port is in host mode?
d
I'm not sure what do you mean
t
Yeah. Not really liking the fact that v1.0.1 doesn't have a web configuration page for BMC networking or just ingest the tpi.ini file from the microSD.
w
I mean really the BMC should behave like any other device and just have a randomly generated MAC that is stable per physical device
like every other device in the world
you don't have to configure a raspberry pi or even ESP32's MAC address
If the chip doesn't have some kind of serial number, it should try to find some source of randomness that is relatively stable per physical device and then store that MAC somewhere
f
It doesn’t have any authentication for the web interface too, which is oddly terrifying
For the firmware update, to use a usb drive for the storage
d
It is going to, though. This firmware is based on the original firmware which is in the alpha state
The BMC only uses the micro USB port and, as much as I am aware, this port cannot be set into the host mode. Update should not require any additional storage, at least for now. Additional BMCs storage is the SD card slot
f
But I’m on 1.0.1 but both it and you said it needs a micro sd card to then firmware above that later or even to the CE version. I don’t have a spare sd card so was wondering what other options were there to move to CE
d
None, SD card is a requirement. This is why I'm mentioning this and encouraging towards CE version of the firmware
w
They can't reflash using the otg port?
s
... it's likely asking a lot, but is there any likely possibility of a kernel/userland update to allow TRIM/DISCARD on the BMC's SD card?
(Is the BMC SoC one of those awkward ones which isn't supported by stock Linux, and needs a heavily-hacked source tree which hasn't been integrated upstream?)
d
The board can be re-flashed using this port and Phoenix Suite, but the process is not the easiest. I meant SD card is required now for OTA with this firmware version
I'm collecting all the ideas and I'll be trying to add them all 🙂
s
As-per https://discord.com/channels/754950670175436841/754950670175436848/1096066131397185566, could the documented support for ext2/3(/4?) filesystems on SD cards please be added/restored?
d
@DhanOS (Daniel Kukiela) request for CE firmware: show the version of the firmware somewhere in the UI? Maybe on the update page "Current version is XXX"?
I am using it, thank you for making it available. I'm excited to see what next you come up with
d
Yeah, this is a good idea and I'm going to do that
There are many things to come, I just need a bit of time to get back to it
d
I get it. And no rush, I just figured this is a relatively small change (ha!) that would make it better
u
fyi for anyone trying to build: sources.redhat.com is not reachable (at least for me) so the LVM2 sources fail to fetch and the build hangs. my workaround is to replace the LVM2 URL with the upstream FTP server URL instead:
Copy code
turing-pi-2-community-edition-firmware on  master [!?] 
❯ head ./buildroot/package/lvm2/lvm2.mk
################################################################################
#
# lvm2
#
################################################################################

LVM2_VERSION = 2.03.14
LVM2_SOURCE = LVM2.$(LVM2_VERSION).tgz
#LVM2_SITE = http://sources.redhat.com/pub/lvm2
LVM2_SITE = https://sourceware.org/pub/lvm2
t
Thanks so much for your work on this. In addition to the excellent suggestions of others, I have three ideas: 1. Ability to set certain hosts to startup automatically on BMC boot. I know others have suggested a custom boot order already, but this would allow unattended restores after a power outage. I have a kludgy workaround via an /etc/init.d/* script but something more accessible is probably more broadly useful. 2. A way to set a custom hostname for the BMC and provide ssh keys. This could be via the web interface or SD card file, and I'm ambivalent but think it would make sense to mirror whatever option you end up with for MAC address setting. 3. At the risk of starting a flame-war, including the nano text editor binary as a vi alternative.
When time permits I want to kick the tires on how exactly the GoAhead web server works. While day-to-day I'm likely to run it through an nginx reverse-proxy on a tpi2-hosted Raspberry Pi (and manage session encryption and authentication from there), I like the idea of having some remote troubleshooting access in case the Pi becomes inaccessible. It might well be better to use certificate-only SSH, instead of the complexity and potential attack surface if I go down the hardening GoAhead and/or using firewall rules route, but I still need to get my head around it. (as a note to future me, and others: basic config is at /mnt/route.txt, usernames and passwords at /mnt/auth.txt, and hosted files at /mnt/www) I'd also like to work out if it's possible to use cron on the BMC. The binary is installed already, but the files it needs are missing and get wiped on reboot if I manually create them. As a workaround I've had some success inserting extra scripts into /etc/init.d/ for some tasks (eg startup host 4) but they occur on start/shutdown only, rather than during routine operation.
d
Thank you for the suggestions. I'm up for all of them. Some of them are already planned (like a config file that will let you set a hostname, mac, ip, node boot order/delay/automatic boot, etc)
I think I need to go through this thread and create issues on GH for each of the ideas - this way it'll be easier to track them
t
Thanks so much!
r
@DhanOS (Daniel Kukiela) just got my board this week, I've plugged in a Jetson Nano but it's a real pain to flash. What would be good is an option to set a node into the boards recovery mode as outlined in this table from the [Nano datasheet](https://developer.download.nvidia.com/assets/embedded/secure/jetson/Nano/docs/JetsonNano_DataSheet_DS09366001v1.1.pdf). unsure if you'd have the ability in software to set a specific pin Low/High?
d
I'm assuming you've seen the official docs. Setting given node into the device mode sets it also into a recovery mode. Flashing in node 1 seems to not always work (some boards have issues with that) so you might try node 2-4
w
I love
micro
as a text editor, if we wanna get into holy wars 😄
(in all seriousness if you like nano for "works like a normal text editor", micro is a breath of fresh air in that it is like that even more so. I don't know of anybody who uses nano because they actually like it, just because it stays out of their way)
d
We're on a strict space constrain before we move to using the SD card, so probably only one of them for now 😉
w
Copy code
wijk% ls -lh $(which micro)
-rwx------+ 1 ubuntu ubuntu 11M Feb 20 02:11 /mnt/seaweed/workspace/bin/micro
wijk% ls -lh $(which nano)
-rwxr-xr-x 1 root root 269K Feb 19  2022 /usr/bin/nano
ok fine
d
11M 😮
No, definitely won't fit
I mean even if I wanted, 11MB is way more than we have left space 🙂
w
I suppose I can easily just
go install
it with my
GOPATH
on SD
t
My day-to-day text editor is VS Code, but I still have muscle memory for nano kb shortcuts
d
I'm using VS Code as well, Sublime Text before, and Nano for the console stuff
With the root mounted on the SD card, we could probably have 2 more things - OS updates via a command and packages installed via a command
Like no more flashing of anything to update the BMC or add some feature/tool
w
\o/
Web based serial console. webserial.io (though that seems to use browser serial ports) or https://github.com/ayushsharma82/WebSerial (but that's ESP32/ESP8266)
d
I started working on an alternative web interface for node management using more modern tooling this weekend and want to discuss potentially rolling it into the CE BMC firmware. Is that something you'd be interested in @DhanOS (Daniel Kukiela) ? I'm currently developing it with a locally running API proxy to get around CORS issues, and can see it running that way to provide an interface to multiple Turing pi boards. The BMC API also leaves much to be desired and am debating adding more work to the proxy server that provides a nicer restful or graphql interface. Don't have much done yet, just re-imagining the interface to allow the user to get a visual state overview and quickly adjust power and usb device/host more quickly.
In many cases users are going to automate a lot of this interaction in different parts of infrastructure, but I like making this pretty for those who just have one node and like pressing buttons.
d
Sure, that'd be great!
I'll re-read it again when I wake up, I'm too tired now to get fully what do you mean by the proxies
No problemo
d
The web server on the BMC is... well, what I can only say is that it is. I'm not yet sure what capabilities do we have
We also want some sort of login before anything can be accessed
d
Yeah, I looked though the webserver c code, very basic in there.
d
Also take a look at a way to show console output or maybe even accept console input? I want to add at least printing the UART things, but being able to actually also input command via web would be epic
I also want to make it possible to edit different BMC settings like MAC, IP, hostname, node power setting, etc, but we'll get to this point I guess
If we need something better for the web server, we'll look into it, but also we only need to be able to serve static files and otherwise query some API endpoints that are working already, so we might have all we need
d
Yup, at the moment, the basics are already exposed, in a non-restful way. Just working around it and thinking of improvements.
d
I'm not sure if we want to go restful way. JSON API seems enough, or maybe we can implement JSON-RPC (which should be a better fit here than a RESTFUL API). I haven't thought much about that either.
w
oh I wondered if you just built a new UI to the existing API
d
The API might and most likely will be modified to better fit theneeds
d
Just to clarify...to update to the CE version, it needs to be done via SD card. SWU version of the ce firmware on an sd card, and it will install from the BMC SD card on power up? Seems like I am missing a fundamental step or two, and the last thing I want to do is take risks with the BMC firmware.
d
You update it the official way, via the web interface which contains the firmware update feature (OTA). If you happened to update the firmware to the version 1.0.1 before, you'll need an SD card inserted, since 1.0.1 of the official firmware has this requirement
d
Ahhh..so nothing on the SD card, it just needs to be in the slot? I do have 1.0.1 installed.
d
Yes. 1.0.1 uses the SD card as a temp folder. A bit unnecessarily IMO
d
Perfect, thank you. I'll give it a try.
d
👍
d
A quick snapshot of the mockup, shows usb selector state, powered state via numbered avatar circle, and then a power toggle. Not much there but a start.
Was thinking of adding a contextual hotkeys for different needs, like power and usb host mode switching with confirmation dialog. Could also be used to chain sequences, like confirm power off and then usb mode switching.
t
The tpi command's functionality lags a bit from the web interface. I'd like to see the "poweron" and "poweroff" options/functions be more granular. Currently, "poweron" turns on SATA power, then each node in order. "poweroff" turns off each node in reverse order, then SATA power. I'd like to be able to power on/off nodes individually or in groups. The SATA power on/off function should be called when any node will be powered on and when all nodes are powered off. The web service performs the former, but does not power off SATA when all nodes are powered down.
The power button logic is pretty dumb. It doesn't examine the current node and SATA power state.
d
A granular way of controlling node power is planned. This is going to be configurable to you can set an order an a delay, auto power on of certain nodes, an ability to turn nodes on/off individually via the cli tool, etc. So indeed this is going to be added
t
Yeah, I can envision lightweight control logic running on the BMC and heavier weight logic using the RPC from a separate orchestration console.
d
As for the SATA power, this is not controllable. The way it works is the BMC is powered via the +5VSB line which is always powered even is the PSU is off. it's the same in the PC and lets it react to the power button, or wake on lan or use USB ports to charge when the PC is off, etc. Whenever the first node is powered, the PSU turns on and then also powers the SATA connectors. It should power off the connectors with the last node being turned off, but this indeed does not work iirc, but it;s a software thing.
The power button currently just turns off all nodes. Do you have any idea here? There's (currently) now ay to turn the node OS power off via this button, with with a help of some scripts, that'd be possible
I thin I like it, but could there be a way to make 2 interfaces? I also kind of like more like a proxmox-style of the GUI and I think I might be not the only one. I'm not asking to make 2 interfaces, but maybe have this in mind that they might be switchable. Also, teh host mode - it'd be nice to make it so you can denote if it's USB power on the back or BMC that the USB is connected to (I might try to make it so we can flash at least CM4s from the BMC to start)
t
I beg to differ on "SATA" power control. If you look at the tpi.c code under the poweron and poweroff functions, you'll find two GPIO operations. poweron does: common_device_gpioWirte(POWER_EN, GPIO_H); (Line 265) This enables power to the nodes, which remain powered off, and SATA. poweron also does: common_device_gpioWirte(SYS_LED, GPIO_L); (Line 267) This turns on the LED attached to S-LED. poweroff does similar common_device_gpioWirte operations to turn off node and SATA power, then the S-LED. These are on Lines 292 and 294.
d
What this does is it turn on the PSU by interacting with pin 16 (PS_ON) of the ATX power connector:
Then, I might be missing what do you mean
With this enabled, not only SATA ports are being supplied with power, but whole PSU starts running and delivers different voltages, also to the Turing Pi 2 board (including node power)
So this is not a SATA power, but power in general
The BMC is being powered by always supplied +5VSB (stand by)
t
I have a modular SFX-L PSU. My fans are attached to a SATA power cable. The POWER_EN GPIO operations turns the circuit on/off. I'm not using the PSU's other outputs at this time. I just want to be able to control nodes and the PSU power more granularly. When running at max, the fans can get annoying. Yes, I can dial down the case fan speed.
d
OK, but I'm still not sure what are you trying to achieve. To be able to power the nodes ON, you need to deliver power to the board by triggering POWER_EN GPIO, which, in your case, will also turn on the fans. You cannot turn the nodes on without turning the fans on in your case
The granularity might mean to be able to trigger POWER_EN separately from the node power to, for example, spin the disks or fans, but if you want to turn on even a single node, POWER_EN has to be triggered since this turns on the PSu and delivers the power to the board
Does this make sense?
Also I apologize, but I have a hard time to fully understand what are you trying to achieve
b
yes, thanks! nice work!!
e
do we think there will ever be more functionality to control the ethernet switch? I.E. add vlans, or bond the two 1gb ports out of the board etc?
d
This is one of the things I want to do and will definitely look into it.
Feature note: an ability to format and mount an SD card through the web interface? not sure how this fits the plan of moving the filesystem root onto the SD card (but also don't know when we move it there)
u
maybe "re-initialize" instead? User must supply a fresh rootFS image to write?
d
I mean, these are 2 things. One is to be able to use an SD card right now and another to move the root filesystem on the card. I haven't yet decided how the flashing process is going to look, but probably something like flash this via OTA and then flash another file via OTA to set the SD card, or the BMC is going to prepare the SD card automatically and copy the files over. If possible, I'd like to save users from flashing the SD card by hand (using their PC/Mac), but I'm not yet sure if this can be done this way. We'll see.
u
what about leaving a minimal BMC image on the included flash, like we have now, but then doing something like an overlay from the SD card? That way a user can still use the board right out of the box, but if they flash the "enhanced" firmware it gives them the option to store extras on the SD card image.
d
This is kind of the plan. The root partition is actually placed twice in the BMC's storage - one if a failover in case flashing did not succedd. I want to replace a root filesystem device only on one of them and move the entire filesystem on the SD card. It's like 20-ish MB in size
The goal then is that if the SD card breaks, the failover filesystem is being used instead and will provide the web interface to flash a new SD card
u
right, ok I see how that would work... that answers the question then: To format the SD card from the web UI, you can clone one of the rootfs partitions on to the SD card and expand it
d
This is one possibility, but the goal is to be able to put more binaries there and they would not fit into the flash partition (this is part of why the filesystem will be moved on to an SD card)
u
hm right...
d
Feature idea: an alternative to microcom? minicom? Is freezing caused by microcom?
u
screen?
p
screen
should be available in buildroot, would be good to give it a try
d
Copy code
login as: root
root@turing's password:
# screen
-sh: screen: not found
p
Have to run
make menuconfig
and it's under the Terminal/Emulators section
d
Oh, you said buldroot
Yeah
But it looks it's 1MB so probably won't get into the firmware for now, but we'll see
p
Everything has been amazing between the community and the devs. Props on your efforts @DhanOS (Daniel Kukiela) !
w
a bit more fully featured
d
I don't think we need much, just to run stable
w
Copy code
wijk% ls -l picocom
-rwxr-xr-x 1 andrew admin 57716 Apr 18 01:33 picocom
wijk% strip picocom
wijk% ls -l picocom
-rwxr-xr-x 1 andrew admin 44036 Apr 18 01:33 picocom
here we go
wijk% make CC=!arm:0 CFLAGS='-static -Os'
d
I'm kind of curious if this is a microcom issue, or the fact that the firmware is also reading UART
w
that's with
-static
because I can't be bothered getting the right glibc where I'm building it
d
40-60kB is great, we definitely can test it out
w
huh, it's not working though, still a glibc issue
d
But then, again, the firmware also reads the UART and a question is if they interfere with microcom/picocom/whatever else
w
maybe I do need to get the right glibc version
d
I did not look into it, but if we can integrate it into the webservice, then no matter the firmware, the web panel or a cli tool could use the same established connection and interact with it
It looks like the picocom got the last update 5 years ago
w
I've been having fun trying to get the right toolchain setup. I'll maybe see about publishing a dockcross image tailored for the BMC ()
=> [4/7] RUN mkdir /dockcross/crosstool && cd /dockcross/crosstool 10833.0s
ok so I have a working toolchain, and successfully built picocom
Copy code
$ docker run werdnum/dockcross-tpi2-bmc >./dockcross-tpi2-bmc
$ chmod +x ./dockcross-tpi2-bmc
$ ./dockcross-tpi2-bmc make
(it's an ARM64 docker image)
took forever to build too
r
I tried to make the build work on my M1 Mac some time ago (there is an old thread lying around somewhere in this discord server) but never got it working as the default toolchain depends on i386 tools. Tried building in docker but that not get that to work either. So
dockcross
gives me some home that I might be able to join the fun without getting an intel PC to build arm images on... Seems to be a problem though:
Copy code
➜  dockcross git:(master) docker run werdnum/dockcross-tpi2-bmc >./dockcross-tpi2-bmc
➜  dockcross git:(master) chmod +x ./dockcross-tpi2-bmc
➜  dockcross git:(master) ./dockcross-tpi2-bmc make
Unable to find image 'containers.andrewgarrett.dev/dockcross/linux-armv7-lts:20230418-484f06a' locally
docker: Error response from daemon: Get "https://containers.andrewgarrett.dev/v2/": Service Unavailable.
See 'docker run --help'.
Any idea what is missing?
d
Is it a private container registry?
w
Oh lol that's my mistake then, I'll have to fix the image
You can manually fix the image name in the script to point to the same correct image
The i386 image is a stupid bug in dockcross. One of the scripts just has if uname -a equals "x86_64" then amd64 else i386. I just had to fix that
r
Stupid me - did not realise it was a script, assumed it was a binary...
w
I forget the directory but it's like install gosu wrapper.sh or something
I also hacked the armv7-lts configuration to use glibc 2,25 which is why I rebuilt the image in the first place
r
So... building I get an error for the
cjson-rebuild
target (following the instructions in the
readme.md
of the official build...)
Copy code
$ ./dockcross-tpi2-bmc make -C buildroot  BR2_EXTERNAL="../br2t113pro"  100ask_t113-pro_spinand_core_defconfig

make: Entering directory '/work/buildroot'
...
#
# configuration written to /work/buildroot/.config
#
make: Leaving directory '/work/buildroot'
$ ./dockcross-tpi2-bmc make -C buildroot cjson-rebuild
make: Entering directory '/work/buildroot'
...
>>> toolchain-external-custom  Configuring
Cannot execute cross-compiler '/work/buildroot/output/host/opt/ext-toolchain/bin/arm-linux-gnueabi-gcc'
make: *** [package/pkg-generic.mk:282: /work/buildroot/output/build/toolchain-external-custom/.stamp_configured] Error 1
make: Leaving directory '/work/buildroot'
Note that I am using the -C switch to enter
buildroot
as the build script can't find the path
../br2t113pro
.
I assume that issue is caused by docker mounting the current dir for the build.
w
Yeah you have to do it in the directory
d
BUG: while turning all nodes off via the web interface (test the tpi tool), this does not turn off the PSU with the last node. It works ok with the
key1
button
j
>test the tpi tool since
tpi
calls the api, same as the web interface, the issue happens in either. I have tested with
tpi
here, both from the BMC and from a remote host.
p
Have been messing around with the BMC API and threw together a simple UI, should be feature complete by this weekend 🕺. It's
60kb
in size vs the
1.4MB
size of the current UI. Built with Picocss + SvelteKit. It also supports connecting to multiple BMC's from the same UI (as well as long polling the updates). Right now, a reverse proxy is required to patch the Headers until the firmware API is updated. Feed back is welcome! (Planning on getting it to 5x100 on Lighthouse) https://turing-pi-ui.vercel.app/ https://github.com/PhearZero/turing-pi-ui
d
You definitely should talk with @destomes who's also working on something: https://discord.com/channels/754950670175436841/1080282784570019942/1097566740625502298
I'm not sure how many designs we can have at once 😄
w
twice as many as half the number
d
Hah, theirs is a lot further along than mine. And both look to have the goal of also supporting external from BMC included support.
Also, there are so many differing frontend methodologies and frameworks, kinda hard to merge or cross develop things. Will take a look though.
d
Ideally, I'd love if you can come into some middle ground or agree on one or another. As I mentioned, I'd love to have something more like Proxmox, or maybe more towards Unraid, but the interface should not be biased with what I like. I guess we could try to have multiple interfaces too
Definitely I do not want you both put time into this and then make us to choose between one or another
I like how it looks like and how compact it is. The USB 2.0 toggle does not let you choose between host and device mode, though, and only one of them at a time can have the USB port assigned, so maybe radio button?
p
Yea I have everything disabled right now, it was mainly just to get a project scaffold. @destomes it should be pretty portable and I'd love to see what you have so far! Since it's all SFC + classless CSS it mostly will be copy-paste. Here is the main css library(also allows Bootstrap Grids): https://picocss.com/
d
I'm thinking of it and maybe we indeed can have a few different frontends so everyone can choose what they like.
p
Yea like a "theme" concept. As long as it's SPA/Static HTML it would be pretty easy to mange too
d
Then, to save space, we can gzip the files which will have an added benefit while serving them over http(s) as we'll only have to change the content type and the browser will automatically ungzip them
p
BTW just sharing for fun, not a big deal what path the different firmwares take from my perspective. I'll probably end up with my own fork of the firmware. Whatever works best for the most people for upstream, IMO. I'm big into HashiCorp/Mitchell Hashimoto and going to be running Consul/Nomad/Firecracker instead of K8's and wouldn't make sense to force that into upstream (unless enough people want it). Love the work everyone is doing! Keeps me motivated, really talented people in all the channels ♥️
d
I'd love to see any contribution, also your design, in this project. Whatever other ideas you have, some people will benefit from them too. The more I think about it the more I like the idea of having themes for the web interface. I'm a bit further with this idea and if we decide to stay with the embedded memory only, if there is a need, we could implement a feature where only a single interface of choice is going to be installed at a time. Choices are always good
So, I hope to see a pull request with your idea, or if you just say we can use your interface, we can make it fit the firmware - whatever works best
d
That's the beauty of open source.
p
Yea I'll definitely keep digging into the REST services side. That's the main reason for the UI is to test in the browser against the BMC. Was a quick and light frontend that won't weigh me down while I'm making API changes. I'll create a specification for a interface this weekend and we all can do some RFC. Maybe next week we can at least get a shared JS/TS library for the existing API until we formalize the spec
d
The thing is we'll have to decide of what to use for the backend. It can be RESTful API, it can be simple JSON or (my favorute) JSON-RPC, or maybe even something else
d
The stack I chose also based off a template and uses a few things I haven't used yet and am learning about. One of those being Vite for the bundling and Recoil for state management. So it has been a bit slow starting up. I also feel like we could have an optimal API voice meeting at some point.
p
Same, ProtoBuff is an oldie but goodie. MessagePack is the new hotness but vanilla JSON is fine for something attached to a LAN.
d
I was going to just make a quick fastAPI server with what I consider as optimal, and have it typed as well. and just temporarily proxy to the old API. but my understanding is the turning pi fork of the original firmware is also being worked on right now. Is it worth it to follow along and contribute to that one discussions instead?
d
I think it should be rather simple so different tools can use it easily. ProtoBuf or MessagePack - don't think we have to turn the communication into binary. Hmmm
p
Check out swagger/openapi if you haven't seen it yet. It's really useful for sharing API designs: https://editor.swagger.io/
d
No serious work is being done already. And indeed it'd be great to decide of the APi type and rework the backend for it
p
Lol yea React finally realized they need state management. Guess they are no longer "just a renderer" lol
d
Yeah, FastAPI supports outputting a generated openapi spec and hosted swagger UI
p
Sweet, pretty much everything these days is on the bandwagon. It's a lot of fun, I work the other way around with the codegen.
d
I'm leaning towards some REST implementation, hmmm. But the way the interface can be constructed would mean multiple calls to the API. this is where JSON-RPC is what I like - it allows you put all the requests in one object and process them this way
Like if you want to turn power on in Node 1 and set USB mode to device for that node at the same time )or even different node), that'll mean 2 API calls that not necessarily have to be successful
p
http/2 or QUIC maintains a single connection, they might be good options as well. Nothing wrong with JSON-RPC as well
d
I do not mean maintaining a connection. Also HTTP/2 might be a bit overkill for the BMC
p
Yea not sure, I've always leveraged nginx and it's been fine. As far KISS, regular REST with "bulk" endpoints could optimize the requests
d
For the purpose of this firmware, we have to keep the HTTP server simple and the communication simple too
d
I'm a big fan of graphql for bundling many typed requests into one. But that may be way overkill
d
We won't cram Nginix into this firmware - it alone would be multiple times bigger than the whole firmware 🙂
As much as I understand and like the ideas, it must to be simple
We're working with a device that's limited with the computational power and we have limited storage. Also 128 MB of RAM does not help either
I think the backend API should be a pretty simple one, REST-based probably, but maybe even not. JSON is native for JS and sending request as JSON objects to the backend might be all we need. But I also haven't thought about it much yet
d
@DhanOS (Daniel Kukiela) related to your UI suggestion. I'm unfamiliar with proxmox but have used vmware esxi, They appear to have similar interfaces? If you have any screens you're interested in emulating, I can take a look. Overall it looks like possibly grouping multiple turing pi2 boards and nodes in a left nav section and a detail section on the right is what you're after? with possible group selection and actions on those selections?
p
I'm guessing a Turing Pi datacenter would be the only users to really complain 🤣
This does look interesting for future research: https://github.com/microsoft/msquic
d
Someone is going to build a giant turing pi2 multi board cluster at some point.
d
I only mean I might be more into raw-looking interfaces instead of the (currently supper common) designs that put form before function. Like some interfaces are wasting too much space to make them look good, but this can also make them more chaotic. I think we can make an interface for me later, when e have the backend and some new frontend working 😄
Maybe not giant, but i want 4 boards and fit them into this case:
😄
d
You saying my interface is too pretty?
d
No, I'm saying people have different taste and this is why I like the approach to have multiple options so everyone can choose what they want. I haven;t seen much of your interface yet to say I like it or not 😛
d
Just poking fun. 😛
d
Speaking of multi-board solutions - I would also love to see the interface to be able to either manage multiple boards at once or the list of machines could redirect to the interfaces of different machines. Managing multiple machines at once sounds better since you could show all nodes as a single list
d
That is part of the intent with my interface, as I have two boards. It looks like @phearzero's supports that as well.
p
Also @destomes checkout #1097689024325492887 I did a info dump the other day. That's at least where I left off mentally
d
We can talk about the backend more, we can voice talk about it too, but then you think about it on your own remember how limited we are with hardware - this is pretty slow chip with just a little bit of RAM and storage and we have to be sane about the implementation. I'm still down for a simple http server crafted specifically for this purpose. I built 2 small servers already - one HTTP server in Python (yes, I know, but python was use there for other things too) where I integrated the GET and POST methods into the other parts of the system and it worked well, but also a SOCKET server into... GTA5 to let different clients (also a Twitch IRC connector) to interact with it. Both worked great.
p
It's a fun target, I'm sure the Disk IO is not the greatest as well. That's freaking awesome! Services are a lot of fun and can be really rewarding! Heads up, I'm have high functioning autism and CompSci is my main special interest. Spent most of my time in telemetry/analytics and crypto. Lot's of system/web whatevers just really enjoy the field and the people are amazing.
d
I must admin this makes me a little bit more excited about this project. I'll still need a few days before I get to that, but can't wait for the development 🙂
t
Based on examination of bmc, webserver and tpi, they all seem to use GPIO directly. There is no central daemon/API that coordinates all requests. IMHO, this should probably be bmc. Obviously, requests need to be serialized with mutexes to keep API requests from stepping on each other. It looks like webserver does this to some, decentralized degree. You can see this by turning on all nodes with tpi/webserver, then press the power button. The power button code doesn't check current state and just blindly turns everything on again (no effect).
d
Yeah, and this is why I thought of integrating of the picocom into the same service so we can have a single connection open and interact with it via command, via web interface or have an ability for the service itself to use it (for additional features like node detection). Indeed some coordination in the web interface is needed and I already thought of this a bit and I am not sure about the implementation - websockets are a bit to much for this, so maybe the front can "ask" the backend for given number of seconds, maybe 1 second and update the states. That'd be useful to also show other information live, for example IP addresses of the nodes which I'd also like to add there, or the BMC system information.
Sky is the limit
And the memory
(both RAM and disk 😛 )
t
Yeah. The RPC/listener should just acknowledge the request and block until the request is complete. The clients should respond with "busy" while a request is being processed. That avoids having to implement queuing and shifts the UI logic to the clients: tpi and webserver. They can respond to the user with an appropriate message to retry in 15 seconds. I'm not certain whether retry should be automatically attempted.
r
Please consider the "No UI" option as well. Personally I am only interested in the API and associated console tools for DevOps and if we end up with a configuration system that allows for the BMC image to be configured with different options, it would be great to be able to skip some components in favour of others.
d
I would opt for a timeout, so the request would wai (if blocked) but up to given amount of time
I would not deny the request unless the state does not match and it has to
r
Also, here is a controversial alternative to an API: why not consider MQTT? If we consider us to be in the IIOT space, MQTT based on Sparkplug B would be a natural and modern choice... 😉 https://sparkplug.eclipse.org/
d
Some sort of GUI will be needed at least for BMC upgrade. A very simple one might be used for this purpose. But good point
It's not controversial. I'm considering different ways of managing teh boards 🙂
r
Any MQTT server has the advantage of handling synchronisation of the commands, comes naturally with the queue/topic concept.
d
But this will have to also integrate nicely with the other ways of controlling the board, so it's not down to the MQTT protocol only
t
Would you accept and queue the request in the centralized service or just tell the client to resend. Timeouts might frustrate users (it's a feature, not a bug.)
r
No, but you could have the API publish commands to the MQTT server and thus handling the queuing aspect naturally.
t
As long as the interface is public, people can create their own "clients".
d
This is quite far into the development and I haven't thought about this well yet. I don't think people are going to send requests to the BMC in different ways at the same time, so some simple system would do IMO (simple synchronization primitive)
But still, you'll have a way to connect and interact with the BMC in other ways, not only by MQTT, which is different with typical implementations of MQTT
t
I don't expect you to implement this stuff in the CE firmware. This is something for the RUST programer(s).
d
This is the place we're discussing the CE firmware features 🙂
r
Yes, of course. What I meant is we need a queuing aspect for the commands whether they come from any of the tools you choose and MQTT servers provide a queuing system OOTB.
t
Would the MQTT run on the BMC or somewhere else? The BMC has both limited memory and storage.
p
I've run MQTT on Pi zeros, tons of implementations out there. The zero has a bit more memory but it hasn't complained yet. It's one of the setups supported by Adafruit and Circuit/MicroPython
r
I believe there are MQTT servers running on the Pico as a python lib and I am pretty sure we can find something similar for Rust as well... 🙂
d
Although we're not going with Rust here, I don't think so 🙂
r
Go?
d
C++ is what's being used currently
r
Ugh...
d
I'd love to have Python, but the memory footprint might be too big
p
Who's on team Zig? lol am I alone?
d
The whole firmware is C++
p
MicroPython was something I wanted to mess with on it at somepoint
d
But this is for microcontrollers, it compiles into the assembly
I will play with MicroPython with raspberry Pi Pico
r
Yeah, I know. But even Linux core developers are now accepting Rust in the build tree... Just saying...
d
But we're expanding what's currently done. Mixing C++ and other language in a single script is rather not possible 🙂
If this was a library or separate tool, then sure
r
I agree, keep it simple and stupid. You will just have very limited help from me with C++ as I will not invest time to (re)learn the language. Call it religious reasons... In any case, the question is moot as I still can't build on my M1 Mac. 🙁
d
I mean I don't know where we are going with this. I would have to learn Rust or Go, or whatever you propose 😉 Then, I don't think we want to start with re-writing the firmware from scratch in a different language - this will put us in a several month of work and the team is working on a new firmware already and most of the people will probably move to it in the future, we'll see. Then, the aim here is to provide features before the new firmware. but who knows when we end up being with this, maybe we'll be developing this firmware further and further indeed and have it as an alternative. then maybe we decide to rewrite it. For now, the goal is to provide more functionality. As for building, I'm using VMWare Player (free) and Ubuntu 22.04 installed in it.
r
Don't worry - I agree we should not rewrite anything. Was just carried away by the mention of Rust.
I will try VM player - would love to be able to join the fun and roll a version of my own.
d
And I'm even personally Keen to try Rust. I've been using so many programming languages in my life and this one seems like something worth looking at
m
Once you get used to Rust, then you will love Rust 😅
d
I've heard this so many times in my life already speaking of all kind of programming languages 😄
m
I've been using C++ for years but now having some affairs with Rust 😍. Then wishing C++ had a simple toolchain like Rust, just cargo install and run 🥹
d
This reminds me how I swapped PHP for Python. Maybe I'll indeed like Rust as much too 🙂
m
well, having a good editor that can show data type of each line of code will help a lot 😄
g
I taught myself Perl years ago but don't remember much anymore. I currently use RPG and C# for work.
k
Has been that way for me with Elixir, love a ton of languages but once I touched Elixir, I'm like, nope, don't need anything else at this point
(Though I do need to give Rust a proper try one day 'cause there's a lot of good synergy between Elixir and Rust)
r
Checked my own statement, and it seems like gcc support for Rust still has some ways to go, target date would appear to be this summer. So even if we would consider it, mixing Rust and C++ would entail mixing the gcc and rustc toolchains... which seems challenging.
Must have been rambling last night, there would be no MQTT server on the BMC. For MQTT support there would be a MQTT client that pulls messages of an MQTT broker and then executes them. Also forget anything about this being helpful for the other means of configuring the BMC and the nodes as the synchronisation I had in my head happens in the MQTT server. Of course we would only pull one message at a time, but to benefit from this we would need the server... (duh). In a pureplay MQTT scenario there would still be a need for bootstrapping the MQTT client configuration (server, credentials, etc) and also for flashing the BMC, so at the very minimum there should be a REST API for this. In summary: MQTT support would be complementary but as we are only talking about a thin client that would rely on the same logic for handling any other management commands, it could probably be very light weight. The advantage of MQTT, in addition to being a modern way of managing your IIOT device, would be that it should also support streaming of logs and any other data we could think of coming out of the RPI.
p
Yay for JS api clients 🎉: https://www.npmjs.com/package/turing-pi-js
r
Looks like Rust is a thing for the official firmware for the BMC at least: > We have a basic firmware built on C which is a starting point for building a state of the art firmware in Rust https://github.com/wenyi0421/turing-pi https://turingpi.com/jobs/firmware-developer/
j
To go just a tad further on this tangent. Do you guys recommend any resource for leaning rust? I have some c/ c++ background from uni days and mostly work with python nowadays.
d
Started learning myself, received 2nd Ed of "The Rust Programming Language", Steve Klabnik and Carol Nichols, (Rust 2021). I am enjoying their presenations
m
You can try Rustlings next https://github.com/rust-lang/rustlings
k
Oh that reminds me (and I promise to stop derailing this thread after this message 🤭) that I wanted to one day learn a language using Exercism only, it’s a brilliant website: https://exercism.org/tracks/rust
p
I've got
rtty
baked into my firmware but
rttys
and
gotty
are pretty heavy. Looking into other options but a lot of them are pretty heavy. Anyone know of any other good
xterm.js
backends that are lightweight?
s
I've noticed that the 12V fan-header only gets powered when the first node is powered-up, but then stays on so long as the system is powered, so far as I can tell: * Can we have independent control of the fan header? * Can there be a run-on option where the fan header is powered for a user-specified number of seconds after the last node is powered-down? (It's possible the the latter is already the case - but not for a customisable period, and I didn't wait for long enough!)
d
This header is connected to +!2V from the power supply. There's no way to control it in any way - neither to control fan speed nor the on/off state. The fact it does not turn off with the last node is a bug in the firmware. The Community Edition firmware is going to fix this soon. If you turn off the nodes using the power button or
key1
on the board, the fan will turn off
m
HI all, I am new here to the discussion. I had a few thoughts though to get more out of the BMC. First, busybox has a builtin httpd server. So could that be used in order to not use more memory? Second, I would like to try building a firmware with dropbear ssh instead of openssh to get a bit more back as well. Any thoughts?
d
About the https server, the web panel is tightly integrated with the firmware so the actions invoked by the panel or the
tpi
tool (which also queries the webservice) can be actually executed. With a separate webserver it's had to communicate withj the firmware somehow anyway. I'm not familiar with dropbear ssh. What would be the advantages over openssh?
m
Dropbear is a much smaller binary than openssh.
d
Even as small as 110kB, hmmm, worth looking at probably. OpenSSH currently is like 6MB iirc
Thank you for the suggestion!
w
I notice mentions around the place of ulibc, musl etc. Seems the existing distro uses ordinary glibc. Not sure how much space it would save but interesting - and might actually make cross compiling easier because there are so many glibc versions, idk
d
Some experiments might have been done
For example, the bmc binary uses about 4 MB in the memory iirc, but I did not check yet how much of that is an actual binary and how much variables (or memory allocations in general)
This is more than 1/3rd of the available memory and even if we still have free memory to work with, thinking of lowering the memory footprint is definitely a good idea since this will let us run more stuff in the future
p
swupdater is pretty heavy and it comes with it's own webserver. I think it's port 8080 on the BMC
That service does have websockets however, might be useful. Not sure if it's goahead or not.
I kill that pid everytime I reboot, going to test if I can flash with USB still and I might rm -rf it completely from my firmware builds
d
Hmmm, having it running all the time might not be the best thing. Thanks for pointing it out. The CE might only run it when necessary. Although I would like to have a way to perform updates more in a linux style than by OTA
Flash with USB? Tou mean the nodes? It's indepemndent from the node flashing
Which process is this?
p
The BMC flash port for PheonixSuite (MicroUSB). Bricked my BMC a few times yesterday and had to recover.
Copy code
bash
/usr/bin/swupdate -v -w -r /var/www/swupdate -p 8080
It's a few MB's in mem, It also has another process that starts using CPU once that service is down. I also normally kill it as well
Copy code
bash
/usr/bin/swupdate-progress
d
OK, thank you. I'll look at it and most likely just make the processes run only on OTA (like when you submit a firmware, run these binaries - since we do not autostart them and the software update causes the BMC restart, they won't start again automatically after)
p
This week I should have more free time, I'm going to update the readme for building on my fork feel free to borrow from it. Been working on web-api changes and experimenting, it's all in WIP shell scripts right now which might be easier for most people. They are all called
fix.sh
🤣
d
Thank you! I'll take a look and see which parts would be useful for the CE version 🙂
Is this a form of my repo or the original one?
p
It can be either one, just little things like looking in the buildroot output before copying the default configs/other libs. All based on the same README.md in both so it should be pretty compatible
d
I'm asking where to look to find your fork 🙂
p
I have both right now: Original: https://github.com/PhearZero/turing-pi CE: https://github.com/Telluric/turing-pi-2-community-edition-firmware My main work is off the Original but planning to port to CE relatively soon.
d
Ok, thank you! I'll take a look
p
It also is still on my workstation, just getting comfy with the project really. I'll ping you when I have something pushed.
Also investigated authentication for the web server. Luckily it's already builtin to goahead and you can find an example route at
http://BMCIP.local/auth/basic/admin/
The routes/configs are stored in the BMC under
/mnt/routes.tx
/mnt/auth.txt
The buildroot defaults are here: https://github.com/Telluric/turing-pi-2-community-edition-firmware/tree/master/br2t113pro/board/100ask/rootfs_overlay/mnt helpful link: https://www.embedthis.com/goahead/doc/ref/api/goahead.html
d
This is helpful. Thank you.
w
Is xinetd still a thing or did it get eaten by systemd?
Feels like "service that starts up when you try to connect to it" is a good fit for this kinda thing
d
Hmmm. In this case we also must start the service after successful firmware file update, so not a problem
w
Btw I haven't successfully done this yet, latest guess is it seems like a CM4 needs to be specially configured to put Bluetooth on mini UART to receive serial breaks
p
The process kicks off in
init.d
. Gives me feeellz from the past, haven't used
init.d
in ages. The process is in
/etc/init.d/S80swupdate
I also question the use of
collectd
and haven't really dug into what exactly it's trying to grab. It's a long lived service on boot as well
Alright, got my frontend running on the BMC. (Simplified it for this iteration) I'll post the docs tomorrow on how to deploy if anyone wants to switch up the web look.
d
May we discuss a way for all of the interfaces to switch between them? I'd love to add this one to the firmware, but we need a way to switch between them
Oh, did you add this already in the top-right corner?
p
That's in the
next
branch right now for a "global" dashboard. Wanted a super simple implementation right now but I can get a very basic redirect switch in there tmrw.
d
No rush, I need a few days still to integrate it
It'll be also good if we can somehow ensure same functionalities of all of the interfaces
p
Yea a good open discussion would be awesome. The challenge is always in replicating state. Super easy to run a service on one of the nodes and just always point to it in the browser. If each of the BMC's have to maintain a list it becomes less trivial. This design is specifically for the local interface on a BMC using the current web server.
d
I can handle the logic of interconnecting the boards, this isbthe easy part for me as opposed to all this HTML/CSS magic. The interfaces should be able to identify the boards and the nodes and I'll make sure to have the right data. I don't have the final idea about thatvyet, though
p
Yea that is exactly what's needed, a "BMC discovery" layer that maintains a peers list. It can be a part of the firmware or as a service that is run anywhere. Creating a service would be a good start and I'm pretty sure that is what @destomes was working on in Python. That way we just have the Python API as an input in the basic UI on the firmware. Once you add the discovery layer/api url, it will populate all the other nodes/full dashboard. If we find it works well we can look at porting it to the BMC in C/C++ with probably limited functionality.
Once we have a list of services we can do 99% of the work in the browsers to fetch state, add a few endpoints to the api to reduce requests. Then start to extend the bmc api to expose more functionality.
d
It must be a part of the firmware
The only thing in Python I've seen was a webservice wrapper, but I could have missed something
p
Sweet then that should be one of our first new endpoints on the BMC. /peers
d
I'll add interconnection directly to the bmc process
I'm not sure autodiscovery is a way, but can be helpful. I want to let people set the BMC hostnames and then we could have a way to cluster the boards by the host names. mDNS should do the rest
So, what I mean is you, as a user, will add hostnames to the cluster of the boards
They will then propagate across all clustered boards
p
Yea that would work great, a lot less to filter too on a busy network. Some kind of name convention or something similar. You enter a TLD like
amazing-cluster.local
and let the boards be subdomains.
d
Then, each boards interface could be used or we can make it so one of the boards is the master. But I like an idea of each boards interface since the master board could be offline and this should not cut you out of the interface
p
100% agree and each one just pulls the current state from the network. Masterless is my favorite topology 🕺. Super easy from the front end perspective, especially once we have cleaner routes/more concise endpoints.
The next version has multi BMC support and hopefully TTY support. Been digging for a few days trying to find a decent WebTTY project for embedded. I'm running a slimmed down
tty-share
on the BMC but it's still really heavy.
d
Hmmm. The idea was to show a log of the UART output so you can scroll through it and see all of the messages. Will this allow for it? I did not think of a ful TTY yet.
p
Yea we can just match WebTTY, it's what vscode/rstudio etc etc uses for term. We can even emulate other terms
d
A TTY with multiple features would be a nice thing. I'll have to see how it works. But we also need to integrate it with the BMC service so it reuses already established serial connections
We definitely should not establish multiple connections and let multiple clients to communicate at once. I see how this can cause issues
p
Yea just one at a time and only for convenience.
w
I volunteer as a beta tester
+1, my primary use case is to check logs, interactive use a distant second
w
lol, systemd is the last thing the BMC needs
p
The simple UI is stable (it doesn't have TTY in this version, just an updated UI): https://github.com/PhearZero/turing-pi-ui
s
Do you have any screenshots, or a demo instance? I'd also strongly suggest moving
/mnt/var/www
out of the way rather than deleting it (there is still ~6.7MB free on the BMC filesystem, plus you could move one or the other directory trees or the
build.tar
to SD card - and you could even
tar -cjpf - build | ssh $BMC_IP 'tar -xjvpf - -C /mnt/var
to avoid the tape archive itself using any of this space), as it gives testers a way to roll-back without having to reflash their firmware...
p
The demo instance doesn't have any mocked data but here it is: https://turing-pi-ui.vercel.app/ Sure thing, I can definitely backup
www
and add the pipe.
s
Ah, that's neat actually! From a UI/UX point of view, are there any additional types of input control available which can differentiate between "USB 2.0" (exactly one must be active at all times) and "Power" (any number zero or more can be active at once)? I guess that SD card details are all that's missing to reach the same functionality as the existing firmware? What might be neat (and already planned?) is to provide a way for nodes to push status updates to the BMC to reflect - for starters - an alarm-state (or even the active run-level?) which might be a case of accepting a POST-request and writing the provided contents to a file to later be read by the web-UI (if the built-in web-server has this capability)?
p
Just pushed the update to the shell script, thanks for the feedback 💗! 1. The USB 2.0 switch act as a single toggle when deployed, only one will ever be active and if something fails it reverts to the current state of the BMC. 2. Good point, I always forget about the sd card! Adding that to the missing features, should be easy to integrate. 3. Yea agreed! I think that is what drove a lot of us into the custom/community firmware side. As it is now, everyone is in "brainstorm" mode. The community firmware is maintained by @DhanOS (Daniel Kukiela) and is the definitive source for all things community firmware related. The main repo is here: https://github.com/daniel-kukiela/turing-pi-2-community-edition-firmware
s
Left a couple of comments!
d
Speaking of the USB, I proposed radio buttons As for the node-BMC communication I've been already thinking of utilizing the UART for this purpose. The nodes can return some statuses even now and we can read them, the same will work in the other way, bit with some service account on the nodes we could even log in (although this might be not necessary, just some service "talking" with the BMC
One more thing as for the USB mode - once we add a way to recognize or set which node has which module inserted, that'll mean we'l have different possibilities for the USB mode settings - jetson devices, for example, do not support the host mode. Ideally I'll split USB host/device from recovery mode and let set them independently. Recovery mode can be enabled/disabled in per-node basis like power and the USB mode can be changed only for CM4 nodes (Jetson devices should have the drop-down list locked to device)
s
Presumably the BMC can't actually read any USB data (only switch the connection) and so can't retrieve any USB device IDs on which to base this?
(Unless it's also possible to briefly switch the USB2.0 connection directly from the node to the BMC, disabling the external port?)
d
This could actually be a way of recognizing them automatically, but could also be in conflict with how someone uses the board. My way is to follow what the original firmware is doing - parse the UART logs and figure out the node type this way
it could be, yes, but this is what I mean - someone might be relying on some USB device availability maybe
p
I'm agnostic on toggle vs radio, anyone have any opinions? I remember the radio suggestion the other day but forgot to try it out.
s
👆This is better!
d
Yeah, I loike it more too 🙂
Things we could add in the future: - board name (liek when we use multiple boards) - host name (the name of host in the OS, read through UART) - ip address (same, read through UART or in other way) - another toggle for the recovery mode (when/if we split USB mode from the recovery mode - currently this toggled them both) - with multiple boards we'll need also a way to group the radio buttons across a single node (maybe using a color?)
Actually, I must admit I like it a lot 🙂
p
For multiple boards I was going to have headers with the server info and subtables. We could put actions/buttons on the header rows as well
d
Hmmm, maybe that'd be better. I don't know. I imagined showing all nodes more like they a part of a bigger cluster, but maybe splitting them by boards is a better idea indeed
p
Won't be much to mock up either one/both once I'm back into the multi-server version.
k
Copy-pasting a note from #754950670175436848 in here – when editing
S99hello.sh
(or anything in the
/etc/init.d
directory), be aware that the scripts in there are run on any run level change What that means, specifically, is that any commands you add in there will be run both on BMC boot as well as just prior to a BMC safe shutdown (via
shutdown -h now
or similar). If you want to run commands only on startup, check the
$1
variable for
start
value, similarly
$1
will equal
stop
when it’s time to shut down the service Ref: https://discord.com/channels/754950670175436841/754950670175436848/1099875572051615794
p
d
You crammed many different things into one PR. I'll review them and let you know. Thank you! Hope you don't mind if I'll have some different perspective on some of your changes (this is why it;s easier to make a PR per each thing separately, now I have all or nothing 😄 )
I'm not a fan of issue templates, btw. This is a small project and I mostly like when people just describe their problem instead of trying to fir it artificially into a template
I have a script to set versions already, you added own breaking mine 🙂
As for the rest - I'll review them later, but there are sadly things I cannot merge. I'm not sure what to do in such situation
p
It's just moved to ./scripts/set_version.sh and added a ./tpi.sh with an easier way for new people to try to build. It's kind of a black box for most people in it's current state
d
I also want to retain raw commands in the main readme - with the scripts file it's harder to find when something errors since another commands are still going to be executed. And breaking occurs a lot
If we want to use scripts, we have to ensure that each command finishes with the right stats and only then execute another. Also, during development, you do not run all the commands, more like use them selectively, hmmm I'm not sure how to handle this via the scripts
p
Check it out first when you get some free time. It's split into 4 parts and the aim is to get every developers working directory to the point where they can easily rebuild. There are checks for errors with tips on what is missing etc.
d
I will check, the only problem I have now is I'm not sure how to handle the fact that I have a different view on some changes that you made... Ehhh, sorry, it's hard for me to find the right words. The thing is I also have own ideas for some of the things and now I'm not feeling comfortable because I don't want to not merge your request, you put time and effort into it, but also some of the things are not fully in-line with how I want to approach them. Hopefully this makes any sense to you.
p
Yea it's no big deal at all, you can manage the project however. It's not wasted energy since I'm already maintaining a fork which will leverage them. It's really ok and appreciate everything you are doing I was just hoping to help other people join the fun on the CE side. Seems like the main issue is discovery, that was something that would be obfuscated by a build script. The compromise would to have both, just add a second markdown with the full line by line details and if something fails then try the line by line guide. For me I need to run both mkfw and set_version constantly so it made sense to me to have an entrypoint that does just that. Figured throwing in the build script would make it less intimidating
d
I'll definitely check your pull request and see about the parts I like. But what should I do, just copy the changes over?
p
We can discus on the pr. Evaluate it and we can even drop that one in favor of one we both like. It's not critical to me to have it merged, if you see something you like I'm more than happy to work towards a solution 🕺!
This is what I do to relax, definitely don't want you to be stressed over the code! 💗
s
Much as it’s a pain - and I’ve definitely tripped over this myself in the past, professionally - it’s best-practice to raise PRs with one intended outcome per PR… so in this case, perhaps one for the GitHub-related updates such as Issue Templates, one for the README, either one per script or one with all of the script updates/additions (and a GHA
shellcheck
of these would be awesome too!), in order to allow each to be considered independently.
w
set -eu
would do the trick right?
Ahh the eternal joy of trying to balance action and forward movement against having that movement go in the right direction. A challenge for any project
I mean you know, at work the standard approach would be to write a design document describing the proposed changes, alternatives considered, considerations and having the decision-maker approve the approach in advance. But even in that corporate environment I tell my reports that they can start development before the doc is approved provided they accept the risk that the approver might want changes
d
There is no single right approach here, but multiple right approaches, each with some different direction would also not lead to anything coherent
I also do not mean writing documents, but once the project will start going one direction, everyone will tune into it
I've worked at a corporation, I'm not right now and this feels better
I'm not sure I understand what do you mean here, especially the
against having that movement go in the right direction
part
Oh, ok, I think I confused the meaning of
against
in this context
b
OIC
Very nice, I'll have to look into this, maybe I can help contribute some of the things to it to get some of those TODO's checked off
d
I'll be putting al feature requests and ideas into the GH issues, make it going and then see where it'll take us 🙂
s
Also, if at all possible, a user should be able to reboot the BMC without any powered-on nodes losing power.
d
This is something we'll check if is possible
p
I'll be maintaining my fork along side CE/Official and feel free to borrow from it. Currently adding a new REST endpoint
v2
which cleans up the API calls as well as staging the API/UI for Authentication. https://github.com/PhearZero/zero-pi-2
I wonder if we could expose the UART over something like
ser2net
https://github.com/longshine/ser2nets <-- same thing but with WebSocket support
d
We probably can, but this would not work for the 1st stage bootloader and who know how big part of the second stage bootloader. And when it starts working SSH will be available already (if you want to use it instead)
w
I mean for the node UARTs
not sure I understand why what stage the node is in booting would impact whether you could forward its serial port over some arbitrary protocol
d
Because you'd have to somehow add this to the firmware
And even more make it work with a network that comes only later, during late 2nd stage bootloader
w
I'm proposing installing ser2net[s] on the BMC, to read the UART from the modules themselves
not the BMC UART, which seems to be what you think I mean
d
No, I actually thought you mean to install this on the nodes. And now I see that'd make no sense. Just woke up 😄
Yes, it'd make sense to put that on the BMC. What could be the use case (like compared to the picocom over ssh)?
w
I mean, not having to ssh into the BMC to read UART
especially if websocket support was available, you could do it from the web
d
I'm asking because people usually ask to keep things "behind" authentication of some sort
Even root logins over SSH are meh
w
indeed, websocket support would mean being able to do the usual mutual TLS or something
d
TLS means encryption, not necessarily authentication. I need to read through that more. ser2nets supports http for websockets, which means it might support authentication and then upgrade the connection to the sockets maybe
If authentication is not supported, that would have to be an option
But yes, definitely something to look at
w
idk I've just been interested in a bunch of these options for setting up remote access to the serial ports
d
And this makes sense indeed. I can, however, see how others might not expose additional things in a manner that does not involve authentication, so we have to take this into account
w
I suppose we're all free to add whatever we want to our own installations
d
I also can see adding this (and other features) more in a way where you choose what do you want to have installed/running
This way you can have this without any authentication. I would personally also use it this way
I'm using UART a lot and also I don't have to have a password on everything
I also like the idea
After a bit of thinking I think we could have the web interface but also a command of some sort that will let us enable/disable and start/stop services
p
I'm actually about to push UART to turing-pi-ui. The firmware already has it baked in and TTY is about 70% feature complete.
It's my weekend goal is to finalize all of the missing features from the current web ui. The main one is uart, I used a few heavy tty protocols and didn't even think that xterm.js could read from /api/bmc?opt=get&type=uart Just a heads up, this does mean that all nodes are exposed on the local network with RPC to the local nodes at the /api/bmc?opt=set&type=uart&node=0&cmd="Do something dangerous". Generally the UART should prompt with login but there could be some cases where the uart does not have shell login
Should work with CE and Official firmware (no dependencies needed at the moment): Here is the demo render: https://turing-pi-ui.vercel.app/ Repo: https://github.com/PhearZero/turing-pi-ui
d
Many people are going to love this 🙂
j
Just posting another fork of the original repository. My aim was to look at if we can extend the networking features of the BMC linux: https://github.com/j0ju/turing-pi/tree/main As an idea I also ported my random persistent MAC feature to the Community Edition and created a PR: https://github.com/daniel-kukiela/turing-pi-2-community-edition-firmware/pull/7
p
I'm hoping to post it in chat this weekend after I get firmware uploads 🕺 and it will be it's official "1.0.0". Super jazzed about it
d
Hi! Thank you for the pull request! There are 4 things I want to mention:. First is you should not use the Allwinner prefix. This is only being used by them to assign MAC to a device. When you are assigning a MAC on your own, you should use one of the ranges for the user-assigned MACs. The ranges are:
Copy code
x2-xx-xx-xx-xx-xx
x6-xx-xx-xx-xx-xx
xA-xx-xx-xx-xx-xx
xE-xx-xx-xx-xx-xx
Second - the goal is to be able to set given (or random) mac via the command and via the web interface. Can you implement similar thing into the
bmc
since this will be the desired way? Third - I only briefly checked your code - isn't it missing
ifup
? Fourth - if you want to modify a file like
/etc/network/interfaces
, it's better to hook the image creation and "regex" the change into the file instead of replacing it entirely. This ensures multiple updates can be done conditionally Additional question - what's the benefit of using the environmental variable compared to simply adding
hwaddr etehr
into the
/etc/network/interfaces
And I am going to integrate it into the firmware in a few coming days so everyone can simply flash new firmware and use it 🙂
j
1. I know that there are user ranges for MACs. Surely I can change that. Any range preference? The Allwinner OUI makes much sense, to me, better would be an OUI from TuringPI itself. 2.
fw_setenv mac_eth0 xx:xx:xx:xx:xx:xx
is the commandline for the shell. This can surely be wrapped into an API call to be set on demand. 3. that point I do not understand. What do you mean be
ifup
is missing? This code is executed by the ifupdown framework itself on
if up -a
eg. during the network init code. 4. Currently it is dependent on the defined
uboot-hwaddress yes
in the supplied
interfaces
file. 5. If the user changes the MAC in the filesystem it is lost after an update. On the other hand with the script if-pre-up.d it is set from uboot env. If the MAC address is persisted in /etc/network/interfaces it would change on updates. The uboot.env is persistend as long as you update only via webinterface.
About the 4th. Root logins for 1.0.1 based releases should work out of the Box, although that what the original firmware is doing, also allows empty passwords for root 😕
Regarding the storage of MAC addresses in UBoot. The original UBoot of TinaLinux, that seems to be used as base for this project. Is using the UBoot environment for storing MAC for LAN, WiFi and BT. The code is still in the UBoot bootcmd, but not used by the BMC Linux nor are there proper values defined. So this is NIH.
d
1. If Turing Pi had own range, it'll most likely be utilized (and this can also be a thing in the future), but no matter Allwinner or Turing Pi, you should only use MACs from the user-assigned ranges since you are setting it by hand. For a quick hack to set a static MAC I used
12:34:56:78:9A:BC
, so probably the first range 🙂 2 Yes, but I also meant this whole part to randomly generate a MAC 3. I mean: ```sh set_MAC() { ip link set down dev "$IFACE" ip link set addr "$MAC" dev "$IFACE" echo "$IFACE: set mac address $MAC" >&2 } ```and I should mention
ip link set up
bot
ifup
4. I understand and this is what I mean, I later provided an example of how you can only modify configuration files for the parts you need without overlay-in a whole file 5. The goal is to have more linux-like updates, the config files will remain. Also, there is going to be initial configuration file that you upload via the web interface or put into the SD card, so the user will keep it for when it'll be needed anyway
They do. I forked the firmware where there was 1.0.1 and added root logins before 1.0.1 was a thing. 1.0.1 also does the same thing - overlays a whole config file instead of modifying it on the fly to change this one thing
I guess if Turing Pi wanted to set MACs from their range, if they had some, they'll use this indeed. We're setting MACs by hand, though and some sort of a new firmware is coming so I might want to avoid using this way, at last for now
Does this make sense?
I want to add so many features to this firmware, but I also want to ensure they're added in a coherent way, otherwise we'll end up in many different ideas that don't work well together
j
1./2. Will update the PR. 3. the ip link up happens during ifup process. This is executed during ifup-pre, before bringing up the interface. The ip link set down is just to ensure I can change the MAC address, as it is only possible if interface is down. 5. Then this solution is best, as it stores the MAC accessible via the uboot env from even if running from SD card, and it is not harmed besides from a full reflash. (.img)
d
3. Right, I understand 5. Well, let me say this again - there will be a single file containing all the settings, not just MAC, but much more. This is why I want o start developing if further again, so the direction is visible.
Ok, maybe not like a single file that contains the settings, but I hope it'll get clear once we add thi to the firmware
j
This makes sense, there are many forks out there, some have barely understandable commits and the feature set and goal is unknown. I started from scratch for debugging fun and found your repo later, and the code a bit diverged.
d
The code has been barely touched by me, yet
We've collected a heck lot of feature requests here
My next step is to create GH issues for each issue or feature request for easier tracking and start implementing them
j
A single file is a bad idea. As the BMC is more a network device with some extras I like the config format and handling of OpenWRT.
d
Then, hopefully, more people will join and follow the same direction
I understand we do not have to agree on things, but there's an idea behind a single file.
The BMC is a general purpose CPU that'll be running all sorts of stuff. It's not just a network device. It'll be able to flash nodes in the future and set them up and manage them, for example
j
If you want a single file in the 1.0.0 is a file /etc/tpi.cfg 😉 And even the BMC initscript using this file, it is tinkering with the network interfaces after they already brought up 😉
d
A single configuration file is meant to let the settings to be centralized and easily manageable via all sorts of interfaces like the web interface, a command of some sort or even via external systems if you wish to use them to set the cluster
/etc/tpi.cfg
is a 1.0.1 thing
j
Jupp, but as there is barely docs on how that works from the BMC OS, it is currently and automotive chip, without HDMI, sound and other the bells and wistles.
d
And I do not want to implement it this way.
j
You are right, but the way it is implemented in for tpi.cfg it is not a good idea IMHO,
d
Well, I've heard this argument so many times already, and still don't know what does this mean. Sorry
I know, because there are as many ways to do one thing as many people you ask
The fact you don't like it does not make it bad automatically
j
eg. How is the Switch ASIC connected to the T113-S3 How the T113-S3 is connected to the Pi sockets.
d
Via I2C Via UART and LAN (through the switch)
j
Cool if you know this, which one of the I2C busses?
There are of course many roads to Rome. Doing the same thing twice and by the latter one by working against the framework(ifupdown) just feels bad from many perspectives. eg. reliability and understandability
I mean on theis page (https://help.turingpi.com/hc/en-us/articles/8685766680477-Specifications-and-I-O-Ports) you find many high level tech specs. Unfortunaltly I did not find more descriptive docs, eg which lanes are connected where.
d
My initial idea for the MAC and other settings is that the bmc process (or some other one) is going to read the config file during the boot procedure and compare various system setting and config file against this central setting repository. If any setting is different, it'll update it accordingly. Speaking of the MAC, for example, if it's set in a config file to be random and static, a single MAC should be generated and saved next to the common config file on the SD card. In case of a static MAC set by the user, this one will be used instead (from the config file). Then it's going to be set in the
/etc/network/interfaces
as
hwaddr ether this_given_mac
and on subsequent boots if this entry is missing or different than what's saved aside, it'll be updated. This means the
/etc/network/interfaces
will always contain the right MAC. the advantage of this is when a board break, this can happen, or for whatever reason it has to be swapped, you only move the SD card to a new board and it's being automatically set to the same settings. If you happen to change anything controlled by this config file later, it'll be changed accordingly and assure the change will be persistent by checking everything on boot. At elast this is the way I see this.
Because not much is released and the Team is working on the entirely new firmware, they recently hired a dev for this. This is only an alpha firmware made only for the boards to work and my fork is only meant to be a thing until the new official firmware is out. If people choose to use it anyway I'll continue my work here. Speaking of hardware, it might be opened in the future fully or partially, this is not known yet
j
That makes sense. my IMHO of MAC of Hardware NICs addresses, they should not change. Normally they are persisted in a kind of EEPROM or anyother close-to-the-hardware storage. Thats why I choose the UBoot env, too. If a user wants to change it later, this is persisted in the operating systems config storage. The can be the OneConfigFile or any other solution.
d
I agree here, but his is something the manufacturer is doing, not a user. Manufacturer sets MACs and they're even closely related to the serial number. If you as a user set it, it should not be stored as persistent IMO (meaning to override or set what you are not supposed to override or set) and should use the user-assigned MAC range. At least this is how I understand and see this.
j
Yeah. IMHO I do not care about MACs, as far there are no collisions on the L2 ethernet segment and the choosen MACs are not from the reserved range. I would love it if somewhere in the EEPROMs or Flash ICs on the TuringPi a unique pre-deployed MAC address per TuringPi2 board could be found.
d
Oh that'd make things so much easier indeed
j
That is a nice trick, looking into this.
w
Nice! Will give it a whirl tonight
d
May I have a few suggestions? And I'm keen to learn what do you think about them. 1.
USB 2.0
->
USB_OTG
2.
TTY
-> `UART`(since this does not have to be
TTY
in my understanding) 3. Could USB Host/Device be replaced with
Host (switch) Device
for easier access? (switch) is the same or similar element to how you toggle the node power 4.
Server
(Section) - I'd call it
BMC
I have a few more, but first I want to find out what do you think about these suggestions
p
Sure thing, I'm not a frontend/UI/UX guy so no real opinions other than needs to be used with one hand while working 🤣. 1. 👍 2. 👍 It's basically 1/2 TTY, saves what you type when you press enter it submits the command to the BMC. UART is closer to what it actually is 3. 👍 Very possible, I'll see where I can sneak it in, maybe the header of Nodes 4. 👍
d
I mentioned some other things, but I'll wait for when we throw more functionality to the firmware since a bit more work will be needed and there's no reason to change a single thing twice
Otherwise I really like the interface. Nicely done
w
Ah, real tty emulation would be neat but ofc this is way better than nothing
p
Yea ran into the same thing a few time, had to stop myself from locking it into any one API. The most widely adopted firmware builds are official and ce so they are a good start! I'm just going to maintain different versions,
main
branch will always be CE/Official until they diverge.
w
I also see a Docker compose file there - bet most people will want Kubernetes manifests to deploy to their cluster - should be an easy fix to turn those 3 directories into config maps and add an ingress
d
Do you think people will want to deploy this interface to the cluster instead having it on the BMC?
p
Yea we are going to have to implement our own protocol, have a few decent ones and going to check out
ser2nets
. The maintainer of tty-share is thinking of porting to Rust/C: Discussion here: https://github.com/elisescu/tty-share/pull/68
w
The proxy. Idk I assume the BMC can't run Docker
d
Why would it have to?
p
That is just to patch the headers while not working on localhost. Until we update the BMC firmware to have the correct headers we need a way to develop frontends
d
Why not to take the interface files and put them i the https server on the BMC?
w
So wait do I need that proxy thing in production?
Or is it just for local development?
p
No proxy in production, only development
BMC API has text/plain and no CORS headers
w
I see ok
So I don't need it as a user
d
And that'll change since I want to also gzip the files, but this is a different story 😄
p
It will error in the browser console because of the headers but the new UI will recover. Once the api is patched then the proxy is no longer needed. We can let users configure CORs at some point, it's best practice to not allow cross-origin requests
w
This is sounding more and more like a good idea
Some kind of plugin architecture
No idea what the interface would be
Maybe just an idempotent shell script that ensures everything is set up
d
I think we just wrap things into the services that we start/stop on demand. But we'll see. Thinking alone won't turn the ideas into the real features 🙂
I want to handle this firmware right so this is why I put so much time into thinking and finding out solutions and what I like
w
I suppose a fully fledged API would end up as basically reimplementing Ansible
d
I don't know Ansible, to be honest. If we get close - let me know 😄
p
There are a lot of interesting paths, any one of the configuration engines would be fun to integrate. Ansible is a good one since it's relatively light(minus the new features). HCL is also another great option. There is the possibility of creating 'recipes" with https://www.packer.io/ and just baking everything into images. Then the BMC just has to manage flashing and shutdown/startup for the most part. We just manage a list of "recipes" and images https://linuxhit.com/build-a-raspberry-pi-image-packer-packer-builder-arm/ I was hoping to get a WebTTY service running on the BMC, that's my current short/long term goal. Then deploy the same service onto the nodes. Most likely going to use tty-share's protocol. That's my current path at least for management.
w
I'm just thinking out loud what kinds of things a plugin or add on would want to do. Run services, create files, modify files, so on and so forth
It quickly approaches the capabilities of a general purpose server configuration tool
So then you have a choice. Structure everything in which case you've reimplemented something like Ansible. Or go free form and everybody has some common free form tool like a shell script that ensures everything is set up
My instinct is the latter is more in the spirit of what we would want to do. YAGNI and all that
p
Had this running on the BMC but it was way to heavy: https://tty-share.com/ even after I stripped it down to just the protocol/wss. Then started porting it to libwebsocket and the original dev is also interested, might get a free as in beer WebTTY
w
The worst option of course would be to try to build one's own generic configuration management system
d
Well, and this is exactly what I may end up doing, at least to start
I would not call it the worst, because we do not have to use what's already developed and try to make it fit our purpose. I don't know yet, we'll see
p
It wouldn't be difficult to just prebake a few images with a configuration agent or let users build their own image. Then the focus is only on "applying" configurations. Let the infrastructure as code frameworks deal with the integrations
d
I have some ideas on how to make the BMC configure the nodes in a way that's easiest for users. This will involve some custom solution I think but would also not need any images crafted specifically for this purpose
p
I think it will be difficult without a standard image since UART/serial settings will be different on every unknown OS. like shell login etc
d
It'll work more like you put a standard image on the BMC's SD card, choose node to flash, choose settings and it'll do everything else for you, even install some services if we end up using some (for example for remove power down of teh whole cluster)
Well, you might not even have to put an image on the card, it can be downloaded for you, one of teh official ones
p
Sounds like a perfect fit for managing prebaked images, then the UI in the BMC just shows what's available on github/etc and pulls from there
d
And out task here is to support as many node and OS combinations as possible. If this firmware lifts off, we'll have the community finding ways and making PRs
Well, I have a lot of ideas of what to do and how to do that already. I want to add node flashing from the BMC. but this is all a lot of custom solutions that have to be developed
If we, however, decide we'd like to support some way of automating the deployments, we'll add this too
Sky is the limit 😉
p
For sure, I generally agree with what you are saying. Node flashing is a really good start. Then it's just a matter of patching the image or offering prebaked images. I think having a repository of configurations would be the best bet. It is a good separation of concerns. Also good place to store images and different use cases in general, doesn't even have to be TP related.
w
Cloud-init is how Ubuntu does this
I mean look if somebody can figure out how to route the flashing USB lanes to the BMC and mount it I'm sure they'll be very popular
Maybe there's a schematic somewhere that would help
d
I did already by reading the firmware
p
Might bake a few images next week just to have around. Cloud-init is a great utility that gets abused in the best possible ways.
w
Oh?
d
I have ideas how to implement flashing from the BMC as I mentioned 🙂
And for the BMC flashing I'm not going to pre-bake any images. I feel there's no need to since I already have some ideas how to configure them on the fly
Well, when I said I'm figuring things out to make the firmware in the right way, I really meant it 🙂
I even found possible buffer-overflows that can cause the bmc process to crash, so I'll fix that too
p
It's a BMC agnostic, just meaning having TPI installed and built for the appropriate architecture. I don't think the BMC is going to be able to do that on the fly
d
I don't want to modify the images in any way, this is what I meant. I don't think there's a need to do so because we have a BMC
w
You're a machine, man
Well, a machine whisperer
p
Let me clarifiy, this isn't for the BMC directly. It's just general use case images. For instance Ubuntu images on the Raspberry Pi imager do not come with
extra-raspi
package installed. Just have a "Turing Ubuntu 20.04" that has it pre-installed so that the sata controller works as expected. Or is the expectation you put in a storage medium and send literally any image and the BMC will just "figure it out"?
d
I've heard both that
extra-raspi
helps and does not help. But this is not the point here. I understand but, like I said, I also have some other ideas I want to try, like using the regular images and configuring them after installations with a some sort of packages. This will let us not have to host any images. I'll write more about this idea when I check it and prove it working or not and when I shape it in a share-able way
So, to make the firmware development happen I bought this:
You can recognize the chip there
Then there is this spot:
And it'll take on of these:
And then one of these:
The last image is the exact SPI Flash that's o the TPi2. I have 10 of them
So basically this is the standalone BMC that I can then hook up to the Waveshare/Nvidia DevBoards for when I need UART or something
This would let me do most of the development and keep it outside of the TPi2
Also Flash wear out won't be a problem
Oh, maybge you actually cannot read the cip name there:
p
Should put a doc together with your dev kit once it's going, it would help people with their shopping lists. I already picked up a few things @DhanOS (Daniel Kukiela) has been experimenting with. Maybe even get some affiliate links going
d
I put links in the chat a few times I think
Let me find them
The last link does not work anymore, but you have a model
Also for the SOP8 socket - it was hard to find exact dimensions and it's a bit too wide for MangoPi and I haven't figured it out yet. But i will
You can also solder the SPI FLash directly, but I want to have a way to swap them easily
I might just extend the pads with some wires and folder the socket a bit further away and make it stick with a bit of hot glue 😄
I'll let you know once I join all of the elements of this puzzle 🙂
Oh, right, I also bought these:
Just in case
Might bprove to be useful
I can de-solder the connector:
Put it flash and use some wires to connect both up
p
Added them to DEVKIT.md in the repo. Never heard of Allwinner up until the TP.
Going to polish the toggle style and add firmware uploads tomorrow, hope everyone enjoys their evening/weekend! Render with @DhanOS (Daniel Kukiela) changes: https://turing-pi-urbirx0n6-telluric.vercel.app/
d
As for
USB_OTG
I kind out thought to keep the
_
there so it's the same as on the board and in the official docs 🙂 OTG Host/Device - I understand it's in works and you moved it, but if this helps I thought of something like this:
w
I imagined two columns of radio buttons all linked
Mode horizontally, node vertically, exactly one of the 8 radio buttons can be selected
Idk maybe radio buttons are passé
So I installed it but nothing changed. Different URL/path?
Ah I don't have an SD card
The deploy script silently fails in this case
hmm, I fixed that but now the web UI isn't loading, console says: http://192.168.124.54/api/bmc?opt=get&type=other net::ERR_EMPTY_RESPONSE window
times out it seems
d
I personally like the switch. Not sure about the position yet, though
w
OK so the community edition does not have this commit in it https://github.com/PhearZero/zero-pi-2/commit/b843660e1a57e4cc64e6ab50c70a5fc5805fedfe
also the BMC webserver firmware never responds to an HTTP request if it doesn't recognize what it is that is being requested
d
It is going to have only part of it, if anything. TO fit the UI on the TPi2 we'll have to make it matching the firmware
And this is something to look at. Should return HTTP 404 (or other error accordingly to situation)
w
I wonder if the UI repo deserves its own thread
well, the UI generally
it does claim to be compatible with your CE firmware
d
When the development start, but i mean really start, there will be so many topics that could be own threads. The problem is I don't want to take over this place with my firmware and I'm thinking of a separate server for the firmware development
By server I mean Discord server, of course
I've never checked that so I don't know if it is or not
w
any idea what license all this stuff is in?
I don't see one specified in GitHub
d
Good question, I don't. The Team calls it open-source
w
I was going to send a PR to quickly return 401/404/whatever in this situation, but by far the most uncomplicated way to do that is if it's released under an open source license
otherwise (or if it's AGPL or similar) I need a special release for my employer to waive their copyright interest in my work
oh well, I faked out the 'other' request for now by hacking the UI code. If I knew anything about newfangled web dev I'd make it optional or something
p
The bmc webserver needs a lot of attention, didn't realize "other" wasn't in CE. Must have forked from a while ago. We can update CE to at least match official api. Then just mount a new action for anything specific to CE
Yea it's a good idea, going to polish it this evening and then probably do a big 1.0.0 ann in a thread tomorrow evening.
w
can I suggest changing the UI code so that one failed request doesn't block the whole UI loading?
like if the 'other' request fails, the UI should load and just have 'unknown' in the relevant fields
n
I know I’m a bit late to this but is there a clear indicator that a change hasn’t been applied yet? With the official firmware, I’ll occasionally toggle a power switch, forget to apply it, and spend a minute trying to figure out why the node isn’t up. I would probably still do that occasionally if there was an indicator but I would figure it out sooner.
c
Hey @DhanOS (Daniel Kukiela) -- figured I'd loop you in since you're maintaining the big fork in the community. I'm hoping to get the structure of the (newer official) firmware repository cleaned up, but since that repository is still tracking the legacy/official one, I suspect that this reorganization may land upstream too, in which case it will probably then get pulled back downstream into your fork (this is such a hot mess I swear)... Anyway, since you might be impacted and likely have opinions about how this stuff is organized, I'd love to get your perspective on this:
j
👍 I like this, you basically are moving stuff that is externally "glued" into buildroot via the set of scripts into the buildroot directory 🙂 I did that partially, too for fun.
c
Thank you for your feedback! Yeah, the goal is that and to make the build process easier. If I'm about to start hacking on the devicetree and kernel drivers, I need to be able to just run
make
-> reflash and not have problems from, say, forgetting to copy over the .dts
j
for fast turn around time I created a flash.sh for my self https://github.com/j0ju/turing-pi/blob/main/flash.sh, but I guess yo uhave your own tooling already
c
This looks faster than what I'm planning on doing. Though, this script looks like it's meant for updating rootfs only (.swu) and not the kernel/dtb?
j
jupp, the generated .swu only contains the rootfs. Kernel upgrading is only possible by flashing the .img via LiveSuit/Phoenix Suit
c
My plan for kernel is to build as a uImage (not Android bootimg, no idea why it uses that currently) that bundles kernel+dtb, then I upload that to microSD, reboot into U-Boot, and use fatload to boot my test kernel.
j
I currently have kernel running that I am fine with. And I did not find yet the way how to do it the same way .
c
That way if I screw something up I just reboot.
j
after
sunxi_card0_probe
the mmc subsystem works fine in the supplied uboot. I tried with kexec to start on demand from sdcard, but that is currently a mess of kernel stacktraces and I do not feel hard enough for that rabbit hole 😕
c
Have you been able to find the "do not autoboot" interrupt key/sequence in the shipped uboot, or is there not one at all? I've been resorting to just removing the
boot_normal
from
bootcmd
, but wasn't sure if you've cracked that particular case.
j
see https://github.com/j0ju/turing-pi#troubleshooting
Copy code
Serial Console: if you cannot enter the serial console of UBoot although a bootdelay>0 is configured flash the .img once.
This suspicion was yesterday confirmed. The uboot envs are indentical between what I dumped from my untouched TuringPi2 vs that is generated out of the build from the vanilla 1.0.1 repo. At least the relevant code path/variables called by runcmd are identical. So it might be another difference. As far I understand the IMG also contains a UBoot
c
"Flash the .img once" meaning the shipped uboot does not respect
bootdelay>0
but .img updates include a uboot that resolves that issue?
j
yes, that is my current working hypothesis
c
I have a freshly-dumped mtd1 if you want to analyze it.
j
I tried to modify set differnt values or delete bootdelay, but it immediatly boots
mtd1 is the Uboot. I have a vanilla dump of that, here, too. Might be interesting if thre multiple versions shipped on different boards, but I will not reverse engineer it, if I have not to
c
Mine has sha256sum:
7e54140e013d66592dd8cd3b34ff5aaf6102df05d8f2d5d4a8d9836d653b742b
j
I used to dump
Copy code
#!/bin/sh

cat /proc/mtd > mtd.txt
while read mtd _ _ name; do
  case "$mtd" in
    mtd[0-9]: )
      mtd="${mtd%:}"
      name="${name#\"}"
      name="${name%\"}"
      dev="/dev/$mtd"ro
      f="$mtd-$name.bin.gz"
      gzip -1 -c < "$dev" > "$f"
      ;;
  esac
done < /proc/mtd

echo "name          size" > ubi.txt
for ubiname in /sys/class/ubi/ubi0_*/name; do
  [ -f "$ubiname" ] || \
    continue

  ubisize="${ubiname%name}/data_bytes"
  read name < "$ubiname"
  read size < "$ubisize"
  printf '%-13s %-d\n' $name $size >> ubi.txt

  ubi="${ubiname%/name}"
  ubi="${ubi##*/}"
  dev="/dev/$ubi"
  f="$ubi-$name.bin.gz"
  gzip -1 -c < "$dev" > "$f"
done
Copy code
3f35d43f10ef9e97a83433776f415350e8899557943ae093f17ebd9126958394  mtd0-boot0.bin.gz
cf6f61ed7b76d0d0628523a11d25afabf31d762111d83ccdd7ed38c24a9dbedc  mtd1-uboot.bin.gz
c1ecaad3d748ba9084d5b686a616fbaa29fb16953fd0eb314d8aeed0fac2fc77  mtd2-secure_storage.bin.gz
927e1a1bdb769278bc0c5ea9c712fe8cbc604c43345c0e7df1855370f1edbb6b  mtd3-sys.bin.gz
ed696d6ea86fc9d15b92cd123d7f3928bc630c082bbe2072bb4b7fe16f39b4af  mtd.txt
d8b2bf7517c044076b76dd6bb614748d7a0ee2277b91035b5d0bcf614bdd4794  ubi0_0-mbr.bin.gz
285557ad2cef6e131645144fcf2b4c2ef08c887e3224a04f43c4207350753f35  ubi0_1-boot-resource.bin.gz
8119bfe5d3b6a1c8a0ce24767d1150ec7fba87f93738a4cea799199b29b6a645  ubi0_2-env.bin.gz
a5a454cbeae2adaae7d8e4fe7d050ad8e42c5ec1af1fb1b8c372a6f70b5acf26  ubi0_3-env-redund.bin.gz
2d443bda03f69367c381a62e2c8c01459e414bca3cbbfe1f124433c288a4f01a  ubi0_4-boot.bin.gz
661699a360f8c882127e3a724e890bbe9f3a8073e8d2e305315abdb2c95ee2b6  ubi0_5-rootfs.bin.gz
1cfc89d37867e4aff2398c784bbac06e33d0fe87d4e6d568467f85ea769231d9  ubi0_6-recovery.bin.gz
43e99eb978f3aeb0e8e100ee626c05ecc777ec2953b9153a0112186e69ad6b70  ubi0_7-dsp0.bin.gz
98c03cfc5852a7caacabbb3dde1b162a120f340bc63797945c580cf62210bd8f  ubi0_8-private.bin.gz
feb473eec89d914f2dbbabc47ff2478d4680a4b8570a4888872c61b5245a2ce8  ubi0_9-UDISK.bin.gz
7b2e27d9149fd0430d7ec57b0dd5025cb8596c2b6c25589da6060366741cc0df  ubi.txt
Vanilla dump with 1.0.1 installed.
c
Mine is a factory dump (no .gz)
j
you are right ...
c
What's the output of
gunzip < mtd1-uboot.bin.gz | sha256sum -
?
j
7e54140e013d66592dd8cd3b34ff5aaf6102df05d8f2d5d4a8d9836d653b742b mtd1-uboot.bin And that means it is identical to your untouched rom contents.
after flashing the .img it is currently 8eae115254642bfe52e5b69572d8979ba9b5c72672a73ea5bbb171b2fc2b1c4c , so it is updated by the .img flashin procedure
If we could get one more confirmation for this hypothesis. This is a possbile way to brick your TPi and should be documented and fixed, e.g. If one would start developing without able to enter UBoot by serial interupt. What would happen if during weird bad update siutation ubi0_5 and ubi0_6 would end up in an reboot cycle without interrupt, The board would loop endlessly. As both known flashing/recovery methods need either Tpi2 in on the state of "linux booted", "adb/Phoenix-/LiveSuit-Update-Mode". In this kind of reboot cycle no interupt would be possible. --> BRICK To prevent this at least once the update procedure with Pheonix/LiveSuit should be done. Alternativly care fully updating mtd1:uboot via console with a known good version (UNTESTED!).
So this should also be a best practice documented somewhere for people starting developing or playing around with the bmc.
c
I'm not 100% clear on this but doesn't the T113 bootrom first check for a particularly-formatted microSD and, if it finds it, it grabs boot0(, u-boot, ...) from that?
So if you royally screw up your SPI-NAND, you can just pop in a microSD and get booted that way, then fix the NAND?
j
If thats the case that would be good. Otherwise, there would be no last resort in case of current boot ubi partition (ubi0_5 or ubi0_6) fucked up for any reasons with and vanilla UBoot.
c
Yeah, and I noticed no pin headers or anything for accessing the NAND, so even with modest tools I'd have no way of debricking (unless the microSD thing works)
j
Ack
Fun fact, the space in ubi0_6 is a litte smaller than in ubi0_5. For know I am unsure why. If you add so much packages or kernel modules that duing the post-build script the rootfs.ubi fits into ubi0_5 but not into ubi0_6. The process that prepares the .img, creates only 8 instead of 9 ubi volumes. And even the partition named "recovery" vanishes, if that resulting .img is flashed.
c
I need to learn about how the .img itself is made. The "tina-pack-tools" are kind of a mystery black box to me right now.
j
For me, too.
c
Could I trouble you to compute the SHA256 for this, but truncated to only the first
950272
bytes?
j
Sure. Is that the UBoot size in flash from beginning? You guess different random flash content in the tail end? 👀
c
I'm looking to see if this matches:
Copy code
turing-pi master $ sha256sum br2t113pro/board/100ask/dragon/u-boot.fex 
e70f8c152195000e5a6f4fd2149926e63f26a8801f301909818b102d17d190e8  br2t113pro/board/100ask/dragon/u-boot.fex
turing-pi master $ ls -l !$
-rwxr-xr-x 1 cfsworks cfsworks 950272 Apr 22 23:50 br2t113pro/board/100ask/dragon/u-boot.fex
j
isn't the .fex also prefixed with some kind of header?
c
One would think but it's copied verbatim into the .img:
Copy code
turing-pi/buildroot/output/images master $ dd if=buildroot_linux_nand_uart3.img bs=256 count=3712 skip=852 of=onlyuboot.fex
3712+0 records in
3712+0 records out
950272 bytes (950 kB, 928 KiB) copied, 0.00892286 s, 106 MB/s
turing-pi/buildroot/output/images master $ sha256sum onlyuboot.fex u-boot.fex 
e70f8c152195000e5a6f4fd2149926e63f26a8801f301909818b102d17d190e8  onlyuboot.fex
e70f8c152195000e5a6f4fd2149926e63f26a8801f301909818b102d17d190e8  u-boot.fex
I'm curious to know if I can just overwrite my /dev/mtd1 with u-boot.fex and whether that is sufficient for resolving the "it does not respect bootdelay" problem.
j
Copy code
~/src/build/turing-pi/bmc-release/turingpi-1.0.1.dump > dd count=1 bs=950272 if=mtd1-uboot.bin status=none | sha256sum
943e4a3eb9cfc624a6b63bbfd1fc0fd4a6ec0cecf357dfd2675f227f89cd9695  -
c
So, probably not a great idea to overwrite my /dev/mtd1 with that just yet
Ah, .img is a specific format for LiveSuit ("IMAGEWTY"), and .fexes are packages that are concatenated in there.
j
and in the mtd1 dump is a prefix "sunxi-package:...HEXGIBBERISH..."
found that while comparing uboot.fex from my last flashed build with the one in flash.
Maybe some hints for the allwinner bootrom?!? I am not that into these embedded arms before
c
Gotta say I am not a fan of prebuilt binaries in the repo... Wonder if my next cleanup step should be to get u-boot building rather than bundling a prebuilt one.
d
There's a recovery SD card image you can use to re-slash the BMC
Thank you. I will take a look. You created a pull request to an old official firmware which not likely is going to be changed/updated in this way. The Team is working on an entirely new firmware. So, I doubt I'll get affected, but I am going to take a look at that anyway and maybe use your pull request in the unofficial firmware. I have had a few days off and I'll finally shape the development of this firmware in a few days. I'll check and think if we can use your work here.
c
Slight clarification: the wenyi repository is the old one, this is the turing-machines repo which is hosting development for the redesigned firmware
d
I feel like I missed something (I was off for a few days) - where does it state this will be the new firmware?
c
(Also second paragraph of )
d
Ok, thank you. I did not see this repo and was unaware of it. It seems like the new firmware is going to be based on the current one and I even see that all of the "community" editions should be merged with this one, which puts a question mark on what we were going to do here
c
"we" being you and I?
d
I mean everything in this thread
c
I would guess "same thing, new location"?
Though fwiw I'm under the impression that the new firmware is going to be a sort of "Ship of Theseus" - it has started off being a fork of the current one but each part will be replaced iteratively, and by the time it launches it won't have much at all in common with the legacy firmware.
(What I mean by that is, whether this counts as the new one being "based on" the old one is as much of a question of perspective as whether the Ship of Theseus is still Theseus's ship)
d
I thought that the new firmware is going to be created from scratch and would take some time to even have a beta version. This meant we could improve this one with the most wanted features as it was always meant to be only a temporary unless the users will want to continue using it. If the changes are going to be more iterative, people will most likely stick with the official firmware and this is the question mark. We'll see 🙂
c
Ah - yeah I guess I have that same question mark then: "How 'usable' does Turing Machines intend to keep it while it's under construction?"
The userland might be redone from scratch, come to think of it, while the boot configuration, kernel, drivers, etc. are updated gradually.
t
as a mere user who might submit opinions or ideas, I too was surprised by this apparent change; I thought the community firmware was going to be a totally community production and the official Rust-based version would supercede it upon it's own completion
I guess it was an assumption
c
It does look like they intend to rip-and-replace all C with Rust
(Though, PR #2 makes me think the replacement will happen in stages.)
t
then I'm confused if the rip-and-replace part basically signals a hold on the community edition development while new API's are solidified?
in order to eliminate duplication of effort
e.g. faster transition to the official Rust-based version
or am I misunderstanding something?
c
I don't really know the plan myself either. I'm just a guy who stumbled on their official repo and started cleaning up around the edges and found that they're quite receptive to those changes.
t
I guess my questions are potentially rhetorical... 😉
c
It may be that the word "firmware" is kind of an overloaded term here, because it can refer either to the literal image on the NAND flash that boots up when you power up your board (including a bootloader, kernel, init system, ...), or it can refer to the "bmc" daemon that hosts a webserver (+ other interfaces???) and dispatches those requests -- i.e., strictly the userland.
And each time they've said "new BMC firmware" the features they talk about are more related to the latter than the former.
t
I guess I was thinking of the bmc daemon in this case
c
Ah yeah. I have no idea what the plan with the bmc daemon is. I see they're working on adding a Rust library that can be called from the existing C code, and the intent seems to be to implement flash-over-network support before they have the new Rust daemon yet (and they want to write the reflash code in Rust so they can reuse it in the new daemon).
Personally if I were doing this, I'd recode the entire
bmc
binary in Rust (there isn't much there, should take a weekend), put the current API in
api_legacy.rs
or similar, and then start working on
api_v1.rs
- but it seems there's a desire to do it differently.
t
I was curious if the prototypes/mockups of the web interface posted earlier might be implemented in the CE version sooner rather than later; the new repo signifies a delay to start merging them... at least from my perspective
d
I want to implement them as soon as possible
c
Haha, hence my remark that "this is such a hot mess I swear"
d
In a few days I'll write down all of the feature ideas and bugs as GH issues and shape v0.2.0 which I want to include a new GUI
I have just had a moment of doubt, but I think continuing what I started here is a good idea still and then we'll see where it gets us
t
and after that you'll reevaluate waiting until this is integrated into the new/official version?
c
Since CE is meant to be temporary, it might also be good to start thinking about what both versions (new/CE) need to have to facilitate easy migration onto the new one once it's ready
d
I won't be waiting for the official version of the firmware. At the end the it's the users who will tell us if our effort here is worth it 🙂
t
seems sensible, only issue is easy migration is hard to plan for if we don't know what we'll be migrating to
I meant after the to-do features on the CE GH are done
c
Well, one might be a request that the new firmware have (at least temporarily) a backwards-compatible implementation of the legacy API.
d
The CE is meant to be temporary unless people will want to continue using it and maybe helping out to make it better. The flashing process will always let people to flash the official one, I'm not going to break this.
c
I wouldn't expect it any other way - but simply being able to wipe and start over back with the official firmware isn't necessarily a "migration path." I was thinking something more along the lines of a mechanism to keep settings interchangeable.
d
I'm not sure what and where is going to be integrated into the official version. If anything. The CE will be living own life for now and then we'll see what's next. The CE version should not wait for the official version on anything since this is not the point of why it exists
t
agreed
d
This is a good idea and we might provide a way to migrate settings to the format that the official firmware will use, but for now we only can wait and see the development progress.
t
I guess I let my anxiety get the best of me
c
That, or we start thinking about a common format that both editions of the firmware can use. Perhaps something stored in the EEPROM so it's kept even across a full NAND reflash?
t
that suggests a common specification needs to be drawn up for use by both editions... 😉
d
What causes your anxiety here?
I'm not even sure if the CE is going to be recognized to the point the dev (or teh devs) will put any time into making anything common (since I read the CE should become a part of the new official firmware)
t
reading about a piece by piece transition, that the new one was still in planning, implied that the more work was done on the CE, the more would have to be reworked for integration into the official one... got my mind racing on if that would basically halt work on the CE
d
So far, I was against touching the EEPROM to not cause any troubles with the future official firmware. My idea was to put the root onto the SD card so the config file(s) could be transitioned across the boards and the BMC OS upgrade was thought more like you update the modern Linux-based systems (without a need of flashing of anything, just run an update command)
t
I support continuing on CE, as @DhanOS (Daniel Kukiela) stated...
d
> if that would basically halt work on the CE This is also kind of my fear
t
wants more pretty features and web UI ;-)
c
Well, I mean, using the EEPROM in the same format that the new firmware's design (eventually) specifies.
(If it does specify using the EEPROM.)
t
yeah, too many assumptions being made at this point
c
What I'm saying in general is we can watch the trajectory of both CE/new repositories with an eye toward making sure they remain "compatible," whatever that ends up meaning.
d
But this would also meant that the CE will always be behind the official firmware which is in the opposite what I want the CE to be)
c
Would it?
d
Hard to say for sure, but I guess so
Whichever planned feature will either need to wait for the official firmware or do things "partially" on it's own
c
I've seen firmware replacements for devices that use large EEPROMs to store config; the official firmware used a
key=value\0key=value\0
format, so the community firmware just invented its own keys for the features not available upstream.
That's not to say the TP2 firmware will use k=v, just that it isn't necessarily the case that sticking to the same format limits the CE.
d
We'll see, I guess
c
It sounds like most of the community wants an open process for the new firmware's development. I'm one of them -- would love that to be a "we'll strive for it" rather than "we'll see I guess" 🙂
d
What do you mean by "new firmware"?
c
"The thing that will be factory-shipped on all TP2 boards hopefully by the end of the year"
d
The "we'll see I guess" was strictly about the CE version following the official firmware EEPROM storage. I don't know about the official firmware, but the CE will always be open and community-driven as long as people will be interested in using it and working on it 🙂
c
Yeah - I'm just trying to encourage not taking a fully passive role about the new official firmware either. I don't know if the settings storage is finalized yet, but if not, maybe think about what you'd like for it to have so users can most easily switch back and forth between it and CE
(Also part of why I ran the reorganization commit by you is to keep CE and the new official firmware from diverging too much - you may end up periodically pulling "new firmware" features down into CE from time to time, and I want to make sure that this process remains smooth.)
d
The question is if your PR will be accepted 🙂
c
Sven's initial feedback makes me think it will be - perhaps with some slight modifications.
d
OK, more important question - when 🙂
c
But, still, if the official firmware is being restructured, and CE ends up needing to restructure itself in the same way, is that organized in a way you find to your taste?
d
These changes should be integrated probably before any other work to avoid having to re-do the features added by the commits
I will make a close look to your changes in a few days. I feel like I might have own ideas about some things, but then, if we want to keep it close to the official firmware, I might need to pull this PR anyway. You also seem to know much much more about it than I do.
c
Looking forward to it! (And if you do have your own ideas about these things, please do post about it on my PR. My whole reason for running it by you is so you don't end up blindsided by any "upstream" change you ultimately don't like.)
j
O cool I did know. Is this publicly available somewhere for fun?
d
c
(^ Earlier today I was thinking about how I would love to have this process be more like... 1. Download the .img 2.
dd
it onto a SD card you don't care about 3. Put it in the TP2 board, power it on, wait for the power LED to start steady blinking 4. Hold KEY1 for 5 seconds (power LED starts rapidly blinking) 5. Once the power LED is no longer rapidly blinking, remove the SD card and push the BMC reset switch. )
j
Oh, I never looked at version 1.0.0, thanx
d
Or you can have the root on the SD card and a fallback OS in the Flash (in case of the SD card issues) and you never have a need to flash anything again 🙂
c
Doesn't that sacrifice /mnt/sdcard?
d
Sacrifice in what meaning? What would be the purpose of it otherwise?
c
Storing persistent data that isn't overwritten by a firmware upgrade?
d
Why could not be these data stored there along with the OS which can by updated by a command (like in a modern Linux-based OS-es)?
c
...what is that command?
j
.... like buildroot ?
d
It does not exist yet, I haven't checked the possibilities of adopting any of the existing systems, yet
c
Well, you just answered your own question then.
d
No, I mean to deliver the compiled binaries, more like
apt
with ubuntu
Elaborate, please?
c
Q: Why could not be these data stored there along with the OS which can by updated by a command (like in a modern Linux-based OS-es)? A: [The command] does not exist yet
j
I like the appraoch. buildroot offers opkg integration. But I then would go more into the openwrt direction ...
d
I have a hard to to understand what do you mean still. The idea is to move the root onto the SD card and be able to update the OS by the command. As much as I'm against storing any data on the system disk, there's not much of a choice here, so why not use /var, or /etc (depending on a use case)? Am I missing something?
My ears are open if you want to say more. You both definitely know much more about that than I do
c
So, the idea is to use the microSD as the NAND is used today, and the NAND is there but 100% vestigial (only used in case of microSD failure, as you said)? The current firmware update process just overwrites the partitions blindly. This is nice and reliable (no chance of some stray file the user left behind conflicting with the upgrade, since all stray files are blown away) but you need to keep files you care about out of the rootfs. The current way of doing that is to keep them on a partition of the SD card instead -- but we don't have partitions anymore because boot0 is now sitting where the partition table "should" be, to make the microSD bootable. So now it sounds like to solve this new problem, your solution is to update the rootfs with a package manager rather than overwriting it every time. But this is Buildroot, which is focused on building one-off images, not packages.
j
openwrt is destined for devices of this class. It offers switch, network, and a good set of packages. It has also a firmware building environment intergrated already with only open source tools. It has already a webinterface (which also needs no binary blob). For now just ideas.
c
I suppose we could reverse the roles of microSD and NAND: erase the NAND and put a jffs2 on it, use that for persistent storage, then swap out the microSD if it fails.
j
you could write a uboot bootcmd for this, 1st try to init sunxi_card0_probe and fatload a zimage or execute a uboot.script from sdcard.
d
The CE version is not even using the SD card for anything (yet). It's based on 1.0.0 and 1.0.1 indeed uses the SD card for this simple config file, which is more of a temporary solution IMO. This does not prevent us from making thngs a bit differently.
c
It would be great to figure out how to get a partition on the NAND that can survive whole-system upgrades. Part of that would entail not using LiveSuit for whole-system upgrades. 😂
d
Well, these are at least my ideas. I not necessarily have all the answers yet.
Well, this is why I would see it the way that you only need to flash the Flash the last time and then all of the updates would be made on the SD card only
c
We shouldn't assume the user wants to have a microSD sitting in their board 24/7 though.
d
I made a pool and 100% of the votes were on this idea
Not many votes, though, but 100% of the interested users
c
...any chance the users who don't want to put in a microSD card are also the uninterested users? 😛
d
I don't see why people would need to swap the SD card and for wat
c
I don't know. I'm not the user.
d
I made it clear and among other options one was only Flash and one was Flash + SD card to have more space for the binaries, and 100% was on the latter. And uninterested people won't run CE anyway
c
I mean, not interested in voting in the poll.
You're drawing a total generalization from a pretty small sample size, which is one of the cardinal sins of statistics. The most you can really say is that there are users that do want it, not that all (or even most) want it.
d
Well, they have had a change to voice out their use-case. There are always people who see things differently and would like to have a different set of features
c
Like, the option to boot CE off of a microSD only is actually a great idea.
But if it becomes "CE assumes you have a microSD sitting in there the whole time it's up, and you can't use it for any other purpose" that may turn into a drawback for some
d
I'm trying to create the CE version of the firmware and I'm listening to the interested people. If these uninterested did not bother to vote, how would we know what they want. I'm not generalizing, I'm trying to star the the CE version of the firmware and try to listen to these who have something to say and want to use the CE. I can also guess what people might want or give them the options, but in some cases there's not a lot that can be done and some decisions have to be made
You can't eat the cake and have the cake (or however it was). I've been thinking on the "lite" version of the firmware containing just most important features that can fit into the Flash and the fully-features one with the root on the SD card. It's not like I don't have alternatives. I just presented to you one of my ideas, not all of them (and I have a bunch)
c
I'm just saying, it wouldn't be good to remove the option of writing CE to the NAND.
Hence "We shouldn't assume the user wants to have a microSD sitting in their board 24/7 though."
d
The thing is all the requested features won't fit into the flash at some point.
c
Ah, then having a "lite" build and a "full" build sounds like the way to resolve that.
d
And it might indeed be like - if you want to use another CE version, you need an SD card, which, to be honest, is not a problematic or expensive requirement. But I don't have the final answers yet and like I said you cannot fit all
Then this raises another questions - what to retain into the lite version, since whomever you ask will have own ideas or set of "required" features
c
I suspect that this is probably something that follows the Pareto rule: 80% of the usecases can be achieved with 20% of the storage space