it is after this command and reboot?....

grep . /sys/class/mtd/mtd*/ecc_failures /sys/class/mtd/mtd*/corrected_bits /sys/class/mtd/mtd*/bad_blocks
cat /sys/class/ubi/ubi0/bad_peb_count
dmesg | head -5

please this

sounds like broken flash

This was after ubivolrm and reboot https://cdn.discordapp.com/attachments/1179360051442487427/1557170526617870408/image.png?backend=b2&ex=6ac6d390&is=6ac58210&hm=8425b0d2b1a16709aee9906eec49ececadce6074b2e76a948616584a303f9860&

Thank you, digging

AI agent is telling that everythign is fine

Good news, that output says the opposite of broken flash: your settings volume is back (/mnt/overlay is ubifs again, no more /tmp/safemode), and the flash's own counters are clean, 0 ECC failures and 1 bad block, which is normal. So it was one damaged write, most likely a power cut during a write at some point, not dying hardware.

Last step: set your password now, check ls /mnt/overlay/upper/etc/shadow exists, then reboot once. If it doesn't ask for the password reset again, you're done. Thanks a lot for the patience and the outputs, this found a real gap: the board falls back to "factory mode in RAM" without telling you, and I'll make it say so.

overlay came back, yeap!

now should be fine

@dingodogz sounds like - everything should be fine now. I am updating web-site with article about that

Thank you so much for all you have and continue to do. Can confirm shadow file in overlay was set and reboot password and cert was fine 🎉

Amazing!

@dingodogz documented your case - https://turingpi.xyz/guides/recover-a-bad-flash/

also added there - https://turingpi.xyz/reference/known-faults/

@dingodogz thank you for help to debug and solve!

honestly, geography confuses me a lot. I checked by some way, and Netherlands is NOT mine requests, somehow Dutch people interesting in custom firmware for Turing Pi like A LOT https://cdn.discordapp.com/attachments/1179360051442487427/1557176034384941206/image.png?backend=b2&ex=6ac6d8b1&is=6ac58731&hm=9e52a5b8928c1f98a34ebb0a38f89941be7eebd17354644cb2eafd221a0e8923&

Also Finland is puzzling me 🙂

this part - no surprizes https://cdn.discordapp.com/attachments/1179360051442487427/1557176372080803850/image.png?backend=b2&ex=6ac6d902&is=6ac58782&hm=6cd339d5860017b8ff26e89312a67088cce045aba29de4fbae8827e0be1fc122&

Oh, to better view visits instead of page requests https://cdn.discordapp.com/attachments/1179360051442487427/1557176950173204541/image.png?backend=b2&ex=6ac6d98b&is=6ac5880b&hm=a032afbfb1d815b58a033058f3d3c1d34820f685aeb35092748647331eccc379&

Ahhhh! I figure out :)))))

There are visits from RUSSIA! VPN!

Russia is blocking internet, and ordinary russian user is user of Dutch VPN 😄

Question on this firmware "upgrade"... Can you upgrade the BMC firmware to this without powering down the box? ie, my homelab is operating 24/7 at the moment doing some long processing. I do want/need the vlan capabilities but can't do anything at the moment at the cost of "rebooting" my RK1 cluster.

Upgrade from Stock firmware to this version impossible without downtime After that each update do NOT touch boxes (they continue to work) and you face only 30-40 seconds network switch downtime during BMC reboot (impossible to fix)

@cheshire.the.cat update once with downtime to my firmware => after that each next update will be without power off nodes

I'll have to pick up another board on ebay or something. Cant power this off for awhile until some of the workload is offset from it.

Report about any issues with firmware, please!

@excavador thanks for this bmc FW. Just upgraded my 2.5.2 board few minutes ago, process was seamless. New features are really great. Thanks again for the good work 😉

You are welcome, and please report bugs or feature requests!