Reading NANDS/48 TSOP - aka the great firmware escape

[darkspr1te]

Hi All,
 So recently i decided to dive into my dead device bin and see what i could play with. The two items that stood out were two $800 tri band routers/AP's both with firmware malfunctions that stop the devices from operating.


My plan of action is to somehow fix the firmware issues both have so I started first with what stores the firmware.
Both device have the same BCM947xx of BCM47xx depending of what document you read, both have jtag ports (i'll start another thread for that road ) & both have same Spansion S34ML01G1 128Mbit TSOP Nands


So I grabbed my hot air gun and proceeded to remove both NANDs and attach them to a couple of dev boards (Namely two VCCGND designed stm32f407zg dev boards)
https://github.com/mcauser/VCC_GND_F407ZG


Find basic nand code on the net is fairly easy , both in arduino format or plain c so i set to dumping my nands.


This is where simple gets thrown out of the window


Initially both dumps contained lots of zero's in random places, I assumed at first it was a assembly issue , code issue but no amount of resoldering or code changes seem to have fixed the data read's using that code base (remember, I have no idea what I am doing here) , how ever all my code can read the ID's (8 bit using ID7-ID0) which is sent in 8bits correctly confirming my soldering is good at least.


So I decided to do more research in to EEC/Spare area's and BBT which got really complicated real fast.


Then I stumbled across NANDO flash reader project, it uses a stm32f103 with FSMC to read nands and being stm32 based it should be easy to port to f407 right? right ? wrong lol
first off was the code base was based on  stm's PRE-Hal &  PRE -LL- libs so almost none of the h/w code would work my what am using currently (stm HAL and LL libs)  so i set out to update it first to HAL/LL and then port to stm32f407 chips I have on hand.


After giving myself a much unneeded bald patch I finally got to a point where the code base can compile, upload and read a NAND


https://github.com/darkspr1te/nand_flasher
(go easy on my coding skills as i am only a hobby programmer)
So progress right ? I can read the nand, read the spare area so we done ?
Well sadly no, you see despite starting a complete new code base , rewriting all the reading routines and such I have still got a batch of random zeros in my dump (remember two different nands from two different hardware but both have the zero's issue )




I have attached the dumps below and provided the source code am currently using, Am wondering if anyone out there could give me a hint on how i can add the Error correction code to dump the nands correctly
I am studying the ASUSWRT-Merlin code base for the CFE (bootloader)  and main kernel drivers for the NAND/MTD controls to see if there is some trick, a example would be is that some nands store spare area at the end, some at the start  plus a whole bunch of other factors am still getting my head around.
but the main thing is why is there always 144 bytes worth of random numbers in my dump ( i say random but it's normally 0x30 or 0x31 which displays as zero)  and this 144 bytes of data is embedded every 1904 bytes , initially just changed the code to ignore this info and not dump it but i feel that maybe i'am missing something here


Here are the two nand dumps , one from asus rt-5300 and the other from Netgear Nighthawk


https://mega.nz/file/6YBCQBwJ#IeaDogYxYWdX2quDbTIy7ndJqZadEIX3vf1iG7d59CI


Before anyone says it, am not worried about stored passwords in these files, both networks they belonged too dont exist anymore nor do the owners,
So am wondering if there is anyone out there that could clue me into what that data is, how do i use spare area to correct dumped data ?


I know it's a long shot and even if nothing comes of this thread i will still continue to dig into it, maybe the info or code could help someone in the future .




Darkspr1te

[darkspr1te]

*Update*
I have fixed the nand read issue, i added a delay in the read page cylce and this gave the nand enough time for switch over and to put the data on the bus.


Next i will undo all of the silly patches fixes i put in to solve the issue , will push the final to github soon, then will add spi and then hopefully it's done for this part for now


:-)




darkspr1te

[amyk]

*Update*
I have fixed the nand read issue, i added a delay in the read page cylce and this gave the nand enough time for switch over and to put the data on the bus.

Async NAND, like a lot of other parts has no minimum frequency. If you suspect timing/signal integrity issues, always slow down as much as you can, even to "human" speeds.

[darkspr1te]

Yeah, I also was thinking that but it seems there is a small turn around time from the last byte sent to the first bytes read. I've now swapped the 1 second delay for a few _NOP's and this has added the much needed turn around time without making it too long


Next is to figure out where the errors are on original nand and see what i can do to recover the firmware. I have a couple of nands laying around so I do have a working replacement for the device. I want to see if i can all the repair work done before i have to start re-soldering nands.




darkspr1te
 

[AVI-crak]

pin7 (R/B#)  Ready Busy. The Ready/Busy output is an Open Drain pin that signals the state of the memory.

This pin will be pulled to ground during the write and read operation, until the transients are completed.

[abyrvalg]

Don’t rely on fixed delays in this case, use R/nB as AVI-crack suggests instead. This delay is not just some “bus turnaround”, during it the entire page data gets transferred from the array to the page buffer, it can even vary from page to page on some NAND types. The same signal also indicates write/erase busy (if you decide to write something back).
Long time ago I’ve built a Cypress FX2-based NAND programmer, designing a GPIF waveform matching the NAND bus. The throughput was great over USB HS.

[DavidAlfa]

The nand needs some time to read a page from the array into the buffer, so first slow it down to 100KHz max, use R/B state, if manually toggling pins ensure to insert a 10us delay between any changes.
When you get good data all the time, try speeding it up.
Few months  ago I also read a parallel nand from a bricked router with a stm32f103 (And programmed a new FW), the code was pretty simple, nand interface is easy.
Will check if I still have the code somewhere.

[darkspr1te]

Yes , pin 7 R/B goes to the MCU's pin PG6 but alas the code just hangs(it is set correctly in the code AFAIK, Input/pull up , also tried other combos like pull down and hi-z ) , I will delve into that now i have other parts working like correct reading and writing.
So far non of the code that i have tested that uses R/B (labeled FSMC_NAND_PCC_WAIT_FEATURE_ENABLE ) has worked, it always hangs at the read state, without 100% knowing what i am doing I do a lot of guess work so disabling the PCC wait feature seemed the obvious choice, now i have to track down why it does not work




darkspr1te

[DavidAlfa]

In the end I was wrong, I used a 32F429 and the FMC!

I recall having this issue, I think it was a bad connection, a bad FMC config... Or maybe too fast FMC clock? I can't remember...

Found the project folder (Attached) but I don't know if it was the final version or a testing thing.
This code was meant to revive a router, quickly made in a Sunday, nothing profesional, it just read a file from the USB and write it to the nand (Or read the nand to a file).
Just check the nand organization is correctly set as per the datasheet (number of pages/planes/dies).

[AVI-crak]

it always hangs at the read state

The memory is waiting for the address to be set, or the address is set above the available limit.
You can check the status of the memory interface by reading the memory configuration registers, and the unique chip number.

[darkspr1te]

Quote from: DavidAlfa on Yesterday at 05:22:01 pm

In the end I was wrong, I used a 32F429 and the FMC!

I recall having this issue, I think it was a bad connection, a bad FMC config... Or maybe too fast FMC clock? I can't remember...

Found the project folder (Attached) but I don't know if it was the final version or a testing thing.
This code was meant to revive a router, quickly made in a Sunday, nothing profesional, it just read a file from the USB and write it to the nand (Or read the nand to a file).
Just check the nand organization is correctly set as per the datasheet (number of pages/planes/dies).

Thanks for the sample code, i will compare to my setup and see if i've missed something obvious, I have enabled the H/w wait state now and yet i still "sometimes" get the 0x30/0x31 errors, so am thinking i need to slow things down to get reliability up ,
sadly my board does not have USB host wired up so i cant use your code directly but i might just swap the USB for sd-card reading code instead.
I will have time to play around tonight , I've made great progress so far but there some niggles i need to fix, then I will be happy and move onto next stage


darkspr1te

[abyrvalg]

You’ve mentioned JTAG at the beginning, have you tried that way? Any problems there? Because raw NAND way can be really long - bring up reading, correct bit errors (either by understanding the ECC or by reading several devices with the same version and majority-deciding. An 1Gbit NAND would contain many correctable errors), understand page/block layout (there can be several zones with different layouts), etc. In contrary, if you manage to load u-boot to RAM via JTAG, it will do most of the low level jobs for you.

[darkspr1te]

Originally when the device's failed I did try JTAG and sadly at that time (2020 ish) I could only get basic JATG up, that is ID code, basic boundary scan . I could not achieve cpu halt let alone be able to upload any code and docs leaks were thin on the ground.
Also at the time it was thought the CFE was encrypted as normally at 0x400 would be text for the basic board parameters but on these boards was what later to be found a custom LZMA compression , also the CFE bootloader itself was LZMA compressed and the decrypt routine I "think" was signed.
However while learning how to figure out the NAND i re-delve into the JTAG side as I think the just dump nand, flash new f/w is not as easy as my initial thinking was , This does not mean that working with the nand is a waste of time as my works have kinda created a man in the middle board , my two target *dead* boards both have full JTAG, including TRST which means i can alter my code to reset loop the cpu while i write new data to the nand and after release TSRT and allow the cpu to boot ,
This in turn will allow me to also work on the jtag side as well and hopefully produce some fruit like hairydairy jtag or my tool for Quallcom cpus via jtag.


but some of the key things this works has produced are
1. and actual full dump of the Asus ac5300 nand which no one has available or has not made publicly available.
2. Another nand reader for stm32s but using current code bases - might be handy for someone down the line  as having a dozen build environments sucks
3. A mad idea to combine this nand reader with jtag via the stm ability to dual wield USB com ports and turn this board into a tool to reverse engineer broadcom cpus (seeing what addresses are read during bootup can give a great insight into code you cannot directly control yet)


As for the speeds of reading the nand, i've been working on the code and i have a full 128Mb dump in 12mins, 14 with spare area , this is fine as when i get further down the line i will be only reading/writing small blocks , so to write just the cfe portion it's 3seconds from button click to complete.
I will prob spend this afternoon soldering some transformer wire between the target board and the nand board , I can micro solder but it wont be a cakewalk though.


Initially i will have to reprogram the stm to be a logic probe reading the address lines and spitting that out via VCP but I already wrote code for that and it works great  as it stores to sdcard/spi flash or eeprom first then when commanded spits it out via the slower vcp , I used this method to hack a STM based OBD reader that used nand for the code lookup db and text ,  once i knew what addresses it read on the nand i was able to replace that data in the update file to cause a overflow and run my code which dumped the flash to com port and from there i grabbed the encryption code for the full firmware.
My hope is a similar trick might work here. 




Yes I put lot of theories up there, we shall see what i can make work.
But compared to my initial work a few years ago which was weeks of compiling different jtag tools ,be it front ends for my jtag device or back end software like urjtag and openocd and such. that was a nightmare of libraries , compile environments and such and back then on my slow internet it ment night after night of downloads only to find it dont work at the end.
This route has produced a little more fruit so to speak


As for your mention of ecc , Thankfully there is a tool out for broamcom nand dumps that uses the actual code from kernel and so long as you have the spare area it 'should' spit out a good file. it's hidden deep in some chinese hacking website but somewhere i have a copy of the source code how ever it needs mods as it ment to work with dumps made by a program called Broadcom studio and the read is done via a SPI to cpu to nand link, but i have some sample dumps to work with so i have worked out most of the format of the file once before, I will have to re-visit that now too i guess.


Lol, all this just to empty out some e-waste from my closet.


darkspr1te