FAT32 getting started

[Jan Audio]

Hi, i like to write a file to a FAT32 SDcard.
To make it simpler i format the card in windows with FAT32.
Now all i want is to write to 1 file in the root.

What do i have to do ?
btw : i am using a 8-bit PIC.
The read/write to SD card is functioning.
I also need to download all the librarys that exist for example, so far i have : sdfat.

thanks

[ucanel]

I wrote an sd card library for dspic30f,
fat16 and fat32 supported,
wrote it in assembler, library size was 1k.
I do not recommend you to write your own library
it takes unnecessaryly long time to understand and implement it.

But it is easy to communicate with an sd card
if we do not think about Fat.
So if it is suitable for you there is an workaround about Fat:
You will create a space filled long  txt file with a pc then
using winhex Hxd or similar
you will learn the start address of this  empty file
and at the mcu side
using this address you can easly write and read this file.

There are so many tutorials about physically reaching to sd card.
As a hint you will use SD SPI mode,
and there is SD Card initialisation sequence
after that there is Sd read and Sd write commands you will need.
ELM Chan's site is a very good starting point.
So many mcu manufacturer uses his FatFS library.
http://elm-chan.org/docs/mmc/mmc_e.html

http://elm-chan.org/docs/mmc/i/sdinit.png


And there is one more hint
sd cards has at least  4 or 5 different types
it is hard to support all of them with just one small library.

[Jan Audio]

Yes, i support V1 and V2 SD cards, i focus on SD-HC only.
The SD card is already initialized, thank you.

And yes i am going to write my own FAT32, with only 1 file to write.

[methods]

A bit off topic but my experience has been that the Arduino libraries handle this beautifully.

I have wired the SPI lines directly to a MicroSD card slot and data logged gigs as easy as writing the terminal.

I strongly encourage all engineers to support the Arduino paradigm.  I got my start with pic, specifically the 16f84 and later the 16F877.  This is back when we did everything and pick asm leveraging the true RISC.

Today I find myself struggling to get an stm32 to even do the most basic of tasks. . .  where for the last decade I've used arduino's to great success.

down selection of a microcontroller should be based on how well supported it is by a community. 

...  Unless it's part of a learning program were you actually wanted to be difficult.

-methods

[Jan Audio]

Yes i downloaded that code also.
The problem is reading the bootsector "bytes per sector" gives zero.
What can i do ?, is this normal ?

[Jan Audio]

...  Unless it's part of a learning program were you actually wanted to be difficult.

I want it to be simple and small, for use with small 8 bitter.
Most important : understandable for myself.

[ucanel]

If i remember correctly bytes per sector is 512 bytes standart for FAT32.
I will check my notes and write it again.

[PlainName]

down selection of a microcontroller should be based on how well supported it is by a community

Having community support is a useful plus, but surely the selection of a micro (indeed, anything) should be on whether it will do the job better than something else. The availability of a ton of code written by someone else is just one factor, and not necessarily the most important.

[Jan Audio]

I attatched a 20x4 LCD, and write hex adress and hex data values 4 in 1 row like this : 00000000 00 00 00 00

If i read the bootsector ( 512 bytes ), it has only some values at the end, it has the 0x55 and 0xAA, and before that some more.
The rest all zeros.

The size in use after formatting is 2K, i overwritten 4 blocks and the card is still working in windows.
btw : i dont know if it matters, i format it with the SD to USB converter from my 3D-printer.

It is formatted FAT32, if i look in those library codes, it gives "invalid card" if the "bytes per sector" reads zero.

[Nominal Animal]

Apologies for inserting Linux in this discussion, but if one just writes data to the SD card sequentially without any file system, it is rather easy to read in Linux: you can treat the block device as a file, and read it directly.  All you need to do is make sure you have the necessary privileges to do so.  (I can show a privileged "opener" written in C, though, that simply verifies the first 512 bytes have an acceptable fingerprint, and if so, pass the descriptor (file handle) back to a normal, unprivileged program, so they can treat the entire device as a single file.  This is safe, and only requires that you leave the first write block intact, containing a fingerprint that does not match any existing filesystem or partition table structure.  (I recommend some initial and final bytes, plus the rest as ASCII text explaining the format.)  It can be used in conjunction with e.g. Graphical User Interface written in Python3 and Qt5, for portability.)

Unfortunately, I do not know of any way to do the equivalent in Windows.

For Linux users, it means you do not need the overhead of any file system, as long as you have a way to determine the beginning and ending of data.  One method I have used, is to have a 64-bit monotonically increasing generation counter in every block (whatever block size you use to write to the device), and use the blocks on the device in sequential order.  In general, it does mean you need to "initialize" the device to all zeroes, unless it is already sufficiently initialized (all counters zeros – it can be easily, albeit slowly, verified).

On a device with N blocks, you then only need to read 2 + log₂ N or so blocks, to find the most recent or oldest block.  On a 8 GB SD card with 512 byte sectors/blocks, or 16,777,216 sectors, this means you only need to read 26 sectors to find the oldest or newest block; and when oldest and newest blocks are known, all other blocks are directly accessible without searching.  So, it is quite fast and efficient, too.  All this is easily written in library form – especially if you have a fixed structure for each block, say containing fixed-size records, or variable-sized records that do not cross a block boundary –, much easier than any filesystem driver.

[magic]

You optimally "initialize" an SD card by erasing it with a special command, though I'm not sure if it's guaranteed whether the result will be all 0x00 or all 0xff or perhaps something else. It tends to be all 0x00 IME.

[Jan Audio]

I need those "bytes per sector".

[Nominal Animal]

You optimally "initialize" an SD card by erasing it with a special command, though I'm not sure if it's guaranteed whether the result will be all 0x00 or all 0xff or perhaps something else. It tends to be all 0x00 IME.

Yeah, CMD38, but issuing it is nontrivial. All zeros or all ones works fine, as long as both 0 and 2⁶⁴-1 are considered invalid generation numbers.  A mix (0xFF00..) is technically not allowed, but one can fiddle a bit with the generation counter to avoid any of the eight bytes ever having zero or one.  It makes the math a bit slower on 8-bit MCUs, though.  So, just writing all zeros to the SD card is easier, but takes a while.  One can optimize that a bit by only clearing sectors whose eight first bytes are nonzero or non-2⁶⁴-1.

I need those "bytes per sector".

What do you mean?  Using a generation counter per block written is at most 1.5625% overhead (8 bytes out of every 512 byte sector; 0.2% at 8 bytes per 4096 byte block), which is definitely less than using any filesystem.

[Jan Audio]

What do you mean?  Using a generation counter per block written is at most 1.5625% overhead (8 bytes out of every 512 byte sector; 0.2% at 8 bytes per 4096 byte block), which is definitely less than using any filesystem.

Talk about What do you mean?, now you got me confused.
Should i ignore the bytespersector and use 512 ?
Should i ignore the whole bootsector and only validate 0x55 and 0xAA ?
I dont think it works.

[Nominal Animal]

Talk about What do you mean?, now you got me confused.

Since you didn't quote the message you answered, I assumed you were responding to me.

I now see you were continuing your previous post, i.e.

I attatched a 20x4 LCD, and write hex adress and hex data values 4 in 1 row like this : 00000000 00 00 00 00

If i read the bootsector ( 512 bytes ), it has only some values at the end, it has the 0x55 and 0xAA, and before that some more.
The rest all zeros.

The size in use after formatting is 2K, i overwritten 4 blocks and the card is still working in windows.
btw : i dont know if it matters, i format it with the SD to USB converter from my 3D-printer.

It is formatted FAT32, if i look in those library codes, it gives "invalid card" if the "bytes per sector" reads zero.

I need those "bytes per sector".

There are two layers here.  One is the physical layer at which the SecureDigital card works.  The other is the filesystem layer, FAT32 in your case.

At the hardware level, when writing to a SD cards, you write in 512-byte aligned blocks.  This is often called a physical sector, because back when storage media had spinning disks, the smallest unit they could read and write was a physical sector (or more properly, a circular arc on the rotating media), usually also 512 bytes.  (Nowadays hard drives with 4096 byte sectors are common, though; and on solid-state media like Flash, things are more complicated.)

At the filesystem level, FAT32 specifies a logical sector size.  This is is a power of two, and at least 512.
FAT32 also specifies a cluster size.  This is a power of two multiple of the logical sector size.

The typical logical sector sizes are 512 and 4096.  The cluster size varies, but 32768 is most common.  You can set either or both when formatting the filesystem, or let the utility implement some heuristic.  Either way, these two numbers are described in the file system boot sector.

I don't know about Windows tools, but in Linux, if I want to find them out, I connect the media (via an USB card reader for SD cards), and then run sudo umount /dev/partition && sudo fsck.ext -nv /dev/partition, which tells me stuff like

Code: [Select]

fsck.fat 4.1 (2017-01-24)
Checking we can access the last sector of the filesystem
Boot sector contents:
System ID "MSDOS5.0"
Media byte 0xf8 (hard disk)
       512 bytes per logical sector
     16384 bytes per cluster
     16448 reserved sectors
First FAT starts at byte 8421376 (sector 16448)
         2 FATs, 32 bit entries
   7577600 bytes per FAT (= 14800 sectors)
Root directory start at cluster 2 (arbitrary size)
Data area starts at byte 23576576 (sector 46048)
   1892901 data clusters (31013289984 bytes)
128 sectors/track, 63 heads
      8064 hidden sectors
  60618880 sectors total
Checking for unused clusters.
Checking free cluster summary.
/dev/sda1: 1064 files, 57794/1892901 clusters
which tells me that this is FAT32 with 512 bytes per logical sector, and 32768 bytes (64 logical sectors) per cluster.  The smallest block this FAT32 filesystem uses is therefore 32768 bytes; even a 1-byte file will reserve 32768 bytes on the media.

Hope this clarifies things a bit.

[Nominal Animal]

What I was telling above, was that when the SD card data is collated using a Linux machine, it is easy to not have any kind of filesystem on it at all, as in Linux you can read the entire SD card as if it was a single continuous file.

Of course, it won't mount, so you won't be able to see its contents on the desktop, but you can write simple tools in C, C++, Python, etc. that let the user examine and export the data to ordinary files – XML and CSV formats are very common.  There are Python modules for exporting the data in other file formats, like Excel or OpenOffice/LibreOffice, or PDF.  Essentially, this moves the complexity from the microcontroller to the management utilities on a computer.

The only "issue" in Linux (and *BSDs) is the permission model.  Ordinary users do not normally have direct access to the block devices, mounted or not.  For this, I have developed a privileged, secure C helper, that examines the underlying block device, and if it looks (has initial/final sector contents matching specific pattern which is not a normal filesystem or partition table, i.e. specifically designed for use with microcontrollers), passes the open file descriptor, file handle, back to the unprivileged ordinary program.  This relies on Unix domain sockets, and passing file descriptors over such a socket, as specified in the POSIX.1 standard.  Basically all except Windows is enough POSIXy today so that the basic idea works.

Note that this is also secure: the unprivileged program does not have privileged access to anything else.  It has access only to the verified-safe-to-access SD card or USB memory stick or whatever media has the correct "signature".

Unfortunately, I have no idea how you would do anything like this in Windows.  I know it is possible in Macs, but I am not familiar enough with Mac OS access restrictions to say for sure whether my C program would work there as well as it does in Linux, or whether it would need some Mac-specific additions; I'd need to check it on a Mac development machine first.  I also have no idea whether Microsoft WSL supports such access at all, because I haven't used Windows in over a decade.  I just don't know.

[nctnico]

Don't bother writing your own FAT implementation. I've used this one in several projects:
http://elm-chan.org/fsw/ff/00index_e.html

It can be tailored to include more or less features depending on what you need versus the amount of flash space you have.

[ucanel]

Can you open sd card with pc Winhex Hxd and add the screenshot of boot sector,
so we may check it for you.
Maybe you are doing something wrong,
BPS information is two bytes long and located at boot sector 0x0C,0x0B

[Jan Audio]

All the sudden it works.
I wrote all zeros to the card, then formatted,
after formatting i found a valid boot sector.

Now i can get started with the fun part.

Don't bother writing your own FAT implementation. I've used this one in several projects:
http://elm-chan.org/fsw/ff/00index_e.html

It can be tailored to include more or less features depending on what you need versus the amount of flash space you have.

Yes i have that and i find it unreadable.
I be using that for inspiration a bit if i,m stuck.

[Jan Audio]

My version allows block to write smaller then 512 ( useless with only 1 SPI port, only i plan reading parallel data )
End of the week i be writing my file, else i be posting here.

[Jan Audio]

I found the root folder, that was easy.
Now i have the file.

All i have to do now is search for how it handles free clusters, my file is bigger then 1 cluster.

[Jan Audio]

All done, it was really simple, i took a few shortcuts.

Read boot-sector.
Find fatsize.
Find root-directory.
I filled a file text with the ammount of bytes i am writing.
Then look up in the FAT for the next cluster in the file.

[coppercone2]

is it possible to make a fat128?

I don't like how they make these marketing names like ORANGE and HAMMER2 that have nothing to do with computer things

[vinayak]

hi Jan audio,

Could you please provide the code which you have written for implementing the FAT32.

Regards,
Vinayak

[darkspr1te]

You didnt really say exactly which pic you are using , but maybe this will help
https://simple-circuit.com/pic18f46k22-sd-card-fat32-mikroc/


darkspr1te