Author Topic: tips and tricks, on using compact flash under linux  (Read 15389 times)

0 Members and 1 Guest are viewing this topic.

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
tips and tricks, on using compact flash under linux
« on: April 22, 2016, 06:28:21 pm »
so, I happen to have a 64Gbyte CF attached to a PCMCIA adaptor, and I need to back up the first slice of my laptop

I am on linux 2.6.39, I can't upgrade it, the CF is handled by "ide-cf"(1) (statically compiled inside the kernel), and dmesg is reporting the CF handled as "HDA"
(cat /proc/devices is confirming the major device, equal to 3, which is "HDA" in term of "device name" under linux)

thus, I'd like to type the following

Code: [Select]
kanojo #sudo dd if=/dev/sda1 of=/dev/hda

which actually works, but ... it's damn too slow

and forking a bigger buffer doesn't solve

Code: [Select]
kanojo #sudo dd if=/dev/sda1 of=/dev/hda bs=32Mbyte

my Compact Flash is made by SanDisk, it's called "Exteme", and claimed to be able to read/write things with a bandwidth of 160Mbyte/sec
thus I'd like to know why? on my laptop I am going something like ~10Mbyte/sec  :-//

there is no filesystem in the middle on the compact flash (which can cause overhead due to the flash needs, e.g. page erase before write)


any trick ? advice ? suggestion ? are the welcome

thanks




(1) ide-cf is p/ATA PCMCIA hard drive support
there is also a s/ATA PCMCIA hard drive support
unfortunately it doesn't work stable on my hardware



 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #1 on: April 22, 2016, 06:42:23 pm »
linux 2.6.39

  • Device Drivers -> ATA/ATAPI/MFM/RLL support (DEPRECATED) -> PCMCIA IDE support, "A driver for PCMCIA IDE/ATA disk cards", drivers/ide/ide-cs.c
  • Device Drivers -> Serial ATA and Parallel ATA drivers -> PCMCIA PATA support, "PCMCIA PATA controller driver", drivers/ata/pata_pcmcia.c

edit:
mac-fdisk doesn't recognize the correct amount of blocks in the CF card (it says 2Gbyte, wrong)
while cfdisk and fdisk recognize it correctly (they say 64Gbyte, correct)
« Last Edit: April 22, 2016, 06:44:09 pm by legacy »
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #2 on: April 22, 2016, 06:48:10 pm »
PCMCIA Storage
Today's PCMCIA/CF storage support densities in the gigabyte realm. The storage cards come in different flavors:

  • Miniature IDE disk drives or microdrives. These are tiny versions of mechanical hard drives that use magnetic media. Their data transfer rates are typically higher than solid state memory devices, but IDE drives have spin-up and seek latencies before data can be transferred. The IDE Card Services driver ide_cs, in conjunction with legacy IDE drivers, is used to communicate with such memory cards.
  • Solid-state memory cards that emulate IDE. Such cards have no moving parts and are usually based on flash memory, which is transparent to the operating system because of the IDE emulation. Because these drives are effectively IDE-based, the same IDE Card Services driver (ide_cs) can be used to talk to them.
  • Memory cards that use flash memory, but without IDE emulation. The memory_cs Card Services driver provides block and character interfaces over such cards. The block interface is used to put a filesystem onto card memory, whereas the character interface is used to access raw data. You may also use memory_cs to read the attribute memory space of any PCMCIA card.
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #3 on: April 22, 2016, 06:49:13 pm »
my SanDisk CF card is described by case2, Solid-state memory cards that emulate IDE
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #4 on: April 22, 2016, 06:55:22 pm »
I do not know the state of art with kernel 3.x and 4.x, with 2.6.39 "drivers/ata/pata_pcmcia.c" seems to be "experimental" for CF
here it doesn't work as expected

thus I have chosen "drivers/ide/ide-cs.c", which is known to be rock (even if old and deprecated)

feel free to let me know your experiences with such a kernel modules and devices  :-+
 

Offline stmdude

  • Frequent Contributor
  • **
  • Posts: 479
  • Country: se
Re: tips and tricks, on using compact flash under linux
« Reply #5 on: April 22, 2016, 06:58:36 pm »
Does hdparm give you any clues?

If I recall correctly, 2.6.x was a bit careful about enabling UltraDMA, Multi-sector transfers and other features needed for maximum performance.
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #6 on: April 22, 2016, 07:13:17 pm »
Code: [Select]
*  sys-apps/hdparm
      Homepage:      http://sourceforge.net/projects/hdparm/
      Description:   Utility to change hard drive performance parameters
      License:       BSD GPL-2

Code: [Select]
# hdparm -I /dev/hda

/dev/hda:

CompactFlash ATA device
Model Number:       SDCFXS-064G                             
Serial Number:      GAZ020116120307
Firmware Revision:  HDX12.03
Standards:
Likely used: 6
Configuration:
Logical max current
cylinders 65535 65535
heads 16 16
sectors/track 63 63
--
CHS current addressable sectors:   66059280
LBA    user addressable sectors:  125059072
LBA48  user addressable sectors:  125059072
Logical/Physical Sector size:           512 bytes
device size with M = 1024*1024:       61064 MBytes
device size with M = 1000*1000:       64030 MBytes (64 GB)
cache/buffer size  = 1 KBytes (type=DualPort)
Capabilities:
LBA, IORDY(may be)(cannot be disabled)
bytes avail on r/w long: 4
Standby timer values: spec'd by Vendor
R/W multiple sector transfer: Max = 1 Current = 0
Advanced power management level: disabled
DMA: not supported
PIO: pio0 pio1 pio2 pio3 pio4
     Cycle time: no flow control=120ns  IORDY flow control=120ns
Commands/features:
Enabled Supported:
    Power Management feature set
    Write cache
    WRITE_BUFFER command
    READ_BUFFER command
    NOP cmd
    CFA feature set
    Advanced Power Management feature set
   * 48-bit Address feature set
    Mandatory FLUSH_CACHE
    FLUSH_CACHE_EXT
CFA max advanced io_udma cycle time: 80ns
CFA max advanced mem_udma cycle time: 80ns
   * CFA Power Level 1  (max 500mA)
HW reset results:
CBLID- above Vih
Device num = 1
Integrity word not set (found 0x0000, expected 0xe4a5)
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #7 on: April 22, 2016, 07:15:26 pm »
125059072 LBA address is correct

see mac-fdisk

Code: [Select]
# mac-fdisk /dev/hda
Command (? for help): i
size of 'device' is 4194304 blocks:
new size of 'device' is 4194304 blocks
Command (? for help): p
/dev/hda
        #                    type name                length   base    ( size )  system
/dev/hda1     Apple_partition_map Apple                   63 @ 1       ( 31.5k)  Partition map
/dev/hda2              Apple_Free Extra              4194240 @ 64      (  2.0G)  Free space

Block size=512, Number of Blocks=4194304
DeviceType=0x0, DeviceId=0x0

4194304 instead of 125059072  |O |O |O
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #8 on: April 22, 2016, 07:16:47 pm »
Code: [Select]
R/W multiple sector transfer: Max = 1 Current = 0
 DMA: not supported

this is interesting for an UDMA device  :-//
 

Offline stmdude

  • Frequent Contributor
  • **
  • Posts: 479
  • Country: se
Re: tips and tricks, on using compact flash under linux
« Reply #9 on: April 22, 2016, 07:18:10 pm »
Can you run hdparm without parameters?  It should just spit out the performance related settings then.
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #10 on: April 22, 2016, 07:22:34 pm »
Code: [Select]
# hdparm /dev/hda

/dev/hda:
 multcount     =  0 (off)
 IO_support    =  0 (default)
 unmaskirq     =  0 (off)
 using_dma     =  0 (off)
 keepsettings  =  0 (off)
 readonly      =  0 (off)
 readahead     = 256 (on)
 geometry      = 65535/255/63, sectors = 125059072, start = 0
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #11 on: April 22, 2016, 07:28:10 pm »
Code: [Select]
# hdparm -a512 -d1 -m16 -u1 /dev/hda

/dev/hda:
 setting fs readahead to 512
 setting multcount to 16
 HDIO_SET_MULTCOUNT failed: Invalid argument
 setting unmaskirq to 1 (on)
 setting using_dma to 1 (on)
 HDIO_SET_DMA failed: Operation not permitted
 multcount     =  0 (off)
 unmaskirq     =  1 (on)
 using_dma     =  0 (off)
 readahead     = 512 (on)

it seems I have to recompile the kernel adding more support
 

Offline stmdude

  • Frequent Contributor
  • **
  • Posts: 479
  • Country: se
Re: tips and tricks, on using compact flash under linux
« Reply #12 on: April 22, 2016, 07:35:23 pm »
Yup, that would do it.

That's the most safe settings it could choose, but definitely not the fastest.
Code: [Select]
multcount     =  0 (off)
 IO_support    =  0 (default)
 using_dma     =  0 (off)

So, those are the important ones.

Right now, each 512byte sector is read like this:
1. Send READ_SECTOR command to the ATA controller
2. Twiddle thumbs until IRQ fires
3. Issue 256 IO operations to read the data. (16 bits per read)
4. Done

Now, the "multicount" setting changes how many sectors that can be transfered in one go. I.e, to transfer (for example) 4 sectors (sequential sectors), you only have to issue one command, and wait for one IRQ.  This produces a noticable increase in performance.
You can adjust this value with the "-m" parameter to hdparm

"IO_support" says if the IO operations are 16 or 32 bits (0 versus 1). Going 32 bits halves the number of IO operations needed to read out the data This also produces a noticable increase in performance.You can adjust this value with the "-c" parameter to hdparm

You can adjust this value with the "-c" parameter to hdparm

"using_dma" eliminates the IO operations all together. The data is DMAd into your RAM for you, and ready to use by the time the IRQ fires.
You can adjust this value with the "-d" parameter to hdparm

It seems your controller or disk doesn't like DMA though..

The mismatch in sectors reported from various fdisks is an issue I've seen myself, even on 4.4.x kernels.  Don't know what's up there..
 

Offline stmdude

  • Frequent Contributor
  • **
  • Posts: 479
  • Country: se
Re: tips and tricks, on using compact flash under linux
« Reply #13 on: April 22, 2016, 07:37:05 pm »
it seems I have to recompile the kernel adding more support

That's possible. It's been many years since I configured a 2.6.x kernel, and used (what effectively is) IDE disks.
 

Offline ade

  • Supporter
  • ****
  • Posts: 231
  • Country: ca
Re: tips and tricks, on using compact flash under linux
« Reply #14 on: April 22, 2016, 08:13:35 pm »
I'm pretty sure one cannot do IDE UDMA over PCMCIA.
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #15 on: April 22, 2016, 08:26:48 pm »
I am on IBM X61S, it's an intel core duo2 laptop  :popcorn:
going to recompile the kernel

 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #16 on: April 22, 2016, 08:27:58 pm »
it's PCMCIA v2.1
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #17 on: April 22, 2016, 08:34:17 pm »
Quote
Industrial SLC PCMCIA ATA Card HERCULES-N Series
SLC - NAND type flash technology
Data transfer mode support PIO 0~6, MWDMA 0~4, UDMA 0~7
Compatible with PC card standard and PCMCIA V.2.1
Compatible with ATA/ATAPI-8 standard
PCMCIA Type II form-factor
S.M.A.R.T. (Self-Monitoring, Analysis and Reporting Technology)
Supports PC Card Memory mode or True IDE mode
Optional for standard casing and rugged metal casing
Capacities from 8GB up to 64GB
Performance up to: Read 117.4 MB/sec ; Write 83.8 MB/sec

an example of a product that should work that way
it might be a linux matter (kernel module, configuration),
or an hardware matter (my PCMCIA controller)
who knows?  :-//
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #18 on: April 22, 2016, 08:38:04 pm »
and here it is CF technology
Performance up to: Read 37.8 MB/sec ; Write 27.0 MB/sec

is there any valid reason to consider a bottleneck between the CF interface (inside the card) and PCMCIA?
 

Offline ade

  • Supporter
  • ****
  • Posts: 231
  • Country: ca
Re: tips and tricks, on using compact flash under linux
« Reply #19 on: April 22, 2016, 09:09:47 pm »
Quote
is there any valid reason to consider a bottleneck between the CF interface (inside the card) and PCMCIA?

CF basically supports three different interfaces:  PIO (Port I/O), MMIO (Memory-Mapped I/O) and DMA.

PIO is the slowest, < 10 MB/s.  MMIO can go between 10 MB/s and 20 MB/s depending on the exact transfer method.  DMA is much faster, potentially > 100 MB/s.

However, the interface we typically call "PCMCIA" does not support CF/IDE/ATA DMA (*), so it is limited to MMIO at best.

DMA support only came with "PCMCIA 5.0" which is a completely different 32-bit interface more commonly called CardBus.

So from the above it seems that you are getting MMIO throughput.  There may be a way to optimize it a bit by playing with different timings / settings but not to DMA speeds.

(*) Note: I believe PCMCIA 2.1 optionally specified a type of rudimentary DMA, but it is not directly compatible with IDE DMA and hence rarely implemented.
 

Offline Rasz

  • Super Contributor
  • ***
  • Posts: 2616
  • Country: 00
    • My random blog.
Who logs in to gdm? Not I, said the duck.
My fireplace is on fire, but in all the wrong places.
 

Offline Kilrah

  • Supporter
  • ****
  • Posts: 1852
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #21 on: April 23, 2016, 07:03:07 am »
I played with a CF and PCMCIA maybe 5-6 years ago to find a use for the 32GB CF I had for my camera after I changed to a new one that used SD instead, if I remember well the PCMCIA to CF adapter had to explicitely support UDMA if you wanted that to work and it was very rare, there were like only one or 2 models available that did and they were "complex" devices, i.e. included a controller chip with drivers, not a simple passive adapter.

« Last Edit: April 23, 2016, 07:14:06 am by Kilrah »
 

Offline Kilrah

  • Supporter
  • ****
  • Posts: 1852
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #22 on: April 23, 2016, 07:20:59 am »
Found the reference of the adapter I had in my old emails (was actually 8 years ago):
http://www.mydigitaldiscount.com/delkin-compact-flash-pcmcia-cardbus-32-adapter-ddcfprocb-ad/

I remembered correctly about a driver being needed, so might not be of much use for you as it's likely not supported on linux.
 

Offline Rasz

  • Super Contributor
  • ***
  • Posts: 2616
  • Country: 00
    • My random blog.
Re: tips and tricks, on using compact flash under linux
« Reply #23 on: April 23, 2016, 07:55:32 am »
Found the reference of the adapter I had in my old emails (was actually 8 years ago):
http://www.mydigitaldiscount.com/delkin-compact-flash-pcmcia-cardbus-32-adapter-ddcfprocb-ad/

I remembered correctly about a driver being needed, so might not be of much use for you as it's likely not supported on linux.

its not pcmcia (essentially ISA), its CardBus (pretty much PCI), thats why it needs a driver for a full blown ide controller inside
Who logs in to gdm? Not I, said the duck.
My fireplace is on fire, but in all the wrong places.
 

Offline ade

  • Supporter
  • ****
  • Posts: 231
  • Country: ca
Re: tips and tricks, on using compact flash under linux
« Reply #24 on: April 23, 2016, 08:01:20 am »
Kilrah, the terminology is a bit confusing but what you've linked to is what we'd commonly call "CardBus", not "PCMCIA".

So that adapter will only work on laptops with a 32-bit CardBus slot, not ones with a "PCMCIA" port.

It is confusing because CardBus was technically described in the "PCMCIA 5.0" specification.

PCMCIA stands for "People Can't Memorize Computer Industry Acronyms".   ;D

Edit: Rasz beat me to it!
 

Offline Kilrah

  • Supporter
  • ****
  • Posts: 1852
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #25 on: April 23, 2016, 08:15:26 am »
Kilrah, the terminology is a bit confusing but what you've linked to is what we'd commonly call "CardBus", not "PCMCIA".
So that adapter will only work on laptops with a 32-bit CardBus slot, not ones with a "PCMCIA" port.

Sure, but precisely the reason the whole thing is confusing is that everybody uses "PCMCIA" for anything in that form factor and Cardbus is never really mentioned... and since CardBus is already 20 years old you can be pretty sure that close to 100% of what's referred to as PCMCIA today is actually CardBus. Specifically I doubt that OP is using a 20+ year old machine...
 

Offline ade

  • Supporter
  • ****
  • Posts: 231
  • Country: ca
Re: tips and tricks, on using compact flash under linux
« Reply #26 on: April 23, 2016, 08:55:58 am »
Quote
Specifically I doubt that OP is using a 20+ year old machine...
True... the laptop itself should support CardBus.
 

Online isometrik

  • Regular Contributor
  • *
  • Posts: 54
  • Country: ca
Re: tips and tricks, on using compact flash under linux
« Reply #27 on: April 23, 2016, 09:20:00 am »
I am not sure if that is the case for all drive types, but generally speaking, Linux (and other flavours of *nix) offers two types of block devices: buffered (slower) and raw (faster).

For example, the buffered (slower) block device "/dev/disk0" will have a raw (faster) equivalent block device under "/dev/rdisk0". The raw (faster) block device name is prepended with a "r".

Have a try, it should make the "dd" transfer much faster.
 

Offline Rasz

  • Super Contributor
  • ***
  • Posts: 2616
  • Country: 00
    • My random blog.
Re: tips and tricks, on using compact flash under linux
« Reply #28 on: April 23, 2016, 10:54:04 am »
Specifically I doubt that OP is using a 20+ year old machine...

thats because you dont know the author :P 13 year old kernel should be a good hint ;)
not to mention pcmcia cf converters are common, there is no electronics inside, just two sockets, cardbus ones are full PCI ide controllers.
Who logs in to gdm? Not I, said the duck.
My fireplace is on fire, but in all the wrong places.
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #29 on: April 23, 2016, 11:20:22 am »
PCMCIA stands for "People Can't Memorize Computer Industry Acronyms".   ;D

 :-DD :-DD :-DD
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #30 on: April 23, 2016, 11:23:30 am »
Quote
Specifically I doubt that OP is using a 20+ year old machine...
True... the laptop itself should support CardBus.

yes, I enabled the support, 16 and 32bit
my hardware is supported by (kernel module name) "yenta_cardbus"
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #31 on: April 23, 2016, 11:52:17 am »
Code: [Select]
x61s ~ # lspcmcia -v
Socket 0 Bridge:    [yenta_cardbus] (bus ID: 0000:05:00.0)
Configuration: state: on ready: yes

Voltage: 3.3V
 Vcc: 3.3V
 Vpp: 0.0V

Socket 0 Device 0: [ide-cs] (bus ID: 0.0)
Configuration: state: on
[io  0x4100-0x410f flags 0x110]
[io  0x0000 flags 0x100]
[mem 0x00000000 flags 0x200]
[mem 0x00000000 flags 0x200]
[mem 0x00000000 flags 0x200]
[mem 0x00000000 flags 0x200]

Product Name:   SANDISK
 SDCFXS-064G
 
Identification: manf_id: 0x00f1 card_id: 0x0101
function: 4 (fixed disk)
prod_id(1): "SANDISK
" (0xda4ae7f3)
prod_id(2): "SDCFXS-064G
" (0xd2534b65)
prod_id(3): --- (---)
prod_id(4): --- (---)


Code: [Select]
x61s ~ # pccardctl info
PRODID_1="SANDISK
"
PRODID_2="SDCFXS-064G
"
PRODID_3=""
PRODID_4=""
MANFID=00f1,0101
FUNCID=4
x61s ~ #


interesting, the linux kernel module in S/ATA PCMCI support does not recognized MANFID=00f1,0101 :palm: :palm: :palm:

drivers/ata/pata_pcmcia.c
Code: [Select]
...
static struct pcmcia_device_id pcmcia_devices[] =
{
        PCMCIA_DEVICE_FUNC_ID(4),
        PCMCIA_DEVICE_MANF_CARD(0x0000, 0x0000),        /* Corsair */
        PCMCIA_DEVICE_MANF_CARD(0x0007, 0x0000),        /* Hitachi */
        PCMCIA_DEVICE_MANF_CARD(0x000a, 0x0000),        /* I-O Data CFA */
        PCMCIA_DEVICE_MANF_CARD(0x001c, 0x0001),        /* Mitsubishi CFA */
        PCMCIA_DEVICE_MANF_CARD(0x0032, 0x0704),
        PCMCIA_DEVICE_MANF_CARD(0x0032, 0x2904),
        PCMCIA_DEVICE_MANF_CARD(0x0045, 0x0401),        /* SanDisk CFA */
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #32 on: April 23, 2016, 11:55:11 am »
Bus options (PCI etc.)  --->  PCCard (PCMCIA/CardBus) support  --->

Code: [Select]
--- PCCard (PCMCIA/CardBus) support                                                             ? ? 
  ? ?                         <*>   16-bit PCMCIA support                                                                     ? ? 
  ? ?                         [*]     Load CIS updates from userspace (EXPERIMENTAL)                                          ? ? 
  ? ?                         -*-   32-bit CardBus support                                                                    ? ? 
  ? ?                               *** PC-card bridges ***                                                                   ? ? 
  ? ?                         <M>   CardBus yenta-compatible bridge support
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #33 on: April 23, 2016, 12:17:01 pm »
&& this link seems interesting while this is LoL

 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #34 on: April 23, 2016, 12:28:13 pm »
interesting, a lot of problems with 2.6.24
CF detected as PIO0 is performing 1.6 MByte/sec 
oh my goodness :palm: :palm: :palm:

Quote
Linus' tree is also broken.

I tried a Linus 2.6.24-rc4 and it acts the same way, with a very slow
transfer rate. 

I also tried 2.6.24-rc4 with the older not-libata PATA drivers and it is
broken.  dmesg had a line about the CF card detected as hda,
but /sys/block did not have hda and /dev/hda did not function.

I will try the patches you mentioned, but I think I may also have to
work backward through kernel versions until I find the last one where
the PCMCIA hd{a,b,c,d,e} drivers worked.
( 2.6.24-rc4-mm1 and Very Slow PCMCIA Compact Flash )
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #35 on: April 23, 2016, 12:31:19 pm »
also it looks like pata_pcmcia is always PIO mode 0

Code: [Select]
/*
  * pcmcia_init_one - attach a PCMCIA interface
  * @pdev: pcmcia device
  *
  * Register a PCMCIA IDE interface. Such interfaces are PIO 0 and
  * shared IRQ.
  */

jesus  :palm: :palm: :palm:
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #36 on: April 23, 2016, 12:34:08 pm »
Quote
>>>>> I normally use a USB-2 memory card reader but I also have a PCMCIA
>>>>> CompactFlash adapter that I use occasionally.  During the MM series
>>>>> kernels 2.6.22 and 23 (I am pretty sure) this didn't work at all.  I
>>>>> don't know about vanilla since I don't run that.
>>>>>
>>>>> Now with MM kernels 2.6.24 rc1-4 the PCMCIA adapter works again, but I
>>>>> only get read rates of 1.6 MB/s.  When it used to work in 2.6.20 I got
>>>>> at least 16 MB/s.  The card itself is capable of 30+ in the USB-2
>>>>> reader.


need to check an old kernel 2.6.20
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #37 on: April 23, 2016, 11:47:02 pm »
anyone using CF with CardBus over linux 4.* or 3.* ?

going to apply this setup for my laptop
kernel 3.17.7 (even if … I can't upgrade 2.6.39, I am considering a dual boot with grub)
 

Offline Monkeh

  • Super Contributor
  • ***
  • Posts: 7990
  • Country: gb
Re: tips and tricks, on using compact flash under linux
« Reply #38 on: April 24, 2016, 02:25:49 am »
You can play with your kernel all you like, it won't change the fact that you need an IDE adapter.
 
The following users thanked this post: Kilrah

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #39 on: April 24, 2016, 10:00:43 am »
usb-CF adaptor
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #40 on: April 24, 2016, 10:04:18 am »
anyway, I can't understand why, if cardbus is pci-like, then plugging a microdrive on a cardbus-CF adaptor gives more performances than a compact flash

10Mbyte/sec (a few models give 20Mbyte/sec) vs 1.6Mbyte/sec

it's 1:10..1:20, so what is wrong with compact flash? (oh, declared to be "true IDE")
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #41 on: April 24, 2016, 10:24:27 am »
The speed is limited by your CardBus controller, whether it supports UDMAx or not.

according with hdparm there is no UDMAx flags with micro drive, and I get 10-20Mbyte/sec
while with a compact flash in "True-IDE" I get 1.6Mbyte/sec

physically

Code: [Select]
05:00.0 CardBus bridge: Ricoh Co Ltd RL5c476 II (rev ba)

Code: [Select]
05:00.0 CardBus bridge: Ricoh Co Ltd RL5c476 II (rev ba)
Subsystem: Lenovo Device 20c6
Flags: bus master, medium devsel, latency 168, IRQ 16
Memory at d7eff000 (32-bit, non-prefetchable) [size=4K]
Bus: primary=05, secondary=06, subordinate=07, sec-latency=176
Memory window 0: d8000000-dbffffff (prefetchable)
Memory window 1: 80000000-83ffffff
I/O window 0: 00004000-000040ff
I/O window 1: 00004400-000044ff
16-bit legacy interface ports at 0001
Kernel driver in use: yenta_cardbus



thus, which is the difference between a micro drive interface and a compact flash interface?
(none of them use UDMA, according to the flags I see)
 

Offline Monkeh

  • Super Contributor
  • ***
  • Posts: 7990
  • Country: gb
Re: tips and tricks, on using compact flash under linux
« Reply #42 on: April 24, 2016, 11:04:10 am »
CardBus is a multi-purpose bus that can be configured to be IDE, that is to say, there is not a single chip in a CardBus to CF converter. The speed is limited by your CardBus controller, whether it supports UDMAx or not.

Uh.. no, CardBus is effectively PCI. You need a host controller attached to do DMA, or you're stuck with PIO on a 16-bit bus (PCMCIA mode).
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #43 on: April 24, 2016, 11:40:14 am »
You need a host controller attached to do DMA, or you're stuck with PIO on a 16-bit bus (PCMCIA mode).

may be the problem is related to the Yenta_CardBus kernel drive, more specifically there is no specific driver for the host controller
it my case it's "Ricoh", and it seems they probably just re-branded some other generic components.

I typed  "lspci -v" and looked for the device numbers after "Subsystem: Device" for those device.
Then googled for those device numbers to figure out who actually made the chipsets for the devices.

Unfortunately I have found no good information  :-//


anyway, I can explain why the micro drive achieves better performances within the same configuration
 

Offline Monkeh

  • Super Contributor
  • ***
  • Posts: 7990
  • Country: gb
Re: tips and tricks, on using compact flash under linux
« Reply #44 on: April 24, 2016, 11:42:25 am »
You need a host controller attached to do DMA, or you're stuck with PIO on a 16-bit bus (PCMCIA mode).

may be the problem is related to the Yenta_CardBus kernel drive, more specifically there is no specific driver for the host controller
it my case it's "Ricoh", and it seems they probably just re-branded some other generic components.

I typed  "lspci -v" and looked for the device numbers after "Subsystem: Device" for those device.
Then googled for those device numbers to figure out who actually made the chipsets for the devices.

Unfortunately I have found no good information  :-//

You have the driver for the cardbus interface, it is not an IDE host controller.
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #45 on: April 24, 2016, 11:49:15 am »
it is not an IDE host controller

you are in my ignore list
 

Offline Monkeh

  • Super Contributor
  • ***
  • Posts: 7990
  • Country: gb
Re: tips and tricks, on using compact flash under linux
« Reply #46 on: April 24, 2016, 11:50:39 am »
Uh.. no, CardBus is effectively PCI. You need a host controller attached to do DMA, or you're stuck with PIO on a 16-bit bus (PCMCIA mode).

From Wikipedia,

Quote
CompactFlash is a smaller dimensioned 50 pin subset of the 68 pin PC Card interface. It requires a setting for the interface mode of either "memory" or "ATA storage".

Probably that means PCMCIA has a "memory" or "ATA storage" mode or both.

And PCMCIA is not CardBus..

E: Or more correctly, PCMCIA <v5 is not CardBus.

Yes, you can have a 16-bit PIO ATA interface with PCMCIA. Or you can have an IDE controller attached via CardBus and get UDMA.

it is not an IDE host controller

you are in my ignore list

You can lead a horse to water..
« Last Edit: April 24, 2016, 11:54:00 am by Monkeh »
 
The following users thanked this post: Kilrah

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #47 on: April 24, 2016, 11:58:51 am »
mmm from the documentation I also see that "CFA extensions" can allow to use pio5 or pio6 (>20Mbyte/sec)
while from linux sources (2.6.39) I see that the ide-cs can support only pio0 (~1Mbyte/sec)

and again it's not clear why the micro drive achieves 10Mbyte/sec in the same bus, plugged to the same interface, and handled by the same kernel

going to buy a second usb-cd adapter, but … it's a workaround  :palm:
(probably I'd better to say to my boss that … his solution is defective)
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #48 on: April 24, 2016, 12:00:31 pm »
p.s.
the SSD-miniPCIe was a good tips, faster, cheaper, very good tips
unfortunately I can't open the laptop (it's not mine)
 

Offline Monkeh

  • Super Contributor
  • ***
  • Posts: 7990
  • Country: gb
Re: tips and tricks, on using compact flash under linux
« Reply #49 on: April 24, 2016, 12:44:35 pm »
Congratulations, you finally bothered to look for an IDE controller!

Sadly, it seems performance on those ones is substandard due to lack of documentation, but it's at least faster than PIO.
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #50 on: April 24, 2016, 12:46:18 pm »
Quote
CompactFlash CardBus adapter from Delkin devices : Delkin announced its first shipments of the CardBus 32 UDMA CompactFlash adapter to camera stores. The CardBus 32 UDMA adapter is designed to transfer images to a computer using UDMA (Ultra Direct Memory Access) enabled CF memory cards at over 45 MB/s. The pairing of CardBus and UDMA technologies allows the fastest data transfer from UDMA CF cards to Apple and PC laptops with CardBus slots. “Successive waves of the latest Digital SLRs produce ever-increasing file sizes, which typically result in slower image transfers,” said Eric Richter, Delkin’s Marketing Manager. “That’s unacceptable to digital photographers who work with laptops as a critical part of their workflow process.”

CompactFlash CardBus adapter from Delkin devicesCompactFlash CardBus adapter
With UDMA CompactFlash cards already available in the market, photographers with Delkin’s new CardBus adapter have the fastest and easiest way to transfer images from their memory card to their computer. No other card reader available in the market comes close to competing with the transfer speeds achieved by the CardBus 32 UDMA adapter. The CardBus UDMA adapter greatly improves the efficiency of a photographer’s workflow with transfer speeds over 45 MB/s. These ultra-fast speeds are achieved when using UDMA enabled CF cards. Delkin’s website has drivers available for Windows Vista/XP/2000/ME/98/98 SE and Mac OS X 10.4+. The Delkin Devices CompactFlash CardBus UDMA adapter sells at an MSRP of $59.99.

Delkin PCMCIA adapters
Delkin Devices, has long been the first in connectivity products by offering the widest variety of PCMCIA adapters in the industry, introduced the innovative CardBus adapter to the USA market over three years ago. Now both the digital photography and the computer market are poised to move towards UDMA technology. Once again, Delkin stays ahead of the technology curve, this time by providing an adapter solution for photographers who demand the fastest means to transfer their digital images into a computer.

About Delkin Devices
Delkin Devices, Inc. is based in Poway, California, and its European branch office is located in Birmingham, England. Delkin has been providing “Everything but the Camera” since 1986. From Archival Gold Media and SensorScope cleaning products to Pop-Up Shades, Delkin products deliver premium quality, innovative designs and an ongoing dedication to superior customer service. Delkin Devices and its products have become worldwide industry leaders in both OEM and consumer markets. Delkin branded memory cards continue to be a leading choice among professional photographers and others passionate about digital photography. Delkin Devices’ innovative digital photography products are available worldwide through a network of independent camera stores, and online.
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #51 on: April 24, 2016, 12:52:20 pm »
good rumors from Amazon reviews, going to buy one, I am curious.
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #52 on: April 24, 2016, 02:26:56 pm »
interesting

Code: [Select]
+ * No chip documentation has yet been found,
+ * so these configuration values were pulled from
+ * a running Win98 system using "debug".
+ * This gives around 3MByte/second read performance,
+ * which is about 2/3 of what the chip is capable of.
+ *
+ * There is also a 4KByte mmio region on the card,
+ * but its purpose has yet to be reverse-engineered.

need for reverse-engineering ;D
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #53 on: April 24, 2016, 02:42:11 pm »
2.6.39
drivers/ide/delkin_cb.c
Code: [Select]
*  Basic support for Delkin/ASKA/Workbit Cardbus CompactFlash adapter
 *
 *  Modeled after the 16-bit PCMCIA driver: ide-cs.c
 *
 *  This is slightly peculiar, in that it is a PCI driver,
 *  but is NOT an IDE PCI driver -- the IDE layer does not directly
 *  support hot insertion/removal of PCI interfaces, so this driver
 *  is unable to use the IDE PCI interfaces.  Instead, it uses the
 *  same interfaces as the ide-cs (PCMCIA) driver uses.
 *  On the plus side, the driver is also smaller/simpler this way.
 *
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #54 on: April 24, 2016, 11:40:58 pm »
unbelievable: cardbus to expresscard adapters. they actually exist  :o :o :o
 

Offline Rasz

  • Super Contributor
  • ***
  • Posts: 2616
  • Country: 00
    • My random blog.
Re: tips and tricks, on using compact flash under linux
« Reply #55 on: April 25, 2016, 05:59:53 am »
unbelievable: cardbus to expresscard adapters. they actually exist  :o :o :o

no they dont, this is usb controller with fancy name to scam noobs
Who logs in to gdm? Not I, said the duck.
My fireplace is on fire, but in all the wrong places.
 
The following users thanked this post: Kilrah

Offline Kilrah

  • Supporter
  • ****
  • Posts: 1852
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #56 on: April 25, 2016, 07:27:42 am »
Yup. This
Quote
For Express Card 34 mm with USB controller


is the key, the adapter will only work with "basic" low-performance ExpressCards that only use the USB 2.0 lane, not the high performance ones that actually use the PCI-Express lanes.
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #57 on: April 25, 2016, 06:31:53 pm »
someone experienced with Delkin Cardbus IDE adaptor on linux ?
if so, I'd like to hear your feedback, especially about performances

thanks
 

Offline Kilrah

  • Supporter
  • ****
  • Posts: 1852
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #58 on: April 25, 2016, 06:38:40 pm »
Not on linux, but I can tell you that unless you intend to only store a small number of large files you don't access often you'll likely be disappointed. Not because of the adapter, but because a CF card doesn't have proper caching like a real SSD would, and while peak sequential transfer might be fast small random accesses are still appallingly slow.
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #59 on: April 25, 2016, 07:51:03 pm »
unless you intend to only store a small number of large files

the compact flash card will be used by a custom application that I am developing is an embedded application (non on the laptop, I will use on a Geode embedded board, basically it's a miniPC with CardBus interface, and it runs linux/x86) which needs to use a flash filesystem without folders, everything in the root (like in CB/M), large pictures from an XRAY machine

it could be described, just to exemplify, like the following

Code: [Select]
dd if=pictureXX.bin of=/dev/hda1/pictureXX.bin bs=Buffer_size


but I need 5Mbyte/sec at least, the CF is able to go up to 40/50Mbyte/sec, and potentially the CardBUS IDE controller is able to have the full PCI speed at 32bit@33Mhz, say, in the theory, an average bandwidth of no less than 100Mbyte/sec?
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #60 on: April 25, 2016, 07:52:36 pm »
oh, it's a mini X-ray machine, small object, like teeth, 10-16Mbyte per picture
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #61 on: April 29, 2016, 04:44:50 pm »
it seems we have a problem  :palm: :palm: :palm: 
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #62 on: April 29, 2016, 05:31:49 pm »
what an unbelievable equipment  :o :o :o
 

Offline Rasz

  • Super Contributor
  • ***
  • Posts: 2616
  • Country: 00
    • My random blog.
Re: tips and tricks, on using compact flash under linux
« Reply #63 on: April 29, 2016, 07:44:51 pm »
what an unbelievable equipment  :o :o :o

as mentioned before cardbus IS PCI, this is just a pci bridge in a box
Who logs in to gdm? Not I, said the duck.
My fireplace is on fire, but in all the wrong places.
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #64 on: April 29, 2016, 07:47:23 pm »
it might be very useful for my SGI-IP30 machine, due to its mechanical limitation about the size of PCI-cards
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #65 on: April 30, 2016, 07:44:37 am »
Code: [Select]
scsi0 : Adaptec 152x SCSI
(scsi0:4:0) 01 03 01 32 08
scsi 0:0:4:0: Direct-Access     CF 4GB
sd 0:0:4:0: [sda] 7928928 512-byte hardware sectors (4060 MB)
sd 0:0:4:0: [sda] Write Protect is off
sd 0:0:4:0: [sda] Mode Sense: 03 00 00 08
sd 0:0:4:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sd 0:0:4:0: [sda] 7928928 512-byte hardware sectors (4060 MB)
sd 0:0:4:0: [sda] Write Protect is off
sd 0:0:4:0: [sda] Mode Sense: 03 00 00 08
sd 0:0:4:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
 sda: sda1
sd 0:0:4:0: [sda] Attached SCSI disk
sd 0:0:4:0: Attached scsi generic sg0 type 0

Code: [Select]
# hdparm -t /dev/sda

/dev/sda:
 Timing buffered disk reads:    6 MB in  3.64 seconds =   1.65 MB/sec

My boss installed an Adaptec 152x card in my laptop CardBus slot, attached to a SCSI-pATA converter (claimed 10Mbyte/sec, SCSI Narrow/DE), we used an IDE-to-CF adaptor to interface the compact flash, and ... still miserable performances  :palm: :palm: :palm:
 

Offline legacyTopic starter

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: tips and tricks, on using compact flash under linux
« Reply #66 on: April 30, 2016, 01:02:32 pm »
what an unbelievable equipment  :o :o :o

as mentioned before cardbus IS PCI, this is just a pci bridge in a box

do you happen, guys, to know where can I buy one for cheap?
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf