On a sunny day (22 Jul 2018 18:59:16 +0100 (BST)) it happened Theo
wrote in
:
>Block size does matter, sometimes rather a lot.
>
>There are two components to this: the syscall overhead and write
>amplification. Syscall overhead is the number of times it has to shuttle
>back and forth between userland and kernel. On this Mac:
>
>$ dd if=/dev/zero of=/dev/null bs=1m count=1024
>1024+0 records in
>1024+0 records out
>1073741824 bytes transferred in 0.041791 secs (25693158687 bytes/sec)
>
>$ dd if=/dev/zero of=/dev/null bs=1 count=1g
>1073741824+0 records in
>1073741824+0 records out
>1073741824 bytes transferred in 1781.858768 secs (602596 bytes/sec)
>(the fans also went up to turbo for that time)
>
>$ dd if=/dev/zero of=/dev/null count=2m
>2097152+0 records in
>2097152+0 records out
>1073741824 bytes transferred in 3.446420 secs (311552811 bytes/sec)
>
>- so if you're copying to an SSD it would make a difference.
>(but less so on a Pi, due to its woeful I/O)
>
>However, there's another issue which is write amplification. If you're
>using an unbuffered write, which is what /dev/rdisk on Mac is, you'll be
>transmitting those 512 byte writes through to the SD card. The SD card
>erase block size is probably somewhere 128KB-1MB. If you write a 512 byte
>block, what's actually happening is it might read 1MB into a buffer,
>overwrite 512 bytes, then write 1MB back to a new block in the flash. So
>every 512 byte write amplifies to 1MB. Good SD cards hold some buffering so
>they can coalesce such writes but such buffering is costly for the
>microcontrollers in the card, so nasty SD cards just write the block back,
>doing vastly more work than they needed.
>
>I have here such a nasty 2GB SD card, and I get:
>
>$ sudo dd if=/dev/zero of=/dev/rdisk2 bs=1m
>dd: /dev/rdisk2: Input/output error
>1929+0 records in
>1928+0 records out
>2021654528 bytes transferred in 268.585332 secs (7527047 bytes/sec)
>
>$ sudo dd if=/dev/zero of=/dev/rdisk2
>dd: /dev/rdisk2: Input/output error
>3948545+0 records in
>3948544+0 records out
>2021654528 bytes transferred in 14503.783763 secs (139388 bytes/sec)
>
>- a factor of 54 slower. By this I would assume that each write is
>amplified (roughly) 54 times, and deduce this (small) SD card has a flash
>block size of 32KiB.
>
>In case you think the fault is using unbuffered mode, we can run the same
>test in buffered mode. It's better than 512 byte blocks unbuffered, but
>much worse than 1M unbuffered.
>
>$ sudo dd if=/dev/zero of=/dev/disk2 bs=1m
>dd: /dev/disk2: end of device
>1929+0 records in
>1928+1 records out
>2021658624 bytes transferred in 2222.524882 secs (909622 bytes/sec)
>
>$ sudo dd if=/dev/zero of=/dev/disk2
>dd: /dev/disk2: end of device
>3948545+0 records in
>3948544+0 records out
>2021654528 bytes transferred in 2711.264947 secs (745650 bytes/sec)
>
>
>Theo
Interesting, I do not have a MAC, just Linux PCs.
I find on my cards hardly any difference, reason why I started leaving out the
bs= option.
909622 / 745650 is a factor 1.2199
So probably did not notice that.
The card itself, especially in write more, is the limiting factor.
Do not have a free card ATM, but here the read timings for a 8 GB raspi card on
my Core5 laptop, card in /dev/sdc
~ # hdparm -t /dev/sdc
/dev/sdc:
Timing buffered disk reads: 50 MB in 3.03 seconds = 16.48 MB/sec
Without bs=
~ # dd if=/dev/sdc of=/dev/zero
16003072+0 records in
16003072+0 records out
8193572864 bytes (8.2 GB) copied, 472.769 s, 17.3 MB/s
dd if=/dev/sdc of=/dev/zero 4.94s user 24.64s system 6% cpu 7:52.81 total
With bs=
~ # dd if=/dev/sdc of=/dev/zero bs=1M
7814+0 records in
7814+0 records out
8193572864 bytes (8.2 GB) copied, 474.15 s, 17.3 MB/s
dd if=/dev/sdc of=/dev/zero bs=1M 0.07s user 16.86s system 3% cpu 7:54.45
total
So it is actualy slower with bs=1M specified ;-)
Of course that is pure coincidence, system processes affect things.
It does show the card read speed sets the speed, and matches almost exactly
what hdparm showed.
Writing to SDcards is usually much slower.
Then you get the same uncertainty, factor 1.2 does not mean so much, could be
other way around
man hdparm says to run it a couple of times too for a better idea.
--- SoupGate-Win32 v1.05
* Origin: Agency HUB, Dunedin - New Zealand | FidoUsenet Gateway (3:770/3)
|