TIP: Click on subject to list as thread! ANSI
echo: rberrypi
to: ALL
from: DENNIS LEE BIEBER
date: 2019-10-30 22:06:00
subject: Re: SDCardKiller

On Wed, 30 Oct 2019 21:12:59 -0400, Dennis Lee Bieber
 declaimed the following:


 Additional information: https://kernelnewbies.org/Ext4
"""
Delayed allocation, on the other hand, does not allocate the blocks
immediately when the process write()s, rather, it delays the allocation of
the blocks while the file is kept in cache, until it is really going to be
written to the disk. This gives the block allocator the opportunity to
optimize the allocation in situations where the old system couldn't.
Delayed allocation plays very nicely with the two previous features
mentioned, extents and multiblock allocation, because in many workloads
when the file is written finally to the disk it will be allocated in
extents whose block allocation is done with the mballoc allocator. The
performance is much better, and the fragmentation is much improved in some
workloads.
"""

 So... to exercise the SD card, you may want to explicitly flush the
data (presuming a flush will result in moving the cache contents to the
media; just closing a file may leave it to the OS as to when to flush the
cache).

"""
Larger inodes: Ext3 supports configurable inode sizes (via the -I mkfs
parameter), but the default inode size is 128 bytes. Ext4 will default to
256 bytes. This is needed to accommodate some extra fields (like nanosecond
timestamps or inode versioning), and the remaining space of the inode will
be used to store extend attributes that are small enough to fit it that
space. This will make the access to those attributes much faster, and
improves the performance of applications that use extend attributes by a
factor of 3-7 times.
"""

 Since the inode contains information about where the data of a file is
stored, it will get updated when the file is closed (well, when the journal
is committed). Note the size of the inode structure. That's a lot smaller
even than the 4kB "block" size that is default. So the card "unit" holding
the inodes will get lots of erase/copy/rewrite operations. Also take into
account that Linux "directory" is just a special file containing the file
name and a pointer/index to the inode associated with the file. Creating a
file will result in opening the directory file, adding a record, with an
inode index picked from the inode bitmap (so that's another part of the
media that gets rewritten, as will the data [zone] bitmap to identify
free/used data regions).

 So... Assume an SD card with 6-unit capability.

One unit can hold the inode bitmap
One unit can hold the zone/data bitmap
One unit can hold the directory file inode
One unit can hold the directory file data

So long as those units are held "open" they do not trigger writes to the
flash memory itself... that leave two units that can be used for the
journal inode and journal data...

 If you are lucky, the journal units don't get written to the actual
flash, but stay in the open "units" in the controller. But committing the
journal contents will result in having to modify a data file inode unit and
a data file data unit -- so that means having to close either the bitmap or
the directory file units... How the card determines which to close/write is
unknown. I'd hope it is the directory file, since that gets just the file
name and initial inode index -- likely when the data file is named in the
open call. The bitmaps will be affected as the data is journalled and
committed.

 In contrast, visualize a cheap SD card (one that was optimized for FAT
operation with video cameras -- which are rated for streaming video to a
freshly formatted card, no jumping around in the filesystem). Such a card
only handles two open units. In my hypothetical example, one would have
needed a card with 8 open unit capability to minimize unit erase/rewrite.
With only 2 units available, a lot of unit activity will take place (I'm
not even accounting for the journal in this list).

Open directory file inode (to find directory data location)
Open directory file data
Open inode bitmap to get a new inode for the new file (whoops, have to
close one of the other units)
Write file name/inode record to directory
Reopen directory file inode to update metadata (access time, etc. --
closing some other unit)
Open the new data file inode (close one other other units)
Open the data/zone bitmap to get space for new data (whoops have to close
another unit)
Open data space unit (close one of the open units)
Write the data and close the data file
Reopen data file inode to update the metadata

 EACH of those Open/reopen operations could result in the card
allocating a free unit, copying data from the original unit into the open
buffer, where it can be modified, and written to flash when it the unit is
closed. So -- without the journal, that card has already cycled at least
six erase units PER FILE CREATION. The other card, with 6 open units,
assuming the journal [which was not included in the 2-unit example] and
bitmaps stay in four of the open units and don't get flushed to media,
undergoes the six erase units initially, but then each subsequent file is
only cycling four units (2x directory, 2x data, ...) The journal and
bitmaps only get flushed when the card is dismounted.


--
 Wulfraed                 Dennis Lee Bieber         AF6VN
 wlfraed@ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

--- SoupGate-Win32 v1.05
* Origin: Agency HUB, Dunedin - New Zealand | FidoUsenet Gateway (3:770/3)

SOURCE: echomail via QWK@docsplace.org

Email questions or comments to sysop@ipingthereforeiam.com
All parts of this website painstakingly hand-crafted in the U.S.A.!
IPTIA BBS/MUD/Terminal/Game Server List, © 2025 IPTIA Consulting™.