On Jun 15, 1996 at 08:24, Bob Lawrence of 3:711/934.12 wrote:

 BL> When I profiled it, virtually *all* the time was taken writing to
 BL> file.

This is generally the case (since I/O is always slow);  what I was
proposing was a single read/write operation for things that can support it
(like headers).

 BL> Hence the two-buffer approach. I just read the PKT byte-by-byte and
 BL> translate what's needed into the QWK buffer. One read, one write...

Ick .. use a buffer of arbitrary length - say, 4Kb.  Keep a variable that
indicates how much of the buffer is actually "full".  Read/write
in 4Kb lots, if you get to the end of the file when less than 4Kb remains,
then your variable will contain the number of "valid" bytes in
the buffer, so only process that many bytes.

For a 32Kb file, that's 8 reads/writes instead of 32,768 reads/writes -
unless I'm reading too much into this "byte-by-byte" thing.  Also
cuts down on function calls, which also slows things down.

 BL> What you are saying, is create a struct of five pointers, point at
 BL> whatever is necessary but then write the *struct* to file as one thing..

Basically, yes - this way of thinking is useful for headers where you're
working with "fields" within the data, but "raw message
text" is something different (although I s'pose you could modify the
theory for that circumstance).

 BL> Brilliant!

Basically, yes.

 BL> Why am I so thick?

Err..

 BL> Thanks for that, Dave.

Hope it helps.

 BL> I like the idea of replacing 64K with 40 sets of 5 pointers... waait an
 BL> minute!  I'd also need 40 sets of genuine memory for headers, and
 BL> footers, and all that crap. I'm looking at 15K... and a shambles!

Careful - didn't you say you read things in then dumped them straight out? 
If so, then you shouldn't need to keep "sets" of things in memory
(only the current header);  alternatively, you could just keep a record of
a file offset for each one, then just seek to that offset if you wanted the
header in question .. as always, many ways to skin Billdinger's cat.

As far as it being a shambles - that's the price you pay for performance
(that's why "speed optimised" code is generally larger than
"size optimised" code).  Always a trade-off...

    - dave
    d.begley{at}ieee.org

---