RS> If you are going to do that sort of check it would be much
RS> better to look at the saved archived PKTs with an archiver
RS> etc to get the total size of the PKTs in there from the directory
RS> info in the archive and compare that with the resulting QWK.

BL> Yes. I agree. You need to look at the archived pkt before
BL> it's unzipped, and at the qwk archive after it's zipped.
BL> Wouldn't a size check be good enough?

Nar, the variation in the PKT/QWK ratio, even if you use the unpacked
numbers is too great, it can easily miss a few messages dropped.

RS> I actually meant using the size data thats in the archive file
RS> embedded directory.

BL> I wrote a VB program that does that, for an unzipper
BL> I was making, but it's pretty messy... and slow.

Yeah, its one of the deficiencys of the archive file designs,
the directory data is scattered thru the file. Its done like
that on purpose to make recovery of a grunged archive more feasible,
or atleast to maximise the chances of recovering what you can.

RS> You really should be doing the calculation on the unarchived
RS> size, not the archived size, coz the packing ratios vary quite
RS> a bit between PKT and QWK since the common strings are different.

BL> Yair... but you tend to get a consistent average over various
BL> packets and sizes.

And consistent averages arent any use when trying to detect if a
message or two got lost. With a particular ratio on a particular
PKT/QWK pair you never know if it was an exceptional collection
of messages or some dropped.

BL> Oddly, the QWK ends up a bit smaller, even with the extra
BL> .ndx files and control.dat.

Yeah, thats mostly the PATH and SEENBYS getting dropped, but the
business of unpacked to packed size is surprisingly complex and
you dont even get the same effect over all the archive formats.
Any dictionary style compression has to give odd and hard to
predict behaviour at times.

BL> It's a real drag reading the archive header for file sizes, and
BL> they're all different. It wouldn't be too bad for just zip, say.

Yeah, no argument that its definitely harder and slower. OTOH I dont
believe the other is good enough.

BL> All I want to do, is print a warning message, abort the conversion,
BL> and save the original packets.

RS> Thats a problem in some ways. There is a lot to be said for trying
RS> to carry on regardless if a fixup does appear to have got back into
RS> synch, notify the user and certainly keep the originals.

BL> Well... the only reason it'll stuff up is if the packet is damaged
BL> by the time the converter gets it... or if it runs out of memory .

Sure, its guaranteed to have perfect code with such a simple and rigid
message format.

RS> Fact remains tho, for a reliable safety check message counter
RS> in PKTs, you need a completely different alg, new code, the
RS> works. I'm not convinced its particularly difficult.

BL> That won't help.

Sure it will.

BL> It'd work if I read the files in the archive,

Yes, thats the best approach, tho the most difficult.

BL> but if I have to unzip them and open them, it'll end up the
BL> same as the converter, but less robust.

Nope, you certainly do have to actually scrutinise the unzipper
error code, not just ignore it like is done now, but thats dead easy.

BL> You can't look for any single thing that defines a message
BL> unambiguously, you have ot read it step-by-step.

Yes, but a completely separate message counter is a different thing
to one what also detects the other stuff in a message too. If only
because its much simpler, its a very useful extra check. And certainly
far far better than comparing the sizes of the PKT and QWK since that
just cant detect a small message or two getting lost whatever you do.

BL> The pkt format is mad!

Yeah, its a classic amateur design, pretty pathetic. Corse that doesnt
mean its not usable tho, just that it requires more work to use.

BL> You can't even find anything to double-check a message if you
BL> miss a null terminator.

Yeah, thats its biggest deficiency, no safety nets at all really.
And thats uttelry mad when there are going to be a multitude of
amateurs writing code which generates or uses it.

The other big deficiency is the inconsistency, the netmail being
quite different in detail to the echomail etc.

BL> QWK has a possible double check with the ndx files, but with pkt
BL> there's nothing.

Yeah, tho the one big advantage the PKT format has is that its certainly
extendible. QWK is much worse there. Not that that couldnt have still be
done with PKT and have some redundancy/safety net stuff too.

BL> IMO, the header ought to be a fixed length with spaces for padding.
BL> That way, you coud do it two ways.

The main trouble with that is extending. Which is what bites QWK on the
arse severely. Fixed length is certainly faster to process usually but
without an escape mechanism it can be a total pain in the arse as the
world moves on. An extendible format can respond gracefully to that.

RS> Assuming various things like the code which purports to be doing
RS> that really is doing that at all times.

BL> and if there is no error, it *has* to write an equivalent QWK message.

RS> Software aint like that Bob |-)

BL> This simple stuff is.

You havent been writing enough code yet Bob. It really is quite remarkable
how even the simplest stuff can fang your arse at times, usually when you
least expect it coz it looks so simple. Paul was doing that with that
PKT combiner. Sure the code looks simple. Problem is that you aint looking
at the real code thats run, you are looking at what you tell the compiler
you want to do and you have no idea if its going to choose to fang your
arse in the process at times. Ditto for the OS for that matter on something
as simple as saying 'give me the next file which matches this mask', it
aint guaranteed to always give them all to you at times.

BL> Writing code is very like designing a circuit;

Its not as similar as you might think actually.

BL> you stuff around until you find code that *wants* to work because
BL> it can't do anything else.

(Continued to next message)

--- PQWK202