BL> Your pkt2qwk compares the size of the PKT to the QWK, and I rather
BL> like that. I could check the ratio, and if I lose 10% (say), I
BL> could add a warning and not erase anything until the user says so.

RS> Trouble with the way its done now is that it assumes that
RS> the combined PKT really is the total of the PKT mail which
RS> was in the archived PKTs. There is no guarantee that it is.

BL> It's a pretty silly check, really, they way it's done in sequence.
BL> Each PKT message writes a QWK message, and the totals are going to
BL> be about the same anyway, even with errors.

Thats only true if you count PKT messages in the PKT>QWK code.

RS> If you are going to do that sort of check it would be much
RS> better to look at the saved archived PKTs with an archiver
RS> etc to get the total size of the PKTs in there from the directory
RS> info in the archive and compare that with the resulting QWK.

BL> I agree, but it's a problem under VB, that doesn't want to
BL> know about the size of unopened files. I suppose I could
BL> open then first, and read the length... I'll give it a go.

I actually meant using the size data thats in the archive file embedded
directory. You really should be doing the calculation on the unarchived
size, not the archived size, coz the packing ratios vary quite a bit
between PKT and QWK since the common strings are different.

RS> Trouble with the size comparison tho is that it cant hope to
RS> pick up more than gross problems, wont catch say a single message
RS> lost for some problem with the message boundary detection etc.

BL> I was trying to work out a way to double-check message ends
BL> and begins, but in the end I decided to let it go. If it misses a
BL> field, it soon stuffs up in a big way, and the error routine finds it.

The whole point tho with overall checking is a safety net if it doesnt.

BL> All I want to do, is print a warning message, abort the conversion,
BL> and save the original packets.

Thats a problem in some ways. There is a lot to be said for trying to
carry on regardless if a fixup does appear to have got back into synch,
notify the user and certainly keep the originals.

BL> What I've done is safe, but a bit of a nuisance. I save the original
BL> archived packets to a "sv" directory, and remove them from the input
BL> "rx" directory so the program won't run a second time until you copy
BL> from the sv back to the rx.

Yeah, I think thats the way to go myself. Maybe even allowing some user
specified safety protocol like say keeping the last weeks PKTs etc.

BL> You'd have to find the faulty packet yourself.

Well, I think it should give you its name, particularly if its processing
PKTs individually. In other words if its not using an external PKT join first.

RS> Nope, you missed my point. You want a completely separate standalone
RS> ute which counts the messages in the PKTs, ideally with a completely
RS> separate algorithm too. Then the comparison of what it counts with
RS> what your PKT2QWK converter counts in the QWK produced really is
RS> useful.

RS> The counting ute isnt hard, pretty trivial compared to the converter.

BL> Actually, the counting ute would be nearly the same as the converter.

Nope, nothing like it for many protocols. You just have to detect the
start of a message. You dont give a stuff about anything else. Anyway,
the whole value is a different alg. No point in having the same one.

BL> I was able to combine pktjoin with no overhead at all.

Yeah, its not that hard to process multiple files instead of one often.

BL> PKT format has a 58-byte packet header you ignore, and a 14-byte
BL> message header you also ignore, because there are nulls in there.
BL> Then you count nulls to find the fields and finally the message
BL> terminator. The converter does this anyway, but adds processing that
BL> actually reads the fields, reformats them, relates them to the nulls,
BL> and identifies the message that it writes into the QWK. If you skip
BL> a null or find an extra one, it soon stuffs up and grinds to a halt.
BL> A simple counter would be less robust than the converter itself.

Yeah, like you said, its a pretty fucked format. Rather more fucked
than QWK in many ways. Its main advantage is that its much more
extensible than QWK and not so limited on the field lengths.

Fact remains tho, for a reliable safety check message counter
in PKTs, you need a completely different alg, new code, the works.
I'm not convinced its particularly difficult.

BL> You could test the input packets by counting nulls and dividing
BL> by five, but the converter does this anyway, and more,

Assuming various things like the code which purports to be doing that
really is doing that at all times.

BL> and if there is no error, it *has* to write an equivalent QWK message.

Software aint like that Bob |-)

BL> I think I'll try reading the size of the PKT archives and comparing
BL> them to the QWK, as you say.

The fundamental problem tho is that will never detect a small number
mismatch in the messages. Its just aint ever going to be possible.
Its fine for a gross problem, but even with our current PKTs, its
going to be quite possible to drop a whole PKT from someone who has
only entered a couple of messages and never even notice they have gone.

BL> ...[later]

BL> Well, that was easy. The QWK seems to be about 7% smaller than
BL> the sum of the packets. I'll set the limits at 0% and 15%...
BL> plenty to find a dropped packet if UNZIP buggers it.

Nope, not with a small PKT it aint.

--- PQWK202