On 10 Oct 95 06:49, Rod Speed wrote to Kieran Haughey:

Hi Rod,

RS>> I think one crucial thing is to do a rigorous check that a
RS>> particular pair really are a dupe, once the quick look flags
RS>> a candidate pair, and if there is any doubt, DONT deleted the
RS>> dupe, coz an occasional mistake like that which keeps a dupe
RS>> is FAR better than deleting what isnt actually a dupe at all.

KH>> That's a good idea, I'm wondering if creating a CRC of the messages
KH>> would actually make it easier and quicker to compare 

RS> That gets a tad tricky since you obviously need to only do the CRC on the
RS> part of the message that doesnt change. The most part that might do, but
RS> still qualify as a dupe, is if the PATH and SEENBYs are the only difference
RS> between a pair. OTOH its pretty simple to exclude those from the CRC.

 I think I sorta went a bit overboard for the CRC idea,
comparing the header, control lines and path/seen-by should be the quickest
method, although I am still yet to look at md5 which paul pointed me in the
direction of.. :)

KH>> The only idea I have come up with it checking the MSGID: kludge

RS>> Its the best candidate, but has some real problems. Its still nothing
RS>> like universally used, and there is some potential for accidental reuse
RS>> of a particular MSGID too, mostly when changing message creation system.

KH>> Also the problem is, how do you know that the MSGID will be unique,
KH>> I mean another program could just as easily create the same msgid
KH>> number as say msged so that sorta chucks that out the window...

RS> Well, the theory is that it cant, thats part of the spec of the MSGID,
RS> that it shouldnt use the same MSGID again for 3 years. OTOH if you apply
RS> that more thorough comparison of the candidate dupe pair if you do get
RS> a pair with the same MSGID, that would fix that residual risk.

Something I didn't realise at the time of writing the message was that
MSGID has an address in it along with the id number.. even if the id number
crops up in two different messages, it's most likely that the addresses
will be different..

RS> And say just dont dupe check messages which dont have any MSGID.
RS> That would then be a hell of a lot better than no dupe checking at all.

That's true.. and prolly what I would implement untill I worked it fully out..

KH>> and if it's equal to another messages
KH>> MSGID: kludge, then it will not process it..

RS>> Thats fine if you ALSO have a check on the other fields on a pair with
RS>> the same MSGID. That doesnt need to be that complex in this particular
RS>> case, if all the main header fields also match too once you have a pair
RS>> with the same MSGID, its pretty safe to assume its a real dupe.

KH>> Possibly, but as I said before, how do you know that it's unique...

RS> You dont need to if you only use the matching MSGID as a situation
RS> where you THEN compare the CRC of the pair of messages with the
RS> same MSGID to ensure that they really are the same message.

 true..
ÿ
Cheers
Kieran

3:711/413.17
@EOT:

--- MsgedSQ 3.25 alpha 14