On 2019 Sep 14 17:02:00, you wrote to me:

 JM> Just out of curiousity, what's the recommended way of dupe checking
 JM> these days, MSGID, TID/PID, CRC, combination there of (including Msg
 JM> Header)?

MSGID is the main way but older software doesn't generate MSGID so other
methods need to be used...

trying to root back through 20 years of memory, i recall one tosser takes the
header + 20 or 40 bytes of the message body and runs that through a CRC32...
that gets the header and generally most of the control lines... with this in
mind, i specifically wrote my message posting tools to put the MSGID at the top
of the control lines so it was pretty much guaranteed to be in that block of
bytes that tosser grabbed for its CRC32... i do no recall any of the posts made
by those tools being reported as dupes by any system in the network...

other tossers take a more involved approach and CRC the header, CRC the control
lines, and CRC the message body... maybe even MD5 instead of CRC... the problem
then comes from those systems that mistakenly reformat the messages as they
process them and write the reformatted messages to new PKTs... now the message
body is different and will fail CRC/MD5 calcs...

some systems specifically CRC the message body to avoid dupes that are
identical even with different headers and control lines... this is apparent on
systems that only get, for example, one posting of an echos rules each month
and only accept new postings of those rules IF the message body is different
than the last... i can understand this reasoning but it thwarts the idea of
monthly rules postings... it also has problems when the message is purged for
age but the CRC database retains the CRC of the purged message... so that
results in there being no area rules in the area even though they are posted
monthly... the only workaround is to add the posting date to the message body
which is kinda rediculous because the header already contains that
information... plus, in my case, there was a new MSGID in each posting as
well... but those monthly posts were still duped out because of the body being
the exact same...

what i would do would be to ask other tosser devs what they use in their
code...

listed in no particular order:

  tobias burchhardt  - fastecho
  rob swindell       - sbbsecho
  nick andre         - d'bridge
  vince coen         - mbse's tosser
  kim heino          - bbbs' tosser
  wilfred van velzen - fmail
  james coyle        - mystic

i'm sure there are others but these seem to be the most common ones being used
today...

)\/(ark

Once men turned their thinking over to machines in the hope that this would set
them free. But that only permitted other men with machines to enslave them.
... Clear out, or I'll set the cats on you
---