04 Feb 18 16:45:10, you wrote to Markus Reschke:

 WV> On 2018-02-04 14:49:26, you wrote to All:

 MR>> The pseudo MSGID is created by a hash of the message text,
 MR>> fromUserName, DateTime, toUserName and subject. Please correct me if
 MR>> I'm wrong. The message text includes all the kludge lines like MSGID
 MR>> and also SEEN-BY and PATH. When I receive the same echomail from two
 MR>> or more echo links and the mail doesn't have a MSGID hpt would create
 MR>> a pseudo MSGID for all copies of the mail. And since every copy of the
 MR>> mail has different PATH and SEEN-BY lines each pseudo MSGID would be
 MR>> different. The result is that all copies of the echomail are
 MR>> considered to be different, i.e. no dupe found. I think PATH and
 MR>> SEEN-BY lines should be removed from the message text before creating
 MR>> the hash. This could be done by temporily exchanging the last char
 MR>> before the first SEEN-BY with a 0. This way the same pseudo MSGID
 MR>> would be created for each mail copy and the copies would be detected
 MR>> as dupes. What do you think?

 WV> Also the seconds in the datetime field should not be used, and the line
 WV> endings in the message body should be considered. Both are sometimes
 WV> changed in transit...

 WV> And the order of the kludge lines on top of the message might change, or
 WV> been added to (The 'RESCANNED' kludge for instance), so you better leave
 WV> those out too.

#3.3.3 DupeBaseType#
#
#Syntax:
#    dupeBaseType 
#Example:
#    dupeBaseType HashDupesWMsgId
#TextDupes
#    stores from, to, subj & msgid as text lines.
#HashDupes
#    stores src32 of from + to + subj + msgid.
#HashDupesWMsgId
#    same as HashDupes, but stores also msgid as text.
#CommonDupeBase
#    stores hashes of from + to + subj + areatag + msgid in one file (hpt_base.dpa)
#
#Default is HashDupesWMsgId.

Perhaps the pseudo msgid should be created only from the body without any
header of klugde lines. To, from and subj are already used even if msgid
exists or not.

'Tommi

---