Hi, Paul.

PE> FM> Of course it is. Is it possible, in general, to scan a PKT and work out
PE> FM> where to put SOT/EOT?

PE> No.

I know believe that to be true, in the very strictest sense. There is
one possible circumstance, which wouldn't even need brain-dead software,
where you could not reliably place a ^aSOT:. There are circumstances
which would make it impossible to reliably place a ^aEOT, but I think
the s/w which originally created the message would need to be very
brain-dead and I would be surprised if there are any messages in
existence to which that would apply. And of course I exclude data
corruption or deliberate construction of a message which conforms to
some possible reading of the spec but to which that comment does not
apply. :-) These comments are explained further below.

PE> FM> If not why not, what stuffs you up from doing that?

PE> The below...

Thank you for the below. I perused it very carefully and then decided to
write a reader so I could get to grips with the problem first hand.
Let me try and put in my own words, so that you can criticise it and
hopefully I'll learn.

1) The SOT/EOT spec is designed to overcome the problem that some
information, useful to a mail reader, is actually indistinguishable from
user text. I use the term 'mail reader' to refer to anything which reads
packets, which may be a full mail reader, a tosser, a program which runs
somewhere in the network to extract and send messages to a(/some)
destination or some utility program such as a statistics keeper or
something which converts .PKTs to say .QWKs.

One of those cases is perhaps slightly different, in that a program
which is initiating or on-forwarding messages into the network must only
generate legal messages. I have not considered any specific problems
that might create with the following, but I *think* none.

2) I assume that 0x00 can never appear in user text because that
terminates a message.

3) I assume that 0x01, when it appears as the first character of a
message or immediately following 0x0d or 0x0d0a, will always indicate a
'kludge' line. You may or may not know what to do with that line, but if
you don't you can bin it (or pass it through). If 0x01 appears within a
line you can ignore it. Maybe it *shouldn't* be there, but you can
ignore it.

4) Therefore all lines starting with 0x01, *wherever* they are between
the start of the message text and the terminating 0x00, can be ignored
for the rest of this discussion. (That doesn't mean you would ignore
them in a program, you will probably extract useful information from
some of them.)

5) That leaves the lines which don't start with 0x01 but which
nevertheless contain information which is useful to a reader program.
AFAIK (at this point) the only lines which fit this case are
  AREA:
  SEEN-BY:

In some of your messages and also in "the below" you include the
tear-line, and maybe some blank lines, as "control" lines, because (if I
understand you correctly) they were not written by the user but
generated by software. IMHO they may (or may not) be useful for locating
other "control" lines but otherwise do not contain information useful
to a reader. I therefore ignore them in this discussion (but I might
have to come back to that, specifically the tear line when it comes to
generating 'legal' messages).

The problem is that those 3 lines (area, origin, seen-by) could also
occur within user text (and should be permitted there), and this has the
potential to stuff up a reader.

Let's look at each case.

6) The AREA: line, if I understand it correctly, must be the first line
if it is present. In FTS-0501.002 you imply that it must be before any
0x01 lines but this could perhaps be relaxed in a reader. A possible
spec for finding AREA: would be
  It's a valid area line if you find AREA: at the start of any line
  before you find a line which doesn't start with AREA:, excluding lines
  which start with 0x01 and maybe blank lines ("lines" consisting of
  0x0d or 0x0d0a only).
This fails in only one case which I can think of - a netmail message,
which therefore doesn't have an area line, but in which the user has
written AREA: at the start of the very first line in his text. Therefore
we must look for AREA: before *any* other lines, and THERE MUST BE AT
LEAST ONE 0x01 LINE. If there is no reason to generate any other 0x01,
then you might as well generate ^aSOT: and therefore you might as well
generate it *all* the time.

I think that is sufficient to make the case for ^aSOT:

(But you could validly argue that logic which says "write a ^aSOT: line
if I haven't written any other 0x01 lines before the user text, and I
haven't written an area line because it's a netmail message" is OK too.

7) Because most software does not generate ^aSOT: (yet, maybe ever) you
can't look for it (and should ignore it if you find it), but because
it's there when it is you can be confident that if you find a line with
AREA: then it's a real area line, if it came from such software. There's
*nothing* you can do with messages from software which doesn't generate
^aSOT:, happens not to generate any other 0x01 lines before the user
text, and the user types AREA: at the beginning of the first line of a
netmail message.

8) SEEN-BY: lines will always be the last lines in a message (remember
we're ignoring 0x01 lines) if I understand correctly (although see next
para). Therefore any line starting with SEEN-BY: after which there are
no other lines except 0x01 lines or other seen-by lines is a valid
seen-by and can be treated as such.

What about a user who types a line starting with SEEN-BY: as the last
line of his message? (And that's not so unlikely, maybe he's imported
another message including the seen-bys.) I note your comment that FTS-4
doesn't actually require that there be an origin line, nor mandate the
order of origin and seen-by lines. So we have 3 cases: new software
which we can make do it "right" (say by generating ^aEOT, or putting the
origin & seen-by in the "accepted" order); old software which follows
the accepted order; and old software that doesn't (and maybe doesn't
even generate origin and/or seen-bys).

There is nothing you can do about the last case. There is nothing you
need to do about the second case. In the first case you can simply (as
the originator of the message) write the origin line last or (as an
intermediate processor) write all seen-bys at the end of the message.

No case for ^EOT: here.

9) As a packet reader you treat any line starting with SEEN-BY: as a
valid seen-by line if it's followed by no lines other than other
seen-bys, 0x01 lines, or blank lines.

it's followed by nothing except what's in the above paragraph.

And you treat any line starting with --- as a valid tear line if it's
followed by nothing except what's in the above 2 paragraphs.

If you do the above the only circumstance which will catch you out is a
line(s) of his text, and is using software which does not create tear,
origin or seen-by lines. And you can't do anything about that anyway
except ask him to update his software.

SUMMARY

There is no case for ^aEOT:. You should generate tear and origin lines
in that order at the end of your message, and add seen-by lines to the
end of a message.

There is a case for ^aSOT:, certainly if you have generated no other
0x01 lines at the beginning of a netmail message.

Paul I will be most interested in your comments. I will now go and read
the rest of this ("the below") again to see if I've covered everything.
I'll probably chomp it unless there's some points I haven't covered
above.

Then I'll go and reply to the other dozen or so messages on this topic.
:-)

Regards, fIM.

[...]

PE> Also some people have
PE> seen fit to exacerbate the problem with control lines such as
PE> "# Origin".

I didn't know that. Should one look for Origin: as well as
Origin:?

PE> 5. FTS-4 does not specify the format of the address in the
PE> origin line.

So one should look for "nD", n = 2..5?  And generate what?

PE> 6. FTS-1 mentions x'8d' when it is totally irrelevant to the
PE> transport layer, and has been interpreted by some people to mean
PE> that x'8d' is allowed to be deleted on in-transit mail.  If it
PE> mentions it at all, it should be "people used to use x'8d' as
PE> a formatting character, it is not required, and this practice
PE> causes problems so do not use it for this purpose in new
PE> products".

PE> 7. x'8d' is used for both soft-CRs and as a national character
PE> by various countries, including Russia and Japan, and there is
PE> no way to tell which is which.

I'll come back to these when I consider that other stuff under the
"funny characters" subject line, and have thoroughly perused FTS-1.

PE> 10. FTS-1 does not specify any maximum length for kludge lines.
PE> This means that every man and his dog defines an artibrary
PE> maximum (and no, it isn't 10 gigabytes) when they write their
PE> program.

Does this actually matter? If a kludge line is one you don't understand,
or it's one you do but is longer than you're prepared to handle, just
bin it or pass it straight to the output, depending on what sort of
program you're writing.

PE> 11. FTS-4 does not specify a maximum length for the AREA,
PE> SEENBY, tear and origin lines.

I thought seen-by was specified? 69 or something? Same as path?

PE> SEENBY - start searching from the bottom of the message.  Go back
PE> one line at a time until you get a SEENBY.  Then go back one line
PE> at a time until you get a line that DOESN'T start with SEENBY.

Well I wouldn't do *that*. Reading backwards is for the birds, and
unnecessary. But I understand what you're saying and agree with your
logic - I just wouldn't be reading backwards, it's not necessary.

 * * My guru told me there would be lifetimes like this...
@EOT:

---