RS> Nope, not the message body, the text the user enters.
RS> THATs what the SOT and EOT do thats different, a completely
RS> rigorous and unambiguous start and finish of the user text
RS> that any message processing software can be quite sure contains
RS> nothing that it needs to analyse for header detail wise.

RS> The principle is fine, the problem is introducing that now
RS> when most messages dont use them.

PE> It is not a great problem, because what it does allow is for
PE> the traditional method to work better.

Sure, I said that a bit cryptically. There certainly isnt any
problem for current message processing code. The problem is
that if you are writing new code, your alg has to still be
written for the messages which dont have SOT and EOT.

You only start to get a big benefit in the code when the vast bulk
of the messages use them and you can treat the ones which dont as an
unusual exception. Even then, you can still argue that the code is
most efficient if it doesnt worry about SOT and EOT until no messages
dont have them. Its a very real problem for any protocol which grows
over time and which needs to have full backward compat. You really do
have to do a lowest common denominator approach most of the time.

PE> Traditionally you have to search from the end backwards,
PE> skipping SEENBY, then Origin line, and then you will probably
PE> find a tearline. Now, if a tearline is missing, you will instead
PE> find an EOT, whereby you are meant to stop looking. Traditionally,
PE> you would have continued looking for a "---" in the user text,
PE> or at least if you had just run over a blank line you would have.

Yes, but thats only for the messages which have an SOT and EOT and
THATs the problem, most dont.

PE> It is really a pigs swill as to what exactly you are meant to do here.
PE> SOT/EOT serves a lot of its purpose just by being there.  E.g. if
PE> SOT/EOT is used, then you will never find "AREA" as the
first line in
PE> a netmail message. Anyone who doesn't use SOT/EOT has to rely on the
PE> user being a really nice person and not sticking "AREA" as the first
PE> line of their netmail messages. That is satisfactory IMO.

Yep, no argument at all about the desirability of a completely definitive
pair of markers bracketing the user text. Its the only way to conveniently
handle the situation where some dork has forwarded a message and its got
the old origin line etc in it.

RS> Nope, its much easier to have something which is guaranteed to
RS> be the end of the user text. Anything else is farting around
RS> detecting when the user text stops and when trailing header
RS> detail starts. Particularly for stuff like forwarded messages
RS> which may well have stuff like real origin lines in the user text.

PE> Yeah, this is the big problem.  Mail readers should be stripping
PE> the Origin and tear lines completely, the same way they strip the
PE> INTL line.  Failure to do so means that people keep shipping these
PE> control lines around with their messages.

True, but thats not that easy to guarantee in a mail reader coz most
allow the use of an external editor and the user can be including a
message manual when in the external editor too. Sure, the mail reader
can in theory scan the message as its handed back from the external
editor to strip or remap some chars in that, but its much more elegant
to just stick an SOT on the front, and EOT on the end and then you
dont give a stuff whats in between them. Far quicker too.

RS> The only problem is that SOT and EOT are not mandated so you
RS> have a real problem when most mail doesnt have them. BUT, like
RS> I say, thats easy enough, just process mail today without relying
RS> on them. If they ever do become universal you can do it differently.

PE> You can simply integrate it as a small part of your processing.

Its not actually that easy as Bob is discovering. You really do get
a significant advantages while ever its not used much to just write
the code as if they dont exist and then just say drop them from the
discovered user text if you arent dropping all kludge lines.

RS> Ditto for taglines. Many people think they are a waste of
RS> bandwidth. Fact remains, heaps use them and THATs what matters.

PE> Taglines are not part of the FTS/FSC standards and are user-text
PE> as far as fidonet is concerned.

Sure, but thats just another manifestation of the same problem, its a
standard which has grown over time, quite a bit of the growth has been
in areas which FTS/FSC isnt concerned about. They arent really user
text tho coz the user didnt usually enter them. Same for the software
brag lines etc too. The really are header type data, not user text.

For example in a message reader it may well be very desirable to strip
that stuff so its not visible when the person is reading the user text
and have it say on a separate page of header type data, auto extracted
into its various fields. For example it would be quite handy to auto
strip the node number and even say look that up in a nodelist so that
at a single keystroke the user could see a display of real zone name
and the nodelist data for that node.

You might want to say have a system which purves at the software brag
lines and auto notifys you when it sees a new version, say sees that
GEdit has jumped a version.

Ditto for the complicated multiline origin like stuff you see with
gated feeds. It might be handy at times to be able to auto chop that
stuff into various fields. And auto netmail reply in a message reader
may want to do that sort of thing.

Obviously if that stuff is clearly delimited off from the true user
text, it can be processed like that if someone chooses to do that.

PE> It's the user's responsibility to not generate taglines if they do not
PE> wish them to come out the other end. Ditto for multi-line signatures.

I wasnt talking about the users desires, I just meant that since
they are commonly used, all message processing software needs to
react gracefully to it.

RS> And the reason its not very useful for that is because you have
RS> what is clearly not normal user text above it. Thats the way the
RS> system has grown over time and there is absolutely nothing you
RS> can do about that now.

PE> It can change Rod.  The INTL was added on as an afterthought,
PE> and not compulsory, as an example.

Its different in the sense that without it you are stuffed to do
some particular thing. Those always do get adopted much more easily.

PE> I would expect SOT/EOT to remain the same - optional, easy to
PE> generate, and can be used as a small part of the logic used to
PE> find origin lines and tear lines etc.

The fundamental problem with it is that the code which analyses message
has to not bother with it much, analyse the messages as if they arent
there for the foreseeable future.

--- PQWK202