RS> The SOT and EOT allows you to know totally unambiguously that
 RS> they are real origin lines etc, coz that are absolutely certain
 RS> to be user text coz they are inside the SOT EOT pair. THATs the
 RS> whole point of them.

  I've been through this with Paul so many times I'm getting sick of
it. The *last* origin line is the real one. It is put in *after* the
user has finished typing, so it always delimits the text... except in
netmail that has nothing anyway so it doesn't need an EOT.

  At the start of the message, the first line will always read
something: either "AREA:" in ordinary email, or INTL, TOPT or FMPT in
netmail. Why is the SOT required? One or all of those will do the same
thing, by being added when the message is processed, and delimiting
the text.

   I don't really object to adding a SOT, because it doesn't do
anything, but the EOT actually interfers with the Origin line that
*has* to be shown in the message field. Without EOT, you can look for
the SEENBY or the * Origin to mark the end fo the message field. With
EOT, you have to do all three, and put the tear line and origin line
back in. Faced by Paul's EOTs in his packets, the first thing I do it
erase them!

  Paul's idea is to protect the message field, so that C code remains
sacrosanct, but SOT/EOT won't even do that, when the origin line has
to be added later. 

  I agree that we need a protected message area, but only because I
look forward to the time when we transmit graphics over the BBS. In
this case, 0x01 will not be protected, and SOT and EOT (and other
kludges) will be useless. If we have to change the system, and we do,
then we should make a change that allows *anything* in the message
field, including graphics. The only way to do this is to expand the
header, and make it a fixed length, with a longer keyword than just a
null to delimit the end. The existing " * Origin" would do that well,
as long as it is used in netmail too. ^aEOT would do alright too, as
long as it is by itself, without tear lines, origin lines, SEENBYs and
PATHs.

 RS> Using that theory there is no point in SOT and EOT at all and
 RS> thats crap. There is some problem with it being introduced now,
 RS> but its quite silly for example to suggest that it gives no
 RS> benefit if it had been say part of the original design.

  Ahh... that's a different question. The problem with PKT format is
that the message header does not allow room for:

  Area identification
  Netmail identification
  Origin and destination Zone
  Origin and destination Point
  SEENBY
  PATH

  All this should go in the header, with room for expansion. It is
interesting to compare it to the 128-byte QWK fixed headers padded by
spaces. When you zip QWK, it ends up smaller than a zipped PKT because
the spaces compress to nothing. If we allowed a fixed 256-byte message
header in PKT, padded by spaces, it would give tons for future change,
and clear out the message field so that only message was in there.

 RS> You are completely mangling the question of the value of that
 RS> approach with what you have to do if its added later, when no
 RS> one is currently using it.

  It has to show a benefit, or no one will use it. If we base a new
standard around graphics, then anyone using the ^a kludges in the
massage field would be inmcompatible for graphics, but able to use
text. This is the way changes should be made... like colour TV. It
doesn't take long before everyone upgrades.

 BL> If someone writes "AREA:" on the first line, or " *
Origin" on
 BL> the last line, it hardly matters. The mailer will write another
 BL> AREA: in front, and another * Origin behind. All we have to do,
 BL> is read the first one, and the last one, and we get exactly the
 BL> same result as SOT and EOT. You are wrong.

 RS> Nope, an origin line in user text does indeed break some mail
 RS> processors and the unambiguous delimiting of user text with SOT
 RS> and EOT allows you to know very easily that its just a quoted
 RS> origin line.

  That's Paul's argument, but it's got a huge hole in it. If the
mailer roots the existing system, then change the mailer. You have to
change it anyway, to incorporate SOT/EOT!

 BL> You're kidding! Add kludges in the message field? Wot rot!

 RS> They arent in the user text, they precede and terminate it.

  The *message field*, not user text. They aren't the same thing. The
message field is delimited top and bottom with nulls, and then filled
up with kludges. We need to get all the kludges out of the message
field and into the header.

 RS> That approach doesnt work because it inevitably breaks when you
 RS> have exhausted that fixed header. Thats precisely the problem
 RS> with the current QWK format, its fixed, its not readily
 RS> expandable.

  QWK relies on fixed 128-byte records right through the packet. I
only want a fixed messager *header*. It would be easy enough to build
expansion into that header. My idea is to use the last "Subject" field
that precedes the message field; expand it, delimit it with ordinary
CRs to add the extra fields at the end, and then fix it with plenty of
spare room up front padded with spaces that compress well.

 BL> The necessary origin line would be constructed from one of the
 BL> header fields,

 RS> Thats got some advantages for the fixed fields in it, the node
 RS> details, but doesnt help with the rest.

  The "rest" has to come from the mailer cfg file, which actually
writes the header anyway. This is the *message* header. Most of it
comes from the original mailer. Only the seenby and path lines would
vary as the message goes the rounds.

 RS> AND your idea isnt readily usable in tandem with the current
 RS> system anyway. Yes, its possible to design a whole new message
 RS> format from scratch, but we arent looking at one of those and
 RS> how to do it, we are actually looking at something which can
 RS> enhance and be used with the current one.

  I agree. It has to be backwards compatible, like colour TV. Old
mailers would work with the new header. All we do is lengthen the
"Subject" field. The extra stuff goes on the end, delimited by CRs.
Old mailers wouldn't even know. New mailers would write both the new
header and the old kludge, but old mailers would break high ASCII,
control codes, and graphics.

 RS> Coz a whole new one from scratch has buckleys of ever getting
 RS> everyone switching to it. And if you are going to make a
 RS> complete step away from the current one, you first have to show
 RS> that one of the true standards like X.400 isnt usable. No point
 RS> in generating a whole new one if it is.

  It would have to be Type-2 compatible. An entirely new standard
would be the preferred option, but these things have to happen slowly,
in stages...  either that, or a completely new net.

 RS> It would actually be quite interesting to do some tests and see
 RS> if there really is any point in the CONTROL.DAT having area
 RS> names in it.

  QWK identifies the message AREA with a simple binary number that is
just two bytes, but you need to carry the conversion list in the
packet whether the whole list is used or not, or establish a unique
number for each AREA, world-wide. PKT uses "AREA:LOCUSER" which takes
up 12 bytes per message, regardless. If your average is 100 messages,
that's 1200 bytes for AREA in PKT against 200 in QWK, and a typical
control.dat is 350 bytes. But who gives a shit? We're talking about
1000 bytes in 100Kb. Processing time is about the same.

  I prefer QWK because it defines netmail as just another area. PKT is
a bit shaky on netmail, and the AREA: kludge spoils the message field.
     

 RS> There isnt all that much in it with the user text proper
 RS> whether its done with fixed fields padded on the end or a
 RS> delimited variable length field. The specified number of fixed
 RS> 128 byte fields is just another way of specify the length of
 RS> the user text so it can have anything at all in it. But you
 RS> could have the user text variable length with the actual number
 RS> of bytes in the user text specified instead, no need for fixed
 RS> fields.

  Yep. No problem with that. QWK is a bit queer with its ndx files.
Knowing the position of each message in a conference makes it easy to
sort, but the essential data is in the headers anyway. Processing the
ndx is surprisingly fast (it surpised me, anyway), and they take up
very little room. On the whole, I think I prefer QWK as being more
robust, but like PKT format it lacks room for seenby and path, and the
128-byte header is full up. Its problem is that with multiple QWKs,
you are odds-on to get the control.dats and ndx files mixed up! At
least PKT is self-contained.

 RS> And you can even argue that if you are going to do a whole new
 RS> message format, you really need to consider why its not better
 RS> to use one of the database formats. Coz they should be stored
 RS> at each end like that anyway, so whats the point in something
 RS> different in the packets which are transported between them ?
 RS> Not much basically.

  Yes... but I think any new system ought to be able to handle
graphics. The NAPLPS stuff meant for Teletext looks good to me.

 BL> That's what I mean by "reading right through." In my final
 BL> program, it takes 30mS to convert a message. 15mS of that is
 BL> spent converting the PKT cr's to Pi in QWK. 5mS is spent
 BL> erasing SOT and EOT, or their little ^a friends.

 RS> This is where you are having a brain fart. You have to drop all
 RS> the other kludge lines anyway. It takes no longer to drop SOT
 RS> and EOT TOO.

  Har! That's quite right, but the only use for SOT and EOT is when
none of the others are there.

 BL> What the point of converting the CR's to Pi, anyway? I'm
 BL> tempted to leave it out, and send the QWKs with embedded cr's.

 RS> Its a rather more complicated question that it looks. Some QWK
 RS> readers do use them sensibly. There is a difference.

  What's the reason then? I've never seen an explanation that made any
sense. It certainly gets stuffed up, if another editor adds its own
CRs and re-wraps the existing text.  If the Pi-copnversion is meant to
prevent that, it doesn't work. 

Regards,
Bob



___ Blue Wave/QWK v2.12
@EOT:

---