On Thursday, 96/11/28, Jonathan de Boyne Pollard wrote to David Noon
about "blockreading from a pipe" as follows:

JP>   It isn't the behaviour of any read API call on just about any
JP>   widely-used operating system on the planet.

Hi Jonathan,

This is the way synchronous reads work on MVS, nowadays called OS/390.
They block until the read buffer is full or some logical endpoint for
the device is reached. There are _many_ copies of OS/390 or MVS in use
on my planet. ... :-)

JP>   Why should you want OS/2 to
JP>   be so different ?  The POSIX 1003.1 read() call can return incomplete
JP>   buffers.  Why should OS/2's DosRead() not do so ?

Like most people who work the big iron, I couldn't give a monkey's
about POSIX. [Well, allow me some hyperbole.]

There is no reason why DosRead() should not return a short buffer. But
there should be a reason for it doing so, and one that is defined by
the reading thread, not the writing thread. The pipe should function
as a speed matching buffer, and if the writer is slow the reader
should be slowed down to match. More about this later.

JP>   Emptying the pipe's internal buffer *is* a logical endpoint.  As I said
JP>   in the last message, there is no way for the thread calling DosRead to
JP>   determine that there will _be_ any more data to be written.

The pipe's internal buffers are not a logical endpoint to the reader,
only to the pipe management routine.

There might not be any more data to read, but there is _always_ more
to come along a pipe that is still connected; there is at least a
disconnect signal. The read thread should be blocked until either the
read buffer is full or the pipe is disconnected. [I think I'll make
that sentence into a keyboard macro, I've typed it so often in this
thread.]

JP>   TEE command until the first program has run to completion and exited
JP>   (unless it fills the TEE command's read buffer).  This isn't very
JP>   satisfactory, especially if the program takes a long time to run.  Data
JP>   shouldn't "stall" in a pipe like that.

The read thread should stall. A pipe is an inherently serial stream,
so the reader cannot execute faster than the writer. This is why the
pipe should match the speeds of the 2 threads. It already does, in a
round-about way. The current semantics require that the reader make 2
visits to the API, if the writer is slow, to read what would otherwise
be a single block of data. By blocking the reader the data are
transferred in a single visit to the API.

The present semantics can be implemented under my suggested scheme by
using DosPeekNPipe() and reading all available bytes, then processing
the shorter-than-normal buffer.

To return to the MVS parallel, in days of yore datasets were read from
reels of tape. Large datasets occupied multiple reels. When a read of
a spanned record encountered the end of one reel and required the
mounting of the next reel, the read task (thread) remained blocked,
even though this required human intervention (SLOW!). There existed
control information (Segment Descriptor Words) that would allow the
system to return parts of the record, but the synchronous API (called
QSAM) remained blocked until the full record was available. Only the
asynchronous API (called BSAM) and lower level interfaces would return
the partial records. [This is an example of where a physical endpoint
is _not_ a logical endpoint for the device involved.]

This approach made I/O programming much simpler than performing
multiple I/O's and piecing together the record segments into a complete
record. The facility to read partial records without waiting for an
operator to mount a new reel of tape was still there, but only
desperados used it; I speak from first-hand experience. ... ;-)

DosRead() is much more like QSAM than BSAM, and should remain so.

JP>   For another example :  If you redirected the output of your C++ compiler
JP>   through a pipe to the TEE command, and the compiler produced one error
JP>   message early on, and spent a further five minutes processing the rest
JP>   of the file, would you want to see that error message as soon as
JP>   it was generated, or would you want to wait the five minutes until the
JP>   compiler had finished before you saw it ?

Actually, I wouldn't mind waiting. Perhaps it's my excessive years of
programming, since I come from an era in which we submitted compiles
on decks of Hollerith cards and received our compilation listings
hours or even days later. [ ... and getting to/from school was uphill
both ways, etc.]

I am accustomed to receiving all my compiler diagnostics that way, en
bloc.

JP>   Not true.  A serial device timeout is not a "logical end of
record".
JP>   It's just a gap.  It *could* be caused by the remote device coming to
JP>   the end of a "packet" of data.  But it *also* could be
caused by a delay
JP>   in datacomms equipment (such as a multispeed modem deciding to
JP>   renegotiate, or retransmission due to error-correction).

A serial port has a record length of 1 byte. Unless the modem
renegotiates in the middle of a byte and still returns a partial byte
it is at the end of a record. Packets of data are only physically
meaningful in synchronous comms, and BSC/SDLC/HDLC modems do not
renegotiate in mid packet.

In the asynchronous comms world "packets" are an artifice introduced
by higher level protocols such as zmodem. These take bytes and group
them into packets. The record of physical transfer from the serial
port is still only 1 byte long. The fact that DosRead() will perform
some of the grouping is simply a matter of convenience to the writers
of protocol modules -- _assuming_ they even use DosRead().

Regards

Dave


 * KWQ/2 1.2i * If Windows is User-Friendly, why do you need to read a 678 pg. ma
--- Maximus/2 3.01