Original from  Mike Bilow  to Denis Tonn on 06-09-1996
Original Subject: Toolkit 1.3 from Devcon

                         ---------------------------------------

 MB> been swapped out.  However, the swap manager finds no table 
 MB> entry for the requested page, since it is uncommitted, and 
 MB> gracefully returns an error code.

  DT> Actually, it traps. Trap 000E to be exact.

MB> Well, yes, Trap 000E is "Segment not present."  However, 
MB> you get a Ring 3 trap which affects the process, not a Ring 
MB> 0 trap which kills OS/2.  I consider this to be relatively 
MB> graceful.

 Yes, it is much more graceful than halting the system. 
 BTW: A segment not present is a trap B. Trap E is a page fault (not 
pressent being the most likely cause). 

  DT> I have a lab for my  classes that does exactly this. You cannot 
  DT> access memory that is  uncommitted at all. If you have an 
  DT> exception handler in place, it will receive control with an 
  DT> exception report record and context record  on the stack. Your 
  DT> handler can then decide what to do.. But I  digress.. .. 

MB> Actually, it is perfectly legal to set up a handler to 
MB> intercept Trap 000E at Ring 3 and try to commit the memory 
MB> in question.

 Certainly. Sometimes it is the exception handler that traps while
processing the exception. Sometimes the only way you can determine the
"original" cause of the trap is by looking at the exception
report and context records on the stack. Just like the exception handler
code would be doing. Again, this is digressing from the original topic. 

 MB> This is not quite accurate.  The least significant two bits of 
 MB> a selector code its privilege; any selector ending in 7h is 
 MB> therefore a Ring 3 selector.  This is an Intel hardware issue, 
 MB> not an 

  DT> This exactly what I mean. Here is a small cut from a dump.
  DT> You can  see the same thing under the debug kernel: 

  DT> # dg 28
  DT> 0028  LDT     Bas=a8ea7000 Lim=0000ffff DPL=0 P   

  DT> # dl 7
  DT> 0007  Data    Bas=a8ea7000 Lim=0000ffff DPL=3 P  RO        

  DT> Note that selector 7 (and it *is* an application accessable
  DT> data  descriptor, abet read only) is the same base address
  DT> at the LDT.  

MB> Keep in mind that the debug kernel makes no attempt to 
MB> validate that the selector you are aiming for is actually a 
MB> selector at all.  You can ask for nonsense and get nonsense 
MB> in reply, and it is up to you to know.  That said, and 
MB> without a debug kernel running in front of me, I would 
MB> agree that your dump above does certainly look as if 0007h 
MB> is a RO Ring 3 *alias* for the LDT.

 Yes I am aware of the limitations of the debug kernel and the dump 
formatter. But the above is easily verified by looking at the LDT
index information itself. Just follow the chains of what the hardware 
MUST be using LDTR=28, GDT 28 base address points to the current LDT. 
Index 1 (selector value 7) of the LDT also has this same address (and
limit) as GDT 28. 
 Yes, selector 7 is a RO data segement alais for the LDT. Usable from 
Ring 3 apps. 

 MB> OS/2 issue.  Buffers passed from Ring 3 to Ring 0, as arise 
 MB> in a DosDevIOCtl() call, will generally be mapped to 
 MB> selector 0017h.  Temporary selectors mapped by 

  DT> Selector 17 will tile to a flat address of %20000 - always.

MB> Perhaps that is true in a process context, but OS/2 uses 
MB> 0017h for a number of other things, especially inside Ring 
MB> 0 code.  Several of the DevHelp calls depend upon the 0017h 
MB> being remapped.

 Are you talking about strategy or interrupt? There is no way of predicting
what the current context will be at interrupt time. I would certainly hope
that a device driver does not depend on the context of selector 17 during
interrupt time. 

 Can you give me an example of one of the DevHelps that depend on a
specific mapping of selector 17? You have made me curious. 


 MB> Processes are not supposed to be able to read their own LDTs.

  DT> But in fact they can. It is an analomy, but selector 7
  DT> *does* always  point to the base of the LDT. 

MB> That's an interesting point.

  DT>   I finally did some testing on the subject. All unique
  DT> allocations  still occur on 64K boundries (at least up to
  DT> FP17) even without the  tiled bit.

MB> It is documented that all extent versions of OS/2 on Intel allocate tiled 
MB> memory even if not specifically requested.  This is 
MB> supposedly subject to change, but certainly not in FP17 and 
MB> certainly not without considerable wailing and gnashing of teeth.

  DT> But I experimented a bit more. I allocated 3 blocks of  memory.
  DT>  The first test without the commit flag, and only 1 byte in
  DT> size.  Since the minimum allocation unit is 4K, I still one
  DT> page - but  uncommitted. I then proceeded to setmem 8 pages
  DT> without error.  Note that this goes well beyond the original
  DT> address (yes address!) allocation. 
  DT>  Then I allocated 1 byte WITH the commit flag, I had
  DT> immediate access  to the first page and was able to setmem
  DT> an additional 5 pages (as far as I tested). Still able to
  DT> setmem more than I "allocated".   Then I allocated 32K
  DT> without the commit flag, and commited the first  4 pages of
  DT> this and caused a trapdump (I am getting good at that) to 
  DT> easily see the tiled address in the LDT. The tiled addresses
  DT> only  showed 4 pages mapped in the LDT. 
  DT>  Lastly I allocated 1 byte without the commit flag, then
  DT> tried to  setmem the pages until the setmem failed. I was
  DT> able to cleanly setmem the first 16 pages of the returned
  DT> address range but failed as soon as I crossed the next 64K
  DT> boundry. 

MB> I think what you have proven is that DosSetMem() is broken!  The real 
MB> question is not how DosSetMem() is polluting the PTEs, but 
MB> rather what happens if you actually try to access the 
MB> memory supposedly being committed.  In theory, DosSetMem() 
MB> should return an outright error if you try to commit 
MB> unallocated memory, but I suspect no one made a big effort 
MB> to test this at IBM.  Among other things, the success or 
MB> failure of such an operation will depend upon the 
MB> accidental or coincidental contiguity of allocations.

 "Broken" it may be. This may be what the updated CP Ref is
trying to clarify. But the memory is accessable and usable. I can show you
the PTE's from accessing the locations if you wish. I can also show you
trace entries for trap E and the allocs and sets. 

 DosSetMem *does* return an error if you try and setmem and address beyond
the 64K boundry that you are in. Again, I can show this easily in a trace
or a simple piece of code can be written to show this. 

 What it seems to be doing is that the minmum ADDRESSES allocated are 64K,
while the minimum MEMORY that can be committed is a 4K page. 

  DT> Which leads me to wonder what the above comment is all
  DT> about. I  suspect that they are "preparing" us for changes
  DT> in the memory  allocation method. And possibly removing the
  DT> default (and automatic)  allocation on tiled (64K)
  DT> boundries. 

MB> There is no important reason to remove automatic tiling under OS/2 on Intel.  

 Hmm.. For most programs, but there are some that need larger than the
current defaults. Also, on a LARGE system it is possible to be using a lot
of shared addresses, which reduces the amount or private addresses
available for applications. There is a guarenteed minimum of 64MB available
for each of private and shared addresses, but then they grow toward each
other. 

MB> It doesn't really hurt anything, and changing it would 
MB> probably break a lot of code that is technically incorrect. 

 That depends on how it is implemented. I suspect that when they remove the
automatic tiling it will require a special bit in the EXE header (linker
switch), *and* that all "untiled" addresses will be allocated
above the 512MB line. 

MB>  However, the reason for the documentation warnings about 
MB> automatic tiling being subject to change was because of 
MB> OS/2 for the PowerPC, where neither tiling nor 16-bit 
MB> interoperation were much of an issue.  It would have been 
MB> foolish for OS/2 on PowerPC to have preserved this 64 KB 
MB> tiling in order to accommodate buggy programs which depend 
MB> upon the idiosyncratic behavior of OS/2 on Intel.

 You may very well be right. But the comments in the later CP Ref and some
other rumblings I have heard would lead me to suspect that the changes in
the API description could just as well apply to the removal of
"automatic tiling" for selected applications. In the case of
these applications the comment about the 64K vs 4K may very well be
correct. In fact it may already be implemented in Connect, but for memory
allocated above the 512MB line. I just need to find the switch or DEF file
parameter to correctly link the app so it will allocate memory above the
512MB line (if this is possible under Connect).. 

  More digging to do.. Hmmm... 


   Denis       

 Certified OS/2 Engineer, Certified OS/2 Instructor, Certifiable....
 All opinions are my very own, IBM has no claim upon them

.
--- Maximus/2 3.01