Original from  Herbert Rosenau  to Eddy Thilleman on 01-07-1999
Original Subject: more power

                         ---------------------------------------

 Between being busy and a mail feed interruption, it's been a while 
since I have looked at these discussions. Apologies for a late (and 
possibly inappropriate) response. 


 HR>>   3,5GB +-------------------------------+
 HR>>         |       reserved for            |
 HR>>         :       feature use by          :
 HR>>         |       32 bit apps             |
 HR>>   512MB +-------------------------------+
 HR>>         |       memory for DOS,         |
 HR>>         |       OS/2 16/32 bit apps     |
 HR>>     0MB +-------------------------------+
 
  ET> Denis Tonn told me that the reserved memory (in OS/2 v2 and v3
  ET> GA) is located from 448 MB to 512 MB in the virtual address space.
 
HR> Yes - but not SYSTEM-wide. The table above describes the system addressspace.
HR> It is the whole address room a CPU like I80x86, Pentium... 
HR> can address. Ths space is hidden from each application.
HR> 
HR> There are kernel, driver, .........
HR> 
HR> An application addressroom looks like:
HR> 
HR>  512MB +------------------------------+
HR>        |    shared system addresses   |
HR>        : Both system and application  :
HR>        | have access to       |
HR>  448MB +------------------------------+ (not realy fixed)
HR>        |     nonshared Application    |
HR>               :  code and data      :
HR>        |         |
HR>        +------------------------------+
HR>  xxxMB |     shared Application       |
HR>               :     code and data            :
HR>        |         |
HR>    0MB +------------------------------+

 Actually, what I said was that the area between 448MB and 512MB was 
used for protected shared DLL data areas. 32 bit application (EXE and 
DLLs without protected shared memory requirements) code use a CS, DS, 
ES with the base at zero and the limit at the 448MB line (1BFFFFFF). 
 As of Warp 4 GA and later Warp 3 Fixpacks, applications receive a CS,
DS, ES with a limit of 512MB (1FFFFFFF). The protected shared area is 
gone. 
 The area above the 512MB line (up to a total of 3GB) is only 
available to applications running on Warp Server SMP and Aurora. This 
area is above the compatability region, and ONLY 32 bit code can use 
it. Data buffers or code in this area cannot be accessed by legacy 16 
bit code. 


 HR>> The reserved area is used by NEW 32 bit apps in Aurora.
 
  ET> According to Denis that the reserved (protected) memory is
  ET> doesn't exist anymore in OS2 Warp 3 with recent fixpacks and also
  ET> not in Warp 4 so all OS/2 applications can use the memory between
  ET> 448 MB and 512 MB.
 
HR> No. But the border of 448 ist not a static one. The space from 512 to 448 MB 
HR> contains system code, data and the callgate. The needed 

 No, there is no "system" code (and never has been) in the 448 to 
512MB region. Callgates are allocated from the GDT, and generally
point to addresses in the system region (above BFFFFFFF). In fact,
most point to virtual addresses in the top 4MB..  
 The 448MB limit was a static one, just as 512MB is a static limit
too. It was eliminated because very few DLLs exploited this feature,
and nearly all the ones that did so were supplied from IBM. 
 Since the protected shared data area is gone, DLL code can (and does)
now load into the region between 448 and 512MB. This could be OS 
supplied DLLs (DOSCALL1, PMMERGE, etc) and/or application DLL's. 

HR> space is like system address space calculated in descending 
HR> order. The real under limit my be differ from release to 
HR> releas, from fixpack to fixpack, So the 448 MB border is a 
HR> limit to give you - the application programmer - a hit of 
HR> how many memory you will have available. For 16 bit 
HR> applications you have the same addressrom based on address 
HR> 0 up to about 448 MB tiled memory. 

 I am not quite sure what you are describing here.. 

 HR>> A DLL is nothing than a trick to become more shareable code
 HR>> pages and to have the code reuseable.
 
  ET> Also the same drive:\path\exefile 
 
HR> No. No for ALL loaded *.exe they will link to, Code in an 
HR> *.exe is shared by all instances of that specific program. 
HR> The same code in different *.exe can not shared by default. 
HR> A programmer must do some explicit tricks to become shared 
HR> pages and becomes unmaintetanceable code.

 DLLs are explicitly shared code (and you can have explicitly shared 
data areas too). Code in EXEs pages can/are shared also, but not 
explicitly by the application programmer. It depends on the fact that 
all code pages loaded from an executable are read only (no choice!). 
OS/2 will recognize when a second process is being started using the 
SAME executable and in the interests of ram page reuse will "map" the 
same physical page into multiple processes. For the loader to properly 
recognize that the second (or third, fourth, etc) process is starting
from the SAME EXE and that the code pages can be "page shared", the
full path to the executable MUST be exactly the same.

HR> A dll has its code shared by default for all instances they 
HR> link to. This kind of linking is known as late linking. 

 See DosLoadModule. A DLL is not mapped into the address space of the 
process until this has been called. If the EXE header of the 
application (or another DLL) indicates that there are fixup data 
entries requiring access to the DLL (sometimes called static linking, 
I prefer the term load time linking), the loader will call this API on
the application's behalf (before the first instruction in the app is 
ever called!). 
 An application can call DosLoadModule explicitly itself. When it does
so, it will also have to call DosQueryProcAddr to locate the address 
of the entry point in the DLL. In this kind of situation (I prefer to 
call this run time linking), the loader will have no idea that the 
code requires access to the DLL and will not do anything on it's 
behalf beforehand. 

 HR>> Loading a page is a difficult process. It may be
 HR>> - nowhere               the freed page is zeroed
 
  ET> Explain this please?
 
HR> You have (as programmer) the ability to define discardable 
HR> pages which containing constant 0 at startup. Such pages 
HR> are good for short temporary use. If the memory is 
HR> unattached for long enough it will be discarded if any 
HR> other memory request (start of a program, swapping) needs 
HR> real memory. A new access will simply allocate a free page 
HR> in RAM and fill it again with 0 (zero). So data goes lost.
HR> So knowing the way OS/2 does its memory management it my be a good idea to 
HR> have quick access temporary data while memory (RAM) is low. 
HR> But this kind of page must be handled very carefully so 
HR> nearly nobody uses it.

 I am not quite sure what you mean by the above. Can you supply an 
example or point to some API's that might be used in the above? 
 
 HR>> - discard it            in case of code or empty page
 
  ET> Empty page? An memory page can be freed when it is allocated and
  ET> this memory is de-allocated (pointers to it destroyed and this
  ET> memory marked as free)?
 
HR> No. The pointer to the page is NOT lost. Only the page 
HR> itself. The entry in page descriptor says that page is not 
HR> really existant and must be created.
HR> 
HR> A page descriptor contains flags for each page:
HR> 
HR> - is in memory: address ok.
HR> - is NOT in memory, swapable, writeable: must reload from swapper
HR> - is NOT in memory, swapable, readonly: must reload from original space
HR> - is NOT in memory, discardable: is nowhere,
HR>      find or make free page in memory for it

 You are right in essence, but there appears to be some hazyness in 
the details. 
 Actually, the page table descriptor only contains a flag for Present 
or Not Present. If the page is not resident in RAM (present) the 
virtual address portion of the descriptor is used as a reference to 
the "virtual page" associated with this virtual address (and process).
It is information in the virtual page structure that supplies the 
above information (and where it is located - if it is anywhere). An 
easy way to identify an unallocated virtual address is to examine the 
the page dir/table entry for it, and if it is Not Present *and* the 
Virtual Page ID is zero then it is not allocated (for that process). 

HR> The CPU reads ths flag for each access to an address inside 
HR> the page. If the page is NOT in memory it throws an 
HR> exeption to Operatingsystem to become the page accessable.
HR> 
HR> Other flags are set/written by CPU until each access to a page:
HR> - usage counter
HR> - usage time (relative)
HR> 
HR> The OS will find the least used page in RAM and if it is
HR> - swapable, writeable: write it into swapper, 
HR> - swapable, readonly:  is discardable,
HR> - dicardable: nothing special,
HR> 
HR> and flag it in its description as NOT in memory. Then it will change to the 
HR> page the CPU requests it and fill the now epmty page with 
HR> the requested contens (swap in, read form *.exe, *.dll or 
HR> simply zero it). Then the page is flagged as in memory and 
HR> the exeption is quit. So the CPU repeats the last page 
HR> access successfully.

 Correct. 

HR> You my wounder. Each exception contains an context switch, 
HR> so the CPU changes its context from application code to 
HR> system code to work on the exeption and back if the 
HR> exeption is quit. A context switch exchanges *all* 
HR> registers (including all special registers) and a flush of 
HR> all pipes.

 There may or not be a context switch (between processes). It depends 
on the situation. If there are free pages available, and the 
requirement is for an "empty" ram page then the allocation can occur
immediately, the page zero filled, and the instruction restarted. If
any Disk IO is needed (swap out/in, etc) then it is likely that a
process context switch will take place (involving all the above and
more). 
 This is one of the reasons that OS/2 tries to be "aggressive" in
maintaining a pool of free pages (swapping out before there is any 
real need to do so). The subject of swapping logic is beyond the scope
of this disussion, even though the previous does deserve mentioning.. 


   Denis       

 All opinions are my very own, IBM has no claim upon them
. 
. 
.
 

 




--- Maximus/2 3.01