If this is your first visit, be sure to
check out the FAQ by clicking the
link above. You may have to register
before you can post: click the register link above to proceed. To start viewing messages,
select the forum that you want to visit from the selection below.

You are correct that the memory size is handled correctly. My mistake there. But wouldn't an extra RET be needed at PSP offet 0Ah to get back to the correct address of the CALL since the RET in DOS 21h code would only bring the code back to after the INT 30h call? CALL 5 uses multiple jumps to keep from resetting the return address so only the same RET that normal INT 21h calls would use is required. I don't know if any programs rely on the PSP 0A - 0D INT 22h Termination alias.

Original call works like this:

Code:

where what
------------- ---------------
CP/M program CALL 5
PSP:0005 CALL FAR address which wraps to 0000:00C0
0000:00C0 JMP FAR to compatibility entry point in DOS
DOS pop IP & CS from stack into AX (IP is dropped)
pop real IP from stack into temporary var
now push flags, CS and IP as if program did INT 21
check if function number in CL is valid for CP/M
if no, exit
put function in AH and goto normal INT 21

Now the stack is set up so that DOS will return to the instruction following the CALL 5, instead of the far call. I modified that entry code in DOS to also pop the flags, so it will work with the INT 30.
Note that I didn't test this yet, so there may be some subtle mistake.

That is probably a myth. The OS/2 Museum analyzed the code of WordStar 3.24 (the oldest surviving version) and found that it did not use CALL 5. Instead the blame most likely lies with Microsoft's own Pascal compiler, which did use CALL 5:

Yes, the main reason for the A20 gate is the startup code for very old MS Pascal & EXEPACK versions IIRC (can't access the site at the moment, but I have read these articles before).
If the program is loaded too low in memory, it calculates a segment number that is "negative", and expects it to wrap around.

That is probably a myth. The OS/2 Museum analyzed the code of WordStar 3.24 (the oldest surviving version) and found that it did not use CALL 5. Instead the blame most likely lies with Microsoft's own Pascal compiler, which did use CALL 5:

WordStar does not use CALL 5; the DOS version of WordStar was an adaption of the CP/M-86* port of WordStar which uses bytes 6 and 7 of the PSP to determine the size of the segment. The fact that CP/M-80 used the same block of code to determine memory size and location of the start of the function dispatcher for API caused no end of trouble including the MS-DOS CALL 5 address wraparound** and the multiple layered jumps for MP/M-80 resident extensions. Programming would have been much simpler if DRI had kept the CP/M 1.4 function that provided memory size and CALL 5 merely needed to redirect to the start of the API.

* A quick disassembly of some versions of WordStar 3 will show parallel function calls to INT E0h for CP/M-86 and INT 21h for DOS.

** Bytes 6 and 7 get close to 64 kB because the software needs a big segment but that means that bytes 6 and 7 also define a segment near the top of memory needing the offset (bytes 8 and 9) to wrap around in order to get to the right address. Fortunately, PC-DOS won't run on a system with less than 64kB or the whole CALL 5 house of cards would fall completely apart.