ChangeLog-0.3.11

This is the changelog for ReactOS™ version 0.3.11 (released 16 December, 2009), an open-source operating system designed to be binary executable – and device driver-compatible with Microsoft™ Windows NT™ and subsequent Microsoft operating systems such as Windows 2000™ and Windows XP™. ReactOS is written by the ReactOS development team, and the last previous version of ReactOS was version 0.3.10 (released 5th July, 2009).

RBUILD_HELPER

Core

Bootloader (FreeLoader)

Rework operating system menu, by storing each selection in a structure instead of distinct arrays. As a bonus, entries like LiveCD\reactos="ReactOS" /DEBUGPORT=COM1 /SOS /MININT now work (Hervé Poussineau)

Move KiBootTss and KiBootGdt to freeldr.c as this is where they belong -- they are not required for NTLDR/WINLDR boot style and are only used directly in freeldr.c. (Stefan Ginsberg)

Kernel and Executive (NTOSKRNL)

Make use of the _SEH2_YIELD macro when returning from an exception instead of pre-initializing status codes and checking them after leaving the SEH block. (Stefan Ginsberg)

Don't use ExSystemExceptionFilter in SEH filters if we know the caller is user mode. (Stefan Ginsberg)

Fixed the remaining issues for running the MP kernel (only on UP so far). Implemented stubbed code in the scheduler, DPC handling and context switch code and fixed various bugs in the spinlock implementation. (Stefan Ginsberg)

DBGK

Fixed support for sending second chance exceptions to user mode debuggers. (Stefan Ginsberg)

EX

Fix a critical bug in ExAllocatePoolWithQuota spotted by msvc. Because of the current POOL_TAGGING mess in wdm.h, ExAllocatePoolWithQuotaTag is defined to ExAllocatePoolWithQuota, so when ExAllocatePoolWithQuota called the with-tag version, it ended up calling _itself_. Fix this by undefining the broken macro.(Stefan Ginsberg)

FSTUB

IO

Calling NtLoadDriver on an already loaded driver should fail with STATUS_IMAGE_ALREADY_LOADED. Fixes one test for kmtest driver tests and a bugcheck when loading drivers a second time. (Michael Martin)

KD

Continued the implementation of kernel support for debugging with WinDbg/KD. Implemented missing handling of various functionality in the KD support APIs and fixed missing or broken support in other areas. Debugging ReactOS kernel mode with WinDbg now works quite well. (Stefan Ginsberg)

Implement support for reading and writing physical memory for KD. The implementation uses a reserved mapping page to map the target physical address to. On x86 this page is located at virtual address 0xFFBFF000, and the PTE for this page is the last PTE of the nonpaged pool's PDE. Other architectures may need to reserve the PTE elsewhere. (Stefan Ginsberg)

Fix incorrect cache flag check and set in KdpRead/WritePhysicalmemory for write combined requests. The debugger's Uncached flag was checked instead of the Write Combined flag, and the debuggers Write Combine number (0x3) was set instead of Mm's flag (0x20). (Stefan Ginsberg)

DBGKD_WAIT_STATE_CHANGE64 is used in KD protocol 5, not number 6 that we use. Protocol 6 uses the DBGKD_ANY_WAIT_STATE_CHANGE structure which is sized according to the largest control-report structure (AMD64_DBGKD_CONTROL_REPORT currently), and is larger than DBGKD_WAIT_STATE_CHANGE64 on x86. This worked because our DBGKD_WAIT_STATE_CHANGE32/64 structures contained incorrect DBGKD_CONTROL_REPORT (used) and CONTEXT (unused) members that sized up the wait-state structure to pass WinDbg's length verification! It actually becomes larger than DBGKD_ANY_WAIT_STATE_CHANGE, but WinDbg only seems bail out only if the structure is too small. Remove the incorrect members from the protocol 5 structures and change to DBGKD_ANY_WAIT_STATE_CHANGE everywhere. Stefan Ginsberg)

Use better named and sized variables in KdpTrap for setting the "return register" in the caller's CONTEXT. (Stefan Ginsberg)

Rename the 3 kdsup.c files to kdx86.c, kdx64.c and kdarm.c to differ them from each other. (Stefan Ginsberg)

Implement KdpAllowDisable -- just check if any processor breakpoints are set on any processor in the system and disallow the disable if so. The routine is now architecture dependant, so move it to the appropriate files. (Stefan Ginsberg)

KD uses STRINGs, not ANSI_STRINGs -- they are the same thing, but let's be consistent. (Stefan Ginsberg)

Change the definition of KDSTATUS so it is guaranteed to be 32-bit. (Stefan Ginsberg)

Fix a critical bug in KiRestoreProcessorControlState: it didn't clear the busy flag in the TSS before reloading the task register, resulting in a GPF if we tried to reload the same register. (Stefan Ginsberg)

Add macros for getting and setting special purpose registers (the Program Counter and the "return register") in portable code instead of using #ifdef every time. Do likewise for setting IMAGE_FILE_MACHINE_XXX, using a new IMAGE_FILE_MACHINE_ARCHITECTURE macro. (Stefan Ginsberg)

Don't refer to the Program Counter as "Eip" in portable code. (Stefan Ginsberg)

KDBG

Get rid of the KDB_MODULE_INFO and operate with LDR_DATA_TABLE_ENTRYs directly.This saves us from some copy steps and unneeded search operations. (Colin Finck)

Don't duplicate code in KdbpSymFindModule and KdbpSymFindUserModule. Use a helper function instead and call it twice from KdbpSymFindModule. (Colin Finck)

Get rid of KdbpSymFindModuleByAddress, KdbpSymFindModuleByName and KdbpSymFindModuleByIndex. Instead use a single KdbpSymFindModule for all three variants. (Colin Finck)

Use _WIN64 instead of _M_AMD64 when determining if we need to set the DBGKD_VERS_FLAG_PTR64 flag. (Stefan Ginsberg)

Correct and clarify the comment documenting under what conditions we pass user mode exceptions to the kernel debugger. (Stefan Ginsberg)

KE

Fixed a bug in KeDetachProcess and KeUnstackDetachProcess that caused a switch from the original process to itself instead of switching from the attached process. (Stefan Ginsberg)

Removed a hack from KiGetMachineBootPointers and set the TSS selector properly in KiRosPrepareForSystemStartup instead for FreeLdr boot style. (Stefan Ginsberg)

Fixed a critical bug in KeFindConfigurationEntry; it passed NULL to KeFindConfigurationNextEntry which expected a pointer to a value containing NULL. This was hidden by the FreeLdr boot style as that boot style maps NULL, but got exposed by WinLdr boot style as NULL is not mapped there. (Stefan Ginsberg)

Add generic KeInvalidateTlbEntry to invalidate a single TLB entry for a given address instead of flushing the whole TLB. Used by the debugger physical memory support as invalidating the whole TLB for every map and unmap of its debug PTE would incur significant overhead for large copies. Replace direct usage of __invlpg() with this in x86 code too. (Stefan Ginsberg)

Correct the value of SIZE_OF_FX_REGISTERS -- it was 4 times too low which resulted in KeContextToTrapFrame not properly clearing out the XMM register area. Correct the define and move it out from ke.h to x86's ketypes.h and use it in the FXSAVE format structure. Also remove the IOPM definitions from ke.h as they have been in the NDK for a while. (Stefan Ginsberg)

Use ZwCurrentProcess() instead of -1 or 0xFFFFFFFF (which is incorrect for 64-bit) for the ProcessId parameter of DbgLoad/UnloadImageSymbols to clarify what is being passed. Don't use ZwCurrentProcess() in KeBugCheckWithTf for the pointer parameter of DbgUnLoadImageSymbols either. Use MAXULONG_PTR casted to PVOID instead. (Stefan Ginsberg)

Don't use KeBugCheck(0) -- 0 is an invalid bugcode. Use a proper bugcode, DbgBreakPoint or UNIMPLEMENTED instead depending on how the old call was used. (Stefan Ginsberg)

Define DBG_STATUS_CONTROL_C for assembly code and use it in KeUpdateSystemTime. (Stefan Ginsberg)

LPC

Implement NtReplyPort based on NtReplyWaitReceivePortEx and LpcReplyPort. (Aleksey Bragin)

Implement NtRequestPort based on NtRequestWaitReplyPort and LpcRequestPort. (Aleksey Bragin)

Fixed recursive spinlock acquisition in the Memory Manager introduced by the rewrite: the old Mm calls certain routines for performing operations on PFNs, and those routines acquire the PFN lock, while the new Mm holds the PFN lock while working with PFNs, including calling those routines. Moved out spinlock acquire/release to the callers of those functions instead. (Stefan Ginsberg)

Make SystemUnloadGdiDriverInformation take the opaque SectionPointer (which is really the Loader Entry) from the SYSTEM_GDI_DRIVER_INFORMATION structure returned in SystemLoadGdiDriverInformation so it doesn't need to loop the loaded module list to find it. (Stefan Ginsberg)

Remove some externs from related variables in the code -- we use something called "headers".(Stefan Ginsberg)

Disable the MiCacheImageSymbols call in MmLoadSystemImage for KD too as this hack is required for rossym rather than KDBG. Fixes detection of non-boot driver images by WinDbg. (Stefan Ginsberg)

Add another hack to freeldr.c to make sure that HAL is the second entry in the Load Order list -- detect if it isn't and insert it as the second entry manually if it isn't. (Stefan Ginsberg)

Add MmIsSessionAddress stub and use it from KD handle session space properly in the Memory Query API, and ASSERT that we are not trying to do a copy to/from session space in MmDbgCopyMemory as we don't handle it properly. Put this in mmdbg.c for now as we don't implement session space, and it is only called from KD right now. (Stefan Ginsberg)

Get rid of the MmFreeLdr* variables too. These have been deprecated for some time now. (Stefan Ginsberg)

The ModuleObject and ImageBaseAddress parameters of MmLoadSystemImage are not optional so don't treat them as such, and don't zero initialize them as callers shouldn't rely on this. (Stefan Ginsberg)

Set LDRP_ENTRY_NATIVE instead of LDRP_COMPAT_DATABASE_PROCESSED to mark the image as a native image. Also fix the value of LDRP_ENTRY_NATIVE. (Stefan Ginsberg)

Fix definition of LDR_DATA_TABLE_ENTRY -- the Checksum member should be in the union too. (Stefan Ginsberg)

Remove some unnecessary externs for stuff we now define globally in the kernel headers. (Stefan Ginsberg)

Move some x86 only stuff from global ke.h and ke_x.h to the x86 dependent ke.h. Remove DR_ACTIVE_MASK as it has been deprecated/unused for a while now. (Stefan Ginsberg)

The physical memory support relies on several Mm variables and structures to be properly set up. Add a new flag, MiDbgReadyForPhysical, and set it when the debugger support can handle physical memory requests. (Stefan Ginsberg)

Protect this page with a Memory Area to make the old Mm keep its dirty hands off it. (Stefan Ginsberg)

Add missing Memory Area for the 2 pages HAL currently uses for its own mappings on x86 -- previously, those pages could have been allocated by other parts of the OS, which would have resulted in serious corruptions. (Stefan Ginsberg)

Add 'simple' implementation of MmDbgCopyMemory to read/write virtual memory in a somewhat safe way (still no support for physical memory). Properly implement KdpCopyMemoryChunks as a wrapper around MmDbgCopyMemory and make most of the remaining unsafe copies use it instead of RtlCopyMemory. This fixes most of the remaining crashes during KD debugging as WinDbg/KD relies on the kernel support to handle bad addresses. (Stefan Ginsberg)

Stub out the remaining missing global variables for the debugger data block -- fixes some cases of WinDbg failures and gives it a chance to handle errors instead of failing on a NULL read. Several of these variables are for functionality we don't yet implement, so I tried to put them where they are "least wrong". Everything besides the MmLoadedUserImageList variable is left unitialized -- KD should mostly be able to handle this properly. (Stefan Ginsberg)

ExceptionRecord32To64 should be available for both 32 and 64 bit builds (and it shouldn't be a forceinline). Get rid of CopyExceptionRecord and determine if we need to convert or can just copy it directly instead. (Stefan Ginsberg)

Get rid of NrSystemPages. Its value was only respected in MmInitializePageList at boot, even though it got updated later. Use a local variable in MmInitializePageList instead. (Stefan Ginsberg)

Fix SystemBasicInformation to use the correct variables for physical page information. Also, don't set ResidentSystemCodePage in the SystemPerformanceInformation query to some random incorrect Mm value. We don't depend this value anywhere in ReactOS currently, so just set it to 0. (Stefan Ginsberg)

Fix recursive spinlock acquisition in Mm caused by locking inconsistency between ARM3 and the old ReactOS Mm. The old Mm calls certain routines to modify PFN entries (lock, unlock, reference, dereference, etc) and acquires/releases the PFN lock inside those functions (which is extremely inefficient as you can't, for example, have to acquire/release the PFN lock twice to reference and lock the same page), while ARM3 synchronizes differently and holds the lock while calling those routines, resulting in a recursive lock attempt on MP (which works on UP because spinlocks are just IRQL raise/lower there). Move out locking from MmAllocPage, MmReference/DereferencePage and MmLock/UnlockPage to the callers to be consistent with ARM3. (Stefan Ginsberg)

Get rid of MmLockPageUnsafe and MmReferencePageUnsafe. The "safe" routines just forwarded to the unsafe versions -- call them directly instead. Remove unused MmAcquirePageListLock/MmReleasePageListLock (Stefan Ginsberg)

MmpDeleteSection can be called for a Section Object before it is fully set up (when ObDereferenceObject is called before all fields are initialized), and it will now crash due to a bogus, nonintialized Segment/ImageSection pointer. Always zero out the whole buffer to prevent this. Fixes the kernel32 loader Winetest. (Stefan Ginsberg)

Paged pool PTEs are demand zero PTEs while the memory hasn't been accessed -- this is the only type of fault supported. (ros-arm-bringup)

Because paged pool PDEs are also demand-paged, added code to handle demand paging of PDEs as well.(ros-arm-bringup)

Also, because paged pool is non-resident, but can be accessed from any process, we need a mechanism to sync up the kernel's page directory with the per-process one, on demand. This is done at startup, but other processes may have paged in paged pool that another process knows nothing about when he faults. (ros-arm-bringup)

Assert if we get a user-mode fault, a transition fault, or a soft fault, since these shouldn't happen.(ros-arm-bringup)

Disable APCs while dispatching faults, and pseudo-use the working set lock.(ros-arm-bringup)

Assert if we get write errors on read-only pages, since we don't use those in ARM3 yet. (ros-arm-bringup)

Assert if we have a paged out PTE, this shouldn't happen yet. (ros-arm-bringup)

Enable test to see if we can touch a paged pool allocation. (ros-arm-bringup)

Fix a bug in memory area creation: Static memory areas had the static flag embedded in their type, so code that was switch()ing on the type would fail to recognize the actual type, because MEMORY_AREA_STATIC was ORed in. (ros-arm-bringup)

Add a new memory area type: MEMORY_AREA_OWNED_BY_ARM3. This will allow us to instruct the ReactOS Memory MAnager to "Back. The Fuck. Off." during page faults and such, so we can handle page faults inside ARM3-owned PTEs ourselves. (ros-arm-bringup)

Create all memory areas in one shot in MmCreateSystemMemoryAreas (get rid of MiInitPageDirectoryMap and MiInitPagedPool memory area creation). (ros-arm-bringup)

Dump the kernel address space after all this is done, in a MmDbg function in mm.This cleans up ARM3 of some ROS-specific code, and also collapses Phase 1 and 2 into a single phase. (ros-arm-bringup)

Fix calculations error in the setup of the paged pool bitmap. (ros-arm-bringup)

Set correct TIB version in the TEB. Due to a historical oddity, the NT TIB identifies itself as '0x1e00', which is 30 shifted 8 bits. 30 is the version identifier of OS/2 3.0, the original 32-bit target of the Windows NT effort. (ros-arm-bringup)

Create a double-mapping PTE for the shared user data region and fault it in whenever a process touches that address. (ros-arm-bringup)

Remove the old hack which used the PCR's page frame number to create a fake PTE each time to reference it, basing on the fact that the shared user data region was on the same page as the PCR on certain architectures. (ros-arm-bringup)

Setup SharedUserData settings during Phase 0, not Phase 1. Fixes an old bug where the Inbv driver assumed ReactOS was in "server mode" simply because the default SharedUserData->NtProductType was "0" (which is an invalid product type, but it's != NtProductTypeNt). Also fixed it such that this reports NtProductServer, which is what ReactOS reports to be (Windows 2003 Server SP 1). (ros-arm-bringup)

This patch has been tested for over 2 months and all known regressions were fixed. (ros-arm-bringup)

DebugPool implementation is not compatible with newer nonpaged pool algorithms, so remove support for nonpaged pool debugging. (ros-arm-bringup)

Implement the algorithms for pool allocation and de-allocation.(ros-arm-bringup)

Initialize MmLowestPhysicalPage to -1, otherwise setting this value will never work properly since we'll never find a page lower than 0 (the default laoder-initialized value). (ros-arm-bringup)

Get rid of the concept and tracking of "mapped pages". The MapCount was never checked anywhere in terms of page accounting -- only the reference count is. (ros-arm-bringup)

Fix the issue with the system attempting to map page 0 into hyperspace. (ros-arm-bringup)

The PTEs for the PFN Database are now created by ARM3. Unlike the old code which create PTE for every page on the machine, ARM3 only creates PTEs to account for pages that should be in the PFN database. (ros-arm-bringup)

A second related change is what "pages should be in the PFN database". Previously, reserved or otherwise non-existing (ie: holes) memory regions would get a PFN entry created and marked as "BIOS". This is wasteful and not compatible with Windows: there should not be PFN entries created at all. (ros-arm-bringup)

Initialize the value of MmBootImageSize in ARM3 now.Also fix its value such that it's PDE aligned -- this makes sure that we don't step on any of the boot loader's PDE mappings and can blow everything away later. (ros-arm-bringup)

Initialize the MmSystem/User/Probe Addresses in ARM3 as well (no functional change). (ros-arm-bringup)

Print out a lot more of the VA ranges in ARM3's Phase 2 initialization. Most of the VA space is now dumped out. (ros-arm-bringup)

Write out the code to initialize session space VA ranges.Image space, view space, working set space and pool space values are all calculated properly. (ros-arm-bringup)

PO

PS

Add missing synchronization for access to PsLoadedModuleList. Inserting and removing entries to the list was protected by a spinlock, but some places didn't acquire the lock before looping the list when searching for module information. For efficiency, use an executive resource for other access than inserting/removing entries so we can have shared locking in some common cases (spinlock stays so access can be synchronized with DPCs). (Stefan Ginsberg)

Fix EngLoad/UnloadImage to do this. Also make EngLoadImage return the pointer to the internal structure as the "handle" so EngUnloadImage doesn't need to loop the driver list to find it. The code is still extremely broken and needs a rewrite.(Stefan Ginsberg)

SE

Factor out working code from SeAccessCheck into SepAccessCheck, the lowest level for SeAccessCheck remains SecurityImpersonation, but for NtAccessCheck it's lowered to SecurityIdentification. (Aleksey Bragin, #4169)

Hardware Abstraction Layer (HAL)

Reimplemented HalpCalibrateStallExecution with a more accurate implementation to determine the stall count factor for KeStallExecutionProcessor. (Stefan Ginsberg)

Implement RtlpDidUnicodeToOemWork to check for unmapped characters. Based on a patch by Daniel Zimmerman. (Aleksey Bragin, #4548)

Instead of removing FORCEINLINE to make GCC happy (it can't inline functions using va_list), use __inline to merely hint that the function should be inlined. This lets you use RtlStringCbPrintfA and RtlStringCbPrintfExA in multiple source files as the functions aren't static. (Stefan Ginsberg)

There is no reason to forceinline all the RtlString* routines -- they are quite big and the compiler should be allowed to optimize here as it likes, so just use __inline. Also add RtlStringCchCopyA and re-enable ASSERTMSG and use it for the stubs. (Stefan Ginsberg)

Remove the hack of checking wether pin create request is passed. Since create items now work nicely, use a different function and handle pin creation requests seperately. (Johannes Anderwald)

TCPIP

UNIATA

Synched the UniATA Inf to the internal support list. Now ROS can detect all supported SATA Controllers even after 1st stage and properly shows them in the HW Wizard. (Daniel Reimer)

Fix a couple of bugs in the uniata ATA code spotted by bearwindows. ICH8/ICH9/ICH10 should be supported now. (Aleksey Bragin)

Revert most of the 41684: "-fix support for ICH7,ICH7M,ICH8,ICH8M,ICH9,ICH9R,ICH9M and ICH10 in compat mode". Those controllers should be supported now, and when set to compat mode the driver falls back to UDMA automatically. (Aleksey Bragin)

Prevent kernel bug check in win32k when calling a hook proc when thread is in cleanup. (James Tabor)

Export global DD context size. Create a container for both PDev and EDDG structures. When allocating PDev and add the DD context size if not zero. (James Tabor)

Working with Michael Martin to fix the LoadMenu callback bug. MenuName is a variable string buffer and not a pointer. The minimum size is always four bytes. Initialization of the structure, zero out the memory, allowing Intersource check to function as it should. To make it Intersource, use the first word character in the buffer. (James Tabor)

Zero byte was not set at correct offset, potentialy leading to a heap overflow (Johannes Anderwald)

MSAFD

MSI

Schedule a rename operation also for ERROR_USER_MAPPED_FILE. Prevents msi from aborting an install when attempting to overwrite font files. Open Office should now install successfully. (Michael Martin)

SHLWAPI

SYSSETUP

Add Ext2, GraphApp and X to the list of acknowledged projects. (James Tabor)

USER32

DIALOG_CreateIndirect: Save the current focused window after sending WM_INITDIALOG as the current window with focus could have been set in the dialog's procedure. Fixes setting focus on Open and Close dialogs to the FileName edit control in OpenOffice. (Michael Martin)

GetDlgItem: Change to WINE's implementation as ours was incorrect. Remove function GetDlgItemEnumProc and struct GETDLGITEMINFO as they are no longer needed. Fixes drawing issues in Open and Save dialogs for OpenOffice. (Michael Martin)