What is Virtual Address Space?

A virtual address identifies a location in virtual memory. The virtual address space is divided into two ranges: user space and system space.

User space is the portion of the virtual address space into which Windows maps user-mode processes, per-process data, and user-mode dynamic link libraries (DLLs). Each process has its own context, that is, the code and data that the process uses. While the process is running, a portion of the context, called the working set, is resident in memory.

System space, also called kernel space, is the portion of the address space in which the operating system and kernel-mode drivers are resident. It is accessible only to kernel-mode code.

The distinction between user space and system space is important in maintaining system security. A user-mode thread can access data only in the context of its own process. It cannot access data within other process contexts or within the system address space. This restriction prevents user-mode threads from reading and writing data that belongs to other processes or to the operating system and drivers, which could result in security vulnerabilities and possibly even a system crash.

Kernel-mode drivers are trusted by the operating system and consequently can access both user space and system space. When a driver routine is called in the context of a user thread, all of the thread’s data remains in the user-mode address space. The driver can access the user-space data for the thread—if it takes the necessary security precautions—in addition to accessing system-space data. The user-mode thread, however, does not have access to the system-space data of the driver.

The sizes of the system and user address spaces depend on whether the hardware is 32-bit or 64-bit and on the options applied to Boot.ini. 

Virtual Address Space on 32-bit Hardware

On 32-bit machines, the lower 2 GB of the virtual address space is user space and the upper 2 GB is reserved as system space by default. If user-mode applications require additional virtual address space, an administrator can reserve 3 GB for user space, thus leaving only 1 GB of system space, by applying the /3GB switch to Boot.ini and then restarting the machine. This switch is also useful during testing to see how a driver performs with limited system address space. The switch is supported on Windows Server 2003 (all editions), Windows XP (all versions), Windows 2000 Advanced Server, and Windows 2000 Datacenter Server.

Figure 1 shows how the virtual address space is organized on 32-bit systems. (The figure is not drawn to scale.)

Figure 1. Layout of virtual address space on 32-bit systems

As the figure shows, the user address space occupies the lower range of the virtual address space and the system address space occupies the upper range. By default, both user and system space are 2 GB. When Windows is started with the /3GB switch, the size of the user space increases to 3 GB. The remaining 1 GB of virtual address space contains the system address space. The process page tables and hyperspace remain at the same addresses, but other components do not. When the system address space is limited to 1 GB, the maximum sizes of the paged and nonpaged pools are reduced and less space is available for system page table entries (PTEs) and the file system cache as well.

The file system cache, which is implemented in software, is not the same as the processor cache, which is implemented in hardware. Keep in mind, however, that this limit applies only to the amount of the cache that is visible at any given time. The system uses all the physical memory in the machine for caching.

The exact sizes of the paged and nonpaged pools, PTEs, and file system cache vary widely from release to release.

On the Intel Pentium Pro and later processors, which support PAE, Windows can support up to 128 GB of physical memory. PAE provides only additional physical memory and does not increase the size of the virtual address space. When PAE is enabled, the virtual address space remains unchanged at 4 GB and the layout of the 2‑GB system space is largely unchanged. 

Operating system support for PAE is provided by a special version of the Windows kernel, which is named Ntkrnlpa.exe (single-processor version) or Ntkrpamp.exe (multi-processor version). In this version of the kernel, PTEs and page directory entries (which are involved in mapping virtual memory to physical memory) are 64 bits wide. If the appropriate device and driver support is present, the PAE kernels support direct I/O for the entire physical address space (including any physical memory above 4 GB).

Virtual Address Space on 64-bit Hardware

On 64-bit machines, the virtual address space is 16 terabytes, with 8 terabytes of user space and another 8 terabytes of system space. The layout is similar to that for 32-bit Windows, except that the sizes are proportionately larger. As with 32-bit hardware, the sizes vary from release to release.

Currently, Windows runs on two 64-bit architectures: Intel Itanium and x64, which includes the AMD64 and the Intel Extended Memory 64 Technology. On x86 and x64 hardware, the page size is 4 KB. On Itanium hardware, the page size is 8 KB. Pointers on both Itanium and x64 architectures are 64 bits long.

Types, Constants, and Macros Used in Addressing

Because of the differences between the x86, x64, and Itanium architectures, drivers should not use hard-coded values or make assumptions about the page size, pointer size, or address ranges of the hardware. For example, on 64-bit hardware, a virtual address in which the highest-order bit is set is not necessarily a system-space address. (The same is true for 32-bit hardware when Windows is started with the /3GB switch.)

Instead, driver code should always use the appropriate constants, types, and macros, which are defined in Wdm.h and Ntddk.h. These include the following:

Constant, type, or macro	Usage
PVOID (or other data-specific pointer type)	Virtual addresses
PHYSICAL_ADDRESS	Physical addresses
PAGE_SIZE	Number of bytes in a page on the current hardware (used in memory allocation)
BYTES_TO_PAGES macro	Number of pages required to hold the input number of bytes on the current hardware
ULONG_PTR	Variables that might hold either an address or data of a scalar type
PVOID64	Pointers on Itanium and x64 hardware only; does not return valid information on x86 hardware

Code that uses these macros and types can depend on Windows to do the right thing, regardless of the underlying hardware platform.

Most drivers run on PAE systems without modification. For PAE compatibility, drivers that perform DMA must use the system’s DMA routines to ensure that address size issues are handled correctly. In addition, drivers should not use ULONG, PVOID, or PVOID64 for physical addresses. On PAE systems, the ULONG and PVOID types are only 32 bits long and therefore cannot store physical addresses above 4 GB. The PVOID64 type does not return valid information on the x86 architecture. Instead of using ULONG, PVOID, or PVOID64, drivers should use ULONG_PTR. Additional limitations apply for certain device types. For further details, see the Windows DDK, which is listed in the Resources section at the end of this paper.

Mapping Virtual Memory to Physical Memory

This section provides a brief overview of how the Windows memory manager maps pages in virtual memory to pages in physical memory. A general understanding of this mapping is helpful in writing a driver, but you do not need to understand the details. For a more detailed explanation, see Inside Windows 2000, which is listed in the Resources section at the end of this paper.

Every page in virtual memory is listed in a page table, which in turn identifies the corresponding physical page. The system and CPU use information from the virtual address to find the correct entry in the page table for a specific page.

Figure 2 shows how the memory manager maps virtual memory to physical memory. In the figure, a virtual address points to a specific location on a virtual page. The virtual address contains a byte offset and several index values that the system and the CPU use to locate the page table entry that maps the virtual page into physical memory. After the memory manager finds the page table entry, it uses the offset to find a byte in physical memory, which is identified by a physical address.

Figure 2. Mapping Virtual to Physical Memory

The length and layout of a virtual address depend on the hardware. Virtual addresses on x86 hardware are 32 bits long and contain two page-table indexes: one is an index into a page directory, which identifies the page table that maps the virtual address, and the other index identifies the page table itself. In PAE mode, the address contains a third index that is 2 bits wide.

Virtual addresses are 48 bits long on x64 hardware and 43 bits long on Itanium hardware, and the number of indexes is also different. However, on all 64-bit hardware, Windows stores and manipulates virtual addresses as 64-bit values. Drivers must not rely on the 43-bit and 48-bit internal address lengths (for example, by attempting to encode additional information in pointers) because Microsoft might change these values at any time.