Windows Internals

[ftvfatboy]

Representation of a file in memory

Physical Device Object (PDO)

- Represents media type objects, device type FILE_DEVICE_DISK,FILE_DEVICE_CDROM, FILE_DEVICE_VIRTUAL_DISK etc.
- Created by FSD by IoCreateDevice(..) to represent a physical or virtual disk obj, that it manages.
- At creation time, a VPB (more about this later) structure is allocated and associated with media type objects, by the I/O M.
- Initially VPB flags indicate tht physical media doesn't have any logical volumes mounted (VPB_MOUNTED)
- Later a mount operation might occur.

Logical Volume Device Object (LVDO)

- Represents instance of a mounted logical volume.
- Created by a FSD implementaion.
- As part of mounting the LV, most OS require certain system defined data structs be created and/or initialised to establish linkage b/w the rest of the I/O M structs and in-mem representaion of mounted vol.
- In NT, the I/O M requires that a Device Object representing the mounted LV be created for physical media be created and initialized. For network redirectors, not required.
- Each VDO is logically associated with a PDO using a VPB struct belonging to the PDO. Occurs at mount time, when the very first Create/Open request is received by the I/O M for an object residing on the PDO.

[ftvfatboy]

[ftvfatboy]

Volume Parameter Block (VPB)

- Creates Logical association b/w PDO & LVDO
- Only exists for Dev Objs that represent physical or virtual media, that can be mounted.
- There is no physical association b/w the PDO & LDO representing a mounted vol, as would typically happen in an attach operation, this is typically done for filters to intercept the requests to the target dev obj.
- Has information abour volume Serial number, Volume Label and Volume label length
- IN a mount op. the only connection b/w PDO and LVDO is VPB
- I/O M is aware of this logical association & always checks the VPB on receipt of a Create/Open call for an on-disk obj. to determine the FS vol dev obj in which the req should actually be directed.
- Each File Obj (FO) struct represents a successful open Op. on an on-disk obj, points to the VPB struct belonging to the PDO on which the opened on-disk obj resides.

[ftvfatboy]

[ftvfatboy]

- The Flags fields in the VPB has one of the three values
1. VPB_MOUNTED - Set by I/O M after successful mount op.
2. VPB_LOCKED - Set by FSD. IF set all Open/Create ops will fail of that LV. FASTFAT code in IFS kit does it, in response to FSCTL_LOCK_VOL
3. VPB_PERSISTENT - If set, I/O M will not del VPB even if ReferenceCount = 0




Volume Control BLock (VCB)

- Created as part of Mount op.
- Contains ptr to the root dir in-mem struct, that is the FCB for the root-dir, a count of the # of file streams currently open on the LV, some flags representing the state of the LV at any given time, sync structs used to maintain the integrity of the VCB itself, may also contain information about available free clusters, a list of allocated clusters etc. Note That I/O M doesn't require this struct, it is for the FSDs own use.


File Control Block (FCB)

- Represents a file stream in memory.
- A typical FS manipulates two types of on-disk objects
1. Files
2. Directories.
- Upon user request to fetch data from disk, or even control information about files, FCB is created. This is highly specific FS inplementaion, as features of the FSDs vary. FS fetches the informaiton & keeps it in system memory somewhere to make it accessible to the user. Maintains in-mem ptrs.
- Multiple users may ask for the same data, and this needs to be satisfied concurrently.
- Sync of threads should be maintained.
- There is one and only one FCB for one on-disk object. FCB is analogus to UNIX vnode.
- Even for FS that support links, there is one FCB per object, even though the two requests make have an entirely different path to the file.
- OS doesn't govern the FCB structure, as it is FS specific, but to integrate with VMM and Cache manager, it requires certain fields to be mandatory. There are 4 of them, namely...
1. FSRTL_COMMON_FCB_HEADER - a struct. Never allocated from paged pool.
2. SECTION_OBJECT_POINTERS - a struct
3. Two sync structs of type ERESOURCE one each for MainResource and PagingIoResource. NEver allocated from paged pool.
- Typically these fields have no use in case of directories.
- FCB is created when a byte stream is opened for the 1st. time and no other FCB representing the same exists. Typically there is a field like a ReferenceCount that determines the number of FOs using the FCB. When the Count is 0 the FCB is destroyed.


Refining our last figure, if FO#1 and FO#2 are handles to the same on-disk obj then the figure becomes...

[ftvfatboy]

[ftvfatboy]

File Object (FO)


- It is the kernel equivalent of an Handle to a file.
- NT executive defined, maintained by I/O M.
- FSD is responsible for initializing FsContext and FsContext2 fields.
- FsContext, theoritically can be pointing to anything, but to maintian symphony with cache manager and VMM, it has to point to FCB_COMMON_HEADER
- FsContext2 might point to something like a context control block.




Other data structures that might be present, depending on the FS to be implemented are...
1. Support for logging ofr fast recovery (NTFS has it)
2. Support for encryption (NTFS has it, and filter drivers might implement it)
3. Support for data compression (Again NTFS has it)
4. SUpport for directory change notification (FASTFAT has it, see code in IFS kit)
5. Support for oplocks and byte range locking.
6. Support for byte streams and dir quotas.
7. SUpport for Dynamic Name Lookup cache.

[ftvfatboy]

System: Windows 2000 professional•free • build • 2195 • X86 • Single CPU

Part I:

In this part, I’ll discuss the fundament of IRQL and the differences/relations between IRQL and CPU rings/thread
priority/hardware IRQ.

Definitions:
For a kernel developer, IRQL is not strange. Almost every kernel support routine has a limit of the working IRQL. But
what on earth an IRQL stands for ? This article is to disclose it’s mysterious veil.
See it’s definition from DDK:
Interrupt Request Level (IRQL)
The priority ranking of an interrupt. A processor has an IRQL setting that threads can raise or lower.
Interrupts that occur at or below the processor's IRQL setting are masked and will not interfere with the
current operation. Interrupts that occur above the processor's IRQL setting take precedence over the current
operation.
The particular IRQL at which a piece of kernel-mode code executes determines its hardware priority.
Kernel-mode code is always interruptible: an interrupt with a higher IRQL value can occur at any time,
thereby causing another piece of kernel-mode code with the system-assigned higher IRQL to be run
immediately on that processor. In other words, when a piece of code runs at a given IRQL, the Kernel masks
off all interrupt vectors with a lesser or equal IRQL value on the microprocessor.
It says that higer IRQL interrupt can interrupt other task or interrupt with lower IRQL and all the interrupts with IRQL
equal to or below current IRQL will be masked and wait until it get opportunity.
Each processor has an independent IRQL, which descripts the current IRQL of the processor, i.e. the IRQL of the
current instructions / codes being executed by this processor.
System support a kernel routine (KeGetCurrentIRQL) to get current processor’s IRQL. Let’s see the assemble

codes:

kd> u KeGetCurrentIrql
hal!KeGetCurrentIrql:
80063124 0fb70524f0dfff movzx eax,word ptr [ffdff024]
8006312b c3 ret


This routine is very simple. It tells us that current IRQL stores at krenel address 0xffdff024. And it’s a WORD (2
bytes), in fact it’s only one byte long, the upper 8 bits are zero.
The value of IRQL can be one of the followings.

Software IRQL:

PASSIVE_LEVEL 0 // Passive release level
LOW_LEVEL 0 // Lowest interrupt level
APC_LEVEL 1 // APC interrupt level
DISPATCH_LEVEL 2 // Dispatcher level
Hardware IRQL:
DIRQL: from 3 to 26 for device ISR
PROFILE_LEVEL 27 (0x1B) // timer used for profiling.
CLOCK1_LEVEL 28 (0x1C) // Interval clock 1 level - Not used on x86
CLOCK2_LEVEL 28 (0x1C) // Interval clock 2 level
SYNCH_LEVEL 28 (0x1C) // synchronization level
IPI_LEVEL 29 (0x1D) // Interprocessor interrupt level
POWER_LEVEL 30 (0x1E) // Power failure level
HIGH_LEVEL 31 (0x1F) // Highest interrupt level

The IRQL values are divided into two groups: Software ( 0,1,2 ) / Hardware IRQL ( >= 3). Hardware IRQL is for
device ISRs and system, it is similar to (but distinguished with) the level of hardware IRQ, which implemented by
i8259, but IRQL is only an action of Windows OS, not hardware’s. It’s realized by Windows OS. But hardware IRQ
level is achieved by 8259A (programmable interrupt controlor). We will detail the hardware IRQ later.
The bigger of it’s value, the higher level the IRQL has.
Let us see an example:

[ftvfatboy]

[ftvfatboy]

Description:
1) Thread A is running at this time.
2) IRQL (0D) interrupt happends, then cpu interrupt the current running thread A and begin to run IRQL (0D).
3) With higher IRQL, IRQL 1A will take over the cpu …
[Here I use this example just to demonstrate the mechanism of IRQL. For a real system, when an hardware IRQ
arises, the eflags IF bit will be set to zero, to mask all maskable-interrupts.]
4) When IRQL 1A is running, IRQL 18 arises, but is masked for it’s IRQL (18)< Current IRQL (1A).
5) IRQL 1A service finishes, then IRQL 18 will run instead of interrupted IRQL 0D routine.
6) Only after IRQL 18 finishes, IRQL 0D can get the cpu.
7) At last, cpu resume the thread A, when IRQL 0D finishes.

IRQL can be prone to be confused with Thread priority, Cpu Rings, and hardware IRQ.


Now let’s discuss the
differences between them.


IRQL vs. Thread priority:

See ddk about Thread priority:

priority

An attribute of a thread that determines when and how often the thread is scheduled to run. For a running
thread, its priority falls into either of two classes, each class with sixteen levels:
Variable priority class has values in the range 0 to 15. This class is used by most threads.
Threads with variable priority are always preemptible; that is, they are scheduled to run round-robin with
other threads at the same level. In general, the Kernel manages a variable-priority thread as follows: when the
thread is interactive with a user, its priority is high (given a boost); otherwise, its priority decays by one level
per quantum the thread runs until it reaches its original programmer-defined base priority level.
Real-time priority class has values in the range 16 to 31. This class is used by time-critical threads,
making such a thread preemptible only by a thread with higher priority.
Note that any thread, whatever its priority attribute, is always preemptible by a software or hardware
interrupt.

A priority of a thread only affects the decisions of system scheduler:
1) When to execute this thread ?
2) How many will this thread take the time slices ?
The scheuler will decide them based on the priority of the thread.
Generally all the threads (including system threads which runs in ring0, kernel space) run at PASSIVE_LEVEL IRQL:
no interrupt vectors are masked. And the scheduler who determinates the state of all the threads runs at
DISPATCH_LEVEL.
System realizes it’s preemption attribution via thread priority, the executor is the scheduler,. but another attribution
“interruptible” of windows nt, is implemented via IRQL.
Windows NT is interruptible, i.e., any codes could be interrupted by higher IRQL interrupt. Every developer should
keep it in mind.

IRQL vs.Cpu Rings:

For x86 cpu, it has 4 rings: 0, 1, 2, 3
Every ring defines it’s privilege, such as memory page access, io access … Windows only uses ring0 and ring3.
Kernel uses ring0, and user routines use ring3.
IRQL only exists in kernel space. For user space, it’s meaningless. All user threads are running at PASSIVE_LEVEL,
though they can result in a task switch to kernel space and change the IRQL. They could not access the IRQL directly.
Understanding IRQL –
4
And also, from assemble codes of KeGetCurrentIRQL, we know that current IRQL locates at [ffdff024]. Address
ffdff024 is in kernel space.

IRQL vs. Hardware IRQs:
For X86 system, it has two 8259 pics (programmable interrupt controller), each could handle 8 IRQs. But the slave
8259 pic is attached to master 8259 ‘s pin 2 (IRQ2). So there are 15 IRQs available.
In general, IRQ 0 has the highest privilege,
the next is IRQ 1, the last is IRQ7. (Irq 8 –
Irq 15 have the same privilage with IRQ2 ?).
For each IRQ, we can mask it by zero the
correspond bit of IMR (interrupt mask
register, port 21h for the master 8259,
0xA1h for the slave.). The IMR register is
only writable, it is not readable. So system
must store it’s current value. For each IRQL,
Windows NT maintains a table of IMR at
hal!KiI8259MaskTable. (Softice’s IRQ
command possibly uses this value, I’m not
sure.)