Windows Ancillary Function Driver (afd.sys) Race-Condition Null-Pointer Dereference
This blog will demonstrate a crash I found in Ancillary Function Driver for winsock (afd.sys). The bug was a race condition in Windows Registered Input/Output component that led to a NULL pointer dereferences which resulted in a DoS of the Windows kernel.
More on Windows Registered Input/Output can be found here and here.
The Vulnerability
The vulnerability is a race condition that leads to a NULL pointer dereference, happens when the user tries to resize the completion queue, and at the same time closes that same completion queue, thus leading to a NULL pointer dereference when searching for a completion queue.
When creating a new completion queue, the completion queue is initialized, and gets inserted into a global table that grows dynamically:
__int64 __fastcall AfdRioCreateCompletionQueue(
PVOID FileObject,
CompletionQueue *PoolChunk,
void *UserMemory,
int QueueSize,
int Unmapped_memory,
KPROCESSOR_MODE AccessMode,
RIO_NOTIFICATION_COMPLETION_TYPE NotificationType,
HANDLE Handle,
PVOID CompletionKey,
PVOID Overlapped)
{
void *LocalUserMemory; // r10
PMDL Mdl; // r14
void *MiniPacket; // r15
CompletionQueue *CompletionQueue; // rdi
ULONG v15; // edx
int v16; // esi
NTSTATUS status; // eax
...
AfdAcquireWriteLock(*((KSPIN_LOCK **)FileObject + 0xC), &LockHandle);
// Insert newly created completion queue to a list.
if ( AfdRioInsertCompletionQueue((__int64)FileObject, CompletionQueue))
{
PoolChunk->CqList = (_LIST_ENTRY *)CompletionQueue;
v16 = 0;
}
else
{
v16 = 0xC0000017;
}
KeReleaseInStackQueuedSpinLock(&LockHandle);
...
}
Then later, it returns an associated handle with that completion queue to the user. This handle can be used for various operations on the queue, like resize or close.
User-mode programs pass the handle to the exposed API, which will be looked up using AfdRioFindCqUnderLock. For example, AfdRioInvalidateCq:
_int64 __fastcall AfdRioInvalidateCq(PVOID afd_endpoint, unsigned int a2)
{
__int64 v2; // rbp
KSPIN_LOCK *v4; // rcx
unsigned int v5; // edi
CompletionQueue *Cq; // rbx
__int64 v7; // r8
struct _MDL *Mdl; // rcx
struct _KLOCK_QUEUE_HANDLE LockHandle; // [rsp+20h] [rbp-38h] BYREF
struct _KLOCK_QUEUE_HANDLE v11; // [rsp+38h] [rbp-20h] BYREF
v2 = a2;
v4 = (KSPIN_LOCK *)*((_QWORD *)afd_endpoint + 0xC);
v5 = 0xC000000D;
memset(&LockHandle, 0, sizeof(LockHandle));
memset(&v11, 0, sizeof(v11));
AfdAcquireWriteLock(v4, &LockHandle);
// search for the queue that the user passes the handle in
Cq = (CompletionQueue *)AfdRioFindCqUnderLock((__int64)afd_endpoint, v2, 0);
...
Similarly, in AfdRioResizeCq, it also searches for the completion queue, then use the results for further processing:
_int64 __fastcall AfdRioResizeCq(__int64 a1, unsigned __int64 a2, unsigned int a3, KPROCESSOR_MODE a4)
{
__int64 v5; // r15
__int64 v6; // r13
unsigned int NewSize; // r12d
ULONG ChunkRemap; // ecx
void *v9; // rax
struct _MDL *Mdl; // rax
struct _MDL *v11; // r14
...
v28 = MappedSystemVa;
if ( !MappedSystemVa )
ExRaiseStatus(0xC000009A);
v13 = AfdRioFindAndReferenceCq(a1, *(_DWORD *)(a2 + 4));
v14 = v13;
if ( !v13 )
{
v25 = 0xC000000D;
goto LABEL_44;
}
v27 = NewSize;
KeAcquireInStackQueuedSpinLock((PKSPIN_LOCK)v13, &LockHandle);
v24 = 1;
CqSize = v14->CqSize;
v31 = CqSize;
MappedMemory = (unsigned int *)v14->QueueCq;
v5 = *MappedMemory; // [1]
v6 = MappedMemory[1];
In AfdRioInvalidateCq, it first acquires a lock to modify the global table that holds the queue:
AfdAcquireWriteLock(v4, &LockHandle); // [2] -> acquire a lock
Cq = (CompletionQueue *)AfdRioFindCqUnderLock((__int64)afd_endpoint, v2, 0); // -> Find the completion queue from handle
if ( Cq )
{
*(_QWORD *)(*((_QWORD *)afd_endpoint + 0xE) + 8 * v2) &= v7; // -> NULL out the associated queue
KeReleaseInStackQueuedSpinLock(&LockHandle); -> release the lock
KeAcquireInStackQueuedSpinLock((PKSPIN_LOCK)Cq, &v11);
Then it proceeds to NULL out the queue member:
Mdl = (struct _MDL *)Cq->Mdl;
Cq->Freed = 1;
MmUnlockPages(Mdl);
IoFreeMdl((PMDL)Cq->Mdl);
Cq->Mdl = 0i64;
Cq->QueueCq = 0i64;
if ( (Cq->NotificationKey & 6) != 0 )
{
ObfDereferenceObject(Cq->NotificationObjectType);
Cq->NotificationKey &= 0xFFFFFFF9;
}
KeReleaseInStackQueuedSpinLock(&v11); // [3] -> release the lock
So what is the vulnerability here? And why does resizing the queue on one thread, while destroying it in another crashes the kernel? Looking at the crash context might give us a better idea of what is happening:
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=0000000000000000 rbx=0000000000000000 rcx=0000000000000042
rdx=0000000000000000 rsi=0000000000000000 rdi=0000000000000000
rip=fffff8064c5fc613 rsp=fffffc8b496e7230 rbp=fffffc8b496e7b20
r8=ffffab85057302d0 r9=0000000000000feb r10=fffff80644cea710
r11=0000000000000001 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei ng nz na po nc
afd!AfdRioResizeCq+0x28b:
fffff806`4c5fc613 448b38 mov r15d,dword ptr [rax] ds:00000000`00000000=????????
Resetting default scope
STACK_TEXT:
fffffc8b`496e6748 fffff806`44f67422 : fffffc8b`496e68b0 fffff806`44c8e920 ffff9b81`a0a4b180 00000000`00000001 : nt!DbgBreakPointWithStatus
fffffc8b`496e6750 fffff806`44f66ae3 : ffff9b81`00000003 fffffc8b`496e68b0 fffff806`44e2fe80 fffffc8b`496e6e60 : nt!KiBugCheckDebugBreak+0x12
fffffc8b`496e67b0 fffff806`44e15ef7 : fffffc8b`496e6fc0 fffff806`44e88f36 000000d1`b98ff838 00000000`00000003 : nt!KeBugCheck2+0xba3
fffffc8b`496e6f20 fffff806`44e2bd29 : 00000000`0000000a 00000000`00000000 00000000`00000002 00000000`00000000 : nt!KeBugCheckEx+0x107
fffffc8b`496e6f60 fffff806`44e27189 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiBugCheckDispatch+0x69
fffffc8b`496e70a0 fffff806`4c5fc613 : 000000d1`b98ff838 00000000`00000000 fffffc8b`496e7b20 00000000`00000000 : nt!KiPageFault+0x489
fffffc8b`496e7230 fffff806`4c5fa0f4 : ffffab85`07d8c810 ffffab85`0a4b40f0 00000000`00000000 000000d1`b98ff701 : afd!AfdRioResizeCq+0x28b
fffffc8b`496e7340 fffff806`4c5ce028 : 00000202`00000001 00000000`00000000 00000000`0007641a 00000000`00000000 : afd!AfdRioFastIo+0x3f8
fffffc8b`496e7430 fffff806`451c4f0e : 00000000`00000000 ffffab85`0a4b40f0 00000000`00000000 00000000`00000000 : afd!AfdFastIoDeviceControl+0x18648
fffffc8b`496e77d0 fffff806`451c4a96 : 00000244`7557c03c 00000000`00000004 00000000`00000000 00000000`00000000 : nt!IopXxxControlFile+0x45e
fffffc8b`496e79c0 fffff806`44e2b408 : 00000000`00000000 00000000`00000000 00000000`00000000 000000d1`b98fef28 : nt!NtDeviceIoControlFile+0x56
fffffc8b`496e7a30 00007ffe`88430084 : 00007ff7`4800262b 00000000`000007a8 000000d1`b94ffa58 cccccccc`cccccccc : nt!KiSystemServiceCopyEnd+0x28
000000d1`b98ff6b8 00007ff7`4800262b : 00000000`000007a8 000000d1`b94ffa58 cccccccc`cccccccc cccccccc`cccccccc : ntdll!NtDeviceIoControlFile+0x14
000000d1`b98ff6c0 00000000`000007a8 : 000000d1`b94ffa58 cccccccc`cccccccc cccccccc`cccccccc 000000d1`b98ff808 : Deregs!reimpl_RIOResizeCq+0x28b [E:\Vuln_text\afd\Deregs\Poc.cpp @ 216]
000000d1`b98ff6c8 000000d1`b94ffa58 : cccccccc`cccccccc cccccccc`cccccccc 000000d1`b98ff808 00000000`0001211b : 0x7a8
000000d1`b98ff6d0 cccccccc`cccccccc : cccccccc`cccccccc 000000d1`b98ff808 00000000`0001211b 000000d1`b98ff838 : 0x000000d1`b94ffa58
000000d1`b98ff6d8 cccccccc`cccccccc : 000000d1`b98ff808 00000000`0001211b 000000d1`b98ff838 cccccccc`00000018 : 0xcccccccc`cccccccc
000000d1`b98ff6e0 000000d1`b98ff808 : 00000000`0001211b 000000d1`b98ff838 cccccccc`00000018 00000000`00000000 : 0xcccccccc`cccccccc
000000d1`b98ff6e8 00000000`0001211b : 000000d1`b98ff838 cccccccc`00000018 00000000`00000000 00000000`00000000 : 0x000000d1`b98ff808
000000d1`b98ff6f0 000000d1`b98ff838 : cccccccc`00000018 00000000`00000000 00000000`00000000 cccccccc`cccccccc : 0x1211b
000000d1`b98ff6f8 cccccccc`00000018 : 00000000`00000000 00000000`00000000 cccccccc`cccccccc 00000000`00000051 : 0x000000d1`b98ff838
000000d1`b98ff700 00000000`00000000 : 00000000`00000000 cccccccc`cccccccc 00000000`00000051 cccccccc`cccccccc : 0xcccccccc`00000018
So it fails at AfdRioResizeCq+0x28b, which looks like this:
KeAcquireInStackQueuedSpinLock((PKSPIN_LOCK)v13, &LockHandle);
v24 = 1;
CqSize = v14->CqSize;
v31 = CqSize;
MappedMemory = (unsigned int *)v14->QueueCq;
v5 = *MappedMemory; -> fails here
This means that the MappedMemory member has already been NULL out, thus leading to the NULL dereference, crashing the kernel. This can be explained like so:
When resizing the queue, it searches for the queue that is requested from user-mode application, then continues with other work. However, in another thread, in trying to destroy the same queue in
AfdRioInvalidateCq, it also searches for the same queue. This eventually lands at [2], where the destroy thread acquires the lock, thereby putting the resizing thread on hold.When the resizing thread is on hold, the destroying thread continues to remove the associated queue from the table. Then it releases the lock, but quickly acquires another one for zeroing out the member of the queue. At this stage, the resizing thread is still on hold.
After the destroy thread is finished, the lock is released at [3], the resizing thread continues its execution. However, the queue now is destroyed, meaning all its member are now zero. It continues to run to [1], but now
MappedMemoryis NULL, thus it dereferences a NULL pointer, leading to the crash in the crash context above.
Challenges
Just resizing and destroying the same completion queue sounds simple right? We just need to utilize the API, then run it on 2 different threads. When I first tried to craft the proof-of-concept for this vulnerability, I found that the crash did happen, but very rarely. Sometimes, it did not even cause a crash. Does this mean that the crash is there, but it is an unstable crash?
This is when my mentor came to the rescue. He looked at the DLL that provided the entrypoint for the Registered Input/Output API, mswsock.dll, and pointed out that there was a race condition in the DLL itself:
int64_t RIOResizeCompletionQueue(int128_t* arg1, int32_t arg2)
{
void var_b8;
int64_t rax_1 = (__security_cookie ^ &var_b8);
uint128_t* rbx = arg1;
if ((arg1 == 0 || (arg2 - 1) > 0x7ffffff))
SetLastError(0x2726);
...
RtlFreeHeap(SockPrivateHeap, 0, var_40_1); // Global heap variable used in multiple functions
}
}
return __security_check_cookie((rax_1 ^ &var_b8));
}
In this resize function, after some processing, it frees the SockPrivateHeap without any multi-threading protection.
We can also see that in RIOCloseCompletionQueue, it also frees that same variable, without any locking as well:
void RIOCloseCompletionQueue(int32_t* arg1)
{
if (arg1 != 0)
{
if (*(uint32_t*)arg1 != 0xa1b2c3d4)
trap(0xd);
arg_c = arg1[2];
int64_t MswRioRegDomain_1 = MswRioRegDomain;
arg_8 = 1;
if (MswRioRegDomain_1 == -1)
MswRioRegDomain_1 = CreateRegDomain();
void var_18;
NtDeviceIoControlFile(MswRioRegDomain_1, 0, 0, 0, &var_18, 0x1211b, &arg_8, 8, 0, 0);
int64_t r8_1 = *(uint64_t*)((char*)arg1 + 0x10);
int64_t SockPrivateHeap_1 = SockPrivateHeap;
*(uint32_t*)arg1 = 0;
RtlFreeHeap(SockPrivateHeap_1, 0, r8_1);
RtlFreeHeap(SockPrivateHeap, 0, arg1);
}
}
This leads to an unstable race condition since the user-mode program is likely to crash first, before it can cause a race condition in the driver. Thus, only by re-implementing the Registered Input/Output API will we have a stable crash.
Timeline
- 17 Mar 2024: Reported to MSRC
- 18 Mar 2024: Assigned MSRC Case 86495
- 02 Apr 2024: MSRC classified this as Moderate-severity DoS and closed case
Acknowledgements
I would like to thank my mentor @b1thvn_ for guiding and assisting me on finding this crash, and Yong for giving me the opportunity to work with all the wonderful people in the team.