As I've been developing the methodology I talked about previously , one of the problem areas that's arisen is determining system impact. One component of impact is determining memory consumption. There's a lot of work to be done here because of the complexity of memory management, and there's a lot of work being done currently. If we're after precision, we need to know how many pages have been allocated to a process, how many are used, if they are in swap or resident in memory, what was previously in those pages, and what's in them now.
I started working on a primitive method to determine memory consumption. Note I said consumption rather than displacement, since determining displacement requires a lot of wrench time.
One such method to maybe start determining consumption is to use the debugging tools from Microsoft. The two tools I'll use here are windbg and cdb(same tool actually..just one is command line).
The tool I'm checking out, is another tool included in the debugging toolkit, also used in incident response and live response scenarios.
So..here goes.
fire up a command prompt and run this:
C:\Program Files\Debugging Tools for Windows>cdb.exe -y http://msdl.microsoft.com/download/symbols -o tlist.exe -t
Microsoft (R) Windows Debugger Version 6.7.0005.1
Copyright (c) Microsoft Corporation. All rights reserved.
CommandLine: tlist -t
Symbol search path is: http://msdl.microsoft.com/download/symbols
Executable search path is:
ModLoad: 01000000 012dd000 tlist.exe
ModLoad: 7c900000 7c9b0000 ntdll.dll
ModLoad: 7c800000 7c8f4000 C:\WINDOWS\system32\kernel32.dll
ModLoad: 77c10000 77c68000 C:\WINDOWS\system32\msvcrt.dll
ModLoad: 03000000 03116000 C:\Program Files\Debugging Tools for Windows\dbghel
p.dll
ModLoad: 77dd0000 77e6b000 C:\WINDOWS\system32\ADVAPI32.dll
ModLoad: 77e70000 77f01000 C:\WINDOWS\system32\RPCRT4.dll
ModLoad: 77c00000 77c08000 C:\WINDOWS\system32\VERSION.dll
ModLoad: 7e410000 7e4a0000 C:\WINDOWS\system32\USER32.dll
ModLoad: 77f10000 77f57000 C:\WINDOWS\system32\GDI32.dll
ModLoad: 774e0000 7761d000 C:\WINDOWS\system32\ole32.dll
ModLoad: 77120000 771ac000 C:\WINDOWS\system32\OLEAUT32.dll
(7c.f90): Break instruction exception - code 80000003 (first chance)
eax=00181eb4 ebx=7ffd5000 ecx=00000001 edx=00000002 esi=00181f48 edi=00181eb4
eip=7c901230 esp=0006fb20 ebp=0006fc94 iopl=0 nv up ei pl nz na po nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000202
ntdll!DbgBreakPoint:
7c901230 cc int 3
0:000>
Ok so that's good. Now we're stepping in to the tlist process and we've reached the first breakpoint. Now, fire up windbg and attach to the kernel (CTRL+K), select local and start with !process 0 0 to get a list of processes that are active.
lkd> !process 0 0
PROCESS 82b7c020 SessionId: 0 Cid: 007c Peb: 7ffd5000 ParentCid: 0da8
DirBase: 03660360 ObjectTable: e12f6a48 HandleCount: 7.
Image: tlist.exe
Aha, here's tlist. Now, we can key in on the process (output truncated):
lkd> !process 82b7c020
PROCESS 82b7c020 SessionId: 0 Cid: 007c Peb: 7ffd5000 ParentCid: 0da8
DirBase: 03660360 ObjectTable: e12f6a48 HandleCount: 7.
Image: tlist.exe
VadRoot 829ff7f8 Vads 27 Clone 0 Private 46. Modified 0. Locked 0.
DeviceMap e1b2efd0
Token e1165d48
ElapsedTime 00:05:54.546
UserTime 00:00:00.015
KernelTime 00:00:00.062
QuotaPoolUsage[PagedPool] 27892
QuotaPoolUsage[NonPagedPool] 1080
Working Set Sizes (now,min,max) (236, 50, 345) (944KB, 200KB, 1380KB)
PeakWorkingSetSize 236
VirtualSize 12 Mb
PeakVirtualSize 12 Mb
PageFaultCount 228
MemoryPriority BACKGROUND
BasePriority 8
CommitCharge 829
DebugPort 82e50a60
Look at the bolded text. We can see the process working set size. At present, the tlist process has been allocated 236 pages of memory at 4k. Multiply 236 * 4 and you'll get 944K. So, at the initial BP we see that tlist is "using" 944K of memory.
In the CDB window, if you tell it to 'go' by typing 'g' we'll see what happens to memory usage.
0:000> g
System Process (0)
System (4)
smss.exe (604)
csrss.exe (652)
winlogon.exe (676)
services.exe (720)
svchost.exe (904)
svchost.exe (1004)
svchost.exe (1116)
wuauclt.exe (3272)
svchost.exe (1232)
svchost.exe (1412)
ccSetMgr.exe (1608)
ccEvtMgr.exe (1656)
SPBBCSvc.exe (1772)
spoolsv.exe (428)
DefWatch.exe (332)
Rtvscan.exe (1100) Scan
VMwareService.exe (1384)
alg.exe (1976)
svchost.exe (1916)
wmiapsrv.exe (1444)
lsass.exe (740)
explorer.exe (1528) Program Manager
VMwareTray.exe (632)
VMwareUser.exe (940)
ccApp.exe (944)
VPTray.exe (816) Missing Virus Definitions
PicasaMediaDetector.exe (936) Picasa Media Detector
taskmgr.exe (3252) Windows Task Manager
cmd.exe (3904) Command Prompt - cdb.exe -y http://msdl.microsoft.com/download
/symbols -o tlist -t
windbg.exe (3500) Windows Driver Kit: Debugging Tools
cdb.exe (3496)
tlist.exe (124)
mmc.exe (3216) Performance
hh.exe (4092) Windows Driver Kit: Debugging Tools
eax=0300b7f8 ebx=00000000 ecx=002643e8 edx=00260608 esi=7c90e88e edi=00000000
eip=7c90eb94 esp=0006fe44 ebp=0006ff40 iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
ntdll!KiFastSystemCallRet:
7c90eb94 c3 ret
Groovy, now we get our tlist output and the tool is done running. However, since we haven't freed the process from the debugger, we have a thread in wait state, which means we can figure out how much memory our process actually consumed:
key in on the same process as before in windbg and you'll see the following:
lkd> !process 82b7c020
PROCESS 82b7c020 SessionId: 0 Cid: 007c Peb: 7ffd5000 ParentCid: 0da8
DirBase: 03660360 ObjectTable: e12f6a48 HandleCount: 22.
Image: tlist.exe
VadRoot 82bc3840 Vads 36 Clone 0 Private 380. Modified 0. Locked 0.
DeviceMap e1b2efd0
Token e1165d48
ElapsedTime 00:12:59.265
UserTime 00:00:00.015
KernelTime 00:00:00.140
QuotaPoolUsage[PagedPool] 38236
QuotaPoolUsage[NonPagedPool] 1440
Working Set Sizes (now,min,max) (771, 50, 345) (3084KB, 200KB, 1380KB)
PeakWorkingSetSize 771
VirtualSize 18 Mb
PeakVirtualSize 19 Mb
PageFaultCount 765
MemoryPriority BACKGROUND
BasePriority 8
CommitCharge 1120
DebugPort 82e50a60
So, we see 3084K has been "used", but can that possibly be accurate? The answer of course is no. There are several factors at play.
1) This was run from a debugger - this adds memory allocated to the process to account for debugging.
2) There is shared memory being used by other DLL's.
3) The working set is not representative of memory actually used by the process. It's representative of the memory(shared, virtual, physical) allocated to the process, but not necessarily that which is consumed(or resident). In addition, Microsoft's documentation on MSDN is inconsistent(surprise, surprise) in describing what's actually included in the working set.
4) The working set is what is recorded by the task manager, which is inaccurate.
So, where does this leave us? Well, in terms of precision and accuracy for determining memory consumption for a process, the working set is not the answer because the allocation hasn't necessarily been consumed, maybe the private bytes associated with a process is what we need to focus on but the real question is, and this has yet to be answered..what's good enough? I'm thinking that maybe the working set size is good enough, but not necessarily precise.
Next steps? Boot a system in debug mode and attach a kernel debugger to it while it's live.
Thoughts?
EDITS: the url is download, not downloads.
Showing posts with label impact. Show all posts
Showing posts with label impact. Show all posts
Thursday, July 12, 2007
Subscribe to:
Posts (Atom)