Friday, May 4, 2007

Analyzing an Intrusion Part III - Corporal and anamnestic

It's been a few days but I've had some time to delve in to the image of the compromised honeypot. I've come across a few surprises and struggled with an evaluation of some software, but it's coming along so here we go...

Corporal and Anamnestic evidence related to the incident that was found on the honeypot:

Time Zone Information:
Time is incredibly important so we need to establish if the TZ settings were correct.
The active time zone information was -4 hours from GMT which is correct.
System Log:

Event Type: Error Date: 04/22/2007 Time: 06:10:10 PM Source: Service Control Manager
Event ID: 7034 EventDescription:The DNS Server service terminated unexpectedly. It has done this 1 time.

AHA! so as suspected, this event was captured by the event log. Note how innocuous this message is. It doesn't say "the DNS service was killed". Also note the time the service was killed because this becomes important as we try to understand the script the attacker executed. Pay attention to the timing as you go through the rest of this.

Security Logs:
To cull the security logs, I tried a number of methods. As this is a honeynet, I use it to test new and different methods and software on a regular basis.

Harlan Carvey details a few methods in his new book Windows Forensic Analysis and I decided to give them a whirl. As an aside, I'd like to plug this book. It's a fantastic resource rich with accurate information, and if you do any sort of windows forensics, it's a must have

First off, the Security logs were corrupted, so that added a modicum of difficulty to the process.
Harlan's most wonderful lsevt2 perl script was able to parse file even though it was corrupted. However, even using csv output, I was unable to import it in to a database or excel because some of some apparent font or type issues, or perhaps they are parsing issues but I've yet to determine that.

The second method was to repair the event logs as detailed here. This site also happens to be mentioned in Harlan's book. This worked and I was able to view the logs as normal in the event viewer.

The third method I tried was using Prodiscover IR. I am impressed that they included an event log viewer in the program, however it crashed reliably when processing the security event logs. Maybe it was too large for the program to handle, but it would just exit, with no warning.

After I repaired the security logs, I used logparser(which I'm still getting the hang of) to narrow down what I wanted to look at.

logparser "select * from SecEvent.EVT where eventid in (592;593) and to_string(timegenerated, 'yyyy-MM-dd HH:mm:ss') like '2007-04%'"
-i:EVT -o:csv and heavily edited the output to make it fit here.

The following shows the execution timeline of the attackers script. You can see the definitive path of the script as it executed. First, a quick explanation. Event ID 592, and 593 are created because process tracking audting is enabled. 592 is for process creation, and 593 is when a process exits. In order for the strings column to make sense you need to know that the far right PID is what spawned the process(parent process), and the PID on the left is the PID of the listed process.

So, we can see cmd.exe was spawned by PID 1732, and was assigned a PID of 4028 in the first line. If you go through the list diagonally like you're tying shoes, you'll see the pattern rather quickly. I've color coded the like PIDS to try to help.

4/22/2007 18:09:53,592,8,"4028,C:\WINDOWS\system32\cmd.exe,1732,"
4/22/2007 18:09:56,592,8,"2124,C:\WINDOWS\system32\net.exe,4028,"
4/22/2007 18:09:56,592,8,"2728,C:\WINDOWS\system32\net1.exe,2124"
4/22/2007 18:09:56,593,8,"2728,C:\WINDOWS\system32\net1.exe,"
4/22/2007 18:09:56,593,8,"2124,C:\WINDOWS\system32\net.exe,"
4/22/2007 18:10:03,592,8,"3836,C:\WINDOWS\system32\cscript.exe,4028,"
4/22/2007 18:10:09,593,8,"3836,C:\WINDOWS\system32\cscript.exe,"
4/22/2007 18:10:09,592,8,"832,C:\WINDOWS\system32\reg.exe,4028,"
4/22/2007 18:10:09,593,8,"832,C:\WINDOWS\system32\reg.exe,"
4/22/2007 18:10:09,592,8,"3244,C:\WINDOWS\system32\reg.exe,4028,"
4/22/2007 18:10:09,593,8,"3244,C:\WINDOWS\system32\reg.exe,"
4/22/2007 18:10:09,592,8,"4036,C:\WINDOWS\system32\reg.exe,4028,"
4/22/2007 18:10:09,593,8,"4036,C:\WINDOWS\system32\reg.exe,"
4/22/2007 18:10:09,592,8,"2668,C:\WINDOWS\system32\reg.exe,4028,"
4/22/2007 18:10:09,593,8,"2668,C:\WINDOWS\system32\reg.exe,"
4/22/2007 18:10:09,592,8,"328,C:\WINDOWS\system32\reg.exe,4028"
4/22/2007 18:10:09,593,8,"328,C:\WINDOWS\system32\reg.exe,"
4/22/2007 18:10:09,592,8,"1140,C:\WINDOWS\system32\reg.exe,4028,"
4/22/2007 18:10:09,593,8,"1140,C:\WINDOWS\system32\reg.exe,"
4/22/2007 18:10:09,592,8,"3996,C:\WINDOWS\system32\reg.exe,4028,"
4/22/2007 18:10:09,593,8,"3996,C:\WINDOWS\system32\reg.exe,"
4/22/2007 18:10:09,592,8,"1012,C:\WINDOWS\system32\pwdump.exe,4028,"
4/22/2007 18:10:09,593,8,"1012,C:\WINDOWS\system32\pwdump.exe,"
4/22/2007 18:10:10,593,8,"1732,C:\WINDOWS\system32\dns.exe,"
4/22/2007 18:10:25,593,8,"4028,C:\WINDOWS\system32\cmd.exe,"

So, just what is happening? We see that PID 1732 started it all. If you jump down to the second to last line, you'll see what PID 1732 was - that's right dns.exe, which is where system level access was granted and the reverse shell was executed from. Every other process listed was called from PID 4028 which is cmd.exe. The reg.exe calls were successful in changing the ErrorReporting values I listed in Part II, so ErrorReporting was effectively disabled.

However, we know that something was missed. Taskkill.exe was listed in the script in many places, and that's what looked like the cause of dns exiting is, however as evidenced by the process tracking log, we can see that taskkill was never called. Is this because it was never called, or did windows miss the boat and not log it? The answer is evident, but I'll make this somewhat interactive. What killed dns.exe? *answer at the end of this entry*

Application Log:
Nothing of value found here.

I also grabbed the IE history using Prodiscover and found the following.
Internet Activity Information file: E:\\PhysicalDrive0.eve\C:\Documents and Settings\Default User\Local Settings\Temporary Internet Files\Content.IE5\index.dat
File Deleted: No
Browser Type: Internet Explorer
File Name: pwdump[1].exe
Last Modified: 04/22/2007 14:06:33
Last Accessed: 04/22/2007 18:10:09
Local Directory:
Type: URL
HTTP Header: HTTP/1.1 200 OK
X-Powered-By: ASP.NET
Content-Type: application/octet-stream
ETag: "809a98f5885c71:ade"
Content-Length: 90624

As we can see by the bolded text, the user SYSTEM downloaded pwdump, as Harlan pointed out.

MAC times:
File Name,File Type,Cr Date,Acc Date,Mod Date
rasadhlp.dll,Executable File,1/4/2006 16:30,4/22/2007 18:09,3/24/2005 19:12
vgaoem.fon,Executable File,3/25/2003 8:00,4/22/2007 18:09,3/25/2003 8:00
cmd.exe,Executable File,3/25/2003 8:00,4/22/2007 18:09,3/24/2005 18:57
dosapp.fon,Executable File,3/25/2003 8:00,4/22/2007 18:09,3/25/2003 8:00
ega40woa.fon,Executable File,3/25/2003 8:00,4/22/2007 18:09,3/25/2003 8:00
cga80woa.fon,Executable File,3/25/2003 8:00,4/22/2007 18:09,3/25/2003 8:00
cga40woa.fon,Executable File,3/25/2003 8:00,4/22/2007 18:09,3/25/2003 8:00
dbg0.log,Unknown File Type,4/22/2007 18:09,4/22/2007 18:09,4/22/2007 18:09
net1.exe,Executable File,1/4/2006 16:31,4/22/2007 18:09,3/24/2005 19:07
net.exe,Executable File,1/4/2006 16:31,4/22/2007 18:09,3/24/2005 19:07
R00000000000a.clb,Unknown File Type,1/4/2006 15:00,4/22/2007 18:10,1/4/2006 15:00
imm32.dll,Executable File,1/4/2006 16:32,4/22/2007 18:10,3/24/2005 19:05
wshext.dll,Executable File,1/4/2006 16:30,4/22/2007 18:10,3/24/2005 19:26
sensapi.dll,Executable File,3/25/2003 8:00,4/22/2007 18:10,3/25/2003 8:00
msisip.dll,Executable File,1/4/2006 16:31,4/22/2007 18:10,3/24/2005 19:07
vbscript.dll,Executable File,1/4/2006 16:30,4/22/2007 18:10,3/24/2005 19:26
rtutils.dll,Executable File,1/4/2006 16:30,4/22/2007 18:10,3/24/2005 19:13
msdart.dll,Executable File,1/4/2006 16:31,4/22/2007 18:10,3/24/2005 19:07
scrobj.dll,Executable File,1/4/2006 16:30,4/22/2007 18:10,3/24/2005 19:13
iphlpapi.dll,Executable File,1/4/2006 16:32,4/22/2007 18:10,3/24/2005 19:05
winmm.dll,Executable File,1/4/2006 16:30,4/22/2007 18:10,3/24/2005 19:26
msado15.dll,Executable File,1/4/2006 16:33,4/22/2007 18:10,3/24/2005 19:07
tapi32.dll,Executable File,1/4/2006 16:30,4/22/2007 18:10,3/24/2005 19:25
msxml3r.dll,Executable File,1/4/2006 16:31,4/22/2007 18:10,3/24/2005 19:07
mscorie.dll,Executable File,1/2/2006 13:15,4/22/2007 18:10,3/24/2003 22:31
9PMKWROZ,Folder,1/2/2006 13:20,4/22/2007 18:10,4/22/2007 18:10
$I30,Unknown File Type,1/2/2006 13:15,4/22/2007 18:10,1/4/2006 16:41
msvcr71.dll,Executable File,1/2/2006 13:15,4/22/2007 18:10,2/24/2003 16:42
v1.1.4322,Folder,1/2/2006 13:15,4/22/2007 18:10,1/4/2006 16:41
mscoree.dll,Executable File,3/24/2005 21:41,4/22/2007 18:10,3/24/2005 21:41
reg.exe,Executable File,1/4/2006 16:30,4/22/2007 18:10,3/24/2005 19:13
cscript.exe,Executable File,1/4/2006 16:32,4/22/2007 18:10,3/24/2005 18:58
msxml3.dll,Executable File,1/4/2006 16:31,4/22/2007 18:10,3/24/2005 19:07
shimeng.dll,Executable File,1/4/2006 16:30,4/22/2007 18:10,3/24/2005 19:13
pwdump[1].exe,Executable File,4/22/2007 18:10,4/22/2007 18:10,4/22/2007 18:10
wbemcons.dll,Executable File,1/4/2006 16:33,4/22/2007 18:10,3/24/2005 19:26
default.LOG,Unknown File Type,1/2/2006 8:06,4/22/2007 18:10,4/22/2007 18:10
system.LOG,Unknown File Type,1/2/2006 8:06,4/22/2007 18:10,4/22/2007 18:10
httperr1.log,Unknown File Type,6/28/2006 0:20,4/22/2007 18:10,4/22/2007 18:10

Anything listed here was touched during the attack. You'll see folders, dll's, exe's, fonts and a few odd files. While these files don't tell us who accessed them, it gives us a blueprint for testing. If we created a virtual machine and ran the attackers scripts, most if not all of these should show up as being accessed.

Of some note is this file: R00000000000a.clb It lives in C:\Windows\Registration and it appears to be related to COM+ applications or having debug rights on a system, which isn't typical for most users. If anyone has more information about this, please share it.

Other locations of artifacts:
$LogFile - If one were so inclined you could reconstruct a lot of the event by using this file alone.
File slack and unallocated space
httperr1.log contained related information as the attacker first scanned my honeypot looking for open ports.

Where there were no artifacts:

I hope this series of entries has shown why we need to use multiple sources of evidence to corroborate and build a strong case and how to do it as well. Digital evidence is largely circumstantial, and we can rarely prove "who" did something, but when evidence from multiple sources is collected and compared, we can reconstruct the events that took place.

*memory analysis coming soon - I hope*

*dns.exe answer*

To answer this question we need to look at the binary pwdump.exe. The strings listed previously indicate some of the capabilities of this particular pwdump binary and provides us enough clues so we can create a hypothesis and test it. I hypothesized that the script didn't actually kill dns, and the script doesn't directly patch the dns hole.
To test this I created an exemplar system running Windows Server 2003 SP1 and DNS to test my hypothesis. The test system was created using VMware Server 1.02 build 39867 running on an Ubuntu Server system.

The test:
copy pwdump[1].exe to the test system, and execute it using: pwdump[1].exe %RANDOM%.

Upon Execution, we can see the following at the command prompt:

The system was then restored to a snapshot and pwdump was run again, this time by Inctrl5.

InCtrl5 excerpt:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\DNS\Parameters "RpcProtocol"
Old type: REG_DWORD
New type: REG_DWORD
Old data: 01, 00, 00, 00
New data: 04, 00, 00, 00

System Event log:
Event Type: Error Date: 05/04/2007 Time: 12:38:13 PM Source: Service Control Manager
Event ID: 7034 EventDescription:The DNS Server service terminated unexpectedly. It has done this 1 time.

pwdump kills dns.exe and "patches" the registry by using the Microsoft workaround.
This pwdump binary also unpacks the samdump.dll that is contained within it.
If anyone is interested in this file for review, let me know.


Anonymous said...

I learned a lot from this series. I hope you're able to make similar posts in the future. Thanks hogfly!

hogfly said...

Great! Can you tell me what was helpful?

Anonymous said...

Well, you took evidence of an incident collected from multiple sources and walked us through analyzing it. I hope to someday get a job as an intrusion analyst, and you've given some great insight as to what a real incident looks like, where evidence can be found, and how to analyze it. It's cool stuff!

Keydet89 said...

However, even using csv output, I was unable to import it in to a database or excel because some of some apparent font or type issues, or perhaps they are parsing issues but I've yet to determine that.

I've been looking at that issue, as some event records contain strings with separators that I haven't been able to come up with a good RegEx for handling. So, what I've done is used the Spreadsheet::WriteExcel module to dump the output directly into a spreadsheet.

As an aside, I'm working on a forensic preprocessor that will in corporate not only the updated code to put the event records into a spreadsheet, but also generate statistics reports based on the Event Logs themselves; such as frequency of certain event IDs, etc.