I've been kind of quiet this month and that's due to some personal stuff.
I received my degree yesterday in Computer Forensics. It's been an interesting experience going back to school. I entered the industry around the time of the dotcom craziness as a kid and school was an afterthought at the time. Family became important and school was not. Needless to say when the Forensics programs began across the country, my interest was piqued and voila - here I am.
Until this point I've been self taught. I devour books and constantly study on my own. I think the funniest part of school was that I already had the required books in my bookshelf. I guess you could say I'll never stop being a student...
Now that school is out of the way I'll have some upcoming posts:
The faces of digital forensic science
Peer Review - DOJ forensic analysis methodology
Using KntDD and KntDD Enterprise
More Vista work - Bitlocker(I've not forgotten it) and other goodies
My talk with an EMT
Dissecting Office 2007
Friday, May 18, 2007
Thursday, May 10, 2007
Digital Forensic Science
For a few years now I've been contemplating the discipline called "digital forensic science" and I constantly wonder and ask myself "Is this a science?" and in fact many of my entries here contain something to the effect of "if we are indeed a science".
This needs to start with some definitions.(If someone wants to give me a better definition than what I have below, then please do)
Wikipedia defines science as: In the broadest sense, science (from the Latin "to know") refers to any systematic methodology which attempts to collect accurate information about reality and to model this in a way which can be used to make reliable, concrete and quantitative predictions about future events and observations.
Wikipedia defines Computer Science as: Computer science, or computing science, is the study of the theoretical foundations of information and computation and their implementation and application in computer systems.
Digital Forensic Science as defined by the DFRWS is:
The use of scientifically derived and proven methods toward the preservation,
collection, validation, identification, analysis, interpretation, documentation and
presentation of digital evidence derived from digital sources for the purpose of
facilitating or furthering the reconstruction of events found to be criminal, or
helping to anticipate unauthorized actions shown to be disruptive to planned
operations.
Now that the definitions are out of the way, let's take a deeper look in to this thing called "digital forensic science" or DFS.
DFS falls under the forensic science umbrella which is the application of science to satisfy the questions of the legal system. However, there is a component missing from DFS, and that's the foundation of science. Let me explain. Every other forensic science has a founding science that backs its use in a legal setting. Entomology is the foundation for forensic entomology, odontology is the foundation for forensic odontology and so on. Digital Forensic Science in practice has no scientific foundation. It is a discipline of Computer Science perhaps, but there doesn't appear to be a clear scientific foundation for what is done in the name of DFS.
Based on the definition of DFS per the DFRWS, it is directly related to the legal system and the law. Therefore, we can look at what qualifies as forensic testimony of a digital forensic scientist. Daubert is the predominant qualifier so let's look take a look at it.
Daubert asks us 4 main questions related to scientific forensic testimony.
DFS fails these questions in many cases because these questions assume one thing and that is the scientific foundation of the application of the technique or method being used by an examiner.
To address them in order...
1) Testing.
This is not just testing the theory, but testing in a repeatable manner. The conducted experiment/test must be documented well enough such that it can be repeated and the results should be the same for each test.
This is probably the easiest part of the daubert challenge. However there are many variables that need to be taken in to account that could affect the outcome. When it comes to testing a theory or method, one must attempt to take every variable in to account, lest it be disproved by another scientist. We have the CFTT, but that's tool testing, and there is no standard for theory or method testing. That's not to say I think there should be a hard coded standard, but a framework should exist. The SWGDE and CFTT both have frameworks but as I said, they are for tool testing, not method or process testing and as such these can not be used without modification and modification of the testing method invalidates the procedure as it is no longer "scientifically derived".
2) Peer review.
As I've said before, we have no formal peer review in the field. We have a few journals but practitioners of DFS do not have a body to submit their procedures to for peer review and in many cases they can not because of a policy. Labs rarely receive a certification from a governing body and until the day comes when all computer forensic labs require a certification, the processes and procedures that take place within said lab could be suspect. How many can claim that their procedures and processes have been peer reviewed?
3) Error rates.
There are no published error rates for procedures, tools, methods etc, and often times we must answer "no" to this question, yet errors occur frequently. Vendors place the responsibility of determining rate of error on the user of their product while I tend to believe that the software should have to be submitted to a review group in order to receive a "forensic" stamp of approval. Beta testing is nice, but it's not and should not be a method of software certification, especially when the lives and well being of individuals is at risk.
FTK crashes on a semi-regular basis if you overload it, I've had cases where it would not process an image regardless of what I had done, and by many accounts Encase 6 has too many bugs to count. This is relatively simple to pick at.
Q: "Mr examiner, how can you trust the results of your tools when they crash in the middle of processing the evidence?"
A: "I re-ran the examination procedures and they concluded without fault".
Q: "So you are in the habit of re-testing until you are satisfied with the result?"
Q: "Did you validate your findings with other software?"
A: "Yes I validated my findings using another tool"
Q: "And what is the error rate with that software?"
A: "I don't know of an error rate with that software"
Let me also pick on Anti-malware software. Once upon a time, NIST approved Symantec and Mcafee for virus scanning. If you used these and nothing was found, then you could effectively claim the system was malware free. As we all know the error rates in these tools are absolutely tremendous. How can anyone claim that the system is malware free unless you scan it with every tool available? As a test of what's out there I submitted the pwdump binary from the recent compromise of my honeynet to Norman, virustotal, and jotti's virus scan and out of the 31 scanners at virustotal, only 5 picked it up as malware, only two from jotti(which are included in the 5 from virustotal), and norman claimed it wasn't malware. To boot, none of them had a specific definition for it. In addition, neither symantec or mcafee recognized the binary. We all know that this is a regular occurrence with AV software but unless we scan the file system with every available tool the best we can ever claim is that "given the tools in use and the current definition files, I can say that with a small degree of certainty, this system is malware free." This will permanently leave the door to the trojan defense wide open.
My point is that the tools available today will always have a rate of error, and we must know them, but how many can answer something other than "no" when the question "Is there a known rate of error for the procedure/tool/process you used?" is asked.
In many ways it would seem that DFS fails Daubert or should fail Daubert, because of the lack of scientific foundation in the discipline. However, as we know, examiners regularly survive Daubert and are allowed to testify.
Heading back to the definition, DFS is not strictly about reconstructing events found to be criminal or helping to anticipate unauthorized actions. While it's largely used in a legal or criminal setting, or to determine unauthorized actions, these are merely applications of DFS. Given this, is it reasonable for the definition to describe the application of the science and define it?
As Mark Pollitt suggested in his 2004 keynote at the DFRWS, perhaps Roles need to be added to the Framework. Given this, I think the definition needs to be re-worked.
The science in the case of DFS is not about proof, as we can't actually prove anything in a concrete manner. If you are ever asked for proof of something, can you make a claim that your conclusion is 100% accurate? For instance, can we prove who was actually sitting at the computer unless we actually saw them? The field is largely conjecture and any argument we make is based on circumstantial evidence. Yes, someone with that IP, at that time, logged in using that user name, and visited that website, and downloaded or viewed that image, and created that file. You can see how circumstantial evidence can mount to build a strong case, but is it strong enough? In many cases yes, but we're seeing instances of "the trojan defense", or "The defendant didn't actually knowingly store those CP files in his browser cache". While that's a bit of clever lawyering, it shows that there are holes in our processes and procedures. I will of course submit that there are always going to be holes as nothing is bulletproof but the lack of standardized scientific methods (as the DFRWS claims are used in DFS) creates situations like this.
As such I wonder if we shouldn't rely on something more concrete to help us effectively conclude that what we say happened, actually happened in the way we claim. Under current circumstances, the best we can do is present a degree of certainty or a level of confidence that what happened actually happened as we say it did. Eoghan Casey was on to something in his Second edition of Digital Evidence when he defines a certainty scale for the trustworthiness of digital evidence. I think that we could benefit from a formal use of this type of scale for our findings. I'm also beginning to think that by using math - either statistical confidence intervals or Bayesian credible intervals we can build a stronger model for Digital Forensic Science and therefore we can strengthen the presentation of digital evidence and the use of it in the legal system.
Thoughts?
This needs to start with some definitions.(If someone wants to give me a better definition than what I have below, then please do)
Wikipedia defines science as: In the broadest sense, science (from the Latin "to know") refers to any systematic methodology which attempts to collect accurate information about reality and to model this in a way which can be used to make reliable, concrete and quantitative predictions about future events and observations.
Wikipedia defines Computer Science as: Computer science, or computing science, is the study of the theoretical foundations of information and computation and their implementation and application in computer systems.
Digital Forensic Science as defined by the DFRWS is:
The use of scientifically derived and proven methods toward the preservation,
collection, validation, identification, analysis, interpretation, documentation and
presentation of digital evidence derived from digital sources for the purpose of
facilitating or furthering the reconstruction of events found to be criminal, or
helping to anticipate unauthorized actions shown to be disruptive to planned
operations.
Now that the definitions are out of the way, let's take a deeper look in to this thing called "digital forensic science" or DFS.
DFS falls under the forensic science umbrella which is the application of science to satisfy the questions of the legal system. However, there is a component missing from DFS, and that's the foundation of science. Let me explain. Every other forensic science has a founding science that backs its use in a legal setting. Entomology is the foundation for forensic entomology, odontology is the foundation for forensic odontology and so on. Digital Forensic Science in practice has no scientific foundation. It is a discipline of Computer Science perhaps, but there doesn't appear to be a clear scientific foundation for what is done in the name of DFS.
Based on the definition of DFS per the DFRWS, it is directly related to the legal system and the law. Therefore, we can look at what qualifies as forensic testimony of a digital forensic scientist. Daubert is the predominant qualifier so let's look take a look at it.
Daubert asks us 4 main questions related to scientific forensic testimony.
- Whether the theory used by the expert can be and has been tested.
- Whether the theory or technique has been subjected to peer review.
- The known or potential rate of error of the method used.
- The degree of the methods or conclusion's acceptance within the relevant scientific community.
DFS fails these questions in many cases because these questions assume one thing and that is the scientific foundation of the application of the technique or method being used by an examiner.
To address them in order...
1) Testing.
This is not just testing the theory, but testing in a repeatable manner. The conducted experiment/test must be documented well enough such that it can be repeated and the results should be the same for each test.
This is probably the easiest part of the daubert challenge. However there are many variables that need to be taken in to account that could affect the outcome. When it comes to testing a theory or method, one must attempt to take every variable in to account, lest it be disproved by another scientist. We have the CFTT, but that's tool testing, and there is no standard for theory or method testing. That's not to say I think there should be a hard coded standard, but a framework should exist. The SWGDE and CFTT both have frameworks but as I said, they are for tool testing, not method or process testing and as such these can not be used without modification and modification of the testing method invalidates the procedure as it is no longer "scientifically derived".
2) Peer review.
As I've said before, we have no formal peer review in the field. We have a few journals but practitioners of DFS do not have a body to submit their procedures to for peer review and in many cases they can not because of a policy. Labs rarely receive a certification from a governing body and until the day comes when all computer forensic labs require a certification, the processes and procedures that take place within said lab could be suspect. How many can claim that their procedures and processes have been peer reviewed?
3) Error rates.
There are no published error rates for procedures, tools, methods etc, and often times we must answer "no" to this question, yet errors occur frequently. Vendors place the responsibility of determining rate of error on the user of their product while I tend to believe that the software should have to be submitted to a review group in order to receive a "forensic" stamp of approval. Beta testing is nice, but it's not and should not be a method of software certification, especially when the lives and well being of individuals is at risk.
FTK crashes on a semi-regular basis if you overload it, I've had cases where it would not process an image regardless of what I had done, and by many accounts Encase 6 has too many bugs to count. This is relatively simple to pick at.
Q: "Mr examiner, how can you trust the results of your tools when they crash in the middle of processing the evidence?"
A: "I re-ran the examination procedures and they concluded without fault".
Q: "So you are in the habit of re-testing until you are satisfied with the result?"
Q: "Did you validate your findings with other software?"
A: "Yes I validated my findings using another tool"
Q: "And what is the error rate with that software?"
A: "I don't know of an error rate with that software"
Let me also pick on Anti-malware software. Once upon a time, NIST approved Symantec and Mcafee for virus scanning. If you used these and nothing was found, then you could effectively claim the system was malware free. As we all know the error rates in these tools are absolutely tremendous. How can anyone claim that the system is malware free unless you scan it with every tool available? As a test of what's out there I submitted the pwdump binary from the recent compromise of my honeynet to Norman, virustotal, and jotti's virus scan and out of the 31 scanners at virustotal, only 5 picked it up as malware, only two from jotti(which are included in the 5 from virustotal), and norman claimed it wasn't malware. To boot, none of them had a specific definition for it. In addition, neither symantec or mcafee recognized the binary. We all know that this is a regular occurrence with AV software but unless we scan the file system with every available tool the best we can ever claim is that "given the tools in use and the current definition files, I can say that with a small degree of certainty, this system is malware free." This will permanently leave the door to the trojan defense wide open.
My point is that the tools available today will always have a rate of error, and we must know them, but how many can answer something other than "no" when the question "Is there a known rate of error for the procedure/tool/process you used?" is asked.
In many ways it would seem that DFS fails Daubert or should fail Daubert, because of the lack of scientific foundation in the discipline. However, as we know, examiners regularly survive Daubert and are allowed to testify.
Heading back to the definition, DFS is not strictly about reconstructing events found to be criminal or helping to anticipate unauthorized actions. While it's largely used in a legal or criminal setting, or to determine unauthorized actions, these are merely applications of DFS. Given this, is it reasonable for the definition to describe the application of the science and define it?
As Mark Pollitt suggested in his 2004 keynote at the DFRWS, perhaps Roles need to be added to the Framework. Given this, I think the definition needs to be re-worked.
The science in the case of DFS is not about proof, as we can't actually prove anything in a concrete manner. If you are ever asked for proof of something, can you make a claim that your conclusion is 100% accurate? For instance, can we prove who was actually sitting at the computer unless we actually saw them? The field is largely conjecture and any argument we make is based on circumstantial evidence. Yes, someone with that IP, at that time, logged in using that user name, and visited that website, and downloaded or viewed that image, and created that file. You can see how circumstantial evidence can mount to build a strong case, but is it strong enough? In many cases yes, but we're seeing instances of "the trojan defense", or "The defendant didn't actually knowingly store those CP files in his browser cache". While that's a bit of clever lawyering, it shows that there are holes in our processes and procedures. I will of course submit that there are always going to be holes as nothing is bulletproof but the lack of standardized scientific methods (as the DFRWS claims are used in DFS) creates situations like this.
As such I wonder if we shouldn't rely on something more concrete to help us effectively conclude that what we say happened, actually happened in the way we claim. Under current circumstances, the best we can do is present a degree of certainty or a level of confidence that what happened actually happened as we say it did. Eoghan Casey was on to something in his Second edition of Digital Evidence when he defines a certainty scale for the trustworthiness of digital evidence. I think that we could benefit from a formal use of this type of scale for our findings. I'm also beginning to think that by using math - either statistical confidence intervals or Bayesian credible intervals we can build a stronger model for Digital Forensic Science and therefore we can strengthen the presentation of digital evidence and the use of it in the legal system.
Thoughts?
Friday, May 4, 2007
Analyzing an Intrusion Part III - Corporal and anamnestic
It's been a few days but I've had some time to delve in to the image of the compromised honeypot. I've come across a few surprises and struggled with an evaluation of some software, but it's coming along so here we go...
Corporal and Anamnestic evidence related to the incident that was found on the honeypot:
Time Zone Information:
Time is incredibly important so we need to establish if the TZ settings were correct.
The active time zone information was -4 hours from GMT which is correct.
System Log:
Event Type: Error Date: 04/22/2007 Time: 06:10:10 PM Source: Service Control Manager
Event ID: 7034 EventDescription:The DNS Server service terminated unexpectedly. It has done this 1 time.
AHA! so as suspected, this event was captured by the event log. Note how innocuous this message is. It doesn't say "the DNS service was killed". Also note the time the service was killed because this becomes important as we try to understand the script the attacker executed. Pay attention to the timing as you go through the rest of this.
Security Logs:
To cull the security logs, I tried a number of methods. As this is a honeynet, I use it to test new and different methods and software on a regular basis.
Harlan Carvey details a few methods in his new book Windows Forensic Analysis and I decided to give them a whirl. As an aside, I'd like to plug this book. It's a fantastic resource rich with accurate information, and if you do any sort of windows forensics, it's a must have
First off, the Security logs were corrupted, so that added a modicum of difficulty to the process.
Harlan's most wonderful lsevt2 perl script was able to parse file even though it was corrupted. However, even using csv output, I was unable to import it in to a database or excel because some of some apparent font or type issues, or perhaps they are parsing issues but I've yet to determine that.
The second method was to repair the event logs as detailed here. This site also happens to be mentioned in Harlan's book. This worked and I was able to view the logs as normal in the event viewer.
The third method I tried was using Prodiscover IR. I am impressed that they included an event log viewer in the program, however it crashed reliably when processing the security event logs. Maybe it was too large for the program to handle, but it would just exit, with no warning.
After I repaired the security logs, I used logparser(which I'm still getting the hang of) to narrow down what I wanted to look at.
logparser "select * from SecEvent.EVT where eventid in (592;593) and to_string(timegenerated, 'yyyy-MM-dd HH:mm:ss') like '2007-04%'"
-i:EVT -o:csv and heavily edited the output to make it fit here.
The following shows the execution timeline of the attackers script. You can see the definitive path of the script as it executed. First, a quick explanation. Event ID 592, and 593 are created because process tracking audting is enabled. 592 is for process creation, and 593 is when a process exits. In order for the strings column to make sense you need to know that the far right PID is what spawned the process(parent process), and the PID on the left is the PID of the listed process.
So, we can see cmd.exe was spawned by PID 1732, and was assigned a PID of 4028 in the first line. If you go through the list diagonally like you're tying shoes, you'll see the pattern rather quickly. I've color coded the like PIDS to try to help.
TimeGenerated,EventID,EventType,Strings
4/22/2007 18:09:53,592,8,"4028,C:\WINDOWS\system32\cmd.exe,1732,"
4/22/2007 18:09:56,592,8,"2124,C:\WINDOWS\system32\net.exe,4028,"
4/22/2007 18:09:56,592,8,"2728,C:\WINDOWS\system32\net1.exe,2124"
4/22/2007 18:09:56,593,8,"2728,C:\WINDOWS\system32\net1.exe,"
4/22/2007 18:09:56,593,8,"2124,C:\WINDOWS\system32\net.exe,"
4/22/2007 18:10:03,592,8,"3836,C:\WINDOWS\system32\cscript.exe,4028,"
4/22/2007 18:10:09,593,8,"3836,C:\WINDOWS\system32\cscript.exe,"
4/22/2007 18:10:09,592,8,"832,C:\WINDOWS\system32\reg.exe,4028,"
4/22/2007 18:10:09,593,8,"832,C:\WINDOWS\system32\reg.exe,"
4/22/2007 18:10:09,592,8,"3244,C:\WINDOWS\system32\reg.exe,4028,"
4/22/2007 18:10:09,593,8,"3244,C:\WINDOWS\system32\reg.exe,"
4/22/2007 18:10:09,592,8,"4036,C:\WINDOWS\system32\reg.exe,4028,"
4/22/2007 18:10:09,593,8,"4036,C:\WINDOWS\system32\reg.exe,"
4/22/2007 18:10:09,592,8,"2668,C:\WINDOWS\system32\reg.exe,4028,"
4/22/2007 18:10:09,593,8,"2668,C:\WINDOWS\system32\reg.exe,"
4/22/2007 18:10:09,592,8,"328,C:\WINDOWS\system32\reg.exe,4028"
4/22/2007 18:10:09,593,8,"328,C:\WINDOWS\system32\reg.exe,"
4/22/2007 18:10:09,592,8,"1140,C:\WINDOWS\system32\reg.exe,4028,"
4/22/2007 18:10:09,593,8,"1140,C:\WINDOWS\system32\reg.exe,"
4/22/2007 18:10:09,592,8,"3996,C:\WINDOWS\system32\reg.exe,4028,"
4/22/2007 18:10:09,593,8,"3996,C:\WINDOWS\system32\reg.exe,"
4/22/2007 18:10:09,592,8,"1012,C:\WINDOWS\system32\pwdump.exe,4028,"
4/22/2007 18:10:09,593,8,"1012,C:\WINDOWS\system32\pwdump.exe,"
4/22/2007 18:10:10,593,8,"1732,C:\WINDOWS\system32\dns.exe,"
4/22/2007 18:10:25,593,8,"4028,C:\WINDOWS\system32\cmd.exe,"
So, just what is happening? We see that PID 1732 started it all. If you jump down to the second to last line, you'll see what PID 1732 was - that's right dns.exe, which is where system level access was granted and the reverse shell was executed from. Every other process listed was called from PID 4028 which is cmd.exe. The reg.exe calls were successful in changing the ErrorReporting values I listed in Part II, so ErrorReporting was effectively disabled.
However, we know that something was missed. Taskkill.exe was listed in the script in many places, and that's what looked like the cause of dns exiting is, however as evidenced by the process tracking log, we can see that taskkill was never called. Is this because it was never called, or did windows miss the boat and not log it? The answer is evident, but I'll make this somewhat interactive. What killed dns.exe? *answer at the end of this entry*
Application Log:
Nothing of value found here.
I also grabbed the IE history using Prodiscover and found the following.
Internet Activity Information file: E:\132.236.62.102-04222007\PhysicalDrive0.eve\C:\Documents and Settings\Default User\Local Settings\Temporary Internet Files\Content.IE5\index.dat
File Deleted: No
Browser Type: Internet Explorer
URL: http://www.dit.net/images/pwdump.exe
File Name: pwdump[1].exe
Last Modified: 04/22/2007 14:06:33
Last Accessed: 04/22/2007 18:10:09
Local Directory:
Type: URL
HTTP Header: HTTP/1.1 200 OK
X-Powered-By: ASP.NET
Content-Type: application/octet-stream
ETag: "809a98f5885c71:ade"
Content-Length: 90624
~U:system
As we can see by the bolded text, the user SYSTEM downloaded pwdump, as Harlan pointed out.
MAC times:
File Name,File Type,Cr Date,Acc Date,Mod Date
rasadhlp.dll,Executable File,1/4/2006 16:30,4/22/2007 18:09,3/24/2005 19:12
vgaoem.fon,Executable File,3/25/2003 8:00,4/22/2007 18:09,3/25/2003 8:00
cmd.exe,Executable File,3/25/2003 8:00,4/22/2007 18:09,3/24/2005 18:57
dosapp.fon,Executable File,3/25/2003 8:00,4/22/2007 18:09,3/25/2003 8:00
ega40woa.fon,Executable File,3/25/2003 8:00,4/22/2007 18:09,3/25/2003 8:00
cga80woa.fon,Executable File,3/25/2003 8:00,4/22/2007 18:09,3/25/2003 8:00
cga40woa.fon,Executable File,3/25/2003 8:00,4/22/2007 18:09,3/25/2003 8:00
dbg0.log,Unknown File Type,4/22/2007 18:09,4/22/2007 18:09,4/22/2007 18:09
net1.exe,Executable File,1/4/2006 16:31,4/22/2007 18:09,3/24/2005 19:07
net.exe,Executable File,1/4/2006 16:31,4/22/2007 18:09,3/24/2005 19:07
R00000000000a.clb,Unknown File Type,1/4/2006 15:00,4/22/2007 18:10,1/4/2006 15:00
imm32.dll,Executable File,1/4/2006 16:32,4/22/2007 18:10,3/24/2005 19:05
wshext.dll,Executable File,1/4/2006 16:30,4/22/2007 18:10,3/24/2005 19:26
sensapi.dll,Executable File,3/25/2003 8:00,4/22/2007 18:10,3/25/2003 8:00
msisip.dll,Executable File,1/4/2006 16:31,4/22/2007 18:10,3/24/2005 19:07
vbscript.dll,Executable File,1/4/2006 16:30,4/22/2007 18:10,3/24/2005 19:26
rtutils.dll,Executable File,1/4/2006 16:30,4/22/2007 18:10,3/24/2005 19:13
msdart.dll,Executable File,1/4/2006 16:31,4/22/2007 18:10,3/24/2005 19:07
scrobj.dll,Executable File,1/4/2006 16:30,4/22/2007 18:10,3/24/2005 19:13
iphlpapi.dll,Executable File,1/4/2006 16:32,4/22/2007 18:10,3/24/2005 19:05
winmm.dll,Executable File,1/4/2006 16:30,4/22/2007 18:10,3/24/2005 19:26
msado15.dll,Executable File,1/4/2006 16:33,4/22/2007 18:10,3/24/2005 19:07
tapi32.dll,Executable File,1/4/2006 16:30,4/22/2007 18:10,3/24/2005 19:25
msxml3r.dll,Executable File,1/4/2006 16:31,4/22/2007 18:10,3/24/2005 19:07
mscorie.dll,Executable File,1/2/2006 13:15,4/22/2007 18:10,3/24/2003 22:31
9PMKWROZ,Folder,1/2/2006 13:20,4/22/2007 18:10,4/22/2007 18:10
$I30,Unknown File Type,1/2/2006 13:15,4/22/2007 18:10,1/4/2006 16:41
msvcr71.dll,Executable File,1/2/2006 13:15,4/22/2007 18:10,2/24/2003 16:42
v1.1.4322,Folder,1/2/2006 13:15,4/22/2007 18:10,1/4/2006 16:41
mscoree.dll,Executable File,3/24/2005 21:41,4/22/2007 18:10,3/24/2005 21:41
reg.exe,Executable File,1/4/2006 16:30,4/22/2007 18:10,3/24/2005 19:13
cscript.exe,Executable File,1/4/2006 16:32,4/22/2007 18:10,3/24/2005 18:58
msxml3.dll,Executable File,1/4/2006 16:31,4/22/2007 18:10,3/24/2005 19:07
shimeng.dll,Executable File,1/4/2006 16:30,4/22/2007 18:10,3/24/2005 19:13
pwdump[1].exe,Executable File,4/22/2007 18:10,4/22/2007 18:10,4/22/2007 18:10
wbemcons.dll,Executable File,1/4/2006 16:33,4/22/2007 18:10,3/24/2005 19:26
default.LOG,Unknown File Type,1/2/2006 8:06,4/22/2007 18:10,4/22/2007 18:10
system.LOG,Unknown File Type,1/2/2006 8:06,4/22/2007 18:10,4/22/2007 18:10
httperr1.log,Unknown File Type,6/28/2006 0:20,4/22/2007 18:10,4/22/2007 18:10
Anything listed here was touched during the attack. You'll see folders, dll's, exe's, fonts and a few odd files. While these files don't tell us who accessed them, it gives us a blueprint for testing. If we created a virtual machine and ran the attackers scripts, most if not all of these should show up as being accessed.
Of some note is this file: R00000000000a.clb It lives in C:\Windows\Registration and it appears to be related to COM+ applications or having debug rights on a system, which isn't typical for most users. If anyone has more information about this, please share it.
Other locations of artifacts:
$LogFile - If one were so inclined you could reconstruct a lot of the event by using this file alone.
File slack and unallocated space
httperr1.log contained related information as the attacker first scanned my honeypot looking for open ports.
Where there were no artifacts:
Prefetch
MRU
MUICache
UserAssist
I hope this series of entries has shown why we need to use multiple sources of evidence to corroborate and build a strong case and how to do it as well. Digital evidence is largely circumstantial, and we can rarely prove "who" did something, but when evidence from multiple sources is collected and compared, we can reconstruct the events that took place.
*memory analysis coming soon - I hope*
*dns.exe answer*
To answer this question we need to look at the binary pwdump.exe. The strings listed previously indicate some of the capabilities of this particular pwdump binary and provides us enough clues so we can create a hypothesis and test it. I hypothesized that the script didn't actually kill dns, and the script doesn't directly patch the dns hole.
To test this I created an exemplar system running Windows Server 2003 SP1 and DNS to test my hypothesis. The test system was created using VMware Server 1.02 build 39867 running on an Ubuntu Server system.
The test:
copy pwdump[1].exe to the test system, and execute it using: pwdump[1].exe %RANDOM%.
Result:
Upon Execution, we can see the following at the command prompt:
The system was then restored to a snapshot and pwdump was run again, this time by Inctrl5.
InCtrl5 excerpt:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\DNS\Parameters "RpcProtocol"
Old type: REG_DWORD
New type: REG_DWORD
Old data: 01, 00, 00, 00
New data: 04, 00, 00, 00
System Event log:
Event Type: Error Date: 05/04/2007 Time: 12:38:13 PM Source: Service Control Manager
Event ID: 7034 EventDescription:The DNS Server service terminated unexpectedly. It has done this 1 time.
Conclusion:
pwdump kills dns.exe and "patches" the registry by using the Microsoft workaround.
This pwdump binary also unpacks the samdump.dll that is contained within it.
If anyone is interested in this file for review, let me know.
Corporal and Anamnestic evidence related to the incident that was found on the honeypot:
Time Zone Information:
Time is incredibly important so we need to establish if the TZ settings were correct.
The active time zone information was -4 hours from GMT which is correct.
System Log:
Event Type: Error Date: 04/22/2007 Time: 06:10:10 PM Source: Service Control Manager
Event ID: 7034 EventDescription:The DNS Server service terminated unexpectedly. It has done this 1 time.
AHA! so as suspected, this event was captured by the event log. Note how innocuous this message is. It doesn't say "the DNS service was killed". Also note the time the service was killed because this becomes important as we try to understand the script the attacker executed. Pay attention to the timing as you go through the rest of this.
Security Logs:
To cull the security logs, I tried a number of methods. As this is a honeynet, I use it to test new and different methods and software on a regular basis.
Harlan Carvey details a few methods in his new book Windows Forensic Analysis and I decided to give them a whirl. As an aside, I'd like to plug this book. It's a fantastic resource rich with accurate information, and if you do any sort of windows forensics, it's a must have
First off, the Security logs were corrupted, so that added a modicum of difficulty to the process.
Harlan's most wonderful lsevt2 perl script was able to parse file even though it was corrupted. However, even using csv output, I was unable to import it in to a database or excel because some of some apparent font or type issues, or perhaps they are parsing issues but I've yet to determine that.
The second method was to repair the event logs as detailed here. This site also happens to be mentioned in Harlan's book. This worked and I was able to view the logs as normal in the event viewer.
The third method I tried was using Prodiscover IR. I am impressed that they included an event log viewer in the program, however it crashed reliably when processing the security event logs. Maybe it was too large for the program to handle, but it would just exit, with no warning.
After I repaired the security logs, I used logparser(which I'm still getting the hang of) to narrow down what I wanted to look at.
logparser "select * from SecEvent.EVT where eventid in (592;593) and to_string(timegenerated, 'yyyy-MM-dd HH:mm:ss') like '2007-04%'"
-i:EVT -o:csv and heavily edited the output to make it fit here.
The following shows the execution timeline of the attackers script. You can see the definitive path of the script as it executed. First, a quick explanation. Event ID 592, and 593 are created because process tracking audting is enabled. 592 is for process creation, and 593 is when a process exits. In order for the strings column to make sense you need to know that the far right PID is what spawned the process(parent process), and the PID on the left is the PID of the listed process.
So, we can see cmd.exe was spawned by PID 1732, and was assigned a PID of 4028 in the first line. If you go through the list diagonally like you're tying shoes, you'll see the pattern rather quickly. I've color coded the like PIDS to try to help.
TimeGenerated,EventID,EventType,Strings
4/22/2007 18:09:53,592,8,"4028,C:\WINDOWS\system32\cmd.exe,1732,"
4/22/2007 18:09:56,592,8,"2124,C:\WINDOWS\system32\net.exe,4028,"
4/22/2007 18:09:56,592,8,"2728,C:\WINDOWS\system32\net1.exe,2124"
4/22/2007 18:09:56,593,8,"2728,C:\WINDOWS\system32\net1.exe,"
4/22/2007 18:09:56,593,8,"2124,C:\WINDOWS\system32\net.exe,"
4/22/2007 18:10:03,592,8,"3836,C:\WINDOWS\system32\cscript.exe,4028,"
4/22/2007 18:10:09,593,8,"3836,C:\WINDOWS\system32\cscript.exe,"
4/22/2007 18:10:09,592,8,"832,C:\WINDOWS\system32\reg.exe,4028,"
4/22/2007 18:10:09,593,8,"832,C:\WINDOWS\system32\reg.exe,"
4/22/2007 18:10:09,592,8,"3244,C:\WINDOWS\system32\reg.exe,4028,"
4/22/2007 18:10:09,593,8,"3244,C:\WINDOWS\system32\reg.exe,"
4/22/2007 18:10:09,592,8,"4036,C:\WINDOWS\system32\reg.exe,4028,"
4/22/2007 18:10:09,593,8,"4036,C:\WINDOWS\system32\reg.exe,"
4/22/2007 18:10:09,592,8,"2668,C:\WINDOWS\system32\reg.exe,4028,"
4/22/2007 18:10:09,593,8,"2668,C:\WINDOWS\system32\reg.exe,"
4/22/2007 18:10:09,592,8,"328,C:\WINDOWS\system32\reg.exe,4028"
4/22/2007 18:10:09,593,8,"328,C:\WINDOWS\system32\reg.exe,"
4/22/2007 18:10:09,592,8,"1140,C:\WINDOWS\system32\reg.exe,4028,"
4/22/2007 18:10:09,593,8,"1140,C:\WINDOWS\system32\reg.exe,"
4/22/2007 18:10:09,592,8,"3996,C:\WINDOWS\system32\reg.exe,4028,"
4/22/2007 18:10:09,593,8,"3996,C:\WINDOWS\system32\reg.exe,"
4/22/2007 18:10:09,592,8,"1012,C:\WINDOWS\system32\pwdump.exe,4028,"
4/22/2007 18:10:09,593,8,"1012,C:\WINDOWS\system32\pwdump.exe,"
4/22/2007 18:10:10,593,8,"1732,C:\WINDOWS\system32\dns.exe,"
4/22/2007 18:10:25,593,8,"4028,C:\WINDOWS\system32\cmd.exe,"
So, just what is happening? We see that PID 1732 started it all. If you jump down to the second to last line, you'll see what PID 1732 was - that's right dns.exe, which is where system level access was granted and the reverse shell was executed from. Every other process listed was called from PID 4028 which is cmd.exe. The reg.exe calls were successful in changing the ErrorReporting values I listed in Part II, so ErrorReporting was effectively disabled.
However, we know that something was missed. Taskkill.exe was listed in the script in many places, and that's what looked like the cause of dns exiting is, however as evidenced by the process tracking log, we can see that taskkill was never called. Is this because it was never called, or did windows miss the boat and not log it? The answer is evident, but I'll make this somewhat interactive. What killed dns.exe? *answer at the end of this entry*
Application Log:
Nothing of value found here.
I also grabbed the IE history using Prodiscover and found the following.
Internet Activity Information file: E:\132.236.62.102-04222007\PhysicalDrive0.eve\C:\Documents and Settings\Default User\Local Settings\Temporary Internet Files\Content.IE5\index.dat
File Deleted: No
Browser Type: Internet Explorer
URL: http://www.dit.net/images/pwdump.exe
File Name: pwdump[1].exe
Last Modified: 04/22/2007 14:06:33
Last Accessed: 04/22/2007 18:10:09
Local Directory:
Type: URL
HTTP Header: HTTP/1.1 200 OK
X-Powered-By: ASP.NET
Content-Type: application/octet-stream
ETag: "809a98f5885c71:ade"
Content-Length: 90624
~U:system
As we can see by the bolded text, the user SYSTEM downloaded pwdump, as Harlan pointed out.
MAC times:
File Name,File Type,Cr Date,Acc Date,Mod Date
rasadhlp.dll,Executable File,1/4/2006 16:30,4/22/2007 18:09,3/24/2005 19:12
vgaoem.fon,Executable File,3/25/2003 8:00,4/22/2007 18:09,3/25/2003 8:00
cmd.exe,Executable File,3/25/2003 8:00,4/22/2007 18:09,3/24/2005 18:57
dosapp.fon,Executable File,3/25/2003 8:00,4/22/2007 18:09,3/25/2003 8:00
ega40woa.fon,Executable File,3/25/2003 8:00,4/22/2007 18:09,3/25/2003 8:00
cga80woa.fon,Executable File,3/25/2003 8:00,4/22/2007 18:09,3/25/2003 8:00
cga40woa.fon,Executable File,3/25/2003 8:00,4/22/2007 18:09,3/25/2003 8:00
dbg0.log,Unknown File Type,4/22/2007 18:09,4/22/2007 18:09,4/22/2007 18:09
net1.exe,Executable File,1/4/2006 16:31,4/22/2007 18:09,3/24/2005 19:07
net.exe,Executable File,1/4/2006 16:31,4/22/2007 18:09,3/24/2005 19:07
R00000000000a.clb,Unknown File Type,1/4/2006 15:00,4/22/2007 18:10,1/4/2006 15:00
imm32.dll,Executable File,1/4/2006 16:32,4/22/2007 18:10,3/24/2005 19:05
wshext.dll,Executable File,1/4/2006 16:30,4/22/2007 18:10,3/24/2005 19:26
sensapi.dll,Executable File,3/25/2003 8:00,4/22/2007 18:10,3/25/2003 8:00
msisip.dll,Executable File,1/4/2006 16:31,4/22/2007 18:10,3/24/2005 19:07
vbscript.dll,Executable File,1/4/2006 16:30,4/22/2007 18:10,3/24/2005 19:26
rtutils.dll,Executable File,1/4/2006 16:30,4/22/2007 18:10,3/24/2005 19:13
msdart.dll,Executable File,1/4/2006 16:31,4/22/2007 18:10,3/24/2005 19:07
scrobj.dll,Executable File,1/4/2006 16:30,4/22/2007 18:10,3/24/2005 19:13
iphlpapi.dll,Executable File,1/4/2006 16:32,4/22/2007 18:10,3/24/2005 19:05
winmm.dll,Executable File,1/4/2006 16:30,4/22/2007 18:10,3/24/2005 19:26
msado15.dll,Executable File,1/4/2006 16:33,4/22/2007 18:10,3/24/2005 19:07
tapi32.dll,Executable File,1/4/2006 16:30,4/22/2007 18:10,3/24/2005 19:25
msxml3r.dll,Executable File,1/4/2006 16:31,4/22/2007 18:10,3/24/2005 19:07
mscorie.dll,Executable File,1/2/2006 13:15,4/22/2007 18:10,3/24/2003 22:31
9PMKWROZ,Folder,1/2/2006 13:20,4/22/2007 18:10,4/22/2007 18:10
$I30,Unknown File Type,1/2/2006 13:15,4/22/2007 18:10,1/4/2006 16:41
msvcr71.dll,Executable File,1/2/2006 13:15,4/22/2007 18:10,2/24/2003 16:42
v1.1.4322,Folder,1/2/2006 13:15,4/22/2007 18:10,1/4/2006 16:41
mscoree.dll,Executable File,3/24/2005 21:41,4/22/2007 18:10,3/24/2005 21:41
reg.exe,Executable File,1/4/2006 16:30,4/22/2007 18:10,3/24/2005 19:13
cscript.exe,Executable File,1/4/2006 16:32,4/22/2007 18:10,3/24/2005 18:58
msxml3.dll,Executable File,1/4/2006 16:31,4/22/2007 18:10,3/24/2005 19:07
shimeng.dll,Executable File,1/4/2006 16:30,4/22/2007 18:10,3/24/2005 19:13
pwdump[1].exe,Executable File,4/22/2007 18:10,4/22/2007 18:10,4/22/2007 18:10
wbemcons.dll,Executable File,1/4/2006 16:33,4/22/2007 18:10,3/24/2005 19:26
default.LOG,Unknown File Type,1/2/2006 8:06,4/22/2007 18:10,4/22/2007 18:10
system.LOG,Unknown File Type,1/2/2006 8:06,4/22/2007 18:10,4/22/2007 18:10
httperr1.log,Unknown File Type,6/28/2006 0:20,4/22/2007 18:10,4/22/2007 18:10
Anything listed here was touched during the attack. You'll see folders, dll's, exe's, fonts and a few odd files. While these files don't tell us who accessed them, it gives us a blueprint for testing. If we created a virtual machine and ran the attackers scripts, most if not all of these should show up as being accessed.
Of some note is this file: R00000000000a.clb It lives in C:\Windows\Registration and it appears to be related to COM+ applications or having debug rights on a system, which isn't typical for most users. If anyone has more information about this, please share it.
Other locations of artifacts:
$LogFile - If one were so inclined you could reconstruct a lot of the event by using this file alone.
File slack and unallocated space
httperr1.log contained related information as the attacker first scanned my honeypot looking for open ports.
Where there were no artifacts:
Prefetch
MRU
MUICache
UserAssist
I hope this series of entries has shown why we need to use multiple sources of evidence to corroborate and build a strong case and how to do it as well. Digital evidence is largely circumstantial, and we can rarely prove "who" did something, but when evidence from multiple sources is collected and compared, we can reconstruct the events that took place.
*memory analysis coming soon - I hope*
*dns.exe answer*
To answer this question we need to look at the binary pwdump.exe. The strings listed previously indicate some of the capabilities of this particular pwdump binary and provides us enough clues so we can create a hypothesis and test it. I hypothesized that the script didn't actually kill dns, and the script doesn't directly patch the dns hole.
To test this I created an exemplar system running Windows Server 2003 SP1 and DNS to test my hypothesis. The test system was created using VMware Server 1.02 build 39867 running on an Ubuntu Server system.
The test:
copy pwdump[1].exe to the test system, and execute it using: pwdump[1].exe %RANDOM%.
Result:
Upon Execution, we can see the following at the command prompt:
The system was then restored to a snapshot and pwdump was run again, this time by Inctrl5.
InCtrl5 excerpt:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\DNS\Parameters "RpcProtocol"
Old type: REG_DWORD
New type: REG_DWORD
Old data: 01, 00, 00, 00
New data: 04, 00, 00, 00
System Event log:
Event Type: Error Date: 05/04/2007 Time: 12:38:13 PM Source: Service Control Manager
Event ID: 7034 EventDescription:The DNS Server service terminated unexpectedly. It has done this 1 time.
Conclusion:
pwdump kills dns.exe and "patches" the registry by using the Microsoft workaround.
This pwdump binary also unpacks the samdump.dll that is contained within it.
If anyone is interested in this file for review, let me know.
Tuesday, May 1, 2007
All the King's Horses
Have you ever been asked to tamper with evidence? Asked to destroy it? Compromise the chain of custody? Lie about your findings? Ignore exculpatory evidence?
I have several books(by several I mean at least a dozen) on the subject of digital forensics - both popular literature and those used in education and upon reflection I realized something. Not a single one of them mentions the moral and ethical challenges of working in digital forensics and evidence handling with more than a passing comment. These challenges exist in every science so this is not exactly a new idea, however what's different is the fact that many corporate IT staffs are being asked to conduct digital forensics work and corporate forensics officers are asked to conduct examinations objectively. The last time I checked, administrative assistants weren't being asked to determine the chemical composition of the yellow #2 pencils at their desk so, why is IT being tasked with science(if digital forensic science is indeed a science)? Worse yet, why is there no protection for the forensics teams? How can we be expected to remain objective and provide complete reports with the threat of unemployment hanging over our heads?
I entitled this entry All the King's Horses because I recently read Vonnegut's short story with the same name. If you haven't ever read it, the story is about a group of American soldiers being held captive by a Communist Guerrilla chief Pi Ying. As the story progresses, the reader finds out that the Colonel's family(wife and 2 sons) are also captives. In order for the Americans to gain their freedom, Pi Ying plays Colonel Bryan Kelly in a game of macabre chess. The Americans are the chess pieces on the white team and Pi Ying has wooden pieces. If an American piece is overtaken then the person representing the piece is executed, and if Pi Ying loses a piece, all he loses is a piece of wood. The game reaches a point where Kelly could win the game, if only he could remove the Black Knight from the middle of the board. The only way he can do this is by sacrificing one of his sons(whom he had chosen to be his knights). This sacrifice would allow him to save the lives of everyone else on the game board. Kelly eventually makes the choice to sacrifice his son.
Imagine this scenario as it relates to digital forensic science. You are Colonel Kelly and your son is your set of ethics and morals. Pi Ying is the corporate officer/lawyer of choice. The pieces you must contend with are individual ethical challenges you face at each turn. Eventually, you get backed in to a corner where you must choose to sacrifice your son or the game is over.
Now for a real example or two.
After leading a long and arduous incident involving a team of roughly 10 people I was called in to see the director of HR. I was asked a series of questions regarding the incident, and as the conversation progressed I realized what was going on. I was being prodded to provide information as ammunition in firing the two network administrators that were "responsible" for the incident occuring. I remained responsive, but tried my hardest not to contribute to what was happening. In the end it didn't matter, the two network administrators lost their jobs because of what I consider an organizational problem. I on the other hand, while a little bitter was able to walk away from the situation with my ethics intact.
During another incident involving a data breach I was sitting in the IRT meeting with some administrative types. I was asked if I could just ignore the fact that I was in posession of disk images from breached systems and make them disappear. My answer? "No."
Now, we all know that business people skipped the day they taught ethics but science is founded on adherence to a strict ethical code in order to keep the science pure. I tend to be a pretty hard nosed individual when it comes to violating ethics regarding science, but I know that others simply can't afford to be as hard nosed, so I wonder, has anyone had a situation like the above where your ethics were challenged?
Our positions as digital forensic examiners puts us in a unique position of close proximity to the law. As such, we are at a high level of risk. So, as a member of a few forensics related organizations, I wonder what people think of the idea of having these organizations protect their members when the ethical code of the organization (which we must adhere to) is challenged in the course of employment - whether as a retained expert or as an in-house examiner. What greater benefit could an organization possibly provide than to protect its members?
Thoughts?
A few links on the subject:
Philosophy of science
Adventures in Science&Ethics
I have several books(by several I mean at least a dozen) on the subject of digital forensics - both popular literature and those used in education and upon reflection I realized something. Not a single one of them mentions the moral and ethical challenges of working in digital forensics and evidence handling with more than a passing comment. These challenges exist in every science so this is not exactly a new idea, however what's different is the fact that many corporate IT staffs are being asked to conduct digital forensics work and corporate forensics officers are asked to conduct examinations objectively. The last time I checked, administrative assistants weren't being asked to determine the chemical composition of the yellow #2 pencils at their desk so, why is IT being tasked with science(if digital forensic science is indeed a science)? Worse yet, why is there no protection for the forensics teams? How can we be expected to remain objective and provide complete reports with the threat of unemployment hanging over our heads?
I entitled this entry All the King's Horses because I recently read Vonnegut's short story with the same name. If you haven't ever read it, the story is about a group of American soldiers being held captive by a Communist Guerrilla chief Pi Ying. As the story progresses, the reader finds out that the Colonel's family(wife and 2 sons) are also captives. In order for the Americans to gain their freedom, Pi Ying plays Colonel Bryan Kelly in a game of macabre chess. The Americans are the chess pieces on the white team and Pi Ying has wooden pieces. If an American piece is overtaken then the person representing the piece is executed, and if Pi Ying loses a piece, all he loses is a piece of wood. The game reaches a point where Kelly could win the game, if only he could remove the Black Knight from the middle of the board. The only way he can do this is by sacrificing one of his sons(whom he had chosen to be his knights). This sacrifice would allow him to save the lives of everyone else on the game board. Kelly eventually makes the choice to sacrifice his son.
Imagine this scenario as it relates to digital forensic science. You are Colonel Kelly and your son is your set of ethics and morals. Pi Ying is the corporate officer/lawyer of choice. The pieces you must contend with are individual ethical challenges you face at each turn. Eventually, you get backed in to a corner where you must choose to sacrifice your son or the game is over.
Now for a real example or two.
After leading a long and arduous incident involving a team of roughly 10 people I was called in to see the director of HR. I was asked a series of questions regarding the incident, and as the conversation progressed I realized what was going on. I was being prodded to provide information as ammunition in firing the two network administrators that were "responsible" for the incident occuring. I remained responsive, but tried my hardest not to contribute to what was happening. In the end it didn't matter, the two network administrators lost their jobs because of what I consider an organizational problem. I on the other hand, while a little bitter was able to walk away from the situation with my ethics intact.
During another incident involving a data breach I was sitting in the IRT meeting with some administrative types. I was asked if I could just ignore the fact that I was in posession of disk images from breached systems and make them disappear. My answer? "No."
Now, we all know that business people skipped the day they taught ethics but science is founded on adherence to a strict ethical code in order to keep the science pure. I tend to be a pretty hard nosed individual when it comes to violating ethics regarding science, but I know that others simply can't afford to be as hard nosed, so I wonder, has anyone had a situation like the above where your ethics were challenged?
Our positions as digital forensic examiners puts us in a unique position of close proximity to the law. As such, we are at a high level of risk. So, as a member of a few forensics related organizations, I wonder what people think of the idea of having these organizations protect their members when the ethical code of the organization (which we must adhere to) is challenged in the course of employment - whether as a retained expert or as an in-house examiner. What greater benefit could an organization possibly provide than to protect its members?
Thoughts?
A few links on the subject:
Philosophy of science
Adventures in Science&Ethics
Subscribe to:
Posts (Atom)
