Triage of Agent.BTZ

I'm a huge proponent of triage incident response. So much so that I developed procedures based on the idea that gathering a little information from key data points early can lead to an accurate assessment of the situation without having to conduct laborious processes such as creating a full disk image all the time. Triage saves time and effort. The purpose of triage is not to conduct a full analysis. The purpose is to 1) sort and prioritize and 2) gather enough information to decide whether or not to continue an investigation. It also maximizes the effectiveness of analyst, systems, and tools. Tools like F-response make triage possible. Tools like Responder Pro from HBgary make triage possible. Speaking of Responder pro..

I recently upgraded my copy of Responder 1.5 to Responder 2. I've got some great things to say about this product but I'll save that for another post. I ran an analysis some time ago in 1.5 against a dangerous little piece of malware that got quite a bit of press in 2008. The malware in question is Trojan.Agent.BTZ. This little gem is what ransacked the military and pentagon. Vendors like to call it Autorun malware, as that's really how it works but it's of course more than what a vendor will tell you.

Generally speaking I was looking for a piece of malware that infects removable media, phones home, gives remote control and downloads other malware. I arrived on scene a short while after the alert and after talking to the admin, decided to plug in my USB key, knowing full well what would probably happen to it. USB key defenses aside, I ran FastDump Pro and grabbed an .hpak memory dump.

So now I had a memory dump and I grabbed a disk image from the computer. In this case I decided to take the drive because I would use this later in a lab scenario for training. Triage requirements aside, this was a good case to capture for later use.

Let's analyze it quickly with 2.0 eh?

*note I've already done this in 1.5 I'm just re-doing it in 2.0 and during the middle of this I experienced a licensing hiccup*

My standard technique for beginning memory analysis in Responder is as follows:
1) Evaluate DDNA listing.

DDNA while not perfect, can be used to quickly hone in on oddities and badness. It helps identify WHAT is on the box, WHAT it might be, and HOW it might be working. Add a cross reference listing to the modules running on the box for more detailed information. DDNA is a boon to any analyst looking to conduct rapid analysis.

Here's what the DDNA output looked like for BTZ. Focusing on the left hand side, we don't see a whole lot that sticks out. A sea of Orange really...


2) Evaluate Network connections. This helps me answer two questions. WHAT is talking, and to WHOM? I tend to go for Network connections before Processes as network connections often identify the process I need to investigate. For all intensive purposes, 2&3 are interchangeable.

3) Evaluate Process lists.
Typically I evaluate processes in a number of ways. I'll look for processes that don't belong, those with odd names, those that are 'hidden', all svchost processes since they are a huge target for process and dll injection attacks, those processes that are "red flags" such as ones executing from the wrong directory or with incorrect paths and processes that don't normally exist.

4) Open file handles and Registry keys.
This should be fairly obvious as to the why. It allows me to find out what process has what file handles and reg keys open.

5) Use a DNS blacklist or keyword list. There are great blacklists out there plus I have a few extras. This immediately helps with data reduction in some cases. It can also assist in zeroing in on the malware. This is great for casting a drag net on a network to look for other infections.

6) Other poking - looking for clues that might tip me off to the true nature of the infection or compromise.

This usually does the trick for the overwhelming majority of malware cases I look at. Granted there are more difficult ones but with malware being as templated as it is..this tends to work.
So now let's work through this for real...

DDNA in this case didn't appear to be helpful, or was it? Looking at the listing, there's a wide variety of suspicious looking processes and modules. That's not really that helpful by all appearances. Let's add a little intelligence to this analysis by pulling up the module listing next to the DDNA listing for processes. This is what that looks like:


So, now we have even more orange...GREAT you might be thinking sarcastically..but what do we see if we look closely? Like a simple equation we can rule out common processes and modules that we have a possible explanation for now and I've highlighted something that looks REALLY suspicious..a process loading a module out of the user's profile. As I said DDNA is not perfect, but what it does is raise the interesting stuff to the top by severity and color coding. This is automated analysis and while it has limits, when we add human intelligence to the analysis process we get an immediate bead on the target.


So what are the key indicators?

* The .dll
* rundll32.exe is calling a .dll out of the user profile
* The file path for the .dll.

Yeah that's pretty odd isn't it? What traits does it have?


Nothing sticks out a whole lot, but there are some good clues in there.

So now I've found something odd and definitely worth looking in to a bit further. This happens to be the jackpot but let's keep evaluating.

How about the DNS blacklist for connections to known bad domains?

The list of hits was far too many to show here. The matches numbered upwards of 336 bad domains. That's too many domains to be helpful but it's definitely a sign that the computer was talking to a lot of known bad actors.

And Network Connections?

The network cable in this incident was unplugged when I arrived. No joy for active connections.

And the process listing?

There's one above the rest that sticks out:
C:\WINDOWS\system32\rundll32.exe" "C:\Documents and Settings\ctuser\Application Data\HELP\system32\rtn.dll"


That just about settles it for me. rtn.dll does not belong. Let's go ahead and process it. I always start by right clicking and taking a look at the strings. This allows me to drill right in to what I am interested in.

Immediate strings of interest:
C:\DOCUME~1\ctuser\LOCALS~1\Temp
C:\DOCUME~1\ctuser\LOCALS~1\Temp
C:\DOCUME~1\ctuser\LOCALS~1\Temp\
C:\DOCUME~1\ctuser\LOCALS~1\Temp\~DFD.tmp
C:\Documents and Settings\ctuser\Application Data\Help
C:\Documents and Settings\ctuser\Application Data\Help
C:\Documents and Settings\ctuser\Application Data\HELP\
C:\Documents and Settings\ctuser\Application Data\HELP\\system32\
C:\Documents and Settings\ctuser\Application Data\HELP\\system32\mswmpdat.tlb
C:\Documents and Settings\ctuser\Application Data\HELP\\system32\wmcache.nld
C:\Documents and Settings\ctuser\Application Data\HELP\system32
C:\Documents and Settings\ctuser\Application Data\HELP\system32\rtn.dll

Yep, this tells us a little about the files the malware is using. Point of interest here is that the malware created a directory structure for the user the malware was run under, and one of the few directories it could write to was in the profile as the user had no elevated privileges. Many people still think that Administrator rights is a means of stopping the execution of modern malware. Those people couldn't be more wrong but I digress...

Right now we don't know what the files are are specifically but we will soon...


How about this?
1dM3uu4j7Fw4sjnbcwlDqet4F7JyuUi4m5Imnxl1pzxI6as80cbLnmz54cs5Ldn4ri3do5L6gs923HL34x2f5cvd0fk6c1a0s


My that's odd isn't it? And it's referenced in three different memory locations. So just what is that string? The following code should give you a big hint.


If you can't tell that's an XOR key and function. We can use that bit of information later when we want to do deeper analysis. As we learn from later analysis, this XOR key is used to encode data written to log files by the malware.

When I did this analysis for real I completed the analysis and decoded the log files kept by the malware, and conducted a more thorough disk based analysis. The purpose of this posting is to illustrate a quick analysis method that pays off with the extraction of the XOR key to encode log files created by malware. In all, doing it this way takes about 15 minutes to get actionable intelligence. Think about it. In a just a few minutes we've gathered:

* Filenames on a filesystem. Pass that information off to your windows admins, and they can search desktops for the files.
* The XOR key to decrypt any discovered log files
* There's more to be seen in the memory dump that allows us to create a snort signature if need be(of course one already exists at present time).
* The malware does not require administrative privileges to execute or maintain persistence.

I realize this post is a bit incomplete..hopefully I'll get back to a continuation piece.

We just don't get it

Given all the talk about APT lately I'm still shocked. Shocked that there are those out there on the 'good guy' side that can do nothing but criticize. One recent discussion that's been heavily debated is one of how "new" Advanced Persistent Threats are. My question to everyone out there:

"Does it really matter ?"

Every day these enemy combatants are lifting data. Lifting data from organizations they're not supposed to be lifting data from. These data are then being used against us to gain political, economic and military advantages. I've watched the data pass through systems for months and it turns my stomach to think that it's being done with such ease. Especially considering where the data is from. That these attacks occur is nothing new. That these attacks are taking place on such a broad scope is entirely new. That the enemy elements are moving against so many targets at the same time and in such different industries is alarming.

For years I've investigated cybercrime and done malware analysis and intrusion investigations. I can say with relative ease that while the tactics used in these attacks are not necessarily new, there is a certain 'newness' to this type of enemy. The majority of cybercrime that occurs today is automated. Malware has reached a point of templatization such that these toolkits are sold so others can perpetrate more crimes. While certain high profile attacks are definitely not automated and require a crew of clever individuals, many cybercrime incidents are automated.

These attacks are not very automated. Like a skilled tradesman, they reduce overhead by automating simple things. When the enemy gains access to your networks, reads your email, browses the internet on your computer, pretends to be you to garner more information from your colleagues, ignores your bank statements but takes schematics, ignores your customer credit card database, but steals your organizations futures documents and pilfers from your R&D group there's a difference. When the same group penetrates military systems and networks there's a difference. The difference is due to the global scale, the difference is in our ability to remain a competitive nation. The difference is in our military's ability to remain effective. The difference is that this is not just about money.

Regarding their malware:
Is it any wonder that the malware used by this enemy shares a common trait with other malware? There are a finite number of methods to accomplish a goal in a given programming language. Is there a reason not to re-use code if it works? Is it any wonder we can look at multiple samples of malware and draw comparisons? Give a fool a katana and he'll cut off his nose. Give a Samurai a katana and he'll cut you in half before you can blink your eyes. Malware is a tool of the enemy, not the enemy himself. The right malware in the hands of a skilled opponent is a force multiplier for a real threat, while malware in hands of a lesser opponent is a nuisance. This enemy is more than their malware.

There is no data breach notification when this enemy penetrates a network and steals data. The notification comes when we have another financial crisis and a foreign government is bailing us out. The notification comes when we have another gas shortage like in the '70's. The notification comes when power grids fail. The notification comes when more of our commerce is outsourced and jobs are lost. The notification comes when our companies are being bought by foreign companies because they can no longer compete. The notification comes when our military can not protect our interests. This problem is bigger than the security industry. This problem is bigger than IT. The security and IT industries are impotent in this situation. This problem will take governments to solve.

The people that call it hype have not seen this enemy work. They have not seen the contents of the stolen files. The business that have recently started doing "Anti-APT audits" are missing the point and trying to capitalize on the situation to further their own business.

What should matter is how successful they have been. What should matter is defending ourselves. What should matter is how and where we share this information. What should matter is taking this information to those with the ability to do something about it. What should matter is taking the fight to the enemy.

So I ask again, does it matter if this threat is new?

The APT is on your webserver

One of the key ways APT gets in to your network is through human exploitation. Duh. We are the weakest link and in my experience it's usually those with some form of fiscal responsibility(re: business offices) that are the weakest. The APT also uses remote exploitation as a weapon. If there's a vulnerable system out there, they find it, exploit it and set up shop. This is done quickly and is done often times before public exploits are available and before the related vulnerability is being widely scanned for.

However, they, at least in my experience, are limited. They seem to limit themselves to Windows systems. I've not yet seen (not that it hasn't happened, but I've not seen it) a Unix system compromised by the APT. If you have, chime in at any time. So far, they've all been Windows systems. This is understandable and predictable. One place I've seen the APT establish a presence is on a web server. Yes, the APT is on your web server. In my experience this has been for C2.

Common traits of an APT web server compromise that I've seen:

System traits:
Windows Server 2003
IIS 6

Management traits:
Often poorly managed - the system may be a development system, or one that is in the process of being decommissioned.
Administrator is the most commonly used account for management.
Security logs and auditing is weak and not offloaded or rolled over periodically.
RDP is available

Compromise traits:
They modify forward DNS lookups for their domains to point to your system.
They don't really attempt to hide their presence.
They create files and host them on your webserver.
Excessive use of the Administrator account, often during non-business hours.
Server may begin proxying traffic to/from China.
A pattern change of many to one relationships, meaning your server will begin seeing requests from many hosts that it normally never receives traffic from and requests are for files and pages that didn't exist prior to the incident. This is often a behavioral pattern anomaly.

Anomalies:
Logs on the server will likely indicate the presence of new files in the form of excessive requests to which your server will likely respond with a 404. That is of course, until your server goes active and DNS propagation occurs.

Your webserver may begin to initiate outbound connections to remote systems that it is not cleared to communicate with and may begin acting as a proxy.

The administrator account is being used to browse the web from the web server. This should be a no-no in any environment and is therefore an anomalous event.

Your webserver may resolve to a domain that is not yours.

As mentioned above, you'll note a behavioral change in who is talking to your server and for what.

Detection:

*note these are not "special techniques". This is standard tradecraft.*

Cull your logs for:

Many hits from different IP's to the same page returning a 404. This is not uncommon on today's webservers, but if you exclude commonly searched for vulnerabilities you can easily do data reduction. This can easily be done with Logparser. A good but old article is here.

Administrator logins to your webserver from ip addresses that have no business with your server with administrative rights.

Administrative RDP sessions from external sources. Again a no-no..but if you've got it open, they'll use it.

Inventory your webservers and do DNS lookups (forward and reverse) on them using external DNS servers. If they're resolving odd domains then you've got something to look for.

M-trends reaction

**FTC disclaimer (re: middle finger) I'm not affiliated with Mandiant. I know folks at Mandiant only by name recognition and perhaps a few blog comment exchanges, or mailing list/forums posts. I, like you, have read the M-trends report. I do not have access to anything other than M-trends, a few M-unition blog posts from Mandiant and random interweb babble on the subject. I would love to have a discussion with the folks over at Mandiant but I do not see that happening any time soon.
FTC disclaimer**


Now that the obligatory disclaimer is out of the way..When reports like this come out it's interesting what happens. The reactions range all over the map. We, the good guys, are too busy sizing each other up, calling each other ignorant, pretending to know what we don't and holding on too tight to really discuss the issues. What I find most interesting is how apparently everyone is an APT expert all of a sudden, with 15 years of experience battling them, and yet for all of this experience and worldly knowledge, none of it has been shared beyond the contents of this report. Sure, it's discussed privately, in secrecy and behind closed doors but there is an entire industry that plays a part in this, and I'd estimate that perhaps 10% of it knows what's going on.

I looked at the M-trends report and thought wow this is a good explanation of what happens and how. This is good information for folks up the ladder to have. This report is what security folks have been talking about for years, what we're all actually so paranoid about. Mandiant does a great job of presenting the scope of the issue and provide a good explanation. However, there is little to no information at the tactical level and no information related to actually countering the APT in an organization. I understand this..it's a report and they don't want the Chinese (oh don't act so surprised) to know just how 'on to them' the good guys really are. Mandiant also wants to continue to make money doing consulting work and selling premium services such as "counter-APT" investigations and what not. I understand this and do not begrudge them. They apparently do a great job and I'm sure their services are well worth it.

When vague reports like this get released, very few people attempt to validate the findings. Even fewer have the data to do so. As it so happens I've got a bit of data that's APT related. Well, maybe more than a bit and in short order will be sharing some of my own findings. Counter-APT operations are not simply after the fact. The reason they seem to be solely after the fact is due to the cost of defending an enterprise, the lack of awareness and poor governance in organizations. I do not want to make an APT "splash". I do want to unveil a bit of the mystery behind the Advanced and Persistent part of the APT. As I've said before, they are human, they are fallible, they are an anomaly, they are more than their malware, and they can be detected.

If an identity gets stolen on the internet

Does anyone notice?
How about when 1000 identities get stolen? What about 2000? What about 50,000?

While doing a "routine investigation" of a Qakbot infection I discovered a dropzone for the malware. I say 'a' and not 'the' because the DZ was a configuration option and it could be updated at any time. The DZ I found was full of thousands of keylog files and other data uploaded from infected systems. This information was promptly sent to some contacts in the FBI.

What I begin to wonder is...did any one of these people actually know their identity was at risk or in fact actually stolen? How many of the companies whose users were infected actually knew about the real risk this malware infection posed to their organization? How many FTP servers were abused as a result of this infection, how many webservers were compromised? How many sensitive intranet systems were exposed?

Qakbot, like many current threats, was short lived, with longer lasting effects. It was a sortie if you will, a quick blitz to get out, infect thousands of systems, capture as much information as possible and send it back to the people behind it to sell, or use in other attacks.

This type of blitz happens daily. For every Conficker worm there are thousands of malware samples that do just as much damage as the media friendly worms using Guerilla style tactics, and it is these smaller samples that eat away at individuals and organizations. Certainly there are large breaches that cause massive damages in one fell swoop, but it is more common to see smaller infections get ignored, and they therefore create more of a problem in the long term. It is unfortunate when organizations don't take these small infections seriously. How many malware infections made the media in the past year that weren't conficker or other major media frenzy type worm and caused serious damage?
Like this one(or 8000), or this one, or this one...hopefully you get the point.

Many organizations don't discover these simple compromises for weeks or months and when they do, it's likely because their antivirus product updated definitions (which are largely ignored) or a third party identified the compromise. Do a quick evaluation of your customers and your own internal organization and look at the malware infections that have taken place over the past 6 months. How many affected individuals reset their passwords? Did any of the systems get "cleaned" and not get rebuilt? How many of those individuals had access to company web or ftp servers? How many of those people are in your Business Service Centers or administrative offices? How many of those people work with sensitive data on a daily basis? How many take their laptops home and let little johnny play on it?

OR

Suppose the following:
John Q Public works for your organization..let's call it Booze Brothers inc. John works in the HR department. John is on vacation and logs in to a public kiosk in the business office at the hotel that happens to be infected with Qakbot to do the following five things:

• Visit amazon to see when that special gift for his daughter will arrive at home
• Check his bank account because he was waiting on a reimbursement
• Check personal email account to see how things are going at home
• Log in to twitter to tell everyone how he's doing
• Log in to facebook to update his page
  

Based on some statistics it is likely that John Q Public just exposed your organization because he uses the same password for one of the five sites above as he does at your organization. The lines between personal identity and work identity are blurred because of this password synchronization that is a common practice. His information gets sent to the DZ and later it is culled and re-used, sold or traded. It may be 24 hours, it may be a week but you'll likely see John Q's account used in an attack against your organization - maybe phishing, maybe used to upload javascript to a webserver, or maybe just a brute force attack.

Identities fall each day to the malware infections that plague us, though recall I don't believe in simple malware infections due to the gateway malware theory. As always, if you haven't done so in the past year, it's probably time to revisit your IR plan to address this sort of stuff.

The need for speed

When it comes to incident response we are still struggling to close the gap. It's been mentioned here before, and other places. When a compromise occurs, how quickly do you get to analyze the compromise? Hours, days? Some time even weeks!

And when you do get your hands on a system what are you left with? A lot of trampled data points and an incomplete data set. And..what kind of time frame are we talking about to conduct the analysis, just to make a cut as to whether or not it passes the "who cares" test? Consider the time it takes to acquire and process an image in FTK or Encase. This is commonly referred to as machine time, and it's not uncommon to have too many systems locked up in machine time while the analyst waits for the case to process. And then of course you have the report writing part of the process. All told this takes an Average of 20-40 hours per case, often times more. This equates to two bottlenecks in the process.

The two primary bottlenecks are

1. The time it takes to get to the point where analysis data points are collected, processed and analyzed. There is a lot of time wasted when the analyst is waiting for "machine time" to complete.
   
2. The time it takes to generate a report.
   

This is the traditional model and its inefficiencies. If this doesn't make sense, perhaps the picture below will help.


So now let's look at a model that involves a triage phase.

By including a triage process that collects primary analysis datapoints while the system is live we increase our efficiencies by multi-tasking. Collection of primary data points can be largely automated. The analyst can then focus on analysis of the collected datapoints while the traditional acquisition and case processing takes place. Essentially what this does is take advantage of all available resources - human time and machine time by changing the model.

The benefits of this model are

1. Not always having to acquire a disk image. If the "who cares" test isn't passed with the datapoints that are collected early, then you can easily move on.
2. This is a one-to-many relationship. An analyst can quickly collect datapoints from several systems and conduct triage analysis instead of waiting for a linear acquisition process to progress.
3. It uses all available resources at the same time instead of waiting on one component to complete.

Here is another illustration.

Again...it's not like this is built around F-response or anything. I'm not saying..I'm just saying.

What are your thoughts? I *am* looking for feedback.

Active Directory Snapshots

With Vista, Microsoft finally made proper use of Volume Shadow Copy and the Volume Shadow Copy Service. A lot of great work was done to help others use this during analysis. Server 2008 continues this model but it applies it to Active Directory. Sounds cool eh?

First off, let me say that this is well known to sysadmins, but I'm fairly certain it's not well known in this part of the industry. I've not seen it discussed on any list or forum I pay attention to at least.

For background - Read these pages here and here...I'll wait.

And now, go read this page here....I'll wait for you again.

So, now that you've read about creating Active Directory Snapshots and how to mount a VHD file in windows, let's discuss it.

When performing incident response in an Active Directory Environment, you're most likely going to want to look at a domain controller, especially if the domain controller is compromised, or there is something funky happening in the directory itself. Any self respecting sysadmin is going to have a regular system state backup of the domain controller. This is done so restores can occur if objects are inadvertently deleted, and also as a good practice. In server 2008, this backup is stored as a .VHD file. In a response scenario involving AD, we want to maintain our methodology of not modifying the system any more than we have to, so, we don't want to work on a live copy of Active Directory, we want to work from a snapshot of it.
Here's a pseudo scenario.

A compromise is believed to have occured in Active Directory.
Logging was disabled by the attacker on the domain controller, or the attacker covered his tracks in the logs.
You have been tasked with figuring out what was changed.
You have a recent system state backup.
You mount the system state backup and recover the AD core files.
You create an Active Directory Snapshot.
You load up Sysinternals Active Directory Explorer
You load the snapshot and the AD core files and Diff them in AD explorer.
You now have a smaller dataset to work with and you have a point in time diff of "what changed".

I'll be putting this together in a more formal manner..but I wanted to throw this out there for anyone that deals in Active Directory Compromises, especially with server 2008 domain controllers.

Memory Acquisition for First Responders

Not long ago I sat down with a group of First Responders to discuss triage of security incidents. I discussed leaving the network connection up so I could remotely access the drive and physical memory. Their response is one that I expect many to have come across.

"If we leave the system up, even if we tell the user to not use the computer, the minute we walk away, the computer will be used."

That was kind of interesting to me considering what's at stake but I completely understood their point of view. Too many organizations can't trust their users. So then I thought hmmm....well memory acquisition has come so far so fast that I can simply teach tech staff at any level to collect physical memory. With targeted training and proper documentation it's a fairly straightforward process to follow on contained systems.

Here's a sample from a doc I drafted detailing use of mdd from mantech.

ManTech DD (MDD)

Limitations:
Less than 4GB memory
32 bit Windows Operating System

Installation
Download mdd from Mantech
You can download the standalone executable (recommended) or a .zip file.
Copy the file(s) to a directory on your USB key
Rename the mdd executable to mdd.exe

Usage
Log in to the compromised system
Insert USB drive
Create a directory for the incident on the USB key or SMB share
Open the trusted command prompt for the operating system
Change directories to where mdd is installed
Execute mdd

Command line
E:\IR\mdd>mdd.exe –o E:\00000\memorydump.img

Where 00000 is the case number you've been given.

Notes
Mdd creates an md5 hash of the output of the memory dump. It’s important to capture this information. You can take a screenshot of the window using Ctrl+Alt+Print Screen or copy/paste from within the command line to a text file. Both forms of output are acceptable. Save this file as memorydump.md5


Training first responders to do a memory acquisition is much easier these days.

START methodology

START is a methodology applied to Mass Casualty Incidents or triage centers and frequently it is applied to battlefield medicine. START stands for Simple Triage and Rapid Treatment. I will focus primarily on Mass Casualty Incidents and triage centers. This methodology has a direct tie to The Golden Hour.

It is my humble opinion that START can be applied easily to Computer Security Incidents; Those of both the mass casualty and triage center variety. In a Mass Casualty Incident you are typically confronted by several potential issues ranging from sensitivity of data to criticality of the resource and the threat posed by the compromise. These casualties come from all sides of the organization. The same holds true when you have an influx of dissimilar incidents and you need to prioritize them - think the ER at a major hospital on a warm Friday or Saturday night.

That said I humbly present my adapted START methodology.


Stage 1 Triage
Stage 1 triage is completed on a live system. This stage requires a network connection.

Conduct Rapid Triage

• Collect Volatile Data
• PII search non system created directories
• Limited malware scan
• FLS & MACTIME

Conduct Rapid Assessment

• Preliminary memory analysis
• Review PII tool logs
• Review Antimalware logs
• Review FLS & MACtime logs
• Establish Time of Compromise

Influential factors in Stage 1

MADtime

• MAD time is the Maximum Allowable Downtime an organization can withstand the loss of a resource. Or more the point…the time it takes for someone to get pissed off(MAD).

Initial Threat assessment

• Is it known
• Identify any knowns
  
• Is it attacking other systems
• Is it spreading

Initial Risk assessment

• Sensitive Data presence
• System Profile

Stage 2 Triage

Stage 2 triage is completed on a disk image either after Stage 1 has been completed, or in place of Stage 1 in the case of a physical drive being delivered or acquired from an unpowered system.

Conduct Rapid Triage

• PII search disk image

• Data point collection

1. Network Logs
2. Prefetch
3. Registry
4. Browser History
   
5. MACtime data
6. Malware scan
7. Event Logs
8. Application Logs

Log the case and turn it over for analysis. The combination of the above data points is more than enough to get an examiner started.


This sure looks like a case for F-response especially if you combine Stage 1 and Stage 2 triage...I'm not saying I built this around it or anything...I'm just saying.

Responder Pro use case

And then conficker woke up...

It was April 10th and there was a bad moon rising. Three systems (one laptop, two desktops) woke up, updated and began attacking the network infrastructure at a customer site. Within minutes, hundreds of accounts were locked out. Within hours, all were locked out. After the initial screams for help and DoS complaints I got onsite, got the interview out of the way I was off to do the field work.

After collecting memory dumps - using FastDump Pro - I went back to the office of the local system administrator and fired up my mac and my XP virtual machine with Responder Pro installed.

After running the memory dumps through Responder Pro I did a quick analysis and compare and contrast. The questions I had to answer immediately was "HOW the hell did this happen?" and "HOW do we prevent it from happening again?"

I won't bore you with gritty details.

To answer the first question of "HOW the hell did this happen?" let's take a look. Two desktop systems and one laptop. I had my suspicions but me being me..I had to know. Using DDNA I quickly identified the injected svchost.



Next I quickly viewed the strings of the binary.


Note the telltale signature of the HTTP string http://%s/search?q=%d. That's confirmation of Conficker.

And for the coup de grace I looked at the Internet History.


That was easy. HOW the hell did this happen? The laptop user who had been out of the office for a while returned, plugged in and nailed two unpatched systems, and instructed them to download a copy from the laptop on port 4555. All told it was 15 minutes to Root Cause Analysis. That's what I like to call Rapid Assessment.

HOW the hell do we keep this from happening again? Well that's an exercise I leave to you..needless to say NAC has gained traction at this customer site.

Disaster averted

It's a rare day when I have truly exciting things happen. Tonight of course was the exception. A few months ago I had a hot water heater installed by so called "professionals"..you know, the factory trained kind. I use a night rate unit that controls when the unit is active. This evening when the unit turned on all was well or so I thought. When I went to the basement to look at something, I noticed an acrid chemical smell of something metallic and plastic burning. Having had "some" experience in this arena, I could tell that it was an electrical fire. If you've never smelled an electrical fire, there's nothing else like it. The smell of the metal wire, and the plastic shielding produces a smell and taste that doesn't leave your mouth or nostrils any time soon. Anyways, I had to locate the smell. The problem with electrical fires when you're in a room full of electrical wiring, is trying to locate exactly where the smell is coming from. For this, unless you have a "hot spot" detector, you usually have to rely on the tried and true sniff test.

So, there I was sniffing around my basement like a bloodhound trying to locate the source. Finally I reached the hot water heater. When you find the source, boy...you find the source. Getting that close to the source of an electrical fire creates a bit of a gag factor but it's temporary. Needless to say I turned off the breaker and called the Fire department. The problem was contained, but I wanted to make sure there were no hot spots growing in the conduit.

Not wanting to lose the opportunity to learn, I tried to pay attention to every detail - you know, that whole "study the methods used by others" idea that I mention quite a bit. The Captain was the first on the scene. I showed him where the fire was and got out of his way. He surveyed the area, asked me a few questions and previewed the hot water heater - meaning he did a sniff test too. When the rig arrived, I went out to let them in, and I showed them where to go. They checked the area with the hot spot detector and validated my findings, then proceeded to tear apart the wiring to determine the scope of the damage. You've already seen the wiring from inside the water heater. Here's the wiring from inside the conduit.



Suffice it to say the wiring is just destroyed. The root cause was a short within the wire nut, caused by poor installation. That smoldered lump of plastic in the first picture is what used to be the wiring nut. Anyways all is well and the aftermath begins tomorrow.

Naturally, this post isn't about the fire in my hot water heater tonight. It's about incident response and a few of the things that contribute to, and separate a good outcome from a bad outcome.

1) Knowing the environment you're dealing with. In this case, this was my house. I knew what I done today that could have created the situation, I knew where each electrical item was in my basement, I knew my wiring panel, and had it labeled.

In the digital world, this is the same as knowing your organization. You need to know where your assets are, what the assets are, how they are connected and you should have an updated topological diagram.

2) Experience and awareness. I've dealt with electrical fires before and knew what the smell was. I knew that a fire was nothing I was qualified to deal with, so I called the professionals without poking around more than was necessary. I also knew that once I described the problem, answered questions and showed them the location of the fire, I should get out of their way and let them work.

In the digital world, if you're the first responder or discover the incident, if you can't solve the problem yourself and you have someone on the way, don't meddle with the system and when the IRT arrives, show them where to go, answer their questions and get out the way. Hovering when an IRT is working does not help the situation. If your assistance is required, you'll be asked to help.

3) Factory trained professionals don't always do the right thing and cut corners. As the firefighters worked they were talking to one another and discussing their findings and theorizing the root cause. The root cause was the people that installed my hot water heater.

In the digital world, consultants are well paid but don't always do the right thing. I've dealt with many cases where the root cause was the consultant's poor choices during installation. Dropping firewalls, poor password security etc. When entering an engagement with a consultant, be sure that you know what you're getting.

These are just a few of the things that you should be aware of in the world of incident response. The biggest lessons of the night for anyone out there that has an Incident Response Team at your disposal is:

• If you are unsure, call the trained people that do know, before you do anything.
• There's no shame in admitting you don't know everything and can't solve the problem.
• If you know something is out of the ordinary, call quickly.

A safe evening to all.

reasonable belief

Just about every state now has a law that addresses data breaches and notification thereof. One thing they pretty much fail to do though is provide criteria for establishing reasonable belief. Well, what is it you might be asking?

Troy Larson provided the following to me for a definition: "As a legal standard, reasonable belief is defined as what an average person in similar circumstances might believe."

Ok, so that's easy enough. What would a layperson believe if presented with the circumstances. As it pertains to data loss investigations, we are never able to present our findings to an "objective" jury. Instead, we present our findings to a subjective group of individuals that have a stake in the data loss process. Sometimes you will be lucky enough to find yourself presenting your findings to a group or person with high ethical and moral standards who wants to do the 'right thing'(TM). If you are lucky enough to find yourself in front of a decision making group, what do you present? Of course, you present your findings in a factual manner, without attempting to inject bias or opinion (unless asked to render one). The role of decision maker is not ours afterall. However, we must take great care to not poison or influence the decision making process. Our analysis must be thorough and complete. It should not be based on assumption or speculation of "what if" or "they could have". That is not our role. Our role is to present what we found, and if something we expected to find, is not found, then we may have reason to suspect something is wrong.


So we must ask ourselves this question - Given normal circumstances, what would a layperson base their decision on? How is reasonable belief actually established?

I'm attempting to answer this very question. To do so, I pored over numerous reports and analyses and their resulting decisions. I did some other research and came up with the following areas that I think influence how a person develops a reasonable belief when weighing the decision to notify as a result of a data loss investigation.
*note these are high level and not intended to be 100% complete. The idea is to highlight areas of influence*



MAC times – Access times post compromise date, not explained by business processes or applications, not attested to by a user, not explained by registry analysis. No sign of MAC time tampering.

Depth/Breadth of penetration - System/root/administrative level access obtained on a system or obtained on multiple systems having access to sensitive data. Attacker had access to files or databases containing sensitive data. Stolen credentials used to log in to business systems and user account has access to sensitive data.

System - Log files suggest data was acquired. Registry analysis shows signs of searching for, or looking through files, opening files containing sensitive data, USB history shows signs of unrecognized devices being used. Internet history shows attacker activity indicating data exfiltration. In other words, this is the typical forensic analysis of a system.

Attack Profile - Targeted attacks, spear phish against specific group or individuals having access to sensitive data. Attack directed at a singular and specific target containing sensitive data.

Detection - I've discussed time previously so I won't cover it, but I will summarize it by saying that when the window of time from time of compromise to time of containment is longer than 3 months, the decision maker tends to be influenced by this fact. The same applies if the window is very small, say 24 hours. The speed with which an incident is detected is a large factor for the decision maker.

Network - Flows/packet captures suggest that data traveling to external entities involved in the incident contained sensitive information. Encrypted traffic flows to/from attack related IP addresses that can not be explained by configuration file.

Malware - Sophistication of malware suggests the ability to log keystrokes, sniff network traffic, modify timestamps, search for and/or steal data. Malware related artifacts show sensitive data being accessed. Malware is designed for theft of sensitive data.

Of course there will be corner cases where companies *SHOULD* automatically notify as in the case of a stolen or lost laptop/tape/hard drive, and data is unenecrypted. This is a huge topic so I'll be discussing it again...

Outbreak!

I have briefly mentioned Mass Casualty Incidents in the past. It's time to delve in to this a little and see where we end up. I'll likely spread this out over a few posts.

One of the most widespread diseases in existence is malaria. There's an estimated 200 to 300 million cases worldwide each year. 2-3 million of those result in death. There is currently no vaccine.

Let's focus on malaria for the time being. Malaria is primarily spread through female mosquitos that pass a parasite to the victim. That is to say a mosquito attaches itself to a victim, and injects saliva in to the wound to keep blood from clotting and the blood flowing. There are areas of the world where mosquitos are highly prevalent, and these are also places of high infection rates.

Wait a second. Let's summarize. An infectious disease, spread worldwide, causes death, and there is no vaccine, only treatment?

Sounds a bit like a malware infection, or rather a malware outbreak doesn't it? What if I were to tell you this is like the USB malware infections that spread all over, and caused the military to take a draconian approach of banning USB keys?

I say this quite a bit but the best way to master your field is to study the methods used in other fields. For an outbreak of this nature I refer to treating and preventing of malaria.

Think about it. Infected USB media is exactly like a mosquito, they contain a parasite and infect computers by injecting the executable referenced in their autorun files.

Let me spell this out for you. When you're faced with a transient population in the tens of thousands and a computer population of twice that number, and you have malware that spreads from one population to another, what do you do? That is to say you've got mobile people with infected USB keys and systems that are either infected or about to be infected.

Think malaria. Kill the mosquitos, innoculate and protect the uninfected, treat the infected. Unfornately this is a problem. Ever tried to track down thousands of USB keys? How do you get a hold of the USB keys? How do you kill the infection on their USB keys?

The answer is obvious. You can't track them down. So, let's focus on the second and third problem. The solution, as is often the case, presented itself.

In a highly distributed and decentralized environment (as many large organizations are), what needs to occur? Coordination, Communication, Information. This is step 1. Without this, everything else fails.

Consider calling emergency gathering of key staff to establish the process and procedure for dealing with the threat. Once the scope of the threat is conveyed, the action plan is established and off you go. Instructions and ideas get shared and the uninfected population is already in the process of being further protected by local IT staff.

What about the infected and the unknown?

In the digital world, you can't kill USB keys by spraying them with repellent and other chemicals and you can't compel tens of thousands of people to turn over their USB keys. But you can establish a triage center for the people in posession of them, and ask them to bring them in. There are two problems with this approach.

1) Scope of population. There is of course a realization that not all USB keys will be accounted for, but through the coordination of efforts to innoculate and protect the uninfected, while treating the infected, an intersection occurs, whereby both populations get protected and treated.

2)Laziness. People will not go out of their way to get a flu shot and they will not go out of their way to get their USB key checked. So, do what the medical field does, establish triage centers in multiple, high traffic areas.

So just what is a triage center for USB keys? It consists of uninfectable systems (mac and/or linux systems), and scripts to detect infected USB keys. Simply have an individual insert their usb stick and within seconds you know if you've got an infection. Then you innoculate the USB stick and make changes to attempt to prevent a recurring infection. In addition, you provide the person with the equivalent of a flyer that has detailed instructions to follow to innoculate and prevent infection of their computer.

But wait..what's missing here? Knowledge of the threat. In the midst of all of this, signature development needs to occur and threat assessments must continue. This is all about continuous information gathering. Samples need to be gathered and analyzed to determine the types and functionality of the malware. A line must be drawn that differentiates high value assets from the assets of little to no value. This is where further triage takes place.

More on this later.

Why your antivirus can't tell you anything useful.

I wrote about your antivirus being unable to tell you anything here and here. I want to take a quick minute to tell you why your antivirus product can't tell you the things you want to know. Or how about you hear the AV industry tell you why they can't tell you anything? The following two quotes are from an AV vendor.

"The most effective detection nowadays is either generic (detection of whole families and sub-families), proactive (heuristics, sandboxing, emulation etc), or hybrid.”

“In the 90s, a good heuristic scanner could claim to detect something like 70-80% of new malware: clearly, that's no longer the case.“


That sort of explains some things I think. The primary detection method is generic, followed by a proactive and hybrid detection model. In short, Antivirus products can't do what they claim to - which is protect your system from malware infections. And when they do detect malware, they are unable to tell you a whole lot about it since the method of detection is nonspecific. Harlan's been on a rampage lately discussing how antivirus vendors are unable to provide adequate information to Incident Responders and I tend to think this explains the source of the problem.

That AV vendors are admitting they can't do the same thing they used to, I tend to think it's past time organizations move beyond antivirus products and in to new markets. Antivirus products are now marginalized and they can't keep up with the malware onslaught. I don't say this to be pessimistic, but I do tend to think it's true. The battle is not being lost, it already is lost. Relying on Antivirus products alone is simple negligence.

Who dropped their pants?

I am developing a new reality game show for intrusion analysts and investigators. I'm calling it
"WHO DROPPED THEIR PANTS?"

There are several ways systems get compromised, but more often than not it's due to a misconfiguration or sloppy management of controls. I constantly refer back to something Charl van der walt of sensepost said a few years ago about sysadmins being able to screw up only once. That has stayed true. I've analyzed countless incidents where the root cause was determined to be gross misconfiguration leading to compromise. The vendor could come in and instruct the sysadmin to disable the host firewall to make a specific piece of software able to function. A sysadmin could tire of testing and rush a system in to production before it was ready. A list of passwords could be stored on root of the drive in cleartext. You get the idea. Someone is dropping the pants on a system right now in your organization in order to get something to work. Why, for many people what's the first troubleshooting step when there's a firewall involved? Disable the firewall of course. Well, did they turn it back on? My most favorite line in the past has been "That system is a Mac running OSX, are you sure it's compromised?"..this being a question I receive from clients when I alert them that something is amiss.

The first thing I want to know is why in this day and age are people still being given absolute control over a system when their job is to manage only one functional role held by the system? Take a database server for instance. Does the DBA need full admin rights over the operating system, or do they need limited or no rights to the operating system, and full rights over the database they are responsible for? When the untrained has more access than required and they are knowledgeable enough to inadvertently do damage the organization is begging for trouble. Unfortunately this is an all too common occurrence in the IT field. Hence the name of my new game "WHO DROPPED THEIR PANTS?" So for us as responders what do we do?

Well, you want to find out who has access to a system and what rights do they have. Then you want to find out when they were logged in to the system and were they logged in at or around the time of compromise. In addition, is the proper logging in place to determine their actions at the time of compromise? These are just a few things to keep in mind when determining "WHO DROPPED THEIR PANTS?"

The internet is for porn

It's a well known saying to be sure. The internet is for porn. Perhaps the funniest depiction of this was a chappelle show episode . It's probably NSFW if you've never seen it, and it's absolutely hilarious, but it definitely raises a number of points that speak to the industry we work in.

Here's a scenario for you..

Joe the finance executive at a bank is browsing the web. He visits a news site, and a link to a site that suggest adult conversations is flashing in the ad banner space. Joe is happily married yet he's curious, and temptation overrules logical thought. He's acting completely right brained. Living in the moment for the moment, not thinking about the future. He visits the "adult conversation" site and bam! he's assaulted with pictures and popups of all forms of pornography. Now Joe's in a whole other world. His basal instincts have taken over and what was supposed to be a quick check in of the local news turned in to a trip down porn lane. A few clicks later and an install of flash player, and he's merrily watching some streaming porn on his laptop at work.

Joe is happy, Joe is enjoying himself.


You, sitting in your position of overwatch, looking for strange and outlandish network behavior notice Joe's computer doing something like this:

111.222.33.44,FSPA,27289,72.213.167.190,FSA,80,909,573,11,6,0,0,TCP,POST / HTTP/1.1..Host: iexujguw.com..Content-Length: 116..Connection: close.....,HTTP/1.1 200 OK..Server: nginx/0.5.33..Date: Fri. 05 June 2008 16:20:27 GMT.

111.222.33.44,FSPA,42583,212.55.163.216,FSA,80,784,687,10,6,0,0,TCP,POST /4D3D07E3ABDFC3C5/qxUX4xETUFYBWqc0kaWCzvoCcAQCYSNwZgcyFiBAByC73XXm0CcAYgVSB,HTTP/1.1 200 OK..Server: nginx/0.5.35..Date: Fri. 05 June 2008 16:40:16 GMT.

111.222.33.44,FSPA,16197,212.55.163.216,FSA,80,848,1054,11,7,0,0,TCP,POST /4D3D07E3ABDFC3C5/qxUX4xETUFYBWqc0kaWCzvoCcAQCYSNwZgcyFiBAByC73XXm0CcAYgVSB,HTTP/1.1 200 OK..Server: nginx/0.5.35..Date: Fri. 05 June 2008 17:00:17 GMT.

111.222.33.44,FSPA,3884,66.102.1.101,FSPA,80,1334,12549,13,14,0,0,TCP,POST /safebrowsing/downloads?client=navclient-auto-ffox&appver=3.0.5&pver=2.2&wr,HTTP/1.1 200 OK..Content-Type: application/vnd.google.safebrowsing-update..Date:,,1010,,

111.222.33.44,FSPA,4124,66.102.1.100,FSPA,80,1322,12549,13,14,0,0,TCP,POST /safebrowsing/downloads?client=navclient-auto-ffox&appver=3.0.5&pver=2.2&wr,HTTP/1.1 200 OK..Content-Type: application/vnd.google.safebrowsing-update..Date:,,1010,,

111.222.33.44,FSPA,59619,212.55.163.216,FSA,80,784,687,10,6,0,0,TCP,POST /4D3D07E3ABDFC3C5/qxUX4xETUFYBWqc0kaWCzvoCcAQCYSNwZgcyFiBAByC73XXm0CcAYgVSB,HTTP/1.1 200 OK..Server: nginx/0.5.35..Date: Fri. 05 June 2008 17:20:18 GMT.

111.222.33.44,FSPA,51889,212.55.163.216,FSA,80,784,687,10,6,0,0,TCP,POST /4D3D07E3ABDFC3C5/qxUX4xETUFYBWqc0kaWCzvoCcAQCYSNwZgcyFiBAByC73XXm0CcAYgVSB,HTTP/1.1 200 OK..Server: nginx/0.5.35..Date: Fri. 05 June 2008 17:40:18 GMT.

111.222.33.44,FSPA,4392,66.102.1.101,FSPA,80,1438,12549,13,14,0,0,TCP,POST /safebrowsing/downloads?client=navclient-auto-ffox&appver=3.0.5&pver=2.2&wr,HTTP/1.1 200 OK..Content-Type: application/vnd.google.safebrowsing-update..Date:,,1010,,

111.222.33.44,FSPA,58415,212.55.163.216,FSA,80,784,687,10,6,0,0,TCP,POST /4D3D07E3ABDFC3C5/qxUX4xETUFYBWqc0kaWCzvoCcAQCYSNwZgcyFiBAByC73XXm0CcAYgVSB,HTTP/1.1 200 OK..Server: nginx/0.5.35..Date: Fri. 05 June 2008 18:00:20 GMT.

Joe has managed to visit one of the countless porn sites that is actually owned and/or operated by a sub-group in organized crime, or hosts malicious flash or other malware.

Joe, in his quest for local news, and following his temptations has opened up the organization to a whole new world of risk.

Joe is compromised.

Not only is he compromised but he's managed to get a copy of Sinowal loaded on to his computer. Joe, being the finance director at the bank has access to all of the financial information of all of the bank's customers, and he uses this access to run reports. Joe is now responsible for exposing the records for all of the customers of the bank.

Ok, enough about Joe.

What I find interesting about this all is how in a matter of a few seconds, one can go from a nice clean site to an awful bodega of porn in a matter of a few clicks. Like six degrees of separation, the internet appears to be '6 clicks to porn', as in from any site you can end up at a porn site in 6 clicks. It's like walking down a street in a major city and from block to block, you can go from the best part of the city, to the worst and most dangerous. I don't know many people that would willingly walk down a dark dank avenue known to have muggers and other dangerous people. Yet, people do it daily on the internet. Most users don't seem to put the two together. For some reason it's as if people still believe that computers are in a separate reality and whatever happens on a computer does not have the ability to affect real people or their lives.

If the saying is to be believed, that computers are deterministic then it can easily be stated that computers don't do bad things. People using computers doing stupid things leads to computers doing bad or stupid things.

That said, in the case of Joe, do you think he should be punished or should you simply investigate the computer intrusion? Do your intrusion investigations lead to investigation of the people using the computer? Is Joe the Witness, the Perpetrator, or the Victim? What's your decision making process?


There is a bit of intel to be gained from this post. Sinowal has definite characteristics on the network. In my experiences they are as follows:

• Always communicates with nginx webservers, acting as proxies.
• Uses a 20 minute timer, and will skew on occasion but not by much.
• Uses a static HTTP POST with 16 hex characters followed by a trailing slash.
• Uses a domain generation routine much like Srizbi and will do an HTTP POST to / at that domain name.

sticking out

Commonly, when an intrusion occurs, an attacker will leave behind various types of detritus. Often times this is in the form of malware or a toolkit. In a recent compromise the attacker downloaded and left 80 MB of tools behind! This is admittedly not par for the course. Typically I'll see about 5-10 MB of detritus per incident. Identifying these files and classifying them is increasingly difficult. Consider the picture I've placed here. In a world where all items are of approximately the same color, this ball sticks out like an eye sore against the backdrop. How about in a filesystem where good files are known, an unknown file will stick out. Ah...if only that utopia existed. It can, if you invest in whitelisting a common build, but for most people out there that's just not reality. Unfortunately not only is identifying malware difficult, but what do you do with something you suspect of being malicious?

Suppose you are a technician checking out a system that you suspect has been compromised. You check with your antivirus program and fail to detect anything strange. However, you notice something looks out of place. You could submit the malware to your antivirus vendor of choice, but depending on your licensing you may not get a response for 4-8 hours. Imagine malware living in your intranet for 4-8 hours because your vendor is slow. Believe me when I say that makes for a long day if you've never had to do it.

You could also bypass your vendor, get results quickly, develop a solution and roll it out in the time the vendor is working on a new definition. How would you do this?

As far as identification goes, what options are available? There are of course the more well known websites for submission. These sites have been mentioned and used by many for a long time now:
Virustotal
Anubis
CWSandbox
Norman
Jotti
Bit9

That's all well and good. I use these sites with some regularity, and the results have been good to date. There are some other options that you may or may not be aware of when you want to determine the badness(TM) of a file. Many of us are familiar with NSRL and other hashing projects and we should all be familiar with hashes and hashing files. Our tools do this for us automagically when we process an image, and if not, it's easy enough to generate a hashlist of the files in an image.

Suppose for a second that you have a list of files and their hashes, such as one generated by FTK Imager's directory listing capability. Suppose you want to take this list and look for possible malware. You could spend an obscene amount of money on a particular tool that matches hashes, or you could try some of these options:

Virus total hash matching
Offensivecomputing hash search
Team Cymru malware hash registry

And what about web based malware such as Javascript or Flash?

You could try these tools and sites:
Iseclabs Wepawet (The people behind Anubis)
Malzilla


For a little fun let's take a test drive of the Cymru malware hash registry.

First I'll hash some files(90% of which are malware):
loki:~ hogfly$ md5 *.exe
MD5 (724L1_setup_e.exe) = 848c95260d147543eff2e2c15acb58f1
MD5 (BR165652.exe) = d77d96af740af6805abcb2c572a758ce
MD5 (BR165680.exe) = 31c1d83de1db1aa5a806434d81183d79
MD5 (DNC-P-Ver.2.7.0.0.exe) = 518bcc3a6633dec8ceae3e0f02b4df60
MD5 (NATEON.exe) = af3c4884f690c48c115c8d9c55998141
MD5 (NVC.exe) = ee862735812241719960f2e069d99680
MD5 (abo.exe) = d2e23dbcbdd9a580b7897add524e4b09
MD5 (autorun.exe) = 3240c08878c7491b85b79c97db5c9204
MD5 (comrepl.exe) = 1d696a5dc70caa34d116344f50854d7f
MD5 (comrereg.exe) = 3619935460ddcb79f1ec9cc5710befc3
MD5 (dw8.exe) = f6da944f7c1ec3f0f8e6d673d9e9ff71
MD5 (envsetup.exe) = a1f8a82aad23a6b44cc92ee2eb1a10ef
MD5 (ff.exe) = 8a1b427981eecf67c60370b599c87dc6
MD5 (file.exe) = 107961dbceea53f729474b43c04302d4
MD5 (flvspeed.exe) = 45ce6d98337e4dba3e87d34adaf6d366
MD5 (hails.exe) = 5ebfe73e4fe237654a6bc07ed1712e7a
MD5 (hails2.exe) = 649d11e8d5676f0cee5c2a4a17f7e1a8
MD5 (index.exe) = 10980f4df2060b86a72eb5e533102980
MD5 (l07.exe) = 072ebc79aa1ff532c0d95f9a1ce4a395
MD5 (mfsl.exe) = 8fe25f71cbda9202995d74686eb5473e
MD5 (msdtc32.exe) = 205ca7ed3e6d8ae218c7fde2c50149f9
MD5 (net.exe) = c9c9a40e8a72907228e6a1bc9b5728ac
MD5 (net1.exe) = b8b857f3b5d8a8ef043fcf80120d0248
MD5 (omg.exe) = 654eef6ff6dbe666c1d9fd1f6049d525
MD5 (palzbn32.exe) = 28f02d257002221d367c0b43202c7a21
MD5 (pinyin.exe) = f9cbef1d67230b3845782b6fa11b976a
MD5 (rsscanner.exe) = a5953f3447a851f665702dd9afa63005
MD5 (scan.exe) = 29e20a4a5df73afee7acb3194f244b8e
MD5 (scann32.exe) = e464fb612104cc1da12c4d501cebe8df
MD5 (sss.exe) = 846790691b6f9717b9a1bf68e0bcd6e5

loki:~ hogfly$ whois -h hash.cymru.com 649d11e8d5676f0cee5c2a4a17f7e1a8
649d11e8d5676f0cee5c2a4a17f7e1a8 1224720131 31

loki:~ hogfly$ dig +short 649d11e8d5676f0cee5c2a4a17f7e1a8.malware.hash.cymru.com TXT
"1224720131 31"

loki:~ hogfly$ whois -h hash.cymru.com e464fb612104cc1da12c4d501cebe8df
e464fb612104cc1da12c4d501cebe8df 1221755478 25


So what do we have here? Two separate methods of doing a malware lookup (this is all explained on their site by the way). We simply feed them a hash and get back a unix datetime and the detection percentage of the malware. The detection percentage is a bit behind the curve and not all that accurate, but I would argue that it doesn't matter that much considering it's in their database and registered as malware. That doesn't mean I wouldn't like to see those numbers updated more frequently though. Cymru does support sending a large list of files via netcat for comparison, and I suspect that will be the most useful method of analysis for many out there.

For more active detection..If you've got about $25k I would strongly suggest you invest in a product called FireEye. I recently completed a demo of their product and all I can say is wow. I was most impressed. If you don't have the money for that you may want to try bothunter.


Here's to hoping you can identify and prosecute the investigation of suspicious files a little faster this year.

The witness, the perpetrator and the victim Part I

When we investigate a system compromise we are often left with only one portion of the cause->effect equation. It's up to us to take what we are presented with, and reconstruct a crime scene in order to determine what happened and often times determine whether or not PII data was acquired.

Using your imagination, try picturing the following scenario:
You arrive at a crime scene at a jewelry store and are lead to a body laying on the floor in a pool of blood. There is a broken lamp on a table, a large dent in the painted wall at about the 5'6" mark near the victim, a hole farther down on the wall at about the two foot height mark. There's blood spatter on the wall, floor and ceiling. Bloody footprints surround the victim and lead away from the victim out the back door of the store. The jewelry cases are smashed and there's blood on some of the glass. The victim is wearing a blue sweater and grey pants, is female, weighs 120lbs and is 5'6" tall.

So what you have is an apparent homicide with many traditional sources of evidence in play. How would you begin to investigate this scenario?

Now imagine the following. You are lead to the scene of a computer intrusion at a local bank. You arrive at the office of a credit card manager and see the following:

A black dell optiplex 755 sits under a desk and a 19" monitor resides on the tabletop. An external hard drive is plugged in to the computer and resides on the tabletop, and you note a USB key plugged in to the USB hub on the monitor. A small HP MFC unit is plugged in and rests on a small table next to the desk. Some papers litter the desk along with a tabletop calendar, a rolodex and a phone and a blackberry. The computer is on, and has Microsoft Outlook 2003 open on the desktop along with excel, Internet Explorer and one of the banks internal applications for credit card management.

This is pretty typical. So, how do you begin your investigation? What's the major difference in the two scenarios ?

In scenario 1, you have appear to have no one to interview. You must examine the deceased, review tapes, interview acquaintances and so on.

In scenario 2, you have a person to interview. The credit card manager was obviously using the computer and someone decided to call you for one reason or another. You must be able to determine if they are the witness to the intrusion, the perpetrator or the victim. Or if they are all three!

Suppose in scenario 2 you conduct your interview before you touch the computer (which I always recommend). What questions do you ask? Questioning a person can be seen as a bit of an arcane art form. The goal is to get the interviewee to be forthcoming with responses. Many people get embarrassed easily and get defensive, especially if they know they did something they probably shouldn't have. We want them to be calm and accepting of us and our questions. So, set some ground rules with your own team first. A few helpful rules of interviewing could be:

1) Never accuse.
2) Keep your cool. Emotions play a larger role in system compromises than people believe.
3) Be aware of your body language. You must always be aware that your face, posture and hand play, are a huge role in gaining the trust of the interviewee.
4) Ask leading questions.
5) Listen. You can't learn anything if you're talking.
6) Be nice.
7) Get them talking and keep them talking until you have enough information to proceed appropriately.

With the information I provided in scenario 2, you have no way of knowing what has happened yet, however, I am willing to bet you have already made some assumptions and perhaps even made some hypotheses. This is a natural occurrence in the brain and it's not a bad thing, unless you fail to view every angle because you develop tunnel vision.

Assuming the credit card manager told you the following how would you proceed?

• They arrived at 7:45am
• They opened Outlook to check email, and read some mail
• They opened an excel attachment containing this month's stats
  
• They plugged in their blackberry to sync it
• They plugged in their usb key to copy files they were working on at home
• They opened IE and visited yahoo.com and started researching colleges for their teenage daughter who is looking at schools.

What your antivirus isn't telling you

Ever look at your antivirus logs or the antivirus logs of a compromised computer and found something like SillyFDC or Trojan.horse? These happen to be generic definitions provided by Symantec, but other vendors have generic detection signatures too. Generic detection is a common method of dealing with malware. While generic detection is generally fantastic, it's a big double edge sword.

Let me explain about the two types of malware above.

SillyFDC is a generic signature for removable media malware.

Trojan.horse has the following caption: Symantec antivirus programs use Trojan horse as a generic detection when detecting many individual but varied Trojan horse programs for which specific definitions have not been created.

So, using these signatures, we call things we don't have signatures for but exhibits trojan like properties a "trojan horse" and something that uses removable media as a spreading mechanism "SillyFDC". Ok, no problem right?

It is in fact a problem.

Antivirus now being the 40% solution against bots, it's likely to miss a recent variant of malware. Additionally, when your clients or users discovered a variant of these types of malware, how are they to know what to do? It's been detected generically. Symantec says that the malware is a low risk. Is it really? Again, how is an organization to know? What about how long it takes for an infection to be detected?

In a real world scenario, I first discovered a variant of removable media malware some 30 days before a definition was made available by Symantec. This malware, not only spread by removable media, but was a key stroke logger as well. Once Symantec generated a definition for it, it was labeled as trojan.horse.

Now, let's look at this from a sysadmin perspective. You run a managed antivirus environment and one day, after your server and clients grab the latest set of definitions, you get an alert for malware called trojan.horse. Great! you say to yourself. My antivirus has done its job. You move on about your day as if nothing happened, afterall your AV product detected and removed the threat. You never bother to look at the file, or the timestamps of the file, and you certainly don't bother to investigate. This is an all too common problem and scenario.

What's my point?

When an antivirus product fires an alert for a generic detection, it always bears investigation. It stands to reason that when something is generically detected, it's much more serious than it appears. Using Trojan.horse as the example, when no existing definition exists, it gets classified as trojan.horse so it can be detected and removed. That's fine, but you have no idea what that malware is actually capable of. An immediate threat assessment should take place, even if you simply submit the malware to an automated sandboxing web site.

What should you look at:

• How long has the malware been on the system?
• What capabilities does it have?
• Has data been exfiltrated as a result of it?
  

Generic detection, while a good thing for the vendor, is a bad thing for the rest of us. It's misleading and provides no information whatsoever. Trojan.horse is a low threat level according to Symantec. I can think of no small amount of people that would consider a key logger a huge threat, especially one that was present on a system for 30 days before a definition was available.

*note I'm not picking on Symantec. This is an issue with all antivirus products*

Beware the key

USB keys are prevalent. They are used heavily by many incident response teams and first responders. They're less fragile than CD's, faster and offer greater storage.

They are also weapons of destruction and can become fast victims of compromised systems. It's been estimated that 10% of malware has the ability to infect removable media devices. Recall if you will that old is new. When you respond in an incident, you'll want to take some precautions if you use USB devices.

1) Make sure your devices are wiped and formatted after each case. If your device infected, your device becomes a weapon.

2) Create a directory named Autorun.inf in the root of your devices. This offers some protection against autorun malware.


To protect your Windows workstations if you haven't already, do the following things.

Copy and paste this in to a .reg file and merge it.

REGEDIT4
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\IniFileMapping\Autorun.inf]
@="@SYS:DoesNotExist"


Follow the instructions here:
http://support.microsoft.com/kb/953252