Wednesday, February 24, 2010

TACTICAL trial by fire

Last week, I received a phone call to perform a sensitive acquisition for Law Enforcement. A tragedy really, but out of it arises a short story of success with modern forensics tools.

When I arrived on scene I was briefed and went to search for the requisite equipment to perform the acquisition. As it turned out, the entire stock of wiped drives was gone. A 500GB drive was located, but it needed to be wiped. Wiping a 500GB drive takes up to a few hours, so that was no good. I did have some clean space on an acquisition RAID device though. Given the sensitivities of the operation I had to do this quickly, efficiently, and right the first time. The margin for error was slim as there was information on the desktop that couldn't be lost.

I went for the Ace up the sleeve. I had up to this point only used it in testing, but I went for a tool I knew could trust. The tool was none other than F-response TACTICAL. Yeah that's right, I went for live imaging in a Law Enforcement case. There are still plenty of those doubters and naysayers out there, so let me be clear. The time to adapt has passed, the need to preserve evidence when lives are at stake is paramount. It's time you adopt modern techniques. There is no such thing as forensic purity, in any forensic discipline when you've got volatile evidence. That's a myth created by those that have never worked in the field.

Photos taken, and requisite documentation completed, I plugged the victim system in to a local switch I had for this purpose. I then proceeded to insert the subject dongle in to the subject computer. I quickly popped the examiner dongle in to my station attached to the acquisition RAID. Configuration, always quick, included physical memory. Then I simply clicked on "auto connect" on the examiner console. Just like that, the disk and memory objects I needed were exposed. Firing up FTK imager, I made the acquisitions I needed. The case proceeded as many do, with hurried phone calls and stress like no normal incident can create. The evidence was secured for examination and the subject laptop was turned over.

I'm an Incident Responder, and a Forensic Examiner. I need tools I can rely on, tools that work in the clutch, tools that don't break the bank, tools to use when life and limb are at stake. For me, that's F-response. A very big thanks to Matt Shannon and the folks at F-response. I'm not sure how the field got along without you and you've made technology available that makes a real difference.

Saturday, February 6, 2010

We just don't get it

Given all the talk about APT lately I'm still shocked. Shocked that there are those out there on the 'good guy' side that can do nothing but criticize. One recent discussion that's been heavily debated is one of how "new" Advanced Persistent Threats are. My question to everyone out there:

"Does it really matter ?"

Every day these enemy combatants are lifting data. Lifting data from organizations they're not supposed to be lifting data from. These data are then being used against us to gain political, economic and military advantages. I've watched the data pass through systems for months and it turns my stomach to think that it's being done with such ease. Especially considering where the data is from. That these attacks occur is nothing new. That these attacks are taking place on such a broad scope is entirely new. That the enemy elements are moving against so many targets at the same time and in such different industries is alarming.

For years I've investigated cybercrime and done malware analysis and intrusion investigations. I can say with relative ease that while the tactics used in these attacks are not necessarily new, there is a certain 'newness' to this type of enemy. The majority of cybercrime that occurs today is automated. Malware has reached a point of templatization such that these toolkits are sold so others can perpetrate more crimes. While certain high profile attacks are definitely not automated and require a crew of clever individuals, many cybercrime incidents are automated.

These attacks are not very automated. Like a skilled tradesman, they reduce overhead by automating simple things. When the enemy gains access to your networks, reads your email, browses the internet on your computer, pretends to be you to garner more information from your colleagues, ignores your bank statements but takes schematics, ignores your customer credit card database, but steals your organizations futures documents and pilfers from your R&D group there's a difference. When the same group penetrates military systems and networks there's a difference. The difference is due to the global scale, the difference is in our ability to remain a competitive nation. The difference is in our military's ability to remain effective. The difference is that this is not just about money.

Regarding their malware:
Is it any wonder that the malware used by this enemy shares a common trait with other malware? There are a finite number of methods to accomplish a goal in a given programming language. Is there a reason not to re-use code if it works? Is it any wonder we can look at multiple samples of malware and draw comparisons? Give a fool a katana and he'll cut off his nose. Give a Samurai a katana and he'll cut you in half before you can blink your eyes. Malware is a tool of the enemy, not the enemy himself. The right malware in the hands of a skilled opponent is a force multiplier for a real threat, while malware in hands of a lesser opponent is a nuisance. This enemy is more than their malware.

There is no data breach notification when this enemy penetrates a network and steals data. The notification comes when we have another financial crisis and a foreign government is bailing us out. The notification comes when we have another gas shortage like in the '70's. The notification comes when power grids fail. The notification comes when more of our commerce is outsourced and jobs are lost. The notification comes when our companies are being bought by foreign companies because they can no longer compete. The notification comes when our military can not protect our interests. This problem is bigger than the security industry. This problem is bigger than IT. The security and IT industries are impotent in this situation. This problem will take governments to solve.

The people that call it hype have not seen this enemy work. They have not seen the contents of the stolen files. The business that have recently started doing "Anti-APT audits" are missing the point and trying to capitalize on the situation to further their own business.

What should matter is how successful they have been. What should matter is defending ourselves. What should matter is how and where we share this information. What should matter is taking this information to those with the ability to do something about it. What should matter is taking the fight to the enemy.

So I ask again, does it matter if this threat is new?

Thursday, February 4, 2010

The APT is on your webserver

One of the key ways APT gets in to your network is through human exploitation. Duh. We are the weakest link and in my experience it's usually those with some form of fiscal responsibility(re: business offices) that are the weakest. The APT also uses remote exploitation as a weapon. If there's a vulnerable system out there, they find it, exploit it and set up shop. This is done quickly and is done often times before public exploits are available and before the related vulnerability is being widely scanned for.

However, they, at least in my experience, are limited. They seem to limit themselves to Windows systems. I've not yet seen (not that it hasn't happened, but I've not seen it) a Unix system compromised by the APT. If you have, chime in at any time. So far, they've all been Windows systems. This is understandable and predictable. One place I've seen the APT establish a presence is on a web server. Yes, the APT is on your web server. In my experience this has been for C2.

Common traits of an APT web server compromise that I've seen:

System traits:
Windows Server 2003

Management traits:
Often poorly managed - the system may be a development system, or one that is in the process of being decommissioned.
Administrator is the most commonly used account for management.
Security logs and auditing is weak and not offloaded or rolled over periodically.
RDP is available

Compromise traits:
They modify forward DNS lookups for their domains to point to your system.
They don't really attempt to hide their presence.
They create files and host them on your webserver.
Excessive use of the Administrator account, often during non-business hours.
Server may begin proxying traffic to/from China.
A pattern change of many to one relationships, meaning your server will begin seeing requests from many hosts that it normally never receives traffic from and requests are for files and pages that didn't exist prior to the incident. This is often a behavioral pattern anomaly.

Logs on the server will likely indicate the presence of new files in the form of excessive requests to which your server will likely respond with a 404. That is of course, until your server goes active and DNS propagation occurs.

Your webserver may begin to initiate outbound connections to remote systems that it is not cleared to communicate with and may begin acting as a proxy.

The administrator account is being used to browse the web from the web server. This should be a no-no in any environment and is therefore an anomalous event.

Your webserver may resolve to a domain that is not yours.

As mentioned above, you'll note a behavioral change in who is talking to your server and for what.

*note these are not "special techniques". This is standard tradecraft.*

Cull your logs for:
Many hits from different IP's to the same page returning a 404. This is not uncommon on today's webservers, but if you exclude commonly searched for vulnerabilities you can easily do data reduction. This can easily be done with Logparser. A good but old article is here.

Administrator logins to your webserver from ip addresses that have no business with your server with administrative rights.

Administrative RDP sessions from external sources. Again a no-no..but if you've got it open, they'll use it.

Inventory your webservers and do DNS lookups (forward and reverse) on them using external DNS servers. If they're resolving odd domains then you've got something to look for.

Wednesday, February 3, 2010

M-trends reaction

**FTC disclaimer (re: middle finger) I'm not affiliated with Mandiant. I know folks at Mandiant only by name recognition and perhaps a few blog comment exchanges, or mailing list/forums posts. I, like you, have read the M-trends report. I do not have access to anything other than M-trends, a few M-unition blog posts from Mandiant and random interweb babble on the subject. I would love to have a discussion with the folks over at Mandiant but I do not see that happening any time soon.
FTC disclaimer**

Now that the obligatory disclaimer is out of the way..When reports like this come out it's interesting what happens. The reactions range all over the map. We, the good guys, are too busy sizing each other up, calling each other ignorant, pretending to know what we don't and holding on too tight to really discuss the issues. What I find most interesting is how apparently everyone is an APT expert all of a sudden, with 15 years of experience battling them, and yet for all of this experience and worldly knowledge, none of it has been shared beyond the contents of this report. Sure, it's discussed privately, in secrecy and behind closed doors but there is an entire industry that plays a part in this, and I'd estimate that perhaps 10% of it knows what's going on.

I looked at the M-trends report and thought wow this is a good explanation of what happens and how. This is good information for folks up the ladder to have. This report is what security folks have been talking about for years, what we're all actually so paranoid about. Mandiant does a great job of presenting the scope of the issue and provide a good explanation. However, there is little to no information at the tactical level and no information related to actually countering the APT in an organization. I understand's a report and they don't want the Chinese (oh don't act so surprised) to know just how 'on to them' the good guys really are. Mandiant also wants to continue to make money doing consulting work and selling premium services such as "counter-APT" investigations and what not. I understand this and do not begrudge them. They apparently do a great job and I'm sure their services are well worth it.

When vague reports like this get released, very few people attempt to validate the findings. Even fewer have the data to do so. As it so happens I've got a bit of data that's APT related. Well, maybe more than a bit and in short order will be sharing some of my own findings. Counter-APT operations are not simply after the fact. The reason they seem to be solely after the fact is due to the cost of defending an enterprise, the lack of awareness and poor governance in organizations. I do not want to make an APT "splash". I do want to unveil a bit of the mystery behind the Advanced and Persistent part of the APT. As I've said before, they are human, they are fallible, they are an anomaly, they are more than their malware, and they can be detected.

Back for another year.

Yeah I've been quiet..really quiet. I've got a lot of ground to make up. I've got products to write reviews about, important issues to discuss, things to say and share. Welcome 2010, it's February already and it's time to catch up.