Late night thoughts

• We're in an OE we created and don't control.

• Cyber is the new Urban and the adversary is the insurgent.

• The adversary looks like the populace, sounds like them, lives in their midst, and hides his activities among the normal and legitimate activities of the populace.

• Regular tactics don't work against irregular adversaries.

• Know your doctrine, study the adversary's.

• If someone punches you in the face, you're in a fight.

• If you stand still you will continue getting punched in the face.

• Espionage is a peacetime effort to us. To them, it's used as an opening salvo to position the pieces to control the center.

• There is a lot of room for deception in the modern computing environment.

• A mix of unorthodox and orthodox strategies is the only way to succeed.

• Know and understand the needs, capabilities, tactics, tools and methodologies of the adversary. This is asymmetry.

• Predictive capability can only come from studying history, yours and theirs.

• The use of malware, viruses, worms and other destructive software is encouraged, and condoned. This is killing with a borrowed knife.

• If I was in a different country, I would be expected to use my computer as a weapon.

• The siegfried line was overrun through cunning and persistence.

• The maginot line was flanked.

• A hardened structure can only protect you from that which it is hardened against.

• Siege warfare mentality no longer applies, yet it is practiced.

• A stationary target will always succumb to cunning and persistence, if it remains stationary.

• Counter offensives launched from stationary positions will hardly be effective.

• You must move as quickly as the adversary.

• Your culture has shaped your entire life. Study a different culture and adapt.

• When a country only wants to buy two of your products, it's so they can reverse engineer and copy them. Russia learned this the hard way.

• The farewell dossier event occured nearly 30 years ago.

• Master your own perception before manipulating the adversary's.

• If the adversary is hungry. He can be easily manipulated.

• Learn to build snowmobiles.

• The world is non-linear. Think in conceptual spirals.

• True intelligence is the result of synthesis.

• Once a target, always a target. Once a target, always a victim. Once a victim, always a victim.

checkmate

After reading the review by Kasparov on Chess Metaphors I began thinking of a few completely unrelated subjects.

On chess,

In the great game of chess, outright domination is not the goal - unless you outclass your opponent so badly that they have no chance of winning. Winning at chess is a mental game. The game is won through feints, sacrifices, and outsmarting your opponent through the use of strategy and tactics depending on the state of the board. It requires an immense number of calculations per second to be able to not only assess the current situation, but the results of the move you are about to make, and your opponents response. Not simply in the action->result mode of thinking, but in the symbiotic relationship that occurs when two grandmasters are locked in a titanic battle of the mind.

Plays are not made, they are developed. This is a guiding principle. No chess move is made for the sake of making the move. A chess move is made to develop a play that may not occur unless 15 other moves take place. This development comes in the form of moving pieces to positions on the board where they will have a greater overall impact in the middle and endgame.

Yet another guiding principle is that of controlling the center. Controlling the center refers to the squares in the center of the board. Controlling these squares, directly, or through pieces with direct access to them from afar can have a huge impact on the success of your game. Unsurprisingly, controlling the center influences your opponents maneuvers, cramps the available space on the board, and may ultimately provide openings for attack if and when your opponent makes a mistake due to your positional influence.

Paraphrased from the article: It used to be that becoming a chess expert would take years of study and practice. With the advent of computers and chess software, the game has changed in that young children are attaining a very high status in the field. What we now see in the game of chess is that the computer has leveled the playing field. In 2005, a "freestyle" tournament took place where competitors were allowed to use computers and the winners were not grandmasters. They were amateurs using three computers at the same time. It was their skill in manipulating the computers that allowed them to win. He summarizes this nicely with the following: "Weak human + machine + better process was superior to a strong computer alone and, more remarkably, superior to a strong human + machine + inferior process."

Better process wins.


Kasparov also discusses his experiences in using computers when he battled Topalov in a heads up competition.
"[..]With that taken care of for us, we could concentrate on strategic planning instead of spending so much time on calculations. Human creativity was even more paramount under these conditions."

Kasparov's tactical advantages were nullified by the computer and its ability to perform more calculations. Here he suggests that the ability of humans to think on their feet, to adapt to unfolding situations, the ability to innovate is what allows the human-computer combination to prevail.

"[..]Correctly evaluating a small handful of moves is far more important in human chess, and human decision-making in general, than the systematically deeper and deeper search for better moves—the number of moves “seen ahead”—that computers rely on."

Again, here he suggests that our ability to make the correct decision when faced with evaluating a small set of moves is a vital component to our decision making processes. All in all, Kasparov's review of the book is quite possibly just as fascinating as the book itself.

And now I leave Kasparov and the world of chess for a world of six legged pests known as ants.

Where I reside, it's getting warm again. When it gets warm in the house, I open up my windows at night to allow cold air in and warm air out. Unfortunately, this permits bugs to enter my residence. Ants are a fascinating little creature. They are full of a complex set of communication and societal roles and responsibilities. One day this past week I saw one of the forager ants making its way through my hallway. As you might be aware, when foragers are out exploring, they mark a path for others to follow. So as it was making its way through my house it was leaving invisible breadcrumbs for other foragers to follow. I quickly squashed the ant and went on my merry way, before it could continue to forage and perhaps find a food source. How could I know if it had already found a food source? How could I know if it wasn't already on its way back to the food source? How could I know it wasn't leading a more serious infiltration? Was it alone? I couldn't answer these questions, I squished it without bothering. I had work to do and no time to bother with ants. Working my way through the house..I spotted another ant. I squashed that one too, its carapace crunching beneath my shoe! I was victorious, two ants detected and killed in a few minutes. I was the defender of my home and I'd be damned if some ants were going to infiltrate my home!

Of course, having some experience with ants I knew full well that by the time I'd spotted those two, I had already been infiltrated. Over the next few days I spotted and squashed several more of the foragers, knowing there would be more. You see, ants are clever little creatures and the foragers may penetrate the house via different means and locations, they may work together or individually but their goal is a common one. Identify a food source, communicate it back to the colony, and begin exfiltrating in an effort to keep the colony healthy, strong and growing and continue foraging. Based on my experiences I had prepared for the inevitable infiltrators success. I had ant traps at my disposal and a plan to take care of them. The thing to remember about ants is that they need forage to find food sources worthy of taking back to the colony. I, as owner of the residence know where the food sources are, how the food moves throughout my house, how frequently the trash goes out, what windows were open, and based on other experiences what other ways the ants might be getting in. Of course there are ways in to the house that I have not yet identified. Ants are tiny little things and are very adept at crawling through the smallest of spaces! My battle with them persists, though I've deployed bait, traps and other active defensive measures. Here, take my poisoned food back to the colony, feed it to your queen and larvae.

So what does all of this have to do with anything? I'm not talking about chess and ants. I'm talking about the APT.

As it relates to chess, the APT are much like an advanced player thinking several moves ahead. Their process is well thought out. They have developed tactics to not only attack but defend their positions once they've made a move as part of a larger strategy to gain and keep access for extended periods of time. They may use complex moves to attempt to outsmart the opposition, they may use simple moves to lull us in to a sense of over confidence about the state of the board. They have a definitive offensive advantage. They can penetrate defenses pretty easily. They may have studied our culture to learn our biases to use them against us. They may use this to convince you to think that you can't stop them, you can only hope to contain them. They may be amateurs in some cases, but their ability to manipulate computers and follow a strong process creates situations where they can win just about every game they play. Their tactics may not be all that different than that of an ant. It can take an awful lot of effort to defend the home-front against the infiltrators. If, like me you find yourself too busy trying to take care of business by defending against them, you will have already lost, and eventually you may not have a business to attend to. Your product may no longer be yours. Your food will be exfiltrated, and you will continue to suffer infiltrations all season long. A good plan, and strong process for defending what's yours is in order. This is why things like IPB exist. To support decision making and allow your boss to apply appropriate resources to defend the enterprise at critical paths when and how he/she chooses. Nobody should know your environment better than you. If you don't know your own environment, stop and take the time to learn about it, otherwise your environment is not yours. Learn what defenses and resources are available and how to apply each one ahead of time. This is not something you want to learn about while under fire. Given the situation at hand, decide which measures to apply. Developing templates may or may not be the way to go. They can aid you greatly by removing a lot of the judgment calls, and by identifying available resources so you don't have to think about it. I know that in my current environment they have helped greatly, particularly around phishing attacks. Defending against advanced intruders takes an advanced defense, effective manipulation of the systems under your control and human creativity. Remember, it takes People, Ideas, and Hardware. The bad guys work in shifts around the clock to attack you. Are you working nearly as hard to defend?

The Tiger and the Ghost

Companies like Mandiant have placed themselves in the lead of the counter-APT fight in a lot of people's eyes. I respect this, and they certainly have teams with great skill and experience. They have done a great job of stirring up a lot of discussion, and have caused a lot of debate. An unfortunate side effect of this is that a lot of people have put on their firefighter hats, and are chasing ghosts. No, not ghosts that pop out of closets and say "boo!". I mean ghosts as in invisible warriors. Unfortunately, this effect is precisely what is expected, and possibly even wanted by our enemies.

A quick word about Mandiant's claims about our adversaries tactics. They are spot on.

Yes, this post is about the APT. However, it is not about their specific tactical assaults. I would submit that though this is important, the most important aspect of countering the APT is understanding him. I've asked before if it really matters how new this threat is. I still contend it does not. Walking that path is wasted effort. It's time to step up our game to understand the enemy.

For several years there have been warnings and warning signs that this adversary was up to something. Several ally countries were penetrated by the ghosts. The U.S. was hit hard during this time as well. While all of this is going on, officials are disavowing and claiming no knowledge of these attacks. Meanwhile, the military general standing behind said official is trying hard to keep his lip from curling in to a tiger's smile.

Stratagem 1:
Deceive the heaven to cross the sea.

For years, we have invited the tiger in for dinner. And why not? He knocked at the door and asked nicely. Only he didn't outright kill us. He learned about us, in the open, and with our invitation. It was accomplished through foreign exchange studies, open trade agreements, imports & exports, business mergers, the legal system, and watching us fight in Kosovo, Iraq, and Afghanistan. They watched us unleash our strike packages, and watched others defend against it. This was done until he felt he could learn no more. He took all he learned and used it for further study.

Stratagem 3:
Kill with a borrowed knife.

Our enemy is no doubt using the works of others to strengthen himself while wearing us down. If your army is not strong enough for attack, let the works of others weaken the enemy. They let our industries get worn down by the daily barrage of malware infections, lesser intrusions, and perhaps some more skilled adversaries. While all of this is going on, they conserve strength.

Stratagem 4:
Wait at ease for the enemy.

Our networks are under constant barrages by lesser opponents, or skilled opponents using simple techniques and tactics to wear down defenses and tie down huge numbers of opponents. Exhaust his will to fight before the real fight comes. This enemy clearly practices this strategy. These ghosts do not step up their game until they have to, they do not reveal the full breadth and depth of their plans until we match them.

What does our enemy want?
To establish the links between political, economic and military installations. To exploit ways to control & disable our ability maintain C2 or C4I. To identify key systems and perform what Mr. Tim Thomas calls "acupuncture warfare" with precision strikes. The adversary is aiming to close market gaps and gain the information advantage. This allows him to control and predict our responses and behaviors. Based on the study of "three-three" this adversary focuses on obtaining, transmitting, handling, and protecting information. He defends himself while controlling our actions or attempting to control our actions with the incursions we are seeing. He seeks to level the playing field through the use of information.

Too much reliance on technology to do the work has led us to a situation where many don't know how to begin fighting this adversary. He has been honing his skills in this space for nearly two decades. If you don't know yourself you will lose. That is, what are your key systems, what relationships do you have with other organizations and who maintains these relationships? What are your true capabilities? Do they match your requirements? These are just some of the questions that need to be answered.

This has nothing to do with what technology you can buy. This has everything to do with how you think, how your boss thinks, how their boss thinks and so on, and how your enemy thinks. If you're just joining, welcome to the fight.

Triage of Agent.BTZ

I'm a huge proponent of triage incident response. So much so that I developed procedures based on the idea that gathering a little information from key data points early can lead to an accurate assessment of the situation without having to conduct laborious processes such as creating a full disk image all the time. Triage saves time and effort. The purpose of triage is not to conduct a full analysis. The purpose is to 1) sort and prioritize and 2) gather enough information to decide whether or not to continue an investigation. It also maximizes the effectiveness of analyst, systems, and tools. Tools like F-response make triage possible. Tools like Responder Pro from HBgary make triage possible. Speaking of Responder pro..

I recently upgraded my copy of Responder 1.5 to Responder 2. I've got some great things to say about this product but I'll save that for another post. I ran an analysis some time ago in 1.5 against a dangerous little piece of malware that got quite a bit of press in 2008. The malware in question is Trojan.Agent.BTZ. This little gem is what ransacked the military and pentagon. Vendors like to call it Autorun malware, as that's really how it works but it's of course more than what a vendor will tell you.

Generally speaking I was looking for a piece of malware that infects removable media, phones home, gives remote control and downloads other malware. I arrived on scene a short while after the alert and after talking to the admin, decided to plug in my USB key, knowing full well what would probably happen to it. USB key defenses aside, I ran FastDump Pro and grabbed an .hpak memory dump.

So now I had a memory dump and I grabbed a disk image from the computer. In this case I decided to take the drive because I would use this later in a lab scenario for training. Triage requirements aside, this was a good case to capture for later use.

Let's analyze it quickly with 2.0 eh?

*note I've already done this in 1.5 I'm just re-doing it in 2.0 and during the middle of this I experienced a licensing hiccup*

My standard technique for beginning memory analysis in Responder is as follows:
1) Evaluate DDNA listing.

DDNA while not perfect, can be used to quickly hone in on oddities and badness. It helps identify WHAT is on the box, WHAT it might be, and HOW it might be working. Add a cross reference listing to the modules running on the box for more detailed information. DDNA is a boon to any analyst looking to conduct rapid analysis.

Here's what the DDNA output looked like for BTZ. Focusing on the left hand side, we don't see a whole lot that sticks out. A sea of Orange really...


2) Evaluate Network connections. This helps me answer two questions. WHAT is talking, and to WHOM? I tend to go for Network connections before Processes as network connections often identify the process I need to investigate. For all intensive purposes, 2&3 are interchangeable.

3) Evaluate Process lists.
Typically I evaluate processes in a number of ways. I'll look for processes that don't belong, those with odd names, those that are 'hidden', all svchost processes since they are a huge target for process and dll injection attacks, those processes that are "red flags" such as ones executing from the wrong directory or with incorrect paths and processes that don't normally exist.

4) Open file handles and Registry keys.
This should be fairly obvious as to the why. It allows me to find out what process has what file handles and reg keys open.

5) Use a DNS blacklist or keyword list. There are great blacklists out there plus I have a few extras. This immediately helps with data reduction in some cases. It can also assist in zeroing in on the malware. This is great for casting a drag net on a network to look for other infections.

6) Other poking - looking for clues that might tip me off to the true nature of the infection or compromise.

This usually does the trick for the overwhelming majority of malware cases I look at. Granted there are more difficult ones but with malware being as templated as it is..this tends to work.
So now let's work through this for real...

DDNA in this case didn't appear to be helpful, or was it? Looking at the listing, there's a wide variety of suspicious looking processes and modules. That's not really that helpful by all appearances. Let's add a little intelligence to this analysis by pulling up the module listing next to the DDNA listing for processes. This is what that looks like:


So, now we have even more orange...GREAT you might be thinking sarcastically..but what do we see if we look closely? Like a simple equation we can rule out common processes and modules that we have a possible explanation for now and I've highlighted something that looks REALLY suspicious..a process loading a module out of the user's profile. As I said DDNA is not perfect, but what it does is raise the interesting stuff to the top by severity and color coding. This is automated analysis and while it has limits, when we add human intelligence to the analysis process we get an immediate bead on the target.


So what are the key indicators?

* The .dll
* rundll32.exe is calling a .dll out of the user profile
* The file path for the .dll.

Yeah that's pretty odd isn't it? What traits does it have?


Nothing sticks out a whole lot, but there are some good clues in there.

So now I've found something odd and definitely worth looking in to a bit further. This happens to be the jackpot but let's keep evaluating.

How about the DNS blacklist for connections to known bad domains?

The list of hits was far too many to show here. The matches numbered upwards of 336 bad domains. That's too many domains to be helpful but it's definitely a sign that the computer was talking to a lot of known bad actors.

And Network Connections?

The network cable in this incident was unplugged when I arrived. No joy for active connections.

And the process listing?

There's one above the rest that sticks out:
C:\WINDOWS\system32\rundll32.exe" "C:\Documents and Settings\ctuser\Application Data\HELP\system32\rtn.dll"


That just about settles it for me. rtn.dll does not belong. Let's go ahead and process it. I always start by right clicking and taking a look at the strings. This allows me to drill right in to what I am interested in.

Immediate strings of interest:
C:\DOCUME~1\ctuser\LOCALS~1\Temp
C:\DOCUME~1\ctuser\LOCALS~1\Temp
C:\DOCUME~1\ctuser\LOCALS~1\Temp\
C:\DOCUME~1\ctuser\LOCALS~1\Temp\~DFD.tmp
C:\Documents and Settings\ctuser\Application Data\Help
C:\Documents and Settings\ctuser\Application Data\Help
C:\Documents and Settings\ctuser\Application Data\HELP\
C:\Documents and Settings\ctuser\Application Data\HELP\\system32\
C:\Documents and Settings\ctuser\Application Data\HELP\\system32\mswmpdat.tlb
C:\Documents and Settings\ctuser\Application Data\HELP\\system32\wmcache.nld
C:\Documents and Settings\ctuser\Application Data\HELP\system32
C:\Documents and Settings\ctuser\Application Data\HELP\system32\rtn.dll

Yep, this tells us a little about the files the malware is using. Point of interest here is that the malware created a directory structure for the user the malware was run under, and one of the few directories it could write to was in the profile as the user had no elevated privileges. Many people still think that Administrator rights is a means of stopping the execution of modern malware. Those people couldn't be more wrong but I digress...

Right now we don't know what the files are are specifically but we will soon...


How about this?
1dM3uu4j7Fw4sjnbcwlDqet4F7JyuUi4m5Imnxl1pzxI6as80cbLnmz54cs5Ldn4ri3do5L6gs923HL34x2f5cvd0fk6c1a0s


My that's odd isn't it? And it's referenced in three different memory locations. So just what is that string? The following code should give you a big hint.


If you can't tell that's an XOR key and function. We can use that bit of information later when we want to do deeper analysis. As we learn from later analysis, this XOR key is used to encode data written to log files by the malware.

When I did this analysis for real I completed the analysis and decoded the log files kept by the malware, and conducted a more thorough disk based analysis. The purpose of this posting is to illustrate a quick analysis method that pays off with the extraction of the XOR key to encode log files created by malware. In all, doing it this way takes about 15 minutes to get actionable intelligence. Think about it. In a just a few minutes we've gathered:

* Filenames on a filesystem. Pass that information off to your windows admins, and they can search desktops for the files.
* The XOR key to decrypt any discovered log files
* There's more to be seen in the memory dump that allows us to create a snort signature if need be(of course one already exists at present time).
* The malware does not require administrative privileges to execute or maintain persistence.

I realize this post is a bit incomplete..hopefully I'll get back to a continuation piece.

TACTICAL trial by fire

Last week, I received a phone call to perform a sensitive acquisition for Law Enforcement. A tragedy really, but out of it arises a short story of success with modern forensics tools.

When I arrived on scene I was briefed and went to search for the requisite equipment to perform the acquisition. As it turned out, the entire stock of wiped drives was gone. A 500GB drive was located, but it needed to be wiped. Wiping a 500GB drive takes up to a few hours, so that was no good. I did have some clean space on an acquisition RAID device though. Given the sensitivities of the operation I had to do this quickly, efficiently, and right the first time. The margin for error was slim as there was information on the desktop that couldn't be lost.

I went for the Ace up the sleeve. I had up to this point only used it in testing, but I went for a tool I knew could trust. The tool was none other than F-response TACTICAL. Yeah that's right, I went for live imaging in a Law Enforcement case. There are still plenty of those doubters and naysayers out there, so let me be clear. The time to adapt has passed, the need to preserve evidence when lives are at stake is paramount. It's time you adopt modern techniques. There is no such thing as forensic purity, in any forensic discipline when you've got volatile evidence. That's a myth created by those that have never worked in the field.

Photos taken, and requisite documentation completed, I plugged the victim system in to a local switch I had for this purpose. I then proceeded to insert the subject dongle in to the subject computer. I quickly popped the examiner dongle in to my station attached to the acquisition RAID. Configuration, always quick, included physical memory. Then I simply clicked on "auto connect" on the examiner console. Just like that, the disk and memory objects I needed were exposed. Firing up FTK imager, I made the acquisitions I needed. The case proceeded as many do, with hurried phone calls and stress like no normal incident can create. The evidence was secured for examination and the subject laptop was turned over.

I'm an Incident Responder, and a Forensic Examiner. I need tools I can rely on, tools that work in the clutch, tools that don't break the bank, tools to use when life and limb are at stake. For me, that's F-response. A very big thanks to Matt Shannon and the folks at F-response. I'm not sure how the field got along without you and you've made technology available that makes a real difference.

We just don't get it

Given all the talk about APT lately I'm still shocked. Shocked that there are those out there on the 'good guy' side that can do nothing but criticize. One recent discussion that's been heavily debated is one of how "new" Advanced Persistent Threats are. My question to everyone out there:

"Does it really matter ?"

Every day these enemy combatants are lifting data. Lifting data from organizations they're not supposed to be lifting data from. These data are then being used against us to gain political, economic and military advantages. I've watched the data pass through systems for months and it turns my stomach to think that it's being done with such ease. Especially considering where the data is from. That these attacks occur is nothing new. That these attacks are taking place on such a broad scope is entirely new. That the enemy elements are moving against so many targets at the same time and in such different industries is alarming.

For years I've investigated cybercrime and done malware analysis and intrusion investigations. I can say with relative ease that while the tactics used in these attacks are not necessarily new, there is a certain 'newness' to this type of enemy. The majority of cybercrime that occurs today is automated. Malware has reached a point of templatization such that these toolkits are sold so others can perpetrate more crimes. While certain high profile attacks are definitely not automated and require a crew of clever individuals, many cybercrime incidents are automated.

These attacks are not very automated. Like a skilled tradesman, they reduce overhead by automating simple things. When the enemy gains access to your networks, reads your email, browses the internet on your computer, pretends to be you to garner more information from your colleagues, ignores your bank statements but takes schematics, ignores your customer credit card database, but steals your organizations futures documents and pilfers from your R&D group there's a difference. When the same group penetrates military systems and networks there's a difference. The difference is due to the global scale, the difference is in our ability to remain a competitive nation. The difference is in our military's ability to remain effective. The difference is that this is not just about money.

Regarding their malware:
Is it any wonder that the malware used by this enemy shares a common trait with other malware? There are a finite number of methods to accomplish a goal in a given programming language. Is there a reason not to re-use code if it works? Is it any wonder we can look at multiple samples of malware and draw comparisons? Give a fool a katana and he'll cut off his nose. Give a Samurai a katana and he'll cut you in half before you can blink your eyes. Malware is a tool of the enemy, not the enemy himself. The right malware in the hands of a skilled opponent is a force multiplier for a real threat, while malware in hands of a lesser opponent is a nuisance. This enemy is more than their malware.

There is no data breach notification when this enemy penetrates a network and steals data. The notification comes when we have another financial crisis and a foreign government is bailing us out. The notification comes when we have another gas shortage like in the '70's. The notification comes when power grids fail. The notification comes when more of our commerce is outsourced and jobs are lost. The notification comes when our companies are being bought by foreign companies because they can no longer compete. The notification comes when our military can not protect our interests. This problem is bigger than the security industry. This problem is bigger than IT. The security and IT industries are impotent in this situation. This problem will take governments to solve.

The people that call it hype have not seen this enemy work. They have not seen the contents of the stolen files. The business that have recently started doing "Anti-APT audits" are missing the point and trying to capitalize on the situation to further their own business.

What should matter is how successful they have been. What should matter is defending ourselves. What should matter is how and where we share this information. What should matter is taking this information to those with the ability to do something about it. What should matter is taking the fight to the enemy.

So I ask again, does it matter if this threat is new?

The APT is on your webserver

One of the key ways APT gets in to your network is through human exploitation. Duh. We are the weakest link and in my experience it's usually those with some form of fiscal responsibility(re: business offices) that are the weakest. The APT also uses remote exploitation as a weapon. If there's a vulnerable system out there, they find it, exploit it and set up shop. This is done quickly and is done often times before public exploits are available and before the related vulnerability is being widely scanned for.

However, they, at least in my experience, are limited. They seem to limit themselves to Windows systems. I've not yet seen (not that it hasn't happened, but I've not seen it) a Unix system compromised by the APT. If you have, chime in at any time. So far, they've all been Windows systems. This is understandable and predictable. One place I've seen the APT establish a presence is on a web server. Yes, the APT is on your web server. In my experience this has been for C2.

Common traits of an APT web server compromise that I've seen:

System traits:
Windows Server 2003
IIS 6

Management traits:
Often poorly managed - the system may be a development system, or one that is in the process of being decommissioned.
Administrator is the most commonly used account for management.
Security logs and auditing is weak and not offloaded or rolled over periodically.
RDP is available

Compromise traits:
They modify forward DNS lookups for their domains to point to your system.
They don't really attempt to hide their presence.
They create files and host them on your webserver.
Excessive use of the Administrator account, often during non-business hours.
Server may begin proxying traffic to/from China.
A pattern change of many to one relationships, meaning your server will begin seeing requests from many hosts that it normally never receives traffic from and requests are for files and pages that didn't exist prior to the incident. This is often a behavioral pattern anomaly.

Anomalies:
Logs on the server will likely indicate the presence of new files in the form of excessive requests to which your server will likely respond with a 404. That is of course, until your server goes active and DNS propagation occurs.

Your webserver may begin to initiate outbound connections to remote systems that it is not cleared to communicate with and may begin acting as a proxy.

The administrator account is being used to browse the web from the web server. This should be a no-no in any environment and is therefore an anomalous event.

Your webserver may resolve to a domain that is not yours.

As mentioned above, you'll note a behavioral change in who is talking to your server and for what.

Detection:

*note these are not "special techniques". This is standard tradecraft.*

Cull your logs for:

Many hits from different IP's to the same page returning a 404. This is not uncommon on today's webservers, but if you exclude commonly searched for vulnerabilities you can easily do data reduction. This can easily be done with Logparser. A good but old article is here.

Administrator logins to your webserver from ip addresses that have no business with your server with administrative rights.

Administrative RDP sessions from external sources. Again a no-no..but if you've got it open, they'll use it.

Inventory your webservers and do DNS lookups (forward and reverse) on them using external DNS servers. If they're resolving odd domains then you've got something to look for.

M-trends reaction

**FTC disclaimer (re: middle finger) I'm not affiliated with Mandiant. I know folks at Mandiant only by name recognition and perhaps a few blog comment exchanges, or mailing list/forums posts. I, like you, have read the M-trends report. I do not have access to anything other than M-trends, a few M-unition blog posts from Mandiant and random interweb babble on the subject. I would love to have a discussion with the folks over at Mandiant but I do not see that happening any time soon.
FTC disclaimer**


Now that the obligatory disclaimer is out of the way..When reports like this come out it's interesting what happens. The reactions range all over the map. We, the good guys, are too busy sizing each other up, calling each other ignorant, pretending to know what we don't and holding on too tight to really discuss the issues. What I find most interesting is how apparently everyone is an APT expert all of a sudden, with 15 years of experience battling them, and yet for all of this experience and worldly knowledge, none of it has been shared beyond the contents of this report. Sure, it's discussed privately, in secrecy and behind closed doors but there is an entire industry that plays a part in this, and I'd estimate that perhaps 10% of it knows what's going on.

I looked at the M-trends report and thought wow this is a good explanation of what happens and how. This is good information for folks up the ladder to have. This report is what security folks have been talking about for years, what we're all actually so paranoid about. Mandiant does a great job of presenting the scope of the issue and provide a good explanation. However, there is little to no information at the tactical level and no information related to actually countering the APT in an organization. I understand this..it's a report and they don't want the Chinese (oh don't act so surprised) to know just how 'on to them' the good guys really are. Mandiant also wants to continue to make money doing consulting work and selling premium services such as "counter-APT" investigations and what not. I understand this and do not begrudge them. They apparently do a great job and I'm sure their services are well worth it.

When vague reports like this get released, very few people attempt to validate the findings. Even fewer have the data to do so. As it so happens I've got a bit of data that's APT related. Well, maybe more than a bit and in short order will be sharing some of my own findings. Counter-APT operations are not simply after the fact. The reason they seem to be solely after the fact is due to the cost of defending an enterprise, the lack of awareness and poor governance in organizations. I do not want to make an APT "splash". I do want to unveil a bit of the mystery behind the Advanced and Persistent part of the APT. As I've said before, they are human, they are fallible, they are an anomaly, they are more than their malware, and they can be detected.

Back for another year.

Yeah I've been quiet..really quiet. I've got a lot of ground to make up. I've got products to write reviews about, important issues to discuss, things to say and share. Welcome 2010, it's February already and it's time to catch up.