Friday, June 1, 2007

Live Response Tool Testing

Over on the Windows Forensic Analysis group there was a recent discussion about live response tool invasiveness and some methods that people currently use. I started thinking about how to codify the methods used so that a standard methodology is created that can be used by anyone.

Below is a copy of my message to that list. I hope readers of this blog can respond with suggestions and any comments.
--

In order to effectively begin to measure the effects our tools have on systems we need to devise a testing methodology. Harlan recently asked "what's meant by testing of tools"?

I'd like to take a first whack at this one to see what others think. I'm hoping people add something to this. One thing to understand is that live response by its nature is not repeatable. However, testing in a controlled environment is as long as variables are identified and documented. Using a standard methodology will allow us to create a system by which our efforts can be measured, and our actions will be much more defensible as a result.

Goals:
To create a standard methodology for measuring the effects of live response tools on Windows Operating Systems.

Constraints and scope:
1) This methodology will be for live response tools only.

2) The methodology will measure the effects of the tool on: physical & virtual memory, the file system, registry, and network state. Modification will be measured in two categories. Type of data being modified and a quantifiable amount of each type of data being modified(i.e, a list of file modifications, amount of memory displaced etc).

3) The control system must be isolated and an accurate baseline must be established.

Needs:
Define an accurate baseline.
Identify tools and current methodogies for using each.
Create a standard baseline image.
Define the process for measuring the effects of the tool.

1 comments:

H. Carvey said...

Hogfly,

Based on your comment to my blog, I thought I'd comment on your blog... ;-)

Okay, I'm on-board with the goals...testing the effects of live response tools on Windows OSs. As defined, the effects include ...physical & virtual memory, the file system, registry, and network state. Modification will be measured in two categories. Type of data being modified and a quantifiable amount of each type of data being modified(i.e, a list of file modifications, amount of memory displaced etc).

As to the rest of it, I'm currently reading a paper that actually goes into this...it's actually a draft of a master's thesis. I'll have to get permission from the author to comment, but to be honest, I think that a lot of this information is already out there. A lot of it is stated in detail in my book, albiet not necessarily as a tool testing process.

You mentioned the Windows group in your comment on my blog...you should be aware from experience (even someone elses), what it's like when you try to involve others in such a process. ;-)