Snapshot, Image, Hash

What to collect, in what order, and why a hash taken at the start is what makes everything after it believable.

3 min read

Not yet reviewed

Evidence decays at different speeds. Network connections vanish in seconds, running processes in minutes, a disk image not at all. So there is an order, and it is simply *most perishable first* — collect the thing that will be gone soonest, before the thing that will still be there tomorrow.

Put these in the order you would collect them, from the thing that disappears fastest to the thing that will keep.

Open network connections
Running processes and their arguments
Logged-in users and their sessions
Contents of /tmp
Log files on disk
File modification times
A disk snapshot
A hash of that snapshot

Show the answer

Seconds — collect firstOpen network connections, Running processes and their arguments
Minutes to hoursLogged-in users and their sessions, Contents of /tmp
Days, or until somebody overwrites itLog files on disk, File modification times
Keeps indefinitely once takenA disk snapshot, A hash of that snapshot

In practice on a single VPS this collapses to three moves, and you can do all three in about five minutes.

One — snapshot the disk

Almost every provider will take a snapshot of a running volume from their control panel. Do this first even though disk is the *least* perishable thing on the list, because it is the one action that takes a minute of waiting and needs no decisions from you. Start it, then carry on.

Two — capture what is only in memory

five commands, in this order

$ ss -tunap > /evidence/connections.txt
$ ps auxww --forest > /evidence/processes.txt
$ w > /evidence/sessions.txt
$ ls -l /proc/*/exe 2>/dev/null > /evidence/binaries.txt
$ date -u >> /evidence/collected-at.txt

Write them somewhere that is not the compromised disk if you can — a mounted volume, or straight over SSH to your laptop. Anything you write to the box itself changes the box, and you are about to spend a day reasoning about which files changed and when.

Note

ps auxww --forest rather than plain ps aux: the tree shows you what started what. A shell whose parent is your web server is a very different fact from a shell whose parent is your login session, and the flat listing cannot tell them apart.

Three — hash everything immediately

A hash taken now is what makes everything you do afterwards believable, including to yourself. It is the difference between "this is the log from that morning" and "this is a log I have been grepping and editing for six hours and I think it is the original".

hash first, work on copies

$ sha256sum /evidence/*.txt > /evidence/MANIFEST
$ sha256sum -c /evidence/MANIFEST
/evidence/connections.txt&#58; OK
/evidence/processes.txt&#58; OK
/evidence/sessions.txt&#58; OK

You have five minutes before you must bring the service back. What do you do with them?

From here on, you work on copies. The box can go back into service, be rebuilt, or be thrown away entirely, and none of your evidence changes — which is exactly the freedom the five minutes bought you.

Tip

Write the collection commands into a script now, before you need them, and keep it somewhere that is not the server. At 7am you will not remember ss -tunap, and the whole point of this chapter is that those minutes are the ones you cannot have back.