Malware Analysis workflows: know your volatile memory-forensics, get the job done, and have fun with RAM

Tags: #<Tag:0x00007f8a1464d9f8> #<Tag:0x00007f8a1464d908> #<Tag:0x00007f8a1464d818> #<Tag:0x00007f8a1464d750> #<Tag:0x00007f8a1464d688> #<Tag:0x00007f8a1464d5c0>

This is a draft

What forensics now... volatile-memory?

The first time I ever heard of it was in a discussion with forensic analysts about “chain of custody”. They were discussing whether a court would accept memory as evidence. That was about 5 years ago.

In reality it’s that courts prefer “chain of custody” and control of evidence by the investigator. There is no perfect way to gather and store evidence. Memory Forensics is a forensics investigation approach to gather as much good records as possible. Maybe these records will be used.
But as an investigator you don’t know how a court is going to use these records. Depending on the court there are different thresholds of “foundation”, of the evidence being what it claims to be. Whether it gets admitted or not is therefore depending on the court. Formality varies.

Memory Forensics here is just another perspective, not just for Malware Analysis. Whether we can use it, or not, depends on the circumstances as well. Formality varies.

Regardless of whether it’s for Malware Analysis or Criminal Investigation, strive for control and discipline in record creation.

If you are interested in Malware Analysis, Forensics or Reverse Engineering you can check the other wiki and blog articles.


I briefly mention that malicious office documents may still carry artifacts from development, when the attackers leave their old Macro drafts in the OLE cache. Such caches are volatile, and analyzing them is volatile-memory forensics. Sometime ago I wrote a bad article about it as well… but that had to do with Digital Signal Processing and software modulation artifacts. If you dig it up, please let me know…

We can also look at less persistent and thus more volatile memory. Like RAM. This isn’t limited to workstations and servers. Memory forensics has become very common on mobile systems, such as Smartphones. I don’t know about the status of embedded and IOT memory forensics, but I will find that out.

RAM RAM RAM your boat - passwords on the streets

Passwords are just artifacts in memory-forensics. Other artifacts can add environmental and other contextual information to the infection chain analysis. You can get memory images from live systems, or hibernated systems. There are different memory acquisition tools for different systems and purposes. Memory acquisition tools need to be chosen carefully, because they can create evidence contamination issues.

The artifacts in memory forensics may include:

  • passwords, encoding keys, encryption keys, certificates
  • Open files
  • unpacked / decoded / encrypted executable segments
  • Registry Hives on Windows
  • File and web site parts
  • contextual process information, such as network socket data structures

Memory acquisition tools

  • FTK Imager - there is a Light version. This is a tool I personally use for this.
  • Newer versions of EnCase can do this.
  • The Pmem suite from the Rekall Memory Forensic Framework
  • Mandiant Memorize - I used that at work. They also have Redline.
  • WinDBG is quite good for this, actually. More targeted towards Malware Analysis.
  • Some VMware products, such as VMware workstation, can create a VM snapshot which has got a VMEM file
  • special hardware, debuggers (also for mobile devices)
  • electrical engineering…

Here is what’s important about WinPmem (which is OpenSource):

On Windows, one must insert a signed driver in order to gain access to physical memory. WinPmem from version 2.0 is built on top of the AFF4 imager technology, and is packaged bundled with the appropriate memory drivers.

That means it may be fine for Malware Analysis, and it’s free. Next step would be to throw the memory image into frameworks, such as Volatility.

Volatility basics

A good start is to start the examination with:

volatility -f mem.img --profile=Win7SP0x64 imageinfo

This means volatility will detect what kind of image this is. And if this concludes with at least some options, chances are good that the image isn’t garbage.

And then it’s up to you.

Let's find a Rootkit (hidden process) with(out) volatility and command-line Kung Fu

Generally the scan routines are more thorough than the listing routines in volatility. If there are differences, that means something tries to hide. Can be Malware, can be DRM, can be image issues.

volatility -f mem.img --profile=Win7SP0x64 pslist > list.txt
volatility -f mem.img --profile=Win7SP0x64 psscan > scan.txt

If you find hidden processes with psscan which pslist does not reveal, you have a good indication that something is odd. You can pipe the column to uniq with awk and diff the entries.

Here’s a hacky one-liner… ready?

awk 'FNR>2{ print $2 }' psscan.txt | sort -u > psscan
awk 'FNR>2{ print $2 }' pslist.txt | sort -u > pslist
diff psscan pslist


[email protected]:/tmp$ diff psscan pslist
< wmitcdc.exe

No, the one-liner is:

volatility -f mem.img --profile=Win7SP0x64 psxview --apply-rules

Volatility will also give you the offset within the memory file where the artifact resides. In this case, it’s Windows.

Windows has got a ressource tracking data-structure, called EPROCESS. For processess it’s basically a double-linked list called EPROCESS::ActiveProcessLinks.
This is explained in a freely available sample chapter of Windows Internals. You can get the list structure with WinDBG. Rootkits would not only dt nt!_eprocess, they’d edit the PEB of a process to remove the links to the neighbors. Then link the neighbors together instead. They need access to kernel memory to do this; be a kernel module or a device driver. The Windows scheduler works based on threads, not based on processes. Therefore removing the PEB block from the linked list doesn’t terminate the activity.
The Malware term for this is a DKOM rootkit. Memory forensics can be used find these.

volatility -f mem.img --profile=Win7SP0x64 driverscan
volatility -f mem.img --profile=Win7SP0x64 modules
volatility -f mem.img --profile=Win7SP0x64 modscan

Note: if you spot a process without a name, that can also be a proprietary driver or some missing data in the memory.

Active network connection enumeration from the memory image with volatility

volatility -f mem.img --profile=Win7SP0x64 connections
volatility -f mem.img --profile=Win7SP0x64 connscan

This will also list the endpoint the process initiated a connection to. sockets & sockscan will work for (hidden) sockets. And here some awk and diff might come in handy, for real.

Dumping a process from a memory image with volatility, and get the exe into IDA Pro

The offset is in the psxview output.

volatility -f mem.img procdump --profile=Win7SP0x64 -D /tmp/ -p 123
volatility -f mem.img procdump --profile=Win7SP0x64 -D /tmp/ -o 0x424242

In many cases this will be a non-packed non-encoded non-obfuscated process dump. You can run peframe (OpenSource, written by Gianni Amato). But the dumped executable has no imports table.

Search for traces of DLL injection or LDR list unlinking

A technique to detect this is called VAD tree walking[1]. VAD stands for Virtual Address Descriptor. Volatility will look for executable segments, and enumerate the VadS tagged nodes. A Volatility command, which also implements this, is malfind.

volatility -f mem.img malfind --profile=Win7SP0x64 -D /tmp/ > /tmp/malfind.txt
vim malfind.txt
egrep -B 35 -A 35 'explorer' malfind.txt
egrep -B 35 -A 35 'lsass' malfind.txt

Another command which can come in handy is ldrmodules.

There are many ways to hide a DLL. One of the ways involves unlinking the DLL from one (or all) of the linked lists in the PEB. However, when this is done, there is still information contained within the VAD (Virtual Address Descriptor) which identifies the base address of the DLL and its full path on disk. To cross-reference this information (known as memory mapped files) with the 3 PEB lists, use the ldrmodules command.

Unless you need to report this in detail, you don’t need to dive into the details here. I find it fascinating, but most people don’t.

[1] Proceedings of the Digital Forensics Research Conference, The VAD Tree: A Process-Eye View of Physical Memory by Brendan Dolan-Gavitt

It’s important to understand that these analysis commands can produce False Positives. One rule of thumb for me is, that if the injected assembly doesn’t look like malicious code, it’s an FP. If I don’t see a function prologue, or at least some MOV instructions which make sense, I go to the next possible finding.

In some cases it’s data, lots of 0s. This analysis step needs some experience. Another rule of thumb is, that attackers will prefer processes like lsass.exe or explorer.exe. Then I note the virtual address where the suspicious segment was detected, and look for the appropiate .dmp in the /tmp folder.

file /tmp/mem-0x424242-0x434343.dmp
peframe mem-0x424242-0x434343.dmp

Detect hook artifacts with volatility

  • Inline hooks (trampolines, API hooks) will modify the beginning of a function, and redirect the start of it toward the rootkit.
  • Then there are layering attacks with filter functions. These are usually archived in conjunction with reflective DLL injection techniques.
  • Call Table hooks will replace addresses of functions in a table

If a rootkit injects its payload into a running process, the respective code (of a DLL) isn’t backed by disk, because the injection isn’t persistent. If that’s the case, the Windows memory manager may flag these process pages as “dirty”. Now trampolines are common, many dev tools and debuggers use this as well. Volatility can detect this legit and malicious behavior.

volatility -f mem.img --profile=Win7SP0x64 apihooks

Few legit hooks use kernel32.dll to replace Windows functionality.

One of my more obscure tools...

  • You can use GNU objdump for a quick look at the PE dump.
    objdump -x -D mem-0x424242-0x434343.dmp | vim -c 'set syntax asmx86' -

  • I also use a tool called biew / beye on Linux, which is similar to Hiew. These were more popular before IDA Pro had a GUI port to Linux.

Working with dumped executables - search through their environment for secrets

Imports will be missing, some data or code gaps. Using the .dmp for code analysis is hard. It’s possible to look for encoding keys, passwords and other application secrets. These can be in ASCII or Unicode.

A tool which makes searching easier is The Sleuth Kit (TSK). Installing this is a lot of work, so you can just use SiFT, from SANS. Originally the RAM dump analysis was only present in PTK, which was commercial. This seems to be different now.

To get all strings (ASCII and Unicode) with TSK (>=4):

srch_strings -a -t 
String analysis, Base64 and other traces

If the Malware sample wants to connect to a CnC server or even a Dropzone via HTTP, it’s likely that the author implemented authentication. Often Base64 is used to encode the passwords. That means you’d also look for Base64 strings in memory and in the Malware samples.

It’s also possible that you find NTLM hashes. I made good experiences cracking these with Hashcat and a GPU.

Search through virtual memory of one specific process

Processes on Windows are segmented into Virtual Memory, so that the OS can manage them. In memory forensics we have one image, from the perspective of the OS. We acquired that image via a driver, that was interacting with the kernel. From the perspective of the kernel, page fragmentation is not an issue. There is a linked list, to piece it all together. From the perspective of Volatility though, things are different.

It might be necessary to go into the Virtual Memory of a selected interesting process. Here is a sample invocation to go deeper for one specific process:

volatility -f mem.img --profile=Win7SP0x64 memdump -D /tmp/ -p 123

  • the PID 123 is from psscan and / or pslist

You can find Downloads and Drops this way, in hex. Hopefully in a complete form.

Memory extraction with the help of the VAD tree with volatility

It’s possible that the apihooks and malfind results indicate that a certain process is affected by malicious behavior. memdump creates one dump file. There is a way to create on file per section.

volatility -f mem.img --profile=Win7SP0x64 vaddump -D /tmp/ -p 123

That means if there is an injected segment, this will be an extra .dmp. And we have not only detected a potentially malicious routine. We have isolated it as well. Next question is: why are the patched processes jumping into that segment. Ah, because someone injected code… right. :slight_smile:

Using IDA Pro on VAD dump segments

You can use IDA Pro to disassemble the .dmp, but since this is just an artifact you should rebase the regions. Edit → Segments → Rebase and set the Target to 0x525252 or what ever vaddump found.

You’d go to the jump addresses from the vaddump or malfind output, mark that area as code and force zero offsets. That will usually end with gaps, and it’s only worth it if you think there are new exploits in the dumps. For example if you know that the infected system was fully patched, and you also know that the infection happened recently.

Examine the Windows Registry with volatility

This is straight forward, and very common in forensics. The Windows Registry is a gold mine.

volatility -f mem.img --profile=Win7SP0x64 printkey

You can diff the command from before and after the infection, when you do Malware Analysis. At some point you know that something like windos_srv32.exe does not belong into HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services. That looks like a persistence mechanism.

Volatility diffing - automated analysis with VolDiff

If you want to deep dive into the before / after infection comparison of memory images, check out VolDiff by Houcem Hachicha. I personally don’t like it, because it reports a lot of non-malicious activity as well.

Scan for Malware persistence mechanisms with volatility

You can use:

volatility -f mem.img --profile=Win7SP0x64 autoruns --output=table | vim -

Search for commands with volatility

Some Malicious Documents tend to trigger cmd.exe commands. Some Shellcode does this. Sometimes the commands remain in the memory.

volatility -f mem.img --profile=Win7SP0x64 consoles

Summary : all your memory are belong to us

Making memory acquisition part of Incident Response on important workstations and servers can be very helpful. There are powerful new approaches to reveal malicious activity. These are artifacts of what really happened. Memory forensics builds a solid bridge between Malware Analysis and Forensics these days.

It’s possible to find kernel mode and user mode rootkits. With volatile memory we can add environmental and contextual information to our analysis.