Export x Number of bytes around selected search hits - categorized by keyword hit

Updated - December 15, 2009 - Third version now available below

This EnScript is an update to the previous post here.

Changes & updates in this version:

1. Now includes the MD5 hash of the file the hit is located in (internal and unallocated files are excluded).

2. Keyword column now shows the hit text as well as the keyword. This is in case you used a GREP expression, it will show the expression and the hit.

3. The AFTER count now starts at the end of the keyword hit instead of the the beginning of the keyword hit.

4. The red highlighted keyword hit should now be accurate and only show the exact characters in the keyword hit.

There are now three versions of this EnScript available.

The first version is from the original EnScript that creates one "Proximity Report" with all the hits you selected from the search hits pane.

The second version is an adaptation from the original based on a reader's request. This second version creates one proximity report for each unique keyword hit. This version was created to easily facilitate redaction of certain hits.

This third version creates one proximity report per keyword. i.e. if you have five different keywords that you have selected in the search hit pane, then five different folders are created and in each foler is one report containing all the hits for that keyword.
Download here - updated original version
Download here - updated adapted version
Download here - updated third version

Harvest Keywords EnScript

This is a follow-up to the post I made on April 28 regarding the "Maine State Police - Keyword Search" EnScript.

This EnScript harvests keywords from selected files in EnCase. If you have a collection of contraband images, movies or whatever, you can load them into EnCase and then use this EnScript to generate a keyword list from a specific offset in each file. The original concept is to extract a unique keyword from somewhere in the middle of each contraband file to be used to positively identify it. Avoid using generic locations such at the header, which would get you hits of that file type, but they may not be contraband. The concept relies on the fact that you have a keyword from the original contraband file(s) that you can use to generate the keyword list (a kind of unique signature), then the original "Main State Police - Keyword Search" EnScript searches each cluster, in just the offset your keyword was harvested from to help reduce the time it takes and positively identify contraband, reducing the need to review every hit.

To use, just blue-check whatever files you wish to harvest keywords from:



Once selected, run the EnScript and pick the offset and size of the keyword. The longer the keyword harvested, the more unique and less chance of false positive hits.



A text file is created with whatever name you specify and the Length (LEN) and Offset (OFF) are appended to the filename, as well as the date and time to avoid accidentally overwriting an existing keyword list:



The list of keywords generated are displayed in the Console Tab of EnCase and can be viewed with notepad:





The generated keyword list can then be used with the EnScript posted on April 28th.

Download here

Maine State Police - Keyword Search & Export EnScript

Maine State Police Keyword Search and export EnScript – v1.0
April 28, 2009
EnCase v 6.13

This post is an update to a previous post here. This is an updated EnScript with new features and an official v1.0 project release. The following description is from the instruction document in the zip file linked below.

1. Introduction

This EnScript was born out of a concept from Sgt. Glenn Lang with the Maine State Police. Sgt. Lang needed a way to quickly and effectively identify known multimedia files that are commonly possessed by persons trafficking and possessing child pornography.

2. How it works

Sgt. Lang had an application named "harvester" written for the purposes of extracting a 10-byte string of hex values from inside any file. The thought was that instead of relying on file headers, to instead grab 10 hex values from the middle of a known video file or graphic file as a "mini" known signature for that specific file. The "harvester" program creates a simple text file with one 10-byte keyword per line (CRLF delimited). Investigators can use the "harvester" program to scan all their known bad media files and extract a mini "signature" for each of them, placing them into a small text file.

This EnScript was designed to read the text file created by "harvester" (or by any other means) and then begins searching the disk for those keywords. TheEnScript was designed to work in two ways. The first way is to search unallocated space, the second it to search all areas of the disk(s) (allocated files and unallocated clusters).

3. Configuration

When you start the EnScript, the investigator is presented with the following initial screen :



A. The first option asks for the text file where the keywords are stored. This text file should be a simple ASCII text file, one 10-byte keyword per line, in the format of:



B. This is the offset into the known file where the keyword was harvested from. The EnScript will search and export using this offset.

C. Total size of export - When a keyword is found, the EnScript will back up “x” bytes, as dictated by the offset value described above. The EnScript will then export from that offset (presumably the beginning of the file) for a total of “x” megabytes, as indicated by this value.

D. File Extension - When a keyword is found and the data around the keyword is exported (as described above), the exported data will be placed into the case default export folder and given the extension as indicated by this value. This is so if the investigator is searching for movie files and the data is carved from unallocated space into the default export folder, the investigator can quickly double click and use a viewer, such as VLC, to view the contents.

E. Comprehensive search - This checkbox dictates how the EnScript will search for the keywords. The normal built-in keyword search process in EnCase searches every byte of the disk (or unallocated cluster object). This EnScript, in an effort to speed this process up, by default, will search for the keyword only at the specified offset of each cluster, then move to the next cluster and look at the specific offset in that cluster and then move again to the next cluster. A typical Windows installation uses the NTFS file system and defaults to a cluster size of 4096 bytes (8 sectors per cluster). This means you are only searching 10 bytes out of those 4096 bytes, effectively only .2% of a cluster. The purpose of this is speed. If you think about how a file will always be saved on disk starting at a cluster boundary, then the keyword your looking for will always be found at the offset you specified in option B into a cluster. Searching the other areas of the cluster is unnecessary.

By checking this box (comprehensive search), the EnScript will instead search every sector. The reason for this option is in case the target drive had some files you are searching for and then the volume was formatted at some point in the past. The formatting process may inadvertently either change the number of sectors per cluster (i.e. was FAT, now is NTFS) or the boundaries of the volume have changed. Therefore, by selecting this option, the EnScript will search for the 10-byte keyword at a specific place in the sector, then move to the next sector and search again at the specific offset. This will increase the amount of time it takes to complete the search, but is still faster than a traditional keyword search where every byte of every sector is searched.

F. Bookmark - This will cause the specific keyword hit to be bookmarked when it found. Check in the bookmarks folder for a folder named “Found keywords in Unallocated – DATE & TIME”.

G. Search all files - The default is for this EnScript to only search unallocated space for the supplied keywords. Checking this box forces the EnScript to search unallocated, as well as every file in the case (allocated files).

4. Console - Real-time information is displayed in the console as the EnScript is running. If a keyword is found, the offset as well as full path of the file of where it was found is displayed.



5. Alert Sound - The EnScript will automatically check for the presence of a .wav file named "alarm.wav" in the “C:\Program Files\EnCase6\” root folder. If this file exists, this .wav file will be played every time a keyword is found during the search process. If the .wav file does not exist, the alert sound function will be skipped, but real-time information is still displayed in the console tab of EnCase.

Project information:
Sgt. Glenn Lang, Maine State Police
glang (at) mcctf.org

Download Here

Keyword List

Video demonstration #1
Video Demonstration #2

Search for keyword in selected file(s) and then parse till double CRLF

A friend contacted me this past week about a problem he was having parsing a large amount of data in unallocated. He had been searching for specific data that used to be in a text file and had since been deleted, but was still in unallocated. The data had a pretty logical structure, something like this:

label1:field1 label2:field2 label3:field3
label4:field4 label5:field5 label6:field6

label1:field1 label2:field2 label3:field3
label4:field4 label5:field5 label6:field6

.....

He wanted to parse out the data back into a text file so he could process it some more, but it needed to be one complete record per line. I wrote an EnScript that asks for a keyword. The keyword should be a unique keyword, that in this case was the text "label1" found at the begining of each record. The EnScript then parses from the keyword hit until it reaches a double CRLF. It then prints out the parsed data on one line to the Console tab.

Here is an example of the text in text view within EnCase:



So in this example, you would run the EnScript and enter a unique keyword that is found at the begining of each line, in this case "label1" appears at the beginning of each line and the EnScript will parse from the keyword hit until a double CRLF is encountered.



The result looks like this in the console tab:



I figured I would post the EnScript in case anyone else has a use for it.

Download Here

Keyword search with exclusion list - Beta

**Updated version now available - v1.5 (case sensitive & user-definable bookmark folder name)

An EnCase user contacted me about a problem he was having searching for a fairly common keyword and how he could exclude certain preceding or proceeding words that were adjacent to the core keyword.

Regular GREP expressions allow you to do this, but the EnCase version of GREP currently does not support using the NOT operator ("^") against multiple characters. So for example if you wanted to search for the keyword of "install". When the search was done, you may have thousands of hits, but see that the majority of them are a word like "clientinstall", and/or 10 other combinations of the keyword of "install" like windowsinstall, installapp, applicationinstall etc.. The EnCase version of GREP DOES allow the NOT character to be used such as this example:

[^t]install

This would find the keyword "install" as long as there was no preceding "t" character before it. This example would eliminate the term clientinstall" as I indicated above, but it would also eliminate all other possible keyword hits that may have the letter "t" preceding the keyword of "install", such as "statinstall", which I may want.

So to get around this scenario, I wrote an EnScript that asks for a keyword. It then also asks for a list of exclusion words (it can be pasted into the window). The EnScript then search ALL SELECTED items for ANSI and UNICODE occurrences of the keyword. It then reads the characters before and after the keyword hit to see if the hit is one of the exclusions you provided, if so, it skips the hit and moves on to the next hit. If the characters before and after do not match what you provided in the exclusion list, then it will bookmark it.

Using the example above, I ran the EnScript and entered "install" as my keyword. I then entered a list of exclusion words, such as:

productinstall
uninstall
appinstall
installapplication

Here is a screenshot of the three fields that require input:



The results exclude any keyword hits that are part of the exclusion list:



This is a beta version and it has not been tested completely. Please feel free to download and try and post any comments, bugs, suggestions.

Download v6 version here