Sign in Subscribe

Plaso Windows Build

Mike

• 8 min read

I'm trying to nail down the steps to build the Log2Timeline Plaso Windows executable. Part of this is to make it more accessible to the community (for example, Autopsy, which still uses a 2018 version). Part is to make my job faster and easier (rather than having to stand up docker, or run it via WSL which seems significantly slower). If you scoff at both of those reasons though, then most of it is just because I'm stubborn.

There's multiple requests on the Plaso GiHub issues asking for the Windows build. People are generally pointed to this statement, where it is mentioned that as of version 20200121, the packaged versions are no longer being built (the last Windows build was 20191203). People are also pointed at this documentation to build their own, which, pardon the blutness, is a bit of a "draw the rest of the fucking owl" article. But hey, at least it's a place to start, and I fully recognize that open source software owes me nothing. Still, it's been nearly four years and PyInstaller (the state of which is blamed for no more Windows bundles), has gone from version 3.5 to 5.11. I hope this prompts someone within the Log2Timeline/Plaso group to rethink the Windows bundle, or at least try to update the documentation.

So lets get started on what I've found and what still doesn't work. As of this writing, I have a completed and running build, but things are still wrong at a level that provides non-functional results. Maybe this will be the groundwork for someone else to get things running, and if so, I'll update this article at that point.

Requirements

Step #1 - Lab setup/pre-reqs/requirements

Git for Windows is an all default install (next -> next -> etc. -> finished), as is Microsoft C++ once you select the "Desktop development with C++" selection on the main menu.

For Python, we're going to select "Customize Installation" so we can install it to a different directory, and select the option to "Add python.exe to PATH". No changes to the next optional features screen. On the advacned options, change the install location to "C:\Python311". Once the install is finished, select "Disable path length limit" and then close.

I'm going to create a folder under the root of C named "working" just to keep things simple (C:\working). I'm going to open a command prompt, change to that directory, and git clone some necessary things

C:\working>git clone https://github.com/log2timeline/plaso
C:\working>git clone https://github.com/log2timeline/l2tdevtools
C:\working>git clone https://github.com/log2timeline/l2tbinaries
C:\working>git clone https://github.com/pyinstaller/pyinstaller

Plaso is the tool we're trying to build. l2tdevtools is ideally used for grabbing the pre-reqs, those being the l2tbinaries. Neither of the latter seem to work, but I'm going to walk thorugh them just to see. PyInstaller is going to be used to build Plaso, we can go ahead and take care of that.

C:\working>cd pyinstaller
C:\working\pyinstaller>python -m pip install .

Step #2 - l2tdevtools (problem 1)

Hint: Read this, but skip these steps if you're aiming the least broken build.

We're now to the Preparations stage of the Plaso packaging documentation I mentioned above. Let's move into the l2tdevtools directory, install the wmi pre-req, set the PYTHONPATH as instructed, and run the update.py function and...

C:\working\pyinstaller>cd ..\l2tdevtools
C:\working\l2tdevtools>python -m pip install wmi
C:\working\l2tdevtools>set PYTHONPATH=.
C:\working\l2tdevtools>python tools\update.py --preset plaso --track stable
Undefined preset: plaso


C:\working\l2tdevtools>

oops, we've hit our first snag, Undefined preset: plaso. I don't know why this won't work. I've raised this as an issue and tried quite a bit of troubleshooting. I've tried Python 3.10, I've tried setup.py build and install first, I've tried tracking down exactly where the issue is (seens like user_defined_project_names is returning an empty set) and manually setting that value, to no real effect.

In any case, this should just automate the l2tbinaries appearently, so lets just go do that.

Step #3 - l2tbinaries

Hint: Read this, but skip these steps if you're aiming the least broken build.

After step 2 fails, I tried installing the binaries in C:\working\l2tbinaries\win64 manually. All 85 of them, pointing them at our C:\Python311 installation directory. Mind numbing, but not as bad as I figured. Things did build after this, but I seemed to get better results using pip to install the majority of these packages. After just using the MSI's, psteal.exe failed with multiple errors.

C:\plasotest>psteal.exe --source "C:\\Windows 11 Pro.vmdk" -o dynamic -w 20230515test.csv
Traceback (most recent call last):
  File "tools\psteal.py", line 21, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 352, in exec_module
  File "plaso\cli\psteal_tool.py", line 11, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 352, in exec_module
  File "plaso\cli\extraction_tool.py", line 20, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 352, in exec_module
  File "plaso\parsers\__init__.py", line 35, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 352, in exec_module
  File "plaso\parsers\spotlight_storedb.py", line 15, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 352, in exec_module
  File "lz4\__init__.py", line 14, in <module>
ModuleNotFoundError: No module named 'lz4._version'
[36844] Failed to execute script 'psteal' due to unhandled exception!

C:\plasotest>

At this point I kind of shugged and gave up. I've seemingly gotten further via other methods.

Step #4 - Plaso Pre-reqs

Let just... forge our own path without l2tdevtools. At this point we're somewhere between Preperations and Packaging in the build doc. Let's move back into our plaso folder and use the requirements.txt file there. We'll get some warnings if we don't install wheel first, so lets do that as well.

C:\working\l2tdevtools>cd ..\plaso
C:\working\plaso>python -m pip install wheel
C:\working\plaso>python -m pip install fakeredis
C:\working\plaso>python -m pip install mock
C:\working\plaso>python -m pip install -r requirements.txt

This part takes a while, go get a beer. At the end of this, if you use run_tests.py you should get a few failure messages. I've stripped out all the [ok]s

C:\working\plaso>python run_tests.py
Using Python version 3.11.3 (tags/v3.11.3:f3909b8, Apr  4 2023, 23:49:59) [MSC v.1934 64 bit (AMD64)]
Checking availability and versions of dependencies.
[OPTIONAL]      unable to determine version information for: flor
[OPTIONAL]      missing: xattr

Checking availability and versions of test dependencies.
[OPTIONAL]      missing: fakeredis
[FAILURE]       missing: mock

We can pip install fakeredis and mock. Flor I'm going to assume is fine (and it's optional anyway). xattr appearently doesn't work on Windows.

C:\working\plaso>python -m pip install fakeredis
C:\working\plaso>python -m pip install mock
C:\working\plaso>python run_tests.py

This time run_tests.py produces a ton of results. At the end I get the following results:

Ran 1334 tests in 440.332s

FAILED (failures=7, skipped=2)

For the 7 failed tests, these all appear to be issues with line endings, e.g. AssertionError: b'place,user,password\r\n' != b'place,user,password\n' Note the \r\n vs \n. In other words, I think things are working, we're just not quite strict enough. Maybe this is a problem in the future, but for now I'm aiming just to get -anything- that works. Lets call 1327 success and 7 failure an overall success.

Step #5 - Build and Install

C:\working\plaso>python setup.py build
C:\working\plaso>python setup.py install

Not much else here, should be no issues with either one of these.

Step #6 - Build and Install

We're now back in line with the Packaging step in the original Plaso help document. Lets start by opening make_release.ps1 in C:\working\l2tdevtools\data\pyinstaller in your editor of choice. On line 20 and 24 we're going to change C:\Python37 to C:\Python311 so that the script won't bomb.

Now open an administrator PowerShell window and move into the plaso source directory. We'll then allow script execution, and run make_release.ps1

PS C:\Windows\system32> cd C:\working\plaso\
PS C:\working\plaso> Set-ExecutionPolicy unrestricted -force
PS C:\working\plaso> ..\l2tdevtools\data\pyinstaller\make_release.ps1

You'll probably see a couple of warnings that remove-item failed for a coulple of license files, but the build should have been successful and you should have a  plaso-20230413-amd64.zip file (or whatever your build number, NOT the current date, is). I'm now going to move this out of my VM and onto my primary desktop to test.

Step #7 - Testing and a few minor adjustments

Lets start with the most basic method we can use, psteal. The first time I run this, I get an error that time_zone_information.yaml is missing. Looking back on my test VM, this file does exist within C:\working\plaso\plaso\preprocessors. I'm going to grab that, and the mounted_devices.yaml file, because I can read into the future and know that this will be the next error, and move both of those over into C:\plaso-20230413-amd64\plaso\preprocessors (you'll have to create the preprocessors folder)

C:\plaso-20230413-amd64>psteal.exe --source "Windows 11 Pro.vmdk" -o dynamic -w test20230517.csv
[..SNIP..]
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\plasotest\\plaso\\preprocessors\\time_zone_information.yaml'
[6508] Failed to execute script 'psteal' due to unhandled exception!

C:\plaso-20230413-amd64>

Try to run it again and we get an error that pycreg is missing pycreg. Again going back to my build vm, I find a pycreg pyd file in the following location: C:\Python311\Lib\site-packages\pycreg.cp311-win_amd64.pyd . Copying that to my primary desktop, in the root of my new plaso build (C:\plaso-20230413-amd64)with all the other pyd files, seems to get things working.

C:\plaso-20230413-amd64>psteal.exe --source "Windows 11 Pro.vmdk" -o dynamic -w .test20230517.csv
2023-05-17 20:49:24,507 [INFO] (MainProcess) PID:5208 <data_location> Determined data location: C:\plaso-20230413-amd64\data
2023-05-17 20:49:24,516 [INFO] (MainProcess) PID:5208 <artifact_definitions> Determined artifact definitions path: C:\plaso-20230413-amd64\artifacts
Checking availability and versions of dependencies.
[OPTIONAL]      unable to determine version information for: flor
[FAILURE]       missing: pycreg.
[OPTIONAL]      missing: xattr.


C:\plaso-20230413-amd64>

Step #8 - It works, sort of...

Running it yet again, and things seem to be working. I get the expected message to select the partition, and then threads start craking away.

plaso - psteal version 20230413

Source path             : C:\plaso-20230413-amd64\Windows 11 Pro.vmdk
Source type             : storage media image
Processing time         : 00:10:15

Tasks:          Queued  Processing      Merging         Abandoned       Total
                1       0               17              0               44497

Identifier      PID     Status          Memory          Sources         Event Data      File
Main            9916    running         0 B             126695 (40)     60470 (2709)    ab31efed3ae748c193c9765802e06cbb
Worker_00       27572   idle            0 B             7210 (78)       231830 (173)    NTFS:\Windows\servicing\Packages\Microsoft-Windows-SMB1Server-D-Opt-Package~31bf3856ad364e35~amd64~~10.0.22621.1.cat
[...etc...]

HOWEVER, if we look at the resulting file (in Timeline Explorer), things are wrong. Every Message line is prepended with <WARNING DEFAULT FORMATTER>. While this column does contain some other data, it doesn't appear complete. Compaired to a good timeline run with the 2019 binaries, it looks like it has the correct number of rows. The Source and Source Long columns aren't correct at all (both just say "N/A").  Other columns do appeare complete and correct though, such as Timestamp, Parser, and Display Name.

I found a semi interesting event with a unique timestamp to show the differences side by side, here's the columns for each:

Timestamp	Color	Timestamp Description	Source	Source Long	Message	Parser	Display Name	Tag Info
2024-03-23 17:33:41.994695		Expiration Time	N/A	N/A	<WARNING DEFAULT FORMATTER> Attributes: cookie_name: bcookie data:  host: linkedin.com httponly: False path: / persistent: True secure: True url: https://linkedin.com/	sqlite/chrome_66_cookies	NTFS:\Users\user\AppData\Local\Microsoft\Edge\User Data\Default\Network\Cookies	-


Timestamp	Color	Timestamp Description	Source	Source Long	Message	Parser	Display Name	Tag Info
2024-03-23 17:33:41.994695	WebHistory	Expiration Time	WEBHIST	Chrome Cookies	https://linkedin.com/ (bcookie) Flags: [HTTP only] = False [Persistent] = True	sqlite/chrome_cookies	TSK:/Users/user/AppData/Local/Microsoft/Edge/User Data/Default/Network/Cookies	-

So yea, parsers are working, it's grabbing the right things, we're not making stuff up, timestamps are correct. Something is wrong with the output though.

Wrapping up

I'm going to publish this as is at this point. I've performed this install 6 or 7 times, getting further along or trying something different each time. This is the closest I've gotten. I certainly haven't hit a brick wall, but I'm hoping some community involvement will help speed things up and lead to better results.

Sign up for more like this.

Enter your email
Subscribe