v2024.06

This release is mostly behind-the-scenes changes, with some refactorings to make it easy to use dfir-unfurl as a library.

What's Changed

• Update protobuf version to 4, regenerate pb-parsing code, and updat… by @obsidianforensics in #189
• Split out the requirements into different extras; move to pyproject.toml for packaging by @obsidianforensics in #196
• Separate out core and api logic. Make standalone "script" files f… by @obsidianforensics in #197
• Split out cli so it can be used without installing the web_app deps by @obsidianforensics in #198
• Move the run function into core.py by @obsidianforensics in #199

Full Changelog: v2023.09.05...v2024.06.27

v2023.09

What's Changed

• Tighten base64 parsing to skip input that's all letters (to reduce fa… by @obsidianforensics in #172
• Update Mastodon edge styling and add 200+ more Mastodon servers. by @obsidianforensics in #174
• Add parse_jwt.py to parse JSON Web Tokens. Add supporting functions… by @obsidianforensics in #175
• Add parser (and test) for DNS DoH URLs by @obsidianforensics in #176
• Adjust for changes in pymispwarninglists by @obsidianforensics in #178

Full Changelog: v2022.11.01...v2023.09.05

v2022.11

What's Changed

🚀 Features

• Use flask-restx for the API by @Rafiot in #137
• Add description nodes for some analytics and tracking query string parameters by @obsidianforensics in #154
• Add expansion of Substack redirects to link shortening parser. by @obsidianforensics in #158
• Add parsing of LinkedIn URLs: Messages (v1 and v2), Profile IDs, and … by @obsidianforensics in #163
• Add expanded list of share codes from Twitter (s param). Make image… by @obsidianforensics in #164
• Add run and print_tree functions to make it easier to use in Jupy… by @obsidianforensics in #167

🛠️ Minor Changes & Fixes

• If a URL is the whole query string or fragment, recognize as data_typ… by @obsidianforensics in #147
• Up required pymispwarninglists version and remove workaround by @obsidianforensics in #148
• Update parsing of Google's RLZ parameter to handle 1-4 digit cohort weeks by @obsidianforensics in #150
• URL parsing fixes by @obsidianforensics in #155
• fix issue #156 by @jkppr in #157
• Fix issue #160 by @jkppr in #161
• Require version of flask-restx that fixes compatibility issue with fl… by @obsidianforensics in #168

New Contributors

• @Rafiot made their first contribution in #137
• @jkppr made their first contribution in #157

Full Changelog: v2022.02...v2022.11

v2022.02

What's Changed

🚀 Features

• Add parser for Brave Search by @obsidianforensics in #127
• Add parsing for Google Search "aqs" parameter by @obsidianforensics in #131
• Use MISP "warning lists" to enrich domain names by @obsidianforensics in #134
• Add MISP's list of URL shortener domains to parse_shortlink.py; try t… by @obsidianforensics in #135
• Names of image attachments on Twitter contain a snowflake; decode it. by @obsidianforensics in #139
• Adds supports for nodes to have multiple parents by @obsidianforensics in #142
• Add URL redirect lookup for another type of Linkedin link shortener. by @obsidianforensics in #145

🛠️ Minor Changes & Fixes

• Parse pipe (|) delimited values (a|b|c|d); different than already-par… by @obsidianforensics in #128
• Fix error when decoding datetime_ticks due to Python dividing large n… by @obsidianforensics in #132
• Switch to embedded blackboxprotobuf to use updated code by @obsidianforensics in #133

Full Changelog: v2021.06.15...v2022.02

v2021.06.15

A new Unfurl release is here! See the blog post for more details.

New Features

• Add hash identification and lookups to online DBs
• Add option to allow remote lookups (-l); disabled by default. @obsidianforensics (#122)
• Identify random MAC addresses in UUID v1 @obsidianforensics (#121)
• Parse (some) Metasploit URLs @obsidianforensics (#116)
• Added two more link shortener sites: t.ly and urlwee.com @obsidianforensics (#115)

Fixes & Improvements

• Small assorted cleanups @obsidianforensics (#119)
• Add more checks, better logging around invalid values, and remove ext… @obsidianforensics (#123)
• Make parsing more consistent when url.path has a trailing slash @obsidianforensics (#124)
• Add tests for Metasploit URLs and hash detection (with and without lo… @obsidianforensics (#125)

v2021.03.11

Changes

• Add logging @obsidianforensics (#110)
• Swap out embedded blackboxprotobuf version for one in pip @obsidianforensics (#108)
• Add support for Yahoo Search @moshekaplan (#70)
• Adds Laravel Ordered UUID parsing @obsidianforensics (#106)
• Add parsing of Laravel "Ordered UUIDs", which also necessitated add E… @obsidianforensics (#105)
• Add parsing of file name & extension if part of URL path @obsidianforensics (#104)

20201102

The 20201102 release adds:

• New examples page on main Unfurl screen
• Improved parsing of Bing searches
• Parsing "biw" & "bih" for Google searches
• Parsing Epoch Hex and FileTime Hex timestamp formats
• Parsing bare TikTok IDs (not in the URL)
• Parsing "country" variations as "generic" QSP
• Parsing a few Dropbox URLs

This release is also on PyPI as 20201102.

20200812

The 20200812 release adds:

• Parsing TikTok URLs, including extracting embedded creation timestamps from IDs (https://dfir.blog/tinkering-with-tiktok-timestamps/)
• Parsing the "continue_time" for YouTube URLs (thanks kistee!)
• Parsing Sonyflake IDs (https://github.com/sony/sonyflake)
• Parsing of "generic" QSPs; for now, just "lang" and "language"

This release is also on PyPI as 20200812.