| @@ -160,8 +160,9 @@ The coding period ran from June through August. This is how the work above lande | |||
| 160 | 160 | - The [viewer asset path fix](https://github.com/mandiant/flare-floss/pull/1398) repaired the logo on GitHub Pages. | |
| 161 | 161 | - An [offline page](https://github.com/mandiant/flare-floss/pull/1399) made the viewer downloadable as a standalone file, with a getting-started landing page and a version-stamped, fully self-contained download. | |
| 162 | 162 | - A simpler [static HTML integration](https://github.com/mandiant/flare-floss/issues/1371) replaced the server approach, closing [#1395](https://github.com/mandiant/flare-floss/pull/1395). | |
| 163 | - | - A [docs update](https://github.com/mandiant/flare-floss/pull/1403) bringing the usage guide up to date with the new CLI is open for review. | |
| 163 | + | - A [docs update](https://github.com/mandiant/flare-floss/pull/1403) brought the usage guide up to date with the new flags, the summary output, and the cache environment variables. | |
| 164 | 164 | - A [CI tweak](https://github.com/mandiant/flare-floss/pull/1404) stopped PyInstaller builds from re-running on docs-only changes. | |
| 165 | + | - Shell completion scripts ([#1407](https://github.com/mandiant/flare-floss/pull/1407), open) generate tab completion for five shells directly from the argparse parser, closing [#1350](https://github.com/mandiant/flare-floss/issues/1350) once merged. | |
| 165 | 166 | ||
| 166 | 167 | ## Reviews I contributed | |
| 167 | 168 | ||
| @@ -170,6 +171,7 @@ Part of the work was reviewing others. The notable PRs I was asked to review: | |||
| 170 | 171 | - [#1319](https://github.com/mandiant/flare-floss/pull/1319): reviewed ELF parsing robustness, including fallback for corrupt binaries. I checked the new fallback logic and requested a real corrupted test file over a mock. | |
| 171 | 172 | - [#1326](https://github.com/mandiant/flare-floss/pull/1326): reviewed the global prevalence database update. I asked whether the large size gap between the Go and Rust sub-databases was intentional before approving. | |
| 172 | 173 | - [#1366](https://github.com/mandiant/flare-floss/pull/1366): reviewed the summary view revamp. I caught that the new filtering let untagged generic strings crowd out tagged interesting ones, and flagged now-dead code, before it merged. | |
| 174 | + | - [#1405](https://github.com/mandiant/flare-floss/pull/1405): reviewed my mentor's performance optimizations for static string extraction and rendering. | |
| 173 | 175 | - [#1273](https://github.com/mandiant/flare-floss/pull/1273), [#1294](https://github.com/mandiant/flare-floss/pull/1294), [#1328](https://github.com/mandiant/flare-floss/pull/1328), and [#1240](https://github.com/mandiant/flare-floss/pull/1240): robustness, performance, and cleanup reviews. | |
| 174 | 176 | ||
| 175 | 177 | ## Future work | |
| @@ -179,7 +181,6 @@ What to do next, roughly in order: | |||
| 179 | 181 | - **Finish the UI release.** The web viewer is deployed at https://mandiant.github.io/flare-floss/ on every change, and [#1399](https://github.com/mandiant/flare-floss/pull/1399) added the downloadable offline page with a getting started view. The standalone binary will ship that static HTML page instead of hosting a server ([#1371](https://github.com/mandiant/flare-floss/issues/1371)). | |
| 180 | 182 | - **Cut a beta release of FLOSS 5.** Once the open work merges, a beta gives people something to try and collect feedback on. | |
| 181 | 183 | - **Resolve the tracked product decisions.** [Viewer tag filtering](https://github.com/mandiant/flare-floss/issues/1377), [version bump and aliases](https://github.com/mandiant/flare-floss/issues/1378), and [deobfuscation default](https://github.com/mandiant/flare-floss/issues/1379). | |
| 182 | - | - **Document the new interface.** [#1403](https://github.com/mandiant/flare-floss/pull/1403) brings the usage guide up to date with the new flags, the summary output, and the cache environment variables. | |
| 183 | 184 | - **Grow the cache.** A v1 tradeoff: no eviction or quota on the cache directory yet, and very large result documents load fully into memory. | |
| 184 | 185 | - **Expand database coverage.** More languages and runtime sub-databases (the Go and Rust ones landed) would tighten noise filtering further. | |
| 185 | 186 | ||
vee1e / d561fcadc2627cc3dcb115d18847a236
Last active 4 hours ago
Lakshit Verma revised this gist 1 week ago · 89c6fe3
1 file changed, 3 insertions, 2 deletions
Lakshit Verma revised this gist 1 week ago · dc07253
1 file changed, 10 insertions, 8 deletions
Diff is too large to be shown
| @@ -153,7 +153,7 @@ The coding period ran from June through August. This is how the work above lande | |||
| 153 | 153 | - Mid-August: the [new command line](https://github.com/mandiant/flare-floss/pull/1355) and the [filters and summary](https://github.com/mandiant/flare-floss/pull/1360). The [web viewer](https://github.com/mandiant/flare-floss/pull/1358) was reworked to match the new results schema. | |
| 154 | 154 | - Late August: [result caching](https://github.com/mandiant/flare-floss/pull/1367), the [qs naming cleanup](https://github.com/mandiant/flare-floss/pull/1370), and the [final merge](https://github.com/mandiant/flare-floss/pull/1372), which completed the project. | |
| 155 | 155 | ||
| 156 | - | ### Post-merge | |
| 156 | + | **Post-merge** | |
| 157 | 157 | ||
| 158 | 158 | - The [OSS database rebuild](https://github.com/mandiant/flare-floss/actions/runs/32274349763) succeeded with its auto-opened update PR ([#1385](https://github.com/mandiant/flare-floss/pull/1385)). | |
| 159 | 159 | - A [build workflow fix](https://github.com/mandiant/flare-floss/pull/1384) unblocked the CI. | |
| @@ -192,3 +192,4 @@ The planning document was the other hard part. A discussion doc and an implement | |||
| 192 | 192 | ## Acknowledgements | |
| 193 | 193 | ||
| 194 | 194 | Thanks to my mentor Moritz Raabe for the straight talk and the support, and for reviewing far more than his share of large diffs. Thanks to Willi Ballenthin for the sharp secondary reviews. Finally, thanks to the rest of the FLARE team for their work in creating FLOSS. This project would not have happened without them. | |
| 195 | + | ||
| @@ -152,13 +152,14 @@ The coding period ran from June through August. This is how the work above lande | |||
| 152 | 152 | - Early August: the [unified pipeline](https://github.com/mandiant/flare-floss/pull/1347) became the default output path. | |
| 153 | 153 | - Mid-August: the [new command line](https://github.com/mandiant/flare-floss/pull/1355) and the [filters and summary](https://github.com/mandiant/flare-floss/pull/1360). The [web viewer](https://github.com/mandiant/flare-floss/pull/1358) was reworked to match the new results schema. | |
| 154 | 154 | - Late August: [result caching](https://github.com/mandiant/flare-floss/pull/1367), the [qs naming cleanup](https://github.com/mandiant/flare-floss/pull/1370), and the [final merge](https://github.com/mandiant/flare-floss/pull/1372), which completed the project. | |
| 155 | - | - Post-merge: | |
| 156 | - | - the successful [database rebuild](https://github.com/mandiant/flare-floss/actions/runs/32274349763) with its auto-opened update PR ([#1385](https://github.com/mandiant/flare-floss/pull/1385)) | |
| 157 | - | - a [build workflow fix](https://github.com/mandiant/flare-floss/pull/1384) | |
| 158 | - | - the [viewer asset path fix](https://github.com/mandiant/flare-floss/pull/1398) | |
| 159 | - | - an [offline page](https://github.com/mandiant/flare-floss/pull/1399) | |
| 160 | - | - an [offline page](https://github.com/mandiant/flare-floss/pull/1399) | |
| 161 | - | - a simpler [static HTML integration](https://github.com/mandiant/flare-floss/issues/1371), closing [#1395](https://github.com/mandiant/flare-floss/pull/1395) | |
| 155 | + | ||
| 156 | + | ### Post-merge | |
| 157 | + | ||
| 158 | + | - The [OSS database rebuild](https://github.com/mandiant/flare-floss/actions/runs/32274349763) succeeded with its auto-opened update PR ([#1385](https://github.com/mandiant/flare-floss/pull/1385)). | |
| 159 | + | - A [build workflow fix](https://github.com/mandiant/flare-floss/pull/1384) unblocked the CI. | |
| 160 | + | - The [viewer asset path fix](https://github.com/mandiant/flare-floss/pull/1398) repaired the logo on GitHub Pages. | |
| 161 | + | - An [offline page](https://github.com/mandiant/flare-floss/pull/1399) made the viewer downloadable as a standalone file. | |
| 162 | + | - A simpler [static HTML integration](https://github.com/mandiant/flare-floss/issues/1371) replaced the server approach, closing [#1395](https://github.com/mandiant/flare-floss/pull/1395). | |
| 162 | 163 | ||
| 163 | 164 | ## Reviews I contributed | |
| 164 | 165 | ||
| @@ -129,7 +129,7 @@ The module is `floss/cache.py`, with tests in `tests/test_cache.py`. | |||
| 129 | 129 | - [#1398](https://github.com/mandiant/flare-floss/pull/1398) (merged): fixed broken logo and favicon on GitHub Pages by switching to relative asset paths. | |
| 130 | 130 | - [#1399](https://github.com/mandiant/flare-floss/pull/1399) (open): added a download button that saves the viewer as one self-contained HTML file people can keep and open offline. | |
| 131 | 131 | ||
| 132 | - | The viewer lives in the `viewer/` directory and is now deployed at https://mandiant.github.io/flare-floss/ on every change. Serving it from the standalone binary is implemented in [#1395](https://github.com/mandiant/flare-floss/pull/1395), which adds a `--server` flag so the binary serves the bundled viewer and hands it the analysis results directly. | |
| 132 | + | The viewer lives in the `viewer/` directory and is now deployed at https://mandiant.github.io/flare-floss/ on every change. [#1395](https://github.com/mandiant/flare-floss/pull/1395) prototyped serving the viewer from the standalone binary via a `--server` flag; after review we decided on a simpler static HTML integration, so it is being closed in favor of that approach. | |
| 133 | 133 | ||
| 134 | 134 | ### 7. Reproducibility and CI hygiene (coding period) | |
| 135 | 135 | ||
| @@ -157,7 +157,8 @@ The coding period ran from June through August. This is how the work above lande | |||
| 157 | 157 | - a [build workflow fix](https://github.com/mandiant/flare-floss/pull/1384) | |
| 158 | 158 | - the [viewer asset path fix](https://github.com/mandiant/flare-floss/pull/1398) | |
| 159 | 159 | - an [offline page](https://github.com/mandiant/flare-floss/pull/1399) | |
| 160 | - | - [binary viewer serving](https://github.com/mandiant/flare-floss/pull/1395) | |
| 160 | + | - an [offline page](https://github.com/mandiant/flare-floss/pull/1399) | |
| 161 | + | - a simpler [static HTML integration](https://github.com/mandiant/flare-floss/issues/1371), closing [#1395](https://github.com/mandiant/flare-floss/pull/1395) | |
| 161 | 162 | ||
| 162 | 163 | ## Reviews I contributed | |
| 163 | 164 | ||
| @@ -172,7 +173,8 @@ Part of the work was reviewing others. The notable PRs I was asked to review: | |||
| 172 | 173 | ||
| 173 | 174 | What to do next, roughly in order: | |
| 174 | 175 | ||
| 175 | - | - **Finish the UI release.** The web viewer is deployed at https://mandiant.github.io/flare-floss/ on every change, and [#1399](https://github.com/mandiant/flare-floss/pull/1399) adds a downloadable offline page. Serving the viewer from the standalone binary is implemented in [#1395](https://github.com/mandiant/flare-floss/pull/1395), awaiting review and merge. | |
| 176 | + | - **Finish the UI release.** The web viewer is deployed at https://mandiant.github.io/flare-floss/ on every change, and [#1399](https://github.com/mandiant/flare-floss/pull/1399) adds a downloadable offline page with a getting started view. The standalone binary will ship that static HTML page instead of hosting a server, closing [#1395](https://github.com/mandiant/flare-floss/pull/1395). | |
| 177 | + | - **Cut a beta release of FLOSS 5.** Once the open work merges, a beta gives people something to try and collect feedback on. | |
| 176 | 178 | - **Resolve the tracked product decisions.** [Viewer tag filtering](https://github.com/mandiant/flare-floss/issues/1377), [version bump and aliases](https://github.com/mandiant/flare-floss/issues/1378), and [deobfuscation default](https://github.com/mandiant/flare-floss/issues/1379). | |
| 177 | 179 | - **Document the new interface.** The usage guide does not yet cover the new flags, the summary output, or the cache environment variables. | |
| 178 | 180 | - **Grow the cache.** A v1 tradeoff: no eviction or quota on the cache directory yet, and very large result documents load fully into memory. | |
| @@ -40,7 +40,7 @@ This submission is split into two parts. The pre-GSoC work below landed before t | |||
| 40 | 40 | ||
| 41 | 41 | ## Project status | |
| 42 | 42 | ||
| 43 | - | **The project is fully complete.** QS was refactored into first-class FLOSS modules, the unified pipeline is the default output path, the new command line is merged, caching ships with the tool, and the web viewer is merged. A source-level review of the merge [surfaced ten issues](https://github.com/mandiant/flare-floss/issues/1375), and I [fixed the code issues](https://github.com/mandiant/flare-floss/pull/1376). **QuantumStrand was then [merged into master](https://github.com/mandiant/flare-floss/pull/1372), [closing the project issue](https://github.com/mandiant/flare-floss/issues/943).** Three product decisions from the review are still tracked as open issues: the [viewer tag filter](https://github.com/mandiant/flare-floss/issues/1377), the [version bump and flag aliases](https://github.com/mandiant/flare-floss/issues/1378), and the [default deobfuscation behavior](https://github.com/mandiant/flare-floss/issues/1379). | |
| 43 | + | **The project is fully complete.** QS was refactored into first-class FLOSS modules, the unified pipeline is the default output path, the new command line is merged, caching ships with the tool, and the web viewer is merged. A source-level review of the merge [surfaced ten issues](https://github.com/mandiant/flare-floss/issues/1375), and I [fixed the code issues](https://github.com/mandiant/flare-floss/pull/1376). **QuantumStrand was then [merged into master](https://github.com/mandiant/flare-floss/pull/1372), [closing the project issue](https://github.com/mandiant/flare-floss/issues/943).** Three product decisions from the review are still tracked as open issues: the [viewer tag filter](https://github.com/mandiant/flare-floss/issues/1377), the [version bump and aliases](https://github.com/mandiant/flare-floss/issues/1378), and the [deobfuscation default](https://github.com/mandiant/flare-floss/issues/1379). | |
| 44 | 44 | ||
| 45 | 45 | ## Pre-GSoC work | |
| 46 | 46 | ||
| @@ -141,23 +141,23 @@ The viewer lives in the `viewer/` directory and is now deployed at https://mandi | |||
| 141 | 141 | The coding period ran from June through August. This is how the work above landed, in order. | |
| 142 | 142 | ||
| 143 | 143 | **June** | |
| 144 | - | - Early June: [AGENTS.md for AI contributors](https://github.com/mandiant/flare-floss/pull/1308) and the [move of the string databases to Git LFS](https://github.com/mandiant/flare-floss/pull/1309). | |
| 144 | + | - Early June: [AGENTS.md](https://github.com/mandiant/flare-floss/pull/1308) and the [Git LFS migration](https://github.com/mandiant/flare-floss/pull/1309). | |
| 145 | 145 | - Mid-June: [native ELF parsing](https://github.com/mandiant/flare-floss/pull/1312) completed the format coverage started before the coding period. | |
| 146 | 146 | ||
| 147 | 147 | **July** | |
| 148 | - | - Early July: the [automated string database pipeline](https://github.com/mandiant/flare-floss/pull/1327) landed, followed by [dependency fixes](https://github.com/mandiant/flare-floss/pull/1332) and [database-change diffs](https://github.com/mandiant/flare-floss/pull/1334) in the auto-generated PRs. | |
| 149 | - | - Late July: the first big integration step, [splitting QS into first-class modules](https://github.com/mandiant/flare-floss/pull/1337). | |
| 148 | + | - Early July: the [database pipeline](https://github.com/mandiant/flare-floss/pull/1327) landed, followed by [dependency fixes](https://github.com/mandiant/flare-floss/pull/1332) and [database-change diffs](https://github.com/mandiant/flare-floss/pull/1334) in the auto-generated PRs. | |
| 149 | + | - Late July: the first big integration step, [module split](https://github.com/mandiant/flare-floss/pull/1337). | |
| 150 | 150 | ||
| 151 | 151 | **August** | |
| 152 | 152 | - Early August: the [unified pipeline](https://github.com/mandiant/flare-floss/pull/1347) became the default output path. | |
| 153 | - | - Mid-August: the [new command line in two parts](https://github.com/mandiant/flare-floss/pull/1355) and the [render-time filters and summary output](https://github.com/mandiant/flare-floss/pull/1360). The [web viewer](https://github.com/mandiant/flare-floss/pull/1358) was reworked to match the new results schema. | |
| 154 | - | - Late August: [result caching](https://github.com/mandiant/flare-floss/pull/1367), the [cleanup that deprecated the old `qs` naming](https://github.com/mandiant/flare-floss/pull/1370), and the [final merge of QuantumStrand into master](https://github.com/mandiant/flare-floss/pull/1372), which completed the project. | |
| 153 | + | - Mid-August: the [new command line](https://github.com/mandiant/flare-floss/pull/1355) and the [filters and summary](https://github.com/mandiant/flare-floss/pull/1360). The [web viewer](https://github.com/mandiant/flare-floss/pull/1358) was reworked to match the new results schema. | |
| 154 | + | - Late August: [result caching](https://github.com/mandiant/flare-floss/pull/1367), the [qs naming cleanup](https://github.com/mandiant/flare-floss/pull/1370), and the [final merge](https://github.com/mandiant/flare-floss/pull/1372), which completed the project. | |
| 155 | 155 | - Post-merge: | |
| 156 | - | - the successful [OSS database rebuild](https://github.com/mandiant/flare-floss/actions/runs/32274349763) with its auto-opened update PR ([#1385](https://github.com/mandiant/flare-floss/pull/1385)) | |
| 157 | - | - a [CI fix for the build workflow](https://github.com/mandiant/flare-floss/pull/1384) | |
| 158 | - | - the [viewer asset path fix on GitHub Pages](https://github.com/mandiant/flare-floss/pull/1398) | |
| 159 | - | - a [downloadable offline viewer page](https://github.com/mandiant/flare-floss/pull/1399) | |
| 160 | - | - [serving the viewer from the standalone binary](https://github.com/mandiant/flare-floss/pull/1395) | |
| 156 | + | - the successful [database rebuild](https://github.com/mandiant/flare-floss/actions/runs/32274349763) with its auto-opened update PR ([#1385](https://github.com/mandiant/flare-floss/pull/1385)) | |
| 157 | + | - a [build workflow fix](https://github.com/mandiant/flare-floss/pull/1384) | |
| 158 | + | - the [viewer asset path fix](https://github.com/mandiant/flare-floss/pull/1398) | |
| 159 | + | - an [offline page](https://github.com/mandiant/flare-floss/pull/1399) | |
| 160 | + | - [binary viewer serving](https://github.com/mandiant/flare-floss/pull/1395) | |
| 161 | 161 | ||
| 162 | 162 | ## Reviews I contributed | |
| 163 | 163 | ||
| @@ -173,7 +173,7 @@ Part of the work was reviewing others. The notable PRs I was asked to review: | |||
| 173 | 173 | What to do next, roughly in order: | |
| 174 | 174 | ||
| 175 | 175 | - **Finish the UI release.** The web viewer is deployed at https://mandiant.github.io/flare-floss/ on every change, and [#1399](https://github.com/mandiant/flare-floss/pull/1399) adds a downloadable offline page. Serving the viewer from the standalone binary is implemented in [#1395](https://github.com/mandiant/flare-floss/pull/1395), awaiting review and merge. | |
| 176 | - | - **Resolve the tracked product decisions.** [Viewer tag filtering](https://github.com/mandiant/flare-floss/issues/1377), [version bump and flag aliases](https://github.com/mandiant/flare-floss/issues/1378), and [default deobfuscation behavior](https://github.com/mandiant/flare-floss/issues/1379). | |
| 176 | + | - **Resolve the tracked product decisions.** [Viewer tag filtering](https://github.com/mandiant/flare-floss/issues/1377), [version bump and aliases](https://github.com/mandiant/flare-floss/issues/1378), and [deobfuscation default](https://github.com/mandiant/flare-floss/issues/1379). | |
| 177 | 177 | - **Document the new interface.** The usage guide does not yet cover the new flags, the summary output, or the cache environment variables. | |
| 178 | 178 | - **Grow the cache.** A v1 tradeoff: no eviction or quota on the cache directory yet, and very large result documents load fully into memory. | |
| 179 | 179 | - **Expand database coverage.** More languages and runtime sub-databases (the Go and Rust ones landed) would tighten noise filtering further. | |
| @@ -152,7 +152,12 @@ The coding period ran from June through August. This is how the work above lande | |||
| 152 | 152 | - Early August: the [unified pipeline](https://github.com/mandiant/flare-floss/pull/1347) became the default output path. | |
| 153 | 153 | - Mid-August: the [new command line in two parts](https://github.com/mandiant/flare-floss/pull/1355) and the [render-time filters and summary output](https://github.com/mandiant/flare-floss/pull/1360). The [web viewer](https://github.com/mandiant/flare-floss/pull/1358) was reworked to match the new results schema. | |
| 154 | 154 | - Late August: [result caching](https://github.com/mandiant/flare-floss/pull/1367), the [cleanup that deprecated the old `qs` naming](https://github.com/mandiant/flare-floss/pull/1370), and the [final merge of QuantumStrand into master](https://github.com/mandiant/flare-floss/pull/1372), which completed the project. | |
| 155 | - | - Post-merge: the successful [OSS database rebuild](https://github.com/mandiant/flare-floss/actions/runs/32274349763) with its auto-opened update PR ([#1385](https://github.com/mandiant/flare-floss/pull/1385)), a [CI fix for the build workflow](https://github.com/mandiant/flare-floss/pull/1384), the [viewer asset path fix on GitHub Pages](https://github.com/mandiant/flare-floss/pull/1398), a [downloadable offline viewer page](https://github.com/mandiant/flare-floss/pull/1399), and [serving the viewer from the standalone binary](https://github.com/mandiant/flare-floss/pull/1395). | |
| 155 | + | - Post-merge: | |
| 156 | + | - the successful [OSS database rebuild](https://github.com/mandiant/flare-floss/actions/runs/32274349763) with its auto-opened update PR ([#1385](https://github.com/mandiant/flare-floss/pull/1385)) | |
| 157 | + | - a [CI fix for the build workflow](https://github.com/mandiant/flare-floss/pull/1384) | |
| 158 | + | - the [viewer asset path fix on GitHub Pages](https://github.com/mandiant/flare-floss/pull/1398) | |
| 159 | + | - a [downloadable offline viewer page](https://github.com/mandiant/flare-floss/pull/1399) | |
| 160 | + | - [serving the viewer from the standalone binary](https://github.com/mandiant/flare-floss/pull/1395) | |
| 156 | 161 | ||
| 157 | 162 | ## Reviews I contributed | |
| 158 | 163 | ||
| @@ -126,8 +126,10 @@ The module is `floss/cache.py`, with tests in `tests/test_cache.py`. | |||
| 126 | 126 | ### 6. The web viewer | |
| 127 | 127 | ||
| 128 | 128 | - [#1358](https://github.com/mandiant/flare-floss/pull/1358): reworked the QS web viewer to match the unified `ResultDocument` schema, added light and dark themes, fuzzy search, and virtualized rendering for very large string corpora. Deployed live for iteration during development. | |
| 129 | + | - [#1398](https://github.com/mandiant/flare-floss/pull/1398) (merged): fixed broken logo and favicon on GitHub Pages by switching to relative asset paths. | |
| 130 | + | - [#1399](https://github.com/mandiant/flare-floss/pull/1399) (open): added a download button that saves the viewer as one self-contained HTML file people can keep and open offline. | |
| 129 | 131 | ||
| 130 | - | The viewer lives in the `viewer/` directory and is now deployed at https://mandiant.github.io/flare-floss/ on every change. Serving it from the standalone binary is tracked as an [open issue](https://github.com/mandiant/flare-floss/issues/1371). | |
| 132 | + | The viewer lives in the `viewer/` directory and is now deployed at https://mandiant.github.io/flare-floss/ on every change. Serving it from the standalone binary is implemented in [#1395](https://github.com/mandiant/flare-floss/pull/1395), which adds a `--server` flag so the binary serves the bundled viewer and hands it the analysis results directly. | |
| 131 | 133 | ||
| 132 | 134 | ### 7. Reproducibility and CI hygiene (coding period) | |
| 133 | 135 | ||
| @@ -149,7 +151,8 @@ The coding period ran from June through August. This is how the work above lande | |||
| 149 | 151 | **August** | |
| 150 | 152 | - Early August: the [unified pipeline](https://github.com/mandiant/flare-floss/pull/1347) became the default output path. | |
| 151 | 153 | - Mid-August: the [new command line in two parts](https://github.com/mandiant/flare-floss/pull/1355) and the [render-time filters and summary output](https://github.com/mandiant/flare-floss/pull/1360). The [web viewer](https://github.com/mandiant/flare-floss/pull/1358) was reworked to match the new results schema. | |
| 152 | - | - Late August: [result caching](https://github.com/mandiant/flare-floss/pull/1367) and the [cleanup that deprecated the old `qs` naming](https://github.com/mandiant/flare-floss/pull/1370). The [final merge of QuantumStrand into master](https://github.com/mandiant/flare-floss/pull/1372) then completed the project. | |
| 154 | + | - Late August: [result caching](https://github.com/mandiant/flare-floss/pull/1367), the [cleanup that deprecated the old `qs` naming](https://github.com/mandiant/flare-floss/pull/1370), and the [final merge of QuantumStrand into master](https://github.com/mandiant/flare-floss/pull/1372), which completed the project. | |
| 155 | + | - Post-merge: the successful [OSS database rebuild](https://github.com/mandiant/flare-floss/actions/runs/32274349763) with its auto-opened update PR ([#1385](https://github.com/mandiant/flare-floss/pull/1385)), a [CI fix for the build workflow](https://github.com/mandiant/flare-floss/pull/1384), the [viewer asset path fix on GitHub Pages](https://github.com/mandiant/flare-floss/pull/1398), a [downloadable offline viewer page](https://github.com/mandiant/flare-floss/pull/1399), and [serving the viewer from the standalone binary](https://github.com/mandiant/flare-floss/pull/1395). | |
| 153 | 156 | ||
| 154 | 157 | ## Reviews I contributed | |
| 155 | 158 | ||
| @@ -164,7 +167,7 @@ Part of the work was reviewing others. The notable PRs I was asked to review: | |||
| 164 | 167 | ||
| 165 | 168 | What to do next, roughly in order: | |
| 166 | 169 | ||
| 167 | - | - **Finish the UI release.** The web viewer is already deployed at https://mandiant.github.io/flare-floss/ (deployed on every change via the web-release workflow). What remains is serving it from the standalone binary, tracked as an [open issue](https://github.com/mandiant/flare-floss/issues/1371). | |
| 170 | + | - **Finish the UI release.** The web viewer is deployed at https://mandiant.github.io/flare-floss/ on every change, and [#1399](https://github.com/mandiant/flare-floss/pull/1399) adds a downloadable offline page. Serving the viewer from the standalone binary is implemented in [#1395](https://github.com/mandiant/flare-floss/pull/1395), awaiting review and merge. | |
| 168 | 171 | - **Resolve the tracked product decisions.** [Viewer tag filtering](https://github.com/mandiant/flare-floss/issues/1377), [version bump and flag aliases](https://github.com/mandiant/flare-floss/issues/1378), and [default deobfuscation behavior](https://github.com/mandiant/flare-floss/issues/1379). | |
| 169 | 172 | - **Document the new interface.** The usage guide does not yet cover the new flags, the summary output, or the cache environment variables. | |
| 170 | 173 | - **Grow the cache.** A v1 tradeoff: no eviction or quota on the cache directory yet, and very large result documents load fully into memory. | |
| @@ -88,6 +88,8 @@ FLOSS tags strings by matching them against databases of known library strings. | |||
| 88 | 88 | - [#1384](https://github.com/mandiant/flare-floss/pull/1384): fixed the OSS DB build CI by putting the repo root on PYTHONPATH so the build script can import the `floss` package. | |
| 89 | 89 | - The library list was settled at around 60 open-source libraries, chosen by auditing false-positive rates against a global prevalence database built from 150,000 real-world samples. | |
| 90 | 90 | ||
| 91 | + | The pipeline has run successfully against the merged code: the bi-weekly rebuild completed cleanly ([workflow run](https://github.com/mandiant/flare-floss/actions/runs/32274349763)) and auto-opened the database update PR ([#1385](https://github.com/mandiant/flare-floss/pull/1385)). | |
| 92 | + | ||
| 91 | 93 | Building the databases surfaced several bugs in the upstream lancelot project, which the pipeline depends on. I filed reproductions and shipped fixes for all of them: | |
| 92 | 94 | ||
| 93 | 95 | - **Weak external symbols crash.** Two common C++ libraries (cryptopp and jsoncpp) triggered a crash on weak external symbols. I [filed the reproduction](https://github.com/williballenthin/lancelot/issues/235) and [shipped the fix](https://github.com/williballenthin/lancelot/pull/237), which Willi merged. | |
| @@ -162,7 +164,6 @@ Part of the work was reviewing others. The notable PRs I was asked to review: | |||
| 162 | 164 | ||
| 163 | 165 | What to do next, roughly in order: | |
| 164 | 166 | ||
| 165 | - | - **Rebuild and verify the OSS string databases.** QuantumStrand is in master now, so the databases get regenerated and validated against the merged code, with the [build workflow CI fix](https://github.com/mandiant/flare-floss/pull/1384) merged. | |
| 166 | 167 | - **Finish the UI release.** The web viewer is already deployed at https://mandiant.github.io/flare-floss/ (deployed on every change via the web-release workflow). What remains is serving it from the standalone binary, tracked as an [open issue](https://github.com/mandiant/flare-floss/issues/1371). | |
| 167 | 168 | - **Resolve the tracked product decisions.** [Viewer tag filtering](https://github.com/mandiant/flare-floss/issues/1377), [version bump and flag aliases](https://github.com/mandiant/flare-floss/issues/1378), and [default deobfuscation behavior](https://github.com/mandiant/flare-floss/issues/1379). | |
| 168 | 169 | - **Document the new interface.** The usage guide does not yet cover the new flags, the summary output, or the cache environment variables. | |
| @@ -18,15 +18,15 @@ | |||
| 18 | 18 | ||
| 19 | 19 | ## Table of contents | |
| 20 | 20 | ||
| 21 | - | - [Project overview](#project-overview) | |
| 22 | - | - [Project status](#project-status) | |
| 23 | - | - [Pre-GSoC work](#pre-gsoc-work) | |
| 24 | - | - [Coding-period deliverables](#coding-period-deliverables) | |
| 25 | - | - [Timeline of feature work](#timeline-of-feature-work) | |
| 26 | - | - [Reviews I contributed](#reviews-i-contributed) | |
| 27 | - | - [Future work](#future-work) | |
| 28 | - | - [Challenges](#challenges) | |
| 29 | - | - [Acknowledgements](#acknowledgements) | |
| 21 | + | 1. [Project overview](#project-overview) | |
| 22 | + | 2. [Project status](#project-status) | |
| 23 | + | 3. [Pre-GSoC work](#pre-gsoc-work) | |
| 24 | + | 4. [Coding-period deliverables](#coding-period-deliverables) | |
| 25 | + | 5. [Timeline of feature work](#timeline-of-feature-work) | |
| 26 | + | 6. [Reviews I contributed](#reviews-i-contributed) | |
| 27 | + | 7. [Future work](#future-work) | |
| 28 | + | 8. [Challenges](#challenges) | |
| 29 | + | 9. [Acknowledgements](#acknowledgements) | |
| 30 | 30 | ||
| 31 | 31 | ## Project overview | |
| 32 | 32 | ||