Skip to content

feat: Motorola S-Record and Intel HEX to binary conversion (#257)#633

Open
gradams42 wants to merge 3 commits into
llnl:mainfrom
gradams42:feat/srec-hex-conversion
Open

feat: Motorola S-Record and Intel HEX to binary conversion (#257)#633
gradams42 wants to merge 3 commits into
llnl:mainfrom
gradams42:feat/srec-hex-conversion

Conversation

@gradams42

Copy link
Copy Markdown
Contributor

Summary

Issue #257 (Add ability to convert Motorola S-Record and Intel Hex files into binary files) was being worked on in the upstream branch srec-and-hex-support. After reviewing the progress made, I ported the implementation into a new infoextractor plugin (srec_hex.py) and fixed two bugs present in that branch. I tried to keep as much of the current implementation as possible, in order to align with the current development thought processes.

Bug fixes:

  • ConfigManager at import timeMAX_FILE_SIZE, STRIP_LEADING_ZEROS, and EXTRACT_DIR were evaluated as module-level globals at import time, preventing test isolation and conflicting with issue Global settings can influence tests #470. Moved all three reads inside extract_file_info() so they reflect the current config at call time.

  • Incorrect gap-padding — padding between non-contiguous data records was written as ASCII '0' (b"0") instead of null bytes (b"\x00"), producing incorrect binary output.

The extractor respects three config options:

  • srec_hex.max_file_size — skip conversion if output would exceed this size (default: 1MB)
  • srec_hex.strip_leading_zeros — trim leading zero-padding before the first data address
  • srec_hex.output_path — directory to write converted binaries (default: system temp)

Testing

Ran the new tests directly to verify SREC and HEX decoding:

pytest tests/file_types/test_srec_hex_files.py -v  →  2 passed

Ran the full test suite to confirm no regressions:

pytest tests/ -v  →  179 passed, 4 skipped

Two bugs in resolve_link() caused a crash when processing software
packages that use the common soname symlink pattern (e.g.
libhelics.so -> libhelics.so.3 -> libhelics.so.3.5.3):

1. `while current_path.is_symlink:` referenced the bound method as an
   attribute rather than calling it. A method reference is always
   truthy, so the loop never terminated based on the symlink check and
   instead called readlink() on the final regular file, raising
   OSError: [Errno 22] Invalid argument.

2. `cur_dir = current_path.parent` assigned a Path object back to a
   variable typed and used as str. On the next loop iteration,
   `cur_dir[len(common_path):]` raised TypeError because Path does not
   support string subscripting.

Fix: call is_symlink() with parentheses and wrap current_path.parent
in str() so cur_dir stays a string throughout the loop.
@gradams42
gradams42 force-pushed the feat/srec-hex-conversion branch from f4febbc to b0ca07d Compare June 12, 2026 03:27
I saw that issue llnl#257 (Add ability to convert Motorola S-Record and
Intel Hex files into binary files) was being worked on in the upstream
branch srec-and-hex-support. After looking at the progress made thus
far, I ported the implementation into a new infoextractor plugin
(srec_hex.py) and fixed two bugs present in the upstream branch:

- ConfigManager reads were evaluated at module import time as global
  constants, preventing test isolation and conflicting with issue llnl#470.
  Moved all three config reads inside extract_file_info() so they
  reflect the current config at call time.

- Gap-padding between data records was written as ASCII '0' (b"0")
  instead of null bytes (b"\x00"), producing incorrect binary output
  for any file with non-contiguous address ranges.

The extractor supports three config options:
  srec_hex.max_file_size       - skip if converted output exceeds this (default 1MB)
  srec_hex.strip_leading_zeros - trim leading padding before first data address
  srec_hex.output_path         - directory for converted binaries (default: system temp)

Testing:
  Ran the new tests directly to verify SREC and HEX decoding:
    pytest tests/file_types/test_srec_hex_files.py -v  →  2 passed
  Ran the full test suite to confirm no regressions:
    pytest tests/ -v  →  179 passed, 4 skipped
@gradams42
gradams42 force-pushed the feat/srec-hex-conversion branch from 1dc64da to e2ef995 Compare June 12, 2026 03:30
@nightlark nightlark added the enhancement New feature or request label Jun 12, 2026
@github-actions

Copy link
Copy Markdown

🆕 New Folders (1)

  • binary

🧪 SBOM Results (1/17)

binary (Link)

---
+++
@@ -0,0 +1,256 @@
+{
+  "authors": null,
+  "bomDescription": "",
+  "bomFormat": "cytrics",
+  "bomUUID": "52f1e504-b79c-4e6d-b0d4-397882a55ee6",
+  "hardware": [],
+  "relationships": [
+    {
+      "relationship": "Contains",
+      "xUUID": "ff94c3a0-6237-42fd-b041-20f7389b1077",
+      "yUUID": "b91c1fab-4e51-4c68-8014-10b9b6d36fec"
+    }
+  ],
+  "software": [
+    {
+      "UUID": "39b66a99-f18e-40ee-b76d-146d4643dde4",
+      "captureTime": "2021-01-01T00:00:00Z",
+      "comments": [],
+      "containerPath": [],
+      "description": "",
+      "fileName": [
+        "test.srec"
+      ],
+      "installPath": [
+        "/home/runner/work/Surfactant/Surfactant/tests/data/binary/test.srec"
+      ],
+      "md5": "8d945b70c74ae0bc0f9a87bc562fffba",
+      "metadata": [
+        {
+          "collectedBy": "Surfactant",
+          "collectionPlatform": "Linux-6.17.0-1018-azure-x86_64-with-glibc2.39",
+          "fileInfo": {
+            "hidden": false,
+            "mode": "-rw-r--r--"
+          }
+        }
+      ],
+      "name": null,
+      "relationshipAssertion": "Unknown",
+      "sha1": "5c6d627e2e4d881700af787171746eb1e4ae451f",
+      "sha256": "f85844d0cd99f80afdec5b839f189bf3b81c8d9229a918411ae180aac5406408",
+      "size": 28884,
+      "softwareType": null,
+      "supplementaryFiles": [],
+      "vendor": [],
+      "version": ""
+    },
+    {
+      "UUID": "ef7fb89c-8195-4c60-a2a3-43baff01bbc5",
+      "captureTime": "2021-01-01T00:00:00Z",
+      "comments": [],
+      "containerPath": [],
+      "description": "",
+      "fileName": [
+        "test.hex"
+      ],
+      "installPath": [
+        "/home/runner/work/Surfactant/Surfactant/tests/data/binary/test.hex"
+      ],
+      "md5": "792ce4a8260e6cccf2b34c70490545d3",
+      "metadata": [
+        {
+          "collectedBy": "Surfactant",
+          "collectionPlatform": "Linux-6.17.0-1018-azure-x86_64-with-glibc2.39",
+          "fileInfo": {
+            "hidden": false,
+            "mode": "-rw-r--r--"
+          }
+        }
+      ],
+      "name": null,
+      "relationshipAssertion": "Unknown",
+      "sha1": "415c68ff7c5cba1f11c8ac1804158724e0200fe1",
+      "sha256": "34e9127f38ea35cbda9405d25a1a3e85bca1f9138b3659ad2ef9d7c8412f2b24",
+      "size": 29212,
+      "softwareType": null,
+      "supplementaryFiles": [],
+      "vendor": [],
+      "version": ""
+    },
+    {
+      "UUID": "ff94c3a0-6237-42fd-b041-20f7389b1077",
+      "captureTime": "2021-01-01T00:00:00Z",
+      "comments": [
+        {
+          "author": null,
+          "comment": "This is a test file to create an msi.",
+          "fieldName": null,
+          "timestamp": null
+        }
+      ],
+      "containerPath": [],
+      "description": "",
+      "fileName": [
+        "test.msi"
+      ],
+      "installPath": [
... and 159 more lines

For commit fa84b89 (Run 27392631969)
Compared against commit 1652a5c (Run 27367245593)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants