Skip to content

Incremental --update silently loses cross-file edges; --force does not repair it, only deleting graphify-out does #3776

Description

@BloodyBeard

Summary

A graph maintained with --update ends up missing edges that a clean build produces — predominantly cross-file imports / imports_from, plus some calls. --force does not repair it. Only deleting the graphify-out directory and rebuilding yields a graph equal to a clean build.

This is silent: nothing in the output indicates the graph is incomplete, and graph.json gives a consumer no way to tell.

Measurement

Project: ~130 Python files, 3,277 nodes. Comparison is against a full build of the same source copied to a clean directory, diffing on (source_file, label) pairs rather than node ids (ids are root-dependent, so an id diff is meaningless across roots).

Build method Node difference Edge difference
maintained with --update 0 65 missing
--force over that same directory 0 64 missing
graphify-out deleted, then rebuilt 0 0

Relation breakdown of the 65 missing edges:

imports        41
imports_from   19
calls           5

Every missing edge originates in a file that had been re-extracted by an
earlier --update. Examples:

reachability.py --imports--> _parser_for()          (source_spans.py)
reachability.py --imports--> function_spans()       (source_spans.py)
validation.py   --imports--> _parser_for()          (source_spans.py)
attacker_value_ranking.py --imports--> is_file_node()  (graph_data.py)

Expected

--update should produce a graph equivalent to a full build of the same source, or --force should repair one that has diverged.

Actual

Re-extracting a changed file appears not to re-resolve that file's edges into other files. The stale state survives --force and is only cleared by removing the output directory.

Why it matters

Any consumer that walks imports/imports_from (dependency analysis, supply-chain reach) undercounts, and anything walking calls can produce false negatives — a reachability query returning "no path" when a path exists in the source. Since the loss happens during the build, it leaves no trace in graph.json, so it cannot be detected by inspecting the graph alone.

Reproduction

  1. Build a graph for a multi-file project.
  2. Edit a file that imports from several others; run graphify <path> --update --code-only. Repeat a few times across different files.
  3. Copy the source to a clean directory and build it fresh.
  4. Diff the two graphs on (source_file, label) node keys and (source_key, target_key, relation) edge keys.

Environment

  • graphify 0.9.65 (PyPI graphifyy)
  • Python 3.14, Windows 10

Possibly related

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions