Repository navigation
Conversation
Replace the example's own PTv3 implementation (ptv3_fvdb.py and fvdb_utils.py) with fvdb.nn.PointTransformerV3 from fvdb-core (openvdb/fvdb-core#808, #809). The PT-v3fvdb Pointcept backbone becomes a thin wrapper that builds a GridBatch from Pointcept's batch, runs the core model, and maps voxel features back to points. The backbone takes the core model's arguments, which follow Pointcept's PT-v3m1, so the training config uses Pointcept's names and shallow-to-deep decoder lists. The config serializes the raw grid_coord (serialization_offset="none"), as Pointcept does. The adapter imports Pointcept by its installed name, so it registers into the registry Pointcept's training uses. minimal_inference.py builds its model configurations with the core model. The flash-attn requirement is now optional, and only needed to run Pointcept's own model for comparison. Remove test_order_type_validation.py, which tested the removed serialization dispatch, and add tests for the new backbone. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the example's own Point Transformer V3 implementation with
fvdb.nn.PointTransformerV3from fvdb-core (openvdb/fvdb-core#808, openvdb/fvdb-core#809). ThePT-v3fvdbPointcept backbone becomes a thin adapter that builds aGridBatchfrom Pointcept's batch, runs the core model, and maps voxel features back to points.Draft until openvdb/fvdb-core#809 merges and ships in a release. Until then this needs an fvdb-core build from that branch and PyTorch >= 2.11 for patch attention.
Part of the work tracked in #66.
What changes
fvdb_extensions/models/ptv3_fvdb.pyandfvdb_utils.pyare removed. They stored serialization orders as attributes onGridBatch, which current fvdb no longer allows, rebuilt the serialization and the convolution plan in every block, and requiredflash_attn.point_transformer_v3m1_fvdb.pywraps the core model. Its arguments are the core model's, which follow Pointcept'sPT-v3m1, so decoder lists run shallow to deep and a Pointcept config switches models by changingtype. It imports Pointcept by its installed name so it registers into the registry Pointcept's training uses.semseg-pt-v3m1-0-fvdb-test.pyuses the Pointcept names and setsserialization_offset="none"so the curve orders are computed on the rawgrid_coord, as Pointcept does. The comparison settings stay the same (linear stem, batch norm, convolution in the position encoding). A checkpoint trained with Pointcept also needspointcept_unpool_cpe=True; see the PTv3: fvdb model diverges from official Pointcept PTv3 when the CPE convolution is enabled #66 findings.scripts/test/minimal_inference.pybuilds its model modes with the core model.requirements.txtdropsflash_attnas a requirement. It is only needed to run Pointcept's own model for comparison.test_order_type_validation.pytested the removed dispatch and is gone.test_fvdb_backbone.pycovers point-to-voxel mapping, forward and backward, the Pointcept options, and thecls_moderejection.Net change is about 210 lines added and 1,480 removed.
Test plan
Run from
point_transformer_v3/with an fvdb-core build of openvdb/fvdb-core#809 on the path:14 tests pass locally (RTX PRO 6000 Blackwell, PyTorch 2.13). I also built the backbone from the training config through
MODELS.buildand ran a forward pass; it has 46,142,496 parameters and returns finite per-point features.Before taking this out of draft I'll rerun the
#12harness comparison against official PT-v3m1 with this config, as in the #66 comment.🤖 Generated with Claude Code