Skip to content

BCI Application - moving exponential average and alternate visualization #1380

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
gabelerner-kernel wants to merge 13 commits into
base: main
Choose a base branch
from
Open

BCI Application - moving exponential average and alternate visualization #1380

Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
fcc2cfa
bci application nilearn viz
gabelerner-kernel Jan 26, 2026
89540b3
make own surf
gabelerner-kernel Jan 26, 2026
c5d03a9
switch to mpl toolkit and faces
gabelerner-kernel Jan 26, 2026
ab0c1be
only flush
gabelerner-kernel Jan 26, 2026
eaa7c33
simpler mesh
gabelerner-kernel Jan 27, 2026
be3aa7b
adjust by min idx
gabelerner-kernel Jan 27, 2026
9642449
add exponential moving average high pass
gabelerner-kernel Jan 28, 2026
bf68818
different viz modes
gabelerner-kernel Jan 28, 2026
a462d0e
conditional import of operator
gabelerner-kernel Jan 28, 2026
46036f6
missing flow link
gabelerner-kernel Jan 28, 2026
f8e8261
Merge branch 'main' into feature/nilearn-viz
bhashemian Jan 29, 2026
7d81690
nicer viz
gabelerner-kernel Feb 4, 2026
cbe5351
Merge branch 'main' into feature/nilearn-viz
bhashemian Feb 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion applications/bci_visualization/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,16 @@ ARG DEBIAN_FRONTEND=noninteractive
# Install curl for downloading data files
RUN apt-get update && apt-get install -y curl && rm -rf /var/lib/apt/lists/*

# Install python3-tk (needed for matplotlib GUI support)
ARG WITH_MATPLOTLIB=no
RUN if [ "$WITH_MATPLOTLIB" = "yes" ]; then apt-get update && apt-get install -y python3-tk && rm -rf /var/lib/apt/lists/*; fi

Comment on lines +27 to +30

@bhashemian bhashemian Jan 29, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great to move this to the end to have all matplotlib related dependencies in one place. In this way, both images, with and without matplotlib, can share the initial cached docker layers. Thanks

ENV HOLOSCAN_INPUT_PATH=/workspace/holohub/data/bci_visualization

# Install Python dependencies
COPY applications/bci_visualization/requirements.txt /tmp/requirements.txt
RUN pip install -r /tmp/requirements.txt --no-cache-dir


# Install optional matplotlib dependencies
COPY applications/bci_visualization/requirements-matplotlib.txt /tmp/requirements-matplotlib.txt
RUN if [ "$WITH_MATPLOTLIB" = "yes" ]; then pip install -r /tmp/requirements-matplotlib.txt --no-cache-dir; fi
Comment thread
bhashemian marked this conversation as resolved.

@bhashemian bhashemian Jan 29, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
RUN if [ "$WITH_MATPLOTLIB" = "yes" ]; then pip install -r /tmp/requirements-matplotlib.txt --no-cache-dir; fi
RUN if [ "$WITH_MATPLOTLIB" = "yes" ]; then pip install -r /tmp/requirements-matplotlib.txt --no-cache-dir; fi

172 changes: 100 additions & 72 deletions applications/bci_visualization/bci_visualization.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,10 +49,12 @@ def __init__(
coefficients_path: Path | str,
mask_path=None,
reg: float = RegularizedSolverOperator.REG_DEFAULT,
viz: str = "clara_viz",
**kwargs,
):
self._rendering_config = render_config_file
self._mask_path = mask_path
self._viz = viz

# Reconstruction pipeline parameters
self._stream = stream
Expand Down Expand Up @@ -109,40 +111,44 @@ def compose(self):
xyz=pipeline_assets.xyz,
)

# ========== Visualization Pipeline Operators ==========
# Get volume_renderer kwargs from YAML config to extract density range
volume_renderer_kwargs = self.kwargs("volume_renderer")
density_min = volume_renderer_kwargs.get("density_min", -100.0)
density_max = volume_renderer_kwargs.get("density_max", 100.0)

voxel_to_volume = VoxelStreamToVolumeOp(
self,
name="voxel_to_volume",
pool=volume_allocator,
mask_nifti_path=self._mask_path,
density_min=density_min,
density_max=density_max,
**self.kwargs("voxel_stream_to_volume"),
)

volume_renderer = VolumeRendererOp(
self,
name="volume_renderer",
config_file=self._rendering_config,
allocator=volume_allocator,
cuda_stream_pool=cuda_stream_pool,
**volume_renderer_kwargs,
)

# IMPORTANT changes to avoid deadlocks of volume_renderer and holoviz
# when running in multi-threading mode
# 1. Set the output port condition to NONE to remove backpressure
volume_renderer.spec.outputs["color_buffer_out"].condition(ConditionType.NONE)
# 2. Use a passthrough operator to configure queue policy as POP to use latest frame
color_buffer_passthrough = ColorBufferPassthroughOp(
self,
name="color_buffer_passthrough",
)
if self._viz == "matplotlib":
from operators.mpl_visualization_operator import MplVisualizationOperator
mpl_visualization_operator = MplVisualizationOperator(fragment=self)
else:
# ========== Visualization Pipeline Operators ==========
# Get volume_renderer kwargs from YAML config to extract density range
volume_renderer_kwargs = self.kwargs("volume_renderer")
density_min = volume_renderer_kwargs.get("density_min", -100.0)
density_max = volume_renderer_kwargs.get("density_max", 100.0)

voxel_to_volume = VoxelStreamToVolumeOp(
self,
name="voxel_to_volume",
pool=volume_allocator,
mask_nifti_path=self._mask_path,
density_min=density_min,
density_max=density_max,
**self.kwargs("voxel_stream_to_volume"),
)

volume_renderer = VolumeRendererOp(
self,
name="volume_renderer",
config_file=self._rendering_config,
allocator=volume_allocator,
cuda_stream_pool=cuda_stream_pool,
**volume_renderer_kwargs,
)

# IMPORTANT changes to avoid deadlocks of volume_renderer and holoviz
# when running in multi-threading mode
# 1. Set the output port condition to NONE to remove backpressure
volume_renderer.spec.outputs["color_buffer_out"].condition(ConditionType.NONE)
# 2. Use a passthrough operator to configure queue policy as POP to use latest frame
color_buffer_passthrough = ColorBufferPassthroughOp(
self,
name="color_buffer_passthrough",
)

holoviz = HolovizOp(
self,
Expand Down Expand Up @@ -181,45 +187,56 @@ def compose(self):
("result", "result"),
},
)
self.add_flow(
convert_to_voxels_operator,
voxel_to_volume,
{
("affine_4x4", "affine_4x4"),
("hb_voxel_data", "hb_voxel_data"),
},
)

# ========== Connect Visualization Pipeline ==========
# voxel_to_volume → volume_renderer
self.add_flow(
voxel_to_volume,
volume_renderer,
{
("volume", "density_volume"),
("spacing", "density_spacing"),
("permute_axis", "density_permute_axis"),
("flip_axes", "density_flip_axes"),
},
)
# Add mask connections to VolumeRendererOp
self.add_flow(
voxel_to_volume,
volume_renderer,
{
("mask_volume", "mask_volume"),
("mask_spacing", "mask_spacing"),
("mask_permute_axis", "mask_permute_axis"),
("mask_flip_axes", "mask_flip_axes"),
},
)

# volume_renderer ↔ holoviz
self.add_flow(
volume_renderer, color_buffer_passthrough, {("color_buffer_out", "color_buffer_in")}
)
self.add_flow(color_buffer_passthrough, holoviz, {("color_buffer_out", "receivers")})
self.add_flow(holoviz, volume_renderer, {("camera_pose_output", "camera_pose")})
if self._viz == "matplotlib":
self.add_flow(
convert_to_voxels_operator,
mpl_visualization_operator,
{
("affine_4x4", "affine_4x4"),
("hb_voxel_data", "hb_voxel_data"),
},
)
else:
self.add_flow(
convert_to_voxels_operator,
voxel_to_volume,
{
("affine_4x4", "affine_4x4"),
("hb_voxel_data", "hb_voxel_data"),
},
)

# ========== Connect Visualization Pipeline ==========
# voxel_to_volume → volume_renderer
self.add_flow(
voxel_to_volume,
volume_renderer,
{
("volume", "density_volume"),
("spacing", "density_spacing"),
("permute_axis", "density_permute_axis"),
("flip_axes", "density_flip_axes"),
},
)
# Add mask connections to VolumeRendererOp
self.add_flow(
voxel_to_volume,
volume_renderer,
{
("mask_volume", "mask_volume"),
("mask_spacing", "mask_spacing"),
("mask_permute_axis", "mask_permute_axis"),
("mask_flip_axes", "mask_flip_axes"),
},
)

# volume_renderer ↔ holoviz
self.add_flow(
volume_renderer, color_buffer_passthrough, {("color_buffer_out", "color_buffer_in")}
)
self.add_flow(color_buffer_passthrough, holoviz, {("color_buffer_out", "receivers")})
self.add_flow(holoviz, volume_renderer, {("camera_pose_output", "camera_pose")})


def main():
Expand All @@ -244,6 +261,16 @@ def main():
help="Path to the mask NIfTI file containing 3D integer labels (I, J, K). Optional.",
)

parser.add_argument(
"-v",
"--viz",
action="store",
type=str,
dest="viz",
default="clara_viz",
help="Visualization backend to use (e.g., 'clara_viz', 'matplotlib'). Optional.",
)

parser.add_argument(
"-h", "--help", action="help", default=argparse.SUPPRESS, help="Help message"
)
Expand All @@ -269,6 +296,7 @@ def main():
coefficients_path=kernel_data / "extinction_coefficients_mua.csv",
mask_path=args.mask_path,
reg=RegularizedSolverOperator.REG_DEFAULT,
viz=args.viz,
)

# Load YAML configuration
Expand Down
23 changes: 20 additions & 3 deletions applications/bci_visualization/metadata.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,9 +69,26 @@
}
]
},
"run": {
"command": "python3 bci_visualization.py --renderer_config <holohub_app_source>/config.json --mask_path <holohub_data_dir>/bci_visualization/anatomy_labels_high_res.nii.gz",
"workdir": "holohub_app_source"
"default_mode": "clara_viz",
"modes": {
"clara_viz": {
"description": "Run the application with ClaraViz for volume rendering.",
"run": {
"command": "python3 bci_visualization.py --renderer_config <holohub_app_source>/config.json --mask_path <holohub_data_dir>/bci_visualization/anatomy_labels_high_res.nii.gz",
"workdir": "holohub_app_source"
}
},
"matplotlib": {
"description": "Run the application with matplotlib for volume rendering.",
"build": {
"docker_build_args": ["--build-arg", "WITH_MATPLOTLIB=yes"]
},
"run": {
"command": "python3 bci_visualization.py --renderer_config <holohub_app_source>/config.json --mask_path <holohub_data_dir>/bci_visualization/anatomy_labels_high_res.nii.gz --viz matplotlib",
"workdir": "holohub_app_source"
}
}
Comment on lines +81 to +90

@coderabbitai coderabbitai Bot Jan 28, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Description is inaccurate - matplotlib doesn't perform volume rendering.

The description states "matplotlib for volume rendering" but matplotlib is a 2D plotting library. Based on the PR summary mentioning "surface mesh rendering", consider updating the description to accurately reflect the visualization approach.

📝 Suggested fix 
 			"matplotlib": {
-				"description": "Run the application with matplotlib for volume rendering.",
+				"description": "Run the application with matplotlib for surface mesh visualization.",
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
"matplotlib": {
"description": "Run the application with matplotlib for volume rendering.",
"build": {
"docker_build_args": ["--build-arg", "WITH_MATPLOTLIB=yes"]
},
"run": {
"command": "python3 bci_visualization.py --renderer_config <holohub_app_source>/config.json --mask_path <holohub_data_dir>/bci_visualization/anatomy_labels_high_res.nii.gz --viz matplotlib",
"workdir": "holohub_app_source"
}
}
"matplotlib": {
"description": "Run the application with matplotlib for surface mesh visualization.",
"build": {
"docker_build_args": ["--build-arg", "WITH_MATPLOTLIB=yes"]
},
"run": {
"command": "python3 bci_visualization.py --renderer_config <holohub_app_source>/config.json --mask_path <holohub_data_dir>/bci_visualization/anatomy_labels_high_res.nii.gz --viz matplotlib",
"workdir": "holohub_app_source"
}
}
🤖 Prompt for AI Agents 
In `@applications/bci_visualization/metadata.json` around lines 81 - 90, The
"matplotlib" entry's description incorrectly claims "volume rendering"; update
the description string in metadata.json under the "matplotlib" object to
accurately state that matplotlib is used for 2D plotting (or surface mesh
visualization if that's the intended behavior) rather than volume
rendering—locate the "matplotlib" key and edit the "description" value
accordingly so it reflects 2D/surface mesh visualization instead of volume
rendering.

}

@greptile-apps greptile-apps Bot Jan 28, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trailing whitespace before closing brace

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

}
}
Loading
Loading