Major refactoring by Speierers · Pull Request #121 · mitsuba-renderer/mitsuba-blender

Speierers · 2025-01-27T08:59:05Z

Major refactoring

This pull request introduces a significant refactoring of the Blender add-on, bringing substantial improvements in performance, usability, and features. The changes are designed to enhance the overall user experience, making it more efficient and enjoyable to work with Mitsuba in Blender.

This new version of the add-on relies on some changes to the Mitsuba changes, introduced in this PR

Overview video

mitsuba_blender_example.mov

Key Enhancements

Codebase Restructuring: The codebase has been completely reorganized for easier maintenance, better separation, readability, and scalability.
Performance Improvements: Significant enhancements have been made to the Blender -> Mitsuba scene conversion process, including:
- In-memory texture and mesh loading
- Caching mechanism for converted textures
- Efficient C++ implementation for writing to Blender's framebuffer
Interactive Viewport and Scene Updates: Users can now add and edit materials, geometry, lights, and directly visualize the results using Mitsuba.
Mitsuba Plugin Support: The add-on now ships with many Mitsuba plugins, including support for:
- Mix, math, separate_rgb, hue_saturation, map_range, clamp, rgb_to_bw, combine_color, and noise texture shader nodes
Texture UDIM support
Better Logging: Improved logging mechanisms provide more detailed information and easier debugging.
And more!

💥🔪 The sharp bits 🔪💥

Test suite wasn't updated
Viewport rendering doesn't work on MacOS. It segfaults when trying to run the Mitsuba LLVM within Blender. It seems that Blender ships with its own version of LLVM, which might cause some conflicts here. I did try to build Mitsuba so that it uses the Blender LLVM libraries, but this still didn't fix the crashes.
I just realised that the "pip install mitsuba" feature of the add-on preference got lost during the refactoring. We might wanna add this back once the Mitsuba PR has shipped.

wjakob · 2025-01-27T12:42:29Z

Hi @Speierers,

absolutely spectacular! I'm not sure to what extent we can really review this as a normal PR. Still, it would be great to have @bathal1's thoughts.

One thing that concerns me is the full loss of the testing harness. Experience has shown that projects without one end up unhealthy. Ideally we would have at least basic coverage to ensure that nothing breaks catastrophically. Is that something you could still add?

PS: I am very curious how UDIMs in Mitsuba can work 😆

Speierers · 2025-02-10T08:13:23Z

One thing that concerns me is the full loss of the testing harness. Experience has shown that projects without one end up unhealthy. Ideally we would have at least basic coverage to ensure that nothing breaks catastrophically. Is that something you could still add?

Wouldn't it be enough to revive the previous test suite for this refactored codebase? The testing infrastructure is still here, it is just a matter to getting it to work again.

wjakob · 2025-02-10T08:57:06Z

I would like to at least test rendering of some basic scenes to avoid catastrophic breakage. I am actually not sure what the previous test suite covered (@bathal1 , do you know?)

dvicini · 2025-05-12T09:00:02Z

What's the status on this? I am increasingly interested in improvements to the blender exporter and was wondering if we are expecting to merge a version of this? What is missing?

rtabbara · 2025-05-12T09:04:25Z

Hi @dvicini ,

I've been looking into this and will be close to updating the PR with some more (minor) changes/additions soon.

bathal1

Thank you very much for the PR @Speierers, this is very cool work !

Here is my review, you will find several comments/questions, it would be great if you could clarify them.

My main concern with this PR is that it seems to heavily focus on rendering inside blender, at the expense of exporting. In particular, having python implementation of various Cycles nodes as Mitsuba plugins is great to render blender native scenes, but I am concerned that this will be misleading for users who want to export a scene to XML, where some materials may be exported without raising errors, and later fail when loading the scene in Mitsuba as some plugins aren't implemented. Some functionality should be added to make sure this doesn't happen, as exporting to XML for later use is still AFAIK the main use case for this addon.

A more minor comment: I did not see any remapping of roughness in the new material export code, was that somehow fixed? Blender usually squares the roughness value that the user inputs so some remapping is necessary to make renderings with Mitsuba match Blender.

To merge this PR, we would also need to:

Add the test suite back, which includes round-trip tests to make sure things are consistent
Add support for installing mitsuba with pip, as this is the preferred way
Merge the correesponding PR on the Mitsuba 3 repo.

bathal1 · 2025-05-26T15:19:53Z

+    if radius > 1e-3:
+        return {
+            'type': 'spherelight',
+            'position': position,
+            'radius': radius,
+            'intensity': intensity,
+            'soft_falloff': b_light.data.use_soft_falloff
+        }


This seems dangerous. IIUC it will only work when rendering inside blender, as the corresponding mitsuba plugin will be registered. However, once the scene is exported to XML for downstream applications, this will result in an error as the spherelight plugin does not exist in Mitsuba.

bathal1 · 2025-05-26T15:59:11Z

+    # for color_layer in b_mesh.vertex_colors:
+    #     if color_layer.name in b_mesh.attributes:
+    #         props[f'vertex_{color_layer.name}'] = b_mesh.attributes[color_layer.name].data[0].as_pointer()
+    #     else:
+    #         props[f'vertex_{color_layer.name}'] = color_layer.data[0].as_pointer()


Why was this commented ?

bathal1 · 2025-05-26T16:08:46Z

+            mesh_dict = convert_mesh(ctx, b_mesh, transform, name_clean, 0)
+            converted_parts.append((-1, mesh_dict))


There used to be a sanity check that the mesh was converted correctly and non-empty, why remove it ?

bathal1 · 2025-05-26T16:10:48Z

+                    if displacement:
+                        mesh_dict['displacement'] = displacement
+
+                if mesh_dict is not None:


The mesh dict could also contain an empty mesh, which we shouldn't export

bathal1 · 2025-05-27T06:32:47Z

+                entry = mi.Bitmap(image.filepath_raw)
+        else:
+            key = 'filename'
+            # TODO: don't save packed images but convert them to a mitsuba texture, and let the XML writer save


This TODO is tken care of by the code above, right ?

bathal1 · 2025-05-27T09:25:32Z

+def find_mitsuba():
+    '''
+    Find Mitsuba path by importing Mitsuba on the system Python (python or python3)
+    '''
+    def f(executable):
+        import subprocess
+        cmd = [executable, '-c', 'import mitsuba; print(mitsuba.__path__[0])']
+        return subprocess.run(cmd, capture_output=True, text=True).stdout[:-5]
+    try:
+        return f('python')
+    except:
+        try:
+            return f('python3')
+        except:
+            return ''


Very nice !

bathal1 · 2025-05-27T09:48:41Z

-            uvindices = (el.index(b's'), el.index(b't'))
-            if -1 in uvindices:
-                uvindices = (el.index(b'u'), el.index(b'v'))
+    if True:


This should probably be removed

bathal1 · 2025-05-27T10:02:31Z

+    if mi_texture.has_property('gain'):
+        bl_mixnode = bl_mat_wrap.ensure_node_type([out_socket_id], 'ShaderNodeMixRGB', 'Color')
+        bl_mixnode.blend_type = 'MULTIPLY'
+        bl_mixnode.inputs[0].default_value = 1.0 # Factor
+        bl_mixnode.inputs[2].default_value = bl_shader_utils.rgb_to_rgba([mi_texture.get('gain')] * 3)
+        bl_teximage = bl_mat_wrap.tree.nodes.new(type='ShaderNodeTexImage')
+        bl_mat_wrap.tree.links.new(bl_teximage.outputs[0], bl_mixnode.inputs[1])
+    else:
+        bl_teximage = bl_mat_wrap.ensure_node_type([out_socket_id], 'ShaderNodeTexImage', 'Color')
+


Mitsuba textures don't have a gain property. Maybe this was meant to be deleted ?

bathal1 · 2025-05-27T12:24:48Z

+def mi_area_to_bl_light(mi_context, mi_emitter, mi_props_id):
+    bl_light = bpy.data.lights.new(name=mi_props_id, type='AREA')
+    color, strength = _get_radiance_value(mi_context, mi_emitter, 'radiance', [1.0, 1.0, 1.0])
+    bl_light.color = color
+    bl_light.energy = strength
+    world_matrix = Matrix()
+    return bl_light, mi_context.mi_space_to_bl_space(world_matrix)


This should probably be merged with the function below, it doesn't seem to be used anywhere else

bathal1 · 2025-05-27T12:26:43Z

Why is this not directly defined in the MitsubaRenderEngineclass ?

bathal1 · 2025-05-28T18:52:09Z

I added the default pip install option back. @Speierers I'm interested to hear if you tested the reload of mitsuba when changing the custom path. In my experience it is a huge pain to point to a new build folder, and I couldn't get it to work today. I reverted to requiring the user to restart blender whenever such a change is made.

In general, Blender 3.6 LTS will stop being supported June 2025 so clean-up export/import to focus on v4.0+ * Remove curves export as uses currently uses unsupported shape plugin * Add translucent BSDF export support * Clean-up export principled BSDF * Clean-up export of normal/bump BSDF * Added brightness/contrast texture plugin * Added invert_rgb texture plugin * Added mix_rgb texture plugin * Added color_ramp texture plugin

- Added a checkbox to toggle between pip version and custom path - Require a restart whenever the mitsuba path is changed - Added messages for common errors

wjakob · 2026-02-23T15:34:37Z

Hi @bathal1 and @Speierers,

Leander Cretigny will work on the blender plugin this semester. To me, the most important thing seems to be to merge this PR and add proper support for testing. Do you agree with this assessment? Is there any feedback or suggestions you would have for him?

dvicini · 2026-02-23T17:32:17Z

Baptiste and I had this WIP PR that updates the plugin to the latest Mitsuba version and were blocked due to some Windows crashes that we didn't have time to investigate: #137

The PR also improves the testing setup and instructions a bit. Maybe it would be nice to first do an incremental update (as done in PR #137), and then adapt all these larger changes?

bathal1 requested changes May 27, 2025

View reviewed changes

Speierers and others added 6 commits June 27, 2025 14:30

Major refactoring

5caa17e

Fixed inconsistent package names

273846d

Added pip installation of Mitsuba back:

6809223

- Added a checkbox to toggle between pip version and custom path - Require a restart whenever the mitsuba path is changed - Added messages for common errors

Removed empty io folder

8be75d7

Use scalar_rgb variant for I/O tasks

ce07ab0

bathal1 force-pushed the major_refactoring branch from 6c53584 to ce07ab0 Compare June 27, 2025 12:47

bathal1 added 3 commits June 27, 2025 15:01

Updated CI Mitsuba version logic

c28ebe0

Temporarily disabled CI matrix

4e2f970

Allow all variants to be chosen

9d824c9

bathal1 force-pushed the major_refactoring branch from e8dff3e to 7d0451a Compare June 27, 2025 13:32

bathal1 added 2 commits June 27, 2025 15:52

Try to set a parallel variant on init

ffb4403

Updated mitsuba pref init flag

a5e7a60

bathal1 force-pushed the major_refactoring branch from 80748f6 to a5e7a60 Compare June 27, 2025 13:53

Fixed sensor test

a180421

bathal1 force-pushed the major_refactoring branch from 8a61844 to a180421 Compare June 27, 2025 14:08

		mesh_dict = convert_mesh(ctx, b_mesh, transform, name_clean, 0)
		converted_parts.append((-1, mesh_dict))

Uh oh!

Conversation

Speierers commented Jan 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Major refactoring

Overview video

Key Enhancements

💥🔪 The sharp bits 🔪💥

Uh oh!

wjakob commented Jan 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Speierers commented Feb 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wjakob commented Feb 10, 2025

Uh oh!

dvicini commented May 12, 2025

Uh oh!

rtabbara commented May 12, 2025

Uh oh!

bathal1 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bathal1 commented May 28, 2025

Uh oh!

wjakob commented Feb 23, 2026

Uh oh!

dvicini commented Feb 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Speierers commented Jan 27, 2025 •

edited

Loading

wjakob commented Jan 27, 2025 •

edited

Loading

Speierers commented Feb 10, 2025 •

edited

Loading

dvicini commented Feb 23, 2026 •

edited

Loading