Support custom recursion limits at build time by seanlinsley · Pull Request #785 · tokio-rs/prost

seanlinsley · 2022-12-15T22:22:42Z

This PR allows users to set a custom recursion limit for specific protobuf structures at build time:

let mut config = prost_build::Config::new();
config.recursion_limit("my_messages.MyMessageType", 1000);

seanlinsley · 2022-12-16T02:43:05Z

thoughts @LucioFranco @nrc @danburkert?

seanlinsley · 2023-01-03T18:14:01Z

-// See `encoding::DecodeContext` for more info.
-// 100 is the default recursion limit in the C++ implementation.
-#[cfg(not(feature = "no-recursion-limit"))]
-const RECURSION_LIMIT: u32 = 100;


Since it sounds like this will never change, I opted to inline it everywhere it's used. That allows documentation to say explicitly that the default recursion limit is 100 instead of requiring that users look up this constant.

I actually kinda like having this as a constant. I wonder if we could just make a constants module that contains just this one. And we can then have all the deep dive docs on the recursion implementation there and then we just need to link to there from the lib doc page. I feel like that would make it easier to maintain down the line and centralize it a bit.

🤷 I'm not sure if a constants module adds much. If you still feel a constant is necessary, I'd lean towards simply adding back the constant here in lib.rs

I like it to be a constant as well. It makes it explicit in the code. But I do like it to be in the documentation as well.

seanlinsley · 2023-01-11T18:02:32Z

Here's a downstream PR using this feature: pganalyze/pg_query.rs#17

Note that to prevent a stack overflow from the recursion, I had to increase the stack size in a separate thread. I wonder if that should be documented here.

LucioFranco

Overall LGTM! Some small suggestions and we can get this merged. Thanks for the patience I went on vacation in between :)

seanlinsley · 2023-02-27T15:10:06Z

@LucioFranco this should be ready for re-review

seanlinsley · 2023-04-04T15:04:28Z

@LucioFranco ping. happy to make any other changes if needed

LucioFranco

Sorry for the super long delay on the review here and thank you for the ping (I generally don't mind them if you're waiting for a review). I left a few comments that I would like to see changed but they are pretty minor and I think once those are good we are good to merge. Thank you for pushing through on this!

LucioFranco · 2023-04-05T20:24:15Z

-// See `encoding::DecodeContext` for more info.
-// 100 is the default recursion limit in the C++ implementation.
-#[cfg(not(feature = "no-recursion-limit"))]
-const RECURSION_LIMIT: u32 = 100;


I actually kinda like having this as a constant. I wonder if we could just make a constants module that contains just this one. And we can then have all the deep dive docs on the recursion implementation there and then we just need to link to there from the lib doc page. I feel like that would make it easier to maintain down the line and centralize it a bit.

caspermeijn

Thank you for coming back to this after two years. I see why this is useful and this is a smart solution.

I am a bit concerned whether this could be a breaking change. I would like to see some prove that it is not.

caspermeijn · 2025-06-10T06:34:23Z

                #(#clear;)*
            }
+
+            fn recursion_limit() -> u32 {


Is this a breaking change for messages with a field named recursion_limit?

I think for enum fields, an accessor is created. However, this function is in a trait, so not directly accessible.

I would like to see a test case for that.

I'm not sure I follow: how could this possibly affect a field named recursion_limit? This is implemented on the Message trait, not on the struct itself.

Sorry for another long delay. Well, if a .proto file has a enum field named recursion_limit, it will generate a function named recursion_limit() on the struct. I would like to see a test that proved that the struct function can still be called.

This is tested in tests/src/recursion_limit_field.rs

caspermeijn · 2025-06-10T06:35:55Z

+    /// or it can be disabled entirely using the `no-recursion-limit` feature.
    #[cfg(not(feature = "no-recursion-limit"))]
-    recurse_count: u32,
+    #[doc(hidden)]


Why is this marked as doc hidden?

@LucioFranco previously suggested that it should be doc(hidden) because the field was changed to public. I assume a previous version of this PR needed it to be public but it seems like it's no longer needed. I'll revert this field back to private.

Yes, private is better. My experience is that pub and doc(hidden) makes a sort of secret public API. That will be used by someone.

caspermeijn

Thank you for your contribution and patience. The code needs a rebase with master.

caspermeijn · 2026-02-22T09:21:07Z

+    /// or it can be disabled entirely using the `no-recursion-limit` feature.
    #[cfg(not(feature = "no-recursion-limit"))]
-    recurse_count: u32,
+    #[doc(hidden)]


Yes, private is better. My experience is that pub and doc(hidden) makes a sort of secret public API. That will be used by someone.

caspermeijn · 2026-02-22T09:33:39Z

-// See `encoding::DecodeContext` for more info.
-// 100 is the default recursion limit in the C++ implementation.
-#[cfg(not(feature = "no-recursion-limit"))]
-const RECURSION_LIMIT: u32 = 100;


I like it to be a constant as well. It makes it explicit in the code. But I do like it to be in the documentation as well.

caspermeijn · 2026-02-22T09:42:21Z

+        .iter()
        .any(|a| a.path().is_ident("prost") && a.parse_args::<skip_debug>().is_ok());

+    let mut recursion_limit: u32 = 100;


I think the default should not be named here. If the attribute is not set, then no code should be generated. Which automatically chooses the default implementation of the trait. This makes sure that prost-derive doesn't have to know about the default value,

seanlinsley · 2026-09-08T16:30:11Z

@caspermeijn all review comments should be addressed

seanlinsley force-pushed the recursion-limit-macro branch 3 times, most recently from b2bb84c to 9a50224 Compare December 15, 2022 22:58

LucioFranco requested changes Dec 16, 2022

View reviewed changes

Comment thread src/message.rs Outdated

LucioFranco mentioned this pull request Dec 16, 2022

Support custom recursion limits at runtime #760

Closed

1 task

seanlinsley commented Jan 3, 2023

View reviewed changes

seanlinsley requested a review from LucioFranco January 3, 2023 18:36

seanlinsley mentioned this pull request Jan 11, 2023

Support parsing complex queries with deeply nested ASTs pganalyze/pg_query.rs#17

Open

LucioFranco requested changes Feb 4, 2023

View reviewed changes

Comment thread src/encoding.rs Outdated

Comment thread src/message.rs Outdated

seanlinsley requested a review from LucioFranco February 27, 2023 15:10

LucioFranco requested changes Apr 5, 2023

View reviewed changes

jszwedko mentioned this pull request May 7, 2024

fix(vector source protobuf codec disk buffering) Build prost with no-recursion-limit feature vectordotdev/vector#19413

Closed

seanlinsley force-pushed the recursion-limit-macro branch 5 times, most recently from 08caf43 to 7fc6689 Compare May 29, 2025 17:44

seanlinsley requested a review from LucioFranco May 29, 2025 18:13

LucioFranco requested a review from caspermeijn June 9, 2025 20:47

caspermeijn requested changes Jun 10, 2025

View reviewed changes

caspermeijn requested changes Feb 22, 2026

View reviewed changes

This was referenced Mar 26, 2026

fix(buffers): add nesting depth limit to prevent protobuf decode corr… connoryy/vector#17

Closed

fix(buffers): add nesting depth limit to prevent protobuf decode corruption vectordotdev/vector#25053

Closed

ganelo mentioned this pull request May 12, 2026

fix(buffers): add nesting depth limit to prevent protobuf decode corruption vectordotdev/vector#25417

Closed

3 of 9 tasks

seanlinsley force-pushed the recursion-limit-macro branch from 86b9439 to caf2b36 Compare September 8, 2026 15:48

Support custom recursion limits at build time

85a5c52

seanlinsley force-pushed the recursion-limit-macro branch from caf2b36 to 85a5c52 Compare September 8, 2026 15:58

seanlinsley requested a review from caspermeijn September 8, 2026 16:27

Conversation

seanlinsley commented Dec 15, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

seanlinsley commented Dec 16, 2022

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

seanlinsley commented Jan 11, 2023

Uh oh!

LucioFranco left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

seanlinsley commented Feb 27, 2023

Uh oh!

seanlinsley commented Apr 4, 2023

Uh oh!

LucioFranco left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

caspermeijn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

caspermeijn left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

seanlinsley commented Sep 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

seanlinsley commented Dec 15, 2022 •

edited

Loading