-
-
Notifications
You must be signed in to change notification settings - Fork 730
Add AI tools policy #13726
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Add AI tools policy #13726
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
e27bf3e
docs: Add `AI-TOOLS.md` policy from python/devguide
Turbo87 bdb9e5e
docs/AI-TOOLS: Drop Python-specific testing principles sentence
Turbo87 bdf9316
AGENTS: Link to `AI-TOOLS.md` from reference material list
Turbo87 c333fee
docs/CONTRIBUTING: Add `Using AI tools` section
Turbo87 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| # Guidelines for using AI tools | ||
|
|
||
| The person submitting an issue or PR is responsible for its content, | ||
| regardless of whether AI tools were used in its creation. Generative AI | ||
| tools can produce output quickly, but discretion, good judgment, and | ||
| critical thinking are the foundation of all good contributions. We value | ||
| good code, concise accurate documentation, and well scoped PRs without | ||
| unneeded code churn. | ||
|
|
||
| ## Considerations for success | ||
|
|
||
| Authors must review the work done by AI tooling in detail to ensure it | ||
| actually makes sense before proposing it as a PR or filing it as an issue. | ||
|
|
||
| We expect PR authors and those filing issues to be able to explain their | ||
| proposed changes in their own words. | ||
|
|
||
| Disclosure of the use of AI tools in the PR description is appreciated, | ||
| while not required. Be prepared to explain how the tool was used and what | ||
| changes it made. | ||
|
|
||
| Whether you are using AI tools or not, keep the following principles in | ||
| mind for the quality of your contribution: | ||
|
|
||
| - Consider whether the change is necessary | ||
| - Make minimal, focused changes | ||
| - Follow existing coding style and patterns | ||
| - Write tests that exercise the change | ||
| - Keep backwards compatibility with prior releases in mind. Existing | ||
| tests may be ensuring specific API behaviors are maintained. | ||
|
|
||
| Pay close attention to AI generated recommendations for testing changes. | ||
| Always review the output before opening a pull request or issue, | ||
| including proposed PR or issue titles and descriptions. | ||
|
|
||
| ## Acceptable uses | ||
|
|
||
| Some of the acceptable uses of generative AI include: | ||
|
|
||
| - Assistance with writing comments, especially in a non-native language | ||
| - Gaining understanding of existing code | ||
| - Supplementing contributor knowledge for code, tests, and documentation | ||
|
|
||
| ## Unacceptable uses | ||
|
|
||
| Maintainers may close issues and PRs that are not useful or productive, | ||
|
Turbo87 marked this conversation as resolved.
|
||
| regardless of whether AI tools were used or not. | ||
|
|
||
| If a contributor repeatedly opens unproductive issues or PRs, they may be | ||
| blocked from contributing to the project because it is disruptive and | ||
| disrespectful of the maintainers time. | ||
|
|
||
| It is not acceptable to alter or bypass existing tests, or remove desired | ||
| functionality, in order to make a failing test pass. Such changes are not | ||
| a real fix. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So ultimately I don't expect crates.io to have a lot of PRs or issues so I think that ultimately this is just me giving advice that you're free to ignore. But at least from what I found, even projects which strongly support AI tools have been in favour of disclosing just so they know what they're dealing with, and I think that it's okay to have a hard rule with no punishments for forgetting it.
Like, I figure the rate of issues/PRs is so low that you have the time/resources to just individually talk to each person and figure out what they're working with anyway, but it can be helpful to ask for to see what people are using.
Just a few examples of other policies that feel relevant here:
View changes since the review
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This would also be my preference.
I litigated this at some length within the Foundation when we were drafting our internal AI policy, so my apologies to @Turbo87 for the repeat, but the short version is that I believe that we should require disclosure for two primary reasons:
I don't want this to be some onerous process. I'd be happy with an
Assisted-ByorCo-Authored-Bytrailer in either the commit message or the PR description, much like the Linux kernel policy. But I do think it's important that we require this.Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be interesting to hear from counsel on the first point. I would expect to hear a mixed set of pros and cons for each choice — widespread disclosure seems likely to carry some risks too for the Project. Regarding the second point, the relevant comment in the Python discussions was this one — in short, reviewers need to assume LLM use anyway.
Disclosure has more than minimal cost:
Mandating disclosure has additional costs. It becomes something we need to enforce fairly and consistently. As part of a moderation policy, we'd be talking about banning people from the Project over this. If it turns out the policy is unenforceable and that people choose not to disclose due to these costs, then we'll have created a kind of policy fiction. That's not any good for us.
Similar to Python, we can encourage disclosure without incurring the costs that accompany mandating it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So, it is worth reiterating on the legal point that generally the result of using LLMs is the removal of copyright protections, not the addition of extra burdens, which does not matter given the extremely permissive license crates.io uses. It does matter for proprietary code and copyleft code, but for open source, permissively licensed work, it seems fine.
Additionally, it seems incredibly likely that if any copyright questions came up, e.g. someone was upset that code verbatim was used without authorisation, there would be ample time to fix the issue (removing the offending code) rather than these being immediate repercussions.
Most of the projects that I've seen be concerned with copyright issues are copyleft and thus specifically rely on copyright to enforce licencing, which just isn't the case here. So, while I do think there are other points to discuss, I don't think the legal point is particularly compelling here, which also matches the existing advice given by Foundation counsel on the matter.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I do most of the PR reviews on crates.io at the moment (obviously excluding my own PRs…), and knowing whether AI was involved wouldn't change much how I review. The "explain your changes" requirement is already what I actually care about. For first-time contributors with zero context I agree disclosure can help, but that's a pretty different situation from someone opening a dozens of PRs each month.
On the legal point: the PSF landed on this exact wording without requiring disclosure. I don't know what their counsel actually said about it, so I'm only guessing, but I'd be a little surprised if a project that size shipped an AI policy without running it past their lawyers at some point. If that's roughly right, it makes me want to understand what's specifically different about our situation before treating the legal angle as a strong reason to go further than they did.
And I think Travis is right that disclosure has real downsides in the current climate. Adding a trailer to every PR also adds quite a bit of friction when LLM tooling is opening them, and getting that tooling to reliably add trailers is a whole separate problem.
That said, here are a few things I'd be happy to put in the policy, if it helps:
Two other options to avoid the additional friction for regular contributors:
We can flip the framing: say explicitly that reviewers should assume AI may have been used, and make disclosure of non-use the optional opt-in. At this point the vast majority of recent crates.io PRs had LLM involvement and IMHO it's better to optimize for the 90% case.
One-time disclosure for regulars, recorded privately: contributors who use AI tooling regularly disclose it once (e.g. via a comment on a designated issue in the team's private ops guide repo) instead of repeating it on every PR. The record exists, so reviewers on the team have access and the legal/review-practicalities angle is covered, but the disclosure isn't sitting in a public list that anyone can scrape and turn into a target. For someone like me who opens a lot of PRs that drops the per-PR friction effectively to zero while still leaving the information available where it matters. First-time or external contributors would still get asked by a PR template or maybe by the reviewer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In the opening comment here I read:
The policy discussed here differs in many parts from rust-forge#1040 (for example the disclosure policy) and I appreciate you want something out sooner.
But I want to raise a more general question: this policy is more lenient than the other one, so I think that if you want to adopt rust-forge#1040, you might find yourself deciding to backpedal on some items that are now allowed (or not mentioned), like the disclosure clause which we feel strongly about and many others, like bots reviews, typos fixes, etc.
How does the t-crates.io team feels when (or if) comes the question about adopting the rust-lang/rust policy?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
rust-lang/rust-forge#1040 is specifically only about rust-lang/rust and not about other subprojects/repositories. if that policy should influence the policies of other subprojects then that discussion needs to happen in a Project-wide RFC with buy-in from all teams.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK that partially answers my question, I'll try to fill in the blanks (correct me if I am wrong): you would not adopt rust-forge#1040 if proposed to become a project-wide policy as it is, correct?
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I honestly haven't followed the forge policy that closely explicitly because it was out of scope regarding crates.io. I have not checked what the current state is and where it might be incompatible with what we're doing on this side of the project. I was also not aware that there was even a discussion of this becoming a project-wide policy.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I also was under the impression that the forge policy was explicitly not proposed as project-wide because it would then require the buy-in of all the teams, many of whom manage their repos differently (case in point: here). If a project-wide policy is adopted, team-wide policy can change to accommodate it, but for now, project-wide policy isn't adopted, so, I think it's totally fine for teams to adopt any policy since it helps remove ambiguity in enforcement.