HIP author skill

[gjenkins8]

First pass at a HIP authoring skill:

• Aim to refine, scope and investigate an the idea first, and promotes validating the idea in #helm-dev or developer call before creating the full HIP
• Then a section, by section drafting and refinement process

The HIP template is duplicated and embedded in the skill - enables providing much more detailed guidance for the sections than what is currently in HIP-0001

closes: #436

[TerryHowe]

/lgtm

[scottrigby]

a few thoughts:

1. H4HIP as outlined in https://github.com/helm/community/blob/main/hips/hip-0012.md is missing from this
2. rather than targeted word length for each section, is it better to have instructions that focus more on the spirit of the goal - such as "keep proposals as simple as possible. do not add unnecessary detail other than what's needed" etc?
3. meta question - what is this ultimately adding for an LLM above what a copy of https://github.com/helm/community/blob/main/hips/hip-0001.md would provide? If there are missing pieces from hip 001 (eg, H4HIP defined in 0012) should we instead amend hip 0001 with any new/missing/clarifying information?

[gjenkins8]

1. H4HIP as outlined in https://github.com/helm/community/blob/main/hips/hip-0012.md is missing from this

H4HIP were HIPs targeting Helm 4's development process. We are beyond that now - we now just have "HIPs" IMHO.

Also to be discussed: the HIP frontmatter actually already has a helm-version: field. Which I think we didn't realize when writing the Helm 4 HIP and proposing H4HIPs. So the H4HIP prefix was actually a little redundant.

Finally, H4HIP was simply a thing we prepended to the HIP's PR title. So we can keep doing that with e.g. H5HIP: , without the skill for writing the HIP actually know/caring I think.

---

rather than targeted word length for each section, is it better to have instructions that focus more on the spirit of the goal - such as "keep proposals as simple as possible. do not add unnecessary detail other than what's needed" etc?

Given LLMs tendency to write overly verbosely, "as simple as possible" are much more ambiguous. In fact, when I updated the skill to remove the word counts and add such language, the eval reported:

My interpretation

Removing the inline Target ~N words anchors backfired. A concrete number is a cheap, strong constraint the model actually applies as it drafts. Replacing it with "as simple as possible, the counts live in the template, treat them as a ceiling" is a softer signal — and it looks like the runs didn't re-derive the numbers from the template, so sections drifted longer. The prose principle "the range is a guide, not a hard limit" may even have licensed expansion.

That said, an update to the skill to use the word counts as guides (not targets, and not hard ceilings) helped a little in evaluations:

e.g. 746af84#diff-61b24769e609a18d3995dcd65eecd2cc83143d97c8b2faee5c4637b8e6f37069R74

---

meta question - what is this ultimately adding for an LLM above what a copy of https://github.com/helm/community/blob/main/hips/hip-0001.md would provide? If there are missing pieces from hip 001 (eg, H4HIP defined in 0012) should we instead amend hip 0001 with any new/missing/clarifying information?

The "template" is included inline with the SKILL so it can be enriched with more detailed instructions. So yes, these could (perhaps should) be fed back into the original template. Any instructions should be applicable to both people and LLMs. But initially inline allows the skill off the ground without affecting existing process, it is hard enough to get changes into this repo as is without proposing changes to existing HIPs!