support TraverseSplat for hclwrite.TokensForTraversal

[jlarfors]

I am working on some tooling to generate HCL and found that the hclwrite.TokensForTraversal method does not support the hcl.TraverseSplat traverser.

I could also not find any use of hcl.TraverseSplat in the hcl package, or in the Terraform code base, so not sure where it is actually used.

Any chance this could be added to the core hcl package? Right now I am relying on my fork and it works great.

[apparentlymart]

Hi @jlarfors,

Honestly my memory of the details here is spotty but I think we ended up not supporting TraverseSplat because it wasn't clear how it ought to behave in the normal use of "traversals" for representing an address of a particular object in scope. In the hclsyntax package there are node types representing traversals from the root scope and relative traversals from other objects, but we ended up needing to implement the "splat operator" as a set of specialized node types because their behavior is so unusual compared to other index and attribute access operations.

Honestly I think I just forgot to delete that vestigial type when I removed my attempt to implement splat expressions as just another kind of traversal. As you noticed, nothing is using it.

I feel uncomfortable with the assymetry of using this traversal type only for hclwrite when it doesn't work as a traversal in any other context.

For similar requests so far we've been gradually adding new functions that return values of type Tokens to help with constructing the tokens for various different expression types that can then be used with SetAttributeRaw. The design intent is for the Expression type to eventually encapsulate those in a higher-level abstraction that doesn't expose the raw tokens, but the exact design for that isn't clear yet and so generating the tokens directly has been the compromise to help folks who are building tools today, by cautiously implementing a set of token generating functions that represent broad use-cases.

I think a function like this for the splat operator would probably have a signature like this:

func TokensForSplat(source, perElem Tokens) Tokens

The source argument would be the tokens to the left of the operator and the perElem would be the tokens to the right. This is more general than using a traversal as the whole expression because the source can now be any valid expression, which is true for the splat operator under evaluation.

Of course all this function would really do is join two token sequences together with the hard-coded delimiter [*], and so the value of this isn't as high as for some of the other token generator functions; if it ends up being a relatively simple function then I might suggest that you just do it inline in your own tool source code for now and then we can try to address this more thoroughly with the Expression abstraction later, once we have some more time to spend there.

What do you think?

[jlarfors]

@apparentlymart thanks for the detailed explanation, and sorry for my delayed response. I understand that TraverseSplat is not a normal traversal, in the sense that changes the return type and is like a shorthand for a for loop as explained here.

In my case, we are building a way for a user to create a reference to an attribute from a Resource or Data Resource, and we added a Splat() method which simply adds the TraverseSplat to an existing Traversal, and for us this works quite well.

Right now we have the following type ReferenceValue which represents a reference to some attribute to some resource.

type ReferenceValue struct {
	tr hcl.Traversal
}

This implements a Tokens() hclwrite.Tokens method which is used by our simple HCL encoder we built to spit out some HCL that we run Terraform over. This quite literally treats a TraverseSplat as another element in the traversal and renders the tokens [*]. With my limited knowledge I don't see the reason why TraverseSplat cannot simply be an element in a hcl.Traversal and why it needs to know what comes before and after it, but I assume there is good reason :)

I'd prefer to use the standard HCL library as much as possible, so do you have a recommendation for how we could store and programmatically build a reference to an attribute (i.e. a traversal), and also provide some special features like Splat() using the hcl* packages?

One thing I consider was re-implementing the hclwrite.TokensForTraversal function, but yeah, I don't want do go down a rabbit hole unnecessarily.