feat(cst): sort an object's properties or an array's elements

[dsherret]

Adds CST reordering that carries each member's comments along with it — the thing the AST plus a position-keyed comment map can't express. Motivated by dprint/dprint-plugin-json#40, which wants to write a package.json's properties in a conventional order; doing it on the AST there forced two compromises (files with comments skipped entirely, and a reordered object losing its blank lines) that this removes.

obj.sort_properties().by_key(|prop| prop.decoded_name());
array.sort_elements().by_key(|element| element.to_string());

A sort is started with sort_properties() / sort_elements() and told how to order things with by or by_key. by_key works the key out once per member rather than once per comparison, which matters when the key is a decoded name. Both are stable.

What travels

A container's children read as open [ sep lead element trail ]... tail close.

• lead (the comments and blank lines above a member) and trail (its comma, and a comment written after it on the same line) travel with the member.
• sep — the line break that ended the previous member's line plus the indentation under it, or on a single line the space between the two — stays put. It positions whatever comes next rather than belonging to either member, which is what keeps both multi-line and single-line containers well formed.
• tail, and anything written on the open token's line, belong to no member and don't move.

Commas are re-decided per position, carrying over whichever of a trailing comma or none the container was written with, and one is added where the new order needs it even if the author left it out. The comma is claimed wherever it was written, including on a later line, so leading-comma style can't strand one in front of the member that follows.

Two normalizations keep the result meaning what it did: a blank line landing directly under the open token is dropped, since a gap there reads as belonging to the container; and a line comment that no longer ends its line gains a line break, without which it would comment out the next member or the closing token.

Ownership of a same-line comment follows what remove_comma_separated already does: everything up to and including the comma was written with the member, and a comment after the comma is the member's only when nothing else shares the line. So { "b": 2, /* between */ "a": 1 } treats the comment as written above "a", the same answer remove gives.

Deciding what travels

The default above is right for a comment describing the member beneath it and wrong for one heading a group, so the caller can say which is which:

obj.sort_properties().pin_comment_headers().by_key(..);
obj.sort_properties().pin_comment_headers_with(|member, comments| ..).by_key(..);
obj.sort_properties().within_groups().by_key(..);

• pin_comment_headers() — a comment with a blank line above it heads what follows and stays where it was written, while members sort past it. Without it a heading is carried off to wherever its first member lands:

  {                            {                            {
    "prop": 1,                   "prop": 1,                   "prop": 1,
                                 "prop1": 1,
    // section        default                  pinned         // section
    "prop2": 2,                  // section                   "prop1": 1,
    "prop1": 1                   "prop2": 2                   "prop2": 2
  }                            }                            }
  

• pin_comment_headers_with(rule) — the rule is handed the member and the comments written above it, and returns how many of them, counting from the top, stay put; the rest travel. pin_comment_headers() is if member.has_blank_line_before() { comments.len() } else { 0 }. A count in between splits a block that is partly a heading and partly a note about the member itself, which a boolean can't say:

  // Dependencies          ← heads the section
                           
  // pinned for CVE-1234   ← describes "b"
  "b": "1.2.3",
  

  Handing the rule the comments the sorter itself computed is deliberate: a count against a list the caller rebuilds would disagree with the sorter exactly in the interesting cases (a comment after the previous member's comma, a block comment sharing the member's line), and an off-by-one there is silent.

• within_groups() — each run of members between blank lines sorts on its own and no member crosses one. A blank line and whatever was written under it is the boundary between two groups, and a boundary stays where it is, so this doesn't depend on which header rule was set; a rule still decides what travels with members inside each group.

has_blank_line_before() is public on every node, since a custom rule needs it.

name_decoded → public decoded_name

It was private and gated behind the serde_json feature, and it's the natural sort key, so it had to become public. Renamed while it's still free to rename: decoded_value() is the crate's existing spelling for this operation on CstStringLit and ObjectPropName, and name_decoded inverted that word order.

A rule or comparator that mutates the container

This used to be partly undone and partly kept — a value set through the shared node survived while a removal was reverted over stale ranges, which could write a corrupted tree. The sort now notices the container changed underneath it and leaves it alone instead.

Performance

Splitting a container was building four small Vecs per member. Each is a stretch of the container's own children that reordering only ever copies, so they're ranges now and the nodes are copied straight out at write-back. The one edited run — the trailing one, because the comma has to suit the new position — records where its comma sits and settles it while writing out, which removed the copy and a whole mutating pass. by_key uses sort_by_cached_key.

properties	before	after
100	50.6µs	31.2µs
1,000	420µs	257µs
10,000	5.50ms	3.15ms

Testing

Unit tests cover field order, trailing-comma styles, single-line and compact containers, word and escaped names, duplicate names (stability), CRLF, missing commas, leading-comma style, comments in every position, blank lines on both sides of the rule, partial header splits, group boundaries vs comments inside a group, the mutation guard, and that handles taken before a sort still resolve and remain editable.

tests/sort_fuzz.rs generates 40,000 documents — trivia in every slot, blank lines, CRLF, missing and trailing commas, duplicate and escaped names, nested values — across all four option combinations, asserting each result re-parses, holds the same members in the comparator's order, preserves stability for equal keys, keeps every comment, and sorts idempotently. ~2.7s, and CI already runs --all-features.

That fuzzer is not decoration. It and the review passes found six bugs the hand-written tests missed: a line comment swallowing the member after it, leading-comma style emitting a container that starts with a comma, a resulting doubled ,,, a replaced member being silently reverted and detached, a blank line riding along instead of staying put, and a header splitting part way along a line holding two comments. It also caught a bug in itself — for a while it was building its options and then sorting without them, so only the default sort was really being checked.

Everything is additive, so a patch release covers it. 160 lib tests, 14 doctests and the fuzz test pass; no new clippy warnings (the two in src/serde.rs are pre-existing).