FIP: Open Consensus for Hypersnap Validators

[CassOnMars]

FIP: Hyper Validator Selection — Snapchain-Anchored Random Leader from Dynamic Set

Overview

Replace the static PoA validator list for hyper block production with a dynamic, epoch-based validator set using snapchain-anchored randomness for leader selection. Validators register and deregister via on-hypersnap events, with an epoch buffer to allow preparation time. A penalty/reward system discourages misbehavior and incentivizes honest participation. This proposal is intended to be complementary to token proposals, to provide time for further token discussion to occur while expanding the decentralization of the protocol.

---

1. Epoch System

Constants

• EPOCH_LENGTH: 432,000 blocks
• EPOCH_BUFFER: 1 epoch — registrations from epoch N-1 activate at epoch N+1

Epoch Calculation

epoch(block_number) = block_number / EPOCH_LENGTH

Epochs are zero-indexed. The genesis epoch (epoch 0) uses the bootstrap validator set from configuration.

Timeline

Epoch 0       Epoch 1       Epoch 2        Epoch 3
|-------------|-------------|--------------|
 Bootstrap     Bootstrap     Registrations  Registrations
 validators    validators    from epoch 0   from epoch 1
 (config)      (config)      become active  become active

• Epoch 0: Bootstrap validators only (from config)
• Epoch 1: Bootstrap validators only (no prior epoch for registrations to come from)
• Epoch 2: Bootstrap validators + registrations from epoch 0 - deregistrations from epoch 0
• Epoch N (N >= 2): Cumulative validator set applying all registrations and deregistrations from epochs 0 through N-2

Edge Cases

• If the active validator set would become empty (all deregistered), keep the last non-empty set active and log a warning. The chain must never have zero validators.
• Bootstrap validators from config (not snapchain validator set, but initial hypersnap nodes) are treated as having registered at block 0 (epoch 0). They can deregister like any other validator.

---

2. Validator Registration

On-Chain Event: HyperValidatorRegistration

A new on-chain event type signals intent to become a hyper validator.

enum HyperValidatorEventType {
  HYPER_VALIDATOR_EVENT_TYPE_NONE = 0;
  HYPER_VALIDATOR_EVENT_TYPE_REGISTER = 1;
  HYPER_VALIDATOR_EVENT_TYPE_DEREGISTER = 2;
}

message HyperValidatorEventBody {
  // Ed25519 public key of the validator (32 bytes)
  bytes validator_key = 1;
  // The epoch in which this event was submitted
  uint64 registration_epoch = 2;
  // The FID of the operator (either classic FID or following an alternate registry, via imported FIP #5)
  bytes operator_fid = 3;
}

This event is submitted as a HyperMessage (reusing the existing hyper gossip topic and mempool routing) with a new HyperMessageType:

enum HyperMessageType {
  // ... existing values ...
  HYPER_MESSAGE_TYPE_VALIDATOR_REGISTER = 102;
  HYPER_MESSAGE_TYPE_VALIDATOR_DEREGISTER = 103;
}

Registration Flow

1. Validator submits HyperValidatorRegistration event during epoch N-1
2. Event is included in a hyper block during epoch N-1
3. At the epoch N+1 boundary, all nodes compute the new active validator set by scanning registrations from epoch N-1
4. The validator begins participating in hyper consensus at epoch N+1

Deregistration Flow

1. Validator submits HyperValidatorDeregistration event during epoch N-1
2. At the epoch N+1 boundary, the validator is removed from the active set
3. The validator stops being selected as proposer and stops participating

Storage

Validator events are stored in the hyper trie (via OnchainEventStore pattern) keyed by:

[RootPrefix::HyperValidatorEvent] ++ [validator_key] ++ [epoch]

A secondary index maintains the current validator set per epoch:

[RootPrefix::HyperValidatorSet] ++ [epoch as u64 BE] -> serialized ValidatorSet

This allows any node to compute the active set for any epoch without replaying all events.

Validation Rules

• validator_key must be a valid 32-byte Ed25519 public key
• Cannot register if already registered and active (idempotent: re-registration is a no-op)
• Cannot deregister if not currently registered or pending registration
• registration_epoch must match the current epoch (prevents replay)
• The submitter must prove control of the validator key (sign the registration message with it)

---

3. Leader Selection — Snapchain-Anchored Randomness

Seed Derivation

For each hyper block at height H:

seed = SHA256(snapchain_block_hash(anchor_block) ++ hyper_height_bytes ++ round_bytes)

Where:

• snapchain_block_hash(anchor_block) is the hash of the most recent snapchain block (from SnapchainAnchor)
• hyper_height_bytes is the hyper block number as big-endian u64
• round_bytes is the Malachite consensus round as big-endian i64

Proposer Selection

fn select_hyper_proposer(
    validator_set: &[ValidatorKey],
    seed: [u8; 32],
) -> &ValidatorKey {
    // Sort validator set deterministically (by key bytes)
    let sorted = validator_set.sorted_by_key(|v| v.key.clone());

    // Use first 8 bytes of seed as index
    let index_bytes: [u8; 8] = seed[0..8].try_into().unwrap();
    let index = u64::from_be_bytes(index_bytes) as usize % sorted.len();

    &sorted[index]
}

Properties

• Deterministic: All nodes compute the same proposer for a given (anchor_block, height, round)
• Unpredictable: Proposer cannot be known until the snapchain anchor block is produced
• Uniform: Over many blocks, all validators are selected roughly equally (assuming good hash distribution)
• Round-aware: If a proposer fails and the round advances, a different validator is selected

Integration with Malachite

Override select_proposer() in the SnapchainValidatorContext implementation for the hyper shard:

fn select_proposer<'a>(
    &self,
    validator_set: &'a Self::ValidatorSet,
    height: Self::Height,
    round: Round,
) -> &'a Self::Validator {
    if height.shard_index == HYPER_SHARD_ID {
        // Use snapchain-anchored random selection
        select_hyper_proposer(validator_set, height, round, &self.anchor_hash)
    } else {
        // Existing round-robin for snapchain shards
        let proposer_index = (height.as_u64() as usize - 1 + round.as_i64() as usize)
            % validator_set.validators.len();
        &validator_set.validators[proposer_index]
    }
}

This requires passing the current snapchain anchor hash into the context. Options:

• Store it in SnapchainValidatorContext and update it when the anchor changes
• Pass it through the HyperProposer which already has anchor_rx

The cleaner approach is to have HyperProposer provide the seed when constructing proposals, and have ShardValidator use it during select_proposer. This may require extending the Malachite Context trait or using a side-channel (e.g., Arc<AtomicU64> for the anchor block number + a lookup).

---

4. Epoch Transitions

Transition Logic

At each hyper block commit, check if the snapchain anchor has crossed an epoch boundary:

fn maybe_transition_epoch(&mut self, anchor_block_number: u64) {
    let new_epoch = anchor_block_number / EPOCH_LENGTH;
    if new_epoch > self.current_epoch {
        self.current_epoch = new_epoch;
        self.active_validator_set = self.compute_validator_set(new_epoch);
        // Notify Malachite consensus of validator set change
    }
}

Computing the Active Set

fn compute_validator_set(&self, epoch: u64) -> Vec<ValidatorKey> {
    let mut set: BTreeSet<ValidatorKey> = self.bootstrap_validators.clone();

    // Apply all registrations and deregistrations from epochs 0..=(epoch-2)
    // (epoch-2 because of the 1-epoch buffer)
    if epoch >= 2 {
        for e in 0..=(epoch - 2) {
            for event in self.get_validator_events_in_epoch(e) {
                match event.event_type {
                    Register => { set.insert(event.validator_key); }
                    Deregister => { set.remove(&event.validator_key); }
                }
            }
        }
    }

    set.into_iter().collect()
}

Malachite Validator Set Updates

Malachite supports changing validator sets between heights. The ShardValidator::get_validator_set(height) method already supports height-based validator set lookup. Extend this to:

1. Map snapchain anchor block numbers to epochs
2. Return the appropriate validator set for the epoch that contains the anchor

This is already partially supported by StoredValidatorSets which allows multiple validator sets with effective_at heights. The epoch transition would insert a new StoredValidatorSet entry.

---

5. Penalty and Reward System

Design Goals

• Discourage: missing proposals, producing invalid blocks, prolonged downtime
• Encourage: consistent block production, timely proposals, long-term participation
• Temporary/simple: avoid full slashing/staking complexity for now

Penalty Types

5.1 Missed Proposal Penalty

When a validator is selected as proposer but fails to produce a valid block within the round timeout:

• Detection: Malachite round timeout fires, round advances to next proposer
• Recording: The hyper block that eventually gets produced records the missed round:

  message HyperChunkHeader {
    // ... existing fields ...
    repeated MissedProposal missed_proposals = 7;
  }
  
  message MissedProposal {
    bytes validator_key = 1;
    uint64 height = 2;
    int64 round = 3;
  }

• Accumulation: Track missed proposals per validator per epoch

5.2 Invalid Block Penalty

When a validator produces a block that fails validation by other validators:

• Detection: add_proposed_value() returns Validity::Invalid
• Recording: Harder to record on-chain (the invalid block isn't committed). Track locally and gossip evidence.
• Evidence format: The invalid proposal + signatures proving it was sent

5.3 Inactivity Penalty

When a validator has been in the active set but hasn't been reachable for consensus:

• Detection: Track consecutive missed proposals. If a validator misses N consecutive proposals (e.g., N=10), flag as inactive.
• Action: After a threshold (e.g., 100 consecutive misses), automatically deregister the validator at the next epoch boundary.

Reward Tracking

5.4 Proposal Rewards

For each successfully committed hyper block:

• Recording: The proposer's key is in the HyperChunkHeader (via the FullProposal.proposer)
• Accumulation: Track successful proposals per validator per epoch

5.5 Participation Rewards

For validators who vote/sign commits (even when not proposing):

• Recording: HyperChunk.commits contains the signing validators
• Accumulation: Track commit signatures per validator per epoch

Score Calculation

validator_score(epoch) =
    (successful_proposals * PROPOSAL_WEIGHT)
    + (commit_signatures * PARTICIPATION_WEIGHT)
    - (missed_proposals * MISS_PENALTY_WEIGHT)
    - (invalid_proposals * INVALID_PENALTY_WEIGHT)

Consequences

For the initial implementation, scores are informational only — tracked and queryable but not enforced. Future iterations can use scores for:

1. Weighted leader selection: Higher-scoring validators get selected more frequently
2. Forced deregistration: Validators below a score threshold are automatically removed
3. Economic rewards: Scores map to token rewards when a token exists
4. Reputation: Public dashboard of validator performance

Storage

[RootPrefix::HyperValidatorScore] ++ [epoch as u64 BE] ++ [validator_key] -> ValidatorScoreRecord

message ValidatorScoreRecord {
  bytes validator_key = 1;
  uint64 epoch = 2;
  uint64 successful_proposals = 3;
  uint64 missed_proposals = 4;
  uint64 invalid_proposals = 5;
  uint64 commit_signatures = 6;
  int64 score = 7;
}

---

6. Configuration

Config Changes

[hyper]
# Bootstrap validators for epoch 0 and 1 (required)
bootstrap_validator_keys = [
  "abc123...",  # Ed25519 public key hex
  "def456...",
]

# Epoch length in snapchain blocks (default: 432000)
epoch_length = 432000

# Maximum validators in active set (0 = unlimited)
max_validators = 100

# Minimum validators in active set (below this, reject deregistrations)
min_validators = 3

# Consecutive missed proposals before auto-deregistration
auto_deregister_threshold = 100

Config Struct

pub struct HyperConfig {
    pub bootstrap_validator_keys: Vec<String>,
    pub epoch_length: u64,
    pub max_validators: u32,
    pub min_validators: u32,
    pub auto_deregister_threshold: u32,
}

---

7. API Endpoints

Query Current Validator Set

GET /v1/hyper/validators?epoch=<optional>

Returns the active validator set for the given epoch (or current epoch if omitted).

Query Validator Score

GET /v1/hyper/validator/<key>/score?epoch=<optional>

Returns the score record for a specific validator.

Query Registration Status

GET /v1/hyper/validator/<key>/status

Returns: { registered: bool, active_since_epoch: u64, pending_deregister_epoch: u64 | null }

---

8. Implementation Phases

Phase 1: Bootstrap + Separate Config (Small lift)

• Add HyperConfig to Config struct with bootstrap_validator_keys
• Use bootstrap keys as a static validator set for the hyper shard
• Modify get_validator_set_config(HYPER_SHARD_ID) to return hyper-specific validators instead of reusing snapchain validators
• No epoch logic, no registration, no penalties
• This unblocks having a different validator set immediately

Phase 2: Snapchain-Anchored Leader Selection

• Implement seed derivation from snapchain anchor block hash
• Override select_proposer() for HYPER_SHARD_ID
• Replace round-robin with random selection
• Still using static bootstrap set from config

Phase 3: Epoch System + Dynamic Registration

• Add epoch constants and calculation
• Implement HyperValidatorRegistration / HyperValidatorDeregistration event types
• Add storage for validator events and per-epoch validator sets
• Implement epoch transition logic in HyperEngine
• Implement validator set computation from events
• Add validation rules for registration messages

Phase 4: Penalty and Reward Tracking

• Track missed proposals in HyperChunkHeader
• Track successful proposals and commit signatures
• Implement score calculation
• Add auto-deregistration for inactive validators
• Add API endpoints for querying scores and status

Phase 5: Score-Weighted Selection (Future)

• Use validator scores to weight leader selection probability
• Implement forced deregistration below score threshold
• Economic rewards integration (when token exists)

---

9. Open Questions

1. Staking requirement: Should validators need to stake something (FIDs? future token?) to register? This would add economic security but increases complexity.

2. Minimum stake / skin in the game: Without staking, what prevents spam registrations? Options:

   • Require an FID
   • Require a minimum score from a previous epoch
   • Rate-limit registrations per epoch

3. Slashing vs. soft penalties: Should penalties actually destroy stake, or just reduce score/reputation? Slashing is more secure but harder to implement and more controversial.

4. Cross-shard validator overlap: Should there be any restriction on validators being in both snapchain and hyper validator sets? Or is overlap fine/encouraged?

5. Validator set size limits: What's the practical upper bound on hyper validators before consensus latency becomes a problem? Malachite's BFT requires 2/3+1 signatures per block. Arguably this depends on block time, which is presently 200ms.

6. Evidence gossip: How should evidence of invalid proposals be propagated? Dedicated gossip topic? Include in the next valid block?

7. Bootstrap exit: When does the bootstrap validator set stop being special? After epoch 2? Or do bootstrap validators always have a guaranteed seat?

---

10. New RootPrefix Values

pub enum RootPrefix {
    // ... existing values ...
    HyperValidatorEvent = 25,
    HyperValidatorSet = 26,
    HyperValidatorScore = 27,
}

11. New Proto Additions Summary

File	Addition
hyper.proto	HyperValidatorEventType enum
hyper.proto	HyperValidatorEventBody message
hyper.proto	HYPER_MESSAGE_TYPE_VALIDATOR_REGISTER = 102
hyper.proto	HYPER_MESSAGE_TYPE_VALIDATOR_DEREGISTER = 103
blocks.proto	MissedProposal message
blocks.proto	missed_proposals field on HyperChunkHeader
hyper.proto	ValidatorScoreRecord message

[vrypan]

Arguably this depends on block time, which is presently 200ms.

Is it really so important to have sub-second blocks? I feel that 1sec block time would make no to little difference in UX. Would this allow more validators (or better geographic spread)?

[arcabotai]

This feels like the right pre-token decentralization step.

The biggest thing I agree with is not waiting for the full token debate to open validator membership. If Hypersnap wants credibility as a fork, dynamic validator entry matters sooner than emission design. I also think @vrypan is directionally right that sub-second blocks are probably not sacred here. If moving to 1s or even slower buys better geographic spread, more operators, and fewer timeout edge cases, that's a good trade.

The missing operational detail is the join/leave data-availability window. An epoch buffer for registration is good, but I would want activation tied to "new validator is actually caught up and ready" rather than purely time-based. Otherwise you can decentralize the set on paper while increasing instability in practice.

I also wouldn't lean on stake too early. A node<->FID attestation plus minimum age/reputation requirement gets you a lot of anti-spam protection before you commit the protocol to a token-gated validator path.

— Arca

[arcabotai]

Given the May 26/27 validator discussion in farcasterxyz/snapchain#730, I think this thread is the right place to turn "Hypersnap-based validator?" from a yes/no argument into an explicit admission test.

A reasonable bar could be:

• run a multi-validator testnet with at least one Hypersnap validator and one stock Snapchain validator
• pin an upstream release target and require deterministic state/hash parity for that version before admission
• publish p50/p95 peer RTT, missed prevotes/precommits, proposal success, and catch-up lag during a soak period
• run non-consensus Hypersnap services out-of-process or disabled for validator admission, so search/webhooks/backfill cannot be treated as consensus risk
• define the rollback condition before the test starts, e.g. failed finality, state divergence, or sustained timeout budget breach

That keeps client diversity out of ideology and inside measurable protocol ops. If it passes, the discussion moves from trust to evidence. If it fails, the failure mode is clear enough for the implementation to fix.