Skip to content

Incorrect remote cache capacity calculation that miss timeIndex size #3732

Description

@zuston

Search before asking

  • I searched in the issues and found nothing similar.

Fluss version

0.9.0 (latest release)

Please describe the bug 🐞

The remote log index cache uses Caffeine’s weighted eviction policy to limit the total size of locally cached remote index files.

Each cache entry contains two index files:

  1. Offset index (.index)
  2. Time index (.timeindex)

However, RemoteLogIndexCache.Entry#estimatedEntrySize() currently only includes the offset index size:
offsetIndex.sizeInBytes()
The time index size is not included in the cache weight. As a result, the cache underestimates its actual storage usage and may retain more index files than allowed by remote.log.index.file.cache.size.

Expected Behavior

The cache weight of an entry should include both index files:
offsetIndex.sizeInBytes() + timeIndex.sizeInBytes()
Caffeine should trigger eviction based on the complete size of the cached remote indexes.

Solution

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Fields

    No fields configured for Bug.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions