Skip to content

feat(kubernetes): support paginated list requests with chunk size - #4321

Open
shardool-patil wants to merge 1 commit into
zalando:masterfrom
shardool-patil:feat/k8s-list-chunk-size
Open

shardool-patil wants to merge 1 commit into
zalando:masterfrom
shardool-patil:feat/k8s-list-chunk-size

Conversation

@shardool-patil

Copy link
Copy Markdown
Contributor

Summary

Introduces support for paginated list requests when querying Kubernetes API resources via a new --kubernetes-list-chunk-size flag and configuration parameter.

Key Changes

  • Configuration (config/config.go, skipper.go, kube.go): Added --kubernetes-list-chunk-size flag and wired KubernetesListChunkSize into client options.
  • Pagination Logic (dataclients/kubernetes/clusterclient.go): Implemented generic loadChunkedList[T] using Kubernetes limit and continue query parameters; keeps non-paginated behavior when chunkSize <= 0.
  • Mock API (dataclients/kubernetes/kubernetestest): Extended test API to handle limit constraints and pagination continue tokens.

Testing

  • Unit Tests:
    • Added unit test cases in clusterclient_test.go and endpointslices_test.go verifying chunked listing across Ingresses, Services, Endpoints, and EndpointSlices.
    • Tested pagination edge cases: single-item pages, chunk sizes exceeding total resource count, exact multiples, and fallback behavior when chunkSize <= 0.
    • Confirmed query parameter persistence (e.g., label selectors and namespace scopes) across pagination calls.
  • Passed targeted test suites:
    go test -v ./dataclients/kubernetes/... ./config

Related Issues

Comment thread dataclients/kubernetes/clusterclient.go Outdated
return err
}
defer rsp.Body.Close()
defer func() { _ = rsp.Body.Close() }()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why this?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did that to handle linter warning regarding the unchecked return value of Close(), so should i revert that?

manifest, err := os.Open("testdata/routegroups/convert/missing-service.yaml")
require.NoError(t, err)
defer manifest.Close()
defer func() { _ = manifest.Close() }()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here why wrapper func?


func TestAuthorizeRequestFilterWithS3DecisionLogPlugin(t *testing.T) {
t.Setenv("AWS_ACCESS_KEY_ID", "testing")
t.Setenv("AWS_SECRET_ACCESS_KEY", "testing")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these changes are unrelated

Comment thread go.mod Outdated
github.com/instana/go-sensor v1.79.0
github.com/klauspost/compress v1.20.1
github.com/instana/go-sensor v1.77.0
github.com/klauspost/compress v1.20.0

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please rebase on current master

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed, rebased and handled the code changes across test files

@shardool-patil
shardool-patil force-pushed the feat/k8s-list-chunk-size branch from 5f531b7 to 735bce5 Compare October 5, 2026 10:55
@shardool-patil
shardool-patil marked this pull request as draft October 5, 2026 11:01
Signed-off-by: Shardool Patil <shardoolpatil999@gmail.com>
@shardool-patil
shardool-patil force-pushed the feat/k8s-list-chunk-size branch from 735bce5 to 35f9454 Compare October 5, 2026 11:21
@shardool-patil
shardool-patil marked this pull request as ready for review October 5, 2026 11:23
@szuecs szuecs added the major moderate risk, for example new API, small filter changes that have no risk like refactoring or logs label Oct 6, 2026
@szuecs

szuecs commented Oct 9, 2026

Copy link
Copy Markdown
Member

Please review

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

major moderate risk, for example new API, small filter changes that have no risk like refactoring or logs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use ListMeta to paginate large lists into more chunks

4 participants