chore(inputs.elasticsearch_query): Migrate to official ES client by alespour · Pull Request #19288 · influxdata/telegraf

alespour · 2026-07-15T16:43:04Z

Summary

Refactors plugin code to use official ES client instead of unmaintained olivere/elastic.
Adds ES 5 integration test coverage.

Node discovery caveats

As results of official clients limitations:

ES 5 configurations using enable_sniffer = true will now fail explicitly because the official v5 client does not support node discovery.
ES 6 client in-flight node discovery cannot be cancelled because DiscoverNodes() has no context support. Shutdown waits for it to finish, bounded by the configured HTTP timeout (5s by default).

Checklist

No AI generated code was used in this PR
AI generated code used in this PR follows the InfluxData Policy on AI-Generated Code Contributions

Related issues

resolves #19212

srebhan

Thanks @alespour for your work! I have some comments...

srebhan · 2026-07-16T14:35:08Z

+		Addresses: cfg.urls,
+		Username:  cfg.username,
+		Password:  cfg.password,
+		Transport: roundTripper{client: cfg.httpClient},


Why do we need this? Can't we simply do

Suggested change

Transport: roundTripper{client: cfg.httpClient},

Transport: cfg.httpClient.Transport,

Unlike Transport, http.Client enforces timeout during the whole request-response lifecycle including response body-reading.

Do we really need this?

I think so, without http.Client wrapper, stalled response during discovery could hang (discovery v6 is not cancellable via context). And that would block plugin shutdown via Stop().
But Client also handles cookies. Passing only Transport would lose it. We would have to handle cookies ourselves to cover cases when users enable auth cookie.

srebhan · 2026-07-17T10:15:22Z

@alespour regarding the question if we need the v5 client if code works with v6, I'd say no. If you can cover multiple ES server versions with one client, go for it as long as this doesn't limit the queries or anything...

alespour · 2026-07-17T11:23:04Z

@srebhan Thank you very much for very thorough review. I'm overhauling the PR.

telegraf-tiger · 2026-07-20T09:01:07Z

Download PR build artifacts for linux_amd64.tar.gz, darwin_arm64.tar.gz, and windows_amd64.zip.
Downloads for additional architectures and packages are available below.

⚠️ This pull request increases the Telegraf binary size by 1.94 % for linux amd64 (new size: 314.2 MB, nightly size 308.2 MB)

📦 Click here to get additional PR build artifacts

Artifact URLs

. DEB	. RPM	. TAR . GZ	. ZIP
amd64.deb	aarch64.rpm	darwin_amd64.tar.gz	windows_amd64.zip
arm64.deb	armel.rpm	darwin_arm64.tar.gz	windows_arm64.zip
armel.deb	armv6hl.rpm	freebsd_amd64.tar.gz	windows_i386.zip
armhf.deb	i386.rpm	freebsd_armv7.tar.gz
i386.deb	ppc64le.rpm	freebsd_i386.tar.gz
mips.deb	riscv64.rpm	linux_amd64.tar.gz
mipsel.deb	s390x.rpm	linux_arm64.tar.gz
ppc64el.deb	x86_64.rpm	linux_armel.tar.gz
riscv64.deb		linux_armhf.tar.gz
s390x.deb		linux_i386.tar.gz
		linux_mips.tar.gz
		linux_mipsel.tar.gz
		linux_ppc64le.tar.gz
		linux_riscv64.tar.gz
		linux_s390x.tar.gz

alespour · 2026-07-20T09:37:40Z

@srebhan PR is overhauled. Plugin uses concrete client implementation for each major version (v5, v6). Version-independent logic is now in shared functions (code blocks moved client_v5.go -> client.go, without logic change).

One thing that requires maintainers decision: v5 client does not support discovery. Now, when enable_sniffer = true and v5 backend detected, plugin fails to start with an error. This is a breaking change.

I can think of 3 solutions:

a) document it as breaking change
b) log warning and silently ignore
c) use v6 client, but, there is a but - although all tests (unit and integration) using v6 client against ES v5 backend pass OK, there is no official guarantee about v6 client compatibility with older ES versions. Just empirical validation with tests. REST interface seems stable between v5 and v6 and official client is fairly low-level, so the fact that tests pass is fairly predictable.

srebhan · 2026-07-21T16:05:45Z

Thanks @alespour for the update, I will review in a minute...

Regarding

I can think of 3 solutions:

Please issue a warning and silently ignore discovery for v5. I think this is best as it is unlikely someone is still on 5.x and uses discovery. Maybe also add a short comment in the README.md that v5 does not support discovery.

srebhan

Nice update @alespour! Some more comments from my side...

srebhan · 2026-07-21T16:07:46Z

+		return "", 0, fmt.Errorf("parsing server version %q failed: %w", info.Version.Number, err)
+	}
+
+	return info.Version.Number, int(version.Major()), nil


Can we please simply return the uint64 instead of casting this?

srebhan · 2026-07-21T16:13:03Z

+	if statusCode <= 299 {
+		return nil
+	}


So we do accept 1xx codes as good?

srebhan · 2026-07-21T16:16:18Z

+		if interval <= 0 {
+			return
+		}


Can we please check this outside of this function?!

srebhan · 2026-07-21T16:20:25Z

+	ctx, cancel := context.WithCancel(context.Background())
+
+	var wg sync.WaitGroup
+	wg.Add(1)
+	go func() {


I would prefer to do the goroutine setup on the caller side and pass the context in here. This saves the returned function and makes it clear that the client runs another thread...

srebhan · 2026-07-21T16:21:01Z

+
+// startDiscovery runs node discovery immediately and repeats it when the interval is positive.
+// The returned function stops the loop and waits for any active call to return.
+func startDiscovery(log telegraf.Logger, interval time.Duration, discover func(context.Context) error) func() {


Can you please make log the last argument?! We usually do this in other places...

srebhan · 2026-07-21T16:24:05Z

+	if cfg.enableSniffer {
+		cfg.httpClient.CloseIdleConnections()
+		return nil, errors.New("enable_sniffer is not supported by the official ElasticSearch v5 client")
 	}


Can we please check this on the caller side?

srebhan · 2026-07-21T16:25:54Z

-		}
+	var result map[string]interface{}
+	if err := json.NewDecoder(res.Body).Decode(&result); err != nil {
+		return nil, err


Can we please prefix the error to easier find the spot when we get a bug-report?

Suggested change

return nil, err

return nil, fmt.Errorf("decoding message body failed: %w", err)

srebhan · 2026-07-21T16:30:20Z

 	}
+}
+
+func (e *ElasticsearchQuery) Start(telegraf.Accumulator) error {


Can we please merge this with newClient?!

srebhan · 2026-07-21T16:31:59Z

 				QueryPeriod:     config.Duration(time.Second * 600),
 			},
-			expected: "Error 404 (Not Found): no such index",
+			expected: "received error 404 (Not Found): no such index",


Please revert. The function should use require.ErrorContains which can perfectly handle prefixed errors. This makes the test more robust if we change the prefix later.

Same below.

srebhan · 2026-07-21T16:32:52Z

+	require.ErrorContains(t, err, `parsing server version "invalid" failed`)
+	require.NotContains(t, err.Error(), "getting server version failed")


Can this really occur? I thought those code paths are mutually exclusive.

fix: migrate plugin to official ES clients

04036dc

telegraf-tiger Bot added fix pr to fix corresponding bug plugin/input 1. Request for new input plugins 2. Issues/PRs that are related to input plugins labels Jul 15, 2026

alespour added 2 commits July 16, 2026 11:39

chore: update dependencies licenses file

19d43c9

fix: preserve error details

7c1dd27

alespour changed the title ~~fix(inputs.elasticsearch_query): migrate plugin code to official ES clients~~ fix(inputs.elasticsearch_query): migrate plugin code to official clients Jul 16, 2026

alespour added 2 commits July 16, 2026 12:45

fix: improve code comment

8f763fe

chore: trigger CI workflow

059f78e

alespour marked this pull request as ready for review July 16, 2026 12:01

srebhan changed the title ~~fix(inputs.elasticsearch_query): migrate plugin code to official clients~~ chore(inputs.elasticsearch_query): Migrate to official ES client Jul 16, 2026

telegraf-tiger Bot added the chore label Jul 16, 2026

srebhan reviewed Jul 16, 2026

View reviewed changes

srebhan self-assigned this Jul 16, 2026

srebhan added area/elasticsearch and removed fix pr to fix corresponding bug labels Jul 16, 2026

alespour added 4 commits July 17, 2026 15:30

refactor: simplify official client integration

e8f7e3a

fix: discovery

ac9d16e

test: add dicsovery test

da37742

fix: preserve server version error messages

b318ea4

alespour requested a review from srebhan July 20, 2026 15:07

srebhan reviewed Jul 21, 2026

View reviewed changes

	Transport: roundTripper{client: cfg.httpClient},
	Transport: cfg.httpClient.Transport,

	return nil, err
	return nil, fmt.Errorf("decoding message body failed: %w", err)

		require.ErrorContains(t, err, `parsing server version "invalid" failed`)
		require.NotContains(t, err.Error(), "getting server version failed")

Uh oh!

Conversation

alespour commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Node discovery caveats

Checklist

Related issues

Uh oh!

srebhan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alespour Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alespour Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

srebhan commented Jul 17, 2026

Uh oh!

alespour commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

telegraf-tiger Bot commented Jul 20, 2026

Artifact URLs

Uh oh!

alespour commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

srebhan commented Jul 21, 2026

Uh oh!

srebhan left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

alespour commented Jul 15, 2026 •

edited

Loading

alespour Jul 20, 2026 •

edited

Loading

alespour Jul 22, 2026 •

edited

Loading

alespour commented Jul 17, 2026 •

edited

Loading

alespour commented Jul 20, 2026 •

edited

Loading