lib/logstorage: limit the total memory used by concurrently executed queries

[cuongleqq]

Closes #1551

Issue

• Some pipes can only use up to a fixed fraction of the memory available to VictoriaLogs (memory.Allowed()) for their state, even when there is still free memory available:
  • 40%: stats, running_stats, top, uniq
  • 20%: sort, facets, stream_context
• Each pipe has its own limit, separate from the others. When many queries run at the same time, each one can reach its limit, so the total memory gets too big and the process can be OOM killed.

Fix

• Add one memory pool shared by all running queries (50% of memory.Allowed()).
• The pipes now take memory from this shared pool instead of using their own separate limit. If the pool is full, the query fails with an error instead of being OOM killed.

Design

To keep it simple:

• All the pipes that have their own limits now draw the memory directly from the shared pool.
• Sub-queries initialization previously also had its own limit; it now reports the memory reserved back to where queries are initialized, so the memory is released after the query is released.

Future todo:

• Make subqueries for the in filter draw memory from the pool.
• Could support a per-query memory cap.

[cubic-dev-ai]

No issues found across 20 files

_{Re-trigger cubic}