Skip to content

Support Count and Python len() on user-facing lazy enumerables#9631

Merged
jhonabreul merged 7 commits into
QuantConnect:masterfrom
jhonabreul:feature-enumerables-python-len-support
Jul 21, 2026
Merged

Support Count and Python len() on user-facing lazy enumerables#9631
jhonabreul merged 7 commits into
QuantConnect:masterfrom
jhonabreul:feature-enumerables-python-len-support

Conversation

@jhonabreul

@jhonabreul jhonabreul commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

Description

Python.NET enables len() on any IEnumerable whose runtime type exposes a public Count property, but several user-facing APIs return lazy LINQ/yield enumerables, so Python algorithms get TypeError: object of type '...' has no len() and C# users force a full re-enumeration on every Count() call.

The change, one commit per area:

  • SecurityTransactionManager.GetOrders, GetOrderTickets and GetOpenOrderTickets (all overloads) wrap their results in MemoizingEnumerable, following the same pattern QCAlgorithm.History already uses. Declared return types stay IEnumerable<T>, so IOrderProvider is unchanged.
  • Other lazy enumerables reachable from QCAlgorithm are wrapped the same way: SubscriptionManager.Subscriptions, Cash.SecuritySymbols and ICurrencyConversion.ConversionRateSecurities (both implementations).

No interfaces are modified, so existing implementations (including external ones) are unaffected.

Engine usages of the wrapped members were audited so memoization adds no overhead to hot paths:

  • SecurityCache.GetAll<T> is intentionally NOT wrapped: fill models enumerate it once per open order on every fill scan, so memoizing would only add allocations and per-item locking.
  • SecurityTransactionManager.GetOpenOrdersRemainingQuantity and CancelOpenOrders enumerate a single time, so they go directly to the order processor and skip the memoizing wrapper.
  • Remaining engine callers (AlgorithmManager custom-data setup and split handling, QCAlgorithm.SetHoldingsImpl / Shortable) run once at startup or per order placement, not per data point, and enumerate once.

Related Issue

N/A

Motivation and Context

len() is the idiomatic way for Python algorithm authors to count orders, tickets or subscriptions; it should work on these APIs instead of requiring a list(...) conversion. Memoizing also makes repeated enumeration of a returned enumerable consistent (single snapshot) and cheaper.

Requires Documentation Change

No.

How Has This Been Tested?

  • SecurityTransactionManagerTests.OrderEnumerablesAreMemoizedExposingCount: asserts the returned enumerables are MemoizingEnumerable and Count matches the submitted orders.
  • SecurityTransactionManagerTests.OrderEnumerablesSupportPythonLen: asserts PyObject.Length() (the same mp_length slot as Python's len()) on all overloads, including the PyObject filter ones.
  • SubscriptionManagerTests.SubscriptionsAreMemoizedExposingCount, CashTests.SecuritySymbolsAreMemoizedExposingCount, SecurityCurrencyConversionTests.ConversionRateSecuritiesAreMemoizedExposingCount / ConstantConversionRateSecuritiesAreMemoizedExposingCount: assert type and Count for each wrapped API.
  • Full affected test classes (634 tests) pass locally; the single failure was InternalSubscriptionManagerTests.UniverseSelectionAddAndRemove, a live-mode timing test excluded from CI (TravisExclude) that passes on re-run and touches none of the changed code.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • Refactor (non-breaking change which improves implementation)
  • Performance (non-breaking change which improves performance. Please add associated performance test and results)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Non-functional change (xml comments/documentation/etc)

Checklist:

  • My code follows the code style of this project.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.
  • My branch follows the naming convention bug-<issue#>-<description> or feature-<issue#>-<description>

Wrap the enumerables returned by GetOrders, GetOrderTickets and
GetOpenOrderTickets in MemoizingEnumerable so they expose a Count
property, which also enables len() on them in Python algorithms.
Wrap SubscriptionManager.Subscriptions, SecurityCache.GetAll,
Cash.SecuritySymbols and ICurrencyConversion.ConversionRateSecurities
in MemoizingEnumerable so they expose a Count property, which also
enables len() on them in Python algorithms.
Exposes the count of stored objects without loading their content,
which also enables len() on the object store in Python algorithms.
@jhonabreul
jhonabreul marked this pull request as ready for review July 21, 2026 18:01
- Revert SecurityCache.GetAll memoization: fill models enumerate it once
  per open order on every fill scan, so memoizing only adds allocations
  and per-item locking on a hot path.
- SecurityTransactionManager.GetOpenOrdersRemainingQuantity and
  CancelOpenOrders go directly to the order processor since they
  enumerate a single time, keeping memoization only for the enumerables
  returned to the user.
Replace direct _orderProcessor calls with private GetOrderTickets/
GetOpenOrderTickets overloads taking a required memoize flag, so every
internal call site states whether the enumerable is single-pass (engine
paths) or user-facing (memoized). Also drop the redundant x => true
fallback: IOrderProvider documents null as match-all and every
implementation handles it.
Preserves the previous behavior where IOrderProcessor implementations
never received a null filter from the transaction manager.
@jhonabreul
jhonabreul merged commit 153d0b7 into QuantConnect:master Jul 21, 2026
7 of 8 checks passed
@jhonabreul
jhonabreul deleted the feature-enumerables-python-len-support branch July 21, 2026 21:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants