Trading Bot System - LLM Guide

Trading Bot Framework — Research Briefing
This is a Python-based automated trading framework running on Kubernetes. Bots run as CronJobs, fetch market data, make decisions, and execute paper/live trades stored in PostgreSQL.

Bot Interface — Two Patterns

Pattern A: decisionFunction(row) -> int (simple, backtestable)

Override this method; the base class handles data fetching, looping, and execution
Called once per OHLCV row. Return 1 (buy), -1 (sell), 0 (hold)
Supports local_backtest(), local_optimize(), hyperparameter tuning

Pattern A1 — Single-ticker (symbol=):

The framework buys/sells self.symbol automatically

class MyBot(Bot): def init(self): super().init("MyBot", symbol="QQQ", interval="1d", period="1y")

  def decisionFunction(self, row) -> int:
      if row["momentum_rsi"] < 30:
          return 1
      elif row["momentum_rsi"] > 70:
          return -1
      return 0

Pattern A2 — Multi-ticker (tickers=[...]):

Pass tickers= instead of symbol=; decisionFunction is called per ticker per bar
Position sizing: equal-weight — each ticker targets total_portfolio_value / N
Fully backtestable via local_backtest() and local_optimize()

class MyMultiBot(Bot): def init(self): super().init("MyMultiBot", tickers=["SPY", "QQQ", "GLD"], interval="1d", period="1y")

  def decisionFunction(self, row) -> int:
      if row["momentum_rsi"] < 30:
          return 1   # buy this ticker toward equal-weight target
      elif row["momentum_rsi"] > 70:
          return -1  # sell all holdings of this ticker
      return 0

Pattern B: makeOneIteration() -> int (complex, not backtestable)

Override this method directly for multi-asset bots or external data sources
Manually call self.buy(symbol), self.sell(symbol), self.rebalancePortfolio(weights)
Use when: portfolio rebalancing across N symbols, signals come from a DB table, external API (Fear & Greed), AI agent flows

Data Available

Yahoo Finance OHLCV — via self.getYFDataWithTA(interval, period)

Returns a DataFrame with timestamp, open, high, low, close, volume + ~150 TA indicators
Intervals: 1m, 5m, 15m, 30m, 1h, 4h, 1d, 1wk, 1mo
Periods: 1d, 5d, 7d, 1mo, 3mo, 6mo, 1y, 2y, max (minute data capped at 60 days by Yahoo)
Multi-symbol: self.getYFDataMultiple(symbols, interval, period) — returns long-format DataFrame

Technical Indicators — via ta library (add_all_ta_features) All ~150 indicators are pre-computed and available as columns. Key ones:

Momentum: momentum_rsi, momentum_stoch, momentum_stoch_signal, momentum_macd, momentum_macd_signal, momentum_cci, momentum_williams_r
Trend: trendmacd, trend_macd_signal, trend_macd_diff, trend_sma_fast, trend_sma_slow, trend_ema_fast, trend_ema_slow, trend_adx, trend_adx_pos, trend_adx_neg, trend_ichimoku*, trend_aroon_up/down
Volatility: volatility_bbm, volatility_bbh, volatility_bbl, volatility_bbw, volatility_atr, volatility_kcp, volatility_dcp
Volume: volume_obv, volume_adi, volume_cmf, volume_fi, volume_mfi, volume_em, volume_vpt

PostgreSQL Tables

┌──────────────────────┬───────────────────────────────────────────────────┬────────────────────────┐ │ Table │ Contents │ Used by │ ├──────────────────────┼───────────────────────────────────────────────────┼────────────────────────┤ │ bots │ Portfolio state JSON per bot │ All bots │ ├──────────────────────┼───────────────────────────────────────────────────┼────────────────────────┤ │ trades │ Trade history (symbol, price, qty, isBuy) │ All bots │ ├──────────────────────┼───────────────────────────────────────────────────┼────────────────────────┤ │ run_logs │ Execution history, success/error │ All bots │ ├──────────────────────┼───────────────────────────────────────────────────┼────────────────────────┤ │ portfolio_worth │ Daily portfolio value snapshots │ Dashboard │ ├──────────────────────┼───────────────────────────────────────────────────┼────────────────────────┤ │ historic_data │ Cached OHLCV (avoids re-fetching) │ All bots │ ├──────────────────────┼───────────────────────────────────────────────────┼────────────────────────┤ │ stock_news │ Recent news headlines per symbol from yfinance │ StockNewsSentimentBot │ ├──────────────────────┼───────────────────────────────────────────────────┼────────────────────────┤ │ stock_earnings │ Earnings dates with before-open / after-close timing (after_close), EPS estimate vs actual, surprise%; upserted │ EarningsInsiderTiltBot, option bots (options.earnings_events) │ ├──────────────────────┼───────────────────────────────────────────────────┼────────────────────────┤ │ stock_insider_trades │ Insider buy/sell transactions │ EarningsInsiderTiltBot │ ├──────────────────────┼───────────────────────────────────────────────────┼────────────────────────┤ │ telegram_messages │ Telegram channel messages + AI summaries + symbol │ TelegramSignalsBankBot │ ├──────────────────────┼───────────────────────────────────────────────────┼────────────────────────┤ │ option_quotes │ yfinance option-chain snapshots (bid/ask/IV/OI/spot); daily capture of SPY, QQQ + top-50 S&P 100 from 2026-09-28 │ any bot using option=; future option backtests │ ├──────────────────────┼───────────────────────────────────────────────────┼────────────────────────┤ │ stock_fundamentals │ Daily point-in-time P/E, EV/EBITDA, FCF yield, ownership (S&P 100, from 2026-09-25) │ implied_correlation weights (utils/fundamentals.py) │ ├──────────────────────┼───────────────────────────────────────────────────┼────────────────────────┤ │ vol_surface │ Per name per day from the capture: constant-maturity ATM IV, 25d skew, term slope, HAR fair vol, VRP, put/call, GEX, max pain │ option scan / cross-vol / scanner bots (utils/vol_surface.py) │ ├──────────────────────┼───────────────────────────────────────────────────┼────────────────────────┤ │ implied_correlation │ SPY implied correlation vs the 50 captured names, daily │ option_DispersionBot │ ├──────────────────────┼───────────────────────────────────────────────────┼────────────────────────┤ │ option_mispricing_scan │ Daily ranked vol-mispricing list: vrp z-score, SVI outliers, parity flags │ scanner / cross-vol shortlists (utils/mispricing_scan.py) │ ├──────────────────────┼───────────────────────────────────────────────────┼────────────────────────┤ │ option_risk │ Per option bot and underlying, daily: $ greeks, spot/vol stress P&L, max loss, margin │ monitoring (utils/option_risk.py) │ ├──────────────────────┼───────────────────────────────────────────────────┼────────────────────────┤ │ macro_events │ FOMC / CPI / NFP dates, past and scheduled (FOMC static, CPI/NFP from FRED) │ option_IndexVolBot logging (utils/macro_calendar.py) │ ├──────────────────────┼───────────────────────────────────────────────────┼────────────────────────┤ │ dividend_events │ Ex-dividend dates and amounts, paid and next announced (daily, corporateeventssnapshot) │ options.next_dividend: early assignment, short-call bots │ ├──────────────────────┼───────────────────────────────────────────────────┼────────────────────────┤ │ corporate_event_refresh │ When each symbol's earnings/dividends were last refreshed; readers trust the tables only within 3 days │ utils/corporate_events.py │ ├──────────────────────┼───────────────────────────────────────────────────┼────────────────────────┤ │ split_events │ Stock splits (ratio, ex-date, pre-split close), hourly splitsnapshot scan of held + cached symbols │ utils/splits.py │ ├──────────────────────┼───────────────────────────────────────────────────┼────────────────────────┤ │ applied_splits │ One row per (bot, symbol, ex-date) once a split was applied to a book: never twice │ utils/splits.py │ ├──────────────────────┼───────────────────────────────────────────────────┼────────────────────────┤ │ bot_alpha_report │ Weekly alpha / t / beta / corr / max DD vs Benchmark_QQQ per live bot, with a verdict │ review (alphareport CronJob, utils/alpha_report.py) │ ├──────────────────────┼───────────────────────────────────────────────────┼────────────────────────┤ │ prediction_market_snapshots │ Daily Kalshi / Polymarket prices of curated macro contracts (Fed path + decisions, CPI, payrolls, U3, GDP, recession, shutdown), Kalshi from 2021-07 │ PredictionMarketOverlayBot, not scheduled (utils/prediction_market_features.py) │ └──────────────────────┴───────────────────────────────────────────────────┴────────────────────────┘

AI — via OpenRouter

Cheap model (default openrouter/free): self.run_ai_simple(system, user) — classification, extraction, single-turn
Main model (default deepseek/deepseek-v3.2): self.run_ai(system, user) — multi-turn with tools (portfolio lookup, market data, trade
history, news lookup)
Fallback wrapper: self.run_ai_simple_with_fallback(system, user, sanity_check) — cheap first, retries with main if output fails sanity check

Regime / Sentiment Utilities

utils.regime — detects bull/bear/sideways regime from price data
utils.ta_regime — TA-based regime via ADX + trend filters
utils.sentiment — fear/greed or other sentiment adapters

Tradeable Universe — tradingbot.utils.config.TRADEABLE

Pre-defined list of liquid ETFs/stocks suitable for the portfolio rebalancing bots

Prediction markets — utils/prediction_market_features.py

feature_frame(bar_dates) → point-in-time daily features: recession_prob, fed_next_bps, fed_path_bps (expected change of the fed funds upper bound by ~6 months out), fed_cut_next_prob, shutdown_prob, cpi_next_mean/std, plus Polymarket cross-checks.
u3_next_mean/std (KXU3 ladder) and event_std_z: how unusually wide the CPI / U3 distribution for the next print is. It is log(std) z-scored against the trailing year within days-to-release buckets. Market.event_uncertainty() hands it to option bots, and option_IndexVolBot logs it. As a gate or size-down it lost to the live rules out of sample (2026-09-29), so IndexVolRules.max_event_std_z / event_size_z stay None.
Bar D only sees prices covering day D-1 or earlier (a day's price is observed after its close).
Round 2 findings, all in docs/backtests/prediction-markets-2026-09.md:
- Kalshi's S&P range mispricing is in last-trade prices only. Taker rules at the bid/ask lose after fees; a maker upper bound is flat except in 2026 (onetime_prediction_market_calibration.py --executable).
- Polymarket tariff and election odds move with the ETFs they should move on the same day, but don't lead them (onetime_prediction_event_basket_study.py).
Venue trading (quoting, arbitrage, market making) belongs in a separate long-running module, not here. Its hook into this framework: own a bots row with portfolio {"USD": NAV}. The daily worth calculator then writes portfolio_worth and the weekly alphareport judges it (portfolio_worth.bot_name is an FK to bots).
Series are hand-curated in utils/prediction_market_series.py; never auto-discover markets. Kalshi's "recession in 2026" means two negative GDP quarters in 2025 or 2026.
API quirks (verified 2026-09-28): Kalshi markets settled before /historical/cutoff live only under /historical/..., whose candles use bare keys (price.close) instead of the live tier's close_dollars / volume_fp; legacy tickers (FED-22DEC-T4.75) carry their strike only in the suffix. Polymarket prices-history with interval=max&fidelity=1440 returns the whole daily history in one call (passing both startTs and endTs caps the window at 15 days).

Portfolio Operations

self.buy(symbol, quantityUSD=-1) # -1 = all cash self.sell(symbol, quantityUSD=-1) # -1 = all holdings self.buy("AAPL", option=True) # ~30-DTE ATM call on AAPL (paper only, see "Options") self.sell("AAPL", option=True) # close every AAPL option held self.rebalancePortfolio({"QQQ": 0.6, "GLD": 0.3, "USD": 0.1}) self.getLatestPrice(symbol) # float self.getLatestPricesBatch(symbols) # dict[str, float]

Portfolio state is a JSON dict in PostgreSQL: {"USD": 8432.10, "QQQ": 12.5, "GC=F": 0.03}.

Guardrails & Limitations

Hard limitations:

Event-driven bots (makeOneIteration only) cannot be backtested with the built-in engine. Multi-ticker decisionFunction bots (tickers=[...]) ARE backtestable via local_backtest() and local_optimize().
No short selling — sell() only sells existing holdings; going short is not supported.
No leverage — position sizing is bounded by available cash.
No fractional lot enforcement — the framework buys fractional quantities; fine for crypto/forex, may not reflect reality for equities.
Minute data capped at 60 days — Yahoo Finance hard limit for intervals ≤ 90m.
Backtest warmup skip — first ~26 bars are skipped when trend_adx == 0.0 (TA warmup period); strategies that need very few bars may lose
meaningful data.

Backtest realism (recently fixed):

Slippage: 0.05% per side (configurable via slippage_pct)
Commission: 0% default (configurable via commission_pct)
Risk-free rate: 0% default for Sharpe (configurable via risk_free_rate)

Live execution uses the same cost model (utils/config.py ExecutionConfig, env vars EXECUTION_SLIPPAGE_PCT / EXECUTION_COMMISSION_PCT). Until Aug 2026 PortfolioManager filled every paper trade at the raw mid price, so paper trading was more optimistic than the backtest of the same strategy — which flattered exactly the high-churn bots. Any portfolio_worth row before that changeover is cost-free and not comparable to later ones.

Convention, identical on both sides: on a BUY, quantity_usd is the gross cash budget and commission comes out of it (never on top), so "spend all cash" cannot overdraw; on a SELL, quantity_usd is position notional at the reference price, so the share count is invariant to slippage and a rebalance converges in one step. Trade.price records the execution price and Trade.profit the net proceeds — that field is a misnomer and has never been realized P&L.

The no-trade band is relative (EXECUTION_MIN_TRADE_USD, EXECUTION_REBALANCE_BAND_PCT): trade only if the adjustment exceeds max(min_trade_usd, band_pct * max(target, current)). Full exits always bypass it — otherwise a sub-band position could never be liquidated, and the only_over_50_usd filter (which closes positions by dropping them from the target) would become a no-op. New entries do not bypass. The old flat $1 threshold is why RegimeAdaptiveBot and EarningsInsiderTiltBot were putting 93% / 92% of their trades through at under $25 notional (smallest: $0.011).

No look-ahead bias — bfill() removed from TA computation
QuantStats reports: Automatically generated and uploaded to GCS (if credentials configured) showing Sharpe/return optimization views, drawdown analysis, and performance vs. buy-and-hold benchmark. Local backtest() and local_optimize() both produce reports. Example report

Practical constraints:

Bots run as Kubernetes CronJobs — no real-time streaming, no intra-bar execution
Minimum meaningful trade: quantityUSD > $10 (enforced in signal bots)
All times are UTC; market hours not enforced (strategy must handle weekends/holidays if needed). utils/market_calendar.py has the NYSE sessions, holidays and early closes (is_session, session_close_utc, sessions_between).
acted_on flag pattern is used for event-driven bots (Telegram signals, stock news) to prevent double-execution on crash

Common pitfalls:

decisionFunction is called once per historical row (~252 calls/ticker for 1y daily). Any external lookup (DB query, API call) inside it runs 252 times per ticker. Always cache per-ticker: check a dict before querying, store the result, reuse it for subsequent rows of the same ticker.
SQLAlchemy detached instance: ORM objects become inaccessible after their session closes. When querying inside get_db_session(), extract all needed values as plain Python types (float(), str(), etc.) before the with block exits. Never return an ORM object from a function that closes the session — attributes will raise DetachedInstanceError on access. The framework follows this itself. Since 2026-09-28:
- PortfolioManager keeps no row. Writers lock one inside their own transaction; readers take a dict (_portfolio()).
- Bot.dbBot is a property returning a fresh BotSnapshot(name, portfolio) on every access.
- There is nothing left to refresh by hand. Bind self.dbBot.portfolio to a local when one method needs a single consistent view.
- BotRepository.log_trade without a session returns None.

Existing Strategies (don't duplicate)

┌────────────────────────┬──────────────┬────────────────────────────────────────────────┐ │ Bot │ Asset │ Strategy │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ EURUSDTreeBot │ EURUSD │ Decision tree on TA indicators │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ XAUZenBot │ Gold (GC=F) │ Multi-indicator TA threshold rules │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ XAUAISyntheticMetalBot │ Gold │ AI agent with TA + market data tools │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ SwingTitaniumBot │ configurable │ Swing highs/lows detection on close │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ TARegimeBot │ configurable │ TA regime (ADX + trend) → buy/sell │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ FearGreedBot │ QQQ │ CNN Fear & Greed Index thresholds │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ RegimeAdaptiveBot │ multi │ AI decides allocation by market regime │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ AIHedgeFundBot │ multi │ Full AI hedge fund analysis → rebalance │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ AIDeepSeekToolBot │ configurable │ AI agent with full tool suite │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ EarningsInsiderTiltBot │ multi │ Equal-weight + tilt by earnings/insider scores │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ TelegramSignalsBankBot │ multi │ AI classifies Telegram signals → trade │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ StockNewsSentimentBot │ multi │ AI classifies news headlines → trade │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ SqueezeMomentumBot │ GLD │ EMA/MACD/RSI zone momentum on Gold ETF │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ option_LeapCallBot │ AAPL options │ 60-delta ~1.5y LEAP, >SMA200, trim at 1.5x │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ option_CreditSpreadBot │ AAPL options │ Bull put / bear call spread when IV/HV rich │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ option_IronCondorBot │ AAPL options │ 20-delta condor, $35 wings, 60 DTE, rich IV │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ option_CatalystCallBot │ AAPL options │ 75-delta call into earnings, sold day before │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ option_MispricingBot │ AAPL options │ IV vs HAR fair vol, gap explained; fly/straddle│ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ option_WheelBot │ AAPL + opts │ Cash-secured puts -> covered calls, physical │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ option_PMCCBot │ AAPL options │ 70-delta LEAP + monthly 30-delta short calls │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ option_EarningsCalendarBot │ AAPL options │ ATM call calendar into earnings (live-only) │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ option_CollarBot │ AAPL + opts │ Shares + 25-delta put / 25-delta call collar │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ option_IndexVolBot │ SPY options │ 10-delta condor, 10% wings, when IV >= HAR + 3 │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ option_CrossVolBot │ 50 names │ Condors on the names furthest above forecast │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ option_MispricingScanBot │ SPY/QQQ + 50 │ z-scored IV vs forecast, both ways, vega caps │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ option_EarningsCrushBot │ 50 names │ Iron fly over reports priced >= 1.25x history │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ option_DispersionBot │ SPY + top 10 │ Short SPY fly vs member straddles, rich corr │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ InstitutionalFlowBot │ S&P 100 │ Weekly top-N by institutional mandate filters + volume accumulation (utils/institutional_ta.py) │ ├────────────────────────┼──────────────┼────────────────────────────────────────────────┤ │ PredictionMarketOverlayBot │ SPY/QQQ/IEF/GLD │ NOT SCHEDULED: equity sleeve cut by Kalshi recession odds / hawkish Fed-path repricing; lost to an SMA200 ablation out of sample │ └────────────────────────┴──────────────┴────────────────────────────────────────────────┘

What a New Strategy Should Specify

Pattern: A (decisionFunction) or B (makeOneIteration)?
Symbol(s): single ticker or portfolio?
Interval + period: what data granularity?
Signal logic: what columns/data drive the decision?
Hyperparameter grid (optional): dict of lists for local_optimize() to tune
Schedule: how often to run (cron string)?

This document provides essential information for LLMs working with this trading bot codebase. It explains the architecture, the Bot class system, and how to effectively work with the code.

Repository Overview

This is an automated trading bot system that:

Fetches market data from Yahoo Finance
Executes trading strategies based on technical analysis
Manages portfolios and tracks trades in PostgreSQL
Runs as Kubernetes CronJobs on a schedule
Uses Helm charts for deployment

Architecture

Directory Structure

tradingbot/
├── utils/
│   ├── botclass.py      # Base Bot class (core functionality)
│   └── db.py            # Database models and session management
├── eurusdtreebot.py     # Example bot implementations
├── feargreedbot.py
├── swingtitaniumbot.py
└── ... (other bot files)

kubernetes/
└── helm/
    └── tradingbots/     # Helm chart for deployment
        ├── Chart.yaml
        ├── values.yaml  # Bot configurations
        └── templates/
            ├── cronjob.yaml
            ├── postgresql-secret.yaml
            ├── postgresql-deployment.yaml
            └── postgresql-service.yaml

Key Technologies

Python 3.12+ with type hints
PostgreSQL via SQLAlchemy ORM
yfinance for market data
ta library for technical analysis indicators
Kubernetes CronJobs for scheduled execution
Helm for deployment management

The Bot Class System

Core Concept

The Bot class (tradingbot/utils/botclass.py) is the foundation. All trading bots inherit from it and implement one of the following approaches, in order of preference from simplest to most complex:

1. Simplest (Preferred): Implement only `decisionFunction(row)`

When to use: Your strategy can be expressed as logic on a single data row with technical indicators
How it works: Base class fetches data, applies your function to each row, averages the last N decisions, and executes trades
Examples: xauaisyntheticmetalbot.py, xauzenbot.py, gptbasedstrategytabased.py, eurusdtreebot.py
Best practice: This is the recommended approach for most bots

2. Medium complexity: Override `makeOneIteration()` for custom data sources or simple custom logic

When to use: You need external APIs (e.g., Fear & Greed Index), custom data processing, or different timeframe handling
How it works: You control the entire iteration but still use base class methods for trading
Examples: feargreedbot.py (uses external API instead of market data)

3. Complex: Override `makeOneIteration()` for portfolio optimization or multi-symbol strategies

When to use: Portfolio rebalancing, multiple symbols, complex optimization algorithms, external data sources
How it works: Full control over data fetching, decision logic, and trade execution
Examples:
- sharpePortfoliooptWeekly.py (portfolio optimization with multiple assets)
- aihedgefundbot.py (reads trading decisions from external database and rebalances)
- telegramsignalsbankbot.py (reads telegram_messages table, classifies signals via AI, partial buys at 20% of cash per signal; uses acted_on column for crash-safe deduplication)

Bot Class Lifecycle

1. Bot.__init__(name, symbol, interval="1m", period="1d")
   ├── Creates/retrieves bot from database
   ├── Initializes portfolio with {"USD": 10000} if new
   ├── Sets up symbol and data cache
   └── Stores interval and period for data fetching

2. Bot.run()
   ├── Calls makeOneIteration()
   ├── Executes buy/sell based on decision
   └── Logs result to database (RunLog)

3. Bot.makeOneIteration() [default implementation]
   ├── Fetches data: getYFDataWithTA(saveToDB=True, interval=self.interval, period=self.period)
   ├── Gets decision: getLatestDecision(data) [applies decisionFunction to each row]
   └── Executes trade if decision != 0

`targetWeights` returns a target book, not a signal

A bot may override targetWeights(rows) -> dict[str, float] instead of decisionFunction, and the framework will size, execute and backtest it. Use it whenever the strategy sizes positions rather than merely signalling them — vol targeting, risk parity, cross-sectional ranking. decisionFunction can only say -1/0/1, which becomes equal weights; before this existed, expressing anything else meant makeOneIteration and forfeiting backtesting entirely.

The contract:

rows is the current bar for every ticker in self.tickers, benchmarks included — cross-sectional strategies must see the whole universe. History is self.datas[ticker], already truncated to bars at or before the current one.
Return long-only weights over tradeable tickers summing to <= 1.0. "USD" must not be a key — cash is the derived residual. An omitted ticker means weight 0, i.e. a full exit that bypasses the no-trade band. {} goes fully to cash.
Output is validated by Bot._coerce_target_weights, which clamps and logs rather than raising: over-1.0 sums are rescaled, negatives clamped to 0, non-finite dropped. A weight on a symbol the bot may not trade (a benchmark, or anything outside the universe) is dropped and NOT redistributed, so a buggy bot under-invests visibly instead of silently over-weighting its other legs.
Return None for "no rebalance this bar". The book is left exactly as it is — live returns before rebalancePortfolio, the backtest sets every target to its current value. {} cannot say this, because an omitted ticker is an exit. It is how a weekly strategy runs on daily bars: gate on the bar's date (row.name in the backtest, row["timestamp"] live — the two paths index differently) and return None off-schedule. See InstitutionalFlowBot.
MIN_UNIVERSE_COVERAGE (default 1.0): by default one ticker with no data skips the whole live run, since its absent row would otherwise read as an exit. A large-universe bot may lower it (InstitutionalFlowBot: 0.95); the missing tickers then sit out and any holding in them is pinned at its current weight. A missing benchmark always skips the run.

Precedence is targetWeights > decisionFunction > makeOneIteration, in both backtest_type and makeOneIteration's dispatch. A bot defining both is one whose weights function calls the other as a helper. Ticker count is irrelevant — a single-ticker targetWeights bot is legal and backtestable.

Live and backtest call the same targetWeights and the same _coerce_target_weights; only execution differs. That parity is pinned by tests/test_multi_ticker_iteration.py::test_target_weights_live_and_backtest_agree_on_one_bar.

Related: BACKTEST_PERIOD. _get_backtest_period("1d") is "1y", so a bot whose lookback approaches a year backtests on almost no evaluable bars and reports a confident number derived from a handful of trades. Any such bot must set BACKTEST_PERIOD (e.g. "max"). See TSMOMTrendBot.

Position-sizing semantics (easy to get wrong)

These are the rules for decisionFunction bots. Anything that reasons about what such a bot holds — a meta-bot, a live-trade copier, a backtest, an analysis script — has to reproduce these three rules exactly. They are not conventions; they are what the live code does.

1. decisionFunction returning 0 means "hold, but stay capped". A 0 leg is never funded — the signal is a statement about initiating exposure — but it IS trimmed back if it has drifted above one equal-weight sleeve, because that cap is a risk limit independent of the signal. 1 targets a full sleeve (buying up or trimming down to it); -1 exits fully and bypasses the no-trade band, so a sell signal can always close a position no matter how small.

Do not read 0 as "go to target": KronosTraderBot.decisionFunction returns 0 for "no prediction available", so funding those legs would make it buy every symbol it has no opinion about.

Multi-asset portfolios remain path-dependent — you cannot recompute holdings for a given date from that date's signals alone, you have to simulate forward.

2. Multi-ticker equal-weighting divides by len(self.tradeable_tickers). Pass benchmark_tickers= to Bot.__init__ for tickers that are loaded into self.datas for their data but never traded — GoldenButterflyMomBot carries SPY as its RRG baseline and declares benchmark_tickers=["SPY"]. Benchmarks are excluded from the divisor, are never asked for a decision, and any position held in one is liquidated. A bot that declares no benchmarks is unaffected.

(Before Aug 2026 the divisor was len(self.tickers), so GoldenButterflyMomBot topped out at 5/6 invested and permanently held >=17% cash.)

2b. The multi-ticker path reconciles the whole book in ONE transaction. _run_multi_ticker_iteration computes target weights via the pure _multi_ticker_target_weights() and makes a single rebalancePortfolio call. It previously issued one buy/sell transaction per leg in ticker order, and since PortfolioManager.buy silently clamps to available cash and never retries, a sell that sorted after a buy never funded it — silent under-investment that persisted until a later run happened to order things favourably. rebalance_portfolio executes all sells before any buy, which fixes this structurally. Holdings in symbols outside self.tickers are left untouched by default (set LIQUIDATE_UNTRACKED = True to sell them); KronosTraderBot rebuilds its universe from the predictions table each run, so a one-day gap must not liquidate the book.

3. Single-asset bots are all-in / all-out. The default makeOneIteration calls buy(symbol) with no amount (spends all cash) or sell(symbol) with no amount (dumps the whole position). There is no partial sizing, so a non-zero holding always means "fully invested". PortfolioManager.sell deletes holdings below 1e-6, so any surviving quantity is a real position rather than rounding residue — qty > 1e-6 is the correct "is this bot in the market?" test.

`historic_data` is keyed by interval — keep it that way

HistoricData's primary key is (symbol, interval, timestamp). Every read must filter on interval and every write must set it. HistoricDataRepository enforces this: get_range and get_latest_timestamp require an interval, and bulk_insert_ohlcv rejects rows without one.

This was not always so. The key used to be (symbol, timestamp), so every bar size a bot ever fetched for a symbol landed in one pile. ^XAU was the worst case: xauzenbot runs 1m every 5 minutes with saveToDB=True, so the table held ~74k one-minute rows for it and, verifiably, zero daily rows. Asking for interval="1d", period="10y" returned those 1-minute bars merged with whatever yfinance sent — mixed granularity, wrong window. Nothing raised; the TA indicators were simply computed across the mixture. A 10-year "daily" backtest of xauzenbot came back with 76,093 bars and a 93% in-market rate against a recorded ~10%, which is how it was found. Two paths were broken at once, and the second is the subtle one: add_pd_df_to_db only inserts bars newer than the stored high-water mark, and unscoped that mark was the newest 1-minute bar — always minutes old — so every daily bar looked stale and was silently dropped. That is why no daily row ever accumulated.

Legacy rows were backfilled by inferring each row's bar size from the smallest gap to an adjacent bar of the same symbol. Time-of-day does not work as a proxy: 69 symbols carry daily bars stamped at session open rather than midnight. _migrate_schema() in db.py replays that same inference, guarded on the PK still being the old 2-column form, so it is a no-op on a migrated or fresh DB.

Regression tests: tests/test_historic_interval.py.

Interval and backtestability are strategy properties, not settings

A bot's thresholds are calibrated to the bar size it was tuned on, so running its decisionFunction on a different interval yields a different strategy wearing the same name. XAUZenbotTreeBot defaults to interval="1m" (it passes no interval to super().__init__) and its atr_threshold=0.08 sits just under ^XAU's ~0.21 median 1-minute ATR. On daily bars the median ATR is ~3.8, so that branch can never fire and the tree collapses into a mostly-always-long rule.

The practical consequence: a bot on a 1m interval cannot be backtested beyond yfinance's ~7-day intraday window (_get_backtest_period returns "7d" for minute intervals for exactly this reason). Any strategy combining a 1m bot with one needing long daily history — as XAUZenCarryBot does, since GoldenButterflyMomBot's RRG needs 252 daily bars — has no shared timeframe on which a joint backtest exists. Evaluate those from recorded live holdings instead (scripts/onetime_simulate_zencarry.py), and be patient about sample size rather than manufacturing a long history that measures something else.

Options (paper, opt-in)

Off unless a bot asks. A bot names the underlying and the framework handles the contract (tradingbot/utils/options.py):

self.buy("AAPL", option=True)  # call; option="put" for a put
self.sell("AAPL", option=True)  # all AAPL options; "call"/"put" for one side


class MyBot(Bot):
    USE_OPTIONS = True  # plain buy()/sell() of self.symbol trade calls instead
    OPTION_TARGET_DTE = 30  # buy the first expiry at least this many days out
    OPTION_ROLL_DTE = 7  # roll a held contract with this few days left

Selection: first expiry >= OPTION_TARGET_DTE, strike nearest spot, preferring contracts with a live two-sided market.
Storage: the portfolio key is the OCC symbol (AAPL261030C00200000), and the quantity is in share-equivalents (contracts x 100) at the per-share premium. Every qty * price valuation stays correct without a multiplier. Buys round down to whole contracts, so a $10k book gets few, chunky positions.
Pricing and fills: each snapshot is stored in option_quotes, refetched when older than 15 min. Value is the bid/ask mid. Buys fill at the ask and sells at the bid. Outside regular hours yfinance's bid/ask are zeroed or stale, so they are stored as NULL and fills use last price ± EXECUTION_OPTION_SLIPPAGE_PCT (default 2%).
Expiry: Bot.run() first rolls contracts within OPTION_ROLL_DTE into a fresh one on the same side. A contract found already expired settles off the underlying's close on the expiry date:
- by default, in cash at intrinsic value;
- with OPTION_SETTLEMENT = "physical" (the wheel), into shares at the strike, as a broker would. A short put is assigned stock and a short call delivers it. Any part that would leave the book short stock is cash-settled instead.
Guards:
- buy(<OCC symbol>) raises.
- rebalancePortfolio refuses contract targets, and a held contract missing from the target is sold like any other exit.
- USE_OPTIONS on a multi-ticker or targetWeights bot raises in __init__.
Live copier: it drops option holdings before mapping, so they never reach a broker. SymbolMapper would otherwise type an OCC symbol as "stock". It also drops any negative quantity and the shares of an underlying the same bot holds options on: a covered call's stock or a straddle's hedge, copied alone, would be a naked stock bet. The weight stays cash or goes to SHV.
Backtests hold the underlying, not the option. yfinance has no historical chains; option_quotes is the only history there will ever be.
Daily capture builds that history. The optionchainsnapshot CronJob (19:45 UTC on weekdays, utils/option_capture.py) stores one live snapshot a day for universes.OPTION_CAPTURE_UNIVERSE: SPY, QQQ and the 50 largest S&P 100 names, frozen on 2026-09-26.
- Per name: the ~7–9 listed expiries nearest to 7, 14, 30, 45, 60, 90, 180, 365 and 540 days, with strikes within moneyness_band(T) of spot (±16% at a week, ±25% at a month, ±60% at a year).
- About 35k rows a day, or ~2–2.5 GB a year. The node disk had 18 GB free at launch.
- A closed market writes nothing and exits 0. Every row carries underlying_price.
- To add names, append to the tuple. Never re-rank it: a name that drops out would leave a gap in its history.
New columns sync automatically. db._sync_missing_columns runs in init_db and adds any model column missing from an existing table, rendering the DDL from the model. New columns must be nullable or carry a server_default. _migrate_schema is only for what that cannot express.

Strike by delta, and short legs (defined risk only). The bot still never names a contract:

self.buy("AAPL", 20_000, option="call", delta=0.70, dte=540)  # 70-delta LEAP
self.open_credit_spread("AAPL", "bull", short_delta=0.30, width=10, dte=35, max_risk_usd=25_000)
self.open_iron_condor("AAPL", short_delta=0.16, width=10, dte=35, max_risk_usd=25_000)
book = self.option_book("AAPL")  # legs, credit, pnl, pnl_pct, dte, max_loss, net greeks
self.close_options("AAPL")  # every leg, long and short, one transaction

Short legs are negative quantities, opened only through PortfolioManager.trade_option_legs, which runs in one locked transaction. Valuation is qty * price as everywhere else, so a short leg counts as a liability. calculate_portfolio_worth includes negative holdings for exactly this reason.
Margin: options.margin_requirement(portfolio) is the worst-case expiry payoff of the option legs plus the underlying's shares. A share counts as a zero-strike call (option_math.stock_leg), which gives:
- a spread's width;
- a condor's wider wing;
- a cash-secured put's strike;
- 0 for a covered call or a collar;
- a bounded amount for short stock beside long calls (a hedged straddle);
- infinite for a naked short call or naked short stock.
A trade that grows any position is refused whole if cash left < margin. That is how unbounded structures are rejected. buy() can never spend reserved margin, and sell() refuses shares that cover a short call. Pure reductions always go through. Expiries are ignored: every leg is taken at its own expiry payoff, which is conservative for long-far / short-near calendars and diagonals.
Shares inside a structure: trade_option_legs also trades the underlying's own symbol, but only beside options on it. That covers buy-writes, collars and delta_hedge, and it is the only way to go short stock. In a StructurePick the underlying's leg counts lots of 100 shares.
Opening needs a live chain: a structure is not opened off-hours. Legs' last prices are from different moments, so the credit would be fiction. Schedule option bots during US market hours.
Rolling: OPTION_ROLL_DTE = None disables it. An underlying with any short leg is never auto-rolled, because rolling one wing alone would uncover the short. Expired legs of either sign still cash-settle at intrinsic value.
Other Bot attributes:
- OPTION_TARGET_DELTA picks the strike by delta for buy(option=...) and rolls.
- INITIAL_CAPITAL is the starting cash when the bot's row is created. The option bots use $100k so whole-contract sizing is sane.
rebalancePortfolio refuses a book holding short legs.
IV is solved from prices, not taken from yfinance. Off-hours yfinance reports impliedVolatility = 1e-5 for the whole chain. options.load_chain, atm_iv and with_greeks give IV and delta per contract.
- The dividend yield enters as Merton's q (options.dividend_yield).
- atm_iv(view, american=True) re-solves the put on a binomial tree.

Round 2 building blocks:

view = options.load_chain("AAPL", 35)
self.open_structure(options.select_iron_butterfly(view, width=25), max_risk_usd=20_000)
self.open_structure(options.select_straddle(view), 10_000)
self.delta_hedge("AAPL", band_usd=2_000)  # shares to ~0 net delta (gamma scalping)
self.close_options("AAPL", include_stock=True)  # options and the hedge shares
options.select_short_leg(view, "P", 0.30)  # cash-secured put; "C" + min_strike = covered call
options.select_collar(view, 0.25, 0.25, with_stock=False)
options.select_calendar(front_view, back_view)  # short front, long back, same strike
options.select_diagonal(leap_view, view, 0.70, 0.30, min_short_strike=k)  # poor man's covered call

Other helpers:

options.earnings_history and next_earnings_date.
options.recent_news(u, days): refreshes stock_news for the symbol first.
options.smile_outliers(view): contracts off the fitted smile by more than their half-spread.
options.opened_on and options.structure_flows: a structure's P&L, including what its hedge shares realized.

utils/option_math.py holds the pure formulas:

Black-Scholes-Merton bs_price and the five greeks (theta per day, vega and rho per point);
implied_volatility, strike_for_delta;
historical vol, iv_hv_ratio, iv_rank, iv_percentile;
breakeven, expected_move, probability_itm;
multi-leg payoff_at_expiry, max_loss, max_profit, breakevens, probability_of_profit, stock_leg;
beta, beta_exposure, delta_dollars, and liquidity screens;
fair vol:
- har_rv_forecast: HAR-RV over the days to expiry, with earnings days excluded;
- ewma_volatility;
- earnings_reaction_returns and earnings_jump;
- implied_earnings_move: from one expiry spanning the report plus a base vol, or from two expiries;
- fair_volatility;
binomial_price and implied_volatility_american: a CRR tree for American options;
fit_smile and smile_z.

The option bots (paper, scheduled 15:00–16:00 UTC; EarningsCrush at 14:30, 16:30 and 19:30):

option_LeapCallBot, option_CreditSpreadBot, option_IronCondorBot, option_CatalystCallBot.
Round 2: option_MispricingBot, option_WheelBot, option_PMCCBot, option_EarningsCalendarBot, option_CollarBot.
All of those trade AAPL. option_IndexVolBot trades SPY.
Round 3 trade many underlyings from one book: option_CrossVolBot, option_MispricingScanBot, option_EarningsCrushBot, option_DispersionBot.
Their rules are pure functions in utils/option_rules.py, shared with scripts/onetime_option_bots_backtest.py (and scripts/onetime_index_vol_backtest.py for the index bot).
The index bot and the four round-3 bots go further. Their whole iteration is one pure decision in utils/option_strategies.py, and live, replay and synthetic backtests all execute the same function (see "Round 4" below).
File and Helm names are option_<x>bot. The CronJob template turns _ into -, because Kubernetes names forbid underscores.
Synthetic backtest (Black-Scholes on a ^VXN-based IV proxy with AAPL skew), with a walk-forward re-tune on 2026-09-25 (grids chosen on 2012–2019, judged on 2019–2026):
- LEAP: re-tuned to 0.60Δ with a trim at 1.5x. Out-of-sample t 2.10, but that is mostly AAPL picked with hindsight.
- Iron condor: re-tuned to $35 wings, 60 DTE. It went from t −2.42 to about 0 (break-even).
- Credit spread: no variant survives out of sample. Paused (suspend: true) on 2026-09-26.
- Catalyst: untuned (10 trades).
- Details are in docs/backtests/option-bots-2026-09.md.
Re-priced on real AAPL chains, 2026-09-30 (docs/backtests/option-aapl-real-calibration-2026-09.md):
- Level: the VXN proxy priced AAPL options ~17% too rich. Real IV / proxy is 0.825; the median IV/HV20 is 1.02, not 1.24.
- Shape: calls sit slightly below ATM vol (they had been flat). Expiries that span a report carry a ~4.3% jump.
- Defaults: --calibration real is now the default; original reproduces the older docs.
- Selling bots weaken (out-of-sample t): IronCondor −0.59, with no variant surviving; Wheel 0.85; PMCC 1.33. Mispricing rises to 1.62.
- Still optimistic: the model is unbiased on average but off by ~3 vol points on a typical day. Its wing spreads are 1.5–1.8× tighter than real ones. At real spreads (OPTION_COST_SCALE=1.6), Mispricing falls to H2 t 0.99; the wheel holds (1.44). Treat synthetic t as an upper bound.
- The re-tune: the wheel consensus (40Δ put, 20Δ call, puts only at IV/HV ≥ 1.1) beats its defaults in both halves. Shipped in OptionWheelBot.RULES.
- Data fix: pre-split AAPL rows in option_quotes had split-adjusted spots beside unadjusted strikes (fixed in place).
Round 2, walk-forward on 2026-09-26 (docs/backtests/option-bots-round2-2026-09.md):
- Mispricing: trades IV minus HAR fair vol, after explaining the gap (earnings, market-wide vs AAPL-specific, pending news, VIX).
  - The gap does predict the variance premium.
  - Short at-the-money structures pay for it in single-stock jumps.
  - Re-tuned to 10-point gaps and 1.5σ wings: out-of-sample t 0.82. Unproven.
- PMCC: re-tuned to a 0.70Δ LEAP; out-of-sample t 2.37 vs the LEAP bot's 2.10, with a lower drawdown. Still long AAPL.
- Wheel and collar: defaults kept. Both are AAPL at a lower beta, not alpha. (The wheel was re-tuned on 2026-09-30 on real AAPL pricing; see below.)
- Earnings calendar: live-only, because nothing can price historical earnings IV.
- Not built, with reasons given in the doc: 0DTE SPX, box spreads, dispersion, tail hedges, naked strangles, long straddles into earnings.
Index vol, walk-forward on 2026-09-26 (docs/backtests/index-vol-2026-09.md):
- option_IndexVolBot sells SPY iron condors, but only while ATM IV is at least 3 points above a HAR forecast of SPY's vol.
- Its backtest implied vol is observed rather than proxied: ^VIX × 0.85, with put and call skew calibrated on the live chain. It covers 2000–2026.
- Result: +2%/yr alpha, t 4.28, beta 0.01, max DD −6.4%, positive in both halves.
- It holds at double costs, on QQQ and with half the skew.
- It is the first option bot with uncorrelated alpha, though the alpha is small.
- The single best parameter set from the first half failed out of sample; the consensus of the first half's top 10 ships instead.
- On real SPY option prices (2026-09-29) the edge is gone (docs/backtests/index-vol-real-surface-2026-09.md):
  - 2019–2026: t 0.45; since early 2024: t −1.01. The synthetic said 3.23 over the same window.
  - The synthetic's call skew, calibrated on one calm day, puts 10-delta short calls 9% out of the money. Real ones sit at 7%, and the 2020 and 2022 rallies ran through them.
  - The test is onetime_option_replay_backtest.py --live-expiries over the DoltHub backfill. utils/option_surface.py prices the legs a day did not record off that day's surface.
  - Recalibrated on those chains, the synthetic SPY model's call skew rises with vol: 0.284 + 0.246 ln(ATM/0.20), not a constant 0.08. It is now the default (--calibration original reproduces the old numbers).
  - On it, the live rules give t 0.99 over 2007–2026. The 2000–2026 walk-forward finds no variant that survives (consensus t 0.05; defaults −2.10).
Round 3, 2026-09-28 (docs/backtests/option-round3-2026-09.md):
- Data:
  - utils/vol_indices.py: the VIX family and the term ratio.
  - utils/macro_calendar.py: FOMC static; CPI and NFP from FRED, which needs FRED_API_KEY, an optional secret key.
  - vol_surface, implied_correlation and option_mispricing_scan are derived after every capture. python -m tradingbot.optionchainsnapshot --backfill rebuilds them.
- Risk:
  - option_math.stress_pnl and options.stress_book give spot×vol grids; optionrisksnapshot writes option_risk daily.
  - A short ITM call whose time value is below the next dividend is assigned early the session before the ex-date (PortfolioManager.assign_before_dividends).
  - Wheel, PMCC, collar and the new bots close such calls two sessions before (Bot.dividend_threatened_calls).
- Math:
  - utils/vol_estimators.py: Yang-Zhang, Parkinson, Garman-Klass, Rogers-Satchell, HAR on any daily variance, GARCH(1,1), VRP z-score, Whalley-Wilmott band.
  - utils/svi.py: SVI with butterfly and calendar arbitrage checks.
  - options.parity_violations: the American parity band, used as a bad-quote filter.
  - options.pick_metrics: price, vega, gamma and max loss of a pick before it is opened.
- Hedging: Bot.delta_hedge(..., ww=(cost, risk_aversion)) rehedges to the edge of a Whalley-Wilmott band. MispricingBot uses it (H2 t 0.88 vs 0.82 for the fixed band).
- IndexVolBot gates, all rejected out of sample: VIX/VIX3M, VVIX, FOMC blackout, term-structure unwind, Yang-Zhang and GARCH forecasts, the z-scored signal, and a tail hedge.
  - The z-score loses to the raw 3-point gap on SPY (H2 t 0.98 vs 2.64).
  - Exception, shipped for drawdown: a VIX/VIX3M ≤ 1.0 entry gate (max_term_ratio=1.0).
    - It has lower H2 t (2.65 vs 3.46).
    - It halves max DD (−2.7% vs −6.3%) and keeps t ≥ 2 in both halves.
    - This overrides the walk-forward rule on purpose.
- The four new bots are live-only and unproven. Single-name option history does not exist yet.
  - utils/option_replay.py and scripts/onetime_option_replay_backtest.py replay stored chains at their recorded bid/ask once option_quotes has months, captured or imported with the same columns.
Round 4, 2026-09-28: the framework around the option bots.
- One decision path. utils/option_decide.py defines decide(market, holdings, rules) -> [Open | Close | Hedge].
  - The strategies live in utils/option_strategies.py (decide_indexvol, decide_crossvol, decide_earningscrush, decide_mispricingscan, decide_dispersion).
  - Three executors run the same function:
    - live: OptionStrategyBot, which opens by worst-case loss capped by free cash;
    - replay: option_replay.run_strategy, the same sizing (options.units_for_risk);
    - synthetic: onetime_index_vol_backtest.py --decide, index vol only.
  - The hand-written replay loops had drifted:
    - no inversion unwind in the index-vol loop;
    - close-to-close HAR where the cross-vol bot uses Yang-Zhang;
    - no front-expiry check in the earnings-crush loop.
  - Parity: tests/test_option_decide_parity.py.
  - Running the live IndexVol decision on the synthetic chain found a drift. The bot fitted HAR on 5 years of closes; every backtest uses all history. That alone took gated t from 4.13 to 1.35. Live now fetches period="max", and the shipped rules reproduce: t 4.13 / H1 2.66 / H2 3.26, max DD −2.4% (docs/backtests/option-round3-2026-09.md).
  - The nine AAPL round-1/2 bots are not migrated. Their synthetic sims carry the walk-forward numbers the docs cite, and porting them would re-baseline those numbers with no new data to judge them on.
- OptionUniverseBot: a base for bots trading many underlyings from one book, set up with UNIVERSE, RULES and DECIDE. It needs no dummy symbol.
- Earnings timing and dividends in the DB.
  - corporateeventssnapshot runs at 12:00 UTC on weekdays. It fills stock_earnings.after_close and dividend_events for the option universe, held underlyings and AAPL.
  - options.earnings_events / next_earnings_date / earnings_history / next_dividend read the tables when the symbol was refreshed within 3 days, and fall back to yfinance with a warning otherwise.
  - Earnings rows are upserted, so actual EPS arrives once reported.
- Half days. utils/market_calendar.py wraps XNYS.
  - Business-day counts in option_rules.business_days and options.business_days_between are NYSE sessions.
  - The synthetic scripts keep plain weekdays (option_rules.weekdays) so their documented numbers reproduce.
  - optionchainsnapshot runs at 16:45 and 19:45 UTC and captures within 90 minutes of that day's close. On an early close (13:00 New York), the 16:45 run captures.
  - EarningsCrush enters within 2 hours of the close.
- Weekly alpha report. The alphareport CronJob runs Saturdays at 07:00 UTC and writes bot_alpha_report.
  - It applies the CLAUDE.md formula to every live bot, against Benchmark_QQQ, dropping weekends and >4-day gaps.
  - backtest._compute_alpha_metrics calls the same alpha_stats.
  - t ≤ −2 is logged as a pause candidate. Nothing is paused automatically.
- No ORM row is held.
  - Bot.dbBot is a property returning a fresh BotSnapshot on every access.
  - PortfolioManager locks the row inside each transaction and reads dicts otherwise.
  - This removed a latent DetachedInstanceError on partial option sells.

Stock splits

Portfolios store share counts, so a split must rescale them or the book drops (or, for a reverse split, jumps) by the ratio on the ex-date. Until 2026-09-30 nothing did: VGT's 8:1 split on 2026-04-21 wiped 7/8 of the VGT position from EarningsInsiderTiltBot (−$142) and RegimeAdaptiveBot (−$72), and both then re-bought target weight. Refunded in cash on 2026-09-30, with their portfolio_worth rows from the ex-date restated (scripts/onetime_repair_split.py). Everything lives in utils/splits.py:

Detection: the splitsnapshot CronJob (hourly at :20, 11–21 UTC, weekdays) scans every held symbol and every symbol with cached bars from the last 30 days in one batched yfinance call and writes split_events.
Books: apply_splits(bot) runs first thing in Bot.run, in the portfolio-worth job before valuing, and in the live copier before it reads a bot. It scales the quantity held at the ex-date, reconstructed as the current count minus trades since, so a late application is exact: a bot that "bought the dip" keeps what it bought and gets its missing shares back. applied_splits stops a second application. Only splits within 14 days are applied automatically; an older miss needs a deliberate repair.
Options: a whole-number split re-keys held contracts to strike / ratio and ratio x the quantity, as OCC does. Any other ratio raises SplitError (a non-standard deliverable the book cannot hold), failing the run loudly.
historic_data: yfinance serves split-adjusted bars but the cache only appends, so bars written before a split stay at the old scale (VGT read 809 -> 101). adjust_history finds the prefix of old-scale bars by the jump that matches the ratio, usually but not always at the ex-date, and divides it. It is idempotent.
Not handled: dividend adjustments. They drift cached bars slightly but touch no share counts.

Reading another bot's state

Bots persist their portfolio to bots.portfolio on every run, which makes a parent's live signal readable without recomputing it. Two rules:

Use BotRepository.read_portfolio(name), not create_or_get_bot(name) — the latter materialises a fresh $10k bot row for a name that does not exist, so a typo turns into a silent all-cash "signal".
Check BotRepository.last_successful_run(name) before trusting it. A parent whose CronJob has died leaves its last traded position in the row forever, and a fossil position is indistinguishable from a live one. See tradingbot/xauzencarrybot.py for the pattern (abort loudly on a stale parent rather than trading it).

Key Bot Class Methods

Data Fetching

# Fetch raw market data
data = bot.getYFData(interval="1m", period="1d", saveToDB=True)
# Returns: DataFrame with columns [symbol, timestamp, open, high, low, close, volume]

# Fetch data with technical analysis indicators
data = bot.getYFDataWithTA(interval="1m", period="1d", saveToDB=True)
# Returns: Same DataFrame + ~150+ TA indicators (RSI, MACD, Bollinger Bands, etc.)
# Indicators are prefilled/backfilled to handle NaN values

Important: Data is cached in self.data based on (interval, period) tuple. If you call with the same settings, it returns cached data.

Decision Making

# Standard approach: Implement decisionFunction
def decisionFunction(self, row: pd.Series) -> int:
    """
    Args:
        row: Single row from DataFrame with all TA indicators

    Returns:
        -1: Sell signal
         0: Hold (no action)
         1: Buy signal
    """
    if row["rsi"] < 30:
        return 1  # Oversold, buy
    elif row["rsi"] > 70:
        return -1  # Overbought, sell
    return 0


# The base class then:
# 1. Applies decisionFunction to each row: data.apply(self.decisionFunction, axis=1)
# 2. Takes the mean of the last N rows (default: 1)
# 3. Returns -1, 0, or 1

When to override makeOneIteration():

You need external data sources (e.g., Fear & Greed Index API)
You need portfolio optimization with multiple symbols
You need custom data processing beyond what decisionFunction can handle
See feargreedbot.py (external API) and sharpePortfoliooptWeekly.py (portfolio optimization) for examples

Trading Operations

# Buy with all available cash
bot.buy(symbol="QQQ")

# Buy specific USD amount
bot.buy(symbol="QQQ", quantityUSD=1000)

# Sell all holdings
bot.sell(symbol="QQQ")

# Sell specific USD amount
bot.sell(symbol="QQQ", quantityUSD=500)

Important:

buy() and sell() automatically update the portfolio in the database
They log trades to the trades table
Portfolio is stored as {"USD": 10000, "QQQ": 5.5, ...} in the bots table

Portfolio Management

# Access portfolio
cash = bot.dbBot.portfolio.get("USD", 0)
holding = bot.dbBot.portfolio.get("QQQ", 0)

# dbBot is read fresh from the database on every access (a plain BotSnapshot,
# never an ORM row), so it already reflects the last buy/sell. Writing to it
# persists nothing: trade through buy/sell/rebalancePortfolio/trade_option_legs.

Price Fetching

# Get latest price (uses cached data if available, otherwise fetches fresh)
price = bot.getLatestPrice(symbol="QQQ")

Note: If self.datasettings == ("1m", "1d") and data is loaded, uses cached data. Otherwise fetches fresh from yfinance.

Telegram Channel Monitor

A standalone channel monitor that is not a Bot subclass — it uses Telethon directly.

Architecture

tradingbot/telegram_monitor.py: entry point — parses env vars, connects via Telethon, calls module-level functions
helm/tradingbots/templates/cronjob-telegram-monitor.yaml: optional Helm CronJob, gated on telegramMonitor.enabled

How it runs

Stateless CronJob pattern: connect → fetch last N messages per channel → skip known IDs → summarize new ones with AI → disconnect. No persistent process.

Session is stored as a Telethon StringSession in the TELEGRAM_SESSION_STRING K8s secret.

Key env vars

Variable	Purpose
`TELEGRAM_API_ID`	From my.telegram.org
`TELEGRAM_API_HASH`	From my.telegram.org
`TELEGRAM_SESSION_STRING`	Telethon StringSession
`TELEGRAM_CHANNELS`	Comma-separated channel usernames or IDs
`TELEGRAM_FETCH_LIMIT`	Messages to check per channel per run (default: 50)

AI summarization

summarize_message(text) calls run_ai_simple (cheap LLM) with a prompt that returns JSON:

{ "summary": "1-3 sentence summary...", "symbol": "AAPL" }

Falls back to raw response as summary if JSON parsing fails. symbol is null when no specific asset is mentioned.

Database model: `TelegramMessage`

class TelegramMessage(Base):
    id: int  # Auto-increment primary key
    channel: str  # Channel username or numeric ID (indexed)
    message_id: int  # Telegram message ID — unique per channel
    text: str  # Original text (nullable, max 4000 chars)
    summary: str  # AI summary (nullable)
    symbol: str  # Primary ticker extracted by AI (nullable, indexed)
    acted_on: bool  # Set True before any trade action — prevents duplicate processing
    published_at: datetime  # UTC posting time
    created_at: datetime
    # Unique constraint: (channel, message_id)

Enabling in values.yaml:

telegramMonitor:
  enabled: true
  schedule: '*/30 * * * *'
  channels: 'some_channel,-1001234567890'
  fetchLimit: '50'

Database Models (tradingbot/utils/db.py)

Bot Model

class Bot(Base):
    name: str (primary key)
    description: str (optional)
    portfolio: dict (JSON, default: {"USD": 10000})
    created_at: datetime
    updated_at: datetime

Trade Model

class Trade(Base):
    id: int (auto-increment)
    bot_name: str (foreign key to Bot.name)
    symbol: str
    isBuy: bool
    quantity: float
    price: float
    timestamp: datetime
    profit: float (nullable, for sells)

HistoricData Model

class HistoricData(Base):
    symbol: str (primary key)
    timestamp: datetime (primary key)
    open: float
    high: float
    low: float
    close: float
    volume: float

RunLog Model

class RunLog(Base):
    id: int (auto-increment)
    bot_name: str (foreign key to Bot.name)
    start_time: datetime
    success: bool
    result: str (nullable, contains decision/error info)

PortfolioWorth Model

class PortfolioWorth(Base):
    bot_name: str (primary key, part of composite)
    date: datetime (primary key, part of composite)
    portfolio_worth: float   # total value in USD on that day
    holdings: dict (JSON)     # snapshot e.g. {"USD": 0, "QQQ": 16.43}
    created_at: datetime

Written daily by the portfolio-worth-calculator CronJob. This is the live per-bot track record — one daily valuation row per bot. Benchmark_QQQ, Benchmark_SPY, Benchmark_FTWD rows are buy-and-hold reference series in the same table. Each real bot has a live equity history here (e.g. AdaptiveMeanReversionBot since 2026-03-22, starting from a $10,000 notional).

Accessing production data (trades + portfolio_worth)

The real trade history (trades) and live equity curves (portfolio_worth) live in the production Postgres in Kubernetes, not on localhost. POSTGRES_URI in .env points at localhost:5432, which is not running locally — you must reach the cluster DB.

Namespace: tradingbots-2025
Postgres pod: psql-deployment-* (service psql-service:5432, db postgres)

Exec directly (simplest, no port-forward):

POD=$(kubectl get pods -n tradingbots-2025 -o name | grep psql | head -1)
kubectl exec -n tradingbots-2025 ${POD#pod/} -- \
  psql -U postgres -d postgres -c \
  "SELECT date::date, portfolio_worth FROM portfolio_worth WHERE bot_name='AdaptiveMeanReversionBot' ORDER BY date;"

Or port-forward and use the framework's get_db_session() locally:

kubectl port-forward -n tradingbots-2025 svc/psql-service 5432:5432

See the /k8s-psql-investigate skill for full details.

Database Session Management

Always use the context manager:

from tradingbot.utils.db import get_db_session

with get_db_session() as session:
    # Do database operations
    bot = session.query(Bot).filter_by(name="MyBot").first()
    # Context manager automatically commits on success, rolls back on error

Important: The context manager handles:

Automatic commit on success
Automatic rollback on exceptions
Connection retry logic (3 attempts with exponential backoff)
Proper session cleanup

Connecting to Multiple Databases

If you need to query a different database (e.g., ai_hedge_fund) while the bot's portfolio is stored in the main postgres database:

from sqlalchemy import create_engine, text
from sqlalchemy.orm import sessionmaker
from os import environ


def _get_other_database_session(self):
    """Create a separate connection to another database."""
    # Read POSTGRES_URI but don't modify the environment variable
    base_uri = environ.get("POSTGRES_URI", "")

    # Modify URI to point to different database (e.g., ai_hedge_fund)
    if "/" in base_uri:
        parts = base_uri.rsplit("/", 1)
        other_db_uri = parts[0] + "/other_database"
    else:
        other_db_uri = base_uri + "/other_database"

    # Create separate engine (doesn't affect base Bot class connection)
    database_url = "postgresql+psycopg2://" + other_db_uri
    engine = create_engine(database_url, pool_pre_ping=True, pool_recycle=3600)
    SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
    return SessionLocal()


# Usage with raw SQL
session = self._get_other_database_session()
try:
    result = session.execute(text("SELECT * FROM some_table")).fetchone()
finally:
    session.close()

Key points:

Base Bot class uses main postgres database (via utils/db.py)
Create separate engine for other database connections
Don't modify POSTGRES_URI environment variable
Use text() from SQLAlchemy for raw SQL queries on tables without models

Creating a New Bot

Step 1: Create Bot File

Create tradingbot/{botname}bot.py:

from tradingbot.utils.botclass import Bot


class MyNewBot(Bot):
    # Optional: Define hyperparameter search space for tuning
    param_grid = {
        "rsi_buy": [65, 70, 75],
        "rsi_sell": [25, 30, 35],
    }

    def __init__(self, rsi_buy: float = 70.0, rsi_sell: float = 30.0, **kwargs):
        # Symbol like "QQQ", "EURUSD=X", "^XAU"
        # Optional: interval="1d", period="1mo" for daily/weekly strategies
        super().__init__("MyNewBot", "SYMBOL", interval="1m", period="1d", **kwargs)

        # Store parameters as instance variables
        self.rsi_buy = rsi_buy
        self.rsi_sell = rsi_sell

    def decisionFunction(self, row):
        # Your trading logic here
        # Access TA indicators via row["indicator_name"]
        # Return -1, 0, or 1
        if row["momentum_rsi"] < self.rsi_buy:
            return 1  # Oversold, buy
        elif row["momentum_rsi"] > self.rsi_sell:
            return -1  # Overbought, sell
        return 0  # Hold


# Standard entry point for local development
bot = MyNewBot()

bot.local_development()  # Runs hyperparameter optimization + backtest
# bot.run()  # Uncomment for production (or use environment detection)

Note: If you only need to change the timeframe (interval/period), you can set it in the constructor and don't need to override makeOneIteration(). See gptbasedstrategytabased.py for an example.

Step 2: Add to Helm Chart

Edit kubernetes/helm/tradingbots/values.yaml:

bots:
  - name: mynewbot
    schedule: '*/5 * * * 1-5' # Every 5 minutes, Mon-Fri

Important:

Filename must be {name}bot.py (e.g., mynewbot.py)
Helm automatically uses {name}.py as the script filename
Container name is auto-generated as tradingbot-{name} (removes "bot" suffix)

Step 3: Deploy

The GitLab CI pipeline will:

Build Docker image
Deploy via Helm (creates CronJob automatically)
Bot runs on schedule

Important Patterns and Conventions

1. Bot Naming

Bot class name: CamelCaseBot (e.g., EURUSDTreeBot)
Bot database name: Same as class name (passed to super().__init__())
Filename: {name}bot.py (lowercase, e.g., eurusdtreebot.py)
Helm name: {name}bot (e.g., eurusdtreebot)

2. Decision Function Contract

Must return int: -1, 0, or 1
Receives a pd.Series with all TA indicators
Is called for each row in the DataFrame
Base class averages the last N decisions (default: 1)

3. Data Format

All DataFrames have this structure:

columns = ["symbol", "timestamp", "open", "high", "low", "close", "volume"]
# Plus ~150+ TA indicators after getYFDataWithTA()

4. Portfolio Structure

portfolio = {
    "USD": 10000.0,  # Cash
    "QQQ": 5.5,  # Holdings (quantity, not value)
    "EURUSD=X": 1000.0,  # More holdings
}

5. Error Handling

run() catches all exceptions and logs to RunLog table
Database operations use retry logic automatically
Empty data returns decision 0 (hold)

6. Data Caching

self.data caches the last fetched DataFrame (per-instance cache)
self.datasettings stores (interval, period) tuple
If same settings requested, returns cached data (no API call)
Database persistence: For cross-run data reuse (e.g., hyperparameter tuning), set saveToDB=True when fetching data. Subsequent calls (even from new Bot instances) will check the database first and only fetch from yfinance if data is missing or stale (older than 10 minutes by default).

7. Timezone Handling

Important: Always use timezone-aware datetimes when comparing with database timestamps.

from datetime import datetime, timezone, timedelta

# ❌ Wrong: datetime.utcnow() returns timezone-naive
one_day_ago = datetime.utcnow() - timedelta(days=1)

# ✅ Correct: Use timezone-aware UTC datetime
now_utc = datetime.now(timezone.utc)
one_day_ago = now_utc - timedelta(days=1)

# Handle timezone-naive datetimes from database
if db_datetime.tzinfo is None:
    db_datetime = db_datetime.replace(tzinfo=timezone.utc)  # Assume UTC

8. Wrapper vs Implementation Pattern

Principle: Keep "wrappers" (entry points) in tradingbot/ and application logic in tradingbot/utils/.

Purpose: Separation of concerns - wrappers handle orchestration (env vars, logging setup) while utils contain reusable logic.

Pattern:

tradingbot/telegram_monitor.py         # Wrapper: entry point, env var parsing
tradingbot/utils/telegram_monitor.py   # Implementation: core logic, reusable functions

Wrapper responsibility (tradingbot/telegram_monitor.py):

Parse environment variables
Set up logging
Call utils functions with parsed parameters
Handle script execution (if __name__ == "__main__")

import os
from telethon.sessions import StringSession
from tradingbot.utils.telegram_monitor import monitor_channels


def main():
    api_id = int(os.environ["TELEGRAM_API_ID"])
    api_hash = os.environ["TELEGRAM_API_HASH"]
    session_string = os.environ["TELEGRAM_SESSION_STRING"]
    channels = [c.strip() for c in os.environ.get("TELEGRAM_CHANNELS", "").split(",") if c.strip()]

    monitor_channels(api_id, api_hash, StringSession(session_string), channels)


if __name__ == "__main__":
    main()

Implementation responsibility (tradingbot/utils/telegram_monitor.py):

Core business logic (functions: get_existing_message_ids(), summarize_message(), process_channel())
Database operations
External API calls
Returns data, doesn't know about env vars or logging setup

def monitor_channels(api_id: int, api_hash: str, session_string, channels: list[str]):
    """Core implementation - no env vars, no logging setup."""
    # Connect to Telegram
    # Process channels
    # Store results

Examples in codebase:

calculate_portfolio_worth.py (wrapper) → imports tradingbot.utils.portfolio_utils (impl)
aitools.py (in utils) → provides run_ai_simple(), run_ai_with_tools() — reusable functions

When to apply this pattern:

Creating a new script/cronjob
Extracting reusable logic that other modules might use
Simplifying complex entrypoints

Common Pitfalls

1. DataFrame Mutation

Problem: getLatestDecision() used to mutate input DataFrame Solution: Now works on a copy - safe to reuse DataFrames

2. Redundant Commits

Problem: Explicit session.commit() inside get_db_session() context manager Solution: Context manager commits automatically - removed redundant commits

3. Index Out of Bounds

Problem: Accessing rows when DataFrame is too small Solution: getLatestDecision() now handles empty/small DataFrames gracefully

4. Container Name Mismatch

Problem: Manual container names in CronJobs Solution: Auto-generated from bot name: tradingbot-{name} (removes "bot" suffix)

5. Script Filename Mismatch

Problem: Manual script names in Helm values Solution: Auto-generated from bot name: {name}.py

6. Timezone Comparison Errors

Problem: TypeError: can't compare offset-naive and offset-aware datetimes when comparing database timestamps Solution: Use datetime.now(timezone.utc) instead of datetime.utcnow(), and handle timezone-naive datetimes from database by adding UTC timezone info

7. Bot Name Casing — Never Normalize

Problem: Bot names in the bots table are stored exactly as the bot constructs itself, e.g. "AdaptiveMeanReversionBot" (CamelCase). Lookups via BotRepository.create_or_get_bot(name) use a case-sensitive filter_by(name=...) exact match. If the lookup misses, a new row is silently created with default {"USD": 10000} — no error.

This combination is a footgun: any code that lowercases (or otherwise normalizes) a bot name before lookup will silently spawn a duplicate stub row with $10k of fake cash. The bug surfaces as "no target weights / portfolio appears empty," not as an error.

Solution:

Never call .lower() / .upper() / .strip() etc. on bot names before passing them to BotRepository. Pass user input through verbatim.
When accepting bot names from external input (env vars, JSON config, CLI args), validate against session.query(Bot).all() first and abort with a clear error if the user's name isn't an exact match — do not rely on create_or_get_bot for validation, it will happily create whatever you ask for.
Filename / Helm name: are lowercase by convention (adaptivemeanreversionbot.py, name: adaptivemeanreversionbot), but the DB row name is CamelCase (whatever the bot passes to super().__init__("CamelCaseName", ...)). Don't conflate them.

8. `create_or_get_bot` is Not a Validator

Problem: The name create_or_get_bot suggests safe lookup, but it's actually INSERT IF NOT EXISTS — it never returns None and never raises on a missing bot. Calling it with a typo'd or normalized name pollutes the DB with empty stub rows.

Solution: For paths that should fail loudly on a missing bot (live-trade copier, AI bot weight configs, anything driven by user input), do an explicit existence check first:

with get_db_session() as s:
    if not s.query(Bot).filter_by(name=name).first():
        raise ValueError(f"Bot {name!r} not in DB. Existing: {[b.name for b in s.query(Bot).all()]}")

Only use create_or_get_bot when the caller genuinely owns the bot's identity (i.e., the bot itself, registering on first run).

9. LiveTrade Buy Sizing — Clamp to Cash, Not Equity

Problem: LiveTradeCopier sizes buy orders against total_equity (broker's ModelAccountValue for C2, NetLiquidation for IB). But brokers run a margin/cash check at order submission and reject if cash < notional. Sells in the same sync don't necessarily release cash before the buy batch fires (C2 paper accounts in particular are slow to settle), so a target like 100% QQQ on a previously-diversified account will get rejected even though equity covers it. Symptom on C2: PreMarginCheck api2 b cs — "Current account cash is $X; proposed trade requires cash of $Y".

Solution: _execute_orders re-fetches broker.get_cash() after the settle delay and scales all buys proportionally if their total notional exceeds available cash (with a 2% buffer). Buys that scale below min_order_usd are dropped. Don't skip the cash clamp by sizing buys against equity directly — settlement timing is broker-specific and not something the copier should pretend to know.

If buys are still rejected, bump LIVETRADE_SETTLE_DELAY_SECONDS (defaults to 10s — too short for C2 paper).

10. eToro Broker — API Quirks

The eToro Public API (https://public-api.etoro.com, /api/v1 prefix) has a number of non-obvious requirements. All of these were learned the hard way during the livetrade/etoro.py integration.

Auth: two headers from the API portal's "Generated Keys" section:

x-api-key = the JWT-style token (looks like eyJ...)
x-user-key = the static alphanumeric string
Plus x-request-id (UUID) per request — required, not optional.

Path prefix for demo vs live: api/v1/trading/... for live, api/v1/trading/.../demo/... for paper. The demo segment goes between info/execution and the resource (e.g. api/v1/trading/info/demo/portfolio). Same applies to execution endpoints.

Portfolio response shape (/api/v1/trading/info/{demo/}portfolio):

Top-level envelope is clientPortfolio, not portfolio.
Cash field is credit (not cash or equity).
No equity field — compute as credit + sum(units * currentPrice) from positions.
Position fields use uppercase ID: instrumentID, positionID (not instrumentId).

Search API (/api/v1/market-data/search):

fields query param is required (e.g. fields=instrumentId,internalSymbolFull,symbol). Without it, results come back empty.
Response key is items, not results.
For yfinance crypto tickers like BTC-USD, strip the -USD/-USDT suffix before passing as internalSymbolFull — eToro uses bare BTC.

Price endpoint: /api/v1/market-data/instruments is metadata-only (no price). Use /api/v1/market-data/instruments/rates?instrumentIds=... instead. Response shape: {"rates": [{"lastExecution": ..., "bid": ..., "ask": ...}]}. Supports up to 100 IDs per call.

SELL close orders (POST .../market-close-orders/positions/{positionId}):

Body must include {"InstrumentId": <int>} even though the position ID is in the path. Missing instrument ID → HTTP 400 "InstrumentId: The instrument id does not exist".
httpx with json=None omits the body and the Content-Type header → eToro returns 415 Unsupported Media Type. Always pass at least {} (or the InstrumentId payload above).

11. Darwinex (DXtrade) Broker — API Quirks

The Darwinex DXtrade API (/dxsca-web) has its own set of nuances.

Auth: Session-token based.

POST /login with {"username", "password", "domain": "default"} returns sessionToken.
Subsequent requests require Authorization: DXAPI <token> header.
Tokens expire (typically 24h, but v1 implementation refreshes on 401 or after 2h).

Account Selection: A single user may have multiple accounts.

Use GET /users/{username}/accounts to list them.
If DARWINEX_ACCOUNT_ID is not provided, the broker picks the first one from this list.

Portfolio & Metrics:

Metrics (GET /accounts/{id}/metrics): Returns balance (cash) and equity (MTM value).
Portfolio (GET /accounts/{id}/portfolio): Returns a list of positions.
DXtrade nets positions automatically; get_positions() aggregates by instrumentCode.
Position quantity is signed based on side (BUY = positive, SELL = negative).

Symbol Mapping (CFD Catalog):

Darwinex is a CFD-only broker. Equities are often suffixed with .US (e.g., AAPL.US).
map_symbol() first checks the manual symbol_map.json, then performs a catalog search (GET /instruments?symbol=...) for both the bare ticker and the .US variant.
Catalog search results can be a list or wrapped in an instruments key; implementation handles both.

No Native Last Price:

The REST API does not provide a simple snapshot "last price" endpoint. Quotes are WebSocket-only (/md).
_get_native_price() returns 0.0, triggering the base class's yfinance fallback for all order sizing and equity calculations.

12. `POSTGRES_URI` Required Even for Non-DB Tests

Problem: pytest tests/... fails with KeyError: 'Set POSTGRES_URI or (POSTGRES_HOST + POSTGRES_PASSWORD) for database connection' even when running tests that don't touch the DB (e.g. pure-mock tests under tests/test_livetrade.py).

Cause: tradingbot/utils/__init__.py imports botclass → bot_repository → db, and db.py resolves DATABASE_URL at module import time. Any test that imports anything from tradingbot.utils (or transitively, like the livetrade copier) triggers this.

Solution: Pass a stub URI for non-DB test runs:

POSTGRES_URI="postgresql://x:x@localhost:5432/x" PYTHONPATH=. uv run pytest tests/ -q

The connection isn't opened until a session is actually used, so a syntactically-valid bogus URI is enough to satisfy import.

13. Index Tickers Defeat `LIVETRADE_STRICT_MAPPING`

Problem: A bot whose ticker is an index — XAUZenbotTreeBot trades ^XAU, the PHLX Gold/Silver index — has nothing tradeable to buy, but LIVETRADE_STRICT_MAPPING=true did not catch it. The copier submitted C2 orders for FullSymbol: "^XAU" that the broker rejected on every run, silently.

Cause: SymbolMapper.map_symbol() never returns None. Its default rules translate only ^GSPC/^NDX/^IXIC; every other ticker falls through the elif chain and is returned unchanged as {"symbol": "^XAU", "type": "stock", "source": "default-rule"}. That is a non-None dict with a truthy symbol, so the copier counted it as successfully mapped and the strict-mode abort never fired. Strict mapping only ever protected against a broker's map_symbol returning None — which the shared SymbolMapper never does.

Solution: copier.sync() now also rejects any mapped symbol that still starts with ^, since no broker accepts a caret symbol. Translated indices (^GSPC → SPX) have already lost the caret and are unaffected. When adding a bot that trades an index, map it to a tradeable proxy ETF in symbol_map.json — ^XAU → GDX (VanEck Gold Miners) for collective2.

Generalize: don't assume a non-None return from a mapper means "mapped". Check the mapped value is usable for the target system, not just that a lookup succeeded.

14. Multiple Collective2 Strategies Share One API Key

C2 API keys are account-scoped and every request selects the strategy by numeric StrategyId, so a second C2 strategy needs a second CronJob, not a second credential. helm/tradingbots/templates/cronjob-livetrade-collective2-xauzerifine.yaml is the pattern: same image, same livetrade_collective2.py, different COLLECTIVE2_SYSTEM_ID + per-instance LIVETRADE_BOT_WEIGHTS/LIVETRADE_DRY_RUN via the default <shared> <per-instance> idiom.

Never point two enabled copiers at one StrategyId. copier.sync() is a full target-state reconciliation that liquidates every position not in its target set, so two jobs on one strategy each liquidate the other's positions on every run. The template hard-fails at render time if systemId is empty, or if it duplicates liveTrade.collective2.systemId while both instances are enabled.

Reusing a StrategyId is correct when handing a strategy over to a different bot set — disable the old instance in the same change that enables the new one. That keeps the C2 strategy's existing track record and equity curve; the first run liquidates the old bots' positions and rebuilds from the new ones. Create a fresh C2 strategy instead if the new bot needs a clean record.

Technical Analysis Indicators

After calling getYFDataWithTA(), the DataFrame includes indicators from the ta library:

Categories:

Trend: trend_sma_fast, trend_macd, trend_adx, trend_ichimoku_*, etc.
Momentum: momentum_rsi, momentum_stoch, momentum_roc, etc.
Volatility: volatility_bbh, volatility_bbl, volatility_atr, volatility_kch, etc.
Volume: volume_* indicators

Naming: All lowercase with underscores (e.g., trend_sma_slow, momentum_rsi)

Access: row["indicator_name"] in decisionFunction()

Deployment Architecture

Kubernetes CronJobs

Each bot runs as a separate CronJob
Schedule defined in values.yaml
All use the same Docker image (tagged by branch)

Helm Chart Structure

helm/tradingbots/
├── Chart.yaml              # Chart metadata
├── values.yaml             # Bot configurations
└── templates/
    ├── cronjob.yaml        # Generates CronJobs for each bot
    ├── postgresql-secret.yaml
    ├── postgresql-deployment.yaml
    └── postgresql-service.yaml

GitLab CI Pipeline

build-docker: Builds and pushes Docker image
helm-kubectl-deploy:
- Extracts image repo/tag from $IMAGE
- Deploys via Helm with image overrides
- Creates namespace if needed

Local Development and Hyperparameter Tuning

Local Development Workflow

The Bot class provides convenient methods for local development and optimization:

bot = MyBot()

# Option 1: Full workflow (optimize + backtest)
bot.local_development()
# - Runs hyperparameter optimization using param_grid
# - Backtests the best parameters
# - Prints results in easy-to-copy format

# Option 2: Just optimize
results = bot.local_optimize()
# Returns optimization results dictionary

# Option 3: Just backtest current parameters
results = bot.local_backtest()
# Returns backtest results dictionary

Hyperparameter Tuning

Define param_grid as a class attribute:

class MyBot(Bot):
    # Define hyperparameter search space
    param_grid = {
        "rsi_buy": [65, 70, 75],
        "rsi_sell": [25, 30, 35],
        "adx_threshold": [15, 20, 25],
    }

    def __init__(self, rsi_buy=70.0, rsi_sell=30.0, adx_threshold=20.0, **kwargs):
        super().__init__("MyBot", "QQQ", **kwargs)
        self.rsi_buy = rsi_buy
        self.rsi_sell = rsi_sell
        self.adx_threshold = adx_threshold

Key Features:

Data pre-fetching: Historical data is fetched once and reused for all parameter combinations (dramatically faster)
Database caching: Data is saved to DB on first fetch, subsequent runs reuse cached data
Parallel execution: Uses multiple CPU cores by default (configurable via n_jobs)
Automatic period adjustment: For minute-level intervals, automatically uses 7 days instead of 1 year (respects Yahoo Finance limits)

Optimization Process:

Pre-fetches 1 year of data (or appropriate period based on interval) with TA indicators
Saves data to database for future reuse
Tests all parameter combinations in parallel
Returns best parameters and full results

Backtesting Period Limits:

Minute intervals (1m, 5m, 15m, etc.): Uses 7 days (Yahoo Finance limit: 8 days)
Hourly intervals: Uses 60 days
Daily/weekly/monthly: Uses 1 year

Standard Bot File Pattern

All bot files follow this pattern:

class MyBot(Bot):
    param_grid = {...}  # Optional: for hyperparameter tuning

    def __init__(self, param1=default1, param2=default2, **kwargs):
        super().__init__("MyBot", "SYMBOL", interval="1d", period="1mo", **kwargs)
        self.param1 = param1
        self.param2 = param2

    def decisionFunction(self, row):
        # Trading logic
        return 0


# Local development: optimize and backtest
bot = MyBot()
bot.local_development()
# bot.run()  # Uncomment for production

Production vs Development:

Local development: Use bot.local_development() to optimize and test
Production: Uncomment bot.run() or use environment detection (Kubernetes sets KUBERNETES_SERVICE_HOST)

Working with the Code

When Adding Features

Database changes: Update models in utils/db.py, migrations handled by SQLAlchemy
Bot functionality: Extend Bot class methods
New bots: Follow the pattern in existing bots (include param_grid if tunable)
Deployment: Update values.yaml and redeploy

When Debugging

Check run_logs table for execution history
Check trades table for trade history
Check bots table for portfolio state
Logs are printed to stdout (captured by Kubernetes)

When Modifying Bot Logic

Choose the simplest approach that works for your needs:

Start with decisionFunction() - If your strategy can be expressed as logic on a single row:
- Access TA indicators from row["indicator_name"]
- Return -1, 0, or 1 based on conditions
- Base class handles data fetching, averaging, and trade execution
- Example: "Buy when RSI < 30 and MACD is bullish"
Override makeOneIteration() only if needed - For:
- External APIs (Fear & Greed Index, sentiment data, etc.)
- Portfolio optimization with multiple symbols
- Custom data processing that can't be done row-by-row
- Example: Portfolio rebalancing based on Sharpe ratio optimization
Data fetching: Use getYFData() or getYFDataWithTA() (both support interval/period)
Trading: Use buy() and sell() methods

Key Files Reference

File	Purpose
`tradingbot/utils/botclass.py`	Base Bot class - core functionality
`tradingbot/utils/db.py`	Database models and session management
`kubernetes/helm/tradingbots/values.yaml`	Bot configurations and schedules
`kubernetes/helm/tradingbots/templates/cronjob.yaml`	CronJob template
`.gitlab-ci.yml`	CI/CD pipeline configuration

Summary

This system provides a robust framework for automated trading:

Implementation Strategy (Simple → Complex)

Start simple: Implement only decisionFunction(row) - works for most strategies
Add complexity only if needed: Override makeOneIteration() for external APIs or portfolio optimization
Use constructor parameters: Set interval and period in __init__() to change timeframes without overriding methods

Key Features

Inherit from Bot and implement decisionFunction() (preferred) or makeOneIteration() (when needed)
Use provided methods for data fetching, trading, and portfolio management
Database is handled automatically - portfolio, trades, and logs are persisted
Deployment is template-based - add bot to values.yaml and deploy
Error handling is built-in - exceptions are caught and logged
Hyperparameter tuning - Define param_grid and use local_development() for optimization
Efficient data loading - Pre-fetches and caches data for hyperparameter tuning (avoids redundant API calls)
Smart period adjustment - Automatically adjusts backtest period based on interval (respects Yahoo Finance limits)

The Bot class abstracts away database operations, data fetching, and trade execution, allowing you to focus on the trading strategy logic. Always prefer the simplest approach that works for your needs.

FilesExpand file tree

AGENTS.md

Latest commit

History

AGENTS.md

File metadata and controls

Trading Bot System - LLM Guide

Repository Overview

Architecture

Directory Structure

Key Technologies

The Bot Class System

Core Concept

1. Simplest (Preferred): Implement only decisionFunction(row)

2. Medium complexity: Override makeOneIteration() for custom data sources or simple custom logic

3. Complex: Override makeOneIteration() for portfolio optimization or multi-symbol strategies

Bot Class Lifecycle

targetWeights returns a target book, not a signal

Position-sizing semantics (easy to get wrong)

historic_data is keyed by interval — keep it that way

Interval and backtestability are strategy properties, not settings

Options (paper, opt-in)

Stock splits

Reading another bot's state

Key Bot Class Methods

Data Fetching

Decision Making

Trading Operations

Portfolio Management

Price Fetching

Telegram Channel Monitor

Architecture

How it runs

Key env vars

AI summarization

Database model: TelegramMessage

Database Models (tradingbot/utils/db.py)

Bot Model

Trade Model

HistoricData Model

RunLog Model

PortfolioWorth Model

Accessing production data (trades + portfolio_worth)

Database Session Management

Connecting to Multiple Databases

Creating a New Bot

Step 1: Create Bot File

Step 2: Add to Helm Chart

Step 3: Deploy

Important Patterns and Conventions

1. Bot Naming

2. Decision Function Contract

3. Data Format

4. Portfolio Structure

5. Error Handling

6. Data Caching

7. Timezone Handling

8. Wrapper vs Implementation Pattern

Common Pitfalls

1. DataFrame Mutation

2. Redundant Commits

3. Index Out of Bounds

4. Container Name Mismatch

5. Script Filename Mismatch

6. Timezone Comparison Errors

7. Bot Name Casing — Never Normalize

8. create_or_get_bot is Not a Validator

9. LiveTrade Buy Sizing — Clamp to Cash, Not Equity

10. eToro Broker — API Quirks

11. Darwinex (DXtrade) Broker — API Quirks

12. POSTGRES_URI Required Even for Non-DB Tests

13. Index Tickers Defeat LIVETRADE_STRICT_MAPPING

14. Multiple Collective2 Strategies Share One API Key

Technical Analysis Indicators

Deployment Architecture

Kubernetes CronJobs

Helm Chart Structure

GitLab CI Pipeline

Local Development and Hyperparameter Tuning

Local Development Workflow

1. Simplest (Preferred): Implement only `decisionFunction(row)`

2. Medium complexity: Override `makeOneIteration()` for custom data sources or simple custom logic

3. Complex: Override `makeOneIteration()` for portfolio optimization or multi-symbol strategies

`targetWeights` returns a target book, not a signal

`historic_data` is keyed by interval — keep it that way

Database model: `TelegramMessage`

8. `create_or_get_bot` is Not a Validator

12. `POSTGRES_URI` Required Even for Non-DB Tests

13. Index Tickers Defeat `LIVETRADE_STRICT_MAPPING`