An event driven backtesting engine replays historical data one bar (or tick) at a time and passes each update through the same chain a live system uses: market event, strategy signal, order, fill, portfolio update. Because the strategy only ever sees data up to the current event, look-ahead bias is much harder to introduce, and the same code can switch from simulated fills to a real broker API. This research note lays out how we plan to build OpenEngine on that design, with DuckDB for 1-minute data and the OpenAlgo PlaceOrder API for execution.
Algorithmic trading requires a realistic and robust simulation environment. This note describes how we are planning to build an open source, event‑driven backtesting engine optimized for 1‑minute data using DuckDB for fast historical data storage and querying. In addition, it details the integration of a broker API for order execution, using the OpenAlgo PlaceOrder API, as an example of how to enable live trading with minimal code changes. This modular, unified architecture helps ensure that your strategy’s behavior remains consistent from simulation to production.

- Event-driven engines process data bar by bar through market, signal, order and fill events, mirroring live trading.
- Vectorised backtests are faster for research; event-driven ones are better when the same code must trade live.
- DuckDB stores and queries years of 1-minute candles locally, fast enough for replay without a database server.
- Swapping a simulated broker class for an OpenAlgo PlaceOrder client moves the engine from backtest to live.
- A minimal working engine needs only five small classes and an event queue (example below).
1. Introduction
Before deploying trading strategies live, a high‑quality backtesting engine is essential for understanding performance under real market conditions. In this note, we explain why an event‑driven approach is advantageous compared to a purely vector‑based method, especially when using granular 1‑minute data. We also demonstrate how to integrate with a broker’s order placement API (OpenAlgo PlaceOrder) to seamlessly transition from simulation to live execution.
2. Background and Motivation
2.1 Vector-Based vs. Event-Driven Backtesting
- Vector-Based Backtesting:
- Overview: Processes large data batches using vectorized operations (e.g., with pandas or NumPy).
- Pros: High performance and simplicity for statistical operations.
- Cons: Not ideal for simulating the sequential and dynamic aspects of live trading (e.g., latency, partial fills, order rejections).
- Event-Driven Backtesting:
- Overview: Simulates trading by processing discrete events (market data updates, signals, orders, fills) in order.
- Pros: Closely mirrors live trading, supports stateful strategies, and unifies simulation and live execution within the same framework.
- Cons: More architecturally complex and requires careful management of events, though this is usually manageable at a 1‑minute resolution.
Given the requirement for a unified codebase for both backtesting and live trading, the event‑driven approach is strongly recommended.
2.2 Why Use DuckDB?
DuckDB is an embeddable, high‑performance OLAP database that:
- Provides Speed: Efficiently processes large volumes of columnar data such as 1‑minute historical market data.
- Is Simple to Integrate: With its Python API, it allows you to store, query, and manage data with minimal setup.
- Scales Well: Suitable for local testing and scalable enough for larger production deployments.
3. Key Components of the Backtesting Engine
A modular, event‑driven backtesting engine should consist of the following components:
- Data Handling and Storage:
- Historical Data Loader: Import 1‑minute data into DuckDB.
- DuckDB Integration: Use SQL queries to efficiently retrieve and process data during simulation.
- Real-Time Data Feed: (For live trading) Ingest live data via APIs or WebSockets.
- Event System:
- Event Types: Define events such as Market Data Events, Signal Events, Order Events, Fill Events, and Portfolio Update Events.
- Event Queue and Dispatcher: Ensure events are processed sequentially and routed to the appropriate engine modules.
- Strategy Engine:
- Signal Generation: Process incoming market events to generate buy/sell signals.
- State Management: Maintain necessary context for sequential or stateful strategies.
- Order Management and Execution:
- Order Manager: Translate signals into orders, simulate order fills in backtesting, and handle order status updates.
- Broker Interface: Abstract the connection to broker APIs, ensuring that the same interface is used for both backtesting and live trading.
- API Integration: In live trading, use endpoints like the OpenAlgo PlaceOrder API to send orders to the broker.
- Portfolio and Risk Management:
- Position Sizing and Allocation: Determine trade sizes based on risk controls.
- Risk Controls: Enforce stop-loss, take-profit, and other risk management rules.
- Portfolio Valuation: Continuously update positions and track performance metrics (drawdowns, Sharpe ratio, etc.).
- Reporting and Analytics:
- Metrics Calculation: Generate performance reports and visualizations.
- Logging: Maintain detailed logs for debugging, auditing, and performance analysis.
- Configuration and Environment Management:
- User Settings: Enable customization of strategy parameters, capital allocation, and risk limits.
- Unified Environment: Ensure that settings work identically in both backtesting and live modes.
4. Integrating Broker API: OpenAlgo PlaceOrder Function
To execute orders in live trading (and even simulate them during backtesting), you need a robust API interface. Here’s an overview of the OpenAlgo PlaceOrder API function:
4.1 API Details
- Endpoint URL:
- Local Host:
POST http://127.0.0.1:5000/api/v1/placeorder - Ngrok Domain:
POST https://<your-ngrok-domain>.ngrok-free.app/api/v1/placeorder - Custom Domain:
POST https://<your-custom-domain>/api/v1/placeorder
- Local Host:
- Request JSON Format:
{
"apikey": "<your_app_apikey>",
"strategy": "Test Strategy",
"symbol": "SAIL",
"action": "BUY",
"exchange": "NSE",
"pricetype": "MARKET",
"product": "MIS",
"quantity": "1",
"price": "0",
"trigger_price": "0",
"disclosed_quantity": "0"
}
Parameter Descriptions:
| Parameter | Description | Mandatory/Optional | Default Value |
|---|---|---|---|
| apikey | App API key | Mandatory | – |
| strategy | Strategy name | Mandatory | – |
| exchange | Exchange code | Mandatory | – |
| symbol | Trading symbol | Mandatory | – |
| action | BUY or SELL | Mandatory | – |
| product | Product type | Optional | MIS |
| pricetype | Price type | Optional | MARKET |
| quantity | Order quantity | Mandatory | – |
| price | Price | Optional | 0 |
| trigger_price | Trigger price | Optional | 0 |
| disclosed_quantity | Disclosed quantity | Optional | 0 |
Sample API Response:
{
"orderid": "240307000614705",
"status": "success"
}
Update (2026): the current OpenAlgo documentation lists the same eleven request fields, with pricetype accepting MARKET, LIMIT, SL and SL-M and product accepting MIS, CNC and NRML. The response can also carry a mode field, which is "analyze" when the order runs in OpenAlgo’s sandbox instead of going to the broker (OpenAlgo PlaceOrder docs). That sandbox is a useful middle step between backtest and live.
4.2 Integration in the Order Manager
When designing your Order Manager module, abstract the execution of orders so that in a live trading scenario, it calls the API. In backtesting, you might simulate the API call or use a similar interface for consistency. For example:
import requests
class BrokerInterface:
def __init__(self, base_url, apikey):
self.base_url = base_url
self.apikey = apikey
def place_order(self, strategy, symbol, action, exchange, product="MIS",
pricetype="MARKET", quantity="1", price="0", trigger_price="0", disclosed_quantity="0"):
endpoint = f"{self.base_url}/api/v1/placeorder"
payload = {
"apikey": self.apikey,
"strategy": strategy,
"exchange": exchange,
"symbol": symbol,
"action": action,
"product": product,
"pricetype": pricetype,
"quantity": quantity,
"price": price,
"trigger_price": trigger_price,
"disclosed_quantity": disclosed_quantity
}
response = requests.post(endpoint, json=payload)
if response.status_code == 200:
return response.json()
else:
raise Exception("Order placement failed", response.text)
In your live trading engine, you would call this method when an order signal is generated. In backtesting, you can simulate this call to log the order details and update simulated portfolio values.
5. Step-by-Step Process for Building the Engine
Step 1: Data Acquisition and Storage
- Collect Historical Data:
Obtain your 1‑minute historical market data from Yahoo Finance or other providers. - Integrate DuckDB:
Create a schema to store the data and load it using DuckDB’s Python API:
import duckdb
con = duckdb.connect('market_data.duckdb')
con.execute("""
CREATE TABLE IF NOT EXISTS market_data (
timestamp TIMESTAMP,
open DOUBLE,
high DOUBLE,
low DOUBLE,
close DOUBLE,
volume BIGINT
)
""")
con.execute("COPY market_data FROM '1min_data.csv' (FORMAT CSV, HEADER)")
If you need years of 1-minute data rather than a single CSV, OpenAlgo Historify downloads broker candles into a local DuckDB database and can export CSV, TXT, ZIP and Parquet (Historify docs). I walk through it in how to download 10 years of intraday data in CSV format, and DuckDB basics are covered in the DuckDB tutorial for traders.
Step 2: Design the Event‑Driven Architecture
- Define Event Types:
Create classes (or named tuples) for events such asMarketDataEvent,SignalEvent,OrderEvent, andFillEvent. - Implement the Event Queue and Dispatcher:
Use Python’s built‑in libraries (e.g.,queue.Queue) to manage event sequencing.
Step 3: Develop the Strategy Module
- Base Strategy Interface:
Define an abstract strategy class to generate signals. - Concrete Strategy Implementation:
For instance, a moving average crossover strategy that processes each 1‑minute market event and outputs a signal. - State Management:
Ensure your strategy maintains any necessary state (e.g., moving averages, current positions).
Step 4: Implement Order Management and Execution Simulation
- Order Manager Module:
Build a component to handle signal-to-order translation, using the BrokerInterface (as shown above) to place orders. - Broker Interface Integration:
Integrate the OpenAlgo PlaceOrder API so that order execution is identical for live and simulated environments.
Step 5: Build Portfolio and Risk Management Components
- Portfolio Manager:
Track positions, cash balances, and overall portfolio value. - Risk Engine:
Incorporate stop-loss, take-profit, and maximum exposure controls.
Step 6: Integrate the Simulation Loop
- Data Replay:
Develop a loop that reads 1‑minute data from DuckDB, converts each row into a market data event, and processes it through your event‑driven architecture. - Live Simulation:
Ensure that the loop is capable of running in real time, processing live events and calling the order placement API when required.
Step 7: Reporting, Analytics, and Logging
- Performance Metrics:
Calculate cumulative returns, drawdowns, and other key indicators. - Visualization:
Use libraries such as Matplotlib or Plotly to visualize equity curves and trade performance. - Logging:
Implement detailed logging to trace events, order placements, and strategy decisions.
Step 8: Transitioning to Live Trading
- Unified Engine Design:
Ensure that your event‑processing loop and core modules remain identical whether processing historical data or live feeds. - API Integration:
Replace simulated order fills with real API calls (e.g., using the BrokerInterface to call the OpenAlgo PlaceOrder endpoint). - Testing:
Rigorously test in a paper trading or simulated environment before full live deployment.
A Minimal Event Driven Backtesting Engine in Python
To make the architecture concrete, here is a small working version of Steps 2 to 6. It reads the market_data table created in Step 1, runs a 20/50 moving average crossover, and fills orders on the next bar with slippage. The SimulatedBroker is the only class you replace to go live; its send() method would call the BrokerInterface.place_order() shown in section 4.2.
from collections import deque
from dataclasses import dataclass
import duckdb
@dataclass
class MarketEvent:
ts: object
close: float
@dataclass
class SignalEvent:
ts: object
action: str # "BUY" or "SELL"
@dataclass
class OrderEvent:
ts: object
action: str
quantity: int
@dataclass
class FillEvent:
ts: object
action: str
quantity: int
price: float
class SmaCross:
"""Strategy: reacts only to MarketEvents, emits SignalEvents."""
def __init__(self, fast=20, slow=50):
self.fast, self.slow, self.closes, self.pos = fast, slow, deque(maxlen=slow), 0
def on_market(self, ev):
self.closes.append(ev.close)
if len(self.closes) < self.slow:
return None
f = sum(list(self.closes)[-self.fast:]) / self.fast
s = sum(self.closes) / self.slow
if f > s and self.pos <= 0:
self.pos = 1
return SignalEvent(ev.ts, "BUY")
if f < s and self.pos > 0:
self.pos = 0
return SignalEvent(ev.ts, "SELL")
return None
class SimulatedBroker:
"""Backtest execution: fills at the next bar's close plus slippage.
Swap this class for one that calls OpenAlgo /api/v1/placeorder in live trading."""
def __init__(self, slippage=0.0005):
self.slippage, self.pending = slippage, []
def send(self, order):
self.pending.append(order)
def on_market(self, ev):
fills = []
for o in self.pending:
px = ev.close * (1 + self.slippage if o.action == "BUY" else 1 - self.slippage)
fills.append(FillEvent(ev.ts, o.action, o.quantity, px))
self.pending = []
return fills
def run(db_path="market_data.duckdb", qty=1):
con = duckdb.connect(db_path, read_only=True)
rows = con.execute(
"SELECT timestamp, close FROM market_data ORDER BY timestamp").fetchall()
strategy, broker, events = SmaCross(), SimulatedBroker(), deque()
cash, position, trades = 0.0, 0, []
for ts, close in rows:
events.append(MarketEvent(ts, close))
while events:
ev = events.popleft()
if isinstance(ev, MarketEvent):
events.extend(broker.on_market(ev)) # fill orders from the previous bar
sig = strategy.on_market(ev)
if sig:
events.append(sig)
elif isinstance(ev, SignalEvent):
broker.send(OrderEvent(ev.ts, ev.action, qty))
elif isinstance(ev, FillEvent):
sign = 1 if ev.action == "BUY" else -1
position += sign * ev.quantity
cash -= sign * ev.quantity * ev.price
trades.append(ev)
last = rows[-1][1] if rows else 0
return {"trades": len(trades), "position": position, "pnl": round(cash + position * last, 2)}
if __name__ == "__main__":
print(run())
Notice the order of operations inside the loop: pending orders are filled on the new bar before the strategy sees it, so a signal on bar N can only execute on bar N+1. That one rule removes the most common source of look-ahead bias in hand-written backtests. For a deeper look at how the queue works, see event driven backtesting and event loops for traders.
Event Driven vs Vectorised Backtesting at a Glance
| Vectorised | Event driven | |
|---|---|---|
| Speed | Very fast (whole arrays at once) | Slower (one event at a time) |
| Look-ahead bias risk | Higher; easy to use future rows by mistake | Lower; strategy sees only past events |
| Order and fill realism | Approximate | Can model slippage, partial fills, order types |
| Same code for live trading | Rarely | Yes, by swapping the broker component |
| Best use | Parameter sweeps and idea screening | Final validation and production |
Common Mistakes When Building a Backtesting Engine
- Filling on the signal bar’s close. If the signal uses the close, the fill must come later.
- Ignoring costs. Brokerage, STT, exchange charges and slippage change the result for 1-minute strategies more than for daily ones.
- Mixing adjusted and unadjusted data. Corporate actions in stocks and contract rolls in futures need consistent handling.
- Session boundaries. Indicators and intraday exits should reset at 09:15 and square off before the broker’s cut-off.
- No logging. Log every event so a backtest trade can be traced and compared with live fills.
For a checklist that applies to any backtest, read 7 things a trader should understand to do proper backtesting.
6. Discussion and Future Directions
By following this detailed, step‑by‑step process and leveraging an event‑driven design, you can build a backtesting engine that mirrors live trading conditions. Integration with DuckDB provides efficient data handling for granular 1‑minute data, and incorporating the OpenAlgo PlaceOrder API ensures a smooth transition to live execution.
Future Enhancements Might Include:
- Support for sub‑minute or tick data.
- Advanced order types and more realistic execution simulations.
- Asynchronous event processing to further reduce latency.
- Integration of machine learning models for dynamic strategy adjustments.
7. Summary and Next Steps
Developing a unified backtesting and live trading engine is challenging yet rewarding. By building an event‑driven system with robust data handling via DuckDB and seamless order execution through the OpenAlgo PlaceOrder API, you can create a framework that minimizes discrepancies between simulated and live performance. This research note has outlined the architectural considerations and practical steps needed to get started, providing a solid foundation for building and refining your trading engine.
Frequently Asked Questions
What is event driven backtesting?
It is a backtest that replays data as a sequence of events (new bar, signal, order, fill) and updates the strategy and portfolio after each one, the same way a live trading system runs.
Is event driven backtesting better than vectorised backtesting?
Not always. Vectorised tests are far faster for screening ideas. Event-driven tests are more realistic and let you reuse the code for live trading, so many traders use both.
Why use DuckDB for backtesting data?
DuckDB is an embedded columnar database: no server to run, fast SQL over millions of rows, and direct reading of CSV and Parquet files, which suits 1-minute market data.
Can an event driven backtesting engine trade live with OpenAlgo?
Yes. Replace the simulated broker with a class that calls the OpenAlgo PlaceOrder endpoint, and feed live bars or quotes into the same event queue.
Learn by watching: free videos every week
Market Profile, order flow, Python, AI and OpenAlgo tutorials on YouTube.

