Rajandran R Creator of OpenAlgo Sketchmaker, RagZ. Building GenAI Applications. Telecom Engineer turned Full-time Derivative Trader. Mostly Trading Nifty, Banknifty, High Liquid Stock Derivatives. Trading the Markets Since 2006 onwards. Using Market Profile and Orderflow for more than a decade. Designed and published 100+ open source trading systems on various trading tools. Now the recent curiosity on building Trading AI Agents.

How to Download Last 10 Years of Intraday Data in CSV Format – Python Tutorial

8 min read

To download intraday data to CSV with Python, connect to your broker through OpenAlgo, loop over a list of symbols, call client.history() with a 1-minute interval and save each symbol to its own CSV file. The script in this tutorial does exactly that for up to 10 years of history, with a resume option if the download stops midway.

Intraday data is a treasure trove for traders, analysts, and researchers. This blog will guide you through creating a Python script to download historical intraday data at 1-minute intervals for multiple symbols. You’ll also learn how to customize the time range, resume downloads, and optimize the script for efficiency.

Key takeaways
  • The script downloads 1-minute OHLCV bars for every symbol in symbols.csv and writes one CSV per symbol in DATE, TIME, OPEN, HIGH, LOW, CLOSE, VOLUME format.
  • How far back you can go depends on your broker’s historical data API: 10+ years with Zerodha, around 7 years with Fyers.
  • Zerodha now bundles historical data with the ₹500 per month Kite Connect plan; the free Personal plan does not include it.
  • A checkpoint file lets you resume a large download, and failed symbols are retried three times.
  • Newer OpenAlgo versions include Historify, which stores candles in a local DuckDB database and exports them to CSV or Parquet.

Why Historical Intraday Data Matters

Intraday data is essential for backtesting strategies, analyzing market trends, and developing automated trading algorithms. With the right tools and a bit of programming, you can build a reliable pipeline to fetch and store this data.

Python for traders, the quick way in

BEGINNER FRIENDLY · RATED 5.0

Python for traders, the quick way in

A crash course that gets you from zero to writing your own market data scripts.

  • Python basics with market examples
  • Fetch, clean and chart price data
  • Lifetime access
Start learning →by Marketcalls

What You’ll Need

1. OpenAlgo Installation

  • Install and configure OpenAlgo. Ensure you are running OpenAlgo with Python 3.9 or higher. Refer to the official OpenAlgo documentation for detailed installation instructions.
  • If you have not updated to the latest OpenAlgo, update before proceeding.

2. OpenAlgo Supported Brokers API

  • With Zerodha API + Addon Historical Chart Data (Cost Rs 4000 per month when this tutorial was written), downloading 10+ years of data is possible.
  • With Fyers API, up to 7+ years of data can be downloaded.
  • Different brokers have their own data download limits. Make sure to check your broker’s data policy.
  • Subscribe to the Zerodha API and the Historical Chart Data API. The total cost for accessing these APIs is Rs 4000 per month.

Update on Zerodha pricing: the ₹4,000 figure above was the Kite Connect subscription plus the separate historical data add-on at the time of writing. Zerodha has since made the Personal API plan free (orders only, no market data) and bundled historical candle data into the Connect plan at ₹500 per app per month (Zerodha support). To download intraday data you need the Connect plan. I covered the change in Zerodha makes its trading API free for personal use.

3. Editor and Tools

  • Use any reliable Python editor, such as VS Code, Cursor, or Windsurf.

4. Ensure OpenAlgo is Running

  • Before running the script, verify that OpenAlgo is running and accessible locally at http://127.0.0.1:5000.
  • The Python script ieod.py is available in the download folder inside OpenAlgo. Simply open a new terminal and run ieod.py from OpenAlgo itself.

5. Symbols List

Prepare a file named symbols.csv with a list of symbols, one per line (you can file this sample file inside the /openalgo/downloads folder inside openalgo

RELIANCE
ICICIBANK
HDFCBANK
SBIN
TCS
INFY

Features of the Script

Know Your Broker’s Data Download Limit

This script downloads only 1-minute intraday data. Feel free to adjust the code to download a custom timeframe based on your broker’s available historical data.

  1. User-Friendly Options: Choose between different time ranges like today, last 5 days, 30 days, or even 10 years.
  2. Resume Downloads: If the script stops, it resumes from the last processed symbol using a checkpoint.
  3. Batch Processing: Downloads are handled in smaller batches to optimize memory usage.
  4. Retry Mechanism: Automatically retries failed downloads up to three times.

Running the Script

How to Start the Download Process

  1. Open a terminal.
  2. Navigate to the OpenAlgo download folder.
  3. Run the script using python ieod.py
  4. If you have not updated OpenAlgo, ensure you are on the latest version before running the script.

How It Works

  1. Interactive Mode: Users can choose between a fresh download or resuming a previous session.
  2. Time Period Options: Select a time range from today’s data to the last 10 years.
  3. Checkpointing: If interrupted, the script resumes from the last saved symbol.

Complete Python Code

import pandas as pd
import logging
import os
import time
import gc
from openalgo import api
from datetime import datetime, timedelta

# Initialize the API client
client = api(api_key='your_api_key_here', host='http://127.0.0.1:5000')

# Path to the CSV file
symbols_file = "symbols.csv"
output_folder = "symbols"
checkpoint_file = "checkpoint.txt"

# Create the output folder if it doesn't exist
os.makedirs(output_folder, exist_ok=True)

# Set up logging
logging.basicConfig(
    filename="data_download.log",
    level=logging.INFO,
    format="%(asctime)s - %(levelname)s - %(message)s",
)

# Function to get start date based on user selection
def get_date_range(option):
    today = datetime.now()
    if option == 1:
        return today.strftime("%Y-%m-%d"), today.strftime("%Y-%m-%d")
    elif option == 2:
        return (today - timedelta(days=5)).strftime("%Y-%m-%d"), today.strftime("%Y-%m-%d")
    elif option == 3:
        return (today - timedelta(days=30)).strftime("%Y-%m-%d"), today.strftime("%Y-%m-%d")
    elif option == 4:
        return (today - timedelta(days=90)).strftime("%Y-%m-%d"), today.strftime("%Y-%m-%d")
    elif option == 5:
        return (today - timedelta(days=365)).strftime("%Y-%m-%d"), today.strftime("%Y-%m-%d")
    elif option == 6:
        return (today - timedelta(days=365 * 2)).strftime("%Y-%m-%d"), today.strftime("%Y-%m-%d")
    elif option == 7:
        return (today - timedelta(days=365 * 5)).strftime("%Y-%m-%d"), today.strftime("%Y-%m-%d")
    elif option == 8:
        return (today - timedelta(days=365 * 10)).strftime("%Y-%m-%d"), today.strftime("%Y-%m-%d")
    else:
        raise ValueError("Invalid selection")

# Prompt user for fresh download or continuation
print("Select download mode:")
print("1) Fresh download")
print("2) Continue from the last checkpoint")

try:
    mode_choice = int(input("Enter your choice (1-2): "))
    if mode_choice not in [1, 2]:
        raise ValueError("Invalid selection")
except ValueError:
    print("Invalid input. Please restart the script and select a valid option.")
    exit()

# Prompt user for time period
print("Select the time period for data download:")
print("1) Download Today's Data")
print("2) Download Last 5 Days Data")
print("3) Download Last 30 Days Data")
print("4) Download Last 90 Days Data")
print("5) Download Last 1 Year Data")
print("6) Download Last 2 Years Data")
print("7) Download Last 5 Years Data")
print("8) Download Last 10 Years Data")

try:
    user_choice = int(input("Enter your choice (1-8): "))
    start_date, end_date = get_date_range(user_choice)
except ValueError as e:
    print("Invalid input. Please restart the script and select a valid option.")
    exit()

# Read symbols from CSV
symbols = pd.read_csv(symbols_file, header=None)[0].tolist()

# Handle checkpoint logic
if mode_choice == 2 and os.path.exists(checkpoint_file):
    with open(checkpoint_file, "r") as f:
        last_processed = f.read().strip()
    # Skip symbols up to the last processed one
    if last_processed in symbols:
        symbols = symbols[symbols.index(last_processed) + 1:]
elif mode_choice == 1:
    # Remove existing checkpoint for fresh download
    if os.path.exists(checkpoint_file):
        os.remove(checkpoint_file)

# Process symbols in batches
batch_size = 10  # Adjust this value based on your memory availability
for i in range(0, len(symbols), batch_size):
    batch = symbols[i:i + batch_size]
    for symbol in batch:
        logging.info(f"Starting download for {symbol}")
        try:
            # Skip already downloaded symbols
            output_file = os.path.join(output_folder, f"{symbol}.csv")
            if os.path.exists(output_file):
                logging.info(f"Skipping {symbol}, already downloaded")
                continue

            # Fetch historical data for the symbol
            for attempt in range(3):  # Retry up to 3 times
                try:
                    response = client.history(
                        symbol=symbol,
                        exchange="NSE",
                        interval="1m",
                        start_date=start_date,
                        end_date=end_date
                    )
                    break
                except Exception as e:
                    logging.warning(f"Retry {attempt + 1} for {symbol} due to error: {e}")
                    time.sleep(5)  # Wait before retrying
            else:
                logging.error(f"Failed to download data for {symbol} after 3 attempts")
                continue

            # Convert the response to a DataFrame if it's a dictionary
            if isinstance(response, dict):
                if "timestamp" in response:
                    df = pd.DataFrame(response)
                else:
                    logging.error(f"Response for {symbol} missing 'timestamp' key: {response}")
                    continue
            else:
                df = response

            # Ensure the DataFrame is not empty
            if df.empty:
                logging.warning(f"No data available for {symbol}")
                continue

            # Reset the index to extract the timestamp
            df.reset_index(inplace=True)

            # Rename and split the timestamp column
            df['DATE'] = pd.to_datetime(df['timestamp']).dt.date
            df['TIME'] = pd.to_datetime(df['timestamp']).dt.time

            # Add SYMBOL column and rearrange columns
            df['SYMBOL'] = symbol
            df = df[['SYMBOL', 'DATE', 'TIME', 'open', 'high', 'low', 'close', 'volume']]
            df.columns = ['SYMBOL', 'DATE', 'TIME', 'OPEN', 'HIGH', 'LOW', 'CLOSE', 'VOLUME']

            # Save to CSV file
            df.to_csv(output_file, index=False)
            logging.info(f"Data for {symbol} saved to {output_file}")

            # Save checkpoint after successfully processing the symbol
            with open(checkpoint_file, "w") as f:
                f.write(symbol)

            # Clear DataFrame and force garbage collection
            del df
            gc.collect()

        except Exception as e:
            logging.error(f"Failed to download data for {symbol}: {e}")

        # Delay to avoid rate limiting
        time.sleep(3)

    logging.info(f"Batch of {batch_size} symbols completed.")

logging.info("All data downloaded.")

How to download intraday data in CSV and verify it

Once the script finishes, every symbol has its own file in the symbols folder. Before you backtest anything, check that the file is complete. This short snippet counts the bars for each day and flags short sessions:

import pandas as pd

df = pd.read_csv("symbols/RELIANCE.csv", parse_dates=["DATE"])

bars_per_day = df.groupby("DATE").size()
print("Trading days:", len(bars_per_day))
print("From", df["DATE"].min().date(), "to", df["DATE"].max().date())

# A full NSE cash session (9:15 to 15:29) has 375 one-minute bars
short_days = bars_per_day[bars_per_day < 375]
print("Days with fewer than 375 bars:", len(short_days))
print(short_days.tail(10))

A short day is not always a problem: Muhurat trading sessions and special sessions have fewer bars. But a run of short days in the middle of a normal month usually means a broker API gap or a rate-limit failure, and those dates are worth downloading again.

Convert 1-minute data to 5-minute, 15-minute or hourly bars

Store the 1-minute data once and build every other timeframe from it with pandas:

import pandas as pd

df = pd.read_csv("symbols/RELIANCE.csv")
df["DATETIME"] = pd.to_datetime(df["DATE"] + " " + df["TIME"])

ohlc = {"OPEN": "first", "HIGH": "max", "LOW": "min", "CLOSE": "last", "VOLUME": "sum"}

# 5-minute and 15-minute bars line up with the 9:15 open automatically
df5 = df.set_index("DATETIME").resample("5min").agg(ohlc).dropna(subset=["OPEN"])

# Hourly bars need a 15-minute offset so they start at 9:15, 10:15, ...
df60 = df.set_index("DATETIME").resample("60min", offset="15min").agg(ohlc).dropna(subset=["OPEN"])

df5.to_csv("RELIANCE_5min.csv")
print(df60.head())

How much disk space does 10 years of 1-minute data need?

One NSE session gives 375 one-minute bars. With roughly 250 trading days a year, ten years is about 9 lakh rows per symbol, which works out to roughly 50 to 60 MB per symbol as CSV. For the full Nifty 500 universe, plan for tens of gigabytes, or save to Parquet, which is much smaller and faster to load.

Historify: the built-in alternative in newer OpenAlgo versions

Since this script was published, OpenAlgo has added Historify, a page under Profile, Historify that downloads broker candles into a local DuckDB database. It can run multi-symbol jobs from a watchlist in the background with pause, resume and retry, fill gaps incrementally, and export to CSV, TXT, ZIP or Parquet. Once data is stored, any script can read it without calling the broker again:

# Read candles already stored in OpenAlgo's Historify (DuckDB) database
df = client.history(
    symbol="RELIANCE",
    exchange="NSE",
    interval="1m",
    start_date="2016-01-01",
    end_date="2026-11-06",
    source="db"   # "api" (default) fetches from the broker instead
)
print(df.tail())

The ieod.py script above is still useful when you want a plain folder of CSV files for Amibroker or another tool. If you want to keep the data in sync every day, Historify’s scheduled downloads are easier to maintain. For rate limits on larger downloads, see fetching broker historical data with OpenAlgo without hitting API rate limits, and to chart the data, try TradingView Lightweight Charts with OpenAlgo.

Common mistakes

  • Running without the history add-on: if your broker plan does not include historical data, the API returns an error or an empty response and the script logs “No data available”.
  • Wrong symbol format: use OpenAlgo’s common symbol format (for example RELIANCE, SBIN) and the exchange NSE, not the broker’s own token or trading symbol.
  • Forgetting the checkpoint: choose “Continue from the last checkpoint” after an interruption. A fresh download deletes the checkpoint, although symbols already saved are still skipped.
  • Ignoring corporate actions: intraday data from broker APIs is usually not adjusted for splits and bonuses, so adjust prices before a long backtest.

Key Tips

  1. Optimize Batch Size: Adjust batch_size based on your system’s memory.
  2. Retry Logic: Ensure you have a stable internet connection to avoid retries.
  3. Monitor Resources: Use tools like Task Manager or htop to track memory usage.
  4. Update OpenAlgo: Always ensure you are using the latest version of OpenAlgo for the best performance and compatibility.

Frequently Asked Questions

Where can I download intraday data for NSE stocks?

Your broker’s historical data API is the most reliable source. Through OpenAlgo the same script works with any supported broker; only the history depth changes. For free options and their limits, see Python libraries with historical data APIs.

How do I download intraday data from Zerodha?

Subscribe to the Kite Connect plan (historical data is included), connect it to OpenAlgo, add your symbols to symbols.csv and run python ieod.py. Choose option 8 for the last 10 years.

Can I download intraday data for F&O contracts?

Yes. Change the exchange from NSE to NFO and use OpenAlgo option or futures symbols in symbols.csv. Expired contracts depend on whether your broker’s API serves them.

Why does the CSV have fewer rows than expected?

Brokers limit how much minute data one request can return, and some only keep a few years. Check the log file data_download.log for errors, then rerun the affected symbols for that period.


Happy coding, and may your data-driven strategies thrive!

Python Elite membership

MEMBERSHIP

Python Elite membership

15 Python, AI and algo trading programs in one plan, including QuantFlow and AI Bootcamp live mentorship.

  • Python from fundamentals to trading apps
  • Backtesting, broker APIs and OpenAlgo
  • Live QuantFlow and AI Bootcamp sessions
View Python Elite →by Marketcalls

Learn by watching: free videos every week

Market Profile, order flow, Python, AI and OpenAlgo tutorials on YouTube.

Rajandran R Creator of OpenAlgo Sketchmaker, RagZ. Building GenAI Applications. Telecom Engineer turned Full-time Derivative Trader. Mostly Trading Nifty, Banknifty, High Liquid Stock Derivatives. Trading the Markets Since 2006 onwards. Using Market Profile and Orderflow for more than a decade. Designed and published 100+ open source trading systems on various trading tools. Now the recent curiosity on building Trading AI Agents.

Leave a Reply

Get Notifications, Alerts on Market Updates, Trading Tools, Automation & More