Data Storage in Python

Storing data in a .txt file is the simplest form of data storage in Python. It is perfect for logs, configuration settings, notes, or any simple data that does not need a rigid structure. Python handles text files natively, so no imports are required.

Write to a text file

You open the file in write mode ("w") and use write() to add content. The with statement is the recommended approach because it automatically closes the file for you, even if an error occurs.

with open("data.txt", "w") as file:
    file.write("Hello, World!\n")
    file.write("Welcome to Python.")

Read from a text file

Open the file in read mode ("r") and use read() to pull the whole contents into a string.

with open("data.txt", "r") as file:
    content = file.read()
    print(content)

Understanding file modes

The second argument to open() is the mode, and choosing the right one matters. Using "w" when you meant "a" will silently erase your file.

CSV (Comma-Separated Values) is the ideal format for tabular data: anything that fits neatly into rows and columns, such as spreadsheets, logs, exports and reports. CSV files open directly in Excel and Google Sheets, which makes them a favourite for sharing data with non-programmers. Python includes a built-in csv module, so no installation is needed.

Writing to a CSV file

Each inner list becomes one row. The newline="" argument prevents blank lines from appearing between rows on Windows, a very common beginner gotcha.

import csv

data = [
    ["Name", "Age"],
    ["Alice", 25],
    ["Bob", 30]
]

with open("data.csv", "w", newline="") as file:
    writer = csv.writer(file)
    writer.writerows(data)

Reading from a CSV file

The reader gives you back each row as a list of strings, which you can loop over.

with open("data.csv", "r") as file:
    reader = csv.reader(file)
    for row in reader:
        print(row)

Working with columns by name

When your CSV has a header row, DictReader lets you access each value by its column name instead of a numeric index, which makes your code far easier to read.

import csv

with open("data.csv", "r") as file:
    reader = csv.DictReader(file)
    for row in reader:
        print(row["Name"], "is", row["Age"])

JSON (JavaScript Object Notation) is excellent for hierarchical or structured data, such as dictionaries, nested lists, or data returned from web APIs. It maps almost perfectly onto Python's dictionaries and lists, and it is readable by nearly every programming language, which is why it has become the standard format for exchanging data on the web. Python's built-in json module handles it all.

Writing JSON

json.dump() converts a Python dictionary into JSON text and writes it to a file. Adding indent=4 makes the saved file neatly formatted and human-readable.

import json

data = {"name": "Alice", "age": 25, "skills": ["Python", "SQL"]}

with open("data.json", "w") as file:
    json.dump(data, file, indent=4)

Reading JSON

json.load() reads the file and converts the JSON back into a Python dictionary, which you can then access by key exactly like any other dictionary.

import json

with open("data.json", "r") as file:
    data = json.load(file)
    print(data["name"])
    print(data["skills"])

Sometimes you want to save a Python object exactly as it is, a list, a dictionary, or even a custom class instance, and load it back later without rebuilding it. The pickle module does this by serializing the object into a binary format. Serialization simply means converting an object into a stream of bytes that can be stored and later reconstructed.

Pickle example

Note the binary modes: "wb" for writing bytes and "rb" for reading bytes. Pickle files are not text, so they must be opened in binary mode.

import pickle

my_data = {"username": "admin", "scores": [10, 20, 30]}

# Save (serialize) the object
with open("data.pkl", "wb") as file:
    pickle.dump(my_data, file)

# Load (deserialize) it back
with open("data.pkl", "rb") as file:
    loaded_data = pickle.load(file)

print(loaded_data)

Pickle vs JSON

The two are easy to confuse, so here is the clear difference:

Once your data grows or you need to search, filter and sort it efficiently, files start to feel limiting. This is where a database comes in. SQLite is a lightweight, file-based database built right into Python, no server or installation required. The entire database lives in a single file, which makes it perfect for small to medium applications, prototypes, and mobile apps.

A complete SQLite example

This example connects to a database (creating it if it does not exist), creates a table, inserts a row, and reads it back.

import sqlite3

# Connect to a database, or create one if it does not exist
conn = sqlite3.connect("mydata.db")
cursor = conn.cursor()

# Create a table
cursor.execute("CREATE TABLE IF NOT EXISTS users (name TEXT, age INTEGER)")

# Insert data
cursor.execute("INSERT INTO users VALUES ('Alice', 25)")
conn.commit()

# Read data
cursor.execute("SELECT * FROM users")
print(cursor.fetchall())

conn.close()

The three steps that trip people up

• commit(): changes such as inserts are not saved until you call conn.commit(). Forget this and your data vanishes when the program ends.
• close(): always close the connection when you are done to release the file.
• fetchall(): a SELECT query does not return rows directly. You retrieve them with fetchall() for all rows or fetchone() for a single row.

Use placeholders to stay safe

Never build SQL queries by joining strings with user input, as it opens you up to SQL injection attacks. Use ? placeholders and pass the values separately.

name = "Bob"
age = 30

# Safe: values are passed separately as a tuple
cursor.execute("INSERT INTO users VALUES (?, ?)", (name, age))
conn.commit()

For larger applications, especially multi-user web apps where many people read and write data at the same time, you will move up to a full database server such as MySQL or PostgreSQL. These run as a separate service that your Python program connects to over a network, and they handle far larger volumes of data reliably.

Installing the connectors

Each database has its own Python library, installed with pip:

# For MySQL
pip install mysql-connector-python

# For PostgreSQL
pip install psycopg2

Example: connecting to MySQL

The pattern is very similar to SQLite. The main difference is the connection step, where you provide the host, username, password and database name.

import mysql.connector

conn = mysql.connector.connect(
    host="localhost",
    user="root",
    password="yourpassword",
    database="mydb"
)

cursor = conn.cursor()
cursor.execute("SELECT * FROM users")
print(cursor.fetchall())
conn.close()

When your data needs to be accessible from anywhere, backed up automatically, or shared across web and mobile apps, cloud storage is the answer. Python has mature libraries for every major cloud provider. These all require authentication with the service, so they are a step up in complexity, but the storage concepts remain the same.

These are well suited to web applications, automated backups, and mobile integrations. Because each provider handles authentication differently, always follow the official documentation for the library you choose.

Here is every method side by side, so you can compare them at a glance.

There is no single best way to store data in Python. The right choice depends on your project, and it comes down to four questions:

• What type of data are you storing? Plain notes suit text files, tables suit CSV, nested data suits JSON, and Python objects suit pickle.
• How much data do you have? A handful of records is fine in a file. Thousands of records that need searching call for a database.
• Who or what will access the data? If only your own Python program reads it, pickle is convenient. If other languages or people need it, use JSON, CSV or a database.
• Do you need local or remote access? Local projects can use files or SQLite. Remote and multi-user projects need a database server or cloud storage.

A good rule of thumb for beginners: start simple with files, move to JSON when your data becomes structured, and step up to SQLite the moment you find yourself wanting to search or filter your data. Master these and you will have the right tool for almost any project.