readwise-sync

Sync your saved Readwise Reader library to local Markdown files, organized by category for offline search and indexing (e.g. with qmd).

By default, feed documents are excluded — feed is an ingestion stream, not your saved library. Pass --include-feed to opt in, or --location feed to sync just the feed in a one-off run.

Features

• Fetches saved-library documents by default (new, later, archive, shortlist); feed is opt-in
• Stitches highlights to parent documents
• Converts HTML to clean Markdown
• Organizes by category (article, book, pdf, etc.)
• Incremental sync (only updates changed documents)
• Tracks new highlights on existing documents
• Staged deletion (moved to .trash/, permanently deleted after 30 days)
• Optional webhook listener for instant event delivery from Readwise
• File-locked state to keep scheduled and webhook-triggered syncs safe

Installation

pip install -e .

This installs two console scripts:

• readwise-sync — pull a snapshot of your library to local Markdown
• readwise-webhook — start the HTTP listener for instant event delivery

You can also install from a checkout without the editable mode:

pip install .

Configuration

Both commands read from environment variables:

Variable	Purpose	Default
READWISE_TOKEN	Readwise API access token	(required)
READWISE_SYNC_DIR	Where to write Markdown files	~/knowledge/readwise
READWISE_WEBHOOK_SECRET	Shared secret for the webhook listener	(unset = no verification)
HOST / PORT	Bind address for the webhook listener	127.0.0.1:8080
READWISE_WEBHOOK_LOG_FILE	Log file for the webhook listener	~/readwise-sync/webhook.log
READWISE_HIGHLIGHT_DELAY_SECONDS	Wait before checking a new highlight persists	3

Get a token at https://readwise.io/access_token.

Usage

One-shot sync

# Preview changes without writing
readwise-sync --dry-run

# Full sync of saved-library locations
readwise-sync

# Sync one location only (no deletion handling, no highlight backfill)
readwise-sync --location archive

# Include feed in a default multi-location run
readwise-sync --include-feed

# Refresh highlights for every document (slow backfill mode)
readwise-sync --refresh-highlights

# Re-fetch one document by ID
readwise-sync --id <document_id> --force

Webhook listener

The webhook listener is optional. It runs on 127.0.0.1:8080 by default and expects to receive Readwise webhook events at POST /webhook/readwise. Expose it through a tunnel (e.g. Cloudflare Tunnel, ngrok, Tailscale Funnel) to receive instant delivery events.

readwise-webhook

The listener only enqueues single-document syncs for events that change saved content (reader.non_feed_document.created, reader.document.archived). Feed-only and reader.any_document.* events are logged and ignored, so the listening service cannot accidentally sync your feed into the local mirror.

File layout

~/knowledge/readwise/
├── article/
│   └── some-article-abc123.md
├── book/
│   └── thinking-fast-and-slow-def456.md
├── pdf/
├── epub/
├── tweet/
├── video/
├── podcast/
├── .trash/
│   └── 2025-02-01/
│       └── deleted-doc-xyz789.md
└── sync_state.json

Document format

---
title: "Article Title"
author: "Author Name"
published_date: 2024-03-15
category: article
source_url: https://example.com/article
readwise_url: https://read.readwise.io/...
readwise_id: 01abc123def456...   # Readwise ULID
synced_at: 2025-02-01T12:00:00Z
---

## Highlights

> First highlighted passage

*My annotation on this highlight*

> Second highlighted passage

## Content

Full document text...

Searching with qmd

After the first sync, register the local mirror as a QMD collection:

# Register the whole mirror
qmd collection add ~/knowledge/readwise --name readwise --mask "**/*.md"

# Optional: per-category sub-collections
qmd collection add ~/knowledge/readwise/article --name readwise-articles --mask "**/*.md"
qmd collection add ~/knowledge/readwise/book    --name readwise-books    --mask "**/*.md"
qmd collection add ~/knowledge/readwise/pdf     --name readwise-pdfs     --mask "**/*.md"
qmd collection add ~/knowledge/readwise/tweet   --name readwise-tweets   --mask "**/*.md"
qmd collection add ~/knowledge/readwise/video   --name readwise-videos   --mask "**/*.md"

Or run ./setup_qmd.sh to do the same thing.

Then:

qmd search "machine learning" -c readwise         # All categories
qmd search "productivity"      -c readwise-books   # Just books
qmd search "AI alignment"      -c readwise-articles
qmd get /absolute/path/to/document.md

After a sync, refresh the index:

qmd update -c readwise       # fast keyword refresh
qmd embed -c readwise        # slower, adds embeddings for semantic search

Deployment

Local cron

# Sync twice daily at 8am and 8pm
0 8,20 * * * READWISE_TOKEN=... /usr/bin/env readwise-sync >> /var/log/readwise_sync.log 2>&1

launchd (macOS)

Two LaunchAgents that share the same env file:

• com.readwise.readwise-sync — StartInterval: 21600 (6h), runs readwise-sync in the foreground
• com.readwise.readwise-webhook — KeepAlive: true, runs readwise-webhook on 127.0.0.1:8080. Pair with a Cloudflare Tunnel or similar so Readwise can reach it.

The bundled start.sh and start-webhook.sh are thin wrappers that source an env file before exec'ing the console scripts, useful if you want a single file that knows about your ~/.config/readwise-sync/env layout.

systemd (Linux)

# /etc/systemd/system/readwise-webhook.service
[Unit]
Description=Readwise sync webhook listener
After=network-online.target

[Service]
EnvironmentFile=/etc/readwise-sync/env
ExecStart=/usr/local/bin/readwise-webhook
Restart=on-failure

[Install]
WantedBy=multi-user.target

Incremental updates

The sync script tracks:

• Which documents have been synced (sync_state.json)
• Which highlights exist on each document

On subsequent runs:

• New documents → created
• Documents with new highlights → regenerated with all highlights
• Unchanged documents → skipped
• Deleted from Readwise → moved to .trash/

Trash management

Deleted documents are moved to .trash/{date}/ rather than permanently deleted. After 30 days (configurable via the TRASH_RETENTION_DAYS constant in src/readwise_sync/sync.py), they're permanently removed.

To restore a deleted document, move it back to its category folder and delete its entry from sync_state.json.

Development

Run the test suite:

python -m unittest discover -s tests -v

The project uses a src/ layout with the readwise_sync package. Console scripts are defined in pyproject.toml and point at the main functions in src/readwise_sync/sync.py and src/readwise_sync/webhook_server.py.

License

MIT — see LICENSE.