Cloud Storage Sync (Mac ↔ Server)

Set up bidirectional file sync between Joseph's Mac and the Hermes Linux server using cloud storage providers and rclone.

Architecture Pattern

Key insight: Only run rclone on the server. The Mac uses the provider's official app.

Mac (official app) ←→ Cloud Storage ←→ Server (rclone bisync)

Mac side: Dropbox/pCloud official client handles sync natively (better macOS integration, selective sync, conflict resolution)
Server side: rclone provides the sync capability where no official Linux client exists or is reliable
Cloud storage: The bridge between both sides

Why not rclone on Mac? The official Dropbox app is more reliable on macOS — native, handles conflicts better, supports selective sync. rclone fills the gap where official clients don't exist or are unreliable (Linux servers).

Provider Notes

Dropbox (Recommended for active sync)

Official Mac app works well
rclone backend is first-class and well-tested
OAuth setup requires creating an app at https://www.dropbox.com/developers/apps
Use "App folder" access type (not "Full Dropbox") — more secure, sufficient for single-user sync
App permissions needed: files.metadata.read, files.metadata.write, files.content.read, files.content.write
Do NOT add sharing.read/write — not needed for basic file sync, only for shared links/collaborators
Basic plan (2 GB) sufficient for wiki (markdown files are tiny)
Development status is fine — no need for production approval when you're the only user

pCloud (Better for bulk/archive storage)

Lifetime 2 TB plan available — good for cold storage
Gotcha: pCloud has restricted API access for rclone due to "severe abuse"
pCloud doesn't officially support rclone — no technical support
OAuth can be finicky — requires creating app in pCloud developer portal
Use for: large files, media, cold storage where reliability matters less
Do not use for: active wiki sync (treat as best-effort)

Google Drive

More complex OAuth setup (Google Cloud Console, consent screen, API enablement)
Multiple APIs (Drive, Docs, Sheets) add complexity
rclone works but setup is heavier than Dropbox

rclone Setup on Headless Linux

Installation

rclone may not be in system package managers. Manual binary install:

# Download (use specific version URL, not 'current' which may redirect oddly)
cd /tmp
curl -L "https://downloads.rclone.org/v1.75.0/rclone-v1.75.0-linux-amd64.zip" -o rclone.zip

Extract (unzip may not be available — use Python)
python3 -c "import zipfile; zipfile.ZipFile('rclone.zip').extractall('.')"

Install to user bin (no sudo needed)
mkdir -p ~/bin
cp rclone-*/rclone ~/bin/
chmod +x ~/bin/rclone
export PATH="$HOME/bin:$PATH"

Verify
rclone version

Gotcha: unzip is often not installed on headless servers. Use Python's zipfile module instead. Gotcha: /usr/local/bin/ may require root. Use ~/bin/ and add to PATH.

Configuration

# Interactive config — follow prompts
rclone config

Or create ~/.config/rclone/rclone.conf directly:
[dropbox]
type = dropbox
token = {"access_token":"...","token_type":"bearer","refresh_token":"...","expiry":"..."}

OAuth Flow

1. Create app at provider's developer console

2. Get app key + secret

3. Set Redirect URL in app settings: Add http://localhost:53682/ to OAuth 2.0 redirect URIs (required for rclone authorize)

4. Run rclone config → choose Dropbox → enter key/secret

5. rclone opens browser for OAuth authorization

6. On headless server: rclone shows a URL to visit manually, paste auth code back

For headless servers: rclone can generate an auth URL you visit on your Mac, then paste the code back into the terminal.

Bidirectional Sync

rclone bisync (Recommended)

# Initial sync (must run once before bisync)
rclone sync /opt/data/wiki dropbox:/wiki --dry-run  # preview first
rclone sync /opt/data/wiki dropbox:/wiki             # actual sync

Initialize bisync (required once)
rclone bisync /opt/data/wiki dropbox:/wiki --resync

Normal bidirectional sync
rclone bisync /opt/data/wiki dropbox:/wiki

Cron Setup

rclone is not a daemon — it runs once and exits. Schedule with cron:

# Every 5 minutes
/5    * ~/bin/rclone bisync /opt/data/wiki dropbox:/wiki --quiet

Why cron? rclone doesn't watch for changes. Cron triggers periodic sync. 5-minute intervals are fine for markdown files.

Alternative: rclone mount

rclone mount dropbox:/wiki /mnt/wiki --vfs-cache-mode writes

More "real-time" but slower for writes
Better for read-heavy use cases
Not recommended for active bidirectional sync

Integration with Wiki Workflow

See wiki-input-output-split skill for where content lives. Cloud sync affects:

produce/blogs/jjude/ and produce/blogs/ctofieldnotes/ — synced FROM Mac via Dropbox
produce/hermes/ — Hermes writes here, syncs TO Mac via Dropbox
consume/ — Hermes writes here, optionally syncs to Mac

Sync direction matters: Some directories are read-only from Mac's perspective (Mac syncs to server), others are write-only from Hermes's perspective (Hermes syncs to Mac). Plan which directories participate in bidirectional sync.

Pitfalls

rclone bisync requires --resync on first run — without it, bisync refuses to start
pCloud API restrictions — treat pCloud sync as best-effort, not reliable infrastructure
Permission errors on headless Linux — use ~/bin/ not /usr/local/bin/
Zip extraction without unzip — use Python's zipfile module
Token refresh — rclone handles this automatically for Dropbox, but verify after long idle periods
Conflict resolution — rclone bisync handles conflicts, but review periodically for wiki content
Use existing .env files — Don't create new .env.dropbox or similar; add credentials to the existing /opt/data/.env file
No server bounce needed — .env changes take effect on next rclone run, no service restart required
Redirect URL is required — rclone authorize won't work without setting http://localhost:53682/ in Dropbox app settings first

cloud-sync