Hledger Analysis

When to use

Joseph asks "how much dividend?", "what were my expenses last month?", "which category was highest?"
Monthly/quarterly/annual financial reports (FY 2025-26, current FY, etc.)
Income vs expense tracking
Budget vs actual analysis
Comparing hledger holdings vs external holdings.csv
"What's my balance as of [date]?"
"Show me monthly expenses for this FY"
Reconciliation questions (what are Equity:Reconciled entries, how to verify)

System location

wiki/wealth/hledger/ — full hledger setup with 50+ files across journals, scripts, import rules, and source files.

Architecture

Import: Runs locally on Joseph's Mac (run-pipeline.sh). Downloads CSVs → preprocesses → hledger import → dedup → check.
Sync: Journal files sync to server via Dropbox.
Analysis: hledger on server (~/bin/hledger v1.52.4) for reporting. Python scripts for heavy post-processing or custom analysis.

Key directories

hledger/
├── journals/              ← Transaction data (THE source of truth)
│   ├── main.journal       ← Hub file (includes everything)
│   ├── accounts.journal   ← Chart of accounts (239 lines)
│   ├── balances.journal   ← Opening + reconciled balances
│   ├── 2025/              ← Per-account journals (14 bank accounts + cards + investments)
│   └── 2026/              ← Current year
├── scripts/               ← Python analysis + import scripts
├── import-rules/          ← hledger CSV import rules (per bank)
├── source-files/          ← Raw CSVs + PDFs from bank websites
└── reports/               ← Generated financial reports (FY expense/income summaries)

Accounts of interest

Dividends: Income:Dividend:JJU:* (per-stock: ITC, COAL, INFY, ASHLEY, etc.)
Income: Income:Consulting:, Income:Interest:, Income:Rent
Expenses: Expenses:Home:, Expenses:Car:, Expenses:Travel:, Expenses:Tax:, etc.
Transfers: Assets:Bank:Transfers (inter-account moves)
Stocks (ICICI): Assets:Invest:Stocks:ICICIDirect:* (per-stock commodity tracking)
Stocks (Zerodha): Assets:Invest:Stocks:Zerodha:{Person}:{TICKER} (JJU, Rani, Jerry, Josh)
Mutual Funds: Assets:Invest:MFs:*

Journal format (for parsing)

2026-04-18 ACH/TVS MOTOR IDIV26/...
    Assets:Bank:ICICI:Demat              6000.00
    Income:Dividend:JJU:TVSMOT        -6000.00

Header: YYYY-MM-DD [*!]? description
Postings: Account:Path amount (4+ spaces indent)
Blank line = transaction separator
Comments: lines starting with ;
Convention: income = negative amounts, expenses = positive amounts

Stock transactions (commodity notation):

2020-03-09 ICICIDirect BUY ITC (BSE)
    Assets:Invest:Stocks:ICICIDirect:ITC    250 ITC {177.65 INR}
    Expenses:Charges:Invest    26.64 INR
    Assets:Bank:ICICI:Demat    -44439.14 INR

Commodity: QTY TICKER {PRICE INR} for buys, -QTY TICKER {PRICE INR} for sells
Zero-price (bonus/demerger): QTY TICKER @ 0 INR

Parsing rules

1. Transaction header matches ^(\\d{4}-\\d{2}-\\d{2})\\s+([!]?\\s.+)

2. Postings match ^(\\s{4,})([\\w:]+)\\s+(-?[\\d,]+\\.?\\d*)

3. Blank line or another header ends a transaction

4. Skip comment-only lines (starting with ;)

Reporting commands (hledger on server)

All commands run from wiki/wealth/hledger/. Use ~/bin/hledger -f journals/main.journal.

Balance at a date

# All accounts at a specific date
~/bin/hledger -f journals/main.journal bal --end 2026-03-31 --flat -S

Depth-limited (top-level categories)
~/bin/hledger -f journals/main.journal bal --end 2026-03-31 --flat --depth 3 -S

Specific account type
~/bin/hledger -f journals/main.journal bal --end 2026-03-31 Assets:Bank --flat -S

Monthly income/expense reports

# Monthly expenses for FY 2025-26
~/bin/hledger -f journals/main.journal bal --begin 2025-04-01 --end 2026-03-31 --period monthly --flat Expenses --depth 2 -S

Monthly income for FY 2025-26
~/bin/hledger -f journals/main.journal bal --begin 2025-04-01 --end 2026-03-31 --period monthly --flat Income --depth 2 -S

Current FY (just change start date)
~/bin/hledger -f journals/main.journal bal --begin 2026-04-01 --period monthly --flat Expenses --depth 2 -S

Annual totals

# Total expenses for FY 2025-26
~/bin/hledger -f journals/main.journal bal --begin 2025-04-01 --end 2026-03-31 --flat Expenses --depth 1 -S

Total income for FY 2025-26
~/bin/hledger -f journals/main.journal bal --begin 2025-04-01 --end 2026-03-31 --flat Income --depth 1 -S

Investigating transactions

# See all transactions for an account
~/bin/hledger -f journals/main.journal print "Expenses:Gifts"

See reconcile entries
~/bin/hledger -f journals/main.journal print --end 2026-03-31 | grep -A 3 "reconcile"

Check Equity:Reconciled balance (should always be 0)
~/bin/hledger -f journals/main.journal bal Equity:Reconciled

Period syntax

--period monthly — monthly columns
--period quarterly — quarterly columns
--begin YYYY-MM-DD --end YYYY-MM-DD — date range
--end YYYY-MM-DD — everything up to that date
-S — sort by amount (descending)
--depth N — limit account hierarchy depth
--flat — don't indent sub-accounts

Analysis approach (no hledger on server)

Python scripts parse journal files directly:

Filter by account pattern (Income:Dividend:, Expenses:)
Filter by date range
Group by category/month/stock
Aggregate (sum, count, average)

For stock holdings comparison, use hledger balance commands:

~/bin/hledger -f journals/main.journal bal Assets:Invest:Stocks:ICICIDirect

See references/journal-format.md for detailed format spec.

See references/existing-system.md for documentation of all existing scripts and their capabilities.

See references/legacy-import.md for beancount conversion and legacy data import.

See scripts/beancount-to-hledger.py for the beancount → hledger converter.

Legacy Data Import (Beancount → hledger)

When importing data from a different ledger format (e.g., beancount), follow the sandbox-first pattern:

1. Convert to a separate directory (journals/beancount-legacy/), NOT into the main journals

2. Validate each output file with ~/bin/hledger -f <file> print and bal

3. Check for duplicates against existing journals before merging

4. Only add include lines to year hub files after validation passes

5. Re-run hledger -f main.journal bal after each year merge

See references/legacy-import.md for beancount-specific details.

Pitfalls

1. Hledger IS on the server — ~/bin/hledger (v1.52.4). Use it directly for reporting; Python parsing is only needed when hledger output needs heavy post-processing.

2. Hledger convention: Income amounts are negative, expenses are positive. When summing, use abs() for income.

3. Reconciled entries have balance assertions in balances.journal — these are ground truth.

4. Multi-year data: Journals span 2005–2026. Always check the relevant year directory.

5. Import is local: The server only has whatever synced via Dropbox. If data looks stale, Joseph needs to run import locally first.

6. Credit card journals are generated by import-cc-2025.py (different format than bank journals) — parser must handle both.

7. Inter-account transfers appear in both account journals — dedup-transfers.py removes duplicates after import.

8. Stock ticker mapping: hledger uses ICICI Direct tickers (ASHLEY, COALIN), while holdings.csv uses NSE tickers (ASHOKLEY, COALINDIA). Map between them when comparing.

9. ETFs vs stocks: holdings.csv has asset_class field — ETFs (etf) should be included when comparing stock holdings, not just stock type.

10. Quoting tickers with numbers: Tickers containing digits (e.g., MON100, NIFBEE) must be quoted in hledger commodity notation: "MON100" not MON100. Unquoted causes parse errors.

11. Balance assertions and multi-commodity accounts: Balance assertions (0 = AMOUNT) only work for single-commodity accounts. If an account has transactions in both main INR and explicit INR commodity, the assertion will fail. Fix by removing explicit INR suffix from bank posting lines to consolidate to main commodity.

12. Converting flat entries to commodity tracking: When replacing flat INR entries with commodity notation (e.g., 500 ASHLEY {23.8 INR}), remove the flat entries FIRST, then add commodity entries. Having both causes double-counting — the flat entries inflate the INR balance while commodity quantities are correct. Verify after removal: hledger bal ACCOUNT --no-total should show only the commodity quantities and any residual flat amounts from entries not yet converted. The INR balance should drop by the sum of removed flat entries.

13. Flat output with account paths: Use -l flag for flat output where account path appears on same line as quantity: ~/bin/hledger bal Assets:Invest:Stocks -l. Without -l, quantity and account path are on separate lines.

14. Cost basis parsing from hledger output: Posting lines have format Assets:... 100 TICKER {23.8 INR}. Quantity is after the account name, not first. Use regex \s(\d+)\s+\S+\s+\{[\d.]+ INR\} to extract.

15. Dual-commodity accounts break balance assertions: If an account has transactions in both plain numbers and explicit INR commodity (e.g., 10410.51 and -4923512.8200 INR), hledger shows them as separate lines that don't net. Ensure all postings in an account use consistent commodity notation.

16. Expenses with negative totals indicate credits (received money), not spending. Check if the transaction is misclassified — e.g., insurance maturity proceeds posted to Expenses:Insurance:Life should be Income:* instead.

17. expenses:unknown and income:unknown are uncategorized transactions from import. Review and reclassify periodically to keep reports accurate.

18. Beancount parsing requires flexible indent detection: posting lines may use 1-space, tab, or 4-space indent. Use stripped[0].isalpha() after lstrip() to identify posting lines, not just startswith("\t").

19. Beancount dates may have leading whitespace: some files have 2019-06-28 ... with a space before the date. Transaction regex must allow ^\s prefix.

20. Sandbox-first for legacy imports: never add converted files directly to main journals. Use a separate directory, validate with hledger parse + balance, then merge one year at a time via include lines.

hledger-analysis