Back to catalog

screener-in-extraction

Extract Screener.in summary + CAGR per NSE ticker.

Category 💰 Wealth

Screener.in Batch Extraction (one line per ticker)

When to use

Pull live Screener.in key stats for many NSE-listed tickers at once — screening sweeps, delegated subagent tasks ("extract data for these 10 tickers"), watchlist refreshes. Output is a compact per-ticker line, NOT the full Joseph analysis (that lives in indian-stock-analysis). No login needed; standalone page works fine.

Workflow (per ticker — serialize, don't parallelize navigations)

The browser is a single session: navigate → extract → next ticker. Batching navigations in one turn fails.

Step 0: Choose consolidated vs standalone

Always start with consolidated (/consolidated/ suffix). If the consolidated page has incomplete/old financial data (e.g., P&L only shows 2015-2016, CAGR fields blank), fall back to standalone (no suffix). Symptoms of bad consolidated data:
  • Quarterly results show dates >5 years old
  • Annual P&L has only 2-3 years of data
  • CAGR fields mostly blank
  • Summary stats show dramatically different P/E (e.g., 521 vs 50.6)

This happened with PRECWIRE (Sep 2026): consolidated page was stale, standalone had full 10Y data.

Step 1: Navigate and read summary stats

Navigate to https://www.screener.in/company/{TICKER}/ (or /consolidated/). The compact snapshot from browser_navigate contains the summary stat list near the top. Read directly:

  • Market Cap (₹ Cr), Current Price, Stock P/E, ROCE %, ROE %, Dividend Yield %, Face Value, 52W High/Low
  • PROS / CONS machine text

Step 2: Extract CAGR + summary + PROS/CONS — USE RELIABLE METHOD

The extract_screener_summary.js script regex has repeatedly failed (Aug 2026, Sep 2026) — summary stats return ~ despite documented fixes. Do NOT trust the one-shot regex script for summary stats. Instead, use this reliable two-step approach:

Step 2a: Summary stats — parse from browser_snapshot compact view (they ARE in the top section), OR use the raw text block approach:
(() => {
  const b = document.body.innerText;
  const extract = (lbl) => {
    const i = b.indexOf(lbl);
    if (i < 0) return '~';
    const m = b.slice(i, i+80).match(/₹\s*([\d,]+(?:\.\d+)?)/);
    return m ? m[1].replace(/,/g, '') : '~';
  };
  const name = (document.querySelector('h1')?.textContent||'').trim().replace(/\s+/g, ' ');
  const pidx = b.indexOf('PROS'), cidx = b.indexOf('CONS');
  let pros='', cons='';
  if (pidx >= 0 && cidx > pidx) pros = b.slice(pidx+4, cidx).split('\n').map(s=>s.trim()).filter(Boolean).join(' | ');
  if (cidx >= 0) { const end = b.indexOf('* The pros', cidx); cons = b.slice(cidx+4, end>cidx?end:cidx+2000).split('\n').map(s=>s.trim()).filter(Boolean).join(' | '); }
  return JSON.stringify({name, mc:extract('Market Cap'), pe:extract('Stock P/E'), roce:extract('ROCE'), roe:extract('ROE'), dy:extract('Dividend Yield'), pros, cons});
})()
Step 2b: CAGR tables — use the reliable th table-scoped approach (this always works):
(() => { const tbl=(t)=>{const th=[...document.querySelectorAll('th')].find(x=>x.textContent.trim()===t); if(!th)return null; const tb=th.closest('table'); return tb.innerText;}; return JSON.stringify({sales:tbl('Compounded Sales Growth'),profit:tbl('Compounded Profit Growth'),price:tbl('Stock Price CAGR')});})()

Output shape: each table's innerText, e.g. 5 Years:\t27%\n3 Years:\t20%\nTTM:\t0% plus 10 Years (often blank for recently-listed cos — that's normal, report ~). Stock Price CAGR rows are 10Y/5Y/3Y/1 Year — the 1-year figure is the PriceCAGR_1Y field.

3. Assemble the output line in the format the parent requested, e.g.:

TICKER|name|MCap|P/E|ROCE|ROE|DivYld|SalesCAGR_5Y|SalesCAGR_3Y|SalesCAGR_TTM|PAT_CAGR_5Y|PAT_CAGR_3Y|PAT_CAGR_TTM|PriceCAGR_1Y|pros_notes

Use ~ for missing values. Never fabricate.

Key gotchas

  • Custom screen query field names ≠ company page column headers. The custom screen query builder on Screener.in uses different field names than the labels visible on company pages. Example: the company page shows "Stock P/E" but the screen query syntax is Price to Earning. If the user gives you a working Screener.in screen query, reproduce it VERBATIM — do not "helpfully" reword field names. This was confirmed when a user's query (Price to Earning < 19) broke after being "corrected" to Stock P/E < 19. When in doubt, use the exact field names from the user's query, not what appears on the page.
  • CAGR figures are NOT reachable by heading traversal. Traversing sibling h2/h3 headings to find the tables returns nothing (the labels live inside table th cells, not as standalone headings — verified empty on all pages). Two validated approaches: (a) th exact-text match + th.closest('table').innerText (CAGR only), or (b) regex over document.body.innerText anchored at section labels — the recommended one-shot script in step 2, which also pulls summary stats + PROS/CONS in the same call.
  • Verify implausible CAGR values by re-navigating and dumping the raw text block: browser_console with (() => { const b=document.body.innerText; const i=b.indexOf('Compounded Sales Growth'); return b.slice(i,i+520); })() — shows the literal 5 Years:\t24% rows, no parsing. Use when a value looks off (e.g. a huge 1-Yr price CAGR). Exception: High 3Y CAGR (300%+) is valid for companies with very low base sales 3 years ago (confirmed for SIGMAADV 331%, KERNEX 497%).
  • Full browser_snapshot truncates (~15k chars) and the CAGR tables sit in the truncated zone. Do NOT page through the snapshot file (/opt/data/cache/web/browser-snapshot-*.txt) to reach CAGR — the console snippet returns the same data in a few hundred chars. Summary stats, by contrast, ARE in the compact navigate snapshot (top of page).
  • Blank 5Y/10Y CAGR = company listed < 5/10 years ago. Report the missing figures as ~, don't guess. Confirmed: AEQUS, OMNI, VIDYAWIRES have blank 5Y CAGR due to recent listing.
  • Consolidated vs standalone: the standalone page (no /consolidated/ suffix) is the default and matches the task spec "no login needed". If a task needs consolidated figures, use /consolidated/ — same extraction works.
  • Machine PROS/CONS quirks to pass through verbatim but flag when obviously misleading for financials cos (e.g., "poor sales growth" computed off a screener revenue line that excludes fee/interest income — seen with wealth managers like 360ONE where Sales CAGR showed -23%/-31% while the business compounded fine). Pass the numbers through but add a one-line caveat in notes when ROCE/ROE/sales patterns contradict the narrative.
  • Numbers are in ₹ Crores unless stated; percentages as given (e.g. 27 for 27%). Keep raw values, don't round to the nearest integer if decimals are shown (ROCE 33.2, ROE 7.96).
  • Summary stat extraction regex is UNRELIABLE (confirmed Sep 2026): The one-shot regex in the old extract_screener_summary.js has failed in at least 3 sessions despite multiple documented fixes. The regex pattern with escaped backslashes is fragile across different Screener.in page renders. Always use Step 2a (raw text block with simple indexOf+slice) for summary stats and Step 2b (th table match) for CAGR. Do NOT use the one-shot script for summary stats.
  • Consolidated page may have stale/missing data: Some tickers (confirmed: PRECWIRE) have consolidated pages with old financial data (2015-2016 era only) while the standalone page has full 10Y data. Symptoms: P/E wildly different between consolidated (521) and standalone (50.6), annual P&L only has 2-3 years, CAGR fields blank. Always check the annual P&L date range — if latest year is >3 years old, fall back to standalone. This is Step 0 in the workflow.
  • web_extract vs browser extraction: For batch extraction of 3+ tickers, web_extract is faster than browser navigation (confirmed working for 5 tickers, Sep 2026) but outputs markdown tables with | separators. The browser extraction script does NOT work with web_extract output — use different regex patterns: r'Dividend\\s+Yield\\s+([0-9.]+)\\s%' for Dividend Yield, and r'5 Years:\\s\\|\\s*([0-9.]+)%' for CAGR in markdown tables. Some summary stats (e.g., Dividend Yield for AZAD) may be missing from web_extract output if the page structure differs — default to 0.00 for missing Dividend Yield. Prefer web_extract for speed; fall back to browser only when summary stats are critical and missing from the markdown output.

Alternative: web_extract batch approach (faster for 3+ tickers)

For ticker lists of 3 or more, web_extract is ~4x faster than browser navigation because it fetches all pages in parallel. Even for small batches (3-5 tickers), web_extract is preferable — it avoids browser session overhead and returns clean markdown. However, the output is markdown with | separators, requiring different parsing.

Extraction pattern (Python with web_extract):
import re

def parse_screener(content, ticker):
    dy = "0.00"  # Default if not found
    s5 = "~"
    s3 = "~"
    
    # Dividend Yield: "Dividend Yield   0.14 %"
    dy_match = re.search(r'Dividend\s+Yield\s+([0-9.]+)\s*%', content)
    if dy_match:
        dy = dy_match.group(1)
    
    # Sales CAGR: markdown table with |
    sales_match = re.search(r'Compounded Sales Growth.?\n\| ---.?\n(.*?)(?:Compounded Profit Growth|Stock Price CAGR)', content, re.DOTALL)
    if sales_match:
        sales_text = sales_match.group(1)
        s5_match = re.search(r'5 Years:\s\|\s([0-9.]+)%', sales_text)
        s3_match = re.search(r'3 Years:\s\|\s([0-9.]+)%', sales_text)
        if s5_match:
            s5 = s5_match.group(1)
        if s3_match:
            s3 = s3_match.group(1)
    
    return dy, s5, s3
Known limitations:
  • Some summary stats (e.g., Dividend Yield for AZAD) may be missing from web_extract output if the page structure differs
  • Default missing Dividend Yield to 0.00 (company likely doesn't pay dividends)
  • Blank 5Y CAGR (~) = company listed < 5 years ago, not a parsing error

Output discipline

  • One line per ticker, pipe-separated, exactly matching the parent's requested schema (they merge these into a table).
  • Lead the summary with all lines, then terse notes on anomalies — parent context windows are precious.
  • If a ticker 404s or redirects to a search page, say so explicitly rather than emitting a fabricated row. Try the alternate name (e.g., NAM-INDIA for Nippon Life) before giving up. Confirmed renames: AMARAJABATARE&M (Amara Raja Energy & Mobility, ex-Amara Raja Batteries, renamed Sep 2023) and DEEPAKNITRDEEPAKNTR (Deepak Nitrite — correct NSE symbol is DEEPAKNTR).