Available .com Growth Keywords Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Complete eight July 2026 Worldwide categories and generate a standalone list of query.domains-confirmed unregistered .com keyword domains with SimilarWeb traffic change above 1000%, 28-day traffic above 300K, and at most three words.

Architecture: Reuse the resumable category exporter and monthly catalog, add three source contracts, expose a query.domains check-only client path, and build a focused report in a separate module/template so the existing broad report remains unchanged. Validated per-domain caches and sanitized query.domains caches are the durable restart boundary.

Tech Stack: Python 3 standard library, local Express/Puppeteer SimilarWeb wrapper, query.domains HTTPS API, dependency-free HTML/CSS/JavaScript, Node and Python unit tests, Chromium QA.

Repository note: /Users/project is not a Git worktree. Do not fabricate commit steps; use passing tests and generated-file audits as checkpoints.

---

Task 1: Add Reference, Real Estate, and Pets & Animals source contracts

Files: - Modify: /Users/project/1B/sites/关键词/ahrefs-top-websites-worldwide-2026-07/test_export_category.py - Create: /Users/project/1B/sites/关键词/ahrefs-top-reference-worldwide-2026-07/01_domains.csv - Create: /Users/project/1B/sites/关键词/ahrefs-top-real-estate-worldwide-2026-07/01_domains.csv - Create: /Users/project/1B/sites/关键词/ahrefs-top-pets-animals-worldwide-2026-07/01_domains.csv

• [ ] Step 1: Extend the failing source-contract test

Add these entries to NEW_CATEGORY_SOURCES:

"Reference": ROOT / "ahrefs-top-reference-worldwide-2026-07" / "01_domains.csv",
"Real Estate": ROOT / "ahrefs-top-real-estate-worldwide-2026-07" / "01_domains.csv",
"Pets & Animals": ROOT / "ahrefs-top-pets-animals-worldwide-2026-07" / "01_domains.csv",

• [ ] Step 2: Run the source test and verify it fails

Run:

python3 -m unittest -v test_export_category.ExportCategoryTests.test_new_category_sources_have_complete_top_100_metadata

Expected: failure because the three CSV files do not exist.

• [ ] Step 3: Create all three exact 100-row source CSVs

Use this header and transcribe ranks 1–100 from the user-provided July 2026 tables:

source_rank,domain,category,source,source_period,ahrefs_search_traffic,ahrefs_traffic_change,ahrefs_rank_change

Represent unchanged rank as 0, upward movement as a positive integer, downward movement as a negative integer, and NEW as NEW. Preserve Unicode domains in source form.

• [ ] Step 4: Run source tests and an independent rank/domain audit

Run:

python3 -m unittest -v test_export_category.py

Expected: all source and exporter tests pass; each new list has 100 unique domains and ranks exactly 1–100.

Task 2: Resume Science and export all new SimilarWeb categories

Files: - Generate: /Users/project/1B/sites/关键词/ahrefs-top-science-worldwide-2026-07/raw/{site_keywords,landing_pages}/*.json - Generate under each new category: raw/{site_keywords,landing_pages}/*.json, 02_site_keywords_all.csv, 03_landing_pages_all.csv, errors.csv, summary.json

• [ ] Step 1: Start a persistent SimilarWeb server and validate fresh quota/session state

Run sim login-auto only if a real endpoint returns HTTP 401. Start sim start in a persistent PTY and require a known domain to return non-empty rows. HTTP 429 is a quota pause, never success.

• [ ] Step 2: Resume Science from its validated 45/46 cache boundary

Run:

python3 export_category.py /Users/project/1B/sites/关键词/ahrefs-top-science-worldwide-2026-07 --timeout 120 --retries 2 --sleep 0.2

Expected final summary: domain_count=100, site_keywords_cached_domains=100, landing_pages_cached_domains=100.

• [ ] Step 3: Export the three new categories sequentially

Run the same command for reference, real-estate, and pets-animals. Do not overlap browser requests. If HTTP 429 occurs, stop the active exporter and preserve the exact cache boundary for the next quota window.

• [ ] Step 4: Rebuild and audit every completed category

For each category run --rebuild-only, then assert aggregate CSV row counts equal the sum of validated JSON row counts. Require errors.csv to contain only its header before treating a category as complete.

Task 3: Add query.domains check-only lookup

Files: - Modify: /Users/project/1B/sites/关键词/ahrefs-top-websites-worldwide-2026-07/test_growth_domain_report.py - Modify: /Users/project/1B/sites/关键词/ahrefs-top-websites-worldwide-2026-07/build_growth_domain_report.py

• [ ] Step 1: Write a failing check-only test

Add a test that supplies a registered domain and verifies /check is called while /whois and /traffic/<domain> are never called:

def test_lookup_checks_never_requests_registered_details(self):
    calls = []
    def fake_request(path, params):
        calls.append(path)
        return {"data": {"domains": [{"domain": "taken.com", "status": "registered"}]}}
    client = QueryDomainsClient("secret", cache_dir, request_json=fake_request, request_gap=0)
    records = client.lookup_checks(["taken.com"])
    self.assertFalse(records["taken.com"]["check"]["available"])
    self.assertEqual(calls, ["/check"])

• [ ] Step 2: Run the focused test and verify it fails

Expected: AttributeError because lookup_checks does not exist.

• [ ] Step 3: Implement lookup_checks and reuse it from lookup

Implement:

def lookup_checks(self, domains: Iterable[str]) -> dict[str, dict[str, Any]]:
    unique_domains = list(dict.fromkeys(str(domain).strip().lower() for domain in domains if domain))
    records = {domain: self._load_cache(domain) for domain in unique_domains}
    self._fetch_check_batches(unique_domains, records)
    return records

Make lookup() call lookup_checks() before its registered-domain detail phase.

• [ ] Step 4: Run all broad-report tests

Run python3 -m unittest -v test_growth_domain_report.py; expected: all tests pass and existing WHOIS/traffic behavior is unchanged.

Task 4: Build the focused available .com report

Files: - Create: /Users/project/1B/sites/关键词/ahrefs-top-websites-worldwide-2026-07/test_available_com_report.py - Create: /Users/project/1B/sites/关键词/ahrefs-top-websites-worldwide-2026-07/build_available_com_report.py - Create: /Users/project/1B/sites/关键词/ahrefs-top-websites-worldwide-2026-07/available_com_template.html

• [ ] Step 1: Write failing filter and aggregation tests

Cover strict boundaries (1000% excluded, 300K excluded), K/M/B parsing, 1–3 words, .com only, representative-row ordering, unique candidates, and available-only final rows.

Required interface:

parse_metric(value: Any) -> float | None
select_source_rows(rows, threshold=1000.0, min_traffic=300000.0, max_words=3) -> list[dict]
build_keyword_summaries(rows) -> list[dict]
available_domain_rows(summaries, records) -> list[dict]

• [ ] Step 2: Run focused tests and verify they fail because the module is absent

Run python3 -m unittest -v test_available_com_report.py.

• [ ] Step 3: Implement strict filtering and representative aggregation

parse_metric removes commas, an optional %, and applies K=1e3, M=1e6, B=1e9. select_source_rows enriches rows with numeric traffic_change_percent, numeric traffic_28d_value, and word_count, then sorts by change descending, traffic descending, and source order. build_keyword_summaries keeps the first representative after this sort and aggregates categories/source domains/source occurrence count.

• [ ] Step 4: Implement query and output flow

Use candidate_domains(keyword, ("com",)), QueryDomainsClient.lookup_checks, and load_query_domains_api_key. Write only records where check.available is True to:

09_available_com_growth_over_1000_traffic_over_300k.csv
10_available_com_growth_over_1000_traffic_over_300k.html

• [ ] Step 5: Implement the standalone table template

Require fixed controls, sticky headers, no pagination, 200-row progressive rendering, filters, full-result sorting, and exact Google/query.domains/Trends URL builders. Visible fields must include candidate domain and all requested clicks/traffic metrics.

• [ ] Step 6: Run focused tests until green

Expected: all strict-boundary, deduplication, available-only, output-name, and HTML-contract tests pass.

Task 5: Build the eight-category catalog and query the final candidates

Files: - Regenerate all completed category indexes and monthly files 01–05. - Generate monthly files 09 and 10.

• [ ] Step 1: Run the monthly catalog builder

python3 build_catalog.py --root /Users/project/1B/sites/关键词 --period 2026-07

Expected: exactly 8 categories and 800 domain rows.

• [ ] Step 2: Run focused report preparation without API calls

python3 build_available_com_report.py --prepare-only

Audit strict boundaries, candidate uniqueness, and exact .com suffix before API work.

• [ ] Step 3: Query query.domains and generate final files

python3 build_available_com_report.py

The credential is loaded at runtime. Existing sanitized .com cache records are reused; pending/transient records are rechecked. Final CSV/HTML contain only confirmed available domains.

Task 6: Verify data, secrets, and browser behavior

Files: - Modify documentation: README.md, DESIGN.md, IMPLEMENTATION_PLAN.md

• [ ] Step 1: Run complete tests and compilation checks

Run all Python unit tests, python3 -m py_compile on builders/exporter, and npm test under /Users/project/脚本/similarweb. Expected: zero failures.

• [ ] Step 2: Run cross-file integrity audits

Verify 800 source rows, eight complete summaries, category-to-master CSV sum equality, unique final candidate domains, strict >1000, strict >300000, 1–3 words, .com suffix, and available == true for every 09 row.

• [ ] Step 3: Scan generated artifacts for the exact API secret

Load the key through api_keys.find_record("QUERY_DOMAINS_API_KEY"), compare it as bytes against generated CSV/HTML/JavaScript/query-cache files, and print counts only. Expected: zero matches.

• [ ] Step 4: Browser-QA final indexes and report

Open local index.html and 10_available_com_growth_over_1000_traffic_over_300k.html in Chromium. Verify source/DOM counts, 200-row progressive rendering when applicable, filters, every sortable header, sticky position before/after scroll, exact encoded actions, mobile horizontal scrolling, and zero console errors.

• [ ] Step 5: Stop the SimilarWeb server/browser and update documentation

Record exact final counts and paths. Stop the dedicated SimilarWeb PTY and Chrome process after all API-dependent verification completes.

本文档为站内渲染。原始文件本地路径：saas/source/keywords/关键词-ahrefs-top-worldwide-2026-07-all-ahrefs-top-websites-wor-d5bf26.md（仅本地保留，不入库不部署）