Initial commit: App Store + Google Play scraper
- discover.py: find app IDs (Play search, iTunes Search API, RSS charts) -> apps.json - scraper.py: fetch metadata + ratings per app/country, threaded, shardable, resumable - netutil.py: shared retry/backoff + adaptive circuit breaker (rate-limit safe) - merge_csv.py: combine sharded outputs from multiple machines - ARCHITECTURE.html: full architecture & flow documentation - english_words.txt: deep search-term wordlist Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
+22
@@ -0,0 +1,22 @@
|
||||
# Python
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*.egg-info/
|
||||
.venv/
|
||||
venv/
|
||||
env/
|
||||
|
||||
# Generated output (CSVs can get very large)
|
||||
output/
|
||||
*.log
|
||||
|
||||
# Backups
|
||||
backups/
|
||||
apps_backup*.json
|
||||
|
||||
# Local tool / editor config
|
||||
.claude/
|
||||
.vscode/
|
||||
.idea/
|
||||
*.swp
|
||||
.DS_Store
|
||||
Reference in New Issue
Block a user