# App Store Scraper (Google Play + Apple App Store)

For a **specific list of apps** (defined in `apps.json`) this produces **one combined CSV**
with core metadata, the **last-updated date**, and the **final rating** (average + total
number of ratings). Python, free libraries only.

## What it uses

| Store | Source |
|-------|--------|
| Google Play | `google-play-scraper` (scrapes public pages) |
| Apple App Store | iTunes Lookup API (**official**, free) |

No paid services, no API keys.

## Setup

```powershell
python -m pip install -r requirements.txt
```

## Configure — `apps.json`

```jsonc
{
  "settings": {
    "country": "us",        // store country code
    "lang": "en",           // language (Play only)
    "delay_seconds": 1.0    // pause between requests (avoid blocks)
  },
  "google_play": [
    { "app_id": "com.whatsapp" }        // the "id=" value in the Play URL
  ],
  "app_store": [
    { "app_id": 310633997, "name": "WhatsApp" }   // the numeric "id" in the App Store URL ("name" is just a label)
  ]
}
```

**Finding IDs**
- Play Store: `play.google.com/store/apps/details?id=`**`com.whatsapp`** → use `com.whatsapp`
- App Store: `apps.apple.com/us/app/whatsapp-messenger/id`**`310633997`** → use `310633997`

## Run

```powershell
python scraper.py
```

## Output — `output/app_data.csv` (one row per app, both stores together)

| Column | Meaning |
|--------|---------|
| `store` | `google_play` or `app_store` |
| `app_id`, `title`, `developer`, `category` | identity |
| `price`, `currency`, `free` | pricing |
| `avg_rating` | final average star rating |
| `total_ratings` | total number of ratings |
| `text_review_count` | number of text reviews (Play only) |
| `last_updated` | date the app was last updated |
| `version` | latest version |
| `url` | store listing URL |

UTF-8 with BOM — opens cleanly in Excel.

## Limitations (important)

- **No "all apps" list exists.** Neither store exposes a full catalog; you scrape only
  the apps you list here. Use `discover.py` to auto-fill `apps.json` from top charts.
- **Rate limits / blocking** — keep `delay_seconds > 0`; heavy use may need proxies.
- **Terms of Service** — both stores restrict automated scraping. Fine for internal research
  at small scale; get legal sign-off before commercial redistribution.
- **Per-country data** — results differ by `country`.