> ## Documentation Index
> Fetch the complete documentation index at: https://docs.go-fetch.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Jobs and results

> How a job runs, the statuses it moves through, and how to read its results.

Everything you collect is a job: one scraper, one input. A job runs in the background, so you create it, wait for
it to finish, then read its results.

## Create a job

`POST /api/v1/jobs/create/` with a `scraper_type` and its input in `config`:

```bash theme={null}
curl https://go-fetch.io/api/v1/jobs/create/ \
  -H "X-API-Key: $GOFETCH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "scraper_type": "youtube",
    "config": {"channelUrls": ["https://www.youtube.com/@NobelPrize"], "limit": 25},
    "timeout": 3600
  }'
```

| Field | Required | What it does |
| - | - | - |
| `scraper_type` | yes | Which scraper runs. See the [scraper guides](https://docs.go-fetch.io/scrapers/instagram.md). |
| `config` | yes | The scraper's input. `input` is accepted as another name. |
| `timeout` | no | Seconds the job may run, from 300 to 39,600. The default depends on the scraper: 8 hours for most, 2 hours for `tiktok_search`, 1 hour for `tiktok_profile` and `google_serp`. |
| `webhooks` | no | Webhooks for this job only. See [Webhooks](https://docs.go-fetch.io/webhooks.md#per-job-webhooks). |

The answer is `201` with the job. The price is locked at this moment, and the job's credit is checked. A request is
refused with `402` if the balance can't cover its estimate, or if that estimate plus the quoted `estimated_cost`
of the organization's pending and running jobs exceeds 10 times the balance ([Billing](https://docs.go-fetch.io/billing.md#credit-check-402)).
A 402 creates no job, so retrying the request after backing off is safe.

How `config` is checked:

* A field the scraper doesn't know is ignored, not refused. Check spelling and case: a misspelt limit runs with the
  default.
* An input a scraper accepts but can't act on with the value you sent is ignored too. Each guide lists them.
* Either way the job still runs, and its log (`GET /api/v1/jobs/{id}/logs/`) gets one `WARNING` naming the
  fields, for example `Input(s) limitt are not recognised for instagram and had no effect; check the spelling.`
* An empty list (`"usernames": []`) is refused with `400`.
* Numbers, links and dates the scraper guide gives a range for are checked, and a value outside it is refused with
  `400` naming the field.

## Statuses

| `status` | Means |
| - | - |
| `pending` | Created, not yet handed to a scraper. |
| `running` | Collecting. Results already delivered can be read. |
| `completed` | Finished. `items_scraped` holds the billed result count and `actual_cost` the charge. |
| `failed` | Stopped by an error. `error_message` and `error_code` say why. |
| `timed_out` | Ran past its `timeout`. |
| `cancelled` | Stopped by you (not charged), or by its [spending cap](https://docs.go-fetch.io/billing.md#cap-a-job-with-maxtotalchargeusd) (billed for the rows it delivered, at most the cap). |

**Wait on `is_terminal`, not on `status`.** A job that fails while it still has automatic retries left reads
`"status": "failed"` with `"is_terminal": false` and `"will_auto_retry": true`: it is about to run again, under the
same id. `is_terminal` is `true` once the job is `completed` or `cancelled`, or `failed` or `timed_out` with no retry
coming. The SDK's `call()` and `wait_for_finish()` already wait for it.

Each earlier attempt is listed in `attempt_history` with its `status`, `error_message` and `failed_at`.

### `error_code`

A job that did not complete has an `error_code`, so your code can decide what to do without reading
`error_message`:

| `error_code` | Means | What to do |
| - | - | - |
| `INPUT_ERROR` | The input can't work, for example an account that does not exist. | Fix the input. |
| `BLOCKED` | The platform refused the requests. | Try again later. |
| `RATE_LIMITED` | The platform throttled the requests. | Try again later. |
| `TIMEOUT` | The job ran past its `timeout`. | Ask for less, or raise `timeout`. |
| `INFRA_TERMINATED` | Our side stopped the job. | Retry it. |
| `PLATFORM_ERROR` | The scraper is not available on our side. | Write to support. |
| `PROFILE_EXCLUDED` | Every target is on our exclusion list. | Remove those targets. |
| `CANCELLED` | You cancelled the job, or its spending cap did. | |
| `UNKNOWN` | None of the above. | Read `error_message`. |

## A finished job

A real completed job, from a test run of the quickstart on a local stack. Amounts are shown as `"…"`: rates
change, so the docs leave them out.

```json theme={null}
{
  "id": "d1c519f1-9ae0-44c0-8362-1afe2f01a0f5",
  "organization": "0d17a952-5d4f-457c-95a0-1812bb75aa86",
  "organization_name": "Local Test",
  "environment": "919aed5c-b794-4ca0-a22a-0a6fc0d47a80",
  "environment_name": "Docs samples",
  "scraper_type": "instagram",
  "config": {
    "limit": 10,
    "timeout": 28800,
    "usernames": [
      "nasa"
    ]
  },
  "status": "completed",
  "is_terminal": true,
  "will_auto_retry": false,
  "items_scraped": 2,
  "output_dataset_url": "/api/v1/jobs/d1c519f1-9ae0-44c0-8362-1afe2f01a0f5/results/",
  "error_message": "",
  "error_code": null,
  "attempt_history": [],
  "started_at": "2026-10-15T08:24:49.206473Z",
  "completed_at": "2026-10-15T08:24:52.146760Z",
  "duration_seconds": 2,
  "created_at": "2026-10-15T08:24:49.202036Z",
  "updated_at": "2026-10-15T08:24:52.146802Z",
  "retry_count": 0,
  "max_retries": 3,
  "can_retry": false,
  "can_resume": true,
  "partial_results_count": 0,
  "last_progress_at": null,
  "ad_hoc_webhooks": null,
  "scraper_metadata": null,
  "started_from": null,
  "is_initial_crawl": true,
  "price_per_1000_at_creation": "…",
  "platform_prices_at_creation": null,
  "minimum_charge_at_creation": true,
  "estimated_cost": "…",
  "actual_cost": "…",
  "credits_deducted": true
}
```

Fields worth knowing:

| Field | What it holds |
| - | - |
| `items_scraped` | Billed result rows, excluding \$0 status rows. |
| `actual_cost`, `credits_deducted` | The final charge in USD, and whether it has been taken from your balance. |
| `estimated_cost` | The estimate the credit check used when the job was created. |
| `price_per_1000_at_creation`, `minimum_charge_at_creation` | The rate locked for this job, and whether its scraper type has a minimum charge per job. |
| `started_from`, `is_initial_crawl` | The start of the job's date filter, and whether the job counts as a full crawl or a recent one ([rates](https://docs.go-fetch.io/billing.md#rates)). |
| `partial_results_count`, `last_progress_at` | Progress while the job runs. |
| `can_retry`, `retry_count`, `max_retries` | Whether `POST /api/v1/jobs/{id}/retry/` will run it again. |
| `scraper_metadata` | What the scraper reported about completeness. See below. |
| `environment`, `environment_name` | The [environment](https://docs.go-fetch.io/authentication.md#environments) of the key that created it. |
| `ad_hoc_webhooks` | The job's own webhooks. |
| `output_dataset_url` | The job's results endpoint (relative path), served with your API key. |

## Read the results

`GET /api/v1/jobs/{id}/results/?offset=0&limit=100` returns a page of rows:

```bash theme={null}
curl "https://go-fetch.io/api/v1/jobs/$JOB_ID/results/?offset=0&limit=100" \
  -H "X-API-Key: $GOFETCH_API_KEY"
```

* `limit` is 50 by default and at most 100. Page with `offset` until you have `total` rows, or a page comes back with
  `count` 0.
* `total` includes \$0 status rows, such as Instagram's `profile_error`, so it can exceed `items_scraped`.
* A running job already returns the rows delivered so far.
* Each row is the scraper's own record. Its fields are in the scraper's guide.
* Rows don't always come back in input order. Match them to your input with the field the guide names (`input`,
  `facebookUrl`, `search_term`, ...).

With the SDK, `iterate_items()` pages for you:

```python theme={null}
import os

from gofetch import GoFetchClient

client = GoFetchClient(api_key=os.environ["GOFETCH_API_KEY"])

items = client.dataset(os.environ["JOB_ID"]).list_items()
print(items.total, "rows")
for item in client.dataset(os.environ["JOB_ID"]).iterate_items():
    print(item.get("type"), item.get("url"))
```

> [!WARNING]
>
> Results are deleted 5 days after collection. After that, `/results/` returns `total: 0`. Job records
> (configuration, status, counts and usage) stay while your account is active. Fetch the results, or take them by
> webhook, within that window.

## Is it complete?

`completed` means the job ran to its end. It does not always mean every target delivered everything: a private
account, a page that shows nothing to signed-out visitors, or a platform that stopped answering partway leaves gaps.
`scraper_metadata` says what happened, with keys that depend on the scraper:

| Key | What it says |
| - | - |
| `completeness` | How much of what you asked for was collected, per target. |
| `coverage` | For comment scrapers: comments collected against the count each post shows. |
| `restrictions` | Targets the platform withheld from signed-out visitors, and why. |
| `targets_failed` | Targets that returned nothing, with a `reason` and whether retrying can help (`retriable`). |
| `excluded_targets` | Targets removed because they are on our exclusion list: people who asked not to be collected. Their rows are never returned. |
| `profiles_scraped`, `profiles_failed` | Targets collected and targets that failed. |

Each scraper guide lists what it reports.

## Logs, cancel and retry

| Request | Does |
| - | - |
| `GET /api/v1/jobs/{id}/logs/` | The progress messages the scraper reported, oldest first, and any warning about the job's input. |
| `POST /api/v1/jobs/{id}/cancel/` | Stops a pending or running job, which is then not charged. A finished job is returned as it is. |
| `POST /api/v1/jobs/{id}/retry/` | Runs a `failed`, `timed_out` or `cancelled` job again, or a `completed` one that delivered nothing, while `can_retry` is `true`. Its credit is checked again. A job its spending cap stopped can't be retried (`400`): create a new job instead. A job you cancelled yourself can. |
| `GET /api/v1/jobs/?status=completed&scraper_type=tiktok` | Your organization's jobs, newest first, filtered by `status`, `scraper_type` or `environment`. |

```bash theme={null}
curl https://go-fetch.io/api/v1/jobs/$JOB_ID/logs/ \
  -H "X-API-Key: $GOFETCH_API_KEY"
```

## Lists and paging

Every list answers `{"total", "offset", "limit", "count", "results"}`. Page with `offset` and `limit`: 25 rows by
default, at most 100. The audit log is the exception: it pages with `page` and `page_size` (100 by default) and
answers `{"count", "next", "previous", "results"}`.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.