# MediaMatrix API guide

Use this guide to connect an agent, Colab notebook, or HTTP client to MediaMatrix.
The API adds channels, lists videos needing transcripts, saves transcript text,
reads saved transcripts, and searches their content. Every endpoint uses one
shared key in the `X-API-Key` header. No account, cookie session, or OAuth flow
is needed.

The website imports channel/video metadata through its YouTube API integration.
**Colab fetches transcript text from YouTube** and submits it to MediaMatrix.
Transcript endpoints read and write the database; they do not fetch from YouTube.

## Read the guides on the website

After deploying the updated website, open [API Docs](https://mediamatrix.reglitch.com/docs)
or use **API Docs** in the navigation bar. The guides are also available directly
as browser-readable Markdown text:

- [API guide](https://mediamatrix.reglitch.com/docs/agent-api.md)
- [Transcript guide](https://mediamatrix.reglitch.com/docs/transcript-api.md)
- [OpenAPI definition](https://mediamatrix.reglitch.com/agent-api.openapi.json)

Reading these documentation pages requires no API key. API requests still need
the shared key. The published guides are included when the website is built.

## Endpoint reference

The API base URL is **https://mediamatrix.reglitch.com**. All paths below are
relative to that address.
Use the site's base address without adding `/api` twice. Colab runs remotely,
so it needs an address it can reach; your Mac's `localhost` is not that address.

| Method | Path | Purpose | Reference |
| --- | --- | --- | --- |
| POST | `/api/channels` | Add one channel; optionally import videos. | [Add a channel](#add-a-channel) |
| POST | `/api/channels/bulk` | Add 1–50 channels in one request. | [Bulk channels](#add-channels-in-bulk) |
| GET | `/api/channels/{channelId}` | Read a stored channel and counts. | [Read a channel](#read-a-stored-channel) |
| GET | `/api/videos/missing-transcripts` | List videos without transcripts, with flags. | [Queue and filters](transcript-api.md#list-videos-missing-transcripts) |
| POST | `/api/videos/{videoId}/transcripts` | Save transcript text for a stored video. | [Submission fields](transcript-api.md#add-transcript-text) |
| GET | `/api/videos/{videoId}/transcripts` | Read a video's saved transcripts. | [Saved reads](transcript-api.md#read-saved-transcripts) |
| GET | `/api/transcripts/search` | Search saved text and return matching excerpts. | [Search and filters](transcript-api.md#search-saved-transcript-text) |

Replace `{channelId}` and `{videoId}` with actual IDs; do not send the braces.
A channel ID starts with `UC`. A YouTube video ID is 11 characters. A transcript
ID is a database integer, such as 123. Channel creation also accepts handles and
channel URLs; transcript paths require video IDs.

## Configure once

Run local `dotnet` commands from the `MediaMatrix.Web` folder. The website,
API guides, and .NET checks are all contained in that folder.

Add or update `AgentApi:ApiKey` in `appsettings.json`, or set the
environment variable `AgentApi__ApiKey` on the server. Merge this entry into the
existing configuration rather than replacing the whole file:

```json
{
  "AgentApi": {
    "ApiKey": "YOUR_SHARED_API_KEY"
  }
}
```

Give the same value to your agent as the `X-API-Key` header. This is separate from
`YouTube:ApiKey`, which stays in the server configuration along with the existing
database settings. Send the **AgentApi key** to these endpoints, not the YouTube
API key or database password. If the shared key is empty, the endpoints return
503; if the supplied key is missing or wrong, they return 401.

The website needs a working `ConnectionStrings:DefaultConnection` for database
operations. Channel imports additionally need `YouTube:ApiKey`. The transcript
queue, reads, saves, and searches do not require a YouTube API key or Python on
the web server. After changing server environment variables, restart the website
so it receives the new values.

Before deploying the channel-exclusion features, apply the pending migrations
from this version of the project:

```bash
dotnet ef database update
```

This adds `Channels.IsExcluded` and `Channels.ExcludeTranscripts`, both defaulting
to false. Existing channels remain active with transcripts allowed. Apply both
`AddChannelExclusion` and `AddChannelTranscriptExclusion` before running the
updated website or Colab scripts.
The API does not apply migrations automatically. Then deploy the website as usual.
POST requests save to the website's configured database, including when the
website runs locally.

## Make your first request

In Terminal, use this read-only request to check the address, key, and database:

```bash
curl -i --get 'https://mediamatrix.reglitch.com/api/transcripts/search' \
  -H 'X-API-Key: YOUR_SHARED_API_KEY' \
  --data-urlencode 'query=example' \
  --data-urlencode 'limit=1'
```

A successful response is HTTP 200 with JSON containing `query`, `items`, and
`nextAfterId`. An empty `items` array is valid when no saved transcript matches.
`-i` includes HTTP status and headers; remove it if you want just the JSON.
Search changes no records and makes no YouTube request.

To run locally yourself, from the `MediaMatrix.Web` folder:

```bash
AgentApi__ApiKey='local-test-key' dotnet run --urls http://localhost:5050
```

Keep that Terminal open. In a second Terminal, repeat the read-only request with
base URL `http://localhost:5050` and header `X-API-Key: local-test-key`. This
temporary environment setting applies to that process; it does not edit
`appsettings.json`. All commands in this guide run from `MediaMatrix.Web`.

In Postman, choose the method and URL from the endpoint table, add `X-API-Key`
under Headers, and use Params for GET filters. For POST, choose **Body → raw →
JSON** and paste the request body. POST requests need
`Content-Type: application/json`. Opening a protected URL in a browser address
bar does not supply the key; use an HTTP client that can set headers.

## Add a channel

`POST /api/channels`

```bash
curl --request POST 'https://mediamatrix.reglitch.com/api/channels' \
  --header 'X-API-Key: YOUR_SHARED_API_KEY' \
  --header 'Content-Type: application/json' \
  --data '{"channel":"https://www.youtube.com/@GoogleDevelopers"}'
```

Supported references:

- A canonical channel ID, such as `UC_x5XG1OV2P6uZZ5FSM9Ttw`.
- A handle, such as `@GoogleDevelopers`.
- A `youtube.com/@handle` or `youtube.com/channel/ID` URL, including channel tabs
  such as `/videos` and shared links with query parameters.
- A legacy `youtube.com/user/name` URL.

The API uses exact channel lookups. It does not guess from channel names or old
`/c/` custom URLs; provide the handle or canonical ID for those channels instead.

By default only the channel, metadata, and a statistics snapshot are added.
To also import up to 50 of its videos:

```json
{
  "channel": "@GoogleDevelopers",
  "importVideos": true,
  "maxVideos": 50
}
```

`maxVideos` defaults to 50 and accepts 1–500. It limits videos fetched from the
uploads playlist, not the number guaranteed to be newly added. Existing videos
are skipped, and inaccessible videos may not be returned by YouTube. This API
does not fetch transcripts.

## Keep a channel without processing it

On a channel's Details page, click **Exclude from processing**. The channel
remains visible, with its ID, metadata, videos, and transcripts preserved. Adding
it again returns the stored channel rather than creating a duplicate. Excluded
channels skip metadata/statistics updates, video imports, availability checks,
Colab analysis/export selection, and automatic transcript fetching. Existing
transcripts remain readable and searchable. Click **Include again** to resume, subject to the separate transcript setting.
The Channels page has **Active** and **Excluded** filters.

You can also check **Exclude from processing** when importing a new channel in
the website. Its initial channel metadata is saved, but no videos are imported.

Through the API, add or mark a channel as excluded:

```json
{
  "channel": "@GoogleDevelopers",
  "isExcluded": true
}
```

Send this to `POST /api/channels` using the same API key. New channels default
to active when `isExcluded` is omitted. For existing channels, omitting the field
**preserves** their current setting. Explicit `true` excludes; explicit `false`
includes again. Exclusion takes priority over `importVideos: true`.
Channel responses include `isExcluded` and `excludeTranscripts`.

For bulk requests, `isExcluded` applies to every channel in the batch:

```json
{"channels":["@GoogleDevelopers","@YouTube"],"isExcluded":true}
```

### Skip only a channel's transcripts

On the channel's Details page, click **Skip transcripts**. This skips automatic
transcript fetching and new transcript submissions through the API/Colab for
**all current and future videos**. Channel information, statistics, video imports,
and availability checks continue normally. Existing transcripts remain readable
and searchable, and manual transcript entry/editing through the website remains
available. Click **Allow transcripts** to reverse the setting. The Channels list
shows a **Transcripts skipped** badge.

You can also select this option when importing a channel, or send:

```json
{"channel":"@GoogleDevelopers","excludeTranscripts":true}
```

Use `POST /api/channels` with the API key. `excludeTranscripts` is independent of
`isExcluded`: full exclusion still stops all automatic work. Either option
preserves the other when changed. Omitted `excludeTranscripts` preserves the
stored value; new channels default to false. Explicit false allows transcripts
again, subject to full exclusion and individual video flags. Channel responses
include `excludeTranscripts`. For bulk requests, the setting applies to every
channel in the batch, for example:

```json
{"channels":["@GoogleDevelopers","@YouTube"],"excludeTranscripts":true}
```

No video flags are rewritten. Updated queues return `channelExcludesTranscripts`
and `shouldFetch: false`; `includeBlocked=false` omits these videos. New API
transcript submissions return 409 while this setting is enabled. Identical
already-saved transcript retries still return 200.

Updated Colab notebooks recheck the channel setting before fetching and saving.
Upload the updated notebook copies and rerun their setup/definition cells; code
already loaded in an older Colab session does not receive these changes by itself.

New channels return **201 Created** with a `Location` header pointing to their
API lookup. A retry for a stored channel returns **200 OK**. The canonical ID is
used for duplicate detection even when the original request was a handle or URL.

```json
{
  "status": "created",
  "channelId": "UC_x5XG1OV2P6uZZ5FSM9Ttw",
  "isExcluded": false,
  "name": "Google for Developers",
  "youTubeUrl": "https://www.youtube.com/channel/UC_x5XG1OV2P6uZZ5FSM9Ttw",
  "websitePath": "/Channels/Details/UC_x5XG1OV2P6uZZ5FSM9Ttw",
  "videosInDatabase": 0,
  "videosAdded": 0,
  "videosSkipped": 0,
  "subscriberCount": 1200,
  "publicVideoCount": 140
}
```

The counts above are examples. Stored channels return `status: "already_exists"`
and retain existing channel metadata and statistics. If `importVideos` is true,
the request can add missing videos to an existing channel. Existing video
metadata, transcripts, and flags are preserved. Counts describe this request;
`videosInDatabase` is the channel's total stored video count.

Fetching finishes before saving. The channel and optional videos are written in
one database save, so a failed write rolls back that import. If the connection
ends after a successful save, repeating the request is safe.

## Add channels in bulk

`POST /api/channels/bulk` uses the same `X-API-Key` header. Submit **1–50** channel
URLs, handles, or IDs in one request:

```bash
curl --request POST 'https://mediamatrix.reglitch.com/api/channels/bulk' \
  --header 'X-API-Key: YOUR_SHARED_API_KEY' \
  --header 'Content-Type: application/json' \
  --data '{"channels":["@GoogleDevelopers","https://www.youtube.com/@YouTube","UC_x5XG1OV2P6uZZ5FSM9Ttw"]}'
```

Bulk requests add **channels only**, with their metadata and statistics. Use the
single-channel endpoint separately if you want videos. Channels are processed
sequentially and saved independently, so earlier successes remain saved if a
later channel fails. Repeating a batch is safe; stored channels are returned as
`already_exists`, including duplicate references in the same batch.

A valid batch returns **200 OK** with a summary and one result per input in the
original order. Check the per-channel status even when the HTTP response is 200:

```json
{
  "requested": 3,
  "created": 1,
  "alreadyExists": 1,
  "failed": 1,
  "notProcessed": 0,
  "retryAfterSeconds": null,
  "results": [
    {"channel":"@example","status":"created","statusCode":201,"data":{"channelId":"UC_EXAMPLE_ID","status":"created"},"error":null},
    {"channel":"@stored","status":"already_exists","statusCode":200,"data":{"channelId":"UC_STORED_ID","status":"already_exists"},"error":null},
    {"channel":"not a channel","status":"failed","statusCode":400,"data":null,"error":{"status":400,"title":"Invalid channel reference."}}
  ]
}
```

The example abbreviates successful `data` objects; actual results contain all
the single-channel response fields. Invalid references and missing channels are
reported individually and processing continues. A server or upstream error
(500, 502, 503, or 504) stops the batch; remaining results have
`status: "not_processed"`, with null `statusCode`, `data`, and `error`. If YouTube
rejects or times out, the response includes `Retry-After: 900` and
`retryAfterSeconds: 900`. Pause before resubmitting failed/unprocessed channels.
Do not immediately send the next batch. For configuration errors, fix the server
settings before retrying.

For large lists, split them into batches of at most 50; use smaller batches if
your hosting provider has a short HTTP request timeout. A disconnected request
may have saved some channels; safely resubmit the same batch to check results.

## Read a stored channel

`GET /api/channels/{channelId}`

```bash
curl 'https://mediamatrix.reglitch.com/api/channels/UC_x5XG1OV2P6uZZ5FSM9Ttw' \
  --header 'X-API-Key: YOUR_SHARED_API_KEY'
```

This returns the same fields with `status: "exists"` and zero added/skipped
counts. It reads the database without refreshing YouTube statistics.

## Colab transcript workflow

1. Make sure the videos are stored first. Channel creation is channel-only by
   default, so a new channel may have no stored videos. To import videos, call
   `POST /api/channels` with `importVideos: true` (including for an existing
   channel), or use the website's existing video-import workflow.
2. Get the processing queue:

   ```bash
   curl 'https://mediamatrix.reglitch.com/api/videos/missing-transcripts?includeBlocked=false&limit=100' \
     -H 'X-API-Key: YOUR_SHARED_API_KEY'
   ```

   The JSON contains `items` and `nextAfterId`. Each item includes `videoId`,
   `channelId`, `title`, `videoUrl`, `flag`, `channelIsExcluded`,
   `channelExcludesTranscripts`, and `shouldFetch`. Omitting
   `includeBlocked=false` also shows flagged videos needing manual work. Known
   blockers include `youtube_unavailable` (private, deleted, or inaccessible),
   `age_restricted`, `no_captions`, `captions_disabled`, and `manual_skip`.
   Excluded parent channels also have `shouldFetch: false` and are omitted from
   the eligible-only queue, regardless of their video's own flag.
3. Use the existing Colab code to fetch transcripts for eligible items. Keep
   YouTube rejection handling and cooldowns there. This API exposes stored flags;
   it does not provide an endpoint to change them.
4. After a successful fetch, submit the actual text, language, and generation
   type. Replace `VIDEO_ID` with the item's video ID:

   ```bash
   curl -i --request POST 'https://mediamatrix.reglitch.com/api/videos/VIDEO_ID/transcripts' \
     -H 'X-API-Key: YOUR_SHARED_API_KEY' \
     -H 'Content-Type: application/json' \
     --data '{"language":"en","content":"Transcript text here.","source":"youtube","isAutoGenerated":true}'
   ```

   New text returns 201; identical text already stored for that language returns
   200. Different existing text returns 409 and is preserved. Set
   `isAutoGenerated` to false for a manually authored YouTube caption track.
5. Finish the page and use its `nextAfterId` for the next queue request. When it
   is null, stop that scan. Saved videos leave the corresponding queue, and
   cursor pagination keeps this from skipping the remaining videos.

Full field definitions, example responses, flag behavior, and Python helpers are
in the [transcript API guide](transcript-api.md).

## Search and read transcript text

Search saved content, with URL encoding for phrases and punctuation:

```bash
curl --get 'https://mediamatrix.reglitch.com/api/transcripts/search' \
  -H 'X-API-Key: YOUR_SHARED_API_KEY' \
  --data-urlencode 'query=climate change' \
  --data-urlencode 'language=en' \
  --data-urlencode 'limit=20'
```

Each result includes `transcriptId`, `videoId`, `videoTitle`, `channelId`,
`language`, `isAutoGenerated`, `flag`, `excerpt`, `videoUrl`, and `websitePath`.
Matches are case-insensitive literal words/phrases, with one result per
transcript and an excerpt of up to 300 characters. You can also filter by
`channelId` or `videoId`.

To read the full saved text for one video:

```bash
curl 'https://mediamatrix.reglitch.com/api/videos/VIDEO_ID/transcripts?language=en' \
  -H 'X-API-Key: YOUR_SHARED_API_KEY'
```

This returns an **array** of saved transcript objects. Omit `language` to read all
saved languages. An empty array means no matching transcript; 404 means the
video itself is not stored. Saved reads use an exact language filter, while
queue/search filters include regional variants: `language=en` in search also
matches `en-US`. Use the actual language from a search result for an exact read.
Relative `websitePath` links are opened using your website's base URL.

## Pagination and limits

| Operation | Default / maximum | Cursor |
| --- | --- | --- |
| Bulk channel creation | 1–50 channels per request | Split larger input lists into separate requests. |
| Optional channel video import | 50 / 500 videos fetched | `maxVideos` is a fetch limit; existing videos are skipped. |
| Missing-transcript queue | 100 / 500 items | `nextAfterId` is a **video ID string**; send it as `afterId`. |
| Transcript search | 20 / 100 results | `nextAfterId` is an **integer transcript ID**; send it as `afterId`. |

Both paginated endpoints return `items` and `nextAfterId`. Start without
`afterId`, keep all filters fixed, and stop when `nextAfterId` is null. These
endpoints use cursors rather than page numbers and do not return a total count.
They return items in ID order; transcript search does not rank results by
relevance. Saved transcript reads return an array, without a cursor envelope.

## Errors and retries

Errors are JSON Problem Details with `status`, `title`, and optional `detail`.
Validation errors also include an `errors` object describing invalid fields.

| Status | Meaning | Agent action |
| --- | --- | --- |
| 400 | Missing/invalid fields, malformed JSON, or invalid filters/limits | Correct the request; inspect `errors` if present. |
| 401 | Shared key is missing or incorrect | Correct `X-API-Key`. |
| 404 | Channel not found, or requested channel/video is not stored | Check the reference and whether it has been imported. |
| 409 | Concurrent import/save, different transcript text already exists, or parent channel is excluded or skips transcripts | Inspect the title. Allow channel transcripts before new submissions; preserve different stored text; retry races only. |
| 415 | POST body has an unsupported content type | Send JSON with `Content-Type: application/json`. |
| 502 | Incomplete or invalid YouTube response | Retry later. |
| 503 | Missing server configuration or rejected/unavailable YouTube request | Read the error title; fix configuration or pause retries. |
| 504 | YouTube request timeout | Retry later. |
| 500 | Storage or unexpected server failure | Check the website logs; retrying does not duplicate records. |

Temporary YouTube failures include `Retry-After: 900` (15 minutes). The agent
should pause its channel-import queue when it receives this header. Limit
retries instead of repeatedly submitting more channels while YouTube rejects
requests. Configuration errors need a server setting changed, not repeated calls.

For example, an invalid key returns HTTP 401 with a body like:

```json
{
  "status": 401,
  "title": "Invalid API key.",
  "detail": "Send the shared key in the X-API-Key header."
}
```

An HTML page, redirect, or ordinary 404 for `/api/transcripts/search` can mean
you are using the wrong website address or have not deployed the API changes.
Confirm the running version and URL. A bulk channel response can contain
individual failures even when its outer HTTP status is 200; inspect `results`.

Transcript queue/read/search requests make no YouTube calls. If Colab's YouTube
fetch fails, handle that in Colab; submitting a transcript to this API does not
retry or perform the fetch.

## Agent handoff

Give your agent the website base URL and shared key, plus this guide. For agents
that import API schemas, use the [OpenAPI definition](/agent-api.openapi.json),
served by the website at `/agent-api.openapi.json`. The definition lists all
seven operations. It describes the API; it does not configure the shared key on
the server or replace deployment.

Tell your agent:

> Use https://mediamatrix.reglitch.com as the API base URL.
> Add channels with POST /api/channels and send my shared key in X-API-Key.
> For multiple channels use POST /api/channels/bulk with a channels array of up
> to 50 URLs, handles, or IDs. Inspect every result; 200 does not mean all succeeded.
> Supply the channel URL, @handle, or canonical ID in the JSON channel field.
> Import only the channel unless I explicitly request videos.
> Treat 201 created and 200 already_exists as success.
> Respect Retry-After and report failures instead of retrying indefinitely.
> When a batch stops, pause and retry failed/unprocessed entries after the issue
> is resolved. Earlier successes are already saved.
> For transcripts, GET /api/videos/{videoId}/transcripts reads saved text;
> POST to that URL submits text. GET /api/videos/missing-transcripts lists
> missing transcripts with flags and shouldFetch. Use includeBlocked=false for
> the Colab queue. Colab handles YouTube fetching. Respect 409 conflicts; videos
> must already exist in MediaMatrix.
> Search saved transcript content with GET /api/transcripts/search?query=TEXT.
> Results include excerpts, video details, and flags; use nextAfterId to page.

After deployment, the OpenAPI schema is available at
[mediamatrix.reglitch.com/agent-api.openapi.json](https://mediamatrix.reglitch.com/agent-api.openapi.json)
and is stored in `wwwroot/agent-api.openapi.json`. Its server URL
is `https://mediamatrix.reglitch.com`, ready for an agent to import. For local
testing or another deployment, change `servers[0].url` in the schema copy you
import to that deployment's base address.

## Offline checks

Run from the `MediaMatrix.Web` folder:

```bash
dotnet run --project tests/MediaMatrix.AgentApiChecks
```

Checks use fake YouTube responses and an in-memory SQLite database. They do not
start the website or connect to the hosted database.