Segmentation & AI custom profiles

Segmentation breaks a single asset into meaningful, labeled time ranges — the beats you'd reach for when building a cut. Studio uses Pegasus 1.5 to produce these segmentations, and you can both consume the five built-in analyses and author your own custom profiles via the in-app AI assistant.

Open any asset from the Library and use the Segmentation strip on the right.

Built-in analyses

Five Pegasus 1.5 jobs run automatically per asset (when auto-process is on in Settings → Embedding). Each is its own row in the Job Monitor, can be cancelled, retried, or re-run independently, and produces a distinct segmentation strip:

recap — single-shot summary

A concise prose summary of the whole asset (no time ranges). Runs in Pegasus's general analysis mode with a JSON-schema response. Useful as a one-line summary on the asset card and as context for the Director's Story Editor when planning a cut.

structure — high-level sections

Segments the asset into broad sections — intro, opening_credits, recap, main_content, commercial_break, transition, closing, end_credits, post_credits. Powers the structural strip and helps the Director skip filler when assembling a cut.

visual — visual beats

Segments by dominant action + visual change. Each segment carries a description, a free-text dominant_action (e.g. "man speaking at podium", "crowd cheering"), and a location (interior / exterior). The Director leans on this segmentation hardest when picking action shots.

spoken — speech beats

Segments by utterance / speaker turn. Each segment summarizes what was said in that beat — coarser than the raw transcript, but bounded to coherent thoughts. Useful for dialogue-heavy cuts where the Director needs to know "where does this speaker pause."

transcript — word-aligned text

Time-aligned spoken text, segment by segment. Closer to a captions format than a summary. Powers transcript-based search and the cohesion enforcer's "don't cut mid-sentence" rule when the Director assembles a cut.

How they render

Each segmented kind renders as a timeline strip under the player: horizontal bands of segments, click-to-jump, scrubbing across the strip plays the player. Every segment row has a + to add that exact range to your project Timeline — one click from "I noticed this beat" to "it's in my cut."

How Pegasus is asked

Studio uses Pegasus 1.5's `time_based_metadata` analysis mode with a segment_definitions response format. Each kind has a hand-tuned description (the prompt that tells Pegasus where to put boundaries) and a small set of fields that differ per kind — for example, the visual kind fills in description / dominant_action / location, while the structure kind fills in section_type / description / contains_graphics.

The default prompts are baked in. You can override them per kind in Settings → Pegasus, then re-run the analysis on any asset to apply your prompt. The result is keyed by (asset, kind) and upserts on re-run — so the latest prompt wins without an explicit clear step.

AI custom segmentation profiles

Beyond the built-ins, you can author your own segmentation profile — a custom Pegasus 1.5 instruction that tells the model where to put boundaries and what to label. This is the powerful part: you describe the kind of beats you care about ("each goal attempt", "every product close-up", "scene changes in the kitchen", "every objection raised in the call") and Studio's AI assistant writes a proper Pegasus segment definition for you.

Creating a profile

1. Open Settings → Segmentation and click + New profile.
2. Describe what you want segmented, in your own words. ("Segment by each time a different speaker takes the microphone." or "Each rally in the tennis match, with the winner labeled.")
3. Click Generate. The LLM drafts a Pegasus segment definition — a boundary description + a small set of typed fields (string, boolean, array). Review the draft; edit anything that looks off.
4. Save the profile. It's stored with your settings and is reusable across assets.
5. To run it, open an asset and pick the profile from the asset view's Segment menu — clicking a profile name runs it immediately. Studio enqueues a Pegasus job and renders the resulting custom strip when it completes.

What's in a profile

A profile is, structurally, a Pegasus segment definition:

{
  "id": "goal_attempts",
  "description": "Generate a segment for every goal attempt — the moment a player visibly winds up for a shot through to either the ball entering the goal, the keeper saving, or the shot going wide.",
  "fields": [
    { "name": "description", "type": "string" },
    { "name": "outcome", "type": "string", "enum": ["scored", "saved", "missed", "blocked"] },
    { "name": "team", "type": "string" },
    { "name": "from_set_piece", "type": "boolean" }
  ]
}

The description is the most important field — it tells Pegasus where to draw segment boundaries. The fields are the columns it fills in for each segment.

Re-running a profile

Profiles are reusable across assets. From any asset's Segment menu, click an existing profile name to run it immediately. The Pegasus job is enqueued; per-asset run results live alongside the built-in analyses in the asset's segmentation panel, keyed by the profile id. You can keep several custom views of the same video.

If the result looks off, refine the profile's description and re-run — it overwrites the previous result for that (asset, profile) pair.

Tips for good profiles

• Be specific about boundaries. "Start a new segment whenever the camera cuts to a different speaker" is much better than "Find the important moments." Boundaries are the single biggest quality lever.
• Keep field sets small. A description + one enum + one boolean outperforms a dozen vague fields. Pegasus is better at filling in a tight schema than a sprawling one.
• Quote any explicit language. If the field is meant to capture what was said, instruct the model to quote verbatim. ("Quote the offending word(s) in the language field.") Compliance-style segmentation depends on this.
• Use enums for categorical fields. A string field that carries an enum of allowed values (outcome: "scored" | "saved" | "missed") is more useful than a free-form string because Studio can filter and group on a closed vocabulary.
• Don't ask Pegasus to do two things at once. One profile = one segmentation question. If you want both "goal attempts" and "speaker changes," that's two profiles.

Reusing custom segments in your cut

The Director does not automatically pick up custom-profile segments — its automatic candidate pool leans on the five built-in analyses. To use a custom profile's segments, add them to your manual Project Timeline via the + button on the segment rows. That timeline is a separate, hand-built cut, independent of the Director's automatic candidate pool — see Timeline for how to assemble and export it.

A note on prompt overrides vs. custom profiles

These are different tools:

• Settings → Pegasus prompt overrides — replace the built-in prompts for the five canonical kinds (recap / structure / visual / spoken / transcript). The result still slots into the built-in segmentation strips and the Director's pipeline.
• Custom profiles — author new segmentation kinds with their own schemas. The result lives alongside the built-ins as a separate strip.

Use prompt overrides when you want the canonical analyses tuned to your domain. Use custom profiles when you want a different question asked of the footage.