LogoColorMon

Replicate Models — Image Generation

Overview

  • Flow: Client UI → POST /api/demo/gen-image → AI SDK generateImage → @ai-sdk/replicate adapter → object storage upload → response with public URLs.
  • Provider options: Provider‑specific fields live under providerOptions.replicate and are flattened into Replicate's input object (and override top‑level mappings like aspectRatio, seed).
  • Model routing: When no input image is provided, the API uses black-forest-labs/flux-schnell; when an input image is provided (image‑only or image+text), it uses black-forest-labs/flux-kontext-pro.

Request Shape

  • Common fields (top‑level):
    • prompt string (required)
    • provider = replicate
    • model = Replicate model id, e.g. bytedance/seedream-3, google/imagen-3-fast, google/imagen-4-fast
    • n number (optional; default 1)
    • aspectRatio string (optional; e.g. "4:3")
    • seed number (optional)
    • options object (optional; merged into providerOptions.replicate verbatim)

Replicate Mapping

  • The adapter sends a JSON body like: 
    {
  "input": {
    "prompt": "...",
    "num_outputs": 1,
    "aspect_ratio": "4:3",
    "seed": 123,
    "... providerOptions.replicate fields ..."
  }
}
  • Keys from providerOptions.replicate are spread last and override the top‑level mappings. Use this to pass model‑specific parameters.

Seedream 3 (bytedance/seedream-3)

  • Supported fields (per model schema):
    • seed number
    • size enum: "small" | "regular" | "big"
    • width number, height number (used when aspect_ratio is "custom")
    • aspect_ratio enum: "1:1" | "3:4" | "4:3" | "16:9" | "9:16" | "2:3" | "3:2" | "21:9" | "custom"
    • guidance_scale number
  • Example request: 
    {
  "provider": "replicate",
  "model": "bytedance/seedream-3",
  "prompt": "A cute cat",
  "aspectRatio": "16:9",
  "seed": 42,
  "options": {
    "size": "regular",
    "guidance_scale": 2.5
  }
}

Google Imagen (google/imagen-3-fast, google/imagen-4-fast)

  • Supported fields (per model schema):
    • aspect_ratio enum (e.g. "1:1" | "9:16" | "16:9" | "3:4" | "4:3")
    • output_format enum ("jpg" | "png")
    • safety_filter_level enum ("block_low_and_above" | "block_medium_and_above" | "block_only_high")
  • Example request: 
    {
  "provider": "replicate",
  "model": "google/imagen-3-fast",
  "prompt": "Cinematic portrait, neon lights",
  "aspectRatio": "4:3",
  "options": {
    "output_format": "jpg",
    "safety_filter_level": "block_only_high"
  }
}

Extending With New Replicate Models

  1. Choose the model id and read its Input schema on Replicate.
  2. Pass model‑specific fields via options in the request body; they will be merged into providerOptions.replicate and flattened into input.
  3. If you need type‑safe mapping or defaults, extend the replicate branch in src/app/api/demo/gen-image/route.ts with a case for your model and map fields explicitly (e.g. rename guidanceScale → guidance_scale).

Notes

  • Top‑level size in AI SDK is "{width}x{height}" and maps to input.size. Many Replicate models use different semantics (e.g. Seedream 3 expects "small" | "regular" | "big"). Prefer providerOptions for those.
  • Unknown fields must be sent via options to avoid being dropped.
  • Auth: set REPLICATE_API_TOKEN. Default model: REPLICATE_IMAGE_MODEL. Default provider: IMAGE_PROVIDER.

FLUX Schnell (black-forest-labs/flux-schnell)

  • Summary:

    • Used by default when there is no input image.
    • Fast text→image model with configurable speed/quality knobs.

  • Supported fields (per model schema):

    • prompt string (required)
    • seed integer (optional) — random seed
    • aspect_ratio enum: "1:1" | "16:9" | "21:9" | "3:2" | "2:3" | "4:5" | "5:4" | "3:4" | "4:3" | "9:16" | "9:21" (default "1:1")
    • output_format enum: "webp" | "jpg" | "png" (default "webp"; our API defaults to "png" if not specified)
    • output_quality integer 0–100 (default 80; not relevant for PNG)
    • num_inference_steps integer 1–4 (default 4)
    • go_fast boolean (default true) — enable faster fp8 run; not deterministic even with seed when enabled
    • megapixels enum "1" | "0.25" (default "1")
    • disable_safety_checker boolean (default false)

  • Example request (text → image):

    {
  "provider": "replicate",
  "model": "black-forest-labs/flux-schnell",
  "prompt": "Kid‑friendly coloring page, baby giraffe, bold outlines, minimal background",
  "aspectRatio": "3:2",
  "seed": 42,
  "options": {
    "output_format": "png",
    "num_inference_steps": 4,
    "go_fast": true,
    "megapixels": "1"
  }
}

FLUX Kontext Pro (black-forest-labs/flux-kontext-pro)

  • Supported fields (per model schema):

    • prompt string (required)
    • seed integer (nullable) — random seed for reproducibility
    • input_image uri (nullable) — reference image url; must be jpeg, png, gif, or webp
    • aspect_ratio enum: "match_input_image" | "1:1" | "16:9" | "9:16" | "4:3" | "3:4" | "3:2" | "2:3" | "4:5" | "5:4" | "21:9" | "9:21" | "2:1" | "1:2" (default "match_input_image")
    • output_format enum: "jpg" | "png" (default "png")
    • safety_tolerance integer from 0–6 (default 2; 2 is the max when using input images)
    • prompt_upsampling boolean (default false) — automatic prompt improvement

  • Example request (text → image):

    {
  "provider": "replicate",
  "model": "black-forest-labs/flux-kontext-pro",
  "prompt": "Kid-friendly coloring page, panda eating bamboo, bold outlines, minimal background",
  "aspectRatio": "3:4",
  "seed": 123,
  "options": {
    "output_format": "png",
    "safety_tolerance": 2,
    "prompt_upsampling": false
  }
}

  • Example request (image + text):

    {
  "provider": "replicate",
  "model": "black-forest-labs/flux-kontext-pro",
  "prompt": "Convert to clean black-line coloring page, kid style, bold outlines",
  "options": {
    "input_image": "https://your-cdn.example.com/path/to/image.png",
    "aspect_ratio": "match_input_image",
    "output_format": "png",
    "safety_tolerance": 2
  }
}