# Core.Today AI API — Vision & Utility Models (STT, OCR, Embeddings, 3D, Lipsync)

> Speech-to-text (Whisper), OCR, image captioning, embeddings, 3D generation, lipsync and video enhancement models available through the Core.Today unified API.

Source: https://console.core.today/docs/utilities
This file: https://console.core.today/llms/utilities.txt
All models: https://console.core.today/llms.txt

## API Usage

Base URL: https://api.core.today/v1
Auth header: X-API-Key: YOUR_API_KEY

1. POST /predictions with {"model": "<model id>", "input": {...}} -> returns job_id
2. GET /predictions/{job_id} -> poll until status is "completed" or "failed"
3. result contains output URL(s); add "is_public": true for a permanent public URL

## Models (18)

### Incredibly Fast Whisper — `vaibhavs10/incredibly-fast-whisper`

Whisper large-v3 optimized for speed (38M+ runs) — transcribes roughly 150 minutes of audio in under 100 seconds using batched inference. Chunk-level or word-level timestamps.

- Provider: Whisper
- Credits: 10 (per run)
- Speed: fast / Quality: ultra
- Required params: audio (string)
- Optional params: task=transcribe, language=None, batch_size=24, timestamp=chunk, diarise_audio=false, hf_token
- Example input:
```json
{
  "model": "vaibhavs10/incredibly-fast-whisper",
  "input": {
    "audio": "https://example.com/podcast-episode.mp3",
    "task": "transcribe",
    "timestamp": "word"
  }
}
```
- Full docs: https://console.core.today/docs/models/vaibhavs10/incredibly-fast-whisper

### Whisper — `openai/whisper`

OpenAI's Whisper large-v3 speech recognition (144M+ runs) — the standard for transcription. Automatic language detection across ~100 languages, English translation, and plain text / SRT / VTT output formats.

- Provider: OpenAI
- Credits: 10 (per run (runtime grows with audio length))
- Speed: medium / Quality: ultra
- Required params: audio (string)
- Optional params: transcription=plain text, translate=false, language=auto, temperature=0, patience, suppress_tokens=-1, initial_prompt, condition_on_previous_text=true, temperature_increment_on_fallback=0.2, compression_ratio_threshold=2.4, logprob_threshold=-1, no_speech_threshold=0.6
- Example input:
```json
{
  "model": "openai/whisper",
  "input": {
    "audio": "https://example.com/meeting.mp3",
    "language": "auto",
    "transcription": "plain text"
  }
}
```
- Full docs: https://console.core.today/docs/models/openai/whisper

### Whisper Diarization — `thomasmol/whisper-diarization`

Whisper transcription with speaker diarization (8M+ runs) — returns who said what, with per-segment speaker labels and timestamps. The go-to for meetings and interviews.

- Provider: Whisper
- Credits: 7 (per run)
- Speed: fast / Quality: high
- Optional params: prompt, file_string, file_url, file, num_speakers, translate=false, language
- Example input:
```json
{
  "model": "thomasmol/whisper-diarization",
  "input": {
    "file_url": "https://example.com/interview.mp3",
    "num_speakers": 2,
    "language": "ko",
    "prompt": "코어닷투데이, API, 크레딧"
  }
}
```
- Full docs: https://console.core.today/docs/models/thomasmol/whisper-diarization

### BLIP — `salesforce/blip`

Salesforce BLIP (173M+ runs) — image captioning, visual question answering, and image-text matching in one model. The classic choice for bulk captioning at 1 credit per image.

- Provider: Salesforce
- Credits: 1 (per image)
- Speed: fast / Quality: standard
- Required params: image (string)
- Optional params: task=image_captioning, question, caption
- Example input:
```json
{
  "model": "salesforce/blip",
  "input": {
    "image": "https://example.com/photo.jpg",
    "task": "image_captioning"
  }
}
```
- Full docs: https://console.core.today/docs/models/salesforce/blip

### CLIP Features — `andreasjansson/clip-features`

CLIP ViT-L/14 embeddings for text AND images (163M+ runs) — puts both in the same vector space for cross-modal search, image dedup, and zero-shot classification. 1 credit per run.

- Provider: CLIP
- Credits: 1 (per run (batch inputs with newlines))
- Speed: fast / Quality: high
- Optional params: inputs=a
b
- Example input:
```json
{
  "model": "andreasjansson/clip-features",
  "input": {
    "inputs": "a photo of a red sneaker\nhttps://example.com/sneaker.jpg"
  }
}
```
- Full docs: https://console.core.today/docs/models/andreasjansson/clip-features

### Hunyuan 3D 3.1 — `tencent/hunyuan-3d-3.1`

Tencent's flagship 3D generation — create high-polygon textured 3D models from a text prompt OR an image, with optional PBR (physically based rendering) materials.

- Provider: Tencent
- Credits: 1160 (per generation)
- Speed: slow / Quality: ultra
- Optional params: prompt, image, enable_pbr=false, face_count=500000, generate_type=Normal
- Example input:
```json
{
  "model": "tencent/hunyuan-3d-3.1",
  "input": {
    "prompt": "A cute low-poly cartoon fox sitting, game asset style",
    "generate_type": "Normal",
    "enable_pbr": true
  }
}
```
- Full docs: https://console.core.today/docs/models/tencent/hunyuan-3d-3.1

### Moondream2 — `lucataco/moondream2`

Small but capable vision-language model (14M+ runs) — ask free-form questions about any image and get detailed answers. Efficient VQA for tagging, moderation prep, and rich alt-text.

- Provider: Moondream
- Credits: 5 (per image)
- Speed: fast / Quality: high
- Required params: image (string)
- Optional params: prompt=Describe this image
- Example input:
```json
{
  "model": "lucataco/moondream2",
  "input": {
    "image": "https://example.com/scene.jpg",
    "prompt": "Describe this image in detail, including colors, objects, and mood."
  }
}
```
- Full docs: https://console.core.today/docs/models/lucataco/moondream2

### Multilingual E5 Large — `beautyyuyanli/multilingual-e5-large`

Multilingual text embeddings (74M+ runs) — 1024-dimension vectors across 100 languages including Korean. Pairs perfectly with Core.Today customer databases' vector search (knn_vector).

- Provider: E5
- Credits: 2 (per run (batch multiple texts in one call))
- Speed: fast / Quality: high
- Optional params: texts=["In the water, fish are swimming.", "Fish swim in the water.", "A book lies open on the table."], batch_size=32, normalize_embeddings=true
- Example input:
```json
{
  "model": "beautyyuyanli/multilingual-e5-large",
  "input": {
    "texts": "[\"코어닷투데이는 AI API 게이트웨이입니다.\", \"크레딧 기반 과금을 지원합니다.\"]",
    "normalize_embeddings": true
  }
}
```
- Full docs: https://console.core.today/docs/models/beautyyuyanli/multilingual-e5-large

### Text Extract OCR — `abiruyt/text-extract-ocr`

Simple, massively-used OCR (91M+ runs) — extracts text from an image with a single input and returns plain text. Great default for receipts, screenshots, and scanned documents at just 1 credit.

- Provider: OCR
- Credits: 1 (per image)
- Speed: fast / Quality: standard
- Required params: image (string)
- Example input:
```json
{
  "model": "abiruyt/text-extract-ocr",
  "input": {
    "image": "https://example.com/receipt.jpg"
  }
}
```
- Full docs: https://console.core.today/docs/models/abiruyt/text-extract-ocr

### TRELLIS — `firtoz/trellis`

Microsoft's TRELLIS image-to-3D (836K+ runs — the most-used 3D model on Replicate). Turns one or more images into a textured GLB 3D asset, with turntable render videos and optional Gaussian PLY.

- Provider: Microsoft
- Credits: 77 (per generation)
- Speed: medium / Quality: high
- Required params: images (array)
- Optional params: seed=0, randomize_seed=true, generate_color=true, generate_normal=false, generate_model=false, save_gaussian_ply=false, return_no_background=false, ss_guidance_strength=7.5, ss_sampling_steps=12, slat_guidance_strength=3, slat_sampling_steps=12, mesh_simplify=0.95, texture_size=1024
- Example input:
```json
{
  "model": "firtoz/trellis",
  "input": {
    "images": [
      "https://example.com/product.png"
    ],
    "generate_model": true,
    "texture_size": 1024
  }
}
```
- Full docs: https://console.core.today/docs/models/firtoz/trellis

### Add Watermark — `charlesmccarthy/addwatermark`

Add a text watermark to any video — simple, fast brand protection for generated or user content at 2 credits per video.

- Provider: FullJourney
- Credits: 1 (per video)
- Speed: fast / Quality: standard
- Required params: video (string)
- Optional params: size=40, watermark=FULLJOURNEY.AI
- Example input:
```json
{
  "model": "charlesmccarthy/addwatermark",
  "input": {
    "video": "https://example.com/generated.mp4",
    "watermark": "CORE.TODAY",
    "size": 48
  }
}
```
- Full docs: https://console.core.today/docs/models/charlesmccarthy/addwatermark

### CogVLM2 Video — `chenxwh/cogvlm2-video`

Video understanding and captioning — ask free-form questions about a video and get detailed answers about actions, scenes, and content. Great for video search indexing and moderation prep.

- Provider: CogVLM
- Credits: 28 (per video query)
- Speed: medium / Quality: high
- Required params: input_video (string)
- Optional params: prompt=Describe this video., top_p=0.1, temperature=0.1, max_new_tokens=2048
- Example input:
```json
{
  "model": "chenxwh/cogvlm2-video",
  "input": {
    "input_video": "https://example.com/clip.mp4",
    "prompt": "Describe this video in detail, including actions and scene changes."
  }
}
```
- Full docs: https://console.core.today/docs/models/chenxwh/cogvlm2-video

### LatentSync — `bytedance/latentsync`

ByteDance's open-source lipsync — re-syncs a video's mouth movements to any audio track using latent diffusion. State-of-the-art open lipsync quality for dubbing and localization.

- Provider: ByteDance
- Credits: 220 (per run (runtime grows with video length))
- Speed: slow / Quality: high
- Optional params: video, audio, guidance_scale=1, seed=0
- Example input:
```json
{
  "model": "bytedance/latentsync",
  "input": {
    "video": "https://example.com/presenter.mp4",
    "audio": "https://example.com/korean-voiceover.mp3"
  }
}
```
- Full docs: https://console.core.today/docs/models/bytedance/latentsync

### MMAudio — `zsxkib/mmaudio`

Add AI-generated sound to any video (5M+ runs) — synthesizes synchronized audio (ambience, effects, foley) from the video content and an optional text prompt. The perfect finisher for silent AI-generated clips.

- Provider: MMAudio
- Credits: 11 (per generation)
- Speed: medium / Quality: high
- Optional params: prompt=, negative_prompt=music, video, duration=8, num_steps=25, cfg_strength=4.5, seed, image
- Example input:
```json
{
  "model": "zsxkib/mmaudio",
  "input": {
    "video": "https://example.com/generated-clip.mp4",
    "prompt": "ocean waves, seagulls, gentle wind"
  }
}
```
- Full docs: https://console.core.today/docs/models/zsxkib/mmaudio

### Real-ESRGAN Video — `lucataco/real-esrgan-video`

Video upscaling with Real-ESRGAN — enhance videos to FHD, 2K, or 4K frame by frame. The go-to open-source video upscaler for old footage and AI-generated clips.

- Provider: Real-ESRGAN
- Credits: 1280 (per run (cost varies strongly with video length/resolution))
- Speed: slow / Quality: high
- Required params: video_path (string)
- Optional params: resolution=FHD, model=RealESRGAN_x4plus
- Example input:
```json
{
  "model": "lucataco/real-esrgan-video",
  "input": {
    "video_path": "https://example.com/generated-clip.mp4",
    "model": "RealESRGAN_x4plus",
    "resolution": "4k"
  }
}
```
- Full docs: https://console.core.today/docs/models/lucataco/real-esrgan-video

### SadTalker — `cjwbw/sadtalker`

Talking-head video from a single photo and an audio track — animates the face with natural head motion and eye blinks, with optional GFPGAN face enhancement.

- Provider: SadTalker
- Credits: 91 (per run (runtime grows with audio length))
- Speed: slow / Quality: high
- Required params: source_image (string), driven_audio (string)
- Optional params: use_enhancer=false, pose_style=0, expression_scale=1, use_eyeblink=true, preprocess=crop, size_of_image=256, facerender=facevid2vid, still_mode=true
- Example input:
```json
{
  "model": "cjwbw/sadtalker",
  "input": {
    "source_image": "https://example.com/portrait.jpg",
    "driven_audio": "https://example.com/narration.wav",
    "preprocess": "full",
    "still_mode": true,
    "use_enhancer": true
  }
}
```
- Full docs: https://console.core.today/docs/models/cjwbw/sadtalker

### Veed Lipsync v2 — `veed/lipsync/v2`

Veed Lipsync v2 via Fal.AI. Replaces a source video's mouth movements to articulate a new audio track — takes any source video + a new audio track and produces a lip-synced output video.

- Provider: Veed
- Credits: 2440 (per video (flat rate, regardless of length))
- Speed: medium / Quality: high
- Required params: video_url (string), audio_url (string)
- Example input:
```json
{
  "model": "veed/lipsync/v2",
  "input": {
    "video_url": "https://v3.fal.media/files/monkey/q1fDPhrpfjfsaRmbhTed4_influencer.mp4",
    "audio_url": "https://v3.fal.media/files/rabbit/Ql3ade3wEKlZXRQLRbhxm_tts.mp3"
  }
}
```
- Full docs: https://console.core.today/docs/models/veed/lipsync/v2

### Video Utils — `nicolascoutureau/video-utils`

FFmpeg-powered video utilities (20M+ runs) — convert to mp4/gif, extract audio as mp3, or dump zipped frames, all with one task parameter. The Swiss-army knife for media pipelines.

- Provider: FFmpeg
- Credits: 5 (per operation)
- Speed: fast / Quality: standard
- Required params: task (string), input_file (string)
- Optional params: fps=0
- Example input:
```json
{
  "model": "nicolascoutureau/video-utils",
  "input": {
    "input_file": "https://example.com/meeting-recording.mp4",
    "task": "extract_video_audio_as_mp3"
  }
}
```
- Full docs: https://console.core.today/docs/models/nicolascoutureau/video-utils