Types

Mixtrain provides typed values that enable rich rendering in the UI. When models or workflows return typed values, the frontend renders them appropriately (images, videos, audio players, etc.).

Media types can be used as both inputs and outputs. When used as inputs, they provide convenient helper methods to download and process files.

Media Types

All media types (Image, Video, Audio, File) support:

URLs from any source: https://, gs://, s3://
Automatic previews
Helper methods for common operations

File

Base class for file references. Use for generic binary files.

from mixtrain import File

# As output
return {"data": File(url="gs://bucket/data.bin")}

# As input in model
def run(self, data: File):
    local_path = data.to_file()  # Downloads to temp file

Field	Type	Description
`url`	`str`	URL to the file (https://, gs://, s3://)
`content_type`	`str \| None`	MIME type
`filename`	`str \| None`	Original filename

Methods:

to_file(path=None) - Download to local file. Returns path. Cached for subsequent calls.

Image

Renders an image inline. Extends File with image-specific helpers.

from mixtrain import Image

# As output
return {"result": Image(url="https://...", width=1024, height=1024)}

# As input in model
def run(self, image: Image):
    pil_img = image.to_pil()      # PIL.Image object
    arr = image.to_numpy()         # numpy array (H, W, C)

Field	Type	Description
`url`	`str`	URL to the image
`width`	`int \| None`	Width in pixels
`height`	`int \| None`	Height in pixels
`format`	`str \| None`	Format (png, jpg, webp)

Methods:

to_file(path=None) - Download to local file
to_pil() - Return as PIL.Image object
to_numpy() - Return as numpy array (H, W, C)

Video

Renders a video with a player. Extends File with video-specific helpers.

from mixtrain import Video

# As output
return {"result": Video(url="https://...", duration_seconds=5.0)}

# As input in model
def run(self, video: Video):
    cap = video.to_cv2()           # cv2.VideoCapture
    for frame in video.to_frames(): # iterate frames as numpy
        process(frame)

Field	Type	Description
`url`	`str`	URL to the video
`duration_seconds`	`float \| None`	Duration in seconds
`width`	`int \| None`	Width in pixels
`height`	`int \| None`	Height in pixels
`format`	`str \| None`	Format (mp4, webm)

Methods:

to_file(path=None) - Download to local file
to_cv2() - Return as cv2.VideoCapture
to_frames() - Yield frames as numpy arrays (H, W, C) in BGR

Audio

Renders an audio player. Extends File with audio-specific helpers.

from mixtrain import Audio

# As output
return {"result": Audio(url="https://...", duration_seconds=30.0)}

# As input in model
def run(self, audio: Audio):
    samples, sample_rate = audio.to_numpy()

Field	Type	Description
`url`	`str`	URL to the audio file
`duration_seconds`	`float \| None`	Duration in seconds
`format`	`str \| None`	Format (mp3, wav)

Methods:

to_file(path=None) - Download to local file
to_numpy() - Return as (samples array, sample_rate) tuple

Model3D

Renders a 3D model viewer.

from mixtrain import Model3D

return {"result": Model3D(url="https://...", format="glb")}

Field	Type	Description
`url`	`str`	URL to the 3D model file
`format`	`str \| None`	Format (glb, gltf, obj)

Text Types

Text

Plain text content.

from mixtrain import Text

return {"message": Text(content="Processing complete!")}

Markdown

Markdown formatted content with rich rendering.

from mixtrain import Markdown

return {"report": Markdown(content="# Results\n- Item 1\n- Item 2")}

JSON

JSON data with syntax highlighting.

from mixtrain import JSON

return {"config": JSON(data={"key": "value", "items": [1, 2, 3]})}

Data Types

Embedding

ML embedding vectors for semantic representations.

from mixtrain import Embedding

return {"embedding": Embedding(
    values=[0.1, 0.2, 0.3, ...],
    dimension=1536,
    model="text-embedding-3-small"
)}

Field	Type	Description
`values`	`list[float]`	The embedding vector
`dimension`	`int \| None`	Dimension (auto-calculated if not provided)
`model`	`str \| None`	Model that generated the embedding

Using Types in Models

Return types from your model's run method to enable rich UI rendering:

from mixtrain import MixModel, Image

class ImageGenerator(MixModel):
    def run(self, inputs: dict) -> dict[str, Image]:
        url = generate_image(inputs["prompt"])
        return {"image": Image(url=url, width=1024, height=1024)}

See Models for more details.

Using Types in Workflows

Workflows can return typed values between steps and as final outputs:

from mixtrain import MixFlow, Video

class VideoWorkflow(MixFlow):
    def run(self, inputs: dict) -> dict[str, Video]:
        url = process_video(inputs["source"])
        return {"video": Video(url=url, duration_seconds=10.0)}

See Workflows for more details.

On this page