The VideoIO API is to video what the ImageIO API is to still images, but it goes deeper. It uses the platform’s native codecs to:

  1. Encode application rendered frames and audio into a standard video file, with deep control over dimensions, frame rate, codec, bitrate, key frame interval and the audio track.

  2. Decode an existing clip into frame accurate RGBA frames and PCM audio.

The decode side is something the Media player API doesn’t provide. Media is a player: its setTime() snaps to key frames and gives no exact frame guarantee. VideoReader, by contrast, decodes the precise frame at a requested time, which is what you need when you want to read an imported clip (for example a screen recording) frame by frame.

VideoIO is supported on iOS, macOS, Android, Windows, Linux, JavaScript and the desktop simulator. It’s not available on the TV, Watch or Car targets.

Checking for support

As with other optional platform features, always gate usage with isSupported():

if (!VideoIO.isSupported()) {
    // Video encoding / decoding is not available on this platform
    // (for example TV, Watch or Car), so fall back gracefully.
    return;
}
VideoIO io = VideoIO.getVideoIO();
On the desktop simulator VideoIO is backed by an ffmpeg/ffprobe binary. The simulator bundles one automatically; if isSupported() returns false on the desktop, make sure ffmpeg is on your PATH or set the ffmpeg.dir system property.

Encoding a video

Build a VideoWriter with a VideoWriterBuilder, then push frames and (optionally) audio. Because every frame is an ordinary Image that you draw into, you have complete control over the pixels - render charts, overlays, transformed camera frames, generated animation, anything.

String out = FileSystemStorage.getInstance().getAppHomePath() + "/generated.mp4";
int w = 640, h = 480;
float fps = 30;

VideoWriter writer = new VideoWriterBuilder()
        .path(out)
        .width(w).height(h).frameRate(fps)
        .videoCodec(VideoIO.CODEC_H264).videoBitRate(4_000_000)
        .build();

// Each frame is just an Image you fully control: draw whatever you like.
for (int i = 0; i < 90; i++) {                 // 3 seconds at 30fps
    Image frame = Image.createImage(w, h, 0xff000000);
    Graphics g = frame.getGraphics();
    g.setColor(0xffffff);
    g.fillRect((i * 8) % w, h / 2 - 20, 60, 40);
    writer.writeFrame(frame, Math.round(i * 1000f / fps));
}
writer.close();

To add an audio track, enable it on the builder (.hasAudio(true)) and push interleaved 16 bit PCM with VideoWriter.writeAudio(…​), or hand it an AudioBuffer.

Decoding a video frame by frame

VideoReader exposes the clip’s duration, dimensions and frame rate, a frame accurate frameAt(long) and a variable-to-constant frame rate resampler readFrames(fps, callback) that walks the whole clip emitting evenly spaced frames:

VideoReader reader = VideoIO.getVideoIO().openReader(videoPath);
System.out.println(reader.getWidth() + "x" + reader.getHeight()
        + " " + reader.getFrameRate() + "fps, "
        + reader.getDurationMillis() + "ms");

List<Image> thumbnails = new ArrayList<>();

// Frame accurate single frame (unlike Media.setTime which snaps to key frames):
VideoFrame oneSecond = reader.frameAt(1000);
if (oneSecond != null) {
    thumbnails.add(oneSecond.toImage());
}

// Resample the (possibly variable frame rate) clip to a constant 10fps stream:
reader.readFrames(10, f -> {
    // f.getARGB() / f.toImage() give you the decoded RGBA pixels
    thumbnails.add(f.toImage());
    return thumbnails.size() < 50;   // stop after 50 frames
});
reader.close();

The audio track, when present, is decoded to interleaved PCM:

VideoReader reader = VideoIO.getVideoIO().openReader(videoPath);
if (reader.hasAudio()) {
    AudioBuffer pcm = reader.readAudio();
    System.out.println("Decoded " + pcm.getSize() + " PCM samples at "
            + reader.getAudioSampleRate() + "Hz, "
            + reader.getAudioChannels() + " channels");
}
reader.close();

Discovering available codecs

The set of codecs is device dependent. Enumerate it to pick the best one (and to detect hardware accelerated encoders):

VideoIO io = VideoIO.getVideoIO();
for (VideoCodec codec : io.getAvailableEncoders()) {
    System.out.println(codec.getId()
            + " (" + codec.getName() + ")"
            + (codec.isHardwareAccelerated() ? " [hardware]" : ""));
}
boolean canEncodeH264 = io.isEncoderSupported(VideoIO.CODEC_H264);

Platform support

PlatformBackend

iOS / macOS

AVFoundation (AVAssetReader / AVAssetWriter, hardware H.264/HEVC)

Android

MediaCodec + MediaMuxer / MediaExtractor + MediaMetadataRetriever

JavaScript

HTML5 <video> + <canvas> decode + WebCodecs encode

Windows

Media Foundation (IMFSourceReader / IMFSinkWriter)

Linux

GStreamer (appsrc / appsink)

Simulator

ffmpeg / ffprobe

TV / Watch / Car

Not supported (isSupported() returns false)