Introduction

A robot can only react to what it can measure.

Encoders can tell you how far a motor has turned. An IMU can tell you how the robot is rotating. Distance sensors can tell you how far away an object is.

But sometimes, you need your robot to see.

Computer vision allows a robot to analyze images from a camera and extract useful information. This can allow a robot to identify objects, locate itself on the field, align with a target, or make decisions based on what is in front of it.

Why use vision?

Vision is particularly useful when the information you need cannot easily be obtained from traditional sensors.

Common applications include:

• Detecting AprilTags
• Localizing the robot on the field
• Aligning with scoring targets
• Detecting game elements
• Identifying objects by color or shape
• Estimating distance and orientation
• Tracking objects

Vision is not always the best solution. Cameras add complexity and can be affected by lighting, camera placement, motion blur, and processing limitations.

Use vision when the information provided by a camera is worth that additional complexity.

How vision works

At a high level, a vision system follows a simple process:

Camera
   ↓
Image
   ↓
Vision Processing
   ↓
Useful Information
   ↓
Robot Code

The camera captures an image. A vision system processes that image and identifies something useful. Your robot code can then use that information to make decisions.

For example:

Camera sees AprilTag
        ↓
AprilTag processor detects the tag
        ↓
Pose is estimated
        ↓
Robot determines its position relative to the tag
        ↓
Robot uses that information to align

The important idea is that vision turns images into data.

Vision Hardware

Before processing an image, your robot needs a way to capture one.

FTC robots can use several types of cameras and vision hardware, ranging from inexpensive USB webcams to dedicated vision processors.

Common options include:

• Logitech C270
• Logitech C310
• Logitech C920
• Global shutter cameras
• Android phone cameras
• Limelight 3A

Different cameras have different tradeoffs in resolution, field of view, frame rate, shutter type, and calibration.

For example, a global shutter camera can reduce motion distortion on a fast-moving robot, while a Logitech C920 provides a good general-purpose webcam with extensive FTC calibration support.

The Limelight 3A takes a different approach by performing vision processing on dedicated hardware rather than sending the camera feed to the Robot Controller for processing.

See Cameras for a comparison of camera and vision hardware available for FTC.

Vision in FTC

Once a camera is connected, FTC provides several technologies for turning its images into useful information.

VisionPortal

VisionPortal is the FTC SDK's framework for working with cameras and vision processors.

It manages the camera stream and provides frames to vision processors. Multiple processors can use the same camera.

AprilTags

AprilTags are visual markers that can be detected by a camera.

The FTC SDK can identify a tag's ID and estimate its position and orientation relative to the camera.

This makes AprilTags particularly useful for localization, navigation, and alignment.

OpenCV

OpenCV is a computer vision library that provides tools for processing images.

In FTC, OpenCV can be used to create custom vision pipelines, such as detecting objects based on color, shape, or location.

Limelight

Limelight is a dedicated vision system that performs image processing on its own hardware and sends the results to your robot.

It can be used for AprilTags, object detection, localization, and other vision applications.

How the Pieces Fit Together

These technologies solve different parts of the vision problem.

Camera
   ↓
Captures images
   ↓
Vision Framework
   ↓
VisionPortal / Limelight
   ↓
Vision Processing
   ↓
AprilTags / OpenCV / Other Processors
   ↓
Useful Data
   ↓
Robot Code

For example, a conventional FTC webcam might use:

Webcam
   ↓
VisionPortal
   ↓
AprilTag Processor
   ↓
AprilTag Detection
   ↓
Robot Code

A Limelight follows a different architecture:

Limelight Camera
   ↓
Limelight Processor
   ↓
AprilTag / Vision Pipeline
   ↓
Vision Result
   ↓
Robot Code

Understanding these layers makes it easier to choose the right hardware and software for your robot.

The Vision Section

This section focuses on the major parts of FTC vision:

1. Cameras: Cameras and vision hardware you can use on your robot
2. VisionPortal: The FTC SDK's vision framework
3. AprilTags: A powerful way to detect known visual markers
4. OpenCV: A toolkit for creating custom vision pipelines
5. Limelight: Dedicated vision hardware for advanced vision processing

Each technology solves a slightly different problem. Understanding how they fit together is more important than memorizing a particular API.