Lesson 03: Manual Annotation Deep Dive

Before you lean on AI auto-labeling, you need to be fast with manual annotation. Every AI model makes mistakes, and your correction speed determines whether AI-assisted labeling is 3x faster or barely faster at all.

This lesson covers every annotation type, when to use each one, and the keyboard patterns that make it efficient.

Rectangle (Bounding Box)

The most common annotation type. Used for object detection tasks (YOLO, SSD, Faster R-CNN).

How to draw:

1. Press R or select Rectangle from the toolbar.
2. Click the top-left corner of the object.
3. Drag to the bottom-right corner and release.
4. Type the label name and press Enter.

When to use: Object detection, image classification with localization, any task where a tight bounding box is sufficient. If your model only needs to know "there is a car here," rectangles are the right choice.

Tips for tight boxes:

• Zoom in (Ctrl + scroll) before drawing. A box that looks tight at 100% zoom often has 10-20 pixels of slack.
• Include the full object. If part of a car is occluded by another car, the box should cover the visible portion, not extend to where you think the full car would be — unless your annotation guidelines say otherwise.
• Grab the corner handles to resize after drawing. You do not need to get it perfect on the first drag.

Polygon

Pixel-level annotation for instance segmentation (Mask R-CNN, YOLO-Seg, Detectron2).

How to draw:

1. Press P or select Polygon from the toolbar.
2. Click to place vertices around the object boundary.
3. Each click adds a point. The line follows your cursor.
4. Close the polygon by clicking the first point, or right-click to auto-close.
5. Type the label and press Enter.

When to use: Instance segmentation, semantic segmentation, any task where you need the precise shape of the object rather than just its bounding box.

Tips for fast polygons:

• You do not need 100 vertices. For a car, 15-25 vertices capture the shape well enough for most segmentation models. More vertices mean more time and more noise.
• Use straight edges where the object has straight edges. Do not add extra vertices on a flat surface.
• For complex boundaries (tree canopy, hair), get the rough shape right and let the model learn the details. Over-annotating complex edges wastes time and does not help the model.
• After closing, drag individual vertices to adjust. You can also add vertices by clicking on an edge.

Circle

Less common, but useful for specific tasks.

How to draw:

1. Select Circle from the toolbar.
2. Click the center of the circle.
3. Drag outward to set the radius and release.

When to use: Annotating circular objects (wheels, buttons, cells in microscopy), or marking regions of interest. Some medical imaging tasks use circles for lesion annotation.

Line

Two-point annotation for directional or edge-based tasks.

How to draw:

1. Select Line from the toolbar.
2. Click the start point.
3. Click the end point.
4. Label and confirm.

When to use: Lane detection, crack detection, measuring distances or angles, annotating linear features like roads or pipelines in aerial imagery.

Point (Keypoint)

Single-point annotation for landmark and pose tasks.

How to draw:

1. Select Point from the toolbar.
2. Click the exact location.
3. Label and confirm.

When to use: Facial landmark detection (68-point faces), body pose estimation (17 COCO keypoints), dental landmark detection in X-rays, or any task requiring precise point localization.

Tips for keypoints:

• Zoom in. A 2-pixel error at 100% zoom becomes a 10-pixel error in training data.
• Use consistent ordering. If you are annotating pose keypoints, always label them in the same order (e.g., nose, left_eye, right_eye, ...).
• For occluded keypoints, follow your annotation guideline. Some protocols mark them as "not visible" with a flag; others skip them entirely.

Rotation (Oriented Bounding Box)

For objects at arbitrary angles — aerial imagery, document analysis, text detection. The DOTA dataset is the standard benchmark for oriented object detection.

How to draw:

1. Select the Rotation tool from the toolbar.
2. Draw a rectangle aligned with the object's orientation.
3. Adjust the rotation angle by dragging the rotation handle.
4. Label and confirm.

When to use: Aerial/satellite object detection (DOTA dataset format), text detection in natural scenes, detecting objects that are not axis-aligned (ships, vehicles from overhead).

Editing Annotations

After drawing, you can always refine:

Labeling Workflow for Speed

The fastest manual annotators follow a pattern:

1. One class at a time. Label all car instances in an image before moving to pedestrian. This reduces context switching and improves consistency.
2. Coarse to fine. Draw rough shapes first, then refine. Getting all objects placed is more important than getting each one perfect on the first pass.
3. Keyboard-first navigation. D for next image, A for previous. Keep your left hand on the keyboard and right hand on the mouse.
4. Consistent zoom level. Enable View > Keep Previous Zoom Ratio for datasets with similar-sized objects.
5. Label reuse. After labeling the first instance, AnyLabeling remembers the label. For subsequent shapes of the same class, just press Enter to confirm the same label.

Group IDs

For tasks that require associating multiple shapes (e.g., linking a bounding box with its keypoints, or connecting detected text with its transcription), AnyLabeling supports group_id. All shapes with the same group ID are treated as belonging to the same entity.

Set the group ID in the shape properties dialog after creating the annotation.

Key Takeaways

• Master rectangle and polygon first — they cover 90% of annotation tasks.
• Zoom in before drawing. Precision at annotation time saves debugging time later.
• One class at a time, coarse to fine. This workflow minimizes errors and maximizes speed.
• You do not need pixel-perfect polygons. 15-25 vertices capture most object shapes well enough for training.

In the next lesson, we bring in the AI. Segment Anything will do the polygon drawing for you.