Instance segmentation
What it does: Detects and segments each individual object separately, even when they overlap or touch.
Key characteristics:
- Instance-level separation – Each object is a separate entity
- Handles overlaps – Can distinguish touching/overlapping objects
- Rich information – Provides class, location, size, shape for each object
- Most versatile – Can be used for almost any detection task
Input
Input:
- Whole images (e.g., photos of products, components, or scenes)
- Corresponding pixel-level masks for training, with separate instances labeled individually
Labels
- Each object must be labeled separately with a unique instance ID
- Assign a class to each object
- Overlapping or touching objects must have separate masks
- Maintain consistent labeling conventions across the dataset
- Use precise pixel-level boundaries for each object
- Ensure no two overlapping objects share the same label
- Keep instance IDs consistent if you track objects across frames
- Review labels carefully to avoid merged instances
Output
Output:
- Individual masks for each detected object
- Class labels and bounding boxes for each instance
- Optionally, confidence scores per object
How it looks in the platform
- Each object is highlighted separately
- Users can view instance masks overlayed on the original image
- Bounding boxes and class names are displayed for each instance
When to use instance segmentation
Use instance segmentation when you need to:
✅ Multiple separate objects – Segment each object individually
✅ Object counting – Know how many objects are present
✅ Different classes per object – Objects may belong to different categories
✅ Overlapping objects – Distinguish touching or overlapping items
✅ Precise size measurement – Exact area of each object
✅ Exact positioning – Location of each object
✅ Individual tracking – Analyze each object separately
Example use cases
| Application | What It Detects & Segments |
|---|---|
| Defect inspection | Each individual defect with size and location |
| Object counting | Count bottles, pills, products on a line |
| Quality control | Measure each product's dimensions individually |
| Assembly verification | Detect and verify each component separately |
| Particle analysis | Identify and measure individual particles |
Instance segmentation is the most powerful and flexible model type.
It provides all the information that classification or segmentation provide, plus individual object details.
When in doubt, choose instance segmentation.
Limitations:
- ❌ Requires detailed labeling per object – more time-consuming
- ❌ Computationally heavier than regular segmentation or classification
- ❌ Larger datasets often needed for good generalization
- ❌ More memory and GPU resources required for training and inference
Better alternatives:
- Use Segmentation if you only need class-level regions without separating individual objects
- Use Classification if you only need a single label per image
Considerations for training
- Input size: Typically 256x256 or 512x512 px; larger sizes improve instance accuracy
- Data augmentation: Rotation, flipping, scaling, color jitter
- Class balance: Ensure enough labeled instances per class
- Loss function: Combination of mask loss and bounding box/classification loss
- Batch size and learning rate: Adjust according to dataset size and GPU memory
- Regularization: Dropout or weight decay may help reduce overfitting
Additional configuration
- Label consistency: Ensure unique instance IDs and consistent class labeling across the dataset