Hugging Face's logo

qualcomm
/
Segment-Anything-Model-2

Image Segmentation
PyTorch
foundation
android
Model card Files
xet
 Community

Segment-Anything-Model-2: Optimized for Qualcomm Devices

SAM 2, the successor to Meta's Segment Anything Model (SAM), is a cutting-edge tool designed for comprehensive object segmentation in both images and videos. It excels in handling complex visual data through a unified, promptable model architecture that supports real-time processing and zero-shot generalization.

This is based on the implementation of Segment-Anything-Model-2 found here. This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the Qualcomm® AI Hub Models library to export with custom configurations. More details on model performance across various devices, can be found here.

Qualcomm AI Hub Models uses Qualcomm AI Hub Workbench to compile, profile, and evaluate this model. Sign up to run these models on a hosted Qualcomm® device.

Getting Started

There are two ways to deploy this model on your device:

Option 1: Download Pre-Exported Models

Below are pre-exported model assets ready for deployment.

Runtime Precision Chipset SDK Versions Download
ONNX float Universal QAIRT 2.45, ONNX Runtime 1.27.1 Download
ONNX w8a8 Universal QAIRT 2.45, ONNX Runtime 1.27.1 Download
QNN_DLC float Universal QAIRT 2.45 Download
QNN_DLC w8a8 Universal QAIRT 2.45 Download
TFLITE w8a8 Universal QAIRT 2.45 Download

For more device-specific assets and performance metrics, visit Segment-Anything-Model-2 on Qualcomm® AI Hub.

Option 2: Export with Custom Configurations

Use the Qualcomm® AI Hub Models Python library to compile and export the model with your own:

  • Custom weights (e.g., fine-tuned checkpoints)
  • Custom input shapes
  • Target device and runtime configurations

This option is ideal if you need to customize the model beyond the default configuration provided here.

See our repository for Segment-Anything-Model-2 on GitHub for usage instructions.

Model Details

Model Type: Model_use_case.semantic_segmentation

Model Stats:

  • Model checkpoint: sam2.1_hiera_t
  • Input resolution: 720p (720x1280)
  • Number of parameters (encoder): 33.5M
  • Model size (encoder) (float): 128 MB
  • Number of parameters (decoder): 6.22M
  • Model size (decoder) (float): 23.7 MB

Performance Summary

Model Runtime Precision Chipset Inference Time (ms) Peak Memory Range (MB) Primary Compute Unit
decoder ONNX float Snapdragon® X2 Elite 3.384 ms 16 - 16 MB NPU
decoder ONNX float Qualcomm® Dragonwing™ Q-8750 3.755 ms 1 - 222 MB NPU
decoder ONNX float Snapdragon® 8 Elite Mobile 3.755 ms 1 - 222 MB NPU
decoder ONNX float Snapdragon® 8 Elite Gen 5 Mobile 3.185 ms 3 - 226 MB NPU
decoder ONNX w8a8 Snapdragon® X2 Elite 1.332 ms 4 - 4 MB NPU
decoder ONNX w8a8 Snapdragon® X Elite 2.733 ms 4 - 4 MB NPU
decoder ONNX w8a8 Snapdragon® 8 Gen 3 Mobile 1.755 ms 4 - 162 MB NPU
decoder ONNX w8a8 Snapdragon® 8 Gen 1 Mobile 3.464 ms 4 - 166 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ QCS6490 13.018 ms 4 - 7 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ QCS8550 (Proxy) 2.605 ms 2 - 14 MB NPU
decoder ONNX w8a8 Qualcomm® QCS8450 3.464 ms 4 - 166 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ IQ-9075 3.134 ms 4 - 7 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ IQ-X7181 2.733 ms 4 - 4 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ Q-8750 1.443 ms 0 - 132 MB NPU
decoder ONNX w8a8 Snapdragon® 8 Elite Mobile 1.443 ms 0 - 132 MB NPU
decoder ONNX w8a8 Snapdragon® 8 Elite Gen 5 Mobile 1.25 ms 0 - 158 MB NPU
decoder QNN_DLC float Snapdragon® X2 Elite 3.085 ms 16 - 16 MB NPU
decoder QNN_DLC float Snapdragon® 8 Gen 1 Mobile 10.013 ms 4 - 219 MB NPU
decoder QNN_DLC float Qualcomm® Dragonwing™ QCS8550 (Proxy) 5.342 ms 16 - 18 MB NPU
decoder QNN_DLC float Qualcomm® QCS8450 10.013 ms 4 - 219 MB NPU
decoder QNN_DLC float Qualcomm® Dragonwing™ Q-8750 2.918 ms 4 - 191 MB NPU
decoder QNN_DLC float Qualcomm® SA8295P 8.191 ms 0 - 195 MB NPU
decoder QNN_DLC float Snapdragon® 8 Elite Mobile 2.918 ms 4 - 191 MB NPU
decoder QNN_DLC float Snapdragon® 8 Elite Gen 5 Mobile 2.594 ms 4 - 192 MB NPU
decoder QNN_DLC w8a8 Snapdragon® X2 Elite 1.317 ms 4 - 4 MB NPU
decoder QNN_DLC w8a8 Snapdragon® X Elite 2.369 ms 4 - 4 MB NPU
decoder QNN_DLC w8a8 Snapdragon® 8 Gen 3 Mobile 1.297 ms 0 - 109 MB NPU
decoder QNN_DLC w8a8 Snapdragon® 8 Gen 1 Mobile 2.558 ms 0 - 111 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ QCS6490 9.611 ms 3 - 9 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ QCS8275 3.909 ms 4 - 89 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ QCS8550 (Proxy) 1.987 ms 4 - 6 MB NPU
decoder QNN_DLC w8a8 Qualcomm® SA8775P 2.607 ms 4 - 90 MB NPU
decoder QNN_DLC w8a8 Qualcomm® SA8650P 2.607 ms 4 - 90 MB NPU
decoder QNN_DLC w8a8 Qualcomm® SA8255P 2.607 ms 4 - 90 MB NPU
decoder QNN_DLC w8a8 Qualcomm® QCS8450 2.558 ms 0 - 111 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ IQ-9075 2.432 ms 3 - 9 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ IQ-X7181 2.369 ms 4 - 4 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ Q-6690 10.206 ms 4 - 275 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ Q-7790 2.663 ms 4 - 215 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ Q-8750 1.047 ms 1 - 91 MB NPU
decoder QNN_DLC w8a8 Qualcomm® SA7255P 3.909 ms 4 - 89 MB NPU
decoder QNN_DLC w8a8 Qualcomm® SA8295P 2.955 ms 4 - 88 MB NPU
decoder QNN_DLC w8a8 Snapdragon® 8 Elite Mobile 1.047 ms 1 - 91 MB NPU
decoder QNN_DLC w8a8 Snapdragon® 8 Elite Gen 5 Mobile 0.902 ms 1 - 128 MB NPU
decoder QNN_DLC w8a8 Snapdragon® 7 Gen 4 Mobile 2.663 ms 4 - 215 MB NPU
decoder TFLITE float Snapdragon® 8 Gen 3 Mobile 3.681 ms 0 - 216 MB NPU
decoder TFLITE float Snapdragon® 8 Gen 1 Mobile 10.042 ms 0 - 220 MB NPU
decoder TFLITE float Qualcomm® Dragonwing™ QCS8550 (Proxy) 5.337 ms 0 - 8 MB NPU
decoder TFLITE float Qualcomm® QCS8450 10.042 ms 0 - 220 MB NPU
decoder TFLITE float Qualcomm® SA8295P 8.189 ms 0 - 196 MB NPU
decoder TFLITE float Snapdragon® 8 Elite Gen 5 Mobile 2.631 ms 0 - 183 MB NPU
decoder TFLITE w8a8 Snapdragon® 8 Gen 3 Mobile 5.781 ms 0 - 265 MB NPU
decoder TFLITE w8a8 Snapdragon® 8 Gen 1 Mobile 10.161 ms 0 - 246 MB NPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ QCS6490 36.571 ms 12 - 58 MB NPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ QCS8275 13.6 ms 0 - 237 MB NPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ QCS8550 (Proxy) 7.692 ms 0 - 23 MB NPU
decoder TFLITE w8a8 Qualcomm® SA8775P 8.562 ms 0 - 240 MB NPU
decoder TFLITE w8a8 Qualcomm® SA8650P 8.562 ms 0 - 240 MB NPU
decoder TFLITE w8a8 Qualcomm® SA8255P 8.562 ms 0 - 240 MB NPU
decoder TFLITE w8a8 Qualcomm® QCS8450 10.161 ms 0 - 246 MB NPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ IQ-9075 8.27 ms 0 - 14 MB NPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ Q-6690 32.622 ms 12 - 210 MB NPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ Q-7790 14.22 ms 12 - 201 MB NPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ Q-8750 3.537 ms 0 - 235 MB NPU
decoder TFLITE w8a8 Qualcomm® SA7255P 13.6 ms 0 - 237 MB NPU
decoder TFLITE w8a8 Qualcomm® SA8295P 9.437 ms 0 - 240 MB NPU
decoder TFLITE w8a8 Snapdragon® 8 Elite Mobile 3.537 ms 0 - 235 MB NPU
decoder TFLITE w8a8 Snapdragon® 8 Elite Gen 5 Mobile 3.118 ms 0 - 254 MB NPU
decoder TFLITE w8a8 Snapdragon® 7 Gen 4 Mobile 14.22 ms 12 - 201 MB NPU
encoder ONNX float Snapdragon® X2 Elite 86.793 ms 70 - 70 MB NPU
encoder ONNX float Qualcomm® Dragonwing™ Q-8750 98.568 ms 44 - 1798 MB NPU
encoder ONNX float Snapdragon® 8 Elite Mobile 98.568 ms 44 - 1798 MB NPU
encoder ONNX float Snapdragon® 8 Elite Gen 5 Mobile 79.185 ms 24 - 1694 MB NPU
encoder ONNX w8a8 Snapdragon® X2 Elite 35.619 ms 18 - 18 MB NPU
encoder ONNX w8a8 Snapdragon® X Elite 90.652 ms 51 - 51 MB NPU
encoder ONNX w8a8 Snapdragon® 8 Gen 3 Mobile 63.217 ms 0 - 1928 MB NPU
encoder ONNX w8a8 Snapdragon® 8 Gen 1 Mobile 125.094 ms 14 - 2403 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ QCS6490 675.867 ms 5 - 11 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ QCS8550 (Proxy) 88.431 ms 0 - 57 MB NPU
encoder ONNX w8a8 Qualcomm® QCS8450 125.094 ms 14 - 2403 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ IQ-9075 75.706 ms 11 - 17 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ IQ-X7181 90.652 ms 51 - 51 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ Q-8750 40.216 ms 11 - 2123 MB NPU
encoder ONNX w8a8 Snapdragon® 8 Elite Mobile 40.216 ms 11 - 2123 MB NPU
encoder ONNX w8a8 Snapdragon® 8 Elite Gen 5 Mobile 34.193 ms 10 - 1679 MB NPU
encoder QNN_DLC float Snapdragon® X2 Elite 85.524 ms 12 - 12 MB NPU
encoder QNN_DLC float Snapdragon® 8 Gen 1 Mobile 342.061 ms 13 - 1866 MB NPU
encoder QNN_DLC float Qualcomm® Dragonwing™ QCS8550 (Proxy) 179.33 ms 12 - 19 MB NPU
encoder QNN_DLC float Qualcomm® QCS8450 342.061 ms 13 - 1866 MB NPU
encoder QNN_DLC float Qualcomm® Dragonwing™ Q-8750 100.354 ms 11 - 2520 MB NPU
encoder QNN_DLC float Qualcomm® SA8295P 282.46 ms 0 - 1581 MB NPU
encoder QNN_DLC float Snapdragon® 8 Elite Mobile 100.354 ms 11 - 2520 MB NPU
encoder QNN_DLC float Snapdragon® 8 Elite Gen 5 Mobile 78.443 ms 0 - 1707 MB NPU
encoder QNN_DLC w8a8 Snapdragon® X2 Elite 36.402 ms 3 - 3 MB NPU
encoder QNN_DLC w8a8 Snapdragon® X Elite 83.198 ms 3 - 3 MB NPU
encoder QNN_DLC w8a8 Snapdragon® 8 Gen 3 Mobile 55.57 ms 3 - 1881 MB NPU
encoder QNN_DLC w8a8 Snapdragon® 8 Gen 1 Mobile 137.17 ms 3 - 1901 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ QCS6490 678.512 ms 3 - 13 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ QCS8275 136.505 ms 3 - 1580 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ QCS8550 (Proxy) 79.851 ms 3 - 6 MB NPU
encoder QNN_DLC w8a8 Qualcomm® SA8775P 81.817 ms 3 - 1581 MB NPU
encoder QNN_DLC w8a8 Qualcomm® SA8650P 81.817 ms 3 - 1581 MB NPU
encoder QNN_DLC w8a8 Qualcomm® SA8255P 81.817 ms 3 - 1581 MB NPU
encoder QNN_DLC w8a8 Qualcomm® QCS8450 137.17 ms 3 - 1901 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ IQ-9075 111.122 ms 3 - 12 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ IQ-X7181 83.198 ms 3 - 3 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ Q-6690 1197.518 ms 3 - 1919 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ Q-7790 95.004 ms 3 - 2357 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ Q-8750 39.9 ms 3 - 1621 MB NPU
encoder QNN_DLC w8a8 Qualcomm® SA7255P 136.505 ms 3 - 1580 MB NPU
encoder QNN_DLC w8a8 Qualcomm® SA8295P 95.741 ms 0 - 1591 MB NPU
encoder QNN_DLC w8a8 Snapdragon® 8 Elite Mobile 39.9 ms 3 - 1621 MB NPU
encoder QNN_DLC w8a8 Snapdragon® 8 Elite Gen 5 Mobile 35.304 ms 3 - 1662 MB NPU
encoder QNN_DLC w8a8 Snapdragon® 7 Gen 4 Mobile 95.004 ms 3 - 2357 MB NPU
encoder TFLITE float Snapdragon® 8 Gen 3 Mobile 129.585 ms 19 - 2228 MB NPU
encoder TFLITE float Snapdragon® 8 Gen 1 Mobile 346.06 ms 21 - 1956 MB NPU
encoder TFLITE float Qualcomm® Dragonwing™ QCS8550 (Proxy) 179.198 ms 20 - 24 MB NPU
encoder TFLITE float Qualcomm® QCS8450 346.06 ms 21 - 1956 MB NPU
encoder TFLITE float Qualcomm® SA8295P 282.725 ms 20 - 1665 MB NPU
encoder TFLITE float Snapdragon® 8 Elite Gen 5 Mobile 78.835 ms 20 - 1776 MB NPU
encoder TFLITE w8a8 Snapdragon® 8 Gen 3 Mobile 69.696 ms 4 - 2222 MB NPU
encoder TFLITE w8a8 Snapdragon® 8 Gen 1 Mobile 185.431 ms 5 - 2284 MB NPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ QCS6490 1723.37 ms 78 - 340 MB CPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ QCS8275 162.93 ms 6 - 1874 MB NPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ QCS8550 (Proxy) 97.837 ms 5 - 9 MB NPU
encoder TFLITE w8a8 Qualcomm® SA8775P 98.023 ms 5 - 1874 MB NPU
encoder TFLITE w8a8 Qualcomm® SA8650P 98.023 ms 5 - 1874 MB NPU
encoder TFLITE w8a8 Qualcomm® SA8255P 98.023 ms 5 - 1874 MB NPU
encoder TFLITE w8a8 Qualcomm® QCS8450 185.431 ms 5 - 2284 MB NPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ IQ-9075 96.78 ms 4 - 69 MB NPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ Q-6690 2390.766 ms 57 - 290 MB CPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ Q-7790 1144.641 ms 20 - 230 MB CPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ Q-8750 54.759 ms 3 - 1792 MB NPU
encoder TFLITE w8a8 Qualcomm® SA7255P 162.93 ms 6 - 1874 MB NPU
encoder TFLITE w8a8 Qualcomm® SA8295P 117.58 ms 5 - 1882 MB NPU
encoder TFLITE w8a8 Snapdragon® 8 Elite Mobile 54.759 ms 3 - 1792 MB NPU
encoder TFLITE w8a8 Snapdragon® 8 Elite Gen 5 Mobile 34.481 ms 4 - 1787 MB NPU
encoder TFLITE w8a8 Snapdragon® 7 Gen 4 Mobile 1144.641 ms 20 - 230 MB CPU

License

  • The license for the original implementation of Segment-Anything-Model-2 can be found here.

References

Community

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
Image Segmentation
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Paper for qualcomm/Segment-Anything-Model-2