⚡�FastDeploy是一款简�易用的推�部署工具箱。覆盖业界主�优质预训练模型并�供开箱�用的开�体验，包括图�分类�目标检测�图�分割�人脸检测�人脸识别�人体关键点识别�文字识别等多任务，满足开�者多场景，多硬件�多平�的快速部署需求。

近期更新

• 🔥 2022.8.18：å�‘布FastDeploy release/v0.2.0
  
  • æœ�务端全新å�‡çº§ï¼šä¸€å¥—SDK，覆盖全é‡�模型
    • å�‘布基于x86 CPUã€�NVIDIA GPU的易用ã€�高性能推ç�†å¼•æ“ŽSDK，推ç�†é€Ÿåº¦å¤§å¹…æ��å�‡
    • 支æŒ�ONNXRuntimeã€�Paddle Inferenceã€�TensorRT推ç�†å¼•æ“Ž
    • 支æŒ�YOLOv7ã€�YOLOv6ã€�YOLOv5ã€�PP-YOLOE等目标检测最优模型å�ŠDemo示例
    • 支æŒ�人脸检测ã€�人脸识别ã€�实时人åƒ�抠图ã€�图åƒ�分割等40+é‡�点模型å�ŠDemo示例
    • 支æŒ�Python API å’Œ C++ API
    • å¼€å�‘AI模型部署代ç �é‡�å‡�少～60%
  • 端侧继ARM CPUå�Žï¼Œå»¶ä¼¸è‡³ç‘žèŠ¯å¾®ã€�晶晨ã€�æ�©æ™ºæµ¦ç­‰NPU能力
    • å�‘布轻é‡�化目标检测Picodet-NPU部署Demo，æ��供低门槛INT8å…¨é‡�化能力

内容目录

• æœ�务端
  • æœ�务端快速开始
    • 快速安装
    • Python预测示例
    • C++预测示例
  • æœ�务端模型列表
• 端侧
  • 端侧文档
    • ARM CPU端部署
    • ARM CPU移动端部署
    • ARM CPU自定义模型
    • NPU端部署
  • 端侧模型列表
• 社区交æµ�
• Acknowledge
• License

1. �务端快速开始

1.1 快速安装 FastDeploy Python/C++ 库

环境�赖

• Linux x64/aarch64
• Windows 10
• Mac OSX x86/arm64
• cuda >= 11.2
• cudnn >= 8.0
• python 3.6~3.9(Windows 10 3.8~3.9)

安装 CPU Python 版本

pip install numpy opencv-python fastdeploy-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html

安装 GPU Python 版本

pip install numpy opencv-python fastdeploy-gpu-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html

安装 C++ 版本

• å�‚考C++预编译库下载文档

准备目标检测模型和测试图片

wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg

1.2 Python预测示例

# GPU/TensorRT部署�考 examples/vision/detection/paddledetection/python
import cv2
import fastdeploy.vision as vision

model = vision.detection.PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel", 
                                 "ppyoloe_crn_l_300e_coco/model.pdiparams", 
                                 "ppyoloe_crn_l_300e_coco/nfer_cfg.yml")
im = cv2.imread("000000014439.jpg")
result = model.predict(im.copy())
print(result)

vis_im = vision.vis_detection(im, result, score_threshold=0.5)
cv2.imwrite("vis_image.jpg", vis_im)

1.3 C++预测示例

// GPU/TensorRT部署�考 examples/vision/detection/paddledetection/cpp
#include "fastdeploy/vision.h"

int main(int argc, char* argv[]) {
  namespace vision = fastdeploy::vision;
  auto model = vision::detection::PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel", 
                                          "ppyoloe_crn_l_300e_coco/model.pdiparams", 
                                          "ppyoloe_crn_l_300e_coco/infer_cfg.yml");
  auto im = cv::imread("000000014439.jpg");

  vision::DetectionResult res;
  model.Predict(&im, &res)

  auto vis_im = vision::Visualize::VisDetection(im, res, 0.5);
  cv::imwrite("vis_image.jpg", vis_im);
}

更多部署案例请�考视觉模型部署示例 .

2. �务端模型列表 🔥🔥🔥

符�说明: (1) ✅: 已�支�; (2) �: 未�支�; (3) �: 暂�支�; (4) --: 暂�考虑;
链接说明：「模型列�会跳转到模型推�Demo代�

任务场景	模型	API	Linux	Linux	Win	Win	Mac	Mac	Linux	Linux
---	---	---	X86 CPU	NVIDIA GPU	Intel CPU	NVIDIA GPU	Intel CPU	Arm CPU	AArch64 CPU	NVIDIA Jetson
Classification	PaddleClas/ResNet50	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Classification	PaddleClas/PPLCNet	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Classification	PaddleClas/PPLCNetv2	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Classification	PaddleClas/EfficientNet	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Classification	PaddleClas/GhostNet	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Classification	PaddleClas/MobileNetV1	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Classification	PaddleClas/MobileNetV2	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Classification	PaddleClas/MobileNetV3	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Classification	PaddleClas/ShuffleNetV2	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Classification	PaddleClas/SqueeezeNetV1.1	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Classification	PaddleClas/Inceptionv3	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Classification	PaddleClas/PPHGNet	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Classification	PaddleClas/SwinTransformer	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Detection	PaddleDetection/PPYOLOE	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Detection	PaddleDetection/PicoDet	Python/C++	✅	✅	✅	✅	��…	✅	✅	â�”
Detection	PaddleDetection/YOLOX	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Detection	PaddleDetection/YOLOv3	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Detection	PaddleDetection/PPYOLO	Python/C++	✅	✅	✅	✅	✅	�	�	�
Detection	PaddleDetection/PPYOLOv2	Python/C++	✅	✅	✅	✅	✅	�	�	�
Detection	PaddleDetection/FasterRCNN	Python/C++	✅	✅	✅	✅	✅	�	�	�
Detection	Megvii-BaseDetection/YOLOX	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Detection	WongKinYiu/YOLOv7	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Detection	meituan/YOLOv6	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Detection	ultralytics/YOLOv5	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Detection	WongKinYiu/YOLOR	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Detection	WongKinYiu/ScaledYOLOv4	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Detection	ppogg/YOLOv5Lite	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Detection	RangiLyu/NanoDetPlus	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Segmentation	PaddleSeg/PPLiteSeg	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Segmentation	PaddleSeg/PPHumanSegLite	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Segmentation	PaddleSeg/HRNet	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Segmentation	PaddleSeg/PPHumanSegServer	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Segmentation	PaddleSeg/Unet	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Segmentation	PaddleSeg/Deeplabv3	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
FaceDetection	biubug6/RetinaFace	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
FaceDetection	Linzaer/UltraFace	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
FaceDetection	deepcam-cn/YOLOv5Face	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
FaceDetection	deepinsight/SCRFD	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
FaceRecognition	deepinsight/ArcFace	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
FaceRecognition	deepinsight/CosFace	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
FaceRecognition	deepinsight/PartialFC	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
FaceRecognition	deepinsight/VPL	Python/C++	✅	✅	✅	✅	✅	✅	✅	�
Matting	ZHKKKe/MODNet	Python/C++	✅	✅	✅	✅	✅	✅	✅	�

3. 端侧文档

3.1 端侧部署

• ARM Linux 系统
  • C++ Inference部署（å�«è§†é¢‘æµ�）
  • C++ æœ�务化部署
  • Python Inference部署
  • Python æœ�务化部署

3.2 移动端部署

• iOS 系统部署
• Android 系统部署

3.3 自定义模型部署

• 快速实现个性化模型替æ�¢

3.4 NPU部署

• 瑞芯微-NPU/晶晨-NPU/æ�©æ™ºæµ¦-NPU

4. 端侧模型列表

任务场景	模型	大�(MB)	Linux	Android	iOS	Linux	Linux	Linux	更新中...
---	---	---	ARM CPU	ARM CPU	ARM CPU	瑞芯微NPU RV1109 RV1126 RK1808	晶晨NPU A311D S905D C308X	�智浦NPU i.MX 8M Plus	更新中...｜
Classification	PP-LCNet	11.9	✅	✅	✅	--	--	--	--
Classification	PP-LCNetv2	26.6	✅	✅	✅	--	--	--	--
Classification	EfficientNet	31.4	✅	✅	✅	--	--	--	--
Classification	GhostNet	20.8	✅	✅	✅	--	--	--	--
Classification	MobileNetV1	17	✅	✅	✅	--	--	--	--
Classification	MobileNetV2	14.2	✅	✅	✅	--	--	--	--
Classification	MobileNetV3	22	✅	✅	✅	�	�	�	�
Classification	ShuffleNetV2	9.2	✅	✅	✅	--	--	--	--
Classification	SqueezeNetV1.1	5	✅	✅	✅
Classification	Inceptionv3	95.5	✅	✅	✅	--	--	--	--
Classification	PP-HGNet	59	✅	✅	✅	--	--	--	--
Classification	SwinTransformer_224_win7	352.7	✅	✅	✅	--	--	--	--
Detection	PP-PicoDet_s_320_coco	4.1	✅	✅	✅	--	--	--	--
Detection	PP-PicoDet_s_320_lcnet	4.9	✅	✅	✅	✅	✅	✅	�
Detection	CenterNet	4.8	✅	✅	✅	--	--	--	--
Detection	YOLOv3_MobileNetV3	94.6	✅	✅	✅	--	--	--	--
Detection	PP-YOLO_tiny_650e_coco	4.4	✅	✅	✅	--	--	--	--
Detection	SSD_MobileNetV1_300_120e_voc	23.3	✅	✅	✅	--	--	--	--
Detection	PP-YOLO_ResNet50vd	188.5	✅	✅	✅	--	--	--	--
Detection	PP-YOLOv2_ResNet50vd	218.7	✅	✅	✅	--	--	--	--
Detection	PP-YOLO_crn_l_300e_coco	209.1	✅	✅	✅	--	--	--	--
Detection	YOLOv5s	29.3	✅	✅	✅	--	--	--	--
FaceDetection	BlazeFace	1.5	✅	✅	✅	--	--	--	--
FaceDetection	RetinaFace	1.7	✅	�	�	--	--	--	--
KeypointsDetection	PP-TinyPose	5.5	✅	✅	✅	�	�	�	�
Segmentation	PP-LiteSeg(STDC1)	32.2	✅	✅	✅	--	--	--	--
Segmentation	PP-HumanSeg-Lite	0.556	✅	✅	✅	--	--	--	--
Segmentation	HRNet-w18	38.7	✅	✅	✅	--	--	--	--
Segmentation	PP-HumanSeg-Server	107.2	✅	✅	✅	--	--	--	--
Segmentation	Unet	53.7	�	✅	�	--	--	--	--
OCR	PP-OCRv1	2.3+4.4	✅	✅	✅	--	--	--	--
OCR	PP-OCRv2	2.3+4.4	✅	✅	✅	--	--	--	--
OCR	PP-OCRv3	2.4+10.6	✅	✅	✅	�	�	�	�
OCR	PP-OCRv3-tiny	2.4+10.7	✅	✅	✅	--	--	--	--

5. 社区交�

• 加入社区👬： 微信扫æ��二维ç �å�Žï¼Œå¡«å†™é—®å�·åŠ å…¥äº¤æµ�群，与开å�‘者共å�Œè®¨è®ºæŽ¨ç�†éƒ¨ç½²ç—›ç‚¹é—®é¢˜

6. Acknowledge

本项目中SDK生�和下载使用了EasyEdge中的�费开放能力，�次表示感谢。

7. License

FastDeploy�循Apache-2.0开��议。