diff --git a/README_CN.md b/README_CN.md
index ab6030aa23..b8eed04585 100644
--- a/README_CN.md
+++ b/README_CN.md
@@ -16,59 +16,98 @@
-**⚡️FastDeploy**是一款**易用高效**的推理部署开发套件。覆盖业界🔥**热门AI模型**并提供📦**开箱即用**的部署体验，包括图像分类、目标检测、图像分割、人脸检测、人脸识别、人体关键点识别、文字识别、语义理解等多任务，满足开发者**多场景**，**多硬件**、**多平台**的产业部署需求。
+**⚡️FastDeploy**是一款**易用高效**的推理部署开发套件。覆盖业界🔥**热门CV、NLP、Speech的AI模型**并提供📦**开箱即用**的部署体验，包括图像分类、目标检测、图像分割、人脸检测、人脸识别、人体关键点识别、文字识别、语义理解等多任务，满足开发者**多场景**，**多硬件**、**多平台**的产业部署需求。
-| Potrait Segmentation | Image Matting | Semantic Segmentation | Real-Time Matting |
+| [Object Detection](examples/vision) | [3D Object Detection](https://github.com/PaddlePaddle/FastDeploy/issues/6) | [Semantic Segmentation](examples/vision/segmentation/paddleseg) | [Potrait Segmentation](examples/vision/segmentation/paddleseg) |
|:---------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
-| 
| 
| 
| 
|
-| **OCR** | **Behavior Recognition** | **Object Detection** |**Pose Estimation**
-| 
|
| 
|
|
-| **Face Alignment** | **3D Object Detection** | **Face Editing** | **Image Animation**
-| 
| 
| 
| 
+| | | | |
+| [**Image Matting**](examples/vision/matting) | [**Real-Time Matting**](examples/vision/matting) | [**OCR**](examples/vision/ocr) |[**Face Alignment**](examples/vision/ocr)
+| | | | |
+| [**Pose Estimation**](examples/vision/keypointdetection) | [**Behavior Recognition**](https://github.com/PaddlePaddle/FastDeploy/issues/6) | [**NLP**](examples/text) |[**Speech**](examples/audio/pp-tts)
+| | | 
| **input** ：早上好，今天是2020
/10/29，最低温度是-3°C。
**output**: [
](https://paddlespeech.bj.bcebos.com/Parakeet/docs/demos/parakeet_espnet_fs2_pwg_demo/tn_g2p/parakeet/001.wav)
|
+
## 近期更新
-- 🔥 **2022.10.15：Release FastDeploy [release v0.3.0](https://github.com/PaddlePaddle/FastDeploy/tree/release%2F0.3.0)**
- - **New server-side deployment upgrade:更快的推理性能，一键量化，更多的视觉和NLP模型**
- - 集成 OpenVINO 推理引擎，并且保证了使用 OpenVINO 与 使用 TensorRT、ONNX Runtime、 Paddle Inference一致的开发体验；
- - 提供[一键模型量化工具](tools/quantization)，支持YOLOv7、YOLOv6、YOLOv5等视觉模型，在CPU和GPU推理速度可提升1.5～2倍；
- - 新增加 PP-OCRv3, PP-OCRv2, PP-Matting, PP-HumanMatting, ModNet 等视觉模型并提供[端到端部署示例](examples/vision)；
- - 新增加NLP信息抽取模型 UIE 并提供[端到端部署示例](examples/text/uie).
- -
+- 🔥 **【直播分享】2022.11.09 20:30～21:30，《覆盖云边端全场景，150+热门模型快速部署》。微信扫码报名**
+- 🔥 **【直播分享】2022.11.10 20:30～21:30，《瑞芯微、晶晨、恩智浦等10+AI硬件部署，直达产业落地》。微信扫码报名**
+- 🔥 **【直播分享】2022.11.10 19:00～20:00，《10+热门模型在RK3588、RK3568部署实战》。微信扫码报名**
+
+
+
- - **服务端部署全新升级：更快的推理性能，更多的视觉模型支持**
- - 发布基于x86 CPU、NVIDIA GPU的高性能推理引擎SDK，推理速度大幅提升
- - 集成Paddle Inference、ONNX Runtime、TensorRT等推理引擎并提供统一的部署体验
- - 支持YOLOv7、YOLOv6、YOLOv5、PP-YOLOE等全系列目标检测模型并提供[端到端部署示例](examples/vision/detection/)
- - 支持人脸检测、人脸识别、实时人像抠图、图像分割等40+重点模型及[Demo示例](examples/vision/)
- - 支持Python和C++两种语言部署
- - **边缘移动端部署新增瑞芯微、晶晨、恩智浦等NPU芯片部署能力**
- - 发布轻量化目标检测[Picodet-NPU部署Demo](https://github.com/PaddlePaddle/Paddle-Lite-Demo/tree/develop/object_detection/linux/picodet_detection)，提供极致的INT8全量化推理能力
+- 🔥 **2022.10.31：Release FastDeploy [release v0.5.0](https://github.com/PaddlePaddle/FastDeploy/tree/release/0.5.0)**
+ - **🖥️ 服务端部署：支持推理速度更快的后端，支持更多的模型**
+ - 集成 Paddle Inference TensorRT后端，并保证其使用与Paddle Inference、TensorRT、OpenVINO、ONNX Runtime、Paddle Lite等一致的开发体验；
+ - 支持并测试 Graphcore IPU 通过 Paddle Inference后端;
+ - 优化[一键模型量化工具](tools/quantization)，支持YOLOv7、YOLOv6、YOLOv5等视觉模型，在CPU和GPU推理速度可提升1.5～2倍；
+ - 新增 [PP-Tracking](./examples/vision/tracking/pptracking) 和 [RobustVideoMatting](./examples/vision/matting) 等模型；
+
+- 🔥 **2022.10.24：Release FastDeploy [release v0.4.0](https://github.com/PaddlePaddle/FastDeploy/tree/release/0.4.0)**
+ - **🖥️ 服务端部署：推理速度大升级**
+ - 升级 GPU 端到端的优化，在YOLO系列上，模型推理速度从 43ms 提升到 25ms；
+ - 新增 [TinyPose](examples/vision/keypointdetection/tiny_pose) and [PicoDetji lianTinyPose](examples/vision/keypointdetection/det_keypoint_unite)Pipeline部署能力；
+ - **📲 移动端和端侧部署：移动端后端能力升级，支持更多的CV模型**
+ - 集成 Paddle Lite，并保证其使用与服务端常用推理引擎 Paddle Inference、TensorRT、OpenVINO、ONNX Runtime 等一致的开发体验；
+ - 新增 [轻量化目标检测模型](examples/vision/detection/paddledetection/android)和[分类模型](examples/vision/classification/paddleclas/android)的安卓端部署能力；
+ - **
Web和小程序部署：新增Web端部署能力**
+ - 集成 Paddle.js部署能力，新增 OCR、目标检测、人像分割背景替换、物体识别等Web端部署能力和[Demo](examples/application/js)；
## 目录
-* **服务端部署**
+
+* 📖 文档教程（点击可收缩）
+
+ - 安装文档
+ - [预编译库下载安装](docs/cn/build_and_install/download_prebuilt_libraries.md)
+ - [GPU部署环境编译安装](docs/cn/build_and_install/gpu.md)
+ - [CPU部署环境编译安装](docs/cn/build_and_install/cpu.md)
+ - [IPU部署环境编译安装](docs/cn/build_and_install/ipu.md)
+ - [Jetson部署环境编译安装](docs/cn/build_and_install/jetson.md)
+ - [Android平台部署环境编译安装](docs/cn/build_and_install/android.md)
+ - 快速使用
+ - [Python部署示例](docs/cn/quick_start/models/python.md)
+ - [C++部署示例](docs/cn/quick_start/models/cpp.md)
+ - [Runtime Python使用示例](docs/cn/quick_start/runtime/python.md)
+ - [Runtime C++使用示例](docs/cn/quick_start/runtime/cpp.md)
+ - API文档(进行中)
+ - [Python API文档](https://www.paddlepaddle.org.cn/fastdeploy-api-doc/python/html/)
+ - [C++ API文档](https://www.paddlepaddle.org.cn/fastdeploy-api-doc/cpp/html/)
+ - 性能调优
+ - [量化加速](docs/cn/quantize.md)
+ - 常见问题
+ - [1. 如何配置模型部署的推理后端](docs/cn/faq/how_to_change_backend.md)
+ - [2. Windows上C++ SDK如何使用](docs/cn/faq/use_sdk_on_windows.md)
+ - [3. Android上如何使用FastDeploy](docs/cn/faq/use_sdk_on_android.md)(进行中)
+ - [4. TensorRT使用中的一些技巧](docs/cn/faq/tensorrt_tricks.md)
+ - [5. 如何增加新的模型](docs/cn/faq/develop_a_new_model.md)(进行中)
+ - 更多FastDeploy部署模块
+ - [服务化部署](../serving)
+ - [Benchmark测试](../benchmark)
+
+
+* **🖥️ 服务器端部署**
* [Python SDK快速开始](#fastdeploy-quick-start-python)
* [C++ SDK快速开始](#fastdeploy-quick-start-cpp)
* [服务端模型支持列表](#fastdeploy-server-models)
-* **端侧部署**
+* **📲 移动端和端侧部署**
* [Paddle Lite NPU部署](#fastdeploy-edge-sdk-npu)
- * [端侧模型支持列表](#fastdeploy-edge-sdk)
-* [社区交流](#fastdeploy-community)
-* [Acknowledge](#fastdeploy-acknowledge)
-* [License](#fastdeploy-license)
+ * [端侧模型支持列表](#fastdeploy-edge-models)
+* ** Web和小程序部署**
+ * [Web端模型支持列表](#fastdeploy-web-models)
+* [**社区交流**](#fastdeploy-community)
+* [**Acknowledge**](#fastdeploy-acknowledge)
+* [**License**](#fastdeploy-license)
-## 服务端部署
+## 🖥️ 服务端部署
-### Python SDK快速开始
+ Python SDK快速开始（点击可收缩）
+
#### 快速安装
##### 前置依赖
-- CUDA >= 11.2
-- cuDNN >= 8.0
-- python >= 3.6
+- CUDA >= 11.2、cuDNN >= 8.0、Python >= 3.6
- OS: Linux x86_64/macOS/Windows 10
##### 安装GPU版本
@@ -112,12 +151,15 @@ print(result)
vis_im = vision.vis_detection(im, result, score_threshold=0.5)
cv2.imwrite("vis_image.jpg", vis_im)
```
-
-C++ SDK快速开始
-
-### C++ SDK快速开始
+
+
+
+
+C++ SDK快速开始（点开查看详情）
+
+
#### 安装
- 参考[C++预编译库下载](docs/quick_start/CPP_prebuilt_libraries.md)文档
@@ -157,134 +199,172 @@ int main(int argc, char* argv[]) {
更多部署案例请参考[视觉模型部署示例](examples/vision) .
-### 服务端和web模型支持列表 🔥🔥🔥
-
-符号说明: (1) ✅: 已经支持; (2) ❔: 未来支持; (3) ❌: 暂不支持; (4) --: 暂不考虑;
+### 服务端模型支持列表 🔥🔥🔥🔥🔥
+
+符号说明: (1) ✅: 已经支持; (2) ❔: 正在进行中; (3) N/A: 暂不支持;
链接说明：「模型列」会跳转到模型推理Demo代码
+
+
+
RV1109
RV1126
RK1808 | 晶晨NPU
A311D
S905D
C308X | 恩智浦NPU
i.MX 8M Plus |更新中...｜
-| Classification | PP-LCNet | 11.9 | ✅ | ✅ | ✅ |-- | -- | -- |--|
-| Classification | PP-LCNetv2 | 26.6 | ✅ | ✅ | ✅ |-- | -- | -- |--|
-| Classification | EfficientNet | 31.4 | ✅ | ✅ | ✅ |-- | -- | -- |--|
-| Classification | GhostNet | 20.8 | ✅ | ✅ | ✅ |-- | -- | -- |--|
-| Classification | MobileNetV1 | 17 | ✅ | ✅ | ✅ |-- | -- | -- |--|
-| Classification | MobileNetV2 | 14.2 | ✅ | ✅ | ✅ |-- | -- | -- |--|
-| Classification | MobileNetV3 | 22 | ✅ | ✅ | ✅ |❔ | ❔ | ❔ |❔|
-| Classification | ShuffleNetV2 | 9.2 | ✅ | ✅ | ✅ |-- | -- | -- |--|
-| Classification | SqueezeNetV1.1 | 5 | ✅ | ✅ | ✅ |
-| Classification | Inceptionv3 | 95.5 | ✅ | ✅ | ✅ |-- | -- | -- |--|
-| Classification | PP-HGNet | 59 | ✅ | ✅ | ✅ |-- | -- | -- |--|
-| Classification | SwinTransformer_224_win7 | 352.7 | ✅ | ✅ | ✅ |-- | -- | -- |--|
-| Detection | PP-PicoDet_s_320_coco | 4.1 | ✅ | ✅ | ✅ |-- | -- | -- |--|
-| Detection | PP-PicoDet_s_320_lcnet | 4.9 | ✅ | ✅ | ✅ |✅ | ✅ | ✅ | ❔|
-| Detection | CenterNet | 4.8 | ✅ | ✅ | ✅ |-- | -- | -- |--|
-| Detection | YOLOv3_MobileNetV3 | 94.6 | ✅ | ✅ | ✅ |-- | -- | -- |--|
-| Detection | PP-YOLO_tiny_650e_coco | 4.4 | ✅ | ✅ | ✅ |-- | -- | -- |--|
-| Detection | SSD_MobileNetV1_300_120e_voc | 23.3 | ✅ | ✅ | ✅ |-- | -- | -- |--|
-| Detection | PP-YOLO_ResNet50vd | 188.5 | ✅ | ✅ | ✅ |-- | -- | -- |--|
-| Detection | PP-YOLOv2_ResNet50vd | 218.7 | ✅ | ✅ | ✅ |-- | -- | -- |--|
-| Detection | PP-YOLO_crn_l_300e_coco | 209.1 | ✅ | ✅ | ✅ |-- | -- | -- |--|
-| Detection | YOLOv5s | 29.3 | ✅ | ✅ | ✅ |-- | -- | -- |--|
-| Face Detection | BlazeFace | 1.5 | ✅ | ✅ | ✅ |-- | -- | -- |--|
-| Face Detection | RetinaFace | 1.7 | ✅ | ❌ | ❌ |-- | -- | -- |--|
-| Keypoint Detection | PP-TinyPose | 5.5 | ✅ | ✅ | ✅ |❔ | ❔ | ❔ |❔|
-| Segmentation | PP-LiteSeg(STDC1) | 32.2 | ✅ | ✅ | ✅ |-- | -- | -- |--|
-| Segmentation | PP-HumanSeg-Lite | 0.556 | ✅ | ✅ | ✅ |-- | -- | -- |--|
-| Segmentation | HRNet-w18 | 38.7 | ✅ | ✅ | ✅ |-- | -- | -- |--|
-| Segmentation | PP-HumanSeg-Server | 107.2 | ✅ | ✅ | ✅ |-- | -- | -- |--|
-| Segmentation | Unet | 53.7 | ❌ | ✅ | ❌ |-- | -- | -- |--|
-| OCR | PP-OCRv1 | 2.3+4.4 | ✅ | ✅ | ✅ |-- | -- | -- |--|
-| OCR | PP-OCRv2 | 2.3+4.4 | ✅ | ✅ | ✅ |-- | -- | -- |--|
-| OCR | PP-OCRv3 | 2.4+10.6 | ✅ | ✅ | ✅ |❔ | ❔ | ❔ |❔|
-| OCR | PP-OCRv3-tiny | 2.4+10.7 | ✅ | ✅ | ✅ |-- | -- | -- |--|
+
+
+
RK3568/RK3588 | 瑞芯微NPU
RV1109/RV1126/RK1808 | 晶晨NPU
A311D/S905D/C308X | 恩智浦NPU
i.MX 8M Plus | 更新中...｜ |
+| Classification | [PaddleClas/PP-LCNet](examples/vision/classification/paddleclas) | 11.9 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
+| Classification | [PaddleClas/PP-LCNetv2](examples/vision/classification/paddleclas) | 26.6 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
+| Classification | [PaddleClas/EfficientNet](examples/vision/classification/paddleclas) | 31.4 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
+| Classification | [PaddleClas/GhostNet](examples/vision/classification/paddleclas) | 20.8 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
+| Classification | [PaddleClas/MobileNetV1](examples/vision/classification/paddleclas) | 17 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
+| Classification | [PaddleClas/MobileNetV2](examples/vision/classification/paddleclas) | 14.2 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
+| Classification | [PaddleClas/MobileNetV3](examples/vision/classification/paddleclas) | 22 | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ | ❔ | -- |
+| Classification | [PaddleClas/ShuffleNetV2](examples/vision/classification/paddleclas) | 9.2 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
+| Classification | [PaddleClas/SqueezeNetV1.1](examples/vision/classification/paddleclas) | 5 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
+| Classification | [PaddleClas/Inceptionv3](examples/vision/classification/paddleclas) | 95.5 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
+| Classification | [PaddleClas/PP-HGNet](examples/vision/classification/paddleclas) | 59 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
+| Classification | [PaddleClas/SwinTransformer_224_win7](examples/vision/classification/paddleclas) | 352.7 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
+| Detection | [PaddleDetection/PP-PicoDet_s_320_coco](examples/vision/detection/paddledetection) | 4.1 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
+| Detection | [PaddleDetection/PP-PicoDet_s_320_lcnet](examples/vision/detection/paddledetection) | 4.9 | ✅ | ✅ | ❔ | ❔ | ✅ | ✅ | ✅ | -- |
+| Detection | [PaddleDetection/CenterNet](examples/vision/detection/paddledetection) | 4.8 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
+| Detection | [PaddleDetection/YOLOv3_MobileNetV3](examples/vision/detection/paddledetection) | 94.6 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
+| Detection | [PaddleDetection/PP-YOLO_tiny_650e_coco](examples/vision/detection/paddledetection) | 4.4 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
+| Detection | [PaddleDetection/SSD_MobileNetV1_300_120e_voc](examples/vision/detection/paddledetection) | 23.3 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
+| Detection | [PaddleDetection/PP-YOLO_ResNet50vd](examples/vision/detection/paddledetection) | 188.5 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
+| Detection | [PaddleDetection/PP-YOLOv2_ResNet50vd](examples/vision/detection/paddledetection) | 218.7 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
+| Detection | [PaddleDetection/PP-YOLO_crn_l_300e_coco](examples/vision/detection/paddledetection) | 209.1 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
+| Detection | YOLOv5s | 29.3 | ❔ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
+| Face Detection | BlazeFace | 1.5 | ❔ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
+| Face Detection | RetinaFace | 1.7 | ❔ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
+| Keypoint Detection | [PaddleDetection/PP-TinyPose](examples/vision/keypointdetection/tiny_pose) | 5.5 | ✅ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | -- |
+| Segmentation | [PaddleSeg/PP-LiteSeg(STDC1)](examples/vision/segmentation/paddleseg) | 32.2 | ✅ | ❔ | ❔ | ✅ | -- | -- | -- | -- |
+| Segmentation | [PaddleSeg/PP-HumanSeg-Lite](examples/vision/segmentation/paddleseg) | 0.556 | ✅ | ❔ | ❔ | ✅ | -- | -- | -- | -- |
+| Segmentation | [PaddleSeg/HRNet-w18](examples/vision/segmentation/paddleseg) | 38.7 | ✅ | ❔ | ❔ | ✅ | -- | -- | -- | -- |
+| Segmentation | [PaddleSeg/PP-HumanSeg](examples/vision/segmentation/paddleseg) | 107.2 | ✅ | ❔ | ❔ | ✅ | -- | -- | -- | -- |
+| Segmentation | [PaddleSeg/Unet](examples/vision/segmentation/paddleseg) | 53.7 | ✅ | ❔ | ❔ | ✅ | -- | -- | -- | -- |
+| Segmentation | [PaddleSeg/Deeplabv3](examples/vision/segmentation/paddleseg) | 150 | ❔ | ❔ | ❔ | ✅ | | | | |
+| OCR | PaddleOCR/PP-OCRv1 | 2.3+4.4 | ❔ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
+| OCR | [PaddleOCR/PP-OCRv2](examples/vision/ocr/PP-OCRv2) | 2.3+4.4 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
+| OCR | [PaddleOCR/PP-OCRv3](examples/vision/ocr/PP-OCRv3) | 2.4+10.6 | ✅ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | -- |
+| OCR | PaddleOCR/PP-OCRv3-tiny | 2.4+10.7 | ❔ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
+
+
+## Web和小程序部署
+
+
+
+| 任务场景 | 模型 | [web_demo](examples/application/js/web_demo) |
+|:------------------:|:-------------------------------------------------------------------------------------------:|:--------------------------------------------:|
+| --- | --- | [Paddle.js](examples/application/js) |
+| Detection | [FaceDetection](examples/application/js/web_demo/src/pages/cv/detection) | ✅ |
+| Detection | [ScrewDetection](examples/application/js/web_demo/src/pages/cv/detection) | ✅ |
+| Segmentation | [PaddleSeg/HumanSeg](./examples/application/js/web_demo/src/pages/cv/segmentation/HumanSeg) | ✅ |
+| Object Recognition | [GestureRecognition](examples/application/js/web_demo/src/pages/cv/recognition) | ✅ |
+| Object Recognition | [ItemIdentification](examples/application/js/web_demo/src/pages/cv/recognition) | ✅ |
+| OCR | [PaddleOCR/PP-OCRv3](./examples/application/js/web_demo/src/pages/cv/ocr) | ✅ |
+
+
## 社区交流
-
-
- **加入社区👬：** 微信扫描二维码，进入**FastDeploy技术交流群**
-
+
**input** ：Life was like a box of chocolates, you never know what you're gonna get.
**output**: [](https://paddlespeech.bj.bcebos.com/Parakeet/docs/demos/tacotron2_ljspeech_waveflow_samples_0.2/sentence_1.wav)
|
-| Potrait Segmentation | Image Matting | Semantic Segmentation | Real-Time Matting |
-|:----------------------------------------------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------------------------------------------:|
-| | | | |
-| **OCR** | **Behavior Recognition** | **Object Detection** | **Pose Estimation** |
-| | | | |
-| **Face Alignment** | **3D Object Detection** | **Face Editing** | **Image Animation** |
-| | | | |
## 📣 Recent Updates
+- 🔥 **【Live Preview】2022.11.09 20:30～21:30，《Covering the full spectrum of cloud-side scenarios with 150+ popular models for rapid deployment》**
+- 🔥 **【Live Preview】2022.11.10 20:30～21:30，《10+ AI hardware deployments from Rockchip, Amlogic, NXP and others, straight to industry landing》**
+- 🔥 **【Live Preview】2022.11.10 19:00～20:00，《10+ popular models deployed in RK3588, RK3568 in action》**
+ - Scan the QR code below using WeChat, follow the PaddlePaddle official account and fill out the questionnaire to join the WeChat group
+
+
+
- - **New deployment upgrade: Support support more backend, support more cv models**
- - Support Paddle Inference TensorRT, and provide a seamless deployment experience with other inference engines include TensorRT、OpenVINO、ONNX Runtime、Paddle Lite、Paddle Inference；
+ - **🖥️ Data Center and Cloud Deployment: Support more backend, Support more CV models**
+ - Support Paddle Inference TensorRT, and provide a seamless deployment experience with other inference engines include Paddle Inference、Paddle Lite、TensorRT、OpenVINO、ONNX Runtime；
- Support Graphcore IPU through paddle Inference;
- - Support tracking model [PP-Tracking](./examples/vision/tracking/pptracking) and [RobustVideoMatting](./examples/vision/matting) model
+ - Support tracking model [PP-Tracking](./examples/vision/tracking/pptracking) and [RobustVideoMatting](./examples/vision/matting) model；
+ - Support [one-click model quantization](tools/quantization) to improve model inference speed by 1.5 to 2 times on CPU & GPU platform. The supported quantized model are YOLOv7, YOLOv6, YOLOv5, etc.
- 🔥 **2022.10.24：Release FastDeploy [release v0.4.0](https://github.com/PaddlePaddle/FastDeploy/tree/release/0.4.0)**
- - **New server-side deployment upgrade: support more CV model and NLP model**
+ - **🖥️ Data Center and Cloud Deployment: end-to-end optimization, Support more CV and NLP model**
+ - end-to-end optimization on GPU, [YOLO series](examples/vision/detection) model end-to-end inference speedup from 43ms to 25ms;
+ - Support CV models include PP-OCRv3, PP-OCRv2, PP-TinyPose, PP-Matting, etc. and provides [end-to-end deployment demos](examples/vision/detection/);
+ - Support information extraction model is UIE, and provides [end-to-end deployment demos](examples/text/uie);
+ - Support [TinyPose](examples/vision/keypointdetection/tiny_pose) and [PicoDet and TinyPose](examples/vision/keypointdetection/det_keypoint_unite)Pipeline deployment.
+ - **📲 Mobile and Edge Device Deployment: support new backend，support more CV model**
- Integrate Paddle Lite and provide a seamless deployment experience with other inference engines include TensorRT、OpenVINO、ONNX Runtime、Paddle Inference；
- - Support [Lightweight Detection Model](examples/vision/detection/paddledetection/android) and [classification model](examples/vision/classification/paddleclas/android) on Android Platform，Download to try it out
- - end-to-end optimization on GPU, [YOLO series](examples/vision/detection) model end-to-end inference speedup from 43ms to 25ms;
- - Web deployment and Mini Program deployment New [OCR and other CV models](examples/application/js) capability；
- - Support [TinyPose](examples/vision/keypointdetection/tiny_pose) and [PicoDet+TinyPose](examples/vision/keypointdetection/det_keypoint_unite)Pipeline deployment；
- - New CV models include PP-OCRv3, PP-OCRv2, PP-TinyPose, PP-Matting, etc. and provides [end-to-end deployment demos](examples/vision/detection/)
- - New information extraction model is UIE, and provides [end-to-end deployment demos](examples/text/uie).
-
+ - Support [Lightweight Detection Model](examples/vision/detection/paddledetection/android) and [classification model](examples/vision/classification/paddleclas/android) on Android Platform，Download to try it out.
+ - **Web-Side Deployment: support more CV model**
+ - Web deployment and Mini Program deployment New [OCR and other CV models](examples/application/js) capability.
+
## Contents
-* **Data Center and Cloud Deployment**
+* 📖 Tutorials（click to shrink）
+
+ - Install
+ - [How to Install FastDeploy Prebuilt Libraries](docs/en/build_and_install/download_prebuilt_libraries.md)
+ - [How to Build and Install FastDeploy Library on GPU Platform](docs/en/build_and_install/gpu.md)
+ - [How to Build and Install FastDeploy Library on CPU Platform](docs/en/build_and_install/cpu.md)
+ - [How to Build and Install FastDeploy Library on IPU Platform](docs/en/build_and_install/ipu.md)
+ - [How to Build and Install FastDeploy Library on Nvidia Jetson Platform](docs/en/build_and_install/jetson.md)
+ - [How to Build and Install FastDeploy Library on Android Platform](docs/en/build_and_install/android.md)
+ - A Quick Start - Demos
+ - [Python Deployment Demo](docs/en/quick_start/models/python.md)
+ - [C++ Deployment Demo](docs/en/quick_start/models/cpp.md)
+ - [A Quick Start on Runtime Python](docs/en/quick_start/runtime/python.md)
+ - [A Quick Start on Runtime C++](docs/en/quick_start/runtime/cpp.md)
+ - API (To be continued)
+ - [Python API](https://baidu-paddle.github.io/fastdeploy-api/python/html/)
+ - [C++ API](https://baidu-paddle.github.io/fastdeploy-api/cpp/html/)
+ - Performance Optimization
+ - [Quantization Acceleration](docs/en/quantize.md)
+ - Frequent Q&As
+ - [1. How to Change Inference Backends](docs/en/faq/how_to_change_backend.md)
+ - [2. How to Use FastDeploy C++ SDK on Windows Platform](docs/en/faq/use_sdk_on_windows.md)
+ - [3. How to Use FastDeploy C++ SDK on Android Platform](docs/en/faq/use_sdk_on_android.md)(To be Continued)
+ - [4. Tricks of TensorRT](docs/en/faq/tensorrt_tricks.md)
+ - [5. How to Develop a New Model](docs/en/faq/develop_a_new_model.md)(To be Continued)
+ - More FastDeploy Deployment Module
+ - [deployment AI Model as a Service](./serving)
+ - [Benchmark Testing](./benchmark)
+
+
+* **🖥️ Data Center and Cloud Deployment**
* [A Quick Start for Python SDK](#fastdeploy-quick-start-python)
* [A Quick Start for C++ SDK](#fastdeploy-quick-start-cpp)
* [Supported Data Center and Cloud Model List](#fastdeploy-server-models)
-* **Mobile and Edge Device Deployment**
+* **📲 Mobile and Edge Device Deployment**
* [Paddle Lite NPU Deployment](#fastdeploy-edge-sdk-npu)
- * [Supported Mobile and Edge Model List](#fastdeploy-edge-sdk)
-* [Community](#fastdeploy-community)
-* [Acknowledge](#fastdeploy-acknowledge)
-* [License](#fastdeploy-license)
+ * [Supported Mobile and Edge Model List](#fastdeploy-edge-models)
+* **Web and Mini Program Deployment**
+ * [Supported Web and Mini Program Model List](#fastdeploy-web-models)
+* [**Community**](#fastdeploy-community)
+* [**Acknowledge**](#fastdeploy-acknowledge)
+* [**License**](#fastdeploy-license)
-## Data Center and Web Deployment
-
-### A Quick Start for Python SDK
+## 🖥️ Data Center and Cloud Deployment
+
+A Quick Start for Python SDK（click to shrink）
+
+
#### Installation
##### Prerequisites
-- CUDA >= 11.2
-- cuDNN >= 8.0
-- python >= 3.6
+- CUDA >= 11.2 、cuDNN >= 8.0 、 Python >= 3.6
- OS: Linux x86_64/macOS/Windows 10
##### Install Fastdeploy SDK with CPU&GPU support
@@ -121,9 +166,12 @@ print(result)
vis_im = vision.vis_detection(im, result, score_threshold=0.5)
cv2.imwrite("vis_image.jpg", vis_im)
```
+
+
+
-A Quick Start for C++ SDK
+A Quick Start for C++ SDK（click to expand）
#### Installation
@@ -165,124 +213,158 @@ int main(int argc, char* argv[]) {
For more deployment models, please refer to [Vision Model Deployment Examples](examples/vision) .
-
-### Supported Data Center and Web Model List🔥🔥🔥
-
-Notes: ✅: already supported; ❔: to be supported in the future; ❌: not supported now;
+### Supported Data Center and Web Model List🔥🔥🔥🔥🔥
+Notes: ✅: already supported; ❔: to be supported in the future; N/A: Not Available;
+
+
+
+
+
RV1109
RV1126
RK1808 | Amlogic-NPU
A311D
S905D
C308X | NXPNPU
i.MX 8M Plus | TBD...｜ |
-| Classification | PP-LCNet | 11.9 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
-| Classification | PP-LCNetv2 | 26.6 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
-| Classification | EfficientNet | 31.4 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
-| Classification | GhostNet | 20.8 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
-| Classification | MobileNetV1 | 17 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
-| Classification | MobileNetV2 | 14.2 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
-| Classification | MobileNetV3 | 22 | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
-| Classification | ShuffleNetV2 | 9.2 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
-| Classification | SqueezeNetV1.1 | 5 | ✅ | ✅ | ✅ | | | | |
-| Classification | Inceptionv3 | 95.5 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
-| Classification | PP-HGNet | 59 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
-| Classification | SwinTransformer_224_win7 | 352.7 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
-| Detection | PP-PicoDet_s_320_coco | 4.1 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
-| Detection | PP-PicoDet_s_320_lcnet | 4.9 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
-| Detection | CenterNet | 4.8 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
-| Detection | YOLOv3_MobileNetV3 | 94.6 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
-| Detection | PP-YOLO_tiny_650e_coco | 4.4 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
-| Detection | SSD_MobileNetV1_300_120e_voc | 23.3 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
-| Detection | PP-YOLO_ResNet50vd | 188.5 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
-| Detection | PP-YOLOv2_ResNet50vd | 218.7 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
-| Detection | PP-YOLO_crn_l_300e_coco | 209.1 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
-| Detection | YOLOv5s | 29.3 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
-| Face Detection | BlazeFace | 1.5 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
-| Face Detection | RetinaFace | 1.7 | ✅ | ❌ | ❌ | -- | -- | -- | -- |
-| Keypoint Detection | PP-TinyPose | 5.5 | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
-| Segmentation | PP-LiteSeg(STDC1) | 32.2 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
-| Segmentation | PP-HumanSeg-Lite | 0.556 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
-| Segmentation | HRNet-w18 | 38.7 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
-| Segmentation | PP-HumanSeg-Server | 107.2 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
-| Segmentation | Unet | 53.7 | ❌ | ✅ | ❌ | -- | -- | -- | -- |
-| OCR | PP-OCRv1 | 2.3+4.4 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
-| OCR | PP-OCRv2 | 2.3+4.4 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
-| OCR | PP-OCRv3 | 2.4+10.6 | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
-| OCR | PP-OCRv3-tiny | 2.4+10.7 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
+| Task | Model | Size (MB) | Linux | Android | iOS | Linux |Linux | Linux | Linux | TBD... |
+|:------------------:|:----------------------------:|:---------:|:-------:|:-------:|:-------:|:-----------------------------------------:|:---------------------------------------:|:------------------------:|:-------:|:-------:|
+| --- | --- | --- | ARM CPU | ARM CPU | ARM CPU |Rockchip-NPU
RK3568/RK3588 |Rockchip-NPU
RV1109/RV1126/RK1808 | Amlogic-NPU
A311D/S905D/C308X | NXP-NPU
i.MX 8M Plus | TBD...｜ |
+| Classification | [PaddleClas/PP-LCNet](examples/vision/classification/paddleclas) | 11.9 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
+| Classification | [PaddleClas/PP-LCNetv2](examples/vision/classification/paddleclas) | 26.6 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
+| Classification | [PaddleClas/EfficientNet](examples/vision/classification/paddleclas) | 31.4 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
+| Classification | [PaddleClas/GhostNet](examples/vision/classification/paddleclas) | 20.8 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
+| Classification | [PaddleClas/MobileNetV1](examples/vision/classification/paddleclas) | 17 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
+| Classification | [PaddleClas/MobileNetV2](examples/vision/classification/paddleclas) | 14.2 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
+| Classification | [PaddleClas/MobileNetV3](examples/vision/classification/paddleclas) | 22 | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ | ❔ | -- |
+| Classification | [PaddleClas/ShuffleNetV2](examples/vision/classification/paddleclas) | 9.2 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
+| Classification | [PaddleClas/SqueezeNetV1.1](examples/vision/classification/paddleclas) | 5 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
+| Classification | [PaddleClas/Inceptionv3](examples/vision/classification/paddleclas) | 95.5 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
+| Classification | [PaddleClas/PP-HGNet](examples/vision/classification/paddleclas) | 59 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
+| Classification | [PaddleClas/SwinTransformer_224_win7](examples/vision/classification/paddleclas) | 352.7 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
+| Detection | [PaddleDetection/PP-PicoDet_s_320_coco](examples/vision/detection/paddledetection) | 4.1 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
+| Detection | [PaddleDetection/PP-PicoDet_s_320_lcnet](examples/vision/detection/paddledetection) | 4.9 | ✅ | ✅ | ❔ | ❔ | ✅ | ✅ | ✅ | -- |
+| Detection | [PaddleDetection/CenterNet](examples/vision/detection/paddledetection) | 4.8 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
+| Detection | [PaddleDetection/YOLOv3_MobileNetV3](examples/vision/detection/paddledetection) | 94.6 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
+| Detection | [PaddleDetection/PP-YOLO_tiny_650e_coco](examples/vision/detection/paddledetection) | 4.4 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
+| Detection | [PaddleDetection/SSD_MobileNetV1_300_120e_voc](examples/vision/detection/paddledetection) | 23.3 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
+| Detection | [PaddleDetection/PP-YOLO_ResNet50vd](examples/vision/detection/paddledetection) | 188.5 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
+| Detection | [PaddleDetection/PP-YOLOv2_ResNet50vd](examples/vision/detection/paddledetection) | 218.7 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
+| Detection | [PaddleDetection/PP-YOLO_crn_l_300e_coco](examples/vision/detection/paddledetection) | 209.1 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
+| Detection | YOLOv5s | 29.3 | ❔ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
+| Face Detection | BlazeFace | 1.5 | ❔ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
+| Face Detection | RetinaFace | 1.7 | ❔ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
+| Keypoint Detection | [PaddleDetection/PP-TinyPose](examples/vision/keypointdetection/tiny_pose) | 5.5 | ✅ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | -- |
+| Segmentation | [PaddleSeg/PP-LiteSeg(STDC1)](examples/vision/segmentation/paddleseg) | 32.2 | ✅ | ❔ | ❔ | ✅ | -- | -- | -- | -- |
+| Segmentation | [PaddleSeg/PP-HumanSeg-Lite](examples/vision/segmentation/paddleseg) | 0.556 | ✅ | ❔ | ❔ | ✅ | -- | -- | -- | -- |
+| Segmentation | [PaddleSeg/HRNet-w18](examples/vision/segmentation/paddleseg) | 38.7 | ✅ | ❔ | ❔ | ✅ | -- | -- | -- | -- |
+| Segmentation | [PaddleSeg/PP-HumanSeg](examples/vision/segmentation/paddleseg) | 107.2 | ✅ | ❔ | ❔ | ✅ | -- | -- | -- | -- |
+| Segmentation | [PaddleSeg/Unet](examples/vision/segmentation/paddleseg) | 53.7 | ✅ | ❔ | ❔ | ✅ | -- | -- | -- | -- |
+| Segmentation | [PaddleSeg/Deeplabv3](examples/vision/segmentation/paddleseg) | 150 | ❔ | ❔ | ❔ | ✅ | | | | |
+| OCR | PaddleOCR/PP-OCRv1 | 2.3+4.4 | ❔ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
+| OCR | [PaddleOCR/PP-OCRv2](examples/vision/ocr/PP-OCRv2) | 2.3+4.4 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
+| OCR | [PaddleOCR/PP-OCRv3](examples/vision/ocr/PP-OCRv3) | 2.4+10.6 | ✅ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | -- |
+| OCR | PaddleOCR/PP-OCRv3-tiny | 2.4+10.7 | ❔ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
+
+## Web and Mini Program Deployment
+
+
+
+| Task | Model | [web_demo](examples/application/js/web_demo) |
+|:------------------:|:-------------------------------------------------------------------------------------------:|:--------------------------------------------:|
+| --- | --- | [Paddle.js](examples/application/js) |
+| Detection | [FaceDetection](examples/application/js/web_demo/src/pages/cv/detection) | ✅ |
+| Detection | [ScrewDetection](examples/application/js/web_demo/src/pages/cv/detection) | ✅ |
+| Segmentation | [PaddleSeg/HumanSeg](./examples/application/js/web_demo/src/pages/cv/segmentation/HumanSeg) | ✅ |
+| Object Recognition | [GestureRecognition](examples/application/js/web_demo/src/pages/cv/recognition) | ✅ |
+| Object Recognition | [ItemIdentification](examples/application/js/web_demo/src/pages/cv/recognition) | ✅ |
+| OCR | [PaddleOCR/PP-OCRv3](./examples/application/js/web_demo/src/pages/cv/ocr) | ✅ |
+
+
## Community
@@ -293,7 +375,7 @@ Notes: ✅: already supported; ❔: to be supported in the future; ❌: not supp
- **WeChat**：join our **WeChat community** and chat with other community members about ideas
-
+