apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2026-04-23 00:17:25 +08:00
Code Issues Actions 19 Packages Projects Releases Wiki Activity

Modified to support custom all reduce by default (#3538)

Browse Source
This commit is contained in:
zhink
2025-08-22 16:59:05 +08:00
committed by GitHub
parent 27666ee586
commit df7c31012b
15 changed files with 18 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/zh/best_practices/ERNIE-4.5-0.3B-Paddle.md
+1 -2
View File
@@ -77,8 +77,7 @@ CUDAGraph 是 NVIDIA 提供的一项 GPU 计算加速技术，通过将 CUDA 操
```
注：
1. 通常情况下不需要额外设置其他参数，但CUDAGraph会产生一些额外的显存开销，在一些显存受限的场景下可能需要调整。详细的参数调整请参考[GraphOptimizationBackend](../features/graph_optimization.md) 相关配置参数说明
2. 开启CUDAGraph时，如果是TP>1的多卡推理场景，需要同时指定 `--enable-custom-all-reduce`
3. 开启CUDAGraph时，暂时不支持`max-model-len > 32768`的场景。
2. 开启CUDAGraph时，暂时不支持`max-model-len > 32768`的场景。
#### 2.2.5 拒绝采样
**原理：**