apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2026-05-07 16:08:58 +08:00
Code Issues Actions 4 Packages Projects Releases Wiki Activity
4,676 Commits 41 Branches 17 Tags
54f7d9f62128cf8317d170eb87605914f5d059de
Commit Graph

185 Commits

Author SHA1 Message Date
周周周 1503443871 add dsv3 mixed deploy as EP16 TP8 (#6525) 2026-02-27 14:08:25 +08:00
sunxin 53aaac69da [Optimization] Enable BF16 gate computation for GLM and Qwen (#6457) 
* gate bf16

* add gate-fp32

* fix

* update baseline

* update

* update

* fix
 2026-02-26 21:08:46 -08:00
jackyYang6 38c3e02470 fix paddleformers fallback (#6465) 2026-02-23 15:29:13 +08:00
bukejiyu dc5917289d [loader]supoort wint2 backend (#6139) 
* support wint2

* update
 2026-02-08 22:42:36 -08:00
chen 72fe94cb13 [Feature] support glm tp+dp+ep (#6317) 2026-02-05 21:47:01 +08:00
GoldPancake 183b8d325a [RL] Support GLM MTP RL Model (#6267) 2026-02-04 20:14:35 +08:00
GoldPancake fb374238e1 Revert "[RL] Support GLM MTP RL Model (#6223)" (#6301) 
This reverts commit af6c84d48d.
 2026-02-02 14:08:13 +08:00
GoldPancake af6c84d48d [RL] Support GLM MTP RL Model (#6223) 
* support glm mtp rl model

* fix

* fix

* fix ut

* update baseline
 2026-01-28 08:28:03 -08:00
ddchenhao66 6d33d5e370 [Models][BugFix] shared experts and dense mlp layer do not require TP split (#6180) 
Co-authored-by: ddchenhao66 <dhaochen163.com>
 2026-01-28 18:58:19 +08:00
Haonan Luo 82057cb71f Support MXFP4 for GPT-OSS (#5435) 
* support mxfp4 in gpt-oss

* support mxfp4 in gpt-oss

* add scope for flashinfer

* remove torch code

* update envs.FD_MXFP4_BACKEND

* update process_weights_after_loading

* update env name

* support tp in gpt-oss, add e2e test

* add flashinfer-python-paddle in requirements

* fix import error

* add test

* add test

* add test

* add test
 2026-01-22 14:21:01 +08:00
jackyYang6 988e0bc338 [Feature] Add PaddleFormers fallback backend (#5999) 
* feat(paddleformers): add dense text model fallback backend

* docs(paddleformers): add user guide and fix code review issues

* add fallback unit test

* precommit format

* fix pre-commit

* fix: address code review feedback

* docs: add PaddleFormers backend documentation (EN) and simplify installation

---------

Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>
Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>
 2026-01-19 21:50:50 +08:00
GoldPancake 879e45f6b3 fix compute logits problem (#6093) 2026-01-19 20:12:14 +08:00
sunxin 9dc1c74d36 fix opt qknorm (#6080) 2026-01-19 12:07:20 +08:00
GoldPancake bda38aa519 [Speculative Decoding] Support MTP for GLM-4.5-Air (#6047) 
* glm mtp
* add spec neox partial rope
 2026-01-16 14:35:24 +08:00
Cheng Yanfei fbcccaa750 [Intel HPU] enable MoE EP for hpu (#5855) 
* enable HPU MoE EP

* MoE intermediate_scale stack

* enable loader_v1 esp for tensor_wise_fp8 TP or EP

* modify activation_scale name
 2026-01-15 13:08:00 +08:00
xiaoxiaohehe001 6f72be7c3e [Optimize] Qwen2.5-VL vision model with merged linear layers and unif… (#6037) 
* [Optimize] Qwen2.5-VL vision model with merged linear layers and unified normalization

* [Optimize] Qwen2.5-VL vision model with merged linear layers and unified normalization
 2026-01-14 19:21:31 +08:00
sunxin 2533836dbb [Optimization] Accelerate Qwen3 QK RMSNorm via Fused Triton Kernel (#5880) 
* qk rmsnorm fused

* inplace

* glm

* fix

* add qknorm layer

* fix

* update

* fix qwen3 vl

* update rl baseline

* fix qwen3 vl moe

* test

* fix qwen vl moe rl

* fix
 2026-01-12 05:10:21 -08:00
xiaoxiaohehe001 00a01ae024 [Feature] Support redundant expert for eplb (#5918) 
* [BugFix] support redundant expert for eplb

* support redundant expert for eplb

* support redundant expert for eplb

* update

* fix ci eplb
 2026-01-09 17:13:24 +08:00
CSWYF3634076 e6cdea4492 [Models] Qwen3VL and Qwen3VL-Moe CUDA graph Support (#5962) 
* [Models] add Qwen3VL and Qwen3VL-Moe CUDA graph support

* [Models] add Qwen3VL and Qwen3VL-Moe CUDA graph support v2

* [Models] add Qwen3VL and Qwen3VL-Moe CUDA graph support v3
 2026-01-09 17:09:02 +08:00
Yuanle Liu d4a386dfc4 Revert "Revert "[TSP] last_norm allgather move to model.py (#5924)" (#5961)" (#5972) 
This reverts commit 8c3513a410.
 2026-01-09 15:58:22 +08:00
Yuanle Liu 8c3513a410 Revert "[TSP] last_norm allgather move to model.py (#5924)" (#5961) 
This reverts commit 2bb838fed9.
 2026-01-09 15:20:40 +08:00
xiaoluomi 2bb838fed9 [TSP] last_norm allgather move to model.py (#5924) 
* support_lastnorm_gather_split_dev

* support_lastnorm_gather_split_dev1

* support_lastnorm_gather_split_dev3

* support_lastnorm_gather_split_dev4

* support_lastnorm_gather_split_dev5
 2026-01-07 23:36:33 -08:00
CSWYF3634076 d8fcb7c07d [Models] Add Qwen3-VL Moe Model Support (#5913) 
* [Model] add Qwen3vl moe model support

* [Model] add Qwen3vl moe model support remove log

* [Model] add Qwen3vl moe model support unittest
 2026-01-08 11:36:42 +08:00
周周周 83ae59431e [BugFix] fix BatchMLAWithPagedKVCacheKernel O_tmp (#5895) 2026-01-06 15:39:06 +08:00
lizexu123 44a13e4557 [Feature] support w4afp8 v1_loader and v0_loader(tp>1) (#5757) 
* support

* fix

* support w4afp8 v1_loader and v0_loader

* fix

* fix test

* fix test

* fix test

* fix moe.py

* add test_ernie_4_5_w4afp8

* add test

* delete tensor

* fix test

* fix

* add

* fix test
 2025-12-30 14:11:52 +08:00
CSWYF3634076 9286403570 [Models] Add Qwen3-VL Model Support (#5763) 
* support v1 loader

* remove useless code

* remove useless

* [Model] support Qwen3VL images success

* [Model] support Qwen3VL rope_3d

* [Model] support Qwen3VL remove log

* [Model] support Qwen3VL RL

* [Model] support Qwen3VL tp

* [Model] support Qwen3VL video

* [Model] support Qwen3VL fix ernievl

* [Model] support Qwen3VL fix get_image_boundaries.cc array out of bounds

* [Model] support Qwen3VL fix multi card

* [Model] support Qwen3VL file close

* [Model] support Qwen3VL fix ce

* [Model] support Qwen3VL fix unittest

* [Model] support Qwen3VL add unittest

---------

Co-authored-by: Ayakouji <yuhongh@qq.com>
 2025-12-29 17:39:33 +08:00
Nyakku Shigure da9ea88a3b [BugFix] Correct condition for reversed_window_indices in SiglipEncoder (#5795) 2025-12-26 19:16:07 +08:00
bukejiyu f0bbdce849 [Loader]Fix bug in MTP weight loading (#5744) 
* fix torch mtp

* fix

* update
 2025-12-25 11:32:17 +08:00
bukejiyu ba4b7afb3a [Others] Rename tensor_parallel_degree to tensor_model_parallel_size for paddleformers 0.4.1 (#5727) 2025-12-23 23:19:11 -08:00
bukejiyu d1c6e57341 [Others] upgrade paddleformer to 0.4.0 (#5599) 2025-12-23 05:08:01 -08:00
yzwu ac013803f3 [Iluvatar] Support V1_KVCACHE_SCHEDULER and paddleocr-vl rope mode (#5555) 2025-12-18 02:14:25 -08:00
fmiao2372 404cf0ece4 [Intel HPU] enable tensor_wise_fp8 (#5324) 
* [Intel HPU] enable tensor_wise_fp8

* update code based on comments

* fix code style issue

* fix bug about RP 5138

* mv kv_cache modifications to HPU backend

* fix FP8 Precision Issues

* fix FP8 Precision Issues

* Add quantization UT

---------

Co-authored-by: yanfeich <yanfei.cheng@intel.com>
Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>
 2025-12-17 16:45:03 +08:00
Ryan 4eb55332f6 [Models] Add forward_meta to VocabParallelEmbedding of all models (#5524) 2025-12-12 14:11:31 +08:00
kevin 954a145d57 [Optimization] support mm prefill batch (#5313)
CE Compile Job / ce_job_pre_check (push) Has been cancelled
Details
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
Details
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
Details
CE Compile Job / Show Code Archive Output (push) Has been cancelled
Details
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
Details
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
Details
CE Compile Job / CE_UPLOAD (push) Has been cancelled
Details
Deploy GitHub Pages / deploy (push) Has been cancelled
Details 
* support mm prefill batch

* update code

* update code

* update code

* update code

* fix encoder cache bug

* update code

* update code

* fix bug

* fix paddle ocr bug

* fix xpu bug

* update code
 2025-12-11 22:21:14 +08:00
Ryan e58fed3665 [Graph Optimization][BugFix][CI] Fix 0size bug && add unitest (#5495) 2025-12-11 16:25:26 +08:00
Haonan Luo e397c4fba6 [Others] remove add_bias option (#5425) 2025-12-09 17:39:35 +08:00
Nyakku Shigure e1c4a12e34 [Graph Optimization][CINN] Use CINN in PaddleOCR-VL ViT part (#5223) 
---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-12-09 14:37:00 +08:00
xiaozude c06a6234b9 [Metax] optimize mla attention (#5258) 2025-12-09 11:18:19 +08:00
Nyakku Shigure 0c66163dfd [Loader][BugFix] Fix some parameters place on CPU in PaddleOCR-VL (#5413) 
* [BugFix] Fix some parameter place on CPU in PaddleOCR-VL

* clean log

* fix codestyle
 2025-12-08 10:01:00 +08:00
RAM b2908b8e82 [New][RL] Support Rollout Routing Replay (#5405) 
* [RL] Support Rollout Routing Replay

* add routing indices cache

* fix config bug and moe forward bug

* R3 Support GLM

* support eb4.5

* fix merge bug

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* add routing replay ci

* support glm topk

* support orther top_k

* fix ci bug

* pre-commit

* only support chatcmpl

* Revert "Revert "[RL] Support Rollout Routing Replay (#5321)" (#5402)"

This reverts commit c45e064f3d.

* Fix XPU and NPU bug

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Yuanle Liu <yuanlehome@163.com>
 2025-12-05 22:06:26 +08:00
Jiang-Jia-Jun c45e064f3d Revert "[RL] Support Rollout Routing Replay (#5321)" (#5402) 
This reverts commit 96d2d4877b.
 2025-12-05 20:19:39 +08:00
RAM 96d2d4877b [RL] Support Rollout Routing Replay (#5321) 
* [RL] Support Rollout Routing Replay

* add routing indices cache

* fix config bug and moe forward bug

* R3 Support GLM

* support eb4.5

* fix merge bug

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* add routing replay ci

* support glm topk

* support orther top_k

* fix ci bug

* pre-commit

* only support chatcmpl

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Yuanle Liu <yuanlehome@163.com>
 2025-12-05 20:01:33 +08:00
Longzhi Wang 5cd17fd662 [Models] Add forward_meta to moe models' forward function (#5138) 
* [Models] Add forward_meta to moe models' forward function

* fix missing param

* fix

* fix

* fix forward_meta

* fix test and remove chunked MoE releated in config

* fix test

* fix

* fix
 2025-12-04 13:26:58 +08:00
lizexu123 c563eca791 [Feature] support reward model (#5301) 
* Your commit message here

* add test

* update develop

* support reward

* support enable_chunk_prefill

* support bingfa

* support convert is reward

* update test

* delete print

* fix enable_thinking

* add document

* fix place

* fix test

* fix

* support enable_prefix_caching

* add no-enable_prefix-caching test

* fix

* support enable_prefix_caching

* delete print

* fix document

* fix

* fix test

* fix document and delete chinese

* udpate

* enable_thinking

* fix test
 2025-12-02 14:55:31 +08:00
K11OntheBoat 2e1680838f [PD Disaggregation] Support PD deployment of DeepSeekv3. (#5251) 
* Support deepseekv3 cache transfer for PD deploy

* clean some log info

---------

Co-authored-by: K11OntheBoat <“ruianmaidanglao@163.com”>
 2025-12-02 14:11:50 +08:00
chen aa35ce449d [Optimization] EP empty_input_forward Remove Communication (#5254) 2025-12-01 21:10:40 +08:00
bukejiyu 1539fd6056 [BugFix]Set default OMP_NUM_THREADS=3 and fix extra GPU memory usage in DeepSeek (#5219) 
* fix bug

* update

* update

* update

* fix copy

* update
 2025-11-28 14:22:04 +08:00
chen 35f85baf09 [BugFix]fix v1 loader lm head fp32 (#5270) 2025-11-27 20:12:56 +08:00
Yuanle Liu cb56d46694 [Optimization] Refine row parallel bias and nranks and moe all_reduce (#5247)
CE Compile Job / ce_job_pre_check (push) Has been cancelled
Details
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
Details
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
Details
CE Compile Job / Show Code Archive Output (push) Has been cancelled
Details
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
Details
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
Details
CE Compile Job / CE_UPLOAD (push) Has been cancelled
Details
Deploy GitHub Pages / deploy (push) Has been cancelled
Details
Publish Job / publish_pre_check (push) Has been cancelled
Details
Publish Job / print_publish_pre_check_outputs (push) Has been cancelled
Details
Publish Job / FD-Clone-Linux (push) Has been cancelled
Details
Publish Job / Show Code Archive Output (push) Has been cancelled
Details
Publish Job / BUILD_SM8090 (push) Has been cancelled
Details
Publish Job / BUILD_SM8689 (push) Has been cancelled
Details
Publish Job / PADDLE_PYPI_UPLOAD_8090 (push) Has been cancelled
Details
Publish Job / PADDLE_PYPI_UPLOAD_8689 (push) Has been cancelled
Details
Publish Job / Run FD Image Build (push) Has been cancelled
Details
Publish Job / Run FastDeploy Unit Tests and Coverage (push) Has been cancelled
Details
Publish Job / Run FastDeploy LogProb Tests (push) Has been cancelled
Details
Publish Job / Extracted partial CE model tasks to run in CI. (push) Has been cancelled
Details
Publish Job / Run Base Tests (push) Has been cancelled
Details
Publish Job / Run Accuracy Tests (push) Has been cancelled
Details
Publish Job / Run Stable Tests (push) Has been cancelled
Details
CI Images Build / FD-Clone-Linux (push) Has been cancelled
Details
CI Images Build / Show Code Archive Output (push) Has been cancelled
Details
CI Images Build / CI Images Build (push) Has been cancelled
Details
CI Images Build / BUILD_SM8090 (push) Has been cancelled
Details
CI Images Build / Run FastDeploy Unit Tests and Coverage (push) Has been cancelled
Details
CI Images Build / Run FastDeploy LogProb Tests (push) Has been cancelled
Details
CI Images Build / Extracted partial CE model tasks to run in CI. (push) Has been cancelled
Details
CI Images Build / Run Base Tests (push) Has been cancelled
Details
CI Images Build / Publish Docker Images Pre Check (push) Has been cancelled
Details 
* rename nranks to tp_size and fix bias in v1 loader

* fix

* update
 2025-11-26 05:09:09 -08:00
chen 209970836e [BugFix] BF16 MoE Cutlass Backend Support EP (#5242) 2025-11-26 19:16:22 +08:00