apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2026-04-23 00:17:25 +08:00
Code Issues Actions 19 Packages Projects Releases Wiki Activity
4,718 Commits 41 Branches 17 Tags
02d32eea3b5688bd1eae4a2e92e84257ce4fbbef
Commit Graph

46 Commits

Author SHA1 Message Date
chen 5382fb2c60 [BugFix] lazy enable_torch_proxy for cutlass (#6523) 
* lazy enable_torch_proxy for cutlass

* test init_flash_attn_version
 2026-03-02 10:43:58 +08:00
chen 2d1531f3cb dev opensource model support fa4/flashmasV2/V3 (#6518) 2026-02-26 17:46:05 +08:00
chen d937d6ebfd check (#6424) 2026-02-10 15:55:17 +08:00
chen a8ffcaa068 fix fa4 test (#6408) 2026-02-10 10:57:21 +08:00
周周周 2b4748de4f [MTP] refactor MTP pre_process (#6358) 2026-02-09 10:47:15 +08:00
K11OntheBoat 116e2aea7a Support Norm before Rope (#6332) 
Co-authored-by: K11OntheBoat <“ruianmaidanglao@163.com”>
 2026-02-05 15:28:52 +08:00
chen 29a313a402 [Optimization] Support FA2/FA3/FA4 with attn_mask_q (#6354) 
* support FA4 sm100

* flash attn backend support mask

* flash attn backend run flashmask correct

* add test for flash_attn_backend and flash_attn_func

* check

* add test for fa4

* requirements.txt add fa4 whl

* check test on sm100

* fix CI conflict

* add enable_torch_proxy for flash_mask

* lazy import fa4

* check

* fix tests import

* check test_load_mpt import
 2026-02-05 14:39:00 +08:00
GoldPancake 183b8d325a [RL] Support GLM MTP RL Model (#6267) 2026-02-04 20:14:35 +08:00
bukejiyu 12d4b4cb87 [Feature]Support reorder ids to split prefill and decodes (#5779) 
* support reorder ids

* perfect code

* fix

* fix unittest

* delete code

* fix

* add python api

* delete custom op

* update algorithm

* fix swap

* support condense

* support condense

* support mtp

* delete code

* update

* update

* update

* update

* update for other platfrom

* update

* fix

* fix mtp

* fix ut

* update

* fix ut

* update ut

* fix

* fix encoder_cache

* fix ci

* fix

* fix vl

* Fix performance regression

* fix

* fix

* fix mtp

* fix index->req_id mapping

* fix ut

---------

Co-authored-by: root <root@yqlcc01-sys-rpm12rzmwjd.yqlcc01.baidu.com>
Co-authored-by: K11OntheBoat <“ruianmaidanglao@163.com”>
Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>
 2026-02-03 00:28:02 -08:00
Yuanle Liu 8b05774fad [Others] enhance deep_ep import and support mixed mode flash_mask_attn (#6238) 
* support flashmaskattn mixed and enhance deepep import

* update

* fix
 2026-01-28 00:02:02 +08:00
sunxin adc69c15d0 [Model Runner] Prepare token count and move FA3 initialization into the graph (#6170) 
* prepare for token num and put FA3 init in graph
 2026-01-26 12:16:57 +08:00
周周周 31410415db FA3 support qwen3 (#5441) 2025-12-09 16:16:16 +08:00
周周周 2aea8a3a60 [Others] Remove useless code (#5404) 2025-12-08 13:59:46 +08:00
lizhenyun01 d436640735 [BugFix] Fix flash_attn_backend 2025-12-05 14:33:38 +08:00
Yonghua Li 43097a512a [BugFix] [PD Disaggregation] fix v1 scheduler prefill node profile run & ipc transfer protocol (#5132)
CE Compile Job / ce_job_pre_check (push) Has been cancelled
Details
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
Details
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
Details
CE Compile Job / Show Code Archive Output (push) Has been cancelled
Details
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
Details
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
Details
CE Compile Job / CE_UPLOAD (push) Has been cancelled
Details
Deploy GitHub Pages / deploy (push) Has been cancelled
Details 
* [fix] fix v1 scheduler profile run for append attention in prefill node

* [fix] skip send_signal if kv signal not inited for gpu and xpu

* [fix] extend fix to flash_attn & mla_attn

* [fix] fix v1 pd run in ipc transfer protocol

* [ci] add test for v1 pd profile run using ipc transfer protocol

* [style] fix code style check

* [style] fix code style again

* [fix] fix profile run

* [update] remove --num-gpu-blocks-override in example script

* [chore] rename forward_meta is_profiling to is_dummy_or_profile_run
 2025-11-20 21:39:22 +08:00
周周周 6fa34102e8 [Others]get_block_shape_and_split_kv_block clean code (#5123) 2025-11-20 16:40:04 +08:00
ltd0924 5bf48de999 [KVCache] support unified cache backend (#4903) 
* [Feature] support unified cache backend

* fix

* fix

* fix

* fix

* Update metax_model_runner.py

* fix

* update

* Update test_moba_attention_backend.py

---------

Co-authored-by: ltd0924 <luotingdan@baidu.com>
 2025-11-12 14:54:52 +08:00
Haonan Luo 1b9f351d21 Support GPT-OSS-BF16 (#4240) 
* [Feature] AppendAtten support sinks & HEAD_DIM=64

* fix bug

* fix bug

* fix bug

* fix bug

* [Feature] support gpt-oss

* fix bug

* add mask

* support-gpt-oss

* support-gpt-oss

* fix long seq

* support wint8

* support wint8

* support wint8

* update test

* change sliding windows init pos

---------

Co-authored-by: ming1753 <ideaminghp@163.com>
Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>
Co-authored-by: ming1753 <61511741+ming1753@users.noreply.github.com>
 2025-10-20 14:44:58 +08:00
Sunny-bot1 a751d977bc [Optimization] Fuse get_max_len and get_kv_max_len (#4369)
CE Compile Job / ce_job_pre_check (push) Has been cancelled
Details
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
Details
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
Details
CE Compile Job / Show Code Archive Output (push) Has been cancelled
Details
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
Details
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
Details
CE Compile Job / CE_UPLOAD (push) Has been cancelled
Details
Deploy GitHub Pages / deploy (push) Has been cancelled
Details 
* opt split_q_block

* fuse max_lens and max kv_len
 2025-10-13 20:35:00 +08:00
YuanRisheng a2ec2c4152 [FDConfig]Remove max_model_len in FDConfig (#4350) 
* modify max_model_len

* fix unittest

* fix unittest

---------

Co-authored-by: root <root@yqlcc01-sys-rpm12rzmwjd.yqlcc01.baidu.com>
 2025-10-11 14:04:17 +08:00
YuanRisheng 2e9e53ff7e [FDConfig]Remove max_num_batched_tokens/max_num_seqs in parallel config (#4116) 
* remove max_num_batched_tokens in parallel config

* remove max_num_seqs

* update test case

* fix test

* fix

---------

Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>
 2025-09-17 10:43:35 +08:00
AIbin a7392a0ff9 【Inference Optimize】DeepSeek-V3-model MLA Optimize (#3886) 
* support MLA chunk_size auto search & cuda_graph
 2025-09-11 10:46:09 +08:00
Jundong Liu 3d0aaa5923 [Excutor] Experiment Feature-Support Prefill in cudagraph (#3459) 
* Support prefill in Cudagraph

* Refactor GetBlockShapeAndSplitKVBlock Kernel V2

* Refactor GetBlockShapeAndSplitKVBlock Kernel V2.1

* Refactor GetBlockShapeAndSplitKVBlock Kernel V2.2

* Refactor GetBlockShapeAndSplitKVBlock Kernel V2.3

* Refactor GetBlockShapeAndSplitKVBlock Kernel V2.4

* Refactor GetBlockShapeAndSplitKVBlock Kernel V2.5

* Solve problem about encoder_num_blocks_x_cpu

* Add early-exit mechanism for attention kernel

* fix test case about append-attention

* Update testcode, Add annotations to related tensors

* move get_input_length_list

* solve test_code

* Add annotations about early-exit for attention kernel

* Add annotations about early-exit for attention kernel2

* solve comment

* solve mtp

---------

Co-authored-by: RAM <gstian5555@outlook.com>
 2025-09-08 13:12:24 +08:00
yangjianfengo1 8e1b35a09b 【Fix bug] w4afp8 的nblock固定为256，并且fa3的append attn 增加mask参数 (#3771) 
* fix w4afp8

* 增加集中式配置

* codestyle

* fix fa3 append attn
 2025-09-02 19:17:01 +08:00
co63oc d6369b4d51 fix typos (#3684) 2025-09-01 17:50:17 +08:00
Liumengyuan e93d4cfcdd Add with_output version AppendAttention (#3302) 
* get use_output from fd_config

* add clear TODO description

* add mask_offset para to align with develop

* fix bug

* fix use_output logic

* fix sot bug
 2025-08-28 17:10:18 +08:00
xiaoxiaohehe001 ad319a87cc support fa3 rope3d (#3622) 2025-08-27 11:31:29 +08:00
yangjianfengo1 3a15e0c53e 【Fix Bug】 修复 fa3 支持集中式bug (#3235)
Deploy GitHub Pages / deploy (push) Has been cancelled
Details 
* fix fa3 集中式bug

* 增加qknorm参数
 2025-08-06 16:24:27 +08:00
yangjianfengo1 64d7a3194d 集中式支持fa3 (#3112) 2025-08-01 18:03:36 +08:00
RAM d850660872 [Executor] Refactor GetBlockShapeAndSplitKVBlock Kernel (#2989) 
* reset decoder_block_shape_q buffer

* refactor GetBlockShapeAndSplitKVBlock Kernel and cudagraph padding batch

* update decode_max_tile_size

* fix pre-commit

* update block_multihead_attn_backend

* update flas attn backend

* update MLA Attention

* update XPU Attention

* update gcu,iluvatar model runner

* Update MTP

* fix MTP bug
 2025-07-31 00:09:31 +08:00
YuanRisheng 6ccc10ad47 Unify server-side and model-side Config (Part1) (#3018) 
* move cache config

* fix mtp
 2025-07-28 10:51:52 +08:00
chen 332154f504 [feature] Support FA2 (#3009) 2025-07-25 14:09:00 +08:00
lizhenyun01 29c3292f02 support c4 attn && fix cache 2025-07-24 12:00:52 +08:00
chen 172e69fe17 FA3 fix bug (#2987) 2025-07-23 19:07:43 +08:00
lizhenyun01 e51f018577 support chunk_prefill in fa3 2025-07-23 12:19:20 +08:00
Nyakku Shigure 48e6a0ca26 [SOT] Mark dynamic dims by type annotations (#2771)
Deploy GitHub Pages / deploy (push) Has been cancelled
Details 
* [SOT] Mark dynamic dims by type annotations

* fix conflict of forward_meta

* mark more attn backend

* fix missing annotated and add env SOT_SPECIALIZED_DIM_NUMBERS

* auto infer implicit 0 dim dynamic dim

* revert manual marked dims

* revert missing update

* auto infer can use unsafe code in warmup stage

* check -> type_match

* fix codestyle

* restore blank line

* empty commit

* add need_warmup nonlocal;

* add doc for resolver

* add missing type hints

* unquote "ForwardMeta"
 2025-07-22 00:23:52 -07:00
Zero Rains 25698d56d1 polish code with new pre-commit rule (#2923) 2025-07-19 23:19:27 +08:00
周周周 d306944f4f remove cum_offsets from get_block_shape_and_split_kv_block (#2913) 
* remove padding_offsets from get_padding_offset.cu

* remove padding_offsets from get_padding_offset.cu

* remove padding_offsets from get_padding_offset.cu

* remove cum_offsets from get_block_shape_and_split_kv_block

* remove cum_offsets from get_block_shape_and_split_kv_block
 2025-07-18 16:13:32 +08:00
freeliuzc d49f8fb30a [Feature][MTP] Support cacheKV transfer in per_chunk mode (#2890) 
* support chunk_prefill both normal and speculative_decoding(mtp)

* optimize pd-disaggregation config

* fix bug
 2025-07-17 17:58:08 +08:00
YuanRisheng 4c7b8bc458 Simplify the Config code (#2770) 
* simplify the code

* fix vl

* delete config

* fix

* perfect code

* fix ci

* fix xpu

* fix xpu

* fix server

* resolve conflict

* fix mtp

* resolve conflict

* fix xpu

* fix xpu

* fix vl

* fix log

* fix qwen moe

* fix qwen moe

* fix qwen moe
 2025-07-14 19:50:05 +08:00
littledgg 59071268b6 [Executor] Move forward_meta.py to fastdeploy/model_executor (#2774) 
* Use PEP 563 in attention.py and fix conflict

* merge commit

* Change what was left out last time
 2025-07-10 20:36:51 +08:00
RichardWooSJTU fee544e808 fix ep prefill (#2762) 2025-07-09 14:03:05 +08:00
RichardWooSJTU 6610aa29d0 Revert "[Bug fix] fix attention rank init (#2743)" (#2761) 
This reverts commit e8bbe7244b.
 2025-07-09 10:38:12 +08:00
RichardWooSJTU e8bbe7244b [Bug fix] fix attention rank init (#2743) 
* fix attention rank init

* fix attention rank init
 2025-07-08 17:19:49 +08:00
gaoziyuan 26d5d737dd 【Fearture】support qwen2 some func (#2740) 
* add rl qwen model support

* fix

* fix
 2025-07-08 12:03:04 +08:00
Yuanle Liu 240bdac2a4 [feat] support fa3 backend for pd disaggregated (#2695)
Deploy GitHub Pages / deploy (push) Has been cancelled
Details 
* support fa3 backend run in pd disaggregated

* support fa3 backend run in pd disaggregated

* support fa3 backend run in pd disaggregated

* support fa3 backend run in pd disaggregated

* delete use_fast_ffn
 2025-07-03 22:33:27 +08:00