57 Commits

Author	SHA1	Message	Date
Jingfeng Wu	3b564116d5	[Docs] Add docs for disaggregated deployment (#6700) ... * add docs for disaggregated deployment * pre-commit run for style check * update docs	2026-04-01 19:27:09 +08:00
jackyYang6	05f2d95729	[RL] Adapt async rollout checkpoint update flow (#7042) ... * update checkpoint-transfer flow and control update_weights params * test: add update_weights route validation	2026-03-30 19:19:34 +08:00
Yonghua Li	a7f52c300d	[Feature] support v1 update/clear api for RL (#6761) ... * [Feature] support v1 update/clear api for RL * [fix] fix execute_model and add sleep/wakeup api * [fix] fix mtp and key_prefix * [chore] move _update_key_prefix to resume method * [fix] make the interface safe to call multiple times * [fix] fix some tiny bugs * [chore] make small changes against pr review * [docs] add docs for weight update * [test] add some tests and update docs * [style] fix code style check * [test] fix ci * [fix] fix stale control responses when control method timed out * [chore] remove unused code * [chore] fix code style * [chore] optimize tags and key_prefix * [test] fix ci * [chore] fix code style * [test] fix ci * [fix] fix ep control * [fix] fix ep control for engine cache queue	2026-03-25 19:18:46 +08:00
jackyYang6	634d23a38a	[Bugfix] Align thinking_budget behavior with ERNIE reasoning flow (#6934) ... * [Bugfix] Align thinking_budget behavior with ERNIE reasoning flow * [Docs] Fix thinking_budget markdown formatting * [Test] Align ernie thinking budget test with process_request_dict	2026-03-23 14:15:55 +08:00
sunxin	33e01f22a8	[Feature][Sampling] Extend top-k_top-p sampling to all backends and unify greedy decoding with top_k=1 (#6894) ... * update sampling * fix * fix * fix mtp * fix test	2026-03-19 01:43:10 -07:00
jc	04fde3b227	[PD Disaggregation] Prefill and decode support cache storage (#6768) ... * Prefill and decode support cache storage * up * up * update docs and refine mooncake store * up	2026-03-16 14:44:49 +08:00
freeliuzc	cf7934a4b2	[Speculative Decoding] Unify Spec and non-spec branch (#6685) ... * optimize spec-inference architecture * delete debug log * optimize spec_method usage && fix unit_test * add claude unit-test skill * fix some ugly bug * enhance robustness and bounds check * unify method & spec_method to method to avoid bug * activate CI * fix unit test * Unify logprobs computation for naive and speculative decoding, fix CUDA kernel * fix logprob bug && optimize verify kernel * fix exist_decode() judge	2026-03-10 23:58:44 -07:00
mouxin	049c807d86	[Docs] Update the document (#6539) ... Co-authored-by: mouxin <mouxin@baidu.com>	2026-02-27 19:21:10 +08:00
GoldPancake	2178f2829b	[Speculative Decoding] Support suffix decoding (#6403) ... * support suffix decoding	2026-02-26 11:42:05 +08:00
jackyYang6	a29ee57e15	[Feature] Support ThinkingBudget Logits processor to control thinking content length (#6367) ... * feat: add thinking budget logits processor * add unittest * fix pre-commit * add unittest * docs: clarify operator-level vs logits processor usage and conflict guidance --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2026-02-25 14:17:09 +08:00
mouxin	6e96bd0bd2	[Feature] Fix counter release logic & update go-router download URL (#6280) ... * [Doc] Update prerequisites in the documentation * [Feature] Enhance Router with /v1/completions, docs, scripts, and version info * [Feature] Enhance Router with /v1/completions, docs, scripts, and version info * [Feature] Enhance Router with /v1/completions, docs, scripts, and version info * [Feature] Fix counter release logic * [Feature] Update go-router download URL * [Feature] Update go-router download URL * [Feature] Update go-router download URL * [Feature] Update go-router download URL * [Feature] Update token counter logic and docs * [Feature] Update token counter logic and docs --------- Co-authored-by: mouxin <mouxin@baidu.com>	2026-02-04 15:02:38 +08:00
jackyYang6	00a6a73431	docs: fix pre-commit error of markdown (#6100)	2026-01-20 19:32:05 +08:00
jackyYang6	988e0bc338	[Feature] Add PaddleFormers fallback backend (#5999) ... * feat(paddleformers): add dense text model fallback backend * docs(paddleformers): add user guide and fix code review issues * add fallback unit test * precommit format * fix pre-commit * fix: address code review feedback * docs: add PaddleFormers backend documentation (EN) and simplify installation --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2026-01-19 21:50:50 +08:00
jc	8d384f9fd8	[PD Disaggregation] Update usage of pd disaggregation and data parallel (#5742) ... * Update usage of pd disaggregation * up * up * up * up * up * up * up * up * up * up dp docs * up * up * up * fix unittest	2026-01-05 17:51:29 +08:00
Jiang-Jia-Jun	3bdd54ef6e	Disable unsupported feature in multi-node deployment docs	2025-12-10 20:23:19 +08:00
lizexu123	95eab9f9ee	[Feature] support stop_token_ids (#5399) ... * support stop_token_ids * fix * delete chinese * support both * delete print	2025-12-09 17:49:12 +08:00
Daci	83dbc4e5dd	[Feature] Guided Decoding add LLguidance backend (#5124) ... * llguidance * add requirements_guided_decoding.txt and doc * fix test_guidance_*.py * fix test_guidance_*.py && mv * fix llguidance choice * test_guidance_* * rm lazy loader --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-12-03 20:23:57 +08:00
lizexu123	c563eca791	[Feature] support reward model (#5301) ... * Your commit message here * add test * update develop * support reward * support enable_chunk_prefill * support bingfa * support convert is reward * update test * delete print * fix enable_thinking * add document * fix place * fix test * fix * support enable_prefix_caching * add no-enable_prefix-caching test * fix * support enable_prefix_caching * delete print * fix document * fix * fix test * fix document and delete chinese * udpate * enable_thinking * fix test	2025-12-02 14:55:31 +08:00
Juncai	36822fa49c	[PD Disaggregation] remove splitwise deployment on single node and refine the code (#4891) ... * remove splitwise deployment on single node and refine the code * up * up * up * add test * up	2025-11-14 09:56:53 +08:00
LiqinruiG	3f74281496	[Docs] add ERNIE-4.5-VL-28B-A3B-Thinking instruction (#4937) ... * [Docs] Improve reasoning_out docs * [Docs] Improve reasoning_out docs * [Docs] Improve reasoning_out docs * [Docs] add ERNIE-4.5-VL-28B-A3B-Thinking instruction * [Docs] add ERNIE-4.5-VL-28B-A3B-Thinking instruction * [Docs] add ERNIE-4.5-VL-28B-A3B-Thinking instruction --------- Co-authored-by: liqinrui <liqinrui@baidu.com>	2025-11-11 10:43:44 +08:00
LiqinruiG	aa79e6185a	[Docs] Improve reasoning_out docs (#4901) ... * [Docs] Improve reasoning_out docs * [Docs] Improve reasoning_out docs * [Docs] Improve reasoning_out docs --------- Co-authored-by: liqinrui <liqinrui@baidu.com>	2025-11-10 19:20:38 +08:00
zhuzixuan	8a9e7b53af	[Docs]Supplement the English and Chinese user documentation for Tool calling (#4895) ... * tool calling文档编写，v1.0 * tool calling文档编写，v1.0 * tool calling文档编写，v1.0 * tool calling doc，v1.1 * tool calling doc，v1.1 * tool calling doc，v1.1 * tool calling doc，v1.1	2025-11-08 20:05:14 +08:00
Juncai	08ca0f6aea	[Feature] [PD] add simple router and refine splitwise deployment (#4709) ... * add simple router and refine splitwise deployment * fix	2025-11-06 14:56:02 +08:00
Jiang-Jia-Jun	ca52cadd74	Update multi-node_deployment.md	2025-10-30 16:40:08 +08:00
李泳桦	a012e3608b	[Feature] support logits processors (#4515) ... * [feat] provide an interface for logits processors and a builtin LogitBiasLogitsProcessor * [chore] fix code style * [fix] add unit test & fix existing bugs * [feat] add engine/worker arg --logits-processors * [fix] redefine user args as logits_processors_args and fix some bugs * [fix] fix test_sampler * Update fastdeploy/model_executor/logits_processor/builtin.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update fastdeploy/model_executor/logits_processor/__init__.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/model_executor/test_logits_processor.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * [fix] fix typo * Update fastdeploy/engine/sampling_params.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * [fix] fix bracelet * [chore] redefine logits processor interface: pass the entire share_inputs into LP, do not copy share_inputs and logits * [doc] add docs * [fix] fix logit bias processor not applied when decoding is too fast & add docs and tests * [fix] fix redundant code * [feat] skip apply() if no bias is specified --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-10-29 00:08:53 +08:00
RAM	775edcc09a	[Executor] Default use CUDAGraph (#3594) ... * add start intercept * Adjustment GraphOptConfig * pre-commit * default use cudagraph * set default value * default use cuda graph * pre-commit * fix test case bug * disable rl * fix moba attention * only support gpu * Temporarily disable PD Disaggregation * set max_num_seqs of test case as 1 * set max_num_seqs and temperature * fix max_num_batched_tokens bug * close cuda graph * success run wint2 * profile run with max_num_batched_tokens * 1.add c++ memchecker 2.success run wint2 * updatee a800 yaml * update docs * 1. delete check 2. fix plas attn test case * default use use_unique_memory_pool * add try-except for warmup * ban mtp, mm, rl * fix test case mock * fix ci bug * fix form_model_get_output_topp0 bug * fix ci bug * refine deepseek ci * refine code * Disable PD * fix sot yaml	2025-10-21 14:25:45 +08:00
yangjianfengo1	ba5c2b7e37	[Docx] add language (en/cn) switch links (#4470) ... * add install docs * 修改文档 * 修改文档	2025-10-17 15:47:41 +08:00
yinwei	20c7b741f4	[XPU] Support W4A8C8-TP4-300B Model (#4068) ... * support w4a8 * delete ep block attn * delete moe_topk_select * update note * update * delte useless info * update * add some note * fix some format * update scale info * add ans baseline --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-10-10 15:41:32 +08:00
yangjianfengo1	b650867fff	修改文档 (#4339)	2025-10-09 20:10:58 +08:00
yangjianfengo1	4325b737e7	【FIX】Change the name of sparse attn from moba to plas (#4006) (#4076) ... * 【FIX】Change the name of sparse attn from moba to plas (#4006) * 更新文档 * 【docs】 update readme (#4000) * 更新文档 * update readme * update docs * 【FIX】Change the name of sparse attn from moba to plas (#3845) * 更新文档 * 更新文档 * 更新文档 * 更新文档 * 修改moba为plas * code style * update ci * code style * update ci * code style --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> * fix max_num_seqs * fix test load attn --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-23 10:26:40 +08:00
lizexu123	c86945ef49	[Feature] support pool (#3827) ... * support pool * update pooling * add pooler_config and check * update * support AutoWeightsLoader load weight * fix * update * delete print * update pre-commit * fix * fix xpu * fix ModelRegistry->model_registry * fix Copilot review * fix pooler.py * delete StepPooler * fix abstract * fix default_loader_v1 * fix Pre Commit * support torch qwen3 dense * add test and fix torch-qwen * fix * fix * adapter ci: * fix review * fix pooling_params.py * fix * fix tasks.py 2025 * fix print and logger * Modefy ModelRegistry and delete AutoWeightsLoader * fix logger * fix test_embedding * fix ci bug * ernie4_5 model_registry * fix test * support Qwen3-Embedding-0.6B tp=1 load * fix extra code * fix * delete fix vocab_size * delete prepare_params_dict * fix:	2025-09-22 14:09:09 +08:00
co63oc	17a27170bc	fix typos (#4093)	2025-09-15 18:33:30 +08:00
freeliuzc	46911f903d	[MTP]update hybrid-mtp-with-ngram (#4047)	2025-09-15 17:13:31 +08:00
Jiang-Jia-Jun	c60adf4281	Revert "【FIX】Change the name of sparse attn from moba to plas (#3845)" (#4001) ... This reverts commit e31c8f7336.	2025-09-09 11:08:23 +08:00
yangjianfengo1	e31c8f7336	【FIX】Change the name of sparse attn from moba to plas (#3845) ... * 更新文档 * 更新文档 * 更新文档 * 更新文档 * 修改moba为plas * code style * update ci * code style * update ci	2025-09-09 10:56:50 +08:00
yangjianfengo1	472402bf4e	Update sparse attn documentation (#3954) ... * 更新文档 * 更新文档 * 更新文档 * 更新文档	2025-09-08 12:23:18 +08:00
ltd0924	7643e6e6b2	[Docs] add data parallel (#3883) ... * [Docs] add data parallel * [Docs] add data parallel	2025-09-04 20:33:50 +08:00
kevin	1908465542	[Feature] mm and thinking model support structred output (#2749) ... * mm support structured output * update code * update code * update format * update code * update code * add enable_thinking default * update code * add structured_outputs test case * add ci install xgrammar * add ci timeout time * update test for structured_outputs * update code * add error traceback info * update error msg * update structred output code * update code * update code * update config * update torch version --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-09-02 16:21:09 +08:00
co63oc	d6369b4d51	fix typos (#3684)	2025-09-01 17:50:17 +08:00
Sunny-bot1	c68c3c4b8b	[Feature] bad words support v1 scheduler and specifiy token ids (#3608) ... * support bad_words_token_ids * docs * fix test * fix * bad words support kvcache v1 and token ids * fix	2025-08-25 20:14:51 -07:00
zhink	df7c31012b	Modified to support custom all reduce by default (#3538)	2025-08-22 16:59:05 +08:00
Zhang Yulong	33ff0bfe38	Update disaggregated.md (#3495) ... 修复文档错误	2025-08-20 19:39:18 +08:00
RAM	154308102e	[Docs]Updata docs of graph opt backend (#3442) ... * Updata docs of graph opt backend * update best_practices	2025-08-15 21:30:32 +08:00
ltd0924	5a84324798	[Doc] Add multinode deployment documents (#3417) ... * Create multi-node_deployment.md * Create multi-node_deployment.md * Update mkdocs.yml	2025-08-15 10:37:04 +08:00
Sunny-bot1	789dc67ff7	[Docs]fix sampling docs (#3113) ... * fix sampling docs * fix sampling docs * update	2025-08-11 20:42:27 +08:00
gaoziyuan	4021d66ea5	【Feature】add fd plugins && rm model_classes (#3123) ... * add fd plugins && rm model_classed * fix reviews * add docs * fix * fix unitest ci	2025-08-03 19:53:20 -07:00
LiqinruiG	25005fee30	[Doc] add chat_template_kwagrs and update params docs (#3103) ... * add chat_template_kwagrs and update params docs * add chat_template_kwagrs and update params docs * update enable_thinking * pre-commit * update test case --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-07-31 19:44:06 +08:00
Jiang-Jia-Jun	66304cf921	Update sampling.md	2025-07-31 15:02:57 +08:00
JYChen	bd29b2aaca	add stop_seqs doc (#3090)	2025-07-30 20:36:18 +08:00
李泳桦	b242150f94	[feat] extra parameters are all passed directly via http payload now, or in extra_body if using openai client (#3058) ... * [feat] extra parameters are all passed directly via http payload now, or in extra_body if using openai client * [fix] delete ci test case for enable_thinking * [fix] add reasoning_parser when server starts * [fix] fix ci consistency test error with reasoning parser * [doc] update docs related to metadata * [fix] cancel enable_thinking default value	2025-07-30 19:25:20 +08:00