569 Commits

Author	SHA1	Message	Date
kevin	4fa76296d9	[BugFix] fix mm splitwise scheduler bug (#5604) ... * fix mm splitwise scheduler bug * fix test case bug * update code * update code	2025-12-25 04:08:11 -08:00
Copilot	1cbf448178	[Feature] Add startup version check mechanism for Paddle (#5769) ... * Initial plan * 实现版本检查机制：添加get_version_info函数并在启动时检查Paddle版本 Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> * 修复代码审查反馈：改进错误处理和日志记录 Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> * Change comments and warning messages from Chinese to English Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> * Update fastdeploy/__init__.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-12-25 19:29:04 +08:00
freeliuzc	9018ccf74e	[Speculative Decoding] Fix attn_mask_offset for multi-step MTP in mixed and PD-split modes (#5738) ... * fix attn_mask_offset in mtp with multi-step and pd-split-mode * fix xpu operater register * update pmtp multi-step mtp strategy in d-split -mode * add note * fix xpu register	2025-12-25 01:54:59 -08:00
YuBaoku	7247dc5f3a	[CI] Add retry and robust cleanup for removal (#5725) ... * [CI] Add retry and robust cleanup for removal * [CI] Ensure clean GPU memory by killing leftover processes	2025-12-25 17:08:27 +08:00
Juncai	412867fd99	[Feature] Support KV Cache Storage (#5571) ... * Support Mooncake Store * up * up * add op * fix conflict * fix error * up for comments * avoid thread lock * up * fix unittest * fix unittest * remove debug info * consider tp_size > 1 * add default rdma_nics * add utils * up * fix error --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-12-25 16:30:35 +08:00
memoryCoderC	be3be4913a	[Optimization] refactor(chat_handler,completion_handler): extract base classes and use AsyncLLM (#5195) ... * [Optimization] refactor(chat_handler,completion_handler): extract base classes and use AsyncLLM * [Optimization] refactor(chat_handler,completion_handler): rename class	2025-12-25 16:28:15 +08:00
Jiaxin Sui	8fc789bb3f	[iluvatar][CI] refactor iluvatar_ci (#5588) ... * refactor iluvatar_ci * refactor iluvatar_ci * refactor iluvatar_ci * refactor iluvatar_ci * refactor iluvatar_ci * refactor iluvatar_ci * refactor iluvatar_ci * refactor iluvatar_ci * refactor iluvatar_ci * Update Docker image tag in iluvatar_test workflow * Update default Docker image version in workflow * Update iluvatar_test.yml * Update default Docker image in workflow config * Update model path in run_ernie300B_4layer.py * Update model path in offline inference check * Add model_data directory and copy model files Create model_data directory and copy necessary files. * Update run_ernie_vl_28B.py * Update run_ernie300B_4layer.py * Update paddlepaddle installation method in script * Change wget command to include proxy option * Modify paddle package installation in CI script Updated installation commands for paddle packages. * Update paddlepaddle and paddle-iluvatar-gpu versions * Delete .github/workflows/ci_iluvatar.yml * Rename workflow from ILUVATAR Test to ILUVATAR-CI * Update installation commands for paddlepaddle and iluvatar	2025-12-25 15:10:34 +08:00
YuBaoku	0410c42a9a	[CI] Refactor RL tests to reuse stable_test (#5516) ... * [CI] Refactor RL tests to reuse stable_test	2025-12-24 19:18:00 +08:00
YuBaoku	e75f93d302	[CI] Refactor RL tests to reuse test_metrics (#5741)	2025-12-24 17:08:40 +08:00
Divano	6b0fba8294	Update run.sh	2025-12-24 15:35:17 +08:00
Nyakku Shigure	11227e00bb	[GraphOptimization] Wrap deep gemm and triton as python op (#5673) ... * [GraphOptimization] Wrap deep gemm and triton as python op * add unitest to _base_test && compatibility * paddle.static.MetaTensor -> "paddle.static.MetaTensor" * mv register_custom_python_op * rename yaml --------- Co-authored-by: DrRyanHuang <zihaohuang@aliyun.com>	2025-12-24 15:23:46 +08:00
bukejiyu	ba4b7afb3a	[Others] Rename tensor_parallel_degree to tensor_model_parallel_size for paddleformers 0.4.1 (#5727)	2025-12-23 23:19:11 -08:00
xunyoyo	8acdd9f156	[CI] 【Hackathon 9th Sprint No.41】NO.41 功能模块单测补充 -new ... Add splitwise connector tests Co-authored-by: CSWYF3634076 <wangyafeng@baidu.com> Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-12-24 14:05:32 +08:00
YuBaoku	672620cdfe	Revert "[CI] Adapt vl_model baseline changes due to Paddle update (#5576)" (#5732) ... This reverts commit 63fff8df70.	2025-12-24 11:59:27 +08:00
GoldPancake	23d488c488	[Feature] Entropy calculation support (#5692) ... * support entropy * fix bug --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-12-23 21:19:47 +08:00
bukejiyu	d1c6e57341	[Others] upgrade paddleformer to 0.4.0 (#5599)	2025-12-23 05:08:01 -08:00
ophilia-lee	99258e19c8	[Benchmark]支持Completions接口 (#5700) ... * benchmark工具支持受限解码场景指定response_format * Update backend_request_func.py output.success判断兼容思考内容超长截断时回复内容为空的情况 * Update benchmark_serving.py 更新benchmark_metrics * 支持Completions接口 * 支持Completions接口 * 支持Completions接口 * [Benchmark]支持Completions接口 * [Benchmark]支持Completions接口 --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-12-23 19:46:23 +08:00
kesmeey	f15edbb6ef	[CI]【Hackathon 9th Sprint No.40】功能模块 fastdeploy/entrypoints/openai/api_server.py 单测补充 (#5567) ... * Add tests for openai api_server coverage * update * Update tests for openai api_server * fix bugs * test: disable some api_server lifespan/controller tests for local env * Format test_api_server with black * update * update * test: narrow envs patch in api_server tests to avoid side effects * fix: separate MagicMock creation to avoid missing req argument * fix: patch TRACES_ENABLE env var in api_server tests * fix: use os.environ patch for TRACES_ENABLE * test: use fake fastdeploy.envs in api_server tests * test: pass fake Request into chat/completion routes * test: increase coverage for tracing and scheduler control * fix: set dynamic_load_weight in tracing headers test * ci: add retry and validation for FastDeploy.tar.gz download * ci: fix indentation in _base_test.yml * refactor: simplify test_api_server.py (807->480 lines, ~40% reduction) * fix: restore missing args attributes (revision, etc.) in _build_args * fix: patch sys.argv to prevent SystemExit: 2 in api_server tests * improve coverage * Remove docstring from test_api_server.py Removed unnecessary docstring from test_api_server.py --------- Co-authored-by: CSWYF3634076 <wangyafeng@baidu.com>	2025-12-23 18:06:43 +08:00
Divano	c1aa66df02	Revert "[Optim] Remove limitation of number of kvcache blocks (#5612)" (#5702) ... This reverts commit 9da89a374b.	2025-12-23 15:41:33 +08:00
Jiang-Jia-Jun	9da89a374b	[Optim] Remove limitation of number of kvcache blocks (#5612) ... * [Optim] Remove limitation of number of kvcache blocks * Update fastdeploy/envs.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update fastdeploy/worker/iluvatar_worker.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Add docs * Update fastdeploy/worker/worker_process.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix ci case --------- Co-authored-by: Jiang-Jia-Jun <jiangjiajun@baidu.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-12-23 11:18:29 +08:00
xunyoyo	3aee5c4bf5	[CI] 【Hackathon 9th Sprint No.37】NO.37 功能模块单测补充 (#5059) ... * Add unit tests for TokenProcessor functionality * Add trace stubs for token processor tests * Increase token processor test coverage * Clean up imports in test_token_processor.py Remove unnecessary path manipulation in test file. * Cleanup: Remove unused imports in test_token_processor Removed unused imports from the test file. * Add trace_carrier to task in test cases Added trace_carrier attribute to task in multiple test cases to ensure proper handling of trace information. * Refine token processor tests for safe coverage * Expand postprocess coverage * Add ZMQ logprob parsing test --------- Co-authored-by: CSWYF3634076 <wangyafeng@baidu.com> Co-authored-by: Tao Luo <luotao02@baidu.com>	2025-12-23 10:35:16 +08:00
Jiaxin Sui	f16077a939	[XPU][CI] Xpu ci update (#5690) ... * Enhance run_ci_xpu.sh with caching and prefill options * Update model path and configuration in run_ci_xpu.sh * Add '北朝' keyword to assertion in run_45vl.py * Enhance process termination logic in run_ci_xpu.sh * Set timeout for CI_XPU job to 60 minutes * Remove extra newline in stop_processes function * Update paddlepaddle-xpu installation command Comment out the previous paddlepaddle-xpu installation command and replace it with a specific version installation due to EP parallel error. * Update PaddlePaddle installation command * Remove max_tokens from model response configuration Removed max_tokens parameter from the model response call.	2025-12-23 10:19:39 +08:00
xiaolei373	dfe8ea941c	[log]console log to llm log (#5680)	2025-12-23 10:05:45 +08:00
ddchenhao66	a1535c7e7e	[XPU][CI] xpu add ci test for pd + TP2 (#5653) ... Co-authored-by: ddchenhao66 <dhaochen163.com>	2025-12-22 19:27:10 +08:00
lizexu123	6d323769dd	fix w4afp8 (#5634)	2025-12-22 13:39:41 +08:00
YuBaoku	fe55baae47	[CI] Fix unit_test error of unstable execution (#5660) ... * [CI] Fix unit_test error of unstable execution	2025-12-19 22:59:53 +08:00
MingkunZhang	46d83be065	[Metax] update ci test (#5652)	2025-12-19 17:25:47 +08:00
xunyoyo	669dfe8dca	[CI] 【Hackathon 9th Sprint No.38】NO.38 功能模块单测补充 (#5060) ... * Add tests for global scheduler functionality This file contains tests for the global scheduler, including request handling, load accounting, and response management. * Enhance global scheduler unit tests * Clean up comments in test_global_scheduler.py Removed unnecessary comments and coverage instructions. * Update tests/scheduler/test_global_scheduler.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/scheduler/test_global_scheduler.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/scheduler/test_global_scheduler.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/scheduler/test_global_scheduler.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Translate comments and docstrings to Chinese Updated comments and docstrings in Chinese to enhance clarity for native speakers. * Use monkeypatch for FD_ENABLE_MAX_PREFILL in tests Replaced direct assignment with monkeypatching for FD_ENABLE_MAX_PREFILL in test cases. * Correct test function definitions in test_global_scheduler Fix function definitions and update test cases for scheduler. * Fix typos in global scheduler tests * Translate comments to English in test_global_scheduler.py Updated comments and docstrings in test_global_scheduler.py from Chinese to English for better clarity and consistency. --------- Co-authored-by: CSWYF3634076 <wangyafeng@baidu.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Jiaxin Sui <95567040+plusNew001@users.noreply.github.com>	2025-12-19 16:28:16 +08:00
xunyoyo	d657455616	[CI] 【Hackathon 9th Sprint No.19】NO.19 功能模块单测补充 (#5063) ... * Add pooler unit tests * Refine pooler tests import handling * Refactor pooler tests to use real modules * Clean up test_pooler.py by removing docstring Removed unnecessary docstring and cleaned up code. * Clean up imports in test_pooler.py Removed unnecessary import of sys and related path adjustments. * Update model config and clean up test code * Update test_pooler.py --------- Co-authored-by: CSWYF3634076 <wangyafeng@baidu.com>	2025-12-18 21:32:44 +08:00
Jiaxin Sui	d739af5e6e	Revert "[XPU][CI] xpu add ci test for pd (#5610)" (#5645) ... This reverts commit 80fb530ce2.	2025-12-18 19:59:09 +08:00
yzwu	ac013803f3	[Iluvatar] Support V1_KVCACHE_SCHEDULER and paddleocr-vl rope mode (#5555)	2025-12-18 02:14:25 -08:00
xunyoyo	2d2619d300	[CI] 【Hackathon 9th Sprint No.36】NO.36 功能模块单测补充 （修复） (#5609) ... * Implement unit tests for PrefixCacheManager * Update prefix cache manager tests * Handle get_all_visible_devices in prefix cache manager tests * Add repo root to prefix cache manager tests sys.path * Use pathlib for repo root in prefix cache manager tests * Refine repo root Path import in tests * Handle list-based visible device configuration * Refine PrefixCacheManager test stubs * Run pre-commit on prefix cache manager tests * Remove duplicate pytest import in cache manager tests * Add tests for visible device formatting * Revert * Simplify test stubs in prefix cache manager tests * Refine PrefixCacheManager tests * Adjust prefix cache manager tests per review * Remove ignored tests from coverage configuration * Make prefix cache manager tests runnable without paddle * Use real paddle import in prefix cache manager tests * Clean up imports in test_prefix_cache_manager.py Removed unnecessary import of 'os' and related path manipulation. * Update test_prefix_cache_manager.py * Replace pid_suffix with ipc_suffix in tests * Add local cache queue and RDMA ports to cache config	2025-12-18 16:08:42 +08:00
kesmeey	d81341b9b3	[CI]【Hackathon 9th Sprint No.14】功能模块 fastdeploy/rl/rollout_model.py 单测补充 (#5552) ... * Add rollout model unit tests * test: update rl rollout_model tests * test: fix cache_type_branches unsupported platform case * test: fix rl rollout_model test indent * Delete tests/spec_decode/test_mtp_proposer.py * chore: format test_rollout_model * chore: translate rollout test comments to English * test: guard rollout_model import by disabling auto registry * chore: reorder imports in rl rollout test * test: isolate env for RL rollout tests * style: format rollout RL tests with black * update * test: remove RL rollout unit tests causing collection issues * test: add lightweight rollout_model RL unit tests * fix(coverage): filter test file paths and handle collection failures - Only extract real test file paths (tests/.../test_*.py) from pytest collect output - Filter out ERROR/collecting prefixes to prevent garbage in failed_tests.log - Add proper error handling for pytest collection failures - Exit early if no test files can be extracted - Preserve collection error output for debugging * update * style: fix code style issues in test_rollout_model.py - Remove unused 'os' import - Remove trailing blank lines --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-12-18 10:57:53 +08:00
Zhang Yulong	94be5ebdd1	[CI] Add CI case for MTP accept ratio (#5570) ... * Implement test for MTP accept ratio Add test for MTP accept ratio with assertions on results and metrics. * Update test_ernie_21b_mtp.py * Refactor test_mtp_accept_ratio for baseline comparison Refactor test_mtp_accept_ratio to compare results against baseline file and metrics. * Fix formatting issues in test_ernie_21b_mtp.py --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-12-17 21:35:02 +08:00
kesmeey	ac731653b3	[CI]【Hackathon 9th Sprint No.12】功能模块 fastdeploy/spec_decode/mtp.py 单测补充 (#5533) ... * Add unit tests for MTPProposer class in spec_decode/mtp.py * fix: remove non-existent QuantizationConfig import in test_mtp_proposer * fix: add logprobs_mode attribute to FakeModelConfig * fix: fix test failures in test_mtp_proposer - fix Mock setup, remove arrival_time, add missing keys * fix: add seq_lens_this_time initialization and kv_cache init before insert_tasks_v1 * fix: check pos_emb_type attribute existence before assertion * test: add minimal coverage for mtp cache type, mm init, preempted * test: fix cache_type_branches unsupported platform on 12 * test: refine MTPProposer tests for cache type, requests and chunked prefill * chore: remove stray spec_decode copy	2025-12-17 20:09:45 +08:00
fmiao2372	404cf0ece4	[Intel HPU] enable tensor_wise_fp8 (#5324) ... * [Intel HPU] enable tensor_wise_fp8 * update code based on comments * fix code style issue * fix bug about RP 5138 * mv kv_cache modifications to HPU backend * fix FP8 Precision Issues * fix FP8 Precision Issues * Add quantization UT --------- Co-authored-by: yanfeich <yanfei.cheng@intel.com> Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-12-17 16:45:03 +08:00
freeliuzc	15f5112ecb	[Speculative Decoding]Support different inferseed in speculate decoding (#5568) ... * fix mtp entropy drop in RL * optimize usage and fix unit test * optimize padding_sampling_params speed(vectorized) --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-12-17 16:14:29 +08:00
ddchenhao66	80fb530ce2	[XPU][CI] xpu add ci test for pd (#5610) ... Co-authored-by: ddchenhao66 <dhaochen163.com>	2025-12-17 16:07:44 +08:00
Yonghua Li	0c8c6369ed	[Feature] [PD Disaggregation] simplify configuration for pd-disaggregated deployment, and refactor post-init and usage for all ports (#5415) ... * [feat] simplify configuration for pd-disaggregated deployment, and refactor post-init and usage for all ports * [fix] fix some bugs * [fix] fix rdma port for cache manager/messager * [fix] temporarily cancel port availability check to see if it can pass ci test * [feat] simplify args for multi api server * [fix] fix dp * [fix] fix port for xpu * [fix] add tests for ports post processing & fix ci * [test] fix test_multi_api_server * [fix] fix rdma_comm_ports args for multi_api_server * [fix] fix test_common_engine * [fix] fix test_cache_transfer_manager * [chore] automatically setting FD_ENABLE_MULTI_API_SERVER * [fix] avoid api server from creating engine_args twice * [fix] fix test_run_batch * [fix] fix test_metrics * [fix] fix splitwise connector init * [test] add test_rdma_transfer and test_expert_service * [fix] fix code syntax * [fix] fix test_rdma_transfer and build wheel with rdma script	2025-12-17 15:50:42 +08:00
Yuanle Liu	cdc0004894	Revert "[Feature] add ue8m0 for per_token_quant_fp8 (#5563)" (#5611) ... This reverts commit 73e1d6aa90.	2025-12-17 13:59:06 +08:00
YuBaoku	21fa2baa51	[CI] disable test_prefix_cache_manager.py in unit_test	2025-12-17 10:48:02 +08:00
周周周	e29b005520	[Others] Clean code && remove GPU sync code (#5548)	2025-12-16 21:09:37 +08:00
Yuanle Liu	867803ae10	[BugFix] fix speculate_limit_thinking_content_length (#5590) ... * fix speculate_limit_thinking_content_length * update	2025-12-16 04:31:45 -08:00
xunyoyo	55609a51fc	[CI] 【Hackathon 9th Sprint No.36】NO.36 功能模块单测补充 (#5058) ... * Implement unit tests for PrefixCacheManager * Update prefix cache manager tests * Handle get_all_visible_devices in prefix cache manager tests * Add repo root to prefix cache manager tests sys.path * Use pathlib for repo root in prefix cache manager tests * Refine repo root Path import in tests * Handle list-based visible device configuration * Refine PrefixCacheManager test stubs * Run pre-commit on prefix cache manager tests * Remove duplicate pytest import in cache manager tests * Add tests for visible device formatting * Revert * Simplify test stubs in prefix cache manager tests * Refine PrefixCacheManager tests * Adjust prefix cache manager tests per review	2025-12-16 19:19:03 +08:00
fxyfxy777	73e1d6aa90	[Feature] add ue8m0 for per_token_quant_fp8 (#5563) ... * ue8m0 * add default arg --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-12-16 18:40:12 +08:00
xiaolei373	a30b4da260	[Feature] Tracing: Fine-Grained Tracing for Request Latency Part1 (#5458)	2025-12-16 16:36:09 +08:00
kevin	c9b47f90ce	[BugFix] fix cpu prefix cache bug (#5544) ... * fix_dy_c8_bug * add block_num check * fix test case * update ci case	2025-12-16 14:21:42 +08:00
YuBaoku	5d2b16e6f3	[CI] Remove test_metrics.py due to incompatible forced merge (#5578) ... * [CI] Remove test_metrics.py due to incompatible forced merge	2025-12-16 14:04:46 +08:00
YuBaoku	63fff8df70	[CI] Adapt vl_model baseline changes due to Paddle update (#5576)	2025-12-16 11:42:31 +08:00
xunyoyo	9e8c46c526	[CI] 【Hackathon 9th Sprint No.34】NO.34 功能模块单测补充 (#5057) ... * Add unit tests for SplitWiseScheduler module * Add info and ping to fake redis client for tests * Document fake redis metadata methods in tests * Enhance splitwise scheduler tests * Clean up test_splitwise_scheduler.py Removed copyright notice and documentation comments. * Simplify splitwise scheduler test stubs * Refine splitwise scheduler tests * Handle empty result keys with restored sleep --------- Co-authored-by: CSWYF3634076 <wangyafeng@baidu.com>	2025-12-15 20:29:25 +08:00