28 Commits

Author	SHA1	Message	Date
zhouchong	6e16438a57	[Feature] implement log channel separation and request log level system (#7190) ... * feat: implement log channel separation and request log level system * fix: log system improvements based on review * add request_id to error logs, use RequestLogLevel enum, and unify logger implementation from utils to logger module	2026-04-16 15:13:05 +08:00
qwes5s5	3b7507a4c2	test_abort (#6743)	2026-03-17 14:06:40 +08:00
Yonghua Li	7811eeccaa	[fix] resolve get_save_output_v1 socket name conflicts between multiple instances (#6758)	2026-03-11 15:02:32 +08:00
CSWYF3634076	1c0a2b055f	[Feature] console print statistical metrics (#6339) ... * [Feature] console print statistical data * [Feature] console print statistical data v2 dp_rank * [Feature] console print statistical data v2 unittest * [Feature] console print statistical data v3 unittest	2026-02-05 19:20:36 +08:00
qwes5s5	38378415c7	add token ratio metrics (#6236)	2026-01-27 17:00:49 +08:00
qwes5s5	b2a2e11551	[Feature] Support stopping the inference for the corresponding request in the online service after a disconnection request. (#5320) ... * request disconnect * request disconnect * fix bug * fix bug--amend --------- Co-authored-by: root <root@yq01-sys-rpm26xc1knu.yq01.baidu.com>	2026-01-16 11:46:13 +08:00
chenjian	74d0f1c01f	[Optim] Robust sync status when preempted happens (#5796) ... * [Bug fix] Sync status for caching output cache * fix * fix * fix bug * fix * fix * support xpu * fix * fix * fix * fix * fix * fix ci * fix ci * fix xpu --------- Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2026-01-14 12:07:33 +08:00
GoldPancake	4e10ae5d99	[Speculative Decoding] Optimize draft logprob (#5842) ... * optimize draft logprob * fix ut	2025-12-31 13:35:56 +08:00
memoryCoderC	be3be4913a	[Optimization] refactor(chat_handler,completion_handler): extract base classes and use AsyncLLM (#5195) ... * [Optimization] refactor(chat_handler,completion_handler): extract base classes and use AsyncLLM * [Optimization] refactor(chat_handler,completion_handler): rename class	2025-12-25 16:28:15 +08:00
xunyoyo	3aee5c4bf5	[CI] 【Hackathon 9th Sprint No.37】NO.37 功能模块单测补充 (#5059) ... * Add unit tests for TokenProcessor functionality * Add trace stubs for token processor tests * Increase token processor test coverage * Clean up imports in test_token_processor.py Remove unnecessary path manipulation in test file. * Cleanup: Remove unused imports in test_token_processor Removed unused imports from the test file. * Add trace_carrier to task in test cases Added trace_carrier attribute to task in multiple test cases to ensure proper handling of trace information. * Refine token processor tests for safe coverage * Expand postprocess coverage * Add ZMQ logprob parsing test --------- Co-authored-by: CSWYF3634076 <wangyafeng@baidu.com> Co-authored-by: Tao Luo <luotao02@baidu.com>	2025-12-23 10:35:16 +08:00
xiaolei373	a30b4da260	[Feature] Tracing: Fine-Grained Tracing for Request Latency Part1 (#5458)	2025-12-16 16:36:09 +08:00
GoldPancake	909059c60a	[Feature] Support for request-level speculative decoding metrics monitoring. (#5518) ... * support spec metrics monitor per request * fix bug * remove debug log * fix ut bugs	2025-12-12 12:22:18 +08:00
Juncai	80efe98f8d	[PD Disaggregation] Add timestamp for analyzing splitwise deployment (#5317) ... * Add timestamp for analyzing splitwise deployment * up * up * up * up * up * up * fix format * fix	2025-12-08 10:08:44 +08:00
chenjian	3878a99b69	[Fearture] Support cache kv cache for output tokens (#4535) ... * [Fearture] Support cache kv cache for output tokens * fix bug * fix ci bug * improve coverage * enable output caching by default * fix ci --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-12-04 20:53:08 +08:00
qwes5s5	117980dd4e	[LogProbs]Enable prompt logprobs output and modify data transmission method for the online interface. (#5089) ... * add prompt logprobs * Merge prompt_logprobs_tensors and prompt_logprobs * fix param check * trigger ci * fix unitest * fix logprobs bug	2025-12-02 13:49:51 +08:00
SunLei	c424e08dc5	[Speculative Decoding] split draft_tokens into standalone post-processing path (#5205) ... * refactor(mtp): split draft_tokens into standalone post-processing path for MTP + logprobs * Restore Request.__repr__ implementation * ci * add envs * fix unittest	2025-11-27 11:22:41 +08:00
chenjian	3ea1b44a58	[Optimization] Improve perf for fd response token with internal adapter (#4992) ... * [Optimize] Improve perf for fd response token with internal adapter * fix * fix bug * fix ci * fix ci * fix ci * fix ci	2025-11-21 19:02:03 +08:00
qwes5s5	36216e62f0	[Log] Add trace log and add loggingInstrumentor tool (#4692) ... * add trace logger and trace print * trigger ci * fix unittest * translate notes and add copyright --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com> Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>	2025-11-17 11:08:57 +08:00
qwes5s5	a2d06118e1	[Logprobs]Support prompt_logprobs and max_logprobs (#4897) ... * add prompt logprobs * trigger ci * fix unitest * Update fastdeploy/config.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update fastdeploy/entrypoints/llm.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update fastdeploy/engine/sampling_params.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/engine/test_sampling_params.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/engine/test_sampling_params.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix max_logprobs --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-11-12 19:29:48 +08:00
Echo-Nie	ff653503ff	[Docs] Add License in Unittest (#4957) ... * add copyright * add CopyRight	2025-11-12 10:44:09 +08:00
Echo-Nie	112623e33e	init version, exist some bugs, waiting fix (#4906)	2025-11-10 14:16:09 +08:00
chenjian	cc8f5312f5	[Feature] Add timestamp for profiler (#4726) ... * [Feature] Add timestamp for profiler * fix bug for offine inference * fix for ci * fix * fix ci	2025-11-05 12:04:59 +08:00
Longzhi Wang	b61a272385	[BugFix] fix unittest of get_save_output_v1 (#4701) ... * [BugFix] fix unittest of get_save_output_v1 * [BugFix] fix unittest of get_save_output_v1 * [BugFix] fix unittest of get_save_output_v1 * [BugFix] fix unittest of get_save_output_v1	2025-10-31 11:23:49 +08:00
ApplEOFDiscord	14f8cddaf1	[Feature] add mm token usage (#4570) ... * add mm token usage * fix unit test * fix unit test * fix unit test * fix model path * fix unit test * fix unit test * fix unit test * remove uncomment * change var name * fix code style * fix code style * fix code style * fix code style * fix unit test	2025-10-29 14:37:12 +08:00
SunLei	ee915220bd	[Speculative Decoding] Add draft_logprobs Support for Speculative Decode MTP (#4467) ... * feat: add draft_logprobs for Speculative Decode MTP * feat: add draft_logprobs for Speculative Decode MTP * feat: add draft_logprobs for Speculative Decode MTP * fix: postprocess for speculative decode * test: test_speculative_decoding_use_logprobs * fix: test_completion_echo * fix test_max_streaming_tokens --------- Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>	2025-10-21 14:57:50 +08:00
co63oc	17a27170bc	fix typos (#4093)	2025-09-15 18:33:30 +08:00
Zhang Yulong	349aa6348b	add cache queue port (#3904) ... * add cache queue port * add cache queue port * add cache queue port	2025-09-05 21:17:06 +08:00
Longzhi Wang	e0c9a6c76c	[Feat] Support streaming transfer data using ZMQ (#3521) ... * Support streaming transfer data of ZMQ * fix typo * fix typo * support tp * add unittest * update * update * fix typo * fix typo * fix tp_num in ci machine --------- Co-authored-by: Wanglongzhi2001 <>	2025-09-02 19:52:19 +08:00