Commit Graph

6 Commits

Author SHA1 Message Date
YuBaoku 61ebac49ef [CI] Fix test_communication.py and add port cleanup (#7021) 2026-03-26 10:56:40 +08:00
luukunn c3d8db85c4 [Optimization] Update ZMQ server (#6735)
* add batch zmq send reaponse

* update

* Revert "update"

This reverts commit 0234a25b47.

* update

* remove lock

* fix unit test

* add unit test

* add unit test

* pre commit

* add unit test

* fix unit test

* add unit test

* fix worker>1

* update zmq_worker_pid

* fix unit test

* fix unit test

* fix unit test

* add unit test

* fix unit test

* fix first token time

* fix logprobs

* add unit test

* op

* remore debug log

---------

Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>
2026-03-19 21:53:16 +08:00
zhouchong 5d9b5e4a5b [Engine] [Feature] Refactor async_llm:cross-process with EngineService,based on zmq communication (#4868)
* Refactor async_llm:cross-process with EngineService

* fix: async_llm output process

* fix: return prompt_token_ids and prompt_tokens in first res

* optimize common_engine start func
2025-12-09 10:53:40 +08:00
zhouchong 35286ce31a fix total_block_num init error in worker_process (#4687) 2025-10-30 19:53:09 +08:00
zhouchong 567f61072c [CI][BugFix] fix port conflicts in concurrent ci test and add more unit test on async_llm (#4616)
* fix:port conflicts in concurrent ci test

* add more unit test on async_llm
2025-10-28 19:04:24 +08:00
zhouchong dce988824d [Feature] Support AsyncLLM (#4458)
CE Compile Job / ce_job_pre_check (push) Has been cancelled
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
CE Compile Job / Show Code Archive Output (push) Has been cancelled
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
CE Compile Job / CE_UPLOAD (push) Has been cancelled
Deploy GitHub Pages / deploy (push) Has been cancelled
* add async_llm

* apply review

* update engine config

* Adapt to latest engine.py changes

* add more unit tests

* Increase unit test coverage

---------

Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>
2025-10-22 15:50:12 +08:00