b05a6c4206
[BugFix][KVCache] Add inter-process lock to fix NaN error under DP+EP ( #6724 )
...
* [BugFix] Support to fix NaN bug in EP
* Optimze notion for all the funs
* Fix potential lock contention failure issues
* Update fastdeploy/inter_communicator/ipc_signal.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update envs.py
* Update default value for USE_KVCACHE_LOCK
Change default value of USE_KVCACHE_LOCK from 1 to 0.
* Update worker_process.py
* Fix suffix wrong
* Update test_prefix_cache_manager.py
---------
Co-authored-by: Jiang-Jia-Jun <jiangjiajun@baidu.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-03-10 21:55:32 +08:00
aae87e6ae2
[CI] 【Hackathon 10th Spring No.27】功能模块 fastdeploy/cache_manager/prefix_cache_manager.py单测补充 ( #6297 )
...
* test: update prefix cache manager tests
* test: refine prefix cache manager coverage helpers
* style: apply black formatting to test_prefix_cache_manager.py
Co-authored-by: Cursor <cursoragent@cursor.com >
* tests: update test_prefix_cache_manager
Co-authored-by: Cursor <cursoragent@cursor.com >
* update
---------
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-03-02 20:04:12 +08:00
e2332a1112
[BugFix] fix num_cpu_blocks computation ( #6438 )
...
* [BugFix] fix num_cpu_blocks computation
* [fix] fix syntax and log
* [fix] pre-commit
* [fix] use getattr
* [fix] ci test
2026-02-13 11:05:14 +08:00
5ac5ecd0b0
[BugFix] fix cache transfer tasks failure after cache cleared ( #6202 )
...
* [fix] fix cache transfer tasks failure after cache cleared
* [fix] fix submit_task
* [fix] fix cache manager hang when clearing prefix cache
* [fix] fix list_proxy has no clear method
* [fix] fix barrier
* [fix] add barrier0
* [fix] add cache_task_is_paused_signal
* [fix] fix condition
* [fix] fix cache transfer sync and delay prefix cache tree clearing
* [fix] fix typo
* [chore] polish code
* [fix] revert only rank0 write kv_cache_status_signal
* [fix] fix thread pool and prefix cache manager hang
* [fix] add timeout for task_swapping_event
* [fix] tolerate prefix cache manager error while prefix tree is cleared
* [chore] add more log
* [fix] fix test_prefix_cache_manager
* [fix] fix prefix_cache_status_signal usage
2026-02-08 15:33:56 +08:00
b1698a79cb
[RL] add version to the key of cache storage && refine raising error ( #6160 )
...
* Waiting for cache transfer manager inited
* up
* up
* up
* up
* up
* fix according comments
* fix unittest
* fix
* fix unittest
* fix error
* pass storage_backend to worker
2026-01-27 10:47:46 +08:00
8d27a523e7
[Feature] [KVCache] support attention_store kv cache backend ( #5823 )
...
* [feat] support attention_store kv cache backend
* [fix] fix codestyle
* [chore] optimize log
* [fix] fix write storage task
* [fix] fix read storage
* [fix] fix code conflict after merge develop
* [fix] fix cache bytes and read task token ids
* [chore] add model for cache transfer manager
* [chore] add some log
* [chore] remove launched_cache_manager_signal
* [fix] fix write_back_storage_task match_block_num condition
* [fix] fix swap_cost_time
* [ci] fix ci
* Update fastdeploy/engine/sched/resource_manager_v1.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update fastdeploy/cache_manager/cache_transfer_manager.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update fastdeploy/cache_manager/transfer_factory/mooncake_store/attention_store.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
---------
Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-01-22 21:01:23 +08:00
eabd01cd21
[BugFix] fix eb5 prefix bug ( #5879 )
...
* fix eb5 prefix bug
* update ci test
* update code
* update code
* update code
* update code
* update code
* update code
* update code
2026-01-06 23:50:39 -08:00
9445fbe054
[KVCache] launch cache transfer processes only if hierarchical cache or kv cache storage is enabled ( #5871 )
...
* [fix] temporarily forbid cpu cache in update/clear api
* [fix] stop launching cache transfer manager unless hierarchical cache is enabled
* [fix] fix no attr hierarchical cache
* [fix] fix ci
* [fix] fix test_prefix_cache_manager.py
2026-01-06 14:27:47 +08:00
e911ac2ce7
[BugFix] Refine the preparation of cpu and storage cache ( #5777 )
...
* Refine the preparation of cpu and storage cache
* fix error
* fix error
* up
* fix
* up docs
* fix unittest
* remove debug info
2026-01-05 10:13:30 +08:00
52dc9a7b85
[BugFix] skip mm revert ( #5848 )
...
* skip mm revert
* update code
* update test
2026-01-04 14:25:45 +08:00
412867fd99
[Feature] Support KV Cache Storage ( #5571 )
...
* Support Mooncake Store
* up
* up
* add op
* fix conflict
* fix error
* up for comments
* avoid thread lock
* up
* fix unittest
* fix unittest
* remove debug info
* consider tp_size > 1
* add default rdma_nics
* add utils
* up
* fix error
---------
Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com >
2025-12-25 16:30:35 +08:00
2d2619d300
[CI] 【Hackathon 9th Sprint No.36】NO.36 功能模块单测补充 （修复） ( #5609 )
...
* Implement unit tests for PrefixCacheManager
* Update prefix cache manager tests
* Handle get_all_visible_devices in prefix cache manager tests
* Add repo root to prefix cache manager tests sys.path
* Use pathlib for repo root in prefix cache manager tests
* Refine repo root Path import in tests
* Handle list-based visible device configuration
* Refine PrefixCacheManager test stubs
* Run pre-commit on prefix cache manager tests
* Remove duplicate pytest import in cache manager tests
* Add tests for visible device formatting
* Revert
* Simplify test stubs in prefix cache manager tests
* Refine PrefixCacheManager tests
* Adjust prefix cache manager tests per review
* Remove ignored tests from coverage configuration
* Make prefix cache manager tests runnable without paddle
* Use real paddle import in prefix cache manager tests
* Clean up imports in test_prefix_cache_manager.py
Removed unnecessary import of 'os' and related path manipulation.
* Update test_prefix_cache_manager.py
* Replace pid_suffix with ipc_suffix in tests
* Add local cache queue and RDMA ports to cache config
2025-12-18 16:08:42 +08:00
55609a51fc
[CI] 【Hackathon 9th Sprint No.36】NO.36 功能模块单测补充 ( #5058 )
...
* Implement unit tests for PrefixCacheManager
* Update prefix cache manager tests
* Handle get_all_visible_devices in prefix cache manager tests
* Add repo root to prefix cache manager tests sys.path
* Use pathlib for repo root in prefix cache manager tests
* Refine repo root Path import in tests
* Handle list-based visible device configuration
* Refine PrefixCacheManager test stubs
* Run pre-commit on prefix cache manager tests
* Remove duplicate pytest import in cache manager tests
* Add tests for visible device formatting
* Revert
* Simplify test stubs in prefix cache manager tests
* Refine PrefixCacheManager tests
* Adjust prefix cache manager tests per review
2025-12-16 19:19:03 +08:00
Jiang-Jia-Jun