d659099415
[Cleanup] Replace torch proxy alias with public compat API ( #7348 )
2026-04-13 11:43:26 +08:00
18062c55bb
[BugFix][KVCache] Fix mm hash boundary comparison in get_block_hash_extra_keys ( #6929 )
...
* [BugFix][KVCache] Fix mm hash boundary comparison in get_block_hash_extra_keys
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
* [BugFix][KVCache] Fix test_get_block_hash_extra_keys_boundary_cases assertions
## Motivation
测试用例 `test_get_block_hash_extra_keys_boundary_cases` 中，Block [4,8) 的
调用错误地传入了 `mm_idx=1`，跳过了 img0[2,5)；但 img0 覆盖 token 4，token 4
属于 block [4,8)，应被包含在 hash_keys 中。此外，所有 assertEqual 只校验了
hash_keys，未校验返回的 mm_idx 游标。
## Modifications
- `test_get_block_hash_extra_keys_boundary_cases`：
- 改为链式调用，用上一次返回的 mm_idx 作为下一次入参，模拟真实调用循环
- Block [4,8) 入参从 `mm_idx=1` 改为沿用上次返回的 `mm_idx=0`，期望值从 `[]` 改为 `["hash-0"]`
- 所有断言改为 `assertEqual((mm_idx, hash_keys), (...))` 同时校验游标
- `test_get_block_hash_extra_keys_no_overlap_at_boundaries`：
- Case B 入参从 `mm_idx=1` 改为 `mm_idx=0`（从头遍历，img-a 走 continue）
- 所有断言增加 mm_idx 校验
- `test_get_block_hash_extra_keys_image_crosses_block_boundary`：
- 所有断言增加 mm_idx 校验
- `test_get_block_hash_extra_keys_no_mm_inputs`：
- 断言增加 mm_idx 校验
- `test_get_block_hash_extra_keys_handles_multimodal_segments`：
- call2、call3 断言增加 mm_idx 校验
## Usage or Command
```bash
python -m pytest tests/cache_manager/test_prefix_cache_manager.py::TestPrefixCacheManagerCoverage -v -k "get_block_hash_extra_keys"
```
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
---------
Co-authored-by: chengyanfu <chengyanfu@baidu.com >
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-30 17:13:31 +08:00
a7f52c300d
[Feature] support v1 update/clear api for RL ( #6761 )
...
* [Feature] support v1 update/clear api for RL
* [fix] fix execute_model and add sleep/wakeup api
* [fix] fix mtp and key_prefix
* [chore] move _update_key_prefix to resume method
* [fix] make the interface safe to call multiple times
* [fix] fix some tiny bugs
* [chore] make small changes against pr review
* [docs] add docs for weight update
* [test] add some tests and update docs
* [style] fix code style check
* [test] fix ci
* [fix] fix stale control responses when control method timed out
* [chore] remove unused code
* [chore] fix code style
* [chore] optimize tags and key_prefix
* [test] fix ci
* [chore] fix code style
* [test] fix ci
* [fix] fix ep control
* [fix] fix ep control for engine cache queue
2026-03-25 19:18:46 +08:00
bb881c2c0a
[PD Disaggregation] pd + cache_storage support vl model ( #6906 )
...
* pd + cache_storage support vl model
* support vl model
* fix test
2026-03-23 15:35:20 +08:00
b05a6c4206
[BugFix][KVCache] Add inter-process lock to fix NaN error under DP+EP ( #6724 )
...
* [BugFix] Support to fix NaN bug in EP
* Optimze notion for all the funs
* Fix potential lock contention failure issues
* Update fastdeploy/inter_communicator/ipc_signal.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update envs.py
* Update default value for USE_KVCACHE_LOCK
Change default value of USE_KVCACHE_LOCK from 1 to 0.
* Update worker_process.py
* Fix suffix wrong
* Update test_prefix_cache_manager.py
---------
Co-authored-by: Jiang-Jia-Jun <jiangjiajun@baidu.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-03-10 21:55:32 +08:00
aae87e6ae2
[CI] 【Hackathon 10th Spring No.27】功能模块 fastdeploy/cache_manager/prefix_cache_manager.py单测补充 ( #6297 )
...
* test: update prefix cache manager tests
* test: refine prefix cache manager coverage helpers
* style: apply black formatting to test_prefix_cache_manager.py
Co-authored-by: Cursor <cursoragent@cursor.com >
* tests: update test_prefix_cache_manager
Co-authored-by: Cursor <cursoragent@cursor.com >
* update
---------
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-03-02 20:04:12 +08:00
e2332a1112
[BugFix] fix num_cpu_blocks computation ( #6438 )
...
* [BugFix] fix num_cpu_blocks computation
* [fix] fix syntax and log
* [fix] pre-commit
* [fix] use getattr
* [fix] ci test
2026-02-13 11:05:14 +08:00
5ac5ecd0b0
[BugFix] fix cache transfer tasks failure after cache cleared ( #6202 )
...
* [fix] fix cache transfer tasks failure after cache cleared
* [fix] fix submit_task
* [fix] fix cache manager hang when clearing prefix cache
* [fix] fix list_proxy has no clear method
* [fix] fix barrier
* [fix] add barrier0
* [fix] add cache_task_is_paused_signal
* [fix] fix condition
* [fix] fix cache transfer sync and delay prefix cache tree clearing
* [fix] fix typo
* [chore] polish code
* [fix] revert only rank0 write kv_cache_status_signal
* [fix] fix thread pool and prefix cache manager hang
* [fix] add timeout for task_swapping_event
* [fix] tolerate prefix cache manager error while prefix tree is cleared
* [chore] add more log
* [fix] fix test_prefix_cache_manager
* [fix] fix prefix_cache_status_signal usage
2026-02-08 15:33:56 +08:00
b1698a79cb
[RL] add version to the key of cache storage && refine raising error ( #6160 )
...
* Waiting for cache transfer manager inited
* up
* up
* up
* up
* up
* fix according comments
* fix unittest
* fix
* fix unittest
* fix error
* pass storage_backend to worker
2026-01-27 10:47:46 +08:00
8d27a523e7
[Feature] [KVCache] support attention_store kv cache backend ( #5823 )
...
* [feat] support attention_store kv cache backend
* [fix] fix codestyle
* [chore] optimize log
* [fix] fix write storage task
* [fix] fix read storage
* [fix] fix code conflict after merge develop
* [fix] fix cache bytes and read task token ids
* [chore] add model for cache transfer manager
* [chore] add some log
* [chore] remove launched_cache_manager_signal
* [fix] fix write_back_storage_task match_block_num condition
* [fix] fix swap_cost_time
* [ci] fix ci
* Update fastdeploy/engine/sched/resource_manager_v1.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update fastdeploy/cache_manager/cache_transfer_manager.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update fastdeploy/cache_manager/transfer_factory/mooncake_store/attention_store.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
---------
Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-01-22 21:01:23 +08:00
eabd01cd21
[BugFix] fix eb5 prefix bug ( #5879 )
...
* fix eb5 prefix bug
* update ci test
* update code
* update code
* update code
* update code
* update code
* update code
* update code
2026-01-06 23:50:39 -08:00
9445fbe054
[KVCache] launch cache transfer processes only if hierarchical cache or kv cache storage is enabled ( #5871 )
...
* [fix] temporarily forbid cpu cache in update/clear api
* [fix] stop launching cache transfer manager unless hierarchical cache is enabled
* [fix] fix no attr hierarchical cache
* [fix] fix ci
* [fix] fix test_prefix_cache_manager.py
2026-01-06 14:27:47 +08:00
e911ac2ce7
[BugFix] Refine the preparation of cpu and storage cache ( #5777 )
...
* Refine the preparation of cpu and storage cache
* fix error
* fix error
* up
* fix
* up docs
* fix unittest
* remove debug info
2026-01-05 10:13:30 +08:00
52dc9a7b85
[BugFix] skip mm revert ( #5848 )
...
* skip mm revert
* update code
* update test
2026-01-04 14:25:45 +08:00
412867fd99
[Feature] Support KV Cache Storage ( #5571 )
...
* Support Mooncake Store
* up
* up
* add op
* fix conflict
* fix error
* up for comments
* avoid thread lock
* up
* fix unittest
* fix unittest
* remove debug info
* consider tp_size > 1
* add default rdma_nics
* add utils
* up
* fix error
---------
Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com >
2025-12-25 16:30:35 +08:00
2d2619d300
[CI] 【Hackathon 9th Sprint No.36】NO.36 功能模块单测补充 （修复） ( #5609 )
...
* Implement unit tests for PrefixCacheManager
* Update prefix cache manager tests
* Handle get_all_visible_devices in prefix cache manager tests
* Add repo root to prefix cache manager tests sys.path
* Use pathlib for repo root in prefix cache manager tests
* Refine repo root Path import in tests
* Handle list-based visible device configuration
* Refine PrefixCacheManager test stubs
* Run pre-commit on prefix cache manager tests
* Remove duplicate pytest import in cache manager tests
* Add tests for visible device formatting
* Revert
* Simplify test stubs in prefix cache manager tests
* Refine PrefixCacheManager tests
* Adjust prefix cache manager tests per review
* Remove ignored tests from coverage configuration
* Make prefix cache manager tests runnable without paddle
* Use real paddle import in prefix cache manager tests
* Clean up imports in test_prefix_cache_manager.py
Removed unnecessary import of 'os' and related path manipulation.
* Update test_prefix_cache_manager.py
* Replace pid_suffix with ipc_suffix in tests
* Add local cache queue and RDMA ports to cache config
2025-12-18 16:08:42 +08:00
55609a51fc
[CI] 【Hackathon 9th Sprint No.36】NO.36 功能模块单测补充 ( #5058 )
...
* Implement unit tests for PrefixCacheManager
* Update prefix cache manager tests
* Handle get_all_visible_devices in prefix cache manager tests
* Add repo root to prefix cache manager tests sys.path
* Use pathlib for repo root in prefix cache manager tests
* Refine repo root Path import in tests
* Handle list-based visible device configuration
* Refine PrefixCacheManager test stubs
* Run pre-commit on prefix cache manager tests
* Remove duplicate pytest import in cache manager tests
* Add tests for visible device formatting
* Revert
* Simplify test stubs in prefix cache manager tests
* Refine PrefixCacheManager tests
* Adjust prefix cache manager tests per review
2025-12-16 19:19:03 +08:00
Nyako Shigure