apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2026-04-23 00:17:25 +08:00
Code Issues Actions 19 Packages Projects Releases Wiki Activity

[Feature] [KVCache] support file_store kv cache backend (#6188)

Browse Source 
* fix(examples): comment out stop.sh to avoid error when script is missing

* feat: add file_store support for cache manager

* [fix] fix multi gpu transfer

* [fix] fix global kvcache transfer

* [Feature] [KVCache] support file_store kv cache backend

* chore: update FileStore according to PR comments

* fix: remove comments

* fix: add swap_cache_layout for file store

* fix: remove rank key

* fix: Switch KV cache storage to pure file mode

* Temporarily disable support for Tensor types

* fix: remove args --kvcache_file_path & add envs FILE_BACKEND_STORAGE_DIR

* fixx: Simplify cache_transfer_manager.py

* fix: fix syntax bug

* fix: Simplify file_store.py

* fix: Use the key directly as the filename

* fix: Simplify set()

* fix: Simplify cache_transfer_manager.py & file_store.py

* fix: Only support load to cpu buffer

* feat: add FileStore backend for cache transfer

* fix: guard zmq import
This commit is contained in:
Moonchild1227
2026-02-03 14:37:58 +08:00
committed by GitHub
parent ee77ff9ebe
commit 39dc4b0c2e
6 changed files with 275 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
fastdeploy/engine/args_utils.py
+1 -1
View File
@@ -1086,7 +1086,7 @@ class EngineArgs:
cache_group.add_argument(
"--kvcache-storage-backend",
type=nullable_str,
choices=["mooncake", "attention_store"],
choices=["mooncake", "attention_store", "file"],
default=EngineArgs.kvcache_storage_backend,
help="The storage backend for kvcache storage. Leave empty to disable.",
)