apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2026-04-23 00:17:25 +08:00
Code Issues Actions 19 Packages Projects Releases Wiki Activity

[Feature] Support KV Cache Storage (#5571)

Browse Source 
* Support Mooncake Store

* up

* up

* add op

* fix conflict

* fix error

* up for comments

* avoid thread lock

* up

* fix unittest

* fix unittest

* remove debug info

* consider tp_size > 1

* add default rdma_nics

* add utils

* up

* fix error

---------

Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>
This commit is contained in:
Juncai
2025-12-25 16:30:35 +08:00
committed by GitHub
parent be3be4913a
commit 412867fd99
27 changed files with 1672 additions and 195 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/cache_manager/test_cache_transfer_manager.py
+2
View File
@@ -41,6 +41,8 @@ class Args:
create_cache_tensor = False
cache_dtype = "bfloat16"
default_dtype = "bfloat16"
kvcache_storage_backend = None
write_policy = "write_through"
# ==========================