apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2026-04-23 00:17:25 +08:00
Code Issues Actions 21 Packages Projects Releases Wiki Activity

[PD Disaggregation] Write the cache of preempted req to storage and refine PD Disaggregation (#7107)

Browse Source 
* Write the cache of preempted req to storage

* up

* fix
This commit is contained in:
jc
2026-04-01 13:15:52 +08:00
committed by GitHub
parent 3651113ee5
commit af51fc46d6
5 changed files with 35 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files
fastdeploy/envs.py
+4
View File
@@ -252,6 +252,10 @@ environment_variables: dict[str, Callable[[], Any]] = {
# When v1 is enabled, the legacy /clear_load_weight and /update_model_weight
# will adopt this new communication pattern.
"FD_ENABLE_V1_UPDATE_WEIGHTS": lambda: bool(int(os.getenv("FD_ENABLE_V1_UPDATE_WEIGHTS", "0"))),
# Whether to save the cache of output token for preempted request to storage.
"FD_SAVE_OUTPUT_CACHE_FOR_PREEMPTED_REQUEST": lambda: bool(
int(os.getenv("FD_SAVE_OUTPUT_CACHE_FOR_PREEMPTED_REQUEST", "1"))
),
}