apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2026-04-23 00:17:25 +08:00
Code Issues Actions 19 Packages Projects Releases Wiki Activity
4,979 Commits 41 Branches 17 Tags
bb48bcbaa2d164f3a65d9d55e42cf140877bc740
Commit Graph

52 Commits

Author SHA1 Message Date
bukejiyu f142b486c9 update (#7101) 2026-04-02 16:07:26 +08:00
bukejiyu 586e6f38b1 [Others]Limit transformers version (#6806) 2026-03-12 20:20:15 -07:00
bukejiyu cffa8c246c [Others]update paddleformer 1.0.0 (#6496) 
* update paddleformer 1.0.0

* update
 2026-03-11 15:06:29 +08:00
Jiang-Jia-Jun ae5e872957 Modify gunicorn version in requirements.txt 
Updated gunicorn version specification in requirements.txt.
 2026-02-28 14:15:51 +08:00
Jiang-Jia-Jun 39a5ea66c8 [BugFix] Enable control socket disable option in API server (#6545) 
* [BugFix] Enable control socket disable option in API server

* Update requirements.txt

* Update requirements.txt
 2026-02-28 10:35:35 +08:00
GoldPancake 2178f2829b [Speculative Decoding] Support suffix decoding (#6403) 
* support suffix decoding
 2026-02-26 11:42:05 +08:00
YuBaoku 1405d7d5d7 [CI] Pin gunicorn version to 25.0.3 (#6497) 2026-02-25 09:52:22 +08:00
chen 29a313a402 [Optimization] Support FA2/FA3/FA4 with attn_mask_q (#6354) 
* support FA4 sm100

* flash attn backend support mask

* flash attn backend run flashmask correct

* add test for flash_attn_backend and flash_attn_func

* check

* add test for fa4

* requirements.txt add fa4 whl

* check test on sm100

* fix CI conflict

* add enable_torch_proxy for flash_mask

* lazy import fa4

* check

* fix tests import

* check test_load_mpt import
 2026-02-05 14:39:00 +08:00
RAM b3f59fd9b5 [RL][CI] Support Async R3 And Add Accuracy Test (#5937) 
* add bs1 r3 test case

* async put

* r3 test case 1.0

* success run eb5

* refine test case

* pre-commit

* add eb45 & glm testcase

* format code

* add p2pstore requirements

* support only last turn

* R3 use worker log

* refine code &fix ci bug

* refine error mesg

* fix empty input bug

* Success set acc ci of eb45 and glm45

* refine code

* fix bug
 2026-01-14 04:25:06 -08:00
luukunn 93b7675a64 [Feature]Report FD statistical information (#5646) 
* add usage commit

* update envs and xpu

* add requirements

* fix quantization value

* add unit test

* add unit test

* fix unit test

* add unit test

* add unit test

* add unit test

* add unit test

* add unit test

* add unit test

* fix FD_USAGE_STATS_SERVER

* fix

* fix

* add doc

* add doc

* add doc

* add doc

* add doc

* fix file name
 2026-01-14 17:54:01 +08:00
Bingoo 8e11d719f3 add flashinfer-python-paddle depend (#5912) 
Co-authored-by: Jiaxin Sui <95567040+plusNew001@users.noreply.github.com>
 2026-01-08 15:08:35 +08:00
bukejiyu d1c6e57341 [Others] upgrade paddleformer to 0.4.0 (#5599) 2025-12-23 05:08:01 -08:00
RAM 131defa122 Revert "Revert "[Feature] Use paddle.compat.enable_torch_proxy in `fastdepl…" (#5606) 
This reverts commit 021399f7c9.
 2025-12-22 22:37:51 +08:00
Jiang-Jia-Jun 021399f7c9 Revert "[Feature] Use paddle.compat.enable_torch_proxy in `fastdeploy/__ini…" (#5579) 
This reverts commit ff45ac078e.
 2025-12-16 13:55:27 +08:00
Jundong Liu ff45ac078e [Feature] Use paddle.compat.enable_torch_proxy in fastdeploy/__init__.py (#5211) 
* test feature

* fix xgrammar

* fix paddleformer

* try whitelist

* manual patch PaddlePaddle/Paddle#76706 for test

* remove triton version

* add comment

* Update scripts/run_ci_xpu.sh

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* remove use-triton-in-paddle in requirement.txt

---------

Co-authored-by: Nyakku Shigure <sigure.qaq@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-12-16 11:05:30 +08:00
Yuanle Liu 41c63f6056 remove fastsafetensors (#5371) 2025-12-04 19:22:04 +08:00
Echo-Nie 96ff402d44 [Optimization] Remove version constraints for setuptools, uvicorn, triton and safetensors, del fastsafetensors (#5330) 
* Remove version constraints for setuptools, triton, and fastsafetensors.

* remove version for uvicorn

* fix according to review
 2025-12-04 10:07:31 +08:00
qwes5s5 117980dd4e [LogProbs]Enable prompt logprobs output and modify data transmission method for the online interface. (#5089) 
* add prompt logprobs

* Merge prompt_logprobs_tensors and prompt_logprobs

* fix param check

* trigger ci

* fix unitest

* fix logprobs bug
 2025-12-02 13:49:51 +08:00
Yuanle Liu ef5aa5c03b [BugFix] fix cuda-python requirement (#5261) 
* fix cuda-python requirement

* update

* fix
 2025-11-27 13:58:18 +08:00
Yuanle Liu 66e096d509 [FDConfig] disable use_sequence_parallel_moe default (#5222
CE Compile Job / ce_job_pre_check (push) Has been cancelled
Details
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
Details
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
Details
CE Compile Job / Show Code Archive Output (push) Has been cancelled
Details
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
Details
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
Details
CE Compile Job / CE_UPLOAD (push) Has been cancelled
Details
Deploy GitHub Pages / deploy (push) Has been cancelled
Details
Publish Job / publish_pre_check (push) Has been cancelled
Details
Publish Job / print_publish_pre_check_outputs (push) Has been cancelled
Details
Publish Job / FD-Clone-Linux (push) Has been cancelled
Details
Publish Job / Show Code Archive Output (push) Has been cancelled
Details
Publish Job / BUILD_SM8090 (push) Has been cancelled
Details
Publish Job / BUILD_SM8689 (push) Has been cancelled
Details
Publish Job / PADDLE_PYPI_UPLOAD_8090 (push) Has been cancelled
Details
Publish Job / PADDLE_PYPI_UPLOAD_8689 (push) Has been cancelled
Details
Publish Job / Run FD Image Build (push) Has been cancelled
Details
Publish Job / Run FastDeploy Unit Tests and Coverage (push) Has been cancelled
Details
Publish Job / Run FastDeploy LogProb Tests (push) Has been cancelled
Details
Publish Job / Extracted partial CE model tasks to run in CI. (push) Has been cancelled
Details
Publish Job / Run Base Tests (push) Has been cancelled
Details
Publish Job / Run Accuracy Tests (push) Has been cancelled
Details
Publish Job / Run Stable Tests (push) Has been cancelled
Details
CI Images Build / FD-Clone-Linux (push) Has been cancelled
Details
CI Images Build / Show Code Archive Output (push) Has been cancelled
Details
CI Images Build / CI Images Build (push) Has been cancelled
Details
CI Images Build / BUILD_SM8090 (push) Has been cancelled
Details
CI Images Build / Run FastDeploy Unit Tests and Coverage (push) Has been cancelled
Details
CI Images Build / Run FastDeploy LogProb Tests (push) Has been cancelled
Details
CI Images Build / Extracted partial CE model tasks to run in CI. (push) Has been cancelled
Details
CI Images Build / Run Base Tests (push) Has been cancelled
Details
CI Images Build / Publish Docker Images Pre Check (push) Has been cancelled
Details 
* disable use_sequence_parallel_moe default

* update
 2025-11-25 21:49:10 +08:00
kevin 8e4e3ff510 [Feature] support eplb in api_server (#4782) 
* support eplb in api_server

* update code

* add eplb test case

* update eplb

* support tp+dp eplb

* update test cese

* update code

* update code

* fix bug

* update copilot review

* update test case name
 2025-11-24 20:22:29 +08:00
qwes5s5 36216e62f0 [Log] Add trace log and add loggingInstrumentor tool (#4692) 
* add trace logger and trace print

* trigger ci

* fix unittest

* translate notes and add copyright

---------

Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>
Co-authored-by: YuBaoku <49938469+EmmonsCurse@users.noreply.github.com>
 2025-11-17 11:08:57 +08:00
bukejiyu b09ebb2813 refactor pt loading (#4532)
CE Compile Job / ce_job_pre_check (push) Has been cancelled
Details
CE Compile Job / print_ce_job_pre_check_outputs (push) Has been cancelled
Details
CE Compile Job / FD-Clone-Linux (push) Has been cancelled
Details
CE Compile Job / Show Code Archive Output (push) Has been cancelled
Details
CE Compile Job / BUILD_SM8090 (push) Has been cancelled
Details
CE Compile Job / BUILD_SM8689 (push) Has been cancelled
Details
CE Compile Job / CE_UPLOAD (push) Has been cancelled
Details
Deploy GitHub Pages / deploy (push) Has been cancelled
Details
Publish Job / publish_pre_check (push) Has been cancelled
Details
Publish Job / print_publish_pre_check_outputs (push) Has been cancelled
Details
Publish Job / FD-Clone-Linux (push) Has been cancelled
Details
Publish Job / Show Code Archive Output (push) Has been cancelled
Details
Publish Job / BUILD_SM8090 (push) Has been cancelled
Details
Publish Job / BUILD_SM8689 (push) Has been cancelled
Details
Publish Job / PADDLE_PYPI_UPLOAD_8090 (push) Has been cancelled
Details
Publish Job / PADDLE_PYPI_UPLOAD_8689 (push) Has been cancelled
Details
Publish Job / Run FD Image Build (push) Has been cancelled
Details
Publish Job / Run FastDeploy Unit Tests and Coverage (push) Has been cancelled
Details
Publish Job / Run FastDeploy LogProb Tests (push) Has been cancelled
Details
Publish Job / Extracted partial CE model tasks to run in CI. (push) Has been cancelled
Details
Publish Job / Run Base Tests (push) Has been cancelled
Details
Publish Job / Run Accuracy Tests (push) Has been cancelled
Details
Publish Job / Run Stable Tests (push) Has been cancelled
Details
CI Images Build / FD-Clone-Linux (push) Has been cancelled
Details
CI Images Build / Show Code Archive Output (push) Has been cancelled
Details
CI Images Build / CI Images Build (push) Has been cancelled
Details
CI Images Build / BUILD_SM8090 (push) Has been cancelled
Details
CI Images Build / Run FastDeploy Unit Tests and Coverage (push) Has been cancelled
Details
CI Images Build / Run FastDeploy LogProb Tests (push) Has been cancelled
Details
CI Images Build / Extracted partial CE model tasks to run in CI. (push) Has been cancelled
Details
CI Images Build / Run Base Tests (push) Has been cancelled
Details
CI Images Build / Publish Docker Images Pre Check (push) Has been cancelled
Details
2025-11-11 21:30:39 +08:00
Juncai 08ca0f6aea [Feature] [PD] add simple router and refine splitwise deployment (#4709) 
* add simple router and refine splitwise deployment

* fix
 2025-11-06 14:56:02 +08:00
zhang-prog 24b9505971 add einops dependency (#4633) 2025-10-28 22:17:13 +08:00
Yuanle Liu 3b58310c26 enhance set_stop_value_multi_ends and standardize the registration of some operators (#4525) 
* fix custom_ops

* paddleformers>=0.3.1
 2025-10-21 22:06:06 +08:00
ltd0924 d8841b7b40 [BugFix] fix workers=1 (#4364) 
* [Feature] support prefix cache in DP

* fix

* Update common_engine.py

* Update common_engine.py

* Update common_engine.py

* Update common_engine.py

* [BugFix] fix workers more than 1

* fix

* Update api_server.py

* fix

* Update api_server.py

* fix

---------

Co-authored-by: ltd0924 <luotingdan@baidu.com>
 2025-10-15 17:06:25 +08:00
lizexu123 c86945ef49 [Feature] support pool (#3827) 
* support pool

* update pooling

* add pooler_config and check

* update

* support AutoWeightsLoader load weight

* fix

* update

* delete print

* update pre-commit

* fix

* fix xpu

* fix ModelRegistry->model_registry

* fix Copilot review

* fix pooler.py

* delete StepPooler

* fix abstract

* fix default_loader_v1

* fix Pre Commit

* support torch qwen3 dense

* add test and fix torch-qwen

* fix

* fix

* adapter ci:

* fix review

* fix pooling_params.py

* fix

* fix tasks.py 2025

* fix print and logger

* Modefy ModelRegistry and delete AutoWeightsLoader

* fix logger

* fix test_embedding

* fix ci bug

* ernie4_5 model_registry

* fix test

* support Qwen3-Embedding-0.6B tp=1 load

* fix extra code

* fix

* delete fix vocab_size

* delete prepare_params_dict

* fix:
 2025-09-22 14:09:09 +08:00
YuBaoku c7b7126b20 [CI] update paddleformers==0.2 in develop (#3878) 2025-09-04 20:12:41 +08:00
YuBaoku 65425bf858 [CI] update paddle version to nightly (#3698) 2025-08-29 18:16:13 +08:00
luukunn eda83ca672 add Tool Parser (#3272)
Deploy GitHub Pages / deploy (push) Has been cancelled
Details 
* add tool-parser

* add tool-parser

* add tool parser

* add tool parser

* fix

* add offline

* add offline

* fix

* parsers:tool&reasoning

* 修改tool parser名称·

* update

* fix reasoning-parser

* add requirements

* fix finish reason

* fix

* fix reasoning-parser

* fix

* fix

* fix

* fix

* fix

---------

Co-authored-by: zhuzixuan <zhuzixuan@baidu.com>
 2025-08-13 01:06:55 +08:00
kevin 9627619235 fix uvicorn multi worker error (#3300) 
Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>
 2025-08-11 19:39:41 +08:00
Yzc216 d9e3f88f9e [Feature] multi source download (#3125)
Deploy GitHub Pages / deploy (push) Has been cancelled
Details 
* multi-source download

* multi-source download

* huggingface download revision

* requirement

* style

* add revision arg

* test

* pre-commit

* Change default download

* change requirements.txt

* modify English Documentation

* documentation

* modify model download path

* add requirements

* error optimization

* 连接失败兜底

* 连接失败兜底

* 连接失败兜底

* unit test

* unit test

* unit test

* test

* test
 2025-08-07 00:40:27 +08:00
Yzc216 980126b83a [Feature] multi source download (#3005) 
* multi-source download

* multi-source download

* huggingface download revision

* requirement

* style

* add revision arg

* test

* pre-commit

* Change default download

* change requirements.txt

* modify English Documentation

* documentation
 2025-07-24 17:42:09 +08:00
Yzc216 e14587a954 [Feature] multi-source download (#2986) 
* multi-source download

* multi-source download

* huggingface download revision

* requirement

* style

* add revision arg

* test

* pre-commit
 2025-07-24 14:26:37 +08:00
Jiang-Jia-Jun fbe3547c95 [Feature] Support include_stop_str_in_output in chat/completion (#2910) 
* [Feature] Support include_stop_str_in_output in chat/completion

* Add ci test for include_stop_str_in_output

* Update version of openai

* Fix ci test

---------

Co-authored-by: Jiang-Jia-Jun <jiangjiajun@baidu.com>
 2025-07-18 16:59:18 +08:00
sg263 52aca233e8 [Trace] fix annotation when add opentelemetry (#2869) 
* add opentelemetry

* add opentelemetry

* add opentelemetry on dequeue

* add opentelemetry on dequeue

* add opentelemetry on dequeue

* fix annotation

* fix annotation when add opentelemetry

* fix opentelemetry-instrumentation-fastapi

* fix pentelemetry-bootstrap

---------

Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>
 2025-07-17 10:29:16 +08:00
ltd0924 d245d1ca6c [LLM] support send batch data and aggregate data (#2860) 
* [LLM] support send batch data and aggregate data

* [LLM] fix ci bugs

* [LLM] fix ci bugs

* [LLM] fix ci bugs

* [LLM] fix ci bugs

* [LLM] update
 2025-07-16 23:42:20 +08:00
sg263 42b80182e0 [Trace] add opentelemetry (#2852) 
* add opentelemetry

* add opentelemetry

* add opentelemetry on dequeue

* add opentelemetry on dequeue

* add opentelemetry on dequeue

---------

Co-authored-by: Jiang-Jia-Jun <163579578+Jiang-Jia-Jun@users.noreply.github.com>
 2025-07-16 15:33:25 +08:00
Jiang-Jia-Jun 05c670e593 [Sync] Update to latest code (#2679) 
* [Sync] Update to latest code

* Add new code files

* Add new code files

* update code

* Try to fix build.sh

* Try to fix build.sh

* Update code

* Update requirements.txt

* Update code

---------

Co-authored-by: Jiang-Jia-Jun <jiangjiajun@baidu.com>
 2025-07-03 15:43:53 +08:00
Jiang-Jia-Jun 92c2cfa2e7 Sync v2.0 version of code to github repo 2025-06-29 23:29:37 +00:00
Jiang-Jia-Jun d151496038 Add requirement 2025-06-29 19:11:44 +00:00
jiangjiajun 684703fd72 [LLM] First commit the llm deployment code 2025-06-09 19:20:15 +08:00
Jason 68523be411 Modify file structure to separate python and cpp code (#223) 
Modify code structure
 2022-09-14 15:44:13 +08:00
DefTruth beddcba900 [docs][win] add windows c++ sdk demo to examples (#136) 
* [docs] format docs with markdown with language tags

* [docs][win] add windows c++ sdk demo

* [docs][win] add windows c++ sdk demo to examples

* [docs][api] update runtime_option docs
 2022-08-22 10:53:00 +08:00
Jason 79c3fe3ded Update requirements.txt 2022-08-18 19:30:36 +08:00
Jason 8e7b540d3f Fix python usage on Windows (#60) 
* Polish compilation on Windows

* Fix python usage on Windows
 2022-07-31 16:50:56 +08:00
Jason ed3d6f2187 Fix requirements (#59) 
* Fix bug in ppyoloe

* fix ppyoloe output format

* remove some requirements

* fix conflicts
 2022-07-31 15:05:30 +08:00
huangjianhui 83b8350b96 Fix evaluation import problem (#46) 
* Detection evaluation function

* Add license

* Fix python import problem

* Modify requirement.txt

* Add requirements.txt

Co-authored-by: Jason <jiangjiajun@baidu.com>
 2022-07-27 16:42:45 +08:00
jiangjiajun b69f13b268 Make api of yolov5 be same bettwen Python and C++ 2022-07-06 03:57:40 +00:00