apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2026-04-23 08:21:53 +08:00
Code Issues Actions 19 Packages Projects Releases Wiki Activity
Files
ecc50321760a716cf01e213630aa8ef36a6e744e
FastDeploy/custom_ops/gpu_ops
T
History
gongweibao 30f9f33f34 [Feature][BugFix][OP] Enhance Deterministic Inference Mode with Kernel-level Fixes and Batch-invariant BMM (#6610) 
* add fa deter

* add ut

* add long sentence

* fix basic

* fix bugs

* fix adn

* fix first

* fix single

* fix single

* fix single test

* refine

* add more test

* refine comments

* add comments of bmm

* fix ci

* remove probe

* add

* remove not need

* refine tests

* fix comments and refine code

* refine code

* refine test

* refine test

* mv 4cards tests

* fix tests

* add

* fix comments

* fix cover

* fix cover

---------

Co-authored-by: gongweibao <gognweibao@baidu.com>
2026-03-09 10:27:53 +08:00
..
append_attn
[Feature][BugFix][OP] Enhance Deterministic Inference Mode with Kernel-level Fixes and Batch-invariant BMM (#6610)
2026-03-09 10:27:53 +08:00
common
init (#6642)
2026-03-04 21:55:31 +08:00
custom_all_reduce
[Feature] Add Deterministic Inference Support (#6476)
2026-02-26 19:31:51 -08:00
cutlass_extensions
init (#6642)
2026-03-04 21:55:31 +08:00
cutlass_kernels
init (#6642)
2026-03-04 21:55:31 +08:00
flash_mask_attn
remove unneeded para from flash_mask_attention (#6218)
2026-01-27 14:04:27 +08:00
fp8_gemm_with_cutlass
init (#6642)
2026-03-04 21:55:31 +08:00
glog
init (#6642)
2026-03-04 21:55:31 +08:00
int8_gemm_with_cutlass
init (#6642)
2026-03-04 21:55:31 +08:00
machete
init (#6642)
2026-03-04 21:55:31 +08:00
mla_attn
init (#6642)
2026-03-04 21:55:31 +08:00
moba_attn
init (#6642)
2026-03-04 21:55:31 +08:00
moe
init (#6642)
2026-03-04 21:55:31 +08:00
quantization
init (#6642)
2026-03-04 21:55:31 +08:00
sample_kernels
init (#6642)
2026-03-04 21:55:31 +08:00
speculate_decoding
[Model Runner] Deprecate not_need_stop (#6356)
2026-03-05 10:55:42 +08:00
w4afp8_gemm
[BugFix] fix wint2 (#6109)
2026-01-20 21:46:21 +08:00
wfp8afp8_sparse_gemm
init (#6642)
2026-03-04 21:55:31 +08:00
append_attention.cu
[Feature]Supports SWA based on appendattn (#6547)
2026-03-01 19:02:08 +08:00
beam_search_softmax.cu
[Feature][BugFix][OP] Enhance Deterministic Inference Mode with Kernel-level Fixes and Batch-invariant BMM (#6610)
2026-03-09 10:27:53 +08:00
cpp_extensions.cc
[Model Runner] Deprecate not_need_stop (#6356)
2026-03-05 10:55:42 +08:00
cuda_multiprocess.h
init (#6642)
2026-03-04 21:55:31 +08:00
dequant_int8.cu
init (#6642)
2026-03-04 21:55:31 +08:00
enforce_generation.cu
init (#6642)
2026-03-04 21:55:31 +08:00
env.h
init (#6642)
2026-03-04 21:55:31 +08:00
fused_get_rotary_embedding.cu
init (#6642)
2026-03-04 21:55:31 +08:00
fused_hadamard_quant_fp8.cu
init (#6642)
2026-03-04 21:55:31 +08:00
fused_mask_swiglu_fp8_quant_kernel.cu
[Optimize] optimize mask_quant & swiglu (#6222)
2026-02-02 13:52:38 +08:00
fused_neox_rope_embedding.cu
[Feature] Optim PaddleOCR-VL (#4873)
2025-11-07 14:56:44 +08:00
fused_rotary_position_encoding.cu
init (#6642)
2026-03-04 21:55:31 +08:00
gather_idx.cu
init (#6642)
2026-03-04 21:55:31 +08:00
gelu_tanh.cu
[Metax] adapt to the latest develop (#6282)
2026-01-29 23:21:20 -08:00
get_attn_mask_q.cu
[Optimization] Support FA2/FA3/FA4 with attn_mask_q (#6354)
2026-02-05 14:39:00 +08:00
get_data_ptr_ipc.cu
init (#6642)
2026-03-04 21:55:31 +08:00
get_img_boundaries.cc
[Models] Add Qwen3-VL Model Support (#5763)
2025-12-29 17:39:33 +08:00
get_mm_split_fuse.cc
init (#6642)
2026-03-04 21:55:31 +08:00
get_output_ep.cc
[PD Disaggregation] Distinguish the pipelines for sending kv signal in different prefill (#5514)
2025-12-12 14:05:36 +08:00
get_output_msg_with_topk.cc
init (#6642)
2026-03-04 21:55:31 +08:00
get_output.cc
[PD Disaggregation] support DP via v1 router and decouple DP and EP (#5197)
2025-12-04 15:38:43 +08:00
get_padding_offset_system.cu
init (#6642)
2026-03-04 21:55:31 +08:00
get_padding_offset.cu
fix empty get_padding_offset (#6462)
2026-02-12 12:34:23 +08:00
get_position_ids_and_mask_encoder_batch.cu
init (#6642)
2026-03-04 21:55:31 +08:00
helper.cu
[Feature][BugFix][OP] Enhance Deterministic Inference Mode with Kernel-level Fixes and Batch-invariant BMM (#6610)
2026-03-09 10:27:53 +08:00
helper.h
[Feature][BugFix][OP] Enhance Deterministic Inference Mode with Kernel-level Fixes and Batch-invariant BMM (#6610)
2026-03-09 10:27:53 +08:00
init_signal_layerwise.cc
init (#6642)
2026-03-04 21:55:31 +08:00
ipc_sent_key_value_cache_by_remote_ptr.cu
[Feature] support rdma pd dy-c8 (#5788)
2026-01-07 14:55:25 +08:00
limit_thinking_content_length.cu
[OP][Feature] 统一 limit_thinking_content_length CUDA 算子，支持回复长度限制与注入序列 (#6493)
2026-02-25 21:36:50 +08:00
merge_prefill_decode_output.cu
init (#6642)
2026-03-04 21:55:31 +08:00
msg_utils.h
init (#6642)
2026-03-04 21:55:31 +08:00
multi_head_latent_attention.cu
【Hackathon 9th No.86】autogen MoeFastHardamardImplWrapper template_instantiation (#4592)
2025-10-30 10:28:36 +08:00
ngram_mask.cu
init (#6642)
2026-03-04 21:55:31 +08:00
noaux_tc_redundant.cu
[BugFix]fix handle 4 return values from noaux_tc_redundant op (#6384)
2026-02-09 13:17:47 +08:00
noaux_tc.cu
init (#6642)
2026-03-04 21:55:31 +08:00
noauxtc_kernel.h
[Others] clean code (#5691)
2025-12-24 11:28:47 +08:00
open_shm_and_get_meta_signal.cc
init (#6642)
2026-03-04 21:55:31 +08:00
per_token_quant_fp8.cu
[Feature] FD_USE_PHI_FP8_QUANT (#6320)
2026-02-03 22:33:03 -08:00
read_data_ipc.cu
init (#6642)
2026-03-04 21:55:31 +08:00
read_ids.py
…
read_temp_ids.py
…
reasoning_phase_token_constraint.cu
[Feature] GPU Memory Optimization and Retirement of V0 Scheduler (#6407)
2026-02-28 15:07:43 +08:00
rebuild_padding.cu
fix bug (#6422)
2026-02-10 14:58:50 +08:00
recover_decode_task.cu
init (#6642)
2026-03-04 21:55:31 +08:00
remote_cache_kv_ipc.cc
init (#6642)
2026-03-04 21:55:31 +08:00
remote_cache_kv_ipc.h
[PD Disaggregation] Distinguish the pipelines for sending kv signal in different prefill (#5514)
2025-12-12 14:05:36 +08:00
save_output_msg_with_topk.cc
[Optim] Robust sync status when preempted happens (#5796)
2026-01-14 12:07:33 +08:00
save_with_output_msg.cc
[Optim] Robust sync status when preempted happens (#5796)
2026-01-14 12:07:33 +08:00
save_with_output_msg.h
init (#6642)
2026-03-04 21:55:31 +08:00
save_with_output.cc
init (#6642)
2026-03-04 21:55:31 +08:00
scaled_gemm_f8_i4_f16_gemm.cu
init (#6642)
2026-03-04 21:55:31 +08:00
scaled_gemm_f8_i4_f16_weight_quantize.cu
init (#6642)
2026-03-04 21:55:31 +08:00
seqs2seqs.cu
init (#6642)
2026-03-04 21:55:31 +08:00
set_data_ipc.cu
[BugFix] fix shm opened but not closed in set_data_ipc (#5826)
2025-12-29 23:35:07 +08:00
set_flags.cu
init (#6642)
2026-03-04 21:55:31 +08:00
set_mask_value.cu
init (#6642)
2026-03-04 21:55:31 +08:00
set_value_by_flags_and_idx.cu
[Feature] GPU Memory Optimization and Retirement of V0 Scheduler (#6407)
2026-02-28 15:07:43 +08:00
share_external_data.cu
init (#6642)
2026-03-04 21:55:31 +08:00
step_reschedule.cu
init (#6642)
2026-03-04 21:55:31 +08:00
step_system_cache.cu
init (#6642)
2026-03-04 21:55:31 +08:00
step.cu
init (#6642)
2026-03-04 21:55:31 +08:00
stop_generation_multi_ends.cu
[Feature] GPU Memory Optimization and Retirement of V0 Scheduler (#6407)
2026-02-28 15:07:43 +08:00
stop_generation.cu
init (#6642)
2026-03-04 21:55:31 +08:00
swap_cache_batch.cu
[Feature] dyc8 support prefixcache (#5125)
2025-11-21 19:46:26 +08:00
swap_cache_layout.cu
[KVCache] Storage cache supports c8 model (#6298)
2026-02-06 12:01:17 +08:00
swap_cache.cu
init (#6642)
2026-03-04 21:55:31 +08:00
system2group.cu
init (#6642)
2026-03-04 21:55:31 +08:00
text_image_gather_scatter.cu
…
text_image_index_out.cu
init (#6642)
2026-03-04 21:55:31 +08:00
token_penalty_multi_scores.cu
[Feature][BugFix][OP] Enhance Deterministic Inference Mode with Kernel-level Fixes and Batch-invariant BMM (#6610)
2026-03-09 10:27:53 +08:00
token_penalty_only_once.cu
init (#6642)
2026-03-04 21:55:31 +08:00
token_transfer.hpp
init (#6642)
2026-03-04 21:55:31 +08:00
transfer_output.cc
init (#6642)
2026-03-04 21:55:31 +08:00
tune_cublaslt_gemm.cu
init (#6642)
2026-03-04 21:55:31 +08:00
unset_data_ipc.cu
init (#6642)
2026-03-04 21:55:31 +08:00
update_attn_mask_offsets.cu
[Speculative Decoding] Support suffix decoding (#6403)
2026-02-26 11:42:05 +08:00
update_inputs_beam.cu
…
update_inputs_v1.cu
[Model Runner] Refactor execute_model for GPU async scheduling (#6176)
2026-01-28 14:19:33 +08:00
update_inputs.cu
[Model Runner] Refactor execute_model for GPU async scheduling (#6176)
2026-01-28 14:19:33 +08:00
update_split_fuse_input.cu
init (#6642)
2026-03-04 21:55:31 +08:00