apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2026-04-24 01:29:57 +08:00
Code Issues Actions 11 Packages Projects Releases Wiki Activity
Files
9018ccf74e96fbe4841e9d33dcf9a3f521a9b5dc
FastDeploy/custom_ops/gpu_ops
T
History
freeliuzc 9018ccf74e [Speculative Decoding] Fix attn_mask_offset for multi-step MTP in mixed and PD-split modes (#5738) 
* fix attn_mask_offset in mtp with multi-step and pd-split-mode

* fix xpu operater register

* update pmtp multi-step mtp strategy in d-split -mode

* add note

* fix xpu register
2025-12-25 01:54:59 -08:00
..
append_attn
support glm fa3 (#5586)
2025-12-16 19:33:27 +08:00
common
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
custom_all_reduce
[BugFix] Fix custom_all_reduce overflow (#5662)
2025-12-19 18:24:21 +08:00
cutlass_extensions
fix typos (#3684)
2025-09-01 17:50:17 +08:00
cutlass_kernels
[Feature] Support fusedmoe on Blackwell (#5325)
2025-12-16 11:58:50 +08:00
flash_mask_attn
[Feature] support flash_mask_attention backend (#5134)
2025-11-28 10:12:16 +08:00
fp8_gemm_with_cutlass
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
glog
…
int8_gemm_with_cutlass
fix typos (#3684)
2025-09-01 17:50:17 +08:00
machete
WINT4/WINT8 dense gemm default use Machete (#4451)
2025-10-23 17:57:59 +08:00
mla_attn
[PD Disaggregation] Support PD deployment of DeepSeekv3. (#5251)
2025-12-02 14:11:50 +08:00
moba_attn
enhance set_stop_value_multi_ends and standardize the registration of some operators (#4525)
2025-10-21 22:06:06 +08:00
moe
[FIX BUG] fix bug in TP in permute_x_fp8_kernel (#5350)
2025-12-03 05:17:37 -08:00
quantization
[OPs] MoE support wfp8afp8(channelwise) and improve per_token_quant_fp8 (#4238)
2025-09-24 16:39:51 +08:00
sample_kernels
[Metax] refactor cutlass moe and optimize flash attention (#5361)
2025-12-10 17:15:17 +08:00
speculate_decoding
[Speculative Decoding] Fix attn_mask_offset for multi-step MTP in mixed and PD-split modes (#5738)
2025-12-25 01:54:59 -08:00
w4afp8_gemm
fix w4afp8 (#5634)
2025-12-22 13:39:41 +08:00
wfp8afp8_sparse_gemm
【New Feature】支持Fp8 group Gemm 24稀疏 (#3463)
2025-08-19 02:54:47 -07:00
append_attention.cu
[Others]get_block_shape_and_split_kv_block clean code (#5123)
2025-11-20 16:40:04 +08:00
beam_search_softmax.cu
…
cpp_extensions.cc
[Speculative Decoding] Fix attn_mask_offset for multi-step MTP in mixed and PD-split modes (#5738)
2025-12-25 01:54:59 -08:00
cuda_multiprocess.h
…
dequant_int8.cu
…
enforce_generation.cu
…
env.h
【Inference Optimize】DeepSeek-V3-model MLA Optimize (#3886)
2025-09-11 10:46:09 +08:00
fused_get_rotary_embedding.cu
rename fused_get_rope.cu (#3752)
2025-09-03 10:54:34 +08:00
fused_hadamard_quant_fp8.cu
[Optimize] Optimize tensorwise fp8 performance (#2729)
2025-07-07 20:06:28 +08:00
fused_neox_rope_embedding.cu
[Feature] Optim PaddleOCR-VL (#4873)
2025-11-07 14:56:44 +08:00
fused_rotary_position_encoding.cu
[Feature] DeepseekV3 use pd_build_static_op (#2948)
2025-07-22 15:03:41 +08:00
gather_idx.cu
…
gelu_tanh.cu
[Feature] Optim PaddleOCR-VL (#4873)
2025-11-07 14:56:44 +08:00
get_data_ptr_ipc.cu
Sync v2.0 version of code to github repo
2025-06-29 23:29:37 +00:00
get_img_boundaries.cc
enhance set_stop_value_multi_ends and standardize the registration of some operators (#4525)
2025-10-21 22:06:06 +08:00
get_mm_split_fuse.cc
enhance set_stop_value_multi_ends and standardize the registration of some operators (#4525)
2025-10-21 22:06:06 +08:00
get_output_ep.cc
[PD Disaggregation] Distinguish the pipelines for sending kv signal in different prefill (#5514)
2025-12-12 14:05:36 +08:00
get_output_msg_with_topk.cc
ep support logprob (#4089) (#4151)
2025-09-19 14:07:31 +08:00
get_output.cc
[PD Disaggregation] support DP via v1 router and decouple DP and EP (#5197)
2025-12-04 15:38:43 +08:00
get_padding_offset_system.cu
…
get_padding_offset.cu
[Iluvatar] Support V1_KVCACHE_SCHEDULER and paddleocr-vl rope mode (#5555)
2025-12-18 02:14:25 -08:00
get_position_ids_and_mask_encoder_batch.cu
[Feature] DeepseekV3 use pd_build_static_op (#2948)
2025-07-22 15:03:41 +08:00
helper.cu
[Feature][OP] Append Attn Support CUDA-PDL (#5072)
2025-11-17 20:47:33 +08:00
helper.h
check METAX_GPU (#5114)
2025-11-19 16:02:21 +08:00
init_signal_layerwise.cc
…
ipc_sent_key_value_cache_by_remote_ptr.cu
[BugFix]remove redundant includes (#4312)
2025-09-30 17:54:19 +08:00
limit_thinking_content_length_v1.cu
[BugFix] Fix limit_thinking early return logic in CUDA kernels (#5471)
2025-12-10 11:03:19 +08:00
limit_thinking_content_length_v2.cu
[BugFix] Fix limit_thinking early return logic in CUDA kernels (#5471)
2025-12-10 11:03:19 +08:00
merge_prefill_decode_output.cu
集中式支持fa3 (#3112)
2025-08-01 18:03:36 +08:00
msg_utils.h
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
multi_head_latent_attention.cu
【Hackathon 9th No.86】autogen MoeFastHardamardImplWrapper template_instantiation (#4592)
2025-10-30 10:28:36 +08:00
ngram_mask.cu
…
noaux_tc_redundant.cu
[Feature] Support noaux for eplb (#5143)
2025-11-21 14:10:32 +08:00
noaux_tc.cu
Fix noaux_tc cuda Error 700 in CUDAGraph (#4174)
2025-09-23 18:41:33 +08:00
noauxtc_kernel.h
[Others] clean code (#5691)
2025-12-24 11:28:47 +08:00
open_shm_and_get_meta_signal.cc
Sync v2.0 version of code to github repo
2025-06-29 23:29:37 +00:00
per_token_quant_fp8.cu
Revert "[Feature] add ue8m0 for per_token_quant_fp8 (#5563)" (#5611)
2025-12-17 13:59:06 +08:00
read_data_ipc.cu
…
read_ids.py
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
read_temp_ids.py
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
rebuild_padding.cu
[Feature] support mtp logprob (#4464)
2025-10-20 15:18:12 +08:00
recover_decode_task.cu
support mtp in v1_scheduler mode (#3695)
2025-09-04 17:39:59 +08:00
remote_cache_kv_ipc.cc
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
remote_cache_kv_ipc.h
[PD Disaggregation] Distinguish the pipelines for sending kv signal in different prefill (#5514)
2025-12-12 14:05:36 +08:00
save_output_msg_with_topk.cc
fix typos (#3684)
2025-09-01 17:50:17 +08:00
save_with_output_msg.cc
fix typos (#3684)
2025-09-01 17:50:17 +08:00
save_with_output_msg.h
…
save_with_output.cc
…
scaled_gemm_f8_i4_f16_gemm.cu
…
scaled_gemm_f8_i4_f16_weight_quantize.cu
fix scaled_gemm_f8_i4_f16_weight_quantize input (#3685)
2025-08-29 11:04:04 +08:00
seqs2seqs.cu
…
set_data_ipc.cu
dcu adapter ernie45t (#2756)
2025-07-09 18:56:27 +08:00
set_flags.cu
…
set_mask_value.cu
…
set_value_by_flags_and_idx.cu
Optimizing the performance of think length limit using custom operators (#4279)
2025-10-20 21:09:13 +08:00
share_external_data.cu
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
step_reschedule.cu
fix typos (#3684)
2025-09-01 17:50:17 +08:00
step_system_cache.cu
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
step.cu
Adapt for iluvatar gpu (#2684)
2025-07-07 16:53:14 +08:00
stop_generation_multi_ends.cu
[Feature] support stop_token_ids (#5399)
2025-12-09 17:49:12 +08:00
stop_generation.cu
…
swap_cache_batch.cu
[Feature] dyc8 support prefixcache (#5125)
2025-11-21 19:46:26 +08:00
swap_cache_layout.cu
[Feature] Support KV Cache Storage (#5571)
2025-12-25 16:30:35 +08:00
swap_cache.cu
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
system2group.cu
…
text_image_gather_scatter.cu
[Graph Optimization] Fix IR graph dependency error exposed after enabling SOT by updating the return value of TextImageGatherScatter (#4610)
2025-10-28 18:31:23 +08:00
text_image_index_out.cu
[Feat] ernie4_5_vl_moe support CudaGraph (#3226)
2025-09-10 13:11:57 +08:00
token_penalty_multi_scores.cu
[Feature] support bad_words (#3055)
2025-07-30 09:31:29 +08:00
token_penalty_only_once.cu
…
token_transfer.hpp
fix typos (#3684)
2025-09-01 17:50:17 +08:00
transfer_output.cc
…
tune_cublaslt_gemm.cu
polish code with new pre-commit rule (#2923)
2025-07-19 23:19:27 +08:00
unset_data_ipc.cu
[feat] support prefix cache clearing when /clear_load_weight is called (#4008)
2025-09-28 19:42:53 +08:00
update_attn_mask_offsets.cu
[BugFix]Fix attention mask bug in D-Node of PD-split mode (#5245)
2025-11-26 17:56:28 +08:00
update_inputs_beam.cu
[XPU] bind some OPs for VL model with pybind (#4522)
2025-10-27 10:50:08 +08:00
update_inputs_v1.cu
check (#5736)
2025-12-24 16:49:20 +08:00
update_inputs.cu
[XPU] bind some OPs for VL model with pybind (#4522)
2025-10-27 10:50:08 +08:00
update_split_fuse_input.cu
fix typos (#3684)
2025-09-01 17:50:17 +08:00