FastDeploy

Check optional params before .get() call in gqa_rope_write_cache (#7311 )

2026-04-13 19:05:43 +08:00

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

Support setting communication groups in custom_allreduce and the all-to-all\transpose fused operator during the decoding phase. (#5917 )

2026-01-12 14:09:39 +08:00

cutlass_extensions

fix typos (#3684 )

2025-09-01 17:50:17 +08:00

cutlass_kernels

support moe for sm103 (#7239 )

2026-04-08 20:57:07 +08:00

flash_mask_attn

[BugFix] Fix Async D2H copy bug & flash mash atten cache V out of bound bug (#7221 ) (#7294 )

2026-04-10 13:54:09 +08:00

fp8_gemm_with_cutlass

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

glog

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

int8_gemm_with_cutlass

fix typos (#3684 )

2025-09-01 17:50:17 +08:00

machete

[BugFix] Fix wint4 ep issue caused by empty run (#5870 )

2026-01-05 14:24:37 +08:00

mla_attn

[BugFix] fix BatchMLAWithPagedKVCacheKernel O_tmp (#5895 )

2026-01-06 15:39:06 +08:00

moba_attn

enhance set_stop_value_multi_ends and standardize the registration of some operators (#4525 )

2025-10-21 22:06:06 +08:00

moe

[OP]Unify MoE op with moe_permute path for bf16 GLM (#7164 ) (#7282 )

2026-04-09 21:37:53 +08:00

quantization

solve conflict (#7135 )

2026-04-02 10:55:15 +08:00

sample_kernels

solve conflict (#7135 )

2026-04-02 10:55:15 +08:00

speculate_decoding

solve conflict (#7135 )

2026-04-02 10:55:15 +08:00

w4afp8_gemm

[BugFix] fix wint2 (#6109 )

2026-01-20 21:46:21 +08:00

wfp8afp8_sparse_gemm

【New Feature】支持Fp8 group Gemm 24稀疏 (#3463 )

2025-08-19 02:54:47 -07:00

append_attention.cu

[RL][Cherry-Pick] RoPE without fmad opt (#6901 ) (#6902 )

2026-03-25 10:42:16 +08:00

beam_search_softmax.cu

solve conflict (#7135 )

2026-04-02 10:55:15 +08:00

cccl_compat.h

solve conflict (#7135 )

2026-04-02 10:55:15 +08:00

cpp_extensions.cc

[OP]Unify MoE op with moe_permute path for bf16 GLM (#7164 ) (#7282 )

2026-04-09 21:37:53 +08:00

cuda_multiprocess.h

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

custom_ftok.h

[Cherry-Pick] [BugFix] replace ftok with custom_ftok in get_output/save_output ops (#6822 ) (#6824 )

2026-03-16 14:22:30 +08:00

dequant_int8.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

enforce_generation.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

env.h

【Inference Optimize】DeepSeek-V3-model MLA Optimize (#3886 )

2025-09-11 10:46:09 +08:00

fused_get_rotary_embedding.cu

rename fused_get_rope.cu (#3752 )

2025-09-03 10:54:34 +08:00

fused_hadamard_quant_fp8.cu

[Optimize] Optimize tensorwise fp8 performance (#2729 )

2025-07-07 20:06:28 +08:00

fused_mask_swiglu_fp8_quant_kernel.cu

[Optimize] optimize mask_quant & swiglu (#6222 )

2026-02-02 13:52:38 +08:00

fused_neox_rope_embedding.cu

[Feature] Optim PaddleOCR-VL (#4873 )

2025-11-07 14:56:44 +08:00

fused_rotary_position_encoding.cu

[Feature] DeepseekV3 use pd_build_static_op (#2948 )

2025-07-22 15:03:41 +08:00

gather_idx.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

gelu_tanh.cu

[Metax] adapt to the latest develop (#6282 )

2026-01-29 23:21:20 -08:00

get_attn_mask_q.cu

update attn_mask_q 2 (#7372 )

2026-04-13 23:34:21 +08:00

get_data_ptr_ipc.cu

Sync v2.0 version of code to github repo

2025-06-29 23:29:37 +00:00

get_img_boundaries.cc

[Models] Add Qwen3-VL Model Support (#5763 )

2025-12-29 17:39:33 +08:00

get_mm_split_fuse.cc

enhance set_stop_value_multi_ends and standardize the registration of some operators (#4525 )

2025-10-21 22:06:06 +08:00

get_output_ep.cc

[Cherry-Pick] [BugFix] replace ftok with custom_ftok in get_output/save_output ops (#6822 ) (#6824 )

2026-03-16 14:22:30 +08:00

get_output_msg_with_topk.cc

[Cherry-Pick] [BugFix] replace ftok with custom_ftok in get_output/save_output ops (#6822 ) (#6824 )

2026-03-16 14:22:30 +08:00

get_output.cc

[Cherry-Pick] [BugFix] replace ftok with custom_ftok in get_output/save_output ops (#6822 ) (#6824 )

2026-03-16 14:22:30 +08:00

get_padding_offset_system.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

get_padding_offset.cu

[Cherry-Pick 2.5][BugFix] Fix get_padding_offset in empty run (#6460 )

2026-02-11 20:21:53 +08:00

get_position_ids_and_mask_encoder_batch.cu

[Feature] DeepseekV3 use pd_build_static_op (#2948 )

2025-07-22 15:03:41 +08:00

helper.cu

[RL][Cherry-Pick] RoPE without fmad opt (#6901 ) (#6902 )

2026-03-25 10:42:16 +08:00

helper.h

[RL][Cherry-Pick] RoPE without fmad opt (#6901 ) (#6902 )

2026-03-25 10:42:16 +08:00

init_signal_layerwise.cc

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

ipc_sent_key_value_cache_by_remote_ptr.cu

[Feature] support rdma pd dy-c8 (#5788 )

2026-01-07 14:55:25 +08:00

limit_thinking_content_length.cu

[Cherry-Pick][OP][Feature] 统一 limit_thinking_content_length CUDA 算子，支持回复长度限制与注入序列 (#6511 )

2026-02-26 13:29:38 +08:00

merge_prefill_decode_output.cu

集中式支持fa3 (#3112 )

2025-08-01 18:03:36 +08:00

msg_utils.h

[Cherry-Pick] [BugFix] replace ftok with custom_ftok in get_output/save_output ops (#6822 ) (#6824 )

2026-03-16 14:22:30 +08:00

multi_head_latent_attention.cu

【Hackathon 9th No.86】autogen MoeFastHardamardImplWrapper template_instantiation (#4592 )

2025-10-30 10:28:36 +08:00

ngram_mask.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

noaux_tc_redundant.cu

[BugFix]fix handle 4 return values from noaux_tc_redundant op (#6384 )

2026-02-09 13:17:47 +08:00

noaux_tc.cu

Fix noaux_tc cuda Error 700 in CUDAGraph (#4174 )

2025-09-23 18:41:33 +08:00

noauxtc_kernel.h

[Others] clean code (#5691 )

2025-12-24 11:28:47 +08:00

open_shm_and_get_meta_signal.cc

Sync v2.0 version of code to github repo

2025-06-29 23:29:37 +00:00

per_token_quant_fp8.cu

[Feature] FD_USE_PHI_FP8_QUANT (#6320 )

2026-02-03 22:33:03 -08:00

read_data_ipc.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

read_ids.py

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

read_temp_ids.py

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

reasoning_phase_token_constraint.cu

[MTP] refactor MTP pre_process (#6358 )

2026-02-09 10:47:15 +08:00

rebuild_padding.cu

fix bug (#6425 )

2026-02-10 17:07:51 +08:00

recover_decode_task.cu

support mtp in v1_scheduler mode (#3695 )

2025-09-04 17:39:59 +08:00

remote_cache_kv_ipc.cc

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

remote_cache_kv_ipc.h

[Cherry-Pick] [BugFix] replace ftok with custom_ftok in get_output/save_output ops (#6822 ) (#6824 )

2026-03-16 14:22:30 +08:00

save_output_msg_with_topk.cc

[Cherry-Pick] [BugFix] replace ftok with custom_ftok in get_output/save_output ops (#6822 ) (#6824 )

2026-03-16 14:22:30 +08:00

save_with_output_msg.cc

[Cherry-Pick] [BugFix] replace ftok with custom_ftok in get_output/save_output ops (#6822 ) (#6824 )

2026-03-16 14:22:30 +08:00

save_with_output_msg.h

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

save_with_output.cc

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

scaled_gemm_f8_i4_f16_gemm.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

scaled_gemm_f8_i4_f16_weight_quantize.cu

fix scaled_gemm_f8_i4_f16_weight_quantize input (#3685 )

2025-08-29 11:04:04 +08:00

seqs2seqs.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

set_data_ipc.cu

[BugFix] fix shm opened but not closed in set_data_ipc (#5826 )

2025-12-29 23:35:07 +08:00

set_flags.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

set_mask_value.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

set_stop.cu

[Model Runner] Refactor execute_model for GPU async scheduling (#6176 )

2026-01-28 14:19:33 +08:00

set_value_by_flags_and_idx.cu

Optimizing the performance of think length limit using custom operators (#4279 )

2025-10-20 21:09:13 +08:00

share_external_data.cu

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

step_reschedule.cu

solve conflict (#7135 )

2026-04-02 10:55:15 +08:00

step_system_cache.cu

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

step.cu

solve conflict (#7135 )

2026-04-02 10:55:15 +08:00

stop_generation_multi_ends.cu

[Feature] support stop_token_ids (#5399 )

2025-12-09 17:49:12 +08:00

stop_generation.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

swap_cache_batch.cu

[Feature] dyc8 support prefixcache (#5125 )

2025-11-21 19:46:26 +08:00

swap_cache_layout.cu

[KVCache] Storage cache supports c8 model (#6298 )

2026-02-06 12:01:17 +08:00

swap_cache.cu

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

system2group.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

text_image_gather_scatter.cu

[Graph Optimization] Fix IR graph dependency error exposed after enabling SOT by updating the return value of TextImageGatherScatter (#4610 )

2025-10-28 18:31:23 +08:00

text_image_index_out.cu

[Feat] ernie4_5_vl_moe support CudaGraph (#3226 )

2025-09-10 13:11:57 +08:00

token_penalty_multi_scores.cu

[BugFix] Fix token_penalty kernel (#6069 )

2026-01-28 12:03:05 +08:00

token_penalty_only_once.cu

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

token_transfer.hpp

fix typos (#3684 )

2025-09-01 17:50:17 +08:00

transfer_output.cc

[LLM] First commit the llm deployment code

2025-06-09 19:20:15 +08:00

tune_cublaslt_gemm.cu

polish code with new pre-commit rule (#2923 )

2025-07-19 23:19:27 +08:00

unset_data_ipc.cu

[feat] support prefix cache clearing when /clear_load_weight is called (#4008 )

2025-09-28 19:42:53 +08:00

update_attn_mask_offsets.cu

[Cherry-Pick][Speculative Decoding] Support suffix decoding (#6403 ) (#6967 )

2026-03-23 17:33:58 +08:00

update_inputs_beam.cu

[XPU] bind some OPs for VL model with pybind (#4522 )

2025-10-27 10:50:08 +08:00

update_inputs_v1.cu

[Model Runner] Refactor execute_model for GPU async scheduling (#6176 )

2026-01-28 14:19:33 +08:00

update_inputs.cu

[Model Runner] Refactor execute_model for GPU async scheduling (#6176 )

2026-01-28 14:19:33 +08:00

update_split_fuse_input.cu

fix typos (#3684 )

2025-09-01 17:50:17 +08:00