FastDeploy

[Model Runner] Prepare token count and move FA3 initialization into the graph (#6170 )

2026-01-26 12:16:57 +08:00

…

Support setting communication groups in custom_allreduce and the all-to-all\transpose fused operator during the decoding phase. (#5917 )

2026-01-12 14:09:39 +08:00

cutlass_extensions

…

cutlass_kernels

[Feature] Support fusedmoe on Blackwell (#5325 )

2025-12-16 11:58:50 +08:00

flash_mask_attn

remove unneeded para from flash_mask_attention (#6218 )

2026-01-27 14:04:27 +08:00

fp8_gemm_with_cutlass

…

glog

…

int8_gemm_with_cutlass

…

machete

[BugFix] Fix wint4 ep issue caused by empty run (#5870 )

2026-01-05 14:24:37 +08:00

mla_attn

[BugFix] fix BatchMLAWithPagedKVCacheKernel O_tmp (#5895 )

2026-01-06 15:39:06 +08:00

moba_attn

…

moe

[Feature] Support Ernie FP8 on sm100 ( the fixed version) (#6304 )

2026-02-03 17:47:38 +08:00

quantization

…

sample_kernels

[Metax] refactor cutlass moe and optimize flash attention (#5361 )

2025-12-10 17:15:17 +08:00

speculate_decoding

remove speculate_get_padding_offset op (#6308 )

2026-02-03 15:18:12 +08:00

w4afp8_gemm

[BugFix] fix wint2 (#6109 )

2026-01-20 21:46:21 +08:00

wfp8afp8_sparse_gemm

…

append_attention.cu

[Others]get_block_shape_and_split_kv_block clean code (#5123 )

2025-11-20 16:40:04 +08:00

beam_search_softmax.cu

…

cpp_extensions.cc

[Optimization] Support FA2/FA3/FA4 with attn_mask_q (#6354 )

2026-02-05 14:39:00 +08:00

cuda_multiprocess.h

…

dequant_int8.cu

…

enforce_generation.cu

…

env.h

…

fused_get_rotary_embedding.cu

…

fused_hadamard_quant_fp8.cu

…

fused_mask_swiglu_fp8_quant_kernel.cu

[Optimize] optimize mask_quant & swiglu (#6222 )

2026-02-02 13:52:38 +08:00

fused_neox_rope_embedding.cu

…

fused_rotary_position_encoding.cu

…

gather_idx.cu

…

gelu_tanh.cu

[Metax] adapt to the latest develop (#6282 )

2026-01-29 23:21:20 -08:00

get_attn_mask_q.cu

[Optimization] Support FA2/FA3/FA4 with attn_mask_q (#6354 )

2026-02-05 14:39:00 +08:00

get_data_ptr_ipc.cu

…

get_img_boundaries.cc

[Models] Add Qwen3-VL Model Support (#5763 )

2025-12-29 17:39:33 +08:00

get_mm_split_fuse.cc

…

get_output_ep.cc

[PD Disaggregation] Distinguish the pipelines for sending kv signal in different prefill (#5514 )

2025-12-12 14:05:36 +08:00

get_output_msg_with_topk.cc

…

get_output.cc

[PD Disaggregation] support DP via v1 router and decouple DP and EP (#5197 )

2025-12-04 15:38:43 +08:00

get_padding_offset_system.cu

…

get_padding_offset.cu

remove speculate_get_padding_offset op (#6308 )

2026-02-03 15:18:12 +08:00

get_position_ids_and_mask_encoder_batch.cu

…

helper.cu

…

helper.h

remove unneeded para from flash_mask_attention (#6218 )

2026-01-27 14:04:27 +08:00

init_signal_layerwise.cc

…

ipc_sent_key_value_cache_by_remote_ptr.cu

[Feature] support rdma pd dy-c8 (#5788 )

2026-01-07 14:55:25 +08:00

limit_thinking_content_length_v1.cu

[BugFix] Fix limit_thinking early return logic in CUDA kernels (#5471 )

2025-12-10 11:03:19 +08:00

limit_thinking_content_length_v2.cu

[BugFix] Fix limit_thinking early return logic in CUDA kernels (#5471 )

2025-12-10 11:03:19 +08:00

merge_prefill_decode_output.cu

…

msg_utils.h

…

multi_head_latent_attention.cu

…

ngram_mask.cu

…

noaux_tc_redundant.cu

[Feature] Support noaux for eplb (#5143 )

2025-11-21 14:10:32 +08:00

noaux_tc.cu

…

noauxtc_kernel.h

[Others] clean code (#5691 )

2025-12-24 11:28:47 +08:00

open_shm_and_get_meta_signal.cc

…

per_token_quant_fp8.cu

[Feature] FD_USE_PHI_FP8_QUANT (#6320 )

2026-02-03 22:33:03 -08:00

read_data_ipc.cu

…

read_ids.py

…

read_temp_ids.py

…

reasoning_phase_token_constraint.cu

[Others] Support constrained decoding when enable_thinking is false (#6248 )

2026-01-28 00:05:17 -08:00

rebuild_padding.cu

Remove MTP rebuil_padding useless code (#6336 )

2026-02-05 16:28:44 +08:00

recover_decode_task.cu

…

remote_cache_kv_ipc.cc

…

remote_cache_kv_ipc.h

[PD Disaggregation] Distinguish the pipelines for sending kv signal in different prefill (#5514 )

2025-12-12 14:05:36 +08:00

save_output_msg_with_topk.cc

[Optim] Robust sync status when preempted happens (#5796 )

2026-01-14 12:07:33 +08:00

save_with_output_msg.cc

[Optim] Robust sync status when preempted happens (#5796 )

2026-01-14 12:07:33 +08:00

save_with_output_msg.h

…

save_with_output.cc

…

scaled_gemm_f8_i4_f16_gemm.cu

…

scaled_gemm_f8_i4_f16_weight_quantize.cu

…

seqs2seqs.cu

…

set_data_ipc.cu

[BugFix] fix shm opened but not closed in set_data_ipc (#5826 )

2025-12-29 23:35:07 +08:00

set_flags.cu

…

set_mask_value.cu

…

set_stop.cu

[Model Runner] Refactor execute_model for GPU async scheduling (#6176 )

2026-01-28 14:19:33 +08:00

set_value_by_flags_and_idx.cu

…

share_external_data.cu

…

step_reschedule.cu

…

step_system_cache.cu

…

step.cu

…

stop_generation_multi_ends.cu

[Feature] support stop_token_ids (#5399 )

2025-12-09 17:49:12 +08:00

stop_generation.cu

…

swap_cache_batch.cu

[Feature] dyc8 support prefixcache (#5125 )

2025-11-21 19:46:26 +08:00

swap_cache_layout.cu

[KVCache] Storage cache supports c8 model (#6298 )

2026-02-06 12:01:17 +08:00

swap_cache.cu

…

system2group.cu

…

text_image_gather_scatter.cu

…

text_image_index_out.cu

…

token_penalty_multi_scores.cu

[BugFix] Fix token_penalty kernel (#6069 )

2026-01-28 12:03:05 +08:00

token_penalty_only_once.cu

…

token_transfer.hpp

…

transfer_output.cc

…

tune_cublaslt_gemm.cu

…

unset_data_ipc.cu

…

update_attn_mask_offsets.cu

[BugFix]Fix attention mask bug in D-Node of PD-split mode (#5245 )

2025-11-26 17:56:28 +08:00

update_inputs_beam.cu

…

update_inputs_v1.cu

[Model Runner] Refactor execute_model for GPU async scheduling (#6176 )

2026-01-28 14:19:33 +08:00

update_inputs.cu

[Model Runner] Refactor execute_model for GPU async scheduling (#6176 )

2026-01-28 14:19:33 +08:00

update_split_fuse_input.cu

…