apps/FastDeploy
1
0
Fork 0
mirror of https://github.com/PaddlePaddle/FastDeploy.git synced 2026-04-23 00:17:25 +08:00
Code Issues Actions 19 Packages Projects Releases Wiki Activity

[Quantization] Support w4afp8 MoE dynamic quantization (#5282)

Browse Source 
* support dynamic activation quant for w4afp8

* support dynamic w4afp8

* add test

* fix

* fix

---------

Co-authored-by: zhoutianzi666 <17801055074@163.com>
This commit is contained in:
Sunny-bot1
2025-12-02 18:56:16 +08:00
committed by GitHub
parent 429dd2b1db
commit 3629db4129
9 changed files with 478 additions and 37 deletions
Download Patch File Download Diff File Expand all files Collapse all files
custom_ops/utils/auto_gen_w4afp8_gemm_kernel.py
+1 -1
View File
@@ -85,7 +85,7 @@ void w4afp8_gemm_M{M}_N{N}_G{GROUPSIZE}_K{K}_E{EXPERTS}_P{PADDING}_{TYPE}(
"""
# [M, K, Number of experts, token Padding Size, weight K group size]
gemm_case = [[256, 256, 2, 0, 128]]
gemm_case = [[256, 256, 2, 0, 128], [512, 256, 2, 0, 128]]
dtype = ["BF16"]