2 Commits

Author	SHA1	Message	Date
google-labs-jules[bot]	ddec1b07f8	⚡ Bolt: [performance improvement] Pre-allocate np.full array for padding lists instead of using slow list concatenations in pad_batch_data ... The old implementation uses `[[pad_id] * (max_len - len(inst)) + list(inst) for inst in insts]` to pad list sequences. This performs an $O(N \times \text{max\_len})$ list concatenation, creating many intermediate Python lists and stressing the garbage collector, before finally passing the result to `np.array(..., dtype=np.int64)`. This change updates it to pre-allocate an empty numpy array (`np.full`) and safely populates it using numpy slicing (`padded_insts[i, :l] = inst`). The change results in a ~2x faster performance. This has been verified to be completely logically equivalent to the original un-modified processor output on a comprehensive set of test cases.	2026-04-13 15:14:37 +00:00
luukunn	b9f8873367	[Optimization]Merge Text processor (#7030) ... * merge text processor * update * fix unit test * merge messages2ids * fix unit test * 删除重复代码 * remove redundant code * delete code * fix unit test	2026-03-30 15:02:35 +08:00