mirror of
https://github.com/burrowers/garble.git
synced 2026-04-22 15:47:04 +08:00
Daniel Martí
ef2385ee97
The added benchmark script shows these numbers for building and running
with vanilla go on literal sizes between 16B and 2048B,
showing that vanilla Go isn't affected at all by these sizes:
│ go │
│ sec/op │
Build/16B 118.1m ± ∞ ¹
Build/64B 123.1m ± ∞ ¹
Build/256B 119.7m ± ∞ ¹
Build/1024B 119.1m ± ∞ ¹
Build/2048B 124.3m ± ∞ ¹
Run/16B 1.671m ± ∞ ¹
Run/64B 1.143m ± ∞ ¹
Run/256B 1.190m ± ∞ ¹
Run/1024B 1.222m ± ∞ ¹
Run/2048B 1.080m ± ∞ ¹
Our simple and swap obfuscators scale pretty well to these same sizes,
only causing moderate slow-downs to build and runtime speeds:
│ simple │ swap │
│ sec/op vs base │ sec/op vs base │
Build/16B 268.0m ± ∞ ¹ +126.88% (p=1.000 n=1) 262.0m ± ∞ ¹ +121.80% (p=1.000 n=1)
Build/64B 253.8m ± ∞ ¹ +106.16% (p=1.000 n=1) 252.4m ± ∞ ¹ +105.00% (p=1.000 n=1)
Build/256B 265.4m ± ∞ ¹ +121.78% (p=1.000 n=1) 276.7m ± ∞ ¹ +131.16% (p=1.000 n=1)
Build/1024B 267.4m ± ∞ ¹ +124.44% (p=1.000 n=1) 315.0m ± ∞ ¹ +164.48% (p=1.000 n=1)
Build/2048B 277.4m ± ∞ ¹ +123.11% (p=1.000 n=1) 383.8m ± ∞ ¹ +208.70% (p=1.000 n=1)
Run/16B 1.740m ± ∞ ¹ +4.12% (p=1.000 n=1) 1.463m ± ∞ ¹ -12.47% (p=1.000 n=1)
Run/64B 1.470m ± ∞ ¹ +28.66% (p=1.000 n=1) 1.455m ± ∞ ¹ +27.35% (p=1.000 n=1)
Run/256B 1.729m ± ∞ ¹ +45.25% (p=1.000 n=1) 1.812m ± ∞ ¹ +52.26% (p=1.000 n=1)
Run/1024B 1.315m ± ∞ ¹ +7.62% (p=1.000 n=1) 1.352m ± ∞ ¹ +10.60% (p=1.000 n=1)
Run/2048B 1.425m ± ∞ ¹ +31.93% (p=1.000 n=1) 1.316m ± ∞ ¹ +21.88% (p=1.000 n=1)
However, the other three cause huge slow-downs in both build and runtime speeds:
│ split │ shuffle │ seed │
│ sec/op vs base │ sec/op vs base │ sec/op vs base │
Build/16B 326.6m ± ∞ ¹ +176.53% (p=1.000 n=1) 363.1m ± ∞ ¹ +207.44% (p=1.000 n=1) 217.4m ± ∞ ¹ +84.05% (p=1.000 n=1)
Build/64B 400.6m ± ∞ ¹ +225.34% (p=1.000 n=1) 328.0m ± ∞ ¹ +166.39% (p=1.000 n=1) 262.1m ± ∞ ¹ +112.87% (p=1.000 n=1)
Build/256B 824.7m ± ∞ ¹ +589.08% (p=1.000 n=1) 588.2m ± ∞ ¹ +391.45% (p=1.000 n=1) 873.7m ± ∞ ¹ +630.05% (p=1.000 n=1)
Build/1024B 3257.7m ± ∞ ¹ +2634.84% (p=1.000 n=1) 1671.4m ± ∞ ¹ +1303.15% (p=1.000 n=1) 5000.0m ± ∞ ¹ +4097.53% (p=1.000 n=1)
Build/2048B 5000.0m ± ∞ ¹ +3921.73% (p=1.000 n=1) 3936.4m ± ∞ ¹ +3066.21% (p=1.000 n=1) 5000.0m ± ∞ ¹ +3921.73% (p=1.000 n=1)
Run/16B 1.680m ± ∞ ¹ +0.51% (p=1.000 n=1) 1.426m ± ∞ ¹ -14.67% (p=1.000 n=1) 1.908m ± ∞ ¹ +14.13% (p=1.000 n=1)
Run/64B 1.560m ± ∞ ¹ +36.53% (p=1.000 n=1) 1.345m ± ∞ ¹ +17.74% (p=1.000 n=1) 1.704m ± ∞ ¹ +49.10% (p=1.000 n=1)
Run/256B 2.133m ± ∞ ¹ +79.24% (p=1.000 n=1) 1.833m ± ∞ ¹ +53.98% (p=1.000 n=1) 1.838m ± ∞ ¹ +54.40% (p=1.000 n=1)
Run/1024B 2.863m ± ∞ ¹ +134.21% (p=1.000 n=1) 1.786m ± ∞ ¹ +46.12% (p=1.000 n=1) 5000.000m ± ∞ ¹ +408975.92% (p=1.000 n=1)
Run/2048B 5000.000m ± ∞ ¹ +462837.24% (p=1.000 n=1) 2.900m ± ∞ ¹ +168.54% (p=1.000 n=1) 5000.000m ± ∞ ¹ +462837.24% (p=1.000 n=1)
As such, limit the scope of when we apply our obfuscators in two ways.
First, always apply a limit of 2048 bytes for all obfuscators.
As we can see above, the two cheap obfuscators still add some overhead,
and we aren't testing what happens if they run on truly huge strings.
It's likely that they will still cause unexpected slow-down.
Second, split the obfuscators along this line and call them
"cheap" versus "expensive". The expensive ones are only used
for sizes of up to 256 bytes. We still measure moderate slow-downs
in build times of 400-600%, but this is a reasonable compromise for now.
We will be filing bugs with upstream Go about the compiler overhead.
We adjust generateLiterals accordingly;
it now generates 100 literals between MinSize and MaxSize,
which must be obfuscated in some way, and 5 literals past MaxSize,
which don't need to be obfuscated, to check for issues like crashes.
Fixes #928.