sharedCache is allocated before goVersionOK runs, but CacheDir is
only set once the version check passes. The trim defer in mainErr
only checked sharedCache != nil, so on the too-new-version path
openCache joined an empty CacheDir with "build" and created the
directory in CWD.
Fixes#995.
Preserve reflected names for foreign named types, alias-backed unnamed structs,
and foreign struct fields by hashing names with the declaring package or struct
identity as appropriate. Extend the reflect script coverage to assert the
foreign type and field names that must now remain visible.
Fixes#996.
Co-Autored by: 0xKismetDev <131729061+0xKismetDev@users.noreply.github.com>
The marshaler interface and some of the other types were unnecessary.
The remaining test still fails without the fix,
because the crux of the issue is the separate package
with the Result generic type having Data as a type parameter and field.
When compiling a package that instantiates a generic type from another
package, info.Types only contains the instantiated *types.Named. The
*types.Struct case in recordFieldToStruct returns early on instantiated
structs (field != field.Origin()), so the origin struct was never
recorded in fieldToStruct. At rewrite time, looking up the origin field
then failed with "could not find struct for field X".
Fix by also recursing into typ.Origin() for instantiated named types,
so the uninstantiated struct is visited and its fields are recorded.
Fixes#1027.
The "next obfuscator" logic was split into three funcs for no reason;
it all fits into a single function in less than a dozen lines.
While here, avoid calling the obfRand variables the same as the
type itself, as that leads to significant confusion.
This is an inherently flaky test, because even if we generate
thousands of literal strings of varying sizes,
we could still get unlucky and end up not using one of the obfuscators.
I briefly looked at a unit test instead, but at that point it's
a bit silly as the underlying logic picks a random element from a slice.
So the unit test looks almost exactly as the code it's testing.
Remove the test entirely. The code it's testing - that we select across
all of our available obfuscators - is simple enough where we can just
look at it if we want to double check that it does what we expect.
The main "linux" job is the slowest by far at over 20 minutes.
The CPU on the hosted Actions runners is very slow,
but luckily, we get 20 concurrent Linux runners.
Split the main Linux job further.
Also, now that generics in Go are widespread,
we no longer need a Go generics library for good coverage.
While here, time each of the third party project commands.
Given that the Linux test job is currently the slowest,
the added parallelism will help.
The full or "long" tests aren't needed to look for races,
as our short tests already cover almost all of our code.
This is a better compromise for CI speed.
This is very expensive; it takes over 30s to run on my laptop.
We already cover `garble build std` in gogarble.txtar,
so this is merely about testing literal obfuscation.
Given the huge cost, this doesn't seem worth it.
If any bugs are uncovered in the literal obfuscators,
we can always add more test cases here.
The new script still produces output compatible with benchstat,
but it does the benchmarking directly via os/exec
rather than using `garble test`. This has a few benefits:
1) We can warm up the "base" caches before all runs
2) We can ensure that a run isn't affected by caching from other runs
3) We can still collect data when obfuscators hit a timeout
4) We can produce output including vanilla "go" as baseline
The Go assembler generates go_asm.h with constants like TypeName__size
and TypeName_FieldName for struct types. When garble obfuscates type
and field names, these constants use the obfuscated names, but the
assembly source still refers to the originals, causing build failures.
During transformCompile, save the original-to-obfuscated name mapping
to the build cache. In the second asm pass (after compile has run),
load the mapping and rewrite the .s files with a strings.Replacer.
The work is shared between the compiler and assembler steps
because it would be too much work for the assembler step by itself.
These constants follow patterns like Type_field or Type__size,
so they don't use special characters that we can easily locate.
And the go_asm.h file that the compiler generates already uses
the obfuscated names, which we cannot reverse without the originals.
Fetching the original names and fields would require parsing
and typechecking, which is something that the compiler step does.
Without the fix, the added test fails as follows:
> exec garble build
[stderr]
# test/with.many.dots/main
./.tmp/[...]/IQSArvqP.s:30: expected pseudo-register; found R11
./.tmp/[...]/IQSArvqP.s:31: expected pseudo-register; found R11
./.tmp/[...]/IQSArvqP.s:37: illegal or missing addressing mode for symbol Args__size
asm: assembly of ./.tmp/[...]/IQSArvqP.s failed
Thanks to Golo Roden for figuring out a test reproducer
as well as the overall approach for a fix to the problem.
The test added here is simplified from his code,
and the fix is inspired by his approach.
Fixes#948.
There are no integration tests because writing one is fairly tricky.
Luckily, the added code is fairly simple.
Windows in CI works as a form of regression test.
First, the current iteration of this code only needs to look at
go/types.Info.Types; looking at the Uses and Defs is redundant.
Note that we still need to fill Uses and Defs for ObjectOf to work.
Second, for consistent hashing in the face of generic types,
we have been using uninstantiated or "origin" field vars
when looking up fieldToStruct entries.
computeFieldToStruct tried to only look at these origin types,
but it didn't do so correctly. The test case added in #1016
made this evident given the added inconsistency panic check.
Third, resolve this by only checking the Origin method when we are
looping over the struct fields; if any of them is an instance,
abort as we are currently on an instantiated type.
This is not ideal, hence the TODO, but it works for now.
This resolves intermittent test failures with the test case
added in #1016; the fix there wasn't entirely incorrect,
but the code was buggy as we would loop over go/types.Info.Types
in the random map iteration order, so it was a coin flip
whether we would record the uninstantiated or instantiated types.
We don't add a test because adding one is tricky, but luckily,
the added assertion catches any such inconsistency in our logic,
so that should work as a form of a regression test.
Attempting to fix a failure on GOOS=darwin:
> exec garble -literals build std
[stderr]
# crypto/internal/fips140
<autogenerated>:1: crypto/internal/fips140.YVgfcHoMsU1M.func1.jump7: invalid relocation R_ADDR in fips data (SDATAFIPS)
And other failures I was seeing on Linux relating to `var RODATA`.
Beyond the usual updating of version strings, re-running `go generate`,
and updating the patches for the linker, we needed extra changes.
First, the linker started being stricter about which packages
are allowed to linkname runtime names by import path.
Stop obfuscating import paths in runtimeAndLinknamed.
Second, the compiler's intrinsics code adds an "add" function
much like the existing "addF", so tweak the regular expression.
Third, note that there is a new runtimeAndDeps windows-only package,
which we find thanks to the recent improvement to cover many platforms.
Fourth, the code around magic numbers in the runtime has changed,
so rewrite the code which patches it via go/ast.
Fixes#990.
Make runtimeAndLinknamed a set, like the other tables, for easy lookups
which will come in handy for Go 1.26 support.
Run `go list` across multiple platforms to ensure we cover all bases.
This again will help for Go 1.26 support, which introduces more
packages specific to MacOS and Windows.
This starts causing a failure on Go 1.26, which we only capture
in another test via `go test` by accident, so make sure we cover it.
Without a fix and on go1.26, I now see:
> exec garble build -o=main$exe ./stdimporter
[stderr]
# test/main/stdimporter
link: x9BrC6t: invalid reference to runtime.pprof_goroutineLeakProfileWithLabels
It can be rather slow on multi-megabyte binaries,
and its output can also be pretty massive at times.
Let the user run it separately on the files we store
to the bincmp_output directory.
We were checking for "have we seen this package path before" via i!=0,
but this would always be true for the package unlucky enough
to be matched first in the file.
As such, we would end up not collecting the intrinsic funcs for this
unlucky first package correctly.
Fix the bug, and re-run the script, which finds two funcs we dropped.
Re-run it with go1.25.8 as well, although this causes no changes.
Change getInt helper from panicking to returning an error when
encountering invalid or out-of-range control flow directive parameters.
This gives a clear error message to the user instead of a stack trace.
Check the return value of targetFile.Close() to catch delayed write
errors. The previous defer pattern silently discarded errors from
the final flush, which could hide data corruption on some filesystems.
Return a descriptive error instead of panicking when loadLinkerPatches
encounters an unsupported patch type (new/delete/copy/rename). This
gives a clear error message to the user instead of a stack trace.
- nextLinearTimeObfuscator was indexing linearTimeObfuscators instead
of linearTimePrimeObfuscators, making the prime variant unreachable.
- Fix copy-pasted docstring on byteLitWithExtKey (said "simple xor"
but the function uses an extended key).
- Add comment explaining the XOR self-inverse mapping.
Only the first 8 bytes of the seed are used for deterministic
obfuscation. Warn the user via stderr when extra bytes are provided,
and remove the TODO that noted this was confusing.
encoding/gob.Decoder already has "map copy" semantics;
any decoded map keys get inserted, and other existing map keys are left.
This is exactly the same behavior that the new code did with maps.Copy,
so the new code is unnecessary.
If there is any bug with this code, it needs to be shown via a test.
As far as I can tell, there is no bug, because no two packages
should clash in terms of map keys.
Add a helper to obtain a go/ssa.Function's origin if there is one,
falling back to itself if there is none.
This mimics how go/types.Func.Origin works,
and lets us simplify quite a bit of code.
Finally, fieldToStruct uses origin objects only,
so don't do a double lookup in reverse.go.
While here, fix up the func mentions in two type aliases.
Use `types.Var.Origin()` when mapping and hashing struct fields so selector and
declaration sites share the same canonical field identity. This fixes build
failures for generic aliases and named types over those aliases.
Fixes#924
Ensure reflected arguments keep field names when passing through generic helpers by propagating through ssa.ChangeType and normalizing generic API call handling.
Also merge dependency reflection metadata correctly from cache hits.
fixes#912
This makes the unobfuscated state easily accessible, for debugging.
Obfuscated code is output to: `./<debugdir>/garbled`
The original source code is output to:
`./<debugdir>/source`
Also enable caching for debugdir to allow for faster iteration.
Avoid hashing free type parameters by object/name identity in type
hashing and use traversal-local canonical IDs instead.
This keeps obfuscated field names stable across alpha-renamed generic
signatures (e.g. T vs newT), fixing interface/method mismatches.
The added test case fails without the fix:
> exec garble build
[stderr]
# test/main
YiaBFlNH.go:48: cannot use bT6psGs5O (variable of type *QdwRyqV[aEqmF4M, zpotHfQdnNB]) as E_OKfdHoPH[zpotHfQdnNB] value in return statement: *QdwRyqV[aEqmF4M, zpotHfQdnNB] does not implement E_OKfdHoPH[zpotHfQdnNB] (wrong type for method Redirect)
have Redirect(struct{palTavb zpotHfQdnNB})
want Redirect(struct{g3N8A_R zpotHfQdnNB})
The fix is Paul's; this is rebased on an updated version of typeutil,
and the test is now part of typeparams.txtar and much smaller.
Fixes#991.
Co-authored-by: Paul Scheduikat <lu4p@pm.me>
GODEBUG started being used for configuring the behavior of the Go
toolchain and standard library, for the sake of smoother transitions
in terms of backwards and forwards compatibility.
See: https://go.dev/doc/godebug
As such, it is not right to have `garble build -tiny` ignore all
GODEBUG settings, because many GODEBUG keys nowadays do not actually
involve debugging what a Go binary is doing.
Moreover, the mechanism we were using broke with Go 1.25.2,
which refactored `func parsedebugvars()` into
`func parseRuntimeDebugVars(godebug string)`,
so our test started breaking as our runtime patching was broken.
Otherwise we might do odd things, such as trying to open the linker
patches directory "patches/" given that go/version.Lang
returns an empty string for invalid versions.
See #978.
We don't need to use a regular expression to find "goN.M" anymore,
as go/version seems to deal with "devel" versions from master just fine.
We can then also stop having two separate fields for the version
of the Go toolchain currently being used.
KismetDev