Mutant IR / ByteCode Reference
Mutant compiles to a flat, linear byte stream. There is no intermediate representation between the AST and the bytecode – the compiler walks the AST and emits bytes directly.
This document covers every layer of the bytecode pipeline: instruction encoding, the constant pool, compiler internals, the stack-machine VM, security machinery, and the polymorphic mutation engine.
Status note: this is a deep technical reference. For current feature enablement and rollout status (especially anti-tamper gates and polymorphic activation depth), use:
Table of Contents#
- Architecture Overview
- Instruction Encoding
- Opcode Reference
- ByteCode Object
- Constant Pool
- Compiler Internals
- Symbol Table & Scoping
- VM: Stack Machine Design
- VM: Execution Loop
- Closures & Free Variables
- Structs at the Bytecode Level
- Enums at the Bytecode Level
- Control Flow
- Operand Encryption
- Runtime Security Features
- Polymorphic Mutation Engine
- Object Type System
- VM Constructor Variants
- Common Pitfalls
1. Architecture Overview#
Source text
│
▼
Lexer (token/)
│
▼
Parser (parser/) ──► AST (ast/)
│
▼
Compiler (compiler/)
│ ├─ SymbolTable resolves variable scopes
│ ├─ CompilationScopes one per function nesting level
│ └─ PolymorphicEngine optional bytecode mutation
│
▼
ByteCode struct
│ ├─ Instructions []byte flat bytecode stream
│ ├─ Constants []object.Object literal pool
│ ├─ StructDefs map[string][]*ast.Identifier
│ └─ EnumDefs map[string][]string
│
▼
VM (vm/)
│ ├─ Stack []object.Object (grows dynamically)
│ ├─ Globals []object.Object (grows dynamically)
│ ├─ Frames []*Frame (call stack)
│ └─ Security subsystem
Mutant compiles to a flat, linear byte stream. There is no intermediate representation between the AST and the bytecode – the compiler walks the AST and emits bytes directly. The VM then executes those bytes in a register-less, expression-stack machine.
2. Instruction Encoding#
2.1 Types#
// code/code.go
type Instructions []byte // the raw bytecode stream
type Opcode byte // a single opcode value (0-based iota)
2.2 Instruction Layout#
Every instruction is:
┌──────────┬──────────────────────────────────────────┐
│ 1 byte │ 0, 1, 2, 3, or 4 bytes of operands │
│ opcode │ (widths defined per opcode in Definition) │
└──────────┴──────────────────────────────────────────┘
Multi-byte operands are always big-endian.
2.3 `Definition` Registry#
type Definition struct {
Name string
OperandWidths []int // widths of each operand in bytes (1 or 2)
}
The definitions map in code/code.go is the single source of truth for every
opcode's operand layout. code.Lookup(byte) returns the definition or an error
for unknown opcodes.
2.4 `Make` — Building Instructions#
func Make(op Opcode, operands ...int) []byte
- Allocates
1 + sum(OperandWidths)bytes. - Writes the opcode at
[0]. - Writes each operand using
binary.BigEndian.PutUint16(2-byte) or a directbyte()cast (1-byte).
// Example: emit OpConstant with index 42
ins := code.Make(code.OpConstant, 42)
// Result: [0x00, 0x00, 0x2A] (OpConstant=0, 42 big-endian)
2.5 Reading Operands#
Unencrypted (compiler / test use only):
func ReadOperands(def *Definition, ins Instructions) ([]int, int)
Returns a []int of decoded operand values and the total bytes consumed by
those operands (not including the opcode byte itself).
Encrypted (VM runtime use — the only correct way during execution):
func ReadUint16(ins Instructions, length int64, password string, offset int64) (uint16, error)
func ReadUint8 (ins Instructions, length int64, password string, offset int64) (uint8, error)
Both functions call security.SecureXOR* before decoding. length is the total
byte length of the main function's instruction stream (used as part of the
XOR key derivation). offset is the absolute byte position of the first operand
byte within the instruction stream.
Critical rule (from project memory): Never read encrypted operands with
ins[offset]orbinary.BigEndian.Uint16(ins[offset:])directly inside the VM. Always callReadUint8/ReadUint16. Reading raw bytes gives corrupted values when encryption is active.
2.6 Disassembling#
Instructions.String() produces a human-readable listing:
0000 OpConstant 0
0003 OpConstant 1
0006 OpAdd
0007 OpSetGlobal 0
0010 OpGetGlobal 0
...
It is not safe to call this on encrypted bytecode (it uses the unencrypted
ReadOperands path). Use it only in tests or during non-password compilation.
3. Opcode Reference#
Opcodes are declared as an iota in code/code.go. Their numeric values are
fixed by the declaration order below, so do not reorder them.
3.1 Complete Table#
| # | Name | Operands (bytes) | Stack effect | Description |
|---|---|---|---|---|
| 0 | OpConstant |
idx (2) |
→ val |
Push constants[idx] |
| 1 | OpPop |
— | val → |
Discard top of stack |
| 2 | OpAdd |
— | b, a → a+b |
Integer or float add; string concat |
| 3 | OpSub |
— | b, a → a-b |
Subtract |
| 4 | OpMul |
— | b, a → a*b |
Multiply |
| 5 | OpDiv |
— | b, a → a/b |
Divide |
| 6 | OpMod |
— | b, a → a%b |
Modulo |
| 7 | OpTrue |
— | → true |
Push the singleton True object |
| 8 | OpFalse |
— | → false |
Push the singleton False object |
| 9 | OpEqual |
— | b, a → bool |
a == b |
| 10 | OpUnEqual |
— | b, a → bool |
a != b |
| 11 | OpGreater |
— | b, a → bool |
a > b |
| 12 | OpMinus |
— | a → -a |
Unary negation |
| 13 | OpBang |
— | a → !a |
Unary logical NOT |
| 14 | OpJumpFalse |
target (2) |
cond → |
Pop condition; if falsy jump to target |
| 15 | OpJump |
target (2) |
— | Unconditional jump to target |
| 16 | OpNull |
— | → null |
Push the singleton Null object |
| 17 | OpGetGlobal |
idx (2) |
→ val |
Push globals[idx] (decrypted) |
| 18 | OpSetGlobal |
idx (2) |
val → |
Pop and store to globals[idx] (encrypted) |
| 19 | OpGetLocal |
idx (1) |
→ val |
Push stack[bp+idx] (decrypted) |
| 20 | OpSetLocal |
idx (1) |
val → |
Pop and store to stack[bp+idx] (encrypted) |
| 21 | OpArray |
n (2) |
eN…e0 → arr |
Pop n elements, build Array, push it |
| 22 | OpHash |
n (2) |
vN,kN…v0,k0 → hash |
Pop n key+value pairs (n is even), build Hash, push it |
| 23 | OpIndex |
— | idx, obj → val |
Pop index then object; resolve obj[idx] |
| 24 | OpCall |
argc (1) |
argN…arg0, fn → result |
Call function with argc arguments |
| 25 | OpReturnValue |
— | val → (caller frame) |
Pop return value, restore frame, push value |
| 26 | OpReturn |
— | (caller frame) |
Void return; restores frame, pushes Null |
| 27 | OpMultiValue |
count (2) |
vN…v0 → multi |
Pop count values into one MultiValue |
| 28 | OpDup |
— | a → a, a |
Duplicate the top of the stack |
| 29 | OpDestructure |
count (2) |
multi → v0…vN |
Pop a MultiValue; push count values from it |
| 30 | OpGetBuiltin |
idx (2) |
→ fn |
Push the builtin at BuiltinNames[idx & 0x7FFF] (see §4.1) |
| 31 | OpClosure |
fnIdx (2), numFree (1) |
fN…f0 → closure |
Pop numFree free vars; wrap constants[fnIdx] in a Closure |
| 32 | OpGetFree |
idx (1) |
→ val |
Push currentClosure.Free[idx], through the cell if it is one (decrypted) |
| 33 | OpCurrentClosure |
— | → closure |
Push the currently executing closure (for named recursion) |
| 34 | OpChkDbg |
— | — | Halt (or warn) if a debugger is detected |
| 35 | OpChkSnd |
— | — | Halt (or warn) if a sandbox environment is detected |
| 36 | OpBreak |
— | → Break{} |
Push a Break sentinel onto the stack |
| 37 | OpContinue |
— | → Continue{} |
Push a Continue sentinel onto the stack |
| 38 | OpMakeStruct |
typeIdx (2), fieldCount (1) |
fN…f0 → struct |
Pop fieldCount values, create a Struct |
| 39 | OpGetField |
nameIdx (2) |
struct → val |
Pop struct; push struct.Fields[constants[nameIdx]] |
| 40 | OpSetField |
nameIdx (2) |
val, struct → struct |
Pop value then struct; set field; push struct back |
| 41 | OpEnumValue |
typeIdx (2), tagIdx (2) |
→ EnumValue |
Create EnumValue{TypeName, Tag, ordinal} |
| 42 | OpGreaterEqual |
— | b, a → bool |
a >= b |
| 43 | OpSetIndex |
— | val, idx, obj → obj |
Mutate obj[idx] in place; push the container back |
| 44 | OpBitAnd |
— | b, a → a&b |
Bitwise AND; both operands must be INTEGER |
| 45 | OpBitOr |
— | b, a → a|b |
Bitwise OR; both operands must be INTEGER |
| 46 | OpBitXor |
— | b, a → a^b |
Bitwise XOR; both operands must be INTEGER |
| 47 | OpBitNot |
— | a → ~a |
Bitwise complement; operand must be INTEGER |
| 48 | OpShiftLeft |
— | b, a → a<<b |
Left shift; a negative count is a runtime error |
| 49 | OpShiftRight |
— | b, a → a>>b |
Arithmetic right shift (sign-preserving); negative count errors |
| 50 | OpGetLocalCell |
slot (1) |
→ val |
Push cell(bp+slot).Value (decrypted); the slot must be boxed |
| 51 | OpSetLocalCell |
slot (1) |
val → |
Pop and store into cell(bp+slot).Value (encrypted) |
| 52 | OpCaptureLocal |
slot (1) |
→ cell |
Push the *Cell in bp+slot itself, for OpClosure's list |
| 53 | OpCaptureFree |
idx (1) |
→ cell |
Push currentClosure.Free[idx] unread, for a nested capture |
| 54 | OpSetFree |
idx (1) |
val → |
Pop and store into Free[idx].Value (encrypted) |
| 55 | OpConcat |
n (2) |
pN…p0 → str |
Pop n pieces of an interpolated string; push the one string they spell |
| 56 | OpIterInit |
— | iterable → iter |
Pop a collection; push a cursor over it (hash keys in Inspect order) |
| 57 | OpIterNext |
target (2), binds (1) |
iter → iter[, key], val |
Advance the cursor: push binds values on top of it, or jump to target when spent |
| 58 | OpMatchFail |
— | subject → (error) |
Pop the unmatched subject and raise, naming the value that fell through |
| 59 | OpLess |
— | b, a → bool |
a < b |
| 60 | OpLessEqual |
— | b, a → bool |
a <= b |
3.2 Stack Notation#
The "Stack effect" column uses the convention:
- Values left of
→are consumed (popped), listed bottom-first (sob, ameansais on top). - Values right of
→are produced (pushed). (caller frame)means the caller's frame is restored — no net stack value in the new frame's context.
3.3 Comparison Operands Run In Source Order#
< and <= have opcodes of their own. Like every other binary operator they
compile the left operand first, so the operands' side effects run in the order
they are written:
[compile a] ← left operand first
[compile b] ← right operand second
OpLess
There used to be no OpLess: the compiler emitted a < b as b > a, pushing
the right operand first and reusing OpGreater. That saved two opcodes and
cost the source-order guarantee — next() < next() called the right-hand
next first, while every other operator, and the tree-walking evaluator macro
expansion computes an unquote with, called the left-hand one first. The same
expression then answered differently inline and inside a macro. OpLess and
OpLessEqual were appended to the instruction set and the swap removed;
OpGreater and OpGreaterEqual keep the values and the meanings they always
had, so bytecode compiled before the change still runs.
4. ByteCode Object#
// compiler/compiler.go
type ByteCode struct {
Instructions code.Instructions // Main function byte stream
Constants []object.Object // Constant pool (see §5)
StructDefs map[string][]*ast.Identifier // Field name lists per struct type
EnumDefs map[string][]string // Tag name lists per enum type
LuaPatches map[string]*object.LuaPatch // Lua security hook patches
Version int // Container version (see §4.1)
BuiltinNames []string // Builtins this program calls (§4.1)
OpcodeMap []byte // Undoes opcode permutation (§16)
SourceFile string // Path this was compiled from (§4.2)
SourceText string // The program's own source (§4.2)
LineTable code.LineTable // Offset -> start line/col (§4.2)
EndTable code.LineTable // Offset -> end line/col (§4.2)
MacroTable code.LineTable // Offset -> macro definition site (§4.2)
}
ByteCode is the artifact handed from the compiler to the VM. It is immutable
after construction (the VM clones nothing; it reads the slices directly).
The LuaPatches field is populated by the Lua integration layer, not by the
core compiler. See builtin/lua.go for context.
4.1 Container versions and builtin resolution#
| Version | Emitted by | OpGetBuiltin operand |
|---|---|---|
| absent (decodes to 0, normalised to 1) | up to v2.4.0 | an ordinal into the global builtin.Builtins registry of the release that compiled it |
| 2 | v2.5.0 onward | 0x8000 | i, where i indexes this program's own BuiltinNames |
Version 1 made the registry append-only forever. An ordinal baked into an
artifact means that entry can never be renamed, retired or reordered, because
doing so would rebind every call in every .mu already written — which is why
net_syn_scan stayed registered long after it was documented as deprecated.
Version 2 puts the names in the artifact. The compiler interns each builtin it
references (SymbolTable.ReferenceBuiltin) and emits a position in that table;
vm.resolveBuiltins binds every name to a function at construction, before a
single instruction runs, and reports a name this runtime does not have rather
than failing at whichever call site happens to be reached. Only referenced
builtins are listed, so retiring an unrelated one cannot stop a program loading.
Version 1 artifacts from v2.3.0 to v2.4.x still run. Their operands resolve through
builtin/legacy_ordinals.go, a frozen snapshot of the registry as it stood at
v2.4.0 — frozen because those indices are baked into files that cannot be
recompiled. A builtin that is renamed adds an entry to builtin.Aliases, which
keeps the old name resolvable for old bytecode without keeping it callable from
new source.
One ordinal meant two builtins before v2.5.0. help was appended to the
registry at init, after every other entry, so it sat at whatever the end was:
398 in v2.3.x, and 408 in v2.4.x, whose ten new builtins put pmap at 398.
legacy_ordinals.go therefore also records v2.3.x's registry, as the v2.4.x
snapshot with help at 398: v2.3.x's 399 entries are its first 399, and
v2.2.x's 211 entries are the first 211 of both tables. Neither v2.3.x nor v2.4.x
wrote Version, so the runner reads which table a version 1 artifact needs
from the container itself: gob writes ByteCode's type, naming every exported
field, ahead of the value, and only v2.4.0 and later declare OpcodeMap there
(compiler.LegacyRegistryOf). A stream whose type cannot be read that way is
refused rather than guessed at.
Registries before v2.2.0 ordered their entries differently, and neither table
describes them. A v2.2.x artifact that calls a builtin does not run correctly
either, although its registry matches: v2.3.0 widened OpGetBuiltin's operand
from one byte to two, so this runtime reads a v2.2.x builtin call's operand
with the next instruction's first byte in it, binds whichever builtin the two
bytes number (or stops when they number none), and reads every instruction
after it out of step. v2.2.x wrote the same ByteCode type as v2.3.x, so
nothing in the container tells such an artifact apart, and it is not refused.
Two development commits between the v2.3.0 and v2.4.0 tags, 821bece and
31182f3, had v2.4.0's registry but no OpcodeMap. Their artifacts read as
v2.3.x, so a pmap call in one reaches help; the builtins v2.4.0 added after
pmap still resolve, because the v2.3.x table keeps the snapshot's entries
past 398.
Both stay for the whole 2.x line: retiring version 1, or the net_syn_scan
alias of net_connect_scan, stops artifacts that already exist from running,
which only a major release may do. 3.0 is the first release that may remove
either, and its changelog will say so if it does.
The 0x8000 tag on version 2 operands (code.BuiltinNameTableFlag) is for the
one case the version field cannot cover. A pre-v2.5 runtime does not know to
read Version, so gob hands it a name-table index and it reads that as a
registry ordinal — and a small index is a perfectly plausible ordinal, so it
calls whatever sits there. The tag puts every version 2 operand far above the
registry's length, so the old runtime trips its own bounds check
(OpGetBuiltin: invalid builtin index=32768) instead of silently calling the
wrong builtin. It is permanent: it cannot be retired without breaking the
artifacts it protects.
4.2 Source positions#
LineTable maps an offset in an instruction stream back to the line and column
that produced it. There is one per stream: ByteCode.LineTable covers
Instructions, and every object.CompiledFunction carries its own alongside
its Name. That is what makes a VM frame resolvable — a frame knows its
closure and its offset, and the closure's function owns the table those are
read against.
entry := uvarint(ipDelta) varint(lineDelta) varint(colDelta)
Deltas are against the previous entry; the line and column deltas are signed
because positions move backwards routinely (a loop's jump is emitted after the
body but belongs to the for above it). An entry is written only where the
position changes.
There are two position tables per stream. LineTable gives where a construct
starts, EndTable where it ends. A start alone names a line, which is where
most languages stop; both ends let the reporter underline the span that failed,
so total / count(xs) says which division. EndTable is a separate field in
the same encoding, so it can be dropped on its own.
Statements, calls, infix expressions and index expressions anchor a position. Everything else inherits from the node enclosing it. Attributing a position to every expression node sounds more precise and is not: measured, it costs one entry per instruction, a table the size of the stream it describes. The four kinds that do anchor are the ones a reader lands on: a frame is a call, a statement is the unit a fault sits inside, and infix and index expressions are where a well-formed program actually fails at runtime -- division by zero, a type mismatch across an operator, an index past the end.
Debug info is not cheap and is not meant to be. The tables come to roughly a
fifth of an encoded artifact, and SourceText roughly doubles what is left, so a
local .mu runs about 1.7x the size of the same program built for release. That
is the trade -g makes in a C toolchain, and it is paid only by artifacts that
never leave the machine.
Parameter names travel too, on CompiledFunction.Params, so a traceback can
print the arguments a frame received rather than only its name. They are gated on
debug info for a second reason besides size: stack values are encrypted at rest,
and rendering them decrypts them. A stripped build prints its frames without
printing the program's own runtime values.
Macro expansions record two sites. MacroTable has the same encoding and is
populated only over instructions a macro produced. LineTable gives the call
the user wrote; MacroTable gives the macro's definition. Both are needed: when
generated code is wrong, the call site contains none of the logic that failed.
The parser's ast.Program.MacroExpansions side-table carries this from
expansion to compilation.
An entry whose line is 0 is a reset. A lookup answers with the last entry at
or before an offset, so "populated only over instructions a macro produced"
needs an entry recording where an expansion stops as much as one recording where
it starts -- without it the last macro in a program is reported as the origin of
every instruction compiled after it. MacroTable and EndTable therefore carry
entries whose accumulated line is 0, which mean the previous entry's coverage
ends here: offsets from there on have no macro origin and no recorded end, and a
reporter falls back to a caret rather than underlining a span it would be taking
from another construct. LineTable never carries one -- an instruction with no
position of its own belongs to the construct it was emitted for and inherits its
line, which is what writing no entry achieves. The entry encoding does not
change, so a decoder that knows nothing about resets reads line 0, and every
consumer of a position already treats a zero line as no position at all.
Versioning. These fields needed no Version bump, unlike BuiltinNames.
gob omits zero values and ignores fields it does not know, so a new runtime
reading an old artifact sees empty tables — which is exactly true of it — and an
old runtime reading a new artifact ignores them. Absence is already the correct
reading in both directions. A program with no tables reports frames by name and
no lines.
Stripping. ByteCode.StripDebugInfo() removes the file name, the embedded
source, every table, and every function and parameter name. generator.compile calls it for release builds: those are
what leave the machine, and a map from an artifact's bytecode back to its source
is worth withholding from them. A .mu compiled to run locally keeps its
positions.
Polymorphism does not strip. Mutation is on by default — mutant prog.mut
compiles at level 5 — so dropping positions there would mean no ordinary run
ever had them. Instead PolymorphicEngine.spliceFillers carries the tables
through the same offset remap it already builds to repoint jumps
(LineTable.Remap). A stream the engine declines to pad keeps its table
unchanged, because nothing moved.
5. Constant Pool#
The constant pool is []object.Object stored in ByteCode.Constants. It is
built incrementally by compiler.addConstant:
func (c *Compiler) addConstant(obj object.Object) int {
c.constants = append(c.constants, obj)
return len(c.constants) - 1 // index used as OpConstant operand
}
What lives in the pool:
| Object type | Example source | Notes |
|---|---|---|
*object.Integer |
42, -7 |
int64 |
*object.Float |
3.14 |
float64 |
*object.String |
"hello", field names, type names |
All string literals; also struct/enum type-name strings and field-name strings needed by struct/enum opcodes |
*object.CompiledFunction |
Any fn(…){…} |
The function body's bytecode + metadata |
Booleans and Null are NOT pooled. They use dedicated singleton opcodes
(OpTrue, OpFalse, OpNull) that push global singleton objects.
Maximum pool size: 65 535 entries (uint16 operand range). In practice the limit is never reached in normal programs.
Constant indices are stable within a single compilation. If polymorphic
constant-pool randomization is enabled (see §16), every operand that indexes the
pool is rewritten to match the shuffled order before the ByteCode is returned.
Which operands those are is declared by code.ConstantOperands, not assumed:
OpConstant, OpClosure, OpMakeStruct, OpGetField, OpSetField and both
operands of OpEnumValue.
6. Compiler Internals#
6.1 Compiler Struct#
type Compiler struct {
constants []object.Object // Accumulates the constant pool
symbolTable *SymbolTable // Current scope's symbol table
scopes []CompilationScope // Stack of compilation scopes
scopeIndex int // Index into scopes[]
structDefinitions map[string][]*ast.Identifier
enumDefinitions map[string][]string
loopContexts []LoopContext // Stack of active for-loop contexts
injectSecurityChecks bool // Enables OpChkDbg/OpChkSnd injection
hasChkDbg bool // Tracks whether DbgCheck was emitted
hasChkSnd bool // Tracks whether SndCheck was emitted
polymorphicEngine *PolymorphicEngine // nil = no mutation
}
6.2 Compilation Scope#
A CompilationScope holds:
type CompilationScope struct {
instructions code.Instructions // byte buffer being built
lastInstruction EmittedInstruction // most recently emitted instruction
prevInstruction EmittedInstruction // the one before last (for pop-removal)
unresolvedJumps map[int]unresolvedJump // forward jumps awaiting a target
}
unresolvedJumps is per scope and not per compiler because a jump position is
an offset into one stream and means something else in any other. The three
position tables the scope also builds are omitted above; see section 11.
The main program compiles into scopes[0]. Each function literal pushes a new
scope via enterScope() and pops it with leaveScope().
6.3 `emit` and Instruction Patching#
func (c *Compiler) emit(op code.Opcode, operands ...int) int
Returns the byte offset of the emitted instruction. This offset is saved so jumps can be back-patched:
// Emit jump with placeholder target
jumpPos := c.emitPlaceholder(code.OpJumpFalse)
// ... compile consequence ...
// Back-patch: replace the placeholder with the real destination
c.changeOperand(jumpPos, realTarget)
changeOperand calls replaceInstruction which overwrites the bytes at pos
in the current instruction buffer. This works because Make produces the same
byte length regardless of the operand value. Operands after the first are read
back and re-emitted unchanged, so a wider instruction — OpIterNext carries a
binding count after its target — keeps its tail instead of having it fall off
the end of a shorter rebuilt instruction.
The placeholder is owed a target#
emitPlaceholder emits operand 0 as placeholderTarget (9999) and records the
position in the current scope's unresolvedJumps; changeOperand removes it.
Nothing else writes to that map, so between them it is exactly the set of jumps
the compiler still owes a target.
assertJumpsResolved requires the set to be empty before a stream is finished,
and there are two such moments: leaveScope, after which a function's stream
becomes a constant in the pool and nobody holds offsets into it any more, and
the end of the *ast.Program arm, which is the only ending the main stream has.
Reaching either with an entry left is reported as an internal compiler error
naming the opcode, the offset and the line it was emitted from — no program can
ask for an unpatched jump, so it is a defect in the compiler rather than in the
source.
The check is there because what an unpatched jump does instead is decided by the length of the program it is in, and all three outcomes were measured:
| the stream | what runs |
|---|---|
| shorter than 9999 bytes | jumps past the end; the run is treated as finished and the program answers whatever was last popped — a boolean, for a program whose value is an integer |
| longer, byte 9999 mid-instruction | decodes that byte as an opcode: "unknown opcode 225: the bytecode may be damaged or decrypted with the wrong key, or it may have been built by a newer version of mutant", none of which is true |
| longer, byte 9999 on a boundary | resumes in unrelated code and does not terminate |
assertJumpsResolved takes a mark — only jumps emitted at or after it are
counted — because a program is not always the outermost thing being compiled,
and an enclosing construct may legitimately be holding a forward jump of its
own while everything inside it is compiled.
A backward jump does not go through emitPlaceholder: a loop's head is a
position already known, so emit is given the real target and there is nothing
to owe.
6.4 Pop-Removal Optimisation#
Expression statements emit OpPop after the expression so the stack stays
balanced. But when an expression is the last statement of a function body, the
OpPop must not be emitted (the value becomes the implicit return value).
The compiler detects this with lastInstructionIs(code.OpPop) and either:
- Calls
removeLastPop()— truncates the instruction buffer by one byte. - Calls
replaceLastPopWithReturn()— overwrites theOpPopwithOpReturnValuein-place (same size, no shift needed).
6.5 Function Literal Compilation#
1. enterScope()
2. If named function: symbolTable.DefineFunctionName(name) → OpCurrentClosure support
3. For each parameter: symbolTable.Define(param) → LocalScope
4. Compile body block
5. If last instruction is OpPop → replaceLastPopWithReturn()
6. If last instruction is not OpReturnValue → emit OpReturn
7. Snapshot freeSymbols = symbolTable.FreeSymbols
8. Snapshot numLocals = symbolTable.numDefinitions
9. Snapshot capturedLocals = symbolTable.CapturedLocals() (see §7.3.1)
10. leaveScope() → captures finished instruction bytes
11. boxCapturedLocals(insts, capturedLocals) → rewrites this function's own
OpGetLocal/OpSetLocal for captured slots to their cell forms, in place
12. For each free symbol: emitCapture(sym) → pushes captured cells onto parent stack
13. compiledFn = &CompiledFunction{Instructions, NumLocals, NumParams, CapturedLocals}
14. fnIndex = addConstant(compiledFn)
15. emit(OpClosure, fnIndex, len(freeSymbols))
6.6 Security Opcode Injection#
When injectSecurityChecks = true (enabled by
EnableSecurityOpcodeInjection()):
- After each top-level statement,
maybeEmitRandomSecurityCheckOpcodes()is called. It usescrypto/randto decide with probability1/3whether to emitOpChkDbgand/orOpChkSnd. - When
ByteCode()is called,ensureRequiredSecurityCheckOpcodes()appends whichever of the two opcodes were never randomly inserted, guaranteeing both are present at least once.
The VM enforces that both opcodes exist in the instruction stream when running
in password/secure mode (validated before and after execution via
validateSecurityCheckOpcodes).
7. Symbol Table & Scoping#
7.1 Symbol and Scope Types#
type SymbolScope string
const (
GlobalScope SymbolScope = "GLOBAL" // top-level let bindings
LocalScope SymbolScope = "LOCAL" // function-local let bindings
BuiltinScope SymbolScope = "BUILTIN" // built-in functions
FreeScope SymbolScope = "FREE" // captured from enclosing scope
FunctionScope SymbolScope = "FUNCTION" // self-reference for named functions
)
type Symbol struct {
Name string
Scope SymbolScope
Index int // offset in its storage (globals[], stack[bp+i], Free[i], Builtins[i])
}
7.2 `Define`#
func (st *SymbolTable) Define(name string) Symbol
- If
st.Outer == nil→GlobalScope, index =numDefinitions++ - Otherwise →
LocalScope, index =numDefinitions++
numDefinitions also becomes the CompiledFunction.NumLocals (the VM reserves
this many stack slots for the frame).
7.3 `Resolve` and Free Variable Promotion#
func (st *SymbolTable) Resolve(name string) (Symbol, bool)
Resolution order:
- Look up in
st.store(current scope). - If not found and
st.Outer != nil, recurse into outer scope. - If found in outer scope:
GlobalScopeorBuiltinScope→ return as-is (loaded viaOpGetGlobal/OpGetBuiltin).- Any other scope → call
defineFree(original):- Appends
originaltost.FreeSymbols. - Stores a new
FreeScopesymbol withIndex = len(FreeSymbols)-1. - Returns the free symbol.
- Appends
- If the original was
LocalScope, also mark that slot captured on the table that owns it. AFreeScopeoriginal needs no marking here: the recursiveResolvethat produced it already marked whichever table owns the slot, so the marking propagates outward on its own.
FreeSymbols is consumed at the end of function compilation (step 10 in §6.5)
to emit the instructions that push captured cells onto the stack before
OpClosure.
7.3.1 Boxed captures#
A captured local does not live in its frame slot; the slot holds an
*object.Cell and the value is inside it. Every closure that captures the
variable holds that same pointer, so there is exactly one storage location per
variable per call and a write through any path is a write all of them see. This
is what object.Environment.Update has always done in the tree-walking
evaluator, and before it the VM captured by value: a closure's writes went into
its own copy, and writing free i actually landed on local i — usually a
parameter.
The capture is discovered while the inner literal is compiled, by which time
the enclosing function has already emitted plain OpGetLocal/OpSetLocal for
the slot. So boxCapturedLocals rewrites those to OpGetLocalCell /
OpSetLocalCell once the scope closes. Only the opcode byte changes — the cell
forms take the same one-byte operand — so nothing moves and no jump target, line
table offset or loop patch has to be recomputed. The alternative, boxing every
local, needs no pass and puts a heap allocation and an indirection on the VM's
hottest path.
CompiledFunction.CapturedLocals carries the boxed slot indices, ascending, and
the VM allocates the cells in callClosure after the arguments are already
in their slots — which is why a captured parameter needs no special case.
CapturedLocals is semantic, not debug information, and is never stripped by
ByteCode.StripDebugInfo: a release build that lost it would silently stop
boxing.
Cells never reach gob and are never encrypted. cell.Value is encrypted on the
way in and decrypted on the way out, so the at-rest guarantee locals already had
is unchanged; encrypting the cell itself would return a new object, and a new
cell is a second storage location — the by-value copy the design exists to
remove.
Bytecode compiled before boxing existed has an empty CapturedLocals and no
cell opcodes, so nothing is boxed and OpGetFree finds a plain value and passes
it through. Such a program keeps its original behaviour exactly, including the
by-value capture.
Workers get their own cells. pmap, peach and spawn run the callback on
another VM and another goroutine, where a shared cell is a data race rather than
a shared accumulator. Each worker is handed a closure whose cells have been
copied, one level deep — the same rule as the globals snapshot beside it — so a
worker's writes to a captured variable stay local.
7.4 `DefineFunctionName`#
func (st *SymbolTable) DefineFunctionName(name string) Symbol
Creates a FunctionScope symbol at index 0. When the compiler resolves the
function's own name inside its body, loadSymbol emits OpCurrentClosure,
which pushes the currently-running closure without going through the variable
system. This allows direct recursion without a free-variable round-trip.
7.5 `loadSymbol` — Scope to Opcode Mapping#
func (c *Compiler) loadSymbol(s Symbol) {
switch s.Scope {
case GlobalScope: c.emit(code.OpGetGlobal, s.Index)
case LocalScope: c.emit(code.OpGetLocal, s.Index)
case BuiltinScope: c.emit(code.OpGetBuiltin,
code.BuiltinNameTableFlag|c.symbolTable.ReferenceBuiltin(s.Name))
case FreeScope: c.emit(code.OpGetFree, s.Index)
case FunctionScope: c.emit(code.OpCurrentClosure)
}
}
emitAssignStore is the write side of the same switch, and had been an if on
GlobalScope while the enum grew to five values — the defect §7.3.1 describes.
It writes globals, locals (OpSetLocal, rewritten to OpSetLocalCell if the
slot turns out to be boxed) and captures (OpSetFree), and refuses builtins and
the function's own name, neither of which is storage.
emitCapture is the third form: the capture list OpClosure consumes wants the
storage itself rather than a read through it, so a LocalScope original becomes
OpCaptureLocal, a FreeScope original OpCaptureFree, and the function's own
name stays OpCurrentClosure — captured by value, because it is not storage.
8. VM: Stack Machine Design#
8.1 Constants#
// global/const.go
StackSize = 2048 // initial stack capacity (doubles on overflow)
GlobalSize = 65536 // initial globals capacity (doubles on overflow)
MaxFrames = 2048 // initial call-stack frame capacity (doubles on overflow)
All three slices are dynamically resized using growSize (doubles until
large enough) and copy. There is no hard cap beyond memory limits.
8.2 VM Struct Layout#
type VM struct {
constants []object.Object // constant pool from compilation
stack []object.Object // value stack; TOS = stack[stackPointer-1]
stackPointer int // points one past top-of-stack
globals []object.Object // global variable storage
frames []*Frame // call stack
frameIndex int // index of the *next* free slot (current = frames[frameIndex-1])
inslen int // byte length of the main bytecode stream (key derivation input)
password string // XOR decryption key (empty = no encryption)
stepCount uint64 // instructions executed so far
// ... integrity scheduling fields (see §15) ...
secureMode bool // true = fatal errors on security violations
structDefs map[string]any // struct field name lists
enumDefs map[string]any // enum tag name lists
enforceSecurityCheckOpcodes bool // true when running with a password
}
8.3 Frame#
type Frame struct {
cl *object.Closure // currently executing closure
ip int // instruction pointer; starts at -1
bp int // base pointer: stack[bp..bp+NumLocals-1] are locals
}
func (f *Frame) Instructions() code.Instructions { return f.cl.Fn.Instructions }
The IP is initialised to -1. On every iteration the main loop increments it
before reading. This means the first instruction is read at ip=0 after the
first increment.
8.4 Stack Layout During a Call#
Before OpCall:
stack[...] = captured globals / earlier temps
stack[sp-argc-1] = closure/builtin being called ← fn
stack[sp-argc] = arg[0]
stack[sp-argc+1] = arg[1]
...
stack[sp-1] = arg[argc-1]execCall sets: newFrame.bp = sp - argc
sp = bp + fn.NumLocals (reserves local slots)
Inside function:
stack[bp + 0] = local variable 0 (first parameter fills this slot)
stack[bp + 1] = local variable 1
...
stack[bp + NumLocals-1] = last local
stack[sp..] = temporaries for expression evaluation
On return (OpReturnValue):
returnValue = pop()
sp = bp - 1 (unwinds frame including the fn slot below args)
push(returnValue)
On return (OpReturn / void):
sp = bp - 1
push(Null)
Important: bp - 1 rewinds past the closure slot that was on the stack
before the call, effectively consuming the function object itself.
9. VM: Execution Loop#
9.1 `Run()` Structure#
vm.ensureFrameBoundaries()
vm.validateSecurityCheckOpcodes("before-execution")loop:
while currentFrame.ip < len(currentFrame.Instructions()) - 1:
runIntegrityProbes() // SHA-256 + CFI checks
currentFrame.ip++
stepCount++
ip = currentFrame.ip
ins = currentFrame.Instructions()
opcodeByte = SecureXOROneAt(ins[ip], inslen, password, ip)
op = Opcode(opcodeByte)
switch op:
... (one case per opcode) ...
vm.validateSecurityCheckOpcodes("after-execution")
9.2 IP Advancement Protocol#
Opcodes with operands must advance the IP by the number of operand bytes consumed. This is done explicitly inside each case, after reading the operand. Pattern:
case code.OpConstant:
constIndex, _ := code.ReadUint16(ins[ip+1:], ...)
vm.currentFrame().ip += 2 // skip 2-byte operand
vm.push(vm.constants[constIndex])
Jump opcodes set IP to target - 1 so the main loop's ip++ at the top of
the next iteration lands exactly at the target byte:
case code.OpJump:
pos := int(readUint16(...))
vm.currentFrame().ip = pos - 1 // loop will +1 before fetching
OpJumpFalse still advances by 2 (ip += 2) when the condition is truthy
(i.e., the jump is not taken), to skip past the operand and continue linearly.
9.3 Per-Opcode Behaviour Summary#
| Opcode | IP advance | Push/Pop details |
|---|---|---|
OpConstant |
+2 | push constants[idx] |
OpPop |
0 | pop (discard) |
OpAdd/Sub/Mul/Div |
0 | pop b, pop a, push result |
OpMod |
0 | pop b, pop a, push a % b |
OpTrue/False/Null |
0 | push singleton |
OpEqual/UnEqual/Greater |
0 | pop b, pop a, push Boolean |
OpMinus |
0 | pop a, push -a |
OpBang |
0 | pop a, push !a |
OpBitAnd/BitOr/BitXor |
0 | pop b, pop a, push a op b (INTEGER only) |
OpShiftLeft/ShiftRight |
0 | pop b, pop a, push a op b; b < 0 errors |
OpBitNot |
0 | pop a, push ~a (INTEGER only) |
OpJump |
sets ip = target-1 | — |
OpJumpFalse |
+2 (not taken) or ip=target-1 | pop condition |
OpGetGlobal |
+2 | push decrypted globals[idx] |
OpSetGlobal |
+2 | pop, encrypt, store to globals[idx] |
OpGetLocal |
+1 | push decrypted stack[bp+idx] |
OpSetLocal |
+1 | pop, encrypt, store stack[bp+idx] |
OpArray |
+2 | pop n, push Array |
OpHash |
+2 | pop n (must be even: k,v pairs), push Hash |
OpIndex |
0 | pop index, pop obj, push result |
OpCall |
+1 | pop argc args + fn; push new frame |
OpReturnValue |
— | pop return val; pop frame; push val |
OpReturn |
— | pop frame; push Null |
OpGetBuiltin |
+1 | push the builtin resolved from BuiltinNames[idx & 0x7FFF] |
OpClosure |
+3 | pop numFree values; push Closure |
OpGetFree |
+1 | push Free[idx], through the cell if it is one |
OpSetFree |
+1 | pop, encrypt, store into Free[idx].Value |
OpGetLocalCell |
+1 | push decrypted cell(bp+idx).Value |
OpSetLocalCell |
+1 | pop, encrypt, store into cell(bp+idx).Value |
OpCaptureLocal |
+1 | push the *Cell in stack[bp+idx] itself |
OpCaptureFree |
+1 | push Free[idx] unread |
OpCurrentClosure |
0 | push current Closure |
OpChkDbg |
0 | detect debugger → error or warn |
OpChkSnd |
0 | detect sandbox → error or warn |
OpBreak |
0 | push Break{} sentinel |
OpContinue |
0 | push Continue{} sentinel |
OpMakeStruct |
+3 | pop fieldCount values; push Struct |
OpGetField |
+2 | pop struct; push field value |
OpSetField |
+2 | pop value, pop struct; set field; push struct |
OpEnumValue |
+4 | push EnumValue |
9.4 Global & Local Encryption at Runtime#
Values are encrypted on write and decrypted on read for globals and
locals when a password is set:
// Write path
vm.globals[idx] = vm.encryptForStorage(vm.pop())
vm.stack[bp+idx] = vm.encryptForStorage(vm.pop())
// Read path
vm.push(vm.decryptForUse(vm.globals[idx]))
vm.push(vm.decryptForUse(vm.stack[bp+idx]))
mutil.EncryptObject / mutil.DecryptObject wrap the object in / unwrap from
*object.Encrypted. If encryption fails (e.g., unsupported type), the object is
stored/returned as-is.
A boxed local slot holds an *object.Cell, and the cell is deliberately one
of the types EncryptObject passes through unchanged. Encryption returns a new
object, and a new cell is a second storage location — the by-value capture
boxing exists to remove. cell.Value goes through the same encrypt-on-write and
decrypt-on-read as any other local, so nothing is left in the clear.
10. Closures & Free Variables#
10.1 Compilation#
Given:
let outer = fn() {
let x = 5;
let inner = fn() { x };
inner
};
The compiler for inner:
- Sees
x→ resolves to outer'sLocalScope. - Calls
defineFree(x_local)→FreeSymbols = [{Name:"x", Scope:LOCAL, Index:0}]. - Marks outer's slot 0 captured, so outer's
CapturedLocalsis[0]. - After body compilation,
FreeSymbolshas one entry. - Outer's own stream is rewritten: the
OpSetLocal 0it emitted forlet x = 5becomesOpSetLocalCell 0. Same width, so nothing after it moves. - In the parent scope,
emitCapturefor each free symbol emitsOpCaptureLocal 0— the cell itself, not a copy of the value. - Emits
OpClosure <innerFnIdx> 1.
Bytecode for outer (CapturedLocals: [0], so the VM boxes slot 0 at frame
entry):
0000 OpConstant 0 ; integer 5
0003 OpSetLocalCell 0 ; let x = 5, written through the cell
0005 OpCaptureLocal 0 ; push slot 0's cell (the capture for inner)
0007 OpClosure 1 1 ; wrap constants[1] (inner fn) with 1 free var
0011 OpSetLocal 1 ; let inner = <closure>; nothing captures inner
0013 OpGetLocal 1 ; return inner
0015 OpReturnValue
10.2 Runtime: `pushClosure`#
func (vm *VM) pushClosure(constIndex, numFree int) error {
fn := vm.constants[constIndex].(*object.CompiledFunction)
free := make([]object.Object, numFree)
for i := 0; i < numFree; i++ {
free[i] = vm.stack[vm.stackPointer-numFree+i]
}
vm.stackPointer -= numFree
closure := &object.Closure{Fn: fn, Free: free}
return vm.push(closure)
}
The captured values are read from the stack in forward order (they were
pushed in forward order by the parent's emitCapture calls). What is copied
into Free is the pointer each OpCaptureLocal / OpCaptureFree pushed, so
the closure and the frame end up naming the same cell.
10.3 Accessing Free Variables#
case code.OpGetFree:
freeIndex := ReadUint8(...)
captured := vm.currentFrame().cl.Free[freeIndex]
if cell, ok := captured.(*object.Cell); ok {
captured = cell.Value
}
vm.push(vm.decryptForUse(captured))
Free holds cells, and OpSetFree writes back through them — so a closure's
write is visible to the enclosing frame and to every other closure over the same
variable, which is what makes an accumulator over a callback work.
The type test is not a guess. Nothing a program can compute is a *Cell, and
the only two entries that are not cells are decidable: the enclosing function's
own name, captured by value with OpCurrentClosure for recursion, and every
entry in bytecode compiled before boxing existed.
11. Structs at the Bytecode Level#
11.1 Definition (compile-time only)#
struct Point { x, y }
No bytecode is emitted. The compiler records:
c.structDefinitions["Point"] = []*ast.Identifier{{Value:"x"}, {Value:"y"}}
This map is carried in ByteCode.StructDefs and loaded into vm.structDefs at
VM construction.
11.2 Construction#
let p = Point { x: 1, y: 2 };
Compiled to:
OpConstant <idx of "Point"> ; push type name string
OpConstant <idx of 1> ; push field value 1
OpConstant <idx of 2> ; push field value 2
OpMakeStruct <typeIdx> 2 ; typeIdx = constant index of "Point", fieldCount = 2
OpSetGlobal <p_idx>
OpMakeStruct runtime behaviour:
- Reads
typeIdx(2 bytes) andfieldCount(1 byte). - Looks up
constants[typeIdx]→ must be*object.Stringcontaining the struct name. - Looks up
vm.structDefs[typeName]→ ordered field name list. - Verifies
len(fieldNames) == fieldCount(error if not). - Pops
fieldCountvalues from stack in reverse (last field popped first), mapping them tofieldNames[fieldCount-1-i]so the first field value pushed maps to the first field name. - Creates
&object.Struct{TypeName, Fields: map[string]Object{...}}.
11.3 Field Access / Mutation#
p.x ; OpGetField
p.x = 10 ; OpSetField
Field names are stored as string constants in the pool. The opcodes carry the constant index:
OpGetField <nameIdx> ; pop struct, push struct.Fields[constants[nameIdx]]
OpSetField <nameIdx> ; pop value, pop struct, set field, push struct back
Accessing a non-existent field pushes Null (no error). Setting a field on a
non-struct object is a runtime error.
12. Enums at the Bytecode Level#
12.1 Definition (compile-time only)#
enum Color { Red, Green, Blue }
No bytecode emitted. Compiler records:
c.enumDefinitions["Color"] = []string{"Red", "Green", "Blue"}
Carried in ByteCode.EnumDefs → vm.enumDefs.
12.2 Tag Reference#
let c = Color.Red;
Compiled to:
OpEnumValue <typeIdx> <tagIdx>
Both operands are 2-byte constant indices pointing to string constants "Color"
and "Red" respectively.
OpEnumValue runtime behaviour:
- Reads
typeIdx(2 bytes) andtagIdx(2 bytes); advances IP by 4. - Looks up both strings in the constant pool.
- Looks up
vm.enumDefs[typeName]→[]string. - Scans the tag list for the matching string; ordinal = position index (0-based).
- Returns error if the tag is not found.
- Pushes
&object.EnumValue{TypeName, Tag, Value: &Integer{int64(ordinal)}}.
13. Control Flow#
13.1 For Loop Structure#
for (let i = 0; i < 10; i = i + 1) { body }
Compiler emits (pseudocode):
[init: OpConstant 0, OpSetLocal 0] ; let i = 0
[conditionStart: label A]
[condition: push (i < 10)]
OpJumpFalse → loopEnd ; if false, exit
[body: ...]
[OpPop if last statement is expression]
[post: i = i + 1]
OpJump → A ; loop back
[loopEnd: label B]
A missing condition (for(;;)) emits OpTrue as the condition. A missing init
or post simply emits nothing for those phases.
13.2 `LoopContext` and Back-Patching#
type LoopContext struct {
breakPositions []int // byte offsets of OpJump instructions from break
continuePositions []int // byte offsets of OpJump instructions from continue
pendingAtEntry int // operands the enclosing expressions held at entry
}
When break or continue is encountered:
- Emit one
OpPopfor every operand the enclosing expressions have pushed since the loop was entered —c.pendingOperands - ctx.pendingAtEntryof them. Each target a break or a continue can reach reads the stack at the depth the loop was entered with, so a jump out of the middle of an expression has to drop that expression's operands on the way out. Without the pops,let t = [0, if (c) { break; } else { 1 }];leaves the array's first element behind and theOpPopmeant for a for-in's cursor takes it instead. - Emit
OpJump 9999(placeholder), after those pops. - Append the emitted position to
ctx.breakPositions/ctx.continuePositions.
After the loop body and post-increment are fully compiled:
- Patch all
breakPositions→loopEndaddress. - Patch all
continuePositions→conditionStartaddress (re-runs the condition check, which then falls through to the post-increment).
At runtime, OpBreak / OpContinue push sentinel objects (&object.Break{} /
&object.Continue{}). These are only meaningful in the evaluator (tree-walk)
path; the compiled VM path uses only the jump instructions.
13.3 While Loop Structure#
while (c) { body }
The same shape with the init and post phases absent:
[conditionStart: label A]
[condition: push c]
OpJumpFalse → loopEnd
[body: ...]
OpJump → A
[loopEnd: label B]
continue back-patches to A, which for this loop is the condition rather than
a post section — there is no post section to run.
13.4 For-In Loop Structure#
for (k, v in xs) { body }
[iterable: push xs]
OpIterInit ; [iter]
[head: label A]
OpIterNext → loopEnd, binds ; [iter] → [iter, key, value], or jump
[bind value: OpSetLocal / OpSetGlobal]
[bind key, when the loop binds two names]
[body: ...]
OpJump → A
[loopEnd: label B]
OpPop ; drop the cursor
break back-patches to B — before the OpPop — so the cursor is dropped on
every exit path. continue back-patches to A, so the advance is this loop's
post section.
Two things about the body differ from §13.1 and are not stylistic. The body
must be stack-neutral: the cursor lives underneath whatever the body leaves
behind, so a body that leaks one value per iteration has the next OpIterNext
reading that leftover as the cursor. And binds is read through the VM's
operand reader rather than indexed out of the instruction stream directly —
a raw byte index returns the obfuscated byte, which silently made a
two-binding loop bind one.
13.5 Match Expressions#
match (subject) { p1 | p2 => a, p3 => b, _ => c }
A match is an expression, so every path through it leaves exactly one value:
[subject: push it once]
[per alternative]
OpDup ; [subject, subject]
[pattern]
OpEqual ; [subject, bool]
OpJumpFalse → next alternative
[on a match]
OpPop ; drop the subject
[arm body]
[leave exactly one value]
OpJump → end
[no arm matched, and no `_` arm]
OpMatchFail ; pops the subject and raises
[end]
The subject lives on the stack behind OpDup rather than in a scratch local
because SymbolTable.Define never reuses a slot: a temporary per match would
spend one of the 256 one-byte local slots each time and eventually panic inside
code.Make. A wildcard arm compiles as the body alone — no compare, no jump —
and its presence is what omits the trailing OpMatchFail.
"Leave exactly one value" is decided from the syntax, not from the last
instruction emitted: if the arm body's last statement is an expression
statement, its OpPop is removed, and otherwise OpNull is emitted. The
instruction-stream test that §6.4 describes cannot be used here, because a
for (v in xs) also ends in an OpPop — the one dropping the cursor — and
removing that one leaks the iterator as the value of the arm. if branches go
through the same rule for the same reason.
14. Operand Encryption#
14.1 Key Derivation#
When a password string is provided to the VM, every opcode byte and every
operand byte is XOR-encrypted using security.SecureXOR*. The key material is
derived from:
- The
passwordstring. inslen— the total byte length of the main instruction stream.offset— the absolute byte position of the byte being encrypted/decrypted.
This makes the key unique per byte position in the stream, so the same operand value at different offsets produces different encrypted bytes.
14.2 Encryption in the VM Loop#
Every single byte read from the instruction stream goes through decryption:
// Opcode byte
opcodeByte, _ = security.SecureXOROneAt(ins[ip], int64(vm.inslen), vm.password, int64(ip))
// 2-byte operand
value, _ = code.ReadUint16(ins[ip+1:], int64(vm.inslen), vm.password, int64(ip+1))
// 1-byte operand
value, _ = code.ReadUint8(ins[ip+1:], int64(vm.inslen), vm.password, int64(ip+1))
The length and offset arguments in ReadUint16 / ReadUint8 must match the
actual position of those bytes in the stream. A mismatch produces a wrong value
silently — there is no authentication tag.
14.3 No Encryption Path#
When password == "" (the default for vm.New):
security.SecureXOROneAt(b, ...)is a no-op that returnsbunchanged.ReadUint16/ReadUint8still call the XOR functions but effectively just dobinary.BigEndian.Uint16.
15. Runtime Security Features#
15.1 Integrity Probes#
The VM maintains SHA-256 checksums of every CompiledFunction's instruction
slice:
frameIntegrity map[*object.CompiledFunction][32]byte
The checksum is computed at VM construction for the main function. For closures, it is computed when the frame is first pushed.
Probe schedule (jittered to frustrate timing analysis):
integrityEvery = 64 steps (base interval for instruction-hash checks)
integrityJitter = XorShift64 PRNG seeded from SHA-256(mainInstructions)nextProbeInterval = integrityEvery + (jitter % 31)
nextSweepInterval = 251 + (jitter % 83)
At each probe:
- Recompute
SHA-256(frame.Instructions()). - Compare to the stored checksum.
- If mismatch →
security.RecordIntegrityFailure+ApplyTamperResponse.
15.2 Control-Flow Integrity (CFI)#
frameBoundaries map[*object.CompiledFunction]map[int]struct{}
Before execution begins, buildInstructionBoundaries walks the encrypted
instruction stream and records the byte offset of every opcode (i.e., every
legal IP value). Jump targets are validated by checking whether the resolved
ip (or ip+1 to account for the pre-decrement protocol) is in this set.
CFI is only active when password != "".
15.3 Security Opcodes#
| Opcode | Trigger | Secure mode behaviour | Non-secure mode |
|---|---|---|---|
OpChkDbg |
security.IsDebuggerPresent() returns true |
Return ErrDebuggerDetected |
Log warning to stderr, continue |
OpChkSnd |
security.IsSandboxed() returns true |
Return ErrSandboxDetected |
Log warning to stderr, continue |
secureMode = true for any VM constructed with NewWithPassword*.
secureMode = false for plain vm.New.
15.4 Tamper Response#
security.ApplyTamperResponse(event, stage, secureMode, err) is the central
tamper handler:
- If
secureMode→ return the error immediately (halts the VM). - If not
secureMode→ log a warning, returnnil(execution continues).
15.5 Secure Memory Cleanup#
vm.CleanupRuntimeSensitiveData(clearGlobals bool, clearConstants bool)
vm.CleanupSensitiveData(clearGlobals bool) // alias: always clears constants
These zero-out the stack, optionally the globals, and optionally zero and nil
the CompiledFunction.Instructions byte slices using security.SecureZero
(which prevents the compiler from optimising the wipe away). The password string
is also cleared.
Intended to be called immediately after vm.Run() returns, before the VM
object is GC'd.
15.6 Polymorphic Marker Stripping#
The marker never reaches the VM. generator.encode removes it before anything
is written, using the level Compiler.PolymorphicLevel() reports, so what the
runtime loads is the instruction stream and nothing else.
The VM used to strip it itself, by guessing: it read the last two bytes and
treated [0xFF, n<=10] in either order as a marker. Ordinary bytecode produces
that pattern on its own -- OpConstant 255 encodes as 00 00 FF, and a
following OpPop makes the last two bytes FF 01 -- so real programs had two
real bytes cut off and died on "not enough bytes for operand". There is nothing
in the format that distinguishes a marker from instruction bytes that end the
same way, which is why the decision belongs to the compiler that knows, not to
the reader that has to guess.
If a marker ever did survive, 0xFF is not a defined opcode, so the VM reports
an undecodable instruction rather than silently running a shortened program.
16. Polymorphic Mutation Engine#
All four stages run at any non-zero mutation level:
InsertNOPs,InsertDeadCode,MutateOpcodesandRandomizeConstants.--mutation 0is the only setting that leaves the program byte-identical.
ReorderInstructionsused to be a fifth field with no implementation behind it. It has been removed: reordering needs a reverse mapping the VM has no way to carry, and a named stage that does nothing is how a stale opcode list once survived unnoticed.Until this was fixed,
RandomizeConstantswas the only stage that ran and it was gated at level 6 while the CLI defaulted to 5 -- so the default ran the engine and emitted unmutated bytecode.
16.1 Engine Configuration#
type PolymorphicEngine struct {
mutationLevel int // 0 = disabled, 1–10 = intensity
randomSeed int64 // for reproducible builds
rng *mathrand.Rand
}
Constructed with compiler.EnablePolymorphism(level) (random seed via
crypto/rand) or EnablePolymorphismWithSeed(level, seed) (deterministic).
16.2 Mutation Pipeline#
ByteCode.Mutate(bc):
if InsertNOPs → spliceFillers(bc, config) // every stream, jumps repointed
or InsertDeadCode
if RandomizeConstants → randomizeConstantPool(bc) // pool operands rewritten
if MutateOpcodes → mutateOpcodes(bc) // must be last
append PolymorphicMarker
NOP insertion and dead-code insertion are two stages but one pass. Both
splice a block at an instruction boundary and both need every jump in the stream
repointed afterwards, so running them separately would decode the boundaries and
rewrite the jumps twice to reach the same stream. spliceFillers picks a
generator per insertion point from whichever stages are enabled.
The order is load-bearing. NOP insertion and constant-pool randomization
both walk the instruction stream by operand width, so they have to run while the
opcode bytes still mean what code.Lookup says. Opcode remapping destroys that
and therefore runs last. It used to sit in the middle, which would have left the
constant-pool pass parsing a remapped stream.
16.3 NOP Insertion#
Splices a stack-neutral pair in after a random subset of instruction boundaries:
OpNull + OpPop
OpTrue + OpPop
OpFalse + OpPop
Rate: level × 1.5 % of instruction boundaries, drawn from the seeded RNG,
so --seed reproduces the build. Applied to the main stream and to every
CompiledFunction in the constant pool.
Two properties make this safe, and the absence of either is why the stage was gated off rather than merely unfinished:
- Jump targets are repointed.
OpJumpandOpJumpFalsecarry absolute offsets (code.JumpOperands), so one inserted byte ahead of a target redirects it into the middle of an instruction. Every target is remapped to its instruction's new offset. The old implementation did not do this at all. - Every variant is stack-neutral. A bare
OpPopused to be one of four choices, commented "safe if stack has something". At an arbitrary instruction boundary the stack holds the expression in progress, so that variant discarded a live value -- randomness decided whether the program was correct.
Nothing is inserted after the final instruction of a stream: the last value
popped is what LastPoppedStackElement reports, and a trailing push/pop pair
would make every program's result null.
If a stream cannot be decoded end to end, or a jump target is not an instruction
boundary, or a new offset will not fit a uint16, padding is declined and
the stream is returned untouched. A half-rewritten stream is not recoverable.
16.4 Opcode Remapping#
A Fisher-Yates shuffle of every opcode value using the deterministic RNG
creates a bijective mapping original → shuffled. Every opcode byte in the
instruction stream and in all CompiledFunction constants is rewritten through
this mapping; operand bytes are left exactly as they were.
The rewrite walks by operand width. It used to walk one byte at a time and
replace anything matching a defined opcode, which rewrote operand bytes as
though they were opcodes -- OpConstant 5 encodes as 00 00 05, and all three
of those bytes are defined opcode values.
The opcode set comes from code.AllOpcodes() rather than a list kept in the
engine, which is what keeps the mapping bijective: a hand-written list that
falls behind the code package produces a partial remap, where the opcodes it
covers take new values while the ones it missed keep theirs, and the two
collide. That drift had already happened (OpGreaterEqual, OpSetIndex) and
stayed invisible because the stage was off.
Bytes that are not defined opcodes map to themselves in both directions, so an undecodable byte stays undecodable instead of being laundered into a valid instruction.
Reversal. The inverse table is shipped in ByteCode.OpcodeMap, 256 entries
indexed by the byte found in the stream:
type ByteCode struct {
...
OpcodeMap []byte // mutated byte → real opcode; nil when not remapped
}
It travels with the program rather than being re-derived from the seed, because
nothing that runs a .mu knows the seed. It is gob-encoded with the rest of
ByteCode; an older file simply has no entry and decodes to nil, which is the
unmutated case.
The VM applies it at every point a byte becomes an opcode
(VM.decodeOpcode), which is three places, not one:
| Site | What breaks without it |
|---|---|
execLoop fetch |
wrong instruction, wrong operand width |
buildInstructionBoundaries |
control-flow integrity reports valid jumps as tampering |
scanInstructionsForSecurityCheckOpcodes |
a program carrying OpChkDbg/OpChkSnd is reported as having lost them |
A table that is not exactly 256 entries is treated as absent
(normalizeOpcodeMap) rather than indexed past its end.
Remapping is all or nothing: if any stream cannot be decoded, no stream is rewritten and no table is shipped. A program with some streams remapped and some not cannot be described by one inverse table.
Compatibility: a remapped
.murequires a runtime that understandsOpcodeMap. Standalone releases embed their runtime, so they are inherently matched; a.muhanded to an older CLI is not, and the file format carries no version negotiation.
16.5 Constant Pool Randomisation#
Fisher-Yates shuffle of the constant pool using the deterministic RNG, followed
by a rewrite of every operand that indexes the pool -- in the main instruction
stream and in each CompiledFunction constant.
The operands to rewrite are read from code.ConstantOperands, which names
OpConstant, OpClosure, OpMakeStruct, OpGetField, OpSetField and both
operands of OpEnumValue. The rewrite walks instructions by operand width
rather than scanning for opcode bytes, because an operand byte can hold any
value and would otherwise be misread as an opcode.
Missing an operand from
ConstantOperandsdoes not fail loudly: the operand still lands on a real pool entry, just the wrong one, so the program compiles clean and misbehaves inside the VM.TestWideOperandsAreClassifiedpins every two-byte operand as a constant-pool index, an instruction offset, or explicitly neither, so a new opcode cannot be added without that call being made for bothConstantOperandsandJumpOperands.
16.6 Polymorphic Marker Format#
[... instructions ...][ 0xFF ][ level_byte ]
The marker is appended to Instructions after all mutations. It is
encrypted together with the rest of the bytecode when a password is used.
Detection:
compiler.DetectPolymorphicLevel(instructions) int
Returns the level (0–10) or 0 if no valid marker is found.
17. Object Type System#
17.1 Core Interface#
type Object interface {
Type() ObjectType // string constant, e.g. "INTEGER"
Inspect() string // human-readable value
}
17.2 Object Types#
ObjectType constant |
Go struct | Notes |
|---|---|---|
INTEGER_OBJ |
*Integer |
Value int64 |
FLOAT_OBJ |
*Float |
Value float64 |
BOOLEAN_OBJ |
*Boolean |
Value bool; singletons global.True, global.False |
NULL_OBJ |
*Null |
singleton global.Null |
STRING_OBJ |
*String |
Value string |
BYTES_OBJ |
*Bytes |
Value []byte — a byte buffer; Inspect() is full hex |
ARRAY_OBJ |
*Array |
Elements []Object |
HASH_OBJ |
*Hash |
Pairs map[HashKey]HashPair |
COMPILED_FN_OBJ |
*CompiledFunction |
Instructions, NumLocals, NumParams, CapturedLocals |
CLOSURE_OBJ |
*Closure |
Fn *CompiledFunction, Free []Object — cells (§7.3.1) |
CELL_OBJ |
*Cell |
Value Object — a captured variable's storage; runtime-only, never gob, never encrypted |
BUILTIN_OBJ |
*Builtin |
Fn func(args ...Object) Object |
FUNCTION_OBJ |
*Function |
AST-level function (evaluator path only) |
RETURN_VALUE_OBJ |
*ReturnValue |
Sentinel (evaluator path only) |
ERROR_OBJ |
*Error |
Message string |
STRUCT_OBJ |
*Struct |
TypeName string, Fields map[string]Object |
ENUM_VALUE_OBJ |
*EnumValue |
TypeName, Tag string, Value *Integer (ordinal) |
ENCRYPTED_OBJ |
*Encrypted |
Value []byte — XOR-encrypted payload |
BREAK_OBJ |
*Break |
Control-flow sentinel |
CONTINUE_OBJ |
*Continue |
Control-flow sentinel |
QUOTE_OBJ |
*Quote |
Macro / meta-programming AST wrapper |
MACRO_OBJ |
*Macro |
Macro object (evaluator path only) |
LUA_PATCH_OBJ |
*LuaPatch |
Lua security patch payload |
17.3 `HashKey` Interface#
For types usable as hash keys, the object must implement:
type Hashable interface {
HashKey() HashKey
}
type HashKey struct {
Type ObjectType
Value uint64
Text string
}
Implemented by Integer, Float, Boolean, String and Bytes. Two keys are
one entry only when every field is equal, and every field is exact: a boolean,
an integer or a float's bits sit in Value, a string sits whole in Text, and
a buffer's SHA-256 sits in Text -- never the buffer, which may be classified
plaintext and would otherwise be copied into a string nothing can zero. Before
2.6.0 a string or a buffer was keyed by its FNV-1a-64 digest alone, so two
values with one digest were one entry.
17.4 `CompiledFunction`#
type CompiledFunction struct {
Instructions code.Instructions // bytecode for this function's body
NumLocals int // number of local variables (stack slots)
NumParams int // number of parameters
CapturedLocals []int // slots an inner function closes over (§7.3.1)
}
NumLocals includes the parameters (parameters are the first NumParams
locals). The VM advances stackPointer by NumLocals when entering a frame to
pre-allocate the local variable slots, then boxes the slots named in
CapturedLocals — after the arguments are already in place, so a captured
parameter needs no special case.
CapturedLocals is semantic, not debug information, and is never stripped by
ByteCode.StripDebugInfo: a release build that lost it would silently stop
boxing and resume capturing by value. Empty means "this function captures
nothing", which is also what every artifact compiled before boxing existed says,
and correctly so.
18. VM Constructor Variants#
| Constructor | Password | SecureMode | Globals | Notes |
|---|---|---|---|---|
vm.New(bc) |
— | true |
fresh | Plain execution; no CFI |
vm.NewWithPassword(bc, pw) |
✓ | true |
fresh | Full security; enforces security opcodes |
vm.NewWithPasswordMode(bc, pw, mode) |
✓ | configurable | fresh | |
vm.NewWithGlobalStore(bc, globals) |
— | true |
provided | REPL: share globals across compilations |
vm.NewWithGlobalStoreAndPassword(bc, globals, pw) |
✓ | true |
provided | |
vm.NewWithPasswordAndGlobalStore(bc, pw, globals) |
✓ | true |
provided | |
vm.NewWithPasswordAndGlobalStoreMode(bc, pw, globals, mode) |
✓ | configurable | provided | Maximum flexibility |
"SecureMode = true" means security violations abort execution. "false" means they emit a warning to stderr and continue.
19. Common Pitfalls#
P1: Reading Encrypted Operands Without Decryption#
Wrong:
idx := binary.BigEndian.Uint16(ins[ip+1 : ip+3])
Correct:
idx, err := code.ReadUint16(ins[ip+1:], int64(vm.inslen), vm.password, int64(ip+1))
Applies to every operand read inside the VM's Run() loop, including opcode
bytes themselves.
P2: Jump Off-by-One#
Jumps set ip = target - 1, not ip = target. The main loop does ip++
before fetching. If you set ip = target, the VM skips the first byte of the
target instruction.
P3: `OpHash` Operand is 2× Pair Count#
OpHash n expects n/2 key-value pairs, so n is always even. The compiler
emits len(node.Pairs) * 2. The VM pops exactly n objects to build the hash.
Confusing pair count with element count causes a stack underflow.
P4: Struct Field Pop Order#
OpMakeStruct pops fields in reverse order (last field first). The compiler
must push field values in forward declaration order. The mapping inside the
VM is:
fieldName := fieldNames[fieldCount-1-i] // i=0 → last field; i=fieldCount-1 → first field
P5: `OpSetField` Consumes and Re-Pushes the Struct#
stack: [..., structObj, value]
OpSetField: pop value, pop struct, mutate struct.Fields[name], push struct
The struct reference is consumed and re-pushed. Code after OpSetField has the
(mutated) struct on top of the stack. If the mutation result is not needed, emit
OpPop to discard it.
P6: `inslen` Must Match the Main Function's Byte Length#
The inslen field used in all XOR decryption calls is set once to
len(bc.Instructions) (the main function's length) and never changes, even when
executing inner closures. Inner functions' operands are decrypted with the same
key parameter. This is by design — changing it for inner functions would break
operand decryption.
P7: Do Not Disassemble Encrypted Bytecode with `Instructions.String()`#
Instructions.String() uses the unencrypted ReadOperands path. On encrypted
bytecode it will produce garbage output or crash. Use it only in tests or
non-password compilation paths.