Mutant Capability Reference
This is the canonical, category-grouped catalog of every Mutant builtin. There are currently 694 registered builtins across 41 capability categories.
Generated from the builtin metadata (
builtin/metadata.go) — the source of truth. Regenerate withgo run ./cmd/gendocs; check for drift withgo run ./cmd/gendocs -check. Do not hand-edit the tables below: signatures, parameter types, platforms, and counts are all read from the metadata, and edits here are overwritten.
This is the canonical, category-grouped catalog of every Mutant builtin. There are currently 694 registered builtins across 41 capability categories. For language syntax and keywords see MUTANT_LANGUAGE_REFERENCE.md; deep-dive guides are linked per category below.
How to read this reference#
- Fallible builtins return a
(value, err)pair, matching the language idiomlet value, err = some_call(...);. Checkerrbefore usingvalue. Infallible helpers return a bare value. - Parameter types are shown inline in each signature, e.g.
str_repeat(s: STRING, n: INTEGER). A parameter with no type shown accepts any value. These are the same contracts the language server checks a call against (thebuiltinArgTypediagnostic), and the same words the runtime uses when a call fails. A parameter shown asBYTES|STRINGaccepts either representation and the builtin hands back the one it was given;string_to_bytes(s, "raw")converts losslessly from a builtin that still returns text. - The Platforms column lists the operating systems a builtin actually works on.
allmeans it is pure-Go and cross-platform (it operates on captured artifacts, so it runs on any host). A restricted set (e.g.windows/linux) means the builtin fails honestly elsewhere — and the language server will flag such a call when you are editing on an unsupported OS (theplatformSupportdiagnostic). - Pure-Go, no cgo. The entire standard library builds and runs with
CGO_ENABLED=0on Windows, Linux, and macOS. - Every builtin has two spellings. A name with an underscore in it is also reachable through its family:
hash.blake2ishash_blake2,fs.readisfs_read, and so on for every name in this table. The two are one function — the dotted form is folded into the flat one at compile time rather than looked up in a list — so the tables below list only the flat name. A variable, parameter or imported module that already binds the family name wins, so no existing program changes meaning.
Platform-restricted builtins#
Almost every builtin is cross-platform. The exceptions:
| Builtin | Platforms | Note |
|---|---|---|
process_memory_scan |
windows, linux | fails honestly on other platforms |
process_modules |
windows, linux | fails honestly on other platforms |
cmd_run, exec_argv, exec_string, process_kill, and reg_open work on all platforms but have platform-specific behavior in one path; hover in the editor shows the note.
Standard Library (59)#
Core language primitives: collection and hash operations, first-class higher-order functions (map/filter/reduce/each/sort_by), math helpers, I/O, and runtime/security introspection.
| Builtin | Platforms | Description |
|---|---|---|
abs(x: INTEGER|FLOAT) -> INTEGER|FLOAT |
all | Absolute value (preserves INTEGER/FLOAT type); errors on the smallest INTEGER. |
avg(array: ARRAY) -> FLOAT |
all | Arithmetic mean of a numeric array (FLOAT); errors on empty. |
ceil(x: INTEGER|FLOAT) -> INTEGER |
all | Smallest integer >= x (INTEGER); an INTEGER is returned unchanged. |
clamp(x: INTEGER|FLOAT, lo: INTEGER|FLOAT, hi: INTEGER|FLOAT) -> INTEGER|FLOAT |
all | Constrains x to the range [lo, hi]. |
concat(a: ARRAY, b: ARRAY) -> ARRAY |
all | Returns a new array with the elements of a followed by b. |
contains(array: ARRAY, value) -> BOOLEAN |
all | Returns whether array contains value (by value equality). |
debug_status() -> (HASH, ERROR) |
all | Returns runtime/debugger status information. |
delete(hash: HASH, key: STRING|BYTES|INTEGER|FLOAT|BOOLEAN) -> HASH |
all | Returns a new hash with key removed. |
each(array: ARRAY, fn: FUNCTION) -> NULL |
all | Calls fn for each element for its side effects and returns null. fn takes (element) or (element, index). |
entries(hash: HASH) -> ARRAY |
all | Returns the hash as an array of [key, value] pairs (sorted by key). |
error(message: STRING, context?: STRING, related?: HASH) -> ERROR |
all | Constructs an error value carrying a message, an origin, and any related facts. Single-return: it cannot fail. |
filter(array: ARRAY, fn: FUNCTION) -> ARRAY |
all | Returns a new array of the elements for which fn is truthy. fn takes (element) or (element, index). |
first(array: ARRAY) -> ANY |
all | Returns the first element of an array. |
flatten(array: ARRAY) -> ARRAY |
all | Flattens one level of nested arrays. |
floor(x: INTEGER|FLOAT) -> INTEGER |
all | Largest integer <= x (INTEGER); an INTEGER is returned unchanged. |
get(hash: HASH, key: STRING|BYTES|INTEGER|FLOAT|BOOLEAN, default) -> ANY |
all | Returns hash[key], or default when the key is absent. |
gets() -> STRING |
all | Reads a full line of input from stdin and returns it as a STRING (newline trimmed). Use to_int/to_float/parse_int to convert. |
has_key(hash: HASH, key: STRING|BYTES|INTEGER|FLOAT|BOOLEAN) -> BOOLEAN |
all | Returns whether hash contains key. |
help(topic?: STRING, mode?: STRING) -> STRING |
all | Returns help text: an overview, a topic (keywords/builtins/examples/docs), or details for a specific builtin name. |
index_of(array: ARRAY, value) -> INTEGER |
all | Returns the first index of value in array, or -1. |
int_to_ip(n: INTEGER) -> STRING |
all | Converts a 32-bit integer to an IPv4 dotted-quad string. |
is_null(v) -> BOOLEAN |
all | Returns whether v is NULL. |
keys(hash: HASH) -> ARRAY |
all | Returns the hash keys as an array (sorted for determinism). |
last(array: ARRAY) -> ANY |
all | Returns the last element of an array. |
len(value: STRING|BYTES|ARRAY|HASH) -> INTEGER |
all | Returns the length of a string, buffer, array, or hash. A string is measured in bytes, not characters: len("José") is 5, because é takes two bytes. s[i], a for-in loop, str_substr, str_char_at and the str_pad builtins count characters (runes), so len(s) is not their bound once a string holds a character that takes more than one byte. A byte that is not valid UTF-8 counts as one character, so it does not separate the two counts. |
map(array: ARRAY, fn: FUNCTION) -> ARRAY |
all | Returns a new array of fn applied to each element. fn takes (element) or (element, index). |
max(value: INTEGER|FLOAT, ...values: INTEGER|FLOAT) -> INTEGER|FLOAT |
all | Returns the largest of the numeric arguments (original type preserved). |
merge(a: HASH, b: HASH) -> HASH |
all | Returns a new hash combining a and b (b wins on key conflicts). |
min(value: INTEGER|FLOAT, ...values: INTEGER|FLOAT) -> INTEGER|FLOAT |
all | Returns the smallest of the numeric arguments (original type preserved). |
mod(a: INTEGER|FLOAT, b: INTEGER|FLOAT) -> INTEGER|FLOAT |
all | Returns a modulo b; errors on b=0. Integer mod when both are INTEGER. |
peach(array: ARRAY, fn: FUNCTION, workers?: INTEGER) -> NULL |
all | Like each, but calls fn on elements concurrently for their side effects and returns null. fn takes (element) or (element, index). Each worker runs on its own VM with a snapshot of globals, so fn should report through a shared store (cache_*/db_*) rather than by assigning to a global. |
pmap(array: ARRAY, fn: FUNCTION, workers?: INTEGER) -> ARRAY |
all | Like map, but applies fn to elements concurrently and returns results in the original order. fn takes (element) or (element, index). Each worker runs on its own VM with a snapshot of globals, so fn should be self-contained: it cannot write back to a global. |
pop(array: ARRAY) -> ARRAY |
all | Returns a new array without the last element. |
pow(x: INTEGER|FLOAT, y: INTEGER|FLOAT) -> FLOAT |
all | Returns x raised to the power y (FLOAT). |
push(array: ARRAY, value) -> ARRAY |
all | Returns a new array with value appended. |
putf(format, ...values) -> NULL |
all | Formats and prints values using a format string. |
putln(...values) -> NULL |
all | Prints values separated by spaces, followed by a newline. |
range(start: INTEGER, end: INTEGER, step?: INTEGER) -> []INTEGER |
all | Returns an array of integers from start (inclusive) to end (exclusive); step defaults to 1. |
reduce(array: ARRAY, fn: FUNCTION, initial) -> ANY |
all | Folds the array to a single value: fn(accumulator, element) starting from initial. |
rest(array: ARRAY) -> ARRAY |
all | Returns a new array without the first element. |
reverse(array: ARRAY) -> ARRAY |
all | Returns a reversed copy of an array. |
round(x: INTEGER|FLOAT) -> INTEGER |
all | Nearest integer to x (INTEGER); an INTEGER is returned unchanged. |
sandbox_status() -> (HASH, ERROR) |
all | Returns sandbox-detection status information. |
security_diagnostics() -> (HASH, ERROR) |
all | Returns security diagnostics for the current runtime. |
serve_arg() -> (ANY, ERROR) |
all | Inside a net_serve handler, returns the shared arg passed to net_serve; null otherwise. |
serve_conn() -> (INTEGER|NULL, ERROR) |
all | Inside a net_serve handler, returns the connection handle (INTEGER); null otherwise. |
set(hash: HASH, key: STRING|BYTES|INTEGER|FLOAT|BOOLEAN, value) -> HASH |
all | Returns a new hash with key set to value (original unchanged). |
sleep_ms(ms: INTEGER) -> (BOOLEAN, ERROR) |
all | Blocks the current handler for ms milliseconds. |
slice(array: ARRAY, start: INTEGER, end: INTEGER) -> ARRAY |
all | Returns the sub-array array[start:end] (bounds-clamped). |
sort(array: ARRAY) -> ARRAY |
all | Returns a sorted copy of an array (all numbers or all strings). |
sort_by(array: ARRAY, fn: FUNCTION) -> ARRAY |
all | Returns a new array stably sorted by the key fn returns for each element (INTEGER/FLOAT/STRING keys). |
sqrt(x: INTEGER|FLOAT) -> FLOAT |
all | Returns the square root of x (FLOAT); errors on negative x. |
string_to_bytes(s: STRING, encoding: STRING) -> (BYTES, ERROR) |
all | Converts text to a BYTES buffer under a named encoding: "raw", "utf8" (validated), "latin1", "hex" or "base64". Returns (bytes, err). |
sum(array: ARRAY) -> INTEGER|FLOAT |
all | Sum of a numeric array (INTEGER if all elements are integers); errors if an all-integer total overflows. |
type_of(v) -> STRING |
all | Returns the object type name of v (e.g. INTEGER, STRING, ARRAY). |
unique(array: ARRAY) -> ARRAY |
all | Returns a new array with duplicate values removed (order preserved). |
values(hash: HASH) -> ARRAY |
all | Returns the hash values as an array (ordered by sorted key). |
with_resource(resource, closer: STRING|FUNCTION, fn: FUNCTION) -> (ANY, ERROR) |
all | Calls fn with a resource an open call returned and always closes it afterwards -- whether fn returns a value, returns an error, or fails outright. closer is the name of a closing builtin ("ntfs_close") or a function taking the resource. If the open itself failed, fn never runs and the open's error comes back unchanged, so wrapping an existing call in with_resource does not change what the program sees. Returns (value, err): value is what fn returned, and err is the first failure among the open, an error fn returned, and the close. When fn and the close both fail, fn's error is the one returned and the close's is attached to it as related["close_error"]. |
zip(a: ARRAY, b: ARRAY) -> ARRAY |
all | Returns an array of [a[i], b[i]] pairs up to the shorter length. |
Testing (10)#
What mutant test reads. test(name, fn) names a test and runs it where it is written, so a test file reads top to bottom and a test declared inside another is a subtest of it; before_each/after_each register fixtures for the tests declared after them. The assertions record what they saw against the test that is running AND return it, so a failure is both something the report can name with a file and line and an ordinary value the program can look at. A test that dies is caught at its own boundary and costs the file no other test. See TESTING.md.
| Builtin | Platforms | Description |
|---|---|---|
after_each(fn: FUNCTION) -> NULL |
all | Registers fn to run after each test declared after this call, at this nesting level and inside it. It runs whether the test passed, failed, or ended in an error. |
assert(condition, message?: STRING) -> BOOLEAN |
all | Fails the current test unless condition is truthy. |
assert_contains(container: STRING|ARRAY|HASH, value, message?: STRING) -> BOOLEAN |
all | Fails the current test unless container holds value: a substring of a string, an element of an array, or a key of a hash. |
assert_eq(got, want, message?: STRING) -> BOOLEAN |
all | Fails the current test unless got equals want. Scalars compare by value; arrays, hashes and structs compare part by part, as == compares them, so key order does not matter and floats that print alike can still differ. |
assert_err(value, substring?: STRING) -> BOOLEAN |
all | Fails the current test unless value is an error, optionally requiring its message to contain substring. This is what the second binding of a (value, err) call is checked with. |
assert_ne(got, unwanted, message?: STRING) -> BOOLEAN |
all | Fails the current test when got equals unwanted, compared the way assert_eq compares. |
assert_ok(value, message?: STRING) -> BOOLEAN |
all | Fails the current test when value is an error, quoting the error's own message. This is the check to put on the second binding of a (value, err) call that is expected to succeed. |
before_each(fn: FUNCTION) -> NULL |
all | Registers fn to run before each test declared after this call, at this nesting level and inside it. A failure in fn fails the test it was preparing. |
fail(message: STRING) -> ERROR |
all | Fails the current test unconditionally with the given message. For the branch a test should never reach. |
test(name: STRING, fn: FUNCTION) -> BOOLEAN |
all | Runs fn as a named test and records whether it passed. Tests run where they are written, in order; a test declared inside another is a subtest of it. A runtime error inside fn fails that test and the file keeps going. |
Concurrency (8)#
Run work alongside the rest of the program and pass values between the pieces. spawn starts a closure on its own VM and hands back a handle for task_wait/task_done; chan_* moves values between them. Each task gets a snapshot of globals, so a channel or a return value is the way back, not a shared variable. For applying one callback across an array, reach for pmap/peach in the standard library instead.
| Builtin | Platforms | Description |
|---|---|---|
chan_close(handle: INTEGER) -> (BOOLEAN, ERROR) |
all | Closes a channel, waking every waiting sender and receiver. Returns true when this call did the closing and false when the channel was already closed. Receivers can still drain values that were already queued; the handle is reclaimed once enough other channels have been closed after it, after which it reports as unknown. |
chan_new(capacity?: INTEGER) -> (INTEGER, ERROR) |
all | Creates a channel for passing values between concurrently running code and returns its handle. Capacity 0 (the default) is unbuffered, so a send waits for a receive; a positive capacity lets that many values queue first. At most 1024 channels may be open at once; close each one with chan_close when you are done with it. |
chan_recv(handle: INTEGER, timeoutMs?: INTEGER) -> (HASH, ERROR) |
all | Takes the next value off a channel, waiting for one if the channel is empty. Returns {ok, value, closed, timeout}; check ok before reading value, since null is itself a sendable value. Values already queued are delivered even after the channel is closed. |
chan_send(handle: INTEGER, value, timeoutMs?: INTEGER) -> (BOOLEAN, ERROR) |
all | Puts a value on a channel, waiting for a receiver if the channel is full. Returns true once the value is handed over and false if the timeout ran out first; sending on a closed channel is an error. |
chan_try_recv(handle: INTEGER) -> (HASH, ERROR) |
all | Takes a value off a channel only if one is already waiting, and never blocks. Returns the same {ok, value, closed, timeout} shape as chan_recv. |
spawn(fn: FUNCTION, arg?) -> (INTEGER, ERROR) |
all | Runs fn on its own VM alongside the rest of the program and returns a task handle to collect it with. fn takes no arguments, or one if arg is given. The task sees a snapshot of globals taken at the spawn, so it should report back through its return value or a channel rather than by assigning to a global. |
task_done(handle: INTEGER) -> (BOOLEAN, ERROR) |
all | Reports whether a spawned task has finished, without waiting for it. |
task_wait(handle: INTEGER, timeoutMs?: INTEGER) -> (ANY, ERROR) |
all | Waits for a spawned task and returns the value its function returned. Whatever stopped the task arrives in the error slot. Without a timeout it waits indefinitely; running out of time is reported as an error and leaves the task collectable. Collecting a task releases its handle, so wait for it once. |
Strings (18)#
Rune-aware string manipulation: case, trimming, padding, slicing, joining, and formatting. Pairs with the text_*/regex_* matching family.
| Builtin | Platforms | Description |
|---|---|---|
str_char_at(s: STRING, index: INTEGER) -> STRING |
all | Returns the rune at index as a string. |
str_ends_with(s: STRING, suffix: STRING) -> BOOLEAN |
all | Returns whether s ends with suffix. |
str_format(format: STRING, ...values) -> STRING |
all | Returns a printf-style formatted string (like putf but returns instead of printing). |
str_join(array: ARRAY, sep: STRING) -> STRING |
all | Joins an array of strings with sep (inverse of text_split). |
str_lower(s: STRING) -> STRING |
all | Returns s with all letters lower-cased. |
str_pad_left(s: STRING, width: INTEGER, pad: STRING) -> STRING |
all | Left-pads s with pad until it reaches width runes. |
str_pad_right(s: STRING, width: INTEGER, pad: STRING) -> STRING |
all | Right-pads s with pad until it reaches width runes. |
str_repeat(s: STRING, n: INTEGER) -> STRING |
all | Returns s repeated n times. |
str_reverse(s: STRING) -> STRING |
all | Returns s reversed (rune-aware). |
str_starts_with(s: STRING, prefix: STRING) -> BOOLEAN |
all | Returns whether s begins with prefix. |
str_substr(s: STRING, start: INTEGER, length: INTEGER) -> STRING |
all | Returns length runes of s starting at rune index start (clamped to bounds). |
str_title(s: STRING) -> STRING |
all | Upper-cases the first letter of each word in s. |
str_trim(s: STRING) -> STRING |
all | Returns s with leading and trailing whitespace removed. |
str_trim_left(s: STRING, cutset: STRING) -> STRING |
all | Trims any leading characters in cutset from s. |
str_trim_prefix(s: STRING, prefix: STRING) -> STRING |
all | Removes prefix from s if present. |
str_trim_right(s: STRING, cutset: STRING) -> STRING |
all | Trims any trailing characters in cutset from s. |
str_trim_suffix(s: STRING, suffix: STRING) -> STRING |
all | Removes suffix from s if present. |
str_upper(s: STRING) -> STRING |
all | Returns s with all letters upper-cased. |
Text Analysis (14)#
Substring search, splitting/replacing, regular expressions, and fuzzy matching (Levenshtein, Jaro-Winkler, similarity).
| Builtin | Platforms | Description |
|---|---|---|
regex_capture_groups(pattern: STRING, input: STRING) -> ([]STRING, ERROR) |
all | Returns full regex capture array (full match plus groups). |
regex_find(pattern: STRING, input: STRING) -> (STRING|NULL, ERROR) |
all | Finds the first regex match in input. |
regex_find_all(pattern: STRING, input: STRING, limit?: INTEGER) -> ([]STRING, ERROR) |
all | Finds all regex matches with optional result limit. |
regex_match(pattern: STRING, input: STRING) -> (BOOLEAN, ERROR) |
all | Returns whether regex pattern matches input. |
regex_replace(pattern: STRING, input: STRING, replacement: STRING) -> (STRING, ERROR) |
all | Replaces all regex matches in input with replacement text. |
text_contains(haystack: STRING, needle: STRING) -> BOOLEAN |
all | Returns whether a string contains a substring. |
text_count(haystack: STRING, needle: STRING) -> INTEGER |
all | Counts non-overlapping substring occurrences. |
text_fuzzy_find(query: STRING, candidates: ARRAY, maxDistance?: INTEGER) -> HASH |
all | Finds the closest fuzzy match in an array of candidate strings. |
text_index(haystack: STRING, needle: STRING) -> INTEGER |
all | Returns the byte offset of the first occurrence of needle in haystack, or -1. It counts bytes, as len does, so it is a position for bytes_slice and bytes_get. s[i], str_substr and str_char_at count characters (runes), and the two agree only up to the first character that takes more than one byte: in "José: admin" the colon is byte 5 and character 4. A byte that is not valid UTF-8 counts as one character, so it does not separate them. |
text_jaro_winkler(left: STRING, right: STRING) -> FLOAT |
all | Computes Jaro-Winkler string similarity score. |
text_levenshtein(left: STRING, right: STRING) -> INTEGER |
all | Computes Levenshtein edit distance between two strings. |
text_replace(text: STRING, old: STRING, new: STRING, count?: INTEGER) -> STRING |
all | Replaces substring occurrences in text; count limits how many (all by default). |
text_similarity(left: STRING, right: STRING) -> FLOAT |
all | Computes normalized Levenshtein similarity between two strings. |
text_split(text: STRING, sep: STRING) -> []STRING |
all | Splits text by separator and returns an array of parts. |
Structured Data (46)#
The formats evidence actually arrives in. JSON and NDJSON/JSONL (Zeek, Elastic bulk, OCSF), CSV/TSV (every SIEM export and hash set), XML (Scheduled Tasks, OOXML, Nessus, plist), YAML including multi-document streams (Sigma rulesets), TOML, and the binary serializations -- CBOR for COSE/WebAuthn, MessagePack for agent traffic, plus schemaless walkers for protobuf and DER/ASN.1 that report structure when no .proto or ASN.1 module is at hand. Then base64/base32/hex/URL encoding, gzip/zlib compression, base conversion, and type conversion. Every decoder shares one bridge, so a byte string is a BYTES buffer and a timestamp is RFC 3339 no matter which format it came from. See STRUCTURED_DATA.md.
| Builtin | Platforms | Description |
|---|---|---|
base32_decode(s: STRING) -> (STRING, ERROR) |
all | Decodes standard base32; returns (bytes, err). |
base32_encode(s: STRING|BYTES) -> STRING |
all | Standard base32-encodes s. |
base64_decode(s: STRING) -> (STRING, ERROR) |
all | Decodes standard base64; returns (bytes, err). |
base64_decode_bytes(s: STRING) -> (BYTES, ERROR) |
all | Decodes standard base64 into a BYTES buffer; returns (bytes, err). |
base64_encode(s: STRING|BYTES) -> STRING |
all | Standard base64-encodes s. |
base64url_decode(s: STRING) -> (STRING, ERROR) |
all | Decodes URL-safe base64; returns (bytes, err). |
base64url_encode(s: STRING|BYTES) -> STRING |
all | URL-safe base64-encodes s. |
cbor_encode(value: STRING|BYTES|INTEGER|FLOAT|BOOLEAN|NULL|ARRAY|HASH|STRUCT) -> (BYTES, ERROR) |
all | Serializes a Mutant value as canonical CBOR: map keys are sorted, INTEGER and BOOLEAN keys stay integers and booleans, and integers use their shortest form, so hash_sha256(cbor_encode(v)) is a stable identifier for v. A buffer encodes as a CBOR byte string. |
cbor_parse(data: BYTES|STRING) -> (ANY, ERROR) |
all | Parses CBOR -- COSE, WebAuthn, IoT telemetry. Byte strings decode to buffers, not text; tagged items are preserved as {_cbor_tag, value} rather than dropped; integer map keys are supported (COSE labels them that way); duplicate keys are refused. Nesting is capped at 64 levels. |
csv_parse(data: BYTES|STRING, options?: HASH) -> (ARRAY, ERROR) |
all | Parses CSV/TSV into an array of hashes keyed by the header row. options: delimiter (default ","), comment, header (default true), trim_space, lazy_quotes. A UTF-8 BOM is stripped, duplicate column names are refused rather than silently resolved, and fields beyond the header land in an _extra array. With header:false each row is an array of strings instead. |
csv_stringify(rows: ARRAY, options?: HASH) -> (STRING, ERROR) |
all | Serializes an array of hashes (or arrays) as CSV/TSV. options: delimiter, header (default true), columns (explicit column order), crlf. Without an explicit columns list the header is the sorted union of every row's keys, so a row missing a key writes an empty field rather than shifting the others. |
der_parse(data: BYTES|STRING) -> ([]HASH, ERROR) |
all | Walks DER/ASN.1 structurally, without a schema -- what x509_parse and pem_decode already need internally, and what a certificate extension or a Kerberos ticket needs when no ASN.1 module is at hand. Each node reports {offset, header_len, length, class, tag, constructed, tag_name}, constructed nodes carry children, and primitives carry raw value bytes plus a decoded rendering for universal types (OIDs dotted, big INTEGERs as exact text -- decimal, or 0x-prefixed hexadecimal past 4 KiB -- rather than truncated, times as RFC 3339, with a UTCTime's two-digit year read as RFC 5280 reads it: 50 to 99 are the 1900s). BER indefinite length is refused by name. |
from_base(s: STRING, base: INTEGER) -> (INTEGER, ERROR) |
all | Parses s as an integer in the given base (2–36); returns (int, err). |
gunzip(s: STRING|BYTES, max_bytes?: INTEGER) -> (STRING, ERROR) |
all | Gzip-decompresses s; returns (bytes, err). Refuses to produce more than 1000x its input, capped at 1 GiB, unless max_bytes says otherwise. |
gunzip_bytes(s: STRING|BYTES, max_bytes?: INTEGER) -> (BYTES, ERROR) |
all | Gzip-decompresses s into a BYTES buffer; returns (bytes, err). Refuses to produce more than 1000x its input, capped at 1 GiB, unless max_bytes says otherwise. |
gzip(s: STRING|BYTES) -> STRING |
all | Gzip-compresses s (returns a byte string). |
hex_decode(s: STRING) -> (STRING, ERROR) |
all | Decodes a hex string to bytes; returns (bytes, err). |
hex_decode_bytes(s: STRING) -> (BYTES, ERROR) |
all | Decodes a hex string into a BYTES buffer; returns (bytes, err). |
hex_encode(s: STRING|BYTES) -> STRING |
all | Hex-encodes a byte string to lowercase hex. |
json_parse(text: STRING) -> (ANY, ERROR) |
all | Parses one JSON value into Mutant values. Exactly one: data after the value, a key repeated within one object, bytes that are not UTF-8 and a \u escape that is half a surrogate pair are refused rather than quietly resolved. An integer too wide for INTEGER comes back as its exact decimal text, as it does from every other decoder. |
json_stringify(value: STRING|BYTES|INTEGER|FLOAT|BOOLEAN|NULL|ARRAY|HASH) -> (STRING, ERROR) |
all | Serializes Mutant values into JSON text. |
msgpack_encode(value: STRING|BYTES|INTEGER|FLOAT|BOOLEAN|NULL|ARRAY|HASH|STRUCT) -> (BYTES, ERROR) |
all | Serializes a Mutant value as MessagePack with sorted map keys and compact integers, so the output is deterministic for a given value. INTEGER and BOOLEAN keys are written as integers and booleans, not as text. |
msgpack_parse(data: BYTES|STRING) -> (ANY, ERROR) |
all | Parses MessagePack -- agent check-ins, queue payloads, Fluentd forward traffic. Binary values decode to buffers, not text; map keys keep their type, and a key repeated in one map is refused. Input carrying more than one value is reported rather than ignored: a blob that decodes and keeps going is either a stream or not what it was thought to be. A document nested past 256 levels, or whose containers declare more values than its bytes can hold, is refused before it is decoded. |
ndjson_parse(data: BYTES|STRING) -> ([]ANY, ERROR) |
all | Parses newline-delimited JSON (NDJSON/JSONL) -- the wire format of Zeek, Elastic bulk and OCSF streams. Blank lines are skipped; every other line is read as json_parse reads a document, and a malformed line fails with its line number rather than silently truncating the stream. |
ndjson_stringify(values: ARRAY) -> (STRING, ERROR) |
all | Serializes an array as newline-delimited JSON, one value per line, with a trailing newline so the output concatenates with another stream. |
parse_float(s: STRING) -> (FLOAT, ERROR) |
all | Parses s as a float; returns (float, err). |
parse_int(s: STRING, base: INTEGER) -> (INTEGER, ERROR) |
all | Parses s as an integer in base (0 auto-detects); returns (int, err). |
plist_parse(path: STRING) -> (ANY, ERROR) |
all | Parses an Apple property list (binary bplist00 or XML) into a Mutant value: dict->hash, array->array, string/integer/real/bool as scalars; dates and data become strings. Returns (value, err). |
protobuf_parse(data: BYTES|STRING) -> ([]HASH, ERROR) |
all | Walks protobuf wire format without a .proto -- the situation an analyst holding a gRPC capture is actually in. Each field reports {field, wire_type, offset} plus every reading its bytes admit: a varint as itself, as zigzag and as bool; a length-delimited field as bytes, plus text and message when those parse. Naming the ambiguity is the honest thing a schemaless reader can do. Past 32 levels the message reading is not tried, and message_unchecked: true says so; a group's fields are always read, or the call is refused. One call reports at most 262,144 fields, counted across every nested reading. |
to_base(n: INTEGER, base: INTEGER) -> STRING |
all | Formats integer n in the given base (2–36). |
to_bool(v: BOOLEAN|INTEGER|FLOAT|STRING) -> (BOOLEAN, ERROR) |
all | Converts a bool/number/string to BOOLEAN; returns (bool, err). |
to_float(v: INTEGER|FLOAT|BOOLEAN|STRING) -> (FLOAT, ERROR) |
all | Converts a number/bool/string to FLOAT; returns (float, err). |
to_int(v: INTEGER|FLOAT|BOOLEAN|STRING) -> (INTEGER, ERROR) |
all | Converts a number/bool/string to INTEGER; a STRING is read in base 10, so a zero-padded field such as "08" or "010" is the decimal number it looks like. Use parse_int(s, 0) for a string carrying a 0x, 0b or 0o prefix. Returns (int, err). |
to_string(v) -> STRING |
all | Converts any value to its STRING representation. |
toml_parse(data: BYTES|STRING) -> (HASH, ERROR) |
all | Parses TOML into a hash. Offset date-times become RFC 3339 strings in UTC, so they sort against every other timestamp the language produces; local date-times, dates and times name no instant and keep the form they were written in (1979-05-27T07:32:00, 1979-05-27, 07:32:00). Arrays and inline tables nested past 256 levels, and keys of more than 256 dotted parts, are refused before decoding. |
toml_stringify(value: HASH|STRUCT) -> (STRING, ERROR) |
all | Serializes a hash or struct as TOML. The top level must be a table -- TOML has no other document shape -- and a buffer is written as hex, matching yaml_stringify. |
url_decode(s: STRING) -> (STRING, ERROR) |
all | URL query-unescapes s; returns (value, err). |
url_encode(s: STRING) -> STRING |
all | URL query-escapes s. |
xml_find(node: HASH, selector: STRING) -> ([]HASH, ERROR) |
all | Selects descendants of a parsed element by a slash-separated path. Three rules, not XPath: a name matches an element, * matches any single level, ** matches any number of levels including none (so "**/Task" also finds a direct child). Each element is returned once, in document order, however many ways the path reaches it. |
xml_parse(data: BYTES|STRING) -> (HASH, ERROR) |
all | Parses XML into a node tree: {name, namespace, attrs, text, children}. Comments, processing instructions and directives are skipped, undeclared entities fail rather than expand (closing billion-laughs and XXE), and windows-1252/iso-8859-1/-15/windows-1251 documents are decoded by their declared charset -- an unrecognised charset fails by name rather than being misdecoded. |
yaml_parse(data: BYTES|STRING) -> (ANY, ERROR) |
all | Parses the first YAML document -- Sigma rules, CI config, cloud manifests. A file of ----separated documents needs yaml_parse_all, which is why this one exists as a pair. |
yaml_parse_all(data: BYTES|STRING) -> ([]ANY, ERROR) |
all | Parses every document in a multi-document YAML stream. A Sigma ruleset is one file of ----separated documents, and yaml_parse would return only the first, silently. |
yaml_stringify(value: STRING|BYTES|INTEGER|FLOAT|BOOLEAN|NULL|ARRAY|HASH|STRUCT) -> (STRING, ERROR) |
all | Serializes a Mutant value as YAML. A buffer is written as hex, matching Inspect and json_stringify; string_to_bytes(s, "hex") converts it back. |
zlib_compress(s: STRING|BYTES) -> STRING |
all | Zlib-compresses s (returns a byte string). |
zlib_decompress(s: STRING|BYTES, max_bytes?: INTEGER) -> (STRING, ERROR) |
all | Zlib-decompresses s; returns (bytes, err). Refuses to produce more than 1000x its input, capped at 1 GiB, unless max_bytes says otherwise. |
zlib_decompress_bytes(s: STRING|BYTES, max_bytes?: INTEGER) -> (BYTES, ERROR) |
all | Zlib-decompresses s into a BYTES buffer; returns (bytes, err). Refuses to produce more than 1000x its input, capped at 1 GiB, unless max_bytes says otherwise. |
Math (5)#
Numeric constants and random-number helpers (cryptographically-random bytes via rand_bytes). Arithmetic helpers like abs/min/max/sum live in the standard library.
| Builtin | Platforms | Description |
|---|---|---|
math_e() -> FLOAT |
all | Returns the constant e. |
math_pi() -> FLOAT |
all | Returns the constant pi. |
rand() -> FLOAT |
all | Returns a random FLOAT in [0, 1). |
rand_bytes(n: INTEGER) -> STRING |
all | Returns n cryptographically-random bytes (as a byte string). |
rand_int(lo: INTEGER, hi: INTEGER) -> INTEGER |
all | Returns a random INTEGER in [lo, hi). |
Hashing (11)#
Cryptographic and checksum digests (MD5/SHA-1/SHA-256/SHA-512/CRC-32/BLAKE2), HMAC, and identifier generators (UUID v4/v7, nanoid, random hex).
| Builtin | Platforms | Description |
|---|---|---|
hash_blake2(s: STRING|BYTES) -> STRING |
all | Returns the lowercase hex BLAKE2b-256 digest of s. |
hash_crc32(s: STRING|BYTES) -> STRING |
all | Returns the CRC-32 (IEEE) checksum of s as 8 hex chars. |
hash_md5(s: STRING|BYTES) -> STRING |
all | Returns the lowercase hex MD5 digest of s. |
hash_sha1(s: STRING|BYTES) -> STRING |
all | Returns the lowercase hex SHA-1 digest of s. |
hash_sha256(s: STRING|BYTES) -> STRING |
all | Returns the lowercase hex SHA-256 digest of s. |
hash_sha512(s: STRING|BYTES) -> STRING |
all | Returns the lowercase hex SHA-512 digest of s. |
hmac(key: STRING|BYTES, message: STRING|BYTES, algo: STRING) -> STRING |
all | Returns the hex HMAC of message under key. algo is md5/sha1/sha256/sha512. |
nanoid(n: INTEGER) -> STRING |
all | Returns a URL-safe random identifier of length n. |
random_hex(n: INTEGER) -> STRING |
all | Returns n cryptographically-random bytes as a 2n-char hex string. |
uuid_v4() -> STRING |
all | Returns a random (v4) UUID string. |
uuid_v7() -> STRING |
all | Returns a time-ordered (v7) UUID string. |
Time (7)#
Unix timestamps, formatting, parsing, and arithmetic (UTC).
| Builtin | Platforms | Description |
|---|---|---|
time_add(unix: INTEGER, seconds: INTEGER) -> INTEGER |
all | Returns the Unix timestamp shifted by seconds. |
time_diff(a: INTEGER, b: INTEGER) -> INTEGER |
all | Returns a - b in seconds (both Unix timestamps). |
time_format(unix: INTEGER, layout: STRING) -> STRING |
all | Formats a Unix timestamp (UTC) using a Go reference layout. |
time_ms() -> (INTEGER, ERROR) |
all | Returns the current Unix time in milliseconds. |
time_now() -> HASH |
all | Returns the current UTC time as a hash {unix, iso, year, month, day, hour, minute, second}. |
time_parse(value: STRING, layout: STRING) -> (INTEGER, ERROR) |
all | Parses value with a Go reference layout, against UTC rather than the host's zone; a zone abbreviation other than UTC or GMT is refused rather than read as +0000; returns (unixSeconds, err). |
time_unix() -> INTEGER |
all | Returns the current Unix time in seconds. |
Bytes (31)#
Binary buffer inspection and construction: fixed-width integer reads/writes (LE/BE), a streaming cursor, slicing, and byte/char conversions.
| Builtin | Platforms | Description |
|---|---|---|
bytes_char_from_int(value: INTEGER) -> (STRING, ERROR) |
all | Converts an integer byte value to a single-character string. |
bytes_cstr_at(data: STRING|BYTES, offset: INTEGER, maxLength: INTEGER) -> (STRING, ERROR) |
all | Reads null-terminated string from bytes at offset. |
bytes_cursor_eof(cursor: HASH) -> (BOOLEAN, ERROR) |
all | Returns whether cursor is at end-of-buffer. |
bytes_cursor_new(data: STRING|BYTES) -> (HASH, ERROR) |
all | Creates a cursor for structured byte parsing. |
bytes_cursor_read_u16_be(cursor: HASH) -> (HASH, ERROR) |
all | Reads unsigned 16-bit big-endian integer from cursor. |
bytes_cursor_read_u16_le(cursor: HASH) -> (HASH, ERROR) |
all | Reads unsigned 16-bit little-endian integer from cursor. |
bytes_cursor_read_u32_be(cursor: HASH) -> (HASH, ERROR) |
all | Reads unsigned 32-bit big-endian integer from cursor. |
bytes_cursor_read_u32_le(cursor: HASH) -> (HASH, ERROR) |
all | Reads unsigned 32-bit little-endian integer from cursor. |
bytes_cursor_read_u64_be(cursor: HASH) -> (HASH, ERROR) |
all | Reads unsigned 64-bit big-endian integer from cursor. |
bytes_cursor_read_u64_le(cursor: HASH) -> (HASH, ERROR) |
all | Reads unsigned 64-bit little-endian integer from cursor. |
bytes_cursor_read_u8(cursor: HASH) -> (HASH, ERROR) |
all | Reads one unsigned byte from cursor. |
bytes_cursor_seek(cursor: HASH, offset: INTEGER) -> (HASH, ERROR) |
all | Moves cursor to an absolute offset. |
bytes_cursor_tell(cursor: HASH) -> (INTEGER, ERROR) |
all | Returns current cursor position. |
bytes_get(data: STRING|BYTES, index: INTEGER) -> (INTEGER, ERROR) |
all | Reads one byte at index as integer. |
bytes_hex(value: INTEGER, width: INTEGER) -> (STRING, ERROR) |
all | Formats an integer as a zero-padded uppercase hex string with a 0x prefix (e.g. bytes_hex(4660, 8) -> "0x00001234"). This formats a number; it does not hex-encode a byte string. |
bytes_int_from_char(char: STRING) -> (INTEGER, ERROR) |
all | Converts a single-character string to its integer byte value. |
bytes_len(data: STRING|BYTES) -> (INTEGER, ERROR) |
all | Returns length of a bytes value. |
bytes_read_u16_be(data: STRING|BYTES, offset: INTEGER) -> (INTEGER, ERROR) |
all | Reads unsigned 16-bit big-endian integer from bytes at offset. |
bytes_read_u16_le(data: STRING|BYTES, offset: INTEGER) -> (INTEGER, ERROR) |
all | Reads unsigned 16-bit little-endian integer from bytes at offset. |
bytes_read_u32_be(data: STRING|BYTES, offset: INTEGER) -> (INTEGER, ERROR) |
all | Reads unsigned 32-bit big-endian integer from bytes at offset. |
bytes_read_u32_le(data: STRING|BYTES, offset: INTEGER) -> (INTEGER, ERROR) |
all | Reads unsigned 32-bit little-endian integer from bytes at offset. |
bytes_read_u64_be(data: STRING|BYTES, offset: INTEGER) -> (INTEGER, ERROR) |
all | Reads unsigned 64-bit big-endian integer from bytes at offset. |
bytes_read_u64_le(data: STRING|BYTES, offset: INTEGER) -> (INTEGER, ERROR) |
all | Reads unsigned 64-bit little-endian integer from bytes at offset. |
bytes_slice(data: STRING|BYTES, start: INTEGER, length: INTEGER) -> (STRING|BYTES, ERROR) |
all | Returns a byte sub-slice of the given length starting at start (i.e. data[start:start+length]). |
bytes_to_string(b: STRING|BYTES, encoding: STRING) -> (STRING, ERROR) |
all | Converts a buffer to text under a named encoding: "raw", "utf8" (validated), "latin1", "hex" or "base64". Returns (string, err). |
bytes_write_u16_be(data: STRING|BYTES, offset: INTEGER, value: INTEGER) -> (STRING|BYTES, ERROR) |
all | Writes unsigned 16-bit big-endian integer into bytes at offset. |
bytes_write_u16_le(data: STRING|BYTES, offset: INTEGER, value: INTEGER) -> (STRING|BYTES, ERROR) |
all | Writes unsigned 16-bit little-endian integer into bytes at offset. |
bytes_write_u32_be(data: STRING|BYTES, offset: INTEGER, value: INTEGER) -> (STRING|BYTES, ERROR) |
all | Writes unsigned 32-bit big-endian integer into bytes at offset. |
bytes_write_u32_le(data: STRING|BYTES, offset: INTEGER, value: INTEGER) -> (STRING|BYTES, ERROR) |
all | Writes unsigned 32-bit little-endian integer into bytes at offset. |
bytes_write_u64_be(data: STRING|BYTES, offset: INTEGER, value: INTEGER) -> (STRING|BYTES, ERROR) |
all | Writes unsigned 64-bit big-endian integer into bytes at offset. |
bytes_write_u64_le(data: STRING|BYTES, offset: INTEGER, value: INTEGER) -> (STRING|BYTES, ERROR) |
all | Writes unsigned 64-bit little-endian integer into bytes at offset. |
Archives (10)#
Read evidence containers in place: zip_* for the .zip a KAPE, CyLR or Velociraptor collection arrives as, tar_* for the .tar and .tar.gz a Linux triage script produces (bzip2 and zstd too, detected by magic rather than by extension). Nothing is extracted to disk -- an entry goes straight into a BYTES buffer for whatever parses it next -- so the classic extraction escape cannot be exploited through these builtins. It is still reported: every entry carries unsafe_path, because an archive containing such a name is a finding in its own right. Every decompression is bounded, here and in gunzip/zlib_decompress, at 1000x its input and 1 GiB, which a caller can override per call with max_bytes.
| Builtin | Platforms | Description |
|---|---|---|
tar_close(handle: STRING) -> (HASH, ERROR) |
all | Closes a tar handle and the archive file behind it. |
tar_entries(handle: STRING) -> ([]HASH, ERROR) |
all | Lists an archive's members from the walk tar_open already did: name, size, entry type, link target, POSIX mode/uid/gid/uname/gname, the three timestamps, and unsafe_path -- which covers a link target that escapes as well as a name that does, a symbolic link's target read from the link's own directory and a hard link's from the archive root. A member's position in this list is how tar_read names it when two members share a name. |
tar_open(path: STRING, max_bytes?: INTEGER) -> (HASH, ERROR) |
all | Opens a tar archive -- plain, or wrapped in gzip, bzip2 or zstd, detected by magic rather than by extension, after a first block that is a valid tar header -- and returns a handle. Tar has no central directory, so the whole archive is walked once here to learn what it contains; bodies are skipped, and what is held is the listing, at most 128 MiB of names, link targets and owners. A compressed walk stops at 1000x the archive's size unless max_bytes names another limit, which every later read through the handle keeps. compression names what was detected. Release the handle with tar_close. |
tar_read(handle: STRING, member: STRING|INTEGER, max_bytes?: INTEGER) -> (STRING, ERROR) |
all | Reads one member -- by name, or by its position in tar_entries -- and returns its contents as text. A name two members share is refused rather than resolved to either (a tar appended to holds every version it was given); read one by position. A hard link reads as the member it names; a symbolic link, a directory or a device holds no content and is refused, saying which. Tar has no index, so this walks from the start of the archive: cheap on a plain .tar, but on a compressed one it decompresses everything before the member, which makes reading many members quadratic. Limited to 1 GiB unless max_bytes says otherwise. Nothing is written to disk. |
tar_read_bytes(handle: STRING, member: STRING|INTEGER, max_bytes?: INTEGER) -> (BYTES, ERROR) |
all | Reads one member -- by name, or by its position in tar_entries -- into a BYTES buffer. Same walk, same rules and same limits as tar_read; this is the form to use, since an archive member is binary unless proven otherwise. |
zip_close(handle: STRING) -> (HASH, ERROR) |
all | Closes a zip handle and the archive file behind it. |
zip_entries(handle: STRING) -> ([]HASH, ERROR) |
all | Lists an archive's entries without decompressing any of them: name, sizes, compression method, CRC-32, mode, mtime, encryption flag, and unsafe_path -- which covers a symbolic link whose target escapes as well as a name that does. An entry's position in this list is how zip_read names it when two entries share a name. |
zip_open(path: STRING) -> (HASH, ERROR) |
all | Opens a zip archive (store/deflate/bzip2/zstd) and returns a handle for the other zip_ builtins. Reads the central directory, and the body of a symbolic-link entry to learn where it points, so opening a large collection is cheap. unsafe_path_count reports entries that would write outside a destination directory on extraction -- by their name, or for a link by where it points; a link whose target cannot be read counts too, since nothing vouches for it. An archive that contains one is itself a finding. Release the handle with zip_close. |
zip_read(handle: STRING, member: STRING|INTEGER, max_bytes?: INTEGER) -> (STRING, ERROR) |
all | Reads one entry -- by name, or by its position in zip_entries -- and returns its contents as text. A name two entries share is refused rather than resolved to either; read one by position. Refuses to produce more than 1000x the entry's compressed size, capped at 1 GiB, unless max_bytes says otherwise -- the declared uncompressed size is checked first and the limit is enforced again against what actually decompresses, because a decompression bomb lies about its size. Nothing is written to disk. |
zip_read_bytes(handle: STRING, member: STRING|INTEGER, max_bytes?: INTEGER) -> (BYTES, ERROR) |
all | Reads one entry -- by name, or by its position in zip_entries -- into a BYTES buffer. Same rules and same limits as zip_read; this is the form to use, since an archive member is binary unless proven otherwise. |
Filesystem (20)#
Read/write/manage files and directories, plus file-level forensics: hashing, entropy, string extraction, magic-type detection, carving, diffing, and NTFS deleted-file recovery.
| Builtin | Platforms | Description |
|---|---|---|
fs_append(path: STRING, data: STRING|BYTES) -> (BOOLEAN, ERROR) |
all | Appends data to the end of a file. |
fs_carve(path: STRING, type: STRING) -> ([]HASH, ERROR) |
all | Scans a file for a known artifact signature and returns the byte offsets where it starts. It reports offsets only; it does not extract (carve out) the artifact bytes or determine their length. |
fs_copy(src: STRING, dst: STRING) -> (BOOLEAN, ERROR) |
all | Copies a file from source path to destination path. |
fs_delete(path: STRING) -> (BOOLEAN, ERROR) |
all | Deletes a file from disk. |
fs_deleted(path: STRING) -> (HASH, ERROR) |
all | Enumerates deleted files from an NTFS $MFT (a standalone $MFT file or a full volume image, auto-detected). A record is deleted when its in-use flag is clear but its metadata still parses. Small files with a resident $DATA attribute are fully recovered, as hex (resident_data) and as a buffer (resident_data_bytes) carrying the same bytes; larger non-resident files report metadata only. size is the default $DATA stream's, and $FILE_NAME's lazily updated copy only where the record does not hold the start of that stream. SI/FN times include unix seconds, a sub-second nanosecond fraction (si_*ns/fnns), and an RFC3339Nano iso string. Returns {source_type, deleted_count, skipped, entries:[{record, name, path, size, is_directory, has_data, resident, recoverable, resident_data, resident_data_bytes, si, si_*ns, fn, fn__ns}]}. Returns (result, err). |
fs_diff(leftPath: STRING, rightPath: STRING) -> (HASH, ERROR) |
all | Compares two files (not directories) and reports differences. |
fs_entropy(path: STRING) -> (HASH, ERROR) |
all | Computes file entropy for packed/encrypted artifact detection. |
fs_exists(path: STRING) -> (BOOLEAN, ERROR) |
all | Returns whether a file or directory exists. |
fs_extract_strings(path: STRING, minLen?: INTEGER) -> ([]STRING, ERROR) |
all | Extracts printable strings from a file. |
fs_hash(path: STRING, algo?: STRING) -> (HASH, ERROR) |
all | Computes hash digests for a file. |
fs_list(path: STRING) -> ([]HASH, ERROR) |
all | Lists directory entries for a path. |
fs_magic(path: STRING) -> (HASH, ERROR) |
all | Infers file type/magic from a file's header against a ~40-signature database (executables PE/ELF/Mach-O, images, archives, documents, SQLite/registry/EVTX/pcap, media, and forensic artifacts like lnk/prefetch). Returns {path, type, mime, signature}. |
fs_metadata(path: STRING) -> (HASH, ERROR) |
all | Returns detailed filesystem metadata for a path. |
fs_mkdir(path: STRING) -> (BOOLEAN, ERROR) |
all | Creates a directory path. |
fs_move(src: STRING, dst: STRING) -> (BOOLEAN, ERROR) |
all | Moves or renames a file or directory. |
fs_read(path: STRING) -> (STRING, ERROR) |
all | Reads file contents from disk. |
fs_read_bytes(path: STRING) -> (BYTES, ERROR) |
all | Reads file contents from disk as a BYTES buffer. Use this rather than fs_read whenever the file is not known to be text. |
fs_stat(path: STRING) -> (HASH, ERROR) |
all | Returns file metadata such as size and timestamps. |
fs_walk(root: STRING, maxDepth?: INTEGER) -> ([]HASH, ERROR) |
all | Walks a directory tree and returns discovered paths. |
fs_write(path: STRING, data: STRING|BYTES) -> (BOOLEAN, ERROR) |
all | Writes data to a file, replacing existing contents. |
Network (33)#
Sockets and TLS sessions, an in-process X.509 CA, HTTP-message inspection, listeners/serve loops, WebSocket framing, scanning, and offline pcap analysis. See SECURE_NETWORKING.md.
| Builtin | Platforms | Description |
|---|---|---|
net_accept(listener: INTEGER, timeoutMs: INTEGER) -> (HASH, ERROR) |
all | Accepts one connection; returns {ok, handle, remote_addr, timeout, error}. |
net_banner(address: STRING, timeoutMs: INTEGER) -> (HASH, ERROR) |
all | Collects service banner text from a network endpoint. |
net_capture_raw(pcap_path: STRING) -> (HASH, ERROR) |
all | Reads raw packets from an offline pcap file into a per-packet listing (live interface capture needs cgo/raw sockets and is unavailable; net_pcap_analyze gives the flow summary, this gives the packets). Returns {file, link_type, count, truncated, packets:[{index, ts, timestamp, length, src, dst, protocol, sport, dport}]}. Returns (result, err). |
net_conn_close(handle: INTEGER) -> (BOOLEAN, ERROR) |
all | Closes a connection and releases its handle. |
net_conn_info(handle: INTEGER) -> (HASH, ERROR) |
all | Returns addressing and negotiated TLS session details for a connection. |
net_conn_read(handle: INTEGER, maxBytes: INTEGER, timeoutMs: INTEGER) -> (HASH, ERROR) |
all | Reads up to maxBytes from a connection; returns {data, bytes, eof, error} with I/O failures in the error field. |
net_conn_read_bytes(handle: INTEGER, maxBytes: INTEGER, timeoutMs: INTEGER) -> (HASH, ERROR) |
all | Reads up to maxBytes from a connection with data as a BYTES buffer; otherwise identical to net_conn_read. |
net_conn_write(handle: INTEGER, data: STRING, timeout_ms?: INTEGER) -> (INTEGER, ERROR) |
all | Writes bytes to a connection and returns the number written. A write deadline (default 30s, or timeout_ms; <=0 blocks forever) prevents a stalled peer from hanging the write. |
net_connect(address: STRING, timeoutMs: INTEGER) -> (INTEGER, ERROR) |
all | Opens a persistent TCP connection and returns a connection handle. |
net_connect_scan(host: STRING, startPort: INTEGER, endPort: INTEGER, timeoutMs: INTEGER) -> (HASH, ERROR) |
all | Scans a TCP port range on a host using full connect() probes (net.Dial). Pure-Go and unprivileged; not a half-open SYN scan (which needs raw sockets/privileges). |
net_dial(address: STRING, timeoutMs: INTEGER) -> (HASH, ERROR) |
all | Connectivity probe: dials address, immediately closes, and returns {ok, latency_ms, error}. Does not return a usable connection (use net_connect for that). |
net_dns_query(name: STRING, qtype: STRING) -> (STRING|ARRAY, ERROR) |
all | Queries DNS records for a hostname. |
net_flow_reconstruct(packets: ARRAY) -> ([]HASH, ERROR) |
all | Reconstructs higher-level flows from packet records. |
net_listen(address: STRING) -> (INTEGER, ERROR) |
all | Opens a plain TCP listener and returns a listener handle. |
net_listen_close(handle: INTEGER) -> (BOOLEAN, ERROR) |
all | Closes a listener and releases its handle. |
net_os_fingerprint(pcap_path: STRING) -> (HASH, ERROR) |
all | Passively fingerprints OS families from TCP SYN/SYN-ACK packets in an offline pcap (p0f-style heuristic over TTL, DF, window, and TCP options). Identifies an OS family, not a definitive OS; runs offline with no privileges. Returns {file, link_type, syn_packets, truncated, hosts:[{ip, ip_version, packet_type, os_guess, confidence, observed_ttl, initial_ttl, hops, window, df, mss, window_scale, sack_permitted, timestamps, tcp_options, signature}]}. truncated is true when the capture held more records than the record cap, or a compressed capture expanded past the stream cap; then only the hosts seen in the records read are listed. Returns (result, err). |
net_pcap_analyze(path: STRING) -> (HASH, ERROR) |
all | Analyzes PCAP captures and returns flow/session signals. Returns {file, link_type, packet_count, truncated, bytes_total, ipv4_packets, ipv6_packets, tcp_packets, udp_packets, icmp_packets, other_packets, first_ts, last_ts, duration_ms, flows:[{src, dst, sport, dport, proto, packets, bytes}]}. truncated is true when the capture held more records than the record cap, or a compressed capture expanded past the stream cap; then every count and every flow row covers only the records read. Returns (result, err). |
net_resolve(host: STRING) -> ([]STRING, ERROR) |
all | Resolves a host name to network addresses. |
net_serve(listener: INTEGER, handler_path: STRING, arg?) -> (NULL, ERROR) |
all | Accept loop that dispatches each connection to a fresh VM running handler_path; handler reads its connection via serve_conn() and shared arg via serve_arg(). Concurrent. |
net_spawn(handler_path: STRING, arg?) -> (BOOLEAN, ERROR) |
all | Runs handler_path on a new goroutine with no connection (serve_conn()->0) and arg via serve_arg(). For auxiliary workers, e.g. a WebSocket reverse pump. |
net_syn_scan(host: STRING, startPort: INTEGER, endPort: INTEGER, timeoutMs: INTEGER) -> (HASH, ERROR) |
all | DEPRECATED alias of net_connect_scan, kept for the 2.x line so existing programs keep running; 3.0 is the first release that may remove it. This is a full TCP connect scan, not a half-open SYN scan; use net_connect_scan. |
net_tls_connect(address: STRING, timeoutMs: INTEGER, options?: HASH|STRUCT) -> (INTEGER, ERROR) |
all | Opens a TLS (secure) client connection and returns a connection handle. |
net_tls_fingerprint(address: STRING, timeoutMs: INTEGER) -> (HASH, ERROR) |
all | Collects TLS certificate and handshake fingerprint metadata. |
net_tls_listen(address: STRING, certPem: STRING, keyPem: STRING, options?: HASH|STRUCT) -> (INTEGER, ERROR) |
all | Opens a TLS-terminating listener from a PEM cert/key pair. |
net_tls_upgrade_client(handle: INTEGER, options?: HASH|STRUCT) -> (HASH, ERROR) |
all | Upgrades an open connection to client-side TLS (STARTTLS / upstream leg). |
net_tls_upgrade_server(handle: INTEGER, certPem: STRING, keyPem: STRING, options?: HASH|STRUCT) -> (HASH, ERROR) |
all | Upgrades an accepted connection to server-side TLS (completes a CONNECT intercept). |
net_udp_scan(host: STRING, startPort: INTEGER, endPort: INTEGER, timeoutMs: INTEGER) -> (HASH, ERROR) |
all | Scans a UDP port range on a host. |
tls_generate_ca(options?: HASH|STRUCT) -> (HASH, ERROR) |
all | Creates a self-signed CA certificate and key; returns {cert_pem, key_pem, serial}. |
tls_generate_cert(options?: HASH|STRUCT) -> (HASH, ERROR) |
all | Creates a self-signed leaf/server certificate and key. |
tls_sign_cert(caCertPem: STRING, caKeyPem: STRING, options?: HASH|STRUCT) -> (HASH, ERROR) |
all | Issues a leaf certificate signed by a CA (per-host interception cert). |
ws_accept_key(client_key: STRING) -> (STRING, ERROR) |
all | Computes the Sec-WebSocket-Accept value for an RFC 6455 101 handshake response. |
ws_read_frame(handle: INTEGER, timeoutMs: INTEGER) -> (HASH, ERROR) |
all | Reads one WebSocket frame (unmasked); returns {fin, rsv1, rsv2, rsv3, opcode, payload, masked, length, is_control}. The three RSV bits are reported, not interpreted: RSV1 is permessage-deflate (RFC 7692), which Chrome and Firefox negotiate by default, and a frame carrying it is relayed by handing this whole hash to ws_write_frame. |
ws_write_frame(handle: INTEGER, opcode: INTEGER, payload: STRING, mask: BOOLEAN, timeout_ms_or_flags?: INTEGER|HASH|STRUCT) -> (INTEGER, ERROR) |
all | Writes one WebSocket frame; mask=true for client->server, false for server->client. FIN is set and no RSV bit is unless the fifth argument says otherwise, so a frame read with ws_read_frame is relayed faithfully by passing that hash here. An opcode outside 0 to 15 is refused rather than masked into a different kind of frame, and a control frame (8 and up) with more than 125 bytes of payload or with fin false is refused, because RFC 6455 section 5.5 has a peer fail the connection on either. A write deadline (default 30s, or timeout_ms; <=0 blocks forever) prevents a stalled peer from hanging the write. |
Http (11)#
HTTP client requests and low-level request/response parsing and building for proxy/inspection workflows.
| Builtin | Platforms | Description |
|---|---|---|
http_build_request(request: HASH|STRUCT) -> (STRING, ERROR) |
all | Serialises a request hash into HTTP wire bytes. A Content-Length header is added when absent, and one that disagrees with the body is refused rather than written: drop it after changing a body and a correct one is computed. A header whose value is a list is written as one field line per element, in order. |
http_build_response(response: HASH|STRUCT, method?: STRING) -> (STRING, ERROR) |
all | Serialises a response hash into HTTP wire bytes. A Content-Length header is added when absent, and one that disagrees with the body is refused rather than written: drop it after changing a body and a correct one is computed. A message that carries no body is never given a Content-Length, keeps one the caller supplied, and is refused if a body comes with it: that is any 1xx, 204 or 304, whose length describes the cached representation, and also the response to a HEAD request, which only the method can say -- pass it and a HEAD exchange rebuilds, declared length and all. A header whose value is a list is written as one field line per element, in order, which is how a two-cookie Set-Cookie goes back out as two. |
http_conn_read_request(handle: INTEGER, timeoutMs: INTEGER) -> (HASH, ERROR) |
all | Reads exactly one HTTP request from a connection handle. |
http_conn_read_request_head(handle: INTEGER, timeoutMs: INTEGER) -> (HASH, ERROR) |
all | Reads a request's line+headers without the body (stream it via net_conn_read); adds content_length, chunked. |
http_conn_read_response(handle: INTEGER, timeoutMs: INTEGER, method?: STRING) -> (HASH, ERROR) |
all | Reads exactly one HTTP response from a connection handle. Pass the method of the request this answers -- which a relay read with http_conn_read_request one call earlier -- to read a response to HEAD, whose Content-Length describes the representation and is followed by no body: without it the read waits for that body until the deadline. |
http_conn_read_response_head(handle: INTEGER, timeoutMs: INTEGER, method?: STRING) -> (HASH, ERROR) |
all | Reads a response's status line+headers without the body (stream it via net_conn_read); adds content_length, chunked. Those two say what follows on the connection, so for a response to HEAD -- pass the method -- content_length is 0 and chunked is false however the headers read, and the declared length stays in headers, which is where the evidence belongs. |
http_get(url: STRING) -> (HASH, ERROR) |
all | Performs an HTTP GET request. The result names where the answer came from: final_url is the URL that produced it after any redirects, redirects is how many hops it took, and final_method is the method that URL was asked with, which is a GET when a 301, 302 or 303 redirected a POST. A redirect out of https into plain http is refused. A body over 32 MiB is refused, and a body the peer cuts off mid-stream sets the response's error field and returns an error beside the bytes that did arrive. Set-Cookie arrives as a list, one element per line the server sent. |
http_parse_request(raw: STRING) -> (HASH, ERROR) |
all | Parses a raw HTTP request into {method, url, path, host, proto, query, headers, body}. A header sent on more than one line is combined into one comma-separated value, except Set-Cookie, which RFC 9110 does not permit to be folded and which arrives as a list of one element per line. A body over 32 MiB is refused rather than cut. |
http_parse_response(raw: STRING, method?: STRING) -> (HASH, ERROR) |
all | Parses a raw HTTP response into {status, status_text, proto, headers, body}. A header sent on more than one line is combined into one comma-separated value, except Set-Cookie, which RFC 9110 does not permit to be folded and which arrives as a list of one element per line. A body over 32 MiB is refused rather than cut. Pass the method of the request this answers to read a response to HEAD, whose Content-Length describes the representation and is followed by no body: without it the parse waits for that body and fails. |
http_post(url: STRING, body: STRING|HASH|STRUCT, contentType?: STRING) -> (HASH, ERROR) |
all | Performs an HTTP POST request. contentType defaults to application/octet-stream when omitted. The result names where the answer came from: final_url is the URL that produced it after any redirects, redirects is how many hops it took, and final_method is the method that URL was asked with, which is a GET when a 301, 302 or 303 redirected a POST. A redirect out of https into plain http is refused. A body over 32 MiB is refused, and a body the peer cuts off mid-stream sets the response's error field and returns an error beside the bytes that did arrive. Set-Cookie arrives as a list, one element per line the server sent. |
http_request(method: STRING, url: STRING, body: STRING|HASH|STRUCT, headers: HASH|STRUCT) -> (HASH, ERROR) |
all | Performs an HTTP request with a body and a headers hash. All four arguments are required; the timeout is a fixed 30s (not configurable). The result names where the answer came from: final_url is the URL that produced it after any redirects, redirects is how many hops it took, and final_method is the method that URL was asked with, which is a GET when a 301, 302 or 303 redirected a POST. A redirect out of https into plain http is refused. A body over 32 MiB is refused, and a body the peer cuts off mid-stream sets the response's error field and returns an error beside the bytes that did arrive. Set-Cookie arrives as a list, one element per line the server sent. A Host header is sent, where it used to be dropped for the URL's own host. A Content-Length that disagrees with the body, a Transfer-Encoding, and two header keys that differ only in case are refused rather than quietly replaced by what net/http would have sent. |
Graph Database (20)#
Graph-oriented data modeling: typed nodes/edges, named relations, indexed artifact attributes, BFS traversal, shortest-path, statistics, and timelines. See GRAPH_DATABASE.md.
| Builtin | Platforms | Description |
|---|---|---|
db_add_artifact(db: INTEGER, type: STRING, attrs?: HASH) -> (HASH, ERROR) |
all | Adds a forensic artifact node, in one transaction with everything indexed about it. type is a STRING, indexed as artifact_type; attrs is an optional hash whose entries are indexed as attr_ |
db_add_edge(db: INTEGER, from: INTEGER, to: INTEGER, edgeType?: INTEGER|ENUM_VALUE) -> (INTEGER, ERROR) |
all | Adds an edge between two node IDs. edgeType is an optional integer/enum edge type. Edge property hashes are not supported. |
db_add_node(db: INTEGER, nodeType?: INTEGER|ENUM_VALUE) -> (INTEGER, ERROR) |
all | Adds a DATA node and returns its ID. nodeType is an optional integer/enum node type (0–127; 0 is the DATA type used when omitted). Property hashes are not supported. |
db_add_relation(db: INTEGER, from: INTEGER, to: INTEGER, relation: STRING, attrs?: HASH) -> (HASH, ERROR) |
all | Adds an edge between two node IDs that carries a named relation. The edge and its relation land in one transaction, the relation indexed on the edge under relation, so db_relations and db_schema read it back. attrs is an optional hash stored beside it as attr_ |
db_bfs(db: INTEGER, origin: INTEGER, depth: INTEGER, direction: STRING) -> (HASH, ERROR) |
all | Breadth-first walk from origin up to depth hops. direction is "out", "in" or "both" -- case and surrounding space do not matter -- and any other word is an error. depth 0 is the origin alone, and a negative depth is an error. Every edge the walk crosses is returned, parallel edges included; graphene's own BFS keeps one edge per neighbour. The walk runs under the language's fixed traversal budget (docs/LIMITS_REFERENCE.md) and is refused rather than cut short when it reaches it. An origin that does not exist is an error. Returns (reach, err): nodes in the order reached, origin first, and edges in the order first crossed. |
db_close(db: INTEGER) -> (BOOLEAN, ERROR) |
all | Closes a graph database handle and flushes pending state. |
db_compact(db: INTEGER) -> (HASH, ERROR) |
all | Merges a disk-backed store's pending writes into its image and truncates the write-ahead log, which is what gives the memory back. Everything written since the last compaction stays resident and is replayed at every open, so a store that is never compacted grows in memory and open time with no error to signal it -- db_stats' delta_records and wal_bytes are the figures that say it is due. snapshot_root is the Merkle root of the image the compaction wrote and prev_root the root of the one it replaced -- the store's identity at this moment, which a script can record and a later check can be held to; both are empty for an in-memory handle. Compacting an in-memory handle is a no-op and reports has_storage false. Note that compacting with this build rewrites the store in a newer on-disk format that older mutant builds cannot open. Returns (report, err). |
db_find(db: INTEGER, key: STRING, value: STRING|BYTES, nodeType?: INTEGER|ENUM_VALUE) -> (HASH, ERROR) |
all | Returns the nodes whose indexed key holds value exactly, as {nodes, count, key_indexed}, ids in ascending order. nodeType restricts the answer to one type (0-127); with none, every type matches. key_indexed is false when the store's index has never held the key: graphene answers a query on such a key with nothing and no error, so an empty list alone cannot say whether nothing matched or nothing was ever indexed under that name. value is compared as bytes. Returns (found, err). |
db_index_prop(db: INTEGER, nodeID: INTEGER, key: STRING, value: STRING) -> (BOOLEAN, ERROR) |
all | Indexes a property (key=value) on a node, in a transaction that fails, and indexes nothing, when the node does not exist. Indexing a key again adds a value rather than replacing the one already there; db_node shows every value a key holds and db_find finds the node under each. All four arguments are required. |
db_node(db: INTEGER, id: INTEGER) -> (HASH, ERROR) |
all | Reads one node back: {id, labels, properties, property_count, multi_valued, blob_bytes}. properties holds everything indexed on the node -- by db_add_artifact, db_index_prop or another writer -- as STRING values, or BYTES for a value that is not valid UTF-8. A key indexed more than once holds an ARRAY of every value and is named in multi_valued, because indexing a key again adds a value rather than replacing one. labels are the type numbers given to db_add_node. blob_bytes is the size of any property blob the record itself carries, which db_* never writes and this does not decode. A node that does not exist is an error. Returns (node, err). |
db_open() -> (INTEGER, ERROR) |
all | Creates an in-memory graph database handle. |
db_open_disk(path: STRING, opts?: HASH) -> (INTEGER, ERROR) |
all | Opens or creates a disk-backed graph database. opts is an optional {memory_budget, discover_memory_budget, verify_on_open, read_only} hash: memory_budget caps what the store holds, in bytes, and is a whole-store figure rather than a whole-process one, so leave room for the program using it; discover_memory_budget:true derives that cap from the cgroup or Job Object limit the process is already under, and an explicit memory_budget always wins over it; verify_on_open:true checks the property indexes against the records before the handle is returned. read_only:true opens an existing store without changing it: it never creates one, runs beside other readers, is refused while a writer holds the store, and every write through the handle is an error; a store with no graphene.lock, or on write-protected media, is read without a lock, and db_stats' lock_free says why. An unknown option key is an error rather than ignored. Note that compacting a store with this build rewrites it in a newer on-disk format that older mutant builds cannot open; reading is unaffected. The store's label table is read before it is opened, because graphene registers those names for the whole process: a directory whose table names one of the custom labels 0-127 that db_* writes unnamed -- a symbol graph from mutant graph export names custom 0 Module -- is refused, and so is one that gives a label of the disclosure ledger's schema another name, or whose table is torn. A forensic ledger is refused too; it is opened with ledger_open. A store the program never closes is closed at its end and named on stderr. |
db_query(db: INTEGER) -> ([]INTEGER, ERROR) |
all | Returns all DATA-type node IDs (an alias for db_query_nodes with no type filter). There is no query-expression language. |
db_query_nodes(db: INTEGER, nodeType?: INTEGER|ENUM_VALUE) -> ([]INTEGER, ERROR) |
all | Returns node IDs, optionally filtered to a single node type (integer/enum). |
db_relations(db: INTEGER, node: INTEGER, direction: STRING) -> (HASH, ERROR) |
all | Lists one node's edges in a direction, in the order they were added: {node, direction, relations, count, unlabelled}, each relation {edge_id, src, dst, relation, labels, properties}. relation is the name db_add_relation stored, or null for an edge that carries none -- every edge db_add_edge wrote, and every db_add_relation edge written before 2.6.0, which kept its relation only in db_timeline's in-process journal and cannot be recovered; unlabelled counts them. A self-loop is listed once. direction is "out", "in" or "both"; any other word is an error, and so is a node that does not exist. Returns (relations, err). |
db_schema(db: INTEGER) -> (HASH, ERROR) |
all | Reports what a store holds: {nodes, edges, node_types, edge_types, node_keys, edge_keys, relations}. node_types and edge_types are [{type, count}] in type order; node_keys and edge_keys are the property keys the index has held, sorted -- on a disk store a key whose every entry has since been removed can still be listed; relations is [{relation, count}], how many edges carry each relation. Returns (schema, err). |
db_shortest_path(db: INTEGER, from: INTEGER, to: INTEGER) -> ([]INTEGER, ERROR) |
all | Finds a path with the fewest edges between two nodes and returns the node IDs along it, from first to last. Edges are walked in either direction, so the path may run against an edge; db_bfs with "out" answers what is reachable going forward. Two nodes that are not connected return an empty array, not an error; a node that does not exist is an error. Runs under the fixed traversal budget, and is refused rather than cut short when it reaches it. |
db_stats(db: INTEGER) -> (HASH, ERROR) |
all | Returns graph database statistics: {nodes, edges, has_storage, read_only, lock_free}. read_only is true for a store opened with db_open_disk's read_only option, and lock_free is the reason such a read could take no lock, empty when it took one. Disk-backed handles also report delta_records, csr_records, deleted_nodes, deleted_edges, wal_bytes, commit_seq and last_compact — growing delta_records/wal_bytes means the store is overdue for compaction. |
db_timeline(db: INTEGER) -> (ARRAY, ERROR) |
all | Returns chronological timeline events recorded in the graph. Takes only the handle (no options argument). |
db_verify(db: INTEGER) -> (HASH, ERROR) |
all | Cross-checks the store's indexes against its records: {checked, consistent, problem, values_checked}. checked is false for a store that cannot verify itself, and consistent is then false as well, never a pass. problem is the first inconsistency found, empty when there is none. values_checked is always false and is there to be read: the check covers the index's structure -- postings, adjacency, that no entry outlives its node or edge -- and not whether an indexed value still says what the script meant. Returns (report, err). |
Forensic Ledger (43)#
The same graph engine under a posture that cannot be turned off: every commit signed and attributed, a log containing an unsigned commit refused on replay, the image verified before it loads, the redaction ledger on, and every retired segment kept. The one option in the family is the role ledger_open records, because every other setting is a decision about what the resulting document may claim. Ledger handles are their own space -- no db_ builtin resolves one -- and a ledger directory refuses to open as an ordinary graph. ledger_under_view hands out a second handle through which the reads show only what a view shows, and every other builtin refuses it. A redaction an older build made could have reached the disclosure records without its record saying so; ledger_redactions_review records that an examiner has answered for such redactions, and no grant is issued from the ledger until one has.
| Builtin | Platforms | Description |
|---|---|---|
ledger_add_edge(ledger: INTEGER, from: INTEGER, to: INTEGER, props: HASH, edgeType?: INTEGER|ENUM_VALUE) -> (HASH, ERROR) |
all | Writes one edge between two node IDs into the ledger, attributed to the opening actor and signed as part of its commit. edgeType is an optional integer/enum edge type. Properties are written twice from one map, to the entity's own blob and to the property index, because they are different storage and only the blob is what a later property redaction removes: an entity written with index entries alone -- which is what db_add_artifact and db_index_prop write -- makes ledger_redact_metadata refuse with "carries no properties to redact" while the values stay queryable in the index. redactable reports whether this entity can ever be property-redacted, which is decided here at write time and not visible again until somebody tries. Values must be STRING or BYTES: the index matches exact bytes, so a rendering Mutant chose for a number would be searchable only by a caller who guessed the same one. The handle comes from ledger_open and is not a db_ handle; the two handle spaces are separate so that no db_ builtin can write into a ledger. Refused on a ledger opened as auditor, before anything else is looked at: an auditor writes nothing into what it audits. The role is asserted, so this keeps the record consistent and is not access control. Returns (info, err). |
ledger_add_node(ledger: INTEGER, props: HASH, nodeType?: INTEGER|ENUM_VALUE) -> (HASH, ERROR) |
all | Writes one node into the ledger, attributed to the opening actor and signed as part of its commit. nodeType is an optional integer/enum node type (0-127; 0 is the DATA type used when omitted). Properties are written twice from one map, to the entity's own blob and to the property index, because they are different storage and only the blob is what a later property redaction removes: an entity written with index entries alone -- which is what db_add_artifact and db_index_prop write -- makes ledger_redact_metadata refuse with "carries no properties to redact" while the values stay queryable in the index. redactable reports whether this entity can ever be property-redacted, which is decided here at write time and not visible again until somebody tries. Values must be STRING or BYTES: the index matches exact bytes, so a rendering Mutant chose for a number would be searchable only by a caller who guessed the same one. The handle comes from ledger_open and is not a db_ handle; the two handle spaces are separate so that no db_ builtin can write into a ledger. Refused on a ledger opened as auditor, before anything else is looked at: an auditor writes nothing into what it audits. The role is asserted, so this keeps the record consistent and is not access control. Returns (info, err). |
ledger_checkpoint(ledger: INTEGER) -> (HASH, ERROR) |
all | Binds every history's head into one digest and records it in the ledger's local checkpoint chain. The digest is the value to publish; everything else in the result is what it commits to -- the snapshot root, the attestation id, the segment, audit, redaction and grant heads, and a count of each, so a truncation shows up as more than a changed hash. Anchoring the snapshot root alone leaves the others free, and an adversary who rewrites only the audit log changes no snapshot root. witnessed always comes back false and is not computed: this call publishes nothing, and the field is there so a manifest assembled from this hash cannot claim otherwise by omission. Recording is a promise to publish -- a checkpoint whose digest never reaches an external witness is reported by ledger_verify_anchor as a fatal finding. The digest cannot be looked at before it is recorded, because a checkpoint is stamped with the time of capture before it is hashed and two captures of an unchanged ledger produce two different digests. delta_records is the count of records written since the last compaction: a checkpoint binds the snapshot, and an uncompacted write moves none of the six heads, so those records are covered by no checkpoint however many are published. Refused on a ledger opened as auditor, before anything else is looked at: an auditor writes nothing into what it audits. The role is asserted, so this keeps the record consistent and is not access control. Returns (checkpoint, err). |
ledger_checkpoint_history(ledger: INTEGER) -> (HASH, ERROR) |
all | Returns this ledger's local checkpoint chain, oldest first, with each entry's digest, the heads it bound and the counts beside them. chain_intact is a weak check and is not the interesting one: it recomputes each digest from its contents and each link from its predecessor, which catches a clumsy edit and nothing else, because anyone who rewrites the chain forward passes it. That is precisely why the digests are published and why ledger_verify_anchor exists. A ledger that has never checkpointed returns count 0 and chain_intact true, which is the honest reading of an empty chain rather than a fault. Returns (history, err). |
ledger_classifications(ledger: INTEGER, node?: INTEGER) -> (HASH, ERROR) |
all | Lists how the nodes a script wrote are classified: for each classified node, the classes in force, the views declared in this run that show it (shown_under), whether the node is still in the ledger (present), and every event of its classification chain. With a keyed case open, that case's classifications; without one, every case's, so an auditor reads them with the ledger alone. Given a node, that node's alone. Each chain is checked as it is read: a fork, an event whose uid does not recompute, one naming another node or case, or classes this program does not write refuse the call. Returns (classifications, err). |
ledger_classify(ledger: INTEGER, node: INTEGER, classes: STRING|ARRAY, reason: STRING) -> (HASH, ERROR) |
all | Records which of the open case's declared classes a node a script wrote holds, as the next event of that node's own hash-linked classification chain in the ledger. A read under a view -- ledger_under_view -- shows a script node only when its classification is within the classes the view grants, so a node nobody classified is shown by no view. classes is one label or an ARRAY of them; an empty ARRAY classifies the node as holding nothing, which no view shows. A repeated or undeclared label is refused, and so is a classification that changes nothing. Only a node a script wrote is classified: a record this program writes into the ledger is refused by name. The event names the node by id and no edge joins them, so the node can still be redacted like any other. shown_under names the views declared in this run that show it now. Needs the case key open, because the classes are tagged under it; it does not need the case attached, but a case attached to a ledger is classified there and nowhere else. Refused while the case is in review or disposed, and on a ledger opened as auditor. Not access control: the program reading under a view holds the ledger's own handle too. Returns (classification, err). |
ledger_close(ledger: INTEGER) -> (BOOLEAN, ERROR) |
all | Closes a ledger handle and flushes pending state. The handle comes from ledger_open and is not a db_ handle; the two handle spaces are separate so that no db_ builtin can write into a ledger. A handle from ledger_under_view is dropped, and the ledger it reads stays open. |
ledger_compact(ledger: INTEGER) -> (HASH, ERROR) |
all | Merges the ledger's delta layer into a new signed image and returns the snapshot roots it produced. This is the operation that makes a ledger provable: an entity written but never compacted is live and in no snapshot, so there is nothing for an inclusion proof to resolve against. Under this family's retention the compaction keeps every retired WAL segment, where the engine default would discard the log here along with every commit's actor, timestamp and signature -- retained_segments is how a manifest shows the retention took effect rather than merely being configured. The handle comes from ledger_open and is not a db_ handle; the two handle spaces are separate so that no db_ builtin can write into a ledger. Reads and rewrites the whole store, so it costs O(store size) rather than a seek. Refused on a ledger opened as auditor, before anything else is looked at: an auditor writes nothing into what it audits. The role is asserted, so this keeps the record consistent and is not access control. Returns (report, err). |
ledger_custody(ledger: INTEGER, node: INTEGER) -> (HASH, ERROR) |
all | Accounts for one entity across every history the ledger keeps -- the compacted snapshot and the entity's place in it, the attestation over that snapshot, the retired WAL segments, the audit log, the redaction ledger and the grant ledger -- and reports what could not be accounted for. Returns gaps, never a verdict: "not verified" is useless to somebody holding evidence, where "the segment chain is intact but nothing attested the current snapshot" says what to do next. Each gap carries its layer, whether the chain is broken or was never established, graphene's own detail verbatim, and a remedy in this language -- three of graphene's six details end by naming an option to set, two of which ledger_open already sets and whose real answer is ledger_compact. source says which of the two wrote each line. There is deliberately no complete field: graphene's Complete() is false whenever any gap exists, and this posture records no role grants by decision, so it would read false on every ledger Mutant can open. Unlike ledger_prove_node this does not refuse an id the store never held -- "nothing was found to account for" is an account, and comes back as live false. Every check here compares the store to itself, so the external gap is always present; ledger_custody_anchored and ledger_verify_anchor are what close it. One gap is this tool's own rather than graphene's: a redacted entity still sits, as written, in the retired write-ahead segments this ledger keeps so that attribution survives compaction, which a library cannot know and a directory being handed over must not hide. Returns (report, err). |
ledger_custody_anchored(ledger: INTEGER, node: INTEGER, snapshotRoot: STRING) -> (HASH, ERROR) |
all | ledger_custody with the snapshot additionally checked against a root the caller retained outside this machine. The only form that can tell "internally consistent" from "not tampered with", because it is the only one comparing the store against something an adversary holding the signing key could not change. snapshotRoot is a required 64-character hex argument, from ledger_root_export at the time it was retained; an empty root and an all-zero one are both refused here rather than passed through, because graphene reports a root mismatch as a broken chain and an examiner who supplied zeroes would be told the image had been tampered with. A separate builtin rather than a nullable third argument on ledger_custody, so that going without a retained root is visible in the source instead of being expressed by a value that happens to be empty. Returns (report, err). |
ledger_declare_composite(ledger: INTEGER, keys: ARRAY, target: STRING) -> (HASH, ERROR) |
all | Declares one key tuple, so a conjunction of equality filters over exactly those keys is answered from one posting list rather than by intersecting several. It serves that query and no other: a query naming fewer of the keys, or naming them with any comparison other than equality, is driven some other way, and ledger_explain_query is how to tell which. Unlike a unique declaration this checks nothing about the data and cannot fail on it, and unlike an ordered declaration it changes no comparison rule -- equality is byte equality either way. target is "node" or "edge". Returns (declaration, err). |
ledger_declare_ordered(ledger: INTEGER, key: STRING, target: STRING) -> (HASH, ERROR) |
all | Declares a property key ordered, so range filters and prefix on it are answered by binary search instead of a scan -- and reports what that does to the answers. order_differs is why this returns anything at all. Declaring a key changes how its range predicates compare, from numeric-when-both-sides-parse to byte order, and the two disagree: over 9, 10, 1x, 100, 2 a between of "2" and "100" goes from four records to none. So the key's existing values are read before the declaration is made and checked for a pair the two rules order differently, which is exact rather than sampled -- the rules can only disagree about two values that both parse as numbers, so sorting those numerically and looking for a byte-order inversion between neighbours finds such a pair if one exists. example carries it. Encode values so byte order matches intent (zero-padded fixed width, or a fixed-width integer encoding) and order_differs comes back false. target is "node" or "edge": the two index spaces are separate and a key declared on one is not declared on the other. Entries written before the declaration are absorbed, so the change reaches queries already written. Returns (declaration, err). |
ledger_declare_unique(ledger: INTEGER, key: STRING, target: STRING) -> (HASH, ERROR) |
all | Enforces that at most one live entity holds any given value under key, which is what turns an indexed value into a name. Unlike the ordered and composite declarations this is checked against what is already written, so it is a statement about the evidence rather than about the schema: a ledger where two nodes already share a value is refused, and graphene's message names the value and every id holding it. After it is declared, writing a duplicate is refused by ledger_add_node rather than by this call. target is "node" or "edge". Returns (declaration, err). |
ledger_declare_unique_edge(ledger: INTEGER, edgeType: INTEGER|ENUM_VALUE) -> (HASH, ERROR) |
all | Enforces that at most one live edge of the given type joins any ordered pair of entities -- the structural counterpart to ledger_declare_unique, which constrains a value rather than a relationship. Like the property form it validates the graph as it stands, so a ledger that already holds two such edges is refused. Enforcement afterwards happens at commit: the second edge is refused by ledger_add_edge, naming the edge that already exists. edgeType is the same 0..127 offset ledger_add_edge takes. Returns (declaration, err). |
ledger_edge(ledger: INTEGER, edge: INTEGER) -> (HASH, ERROR) |
all | Reads one edge record back, with its endpoints and its weight. weight is graphene's own field and means a similarity score for its SimilarTo type and zero for everything else; it is not a distance, which is why ledger_path takes the reading to make of it as an argument. Properties come back as BYTES for the reason ledger_node gives. An edge a node redaction took as collateral is refused by name, saying which redaction removed it and that it was not the subject of that decision. Takes a handle from ledger_under_view too, and then reads an edge only when the view shows both its ends and it is a script's edge, an IN_CASE or a CLASSIFIED_AS; any other is refused as one the ledger never held is. Returns (edge, err). |
ledger_explain_query(ledger: INTEGER, query: HASH) -> (HASH, ERROR) |
all | Reports how the planner resolved a query: which index drove it, how many candidates that produced, and how each remaining filter was applied. Takes the same query HASH as ledger_query_nodes. This is diagnostic output and graphene says so plainly -- which index the planner picks may change as its cost model improves, and the results a query returns may not -- so a script must not make an evidentiary decision from a plan. scanned is the reading worth acting on: the driver fell back to examining every entity, which happens when no filter can be served (contains can never be served by any index), when a range names a key that was not declared ordered, and whenever two or more filters are combined with mode any. candidates is what the driving step produced and not the size of the driving set, because a limit can be pushed into the driver, and residual cost is a forecast rather than a measurement; graphene documents both and neither is softened here. Returns (plan, err). |
ledger_indexes(ledger: INTEGER) -> (HASH, ERROR) |
all | Reports every index declaration in force, on both the node and the edge side: the ordered keys, the unique keys, the composite key tuples, and the edge types under a cardinality constraint. Worth reading before a query and not only after one, because ordered_node_keys is what decides whether a range filter compares numerically or byte-wise -- see ledger_declare_ordered for what that costs. Declarations are recorded in the store and survive both a compaction and a reopen, so a ledger handed over carries the schema decisions that shaped whatever was reported from it. Returns (indexes, err). |
ledger_node(ledger: INTEGER, node: INTEGER) -> (HASH, ERROR) |
all | Reads one node record back. Until this family had a read side a ledger Mutant opened was write-only: a script could commit attributed evidence and prove it was there, and could not ask what it said. Properties come back as BYTES rather than STRING because that is what the store holds -- ledger_add_node accepts either and commits bytes, so the two are the same blob afterwards and rendering one as text would invent a distinction the ledger does not keep and lose any value that is not valid UTF-8. bytes_to_string is the conversion for a caller who knows what they wrote, and it is that one rather than to_string, which renders BYTES as hex. labels are the offsets that were written, not graphene internal type numbers, so a label read here can be passed straight back to ledger_add_node; a graphene built-in type, which no Mutant builtin can write, comes back negative so it cannot be mistaken for one. An id the ledger does not hold is refused, and the refusal distinguishes an id that was never written from one a redaction removed by reading the redaction ledger first. Takes a handle from ledger_under_view too, and then reads only what the view shows: a node it does not show is refused as one the ledger never held is, and the refusal does not say which of the two it is. Returns (node, err). |
ledger_open(path: STRING, actor: STRING, options?: HASH) -> (HASH, ERROR) |
all | Opens or creates a forensic ledger: a graphene store in a fixed strict posture -- every commit signed, a log containing an unsigned commit refused on replay, the image verified before it loads, the redaction ledger on, and retention keeping every retired WAL segment. None of that is an option, because each is a decision about what the resulting document may claim and a script that can turn signing off is a script whose store cannot be relied on to have had it on; the one option is the role. actor is the examiner's asserted name: it is recorded, never authenticated, and actor_id is that name hashed into the 64-bit field graphene attributes commits with. role is one of administrator, case_owner, lead_investigator, investigator, reviewer or auditor, recorded on every redaction record and every disclose_* record written through this handle, and never checked; a script's own ledger_add_node or ledger_add_edge commit records the actor but not the role, because graphene keeps no role on a commit. Left out, it is the open case's role when actor is the case's examiner (role_source case), and otherwise unasserted; an actor asserting two different roles in one process is refused, as is a recipient-only role. A ledger opened as auditor refuses every write, and cannot create a ledger that does not exist yet. Signing uses the local Mutant key pair; key_created_for_this_run is true when this process had to generate it, which means every signature under it is by a key younger than the case. role_grants is always false and stays that way on purpose -- filling graphene's grant ledger would manufacture the look of an authorisation model behind a typed-in name -- so ledger_custody keeps reporting that gap. Writes a mutant.ledger marker into the directory, and db_open_disk refuses any directory carrying one. A ledger the program never closes with ledger_close is closed at its end and named on stderr, because a ledger left open when the process exits records its next open as a restart after an unclean shutdown. Returns (info, err). |
ledger_path(ledger: INTEGER, src: INTEGER, dst: INTEGER, costModel: STRING) -> (HASH, ERROR) |
all | Finds the cheapest path between two entities under a named cost model. The model is a name and not a function on purpose. graphene's EdgeCost contract is three obligations -- non-negative, deterministic, cheap -- and a function written in this language satisfies none of them by construction: graphene refuses a negative or NaN cost by edge id, it cannot check determinism, and a probe confirmed that a cost returning a different number each time it is asked about the same edge produces a path with no error that is neither the cheapest nor costed by the number reported beside it. The callback also runs once per incident edge on Dijkstra's inner loop, and builtin/resource.go already records that the VM and the evaluator drive a closure differently, so the same script would cost its evidence differently under the two engines. The three models are hops (every step 1), weight (the edge weight is the distance) and similarity (1 - weight, the reading graphene documents for its SimilarTo type). Two entities that are not connected come back as found false rather than as an error, because "these are not related" is a finding and folding it into the error channel would make a script's error branch mean that and "bad handle" at once. A src or dst the ledger does not hold is a refusal, and says whether a redaction removed it. The search walks an edge in either direction, so a path from a file back to the image that contains it is found like any other; reversed_hops lists the 0-based hops that ran against their edge, and directed is true only when none did. Takes a handle from ledger_under_view too: the search then crosses only nodes the view shows, found false means not connected through what it shows, and view says which view answered (empty on the ledger's own handle). Returns (path, err). |
ledger_patterns(ledger: INTEGER, pattern: HASH, scope: ARRAY|INTEGER, maxMatches: INTEGER) -> (HASH, ERROR) |
all | Finds every subgraph matching a shape. pattern is a HASH with nodes (an ARRAY of {"id": |
ledger_proof_describe(proof: STRING|BYTES) -> (HASH, ERROR) |
all | Decodes an exported proof and reports what it says about itself, checking none of it. Every root field is prefixed stated_ because this builtin is given no root and verifies nothing: those values are the file's own account of itself, and a proof file can state whatever roots its author chose. There is deliberately no verified field, which would be read as a verdict of false rather than as the absence of one. Decoding is not a weaker form of verifying -- a byte flipped inside a proof's roots region still decodes cleanly and then fails verification -- so a proof this describes is still a proof that may be false. ledger_verify_proof is the one that reaches a verdict, and it takes a root. Returns (claims, err). |
ledger_prove_edge_redaction(ledger: INTEGER, edge: INTEGER) -> (HASH, ERROR) |
all | Builds and exports a Merkle proof that the compacted image records an edge as deliberately removed. Separate from ledger_prove_redaction because an edge id and a node id are different namespaces: answering "was 7 redacted?" without knowing which 7 was meant is how a proof about one thing gets read as a proof about another. Proves an edge removed outright, an edge whose properties were stripped, and an edge cascaded out by a node redaction -- the last is why it exists, because an edge taken as collateral is as absent as one removed deliberately and deserves the same standing. Where an entity has more than one tombstone the most recent is proved, because that is the one describing its present state. Verify with ledger_verify_proof against a root obtained independently. Returns (proof, err). |
ledger_prove_node(ledger: INTEGER, node: INTEGER) -> (HASH, ERROR) |
all | Builds and exports a Merkle inclusion proof that one node was in the ledger's current compacted snapshot, and returns it as BYTES ready for fs_write. A proof is what lets somebody who does not have the ledger check a claim about one entity: it carries that entity's leaf, the sibling hashes and the component roots, and nothing about any other entity. It deliberately does not carry a root the verifier should trust -- snapshot_root comes back beside it so the examiner can retain or publish that value out of band, because ledger_verify_proof takes the root as an argument and a proof checked against the root inside itself proves nothing. Proofs are bound to a snapshot: one exported now verifies against the root named here forever, does not verify against the root a later compaction produces, and a fresh proof for the same unchanged entity is different bytes -- so exporting proofs without retaining the roots beside them produces files nobody can check. Refuses with three messages where graphene has one: a ledger never compacted, a node that is live but was written after the last compaction, and a node this ledger does not have. The last two are the same error underneath and have opposite fixes. There is no ledger_prove_edge -- graphene can prove an edge removed but not present. Takes a handle from ledger_under_view too, and then refuses a node the view does not show before anything is proved, as one the ledger never held is refused; a proof of a node it shows still carries the snapshot's size and the leaf's position, as every inclusion proof does. view says which view it was made under (empty on the ledger's own handle). Returns (proof, err). |
ledger_prove_property_redaction(ledger: INTEGER, node: INTEGER) -> (HASH, ERROR) |
all | Builds and exports a proof that an entity's properties were removed and that nothing else about it changed -- and does it without revealing what was removed. The claim is stronger than "this entity's version hash changed" and is checkable by somebody who never sees the content: the entity is in the image with these labels, its properties are now empty, it previously had properties hashing to P, it was otherwise byte-identical, and the removal is recorded under the snapshot root. The prior leaf is reconstructed from the surviving entity's identity plus the destroyed blob's 32-byte digest, so the content appears nowhere in the file. prior_leaf_bytes and surviving_leaf_bytes are equal on a well-formed proof, because the two leaves differ only in their final 32 bytes and that is the entire claim. Refuses an entity removed outright rather than stripped, including one whose properties were stripped and which was later removed -- this proof asserts the entity is still here, which is not true of it. Verify with ledger_verify_proof against a root obtained independently. Returns (proof, err). |
ledger_prove_redaction(ledger: INTEGER, node: INTEGER) -> (HASH, ERROR) |
all | Builds and exports a Merkle proof that the compacted image records a node as deliberately removed, and returns it as BYTES ready for fs_write. The audience is the one that matters most in an evidentiary exchange: somebody handed a single image and nothing else, to whom a redacted entity is simply absent and indistinguishable from one that never existed. A tombstone is the image's own record that the absence was deliberate, and it is Merkle-rooted into the snapshot root, so a root retained out of band commits to what was taken out of the image as well as what is in it. The proof carries no content, no reason and no actor -- only the scope, the entity, the version hash of what was destroyed, and the ledger sequence and record hash to ask for if the circumstances are needed. Verify it with ledger_verify_proof against a root obtained independently. Refuses with three messages where graphene has one: a ledger never compacted, an entity redacted but not yet compacted, and an entity this ledger never redacted. Returns (proof, err). |
ledger_provenance(ledger: INTEGER, node: INTEGER, maxDepth: INTEGER) -> (HASH, ERROR) |
all | Walks inbound edges from one entity back towards the evidence it came from, and says why the walk stopped. stopped_at is the field that matters and it is not graphene's: ProvenanceChain documents that a walk which does not reach a root returns the deepest path it found, so a chain cut short by its depth limit comes back the same shape as one that reached the source. This reads the last node's inbound edges afterwards and reports root, depth or cycle; complete is true for exactly one of the three. branch_points is the second thing graphene does not report: a node with two parents has two ancestries and the walk follows one, so every place the chain had a choice is named along with the parents it did not take -- "this artefact came from that image" must not be read off a result that had another answer. maxDepth must be positive; graphene substitutes 64 for a non-positive depth, which would be reported here as the limit the caller chose. Every inbound edge type is followed, deliberately: filtering by type adds a fourth reason a walk can stop early and the ids are opaque offsets. The walk runs under this language's fixed traversal budget and a walk that exceeds it is refused rather than truncated. Takes a handle from ledger_under_view too: the walk then reads only what the view shows, so root, stopped_at and complete are about what it shows, branch_points names only parents it shows, and view says which view answered (empty on the ledger's own handle). Returns (provenance, err). |
ledger_query_nodes(ledger: INTEGER, query: HASH) -> (HASH, ERROR) |
all | Answers a node query and says which comparison rule answered it. query is a HASH taking types, ids, filters, mode (all or any), order (asc or desc by id), offset and limit; a filter is {"key", "op", "value"} with op one of eq, prefix, contains, gt, gte, lt, lte, between, and between also needs value_upper. comparison is the field this builtin exists for. A key that has not been declared ordered compares range predicates numerically when both sides parse as numbers and byte-wise otherwise; a declared key compares byte-wise throughout. Those are different questions, and over the values 9, 10, 1x, 100, 2 a between of "2" and "100" returns four records undeclared and none at all declared. So comparison reports which rule ran -- none, numeric-then-bytes, bytes, or mixed when the query's range filters straddle both -- and range_keys names the keys it applies to. limited says a limit was reached, which means there may be more. unindexed_keys names every filter key the ledger's index has never held: a filter on one matches nothing whatever the records hold, which is what a misspelt key does and what the disclose_* family's own properties do, since that family indexes only the keys it looks its records up by. index_keys_known is false when the store could not list its keys, and unindexed_keys is then empty because nothing could be said. Takes a handle from ledger_under_view too: ids are then the matches the view shows, offset and limit apply after that, and nothing is said about the index -- index_keys_known is false and unindexed_keys empty, because whether a key is indexed anywhere says whether a withheld node holds it. view says which view answered (empty on the ledger's own handle). Returns (result, err). |
ledger_redact_edge(ledger: INTEGER, edge: INTEGER, reason: STRING) -> (HASH, ERROR) |
all | Destroys one relationship and leaves both endpoints, recording who did it and why. Only what a script wrote can be redacted: a record of the ledger's disclosure schema -- a Case, Disclosure, Withdrawal, ReclassEvent or any other node or edge disclose_* write and read -- is refused, because removing one would take a withdrawal or a supersession out of what those builtins consult; the schema is corrected by appending a record, never by removing one. A node redaction takes every incident edge with it, which is usually more than the decision called for; this removes the single relationship that was actually in scope. Edge ids and node ids are different namespaces that look alike, so this refuses a node id no differently from any other number it cannot find -- an edge redaction performed against a node id would remove a relationship nobody named. The reason is required, is written to the redaction ledger and again to the audit log, and neither is redactable -- so a reason repeating a value this call is about to destroy is refused rather than recorded. Whitespace is not a reason either, which graphene accepts and this does not. provable comes back false until the next compaction binds a tombstone for the edge into the snapshot root. retained_segments counts the retired write-ahead segments this ledger keeps: a segment is an immutable record of the commits it carries, so it still holds this entity as originally written, and a redaction rewrites the live graph and the next compacted image but never a segment. Redacting before the first compaction does not avoid it, because the compaction that makes a redaction permanent is the same operation that rotates the live log into a segment. ledger_open keeps every segment so that each commit's actor, timestamp and signature survive compaction, and the same bytes carry both -- attribution and erasure want the same bytes, and this family reports the trade rather than resolving it. Refused on a ledger opened as auditor, before anything else is looked at: an auditor writes nothing into what it audits. The role is asserted, so this keeps the record consistent and is not access control. Returns (redaction, err). |
ledger_redact_edge_properties(ledger: INTEGER, edge: INTEGER, reason: STRING) -> (HASH, ERROR) |
all | Destroys a relationship's properties and keeps the relationship -- endpoints, labels and weight survive, so the graph's shape is untouched and only the data the order was about goes. Only what a script wrote can be redacted: a record of the ledger's disclosure schema -- a Case, Disclosure, Withdrawal, ReclassEvent or any other node or edge disclose_* write and read -- is refused, because removing one would take a withdrawal or a supersession out of what those builtins consult; the schema is corrected by appending a record, never by removing one. The edge counterpart of ledger_redact_node_properties, and the same all-or-nothing rule applies: the property blob is one value and the index purge is unconditional. The reason is required, is written to the redaction ledger and again to the audit log, and neither is redactable -- so a reason repeating a value this call is about to destroy is refused rather than recorded. Whitespace is not a reason either, which graphene accepts and this does not. The record keeps the edge's version hash before and after plus the separated hash of the destroyed blob. There is no proof builtin for this scope: graphene can prove an edge removed but never prove what an edge still in the image used to carry, so ledger_prove_edge_redaction proves the removal and the property claim rests on the ledger record. provable comes back false until the next compaction. retained_segments counts the retired write-ahead segments this ledger keeps: a segment is an immutable record of the commits it carries, so it still holds this entity as originally written, and a redaction rewrites the live graph and the next compacted image but never a segment. Redacting before the first compaction does not avoid it, because the compaction that makes a redaction permanent is the same operation that rotates the live log into a segment. ledger_open keeps every segment so that each commit's actor, timestamp and signature survive compaction, and the same bytes carry both -- attribution and erasure want the same bytes, and this family reports the trade rather than resolving it. Refused on a ledger opened as auditor, before anything else is looked at: an auditor writes nothing into what it audits. The role is asserted, so this keeps the record consistent and is not access control. Returns (redaction, err). |
ledger_redact_node(ledger: INTEGER, node: INTEGER, reason: STRING) -> (HASH, ERROR) |
all | Destroys an entity and every edge touching it, and records who did it, when, under which key and why. Only what a script wrote can be redacted: a record of the ledger's disclosure schema -- a Case, Disclosure, Withdrawal, ReclassEvent or any other node or edge disclose_* write and read -- is refused, as is a node whose cascade would take a schema edge with it, because removing one would take a withdrawal or a supersession out of what those builtins consult; the schema is corrected by appending a record, never by removing one. The difference between this and db_delete_node is the record: a deletion after a compaction is indistinguishable from evidence that was never ingested, which makes lawful redaction and evidence destruction the same operation. This keeps the fact, the actor, the reason, the shape of what went and the version hash identifying it, and destroys only the content. Cascades: every incident edge goes too, each with its own version hash in the record, so a removal taken as collateral can still be identified. Call ledger_redaction_impact first -- it reports the cascade before anything is destroyed, and the hashes it returns are the ones the record will carry. The reason is required, is written to the redaction ledger and again to the audit log, and neither is redactable -- so a reason repeating a value this call is about to destroy is refused rather than recorded. Whitespace is not a reason either, which graphene accepts and this does not. provable comes back false and is not computed: a redaction is bound into the image by the next compaction and this call is always before it. retained_segments counts the retired write-ahead segments this ledger keeps: a segment is an immutable record of the commits it carries, so it still holds this entity as originally written, and a redaction rewrites the live graph and the next compacted image but never a segment. Redacting before the first compaction does not avoid it, because the compaction that makes a redaction permanent is the same operation that rotates the live log into a segment. ledger_open keeps every segment so that each commit's actor, timestamp and signature survive compaction, and the same bytes carry both -- attribution and erasure want the same bytes, and this family reports the trade rather than resolving it. Refused on a ledger opened as auditor, before anything else is looked at: an auditor writes nothing into what it audits. The role is asserted, so this keeps the record consistent and is not access control. Returns (redaction, err). |
ledger_redact_node_properties(ledger: INTEGER, node: INTEGER, reason: STRING) -> (HASH, ERROR) |
all | Destroys an entity's properties and keeps the entity: its ID, its labels and every edge touching it survive, so the graph's shape is intact and the entity remains available as a subject of provenance. Only what a script wrote can be redacted: a record of the ledger's disclosure schema -- a Case, Disclosure, Withdrawal, ReclassEvent or any other node or edge disclose_* write and read -- is refused, because removing one would take a withdrawal or a supersession out of what those builtins consult; the schema is corrected by appending a record, never by removing one. This is the lawful-erasure case, and removing a whole node to erase one field destroys evidence that was never in scope. All of an entity's properties or none of them -- there is no per-key form, because the property blob is one value and the index purge is unconditional, so an entity with one sensitive field and four innocuous ones loses all five. Write the sensitive field on an entity of its own if it may have to go separately, which is a decision taken at ingest rather than here. The index entries go with the blob, unconditionally, or the destroyed values would stay queryable. The reason is required, is written to the redaction ledger and again to the audit log, and neither is redactable -- so a reason repeating a value this call is about to destroy is refused rather than recorded. Whitespace is not a reason either, which graphene accepts and this does not. The record keeps the version hash the entity had before and the hash it has after, plus a separated hash of the destroyed blob, which is what makes ledger_prove_property_redaction content-free. provable comes back false: until the next compaction the image still holds the entity as it was and records no removal. retained_segments counts the retired write-ahead segments this ledger keeps: a segment is an immutable record of the commits it carries, so it still holds this entity as originally written, and a redaction rewrites the live graph and the next compacted image but never a segment. Redacting before the first compaction does not avoid it, because the compaction that makes a redaction permanent is the same operation that rotates the live log into a segment. ledger_open keeps every segment so that each commit's actor, timestamp and signature survive compaction, and the same bytes carry both -- attribution and erasure want the same bytes, and this family reports the trade rather than resolving it. Refused on a ledger opened as auditor, before anything else is looked at: an auditor writes nothing into what it audits. The role is asserted, so this keeps the record consistent and is not access control. Returns (redaction, err). |
ledger_redaction_impact(ledger: INTEGER, node: INTEGER) -> (HASH, ERROR) |
all | Reports what redacting a node would remove, without removing anything. The point of a preview is to be available while refusing is still possible: a hub node takes an evidentiary subgraph with it, and cascade_count is the number an examiner should see before signing the order rather than after. cascaded_edges and cascaded_hashes are parallel lists in graphene's own order, and the hashes are computed here rather than at deletion time -- so a manifest can record the exact set this preview showed, and a recipient can check that it is the set the redaction record describes. There is no exceeds_policy field: this posture sets no cascade limit, because a limit Mutant invented would refuse a lawful order for a number nobody chose, and a field that is false on every ledger reads as a check that passed. The preview is the limit. protected is true when the redaction would be refused because the node, or an edge its cascade would take, belongs to the ledger's disclosure schema, and protected_reason says which. Refuses an id the ledger never held, and says so differently for one it held and redacted. Returns (impact, err). |
ledger_redactions(ledger: INTEGER) -> (HASH, ERROR) |
all | Returns every redaction this ledger records, oldest first, with the actor, the role, the time, the scope, the reason, the version hash of what was destroyed and the hash chain linking each record to the one before it. The ledger is its own append-only file that compaction never touches, which is why a redaction record outlives the entity it describes. chain_intact recomputes each record's hash from its contents and each link from its predecessor, checking signatures against the keyring this ledger was opened with: it catches an edited or removed record. It does not catch a ledger rewritten forward from the start by whoever holds the signing key -- that is what binding the head into a published checkpoint is for, and an unpublished chain is the store's word about itself. The chain check's explanation is chain_reason rather than reason, because reason is the field beside it and means why a redaction was performed. head is the last record's hash, which is the value to bind or publish. Each record carries the actor's id; actor is the name behind it where one is known -- this session's own, or the name a disclosure-family Actor node pairs with that id -- with actor_source saying which (session, ledger), and empty with actor_source unknown otherwise, never the reader's name. role is the role the redaction was made under, as asserted, and empty on a record made before roles were recorded. unguarded is true on a redaction recorded with no role by a build that may not have refused a redaction of the disclosure schema, unless it stripped the properties of a node or an edge a script wrote that the ledger still holds: it could have removed a withdrawal, a disclosure or a reclassification, and its record cannot say. reviewed_through is how far ledger_redactions_review has answered for them, and unguarded_redactions lists those after it, which stop every grant from this ledger until they are reviewed. A chain of reviews that cannot be read counts for nothing, and reviews_intact and reviews_reason say so; the redactions are listed anyway. Returns (redactions, err). |
ledger_redactions_review(ledger: INTEGER, through_seq: INTEGER, reason: STRING) -> (HASH, ERROR) |
all | Records that an examiner has read the ledger's redactions made before its disclosure schema was guarded, up to the redaction numbered through_seq, and answers for them. The redaction builtins never remove a record disclose_* write; a redaction recorded with no role was made by an earlier build that may have, and its record names what it removed by id and hash, never by label -- so it could have removed a withdrawal, a disclosure or a reclassification with nothing in the ledger to say so. Every such redaction is in question except one that stripped the properties of a node or an edge a script wrote that the ledger still holds. Until each is answered for, disclose_to_passphrase issues no grant from the ledger, and disclose_history, disclose_for_segment, disclose_reclassified and redaction_versions list them in unguarded_redactions; ledger_redactions marks each one. A review is the next event of the ledger's one hash-linked chain of reviews, bound to the hash of the redaction record it names, and it checks nothing: record again first what a reviewed redaction removed that can be -- a withdrawal with disclose_withdraw while its disclosure is still in the ledger, a reclassification with disclose_reclassified. A withdrawal whose disclosure was removed needs nothing: it names the record and the recipient itself, and still stops a new grant. Refused for a number the ledger records no redaction under, one a review already reaches, a range holding no redaction in question, a chain of reviews that cannot be read, and a ledger opened as auditor. Needs no case open: redaction records are the ledger's, not a case's. Returns (review, err). |
ledger_root_export(ledger: INTEGER) -> (HASH, ERROR) |
all | Returns the ledger's current snapshot roots: the value an examiner retains or publishes so a proof can later be checked against a root the proof's author did not supply. All six component roots come back together because the snapshot root binds them and ledger_verify_chain needs the set rather than the one number. Unlike ledger_stats, which reports "never compacted" as a state with empty root strings, this refuses on a ledger with no snapshot: its job is to produce a value that will be retained, and an empty one retained now is an empty one quoted later in a manifest as though it were a root. Returns (roots, err). |
ledger_stats(ledger: INTEGER) -> (HASH, ERROR) |
all | Reports what a ledger holds and what it commits to: node and edge counts, the delta/WAL figures that say a compaction is due, the retained segment count, the audit and redaction entry counts, and the snapshot roots. compacted is false on a ledger that has never been compacted, and the four root fields are empty strings rather than an error -- a store whose contents are still entirely in the log has no roots, which is a state rather than a failure, and nothing in it can be proved to anybody until it does. The handle comes from ledger_open and is not a db_ handle; the two handle spaces are separate so that no db_ builtin can write into a ledger. Returns (stats, err). |
ledger_subgraph(ledger: INTEGER, nodeIds: ARRAY) -> (HASH, ERROR) |
all | Returns the entities named and every relationship among them -- the edges whose two endpoints are both in the set. An id given twice is counted once and reported in duplicates; graphene returns the record once per occurrence, so a list with a repeat in it would otherwise produce a subgraph claiming more entities than were asked about. An id the ledger does not hold refuses the whole call rather than dropping it, because an induced subgraph is the claim that these are all the relationships among these entities, and a missing entity makes that false rather than incomplete -- the refusal names how many were missing and whether a redaction removed the first of them. An empty list is refused too: a subgraph over nothing is a question with no subject, not an empty answer. Takes a handle from ledger_under_view too: an id the view does not show is then refused as a missing one is, the edges are the ones the view shows, and view says which view answered (empty on the ledger's own handle). Returns (subgraph, err). |
ledger_under_view(ledger: INTEGER, view: STRING) -> (HASH, ERROR) |
all | Returns a second handle to an open ledger, through which the reads show it as a view declared in the open case shows it: the script nodes whose classification (ledger_classify) is within the classes the view grants, the case's Case and Record nodes, and the Classification nodes of the classes it grants -- and an edge only when it joins two nodes the view shows and is a script's edge, an IN_CASE or a CLASSIFIED_AS. No walk through it crosses a node the view does not show. It fails closed: ledger_node, ledger_edge, ledger_provenance, ledger_path, ledger_subgraph, ledger_patterns, ledger_query_nodes and ledger_prove_node read through it, ledger_close drops it and leaves the ledger open, and every other builtin refuses it by name. A node the view does not show, one the ledger never held and one a redaction removed get one answer, and no read under a view counts what it withholds. Needs the case key open. Not access control: the program holding it holds the ledger's own handle too, and access_control says so. Returns (handle, err). |
ledger_verify_anchor(ledger: INTEGER, witnessed: ARRAY) -> (HASH, ERROR) |
all | Checks this ledger's checkpoint chain against what an external witness says it holds -- the one check in this family that is not the store vouching for itself. Mutant ships no anchor transport and graphene ships none either: what makes an anchor an anchor is being beyond the reach of whoever can rewrite the store, and that property comes from where it lives, not from code. So the witness is an argument, exactly as the snapshot root is in ledger_verify_proof. witnessed is an ARRAY of hashes carrying digest (64 hex) and unix (INTEGER seconds, as time_unix returns), with an optional ref -- a timestamp-token serial, a transaction hash, a URL, a page number. The time is required because it is the only part of a witness record this store could not have written itself. The check runs both ways and both are fatal: a local checkpoint the witness never saw means the local chain was rewritten, and a witnessed digest with no local checkpoint means the local record was destroyed, which is otherwise indistinguishable from innocence and strictly easier to perform. An empty witness list is a legitimate argument, not an error -- it says nothing was ever published, and every local checkpoint becomes a finding. current_matches_last is about the six heads a checkpoint binds and not about the ledger's contents: an uncompacted write moves none of them, so a ledger with records in flight is confirmed by the witness while those records are covered by nothing, which is what delta_records and its gap report. Returns (audit, err). |
ledger_verify_chain(earlier: HASH, later: HASH) -> (HASH, ERROR) |
all | Checks that a later retained snapshot names an earlier one as its predecessor, given two hashes from ledger_root_export. This is what separates a store's history from a collection of internally consistent files: a substituted snapshot can be perfectly coherent on its own, but it cannot claim a predecessor it never had. Both arguments must carry all six component roots and body_version; a missing one is refused rather than read as zero, because a zero component would fail the snapshot root's own binding check and report the chain as broken when what was broken was the argument. A snapshot does not chain to itself. Two well-formed root sets that do not link come back as chained false with a reason, not as an error. Returns (result, err). |
ledger_verify_proof(proof: STRING|BYTES, snapshotRoot: STRING) -> (HASH, ERROR) |
all | Checks an exported proof against a snapshot root the caller supplies. Needs no ledger, no store and no directory: the bytes and the root are enough, which is the property proofs exist for. snapshotRoot is a required 64-character hex argument and there is no form of this call that reads the root out of the proof -- whoever wrote a proof file supplied both the evidence and the roots it claims, so checking one against the other is circular. An empty root is refused, and so is an all-zero one, which is what an empty value looks like once something has padded it and which graphene records as "this snapshot has no roots" rather than as a root. A file that is not a readable proof comes back as an error; a readable proof that is not true comes back as a value with verified false and a reason, because a corrupt download and a forgery call for different responses. checked_against and stated_snapshot_root are always both present so a reader can see which root the verdict was reached against. Verifies all three proof kinds: node inclusion, redaction and property redaction. Returns (result, err). |
ledger_verify_store(path: STRING) -> (HASH, ERROR) |
all | Recomputes a compacted image's Merkle roots from the records the file actually holds and compares them to the roots the file carries. A different question from the image's own digest: the digest asks whether these bytes are what was written, and this asks whether the roots describe the records in the same file. An adversary who edits a record and recomputes the digest defeats the first and not this -- and if they recompute the roots too, the snapshot root changes, which is what a root retained elsewhere detects. Takes a path rather than a handle, because the party who most needs it is the one holding a copy of a store and no reason to trust the process that wrote it; it runs on an open ledger as well. The path may be the ledger directory or the image file. A directory holding no image is refused as a mistake about which path was passed, not reported as a failed check. Returns (result, err). |
Cache (8)#
In-memory key/value cache with TTLs and hit/miss statistics.
| Builtin | Platforms | Description |
|---|---|---|
cache_clear(name: STRING) -> (INTEGER, ERROR) |
all | Clears all entries and resets relevant cache state. |
cache_close(name: STRING) -> (HASH, ERROR) |
all | Closes a named cache and frees its entries and backend. |
cache_delete(name: STRING, key: STRING) -> (BOOLEAN, ERROR) |
all | Deletes a key from cache and returns whether it existed. |
cache_get(name: STRING, key: STRING) -> (HASH, ERROR) |
all | Reads a value from cache and returns found/value fields. |
cache_keys(name: STRING) -> ([]STRING, ERROR) |
all | Lists sorted cache keys for a cache namespace. |
cache_open(name: STRING) -> (HASH, ERROR) |
all | Opens or creates a named in-memory cache store. |
cache_put(name: STRING, key: STRING, value, ttlSeconds?: INTEGER) -> (BOOLEAN, ERROR) |
all | Stores a value in a named cache key with optional TTL. |
cache_stats(name: STRING) -> (HASH, ERROR) |
all | Returns cache counters such as hits, misses, puts, deletes, and expires. |
Policy (5)#
Load and evaluate allow/deny policies with rule metadata and evaluation traces.
| Builtin | Platforms | Description |
|---|---|---|
policy_allow(policy: HASH|STRING, input: HASH) -> (BOOLEAN, ERROR) |
all | Evaluates and returns allow/deny boolean for a policy. |
policy_eval(policy: HASH|STRING, input: HASH) -> (HASH, ERROR) |
all | Evaluates a loaded policy and returns decision details. |
policy_load(name: STRING, source: HASH|STRING) -> (HASH, ERROR) |
all | Loads a policy module by name from source text or config hash. A policy or query that calls one of OPA's builtins that reach the network -- http.send, json.match_schema, json.verify_schema -- is refused, here and by every policy_* builtin: fetch with http_get and pass what it returns in the input. |
policy_rules(policy: HASH|STRING) -> (ANY, ERROR) |
all | Returns rule metadata exported by a loaded policy. |
policy_trace(policy: HASH|STRING, input: HASH) -> (ANY, ERROR) |
all | Runs policy evaluation with trace output for debugging rule flow. |
Runtime Integration (3)#
Embed and securely execute Lua in a restricted sandbox (no io, dangerous os.* stripped, bounded execution). See RUNTIME_INTEGRATION.md.
| Builtin | Platforms | Description |
|---|---|---|
lua_run_file(path: STRING) -> (HASH, ERROR) |
all | Runs a Lua script from a file. |
lua_run_http(url: STRING) -> (HASH, ERROR) |
all | Fetches and runs a Lua script from an HTTP endpoint in a restricted sandbox (no io, no os.execute/exit/remove; only safe base/math/string/table/os-time libraries). |
lua_run_string(code: STRING) -> (HASH, ERROR) |
all | Runs a Lua script from a string. |
Command Execution (5)#
Run an external command and read back what it printed. What guards one is fixed and not configurable: a 3-second timeout, 8 KiB kept of each output stream, a kill that reaches the whole process tree, and a shell named rather than written as a command line. There is no per-builtin capability gate and none is planned -- SEC-017 in SECURITY_LLD_TRACEABILITY.md records that decision -- so a program that can reach these builtins can run commands, and the lint rule commandInjection is an editor-time warning and not an enforcement.
| Builtin | Platforms | Description |
|---|---|---|
cmd_add(builder: HASH, arg: STRING) -> (HASH, ERROR) |
all | Appends a line to a command builder and returns the builder with it. The builder is a value, not a handle: the line is on the hash this returns and not on the one passed in. |
cmd_builder(shell?: STRING) -> (HASH, ERROR) |
all | Creates a command builder object for step-wise command composition. The shell is recorded as given and decides, at cmd_run, how the lines are joined into one command. |
cmd_run(builder: HASH) -> (HASH, ERROR) |
all | Runs every line a builder collected, in order, and returns the command's output streams, exit code and timeout status. The lines are joined with what the named shell reads as the end of one line: a newline for powershell, pwsh, sh, bash and any other name, and "&" for cmd and batch, whose command line ends at the first newline. The exit code is the last line's in every shell, so a line that fails does not stop the rest and is not reported on its own; a blank line is a no-op, and a builder holding nothing but blank lines is refused rather than run. |
exec_argv(argv: ARRAY) -> (HASH, ERROR) |
all | Runs a program with exactly the arguments given, with no shell in between. |
exec_string(command: STRING, shell?: STRING) -> (HASH, ERROR) |
all | Executes a shell command string via security-guarded command execution. |
Cryptography (6)#
X.509 certificate parsing, JWT decoding, PEM decoding, and authenticated AES-GCM encryption/decryption.
| Builtin | Platforms | Description |
|---|---|---|
aes_decrypt(key: STRING, ciphertext: STRING) -> (STRING, ERROR) |
all | AES-GCM decrypts ciphertext produced by aes_encrypt (nonce-prefixed). Returns (plaintext, err); errors on wrong key or tampering. |
aes_decrypt_bytes(key: STRING, ciphertext: STRING|BYTES) -> (BYTES, ERROR) |
all | AES-GCM decrypts into a BYTES buffer. Recovered plaintext is binary until something proves otherwise, and a buffer is also the form the VM can wipe. Returns (plaintext, err). |
aes_encrypt(key: STRING, plaintext: STRING|BYTES) -> (STRING, ERROR) |
all | AES-GCM encrypts plaintext. key must be 16/24/32 bytes. A random nonce is prepended to the output. Returns (ciphertext, err). |
jwt_decode(token: STRING) -> (HASH, ERROR) |
all | Decodes a JWT's header and claims WITHOUT verifying the signature (verified is always false). Returns {header, claims, algorithm, signature_present, verified}. Returns (result, err). |
pem_decode(s: STRING) -> (HASH, ERROR) |
all | Decodes the first PEM block. Returns {type, headers, der_hex, size, remaining_bytes}. Returns (result, err). |
x509_parse(pem_or_der: STRING) -> (HASH, ERROR) |
all | Parses an X.509 certificate (PEM or DER). Returns {subject, issuer, serial, not_before, not_after, is_ca, version, dns_names, ip_addresses, email_addresses, key_algorithm, signature_algorithm, sha1, sha256}. Returns (cert, err). |
Fingerprinting (4)#
Malware/host fingerprints: PE import hash (imphash), JA3 TLS-client fingerprint, and NTLM/LM password hashes (for authorized credential testing).
| Builtin | Platforms | Description |
|---|---|---|
imphash(pe_path: STRING) -> (HASH, ERROR) |
all | Computes the PE import hash (pefile/Mandiant algorithm) for malware clustering. Returns {imphash, import_count, dll_count}. Note: ordinal-only imports are rendered as ord |
ja3(client_hello: STRING) -> (HASH, ERROR) |
all | Computes the JA3 TLS-client fingerprint from a ClientHello (raw bytes, with or without the TLS record layer). Hashes version,ciphers,extensions,curves,point_formats with GREASE (RFC 8701) removed. Returns {ja3, ja3_hash (md5), tls_version, ciphers[], extensions[], curves[], point_formats[]}. Returns (result, err). |
lm_hash(password: STRING) -> STRING |
all | Returns the legacy LM hash (DES-based; case-insensitive, max 14 chars) as hex. Empty password -> aad3b435b51404eeaad3b435b51404ee. |
nt_hash(password: STRING) -> STRING |
all | Returns the NTLM NT hash (MD4 of the UTF-16LE password) as hex. For authorized credential testing/CTF use. |
Network Intelligence (11)#
IOC handling: defang/refang, IP/CIDR math, domain/eTLD+1 extraction, validation, and IOC extraction from free text.
| Builtin | Platforms | Description |
|---|---|---|
cidr_hosts(cidr: STRING) -> []STRING |
all | Returns all addresses in a CIDR range (capped; errors if >20 host bits). |
defang(ioc: STRING) -> STRING |
all | Defangs an indicator for safe display (http->hxxp, .->[.], @->[at]). |
domain_extract(url: STRING) -> STRING |
all | Extracts the lowercased hostname from a URL or host string. |
extract_iocs(text: STRING) -> HASH |
all | Extracts IOCs from text: {ipv4, urls, domains, emails, md5, sha1, sha256}, each unique and sorted. Defang markers such as [.], (dot), [at] and hxxp are undone before matching; the bare words at and dot are not, because ordinary prose uses them. |
ip_in_cidr(ip: STRING, cidr: STRING) -> BOOLEAN |
all | Returns whether an IP falls within a CIDR range. |
ip_is_private(ip: STRING) -> BOOLEAN |
all | Returns whether an IP is private/loopback/link-local (RFC1918 etc.). |
ip_to_int(ip: STRING) -> INTEGER |
all | Converts an IPv4 address to its 32-bit integer form. |
ip_version(ip: STRING) -> INTEGER |
all | Returns 4, 6, or 0 (invalid) for an IP address. |
is_valid_domain(s: STRING) -> BOOLEAN |
all | Returns whether s is a syntactically valid domain name. |
refang(ioc: STRING) -> STRING |
all | Reverses common defang encodings ([.]/(.)/[dot]->., hxxp->http, [at]->@). |
tld_extract(domain: STRING) -> HASH |
all | Returns {domain, etld1, suffix} using the public suffix list. A leading dot, as a cookie's host has, and a trailing dot are dropped. etld1 is empty for a name with no registrable domain: an IP address, a bare public suffix, localhost. |
Detection (9)#
Two ways to decide that something is worth looking at. The detect_* builtins are heuristic detectors for code injection, C2 beaconing, persistence, privilege escalation and suspicious files, driven by supplied evidence. The sigma_* builtins run real Sigma rules -- the portable YAML detection format -- over events_from() timelines: sigma_parse and sigma_parse_all compile a rule or a whole ruleset, sigma_match asks one rule about one event, and sigma_scan runs a ruleset over a timeline. A rule this engine cannot evaluate is a compile error rather than a silent non-match, and both sigma_match and sigma_scan report the fields a rule read that the evidence never carried -- a rule that could not have fired is not a host that came back clean. See DETECTION_RULES.md.
| Builtin | Platforms | Description |
|---|---|---|
detect_injection(facts: HASH) -> (HASH, ERROR) |
all | Scores probable code injection in a memory image from three kinds of evidence: two or more PE headers (+40), each confirmed the way mem_find_pe confirms one, by an MZ marker whose e_lfanew leads to "PE\0\0"; a NOP sled of 16 bytes or more (+25); and weighted shellcode signatures (GetPC via fnstenv or call-pop, x86 and x64 PEB walks, cld;call, xor/push). The two-byte cld;call counts by density, not presence: only when uniformly random bytes of the same length would reach its hit count in at most one buffer in a thousand. In 1 MiB random bytes average 16 and it takes 31; a single cld;call counts only in a buffer of at most 66 bytes. A cld;call count that chance explains is listed in chance_signatures with the count random bytes hold on average, and adds nothing. The other signatures (all four bytes or longer) and call-pop count on presence. detected is score >= 60, which always takes two or more signals. Returns detected, score, pe_headers (confirmed PE headers), shellcode_hits (the hits of the signatures that counted), matched_signatures, chance_signatures and reasons. |
detect_network_beacon(flows: ARRAY) -> (HASH, ERROR) |
all | Detects C2 beaconing by analyzing inter-arrival interval regularity (low coefficient of variation) and optional transfer-size consistency per destination; each flow may carry ts (epoch/RFC3339) and bytes. Returns per-dst score, interval_cv, and confidence. |
detect_persistence(facts: HASH) -> (HASH, ERROR) |
all | Detects persistence indicators from host evidence facts. |
detect_priv_esc(facts: HASH) -> (HASH, ERROR) |
all | Detects potential privilege-escalation indicators from host facts. |
detect_suspicious_files(paths: ARRAY) -> (HASH, ERROR) |
all | Flags suspicious files via entropy tiers (high/very-high), executable magic under a document extension (extension_mismatch), and disguised double extensions (e.g. invoice.pdf.exe). A path that cannot be read is reported rather than skipped: it lands in unread with the reason the open failed, complete goes false, and the double-extension test still runs on its name, because a name is evidence that does not need the file opened. detected and count describe the files that were examined, so a scan that could not open half the evidence cannot come back as nothing suspicious. |
sigma_match(rule: STRING|HASH, event: HASH) -> (HASH, ERROR) |
all | Asks one rule about one event. A field named in the rule is looked up on the event and then inside extra, where events_from() keeps the source entry verbatim, so a rule written in a source's own taxonomy -- Image, CommandLine, EventID -- runs against a normalized Mutant event with no field-mapping file in between. String comparison is case-insensitive unless |cased, values carry Sigma's * and ? wildcards, and a field holding a list matches when any element does. fields_missing names the fields the rule read and this event did not carry: a rule that did not match because the field was never there is a different answer from a rule that looked and disagreed, and only one of them means clean. Accepts the rule as YAML text or as a sigma_parse hash; the hash is recompiled on every call, which is the cost sigma_scan exists to avoid. An event nesting deeper than a keyword search can read -- past 256 levels, which the file parsers will not build but json_parse and ndjson_parse can -- returns an error instead of a no-match it did not earn. Returns (result, err). |
sigma_parse(rule: STRING|BYTES) -> (HASH, ERROR) |
all | Compiles one Sigma detection rule from YAML. The rule is validated rather than accepted: a condition naming a search identifier the detection block does not define, an all of filter* that matches no identifier, an of them that covers nothing because every search identifier starts with an underscore, an N of ... that asks for more searches than its target covers and so can never match, a search that tests nothing and therefore holds for every event (an empty mapping, an empty value list), a null inside a list of values, a modifier this engine does not implement, a rule collection, timeframe, near and aggregation pipes are all errors. That is deliberate -- a rule this engine cannot evaluate has to fail where you can see it, because a detection that silently never fires reads as coverage on a report and is a blind spot in the evidence. Supported value modifiers: contains, startswith, endswith, all, cased, re (with i/m/s), base64, base64offset, utf16/utf16le/utf16be/wide, windash, cidr, lt/lte/gt/gte, exists, fieldref. |windash permutes the interchangeable dash characters at each position a switch starts, rather than substituting one of them everywhere, and refuses a value that would expand past 625 spellings. The returned hash carries the detection block verbatim, so it is a rule and not a description of one: sigma_match and sigma_scan take it straight back. Returns (rule, err). |
sigma_parse_all(ruleset: STRING|BYTES) -> ([]HASH, ERROR) |
all | Compiles every rule in a multi-document YAML ruleset, which is the shape a Sigma ruleset ships in. One rule that does not compile fails the call rather than being dropped quietly: a ruleset that loads 43 of its 44 rules is a ruleset you believe covers something it does not. Returns (rules, err). |
sigma_scan(rules: STRING|HASH|ARRAY, events: ARRAY|HASH) -> (HASH, ERROR) |
all | Runs a ruleset over a timeline, compiling each rule once and then walking the events. Every hit records which rule fired, which of its searches held, and the event itself, so a hit is reviewable without a second lookup. unmatched_fields names the fields no event in the whole scan carried, which is the honest answer to whether the ruleset had anything to look at: a rule reading Image against a timeline that has no Image field did not clear the host, it never ran. One event a keyword search cannot read whole -- nesting past 256 levels, which the file parsers will not build but json_parse and ndjson_parse can -- fails the scan by name rather than being counted as clean. Returns (report, err). |
Process Forensics (9)#
Live process inspection: enumeration, tree, environment, open files, threads, modules, executable hashing, memory scanning, and signaling.
| Builtin | Platforms | Description |
|---|---|---|
process_env(pid?: INTEGER) -> (HASH, ERROR) |
all | Returns environment variables for a process (cross-platform; other processes may require privileges). A Windows per-drive entry keeps its own name, which begins with '=' ('=C:'). Refuses, rather than reporting an empty environment, when another process's environment could not be read. |
process_hash(pid?: INTEGER) -> (HASH, ERROR) |
all | Computes SHA-256 hash metadata for a process executable. |
process_kill(pid: INTEGER, signal?: INTEGER) -> (BOOLEAN, ERROR) |
all | Sends a signal to a process (default SIGKILL semantics). |
process_list() -> ([]HASH, ERROR) |
all | Lists running processes (pid, ppid, name) natively on Windows, Linux, and macOS. |
process_memory_scan(pid: INTEGER, pattern: STRING) -> (HASH, ERROR) |
windows, linux | Scans another process's readable memory for a byte pattern and returns {pid, pattern, matched, truncated, addresses}. Real scan on Linux (/proc/ |
process_modules(pid?: INTEGER) -> ([]STRING, ERROR) |
windows, linux | Lists loaded module/library paths for a process (memory maps on Linux, Toolhelp32 on Windows; fails honestly on platforms without a backend, e.g. macOS). Refuses pid 0 on Windows, where Toolhelp32 reads 0 as the calling process and would report Mutant's own modules as the System Idle Process's. |
process_open_files(pid?: INTEGER) -> ([]STRING, ERROR) |
all | Lists open file paths for a process (cross-platform; may require privileges for other processes). |
process_threads(pid?: INTEGER) -> (HASH, ERROR) |
all | Returns {pid, count, tids} for a process. The thread count is cross-platform; tids are populated where the OS exposes them (e.g. Linux). |
process_tree(rootPid?: INTEGER) -> (HASH, ERROR) |
all | Returns descendant processes for a root pid (default current process). Cross-platform, using real parent PIDs on every OS. Each pid is reported at most once and the root is never its own descendant, so a parent cycle -- Windows reports pid 0 as its own parent -- ends the walk rather than repeating it. A root pid that is not in the process table is refused rather than reported as a process with no descendants. |
Memory Forensics (7)#
Memory-dump analysis: segmentation with entropy, string extraction, pattern scanning, and PE/shellcode discovery.
| Builtin | Platforms | Description |
|---|---|---|
mem_find_pe(path: STRING) -> (HASH, ERROR) |
all | Finds PE headers in a memory image: carves each MZ marker and confirms real PEs by following e_lfanew to "PE\0\0". Returns {candidates, confirmed, headers:[{mz_offset, confirmed, pe_offset, machine}]}. |
mem_find_shellcode(path: STRING) -> ([]HASH, ERROR) |
all | Scans a memory dump file for common shellcode byte signatures. |
mem_map(path: STRING) -> ([]HASH, ERROR) |
all | Splits a memory dump into fixed-size (4 KiB) segments, each with measured entropy and printable-byte ratio. A raw dump carries no page-protection metadata, so no readable/writable/executable flags are reported. |
mem_read(path: STRING, offset: INTEGER, size: INTEGER) -> (HASH, ERROR) |
all | Reads a byte range from a memory image. |
mem_read_bytes(path: STRING, offset: INTEGER, size: INTEGER) -> (BYTES, ERROR) |
all | Reads a byte range from a memory image as a BYTES buffer. Unlike mem_read it returns the buffer itself rather than a hash: the offset is the one you asked for, and a short read at end-of-image is simply a shorter length. Prefer this whenever the range is going to be parsed rather than printed. |
mem_scan(path: STRING, pattern: STRING) -> (HASH, ERROR) |
all | Scans a memory image for a string/byte pattern. |
mem_strings(path: STRING, minLen?: INTEGER) -> ([]STRING, ERROR) |
all | Extracts printable strings from memory image data. |
Binary Analysis (14)#
PE/ELF/Mach-O/DWARF parsing, imports, sections, strings, entropy, literal signature scanning, and Go-binary metadata recovery (GoReSym).
| Builtin | Platforms | Description |
|---|---|---|
bin_dwarf_parse(path: STRING) -> (HASH, ERROR) |
all | Parses DWARF metadata and reports compile unit information. |
bin_elf_parse(path: STRING) -> (HASH, ERROR) |
all | Parses ELF headers and returns core binary metadata. |
bin_entropy(path: STRING) -> (HASH, ERROR) |
all | Computes binary entropy signal. |
bin_imports(path: STRING) -> (HASH, ERROR) |
all | Returns imported symbols/libraries from a binary. |
bin_is_go(path: STRING) -> (HASH, ERROR) |
all | Quick check whether a binary was produced by the Go toolchain, using three signals (build info blob, Go build ID, and a parseable pclntab — the one that survives stripping). Returns {is_go, go_version, has_buildinfo, has_build_id, has_pclntab}. Returns (result, err). |
bin_macho_parse(path: STRING) -> (HASH, ERROR) |
all | Parses a Mach-O binary (macOS/iOS). Handles thin and fat/universal images. For a thin binary returns {format, fat, magic, cpu, type, flags, num_sections, num_commands, imported_libraries}; for a fat binary returns {format, fat, num_arches, architectures:[{cpu, type, offset, size, align}]}. Returns (result, err). |
bin_pe_parse(path: STRING) -> (HASH, ERROR) |
all | Parses PE headers and returns core binary metadata. |
bin_sections(path: STRING) -> (HASH, ERROR) |
all | Returns binary section table information. |
bin_strings(path: STRING, minLen?: INTEGER) -> ([]STRING, ERROR) |
all | Extracts printable strings from a binary. |
bin_yara_scan(path: STRING, rules: ARRAY, caseInsensitive?: BOOLEAN) -> (HASH, ERROR) |
all | Literal multi-string scan of a file (NOT a real YARA engine — that needs cgo). Reports every offset of each rule string. Case-sensitive unless caseInsensitive is true. Returns {engine, matched, total_hits, hits:[{rule, count, offsets}]}. |
go_build_id(path: STRING) -> (STRING, ERROR) |
all | Extracts the Go build ID from a binary. Returns (build_id, err). |
go_buildinfo(path: STRING) -> (HASH, ERROR) |
all | Extracts Go build info from a binary: go_version, module path, main module, dependencies (path/version/sum), and build settings (GOOS/GOARCH/vcs.*). Returns (info, err). |
go_symbols(path: STRING, mode?: STRING) -> (HASH, ERROR) |
all | Recovers function symbols from a Go binary via the pclntab — works even on STRIPPED binaries. A package counts as the examined program's own code when it is at or under the main module path the binary records, or when its first path segment is not a Go standard library name. Returns {go_version, arch, os, pclntab_va, function_count, user_function_count, std_function_count, functions:[{name, package, start, end, stdlib}]}. mode is "all" (default), "user", or "std". Returns (result, err). |
go_types(path: STRING) -> (HASH, ERROR) |
all | Recovers type and interface definitions from a Go binary via GoReSym typelink/itablink parsing, including reconstructed Go source for structs/interfaces where possible. Returns {go_version, type_count, itab_count, types:[{va, name, kind, reconstructed}], itabs:[...]}. Type recovery needs a parseable moduledata; GoReSym v1.7.1 supports it up to ~Go 1.24 and returns an honest error on newer toolchains. Returns (result, err). |
Reporting (7)#
An investigation ends in a report, not a stdout dump. report_new starts one and report_section/report_text/report_list/report_table fill it in; the report itself is a plain hash, so it can be JSON-encoded, diffed against the last one, or written by hand, and every builder returns a new document rather than changing the one it was given. report_render writes it as HTML, Markdown or CSV. Nearly every string in a report came from the evidence, which is to say it was written by the subject of the investigation, so the model holds values and each format is escaped for the thing that actually goes wrong in it: HTML is the boundary, and never turns evidence text into a link; Markdown is escaped for structure, so a pipe in a filename cannot shift a table column; CSV guards the cells a spreadsheet would execute when the file is opened.
| Builtin | Platforms | Description |
|---|---|---|
report_list(report: HASH, items: ARRAY, opts?: HASH) -> (HASH, ERROR) |
all | Adds a bulleted list, or a numbered one with {"ordered": true}. Items are scalars, rendered the way a CSV cell is, so a count that arrived as an INTEGER needs no conversion. Returns (report, err). |
report_new(title: STRING, opts?: HASH) -> (HASH, ERROR) |
all | Starts a report. The report is a plain hash -- title, generated, and a sections array -- so it can be JSON-encoded, diffed against yesterday's, or written by hand; every builder here returns a new document rather than changing the one it was given. generated defaults to now and is the only field that cannot be derived from the document, so pinning it makes two renders of one investigation the same bytes. Returns (report, err). |
report_render(report: HASH, format: STRING, opts?: HASH) -> (STRING, ERROR) |
all | Renders a report as "html", "markdown" or "csv". The whole document is validated first and a block nothing renders is an error naming its section and index, because a renderer that steps over what it does not understand produces a report that looks complete and is missing a finding. HTML is the format to hand to a person: every string is escaped, the stylesheet is inlined, there is no script, font or image, and evidence text is never turned into a link -- a report that makes the attacker's URL clickable is a report that can be clicked. Markdown is escaped for structure, so a pipe in a filename cannot shift a table column. CSV carries one table, and guards the cells a spreadsheet would execute. Returns (text, err). |
report_section(report: HASH, heading: STRING, opts?: HASH) -> (HASH, ERROR) |
all | Opens a section. Everything added afterwards lands in it, until the next one. Returns (report, err). |
report_table(report: HASH, rows: ARRAY, opts?: HASH) -> (HASH, ERROR) |
all | Adds a table. Rows arrive in either shape csv_stringify takes -- an array of arrays written positionally, or an array of hashes written against a column list -- and go through the same normalization, so a table in a report and the same table written straight to CSV cannot disagree about what a cell contains. A table with no rows and no columns is refused: pass columns to record that a search found nothing. Returns (report, err). |
report_text(report: HASH, text: STRING) -> (HASH, ERROR) |
all | Adds a paragraph. Line breaks inside it survive every format. A paragraph written before the first report_section lands in a lead section that renders without a heading, because a report usually opens with a summary. Returns (report, err). |
report_write(report: HASH, path: STRING, opts?: HASH) -> (HASH, ERROR) |
all | Renders a report and writes it to disk, returning what the file turned out to be: {path, bytes, format, sha256}. The format comes from the path's extension -- .html, .htm, .md, .markdown, .csv -- or from {"format": ...} for a path whose name says nothing. The digest is read back off the disk and checked against the document that was meant to be there, so a short write or a full disk is a refusal rather than a hash of something nobody can reproduce. When a case is open the write becomes a timeline entry carrying the path and the digest, which is the whole of the link between an investigation and the documents it produced. Returns (result, err). |
Chain of Custody (37)#
A case session that records who opened what, when, with which build of the tool. case_open(id, examiner) starts it; from there every evidence opener records its source into the manifest and every builtin that reads through an evidence handle is counted against that source. case_verify re-measures the sources and reports drift; case_write seals the manifest with a SHA-256 over its own contents and an Ed25519 signature, and case_manifest_verify checks both from the file alone. Nothing is recorded until a case is opened, so a program that does not use this is unaffected by it. The read-only guarantee the manifest asserts is machine-checked: see EVIDENCE_HANDLING_POLICY.md. The case key is separate and never travels as an argument: case_key_create, case_key_open, case_key_rotate and case_key_fingerprint take a path to a key file whose passphrase is read from the terminal, and class_define/class_list declare the classification labels a record segment may carry, each tagged by an HMAC keyed to the case key so that two investigations declaring the same label produce different tags. What a classification does and does not promise is stated in DISCLOSURE_POLICY.md. A case also outlives the run in a forensic ledger: case_attach keeps its lifecycle and who holds which role in it, the evidence_* builtins the custody of each exhibit, review_request and review_decide who reviewed the case and what they decided, and retention_set and retention_hold how long it is kept and what holds it -- a legal hold stops every disposal and every erasure until retention_release lifts it. case_key_erase erases the case key, which is the erasure that reaches every copy of every record wrapped under it; erasure_list reads back what was erased; and case_transition disposes of a retained case only when no hold is in force, the retention period has run, every exhibit is returned or disposed of, and the key is erased.
| Builtin | Platforms | Description |
|---|---|---|
audit_head() -> (HASH, ERROR) |
all | Returns the head of the security audit chain: the SHA-256 commitment to every security event this run recorded, in order. The counters in case_manifest say how many times a check tripped; the chain says in what order and at which stage, which is the question a counter cannot be asked afterwards. The head covers every event ever recorded even when older entries have been dropped from memory, so entries and retained are different numbers and both are reported. |
audit_verify(path: STRING, head?: STRING) -> (HASH, ERROR) |
all | Re-walks a written audit log, recomputing every link and the timestamp each entry displays against the one its hash covers, and stops at the first entry that disagrees. Pass the head from the case manifest as the second argument to check the log against a value its own writer did not choose: anchored says whether a head was supplied and anchor_matches whether it agreed, because a log checked only against the head stored inside it has been checked against itself. A log whose entries were dropped for the memory cap begins where the cap rule says a log of its length must: the last entry's seq is inside its hash, so it fixes the first entry the log may start at, and a log missing any other prefix breaks at its first entry. Only that first entry's prev is a claim rather than something the document can check, and only on a capped log. entries, dropped and chain_complete are recomputed from the entries, never copied from the document. An entry with no positive integer seq is itself a break; broken_index is the break's position in the log (1-based) and broken_at its seq, 0 when it has none. |
audit_write(path: STRING) -> (HASH, ERROR) |
all | Writes the audit chain to disk as JSON: every retained entry with its sequence number, timestamp, event, stage, the hash of the entry before it and its own hash. The file is read back and hashed after writing, so a document that is not what was written is an error rather than a digest nobody can reproduce. When a case is open the write is recorded in its timeline, the same way a report is. The log carries no signature of its own -- what ties it to a run is that its head matches the one sealed into that run's manifest. |
case_assign(ledger: INTEGER, examiner: STRING, role: STRING, reason: STRING) -> (HASH, ERROR) |
all | Records the role an examiner holds in the attached case, or ends it with the role none, as the next event of that examiner's assignment chain in the ledger. A later case_attach by that examiner must assert the role the chain's head names. Anybody attached may assign anybody: who may assign is an organisation's policy and this program does not hold one, so the record says who assigned whom, and neither name is authenticated. A recipient-only role is refused -- nobody works a case as one -- as is an assignment that changes nothing. Refused while the case is disposed, and on a ledger opened as auditor. Returns (assignment, err). |
case_attach(ledger: INTEGER) -> (HASH, ERROR) |
all | Binds the open case to a ledger, so that its state outlives the run. The first attach registers the case: a Case node, a registered lifecycle event and the examiner's assignment under the role they asserted. Every later attach reads the case back -- its lifecycle state, every examiner's latest assignment, and the classes, views and recipient role bundles declared under the open key generation, each class tagged again under that key and compared with the tag the ledger holds -- and refuses an examiner the ledger assigns no role, or a different one; that keeps two records from disagreeing about one person and is not access control. Keys and grants are never read back. Needs an open case with its key open, a ledger opened by the case's examiner under an asserted role, and no classes, views or role bundles defined in this run yet: attach before defining, so every definition the case holds is one its ledger holds. While attached, class_define, view_define and role_define write through to the ledger, and the case is written through this ledger and no other: a builtin that takes the ledger is refused one the case is not attached to, which records no lifecycle for it, and the manifest carries a ledger_state block marked source: ledger. Lifecycle and assignment records are hash-linked chains, and a chain with two events after one, or an event whose uid does not recompute, is refused by every reader rather than resolved. Refused on a ledger opened as auditor. Returns (case, err). |
case_bundle(dir: STRING, opts?: HASH) -> (HASH, ERROR) |
all | Writes the handover: manifest.json, report.html, report.md and a SHA256SUMS any sha256sum can check. The reports are written first and the manifest records what they hashed to, so the seal over the manifest -- a SHA-256 over every other field, signed with the local key pair -- covers the reports too: edit a byte of report.html and it no longer matches the case it claims to be from. The binding runs one way on purpose; a report quoting the manifest's hash would be quoting a document that had not been written yet. Returns (result, err). |
case_close() -> (HASH, ERROR) |
all | Closes the case and returns its final manifest. After this, evidence openers stop recording. |
case_evidence(path: STRING, options?: HASH) -> (HASH, ERROR) |
all | Brings a file under custody that no evidence opener will touch -- a carved file, an export, a hash list handed over with the drive. Registers its size, modification time and, under the case's hash policy, its digest. |
case_key_create(path: STRING, options?: HASH) -> (HASH, ERROR) |
all | Mints the case key a classified record is sealed under and writes it to a file that must not already exist. The passphrase is asked for at the terminal and never appears in an argument: key material in program text is key material a traceback can print, and a Go string holding a secret cannot be wiped. Refuses an existing path, because writing a key over a key makes every record sealed under the old one unopenable. |
case_key_erase(ledger: INTEGER, key_path: STRING, reason: STRING, options?: HASH) -> (HASH, ERROR) |
all | Erases the case key, which is the erasure that reaches every copy: with the key gone, no copy of any record wrapped under it opens for anybody who relies on the case key. Left alone it erases every generation -- the key file is overwritten with zeros, forced to the disk and removed -- and zeroes the key the open case holds; a case is disposed of only after this. {"generation": n} erases one generation that is not the current one: its wrapped key is overwritten with zeros in the file, whose MAC is recomputed under the passphrase asked for at the terminal, and the file is rewritten in place, with a copy of what it is becoming kept beside it until the overwrite is on the disk. It reaches neither a copy of the key file kept anywhere else nor a grant already issued, which carries its segments' keys and not the case key, and does_not_erase says so. Recorded as the next event of the case's erasure chain in the ledger -- an ErasureEvent that ERASES the Case, is IN_CASE and PERFORMED_BY the examiner -- naming the generations erased, their fingerprints and the key file's MAC; the key file's path is not recorded. Refused for a key file that is not the open case's, while the case is disposed or under a legal hold, for an examiner not acting as a case_owner or an administrator (asserted, and not access control), while a record handle holds a key an erased generation wrapped, once the ledger records every generation erased, and on a ledger opened as auditor. Returns (erasure, err). |
case_key_fingerprint(path: STRING) -> (HASH, ERROR) |
all | Reads what a key file says about itself, with no passphrase and no unwrapping. authenticated is always false and says so: without the passphrase the file's own MAC cannot be checked, so every field returned is a claim. The signature narrows that to a claim by the holder of a signing key, which is better and is not proof. Each generation says whether it is erased: the zeros case_key_erase writes over an erased generation's wrapped key. |
case_key_open(path: STRING, options?: HASH) -> (HASH, ERROR) |
all | Unwraps a case key and holds it for the life of the open case. Returns no handle and has no closer: the key's lifetime is the case's, and case_close zeroes it. Refuses a key whose case id is not the open case's, a generation case_key_erase erased, and a second key while one is open. |
case_key_rotate(path: STRING, options: HASH) -> (HASH, ERROR) |
all | Changes the passphrase, or mints a new case key. mode is required and has no default because the two do different things. Neither reaches a disclosure already issued: a grant is bytes in someone else's hands, and rotation is not revocation. |
case_manifest() -> (HASH, ERROR) |
all | Returns the case manifest as it stands: the case and examiner, the tool build, every evidence source with its size and digest, every builtin that touched each source with a count, the timeline, and the security telemetry for the run. Readable while the case is open and after it closes. |
case_manifest_verify(path: STRING) -> (HASH, ERROR) |
all | Checks a written manifest: that its contents still hash to the value in its seal, and that the signature over them holds. A function of the file alone -- it needs neither the case that produced it nor any key the reader does not already hold. |
case_note(text: STRING, data?) -> (HASH, ERROR) |
all | Records an examiner's note in the case timeline, with an optional value alongside it. The timeline holds what the analyst did -- opens, notes, verifications, the close -- and never grows with what the program read. |
case_open(id: STRING, examiner: STRING, options?: HASH) -> (HASH, ERROR) |
all | Opens a chain-of-custody session. From here until case_close, every evidence opener records its source into the case manifest and every builtin that reads through an evidence handle is counted against that source. Only one case may be open at a time. |
case_report(opts?: HASH) -> (HASH, ERROR) |
all | Renders the open case as a report value, in the shape report_new builds: the case header, the evidence with its digests, what each builtin touched and how often, the timeline, the integrity statement and the security counters. It is read from the manifest rather than from the session, so the report and the manifest cannot disagree about what was examined; and it is a value, so an examiner's conclusions can be added with report_section and report_text before anything is rendered. |
case_transition(ledger: INTEGER, to: STRING, reason: STRING) -> (HASH, ERROR) |
all | Moves the attached case along its lifecycle -- registered, active, in_review, concluded, retained, disposed -- by appending a lifecycle event to the ledger, hash-linked to the one before it and attributed to the examiner and the role they asserted. It makes the moves that are an examiner's decision alone: registered to active, and reopening a concluded case. It also disposes of a retained case -- recorded by an examiner acting as a case_owner or an administrator, which is asserted and not access control -- but only when the ledger shows everything a disposal needs: no legal hold in force, the retention period run, every exhibit returned or disposed of, and every generation of the case key erased with case_key_erase. A refusal names all of what is missing at once, a disposal deletes nothing, and the move names the erasure of the case key it rests on. The review and retention moves are made by the builtins that record them -- review_request submits an active case for review, review_decide concludes it or sends it back, retention_set retains a concluded case -- and a refusal names the state the case is in, the moves available from it, and the builtin that makes a move case_transition does not. A state refuses what it cannot take: in review, sealing, definitions, reclassifications, disclosures, redaction versions and new evidence; concluded or retained, sealing and new evidence; disposed, everything but a withdrawal and the movement of an exhibit already taken in, which every state takes. Builtins that take the ledger ask the ledger for the state -- the one the case is attached to, and no other -- and the rest ask the case attached in this run. Refused on a ledger opened as auditor. Returns (transition, err). |
case_verify() -> (HASH, ERROR) |
all | Re-measures every source under custody and reports what moved. A case opened with a hash policy compares digests; one opened without compares size and modification time. Each source says which basis was used, so a weaker check is never mistaken for a stronger one. |
case_write(path: STRING, options?: HASH) -> (HASH, ERROR) |
all | Writes the manifest to disk as a signed JSON document. The seal carries a SHA-256 over every field except itself and an Ed25519 signature over the same bytes, from the local key pair Mutant already maintains; the public key travels in the document, so case_manifest_verify needs nothing but the file. |
class_define(label: STRING, options?: HASH) -> (HASH, ERROR) |
all | Declares one classification label and returns the tag its segments will carry. Labels are declared before use so that a typo is an error rather than a new secret class nobody recognises. The tag is keyed to the case key, so two investigations that both declare "restricted" produce different tags and neither can be linked to the other. While the case is attached to a ledger with case_attach, the definition is written into the ledger before the run holds it (in_ledger), and a label a later attach read back is answered with already_defined: true instead of being refused -- unless a description is given that differs from the ledger's, which is refused, because a definition there is written once. Refused while the attached case is in review or disposed. |
class_list() -> (HASH, ERROR) |
all | Returns the classification scheme in force, in declaration order. Answers with the same keys whether or not a case is open, so a program branches on a field rather than on an error: open says a case is open, tagged says a case key is open, which is what makes a tag possible at all. The order is presentation order and carries no authority; nothing here enforces a lattice. |
erasure_list(ledger: INTEGER, options?: HASH) -> (HASH, ERROR) |
all | Reads the erasures a ledger records, in every case or in one: each record_erase and case_key_erase, with what it erased, the digests or the key file MAC it recorded, and who erased it and as what. Needs no case open, so an auditor reads it with the ledger alone. Each case's erasure chain is checked as it is read -- that it does not fork, that every uid recomputes, that each event names its own case and erases a record or the case key in the form this program writes, and that nothing follows the erasure of every generation of the case key -- and a chain that fails is refused rather than resolved. A row answers "", 0 or an empty list for the fields only the other kind of erasure records. Returns (erasures, err). |
evidence_accept(ledger: INTEGER, exhibit: STRING, path: STRING, options?: HASH) -> (HASH, ERROR) |
all | Records the examiner an exhibit was released to taking it. The file they were handed is hashed with SHA-256 and compared with the digest taken at intake. A mismatch is refused and nothing is recorded, unless the examiner states what happened with {"discrepancy": "..."}: the accept is then recorded with both digests and the statement, and matches_intake is false. The file is opened read-only and registered in the run's custody. Taken in every lifecycle state; refused on a ledger opened as auditor. Returns (custody event, err). |
evidence_dispose(ledger: INTEGER, exhibit: STRING, statement: STRING) -> (HASH, ERROR) |
all | Records the holder of an exhibit disposing of it, in a statement of how. It deletes nothing -- not the file the exhibit was taken in from, not any copy -- and says so with deleted: false. Recorded by a case_owner or an administrator who holds the exhibit; the role is asserted, and the refusal says it is not access control. Ends the exhibit's custody chain: a disposed exhibit is not handed on or taken in again. Taken in every lifecycle state; refused while the case is under a legal hold (retention_hold), naming the first hold placed, and on a ledger opened as auditor. Returns (custody event, err). |
evidence_history(ledger: INTEGER, options?: HASH) -> (HASH, ERROR) |
all | Reads the custody of every exhibit in a ledger, from each EvidenceFile node and its custody chain: where the exhibit is, who holds it, and every event with who recorded it and as what. Needs no case open, so an auditor reads it with the ledger alone. Each chain is checked as it is read -- that it does not fork, that every event's uid recomputes, that each event moves the exhibit from where the one before left it, that a hand-off is recorded by the holder and an accept by the person the exhibit was released to, and that every hash check agrees with its own digests -- and a chain that fails is refused rather than resolved. A hash check is two answers per event, hash_checked and matches_intake, and discrepancies counts the accepts and re-intakes whose file did not match the intake. Returns (history, err). |
evidence_intake(ledger: INTEGER, exhibit: STRING, path: STRING, options?: HASH) -> (HASH, ERROR) |
all | Takes an exhibit into the attached case's custody. The file is hashed with SHA-256 whatever the case's hash policy, and the ledger gets an EvidenceFile node that BELONGS_TO the case and the first event of the exhibit's custody chain, held by the examiner who took it in. Each custody event is hash-linked to the one before it, and every reader refuses a chain that forks, skips a state, or holds an event whose uid does not recompute. An exhibit the case returned is taken in again under its own name: the re-intake is the next event of the same chain, and the file is compared with the digest taken at the first intake. A mismatch is refused and nothing is recorded unless {"discrepancy": "..."} says what happened; hash_checked and matches_intake say what the comparison found, and a first intake is compared with nothing. An exhibit that is held, in transit or disposed of is not taken in again. The file is opened read-only to hash it and nothing else, and is registered in the run's custody too, so case_verify re-measures it. Needs the case attached with case_attach and the examiner assigned a role in it. A new exhibit is refused while the case is in review, concluded, retained or disposed; a returned one coming back is a movement of an exhibit the case has taken in, and like a release or an accept is taken in every state. Refused on a ledger opened as auditor. Returns (custody event, err). |
evidence_release(ledger: INTEGER, exhibit: STRING, to: STRING, reason: STRING) -> (HASH, ERROR) |
all | Records the examiner who holds an exhibit releasing it to another examiner the case assigns a role, as the next event of its custody chain; the exhibit is then in transit until they accept it with evidence_accept. Recorded by the holder the ledger names, and to somebody assigned in the case, so a misspelt name is refused rather than left holding an exhibit nobody can accept -- that keeps the record consistent with itself and is not access control. Taken in every lifecycle state, because where an exhibit is, is a fact. Refused on a ledger opened as auditor. Returns (custody event, err). |
evidence_return(ledger: INTEGER, exhibit: STRING, to: STRING, reason: STRING) -> (HASH, ERROR) |
all | Records the holder of an exhibit returning it out of the case -- to its owner, or the agency that submitted it. A returned exhibit is not handed on; if it comes back, evidence_intake takes it in again under its own name and compares what came back with what was first taken in. Taken in every lifecycle state; refused on a ledger opened as auditor. Returns (custody event, err). |
retention_hold(ledger: INTEGER, reason: STRING, options?: HASH) -> (HASH, ERROR) |
all | Places a legal hold on the attached case. While any hold is in force nothing in the case is disposed of or erased: evidence_dispose, record_erase, case_key_erase and the disposal of the case refuse, naming the hold. A hold is placed in any state but disposed, and each is its own event, so two holds for two matters are two, and lifting one leaves the other in force. {"authority": name} records whose authority the hold was placed on; left out, it is the examiner's own. authority_basis says which, the hold is AUTHORISED_BY that authority, and neither is checked. Refused while the case is disposed, and on a ledger opened as auditor. Returns (hold, err). |
retention_list(ledger: INTEGER, options?: HASH) -> (HASH, ERROR) |
all | Reads the retention of every case in a ledger that has any, or of one: the period in force and its basis, whether it has lapsed, the holds in force, and every retention event. Needs no case open, so an auditor reads it with the ledger alone. Each chain is checked as it is read -- that it does not fork, that every uid recomputes, that each event names its own case, that each period names a time, and that each release lifts a hold placed before it and still in force -- and a chain that fails is refused rather than resolved. Returns (retention, err). |
retention_release(ledger: INTEGER, hold: STRING, reason: STRING) -> (HASH, ERROR) |
all | Lifts a legal hold on the attached case, named by the uid retention_hold or retention_list gives it; other holds stay in force. Anybody attached may lift a hold: who may is an organisation's policy and this program does not hold one, so the record says who lifted it and as what. Refused for a hold the case never had, one already lifted (naming when and by whom), a disposed case, and a ledger opened as auditor. Returns (release, err). |
retention_set(ledger: INTEGER, until: STRING, basis: STRING) -> (HASH, ERROR) |
all | Records the date the attached case is kept until, and the basis for keeping it -- a statute, a policy, a court's order. The first period retains a concluded case, moving it to retained in the same commit; a later one changes a retained case's period and says what it was in previous_until. A date is taken as its first instant in UTC and an RFC 3339 time as given; one already passed is recorded and reported lapsed, and nothing is disposed of when a period runs. A case's retention events are one hash-linked chain, which every reader walks. Refused for a case neither concluded nor retained, a period the case already has, a disposed case, and a ledger opened as auditor. Returns (retention, err). |
review_decide(ledger: INTEGER, request: STRING, decision: STRING, reason: STRING) -> (HASH, ERROR) |
all | Records a reviewer's answer to a request for a review -- approved, changes_requested or rejected -- with the reason. A review of the case moves it on: approved to concluded, changes_requested or rejected back to active. A review of a record or a redaction version is recorded and moves nothing. Recorded by an examiner acting as reviewer who is not the one who asked for the review; the role is asserted, and the refusal says it is not access control. A request is decided once: a second decision is refused and names the first, and a new request asks again. The decision is a chain of one event keyed by its request; it REVIEWS the request, is IN_CASE and PERFORMED_BY the reviewer, and carries the hash the request bound. Needs the case attached, so a reviewer attaches under the reviewer role the case assigns them. Refused while the case is disposed, and on a ledger opened as auditor. Returns (decision, err). |
review_list(ledger: INTEGER, options?: HASH) -> (HASH, ERROR) |
all | Reads the review requests in a ledger, each with its decision, from every case or from one. Needs no case open, so an auditor reads it with the ledger alone. Each case's chain of requests and each request's decision is checked as it is read -- that neither forks, that every uid recomputes, that each event names its own case and request, and that no request is decided twice -- and a chain that fails is refused rather than resolved. A row gives what was asked about and the hash it was bound by, who asked and as what, and the decision with who made it; an undecided request says decided: false and answers "" for the decision's fields. Returns (reviews, err). |
review_request(ledger: INTEGER, subject: STRING, note: STRING, options?: HASH) -> (HASH, ERROR) |
all | Asks for a review of the attached case, of one of its records or of one of its redaction versions, and binds what is asked about by a hash a reviewer can hold their copy to. The subject is "case", a record's uid, or a redaction version's uid (from redaction_commit or redaction_versions), told apart by their length. A record is bound by the SHA-256 of the record file the ledger holds, and a redaction version by the digest of what it releases. A case is reviewed as a written manifest shows it: {"manifest": path} names one case_write wrote, which must be this case's, still hash to its seal, carry a signature that holds if it is signed, and have been written with the case's lifecycle where it stands now -- the manifest a run renders in memory changes every time it is looked at, so it binds nothing. A review of the case moves it from active to in_review, where it takes no sealing, definitions, disclosures, redaction versions or new evidence until review_decide answers; a review of a record or a redaction version moves nothing. One request is open at a time for one subject. A case's requests are one hash-linked chain, and each request is IN_CASE, REVIEWS its subject and is PERFORMED_BY the examiner. Anybody attached may ask. Refused while the case is disposed, and on a ledger opened as auditor. Returns (request, err). |
Classified Records (13)#
Evidence encrypted at rest, in a container whose structure is public and whose content is not. record_classify_range names a run of bytes and the class it carries; record_seal writes a .mrec in which each classification span is split into segments of at most 64 KiB, and a segment is at once the AEAD unit, the key-derivation unit and the smallest disclosable unit -- so holding one segment's key can never open bytes of two classifications. A default class is required and covers every byte no range names, because a record with an unlabelled remainder discloses that remainder to everyone. Range boundaries and lengths are public by construction, which is deliberate: a redaction nobody can see is a redaction nobody can challenge. record_seal_quantised rounds boundaries outward where a short secret must not advertise its length; it requires a rounds_to option naming the one class rounding may grow, and records how many bytes changed class beside the class they changed into -- a count with no direction cannot be read, because whether widening a range withholds those bytes or releases them depends on which of the two classes is the more sensitive, and nothing here orders them. record_verify checks a record holding no key at all -- the property a recipient granted nothing still has -- and reports in a does_not_prove field what a valid signature does not establish. record_read refuses a span it cannot fully decrypt and names the segments; record_read_partial is the separate contract that returns the holes as data. record_search finds a literal in what one reader of a record could read -- the segments a grant opens, or those a named view would -- never reports a match that needs a byte it did not read, and is recorded in the case timeline with the pattern only as a digest keyed to the case key. Plaintext any of the three returns -- a search's as the context around each hit -- is marked with the record and the classes it came from, and every builtin that sends a value out of the process -- putln, putf, fs_write, fs_append, http_post, http_request, report_write, report_render, report_table, report_list, case_note, cache_put, ledger_add_node, ledger_add_edge, db_add_artifact and db_add_relation -- refuses a marked buffer wherever it sits in its arguments, naming the record and never a byte; bytes_slice carries the mark to the slice, and + of two buffers to the joined one. record_release(buffer, reason) is the deliberate way out: it requires a reason and an open case, writes the release into the case timeline, and returns an unmarked copy. The mark catches accidents, not adversaries: a conversion to a string, to hex or to JSON drops it, and so does a loop that rebuilds the buffer byte by byte. record_erase destroys the record key in one copy of a record and says what it did not reach -- every other copy, and every grant already issued. What a disclosure can and cannot promise is stated in DISCLOSURE_POLICY.md.
| Builtin | Platforms | Description |
|---|---|---|
record_classify_range(offset: INTEGER, length: INTEGER, label: STRING) -> (HASH, ERROR) |
all | Describes one run of bytes and the class it carries, resolving the label to the tag its segments will actually be bound to. It is a builtin rather than a hash you write out so that a label which was never declared is refused at the line that named it, rather than at the seal -- by which point a program has usually built a list and lost track of which entry was wrong. Ranges do not have to be given in order and must not overlap: which label wins where two ranges disagree is a legal question, not one this tool may answer for you. Returns (range, err). |
record_close(record: INTEGER) -> (BOOLEAN, ERROR) |
all | Closes a record handle, zeroing its key schedule and releasing the file. The handle comes from record_open and is not a ledger or database handle; the three handle spaces are separate so that no builtin of one family can resolve a handle of another. |
record_erase(ledger: INTEGER, path: STRING, reason: STRING, options?: HASH) -> (HASH, ERROR) |
all | Destroys the record key in one copy of a record: the wrapped key and its nonce in the file's header are overwritten in place with zeros as wide, forced to the disk and read back, and the open case key must then unwrap nothing in the file. That is all it destroys, and does_not_erase says so in the result: every other copy of the record -- a disclosure package's included -- carries the same wrapped key and still opens with the case key, a grant already issued still opens the segments it names in any copy, the storage under the file may still hold the overwritten bytes, and the evidence the record was sealed from and the ledger's record of it are untouched. case_key_erase is the erasure that reaches every copy. The erasure is recorded as the next event of the case's erasure chain in the ledger -- an ErasureEvent that ERASES the record's Record node, is IN_CASE and PERFORMED_BY the examiner -- with the file's digest and its header's before and after. The record's signature covered the overwritten bytes, so record_verify reports an erased copy as erased with its signature no longer holding, and a copy that hashes to the digest recorded after is the copy the erasure left. Everything that can refuse runs before a byte is written: the case attached and not disposed, no legal hold in force, an examiner acting as a case_owner or an administrator (the role is asserted, and the refusal says it is not access control), the case key open at the generation that wraps the record, a record that key sealed in the form record_seal writes, one the ledger does not know by another digest, no handle in this process holding its key, a file that can be written, and a ledger that does not record the case key erased. {"preview": true} runs every check and hashes the file as it is and as the erasure would leave it, and writes nothing. Refused on a ledger opened as auditor. Returns (erasure, err). |
record_layout(record: INTEGER) -> (HASH, ERROR) |
all | Reports the record's public structure -- every span, every segment, where each one sits and which class it carries -- and reads no plaintext to do it. boundaries_are_public is returned as a field rather than left to a document, because a reader of a layout is exactly the reader who might otherwise conclude the opposite: offsets and lengths are visible to everyone holding the record, with no key at all. Returns (layout, err). |
record_open(path: STRING, options?: HASH) -> (HASH, ERROR) |
all | Opens a record for reading and returns a handle. By default it needs the case key the record was sealed under: the record key is wrapped under it, and a record sealed by another case is refused by its case uid before any unwrapping is attempted, as is a copy whose key record_erase erased. With {"grant": path} it opens under a disclosure grant instead -- the recipient's side of a disclosure -- which needs no case and no case key, asks at the terminal for the grant's passphrase, and opens exactly the segments the grant carries material for; a grant for a different record, or one whose segments this header describes differently, is refused before a single segment is tried. opened_with says which, and granted_segments how many segments will open. The handle can only ever open -- there is no path from here to sealing, and a record opened under a grant cannot issue one. Record handles are their own space and are not ledger or database handles. Close it with record_close. Returns (record, err). |
record_prove_segment(record: INTEGER, index: INTEGER) -> (HASH, ERROR) |
all | Cites one segment in a form a third party can check against the copy they hold: which segment, at which offset, under which class, with which digest, folding into which signed root. It is deliberately not a Merkle inclusion proof -- a tree buys a compact path to the root, and compactness is worth having only when the verifier does not have the data, but a disclosure hands over a record byte-identical to the one under custody, so every recipient already holds every segment and can recompute the whole fold. The root is recomputed from the bytes on disk rather than read from the footer, so root_matches is about the file and not about what the file says of itself. content_free is true: nothing here reveals plaintext. Returns (citation, err). |
record_read(record: INTEGER, offset: INTEGER, length: INTEGER) -> (BYTES, ERROR) |
all | Returns a span of plaintext, or refuses and names the segments that stood in the way. It is an error and not a short read: a caller who wrote record_read(r, 0, n) believes they are getting n bytes of evidence, and a buffer with zeros where the withheld spans were would look exactly like evidence and not be it. Use record_read_partial for the bytes that are readable, which returns the holes as data instead. The result is BYTES and never a STRING, because a Go string cannot be zeroed and the runtime copies one at will. Returns (bytes, err). |
record_read_partial(record: INTEGER, offset: INTEGER, length: INTEGER) -> (HASH, ERROR) |
all | Returns what is readable plus exactly what is not. The unreadable spans are zero-filled rather than removed, so offsets still line up with the record's own numbering -- the same reason the format refuses to pad, since a recipient who cannot place their fragments at their true positions cannot reconstruct the document they were given. holes_are_zero_filled is a field because zeros are also a thing evidence contains. length and withheld are both reported so a caller does not have to subtract. Returns (result, err). |
record_release(buffer: BYTES, reason: STRING) -> (BYTES, ERROR) |
all | Returns an unmarked copy of classified plaintext and records, in the open case's timeline, that it was released and why. Plaintext from record_read and record_read_partial, and the context of a record_search hit, is marked, and the builtins that send a value out of the process -- putln, putf, fs_write, fs_append, http_post, http_request, report_write, report_render, report_table, report_list, case_note, cache_put, ledger_add_node, ledger_add_edge, db_add_artifact and db_add_relation -- refuse a marked buffer anywhere inside their arguments, naming the record and the classes and never the bytes. This is the deliberate path past that check: an accident is caught, a decision is recorded. The mark is carried by bytes_slice and by + of two buffers and by nothing else, so a conversion to a string, to hex or to JSON also loses it -- it catches accidents, not adversaries. A buffer that carries no mark is refused rather than copied, because a release of nothing would put a release that did not happen in the timeline. Needs an open case to record into. Returns (buffer, err). |
record_seal(source: STRING, record: STRING, ranges: ARRAY, options: HASH) -> (HASH, ERROR) |
all | Encrypts a file into a .mrec at the classification boundaries it was given. The default option is required and names the class every byte no range covers will carry: there is no implicit unclassified, because a record with an unlabelled remainder discloses that remainder to everyone who is disclosed anything, and "the examiner decided this is open" and "the examiner did not think about this" are different statements. Each span splits into segments of at most segment_size, and a segment never straddles a span, so a segment key can never open bytes of two classifications. Range boundaries and lengths are public by construction -- a redaction nobody can see is a redaction nobody can challenge -- and record_seal_quantised is the answer where a short secret must not advertise its length. The destination must not already exist. The record is written to a temporary file beside it and renamed into place only once it is complete, so a seal that is killed leaves under the name an empty file record_open refuses as interrupted, and beside it a .mutant-record-* file of partial ciphertext; delete both and seal again. Returns (record, err). |
record_seal_quantised(source: STRING, record: STRING, ranges: ARRAY, options: HASH) -> (HASH, ERROR) |
all | Seals with span boundaries rounded OUTWARD to a quantum, so that a four-byte secret is indistinguishable from anything else inside its rounded span. rounds_to is required and names the one class rounding may grow, because every byte a range grows over stops carrying the class it had and starts carrying that range's -- and whether that withholds those bytes or releases them depends on which of the two classes is the more sensitive, which nothing here knows: a class is a label and a tag, and nothing orders them. With default restricted and a four-byte open passage, rounding to sixteen releases twelve bytes nobody cleared, so a range whose class is not the one named is sealed at the boundary it was given rather than rounded. quantised_extra is how many bytes changed class, and rounds_to comes back beside it because a count with no direction cannot be read. Everything else matches record_seal. Returns (record, err). |
record_search(record: INTEGER, pattern: BYTES|STRING, options?: HASH) -> (HASH, ERROR) |
all | Finds a literal pattern in what one reader of a record could read, and returns each match with the plaintext around it. A record opened under a grant is searched as its recipient holds it: the segments the grant opens. A record opened with the case key opens every segment, so the search must name a view, and then reads exactly the segments a grant under that view would open -- the partition view_preview reports. No other segment is read from the file. The matcher carries a partial match across the boundary between two readable segments and starts again after every segment it did not read, so every offset the pattern occurs at is reported, overlapping ones included, and a match that would need even one byte it did not read never is. A hit's context is the readable bytes around the match, clipped where they stop and never zero-filled, and it is marked as a read's output is, so the builtins that send a value out of the process refuse it; offsets and counts are integers and carry no mark, and an offset beside the pattern says what the record holds there. count is every match in the searched bytes, hits the first max_hits of them, and complete whether every segment was searched. A segment the reader should open and cannot is listed in failures, not counted as withheld. The search is written into the open case's timeline with the pattern as a digest keyed to the case key, which only a holder of that key can check a guess against; with no case key open the pattern is not written in any form. Literal only: a regular expression over a record read a segment at a time would miss matches without saying so. Returns (result, err). |
record_verify(path: STRING) -> (HASH, ERROR) |
all | Checks a record holding no key at all, which is the property the whole format is arranged around: a recipient who was granted nothing can still establish that the file they hold is the file that was sealed. It rebuilds every segment descriptor from the public header, recomputes every segment digest from the stored ciphertext, folds the root and checks the signature. signed and signature_valid are two bits and never one, because an unsigned record is a normal record written on a machine with no key store and a forged one is not. does_not_prove is a returned field and not a footnote: a signature authenticates the document, not the names in it. A copy whose key record_erase erased says erased: true, and its signature no longer holds, because it covered the bytes the erasure overwrote; the digests the ledger's record of the erasure gives are what show the rest of the file unchanged. Returns (result, err). |
Disclosure (15)#
Who may open which part of a record, decided once and reviewable before anything is handed over. view_define(label, classes) declares a named posture -- the set of classifications a grant issued under that name will open -- and view_list reports the postures in force. A view names the classes it grants and cannot negate: an "everything except" posture would widen by itself every time a class was declared after it, changing what it releases with nothing edited and nothing recorded, which is the one way a disclosure can change invisibly. view_preview(record, view) says exactly what a view would release from a particular record and exactly what it would hold back, as byte ranges on both sides, and it does so as arithmetic over the record's public header -- it opens no segment and needs no plaintext. Where the record was sealed by record_seal_quantised, the preview states the rounding in the terms of the view in front of it, because rounding moves bytes into one named class and therefore releases more or withholds more depending on whether this view grants that class. A preview says what would be disclosed; it does not say that the classification which put those bytes where they are was correct, and it says so in a does_not_say field. The disclose_* family then carries a posture out. disclose_to_passphrase(ledger, record, view, recipient) issues a grant -- the key material for exactly the segments the view selects, 56 bytes a segment, sealed under a passphrase asked for at the terminal -- and writes the disclosure into the forensic ledger before handing anything back; a disclosure the ledger could not record was not issued. The record a recipient receives is byte-identical to the one under custody: a disclosure is a key grant, never a re-encryption. disclose_bundle lays the package down the way case_bundle does, with the ledger's inclusion proof for the disclosure and a signed manifest, and returns the snapshot root to be sent by another route; disclose_verify(dir, root) is the recipient's side, and a null root is a finding, never a pass. record_open(path, {"grant": path}) reads a record under a grant alone, with no case and no case key. disclose_withdraw records a withdrawal and stops further grants -- it is not a revocation, nothing reaches a recipient's copy, and bytes_recoverable is false on every row this family writes -- while disclose_history and disclose_for_segment answer who was given what, listing withdrawn recipients because they still hold what they were given. disclose_reclassified records that a record was sealed again under different classes and reports which recipients hold bytes whose class changed, without saying which way: nothing in this tool orders classes. What a disclosure can and cannot promise is stated in DISCLOSURE_POLICY.md.
| Builtin | Platforms | Description |
|---|---|---|
disclose_bundle(ledger: INTEGER, disclosure: STRING, dir: STRING) -> (HASH, ERROR) |
all | Writes the package for one disclosure into a directory that must be empty or absent: the record byte for byte (re-hashed and compared with the digest taken when the grant was issued), the sealed grant, the ledger's inclusion proof for the Disclosure node, a report, a manifest sealed and signed over the digests of all four, and a SHA256SUMS last. The ledger must have been compacted since the disclosure, because the proof has to resolve against a snapshot. The snapshot root the proof resolves against is returned and is NOT written into the package: a recipient checking a proof against a root its author supplied would be checking nothing, so the root travels by another route. Returns (bundle, err). |
disclose_for_segment(ledger: INTEGER, record_uid: STRING, segment: INTEGER) -> (HASH, ERROR) |
all | Answers who was given one segment of a record: every disclosure of that record whose grant includes the segment, with the segment's byte range and class beside them. A withdrawn disclosure is listed with withdrawn: true rather than dropped, because its recipient still holds what they were given -- this is the question a withdrawal cannot answer and the reason it is a query. The segment is derived from what the ledger recorded about the record by the same derivation the record format uses. A record this ledger has never recorded a disclosure of is an answer, not an error: record_known is false and nobody holds any of it as far as this ledger knows. unguarded_redactions lists the redactions recorded with no role, by a build that may not have refused one of a disclosure record, that no ledger_redactions_review answers for: any of them could have taken a holder out of this answer. Returns (holders, err). |
disclose_history(ledger: INTEGER) -> (HASH, ERROR) |
all | Lists every disclosure this ledger records, oldest first, each joined to its withdrawal if it has one: who it was issued to, under which view, from which record, how many segments were granted and withheld, when and why it was withdrawn, and on whose authority (withdrawal_authorised_by, withdrawal_authority_basis -- not recorded on a withdrawal written before 2.6.0) and under which role (actor_role, withdrawal_role); the role the recipient was assigned (recipient_role) and the version of the record's redaction it was issued under (redaction_version, redaction_uid), both empty on a disclosure issued before 2.6.0. It reads only nodes this family wrote -- a node a script put into the same ledger carries a label a script cannot spell and is not in the history -- and foreign counts nodes that carry a disclosure uid without being a disclosure, because a history that skipped them silently would hide the one thing a reviewer should see. It is complete for this ledger and says nothing about any other. unguarded_redactions lists the redactions recorded with no role, by a build that may not have refused one of a disclosure record, that no ledger_redactions_review answers for: any of them could have taken a disclosure or a withdrawal out of the history. Returns (history, err). |
disclose_reclassified(ledger: INTEGER, record: INTEGER, superseded: INTEGER) -> (HASH, ERROR) |
all | Records that record reclassifies superseded -- the same evidence sealed again under different classes -- and reports every disclosure of the superseded record whose granted bytes changed class. Both records must be opened with the case key: each footer's sealed whole-plaintext digest is opened and the two compared, and records that do not decrypt to the same plaintext are refused, because a reclassification is the same bytes and nothing else. The change is written to the ledger once, as a ReclassEvent and a SUPERSEDES edge from the new record to the old; asking again reports without writing, and the superseded record can no longer be disclosed. The report never says a class was raised or lowered -- nothing in this tool orders classes -- it says which byte ranges moved from which class to which, which recipients hold them (withdrawn or not, since a withdrawal reaches nobody's copy), and whether each recipient's view grants the new class. now_withheld_bytes is what those recipients hold that the classification now in force would not give them. unguarded_redactions lists the redactions recorded with no role, by a build that may not have refused one of a disclosure record, that no ledger_redactions_review answers for: any of them could have taken a disclosure out of affected. Refused on a ledger other than the one the case is attached to, and on a ledger opened as auditor, before anything else is looked at: an auditor writes nothing into what it audits. The role is asserted, so this keeps the record consistent and is not access control. Returns (reclassification, err). |
disclose_to_passphrase(ledger: INTEGER, record: INTEGER, view: STRING, recipient: STRING) -> (HASH, ERROR) |
all | Issues a disclosure: the key material for exactly the segments of record whose class view grants, sealed under a passphrase asked for at the terminal, for a recipient the examiner names. The granted set is computed by the same partition view_preview reports, so the preview is the disclosure. The disclosure is written into the ledger -- a Disclosure node with its record, recipient, view and actor, in one signed transaction -- before anything is handed back, and if that write fails nothing was issued. The record the recipient will receive is byte-identical to the one under custody; a disclosure is a key grant and never a re-encryption. A record opened under a grant cannot issue one. A recipient whose earlier disclosure of this record was withdrawn is refused, because a withdrawal is how further grants are stopped, and so is a record another record has reclassified. So is every grant from a ledger holding redactions recorded with no role, by a build that may not have refused one of a disclosure record, until ledger_redactions_review answers for them: any of them could have removed the withdrawal or the reclassification these refusals read. A grant is issued against the recipient's role, deny by default: it is refused unless the ledger assigns the recipient a recipient role in force (role_assign), the case defines that role's bundle (role_define), and the bundle holds the view; the grant is still exactly the view. Every one of these is refused before any key material is derived or any passphrase asked for, and asked again under the ledger lock before the write. The disclosure names the version of the record's redaction under the view that it carries out (redaction_version): the one in force, or the next, written in the same transaction when what the view releases from this evidence has changed (see redaction_commit). bytes_recoverable is false, as it is of every disclosure: nothing after this call reaches the recipient's copy. The sealed grant is held by this run until disclose_bundle writes it. Refused on a ledger other than the one the case is attached to, and on a ledger opened as auditor, before anything else is looked at: an auditor writes nothing into what it audits. Roles are asserted, so these refusals keep the record consistent and are not access control. Returns (disclosure, err). |
disclose_verify(dir: STRING, root: STRING|NULL) -> (HASH, ERROR) |
all | Checks a disclosure package, as its recipient or anyone else: the manifest's seal and signature; every file's digest; that SHA256SUMS names exactly the manifest's files and the manifest itself, with every digest agreeing, and that the directory holds nothing but those and SHA256SUMS; the record's own signature, with no key at all; that the grant names this disclosure and this record and was issued against its header exactly as it stands; that the grant opens exactly the segments whose class the view names and the manifest's withheld list is every other segment -- complete as authorised; with the grant's passphrase, asked for at the terminal, that every granted segment decrypts with its tag holding; and, against root, that the ledger snapshot contains this disclosure property for property; and that what the ledger records is this package -- its recipient, examiner and view as the manifest states them, and the record and grant files themselves by digest, uid, runs and descriptors root -- so a package relabelled, or holding another disclosure's grant, and re-signed with any key, does not verify. manifest_public_key and record_public_key say which keys signed the manifest and the record: a signature holds over whatever its key signed, and means something only once that key is compared with one the recipient already trusts. root is required and may be null, and null is a finding, never a pass: a root read out of the package would be one its author chose. verified is true only when every check ran and passed. It establishes nothing about whether the classification was right, whether the record is all the evidence there is, or whether the names in it are true -- does_not_cover says so. Returns (verification, err). |
disclose_withdraw(ledger: INTEGER, disclosure: STRING, reason: STRING, options?: HASH) -> (HASH, ERROR) |
all | Records that a disclosure is withdrawn: a Withdrawal node, attributed and signed, pointing at the disclosure, with the reason the examiner gives. It is not a revocation and nothing in this family is named like one -- once the recipient holds the record and the grant, both are theirs and nothing reaches their copy, which bytes_recoverable: false says as a field. What a withdrawal does is stop further grants: disclose_to_passphrase refuses to issue this record to this recipient again, and disclose_bundle refuses to package the withdrawn disclosure. A disclosure is withdrawn once; a second withdrawal is refused and names the first. A withdrawal records on whose authority it was made: the examiner's own when authorised_by is left out or names them (authority_basis self), the named authority otherwise (named), as an AUTHORISED_BY edge to that name's Actor node beside the PERFORMED_BY edge, which carries the role the examiner acted under. Both names are asserted and neither is checked. Needs no open case -- the ledger is the record. Refused on a ledger opened as auditor, before anything else is looked at: an auditor writes nothing into what it audits. The role is asserted, so this keeps the record consistent and is not access control. Returns (withdrawal, err). |
redaction_commit(ledger: INTEGER, record: INTEGER, view: STRING, reason: STRING) -> (HASH, ERROR) |
all | Records how a view divides a record -- the plaintext byte ranges it releases and the ones it withholds -- as a version of that evidence's redaction, ahead of any disclosure under it, so the redaction can be reviewed before it is carried out. The division is the one view_preview reports and a disclosure issues. A view's versions of one piece of evidence are a hash-linked chain that follows it through its reclassifications: the line is found by walking the reclassifications the ledger records back from the record, to the nearest record whose redaction already has versions, or else to the evidence's first record. A version is written only when what the view releases changes: committing the redaction in force again writes nothing and answers already_committed: true, and a reclassification that releases the same bytes is the same redaction. disclose_to_passphrase issues under the version in force and writes the next itself when the redaction has changed. Refused for a record another record has reclassified, while the case is in review or disposed, on a ledger other than the one the case is attached to, and on a ledger opened as auditor. Returns (version, err). |
redaction_versions(ledger: INTEGER, record_uid: STRING) -> (HASH, ERROR) |
all | Lists the versions of a record's redaction, per view, in the line its evidence's versions are kept in, and every disclosure issued under each -- from the RedactionVersion nodes and the REDACTED_AS edge every disclosure carries. A disclosure issued under a version that is no longer the head of its chain is stale: its recipient holds a redaction this evidence is no longer disclosed under, and saying so, like a withdrawal, reaches nobody's copy. unversioned counts disclosures of the record issued before versions were recorded, which disclose_history lists. Needs no case open, so an auditor reads it with the ledger alone; every chain is checked as it is read, and one that forks or holds an event whose uid does not recompute is refused. A record the ledger knows nothing of is answered with record_known: false, not an error. unguarded_redactions lists the redactions recorded with no role, by a build that may not have refused one of a disclosure record, that no ledger_redactions_review answers for: any of them could have taken a disclosure or a reclassification out of this answer. Returns (versions, err). |
role_assign(ledger: INTEGER, recipient: STRING, role: STRING, reason: STRING) -> (HASH, ERROR) |
all | Records the recipient role a named recipient holds in the open case, or ends it with the role none, as the next event of that recipient's own assignment chain in the ledger -- hash-linked like an examiner's, and kept apart from them. A disclosure to them is issued against that role's bundle, and refused while they hold none. The recipient is named exactly as disclose_to_passphrase will name them: the name is fingerprinted, not normalised, so Counsel and counsel are two recipients. Needs the case key open, because a case is found in its ledger by the identity its key file carries; it does not need the case attached, and records in the ledger it is given, as a disclosure does -- but a case attached to a ledger is assigned there and nowhere else. An examiner-only role is refused, and so is an assignment that changes nothing. Anybody may assign anybody: who may is an organisation's policy and this program does not hold one, so the record says who assigned whom and no name in it is authenticated. bundled and grants say what the role's bundle holds in this run. Refused while the case is disposed, and on a ledger opened as auditor. Returns (assignment, err). |
role_define(role: STRING, views: ARRAY, options?: HASH) -> (HASH, ERROR) |
all | Declares which of the case's views a recipient role may be granted: the role's bundle. A disclosure is issued against the recipient's role, deny by default -- disclose_to_passphrase refuses unless the recipient holds a role in force, that role has a bundle, and the bundle holds the view -- and the grant is still exactly the view, so view_preview still says what it releases. Only a recipient role takes a bundle: legal, external_partner, restricted_viewer, and reviewer and auditor, who are on both sides. A bundle holding no view is legal and says so in grants_nothing, because "somebody decided nothing" and "nobody decided" are different answers to a disclosure review. A role's bundle is defined once in a run. Needs the case key open: a view names classes, and a class is tagged under the key. While the case is attached to a ledger, the bundle is written into the ledger before the run holds it (in_ledger) -- a RoleBundle node that BUNDLES each of its views -- and a later attach reads it back; defining again a bundle the ledger already holds is answered with already_defined: true when it holds the same views, and refused when it holds anything else. Refused while the attached case is in review or disposed. Returns (bundle, err). |
role_list(ledger?: INTEGER) -> (HASH, ERROR) |
all | Returns the recipient roles' bundles declared for this case, in declaration order, and -- given a ledger -- every recipient it assigns a recipient role, with the role in force and what that role's bundle holds in this run (bundled, grants). Answers with the same keys whether or not a case is open, so a program branches on a field rather than an error: open says a case is open and tagged that its key is. With a ledger and no keyed case open, it lists the recipients of every case in the ledger, so an auditor reads them with the ledger alone. Each assignment chain is checked as it is read, and one that forks, holds an event whose uid does not recompute, or mixes an examiner's assignments with a recipient's is refused. Returns (roles, err). |
view_define(label: STRING, classes: ARRAY, options?: HASH) -> (HASH, ERROR) |
all | Declares one named disclosure posture: the set of classifications a grant issued under this name will open, and nothing else. Declaring it once means a disclosure review asks what a name grants rather than reading the fourth argument of the sixth call in a script, and it puts the posture in the manifest as a row with a label on it. There is deliberately no negation and no "everything except": a view that granted all classes but one would widen by itself every time a class was declared after it, changing what it releases with nothing edited and nothing recorded. A view granting no class at all is legal and says so in grants_nothing -- it discloses the record, its signature and its shape, and no content. While the case is attached to a ledger with case_attach, the view is written into the ledger before the run holds it (in_ledger); a view a later attach read back is answered with already_defined: true when it grants the same classes, and refused when it grants anything else. Refused while the attached case is in review or disposed. Returns (view, err). |
view_list() -> (HASH, ERROR) |
all | Returns the disclosure postures declared for this case, in declaration order. Answers with the same keys whether or not a case is open, so a program branches on a field rather than on an error: open says a case is open, tagged says a case key is open, which is what makes a class tag and therefore a view possible at all. |
view_preview(record: INTEGER, view: STRING) -> (HASH, ERROR) |
all | Says what a view would disclose from a record, before anything is disclosed. It is arithmetic over the record's public header and the case's class table: it opens no segment, needs no plaintext, and reads_no_plaintext is a field saying so. granted_runs and withheld_runs are the byte ranges on each side, merged where adjacent segments of one class sit next to each other, and both are reported because "what am I holding back" is the question a disclosure review actually asks. Where the record was sealed by record_seal_quantised, rounding_effect states the direction in the terms of this view: rounding moves bytes into one named class, so the same rounding releases more when the view grants that class and withholds more when it does not, and a count printed without a direction is one a reviewer completes themselves. A class the record carries whose tag this case cannot name is counted in unnamed_classes rather than passed off as unclassified. Returns (preview, err). |
Registry Forensics (15)#
Windows registry across three sources via one polymorphic API (regf hive file, hive-JSON, or live Windows registry), plus Amcache and Shimcache execution evidence.
| Builtin | Platforms | Description |
|---|---|---|
amcache_parse(path: STRING) -> (HASH, ERROR) |
all | Parses an Amcache.hve hive (program execution/presence evidence) into {format, count, entries:[{key, path, name, sha1, publisher, version, product, size, last_write}]}. Supports the modern InventoryApplicationFile and legacy Root\File layouts. Returns (result, err). |
hive_close(handle: STRING) -> (BOOLEAN, ERROR) |
all | Closes a hive handle. Returns (bool, err). |
hive_get_value(handle: STRING, keypath: STRING, name: STRING) -> (HASH, ERROR) |
all | Returns {name, type, data} for a single value under keypath. Binary values (REG_BINARY, and types the reader does not recognise) also carry data_bytes, the stored bytes as a BYTES buffer; other types omit it, since data already holds the value faithfully. Returns (result, err). |
hive_key_info(handle: STRING, keypath?: STRING) -> (HASH, ERROR) |
all | Returns {name, last_write, last_write_iso, subkey_count, value_count} for a key (keypath is backslash-separated under the root; default root). Returns (result, err). |
hive_list_keys(handle: STRING, keypath?: STRING) -> ([]STRING, ERROR) |
all | Returns the subkey names under a key (default root) as an array. Returns (array, err). |
hive_list_values(handle: STRING, keypath?: STRING) -> ([]HASH, ERROR) |
all | Returns a key's values as [{name, type, data}] (REG_SZ/DWORD/QWORD/MULTI_SZ decoded; binary as hex). Binary values (REG_BINARY, and types the reader does not recognise) also carry data_bytes, the stored bytes as a BYTES buffer; other types omit it, since data already holds the value faithfully. Returns (array, err). |
hive_open(path: STRING) -> (HASH, ERROR) |
all | Opens a real Windows registry hive (regf binary format — SOFTWARE/SYSTEM/NTUSER.DAT, etc.) and returns {handle, path}. Distinct from the JSON-fixture reg_* family. Returns (result, err). |
reg_close(handle: STRING) -> (HASH, ERROR) |
all | Closes a registry-hive handle opened with reg_open. |
reg_deleted_keys(handle: STRING) -> (ARRAY, ERROR) |
all | Lists deleted-key entries. Populated only for the JSON source (its deleted_keys field); empty for real hive files and live registry (no unallocated-cell carving). |
reg_enum_keys(handle: STRING, keyPath?: STRING) -> ([]STRING, ERROR) |
all | Enumerates subkeys under a key. For hive/live sources keyPath is relative to the opened key (default root); for JSON it is the absolute path. Returns (array, err). |
reg_enum_values(handle: STRING, keyPath?: STRING) -> ([]HASH, ERROR) |
all | Enumerates a key's values as [{name, type, data}] (REG_SZ/DWORD/QWORD/MULTI_SZ decoded; binary as hex). Binary values (REG_BINARY, and types the reader does not recognise) also carry data_bytes, the stored bytes as a BYTES buffer; other types omit it, since data already holds the value faithfully. A JSON source never reports one, having no binary type to transcribe. Works across JSON/hive-file/live sources. Returns (array, err). |
reg_get_value(handle: STRING, keyPath: STRING, valueName: STRING) -> (HASH, ERROR) |
all | Reads a specific registry value with type metadata, across JSON/hive-file/live sources. Binary values (REG_BINARY, and types the reader does not recognise) also carry data_bytes, the stored bytes as a BYTES buffer; other types omit it, since data already holds the value faithfully. A JSON source never reports one, having no binary type to transcribe. A value that does not exist is an error, on every source including the live registry, and is told apart from a value that is present and empty. Returns (result, err). |
reg_open(source: STRING) -> (HASH, ERROR) |
all | Opens a registry data source (polymorphic) and returns {handle, path, source_type, status}. Dispatch: a regf hive file (SOFTWARE/SYSTEM/NTUSER.DAT, …) -> real hive parse; a hive-JSON file -> JSON; otherwise a live Windows registry path (e.g. HKLM\SOFTWARE...) -> live registry (Windows only). Returns (result, err). |
reg_timeline(handle: STRING) -> (ARRAY, ERROR) |
all | Returns timeline entries. Populated only for the JSON source (its timeline field); empty for real hive files and live registry. |
shimcache_parse(path: STRING) -> (HASH, ERROR) |
all | Decodes the Windows AppCompatCache (shimcache) — program execution/presence evidence. Accepts a SYSTEM hive file (locates the value) or a raw AppCompatCache blob. Reads three entry layouts and names the one it read in version: Win8 ('00ts' at a 0x80 header, with package data), Win8.1 ('10ts' at 0x80) and Win10/11 ('10ts' at 0x30 or 0x34). Returns {version, count, entries:[{position, path, last_modified, last_modified_iso}]}. Returns (result, err). |
Filesystem Forensics (115)#
Read-only filesystem parsers for NTFS/FAT/exFAT/ext/HFS+/XFS images and standalone $MFT timelines, with rich MAC timestamps and metadata.
| Builtin | Platforms | Description |
|---|---|---|
ext_capabilities(handle: STRING) -> (HASH, ERROR) |
all | Reports what ext can record, as distinct from what this volume happens to have recorded. Twenty-three capabilities, eleven of them read from this volume's superblock -- more than any other filesystem here, and a property of ext rather than of libext, because ext2, ext3 and ext4 are one on-disk format separated by feature flags. creation_times and sub_second_timestamps both follow the inode size, and that single difference is most of what separates an ext2 volume from an ext4 one; an answer cached from one volume is wrong about the next. journaled is true only when the journal is on this volume, so a volume using an external journal device reports false, meaning the journal exists but not here. deleted_entries_survive is not among the questions libext answers, so it comes back unanswered rather than false; ext_deleted is what reports on that. Every capability reports two bits, not one: a name appears in capabilities only where the library declares it, and the questions it does not answer are listed in unanswered rather than rendered false -- a missing field and a recorded no are different facts and only one of them is evidence. source says whether an answer is a constant of the format or was read from this volume, which is what decides whether it may be cached across volumes; volume_specific is that list. Nothing here describes the contents: a true access_times says the record has the field, not that any timestamp on this volume was maintained. Reads no disk. Returns (result, err). |
ext_close(handle: STRING) -> (HASH, ERROR) |
all | Closes an ext handle and releases its file. |
ext_deleted(handle: STRING) -> (HASH, ERROR) |
all | Reports deleted and orphaned inodes from every source libext consults: the inode table, whose slots keep mode, size, owner and times until reused; the legacy orphan list and the ext4 orphan file, which record inodes unlinked while still open and which a clean unmount would otherwise leave no trace of; and directory slack, which keeps the name the inode never held. ext4 zeroes the extent tree on unlink, so content_state is usually none -- the file is fully described and cannot be located -- and an entry that does carry a map reports it as image-absolute runs rather than raw block numbers, because a volume-relative block scaled to bytes looks exactly like an image offset. Where a map survives, reallocated is answered here rather than taken from libext's confidence, whose partial means either a block in use or a block bitmap that could not be read: an inode the bitmap still marks in use -- unlinked while open -- owns every block its map names, so it is not reallocated, and otherwise every block is asked about, with allocation_checked false when any question went unanswered. examined counts the inode-table slots read, and a slot past the end of the image makes the scan incomplete, because libext passes over a slot it cannot read without a warning. Entries on the legacy orphan chain carry no deletion time by design, because ext stores the next-orphan pointer in that field; an empty deleted_at there is not an absence of evidence. Block groups whose inode tables were never initialised are not scanned, since what is in them predates the filesystem. Warnings raised during this scan are reported; ones already on the handle are not. Returns (result, err). |
ext_dir_slack(handle: STRING, path: STRING) -> (HASH, ERROR) |
all | Recovers the directory records surviving in one directory's slack. Unlinking a file does not erase its record: the preceding record's rec_len is extended to swallow it, leaving the old record intact in the gap between where the live entry's name ends and where its rec_len now reaches. On ext4 this is frequently the only surviving evidence that a name existed at all, because unlink also zeroes the inode's extent tree. Each record is reported at two positions -- dir_offset within the directory's own data stream, which is all libext reports, and offset on the image, which this maps through the directory's data runs because nothing in libext does and a finding that cannot be re-read is not one. Every name is a candidate: the record is real, but the inode it names may since have been reused, so it has to be cross-checked against the inode table before it is called a recovered file. Records that duplicate a live entry are kept and flagged through shadows_live rather than dropped -- they are residue from the directory being rewritten and evidence of no deletion, and ext_deleted filters them out where this deliberately does not. Two limits libext does not report: a record whose inode field was cleared, which is the classic ext2 and ext3 unlink marker, is never recovered, and a block with an implausible record length is abandoned from that point on in silence. An inline directory keeps its records in the inode rather than in blocks and libext's scanner reads blocks only, so it is named as such rather than reported as empty. Returns (result, err). |
ext_extract_file(handle: STRING, path: STRING, dest: STRING) -> (HASH, ERROR) |
all | Streams a file out of an opened ext image and writes it to dest, never holding the whole of it in memory -- which is what makes a file larger than a mutant value, or larger than RAM, extractable at all. The SHA-256 of what was written is computed in the same pass, so nothing has to read the extracted copy back to obtain one. dest must not already exist: two files in an image can share a name, and evidence written over by accident is not recoverable. A copy that fails part way has its partial output removed, because a prefix left under the name of the whole file reads as the whole file. size is the length the volume records and located_bytes is how much of it the library could actually find. The two differ on a FAT or exFAT entry whose cluster chain was broken when the file was deleted: the entry still records the original length and only a prefix of it leads anywhere. When they differ truncated is true and what was written is the recoverable prefix rather than the file -- so read truncated before treating digest as the file's digest. Returns (result, err). |
ext_hash_file(handle: STRING, path: STRING, algorithm: STRING) -> (HASH, ERROR) |
all | Digests a file inside an opened ext image without writing a copy of it anywhere, reading it in fixed chunks so the peak allocation has nothing to do with the size of the file. This is how a file too large to read into a mutant value is still looked up in a hash set. md5 and sha1 are offered because published hash sets are keyed on them, not as security properties. size is the length the volume records and located_bytes is how much of it the library could actually find. The two differ on a FAT or exFAT entry whose cluster chain was broken when the file was deleted: the entry still records the original length and only a prefix of it leads anywhere. When they differ truncated is true and the digest covers only the recoverable prefix, so it will not match a hash set entry for the intact file -- which is the honest answer rather than a defect. Returns (result, err). |
ext_journal(handle: STRING) -> (HASH, ERROR) |
all | Reports the JBD2 journal's transactions, its own superblock and its feature bits. JBD2 journals whole blocks rather than operations, so each descriptor transaction names the filesystem blocks it carries copies of and each commit block carries the time it committed -- a transaction that never committed has no commit block and therefore no time, which has_timestamp says rather than a zero date. The journal is circular and libext reads it linearly from the first block to the last, parsing the superblock's Start and Sequence and then using neither, so where the log has been written round the transactions come back in physical order with stale pre-wrap ones interleaved among new ones; wrapped reports whether the sequence numbers go backwards anywhere in that order. Where it is true, committed must not be read as a finding: a transaction whose commit block sits at a lower physical block than its descriptor has the commit processed first, matched against nothing and discarded, so one that did commit is reported as one that never did, and this builtin raises commit_state_unreliable to say so. Revoke records are identified and not parsed -- they carry no tags and a block count of zero -- and a revoke is precisely the statement that a journalled copy must not be replayed. No checksum is verified anywhere in this journal. external separates a filesystem whose journal is on another device, which is fully journalled with the evidence elsewhere, from one with no journal at all; libext's own status string calls both of them the same thing. Returns (result, err). |
ext_journal_block_copies(handle: STRING, fs_block: INTEGER) -> (HASH, ERROR) |
all | Returns every journalled copy of one filesystem block, each being that block's contents at the moment a transaction was written, with its sha256 and its bytes. This is how a prior state of metadata the live filesystem has since overwritten is recovered -- a group descriptor, a bitmap, a directory block, an inode table block. libext documents the copies as newest first and that holds only while the journal has not been written round, so this walks the journal a second time purely to establish wrapped: a prior state quoted as the state immediately before an event is a claim about time, and where the seam is present the ordering it rests on is one the library declined to make. Copies are not checked against revoke records, which libext identifies and does not parse, so a copy here may be one the filesystem had already declared must not be replayed. A tag naming block zero yields a zero-filled buffer with no error, so an all-zero copy is as likely to be a block that was never read as a block that was zeroed. At most 256 copies are returned because each carries a whole filesystem block; entry_count keeps counting past that. Returns (result, err). |
ext_journal_inode_versions(handle: STRING, inode: INTEGER) -> (HASH, ERROR) |
all | Recovers prior on-disk states of one inode from journalled copies of the inode-table block that holds it. This is the one path in this package that can locate an ext4 file the filesystem itself can no longer locate: unlink zeroes the extent tree in the live inode and leaves everything else, which is why the usual deleted ext4 file comes back from ext_deleted fully described and entirely unfindable with content_state none -- and a journalled copy of the same block from before the unlink still carries the tree. Each version reports content_state in the same vocabulary the *_deleted family uses and runs as image-absolute byte ranges, so a version reporting preserved can be read with raw_read_at_bytes or written out with ext_recover_journalled_file. The ordering is libext's, newest first only while the journal has not been written round, so version 0 is the most recent surviving copy and not necessarily the state just before the deletion; wrapped is checked with a second journal walk for exactly that reason. deleted_at_raw is carried beside deleted_at because ext4 reuses the deletion-time field to hold the next inode number while an inode sits on the legacy orphan list, and that value read as a date is a timestamp somewhere in 1970. Returns (result, err). |
ext_list_files(handle: STRING, dir: STRING) -> ([]HASH, ERROR) |
all | Lists entries under a directory in an opened ext image, with created_at/modified_at/accessed_at/changed_at and deleted. |
ext_metadata(handle: STRING, path: STRING) -> (HASH, ERROR) |
all | Returns metadata for a path in an opened ext image, including created_at/modified_at/accessed_at/changed_at, deleted, and warnings (where the parser judged the answer may be incomplete). |
ext_open(image: STRING, offset?: INTEGER|HASH, length?: INTEGER) -> (HASH, ERROR) |
all | Opens an ext2/3/4 filesystem image and returns a handle. Returns (result, err). A volume that sits inside a larger image is opened where it lies rather than carved out first: pass the partition hash from table_list_partitions, or an explicit byte offset with an optional length. Every byte offset this handle goes on to report then already includes that start, so it addresses the image file itself and can be compared with a range from the table_ family without adding anything -- adding it twice, or comparing a partition-relative offset against a whole-disk one, mislocates evidence without failing. bounded is false when no length was given, which means reads can run past this volume's end into whatever follows it on the disk. |
ext_read_file(handle: STRING, path: STRING) -> (STRING, ERROR) |
all | Reads a file's bytes from an opened ext image. It allocates for the file's whole length before reading a byte of it, so it is bounded by what a library will read into memory: ext_extract_file streams the file to disk instead, ext_hash_file digests it in place, and ext_read_file_at returns one window of it. |
ext_read_file_at(handle: STRING, path: STRING, offset: INTEGER, length: INTEGER) -> (BYTES, ERROR) |
all | Reads one window of a file inside an opened ext image and returns it as a BYTES buffer, so the head of a file far larger than memory is still reachable. length is capped at 32 MiB, which is what the returned value has to fit into; use ext_extract_file for the whole of a larger file. An offset at or past the file's recorded size returns an empty buffer, which is how a caller walks off the end. An offset inside the recorded size but past the bytes the library could locate is an error naming both numbers: that is content the volume says exists and the image cannot produce, and a short buffer or a run of zeroes would hide it. The result is short rather than padded when the window runs past what was located. Returns (bytes, err). |
ext_read_file_bytes(handle: STRING, path: STRING) -> (BYTES, ERROR) |
all | Reads a file from an opened ext image as a BYTES buffer. It allocates for the file's whole length before reading a byte of it, so it is bounded by what a library will read into memory: ext_extract_file streams the file to disk instead, ext_hash_file digests it in place, and ext_read_file_at returns one window of it. |
ext_recover_file(handle: STRING, index: INTEGER, dest: STRING) -> (HASH, ERROR) |
all | Writes out the blocks a deleted ext inode still points at. ext4 zeroes the extent tree on unlink, so most entries report content_state none and are refused here rather than written as an empty file; the ones that survive are ext3-era inodes and files unlinked while still open. The runs come from libext's own DataRuns, because Extent.PhysicalBlock is volume-relative and only that call adds the base offset -- doing the multiplication anywhere else produces a partition-relative offset that looks exactly like an absolute one. confidence is libext's grade, whose partial means either a block in use or a block bitmap that could not be read; reallocated and allocation_checked are answered here instead -- the inode bitmap first, since an inode still in use, unlinked while open, owns its blocks, then every block -- and a caveat names the first block found in use. dest must not already exist: two deleted entries in one image can carry the same name, and evidence written over by accident is not recoverable. A write that fails part way has its partial output removed, because a prefix left under the name of the whole file reads as the whole file. bytes_written is split three ways and the split is the finding: located_bytes were read from the image, sparse_bytes are a hole the filesystem recorded and so are the file's own content, and unlocated_bytes are zeros standing in for ranges the library could not place -- never evidence that the file held zeros there. caveats states in prose what this recovery does not establish, and a report that quotes the digest without it is quoting a number out of its scope. Returns (result, err). |
ext_recover_journalled_file(handle: STRING, inode: INTEGER, version: INTEGER, dest: STRING) -> (HASH, ERROR) |
all | Writes out the blocks a journalled version of an inode points at -- the extent tree ext4's unlink zeroed, read back out of the journal and followed. It takes an inode number and a version index rather than a scan index, because unlike a deleted directory entry this subject has an identifier that survives: the inode number is the inode number, and the version list for it is derived the same way on every call over the same image. What the version index is not is a date, and the result echoes it for that reason -- where the journal has wrapped, libext's newest-first ordering is not newest first, which ext_journal_inode_versions reports. A version recording inline data is refused rather than written empty, because libext reads inline content only through a live inode; a version whose extent tree is already zeroed is refused with the note that an earlier version is where the tree would be. dest must not already exist, and a write that fails part way has its partial output removed. bytes_written splits into located_bytes read from the image, sparse_bytes where a recorded hole means the zeros are the content, and unlocated_bytes written only so later offsets land. The standing caveat is that the filesystem reallocated those blocks freely after the unlink and nothing here checks whether they still hold this file's content. Returns (result, err). |
ext_report(handle: STRING) -> (HASH, ERROR) |
all | Walks the whole inode table once -- not only what the directory tree reaches -- and returns libext's own versioned document, so an inode nothing names any more is listed with whatever path could be reconstructed for it. ext is the only one of the six formats here that records when a file was deleted, and that stamp is carried as deleted_at rather than folded into the other four; it is also the one timestamp that makes a row a deletion event in a timeline. Identity is the inode number with its generation, which is what tells a file replaced at the same path from one never touched. Fragments describe written data only: a preallocated span reads as zeros through the file interface and may still hold prior contents, which is ext_slack's question rather than this document's. Every end is exclusive, as on the other five formats, and a fragment's file_offset is where it sits in the file, read from the extent's logical block, so a run after a hole is placed after the hole; start_offset and end_offset bound the filesystem as its superblock sizes it. A row whose block map could not be read keeps its place with layout.error set, and makes the report incomplete; so does an inode-table slot past the end of the image, which libext passes over without a warning. The warnings libext raised during the walk are in warnings. On a volume whose inodes are 128 bytes there is no birth time and no sub-second fraction at all. One row per entry in the shape all six *_report builtins share: path, name, type, size, the deleted and fragmented bits, an identity and a parent identity, the four times that mean the same thing on every format, the layout the fragments were derived from, and image-absolute fragments whose offsets are -1 with located false wherever a run has no place in the image. Everything this format records that the others do not travels verbatim in each row's extra, so normalising loses nothing. identity_kind names the addressing and identity_stable says whether it survives the slot being reused, read from the same Capabilities *_capabilities reports and carrying identity_stable_answered because not every library declares it. completeness_checked says whether anything reconciled this listing against the filesystem's own record of what exists; completeness_proven means nothing without it. The library's own schema_version, library_version and generated stamp travel with the document. Reads the whole volume, so it costs O(volume) in time and holds its listing in memory rather than streaming it; the listing is capped at 50000 rows with every count beside it honest past the cap. events_from(report, "fs_report") turns it into a supertimeline. Returns (result, err). |
ext_slack(handle: STRING, path: STRING) -> (HASH, ERROR) |
all | Reports the blocks past the end of an ext file and the preallocated ranges inside it. libext exposes no file-slack API: Extents keeps the whole blocks an inode maps including preallocation past the end of the file, DataRuns trims the same map to the recorded size, and the slack is the difference -- so this is arithmetic on the library's own public output rather than a second copy of its block map. An extent flagged unwritten was allocated and never written to; the part of it inside the recorded size is reported as the unwritten class, and it reads back as zeros through the filesystem while the blocks themselves still hold what was there before. A file whose data is inline in its inode owns no blocks and reports zero of both. A sparse extent has no physical backing and so contributes no location. Directory-record slack -- the deleted names left in the gap a live record's rec_len was extended over -- is a different artifact with its own builtin, ext_dir_slack. Returns (result, err). |
ext_verify(handle: STRING) -> (HASH, ERROR) |
all | Reports ext's superblock plausibility check together with the volume's metadata CRC32c checksums, compared. The plausibility check is geometry and range sanity -- block and inode sizes, inodes per group, reserved percentage -- and not a checksum; a pass means nothing implausible was found, not that the superblock is unmodified. A volume made without RO_COMPAT_METADATA_CSUM reports the checksum check unavailable: it carries no CRCs, so a torn or edited structure on it is not detectable by checksum at all. On a volume that has them, the superblock, every group descriptor, the allocation bitmaps of every initialised group and every allocated inode are compared, so the call reads every allocated inode and costs O(inodes in use); an inode whose first 128 bytes are zero is accepted, as e2fsck accepts it, and metadata_readable fails when any of them could not be read. Extent tree blocks, directory blocks, extended attribute blocks and the journal are not compared, because libext compares no checksum on them, and a volume whose csum_seed feature stores a zero seed reports the check unavailable: libext would seed every comparison differently from the kernel. Every check reports checked beside passed, because a check that never ran says nothing about the volume and one boolean cannot tell "intact" from "nothing to compare against"; examined is the denominator it covered. Each check's kind is integrity -- a checksum or mechanism the format itself defines -- or structural: a plausibility test, a flag the volume keeps about itself, or whether a structure could be reached. verified is true only when at least one integrity check actually ran and every check that ran, of either kind, passed, so a volume with nothing checkable is never verified however many structural checks held -- integrity_checks_run is what tells that apart from a failure. scope states what the checks did not reach. Returns (result, err). |
ext_xattrs(handle: STRING, path: STRING) -> (HASH, ERROR) |
all | Lists one ext inode's extended attributes, read from both of the places ext keeps them. Reading only one is the trap here and libext's own documentation names it: GetXAttrs follows the inode's external attribute block and nothing else, and returns an empty list with no error for an inode that has no such block -- while security.selinux and system.posix_acl_access are small enough on a typical modern system never to need one, so a reader that follows only the block reports a file with SELinux labels and POSIX ACLs as having no attributes at all. Both storages are consulted here and every attribute says which it came from; a name appearing in both is kept twice and warned on, because which one the kernel would return is not recorded on disk. A value large enough that ext4 gives it its own inode is followed by libext and reported like any other, and a failure to follow one arrives as a library warning rather than as an empty value. Values read from the external block carry its position on the image; inline values have no block address and report -1. A volume without the ext_attr feature reports supported false, which is a statement about the filesystem rather than about this inode. Deleted inodes are not reachable: this addresses a live file by path. Returns (result, err). |
fat_capabilities(handle: STRING) -> (HASH, ERROR) |
all | Reports what FAT can record, as distinct from what this volume happens to have recorded. Twenty-four capabilities, three of them read from this volume's boot record and all three about what it can be checked or recovered against rather than what it records: a second allocation table to compare, an FSInfo sector to read hints from, and a backup boot sector that may be what let the volume be opened at all. stable_file_identity false is the consequential one -- a directory slot reused after a deletion carries its predecessor's identity exactly, so two readings of a FAT volume cannot be diffed on it. sub_second_timestamps is true of one timestamp only: creation is recorded to 10 milliseconds, modification to 2 seconds and last access to the day, so equal timestamps here are not evidence that nothing changed. Every capability reports two bits, not one: a name appears in capabilities only where the library declares it, and the questions it does not answer are listed in unanswered rather than rendered false -- a missing field and a recorded no are different facts and only one of them is evidence. source says whether an answer is a constant of the format or was read from this volume, which is what decides whether it may be cached across volumes; volume_specific is that list. Nothing here describes the contents: a true access_times says the record has the field, not that any timestamp on this volume was maintained. Reads no disk. Returns (result, err). |
fat_close(handle: STRING) -> (HASH, ERROR) |
all | Closes a FAT handle and releases its file. |
fat_deleted(handle: STRING) -> (HASH, ERROR) |
all | Reports FAT records marked deleted in a reachable directory, records surviving in the first cluster of a deleted directory, and records swept out of clusters nothing references. Content is located for one cluster and no further: deletion frees the FAT chain, so following it from a deleted entry follows whatever owns those clusters now, and the first cluster is the only location the surviving record itself states. Entries therefore report content_state first_cluster_only with located_bytes counting exactly that, against a size the record still claims -- the two are separate fields because a recorded size is not a count of the bytes that are there. Assuming the rest of the file followed contiguously is a hypothesis and is not taken here. Deletion also overwrites the first character of the short name; where it could not be brute-forced back out of the name checksum libfat substitutes an underscore, and those entries report name_source reconstructed. The sweep examines free clusters only and requires the dot records to agree with the cluster they were found in, so a directory fragment whose start was overwritten is not found. A record found both in a deleted directory and by the sweep is one record, listed once under the directory's path with the other sighting in its reasons. libfat has no warnings channel, so warnings_available is false and a silently skipped cluster leaves no trace. Returns (result, err). |
fat_extract_file(handle: STRING, path: STRING, dest: STRING) -> (HASH, ERROR) |
all | Streams a file out of an opened FAT image and writes it to dest, never holding the whole of it in memory -- which is what makes a file larger than a mutant value, or larger than RAM, extractable at all. The SHA-256 of what was written is computed in the same pass, so nothing has to read the extracted copy back to obtain one. dest must not already exist: two files in an image can share a name, and evidence written over by accident is not recoverable. A copy that fails part way has its partial output removed, because a prefix left under the name of the whole file reads as the whole file. size is the length the volume records and located_bytes is how much of it the library could actually find. The two differ on a FAT or exFAT entry whose cluster chain was broken when the file was deleted: the entry still records the original length and only a prefix of it leads anywhere. When they differ truncated is true and what was written is the recoverable prefix rather than the file -- so read truncated before treating digest as the file's digest. Returns (result, err). |
fat_hash_file(handle: STRING, path: STRING, algorithm: STRING) -> (HASH, ERROR) |
all | Digests a file inside an opened FAT image without writing a copy of it anywhere, reading it in fixed chunks so the peak allocation has nothing to do with the size of the file. This is how a file too large to read into a mutant value is still looked up in a hash set. md5 and sha1 are offered because published hash sets are keyed on them, not as security properties. size is the length the volume records and located_bytes is how much of it the library could actually find. The two differ on a FAT or exFAT entry whose cluster chain was broken when the file was deleted: the entry still records the original length and only a prefix of it leads anywhere. When they differ truncated is true and the digest covers only the recoverable prefix, so it will not match a hash set entry for the intact file -- which is the honest answer rather than a defect. Returns (result, err). |
fat_list_files(handle: STRING, dir: STRING) -> ([]HASH, ERROR) |
all | Lists entries under a directory in an opened FAT image. |
fat_metadata(handle: STRING, path: STRING) -> (HASH, ERROR) |
all | Returns metadata for a path in an opened FAT image. |
fat_open(image: STRING, offset?: INTEGER|HASH, length?: INTEGER) -> (HASH, ERROR) |
all | Opens a FAT filesystem image and returns a handle. Returns (result, err). A volume that sits inside a larger image is opened where it lies rather than carved out first: pass the partition hash from table_list_partitions, or an explicit byte offset with an optional length. Every byte offset this handle goes on to report then already includes that start, so it addresses the image file itself and can be compared with a range from the table_ family without adding anything -- adding it twice, or comparing a partition-relative offset against a whole-disk one, mislocates evidence without failing. bounded is false when no length was given, which means reads can run past this volume's end into whatever follows it on the disk. |
fat_read_file(handle: STRING, path: STRING) -> (STRING, ERROR) |
all | Reads a file's bytes from an opened FAT image. It allocates for the file's whole length before reading a byte of it, so it is bounded by what a library will read into memory: fat_extract_file streams the file to disk instead, fat_hash_file digests it in place, and fat_read_file_at returns one window of it. |
fat_read_file_at(handle: STRING, path: STRING, offset: INTEGER, length: INTEGER) -> (BYTES, ERROR) |
all | Reads one window of a file inside an opened FAT image and returns it as a BYTES buffer, so the head of a file far larger than memory is still reachable. length is capped at 32 MiB, which is what the returned value has to fit into; use fat_extract_file for the whole of a larger file. An offset at or past the file's recorded size returns an empty buffer, which is how a caller walks off the end. An offset inside the recorded size but past the bytes the library could locate is an error naming both numbers: that is content the volume says exists and the image cannot produce, and a short buffer or a run of zeroes would hide it. The result is short rather than padded when the window runs past what was located. Returns (bytes, err). |
fat_read_file_bytes(handle: STRING, path: STRING) -> (BYTES, ERROR) |
all | Reads a file from an opened FAT image as a BYTES buffer. It allocates for the file's whole length before reading a byte of it, so it is bounded by what a library will read into memory: fat_extract_file streams the file to disk instead, fat_hash_file digests it in place, and fat_read_file_at returns one window of it. |
fat_recover_file(handle: STRING, index: INTEGER, dest: STRING) -> (HASH, ERROR) |
all | Writes out what is locatable of an entry fat_deleted reported. Deletion frees the cluster chain, so for all but a resident-free contiguous case only the first cluster is known and the rest of the FAT now describes whatever was written to those clusters next. What gets written is therefore usually one cluster and complete is false; fat_recover_file_assuming_contiguous is the builtin that goes further, and it is a separate one because going further is a hypothesis. dest must not already exist: two deleted entries in one image can carry the same name, and evidence written over by accident is not recoverable. A write that fails part way has its partial output removed, because a prefix left under the name of the whole file reads as the whole file. bytes_written is split three ways and the split is the finding: located_bytes were read from the image, sparse_bytes are a hole the filesystem recorded and so are the file's own content, and unlocated_bytes are zeros standing in for ranges the library could not place -- never evidence that the file held zeros there. caveats states in prose what this recovery does not establish, and a report that quotes the digest without it is quoting a number out of its scope. Returns (result, err). |
fat_recover_file_assuming_contiguous(handle: STRING, index: INTEGER, dest: STRING) -> (HASH, ERROR) |
all | Recovers a deleted FAT entry as though the clusters after its first one followed it in order. libfat synthesises ceil(size/cluster) clusters from the first and sets its own Assumed flag; assumed reports that flag rather than the fact that this builtin was called, so an entry whose chain turned out to be walkable comes back assumed false and identical to fat_recover_file. When assumed is true nothing in the filesystem connects these bytes to this file beyond their position, and the assumption is unfalsifiable from inside the volume because the evidence that would test it is the chain deletion freed. dest must not already exist: two deleted entries in one image can carry the same name, and evidence written over by accident is not recoverable. A write that fails part way has its partial output removed, because a prefix left under the name of the whole file reads as the whole file. bytes_written is split three ways and the split is the finding: located_bytes were read from the image, sparse_bytes are a hole the filesystem recorded and so are the file's own content, and unlocated_bytes are zeros standing in for ranges the library could not place -- never evidence that the file held zeros there. caveats states in prose what this recovery does not establish, and a report that quotes the digest without it is quoting a number out of its scope. Returns (result, err). |
fat_report(handle: STRING) -> (HASH, ERROR) |
all | Walks the whole volume once and returns libfat's own versioned document: the reachable directory tree with deleted entries reported at the slots they occupy, and deleted directories descended into where their first cluster is still free and still begins with its own dot records. Nothing is assumed -- a deleted entry's FAT chain has been freed, so its layout past the first cluster is absent here rather than guessed, and fat_recover_file_assuming_contiguous is where that hypothesis is asked for by name. The unreferenced-cluster sweep is not run; fat_deleted is where records no directory reaches are looked for. FAT records no metadata-change time, so metadata_changed is empty on every row for a reason fat_capabilities states, and an identity here names a slot rather than a file. A disagreement between the volume's two allocation tables is raised as a warning, since it is a tamper and damage indicator rather than a reporting problem. One row per entry in the shape all six *_report builtins share: path, name, type, size, the deleted and fragmented bits, an identity and a parent identity, the four times that mean the same thing on every format, the layout the fragments were derived from, and image-absolute fragments whose offsets are -1 with located false wherever a run has no place in the image. Everything this format records that the others do not travels verbatim in each row's extra, so normalising loses nothing. identity_kind names the addressing and identity_stable says whether it survives the slot being reused, read from the same Capabilities *_capabilities reports and carrying identity_stable_answered because not every library declares it. completeness_checked says whether anything reconciled this listing against the filesystem's own record of what exists; completeness_proven means nothing without it. The library's own schema_version, library_version and generated stamp travel with the document. Reads the whole volume, so it costs O(volume) in time and holds its listing in memory rather than streaming it; the listing is capped at 50000 rows with every count beside it honest past the cap. events_from(report, "fs_report") turns it into a supertimeline. Returns (result, err). |
fat_slack(handle: STRING, path: STRING) -> (HASH, ERROR) |
all | Reports the unused tail of a FAT file's last cluster. FAT records no valid-data length, so the unwritten class is reported as unavailable on every file rather than as zero. This does not simply render libfat's SlackRange, because SlackRange returns the same empty result and false from five different situations and only one of them means the file ends on a cluster boundary: a directory, an empty file, a chain that could not be walked, a genuinely aligned file, and a tail that would fall past the end of the volume. The third is the one that matters, and a freed chain looks exactly like it, so the chain is walked here and file_slack_checked is false with file_slack_bytes at -1 when it broke, with a warning naming which way: a reallocated first cluster, a broken chain, a loop, or runs simply shorter than the recorded size. Note what this cannot reach: it addresses a live file by path, and a deleted entry cannot be opened by one, so the slack most worth having needs the scan index ext_deleted's siblings hand out and this does not take it. Directories are outside libfat's slack API altogether and are reported as unchecked rather than as empty, because directory cluster slack is where deleted short and long name records survive. Volume slack, the sectors past the last cluster, is covered by no libfat API. Returns (result, err). |
fat_verify(handle: STRING) -> (HASH, ERROR) |
all | Compares FAT's two allocation tables against each other, which is the only integrity signal the format has -- FAT carries no checksum anywhere. A volume with a single FAT reports the check unavailable rather than passed, because there is no mirror to compare. libfat counts disagreements as entries are read rather than scanning for them, so this walks the live directory tree first to give the comparison a denominator worth quoting; the data chains of files nothing read and clusters no directory references are not covered. The mismatch count is cumulative for the handle and rises again each time the same bad entry is re-read, so it bounds the number of disagreeing entries from above rather than counting them. Every check reports checked beside passed, because a check that never ran says nothing about the volume and one boolean cannot tell "intact" from "nothing to compare against"; examined is the denominator it covered. Each check's kind is integrity -- a checksum or mechanism the format itself defines -- or structural: a plausibility test, a flag the volume keeps about itself, or whether a structure could be reached. verified is true only when at least one integrity check actually ran and every check that ran, of either kind, passed, so a volume with nothing checkable is never verified however many structural checks held -- integrity_checks_run is what tells that apart from a failure. scope states what the checks did not reach. Returns (result, err). |
hfs_capabilities(handle: STRING) -> (HASH, ERROR) |
all | Reports what this HFS volume's kind can record. Nine capabilities, every one of them volume state, because "HFS" names three formats -- classic HFS, HFS+ and HFSX -- and libhfs decides which it opened before answering anything: classic HFS returns two answers and nothing else, HFS+ reads the attributes file and the journal bit off the volume header, and HFSX reads the catalog B-tree header to learn whether its names compare case-sensitively. This is the shortest set of the six: ten of the eighteen questions asked of every filesystem here go unanswered, creation and modification times among them. HFS has recorded both since 1985, so read that silence as a gap in what libhfs declares rather than as an absence on the volume -- hfs_metadata reports the dates themselves. Every capability reports two bits, not one: a name appears in capabilities only where the library declares it, and the questions it does not answer are listed in unanswered rather than rendered false -- a missing field and a recorded no are different facts and only one of them is evidence. source says whether an answer is a constant of the format or was read from this volume, which is what decides whether it may be cached across volumes; volume_specific is that list. Nothing here describes the contents: a true access_times says the record has the field, not that any timestamp on this volume was maintained. Reads no disk. Returns (result, err). |
hfs_close(handle: STRING) -> (HASH, ERROR) |
all | Closes an HFS+ handle and releases its file. |
hfs_deleted(handle: STRING) -> (HASH, ERROR) |
all | Reports HFS+ catalog records recovered from space the volume no longer treats as live: the free tail inside live B-tree nodes, nodes the tree marks free, and catalog nodes carved out of unallocated blocks. All three sources run. The third reads a large part of the image and libhfs leaves it off by default for that reason, but a scan that quietly skipped a source would report the same empty list as one that found nothing. Records still live in the catalog are excluded: a B-tree insert shifts records within a node and leaves the previous bytes behind it, so a live file record routinely appears in slack, and reporting one as a deletion would say a file was removed when it never was. The extent list on a recovered record is the eight inline descriptors only -- libhfs does not consult the extents overflow tree for a deleted record -- so a fragmented file reports located_bytes short of size, and that difference is map that was never read rather than content that is missing. entry_offset is image-absolute only for records carved from unallocated space; the two node-based sources measure from the node instead, and rather than hand back an offset that would seek to unrelated bytes those entries report -1. allocation_checked is true only when the allocation file answered for every block of a record's extents -- a block past the volume, or one it would not read, leaves it false -- and reallocated says one of them is in use. Damage libhfs meets during the walk is warned, including damage it already listed for an earlier call. Returns (result, err). |
hfs_extract_file(handle: STRING, path: STRING, dest: STRING) -> (HASH, ERROR) |
all | Streams a file out of an opened HFS+ image and writes it to dest, never holding the whole of it in memory -- which is what makes a file larger than a mutant value, or larger than RAM, extractable at all. The SHA-256 of what was written is computed in the same pass, so nothing has to read the extracted copy back to obtain one. dest must not already exist: two files in an image can share a name, and evidence written over by accident is not recoverable. A copy that fails part way has its partial output removed, because a prefix left under the name of the whole file reads as the whole file. size is the length the volume records and located_bytes is how much of it the library could actually find. The two differ on a FAT or exFAT entry whose cluster chain was broken when the file was deleted: the entry still records the original length and only a prefix of it leads anywhere. When they differ truncated is true and what was written is the recoverable prefix rather than the file -- so read truncated before treating digest as the file's digest. Returns (result, err). |
hfs_hash_file(handle: STRING, path: STRING, algorithm: STRING) -> (HASH, ERROR) |
all | Digests a file inside an opened HFS+ image without writing a copy of it anywhere, reading it in fixed chunks so the peak allocation has nothing to do with the size of the file. This is how a file too large to read into a mutant value is still looked up in a hash set. md5 and sha1 are offered because published hash sets are keyed on them, not as security properties. size is the length the volume records and located_bytes is how much of it the library could actually find. The two differ on a FAT or exFAT entry whose cluster chain was broken when the file was deleted: the entry still records the original length and only a prefix of it leads anywhere. When they differ truncated is true and the digest covers only the recoverable prefix, so it will not match a hash set entry for the intact file -- which is the honest answer rather than a defect. Returns (result, err). |
hfs_list_files(handle: STRING, dir: STRING) -> ([]HASH, ERROR) |
all | Lists entries under a directory in an opened HFS+ image. |
hfs_metadata(handle: STRING, path: STRING) -> (HASH, ERROR) |
all | Returns metadata for a path in an opened HFS+ image, including created_at/modified_at/accessed_at/changed_at/backup_at, time_source (HFS+ GMT vs classic-HFS local wall clock), compressed, compression_type and resource_fork_size. |
hfs_open(image: STRING, offset?: INTEGER|HASH, length?: INTEGER) -> (HASH, ERROR) |
all | Opens an HFS+ filesystem image and returns a handle. Returns (result, err). A volume that sits inside a larger image is opened where it lies rather than carved out first: pass the partition hash from table_list_partitions, or an explicit byte offset with an optional length. Every byte offset this handle goes on to report then already includes that start, so it addresses the image file itself and can be compared with a range from the table_ family without adding anything -- adding it twice, or comparing a partition-relative offset against a whole-disk one, mislocates evidence without failing. bounded is false when no length was given, which means reads can run past this volume's end into whatever follows it on the disk. |
hfs_read_file(handle: STRING, path: STRING) -> (STRING, ERROR) |
all | Reads a file's bytes from an opened HFS+ image. It allocates for the file's whole length before reading a byte of it, so it is bounded by what a library will read into memory: hfs_extract_file streams the file to disk instead, hfs_hash_file digests it in place, and hfs_read_file_at returns one window of it. |
hfs_read_file_at(handle: STRING, path: STRING, offset: INTEGER, length: INTEGER) -> (BYTES, ERROR) |
all | Reads one window of a file inside an opened HFS+ image and returns it as a BYTES buffer, so the head of a file far larger than memory is still reachable. length is capped at 32 MiB, which is what the returned value has to fit into; use hfs_extract_file for the whole of a larger file. An offset at or past the file's recorded size returns an empty buffer, which is how a caller walks off the end. An offset inside the recorded size but past the bytes the library could locate is an error naming both numbers: that is content the volume says exists and the image cannot produce, and a short buffer or a run of zeroes would hide it. The result is short rather than padded when the window runs past what was located. Returns (bytes, err). |
hfs_read_file_bytes(handle: STRING, path: STRING) -> (BYTES, ERROR) |
all | Reads a file from an opened HFS+ image as a BYTES buffer. It allocates for the file's whole length before reading a byte of it, so it is bounded by what a library will read into memory: hfs_extract_file streams the file to disk instead, hfs_hash_file digests it in place, and hfs_read_file_at returns one window of it. |
hfs_recover_file(handle: STRING, index: INTEGER, dest: STRING) -> (HASH, ERROR) |
all | Writes out the extents of a catalog record hfs_deleted carved out of node slack, a free node or unallocated space. Only the eight extents held in the record itself are used -- anything that overflowed into the extents B-tree is not among them -- and libhfs clamps a corrupt recorded size to what those extents can actually hold, so a short bytes_written against size is the normal shape of a partial record rather than an error. An entry with a block in use is still written, because refusing would deny an examiner data they may need, but reallocated is true and a caveat names the first block in use; the allocation file is asked about every block of the extents here, and allocation_checked is false when it could not answer for one. dest must not already exist: two deleted entries in one image can carry the same name, and evidence written over by accident is not recoverable. A write that fails part way has its partial output removed, because a prefix left under the name of the whole file reads as the whole file. bytes_written is split three ways and the split is the finding: located_bytes were read from the image, sparse_bytes are a hole the filesystem recorded and so are the file's own content, and unlocated_bytes are zeros standing in for ranges the library could not place -- never evidence that the file held zeros there. caveats states in prose what this recovery does not establish, and a report that quotes the digest without it is quoting a number out of its scope. Returns (result, err). |
hfs_report(handle: STRING) -> (HASH, ERROR) |
all | Walks the whole catalog once and returns libhfs's own versioned document: every record on the volume, the filesystem's own metadata included, with no bound on the listing -- a bound would make file_count the cap rather than the number of records the volume holds. This is the one of the six with no per-file layout: libhfs's report carries catalog records and paths and no extents, so fragments_available is false rather than every row claiming a file with no data. metadata_changed is the HFS+ attribute-modification date, which is the field that moves when a record's metadata changes and so is this format's ctime. On a classic HFS volume the stored times are local wall clock with no recorded offset and are read as UTC because there is nothing else to read them as -- each row's time_source says which kind it is, and comparing across the two compares different clocks. The volume's block counts are its header's own claim rather than a count of the allocation bitmap, and a disagreement between the two is itself a finding. One row per entry in the shape all six *_report builtins share: path, name, type, size, the deleted and fragmented bits, an identity and a parent identity, the four times that mean the same thing on every format, the layout the fragments were derived from, and image-absolute fragments whose offsets are -1 with located false wherever a run has no place in the image. Everything this format records that the others do not travels verbatim in each row's extra, so normalising loses nothing. identity_kind names the addressing and identity_stable says whether it survives the slot being reused, read from the same Capabilities *_capabilities reports and carrying identity_stable_answered because not every library declares it. completeness_checked says whether anything reconciled this listing against the filesystem's own record of what exists; completeness_proven means nothing without it. The library's own schema_version, library_version and generated stamp travel with the document. Reads the whole volume, so it costs O(volume) in time and holds its listing in memory rather than streaming it; the listing is capped at 50000 rows with every count beside it honest past the cap. events_from(report, "fs_report") turns it into a supertimeline. Returns (result, err). |
hfs_resource_fork(handle: STRING, path: STRING) -> (HASH, ERROR) |
all | Reports one HFS+ file's resource fork: its recorded size, and the image ranges holding it. This is the companion to hfs_slack's finding that a compressed file's data fork is empty -- it is where those bytes went. On a decmpfs-compressed file the payload is held either in this fork or, when it is small enough, inline in the com.apple.decmpfs attribute that hfs_xattrs reaches, and the two cases are distinguished rather than merged: holds_compressed_payload is true only when the file is compressed and this fork is not empty. The bytes are never decompressed on the way out, which is libhfs's rule and the right one for evidence -- the compressed payload as stored is the artifact. On an ordinary file the resource fork is the classic Macintosh second fork, carrying icons, and on older material frequently carrying the document itself. A fork whose extents describe more space than its recorded size reports the surplus as slack on the range holding it, because an attribute record carries no block total to trim against. Folders have no fork of either kind. Returns (result, err). |
hfs_slack(handle: STRING, path: STRING) -> (HASH, ERROR) |
all | Reports the slack of an HFS+ file's data fork, which libhfs carries on every extent rather than only the last -- the honest shape, because an over-allocated fork has whole extents that are slack from their first byte, and a final range holding no file data at all is a fact about the volume rather than an empty result. HFS+ records no valid-data length, so the unwritten class is unavailable on this format rather than zero. A decmpfs-compressed file's data fork is genuinely empty on disk, its payload living in the resource fork or an extended attribute, so it is reported as unchecked with a warning rather than as a file with no slack; the resource fork is not examined here. An HFS+ directory is a B-tree record rather than an allocation of blocks and so owns no tail. Free space belonging to no file is a different question with its own builtin, hfs_unallocated. Returns (result, err). |
hfs_unallocated(handle: STRING) -> (HASH, ERROR) |
all | Walks an HFS+ volume's allocation file and reports every maximal run of blocks belonging to no live file -- the input to carving, and the only free-space traversal available anywhere in these six filesystem libraries: the others offer a per-cluster or per-block query, a raw bitmap, or a count the volume merely claims. free_blocks is counted bit by bit and free_blocks_claimed is what the volume header records, and the two are reported separately because they are different kinds of statement: a mismatch means the volume was not unmounted cleanly or its metadata is inconsistent, so claim_matches is the field to read before anything carved out of this space is relied on. A free block is not a statement that anything was ever written there, nor that what is there now was deleted rather than never used. Blocks still claimed by a file that was deleted are not free and do not appear here, which is what hfs_deleted finds instead. A run whose start block does not resolve to an offset is reported at -1 rather than at zero, which is a real place on the volume. Returns (result, err). |
hfs_verify(handle: STRING) -> (HASH, ERROR) |
all | Reports what an HFS+ volume can be asked, which is less than the other five filesystems. The format records no checksum over metadata or data, so the checksum check is permanently unavailable and no tool can make it otherwise -- an integrity claim about an HFS+ volume has to rest on the image's own digests, which is what ewf_verify checks, rather than on the filesystem. What is checked here is structural: whether each b-tree the volume header names has a header that parses, since a header that does not means the tree beneath it is unreachable. Alongside that are the anomalies libhfs recorded while parsing on this handle, which accumulate as a side effect of whatever was read, so an empty list can mean little was read rather than that nothing was wrong. Every check reports checked beside passed, because a check that never ran says nothing about the volume and one boolean cannot tell "intact" from "nothing to compare against"; examined is the denominator it covered. Each check's kind is integrity -- a checksum or mechanism the format itself defines -- or structural: a plausibility test, a flag the volume keeps about itself, or whether a structure could be reached. verified is true only when at least one integrity check actually ran and every check that ran, of either kind, passed, so a volume with nothing checkable is never verified however many structural checks held -- integrity_checks_run is what tells that apart from a failure. scope states what the checks did not reach. Returns (result, err). |
hfs_xattrs(handle: STRING, path: STRING) -> (HASH, ERROR) |
all | Lists one HFS+ catalog node's extended attributes, inline or fork-backed, in B-tree key order. com.apple.quarantine, which records that a file was downloaded and by what, and com.apple.metadata:kMDItemWhereFroms, which records where from, both live here. The system attributes com.apple.decmpfs and com.apple.ResourceFork are returned like any other and flagged rather than filtered -- libhfs calls filtering them a policy decision belonging to the caller, and the presence of decmpfs is exactly why that file's data fork reads as empty everywhere else. A fork-backed value carries every range it occupies on the image in ranges, in the order they make up the value, and a single offset only when those ranges follow one another -- a fragmented value reports -1 there and a value_fragmented warning -- so a value too large to render is still reachable with raw_read_at_bytes, range by range; an inline value lives in the B-tree record itself, has no allocation-block address, and reports -1 rather than a zero that would read as the start of the image. Values are read while they fit the render cap and reported by length and ranges when they do not, so the size of an attribute never sets the size of an allocation here; a value that could not be read is a value_unreadable warning and makes the result incomplete. Damage libhfs meets while reading the node is warned and makes the result incomplete too, including damage it already listed for an earlier call, which it lists once per volume and still counts. Classic HFS has no attributes B-tree at all: supported distinguishes that from an HFS+ node that simply carries none. Returns (result, err). |
mft_parse(path: STRING) -> (HASH, ERROR) |
all | Parses an NTFS Master File Table into a per-record timeline. Auto-detects a standalone $MFT file (FILE-signature record stream, e.g. KAPE/FTK/icat) vs a full NTFS volume image. Each entry has $STANDARD_INFORMATION (si_*) and $FILE_NAME (fn_*) MAC times as unix seconds, a sub-second nanosecond fraction (si_*ns/fnns, 0-999999999, at NTFS 100 ns resolution — a whole-second/zero fraction is a timestomping tell), and an RFC3339Nano iso string; plus reconstructed path, size, sequence, and hard-link count. The path follows each $FILE_NAME parent reference -- a record number and that record's sequence number -- up to the root (record 5), and starts with ? where the chain breaks: at a parent other than the root that the table does not hold or that has no name, and at a reference whose parent's record has since been given to another file. NTFS reuses records and adds one to a record's sequence number each time it frees it, so the parent is the same directory when the two numbers agree, or when it is deleted and one past the reference. A walk up the chain also stops with ? once it has followed 257 directories without reaching the root, which bounds a cycle; the directories such a walk passed keep the cut-short prefix it found, so a path under one of them can be cut at the same place even when it is shallower. parent_sequence is the number the reference carries (0 when the record has no $FILE_NAME), parent_checked says it was compared with the record now at parent_record (it is not when the table holds no such record, or when the reference carries 0, NTFS's no-check value), and parent_reused says that record now holds a different file. size is the default $DATA stream's, and so is allocated_size when that stream is non-resident; the copies $FILE_NAME carries, which NTFS updates lazily and are routinely stale or zero, stand in only where the record does not hold the start of that stream. The record size is read from the first record header rather than assumed, and skipped counts records that would not parse. Returns {source_type, record_size, count, skipped, entries:[{record, parent_record, parent_sequence, parent_checked, parent_reused, in_use, is_directory, name, path, size, allocated_size, sequence, hard_links, file_attributes, si, si_*ns, fn, fn__ns}]}. Returns (result, err). |
ntfs_capabilities(handle: STRING) -> (HASH, ERROR) |
all | Reports what NTFS can record, as distinct from what this volume happens to have recorded. Twenty-two capabilities, twenty-one of them constants of the format and one -- change_journal -- read from this volume, because $UsnJrnl can be enabled, filled, deleted and re-created; libntfs cannot tell a journal that was never enabled from an $Extend directory too damaged to read, so a false there is not proof of absence. This is the fullest of the six: NTFS answers all eighteen questions asked of every filesystem here and adds security descriptors, alternate data streams, encryption and a change journal no other format in this tree has. Every capability reports two bits, not one: a name appears in capabilities only where the library declares it, and the questions it does not answer are listed in unanswered rather than rendered false -- a missing field and a recorded no are different facts and only one of them is evidence. source says whether an answer is a constant of the format or was read from this volume, which is what decides whether it may be cached across volumes; volume_specific is that list. Nothing here describes the contents: a true access_times says the record has the field, not that any timestamp on this volume was maintained. Reads no disk. Returns (result, err). |
ntfs_close(handle: STRING) -> (HASH, ERROR) |
all | Closes an NTFS handle and releases its file. |
ntfs_deleted(handle: STRING) -> (HASH, ERROR) |
all | Reports every MFT record whose in-use flag is clear, with the name, timestamps and data runs NTFS left in place. NTFS does not erase a record when a file is unlinked, so this is the one filesystem of the six whose deleted block map is neither zeroed nor guessed: entries report content_state preserved, or resident when the content was small enough to live inside the record itself. What it cannot report is reuse. libntfs exposes no cluster-allocation query, so allocation_checked is false on every entry and the reallocated bit beside it means nobody looked, not that nothing happened. Records that would not parse are counted in unreadable -- a number libntfs itself does not keep -- and complete goes false when there were any, though an unreadable record may equally be a slot never used, one marked BAAD, or one whose fixups failed because it was partly overwritten, and nothing in the library tells those apart. Only the records the image holds are read: the $MFT's run list declares how many there are and nothing bounds it, so records it places outside the image, or in no run, are counted rather than walked one by one, and complete goes false with incomplete_reason saying how many and from which entry. Paths are reconstructed from parent references, so a file whose parent directory is still live gets its full path back. Names surviving in a parent directory index slack after the record itself was reused are not swept here; ntfs_list_files reports those, marked deleted, per directory. Returns (result, err). |
ntfs_extract_file(handle: STRING, path: STRING, dest: STRING) -> (HASH, ERROR) |
all | Streams a file out of an opened NTFS image and writes it to dest, never holding the whole of it in memory -- which is what makes a file larger than a mutant value, or larger than RAM, extractable at all. The SHA-256 of what was written is computed in the same pass, so nothing has to read the extracted copy back to obtain one. dest must not already exist: two files in an image can share a name, and evidence written over by accident is not recoverable. A copy that fails part way has its partial output removed, because a prefix left under the name of the whole file reads as the whole file. size is the length the volume records and located_bytes is how much of it the library could actually find. The two differ on a FAT or exFAT entry whose cluster chain was broken when the file was deleted: the entry still records the original length and only a prefix of it leads anywhere. When they differ truncated is true and what was written is the recoverable prefix rather than the file -- so read truncated before treating digest as the file's digest. Returns (result, err). |
ntfs_extract_stream(handle: STRING, path: STRING, stream: STRING, dest: STRING) -> (HASH, ERROR) |
all | Streams one named $DATA stream out of an image onto local disk without ever holding it in memory, computing the SHA-256 of what was written in the same pass. This is the path for an alternate data stream carrying a payload rather than a label -- an executable parked behind a text file's name is the case the technique is known for, and it is reachable no other way, because the stream's bytes are not part of the file the path addresses. dest must not already exist and a copy that fails part way has its partial output removed, the same rule every builtin here follows when it produces a new piece of evidence out of an image. An empty stream name extracts the unnamed stream. What is written is bounded by the bytes libntfs could locate rather than by the length the record claims, and truncated says whether the two differ. Returns (result, err). |
ntfs_hash_file(handle: STRING, path: STRING, algorithm: STRING) -> (HASH, ERROR) |
all | Digests a file inside an opened NTFS image without writing a copy of it anywhere, reading it in fixed chunks so the peak allocation has nothing to do with the size of the file. This is how a file too large to read into a mutant value is still looked up in a hash set. md5 and sha1 are offered because published hash sets are keyed on them, not as security properties. size is the length the volume records and located_bytes is how much of it the library could actually find. The two differ on a FAT or exFAT entry whose cluster chain was broken when the file was deleted: the entry still records the original length and only a prefix of it leads anywhere. When they differ truncated is true and the digest covers only the recoverable prefix, so it will not match a hash set entry for the intact file -- which is the honest answer rather than a defect. A path with no unnamed $DATA attribute is refused rather than digested: it names no content of its own, and the bytes the library would otherwise serve belong to one of its named streams, so the digest would be that stream's published under the file's name with truncated false and located_bytes agreeing with size. ntfs_streams lists those streams and ntfs_read_stream reads them by name. Returns (result, err). |
ntfs_list_files(handle: STRING, dir: STRING) -> ([]HASH, ERROR) |
all | Lists entries under a directory in an opened NTFS image. size and allocated_size are read from each entry's $DATA attribute rather than from the copy the directory index keeps, because NTFS updates that copy lazily and it is routinely stale and routinely zero -- on a stock 1 GiB volume four live files hold 4,259,940 bytes between them while the index records 0 for every one, and $MFT records 16,384 against an actual 2,097,152. Resolving it costs one open per listed file, which is most of what the call costs. size_source says which number is in hand: data_attribute was read from the unnamed $DATA stream and is authoritative; no_data_attribute means the path addresses no $DATA at all, so its length is zero -- a directory, or a file holding its content only in named streams, which ntfs_streams lists and ntfs_read_stream reads; index_record is the index copy, used only when the $DATA could not be reached, which is the ordinary case for a deleted entry because it no longer opens by path, and it may be stale. |
ntfs_log_records(handle: STRING) -> (HASH, ERROR) |
all | Reports the $LogFile record stream: the metadata operations NTFS journals so it can replay or undo them after a crash, each with its LSN, its redo and undo operation names, and the clusters it touched. Records come back in the order their pages sit in the file, which is the order they were written only until the log was written round, so ordering is physical and wrapped reports whether the LSNs go backwards anywhere in it -- where they do, this array is not a timeline and ntfs_log_transactions, which is ordered by LSN, is the one to read. target_attribute is an offset into the open attribute table rather than an attribute: the table is dumped into the log periodically, and where a dump survives target_attribute_type, target_attribute_name and target_record resolve the operation to the stream and the MFT record it touched. Where the log has been written round past the last dump the table comes back empty rather than as an error, attribute_table_entries is zero, and every record's target stays unresolved. $LogFile carries no wall-clock time anywhere, so an ordering by LSN is the only chronology it can support. restart.open_count is a rough count of the times the volume has been mounted. libntfs drops a page whose update-sequence fixups fail whole, along with the partial record carried into it -- and a fixup failure is a torn write, the page most worth seeing -- so the pages are walked again here by libntfs's own rules: each such page is counted in a log_pages_torn warning and makes the result incomplete, and so does a $LogFile that cannot be read to its end. An LSN names one record, so a record met twice is a copy and is reported once, and a walk holding copies does not judge wrapped: wrap_checked is false. Returns (result, err). |
ntfs_log_transactions(handle: STRING) -> (HASH, ERROR) |
all | Groups the $LogFile records into the transactions they belong to, ordered by the LSN each began at. TransactionID is a slot in NTFS's small transaction table and not an identifier -- the same ID belongs to thousands of unrelated transactions over the life of a log -- so a transaction is closed here when its end record appears and the next record carrying that ID starts a new one, which is what keeps this from collapsing a whole log into a handful of pseudo-transactions. committed and forgotten are two bits and neither is the negation of the other: NTFS ends almost every transaction with ForgetTransaction rather than CommitTransaction, so a log holding tens of thousands of records may contain no commit record at all and committed false across a whole volume is the normal reading rather than evidence that nothing completed. end_state names which of the two was seen. start_present reports whether a transaction's earliest surviving record is one that names no previous record of its own -- where it is false the beginning was overwritten by the wrap and first_lsn is merely the oldest surviving part, which the grouping otherwise presents as the beginning. The records are grouped in LSN order, so a transaction whose records fell on both sides of the wrap is one transaction, and start_present is read from its lowest-LSN record. Pages libntfs drops and records met twice are reported as ntfs_log_records reports them. The log carries no wall-clock time, so this is an ordering and not a timeline. Returns (result, err). |
ntfs_metadata(handle: STRING, path: STRING) -> (HASH, ERROR) |
all | Returns metadata for a file/directory in an opened NTFS image, including the $STANDARD_INFORMATION created_at/modified_at/accessed_at/changed_at times and the readability flags (resident, sparse, compressed, encrypted, blocking_error). A path with no unnamed $DATA attribute names no content of its own, so size is 0, has_data false and the stream flags clear for it, and blocking_error names the alternate streams that hold its bytes -- without that every one of those fields would instead describe whichever named stream the library selected, which on a stock volume makes $Secure report the 264472 bytes of its $SDS stream as the file's own. The four times belong to the entry rather than to any stream and are returned either way. |
ntfs_open(image: STRING, offset?: INTEGER|HASH, length?: INTEGER) -> (HASH, ERROR) |
all | Opens an NTFS filesystem image and returns a handle. Returns (result, err). A volume that sits inside a larger image is opened where it lies rather than carved out first: pass the partition hash from table_list_partitions, or an explicit byte offset with an optional length. Every byte offset this handle goes on to report then already includes that start, so it addresses the image file itself and can be compared with a range from the table_ family without adding anything -- adding it twice, or comparing a partition-relative offset against a whole-disk one, mislocates evidence without failing. bounded is false when no length was given, which means reads can run past this volume's end into whatever follows it on the disk. |
ntfs_read_file(handle: STRING, path: STRING) -> (STRING, ERROR) |
all | Reads a file's bytes from an opened NTFS image. It allocates for the file's whole length before reading a byte of it, so it is bounded by what a library will read into memory: ntfs_extract_file streams the file to disk instead, ntfs_hash_file digests it in place, and ntfs_read_file_at returns one window of it. A path with no unnamed $DATA attribute is refused rather than read: it names no content of its own, and the bytes the library would otherwise serve belong to one of its named streams. ntfs_streams lists those streams and ntfs_read_stream reads them by name. |
ntfs_read_file_at(handle: STRING, path: STRING, offset: INTEGER, length: INTEGER) -> (BYTES, ERROR) |
all | Reads one window of a file inside an opened NTFS image and returns it as a BYTES buffer, so the head of a file far larger than memory is still reachable. length is capped at 32 MiB, which is what the returned value has to fit into; use ntfs_extract_file for the whole of a larger file. An offset at or past the file's recorded size returns an empty buffer, which is how a caller walks off the end. An offset inside the recorded size but past the bytes the library could locate is an error naming both numbers: that is content the volume says exists and the image cannot produce, and a short buffer or a run of zeroes would hide it. The result is short rather than padded when the window runs past what was located. Returns (bytes, err). |
ntfs_read_file_bytes(handle: STRING, path: STRING) -> (BYTES, ERROR) |
all | Reads a file from an opened NTFS image as a BYTES buffer. It allocates for the file's whole length before reading a byte of it, so it is bounded by what a library will read into memory: ntfs_extract_file streams the file to disk instead, ntfs_hash_file digests it in place, and ntfs_read_file_at returns one window of it. |
ntfs_read_stream(handle: STRING, path: STRING, stream: STRING, offset: INTEGER, length: INTEGER) -> (BYTES, ERROR) |
all | Reads one window of a named $DATA stream and returns it as a BYTES buffer. An alternate data stream is usually small and usually the evidence: the Zone.Identifier stream on a downloaded file records the URL it came from and the security zone Windows assigned it, and no amount of reading the file itself will produce either. An empty stream name selects the unnamed stream, which makes this a superset of ntfs_read_file_at; names are matched case-insensitively, as NTFS matches them, and a named stream on a directory opens like any other because a named stream holds content whatever its host entry is. An offset at or past the stream's recorded size returns an empty buffer, which is how a caller walks off the end; an offset inside the recorded size but past what the library could locate is an error naming both numbers rather than a run of zeroes. length is capped at 32 MiB; a stream larger than that is what ntfs_extract_stream is for. Returns (bytes, err). |
ntfs_recover_file(handle: STRING, index: INTEGER, dest: STRING) -> (HASH, ERROR) |
all | Writes out the content of an entry ntfs_deleted reported. NTFS leaves the run list of a deleted file in place, so this is the one filesystem of the six whose recovery reads the map the filesystem itself wrote rather than a remnant or a guess. A resident value is taken from the parsed MFT record and never re-read from the image: an MFT record carries update-sequence fixups, so the last two bytes of each sector hold the record's sequence number and a resident value crossing a sector boundary read off the disk is quietly two bytes wrong. A compressed or EFS-encrypted $DATA attribute is refused rather than written, because its runs describe compression units or ciphertext and not the file's bytes. allocation_checked comes back false on every entry -- libntfs has no cluster-allocation query -- so reallocated false here means nobody looked. dest must not already exist: two deleted entries in one image can carry the same name, and evidence written over by accident is not recoverable. A write that fails part way has its partial output removed, because a prefix left under the name of the whole file reads as the whole file. bytes_written is split three ways and the split is the finding: located_bytes were read from the image, sparse_bytes are a hole the filesystem recorded and so are the file's own content, and unlocated_bytes are zeros standing in for ranges the library could not place -- never evidence that the file held zeros there. caveats states in prose what this recovery does not establish, and a report that quotes the digest without it is quoting a number out of its scope. Returns (result, err). |
ntfs_reparse(handle: STRING, path: STRING) -> (HASH, ERROR) |
all | Reports one entry's reparse point: the tag that says who owns it, the target where the tag names one, and the tag-specific bytes where it does not. A target is decoded for the three tags that carry one -- symbolic links, mount points, and WSL symlinks, each with a different layout -- and for every other tag, which covers deduplication, cloud placeholders, WOF-compressed files and anything shipped next, the bytes are handed back as hex rather than guessed at. A file with no reparse point is reported as having none rather than as one whose target could not be read: is_reparse_point tells the two apart, and they are different findings. The link is never followed, which is libntfs's behaviour and the right one here -- a directory carrying a reparse point lists its own index, usually empty, rather than the target's, and whether the target exists is not checked because on an image of one volume it frequently cannot be. A third-party tag carries an owner GUID, which is reported, and a data layout defined by whoever wrote the filter driver, which is not interpreted. Returns (result, err). |
ntfs_report(handle: STRING) -> (HASH, ERROR) |
all | Walks the whole volume once and returns libntfs's own versioned document: every user-space MFT record, deleted and orphaned entries included, with paths reconstructed from $FILE_NAME parent references and each record's data runs as image-absolute offsets. When the image does not hold the whole MFT -- its run list places records outside the image, or in no run -- no report is built and the call says why: libntfs builds the document by walking every declared record and that walk cannot be bounded from here, so a truncated image was reported complete with records silently missing and a crafted run length kept the call busy for hours; ntfs_deleted and ntfs_verify walk only the records the image holds. The twelve reserved metadata records are not listed -- $MFT, $LogFile and $Bitmap are absent by design rather than missing. Identity is the MFT entry number with its sequence number, which is the one identity of the six that is both stable and carries a reuse counter, so two readings of a volume can be diffed on something sturdier than a path. Times are the $STANDARD_INFORMATION set, which is the set an anti-forensic tool rewrites; the $FILE_NAME copy most tools leave alone is not in this document, and ntfs_metadata is where the two are compared. A resident stream's bytes live inside the MFT record rather than at any cluster, so its run reports located false. One row per entry in the shape all six *_report builtins share: path, name, type, size, the deleted and fragmented bits, an identity and a parent identity, the four times that mean the same thing on every format, the layout the fragments were derived from, and image-absolute fragments whose offsets are -1 with located false wherever a run has no place in the image. Everything this format records that the others do not travels verbatim in each row's extra, so normalising loses nothing. identity_kind names the addressing and identity_stable says whether it survives the slot being reused, read from the same Capabilities *_capabilities reports and carrying identity_stable_answered because not every library declares it. completeness_checked says whether anything reconciled this listing against the filesystem's own record of what exists; completeness_proven means nothing without it. The library's own schema_version, library_version and generated stamp travel with the document. Reads the whole volume, so it costs O(volume) in time and holds its listing in memory rather than streaming it; the listing is capped at 50000 rows with every count beside it honest past the cap. events_from(report, "fs_report") turns it into a supertimeline. Returns (result, err). |
ntfs_security(handle: STRING, path: STRING) -> (HASH, ERROR) |
all | Reports the security descriptor governing one NTFS file: its owner, its group, and the entries of its DACL and SACL as they are stored on the volume. It is a reading, not an access decision, and the difference is the reason dacl_present is a field of its own. A descriptor carrying no DACL at all grants everyone full access; a descriptor whose DACL is present and holds no entries denies everyone. Rendered as an empty array those two are the same value and mean opposite things, so both are reported and both raise a warning naming which case it is. No field here answers whether a particular account could open the file: that needs group memberships, privilege assignments and an inheritance walk which a disk image does not contain. source says where the descriptor came from, because the provenance differs -- one stored on the entry itself belongs to this file alone, while one resolved through the entry's security ID is shared with every other file carrying that ID, and a finding about permissions on a single file has to say which it is. SIDs are rendered in S-R-I-S notation and named only where the SID is a well-known one; a domain account SID names nobody without that domain's directory. Returns (result, err). |
ntfs_security_descriptors(handle: STRING) -> (HASH, ERROR) |
all | Reports every security descriptor in the volume's $Secure:$SDS stream, in ascending security ID order -- the whole vocabulary of permissions the volume uses, read once rather than a lookup per file. This is the question worth asking across an image rather than of one file: which descriptors on this volume grant what, and which security IDs carry no DACL at all and so grant everyone full access, each of which is warned on as it is indexed. Each entry carries the SHA-256 of the descriptor bytes as stored, so the same descriptor can be recognised across images without comparing ACE by ACE. It says which descriptors exist, not which files use them: that join runs through the security ID in each entry's $STANDARD_INFORMATION, which ntfs_security reports per file. A descriptor stored directly on an entry, as older volumes and some system files do, is not in $SDS and so is not here. $SDS is append-only and libntfs indexes what it holds, so a descriptor no file still references is reported like any other rather than distinguished as stale. Returns (result, err). |
ntfs_slack(handle: STRING, path: STRING) -> (HASH, ERROR) |
all | Reports the bytes in an NTFS file's allocation that the file never put there, in two classes that are not the same thing. file_slack is the space between the end of the stream and the end of its last cluster -- the classic slack, and where the cluster's previous occupant survives. unwritten is the space between InitializedSize and RealSize: inside the length the volume records, allocated, readable, and never written by this file, so the filesystem hands back zeros while the clusters themselves still hold what was there before. Every range carries its class, because a valid-data-length gap quoted as slack in a report is a false statement about where the bytes came from. libntfs has no slack API of any kind, so both figures are computed from the non-resident $DATA attribute's AllocatedSize, RealSize and InitializedSize and located through its run list -- including the whole clusters past the end of the stream, which Fragments drops and a file truncated in place still owns. It is refused rather than guessed on a compressed stream, whose runs map compression units rather than stream bytes, and on a sparse one, whose allocated size is smaller than its recorded size so their difference is not a tail. A resident file's content lives inside the MFT record and owns no cluster, so it reports zero; the record's own unused space is record slack, a different artifact, as are alternate data streams. Returns (result, err). |
ntfs_streams(handle: STRING, path: STRING) -> (HASH, ERROR) |
all | Lists the $DATA streams of one NTFS file or directory: the unnamed stream the path addresses, and every alternate data stream beside it that the path cannot name and a directory listing does not show. An alternate data stream is the oldest hiding place on NTFS and still a working one -- its bytes are addressed only as file:stream, they are not counted in the file's size, and a directory, which has no content of its own, can carry one too, which is warned on because nothing that walks a volume by path will ever reach it. Each stream reports both of its lengths, whether it is resident in the MFT record, and whether libntfs can read it at all -- that last is known before the first byte is asked for, so an encrypted or otherwise blocked stream is named here rather than discovered part-way through an extraction that has already created its destination file. Reading a stream is ntfs_read_stream or ntfs_extract_stream. $EA and $EA_INFORMATION, the extended attributes NTFS inherited from OS/2 and that WSL uses to store its own metadata, are a different attribute type and are not listed here. Returns (result, err). |
ntfs_usn_journal(handle: STRING) -> (HASH, ERROR) |
all | Reports the USN change journal, the richest timeline artifact on an NTFS volume: every record carries a wall-clock timestamp, the changed file's MFT reference and sequence number, its parent's, a decoded reason bitmask and, on version 2 and 3 records, the filename. $J is a sparse stream appended to and trimmed from the front rather than overwritten in place, so stream order is the order the changes happened in and wrapped is false as a property of the format rather than as a measurement; lowest_position and highest_position bound the surviving window in USN, which is itself a byte offset into the stream. Version 4 records track extents and carry neither a timestamp nor a name, so they report has_timestamp and has_name false instead of a year-1 date and an empty string that would read as a file with no name. What this cannot see is a journal deleted and recreated -- a classic anti-forensic action -- because the $Max stream holding the journal's identifier, maximum size and lowest valid USN is one libntfs never reads, so a wiped journal looks like a volume with a short history. A record that will not parse is skipped in silence and counted nowhere, and a read error partway through ends the walk with no error at all: warnings_available is false because libntfs has no channel to report either. present distinguishes a volume with no change journal from a journal with no records. Returns (result, err). |
ntfs_verify(handle: STRING) -> (HASH, ERROR) |
all | Checks every MFT record against its update sequence array -- the per-sector mark NTFS writes so that a torn write can be detected. This is the only integrity check NTFS defines: the format carries no checksum over metadata or data, and libntfs exposes no other verifier, so nothing else about the volume can be verified from the volume itself. Records that are unallocated or that the volume marked bad are skipped rather than failed, because a whole-MFT walk meets them constantly and counting them as damage would report every healthy volume as corrupt. Reads every MFT record the image holds, so it costs O(MFT size) rather than a seek -- and no more: the $MFT's own run list declares how many records there are and nothing bounds it, so a record it places outside the image or in no run, one whose read fails and one whose attributes will not parse fail the structural mft_readable check and are named in the findings, rather than being skipped in silence. Every check reports checked beside passed, because a check that never ran says nothing about the volume and one boolean cannot tell "intact" from "nothing to compare against"; examined is the denominator it covered. Each check's kind is integrity -- a checksum or mechanism the format itself defines -- or structural: a plausibility test, a flag the volume keeps about itself, or whether a structure could be reached. verified is true only when at least one integrity check actually ran and every check that ran, of either kind, passed, so a volume with nothing checkable is never verified however many structural checks held -- integrity_checks_run is what tells that apart from a failure. scope states what the checks did not reach. Returns (result, err). |
xfat_capabilities(handle: STRING) -> (HASH, ERROR) |
all | Reports what exFAT can record, as distinct from what this volume happens to have recorded. Twenty-two capabilities, one -- second_fat -- read from this volume's boot record. exFAT is the sibling of FAT that gained an allocation bitmap independent of the table, a valid data length distinguishing written bytes from allocated ones, and a NoFatChain flag declaring contiguity, and that gained no metadata-change time, no owner and no identity surviving reuse. timezone_offsets is the one timestamp question it answers differently from FAT: exFAT stores the offset beside the stamp, so a reading can be anchored to UTC. Every capability reports two bits, not one: a name appears in capabilities only where the library declares it, and the questions it does not answer are listed in unanswered rather than rendered false -- a missing field and a recorded no are different facts and only one of them is evidence. source says whether an answer is a constant of the format or was read from this volume, which is what decides whether it may be cached across volumes; volume_specific is that list. Nothing here describes the contents: a true access_times says the record has the field, not that any timestamp on this volume was maintained. Reads no disk. Returns (result, err). |
xfat_close(handle: STRING) -> (HASH, ERROR) |
all | Closes an exFAT handle and releases its file. |
xfat_deleted(handle: STRING) -> (HASH, ERROR) |
all | Reports exFAT records marked deleted in a reachable directory, records surviving inside a deleted directory, and entry sets carved out of unallocated clusters. exFAT is the one format here that can state a deleted file whole layout as a fact: a stream extension carrying NoFatChain means the volume declared the run contiguous while the file was live and the FAT entries for it were undefined even then, so freeing the chain took nothing away, and those entries report content_state declared_contiguous rather than a guess. Everything else reports first_cluster_only. The carve accepts an entry set only when every secondary record agrees with its primary on allocation state and a stream extension was seen, so a half-overwritten set is dropped rather than reported half-read. A carved entry whose name did not survive is given a placeholder derived from its cluster number and reports name_source synthetic -- it is not a filename and must not be written into a report as one. allocation_checked comes from the allocation bitmap, asked about the entry's first cluster: in use, the entry was reallocated whatever the length of its run; free, that settles a one-cluster run only, and a declared-contiguous run is reported unchecked because only its first cluster was asked about -- xfat_recover_file asks about every cluster of the run. A record found both in a deleted directory and by the carve is one record, listed once under the directory's path with the other sighting in its reasons, and the carve's own /$OrphanFiles directory is not a record and is not listed. Returns (result, err). |
xfat_extract_file(handle: STRING, path: STRING, dest: STRING) -> (HASH, ERROR) |
all | Streams a file out of an opened exFAT image and writes it to dest, never holding the whole of it in memory -- which is what makes a file larger than a mutant value, or larger than RAM, extractable at all. The SHA-256 of what was written is computed in the same pass, so nothing has to read the extracted copy back to obtain one. dest must not already exist: two files in an image can share a name, and evidence written over by accident is not recoverable. A copy that fails part way has its partial output removed, because a prefix left under the name of the whole file reads as the whole file. size is the length the volume records and located_bytes is how much of it the library could actually find. The two differ on a FAT or exFAT entry whose cluster chain was broken when the file was deleted: the entry still records the original length and only a prefix of it leads anywhere. When they differ truncated is true and what was written is the recoverable prefix rather than the file -- so read truncated before treating digest as the file's digest. Returns (result, err). |
xfat_hash_file(handle: STRING, path: STRING, algorithm: STRING) -> (HASH, ERROR) |
all | Digests a file inside an opened exFAT image without writing a copy of it anywhere, reading it in fixed chunks so the peak allocation has nothing to do with the size of the file. This is how a file too large to read into a mutant value is still looked up in a hash set. md5 and sha1 are offered because published hash sets are keyed on them, not as security properties. size is the length the volume records and located_bytes is how much of it the library could actually find. The two differ on a FAT or exFAT entry whose cluster chain was broken when the file was deleted: the entry still records the original length and only a prefix of it leads anywhere. When they differ truncated is true and the digest covers only the recoverable prefix, so it will not match a hash set entry for the intact file -- which is the honest answer rather than a defect. Returns (result, err). |
xfat_list_files(handle: STRING, dir: STRING) -> ([]HASH, ERROR) |
all | Lists entries under a directory in an opened exFAT image, with created_at/modified_at/accessed_at, per-timestamp *_utc_offset_valid flags, attributes and valid_data_size. A nameless entry is reported as "(unnamed)". |
xfat_metadata(handle: STRING, path: STRING) -> (HASH, ERROR) |
all | Returns metadata for a path in an opened exFAT image, including created_at/modified_at/accessed_at with *_utc_offset_valid flags, attributes and valid_data_size (the written portion of size; the remainder is slack). |
xfat_open(image: STRING, offset?: INTEGER|HASH, length?: INTEGER) -> (HASH, ERROR) |
all | Opens an exFAT filesystem image and returns a handle. Returns (result, err). A volume that sits inside a larger image is opened where it lies rather than carved out first: pass the partition hash from table_list_partitions, or an explicit byte offset with an optional length. Every byte offset this handle goes on to report then already includes that start, so it addresses the image file itself and can be compared with a range from the table_ family without adding anything -- adding it twice, or comparing a partition-relative offset against a whole-disk one, mislocates evidence without failing. bounded is false when no length was given, which means reads can run past this volume's end into whatever follows it on the disk. An exFAT volume records in its own boot record the sector its partition began at, and the specification makes that field informational, so a volume imaged at the partition level -- its boot record saying sector 2048 while byte 0 of the file is the volume -- opens rather than being refused. Both numbers travel in xfat_report's volume hash, as partition_offset and base, and a disagreement raises recorded_partition_offset_disagrees there rather than being dropped. An offset you pass is a different matter: that is where the volume is read from, and it is still checked against what the boot record claims. |
xfat_read_file(handle: STRING, path: STRING) -> (STRING, ERROR) |
all | Reads a file's bytes from an opened exFAT image. It allocates for the file's whole length before reading a byte of it, so it is bounded by what a library will read into memory: xfat_extract_file streams the file to disk instead, xfat_hash_file digests it in place, and xfat_read_file_at returns one window of it. |
xfat_read_file_at(handle: STRING, path: STRING, offset: INTEGER, length: INTEGER) -> (BYTES, ERROR) |
all | Reads one window of a file inside an opened exFAT image and returns it as a BYTES buffer, so the head of a file far larger than memory is still reachable. length is capped at 32 MiB, which is what the returned value has to fit into; use xfat_extract_file for the whole of a larger file. An offset at or past the file's recorded size returns an empty buffer, which is how a caller walks off the end. An offset inside the recorded size but past the bytes the library could locate is an error naming both numbers: that is content the volume says exists and the image cannot produce, and a short buffer or a run of zeroes would hide it. The result is short rather than padded when the window runs past what was located. Returns (bytes, err). |
xfat_read_file_bytes(handle: STRING, path: STRING) -> (BYTES, ERROR) |
all | Reads a file from an opened exFAT image as a BYTES buffer. It allocates for the file's whole length before reading a byte of it, so it is bounded by what a library will read into memory: xfat_extract_file streams the file to disk instead, xfat_hash_file digests it in place, and xfat_read_file_at returns one window of it. |
xfat_recover_file(handle: STRING, index: INTEGER, dest: STRING) -> (HASH, ERROR) |
all | Writes out what is locatable of an entry xfat_deleted reported. exFAT is the one format here that can report a deleted file's whole layout as a fact: a stream extension that recorded NoFatChain declared the run contiguous while the file was live, and freeing the chain took nothing away, so those entries recover in full with content_state declared_contiguous. Everything else yields the first cluster alone. An entry past its valid-data length carries a caveat: those bytes were allocated to this file and never written by it. dest must not already exist: two deleted entries in one image can carry the same name, and evidence written over by accident is not recoverable. A write that fails part way has its partial output removed, because a prefix left under the name of the whole file reads as the whole file. bytes_written is split three ways and the split is the finding: located_bytes were read from the image, sparse_bytes are a hole the filesystem recorded and so are the file's own content, and unlocated_bytes are zeros standing in for ranges the library could not place -- never evidence that the file held zeros there. caveats states in prose what this recovery does not establish, and a report that quotes the digest without it is quoting a number out of its scope. Returns (result, err). |
xfat_recover_file_assuming_contiguous(handle: STRING, index: INTEGER, dest: STRING) -> (HASH, ERROR) |
all | Recovers a deleted exFAT entry as though the clusters after its first one followed it in order, the way fat_recover_file_assuming_contiguous does. An entry that recorded NoFatChain needs no hypothesis -- the volume already stated the layout -- and for those this reports assumed false and returns the same bytes as xfat_recover_file. dest must not already exist: two deleted entries in one image can carry the same name, and evidence written over by accident is not recoverable. A write that fails part way has its partial output removed, because a prefix left under the name of the whole file reads as the whole file. bytes_written is split three ways and the split is the finding: located_bytes were read from the image, sparse_bytes are a hole the filesystem recorded and so are the file's own content, and unlocated_bytes are zeros standing in for ranges the library could not place -- never evidence that the file held zeros there. caveats states in prose what this recovery does not establish, and a report that quotes the digest without it is quoting a number out of its scope. Returns (result, err). |
xfat_report(handle: STRING) -> (HASH, ERROR) |
all | Walks the whole volume once and returns libxfat's own versioned document: the reachable directory tree with deleted entries reported at the slots they occupy, no extent assumed, and each row's entry-set checksum carried as two bits -- checked beside verified -- because the comparison does not happen for a synthetic entry and a verified that nothing checked says nothing. The sweep for records carved out of unallocated space is not run here; xfat_deleted is where those are looked for, with the confidence grading this shape has nowhere to put. Each row carries the valid data length beside the size: the bytes between them are allocated, readable and were never written by this file. exFAT records no metadata-change time, so metadata_changed is empty on every row, and each timestamp says whether the volume actually recorded a UTC offset for it. One row per entry in the shape all six *_report builtins share: path, name, type, size, the deleted and fragmented bits, an identity and a parent identity, the four times that mean the same thing on every format, the layout the fragments were derived from, and image-absolute fragments whose offsets are -1 with located false wherever a run has no place in the image. Everything this format records that the others do not travels verbatim in each row's extra, so normalising loses nothing. identity_kind names the addressing and identity_stable says whether it survives the slot being reused, read from the same Capabilities *_capabilities reports and carrying identity_stable_answered because not every library declares it. completeness_checked says whether anything reconciled this listing against the filesystem's own record of what exists; completeness_proven means nothing without it. The library's own schema_version, library_version and generated stamp travel with the document. Reads the whole volume, so it costs O(volume) in time and holds its listing in memory rather than streaming it; the listing is capped at 50000 rows with every count beside it honest past the cap. events_from(report, "fs_report") turns it into a supertimeline. Returns (result, err). |
xfat_slack(handle: STRING, path: STRING) -> (HASH, ERROR) |
all | Reports an exFAT entry's cluster tail and the part of its allocation past ValidDataLength, each range carrying which of the two it is. The unwritten region is a different byte class from slack: it lies inside the recorded size, it is allocated and readable, and it can span several runs because ValidDataLength may be zero on a large allocation -- which is why it is not a flag on the last range. Unlike its FAT sibling libxfat does report directory slack, a divergence its own documentation calls deliberate, because directory cluster slack is where deleted directory records survive. Two library behaviours are corrected here. A tail that SlackRange declines to name is diagnosed rather than rendered as zero. And UnwrittenRanges does not check whether the run walk was truncated before mapping the region, so a short walk silently reports a smaller unwritten region than exists: that case is warned on and the scan marked incomplete, which makes unwritten_bytes a lower bound rather than a total. A deleted entry the volume had recorded as contiguous keeps its layout through deletion, and libxfat would report its slack -- but this addresses an entry by path and a deleted one cannot be opened by path, so that case is not reachable here. Returns (result, err). |
xfat_verify(handle: STRING) -> (HASH, ERROR) |
all | Runs exFAT's two independent integrity checks over the live directory tree: each file entry set's recorded checksum against the checksum computed over the set as read, and each name re-hashed through the volume's own up-case table against the hash stored in its stream extension record. Neither subsumes the other -- a name rewritten with the entry set checksum recomputed but the hash left alone passes one and fails the other -- so they are reported as two checks, not one. An entry whose name is a libxfat placeholder rather than a name read off the volume is reported as a finding, because that name is the library's invention and not evidence. Deleted and carved records are not walked, and libxfat reports no warning for a directory it could not read, so examined is the count of entries actually reached rather than a claim about the volume. Every check reports checked beside passed, because a check that never ran says nothing about the volume and one boolean cannot tell "intact" from "nothing to compare against"; examined is the denominator it covered. Each check's kind is integrity -- a checksum or mechanism the format itself defines -- or structural: a plausibility test, a flag the volume keeps about itself, or whether a structure could be reached. verified is true only when at least one integrity check actually ran and every check that ran, of either kind, passed, so a volume with nothing checkable is never verified however many structural checks held -- integrity_checks_run is what tells that apart from a failure. scope states what the checks did not reach. Returns (result, err). |
xfs_capabilities(handle: STRING) -> (HASH, ERROR) |
all | Reports what XFS can record, as distinct from what this volume happens to have recorded. Twenty-five capabilities, thirteen constants of the format and twelve read from this superblock, the split falling almost exactly on the v4/v5 line: creation_times and metadata_checksums are both "is this a v5 filesystem", and a v4 filesystem has no birth time at all. journaled is true only for an internal log, so a filesystem with an external log device reports false. needs_repair is the one entry here that is not a capability -- it is the superblock's own statement that this filesystem was not cleanly unmounted, quoted where libxfs put it. The format version libxfs also carries in that struct is not repeated here, since xfs_metadata already reports it and the same number in two places invites a reader to wonder which was read. Every capability reports two bits, not one: a name appears in capabilities only where the library declares it, and the questions it does not answer are listed in unanswered rather than rendered false -- a missing field and a recorded no are different facts and only one of them is evidence. source says whether an answer is a constant of the format or was read from this volume, which is what decides whether it may be cached across volumes; volume_specific is that list. Nothing here describes the contents: a true access_times says the record has the field, not that any timestamp on this volume was maintained. Reads no disk. Returns (result, err). |
xfs_close(handle: STRING) -> (HASH, ERROR) |
all | Closes an XFS handle and releases its file. |
xfs_deleted(handle: STRING, path: STRING) -> (HASH, ERROR) |
all | Reports the free slots and carved candidates in one XFS directory data blocks. It takes a path where the other five take only a handle, because libxfs offers no volume-wide sweep and scanning the root instead would be a claim about every directory made from evidence about one. Content is not recoverable through this route and every entry says so with content_state unsupported: XFS clears di_mode when it frees an inode and libxfs refuses to open an unallocated one, with no raw escape hatch, so names come back and bytes do not. xfs_unlinked is the one place XFS yields deleted content. Best-effort parsing is on, so a malformed block is resynchronised past and recorded as an anomaly instead of ending the scan. Short-form directories -- small ones stored inline in the inode -- carry no deleted-entry recovery whatever, and a scan of one reports complete false with the reason rather than an empty list. Truncation is read from the listing rather than from an error, because a cap the caller set returns no error at all. Each entry carries the libxfs confidence grade and the reasons behind it. Returns (result, err). |
xfs_extract_file(handle: STRING, path: STRING, dest: STRING) -> (HASH, ERROR) |
all | Streams a file out of an opened XFS image and writes it to dest, never holding the whole of it in memory -- which is what makes a file larger than a mutant value, or larger than RAM, extractable at all. The SHA-256 of what was written is computed in the same pass, so nothing has to read the extracted copy back to obtain one. dest must not already exist: two files in an image can share a name, and evidence written over by accident is not recoverable. A copy that fails part way has its partial output removed, because a prefix left under the name of the whole file reads as the whole file. size is the length the volume records and located_bytes is how much of it the library could actually find. The two differ on a FAT or exFAT entry whose cluster chain was broken when the file was deleted: the entry still records the original length and only a prefix of it leads anywhere. When they differ truncated is true and what was written is the recoverable prefix rather than the file -- so read truncated before treating digest as the file's digest. Returns (result, err). |
xfs_hash_file(handle: STRING, path: STRING, algorithm: STRING) -> (HASH, ERROR) |
all | Digests a file inside an opened XFS image without writing a copy of it anywhere, reading it in fixed chunks so the peak allocation has nothing to do with the size of the file. This is how a file too large to read into a mutant value is still looked up in a hash set. md5 and sha1 are offered because published hash sets are keyed on them, not as security properties. size is the length the volume records and located_bytes is how much of it the library could actually find. The two differ on a FAT or exFAT entry whose cluster chain was broken when the file was deleted: the entry still records the original length and only a prefix of it leads anywhere. When they differ truncated is true and the digest covers only the recoverable prefix, so it will not match a hash set entry for the intact file -- which is the honest answer rather than a defect. Returns (result, err). |
xfs_list_files(handle: STRING, dir: STRING) -> ([]HASH, ERROR) |
all | Lists entries under a directory in an opened XFS image, with file_type from the directory record. A damaged inode no longer aborts the listing: that entry is reported with inode_error set and size 0. |
xfs_log_records(handle: STRING) -> (HASH, ERROR) |
all | Reports the XFS log's records, sorted by log sequence number. An LSN is a cycle and a block and the cycle is the number of times the log has been written round, so ordering by it is exact and wrapped here reports a fact about the log rather than a hazard in the ordering -- this is the one journal of the three whose own format settles the question. cleared_blocks counts basic blocks holding a cleared record stamp, the well-formed zero-cycle header mkfs.xfs writes across the whole log, which is the difference between a log that has been quiet and one whose records have been overwritten; neither sibling library has anything comparable. checksum_checked and checksum_valid are two bits, and checked is false for four different reasons -- a v4 image, a stored CRC of zero, checksums skipped, or a header shorter than 328 bytes -- so valid means nothing unless checked is true. has_unmount_record false says the image was taken from a running or a crashed system. This goes through libxfs's options form deliberately: the plain call returns only the slice and discards the truncation flag, the cleared-block count, the offsets and every anomaly, so a scan that stopped at a block limit or after eight unreadable blocks would come back looking complete with a nil error. There is no wall-clock time anywhere in an XFS log: it can say what happened and in what order, and can never say when. Returns (result, err). |
xfs_log_transactions(handle: STRING) -> (HASH, ERROR) |
all | Groups the XFS log's operations into transactions, ordered by the LSN each started at, with the buffers and inodes each touched. item_count is what the transaction header claimed and items_recovered is how many were actually rebuilt: libxfs stops at 65536 items in one transaction without raising an anomaly, so the two numbers disagreeing is the only sign of it. committed reports that a commit region was found, and a transaction without one was in flight when the image was captured and never reached the filesystem, which makes it evidence of intent rather than of change -- the strongest thing any of these three journals can say. A LOG_TRANSACTION_RESTARTED anomaly is graded low by libxfs and means a transaction was discarded rather than noted: XFS reuses transaction identifiers aggressively and a second start for a live one replaces the builder, so this builtin raises transaction_replaced beside it. Transactions whose start region was overwritten by the wrap are dropped silently, which makes this a lower bound on what the log recorded. An item's offset is -1 where libxfs reported both ends as zero, because that means the block number was negative rather than that the buffer sits at offset zero, which is the superblock. There is no wall-clock time anywhere in an XFS log. Returns (result, err). |
xfs_metadata(handle: STRING, path: STRING) -> (HASH, ERROR) |
all | Returns metadata for a path in an opened XFS image, including created_at/modified_at/accessed_at/changed_at and needs_repair (the filesystem was left inconsistent and its metadata should be treated with suspicion). A time of exactly zero is reported empty: XFS marks no time as unset, and a v4 inode has no creation time at all. |
xfs_open(image: STRING, offset?: INTEGER|HASH, length?: INTEGER) -> (HASH, ERROR) |
all | Opens an XFS filesystem image and returns a handle. Returns (result, err). A volume that sits inside a larger image is opened where it lies rather than carved out first: pass the partition hash from table_list_partitions, or an explicit byte offset with an optional length. Every byte offset this handle goes on to report then already includes that start, so it addresses the image file itself and can be compared with a range from the table_ family without adding anything -- adding it twice, or comparing a partition-relative offset against a whole-disk one, mislocates evidence without failing. bounded is false when no length was given, which means reads can run past this volume's end into whatever follows it on the disk. |
xfs_read_file(handle: STRING, path: STRING) -> (STRING, ERROR) |
all | Reads a file's bytes from an opened XFS image. It allocates for the file's whole length before reading a byte of it, so it is bounded by what a library will read into memory: xfs_extract_file streams the file to disk instead, xfs_hash_file digests it in place, and xfs_read_file_at returns one window of it. |
xfs_read_file_at(handle: STRING, path: STRING, offset: INTEGER, length: INTEGER) -> (BYTES, ERROR) |
all | Reads one window of a file inside an opened XFS image and returns it as a BYTES buffer, so the head of a file far larger than memory is still reachable. length is capped at 32 MiB, which is what the returned value has to fit into; use xfs_extract_file for the whole of a larger file. An offset at or past the file's recorded size returns an empty buffer, which is how a caller walks off the end. An offset inside the recorded size but past the bytes the library could locate is an error naming both numbers: that is content the volume says exists and the image cannot produce, and a short buffer or a run of zeroes would hide it. The result is short rather than padded when the window runs past what was located. Returns (bytes, err). |
xfs_read_file_bytes(handle: STRING, path: STRING) -> (BYTES, ERROR) |
all | Reads a file from an opened XFS image as a BYTES buffer. It allocates for the file's whole length before reading a byte of it, so it is bounded by what a library will read into memory: xfs_extract_file streams the file to disk instead, xfs_hash_file digests it in place, and xfs_read_file_at returns one window of it. |
xfs_recover_file(handle: STRING, index: INTEGER, dest: STRING) -> (HASH, ERROR) |
all | Writes out an inode from an allocation group's unlinked chain, as xfs_unlinked reported it. This is the only XFS recovery there is and it is the only one in the whole family that is not inference: an inode reaches an unlinked bucket because the filesystem put it there when a file was unlinked while still open, and it is still allocated and still fully readable, so the ordinary inode reader is both available and correct. An entry from xfs_deleted, whose evidence is a directory record and whose inode has had di_mode cleared, reports content_state unsupported and is refused. dest must not already exist: two deleted entries in one image can carry the same name, and evidence written over by accident is not recoverable. A write that fails part way has its partial output removed, because a prefix left under the name of the whole file reads as the whole file. bytes_written is split three ways and the split is the finding: located_bytes were read from the image, sparse_bytes are a hole the filesystem recorded and so are the file's own content, and unlocated_bytes are zeros standing in for ranges the library could not place -- never evidence that the file held zeros there. caveats states in prose what this recovery does not establish, and a report that quotes the digest without it is quoting a number out of its scope. Returns (result, err). |
xfs_report(handle: STRING) -> (HASH, ERROR) |
all | Walks the whole volume once and returns libxfs's own versioned document, with the inode reconciliation turned on. This is the only one of the six that can say its listing is complete: it walks every allocation group's inode b-tree and compares what the directory walk reached against the superblock's counters and the per-group headers -- three independently maintained sources -- and completeness_proven is libxfs's own conclusion from them. It is also what surfaces the inodes that exist and hold data but that no directory walk would ever produce: the unlinked, which are deleted but still open, and the unreferenced, both counted in the volume hash. That costs a second pass over the metadata, which is why libxfs leaves it off; the alternative is a document that cannot say what it missed. Verification is best-effort, so a checksum mismatch is recorded as an anomaly and the walk continues. Carved directory records are not here; xfs_deleted reports those per directory, graded. start_offset and end_offset bound the volume as its superblock sizes it, end exclusive, as on the other five formats. One row per entry in the shape all six *_report builtins share: path, name, type, size, the deleted and fragmented bits, an identity and a parent identity, the four times that mean the same thing on every format, the layout the fragments were derived from, and image-absolute fragments whose offsets are -1 with located false wherever a run has no place in the image. Everything this format records that the others do not travels verbatim in each row's extra, so normalising loses nothing. identity_kind names the addressing and identity_stable says whether it survives the slot being reused, read from the same Capabilities *_capabilities reports and carrying identity_stable_answered because not every library declares it. completeness_checked says whether anything reconciled this listing against the filesystem's own record of what exists; completeness_proven means nothing without it. The library's own schema_version, library_version and generated stamp travel with the document. Reads the whole volume, so it costs O(volume) in time and holds its listing in memory rather than streaming it; the listing is capped at 50000 rows with every count beside it honest past the cap. events_from(report, "fs_report") turns it into a supertimeline. Returns (result, err). |
xfs_slack(handle: STRING, path: STRING) -> (HASH, ERROR) |
all | Reports an XFS inode's block tail and its unwritten extents. libxfs offers no last-block-tail API, but DataRuns does not trim its ranges to the inode's size, so the tail is what the runs hold past it. An unwritten extent is preallocated space that was never written: it reads back as zeros through the filesystem and carries a real location all the same, which libxfs says plainly is the point -- the blocks are real, and where they are is exactly what makes them worth examining. A hole is not the same thing and has no location to report. DataRuns discards the anomalies its own block-number conversion raises, so a range whose fsblock did not resolve arrives looking like a range at offset zero; it is detected here by the rule libxfs documents -- neither sparse nor located -- and reported at -1, with the scan marked incomplete. A directory keeps its leaf and free-index blocks 32 and 64 GiB into its address space, past the data section its size measures; they are live directory metadata, so they count neither toward allocated_bytes nor as slack. Returns (result, err). |
xfs_unlinked(handle: STRING) -> (HASH, ERROR) |
all | Follows the allocation groups unlinked inode buckets and reports the inodes on them. This is the only evidence in any of the six filesystems that the filesystem itself asserts rather than something inferred from surviving bytes: an inode reaches a bucket because it was unlinked while a process still held it open, and until that process closes it the inode stays allocated, keeps its generation number and stays fully readable. Content is therefore genuinely recoverable here and the runs reported are live extents rather than a stale map, so entries report content_state preserved with allocation_checked true; an inode on a chain that will not open is reported with neither, an inode_unreadable warning and unreadable counted, and the scan is incomplete. A time of exactly zero is reported empty: XFS marks no time as unset, and a v4 inode has no creation time at all. What the chain does not carry is a name -- the directory entry is already gone, which is what unlinked means -- so name is empty and name_source is none. A zeroed bucket array is reported once as an anomaly rather than read as sixty-four empty chains. Returns (result, err). |
xfs_verify(handle: STRING) -> (HASH, ERROR) |
all | Checks the XFS superblock's stored CRC32c against the bytes on the volume, and whether the filesystem is marked as needing repair. XFS records metadata CRCs only from v5 onward, so a v4 image reports the CRC check unavailable rather than failed -- superblock_crc_valid would read false on every v4 volume whatever its state, which is why the checked bit carries the meaning here. The needs-repair result is a flag the filesystem set about itself rather than a check of any bytes, so a pass means it never recorded being left inconsistent, not that it is consistent. Anomalies libxfs raised while building the report are carried through as findings with its own severity words normalised onto the three this family uses. Every check reports checked beside passed, because a check that never ran says nothing about the volume and one boolean cannot tell "intact" from "nothing to compare against"; examined is the denominator it covered. Each check's kind is integrity -- a checksum or mechanism the format itself defines -- or structural: a plausibility test, a flag the volume keeps about itself, or whether a structure could be reached. verified is true only when at least one integrity check actually ran and every check that ran, of either kind, passed, so a volume with nothing checkable is never verified however many structural checks held -- integrity_checks_run is what tells that apart from a failure. scope states what the checks did not reach. Returns (result, err). |
xfs_xattrs(handle: STRING, path: STRING) -> (HASH, ERROR) |
all | Lists one XFS inode's extended attributes, short-form or block-backed, in the order the attribute fork holds them. libxfs performs no deduplication and documents that it does not: two records carrying the same fully-qualified name both come back, which is an inconsistency in the fork rather than a rendering artifact, so duplicates are counted and warned on here instead of being silently collapsed into one. A remote value -- one large enough that XFS gives it its own blocks -- is followed by the library and reported like any other. Values carry no location on this format: libxfs exposes the attribute fork's extents nowhere that a per-value offset can be derived from, so offset is -1 throughout, unlike the ext and HFS+ siblings where it is a real address. Namespaces are the three XFS records: user, trusted and security, which is where SELinux labels live. An inode whose attribute fork is empty reports no attributes and consults no storage, which is an answer rather than a gap. Returns (result, err). |
Disk Image Forensics (34)#
Container and partition-table parsers: raw images, EWF/E01, VHD/VHDX (with differencing chains), and MBR/GPT partition tables.
| Builtin | Platforms | Description |
|---|---|---|
ewf_close(handle: STRING) -> (HASH, ERROR) |
all | Closes an EWF handle and its segment files. |
ewf_metadata(handle: STRING) -> (HASH, ERROR) |
all | Returns EWF metadata (version, sectors/chunks, digests, media info, sector_size, compression_method). chunk_tables_invalid counts chunk-table groups that failed both their primary and backup checksum — their data decoded unverified and should be treated as suspect; chunk_tables_recovered, observed_chunk_count and acquisition_error_count report the rest of the integrity picture. compression_method is libewf's own numbering -- 0 none, 1 deflate, 2 bzip2 -- and not the ZIP method numbers this tree uses elsewhere, where 0 is store and 8 is deflate. EWF v1, which is every EnCase 1-7, linen, ewf, smart and ftk image, carries no compression-method field in its header at all, so there the number is a placeholder and not a reading: has_compression_method says which it is, and while it is false compression_method says nothing about how the image was stored. A v1 chunk flagged compressed is always deflate. The compression level ewfinfo also prints is not published. |
ewf_open(segments: STRING|ARRAY, checksum_policy?: STRING) -> (HASH, ERROR) |
all | Opens an EWF/E01 image. Given one segment path, the rest of the set is discovered beside it and all of them are opened -- the extensions run .E01 to .E99 and then continue .EAA rather than .E100, and the path need not be the first segment, so an image named by its .E03 still decodes from its .E01. Given an array of paths -- even an array of one -- those files are taken as given and nothing is discovered, so a set whose members were gathered from elsewhere still opens. A discovered set with a hole in its numbering is refused rather than opened, because decoding a set that is missing a segment yields an image that is not the one acquired; ewf_segments reports which files are absent, and ewf_open_partial proceeds anyway for triage. checksum_policy decides what an unverifiable chunk table means -- a chunk table maps offsets to compressed chunks, so an unverified one means the decoded bytes may not be the bytes written. The returned segments array names every file the image was decoded from, for the report. Returns (result, err). |
ewf_open_partial(segments: STRING|ARRAY, checksum_policy?: STRING) -> (HASH, ERROR) |
all | Opens an EWF/E01 segment set that ewf_open refuses: one with a hole in its numbering, one that does not begin at segment 1, or one whose final segment carries no done-section. libewf places each chunk by its position among the segments it is given, so a segment after a hole would land where the missing segment's chunks belong. A discovered set with a hole is therefore decoded only from segment 1 up to the first missing segment, and the present segments past it are named in undecoded_segments rather than decoded; a set with no segment 1 is refused, because nothing in it can be placed; and a list of paths must hold segments 1 to k with none missing, read from each file's own header. What is decoded is only part of the device -- the image still reports the full size its volume section declares, while reads past the decoded data return end-of-file -- so this is for metadata inspection and triage of damaged evidence, never for content that will be hashed or carved. It is a separate builtin rather than a flag because the difference has to be visible at the call site and in the audit trail. partial is true whenever the decoded image is not the whole acquired one -- a hole, segments left undecoded, a set that does not begin at segment 1 or whose final segment carries no done-section, or fewer chunks decoded than the volume declares -- read from the reader as well as from discovery; missing_segments lists the holes discovery found, and a list of paths is not discovered, so for one partial rests on the reader alone. These fields carry the caveat into whatever report is built from them; they are on ewf_open's result too, so a template written against one cannot silently drop them for the other. Otherwise identical to ewf_open. Returns (result, err). |
ewf_read_at(handle: STRING, offset: INTEGER, length: INTEGER) -> (STRING, ERROR) |
all | Reads length bytes at an offset from an EWF image (length capped at 32 MiB). Three boundary cases, and each is answered differently so a caller can tell them apart. An offset at or past total_logical_bytes returns an empty buffer, which is how a caller walks off the end of the media. A window that starts inside the data this handle located and runs past it returns what is there, short rather than padded. An offset inside total_logical_bytes that this handle cannot serve -- it lies in a segment ewf_open_partial did not load -- is an error naming both the offset and the declared size, because an empty buffer there is indistinguishable from the end of the disk and would make an incomplete image read as a whole, shorter one. Returns (result, err). |
ewf_read_at_bytes(handle: STRING, offset: INTEGER, length: INTEGER) -> (BYTES, ERROR) |
all | Reads length bytes at an offset from an EWF image as a BYTES buffer (length capped at 32 MiB). Three boundary cases, and each is answered differently so a caller can tell them apart. An offset at or past total_logical_bytes returns an empty buffer, which is how a caller walks off the end of the media. A window that starts inside the data this handle located and runs past it returns what is there, short rather than padded. An offset inside total_logical_bytes that this handle cannot serve -- it lies in a segment ewf_open_partial did not load -- is an error naming both the offset and the declared size, because an empty buffer there is indistinguishable from the end of the disk and would make an incomplete image read as a whole, shorter one. Returns (result, err). |
ewf_segments(path: STRING) -> (HASH, ERROR) |
all | Reports the segment set a path belongs to without opening any of it, so an examiner can record which files an image will be decoded from -- and find out what is missing before paying for a full open. contiguous means the numbering has no holes; it does not mean the set is whole, because a set truncated at its end is indistinguishable from a complete one by looking at a directory, and that case is caught only by ewf_metadata's has_done_section on the opened image. When contiguous is false, missing_segments gives the absent segment numbers and missing_files names the files to go and find; segments is empty, because the set was not resolved. A hole is reported rather than raised, since which files are absent is a finding about the evidence and belongs in a report. Returns (result, err). |
ewf_verify(handle: STRING) -> (HASH, ERROR) |
all | Recomputes MD5 and SHA-1 over the whole decoded device and compares them against the digests the acquisition tool stored in the image. This is the strongest self-contained integrity check an EWF image admits, and it is what ewf_metadata's md5_digest and sha1_digest do not do -- those report what the image claims, this checks it. ok is true only when every stored digest was reproduced and no span failed to decode; an image that stores no digest at all is never ok, because there is nothing to verify against. Spans that fail to decode are zero-filled so hashing can continue and are listed in bad_ranges -- any entry there makes the computed digests meaningless, so read bad_ranges before treating a mismatch as evidence of tampering. md5_match and sha1_match report a comparison that happened only where the matching has_stored_md5 or has_stored_sha1 is true: where the image stored no such digest there was nothing to compare against, and the match field is false because nothing was reproduced rather than because something differed -- read the pair, never the match field alone. Reads the entire image, so it costs O(image size) rather than a seek. Returns (result, err). |
raw_close(handle: STRING) -> (HASH, ERROR) |
all | Closes a raw image handle. |
raw_metadata(handle: STRING) -> (HASH, ERROR) |
all | Returns {file_size, assumed_sector_size, sector_size_assumed} (raw images carry no real sector-size metadata). |
raw_open(image: STRING) -> (HASH, ERROR) |
all | Opens a raw disk image and returns a handle. Returns (result, err). |
raw_read_at(handle: STRING, offset: INTEGER, length: INTEGER) -> (STRING, ERROR) |
all | Reads length bytes at an offset from a raw image (length capped at 32 MiB). |
raw_read_at_bytes(handle: STRING, offset: INTEGER, length: INTEGER) -> (BYTES, ERROR) |
all | Reads length bytes at an offset from a raw image as a BYTES buffer (length capped at 32 MiB). |
table_close(handle: STRING) -> (HASH, ERROR) |
all | Closes a partition-table handle. |
table_detect(image: STRING) -> (HASH, ERROR) |
all | Names every partition scheme that parses cleanly, in preference order, and issues no handle. It is the question asked before the decision, so a script that only wants to know what an image is has nothing to release afterwards. The read is still recorded against the open case. |
table_list_partitions(handle: STRING) -> ([]HASH, ERROR) |
all | Lists partitions with LBA ranges, absolute start_byte/length_byte, type, name, flags, and hex type_code/attributes. Use start_byte rather than start_lba * block_size, which mislocates every partition on a table parsed at a non-zero offset. A number an entry cannot give exactly -- an LBA past 2^63, or an LBA times the block size that overflows -- reads -1 rather than a value that wrapped, and every *_open refuses a partition carrying one: a wrapped start_byte named another partition's bytes. The listing maps the whole device, not only its volumes: allocated, unallocated, meta and structure decode flags, occupies_space marks the rows that tile the device exactly once, and has_nested marks a slice holding a partition scheme of its own. A script that hands every entry to fat_open is handing it GPT headers and interior gaps. index numbers the rows of this map and is what table_partition_info and table_nested take; slot_number is the number the partition table itself gives a partition, and reads -1 on every row that is not one. Selecting the row whose slot_number is 3 is how to ask for the partition a table calls number 3, which is not row 3. |
table_nested(handle: STRING, index: INTEGER) -> (HASH, ERROR) |
all | Returns the partition scheme found inside one partition -- a BSD disklabel in an MBR 0xA5 slice is the case that occurs in practice. Its partitions carry absolute byte offsets computed by the inner table, which is the only thing that knows whether the label was written with container-relative or disk-absolute addresses. A partition holding no nested scheme is an error rather than an empty listing, since an empty listing reads as a container that held nothing; has_nested on the partition is how to ask first. The index is a row in the whole-device map table_list_partitions produces rather than a partition number, so a row that is no partition at all -- a GPT header, a copy of the table, an unallocated gap -- refuses with the same message, and that refusal is not evidence that a real partition was looked inside and found to hold nothing. |
table_open(image: STRING) -> (HASH, ERROR) |
all | Opens a disk image and parses its partition table. Each warning carries a stable code -- overlap, out_of_bounds, hybrid_mbr, entry_count_truncated, backup_missing, backup_mismatch, nested -- beside the prose and the LBA it was found at, and warning_codes is the set of them; branch on the code, because the prose is reworded between releases and the code is not. gpt_backup reports the secondary GPT as ok, missing or mismatch, and unknown when the question does not apply -- two copies are written together, so a mismatch means one was rewritten without the other. candidates lists every scheme that parsed cleanly, so ambiguous means this parse picked a winner: table_open_strict refuses to, table_open_as chooses, table_open_all keeps every one. Returns (result, err). |
table_open_all(image: STRING) -> ([]HASH, ERROR) |
all | Returns one handle per scheme that parsed cleanly, rather than one winner. A hybrid MBR -- a disk whose protective MBR also carries real records -- is the case that needs it: table_open reports the GPT, and the MBR entries are reachable by no other route. Each handle owns its own descriptor and is released with its own table_close. |
table_open_as(image: STRING, scheme: STRING) -> (HASH, ERROR) |
all | Parses the image as one named scheme instead of autodetecting, which is how an examiner records having chosen between candidates rather than accepting a default. Returns what table_open returns. A scheme the parser does not have is refused by name rather than attempted. |
table_open_strict(image: STRING) -> (HASH, ERROR) |
all | Opens like table_open, but refuses media on which more than one scheme parses cleanly instead of resolving it by preference order. The error names every candidate and the one preference would have returned. Use it where a silent winner is not an acceptable answer. |
table_partition_info(handle: STRING, index: INTEGER) -> (HASH, ERROR) |
all | Returns details for a single row of the map table_list_partitions produces, including absolute start_byte/length_byte and the decoded allocated/unallocated/meta/structure flags. The index is a row in that map and not a partition number: the map covers the whole device, so its rows include the protective MBR, both GPT headers, both copies of the partition table and the unallocated gaps, and on a GPT disk index 1 is the GPT header rather than the second partition. slot_number is the number the partition table itself gives a partition and reads -1 on every row that is not one, so the way to reach the partition a table calls number 3 is to list the partitions and take the row whose slot_number is 3. start_lba, length_lba, end_lba, start_byte or length_byte reads -1 when it does not fit a signed 64-bit integer, and every *_open refuses a partition carrying one. |
vhdi_chain(handle: STRING) -> (HASH, ERROR) |
all | Names every image file that together constitutes this device, from the disk that was opened outwards to its base, with what produced each link. A differencing disk holds only what was written since its parent, so these files are the device an evidence record has to name -- and they were routinely made by different tools at different times, which is why the creator, the creation time, the footer copy the image was read from and the log replay state are recorded per link rather than flattened into one record. complete is false when a link the device needs was never attached: the chain then stops short of the device, reads into the missing ranges fail rather than returning zeroes, and parent_resolve_error says why. needs_parent is the narrower per-image fact and not a second spelling of complete: it is true only when the disk that was opened had no parent attached at all, so a chain that stops further back reports complete false with needs_parent false. Returns (result, err). |
vhdi_changed_extents(handle: STRING, since_chain_index: INTEGER) -> (HASH, ERROR) |
all | Reports which ranges of the virtual disk were written by the links nearer the leaf than the named chain index, which is the block-level answer to what a checkpoint changed and needs no filesystem knowledge at all. Chain indices count outwards from the disk that was opened: 0 is that disk and 1 its parent, so 1 is what the leaf alone wrote and 2 is what the leaf and the checkpoint below it wrote together; vhdi_chain lists them. A range a child explicitly cleared counts as written, because clearing is a write and a deletion that zeroes its blocks shows up no other way -- filtering on mapped alone under-reports deletions and does it silently. deletions_expressible is false when a VHD link is among those being reported on: VHD has no block state meaning zero, so a region the guest cleared is recorded as belonging to the parent and reads back as the parent's old contents, and across such a link the deletion is absent from the format rather than merely from this answer. An unresolved range is neither changed nor unchanged and is returned as itself. Returns (result, err). |
vhdi_changed_since(handle: STRING, path: STRING) -> (HASH, ERROR) |
all | vhdi_changed_extents addressed by the path of a disk in the chain rather than by its index, which is how a script holding a checkpoint tree asks the question, and the index it resolved to is reported beside the answer. Paths are compared the way the platform's filesystem compares them, so a script does not have to get Windows's case-insensitivity right itself. Naming the disk that was opened yields nothing, because nothing can have been written since the leaf, and that empty answer carries named_disk_is_the_leaf so it is not read as a finding. A path naming no disk in the chain is an error rather than an empty answer. Every boundary of vhdi_changed_extents applies here, deletions_expressible included. Returns (result, err). |
vhdi_close(handle: STRING) -> (HASH, ERROR) |
all | Closes a VHD/VHDX handle. |
vhdi_discover(dir: STRING) -> (HASH, ERROR) |
all | Reads the headers of every disk image in a directory and builds the parent-to-child tree they record, which is the shape a chain walk cannot see: checkpoints branch, and from a leaf a sibling branch is invisible. Children are joined to parents by recorded identity alone -- matching on virtual size would attach any image of the right size, and matching on a recorded filename would follow a name that may have been reused -- so a child whose parent is not in the directory stays a root and says so. branched is true when the directory holds more than one leaf; which branch a virtual machine is using is recorded in the machine's configuration and not in the disks, so it is not guessed here. Files that could not be probed are kept and named rather than dropped, because an unreadable file in a checkpoint directory may be the link that would have joined two halves of the tree, and files that are not images at all are listed apart so they do not bury the real failures. The images are not registered as evidence: naming a directory is not the same as naming the files an examination read. Returns (result, err). |
vhdi_extents(handle: STRING, offset: INTEGER, length: INTEGER) -> (HASH, ERROR) |
all | Maps a window of a VHD/VHDX virtual address space to the runs that back it, which is how a sparse image is read without moving its absent zeroes through memory. Each extent carries its kind -- mapped, zero, zeroed_by_child or unresolved -- and, for a mapped one, the file it lives in and the offset within it; file_offset is -1 everywhere else, because 0 is a real offset and the library leaves the field at zero for a run that has no place in any file. zero and zeroed_by_child both read back as zeroes and are different facts: the first was never written anywhere in the chain, the second was explicitly cleared by a differencing child, and a deletion that clears its blocks appears only as the second -- so they are counted separately and is_write tells them apart. A range resolving to a parent that is not attached is unresolved and is neither, because the disk that would say is missing. The byte totals cover the whole window even when the extent list is capped. A mapped range means bytes exist there, not that they are live data: an image that never returns written blocks to the free pool keeps them mapped after the guest deleted what was in them, and says so as a warning. Pass 0 and the virtual_size from vhdi_metadata to map the whole device, which on a long differencing chain walks every block of every link. Returns (result, err). |
vhdi_map_offset(handle: STRING, offset: INTEGER) -> (HASH, ERROR) |
all | Maps a virtual offset to a backing file offset. Returns {virtual_offset, mapped, file_offset}. |
vhdi_metadata(handle: STRING) -> (HASH, ERROR) |
all | Returns VHD/VHDX metadata (format, disk_type, virtual_size, block/sector size, identifiers), the differencing-chain state (chain_complete, chain_depth, parent_resolve_error), needs_parent for the opened image by itself, and the VHDX log state (is_dirty, has_log, log_replayed). needs_parent and chain_complete answer different questions, and a chain more than two links long is what separates them: needs_parent is true only when this image's own parent was never attached, while chain_complete is false when any link is missing however far back, so an image whose parent attached but whose grandparent did not reports needs_parent false and chain_complete false. chain_complete is the one to test before trusting a read of the whole device; chain_depth counts the links that did attach, this one included, and parent_resolve_error says where the chain stopped. |
vhdi_open(image: STRING) -> (HASH, ERROR) |
all | Opens a VHD/VHDX disk image and returns a handle. Returns (result, err). |
vhdi_probe(image: STRING) -> (HASH, ERROR) |
all | Reads one VHD/VHDX image's headers without opening it: no block allocation table is parsed, no log is replayed and no content is read, which is what makes asking about a terabyte image cheap. It answers what the file says it is, what chain it records, and whether anything about it disagrees with itself -- a checkpoint extension on a disk that is not differencing, an image recording no link identity that no child can ever be matched to, a footer read from a fallback copy because the conformant one could not be. link_identity is reported apart from identifier because they are different fields: a differencing child records its parent's link identity, and conflating the two is what makes a chain fail to join up. The image is registered as evidence, because the program named this file. Returns (result, err). |
vhdi_read_at(handle: STRING, offset: INTEGER, length: INTEGER) -> (STRING, ERROR) |
all | Reads length bytes at a virtual offset from a VHD/VHDX image (length capped at 32 MiB). |
vhdi_read_at_bytes(handle: STRING, offset: INTEGER, length: INTEGER) -> (BYTES, ERROR) |
all | Reads length bytes at a virtual offset from a VHD/VHDX image as a BYTES buffer (length capped at 32 MiB). |
Windows Artifacts (5)#
Execution and shell-activity artifacts: Prefetch, Windows Event Logs (EVTX), shell links (LNK), and Jump Lists.
| Builtin | Platforms | Description |
|---|---|---|
evtx_parse(path: STRING) -> (HASH, ERROR) |
all | Parses a Windows Event Log (.evtx). Walks every chunk and decodes each record's BinXML (templates + substitutions) into the fully-expanded event tree, plus summary fields per record. Returns {source, chunk_count, count, records:[{record_id, timestamp, timestamp_iso, event_id, event_record_id, level, channel, computer, provider, event}]}. Returns (result, err). |
evtx_parse_bytes(path: STRING) -> (HASH, ERROR) |
all | Parses a Windows Event Log (.evtx) exactly as evtx_parse does, except that binary event values (EVTX BinaryType) come back as BYTES buffers instead of hex strings. Everything else -- the record tree, the summary fields, the return shape -- is identical. Use this when an event carries a binary payload you intend to read rather than print. Returns {source, chunk_count, count, records:[{record_id, timestamp, timestamp_iso, event_id, event_record_id, level, channel, computer, provider, event}]}. Returns (result, err). |
jumplist_parse(path: STRING) -> (HASH, ERROR) |
all | Parses a Windows Jump List (recent/pinned destinations). Auto-detects *.automaticDestinations-ms (OLE compound file: numbered shell-link streams + a DestList MRU/metadata stream) and *.customDestinations-ms (concatenated shell links). Each entry merges DestList metadata (last_access, pinned, hostname) with the embedded shell-link target. Returns {type, format_version, entry_count, pinned_count, entries:[{stream_id, target, arguments, working_dir, name, last_access, last_access_iso, pinned, hostname}], unreadable_entries}. unreadable_entries counts what the jump list held and this answer does not: for an automatic list, a numbered stream the parser could not read, since the container declares every one of them to be a shell link; for a custom list, a link that came apart under the parser, but not a byte sequence that was never a link. A non-zero count means the list is damaged or partly recovered, and the entries are the ones that survived. Returns (result, err). |
lnk_parse(path: STRING) -> (HASH, ERROR) |
all | Parses a Windows shell link (.lnk): header (attributes, creation/access/write FILETIME->unix), decoded LinkFlags, LinkInfo local_base_path (target), and StringData (name, relative_path, working_dir, arguments, icon_location). Returns (result, err). |
prefetch_parse(path: STRING) -> (HASH, ERROR) |
all | Decodes a Windows Prefetch (.pf) file — program execution evidence. Transparently decompresses the Win10/11 MAM (Xpress-Huffman) container and parses the SCCA format for XP (v17), Vista/7 (v23), Win8.1 (v26), and Win10/11 (v30/v31). Returns {version, executable, prefetch_hash, run_count, run_times[], files_loaded[], file_count, volumes:[{device_path, serial, created, created_iso}], compressed}. Returns (result, err). |
Unix Artifacts (1)#
Unix log artifacts: RFC 5424 / RFC 3164 syslog parsing.
| Builtin | Platforms | Description |
|---|---|---|
syslog_parse(path: STRING) -> (HASH, ERROR) |
all | Parses a Unix syslog file into structured entries, auto-detecting RFC 5424 (IETF, ISO-8601) and RFC 3164 (BSD) per line; unmatched lines are kept as raw messages. RFC 3164 lines omit the year, so the current year is assumed. Each entry has a ts unix field for timeline_merge/timeline_sort. Returns {count, entries:[{format, priority, facility, severity, timestamp, ts, host, app_name, pid, msgid, structured_data, message}]}. Returns (result, err). |
Browser Artifacts (5)#
Chromium/Firefox history, cookies, and downloads, plus generic read-only SQLite querying. The query runs on a temp copy opened read-only which may attach no other database, so the original is never modified and no statement writes any file.
| Builtin | Platforms | Description |
|---|---|---|
browser_cookies(path: STRING) -> (HASH, ERROR) |
all | Parses a Chromium (Cookies) or Firefox (cookies.sqlite) cookie database. Chromium cookie values are OS-encrypted; such rows are reported with encrypted=true and an empty value (decryption needs OS keys). Returns {browser, count, entries:[{host, name, value, path, expires, expires_iso, secure, http_only, encrypted, browser}]}. Returns (result, err). |
browser_downloads(path: STRING) -> (HASH, ERROR) |
all | Parses download records from a Chromium (History downloads table) or Firefox (places.sqlite moz_annos) database; Firefox support is best-effort (destination file URI). For Chromium, url is where the file came from (the last hop of downloads_url_chains) and url_chain is every hop in order; the page the download started from is tab_url, which url used to hold. Returns {browser, count, entries:[{url, url_chain, tab_url, referrer, site_url, target_path, bytes_total, bytes_received, start_time, end_time, state, mime_type, browser}]}. Returns (result, err). |
browser_history(path: STRING) -> (HASH, ERROR) |
all | Parses a Chromium (History) or Firefox (places.sqlite) history database into normalized visit entries, auto-detecting the schema and converting timestamps to unix. Returns {browser, count, entries:[{url, title, visit_count, last_visit, last_visit_iso, browser}]}. Returns (result, err). |
sqlite_query(path: STRING, sql: STRING, params?: ARRAY) -> (HASH, ERROR) |
all | Runs a read-only SQL query against a SQLite database, pure-Go (no cgo). The database (+ any -wal/-shm sidecars) is copied to a temp file first, so the original is never modified or lock-contended — safe for forensic DBs held open by a running app. The copy is opened read-only and may attach no other database, so ATTACH and VACUUM INTO are refused and no statement writes any file. Optional params is an ARRAY of bind values for a parameterized query: a BYTES value binds as a BLOB, and a value with no SQLite type (an ARRAY, a HASH) is refused. A result with two columns of one name is refused, since a row holds one value per name. TEXT in a column declared DATE, DATETIME or TIMESTAMP comes back as the RFC 3339 time it names, with its fraction and the offset it was written with (UTC when it was written with none, as SQLite's date functions read it); CAST(col AS TEXT) returns the text as stored. Returns {columns, row_count, truncated, rows:[{col: value}]}. Returns (result, err). |
sqlite_query_bytes(path: STRING, sql: STRING, params?: ARRAY) -> (HASH, ERROR) |
all | Runs a read-only SQL query exactly as sqlite_query does, except that BLOB columns come back as BYTES buffers. sqlite_query asks whether a BLOB happens to be valid UTF-8 and returns a string if so and hex if not, so one column's type varies row by row with its content; here a BLOB is a buffer whatever it holds. Other column types are unchanged: INTEGER is an INT, TEXT a STRING, NULL a NULL. Returns {columns, row_count, truncated, rows:[{col: value}]}. Returns (result, err). |
Forensic Timeline (5)#
Normalize timestamps across epochs and merge/sort artifact events into a single supertimeline; Sleuth Kit bodyfile/mactime interop.
| Builtin | Platforms | Description |
|---|---|---|
bodyfile_parse(path: STRING) -> ([]HASH, ERROR) |
all | Parses a Sleuth Kit bodyfile (MD5|name|inode|mode|UID|GID|size|atime|mtime|ctime|crtime) into an array of entry hashes. The name is the only field that may contain the separator, so the md5 and the nine fields after the name are read from their own ends. Refuses a file in which no line held eleven fields. Returns (entries, err). |
mactime(entries: ARRAY) -> []HASH |
all | Builds a chronological MAC-time timeline from bodyfile_parse entries: one row per distinct time with a MACB flag string (m/a/c/b, "." where absent), sorted by ts then name (ts field composes with timeline_merge). |
timeline_merge(sources: ARRAY, field?: STRING) -> ARRAY |
all | Flattens an array of event arrays into one supertimeline sorted by a numeric timestamp field (default "ts"). |
timeline_sort(events: ARRAY, field?: STRING) -> ARRAY |
all | Returns events (array of hashes) sorted ascending by a numeric timestamp field (default "ts"); events missing the field sort last. Stable. |
timestamp_normalize(value: INTEGER|STRING, format?: STRING) -> (HASH, ERROR) |
all | Normalizes a timestamp to {unix, unix_ms, iso, format}. Formats: unix (s/ms/us/ns), filetime (Windows), webkit/chrome, dos (packed 32-bit), iso (RFC3339 string). Default "auto" detects unix magnitude or parses an ISO string. Returns (result, err). |
Schema Interchange (7)#
Normalize any parsed artifact into one event vocabulary, then write that vocabulary out in whichever schema the next tool reads. events_from maps an artifact's entries onto a fixed set of event fields -- one event per timestamp the artifact recorded, with the source entry carried verbatim in extra so nothing is lost -- and event_kinds() reports what each supported artifact maps. ecs_event, ocsf_event and timesketch_event render those events as Elastic Common Schema documents, OCSF events, or the plaso records Timesketch ingests, one event or a whole timeline at a time. Fourteen artifacts and three schemas cost seventeen mappings here rather than forty-two, so a new parser reaches every schema by describing itself once. stix_bundle and stix_pattern take the other road out of an investigation -- indicators rather than events -- rendering extract_iocs output as a STIX 2.1 bundle whose every id is derived from the indicator itself, so a re-run over the same evidence is the same bundle byte for byte. An artifact Mutant does not know is described with a mapping hash rather than waiting for support. See INTERCHANGE_SCHEMAS.md.
| Builtin | Platforms | Description |
|---|---|---|
ecs_event(event: HASH|ARRAY, opts?: HASH) -> (HASH|ARRAY, ERROR) |
all | Renders events_from() events as Elastic Common Schema documents: one ECS document, or one per event when given a whole timeline. Maps the envelope onto @timestamp, event., host/user, file., process., source/destination., url.full and registry.path; keeps the severity word in log.level beside a syslog-scale event.severity; and carries what ECS does not define under a mutant namespace -- including ts_desc, without which the four documents an $MFT record produces are the same document four times, and the verbatim source entry in mutant.extra. A field the artifact did not record is omitted rather than emitted empty. A category ECS has no honest home for, log among them, leaves event.category out rather than filing the document under the wrong one. Returns (document(s), err). |
event_kinds() -> []HASH |
all | Lists the artifact kinds events_from understands, each with the envelope category and action it maps to, the key its entries live under, the timestamps it reads and what each one means, and the envelope fields it fills. |
events_from(artifact: HASH|ARRAY, kind_or_mapping: STRING|HASH) -> ([]HASH, ERROR) |
all | Normalizes a parsed artifact into interchange events: one event per timestamp the artifact records, in a fixed vocabulary the ECS/OCSF/Timesketch emitters are written against. The second argument names a built-in source kind (event_kinds() lists them) or is a mapping hash {kind, category, action, entries, message, times:[{field, desc, format?, ns_field?, array?}], fields:{envelope: source}} describing an artifact this tree does not know. Unknown fields are omitted rather than emitted empty, a zero timestamp yields no event, and extra carries the source entry verbatim so normalizing loses nothing. Accepts a parser's (result, err) pair directly. Returns (events, err). |
ocsf_event(event: HASH|ARRAY, opts?: HASH) -> (HASH|ARRAY, ERROR) |
all | Renders events_from() events as OCSF 1.1 events: one event, or one per event when given a whole timeline. The class comes from the envelope category -- file to 1001 File System Activity, web to 6001, network to 4001, authentication to 3002 -- and anything with no honest fit, execution and registry included, stays on the Base Event 0/0 rather than borrowing a class someone else's detection rule fires on. activity_id is read from what the timestamp records, so a creation time becomes Create and an access time becomes Read; the severity word maps straight onto severity_id, which is why the envelope carries a word. What OCSF has no home for -- the kind, the timestamp description, the verbatim source entry -- goes in unmapped, the field OCSF keeps for exactly that. Returns (event(s), err). |
stix_bundle(iocs: HASH, opts?: HASH) -> (HASH, ERROR) |
all | Renders indicators as a STIX 2.1 bundle, taking extract_iocs() output directly. ipv4 and ipv6 become ipv4-addr and ipv6-addr observables, domains become domain-name, urls url, emails email-addr, and each md5/sha1/sha256 digest becomes a file observable carrying that one algorithm -- three digests of one file are three observables, because nothing in a list of digests says they describe the same file. Every id is a UUIDv5 over the object's ID contributing properties under the namespace STIX fixes for the purpose, so the same indicator is the same object wherever it is seen and a re-run over the same evidence is the same bundle byte for byte. indicators: true adds an Indicator beside each observable, typed unknown rather than malicious-activity: extraction found the value, it did not conclude anything about it. A key no indicator type claims is an error, and so is a value that is not what its key says -- a 40-character digest under sha256 would become an observable nothing else can match. A bundle with no indicators in it has no objects key, because an empty STIX list property must be left out. Returns (bundle, err). |
stix_pattern(type: STRING, value: STRING) -> (STRING, ERROR) |
all | Renders the STIX pattern that matches one indicator: stix_pattern("sha256", digest) gives [file:hashes.'SHA-256' = '...']. Takes the same type names stix_bundle accepts (ipv4, ipv6, domains, urls, emails, md5, sha1, sha256, and their singular spellings), normalizes and checks the value the same way, and escapes it for the pattern. Returns (pattern, err). |
timesketch_event(event: HASH|ARRAY, opts?: HASH) -> (HASH|ARRAY, ERROR) |
all | Renders events_from() events as Timesketch/plaso records, ready for ndjson_stringify: one record, or one per event when given a whole timeline. Fills all three fields Timesketch requires -- message, datetime, timestamp_desc -- and never leaves one empty, because a record missing any of them is dropped on ingest rather than flagged. timestamp counts plaso microseconds. data_type is the plaso string its analyzers key off (mft to fs:stat:ntfs, prefetch to windows:prefetch:execution, the browser kinds by which browser the entry came out of); a kind with no plaso equivalent gets mutant: |
Email Forensics (5)#
Parse raw email into headers/body/attachments/URLs and cryptographically verify DKIM (with SPF/DMARC reporting).
| Builtin | Platforms | Description |
|---|---|---|
email_attachments(raw: STRING) -> ([]HASH, ERROR) |
all | Extracts attachment metadata/content details from raw email input. |
email_headers(raw: STRING) -> (HASH, ERROR) |
all | Parses and returns message headers from raw email input. |
email_parse(raw: STRING) -> (HASH, ERROR) |
all | Parses a raw email message into headers, body parts, and attachments. |
email_spf_dkim(raw: STRING) -> (HASH, ERROR) |
all | Cryptographically verifies DKIM signatures (public key via DNS) and reports SPF/DMARC. SPF is reported as recorded by the receiving MTA; DMARC combines the reported result with DKIM alignment. |
email_urls(raw: STRING) -> ([]HASH, ERROR) |
all | Extracts and normalizes URLs from email headers and body. |
Hash-Set Forensics (3)#
NSRL-style known-file hash sets for include/exclude filtering.
| Builtin | Platforms | Description |
|---|---|---|
hashset_close(handle: STRING) -> (BOOLEAN, ERROR) |
all | Frees a loaded hash set. Returns (bool, err). |
hashset_contains(handle: STRING, hash: STRING) -> (BOOLEAN, ERROR) |
all | Returns whether a hash is in a loaded set (case-insensitive). Returns (bool, err). |
hashset_load(path: STRING) -> (HASH, ERROR) |
all | Loads a file of hashes (one per line, or CSV/NSRL where the hash is the first field) into an in-memory set. Skips headers/comments/non-hex. Returns {handle, count}. Returns (result, err). |