nixir

diff --git a/README.md b/README.mdindex e99dc1e..1048d9c 100644--- a/README.md+++ b/README.md@@ -7,8 +7,6 @@ where the plugin parses and compiles Nix code at runtime, or; **ahead-of-time** compilation where the `nix-irc` tool pre-compiles `.nix` files into `.nixir` files. -The plugin automatically chooses the fastest path based on file availability.- ## Supported Nix Constructs  - Literals: integers, strings, booleans, null, paths@@ -27,6 +25,145 @@ The plugin automatically chooses the fastest path based on file availability.     `||`, `->`   - Unary: `-`, `!` +## Overview++Nixir is a Nix evaluator plugin that compiles Nix expressions to a custom binary+intermediate representation (IR). Think of it like a compiler for Nix: it+translates human-readable Nix code into a compact, fast-to-execute format that+runs in a custom virtual machine.++The plugin works in two ways:++1. **Ahead-of-time**: Use the `nix-irc` tool to compile `.nix` files to `.nixir`+   once, then load them instantly+2. **On-the-fly**: Let the plugin parse and compile Nix code at runtime when you+   need it++While Nixir _is_ designed as a toy research project, I _envision_[^1] a few+_potential_ use cases built around working with Nix. Sure, you _probably_ would+not go work with Nix willingly, science is not about why, it is about _why not_.++Some potential use cases for Nixir _might_ include:++- **CI/CD Acceleration**: Pre-compile stable Nix expressions to `.nixir` for+  faster repeated evaluation in CI pipelines+- **Embedded Nix**: Use Nix as a configuration language in C++ applications+  without bundling the full evaluator+- **Plugin Ecosystem**: Extend Nix with custom evaluation strategies via the+  plugin API+- **Build Caching**: Cache compiled IR alongside source for instant startup of+  Nix-based tools++[^1]: I'm not entirely convinced either, do not ask.++### Architecture++```mermaid+flowchart TD++    subgraph Source["User Source"]+        A[".nix Source Files"]+    end++    subgraph Compiler["External Tool: nix-irc"]+        B1["Parse Nix"]+        B2["Static Import Resolution"]+        B3["Flatten Import Graph"]+        B4["Desugar + De Bruijn Conversion"]+        B5["Emit Versioned IR Bundle (.nixir)"]+    end++    subgraph IR["IR Bundle"]+        C1["Binary IR Format"]+        C2["Versioned Header"]+        C3["No Names, Indexed Vars"]+    end++    subgraph Plugin["nix-ir-plugin.so"]+        D1["Primop Registration"]+        D2["prim_loadIR"]+        D3["prim_compileNix"]+        D4["prim_info"]+    end++    subgraph CompilePath["On-the-fly Path"]+        E1["Parse Source String"]+        E2["IR Generation"]+    end++    subgraph LoadPath["Pre-compiled Path"]+        F1["Deserialize .nixir"]+    end++    subgraph VM["Custom Lazy VM"]+        G1["Heap-Allocated Thunks"]+        G2["Memoization"]+        G3["Cycle Detection"]+        G4["Closure Environments (Array-Based)"]+        G5["FORCE / THUNK Execution"]+    end++    A --> B1+    B1 --> B2+    B2 --> B3+    B3 --> B4+    B4 --> B5+    B5 --> C1++    C1 --> D1++    D2 -->|explicit| F1+    F1 --> G1++    D3 -->|explicit| E1+    E1 --> E2+    E2 --> G1++    G1 -.-> G2 -.-> G3 -.-> G4 -.-> G5+```++The same compiler code runs both in the standalone `nix-irc` CLI tool and inside+the plugin for on-the-fly compilation. This ensures consistent behavior between+pre-compiled and runtime-compiled paths. The intermediate representation (IR)+design uses De Brujin indices instead of names for variable binding, which+eliminates string lookup and the binary format uses a versioned header+(`0x4E495258`). In addition, we make use of string interning for repeated+identifiers and type-tagged nodes for efficient dispatching.++The runtime implements lazy evaluation using heap-allocated thunks. Each thunk+holds a delayed computation and is evaluated at most once through memoization.+Recursive definitions are handled through a blackhole mechanism that detects+cycles at runtime. Variable lookup uses array-based closure environments,+providing O(1) access by index rather than name-based lookup.++The plugin integrates with Nix through the `RegisterPrimOp` API, exposing three+operations: `nixIR_loadIR` for loading pre-compiled `.nixir` bundles,+`nixIR_compile` for on-the-fly compilation, and `nixIR_info` for metadata. This+integration path is compatible with Nix 2.32+.++### IR Format++The `.nixir` files use a versioned binary format:++```plaintext+Header:+  - Magic: 0x4E495258 ("NIRX")+  - Version: 1 (uint32)+  - Source count: uint32+  - Import count: uint32+  - String table size: uint32++String Table:+  - Interned strings for efficient storage++Nodes:+  - Binary encoding of IR nodes+  - Each node has type tag + inline data++Entry:+  - Main expression node index+```+ ## Usage  ### Building@@ -57,6 +194,8 @@ $ nix-irc --no-imports input.nix output.nixir  ### Runtime Evaluation (Plugin) +<!--markdownlint-disable MD013-->+ ```bash # Load the plugin and evaluate IR $ nix --plugin-files ./nix-ir-plugin.so eval --expr 'builtins.nixIR_loadIR "output.nixir"'@@ -68,6 +207,8 @@ $ nix --plugin-files ./nix-ir-plugin.so eval --expr 'builtins.nixIR_compile "1 + $ nix --plugin-files ./nix-ir-plugin.so eval --expr 'builtins.nixIR_info' ``` +<!--markdownlint-enable MD013-->+ ### Running Tests  ```bash@@ -80,36 +221,19 @@ done $ hexdump -C tests/simple.nixir | head -3 ``` -## IR Format--The `.nixir` files use a versioned binary format:--```plaintext-Header:-  - Magic: 0x4E495258 ("NIRX")-  - Version: 1 (uint32)-  - Source count: uint32-  - Import count: uint32-  - String table size: uint32--String Table:-  - Interned strings for efficient storage--Nodes:-  - Binary encoding of IR nodes-  - Each node has type tag + inline data--Entry:-  - Main expression node index-```- ## Contributing -This is a research/experimental project. Contributions welcome!--Areas where help is needed:+This is a research project (with no formal association, i.e., no thesis or+anything) that I'm working on entirely for fun and out of curiousity. Extremely+experimental, could change any time. While I do not suggest running this project+in a serious context, I am happy to receive any kind of feedback you might have.+You will notice _very_ quickly that I'm a little out of my depth, and the code+is in a rough shape. Areas where help is needed: -- Expanding parser to handle more Nix syntax+- Compiler semantics - Performance optimization - Test coverage - Documentation improvements+- Expanding parser to handle more Nix syntax (module system in particular)++Contributions _are_ welcome!