Persistent State Machines: LLM Attention with INT4 In-Memory Cells

Persistent State Machines: Complete Mathematical Proofs and Vivado Implementation Synthesis (Version 8.0)

ABSTRACT We present a formal discrete framework for attention operators in Large Language Models via Persistent State Machines (PSMs). Computation is broadcast to stationary in-memory cells that evaluate local deterministic state transitions. Complete mathematical proofs are given for quantization error bounds, a concrete multi-phase discrete Softmax construction under an explicit bounded-logits assumption, deterministic finite-automaton equivalence with spatial factorization, and membership in DSPACE(O(n)).

We validate the implementation feasibility of the architecture on contemporary programmable logic fabric through two distinct evaluation flows:

1. Low-Power Evaluation (Zynq-7000 xc7z020): A full 1024-cell array (d=128) out-of-context (OOC) block design implementation. By annotating the post-route netlist with a Switching Activity Interchange Format (SAIF) file obtained from a post-route functional simulation, we demonstrate that hardware-level self-activation gating restricts active dynamic switching to sparse cells. The dynamic power of the core logic is estimated below 1.0 mW, yielding a normalized dynamic energy of 3.81 × 10^-5 pJ/op.

   Advertisement

2. System-on-Chip PCIe Integration (UltraScale+ xcvu9p): To verify system-level compatibility, a 256-cell sub-array (representing an integrated 1-head attention system) is integrated within a full SoC. This includes AMBA AXI4 interconnects and an AMD Xilinx PCIe Gen3 x1 Bridge (XDMA v4.2). The complete SoC successfully closed timing at a unified system clock of 62.5 MHz (Worst Negative Slack WNS = +1.854 ns) under the AWS Cloud FPGA Developer environment. The SoC occupies only 0.67% of the device’s logic slices and 0.00% of DSP blocks, showing high scaling potential.

   Advertisement

Functional simulation with more than one thousand random vectors confirmed bit-exact agreement with a fixed-point software reference. All energy figures are simulation-based tool estimates for the synthesised logic; no physical FPGA board measurement was performed, and system-level external memory energy is strictly excluded.

Japanese Patent Application No. 2026-177318 (Patent Pending)

Advertisement

029_Zenodo_ASMA_Paper_v8_0_FullProofMaster.pdf

Advertisement

Files
(17.7 kB)