콜모고로프-아놀드 네트워크를 활용한 FPGA 초고속 머신러닝

Ultrafast machine learning on FPGAs via Kolmogorov-Arnold Networks 07 Jun 2026 This post is a high-level explainer for my Master’s thesis, which involves designing hardware architectures for ultrafast inference and online learning using the Kolmogorov-Arnold Network (KAN) architecture. I’ll assume familiarity with standard machine learning concepts, as well as some understanding of hardware and digital circuits; read my previous post here for the latter. Please read the two papers below for more information, particularly for details on benchmarks and notable results. [FPGA 2026 Best Paper] Duc Hoang * , Aarush Gupta * , and Philip C. Harris. “KANELÉ: Kolmogorov–Arnold Networks for Efficient LUT-based Evaluation.” Proceedings of the 2026 ACM/SIGDA International Symposium on Field Programmable Gate Arrays . ACM, 2026. https://dx.doi.org/10.1145/ 3748173.3779202 [ICML 2026] Duc Hoang * , Aarush Gupta * , and Philip Harris. “Ultrafast on-FPGA Online Learning via Spline Locality in Kolmogorov-Arnold Networks.” arXiv preprint arXiv:2602.02056 , 2026. https://arxiv.org/abs/2602.02056 *equal contribution The case for machine learning on FPGAs Most modern machine learning workloads, whether training or inference, run on graphics processing units (GPUs). Through hardware architectures that support a highly parallel execution model, GPUs can perform simple operations on large amounts of data with extremely high throughput. This makes them ideal for machine learning problems involving large architectures or batch-style training and inference. However, complex GPU architectures cannot meet the demands of applications that require ultra-low latency (e.g. sub-microsecond latency) and high hardware efficiency. Processors (e.g. CPUs and GPUs) incur significant overhead from scheduling and optimizing instructions, dynamically accessing memory, and so on. Extremely specialized workloads with ultralow latency (e.g. $\sim$nanoseconds) and efficiency requirements are instead better served by custom hardware accelerators . Field-programmable gate arrays, or FPGAs, are reconfigurable digital logic devices that are extremely well-suited for this style of custom hardware acceleration. FPGAs contain lookup tables (LUTs), which represent digital functions by enumerating the output value for every combination of binary inputs; flip-flops (FFs), which store state; and other memory and computation primitives. These components and the connections between them are reconfigured to design a custom digital circuit, allowing for low-level hardware architecture and algorithm co-design that enables ultrafast machine learning. Importantly, neural networks are implemented directly as digital logic , rather than as instructions that are sequentially executed on a processor. Background Fixed-point quantization FPGAs and other digital devices fundamentally operate on bits rather than continuous values. However, we often think about arithmetic operations in neural networks (e.g. $\times, +$) as happening over the real numbers $\mathbb R$. We thus need to encode real numbers as bitstrings (sequences of bits), a process known as quantization . Operations like addition and multiplication then become binary functions. One method for doing this is fixed-point quantization . Fixed-point quantization represents numbers in base-2, where some bits (fractional bits) come after the decimal point. To illustrate, if we use 8 bits total with 4 fractional bits after the decimal point, we can represent $2^8$ values from $(-2^7) / 2^4 = -8$ to $(2^7 - 1) / 2^4 = 7.9375$, spaced evenly in increments of $1/2^4 = 0.0625$. We will assume here that the representable range is symmetric about zero. In a fixed-point quantization scheme, we can only represent a discrete set of values in some fixed range, which will lead to approximation error when trying to represent real values. One focus of resource-efficient machine learning is minimizing this approximation error, or quantization error , to enable stable training and inference. Lookup-table neural networks (LUT-NNs) FPGAs implement digital logic primarily through lookup tables (LUTs), which are small components that represent arbitrary binary functions by storing their output for each combination of binary inputs. For example, $\text{AND} : \{0, 1\}^2 \to \{0, 1\}$ 1 is represented with a lookup table Input ($x,y$) $x\text{ AND }y$ 00 0 01 0 10 0 11 1 It then makes sense to learn these binary functions, represented as lookup tables, as core primitives of a neural network: such a network is known as a lookup-table neural network (LUT-NN). However, learning lookup tables through gradient descent or similar approaches is difficult. To address this issue, recall that we can learn real-valued functions $f: \mathbb R \to \mathbb R$ through gradient descent. If we perform fixed-point quantization with $b_i$ input bits and $b_o$ output bits, $f$ becomes a binary function $f_Q : \{0, 1\}^{b_i} \to \{0, 1\}^{b_o}$. We can then learn continuous $f$ and quantize to get our desired lookup tables! To convert $f$ into a LUT, we discretize the function domain and range into $N_i = 2^{b_i}$ and $N_o = 2^{b_o}$ values. The lookup table of $f_Q$ stores, for each of the input values $I \in \{I_0, I_1, \ldots, I_{N_i-1}\}$, the corresponding output. Quantizing a continuous $f$ (dashed line) to a binary function $f_Q$ (orange dots). The example function $f$ above, where $q_{l-1}$ and $q_{l}$ represent quantization of the inputs and outputs, produces a binary function $f_Q$ with LUT: \(q_{l-1}(x_l)\) \(q_l(x_{l+1})\) 00 000 01 011 10 100 11 111 We could also extend this approach to represent multivariate functions as lookup tables, where $f_m: \mathbb{R}^{d_i} \to \mathbb{R}^{d_o}$ would turn into a binary function with $d_i b_i$ input bits and $d_o b_o$ output bits. In this lookup table, we store entries of size $d_o b_o$ for each of the $2^{d_i b_i}$ possible combinations of inputs. This is clearly impractical for any reasonable $d_i$. LUT-based networks therefore combine smaller lookup tables with arithmetic operations to enable architectures that are expressive, resource efficient, and easily trainable. Kolmogorov-Arnold Networks (KANs) KANs replace the learnable weights and fixed activation functions in MLP (multi-layer perceptron) architectures with learnable activation functions . In this work, we demonstrate that KANs are a natural architecture for efficient, expressive LUT-NNs. In a KAN layer, each edge carries a learnable univariate function instead of a scalar weight. For a KAN layer with $n_{\mathrm{in}}$ inputs and $n_{\mathrm{out}}$ outputs, the activation of the $q$-th output is \[y_q = \sum_{p=1}^{n_{\mathrm{in}}} \phi_{q,p}(x_p),\] where $\phi_{q,p} \colon \mathbb{R} \to \mathbb{R}$ are the learned edge activations. Compared to an MLP, which uses \[y_q = \sigma\left( \sum_{p=1}^{n_{\mathrm{in}}} w_{q,p} x_p + b_q \right)\] with fixed $\sigma$, the KAN places the nonlinearity in the edge functions $\{ \phi_{q,p} \}$ and keeps the node operation as a simple sum. The next question is how to learn the KAN activations $\{ \phi_{q,p} \}$. To do so, we parametrize them as linear combinations of some functional basis: \[\phi_{q,p}(x) = \sum_{i=1}^n c_{q,p,i}B_i(x),\] which allows us to treat the coefficients $c_{q,p,i}$ as trainable parameters for gradient descent. The original KAN paper uses B-splines , which form a polynomial basis that is smooth and also local , i.e. only a subset of basis functions are nonzero for any given input. Additionally, B-splines, and therefore the activations $\{ \phi_{q,p} \}$, are defined over a small, finite domain (e.g. $[-1, 1]$), which turns out to be important. Terminology: B-splines refer to the basis functions ${B_i}$, whereas activations refer to the learned $\phi_{q,p}(x) = \sum_{i=1}^n c_{q,p,i}B_i(x)$. While the complete behavior of the KAN architecture has yet to be fully explored, it offers potential improvements over MLPs in scaling