SPB Git

spb/forge Public MIT

Forge — LLM training from scratch in pure C++20 + Metal on Apple Silicon.

llmtrainingcppmetalapple-silicon
C++ 61.2% C 23% Python 7.6% TeX 7.2% CMake 1.1%
478 B · 17 lines cpp
Raw Blame History 
1// Author: Simon-Pierre Boucher — contact@spboucher.ai2#include "nn/linear.h"34#include "ops/ops.h"56namespace forge::nn {78Var Linear::forward(const Var& x) const {9    Var w = mask_.defined() ? ops::mul(weight_, mask_) : weight_;10    if (quant_ != ops::QuantMode::None) w = ops::fake_quant(w, quant_);11    Var y = ops::matmul(x, w, /*transpose_a=*/false, /*transpose_b=*/true);12    if (bias_.defined()) y = ops::add_bias(y, bias_);13    return y;14}1516} // namespace forge::nn17