Forge: LLM training from scratch in C++20 + Metal on Apple Silicon
A complete transformer training stack with no ML dependencies: tensors, autograd, hand-written Metal kernels, flash attention (forward and backward), AdamW, BPE tokenizer, checkpointing and generation. Architecture is fully config-driven — the same binary trains 12M to 205M parameter models. Every Metal kernel is validated against a CPU reference (85 parity checks, <=1e-4, most bit-exact), gradients against central finite differences, and each optimization was accepted only after the training loss trajectory stayed numerically unchanged. Measured findings (M5 Max, documented in RESEARCH.md and paper/forge.tex): - `constant constexpr` for MSL tile constants declares an address-space variable, not a compile-time constant. Loops stop unrolling and every matrix accumulator spills: 0.82 -> 10.21 TFLOPS once switched to enums. - That defect is invisible in the AIR at every -O level, because unrolling happens in the driver back end. Benchmark; do not read the IR. - Register pressure, not bandwidth, dominates attention backward. Guided by measured spill counts, three restructurings took it 107 -> 7.05 ms (15.2x). - On M5, mpp::tensor_ops::matmul2d reaches 51.5 TFLOPS with f16 operands vs 10.6 for a tuned simdgroup_matrix kernel (4.9x), verified numerically. f16 on the simdgroup path alone is worth only +18-22%. - Concurrent dispatch for the optimizer sweep: +22% on the 100M config. Trained the 12.2M config for one epoch over 19.14M TinyStories tokens: loss 8.40 -> 2.99, validation 3.009, perplexity 20.27, ~38.2k tokens/sec.
Showing 122 changed files with +20,001 and −0
added
.gitignore
+15 −0
@@ -0,0 +1,15 @@ | ||
| 1 | +build/ | |
| 2 | +runs/ | |
| 3 | +data/ | |
| 4 | +*.metallib | |
| 5 | +*.air | |
| 6 | +.DS_Store | |
| 7 | +compile_commands.json | |
| 8 | +.cache/ | |
| 9 | + | |
| 10 | +# LaTeX build products | |
| 11 | +paper/*.aux | |
| 12 | +paper/*.log | |
| 13 | +paper/*.out | |
| 14 | +paper/*.toc | |
| 15 | +paper/*.pdf | |
added
CLAUDE.md
+201 −0
@@ -0,0 +1,201 @@ | ||
| 1 | +# CLAUDE.md | |
| 2 | + | |
| 3 | +<!-- | |
| 4 | +Author: Simon-Pierre Boucher | |
| 5 | +Contact: contact@spboucher.ai | |
| 6 | +--> | |
| 7 | + | |
| 8 | +> **Instruction to Claude:** Every source file created in this project (headers, .cpp, .metal, scripts) must begin with a header comment containing: | |
| 9 | +> `Author: Simon-Pierre Boucher — contact@spboucher.ai` | |
| 10 | + | |
| 11 | +--- | |
| 12 | + | |
| 13 | +## ⚠️ MANDATORY FIRST STEP: Initial Web Research Phase | |
| 14 | + | |
| 15 | +**Before writing any code**, Claude must perform a substantial web research session to acquire up-to-date, accurate knowledge. Metal compute programming is niche and evolves with each OS/hardware generation — training data may be outdated or incomplete. Do NOT rely on memory for Metal API details, shader syntax, or performance characteristics. | |
| 16 | + | |
| 17 | +Research the following topics thoroughly (multiple searches and full-page fetches per topic, prioritizing Apple's official documentation, WWDC session notes, and reputable engineering blogs): | |
| 18 | + | |
| 19 | +1. **Metal-cpp**: current setup, headers, memory management (NS::SharedPtr, autorelease pools in C++), how to integrate into a CMake project, compiling `.metal` → `.metallib` with `xcrun metal`. | |
| 20 | +2. **Metal Shading Language (latest version)**: compute kernel syntax, threadgroup memory, `simdgroup_matrix` / `simdgroup_multiply_accumulate` API (exact types, supported shapes like 8x8, f16/f32 rules), simd shuffle/reduction intrinsics, atomics. | |
| 21 | +3. **Metal compute best practices for Apple Silicon / M3 family**: threadgroup sizing, occupancy, memory coalescing, `MTLStorageModeShared` unified memory patterns, command buffer batching, avoiding sync stalls, GPU timestamps/counters for profiling, dynamic caching on M3. | |
| 22 | +4. **State-of-the-art matmul on Metal**: study existing high-performance implementations — llama.cpp/ggml Metal kernels, MLX source code (github.com/ml-explore/mlx, especially `mlx/backend/metal/kernels/`), and any published tiling strategies for Apple GPUs. Extract concrete tile sizes and techniques that work on M-series. | |
| 23 | +5. **Flash attention on Metal**: how MLX and llama.cpp implement fused/tiled attention on Apple GPUs; online softmax algorithm details. | |
| 24 | +6. **Training-specific references**: nanoGPT / llm.c (Karpathy) for training loop structure, mixed-precision training with loss scaling, AdamW implementation details, gradient clipping, warmup+cosine schedules. | |
| 25 | +7. **Current macOS/Xcode toolchain**: any recent changes to Metal 3.x/4 features relevant to compute, bfloat16 support status on M3, and known pitfalls. | |
| 26 | + | |
| 27 | +**Deliverable of this phase**: write a `RESEARCH.md` file at the repo root summarizing key findings — exact API signatures to use, chosen tile sizes with justification, links to all sources, and a list of pitfalls to avoid. Update it whenever new research is done during the project. Only after RESEARCH.md exists should implementation begin (Milestone M1). | |
| 28 | + | |
| 29 | +Additionally, **whenever implementing a new kernel or hitting a Metal API uncertainty during the project, search the web again** rather than guessing. | |
| 30 | + | |
| 31 | +--- | |
| 32 | + | |
| 33 | +## Project: Forge — LLM Training Framework from Scratch (C++ / Metal) | |
| 34 | + | |
| 35 | +A minimal, high-performance framework for **training small-to-medium LLMs from scratch** on Apple Silicon, written in **pure C++20** with **Metal compute kernels** for GPU acceleration. No PyTorch, no MLX, no external ML dependencies. The framework must be **model-agnostic and configurable** so the same codebase can train models of different sizes (10M → 500M+ parameters) by changing a config file only. | |
| 36 | + | |
| 37 | +### Target hardware | |
| 38 | +- Apple Silicon, primary target: **M3 Ultra, 96 GB unified memory** (80-core GPU, ~28 TFLOPs FP16, 800 GB/s bandwidth) | |
| 39 | +- Must still run correctly (slower) on any M-series Mac | |
| 40 | + | |
| 41 | +### Core design principles | |
| 42 | +1. **Correctness first, speed second**: every op gets a CPU reference implementation; Metal kernels are validated against CPU outputs (tolerance ≤ 1e-4) and gradients are validated with numerical gradient checking. | |
| 43 | +2. **Model-agnostic**: architecture defined entirely by a JSON/TOML config (n_layers, d_model, n_heads, n_kv_heads, d_ff, vocab_size, context_length, tied embeddings, etc.). No hardcoded model sizes anywhere. | |
| 44 | +3. **Unified memory advantage**: exploit Apple Silicon's shared CPU/GPU memory — use `MTLStorageModeShared` buffers, avoid copies entirely. | |
| 45 | +4. **Minimal dependencies**: C++20 standard library, Metal-cpp (Apple's official C++ bindings), and nothing else. JSON parsing may use a single-header library (nlohmann/json). | |
| 46 | +5. **Hackability**: code should be readable and modifiable for research experiments (pruning masks, quantized weights, custom attention variants). | |
| 47 | + | |
| 48 | +--- | |
| 49 | + | |
| 50 | +## Repository layout | |
| 51 | + | |
| 52 | +``` | |
| 53 | +forge/ | |
| 54 | +├── CLAUDE.md | |
| 55 | +├── CMakeLists.txt | |
| 56 | +├── configs/ | |
| 57 | +│ ├── gpt-10m.json | |
| 58 | +│ ├── gpt-25m.json | |
| 59 | +│ ├── gpt-100m.json | |
| 60 | +│ └── gpt-200m.json | |
| 61 | +├── src/ | |
| 62 | +│ ├── core/ | |
| 63 | +│ │ ├── tensor.h / tensor.cpp # Tensor class, shapes, strides, dtype | |
| 64 | +│ │ ├── allocator.h / allocator.cpp # MTLBuffer pool, no per-op allocations | |
| 65 | +│ │ ├── device.h / device.cpp # Metal device, queues, pipeline cache | |
| 66 | +│ │ └── autograd.h / autograd.cpp # Dynamic graph, backward() | |
| 67 | +│ ├── ops/ | |
| 68 | +│ │ ├── cpu/ # Reference implementations (all ops) | |
| 69 | +│ │ └── metal/ # Metal dispatch wrappers | |
| 70 | +│ ├── kernels/ # .metal shader sources | |
| 71 | +│ │ ├── matmul.metal # naive → tiled → simdgroup_matrix | |
| 72 | +│ │ ├── softmax.metal | |
| 73 | +│ │ ├── layernorm.metal # (or rmsnorm) | |
| 74 | +│ │ ├── attention.metal # fused attention (flash-style) | |
| 75 | +│ │ ├── elementwise.metal # GELU/SiLU, add, mul, fused bias+act | |
| 76 | +│ │ ├── embedding.metal | |
| 77 | +│ │ ├── cross_entropy.metal # fused softmax + CE loss | |
| 78 | +│ │ └── adamw.metal # fused optimizer update | |
| 79 | +│ ├── nn/ | |
| 80 | +│ │ ├── module.h # base Module, parameter registration | |
| 81 | +│ │ ├── linear.h / linear.cpp | |
| 82 | +│ │ ├── embedding.h | |
| 83 | +│ │ ├── attention.h # MHA + GQA support | |
| 84 | +│ │ ├── mlp.h | |
| 85 | +│ │ ├── transformer.h # block + full model from config | |
| 86 | +│ │ └── config.h # ModelConfig struct ← JSON | |
| 87 | +│ ├── train/ | |
| 88 | +│ │ ├── dataloader.h / .cpp # mmap'd binary token files | |
| 89 | +│ │ ├── optimizer.h / .cpp # AdamW (+ weight decay, grad clip) | |
| 90 | +│ │ ├── scheduler.h # warmup + cosine decay | |
| 91 | +│ │ ├── trainer.h / .cpp # training loop, checkpointing, logging | |
| 92 | +│ │ └── checkpoint.h / .cpp # save/load weights + optimizer state | |
| 93 | +│ ├── tokenizer/ | |
| 94 | +│ │ └── bpe.h / bpe.cpp # BPE encode/decode (load pretrained vocab) | |
| 95 | +│ └── main.cpp # CLI: train / generate / eval | |
| 96 | +├── tools/ | |
| 97 | +│ ├── prepare_data.py # HF dataset → tokenized .bin (uint16) | |
| 98 | +│ └── train_tokenizer.py # train small BPE vocab on corpus | |
| 99 | +└── tests/ | |
| 100 | + ├── test_ops.cpp # CPU vs Metal parity for every op | |
| 101 | + ├── test_gradcheck.cpp # numerical gradient checking | |
| 102 | + └── test_overfit.cpp # sanity: overfit tiny batch to ~0 loss | |
| 103 | +``` | |
| 104 | + | |
| 105 | +--- | |
| 106 | + | |
| 107 | +## Architecture specification | |
| 108 | + | |
| 109 | +### Model (decoder-only transformer, configurable) | |
| 110 | +- Token embedding (optionally tied with output head) | |
| 111 | +- Learned positional embeddings **or** RoPE (config flag; default RoPE) | |
| 112 | +- N × TransformerBlock: | |
| 113 | + - Pre-norm (LayerNorm or RMSNorm, config flag; default RMSNorm) | |
| 114 | + - Multi-head attention with causal mask; support GQA via `n_kv_heads` | |
| 115 | + - MLP: SwiGLU (default) or GELU, `d_ff` from config | |
| 116 | + - Residual connections | |
| 117 | +- Final norm → LM head → fused softmax cross-entropy | |
| 118 | + | |
| 119 | +### Tensor class requirements | |
| 120 | +- Dtypes: `f32` (default for training), `f16`/`bf16` (compute/storage where safe), `u16`/`i32` (tokens/indices) | |
| 121 | +- Backed by `MTLBuffer` with `MTLStorageModeShared`; raw pointer accessible from CPU at all times | |
| 122 | +- Shape + strides; row-major; support views/reshape without copy | |
| 123 | +- Reference-counted buffer ownership through the allocator pool | |
| 124 | + | |
| 125 | +### Autograd requirements | |
| 126 | +- Dynamic tape: each forward op records a node with inputs + backward lambda | |
| 127 | +- `loss.backward()` walks the tape in reverse; gradients accumulate into `.grad` tensors | |
| 128 | +- `no_grad` scope for inference/generation | |
| 129 | +- Gradient accumulation across micro-batches must be supported | |
| 130 | + | |
| 131 | +### Metal execution model | |
| 132 | +- One `MTLCommandQueue`; batch many kernel dispatches per `MTLCommandBuffer` | |
| 133 | +- **Never** call `waitUntilCompleted` per-op; sync only at loss readback / logging boundaries | |
| 134 | +- Precompile all pipelines at startup; cache `MTLComputePipelineState` by kernel name + specialization constants | |
| 135 | +- Threadgroup sizes chosen per-kernel; expose via constants for tuning | |
| 136 | + | |
| 137 | +### Kernel optimization roadmap (implement in this order) | |
| 138 | +1. `matmul` naive (correctness baseline) | |
| 139 | +2. `matmul` tiled with threadgroup memory | |
| 140 | +3. `matmul` with `simdgroup_matrix` (f16 accumulate f32) — this is the perf-critical path | |
| 141 | +4. Fused `softmax` (online, single-pass) | |
| 142 | +5. `rmsnorm` / `layernorm` (parallel reduction) | |
| 143 | +6. Fused attention kernel (flash-attention style: tiled QK^T + online softmax + PV in one kernel, no materialized attention matrix) | |
| 144 | +7. Fused `bias + activation`, fused residual add | |
| 145 | +8. Fused `cross_entropy` (avoid materializing full logits softmax) | |
| 146 | +9. Fused `AdamW` update (one kernel over all params) | |
| 147 | + | |
| 148 | +### Training loop requirements | |
| 149 | +- Mixed precision: weights/master in f32, compute in f16/bf16 with loss scaling (config flag) | |
| 150 | +- AdamW with decoupled weight decay; gradient clipping by global norm | |
| 151 | +- LR schedule: linear warmup → cosine decay (configurable) | |
| 152 | +- Gradient accumulation for large effective batch sizes | |
| 153 | +- Checkpoint every N steps (weights + optimizer + RNG state + step); resumable | |
| 154 | +- Logging: step, loss, tokens/sec, LR, grad norm → stdout + CSV file | |
| 155 | +- Deterministic mode (fixed seeds) for debugging | |
| 156 | + | |
| 157 | +### Data pipeline | |
| 158 | +- `tools/prepare_data.py`: downloads HF dataset, tokenizes with trained BPE, writes `train.bin` / `val.bin` as flat uint16 token arrays | |
| 159 | +- C++ dataloader: `mmap()` the .bin file, sample random contiguous windows of `context_length + 1`, build (input, target) batches directly into shared buffers | |
| 160 | +- No Python at training time | |
| 161 | + | |
| 162 | +### CLI | |
| 163 | +``` | |
| 164 | +forge train --config configs/gpt-25m.json --data data/tinystories --out runs/exp1 | |
| 165 | +forge generate --checkpoint runs/exp1/ckpt_5000 --prompt "Once upon a time" --temp 0.8 --top-k 40 | |
| 166 | +forge eval --checkpoint runs/exp1/ckpt_5000 --data data/tinystories/val.bin | |
| 167 | +``` | |
| 168 | + | |
| 169 | +--- | |
| 170 | + | |
| 171 | +## Research extensibility (design for these now, implement later) | |
| 172 | +- **Sparsity masks**: `Linear` must support an optional binary mask applied to weights (for lottery-ticket / pruning experiments). Checkpoints must be able to store initial weights (for rewind-to-init experiments). | |
| 173 | +- **Quantized weights**: leave a clean seam in `Linear` forward to swap in ternary/int8 weight kernels later (BitNet-style). | |
| 174 | +- **Attention variants**: `attention.h` behind an interface so linear-attention variants can be added without touching the rest. | |
| 175 | + | |
| 176 | +--- | |
| 177 | + | |
| 178 | +## Testing & validation protocol (mandatory) | |
| 179 | +1. Every Metal kernel: parity test vs CPU reference (`tests/test_ops.cpp`), max abs error ≤ 1e-4 (f32) / 1e-2 (f16) | |
| 180 | +2. Every op with parameters: numerical gradient check on small tensors (central differences, rel error ≤ 1e-3) | |
| 181 | +3. End-to-end sanity: overfit a single batch of 64 sequences to loss < 0.05 within 500 steps | |
| 182 | +4. Throughput benchmark target on M3 Ultra: ≥ 100k tokens/sec for the 25M config, ≥ 50k tokens/sec for 200M (batch ≥ 512 sequences of 512 tokens) | |
| 183 | + | |
| 184 | +## Build | |
| 185 | +- CMake ≥ 3.24, clang from Xcode toolchain, C++20 | |
| 186 | +- `.metal` files compiled to a `.metallib` at build time via `xcrun metal` custom command | |
| 187 | +- `make test` runs all parity + gradcheck tests; CI-style: tests must pass before any kernel is considered done | |
| 188 | + | |
| 189 | +## Coding conventions | |
| 190 | +- C++20, no exceptions in hot paths, `snake_case` functions, `PascalCase` types | |
| 191 | +- Every file starts with the author header (see instruction at top) | |
| 192 | +- Comments explain *why*, not *what*; kernel files document their threadgroup layout and memory access pattern | |
| 193 | +- No premature abstraction: three concrete uses before generalizing | |
| 194 | + | |
| 195 | +## Suggested milestones | |
| 196 | +1. **M1 — Skeleton**: Tensor, allocator, device init, naive CPU matmul, build system | |
| 197 | +2. **M2 — CPU training**: full model forward/backward on CPU, overfit tiny batch, gradcheck green | |
| 198 | +3. **M3 — Metal core**: matmul (tiled), softmax, norm, elementwise on GPU; parity tests green | |
| 199 | +4. **M4 — Full GPU training**: train 10M model on TinyStories end-to-end, generation working | |
| 200 | +5. **M5 — Performance**: simdgroup matmul, fused attention, fused CE + AdamW; hit throughput targets | |
| 201 | +6. **M6 — Scale**: 100M–200M configs, mixed precision, resumable long runs, SmolLM-corpus pipeline | |
added
CMakeLists.txt
+109 −0
@@ -0,0 +1,109 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +cmake_minimum_required(VERSION 3.24) | |
| 3 | +project(forge LANGUAGES CXX) | |
| 4 | + | |
| 5 | +set(CMAKE_CXX_STANDARD 20) | |
| 6 | +set(CMAKE_CXX_STANDARD_REQUIRED ON) | |
| 7 | +set(CMAKE_CXX_EXTENSIONS OFF) | |
| 8 | + | |
| 9 | +if(NOT CMAKE_BUILD_TYPE) | |
| 10 | + set(CMAKE_BUILD_TYPE Release) | |
| 11 | +endif() | |
| 12 | + | |
| 13 | +add_compile_options(-Wall -Wextra) | |
| 14 | + | |
| 15 | +# --------------------------------------------------------------------------- | |
| 16 | +# Metal shader library: .metal -> .air -> forge.metallib | |
| 17 | +# Compiled with -std=metal3.2 (macOS 15+ baseline: simdgroup_matrix, bfloat). | |
| 18 | +# Debug builds embed sources for the Xcode GPU debugger. | |
| 19 | +# --------------------------------------------------------------------------- | |
| 20 | +file(GLOB METAL_SOURCES CONFIGURE_DEPENDS ${CMAKE_SOURCE_DIR}/src/kernels/*.metal) | |
| 21 | +set(AIR_FILES "") | |
| 22 | +foreach(shader ${METAL_SOURCES}) | |
| 23 | + get_filename_component(shader_name ${shader} NAME_WE) | |
| 24 | + set(air ${CMAKE_BINARY_DIR}/${shader_name}.air) | |
| 25 | + # Metal Performance Primitives (cooperative tensors) need the Metal 4 | |
| 26 | + # language; everything else targets 3.2 so it runs on macOS 15+. | |
| 27 | + if(shader_name MATCHES "_mpp$") | |
| 28 | + set(metal_std "-std=metal4.0") | |
| 29 | + else() | |
| 30 | + set(metal_std "-std=metal3.2") | |
| 31 | + endif() | |
| 32 | + add_custom_command( | |
| 33 | + OUTPUT ${air} | |
| 34 | + # -frecord-sources is unconditional, not Debug-only: without shader sources | |
| 35 | + # embedded in the metallib, the Metal debugger and gpudebug report | |
| 36 | + # "no source" and every per-shader/per-line profiling view comes back | |
| 37 | + # empty. It costs file size, nothing else. | |
| 38 | + COMMAND xcrun -sdk macosx metal ${metal_std} -O2 | |
| 39 | + -gline-tables-only -frecord-sources | |
| 40 | + -c ${shader} -o ${air} | |
| 41 | + DEPENDS ${shader} | |
| 42 | + COMMAND_EXPAND_LISTS | |
| 43 | + COMMENT "Compiling Metal shader ${shader_name}.metal" | |
| 44 | + VERBATIM) | |
| 45 | + list(APPEND AIR_FILES ${air}) | |
| 46 | +endforeach() | |
| 47 | + | |
| 48 | +add_custom_command( | |
| 49 | + OUTPUT ${CMAKE_BINARY_DIR}/forge.metallib | |
| 50 | + # Link with `metal`, not `metallib`: the sources only survive into the | |
| 51 | + # library if -frecord-sources is passed at link time too, and `metallib` | |
| 52 | + # rejects that flag ("unknown argument"). | |
| 53 | + COMMAND xcrun -sdk macosx metal -frecord-sources ${AIR_FILES} | |
| 54 | + -o ${CMAKE_BINARY_DIR}/forge.metallib | |
| 55 | + DEPENDS ${AIR_FILES} | |
| 56 | + COMMENT "Linking forge.metallib" | |
| 57 | + VERBATIM) | |
| 58 | +add_custom_target(forge_kernels ALL DEPENDS ${CMAKE_BINARY_DIR}/forge.metallib) | |
| 59 | + | |
| 60 | +# --------------------------------------------------------------------------- | |
| 61 | +# Core library | |
| 62 | +# --------------------------------------------------------------------------- | |
| 63 | +add_library(forge_core STATIC | |
| 64 | + src/core/metal_impl.cpp | |
| 65 | + src/core/tensor.cpp | |
| 66 | + src/core/allocator.cpp | |
| 67 | + src/core/device.cpp | |
| 68 | + src/core/autograd.cpp | |
| 69 | + src/ops/cpu/cpu_ops.cpp | |
| 70 | + src/ops/metal/metal_ops.cpp | |
| 71 | + src/ops/ops.cpp | |
| 72 | + src/nn/linear.cpp | |
| 73 | + src/train/optimizer.cpp | |
| 74 | + src/train/dataloader.cpp | |
| 75 | + src/train/checkpoint.cpp | |
| 76 | + src/train/trainer.cpp | |
| 77 | + src/tokenizer/bpe.cpp) | |
| 78 | + | |
| 79 | +target_include_directories(forge_core PUBLIC | |
| 80 | + ${CMAKE_SOURCE_DIR}/src | |
| 81 | + ${CMAKE_SOURCE_DIR}/third_party | |
| 82 | + ${CMAKE_SOURCE_DIR}/third_party/metal-cpp) | |
| 83 | + | |
| 84 | +target_link_libraries(forge_core PUBLIC | |
| 85 | + "-framework Metal" "-framework Foundation" "-framework QuartzCore") | |
| 86 | + | |
| 87 | +add_dependencies(forge_core forge_kernels) | |
| 88 | + | |
| 89 | +# --------------------------------------------------------------------------- | |
| 90 | +# CLI | |
| 91 | +# --------------------------------------------------------------------------- | |
| 92 | +add_executable(forge src/main.cpp) | |
| 93 | +target_link_libraries(forge PRIVATE forge_core) | |
| 94 | + | |
| 95 | +# --------------------------------------------------------------------------- | |
| 96 | +# Tests | |
| 97 | +# --------------------------------------------------------------------------- | |
| 98 | +enable_testing() | |
| 99 | +foreach(t test_ops test_gradcheck test_overfit test_tokenizer) | |
| 100 | + add_executable(${t} tests/${t}.cpp) | |
| 101 | + target_link_libraries(${t} PRIVATE forge_core) | |
| 102 | + add_test(NAME ${t} COMMAND ${t} WORKING_DIRECTORY ${CMAKE_BINARY_DIR}) | |
| 103 | +endforeach() | |
| 104 | + | |
| 105 | +# Benchmarks: built, not run by ctest. | |
| 106 | +foreach(b bench_matmul bench_attention bench_precision mppcheck) | |
| 107 | + add_executable(${b} tests/${b}.cpp) | |
| 108 | + target_link_libraries(${b} PRIVATE forge_core) | |
| 109 | +endforeach() | |
added
LICENSE
+21 −0
@@ -0,0 +1,21 @@ | ||
| 1 | +MIT License | |
| 2 | + | |
| 3 | +Copyright (c) 2026 Simon-Pierre Boucher <contact@spboucher.ai> | |
| 4 | + | |
| 5 | +Permission is hereby granted, free of charge, to any person obtaining a copy | |
| 6 | +of this software and associated documentation files (the "Software"), to deal | |
| 7 | +in the Software without restriction, including without limitation the rights | |
| 8 | +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |
| 9 | +copies of the Software, and to permit persons to whom the Software is | |
| 10 | +furnished to do so, subject to the following conditions: | |
| 11 | + | |
| 12 | +The above copyright notice and this permission notice shall be included in all | |
| 13 | +copies or substantial portions of the Software. | |
| 14 | + | |
| 15 | +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |
| 16 | +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |
| 17 | +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |
| 18 | +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |
| 19 | +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |
| 20 | +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |
| 21 | +SOFTWARE. | |
added
README.md
+275 −0
@@ -0,0 +1,275 @@ | ||
| 1 | +<!-- Author: Simon-Pierre Boucher — contact@spboucher.ai --> | |
| 2 | + | |
| 3 | +<div align="center"> | |
| 4 | + | |
| 5 | +# 🔥 Forge | |
| 6 | + | |
| 7 | +### LLM training from scratch — pure C++20 + Metal on Apple Silicon | |
| 8 | + | |
| 9 | +**No PyTorch. No MLX. No ML dependencies. Just C++, Metal kernels, and a JSON parser.** | |
| 10 | + | |
| 11 | +<br/> | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | +<br/> | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | +<br/> | |
| 31 | + | |
| 32 | +**[📄 Paper](paper/forge.tex)** · **[🔬 Research notes](RESEARCH.md)** · **[📐 Spec](CLAUDE.md)** | |
| 33 | + | |
| 34 | +</div> | |
| 35 | + | |
| 36 | +--- | |
| 37 | + | |
| 38 | +## ✨ What this is | |
| 39 | + | |
| 40 | +A complete, working transformer training stack built from nothing on Apple Silicon. | |
| 41 | +Tensors, autograd, Metal compute kernels, flash attention (forward **and** backward), | |
| 42 | +AdamW, BPE tokenizer, checkpointing, generation — all hand-written, all validated | |
| 43 | +against a CPU reference. | |
| 44 | + | |
| 45 | +Architecture is **entirely config-driven**: the same binary trains a 12M or a 205M | |
| 46 | +parameter model by changing a JSON file. | |
| 47 | + | |
| 48 | +> **Why it matters:** as of July 2026, *no* major open-source framework ships a fused | |
| 49 | +> attention **backward** kernel for Metal. MLX throws `"NYI"`. llama.cpp doesn't | |
| 50 | +> support the op. PyTorch MPS and Candle are forward-only. Forge has one, and it's | |
| 51 | +> 15× faster than the naive version. | |
| 52 | + | |
| 53 | +--- | |
| 54 | + | |
| 55 | +## 🏆 Headline results | |
| 56 | + | |
| 57 | +<div align="center"> | |
| 58 | + | |
| 59 | +| | Metric | Value | | |
| 60 | +|:--|:--|--:| | |
| 61 | +| 🚀 | Training throughput (12M model, ctx 512) | **38.2k tok/s** | | |
| 62 | +| ⚡ | GEMM f32, `simdgroup_matrix` | **10.8 TFLOPS** | | |
| 63 | +| 🔥 | GEMM f16 via `matmul2d` (M5 neural accelerators) | **51.5 TFLOPS** | | |
| 64 | +| 🎯 | Flash attention backward speedup | **15.2×** | | |
| 65 | +| 📉 | Validation perplexity (12M, 1 epoch TinyStories) | **20.27** | | |
| 66 | +| ✅ | CPU↔Metal parity checks | **85 passing** | | |
| 67 | + | |
| 68 | +</div> | |
| 69 | + | |
| 70 | +### 🎓 Trained model — 12.2M params, one epoch, ~8 minutes | |
| 71 | + | |
| 72 | +<div align="center"> | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | +</div> | |
| 81 | + | |
| 82 | +Validation loss falls monotonically with no sign of overfitting: | |
| 83 | + | |
| 84 | +| step | 99 | 199 | 299 | 399 | 499 | final | | |
| 85 | +|:--|--:|--:|--:|--:|--:|--:| | |
| 86 | +| **train** | 5.14 | 4.10 | 3.48 | 3.17 | 3.07 | **2.99** | | |
| 87 | +| **val** | 5.16 | 4.10 | 3.48 | 3.20 | 3.07 | **3.01** | | |
| 88 | + | |
| 89 | +Sampled from the trained checkpoint (`temp 0.8`, `top-k 40`): | |
| 90 | + | |
| 91 | +> *Once upon a time, there was a little dog named Spot. Spot loved to hop and play in | |
| 92 | +> the yard with his toys. One day, it was very cold. Spot wanted to play, but it was | |
| 93 | +> too big for Spot to jump in the yard with the yarn. A big dog saw the yarn and | |
| 94 | +> wanted them too…* | |
| 95 | + | |
| 96 | +Coherent English, consistent characters, story structure — from a 12M-parameter model | |
| 97 | +trained for eight minutes on one machine. | |
| 98 | + | |
| 99 | +--- | |
| 100 | + | |
| 101 | +## 🔬 Four findings worth your time | |
| 102 | + | |
| 103 | +These came out of measurement, and several contradicted what we expected. | |
| 104 | + | |
| 105 | +### 1️⃣ `constant constexpr` cost us 12× | |
| 106 | + | |
| 107 | +```cpp | |
| 108 | +constant constexpr uint TM = 4; // ❌ an address-space VARIABLE, not a constant | |
| 109 | +enum : uint { TM = 4 }; // ✅ a real compile-time constant | |
| 110 | +``` | |
| 111 | + | |
| 112 | +In MSL, `constant` is an **address-space qualifier**, and program-scope variables must | |
| 113 | +live there. So that declaration isn't a compile-time constant — loop bounds built from | |
| 114 | +it don't unroll, `acc[i][j]` becomes dynamic indexing into opaque `simdgroup_matrix` | |
| 115 | +values, and the driver spills all 16 accumulators (256 B each) to the stack. | |
| 116 | + | |
| 117 | +**0.82 → 10.21 TFLOPS.** Before the fix, the "optimized" matrix kernel was *3× slower | |
| 118 | +than a naive one*. | |
| 119 | + | |
| 120 | +### 2️⃣ The IR can't diagnose it — benchmark instead | |
| 121 | + | |
| 122 | +`metal -S -emit-llvm` shows the same 3 `alloca`s and 2 MMA intrinsics at `-O0`, `-O2` | |
| 123 | +and `-O3` — **and after the fix that made it 12× faster**. Unrolling and fragment | |
| 124 | +promotion happen in the driver's AIR→ISA back end. Reading the IR looked like | |
| 125 | +confirmation and sent us the wrong way. | |
| 126 | + | |
| 127 | +### 3️⃣ Register pressure, not bandwidth, owns attention backward | |
| 128 | + | |
| 129 | +| attention backward, per layer | gpt-10m | gpt-25m | | |
| 130 | +|:--|--:|--:| | |
| 131 | +| `k,v,dk,dv` all in registers | 150 ms | — | | |
| 132 | +| read-only `k,v` from device | 118 ms | 611 ms | | |
| 133 | +| `dK`/`dV` split into 2 kernels | 105 ms | 549 ms | | |
| 134 | +| **tiled with `simdgroup_matrix`** | **9.66 ms** | **45.5 ms** | | |
| 135 | +| **+ `dK`/`dV` split again** | **7.05 ms** | **32.9 ms** | | |
| 136 | + | |
| 137 | +Once shader sources were embedded in the metallib, `gpudebug` showed it directly: | |
| 138 | + | |
| 139 | +| kernel | temp registers | **spilled bytes** | | |
| 140 | +|:--|--:|--:| | |
| 141 | +| flash forward, scalar | 126 | **368** | | |
| 142 | +| flash forward, MMA | 85 | **0** ✅ | | |
| 143 | +| flash backward dKV, fused | 111 | **4352** ❌ | | |
| 144 | + | |
| 145 | +That 4352-byte spill was a bug we hadn't suspected. Splitting the kernel recovered | |
| 146 | +another 27%. | |
| 147 | + | |
| 148 | +### 4️⃣ The M5 neural accelerators are worth 4.9× — and f16 is the key | |
| 149 | + | |
| 150 | +`mpp::tensor_ops::matmul2d` (Metal 4 cooperative tensors) vs. our hand-written kernel: | |
| 151 | + | |
| 152 | +| shape | `simdgroup` f32 | `simdgroup` f16 | **`matmul2d` f32** | **`matmul2d` f16** | | |
| 153 | +|:--|--:|--:|--:|--:| | |
| 154 | +| 2048³ | 9.3 T | 10.6 T | 14.9 T | **51.5 T** | | |
| 155 | +| 4096³ | 9.9 T | 11.8 T | 14.6 T | **44.3 T** | | |
| 156 | + | |
| 157 | +Verified numerically, not just timed: f32 is **bit-exact** vs the CPU reference, f16 | |
| 158 | +differs by 3.8e-06, all outputs non-zero, and the transposed variants training needs | |
| 159 | +(`nt`, `tn`) are bit-exact too. | |
| 160 | + | |
| 161 | +**The negative result matters as much:** f16 buys only **+18–22%** on the | |
| 162 | +`simdgroup_matrix` path. So mixed precision looked like a memory-only feature — until | |
| 163 | +it turned out to be the *entry condition* for a 4.9× path. | |
| 164 | + | |
| 165 | +--- | |
| 166 | + | |
| 167 | +## 🚀 Quick start | |
| 168 | + | |
| 169 | +```bash | |
| 170 | +# Build (macOS + Xcode CLT + CMake ≥ 3.24) | |
| 171 | +cmake -B build && cmake --build build -j | |
| 172 | +cd build && ctest --output-on-failure # parity + gradcheck + overfit + tokenizer | |
| 173 | + | |
| 174 | +# Data: download TinyStories, train a BPE vocab, tokenize to uint16 .bin | |
| 175 | +python3 tools/prepare_data.py --out data/tinystories --vocab-size 4096 | |
| 176 | + | |
| 177 | +# Train · generate · eval | |
| 178 | +./build/forge train --config configs/gpt-10m-1epoch.json --data data/tinystories --out runs/exp1 | |
| 179 | +./build/forge generate --checkpoint runs/exp1/ckpt_latest.bin \ | |
| 180 | + --tokenizer data/tinystories/tok4096.model \ | |
| 181 | + --prompt "Once upon a time" --temp 0.8 --top-k 40 | |
| 182 | +./build/forge eval --checkpoint runs/exp1/ckpt_latest.bin --data data/tinystories/val.bin | |
| 183 | +./build/forge info --config configs/gpt-200m.json | |
| 184 | +``` | |
| 185 | + | |
| 186 | +Add `--backend cpu` to run the same model through the CPU reference path — every op is | |
| 187 | +bit-comparable with the GPU path, which is what the parity suite checks. | |
| 188 | + | |
| 189 | +--- | |
| 190 | + | |
| 191 | +## 📊 Benchmarks | |
| 192 | + | |
| 193 | +**GEMM** (`tests/bench_matmul.cpp`, TFLOPS, f32): | |
| 194 | + | |
| 195 | +| kernel | naive | 16×16 tiled | `simdgroup_matrix` | | |
| 196 | +|:--|--:|--:|--:| | |
| 197 | +| 4096³ | 1.29 | 2.53 | **9.49** | | |
| 198 | +| forward MLP `X·W₁ᵀ` | 1.46 | 2.53 | **10.68** | | |
| 199 | +| backward `dX = dY·W` | 1.51 | 2.54 | **10.58** | | |
| 200 | +| backward `dW = dYᵀ·X` | 0.76 | 2.15 | 4.69 ⚠️ | | |
| 201 | + | |
| 202 | +⚠️ `dW` lags — `K = B·T` is huge with few threadgroups. Split-K would fix it (not done). | |
| 203 | + | |
| 204 | +**Scale** — all configs train on one M5 Max: | |
| 205 | + | |
| 206 | +| config | params | context | tok/s | | |
| 207 | +|:--|--:|--:|--:| | |
| 208 | +| `gpt-10m` | 12.2M | 512 | **38.2k** | | |
| 209 | +| `gpt-25m` | 29.9M | 1024 | 22.1k | | |
| 210 | +| `gpt-100m` | 97.5M | 1024 | 9.7k | | |
| 211 | +| `gpt-200m` | 205.5M | 1024 | 1.8k† | | |
| 212 | + | |
| 213 | +† measured before the MMA backward landed — pessimistic. | |
| 214 | + | |
| 215 | +--- | |
| 216 | + | |
| 217 | +## 🏗️ Architecture | |
| 218 | + | |
| 219 | +| path | contents | | |
| 220 | +|:--|:--| | |
| 221 | +| `src/core/` | `Tensor` (shared-storage views), `Allocator` (bucketed MTLBuffer pool), `Device` (queue + pipeline cache), autograd tape | | |
| 222 | +| `src/kernels/` | 12 `.metal` files: GEMM (naive→tiled→simdgroup→`matmul2d`), flash attention (scalar + MMA), softmax, norms, elementwise, embedding, cross-entropy, AdamW | | |
| 223 | +| `src/ops/` | `cpu/` reference impls · `metal/` dispatch + batched `Stream` · `ops.cpp` autograd layer routing to either backend | | |
| 224 | +| `src/nn/` | `Module`, `Linear` (pruning-mask + quantization seams), attention (GQA + RoPE behind an interface), SwiGLU/GELU MLP, `Transformer` | | |
| 225 | +| `src/train/` | mmap dataloader, AdamW, warmup+cosine schedule, trainer, resumable checkpoints | | |
| 226 | +| `src/tokenizer/` | byte-level BPE, verified identical to the Python encoder | | |
| 227 | +| `tests/` | parity · gradcheck · overfit · tokenizer · 3 benchmarks | | |
| 228 | + | |
| 229 | +**Model:** decoder-only transformer · RMSNorm or LayerNorm · SwiGLU or GELU · RoPE | |
| 230 | +(interleaved pairs) or learned positions · GQA · tied embeddings — all from config. | |
| 231 | + | |
| 232 | +**Verified:** 85 CPU↔Metal parity checks (≤1e-4, most bit-exact) · numerical gradient | |
| 233 | +checks on every parameterized op and a full transformer · single-batch overfit to | |
| 234 | +loss < 0.05 in 86 steps · exact checkpoint resume · BPE round-trip vs Python. | |
| 235 | + | |
| 236 | +--- | |
| 237 | + | |
| 238 | +## 🗺️ Roadmap | |
| 239 | + | |
| 240 | +- [ ] **Mixed precision** (f16/bf16 + f32 master weights + loss scaling) — now the top | |
| 241 | + priority, since it gates the 4.9× `matmul2d` path | |
| 242 | +- [ ] **Wire `matmul2d` into `ops::matmul`** behind runtime macOS-26 detection | |
| 243 | +- [ ] **Activation checkpointing** — activation memory, not parameter count, bounds | |
| 244 | + model size today | |
| 245 | +- [ ] **Split-K** for the `dW = dYᵀ·X` GEMM (4.7 vs 10.6 TFLOPS) | |
| 246 | +- [ ] **KV cache** for generation (currently recomputes full context per token) | |
| 247 | +- [ ] Attention tile sweep — BQ/BK unswept; softmax leaves 96/128 threads idle | |
| 248 | + | |
| 249 | +--- | |
| 250 | + | |
| 251 | +## 👤 Author | |
| 252 | + | |
| 253 | +<div align="center"> | |
| 254 | + | |
| 255 | +**Simon-Pierre Boucher** | |
| 256 | + | |
| 257 | +[](mailto:contact@spboucher.ai) | |
| 258 | +[](https://github.com/spboucher-ai) | |
| 259 | + | |
| 260 | +</div> | |
| 261 | + | |
| 262 | +--- | |
| 263 | + | |
| 264 | +## 🙏 Acknowledgements | |
| 265 | + | |
| 266 | +GEMM structure follows **MLX**'s STEEL kernels. Attention follows **FlashAttention-2** | |
| 267 | +as adapted to Apple GPUs by **metal-flash-attention**. Training-loop details — AdamW | |
| 268 | +with ε outside the sqrt, weight decay on rank-≥2 params only, clip folded into the | |
| 269 | +optimizer's gradient read, fused classifier writing logit gradients in place — follow | |
| 270 | +**llm.c** and **nanoGPT**. Built on Apple's **metal-cpp**. | |
| 271 | + | |
| 272 | +<div align="center"> | |
| 273 | +<br/> | |
| 274 | +<sub>Built from scratch on Apple Silicon. Measured, not assumed.</sub> | |
| 275 | +</div> | |
added
RESEARCH.md
+343 −0
@@ -0,0 +1,343 @@ | ||
| 1 | +<!-- Author: Simon-Pierre Boucher — contact@spboucher.ai --> | |
| 2 | + | |
| 3 | +# RESEARCH.md — Forge: Metal/C++ LLM Training Framework | |
| 4 | + | |
| 5 | +Research phase completed 2026-07-31 (6 parallel web-research passes + local toolchain probes). | |
| 6 | +This file records the API signatures, tile sizes, design decisions, and pitfalls that gate | |
| 7 | +implementation. **Update this file whenever new research is done during the project.** | |
| 8 | + | |
| 9 | +--- | |
| 10 | + | |
| 11 | +## 0. Local environment (verified empirically, 2026-07-31) | |
| 12 | + | |
| 13 | +| Item | Value | | |
| 14 | +|---|---| | |
| 15 | +| Dev machine | Apple **M5 Max**, 40-core GPU, macOS **27.0** (build 26A5388g) | | |
| 16 | +| Primary deploy target | **M3 Ultra** (Apple9 family), 80-core GPU, ~28.3 TFLOPS FP32/FP16, 819 GB/s, 96 GB unified | | |
| 17 | +| Xcode | 26.6 (build 17F113), Apple clang 21.0.0 | | |
| 18 | +| Metal compiler | `xcrun -sdk macosx metal` — accepts `-std=metal3.0/3.1/3.2/4.0` (no 3.3); default compiles fine | | |
| 19 | +| CMake | 4.3.2 | | |
| 20 | +| Local probes (compiled OK from `-std=metal3.1` up) | `bfloat`, `simdgroup_bfloat8x8`, `simdgroup_half8x8` → `simdgroup_float8x8` mixed MMA, device `atomic_float` `fetch_add` | | |
| 21 | +| Local reference source | MLX checkout at `~/Desktop/other/mlx` (incl. vendored metal-cpp under `build/DerivedData/.../mlx-swift/Source/Cmlx/metal-cpp`) | | |
| 22 | + | |
| 23 | +**Decision: compile kernels with `-std=metal3.2`** (macOS 15+ baseline: full simdgroup_matrix, | |
| 24 | +bfloat, atomic_thread_fence) and use the **classic Metal 3 compute API** via metal-cpp. Metal 4 | |
| 25 | +(`MTL4*`, MTLTensor, cooperative tensors) is a parallel opt-in API — a future seam, not a dependency. | |
| 26 | + | |
| 27 | +--- | |
| 28 | + | |
| 29 | +## 1. metal-cpp — setup and memory management | |
| 30 | + | |
| 31 | +Source: https://github.com/apple/metal-cpp (header-only, C++17+; releases track OS — use the macOS 26.x/27 release). Vendor into `third_party/metal-cpp`. | |
| 32 | + | |
| 33 | +In exactly **one** .cpp file: | |
| 34 | +```cpp | |
| 35 | +#define NS_PRIVATE_IMPLEMENTATION | |
| 36 | +#define CA_PRIVATE_IMPLEMENTATION | |
| 37 | +#define MTL_PRIVATE_IMPLEMENTATION | |
| 38 | +#include <Foundation/Foundation.hpp> | |
| 39 | +#include <Metal/Metal.hpp> | |
| 40 | +#include <QuartzCore/QuartzCore.hpp> | |
| 41 | +``` | |
| 42 | +Link frameworks: **Foundation, Metal, QuartzCore** (MetalKit not needed for compute). | |
| 43 | + | |
| 44 | +**Ownership (Cocoa rules, no ARC):** | |
| 45 | +- `new*` / `alloc` / `copy` / `Create*` return **retained** objects → caller releases. That covers `MTL::CreateSystemDefaultDevice()`, `newCommandQueue()`, `newBuffer()`, `newLibrary()`, `newFunction()`, `newComputePipelineState()`. | |
| 46 | +- Everything else is **autoreleased** — notably `queue->commandBuffer()` and `cmdBuf->computeCommandEncoder()`. They die when the enclosing `NS::AutoreleasePool` drains. | |
| 47 | +- Smart pointers: `NS::TransferPtr(p)` adopts retained results (no extra retain); `NS::RetainPtr(p)` retains autoreleased/borrowed objects. | |
| 48 | +- **Wrap each training step (or N steps) in an `NS::AutoreleasePool`** (`alloc()->init()` … `drain()`), and one in `main()` — otherwise autoreleased command buffers accumulate unboundedly. Debug with `OBJC_DEBUG_MISSING_POOLS=YES`. | |
| 49 | + | |
| 50 | +**Key compute API (verified against metal-cpp headers):** | |
| 51 | +```cpp | |
| 52 | +MTL::Device* MTL::CreateSystemDefaultDevice(); | |
| 53 | +CommandQueue* Device::newCommandQueue(); | |
| 54 | +Buffer* Device::newBuffer(NS::UInteger length, MTL::ResourceOptions); // MTL::ResourceStorageModeShared | |
| 55 | +Buffer* Device::newBuffer(const void* ptr, NS::UInteger len, MTL::ResourceOptions, void(^dealloc)(void*, NS::UInteger)); // NoCopy: page-aligned | |
| 56 | +Library* Device::newLibrary(const NS::String* filepath, NS::Error** err); // load .metallib by path (CLI tool: do NOT rely on newDefaultLibrary) | |
| 57 | +Function* Library::newFunction(const NS::String* name, const MTL::FunctionConstantValues*, NS::Error**); | |
| 58 | +ComputePipelineState* Device::newComputePipelineState(const MTL::Function*, NS::Error**); | |
| 59 | +ComputeCommandEncoder* CommandBuffer::computeCommandEncoder(MTL::DispatchType); // Serial | Concurrent | |
| 60 | +void Encoder::setComputePipelineState(const MTL::ComputePipelineState*); | |
| 61 | +void Encoder::setBuffer(const MTL::Buffer*, NS::UInteger offset, NS::UInteger index); | |
| 62 | +void Encoder::setBytes(const void* bytes, NS::UInteger length, NS::UInteger index); // ≤ ~4 KB, for param structs | |
| 63 | +void Encoder::setThreadgroupMemoryLength(NS::UInteger length, NS::UInteger index); | |
| 64 | +void Encoder::dispatchThreads(MTL::Size grid, MTL::Size tg); // non-uniform TGs, supported on all M-series — prefer for elementwise | |
| 65 | +void Encoder::dispatchThreadgroups(MTL::Size tgs, MTL::Size tg); // for tiled kernels | |
| 66 | +void CommandBuffer::commit(); addCompletedHandler(const MTL::HandlerFunction&); // std::function<void(CommandBuffer*)> | |
| 67 | +CFTimeInterval CommandBuffer::GPUStartTime() / GPUEndTime(); // per-command-buffer timing, read in completed handler | |
| 68 | +NS::UInteger ComputePipelineState::maxTotalThreadsPerThreadgroup(); // can be < 1024 (register pressure) — always query | |
| 69 | +NS::UInteger ComputePipelineState::threadExecutionWidth(); // 32 on Apple GPUs | |
| 70 | +``` | |
| 71 | + | |
| 72 | +--- | |
| 73 | + | |
| 74 | +## 2. Build system: .metal → .metallib in CMake | |
| 75 | + | |
| 76 | +Two-step pipeline as CMake custom commands (Ninja/Makefiles generator): | |
| 77 | +``` | |
| 78 | +xcrun -sdk macosx metal -std=metal3.2 -O2 [-gline-tables-only -frecord-sources in Debug] -c foo.metal -o foo.air | |
| 79 | +xcrun -sdk macosx metallib foo.air bar.air ... -o forge.metallib | |
| 80 | +``` | |
| 81 | +Load at runtime with `device->newLibrary(path, &err)`. Fast math is **ON by default** | |
| 82 | +(`-fmetal-math-mode=fast`: assumes no NaN/INF, reassociation, auto-FMA contraction). Keep it on | |
| 83 | +for throughput but use `metal::precise::` selectively in numerically sensitive spots and account | |
| 84 | +for FMA contraction in the 1e-4 CPU-parity tests. | |
| 85 | + | |
| 86 | +--- | |
| 87 | + | |
| 88 | +## 3. MSL essentials (from MSL Specification v4.1, 2026-06-04) | |
| 89 | + | |
| 90 | +Prelude for all kernels: `#include <metal_stdlib>` + `using namespace metal;` (umbrella header | |
| 91 | +covers simdgroup, simdgroup_matrix, atomics, compute barriers). | |
| 92 | + | |
| 93 | +### simdgroup_matrix (the perf-critical path) | |
| 94 | +- Types: `simdgroup_half8x8`, `simdgroup_bfloat8x8` (MSL 3.1+), `simdgroup_float8x8`. **8×8 is the only shape.** Available on Apple7+ (all M-series). Each thread holds 2 elements (`vec<T,2>` via `.thread_elements()`). | |
| 95 | +- Signatures (verbatim; threadgroup and device overloads): | |
| 96 | +```metal | |
| 97 | +void simdgroup_load (thread simdgroup_matrix<T,8,8>& d, const {threadgroup|device} T* src, | |
| 98 | + ulong elements_per_row = 8, ulong2 matrix_origin = 0, bool transpose_matrix = false); | |
| 99 | +void simdgroup_store(thread simdgroup_matrix<T,8,8> a, {threadgroup|device} T* dst, | |
| 100 | + ulong elements_per_row = 8, ulong2 matrix_origin = 0, bool transpose_matrix = false); | |
| 101 | +void simdgroup_multiply_accumulate(d, a, b, c); // d = a*b + c | |
| 102 | +simdgroup_matrix<T,8,8> make_filled_simdgroup_matrix<T,8,8>(T value); | |
| 103 | +``` | |
| 104 | +- **Mixed precision**: the spec documents only same-T operands, but **half×half→float accumulate compiles and is production-stable** (llama.cpp `kernel_mul_mm`: `simdgroup_half8x8` inputs, `simdgroup_float8x8` accumulators; confirmed compiling locally). MLX instead converts inputs to float fragments at load and runs all-float MMA (spec-clean). **Decision: MLX-style all-float fragments first (correctness), llama.cpp-style mixed MMA behind a compile-time switch (perf).** | |
| 105 | +- All simdgroup_matrix ops require **uniform SIMD-group control flow** — UB otherwise. | |
| 106 | + | |
| 107 | +### SIMD intrinsics (Apple7+ hardware reductions — all M-series) | |
| 108 | +`simd_sum, simd_product, simd_max, simd_min, simd_prefix_exclusive_sum` — one instruction, result broadcast to active lanes. `simd_shuffle(_up/_down/_rotate)`, `simd_broadcast(_first)`, `simd_ballot`, `simd_all/any`. SIMD width = **32**. Caveats: `simd_shuffle_xor` is **integer-only**; **bfloat is excluded** from all `simd_*` intrinsics; reductions cover *active* threads only. | |
| 109 | + | |
| 110 | +### Threadgroup memory & barriers | |
| 111 | +- Max **32 KB** per threadgroup (API limit; all Apple Silicon). Max 1024 threads/TG (but query the pipeline's `maxTotalThreadsPerThreadgroup`). | |
| 112 | +- Declare tiles inside the kernel (`threadgroup half As[...];`) or as `[[threadgroup(n)]]` arg (runtime-sized). | |
| 113 | +- `threadgroup_barrier(mem_flags::mem_threadgroup)` around shared-tile writes; `simdgroup_barrier(mem_flags::mem_none)` as a cheap scheduling fence between `simdgroup_load` batches and MMA (llama.cpp/MLX idiom). | |
| 114 | + | |
| 115 | +### Atomics | |
| 116 | +`atomic_float` (device memory only, Metal 3+): load/store/exchange/CAS/**fetch_add/sub** only, `memory_order_relaxed` only (below Metal 4.1). No float min/max atomics. Gradient accumulation via atomics is legal but nondeterministic → **use deterministic reduction trees / two-kernel splits instead** (required for deterministic mode anyway). | |
| 117 | + | |
| 118 | +### Numerics | |
| 119 | +- `bfloat`: MSL 3.1+, truncated f32; NOT implicitly convertible to/from half; no simd intrinsic support. Fine as storage + convert-to-float compute. | |
| 120 | +- `fast::exp` error grows as `3 + floor(|2x|)` ulp → **always subtract the row max before exp**. MLX folds `scale * M_LOG2E_F` and uses `fast::exp2` (exp2 is the HW primitive) — adopt this. | |
| 121 | +- `fma()` always correctly rounded; compiler auto-contracts `a*b+c` — a source of CPU/GPU last-ulp differences (tolerances absorb it). | |
| 122 | + | |
| 123 | +--- | |
| 124 | + | |
| 125 | +## 4. Execution model & performance practices (Apple Silicon / Apple9) | |
| 126 | + | |
| 127 | +Architecture facts: 128 ALUs/core (4×32-wide), FP16 and FP32 FMA at the **same** 128 FMA/core/cycle rate — f16 wins via registers/bandwidth and (family 9) FP16∥FP32∥INT dual-issue "up to 2× ALU". M3+ **dynamic caching**: registers/threadgroup/stack share unified on-chip caches; register count no longer sets occupancy; HW auto-throttles occupancy. Consequences: (a) register-heavy small-K GEMM configs win on M3+ (see §5); (b) **threadgroup memory used as a mere read-cache of device data can be a pessimization on family 9** — use it for reuse + layout transformation only; (c) async copies (`simdgroup_event`) help M1/M2 but **hurt M3+** — don't use them. | |
| 128 | + | |
| 129 | +**Command submission (the rules Forge's device layer enforces):** | |
| 130 | +1. One `MTLCommandQueue`. Batch the whole training step (fwd+bwd+optimizer, 100s of dispatches) into **one command buffer**, few **serial** compute encoders. Within a serial encoder, dispatch N+1 sees dispatch N's writes — no barriers needed (`memoryBarrier` is ignored on serial encoders). MLX batches all ops of an eval this way; Candle measured ~13× per-dispatch overhead win. | |
| 131 | +2. **Never `waitUntilCompleted` per-op.** Sync at loss readback only: `addCompletedHandler` + semaphore/`MTLSharedEvent`, keep 1–2 steps in flight, double-buffer the batch-input/loss buffers. | |
| 132 | +3. `MTLStorageModeShared` everywhere. Coherency is defined **at command-buffer boundaries**: CPU writes before `commit` are visible; GPU writes are readable after completion. Never CPU-write a buffer an in-flight command buffer may touch. | |
| 133 | +4. **Buffer pool**: `newBuffer` is a kernel VM allocation — pool by size class, recycle **only after** the referencing command buffer completes (MLX issue #3689 use-after-free class). Suballocate at **256-byte alignment** (always valid; setBuffer offset needs 4 B on Apple GPUs but 256 is free insurance and vector-load friendly). | |
| 134 | +5. Dataloader: `mmap` the token file and wrap with `newBuffer(bytesNoCopy:)` — requires **16384-byte page** alignment/multiple (Apple Silicon page size). True zero-copy. | |
| 135 | +6. Threadgroup sizing: multiples of 32; 128–256 threads for GEMM tiles; 256–1024 for row reductions (simd_sum first, tiny cross-simd threadgroup step second — shuffle bandwidth is 2× threadgroup-memory bandwidth); vectorize memory-bound kernels with `float4`/`half4` (elementwise/AdamW/norm are pure-bandwidth, target ~800 GB/s). | |
| 136 | +7. Specialize kernels with **function constants** (`MTLFunctionConstantValues`) — alignment flags, has_bias, activation type, tile params — and cache pipelines by (kernel name + constants). No runtime branches for these. | |
| 137 | +8. Profiling: `GPUStartTime/GPUEndTime` per command buffer always-on for tokens/sec; `MTLCounterSampleBuffer` (timestamp counter set, encoder-boundary sampling — check `supportsCounterSampling`) for per-kernel tuning; `MTL_DEBUG_LAYER=1` in tests. | |
| 138 | + | |
| 139 | +--- | |
| 140 | + | |
| 141 | +## 5. GEMM design (chosen tiles + justification) | |
| 142 | + | |
| 143 | +Reference implementations studied: MLX STEEL (`mlx/backend/metal/kernels/steel/gemm/`), llama.cpp `kernel_mul_mm`, Philip Turner's metal-flash-attention GEMM. | |
| 144 | + | |
| 145 | +**Structure (STEEL pattern):** `BlockLoader` stages device→threadgroup with one 16-byte vector read per thread per step, **+16 bytes row padding** against bank conflicts (`tgp_padding = 16/sizeof(T)`); tiles consumed as 8×8 `simdgroup` fragments, K-inner loop in steps of 8 with `simdgroup_barrier(mem_none)` fences; **f32 accumulator fragments always**; single-buffered tiles (no double buffering — multiple resident TGs hide latency); epilogue → predicated `store_result_safe` on edges. | |
| 146 | + | |
| 147 | +**Tile configs (starting points, to be autotuned on target):** | |
| 148 | + | |
| 149 | +| Case | BM×BN×BK | WM×WN (simdgroups) | Source/justification | | |
| 150 | +|---|---|---|---| | |
| 151 | +| f16/bf16 default (large M·N) | 64×64×16 | 1×2 (64 thr) … 2×2 (128 thr) | MLX large-device table | | |
| 152 | +| f16 NT (fwd `X·Wᵀ`) | 64×32×32 | 2×2 | MLX nt case | | |
| 153 | +| f16 NN (bwd `dX = dY·W`) | 64×64×16 or 32×64×16 | 1×2 | MLX nn cases | | |
| 154 | +| f32 | 32×64×16 (nt) / 64×32×32 (nn) | 1×2 / 2×2 | MLX f32 table | | |
| 155 | +| M3+ alternative to benchmark | 32×32×8, 1 simdgroup/TG | 1×1 | MFA Apple9 finding: dynamic caching favors register-heavy small-K | | |
| 156 | + | |
| 157 | +- **Transpose variants**: one templated kernel with `transpose_a/transpose_b` bools → nn/nt/tn/tt instantiations; detect stride-1 dim (MLX `check_transpose`), never materialize transposes. Training needs **nt (forward), nn (dX), tn (dW = dYᵀ·X)**. | |
| 158 | +- **Edges**: function constants `align_M/align_N/align_K` at pipeline creation; K-remainder tile handled FIRST with predicated `load_safe` (zero-fill OOB), then aligned main loop; per-element predicated fragment stores. No separate edge kernels. | |
| 159 | +- **Split-K** for dW-shaped GEMMs (small M,N, huge K = B·T): MLX routing `tiles(M,16)*tiles(N,16) ≤ 2048 (Ultra) && K/16 ≥ 8 && K ≥ max(M,N)`; ≤32 partitions accumulate into f32 scratch `[partitions, M, N]` + tiny reduce kernel. No atomics. | |
| 160 | +- **GEMV** for generation (min(M,N)=1): rows-per-simdgroup, float4 dots, `simd_shuffle_down` tree / `simd_sum` (llama.cpp `kernel_mul_mv` pattern). | |
| 161 | +- Grid swizzle for large matrices (`swizzle_log` 1–2), serpentine fragment iteration for register reuse. | |
| 162 | +- **Expected perf**: MFA/STEEL-class kernels hit 80–93% ALU on M-series → ~23–26 TFLOPS f16 on M3 Ultra; naive tiled non-simdgroup ≈ 35–40% — hence the roadmap naive → tiled → simdgroup. | |
| 163 | +- **Operand precision on the simdgroup path is a small win** (`tests/bench_precision.cpp`, M5 Max, f32 accumulator throughout): f16 and bf16 operands give **+18–22%** over f32 (e.g. 9.9 → 12.4 TFLOPs), not the 2× one might expect from Apple's "2× ALU on family 9" framing. This matches metal-benchmarks' claim that f16 and f32 FMA run at the same rate and the gain comes from register/bandwidth pressure. Mixed precision on *this* path is therefore a memory feature. Through MPP (§5b) the same operands are worth 4.9×. | |
| 164 | +- **Measured (this repo, `tests/bench_matmul.cpp`, M5 Max 40-core, f32)**: naive 1.3–1.5 · 16×16-tiled 2.5 · simdgroup 9.5–10.8 TFLOPs. The `nt` forward shapes and `nn` dX shape all land ≈10.6; `tn` (dW = dYᵀ·X, K = B·T = 65536) lags at 4.7 — few threadgroups × very long K, which is exactly the split-K case (MLX routes it that way; not yet implemented here). | |
| 165 | +- llama.cpp codegen gotcha: `sa[64*ib + ...]` (subscript) was "massively slower" than `*(sa + 64*ib + ...)` (pointer deref) — watch for it. | |
| 166 | + | |
| 167 | +--- | |
| 168 | + | |
| 169 | +## 5b. Metal Performance Primitives (`matmul2d`) — measured 4.9× over hand-written MMA | |
| 170 | + | |
| 171 | +Probed and benchmarked locally 2026-07-31 (`src/kernels/matmul_mpp.metal`, | |
| 172 | +`tests/bench_precision.cpp`, `tests/mppcheck.cpp`). **This is the largest single | |
| 173 | +efficiency finding for this project and it changes the roadmap.** | |
| 174 | + | |
| 175 | +`MetalPerformancePrimitives.framework` ships in the macOS 26.5 SDK | |
| 176 | +(`Headers/MPPTensorOpsMatMul2d.h`, 642 lines) and compiles under `-std=metal4.0`. | |
| 177 | +`mpp::tensor_ops::matmul2d` is the cooperative-tensor matmul that targets the | |
| 178 | +per-core **neural accelerators** on M5-class hardware. | |
| 179 | + | |
| 180 | +Measured on M5 Max, 64×32 tile, 4 simdgroups, f32 accumulate, versus this repo's | |
| 181 | +own simdgroup_matrix GEMM at identical shapes: | |
| 182 | + | |
| 183 | +| shape | simdgroup f32 | simdgroup f16 | **MPP f32** | **MPP f16** | | |
| 184 | +|---|---|---|---|---| | |
| 185 | +| 2048³ | 9.3 T | 10.6 T | **14.9 T** | **51.6 T** | | |
| 186 | +| 4096³ | 9.9 T | 11.8 T | **14.1 T** | **44.1 T** | | |
| 187 | +| 65536×512×1408 | 9.9 T | 12.4 T | **15.3 T** | **22.8 T** | | |
| 188 | + | |
| 189 | +So **1.5× for f32 and 4.3–4.9× for f16**. Both verified numerically against the CPU | |
| 190 | +reference (`tests/mppcheck.cpp`): f32 is bit-exact, f16 is 3.8e-06 against a | |
| 191 | +reference fed the same rounded inputs, and every output element is nonzero (i.e. the | |
| 192 | +full K reduction really happens). | |
| 193 | + | |
| 194 | +Note this contradicts the "Rigel" paper's M4 Max finding that `matmul2d` still runs on | |
| 195 | +the shader cores and loses to a hand-fused GEMM — M5 evidently has the accelerator | |
| 196 | +hardware that M4 lacked. Re-measure per generation; do not assume. | |
| 197 | + | |
| 198 | +**API notes that cost time to work out (the header's own example is stale):** | |
| 199 | +- Bind ordinary buffers and build tensors *in-kernel* with the `tensor_inline` | |
| 200 | + descriptor: `tensor<device T, dextents<int32_t,2>, tensor_inline> A(ptr, dextents<int32_t,2>(cols, rows))`. The default `tensor_handle` descriptor wraps an opaque handle that only a host-side `MTLTensor` can supply, so `tensor_inline` avoids all MTLTensor plumbing. | |
| 201 | +- Extents are **(columns, rows)**, and `slice(x, y)` takes column-then-row. | |
| 202 | +- Element type must be **non-const** and one of `uint8_t/int8_t/uint4b_format/int4b_format/float/half/bfloat` — a `const float` tensor fails a `static_assert`. | |
| 203 | +- The destination is a `cooperative_tensor`; zero it via `get_capacity()` + | |
| 204 | + **`is_valid_element(i)`**. The header comment says `get_mask(i)`, which does not | |
| 205 | + exist in this SDK. | |
| 206 | +- `#pragma unroll full` is not valid MSL here; use `#pragma clang loop unroll(full)`. | |
| 207 | +- Requires `-std=metal4.0`, so it is a per-file build flag (CMake keys off the | |
| 208 | + `_mpp` filename suffix) and macOS 26+; the simdgroup_matrix path stays the | |
| 209 | + portable fallback for macOS 15. | |
| 210 | + | |
| 211 | +**Transposes work, so this covers training, not just inference.** `transpose_left` / | |
| 212 | +`transpose_right` are descriptor fields (hence template parameters), and nn / nt / tn | |
| 213 | +are all bit-exact against the CPU reference — i.e. the forward `X·Wᵀ`, the `dX = dY·W` | |
| 214 | +and the `dW = dYᵀ·X` of the backward are all served. Two gotchas: the operand extents | |
| 215 | +must be swapped to match the transpose flag, and `slice()` is (column, row) so a | |
| 216 | +transposed operand is sliced the other way round. Also `op.template | |
| 217 | +get_destination_cooperative_tensor<...>()` once the kernel is itself a template. | |
| 218 | + | |
| 219 | +**Implication for the roadmap.** Mixed precision was previously judged a | |
| 220 | +memory-only win (f16 operands buy just +18–22% on the simdgroup path — measured, | |
| 221 | +see below). Through MPP the same f16 operands are worth **4.9×**. That reverses | |
| 222 | +the priority: f16/bf16 plumbing is now the highest-value remaining work, because | |
| 223 | +it is the gate to the accelerator path, not merely a way to halve activations. | |
| 224 | + | |
| 225 | +## 6. Fused attention design (flash-style) | |
| 226 | + | |
| 227 | +Algorithm: FlashAttention-2 (arXiv 2307.08691). Online softmax per Q-block over KV-blocks j: | |
| 228 | +``` | |
| 229 | +m_new = max(m_old, rowmax(S_j)) S_j = (Q Kᵀ) * scale (f32) | |
| 230 | +P~ = exp2(S_j - m_new) (scale folded: scale * M_LOG2E_F, fast::exp2) | |
| 231 | +factor = exp2(m_old - m_new) (≤ 1; the paper's diag(...)^-1 is a known typo) | |
| 232 | +l = l * factor + rowsum(P~) | |
| 233 | +O = O * factor + P~ · V_j (O in f32 registers; divide by l ONCE after the loop) | |
| 234 | +L = m + log(l) (one f32/row, saved for backward) | |
| 235 | +``` | |
| 236 | +**Layout (MLX steel_attention + llama.cpp flash_attn_ext, adapted):** one threadgroup per (Q-block, head, batch); 4 simdgroups, **split-Q** — each simdgroup owns its Q rows so row statistics need only `simd_max`/`simd_sum` (zero cross-warp softmax traffic). Tiles: **BQ=32, BK=32 (BK=16 at head_dim 128), head dims {64, 80, 128}** (MLX); K loaded via `simdgroup_load(..., transpose=true)`; PV matmul non-transposed. m/l/factors/S/O all **f32** even with f16 Q/K/V. K/V threadgroup tiles share one padded allocation. | |
| 237 | + | |
| 238 | +- **Causal masking**: `kb_lim = ceil((q_max)/BK)` skips fully-masked KV blocks entirely; only diagonal-band blocks (`kb >= kb_min_causal`) pay per-element masking. Mask value = **`-FLT_MAX` / finite_min, never -INFINITY** (`exp(-inf - -inf) = NaN`). Guard final divide: `l == 0 → output 0`. | |
| 239 | +- **GQA**: `kv_head = q_head / (n_heads / n_kv_heads)` index mapping — no repeat_kv copies. | |
| 240 | +- **The `D` term is free and worth a lot.** `D_i = rowsum(dP ∘ P)` — needed by both backward kernels — equals `dO_i · O_i` exactly, since `Σ_j P_ij (dO_i · V_j) = dO_i · (Σ_j P_ij V_j) = dO_i · O_i`. Computing it per (i,j) pair instead of once per query row makes the dk/dv kernel O(T³·hd). Measured in this repo: hoisting it into a one-thread-per-row preprocess kernel took a training step from **18.4k to 57.8k tokens/sec (3.1×)** with bit-identical losses, even in the *unfused* attention path. | |
| 241 | +- **Measured (this repo, `tests/bench_attention.cpp`, M5 Max, f32, causal), scalar vs simdgroup_matrix-tiled**, at gpt-10m shapes (B64 T512 H6 hd64) and gpt-25m shapes (B64 T1024 H8): | |
| 242 | + | |
| 243 | +| | fwd gpt-10m | fwd gpt-25m | bwd gpt-10m | bwd gpt-25m | | |
| 244 | +|---|---|---|---|---| | |
| 245 | +| scalar (one thread/row) | 8.0 ms (1.6 TF) | 40.9 ms (1.7 TF) | 107 ms | 561 ms | | |
| 246 | +| MMA-tiled | **2.3 ms (5.7 TF)** | **10.6 ms (6.5 TF)** | 9.8 ms | 45.5 ms | | |
| 247 | +| MMA-tiled, dK/dV split | — | — | **7.05 ms** | **32.9 ms** | | |
| 248 | + | |
| 249 | + So 3.6× on the forward and **10.9–12.3× on the backward**. End-to-end that took gpt-10m training from 20.1k to 35.2k tokens/sec. The backward gains more than the forward because the scalar backward was also spilling (§8 6b) — MMA fixes the arithmetic *and* the register pressure at once, since an 8×8 fragment is 2 floats per lane where the scalar version held 4 arrays of head_dim. | |
| 250 | +- **Ecosystem survey (2026-07-31): almost nobody has a fused attention backward on Metal.** MLX's `ScaledDotProductAttentionVJP::use_fallback` returns `true` unconditionally and `eval_gpu` throws `"NYI"`; its Metal *forward* also refuses the fused path under grad tracing (`if (is_training) return true;`), and its fused VJP is CUDA/cuDNN-only. llama.cpp's Metal backend does not support `GGML_OP_FLASH_ATTN_BACK` (falls to `default: return false`). PyTorch MPS and Candle ship hand-written MSL forwards only. tinygrad's FA backward is AMD-only. Burn/CubeCL has a 3-kernel backward that is still a scalar scaffold (no tiling, no MMA). The only production prior art is philipturner's MFA and the third-party `mlx-mfa` package, plus an unmerged MLX PR (#3241). | |
| 251 | +- **Both production references use exactly the structure implemented here**: (i) `delta = rowsum(dO ∘ O)` precomputed in **fp32** outside the main kernels, (ii) a dQ kernel with grid `(NQ, H, B)`, one Q tile per threadgroup, looping KV and accumulating dQ in registers, (iii) a dKV kernel with grid `(NK, H_kv, B)`, one KV tile per threadgroup, looping Q. **No atomics anywhere.** They additionally work in the log2 domain (`P = exp2(S·scale·log2e − L_log2)`); the elementwise softmax step is ~1/head_dim of the MMA work here, so that is a micro-optimisation at these shapes. | |
| 252 | +- **Their tuned tile configs, and why BK=32 lost here.** MLX PR #3241 and mlx-mfa both tune D=64 dQ to `BQ=32, BK=32, WM=4` (mlx-mfa reports retuning dQ from BK=64 to 32 was worth 4–14%). Tried it: **fwd 2.27 → 3.63 ms, bwd 9.66 → 11.16 ms — worse.** BK=32 takes K+V tiles from 8.7 to 17.4 KB, so with the S/P tile the kernel crosses 22 KB and residency drops from 2 threadgroups per core to 1. Their configs fit because they alias temporally-disjoint threadgroup buffers (PR #3241 notes aliasing a reduction buffer over `Q_smem+dO_smem`: 23,040 → 14,848 bytes, doubling occupancy). **Conclusion: BK=32 is only reachable after implementing that aliasing — the tile size and the memory plan have to be tuned together, not independently.** | |
| 253 | +- **Register pressure is the binding constraint at head_dim 128**, not bandwidth: PR #3241 measures D=128 dKV at BQ=32/WM=2 needing ~338 registers/thread (over the 256 limit, spills) versus ~202 at BQ=16/WM=2. Worth knowing before extending past hd=64. | |
| 254 | +- **A fused backward is not automatically faster.** PR #3241's own numbers on M3 Max: fused beats MLX's unfused VJP only for causal, D≤96 (1.17–1.37×); dense is 0.36–0.70×, i.e. slower, because the backward needs ~2.5× the forward's FLOPs and 32×32 tiles underuse the MMA units next to large-tile GEMMs. The payoff there is memory (95% peak reduction at L=4096). This does not contradict the 10.9× measured here, which compares fused-scalar against fused-MMA — both already avoid the T² tensor. | |
| 255 | +- **Backward (training)**: forward saves `L`. **Key finding: MLX does not use fused attention for training** (fused VJP is `NYI`; comment says unfused is faster on Metal at their configs). Plan: **M4/M5 milestone = fused forward + unfused-composed backward validated vs CPU; stretch = MFA-style fused backward** — two kernels, no atomics: dQ-kernel parallel over Q rows `(3D+5)N²` FMAs, dK/dV-kernel parallel over KV rows `(4D+5)N²` FMAs, both recompute `P = exp(S - L)` per tile, plus a cheap preprocess `D = rowsum(dO ∘ O)`. MFA measured fwd 91–94% ALU on M3, fwd+bwd ~69–71%. | |
| 256 | +- Decode path (`forge generate`): sdpa_vector-style kernel — 1 Q row, 32-thread strided KV walk, scalar online update, split-K two-pass over long KV. | |
| 257 | + | |
| 258 | +--- | |
| 259 | + | |
| 260 | +## 7. Training loop (nanoGPT / llm.c distilled) | |
| 261 | + | |
| 262 | +**Hyperparameters (defaults for configs):** AdamW β=(0.9, 0.95), eps 1e-8, weight_decay 0.1 **only on dim≥2 params** (weights/embeddings; never biases/norm gains); grad clip 1.0 (global norm); linear warmup `lr*(it+1)/(warmup+1)` → cosine to `min_lr = lr/10`; init: normal(0, 0.02), residual projections `0.02/sqrt(2*n_layers)`; tied wte/lm_head (one grad buffer — embedding and head grads accumulate into the same tensor); vocab padded to a multiple of 64 (matmul tile). | |
| 263 | + | |
| 264 | +**AdamW exact (llm.c, PyTorch-compatible):** t is 1-based; | |
| 265 | +``` | |
| 266 | +m = β1·m + (1-β1)·g ; v = β2·v + (1-β2)·g² | |
| 267 | +m̂ = m/(1-β1ᵗ) ; v̂ = v/(1-β2ᵗ) | |
| 268 | +w -= lr · ( m̂/(sqrt(v̂) + eps) + wd·w ) # eps OUTSIDE sqrt; wd decoupled, never through m/v | |
| 269 | +``` | |
| 270 | +Fused kernel: host precomputes bias corrections and `grad_scale = min(1, clip/global_norm) / loss_scale`; kernel reads grad → scales → m/v update → step → writes m, v, f32 master, and f16/bf16 working copy (llm.c uses **stochastic rounding** for bf16 working weights — low 16 mantissa bits vs random threshold — else sub-ulp updates stagnate). | |
| 271 | + | |
| 272 | +**Memory layout (llm.c pattern):** all params in **one flat f32 buffer** with a table of (offset, shape) per tensor; grads/m/v/master as identically-shaped flat buffers → optimizer is one kernel over `num_parameters`, and global-norm is a two-pass reduction (partials per threadgroup → single-TG reduce; fold the scale into the AdamW grad read, no third pass). | |
| 273 | + | |
| 274 | +**Mixed precision:** f32 master weights + f16/bf16 compute copies; grads accumulated in f32 across micro-batches. Loss scaling **for f16 only** (bf16 needs none): init 2^16, halve + **skip step** on inf/nan, double after 2000 clean steps; unscale before clipping. Keep softmax, norms, CE, and all reductions in f32. | |
| 275 | + | |
| 276 | +**Fused cross-entropy (llm.c fused_classifier):** one pass computes online (max, sumexp); `loss = max + log(sumexp) - logit[target]`; gradient `(softmax - onehot)·dloss` written **in place over the logits buffer** (dloss = 1/(B·T·grad_accum)); full softmax never materialized. Support `ignore_index=-1`. | |
| 277 | + | |
| 278 | +**Model formulas:** | |
| 279 | +- RoPE (θ=10000, **interleaved-pairs / GPT-J convention** — simplest kernel, stride-2 pairs; document it; backward = rotation by −mθ): `x'_{2k} = x_{2k}cos(mθ_k) − x_{2k+1}sin(mθ_k)`, `x'_{2k+1} = x_{2k}sin(mθ_k) + x_{2k+1}cos(mθ_k)`, `θ_k = 10000^{-2k/d_head}`; q and k only. | |
| 280 | +- RMSNorm (eps 1e-6): `y = w ⊙ x / sqrt(mean(x²) + eps)`; backward `dx_i = (1/rms)[g_i w_i − x_i·Σ_j(g_j w_j x_j)/(N·rms²)]`, `dw_i = Σ_rows g_i x_i / rms`. | |
| 281 | +- SwiGLU: `W2(silu(xW1) ⊙ xW3)`, `d_ff = round_up(2/3·4·d_model, multiple)`; `silu'(z) = σ(z)(1 + z(1−σ(z)))`. | |
| 282 | +- Data loader: flat uint16 `.bin`, random contiguous windows of `context+1`, y = x shifted by 1. Adopt llm.c's 256-int32 header `{magic 20240520, version 1, num_tokens}`. uint16 caps vocab at 65535 — fine. | |
| 283 | +- Checkpoint: step + RNG state(s) + params + m + v + master weights (+ dataloader position) → exact resume. | |
| 284 | + | |
| 285 | +**Reference points:** TinyStories = `roneneldan/TinyStories`, ~926M train / ~19M val tokens (GPT-2 tok); 4k–8k custom BPE vocab is the sweet spot for 10–50M models (llama2.c: 4096-vocab ≈ 32k-vocab compression on this domain). llama2.c 110M config: lr 4e-4, 131k tokens/step, dropout 0.1. BPE: minbpe algorithm (byte-level, lowest-merge-rank greedy encode); C++ needs a hand-rolled UTF-8 codepoint classifier if we adopt the GPT-2/GPT-4 regex pre-split (std::regex can't do `\p{L}`) — or skip the regex (BasicTokenizer) for small domain vocabs. | |
| 286 | + | |
| 287 | +--- | |
| 288 | + | |
| 289 | +## 8. Consolidated pitfalls checklist | |
| 290 | + | |
| 291 | +**Metal/metal-cpp** | |
| 292 | +1. `NS_PRIVATE_IMPLEMENTATION` in exactly one TU; missing pool around per-step command buffers = leak. | |
| 293 | +2. Coherency only at command-buffer boundaries; never CPU-write in-flight buffers; never recycle a pooled buffer before its command buffer completes. | |
| 294 | +3. `setBytes` ≤ 4 KB; buffer-offset alignment → use 256 B in the allocator; bytesNoCopy needs 16 KB pages. | |
| 295 | +4. Query `maxTotalThreadsPerThreadgroup` per pipeline (register pressure can shrink it below 1024). | |
| 296 | +5. `memoryBarrier` is silently ignored on serial encoders — if we ever flip an encoder to Concurrent, the barriers must already be written. | |
| 297 | +6. Fast math default: no-NaN/no-INF assumptions, x/0 UB, auto-FMA. Subtract row max before every exp; use `precise::` where parity demands. | |
| 298 | + | |
| 299 | +**Kernels** | |
| 300 | +6f. **Register spill is directly measurable, headlessly, and worth checking on every kernel.** Xcode 26 ships `gpudebug`; `gpucapture start --pid <p> --boundary 1 --count N -o t.gputrace`, then `gpudebug -s <id> -c "profile run --gpu-state high"` / `"profile embed"` / `"profile load"`, then `go performance/shaders` + `info comp0` prints **Temp registers**, **Uniform registers** and **Spilled bytes** per kernel. This requires shader sources in the metallib: pass `-frecord-sources` at compile *and* link, and link with `metal`, not `metallib` (which rejects the flag). Measured here — scalar flash forward 126 registers / **368 spilled**; MMA forward 85 / **0** (that is the mechanism behind its 3.6×); MMA dQ 95 / 0; and the fused MMA dKV **111 / 4352 spilled**, a defect found only by this tooling. Splitting dKV into separate dV and dK kernels cut the backward a further 27% (9.66 → 7.05 ms). | |
| 301 | +6a. Hoist `D = rowsum(dO ∘ O)` out of the attention backward inner loops (see §6) — a 3.1× step-time factor, and it applies to unfused attention too. | |
| 302 | +6b. **Per-thread register pressure dominates the attention backward.** A one-thread-per-row backward that keeps `k`, `v`, `dk`, `dv` in registers is 4·head_dim floats — 1 KB/thread at hd=64 — and spills. Measured on gpt-10m (B64 T512 H6), backward per layer: **150 ms** (all four in registers) → **118 ms** (read-only `k`/`v` re-read from device; they are the same address every iteration, so L1 serves them) → **105 ms** (dK and dV split into separate kernels so each thread carries one accumulator). The split recomputes the `q·k` dot in both kernels and still wins. MFA's parameter tables encode the same tradeoff: at hd ≥ 56 it stops caching `K`, at hd ≥ 80 it drops to `dV`/`dK` only. | |
| 303 | +6d. **Applying a per-row scale to an accumulator held in fragments does not require knowing the lane mapping.** Online softmax needs `O <- diag(corr) · O` each KV block, but MSL leaves the element→lane mapping of `simdgroup_matrix` unspecified (MLX reverse-engineers it in `mma.h::get_coord`). Build the 8×8 diagonal in threadgroup memory and apply it with an ordinary MMA instead: spec-clean, costs head_dim/8 extra MMAs per block (~25% more MMA work), and keeps O in registers — staging O in threadgroup memory would have cost 8 KB and halved residency. The backward needs none of this: `L` from the forward already fixes the normalisation. | |
| 304 | +6e. Get S transposed by swapping the operand roles, not by transposing data: the dK/dV kernel wants `S^T`/`dS^T` (rows = KV, cols = query), which is exactly `S^T = K @ Q^T` and `dP^T = V @ dO^T`. `simdgroup_load(..., transpose=true)` supplies the `Q^T`/`dO^T` operands straight from the staged tiles at no cost. | |
| 305 | +6c. Staging K/V through threadgroup memory in the *forward* changed nothing (8.3 → 8.0 ms): the 32 threads of a simdgroup are consecutive query rows of one head, so they hit the same K/V addresses in the same cycle and the cache already broadcasts them. Tile staging pays off for reuse and layout transformation, not for reads that are already uniform across the simdgroup. | |
| 306 | +7. simdgroup_matrix requires uniform control flow; 8×8 only; mixed half→float MMA is undocumented (keep all-float fallback). | |
| 307 | +7a. **`constant constexpr uint TILE = 4` at program scope is NOT a compile-time constant.** In MSL `constant` is an *address-space qualifier* and program-scope variables are required to live there, so that declaration creates a constant-address-space **variable**. Loop bounds derived from it aren't compile-time known, the fragment loops don't unroll, `acc[i][j]` becomes dynamic indexing into an opaque `simdgroup_matrix` array, and the driver spills all 16 accumulators (256 B each) to the stack — every MMA then pays ~1 KB of stack traffic. Measured on the f32 GEMM: **0.82 → 10.2 TFLOPs (12×)** purely from declaring tile geometry as `enum : uint { ... }` plus `#pragma clang loop unroll(full)` on the fragment loops. Use enumerators (or `#define`) for tile constants, never `constant constexpr`. | |
| 308 | +7b. Do not diagnose this from `metal -S -emit-llvm`: at every `-O` level the AIR keeps the loops rolled and shows the same 3 allocas / 2 MMA intrinsics whether or not the fast path is hit. Unrolling and fragment promotion happen in the driver's AIR→ISA backend at pipeline creation, so **benchmark, don't read the IR**. | |
| 309 | +7c. Stage tiles with a transpose-aware thread→address map (walk the operand's contiguous axis). Mapping `idx` by the fixed logical axis leaves the `nt` case — the one the forward pass issues most — strided and uncoalesced. | |
| 310 | +7d. Budget threadgroup memory for occupancy: an early epilogue that staged the whole 64×64 output block cost 27 KB, capping residency at one threadgroup per core. Store one 8×8 fragment at a time through per-simdgroup scratch (1 KB total, simdgroup-local so no threadgroup barrier) → 10.5 KB, ~3 threadgroups resident. | |
| 311 | +8. Never `-INFINITY` masks (NaN via `exp(-inf − -inf)`); guard `l==0 → 0` on fully-masked rows. | |
| 312 | +9. Accumulate f32 always (matmul, softmax stats, norms, CE); f16 accumulators overflow (max 65504). | |
| 313 | +10. `simd_shuffle_xor` int-only; no `simd_*` on bfloat; reductions cover active lanes only. | |
| 314 | +11. No async copies on M3+; no threadgroup-memory-as-pure-read-cache on M3+. | |
| 315 | +12. +16 B row padding on threadgroup tiles; vectorized loads need alignment checks; pointer-deref vs subscript codegen trap. | |
| 316 | + | |
| 317 | +**Training** | |
| 318 | +11a. **Concurrent dispatch is the cheap encoder win, and it needs no Metal 4.** Metal's default compute encoder orders every dispatch against the previous one. The optimizer is the pathological case: one dispatch per parameter tensor (~100 for a 100M model) and a single-threadgroup `sumsq` per tensor, all mutually independent. Wrapping both optimizer passes in a concurrent-dispatch region took gpt-100m from 7.9k to **9.6k tokens/sec (+22%)** with identical losses. Switching encoder kind ends the encoder, and Metal orders tracked resources across encoders in a command buffer, so the switch is itself the barrier — no explicit `memoryBarrier` needed as long as everything inside a region is independent. An independent investigation measured ~15× on a synthetic batch of small independent dispatches and found Metal 4's encoding path offers no CPU-side advantage once invariant binds are hoisted: the concurrency (available since macOS 10.14) is the whole effect. | |
| 319 | +11b. Apple-silicon default math mode is `.relaxed`, which **honours Inf/NaN** — important, because f16 loss-scaling overflow detection depends on it. Setting the deprecated `fastMathEnabled = true` silently selects `.fast`, which does not. Passing no math flags (as here) keeps the safe default. | |
| 320 | +12a. **Buffer recycling is gated on sync, so the sync cadence sets peak memory.** Pooled buffers released while a command buffer is open park on the allocator's retire list and are only reusable after `sync()`. Syncing once per *optimizer* step means none of the gradient-accumulation micro-batches recycle anything, and peak activation memory scales with `grad_accum_steps` — `gpt-100m` at micro-batch 8 OOM'd on a 37 GB working set until the trainer began syncing once per micro-batch (a ~7 GB peak). Cost is one extra GPU sync per micro-batch. | |
| 321 | +12b. Activation memory, not parameter count, bounds model size without checkpointing: ~490 MB per layer at batch 8 × ctx 1024 × d_model 768, times 12 layers. | |
| 322 | +13. AdamW: eps outside sqrt, 1-based t, wd decoupled and dim≥2 only, clip before m/v (fold into grad read). | |
| 323 | +14. Warmup `(it+1)/(warmup+1)` (no lr=0 step); loss ÷ grad_accum inside micro-loop. | |
| 324 | +15. GradScaler: unscale before clip; skip step on inf/nan; f16 only. | |
| 325 | +16. bf16 working weights need f32 master or stochastic rounding. | |
| 326 | +17. RoPE convention (interleaved vs rotate_half) must be consistent everywhere — Forge uses interleaved (GPT-J). | |
| 327 | +18. Tied embeddings: single grad buffer for wte/lm_head. | |
| 328 | +19. CE in-place logit-gradient needs a sync between loss read and grad overwrite. | |
| 329 | +20. Deterministic mode: no atomic-add gradient accumulation — reduction trees / two-kernel splits. | |
| 330 | + | |
| 331 | +--- | |
| 332 | + | |
| 333 | +## 9. Sources | |
| 334 | + | |
| 335 | +**Apple official**: [MSL Spec v4.1 PDF](https://developer.apple.com/metal/Metal-Shading-Language-Specification.pdf) · [Metal Feature Set Tables](https://developer.apple.com/metal/Metal-Feature-Set-Tables.pdf) · [metal-cpp](https://github.com/apple/metal-cpp) · [metal-cpp setup](https://developer.apple.com/metal/cpp/) · Tech Talks [111375 (M3/A17 GPU)](https://developer.apple.com/videos/play/tech-talks/111375/), [111373 (shader best practices)](https://developer.apple.com/videos/play/tech-talks/111373/) · WWDC [22-10159 (scale compute)](https://developer.apple.com/videos/play/wwdc2022/10159/), [20-10603 (GPU counters)](https://developer.apple.com/videos/play/wwdc2020/10603/), [25-205 (Metal 4)](https://developer.apple.com/videos/play/wwdc2025/205/) · [Metal Best Practices Guide (triple buffering)](https://developer.apple.com/library/archive/documentation/3DDrawing/Conceptual/MTLBestPracticesGuide/TripleBuffering.html) · [precompiling shader libraries](https://developer.apple.com/documentation/metal/building-a-shader-library-by-precompiling-source-files) | |
| 336 | + | |
| 337 | +**Reference kernels**: [MLX STEEL GEMM](https://github.com/ml-explore/mlx/tree/main/mlx/backend/metal/kernels/steel/gemm) · [MLX steel attention](https://github.com/ml-explore/mlx/tree/main/mlx/backend/metal/kernels/steel/attn) · [MLX matmul dispatch](https://github.com/ml-explore/mlx/blob/main/mlx/backend/metal/matmul.cpp) · [MLX sdpa dispatch](https://github.com/ml-explore/mlx/blob/main/mlx/backend/metal/scaled_dot_product_attention.cpp) · [llama.cpp ggml-metal.metal](https://github.com/ggml-org/llama.cpp/blob/master/ggml/src/ggml-metal/ggml-metal.metal) · [metal-flash-attention](https://github.com/philipturner/metal-flash-attention) · [metal-benchmarks](https://github.com/philipturner/metal-benchmarks) · [percisely.xyz/gemm](https://percisely.xyz/gemm) | |
| 338 | + | |
| 339 | +**Papers**: [FlashAttention-2](https://arxiv.org/abs/2307.08691) · [FlashAttention](https://arxiv.org/abs/2205.14135) · [Rigel/Metal-4 matmul](https://arxiv.org/abs/2606.12765) | |
| 340 | + | |
| 341 | +**Training references**: [nanoGPT](https://github.com/karpathy/nanoGPT) (train.py, model.py) · [llm.c](https://github.com/karpathy/llm.c) (train_gpt2.c/.cu, llmc/adamw.cuh, global_norm.cuh, fused_classifier.cuh, dataloader.h) · [minbpe](https://github.com/karpathy/minbpe) · [llama model.py (RoPE/RMSNorm/SwiGLU)](https://github.com/meta-llama/llama/blob/main/llama/model.py) · [llama2.c](https://github.com/karpathy/llama2.c) · [PyTorch AMP docs](https://docs.pytorch.org/docs/stable/amp.html) · [TinyStories](https://huggingface.co/datasets/roneneldan/TinyStories) · [smollm-corpus](https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus) | |
| 342 | + | |
| 343 | +**Community/pitfalls**: [dpogue.ca CMake+Metal](https://dpogue.ca/articles/cmake-metal.html) · [MLX #3689 (buffer use-after-free)](https://github.com/ml-explore/mlx/issues/3689) · [MLX #1864 (command-buffer batching)](https://github.com/ml-explore/mlx/pull/1864) · [MLX discussion #3209 (M3 throughput)](https://github.com/ml-explore/mlx/discussions/3209) · [HF #25199 (RoPE conventions)](https://github.com/huggingface/transformers/issues/25199) · [MoltenVK #1989 (offset alignment)](https://github.com/KhronosGroup/MoltenVK/issues/1989) · [llama.cpp #9094 (bf16 Metal status)](https://github.com/ggml-org/llama.cpp/issues/9094) · [tinygrad #7408 (bf16 simdgroup)](https://github.com/tinygrad/tinygrad/pull/7408) | |
added
configs/gpt-100m.json
+38 −0
@@ -0,0 +1,38 @@ | ||
| 1 | +{ | |
| 2 | + "_comment": "batch_size is the MICRO-batch: activations for the whole tape live at once (no activation checkpointing yet), so ~2.3 GB/layer at batch 32 x ctx 1024. batch_size * grad_accum_steps * context_length is the effective tokens/step. precision is parsed but not yet honored - all kernels are f32 (see README \"Not yet done\").", | |
| 3 | + "model": { | |
| 4 | + "name": "gpt-100m", | |
| 5 | + "n_layers": 12, | |
| 6 | + "d_model": 768, | |
| 7 | + "n_heads": 12, | |
| 8 | + "n_kv_heads": 12, | |
| 9 | + "d_ff": 2048, | |
| 10 | + "vocab_size": 16384, | |
| 11 | + "context_length": 1024, | |
| 12 | + "tied_embeddings": true, | |
| 13 | + "use_rope": true, | |
| 14 | + "rope_theta": 10000.0, | |
| 15 | + "norm": "rmsnorm", | |
| 16 | + "norm_eps": 1e-06, | |
| 17 | + "activation": "swiglu", | |
| 18 | + "dropout": 0.0 | |
| 19 | + }, | |
| 20 | + "train": { | |
| 21 | + "lr": 0.0004, | |
| 22 | + "min_lr_ratio": 0.1, | |
| 23 | + "warmup_steps": 2000, | |
| 24 | + "max_steps": 100000, | |
| 25 | + "beta1": 0.9, | |
| 26 | + "beta2": 0.95, | |
| 27 | + "eps": 1e-08, | |
| 28 | + "weight_decay": 0.1, | |
| 29 | + "grad_clip": 1.0, | |
| 30 | + "batch_size": 8, | |
| 31 | + "grad_accum_steps": 16, | |
| 32 | + "precision": "f32", | |
| 33 | + "checkpoint_every": 2000, | |
| 34 | + "eval_every": 1000, | |
| 35 | + "eval_batches": 20, | |
| 36 | + "seed": 1337 | |
| 37 | + } | |
| 38 | +} | |
added
configs/gpt-10m-1epoch.json
+38 −0
@@ -0,0 +1,38 @@ | ||
| 1 | +{ | |
| 2 | + "_comment": "One epoch over the 19.14M-token TinyStories set: 584 steps x 64 sequences x 512 tokens = 19.1M tokens. Warmup is 10% of the run.", | |
| 3 | + "model": { | |
| 4 | + "name": "gpt-10m", | |
| 5 | + "n_layers": 6, | |
| 6 | + "d_model": 384, | |
| 7 | + "n_heads": 6, | |
| 8 | + "n_kv_heads": 6, | |
| 9 | + "d_ff": 1024, | |
| 10 | + "vocab_size": 4096, | |
| 11 | + "context_length": 512, | |
| 12 | + "tied_embeddings": true, | |
| 13 | + "use_rope": true, | |
| 14 | + "rope_theta": 10000.0, | |
| 15 | + "norm": "rmsnorm", | |
| 16 | + "norm_eps": 1e-06, | |
| 17 | + "activation": "swiglu", | |
| 18 | + "dropout": 0.0 | |
| 19 | + }, | |
| 20 | + "train": { | |
| 21 | + "lr": 0.0006, | |
| 22 | + "min_lr_ratio": 0.1, | |
| 23 | + "warmup_steps": 58, | |
| 24 | + "max_steps": 584, | |
| 25 | + "beta1": 0.9, | |
| 26 | + "beta2": 0.95, | |
| 27 | + "eps": 1e-08, | |
| 28 | + "weight_decay": 0.1, | |
| 29 | + "grad_clip": 1.0, | |
| 30 | + "batch_size": 64, | |
| 31 | + "grad_accum_steps": 1, | |
| 32 | + "precision": "f32", | |
| 33 | + "checkpoint_every": 200, | |
| 34 | + "eval_every": 100, | |
| 35 | + "eval_batches": 20, | |
| 36 | + "seed": 1337 | |
| 37 | + } | |
| 38 | +} | |
added
configs/gpt-10m.json
+37 −0
@@ -0,0 +1,37 @@ | ||
| 1 | +{ | |
| 2 | + "model": { | |
| 3 | + "name": "gpt-10m", | |
| 4 | + "n_layers": 6, | |
| 5 | + "d_model": 384, | |
| 6 | + "n_heads": 6, | |
| 7 | + "n_kv_heads": 6, | |
| 8 | + "d_ff": 1024, | |
| 9 | + "vocab_size": 4096, | |
| 10 | + "context_length": 512, | |
| 11 | + "tied_embeddings": true, | |
| 12 | + "use_rope": true, | |
| 13 | + "rope_theta": 10000.0, | |
| 14 | + "norm": "rmsnorm", | |
| 15 | + "norm_eps": 1e-6, | |
| 16 | + "activation": "swiglu", | |
| 17 | + "dropout": 0.0 | |
| 18 | + }, | |
| 19 | + "train": { | |
| 20 | + "lr": 6e-4, | |
| 21 | + "min_lr_ratio": 0.1, | |
| 22 | + "warmup_steps": 1000, | |
| 23 | + "max_steps": 20000, | |
| 24 | + "beta1": 0.9, | |
| 25 | + "beta2": 0.95, | |
| 26 | + "eps": 1e-8, | |
| 27 | + "weight_decay": 0.1, | |
| 28 | + "grad_clip": 1.0, | |
| 29 | + "batch_size": 64, | |
| 30 | + "grad_accum_steps": 1, | |
| 31 | + "precision": "f32", | |
| 32 | + "checkpoint_every": 1000, | |
| 33 | + "eval_every": 500, | |
| 34 | + "eval_batches": 20, | |
| 35 | + "seed": 1337 | |
| 36 | + } | |
| 37 | +} | |
added
configs/gpt-200m.json
+38 −0
@@ -0,0 +1,38 @@ | ||
| 1 | +{ | |
| 2 | + "_comment": "batch_size is the MICRO-batch; see gpt-100m.json. 4 x 64 x 1024 = 262144 tokens/step. precision is parsed but not yet honored - all kernels are f32 (see README \"Not yet done\").", | |
| 3 | + "model": { | |
| 4 | + "name": "gpt-200m", | |
| 5 | + "n_layers": 16, | |
| 6 | + "d_model": 1024, | |
| 7 | + "n_heads": 16, | |
| 8 | + "n_kv_heads": 8, | |
| 9 | + "d_ff": 2816, | |
| 10 | + "vocab_size": 16384, | |
| 11 | + "context_length": 1024, | |
| 12 | + "tied_embeddings": true, | |
| 13 | + "use_rope": true, | |
| 14 | + "rope_theta": 10000.0, | |
| 15 | + "norm": "rmsnorm", | |
| 16 | + "norm_eps": 1e-06, | |
| 17 | + "activation": "swiglu", | |
| 18 | + "dropout": 0.0 | |
| 19 | + }, | |
| 20 | + "train": { | |
| 21 | + "lr": 0.0003, | |
| 22 | + "min_lr_ratio": 0.1, | |
| 23 | + "warmup_steps": 2000, | |
| 24 | + "max_steps": 200000, | |
| 25 | + "beta1": 0.9, | |
| 26 | + "beta2": 0.95, | |
| 27 | + "eps": 1e-08, | |
| 28 | + "weight_decay": 0.1, | |
| 29 | + "grad_clip": 1.0, | |
| 30 | + "batch_size": 4, | |
| 31 | + "grad_accum_steps": 64, | |
| 32 | + "precision": "f32", | |
| 33 | + "checkpoint_every": 2000, | |
| 34 | + "eval_every": 1000, | |
| 35 | + "eval_batches": 20, | |
| 36 | + "seed": 1337 | |
| 37 | + } | |
| 38 | +} | |
added
configs/gpt-25m.json
+38 −0
@@ -0,0 +1,38 @@ | ||
| 1 | +{ | |
| 2 | + "_comment": "batch_size is the MICRO-batch (8 x 8 x 1024 = 65536 tokens/step); the autograd tape holds all activations. precision is parsed but not yet honored - all kernels are f32 (see README \"Not yet done\").", | |
| 3 | + "model": { | |
| 4 | + "name": "gpt-25m", | |
| 5 | + "n_layers": 8, | |
| 6 | + "d_model": 512, | |
| 7 | + "n_heads": 8, | |
| 8 | + "n_kv_heads": 8, | |
| 9 | + "d_ff": 1408, | |
| 10 | + "vocab_size": 8192, | |
| 11 | + "context_length": 1024, | |
| 12 | + "tied_embeddings": true, | |
| 13 | + "use_rope": true, | |
| 14 | + "rope_theta": 10000.0, | |
| 15 | + "norm": "rmsnorm", | |
| 16 | + "norm_eps": 1e-06, | |
| 17 | + "activation": "swiglu", | |
| 18 | + "dropout": 0.0 | |
| 19 | + }, | |
| 20 | + "train": { | |
| 21 | + "lr": 0.0006, | |
| 22 | + "min_lr_ratio": 0.1, | |
| 23 | + "warmup_steps": 2000, | |
| 24 | + "max_steps": 50000, | |
| 25 | + "beta1": 0.9, | |
| 26 | + "beta2": 0.95, | |
| 27 | + "eps": 1e-08, | |
| 28 | + "weight_decay": 0.1, | |
| 29 | + "grad_clip": 1.0, | |
| 30 | + "batch_size": 8, | |
| 31 | + "grad_accum_steps": 8, | |
| 32 | + "precision": "f32", | |
| 33 | + "checkpoint_every": 1000, | |
| 34 | + "eval_every": 500, | |
| 35 | + "eval_batches": 20, | |
| 36 | + "seed": 1337 | |
| 37 | + } | |
| 38 | +} | |
added
configs/gpt-smoke.json
+38 −0
@@ -0,0 +1,38 @@ | ||
| 1 | +{ | |
| 2 | + "_comment": "Small-context smoke config for end-to-end verification. The unfused attention kernel materializes [B,H,T,T] probabilities, so keep T modest until the fused flash kernel lands.", | |
| 3 | + "model": { | |
| 4 | + "name": "gpt-smoke", | |
| 5 | + "n_layers": 4, | |
| 6 | + "d_model": 256, | |
| 7 | + "n_heads": 4, | |
| 8 | + "n_kv_heads": 2, | |
| 9 | + "d_ff": 704, | |
| 10 | + "vocab_size": 4096, | |
| 11 | + "context_length": 128, | |
| 12 | + "tied_embeddings": true, | |
| 13 | + "use_rope": true, | |
| 14 | + "rope_theta": 10000.0, | |
| 15 | + "norm": "rmsnorm", | |
| 16 | + "norm_eps": 1e-6, | |
| 17 | + "activation": "swiglu", | |
| 18 | + "dropout": 0.0 | |
| 19 | + }, | |
| 20 | + "train": { | |
| 21 | + "lr": 1e-3, | |
| 22 | + "min_lr_ratio": 0.1, | |
| 23 | + "warmup_steps": 100, | |
| 24 | + "max_steps": 2000, | |
| 25 | + "beta1": 0.9, | |
| 26 | + "beta2": 0.95, | |
| 27 | + "eps": 1e-8, | |
| 28 | + "weight_decay": 0.1, | |
| 29 | + "grad_clip": 1.0, | |
| 30 | + "batch_size": 32, | |
| 31 | + "grad_accum_steps": 1, | |
| 32 | + "precision": "f32", | |
| 33 | + "checkpoint_every": 500, | |
| 34 | + "eval_every": 250, | |
| 35 | + "eval_batches": 10, | |
| 36 | + "seed": 1337 | |
| 37 | + } | |
| 38 | +} | |
added
paper/forge.tex
+558 −0
@@ -0,0 +1,558 @@ | ||
| 1 | +% Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +\documentclass[11pt,a4paper]{article} | |
| 3 | + | |
| 4 | +\usepackage[utf8]{inputenc} | |
| 5 | +\usepackage[T1]{fontenc} | |
| 6 | +\usepackage{lmodern} | |
| 7 | +\usepackage{amsmath,amssymb} | |
| 8 | +\usepackage{booktabs} | |
| 9 | +\usepackage{listings} | |
| 10 | +\usepackage{xcolor} | |
| 11 | +\usepackage{graphicx} | |
| 12 | +\usepackage[margin=1in]{geometry} | |
| 13 | +\usepackage{hyperref} | |
| 14 | +\hypersetup{colorlinks=true,linkcolor=blue,citecolor=blue,urlcolor=blue} | |
| 15 | +\usepackage{microtype} | |
| 16 | + | |
| 17 | +\lstdefinestyle{msl}{ | |
| 18 | + basicstyle=\ttfamily\footnotesize, | |
| 19 | + keywordstyle=\color{blue!70!black}\bfseries, | |
| 20 | + commentstyle=\color{green!40!black}\itshape, | |
| 21 | + stringstyle=\color{red!60!black}, | |
| 22 | + numbers=none, breaklines=true, frame=single, framesep=4pt, | |
| 23 | + backgroundcolor=\color{gray!5}, showstringspaces=false, | |
| 24 | + morekeywords={kernel,threadgroup,device,constant,constexpr,simdgroup_float8x8, | |
| 25 | + simdgroup_matrix,enum,uint,template,typename,tensor,dextents,tensor_inline,half,bfloat} | |
| 26 | +} | |
| 27 | +\lstset{style=msl} | |
| 28 | + | |
| 29 | +\title{\textbf{Forge}: Building an LLM Training Framework from Scratch\\ | |
| 30 | +in C++ and Metal on Apple Silicon\\ | |
| 31 | +\large Measured Lessons on Compiler Traps, Register Pressure,\\ | |
| 32 | +and the M5 Neural Accelerators} | |
| 33 | + | |
| 34 | +\author{Simon-Pierre Boucher\\ | |
| 35 | +\texttt{contact@spboucher.ai}} | |
| 36 | + | |
| 37 | +\date{31 July 2026} | |
| 38 | + | |
| 39 | +\begin{document} | |
| 40 | +\maketitle | |
| 41 | + | |
| 42 | +\begin{abstract} | |
| 43 | +We report engineering findings from \emph{Forge}, a transformer training framework | |
| 44 | +written from scratch in C++20 with hand-written Metal compute kernels, with no | |
| 45 | +PyTorch, MLX, or other machine-learning dependency. Every GPU kernel is validated | |
| 46 | +against a CPU reference to $\leq 10^{-4}$, and every optimization reported here was | |
| 47 | +accepted only after the training loss trajectory remained numerically unchanged. | |
| 48 | + | |
| 49 | +Four findings are, we believe, of general interest to practitioners writing Metal | |
| 50 | +compute kernels. First, the idiom \texttt{constant constexpr} for tile constants in | |
| 51 | +the Metal Shading Language silently declares an \emph{address-space variable} rather | |
| 52 | +than a compile-time constant; the resulting loss of loop unrolling spills every | |
| 53 | +matrix accumulator to the stack and cost a factor of \textbf{12$\times$} on our GEMM. | |
| 54 | +Second, this class of defect is invisible in the AIR intermediate representation at | |
| 55 | +every optimization level, because unrolling and fragment promotion occur in the | |
| 56 | +driver's back end --- practitioners must benchmark rather than read the IR. Third, | |
| 57 | +per-thread register pressure, not bandwidth, is the dominant constraint in | |
| 58 | +attention backward kernels; measured spill counts guided three successive | |
| 59 | +restructurings that took the backward pass from 107\,ms to | |
| 60 | +7.05\,ms, a \textbf{15.2$\times$} improvement. Fourth, on | |
| 61 | +M5-generation hardware the Metal Performance Primitives cooperative-tensor | |
| 62 | +\texttt{matmul2d} reaches \textbf{51.5\,TFLOP/s} with \texttt{half} operands against | |
| 63 | +10.6\,TFLOP/s for a well-tuned hand-written \texttt{simdgroup\_matrix} kernel, | |
| 64 | +a \textbf{4.9$\times$} gap that inverts the usual cost/benefit assessment of mixed | |
| 65 | +precision. | |
| 66 | + | |
| 67 | +We also report a negative result that we consider equally useful: \texttt{half} and | |
| 68 | +\texttt{bfloat} operands buy only 18--22\% over \texttt{float} on the | |
| 69 | +\texttt{simdgroup\_matrix} path, so mixed precision on that path is a memory | |
| 70 | +optimization and not a compute one. The framework trains models from 12M to 205M | |
| 71 | +parameters on a single machine. | |
| 72 | +\end{abstract} | |
| 73 | + | |
| 74 | +\tableofcontents | |
| 75 | + | |
| 76 | +\section{Introduction} | |
| 77 | + | |
| 78 | +Apple Silicon offers a large unified memory pool and substantial GPU throughput, but | |
| 79 | +the software ecosystem for \emph{training} on it is thin. Inference frameworks are | |
| 80 | +mature; training-specific kernels, in particular a fused attention backward pass, are | |
| 81 | +largely absent. A survey we conducted on 31 July 2026 (Section~\ref{sec:survey}) | |
| 82 | +found that of the major open-source frameworks, none ships a fused attention backward | |
| 83 | +kernel for Metal. | |
| 84 | + | |
| 85 | +This paper reports what we learned building one. \emph{Forge} is a decoder-only | |
| 86 | +transformer training framework in pure C++20 with Metal compute kernels. Its | |
| 87 | +architecture is entirely configuration-driven, and it has no machine-learning | |
| 88 | +dependency: only Apple's \texttt{metal-cpp} bindings and a single-header JSON parser. | |
| 89 | + | |
| 90 | +Our contribution is not the framework itself but the measurements. Each section below | |
| 91 | +states a hypothesis, the measurement that tested it, and the outcome --- including | |
| 92 | +the cases where the hypothesis was wrong. | |
| 93 | + | |
| 94 | +\subsection{Methodology and reproducibility} | |
| 95 | + | |
| 96 | +All measurements were taken on an Apple M5 Max (40-core GPU) running macOS 27.0 | |
| 97 | +(build 26A5388g) with Xcode 26.6 and Metal toolchain 32023.883. Kernels are compiled | |
| 98 | +at \texttt{-std=metal3.2} except where Metal 4 features are required. | |
| 99 | +Throughput figures are computed from | |
| 100 | +\texttt{MTLCommandBuffer.GPUStartTime}/\texttt{GPUEndTime}, which excludes CPU | |
| 101 | +encoding time. | |
| 102 | + | |
| 103 | +Two disciplines govern every result reported here. | |
| 104 | + | |
| 105 | +\paragraph{Correctness gates optimization.} Every Metal kernel has a CPU reference | |
| 106 | +implementation, and 85 parity assertions compare them (maximum absolute error | |
| 107 | +$\leq 10^{-4}$ in \texttt{float}; most are bit-exact). Gradients are additionally | |
| 108 | +checked against central finite differences. An optimization is accepted only if the | |
| 109 | +full suite still passes. | |
| 110 | + | |
| 111 | +\paragraph{Numerical equivalence, not just correctness.} For the larger kernel | |
| 112 | +rewrites we required that the \emph{training loss trajectory} be unchanged. After | |
| 113 | +replacing the scalar attention kernels with tiled matrix-multiply versions, the loss | |
| 114 | +at every logged step over 250 optimizer steps was identical | |
| 115 | +(8.3618, 7.6130, 6.6455, 5.0902, 4.9845; validation 4.9018). This is a stronger | |
| 116 | +check than parity on a single call, since it accumulates any discrepancy through the | |
| 117 | +optimizer. | |
| 118 | + | |
| 119 | +\section{System overview} | |
| 120 | + | |
| 121 | +The framework comprises roughly 6{,}500 lines of C++ and Metal. A \texttt{Tensor} | |
| 122 | +type provides shared-storage views over pooled \texttt{MTLBuffer} allocations in | |
| 123 | +\texttt{MTLStorageModeShared}, so the same memory is addressable from CPU and GPU | |
| 124 | +with no copies. A dynamic autograd tape records backward closures. Operations | |
| 125 | +dispatch to either a CPU reference backend or the Metal backend behind one interface, | |
| 126 | +which is what makes the parity testing possible. | |
| 127 | + | |
| 128 | +Execution follows the batching discipline recommended for Apple GPUs: the whole | |
| 129 | +training micro-batch is encoded into one command buffer with a small number of | |
| 130 | +long-lived compute encoders, and the GPU is synchronized only at the loss-readback | |
| 131 | +boundary. | |
| 132 | + | |
| 133 | +\section{Finding 1: \texttt{constant constexpr} is not a compile-time constant} | |
| 134 | +\label{sec:constexpr} | |
| 135 | + | |
| 136 | +\subsection{The trap} | |
| 137 | + | |
| 138 | +Tile geometry in a GEMM kernel is naturally expressed as named constants. The | |
| 139 | +apparently idiomatic C++ spelling is: | |
| 140 | + | |
| 141 | +\begin{lstlisting} | |
| 142 | +constant constexpr uint TM = 4; // WRONG in MSL | |
| 143 | +\end{lstlisting} | |
| 144 | + | |
| 145 | +In the Metal Shading Language, however, \texttt{constant} is an \emph{address-space | |
| 146 | +qualifier}, and program-scope variables are \emph{required} to live in it. The | |
| 147 | +declaration above therefore creates a variable in the constant address space, not a | |
| 148 | +compile-time constant. Loop bounds derived from it are not known at compile time, so | |
| 149 | +the fragment loops do not unroll; \texttt{acc[i][j]} becomes dynamic indexing into an | |
| 150 | +array of opaque \texttt{simdgroup\_matrix} values; and the compiler is forced to | |
| 151 | +place all sixteen 8$\times$8 accumulators (256 bytes each) on the stack. Every | |
| 152 | +matrix-multiply-accumulate then pays roughly a kilobyte of stack traffic. | |
| 153 | + | |
| 154 | +The correct spelling uses enumerators, which are true integral constant expressions: | |
| 155 | + | |
| 156 | +\begin{lstlisting} | |
| 157 | +enum : uint { BM = 64, BN = 64, BK = 16, WM = 2, WN = 2, | |
| 158 | + TM = BM / (8 * WM), TN = BN / (8 * WN) }; | |
| 159 | +\end{lstlisting} | |
| 160 | + | |
| 161 | +together with \verb|#pragma clang loop unroll(full)| on the fragment loops. Note also | |
| 162 | +that \verb|#pragma unroll full|, which appears in Apple's own header documentation, | |
| 163 | +is not valid here. | |
| 164 | + | |
| 165 | +\subsection{Measurement} | |
| 166 | + | |
| 167 | +The change is textual and semantically neutral. Its effect is not: | |
| 168 | + | |
| 169 | +\begin{table}[h] | |
| 170 | +\centering | |
| 171 | +\begin{tabular}{lrr} | |
| 172 | +\toprule | |
| 173 | +Shape & \texttt{constant constexpr} & \texttt{enum} + unroll pragma \\ | |
| 174 | +\midrule | |
| 175 | +$2048^3$ & 0.82 & 10.21 \\ | |
| 176 | +$4096^3$ & 0.85 & \phantom{0}9.83 \\ | |
| 177 | +$65536\times512\times1408$ & 0.83 & 10.68 \\ | |
| 178 | +\bottomrule | |
| 179 | +\end{tabular} | |
| 180 | +\caption{f32 GEMM throughput (TFLOP/s), M5 Max. A 12$\times$ difference from the | |
| 181 | +declaration form of tile constants.} | |
| 182 | +\end{table} | |
| 183 | + | |
| 184 | +Before this fix the ``optimized'' \texttt{simdgroup\_matrix} kernel was | |
| 185 | +\emph{three times slower} than a naive 16$\times$16 tiled kernel --- a result that | |
| 186 | +invites the wrong conclusion, namely that the matrix instructions are not worth | |
| 187 | +using. | |
| 188 | + | |
| 189 | +\section{Finding 2: the intermediate representation cannot diagnose this} | |
| 190 | +\label{sec:ir} | |
| 191 | + | |
| 192 | +Having observed the anomaly, our first instinct was to inspect the compiler output: | |
| 193 | + | |
| 194 | +\begin{lstlisting} | |
| 195 | +xcrun metal -std=metal3.2 -O2 -S -emit-llvm -c matmul_simd.metal -o out.ll | |
| 196 | +\end{lstlisting} | |
| 197 | + | |
| 198 | +The AIR showed three \texttt{alloca} instructions for the fragment arrays and only | |
| 199 | +two matrix-multiply intrinsics where thirty-two were expected --- apparently | |
| 200 | +confirming the spill hypothesis. It also, however, showed \emph{exactly the same | |
| 201 | +thing} at \texttt{-O0}, \texttt{-O2} and \texttt{-O3}, and continued to show it after | |
| 202 | +the fix that produced the 12$\times$ speedup. | |
| 203 | + | |
| 204 | +The explanation is that loop unrolling and promotion of \texttt{simdgroup\_matrix} | |
| 205 | +values into registers happen in the driver's AIR-to-ISA back end, at pipeline | |
| 206 | +creation time, not in the front end that \texttt{-emit-llvm} exposes. The IR is | |
| 207 | +therefore uninformative for exactly the class of question one is most tempted to ask | |
| 208 | +of it. | |
| 209 | + | |
| 210 | +\paragraph{Practical rule.} On Metal, benchmark; do not read the IR. We lost time to | |
| 211 | +this and record it because the failure mode is silent: the IR looks like a | |
| 212 | +confirmation. | |
| 213 | + | |
| 214 | +\section{Finding 3: register pressure dominates attention backward} | |
| 215 | +\label{sec:registers} | |
| 216 | + | |
| 217 | +\subsection{Three restructurings, each guided by measurement} | |
| 218 | + | |
| 219 | +Our first fused attention implementation assigned one thread per query row and held | |
| 220 | +$q$, $o$ (forward) or $k$, $v$, $dk$, $dv$ (backward) in per-thread arrays. At head | |
| 221 | +dimension 64 the backward variant holds $4 \times 64 = 256$ floats, or one kilobyte | |
| 222 | +per thread. | |
| 223 | + | |
| 224 | +We tested the hypothesis that this spills, by progressively reducing what each thread | |
| 225 | +holds: | |
| 226 | + | |
| 227 | +\begin{table}[h] | |
| 228 | +\centering | |
| 229 | +\begin{tabular}{lrr} | |
| 230 | +\toprule | |
| 231 | +Attention backward, per layer & gpt-10m shapes & gpt-25m shapes \\ | |
| 232 | +\midrule | |
| 233 | +$k,v,dk,dv$ all in registers & 150 & --- \\ | |
| 234 | +read-only $k,v$ from device & 118 & 611 \\ | |
| 235 | +$dK$ and $dV$ split into two kernels & 105 & 549 \\ | |
| 236 | +\midrule | |
| 237 | +tiled with \texttt{simdgroup\_matrix} & \phantom{0}9.66 & \phantom{0}45.5 \\ | |
| 238 | +\quad + $dK$/$dV$ split again & \textbf{\phantom{0}7.05} & \textbf{\phantom{0}32.9} \\ | |
| 239 | +\bottomrule | |
| 240 | +\end{tabular} | |
| 241 | +\caption{Time in milliseconds. gpt-10m shapes are $B{=}64$, $T{=}512$, $H{=}6$, | |
| 242 | +$d_h{=}64$; gpt-25m shapes are $B{=}64$, $T{=}1024$, $H{=}8$. Total improvement | |
| 243 | +15.2$\times$ and 17.0$\times$ respectively.} | |
| 244 | +\end{table} | |
| 245 | + | |
| 246 | +The read-only operands $k$ and $v$ are re-read from device memory on every iteration | |
| 247 | +in the second row, which sounds wasteful; because a thread reads the same address | |
| 248 | +every iteration, the L1 cache serves it, and removing the two arrays from registers | |
| 249 | +is the larger effect. | |
| 250 | + | |
| 251 | +\subsection{Confirming the mechanism directly} | |
| 252 | + | |
| 253 | +The reasoning above was indirect. Xcode~26 ships \texttt{gpudebug}, a headless | |
| 254 | +command-line GPU debugger that reports compiler statistics per kernel. It requires | |
| 255 | +shader sources to be embedded in the library, which in turn requires | |
| 256 | +\texttt{-frecord-sources} at \emph{both} the compile and link steps. (The link must | |
| 257 | +be performed by \texttt{metal}; \texttt{metallib} rejects the flag.) With that fixed, | |
| 258 | +the mechanism is directly visible: | |
| 259 | + | |
| 260 | +\begin{table}[h] | |
| 261 | +\centering | |
| 262 | +\begin{tabular}{lrrr} | |
| 263 | +\toprule | |
| 264 | +Kernel & Temp registers & Spilled bytes & Cost \\ | |
| 265 | +\midrule | |
| 266 | +flash forward, scalar & 126 & 368 & 38.9\% \\ | |
| 267 | +flash forward, MMA-tiled & \phantom{0}85 & \textbf{0} & 29.6\% \\ | |
| 268 | +flash backward $dQ$, MMA & \phantom{0}95 & \textbf{0} & \phantom{0}8.5\% \\ | |
| 269 | +flash backward $dKV$, MMA (fused) & 111 & \textbf{4352} & 16.4\% \\ | |
| 270 | +\bottomrule | |
| 271 | +\end{tabular} | |
| 272 | +\caption{Compiler statistics from \texttt{gpudebug}, M5 Max.} | |
| 273 | +\end{table} | |
| 274 | + | |
| 275 | +Two things follow. The matrix-tiled forward eliminates the spill entirely | |
| 276 | +($368 \rightarrow 0$ bytes), which is the mechanism behind its speedup: an | |
| 277 | +8$\times$8 fragment occupies two floats per lane, where the scalar formulation held | |
| 278 | +whole arrays of head-dimension length. And the fused $dK/dV$ backward kernel | |
| 279 | +\emph{still spilled} 4352 bytes --- a defect we had not suspected, and which made it | |
| 280 | +the most expensive backward kernel. Splitting it recovered a further 27\%. | |
| 281 | + | |
| 282 | +We regard the tooling lesson as the transferable one: register pressure is | |
| 283 | +measurable, cheaply and without a GUI, and it is worth measuring before restructuring | |
| 284 | +a kernel on intuition. | |
| 285 | + | |
| 286 | +\section{Finding 4: cooperative tensors and the M5 neural accelerators} | |
| 287 | +\label{sec:mpp} | |
| 288 | + | |
| 289 | +\subsection{Mixed precision on the classical path is a memory optimization} | |
| 290 | + | |
| 291 | +The literature disagrees about \texttt{half} on Apple GPUs. Published | |
| 292 | +microbenchmarks report that \texttt{half} and \texttt{float} fused multiply-add | |
| 293 | +execute at the same rate, the benefit arising from register and bandwidth pressure; | |
| 294 | +Apple's material for the M3 generation describes up to twice the arithmetic | |
| 295 | +throughput from co-issue. Since plumbing mixed precision through a training framework | |
| 296 | +is a substantial amount of work, we measured before committing. Holding the tiling | |
| 297 | +and the \texttt{float} accumulator fixed and varying only the operand type: | |
| 298 | + | |
| 299 | +\begin{table}[h] | |
| 300 | +\centering | |
| 301 | +\begin{tabular}{lrrr} | |
| 302 | +\toprule | |
| 303 | +Shape & \texttt{float} & \texttt{half} & \texttt{bfloat} \\ | |
| 304 | +\midrule | |
| 305 | +$2048^3$ & \phantom{0}9.29 & 10.64 & 10.82 \\ | |
| 306 | +$4096^3$ & \phantom{0}9.85 & 11.81 & 12.07 \\ | |
| 307 | +$65536\times512\times1408$ & \phantom{0}9.90 & 12.45 & 12.44 \\ | |
| 308 | +\bottomrule | |
| 309 | +\end{tabular} | |
| 310 | +\caption{\texttt{simdgroup\_matrix} throughput (TFLOP/s) by operand precision, | |
| 311 | +\texttt{float} accumulator throughout.} | |
| 312 | +\end{table} | |
| 313 | + | |
| 314 | +The gain is 18--22\%, consistent with the register/bandwidth explanation rather than | |
| 315 | +with doubled arithmetic rate. On this path, mixed precision halves activation memory | |
| 316 | +--- valuable, since activation memory bounds trainable model size --- but it is not a | |
| 317 | +compute optimization. | |
| 318 | + | |
| 319 | +\subsection{The same operands through \texttt{matmul2d}} | |
| 320 | + | |
| 321 | +\texttt{MetalPerformancePrimitives.framework} ships in the macOS 26.5 SDK and exposes | |
| 322 | +\texttt{mpp::tensor\_ops::matmul2d}, a cooperative-tensor matrix multiply that targets | |
| 323 | +the per-core neural accelerators introduced with the M5 generation. Compiling at | |
| 324 | +\texttt{-std=metal4.0}, with a 64$\times$32 tile over four SIMD groups and a | |
| 325 | +\texttt{float} accumulator: | |
| 326 | + | |
| 327 | +\begin{table}[h] | |
| 328 | +\centering | |
| 329 | +\begin{tabular}{lrrrr} | |
| 330 | +\toprule | |
| 331 | + & \multicolumn{2}{c}{\texttt{simdgroup\_matrix}} & \multicolumn{2}{c}{\texttt{matmul2d}} \\ | |
| 332 | +\cmidrule(lr){2-3}\cmidrule(lr){4-5} | |
| 333 | +Shape & \texttt{float} & \texttt{half} & \texttt{float} & \texttt{half} \\ | |
| 334 | +\midrule | |
| 335 | +$2048^3$ & \phantom{0}9.3 & 10.6 & 14.9 & \textbf{51.5} \\ | |
| 336 | +$4096^3$ & \phantom{0}9.9 & 11.8 & 14.6 & \textbf{44.3} \\ | |
| 337 | +$65536\times512\times1408$ & \phantom{0}9.9 & 12.4 & 14.4 & \textbf{23.8} \\ | |
| 338 | +\bottomrule | |
| 339 | +\end{tabular} | |
| 340 | +\caption{Throughput (TFLOP/s), M5 Max. A 1.5$\times$ gain in \texttt{float} and | |
| 341 | +4.3--4.9$\times$ in \texttt{half}.} | |
| 342 | +\end{table} | |
| 343 | + | |
| 344 | +Because a 4.9$\times$ claim invites scepticism, we verified numerically rather than | |
| 345 | +only timing: against the CPU reference the \texttt{float} path is bit-exact, the | |
| 346 | +\texttt{half} path differs by $3.8 \times 10^{-6}$ from a reference fed identically | |
| 347 | +rounded inputs, and every output element is non-zero, confirming that the full | |
| 348 | +reduction over $K$ occurs. The transposed variants required for training | |
| 349 | +($X W^{\!\top}$ in the forward, $dX = dY W$ and $dW = dY^{\!\top} X$ in the backward) | |
| 350 | +are likewise bit-exact. | |
| 351 | + | |
| 352 | +\paragraph{Generational caveat.} Published work on M4 Max hardware found | |
| 353 | +\texttt{matmul2d} still executing on the shader cores and losing to a hand-fused | |
| 354 | +GEMM. Our M5 result is the opposite. The accelerator hardware differs by generation; | |
| 355 | +this measurement should be repeated per target device rather than assumed. | |
| 356 | + | |
| 357 | +\paragraph{Consequence.} This inverts the priority we had assigned to mixed | |
| 358 | +precision. Judged on the classical path it is worth 20\% and is chiefly a memory | |
| 359 | +feature; as the entry condition for the accelerator path it is worth 4.9$\times$. | |
| 360 | + | |
| 361 | +\subsection{Interface notes} | |
| 362 | + | |
| 363 | +Several details cost us time and are not documented accurately. | |
| 364 | + | |
| 365 | +\begin{itemize} | |
| 366 | + \item Bind ordinary buffers and construct tensors inside the kernel with the | |
| 367 | + \texttt{tensor\_inline} descriptor. The default \texttt{tensor\_handle} | |
| 368 | + descriptor wraps an opaque handle obtainable only from a host-side | |
| 369 | + \texttt{MTLTensor}, which would require substantial host plumbing. | |
| 370 | + \item Extents are ordered (columns, rows), and \texttt{slice()} takes column then | |
| 371 | + row; a transposed operand must be sliced the other way and its extents | |
| 372 | + swapped. | |
| 373 | + \item The element type must be non-\texttt{const}; a \texttt{const float} tensor | |
| 374 | + fails a static assertion. | |
| 375 | + \item Zero the destination cooperative tensor using | |
| 376 | + \texttt{is\_valid\_element(i)}. Apple's own header example calls | |
| 377 | + \texttt{get\_mask(i)}, which does not exist in this SDK. | |
| 378 | +\end{itemize} | |
| 379 | + | |
| 380 | +\section{Finding 5: concurrency, not Metal 4, is the encoding win} | |
| 381 | + | |
| 382 | +Metal's default compute encoder orders every dispatch against its predecessor. For | |
| 383 | +the optimizer step this is pure loss: AdamW issues one dispatch per parameter tensor | |
| 384 | +--- of order one hundred for a 100M-parameter model --- and each gradient-norm | |
| 385 | +reduction is a single threadgroup. These are mutually independent. | |
| 386 | + | |
| 387 | +We introduced a scoped concurrent-dispatch region and applied it to both optimizer | |
| 388 | +passes. On the 100M configuration end-to-end throughput rose from 7.9k to | |
| 389 | +\textbf{9.6k tokens/s}, a 22\% improvement, with identical losses. The change is | |
| 390 | +about twenty lines and requires no Metal 4 adoption. | |
| 391 | + | |
| 392 | +An independent investigation confirmed the general shape of this result --- roughly | |
| 393 | +15$\times$ on a synthetic batch of small independent dispatches --- while also | |
| 394 | +establishing that Metal 4's command-encoding path offers essentially no CPU-side | |
| 395 | +advantage once invariant bindings are hoisted out of the dispatch loop. The | |
| 396 | +concurrency, available since macOS 10.14, is the entire effect. | |
| 397 | + | |
| 398 | +\section{Ecosystem survey} | |
| 399 | +\label{sec:survey} | |
| 400 | + | |
| 401 | +We surveyed the major open-source frameworks on 31 July 2026 for a fused attention | |
| 402 | +backward pass on Metal: | |
| 403 | + | |
| 404 | +\begin{table}[h] | |
| 405 | +\centering | |
| 406 | +\begin{tabular}{ll} | |
| 407 | +\toprule | |
| 408 | +Framework & Fused attention backward on Metal \\ | |
| 409 | +\midrule | |
| 410 | +MLX & No. \texttt{use\_fallback} returns \texttt{true}; \texttt{eval\_gpu} throws | |
| 411 | + \texttt{"NYI"}. The fused backward is CUDA-only. \\ | |
| 412 | +llama.cpp & No. The Metal backend does not support | |
| 413 | + \texttt{GGML\_OP\_FLASH\_ATTN\_BACK}. \\ | |
| 414 | +PyTorch MPS & No. Hand-written Metal forward kernels only. \\ | |
| 415 | +Candle & No. Vendored MLX forward only. \\ | |
| 416 | +tinygrad & No. Its flash-attention backward targets AMD. \\ | |
| 417 | +Burn / CubeCL & Kernels exist but are a scalar scaffold, not wired to autodiff. \\ | |
| 418 | +\bottomrule | |
| 419 | +\end{tabular} | |
| 420 | +\end{table} | |
| 421 | + | |
| 422 | +MLX's Metal forward additionally declines the fused path under gradient tracing, with | |
| 423 | +the comment that unfused is faster for training on Metal. Production prior art is | |
| 424 | +limited to Philip Turner's \emph{metal-flash-attention} and one third-party package. | |
| 425 | +Both use the same three-stage, atomic-free structure we arrived at independently: | |
| 426 | +a preprocessing kernel for $D = \mathrm{rowsum}(dO \circ O)$ in \texttt{float}, a | |
| 427 | +$dQ$ kernel parallel over queries, and a $dK/dV$ kernel parallel over key/value rows. | |
| 428 | + | |
| 429 | +\section{Attention kernel design} | |
| 430 | + | |
| 431 | +Our tiled forward kernel assigns 32 query rows per threadgroup across four SIMD | |
| 432 | +groups, holding $Q$ and the output accumulator in registers as 8$\times$8 fragments | |
| 433 | +and staging $K$ and $V$ through threadgroup memory. The score tile is round-tripped | |
| 434 | +through threadgroup memory so that the softmax reductions run on ordinary threads. | |
| 435 | + | |
| 436 | +\paragraph{A scoped problem worth recording.} Online softmax requires rescaling the | |
| 437 | +output accumulator by a per-row factor at every key/value block, | |
| 438 | +$O \leftarrow \mathrm{diag}(\mathrm{corr})\, O$. But the Metal specification leaves | |
| 439 | +the mapping from \texttt{simdgroup\_matrix} elements to lanes \emph{unspecified}, so a | |
| 440 | +lane cannot determine which row its registers correspond to. MLX resolves this by | |
| 441 | +reverse-engineering the mapping. We instead construct the 8$\times$8 diagonal matrix | |
| 442 | +in threadgroup memory and apply it with an ordinary matrix multiply. This is | |
| 443 | +specification-clean, costs $d_h/8$ additional multiplies per block (about 25\% more | |
| 444 | +matrix work), and keeps the accumulator in registers --- staging it in threadgroup | |
| 445 | +memory would have cost 8\,KB and halved residency. The backward pass needs none of | |
| 446 | +this, since the saved logsumexp already fixes the normalization. | |
| 447 | + | |
| 448 | +A second technique worth noting: the $dK/dV$ kernel requires transposed score | |
| 449 | +matrices, which are obtained not by moving data but by exchanging the operand roles, | |
| 450 | +$S^{\!\top} = K Q^{\!\top}$ and $dP^{\!\top} = V\, dO^{\!\top}$, with | |
| 451 | +\texttt{simdgroup\_load(..., transpose=true)} supplying the transposed operands | |
| 452 | +directly from the staged tiles. | |
| 453 | + | |
| 454 | +\section{Results} | |
| 455 | + | |
| 456 | +\begin{table}[h] | |
| 457 | +\centering | |
| 458 | +\begin{tabular}{lrrr} | |
| 459 | +\toprule | |
| 460 | +Kernel & Naive & Tiled & \texttt{simdgroup\_matrix} \\ | |
| 461 | +\midrule | |
| 462 | +$4096^3$ & 1.29 & 2.53 & \phantom{0}9.49 \\ | |
| 463 | +Forward MLP $XW_1^{\!\top}$ & 1.46 & 2.53 & 10.68 \\ | |
| 464 | +Backward $dX = dY W$ & 1.51 & 2.54 & 10.58 \\ | |
| 465 | +Backward $dW = dY^{\!\top} X$ & 0.76 & 2.15 & \phantom{0}4.69 \\ | |
| 466 | +\bottomrule | |
| 467 | +\end{tabular} | |
| 468 | +\caption{GEMM throughput (TFLOP/s). The $dW$ case lags because $K = BT$ is very large | |
| 469 | +with few threadgroups; split-K would address it and is not implemented.} | |
| 470 | +\end{table} | |
| 471 | + | |
| 472 | +End-to-end training throughput, all configurations on one M5 Max: | |
| 473 | + | |
| 474 | +\begin{table}[h] | |
| 475 | +\centering | |
| 476 | +\begin{tabular}{lrrr} | |
| 477 | +\toprule | |
| 478 | +Configuration & Parameters & Context & Tokens/s \\ | |
| 479 | +\midrule | |
| 480 | +gpt-10m & 12.2M & \phantom{0}512 & 38.2k \\ | |
| 481 | +gpt-25m & 29.9M & 1024 & 22.1k \\ | |
| 482 | +gpt-100m & 97.5M & 1024 & \phantom{0}9.7k \\ | |
| 483 | +gpt-200m & 205.5M & 1024 & \phantom{0}1.8k\rlap{$^\dagger$} \\ | |
| 484 | +\bottomrule | |
| 485 | +\end{tabular} | |
| 486 | +\caption{$^\dagger$ measured before the matrix-tiled backward and concurrent | |
| 487 | +optimizer landed; this figure is pessimistic.} | |
| 488 | +\end{table} | |
| 489 | + | |
| 490 | +\subsection{A memory bug found by scaling} | |
| 491 | + | |
| 492 | +The 100M configuration exhausted GPU memory even at a micro-batch of eight. The cause | |
| 493 | +was an interaction between two individually reasonable decisions: pooled buffers | |
| 494 | +released while a command buffer is open are parked on a retire list until the next | |
| 495 | +synchronization, and the trainer synchronized once per \emph{optimizer} step. With | |
| 496 | +sixteen gradient-accumulation micro-batches, none of the sixteen sets of activations | |
| 497 | +were ever recycled, so peak memory scaled with the accumulation factor. | |
| 498 | +Synchronizing once per micro-batch fixed it. We note this because both decisions are | |
| 499 | +defensible in isolation and the failure appears only at scale. | |
| 500 | + | |
| 501 | +\subsection{End-to-end training run} | |
| 502 | +\label{sec:epoch} | |
| 503 | + | |
| 504 | +To confirm that the optimized kernels train a model and not merely a benchmark, we | |
| 505 | +trained the 12.2M-parameter configuration for exactly one epoch over the | |
| 506 | +19.14M-token TinyStories corpus: 584 steps of 64 sequences $\times$ 512 tokens. | |
| 507 | +Learning rate follows linear warmup over the first 10\% of the run into a cosine | |
| 508 | +decay to one tenth of peak; AdamW with decoupled weight decay 0.1 on parameters of | |
| 509 | +rank at least two, gradient clipping at global norm 1.0. | |
| 510 | + | |
| 511 | +RESULTS_PLACEHOLDER | |
| 512 | + | |
| 513 | +\section{Related work} | |
| 514 | + | |
| 515 | +Our GEMM structure follows MLX's STEEL kernels, and our attention design follows | |
| 516 | +FlashAttention-2 as adapted to Apple GPUs by \emph{metal-flash-attention}. The | |
| 517 | +training-loop details --- decoupled weight decay with $\varepsilon$ outside the square | |
| 518 | +root, weight decay applied only to parameters of rank at least two, the gradient-clip | |
| 519 | +factor folded into the optimizer's gradient read, and the fused classifier that | |
| 520 | +writes the logit gradient in place --- follow \texttt{llm.c} and \texttt{nanoGPT}. | |
| 521 | + | |
| 522 | +\section{Limitations} | |
| 523 | + | |
| 524 | +All measurements come from a single M5 Max. The neural-accelerator result in | |
| 525 | +particular is generation-specific and we expect it not to transfer to the M3 | |
| 526 | +generation. Mixed precision is not implemented: every kernel is \texttt{float}, so | |
| 527 | +the 4.9$\times$ accelerator path is measured but not yet exploited by the training | |
| 528 | +loop. Activation checkpointing is absent, which is what bounds model size. Split-K | |
| 529 | +for the $dW$ GEMM is not implemented. Finally, brief benchmarks on Apple Silicon can | |
| 530 | +execute in a reduced GPU performance state; long-running configurations should be | |
| 531 | +cross-checked against the performance-state trace. | |
| 532 | + | |
| 533 | +\section{Conclusion} | |
| 534 | + | |
| 535 | +The largest factors we encountered were not algorithmic. A twelvefold loss came from | |
| 536 | +a declaration keyword whose meaning differs between C++ and the Metal Shading | |
| 537 | +Language. A fifteenfold gain in the attention backward came from reducing per-thread | |
| 538 | +register pressure, measurable directly once shader sources were embedded in the | |
| 539 | +library. A fivefold opportunity sits behind a Metal 4 interface whose own header | |
| 540 | +documentation is stale. | |
| 541 | + | |
| 542 | +The methodological lesson is narrower and firmer than any single number. Twice in | |
| 543 | +this work a confident inference from static artifacts --- the intermediate | |
| 544 | +representation in Section~\ref{sec:ir}, a published tile configuration in | |
| 545 | +Section~\ref{sec:registers} --- pointed the wrong way, and was corrected only by | |
| 546 | +measurement. On this platform the profiler is cheap, scriptable, and headless; the | |
| 547 | +intuitions are not reliable. | |
| 548 | + | |
| 549 | +\section*{Availability} | |
| 550 | + | |
| 551 | +\emph{Forge} comprises approximately 6{,}500 lines of C++20 and Metal. The | |
| 552 | +benchmarks reported here are \texttt{tests/bench\_matmul.cpp}, | |
| 553 | +\texttt{tests/bench\_attention.cpp} and \texttt{tests/bench\_precision.cpp}; the | |
| 554 | +numerical verification of the cooperative-tensor path is \texttt{tests/mppcheck.cpp}. | |
| 555 | +Research notes with exact interface signatures and a pitfalls list are maintained in | |
| 556 | +\texttt{RESEARCH.md}. | |
| 557 | + | |
| 558 | +\end{document} | |
added
src/core/allocator.cpp
+82 −0
@@ -0,0 +1,82 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#include "core/allocator.h" | |
| 3 | + | |
| 4 | +#include <Metal/Metal.hpp> | |
| 5 | + | |
| 6 | +#include <cstdio> | |
| 7 | +#include <cstdlib> | |
| 8 | + | |
| 9 | +namespace forge { | |
| 10 | + | |
| 11 | +Allocator::Allocator(MTL::Device* device) : device_(device) {} | |
| 12 | + | |
| 13 | +Allocator::~Allocator() { | |
| 14 | + trim(); | |
| 15 | +} | |
| 16 | + | |
| 17 | +size_t Allocator::bucket_size(size_t nbytes) { | |
| 18 | + size_t b = 256; | |
| 19 | + while (b < nbytes) b <<= 1; | |
| 20 | + return b; | |
| 21 | +} | |
| 22 | + | |
| 23 | +MTL::Buffer* Allocator::acquire(size_t nbytes) { | |
| 24 | + const size_t bucket = bucket_size(nbytes); | |
| 25 | + { | |
| 26 | + std::lock_guard<std::mutex> lock(mutex_); | |
| 27 | + auto it = pool_.find(bucket); | |
| 28 | + if (it != pool_.end() && !it->second.empty()) { | |
| 29 | + MTL::Buffer* buf = it->second.back(); | |
| 30 | + it->second.pop_back(); | |
| 31 | + bytes_pooled_ -= bucket; | |
| 32 | + bytes_live_ += bucket; | |
| 33 | + return buf; | |
| 34 | + } | |
| 35 | + } | |
| 36 | + MTL::Buffer* buf = device_->newBuffer(bucket, MTL::ResourceStorageModeShared); | |
| 37 | + if (!buf) { | |
| 38 | + std::fprintf(stderr, "forge: Metal buffer allocation failed (%zu bytes)\n", bucket); | |
| 39 | + std::abort(); | |
| 40 | + } | |
| 41 | + std::lock_guard<std::mutex> lock(mutex_); | |
| 42 | + bytes_live_ += bucket; | |
| 43 | + return buf; | |
| 44 | +} | |
| 45 | + | |
| 46 | +void Allocator::release(MTL::Buffer* buffer) { | |
| 47 | + if (!buffer) return; | |
| 48 | + const size_t bucket = buffer->length(); | |
| 49 | + std::lock_guard<std::mutex> lock(mutex_); | |
| 50 | + if (defer_) { | |
| 51 | + retired_.push_back(buffer); | |
| 52 | + } else { | |
| 53 | + pool_[bucket].push_back(buffer); | |
| 54 | + bytes_pooled_ += bucket; | |
| 55 | + } | |
| 56 | + bytes_live_ -= bucket; | |
| 57 | +} | |
| 58 | + | |
| 59 | +void Allocator::set_defer(bool defer) { | |
| 60 | + std::lock_guard<std::mutex> lock(mutex_); | |
| 61 | + defer_ = defer; | |
| 62 | +} | |
| 63 | + | |
| 64 | +void Allocator::flush_retired() { | |
| 65 | + std::lock_guard<std::mutex> lock(mutex_); | |
| 66 | + for (MTL::Buffer* buf : retired_) { | |
| 67 | + pool_[buf->length()].push_back(buf); | |
| 68 | + bytes_pooled_ += buf->length(); | |
| 69 | + } | |
| 70 | + retired_.clear(); | |
| 71 | +} | |
| 72 | + | |
| 73 | +void Allocator::trim() { | |
| 74 | + std::lock_guard<std::mutex> lock(mutex_); | |
| 75 | + for (auto& [bucket, buffers] : pool_) { | |
| 76 | + for (MTL::Buffer* buf : buffers) buf->release(); | |
| 77 | + } | |
| 78 | + pool_.clear(); | |
| 79 | + bytes_pooled_ = 0; | |
| 80 | +} | |
| 81 | + | |
| 82 | +} // namespace forge | |
added
src/core/allocator.h
+63 −0
@@ -0,0 +1,63 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#pragma once | |
| 3 | + | |
| 4 | +#include <cstddef> | |
| 5 | +#include <mutex> | |
| 6 | +#include <unordered_map> | |
| 7 | +#include <vector> | |
| 8 | + | |
| 9 | +namespace MTL { | |
| 10 | +class Buffer; | |
| 11 | +class Device; | |
| 12 | +} | |
| 13 | + | |
| 14 | +namespace forge { | |
| 15 | + | |
| 16 | +// Size-bucketed MTLBuffer pool. Every tensor allocation goes through here so | |
| 17 | +// the training loop never touches newBuffer (a kernel-level VM allocation). | |
| 18 | +// Buffers use MTLStorageModeShared: the CPU pointer is valid at all times. | |
| 19 | +// | |
| 20 | +// Sizes are rounded up to power-of-two buckets (min 256 bytes, which also | |
| 21 | +// satisfies every setBuffer offset-alignment rule). release() returns the | |
| 22 | +// buffer to its bucket; it is the caller's job (Storage refcounting, and | |
| 23 | +// later the command-buffer retire list) to only release buffers no in-flight | |
| 24 | +// command buffer references. | |
| 25 | +class Allocator { | |
| 26 | +public: | |
| 27 | + explicit Allocator(MTL::Device* device); | |
| 28 | + ~Allocator(); | |
| 29 | + | |
| 30 | + Allocator(const Allocator&) = delete; | |
| 31 | + Allocator& operator=(const Allocator&) = delete; | |
| 32 | + | |
| 33 | + // Returns a retained buffer of capacity >= nbytes. | |
| 34 | + MTL::Buffer* acquire(size_t nbytes); | |
| 35 | + void release(MTL::Buffer* buffer); | |
| 36 | + | |
| 37 | + // While deferring (a command buffer is being encoded / in flight), | |
| 38 | + // released buffers park on a retire list instead of the free pool: | |
| 39 | + // recycling them would let CPU-side writes (zeros, fills) race pending | |
| 40 | + // GPU reads. The Stream flips this on at first encode and flushes at | |
| 41 | + // sync. | |
| 42 | + void set_defer(bool defer); | |
| 43 | + void flush_retired(); | |
| 44 | + | |
| 45 | + // Drop all pooled (free) buffers back to the OS. | |
| 46 | + void trim(); | |
| 47 | + | |
| 48 | + size_t bytes_live() const { return bytes_live_; } | |
| 49 | + size_t bytes_pooled() const { return bytes_pooled_; } | |
| 50 | + | |
| 51 | +private: | |
| 52 | + static size_t bucket_size(size_t nbytes); | |
| 53 | + | |
| 54 | + MTL::Device* device_; // borrowed, owned by forge::Device | |
| 55 | + std::mutex mutex_; | |
| 56 | + std::unordered_map<size_t, std::vector<MTL::Buffer*>> pool_; | |
| 57 | + std::vector<MTL::Buffer*> retired_; | |
| 58 | + bool defer_ = false; | |
| 59 | + size_t bytes_live_ = 0; | |
| 60 | + size_t bytes_pooled_ = 0; | |
| 61 | +}; | |
| 62 | + | |
| 63 | +} // namespace forge | |
added
src/core/autograd.cpp
+24 −0
@@ -0,0 +1,24 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#include "core/autograd.h" | |
| 3 | + | |
| 4 | +#include <cstdio> | |
| 5 | +#include <cstdlib> | |
| 6 | + | |
| 7 | +namespace forge { | |
| 8 | + | |
| 9 | +Tape& Tape::get() { | |
| 10 | + static Tape tape; | |
| 11 | + return tape; | |
| 12 | +} | |
| 13 | + | |
| 14 | +void Tape::backward(const Var& loss) { | |
| 15 | + if (loss.value().numel() != 1) { | |
| 16 | + std::fprintf(stderr, "forge: backward() needs a scalar loss\n"); | |
| 17 | + std::abort(); | |
| 18 | + } | |
| 19 | + loss.grad().fill_(1.0f); | |
| 20 | + for (auto it = nodes_.rbegin(); it != nodes_.rend(); ++it) (*it)(); | |
| 21 | + clear(); | |
| 22 | +} | |
| 23 | + | |
| 24 | +} // namespace forge | |
added
src/core/autograd.h
+95 −0
@@ -0,0 +1,95 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#pragma once | |
| 3 | + | |
| 4 | +#include "core/tensor.h" | |
| 5 | + | |
| 6 | +#include <functional> | |
| 7 | +#include <memory> | |
| 8 | +#include <vector> | |
| 9 | + | |
| 10 | +namespace forge { | |
| 11 | + | |
| 12 | +// A Var is a Tensor plus (lazily allocated) gradient storage. Copies share | |
| 13 | +// the same impl, so parameters handed to modules, the optimizer and the | |
| 14 | +// tape all see one .grad — which is also what makes tied embeddings and | |
| 15 | +// gradient accumulation across micro-batches work for free. | |
| 16 | +class Var { | |
| 17 | +public: | |
| 18 | + Var() = default; | |
| 19 | + explicit Var(Tensor value, bool requires_grad = false) | |
| 20 | + : impl_(std::make_shared<Impl>(Impl{std::move(value), Tensor{}, requires_grad})) {} | |
| 21 | + | |
| 22 | + bool defined() const { return impl_ != nullptr; } | |
| 23 | + // Stable identity of the underlying storage — two Vars with the same id | |
| 24 | + // are the same parameter (tied embeddings/lm_head). | |
| 25 | + const void* id() const { return impl_.get(); } | |
| 26 | + Tensor& value() const { return impl_->value; } | |
| 27 | + bool requires_grad() const { return impl_ && impl_->requires_grad; } | |
| 28 | + | |
| 29 | + // Gradient, allocated as zeros on first touch. Grads are f32 always. | |
| 30 | + Tensor& grad() const { | |
| 31 | + if (!impl_->grad.defined()) { | |
| 32 | + impl_->grad = Tensor::zeros(impl_->value.shape(), DType::F32); | |
| 33 | + } | |
| 34 | + return impl_->grad; | |
| 35 | + } | |
| 36 | + bool has_grad() const { return impl_ && impl_->grad.defined(); } | |
| 37 | + void zero_grad() const { | |
| 38 | + if (impl_ && impl_->grad.defined()) impl_->grad.zero_(); | |
| 39 | + } | |
| 40 | + | |
| 41 | + // Shape-only view: shares BOTH value and grad storage with this Var, so | |
| 42 | + // gradient flow needs no tape node. | |
| 43 | + Var reshaped(std::vector<int64_t> shape) const { | |
| 44 | + Var out(value().view(shape), requires_grad()); | |
| 45 | + if (requires_grad()) out.impl_->grad = grad().view(std::move(shape)); | |
| 46 | + return out; | |
| 47 | + } | |
| 48 | + | |
| 49 | +private: | |
| 50 | + struct Impl { | |
| 51 | + Tensor value; | |
| 52 | + Tensor grad; | |
| 53 | + bool requires_grad; | |
| 54 | + }; | |
| 55 | + std::shared_ptr<Impl> impl_; | |
| 56 | +}; | |
| 57 | + | |
| 58 | +// Dynamic tape. Ops that produce grad-requiring outputs push a backward | |
| 59 | +// lambda; backward() runs them in reverse and clears the tape. Not | |
| 60 | +// thread-safe by design: one training thread. | |
| 61 | +class Tape { | |
| 62 | +public: | |
| 63 | + static Tape& get(); | |
| 64 | + | |
| 65 | + bool enabled() const { return enabled_; } | |
| 66 | + void set_enabled(bool e) { enabled_ = e; } | |
| 67 | + | |
| 68 | + void record(std::function<void()> backward_fn) { | |
| 69 | + if (enabled_) nodes_.push_back(std::move(backward_fn)); | |
| 70 | + } | |
| 71 | + | |
| 72 | + // Seeds d(loss)/d(loss) = 1 and walks the tape in reverse. | |
| 73 | + void backward(const Var& loss); | |
| 74 | + | |
| 75 | + void clear() { nodes_.clear(); } | |
| 76 | + size_t size() const { return nodes_.size(); } | |
| 77 | + | |
| 78 | +private: | |
| 79 | + Tape() = default; | |
| 80 | + std::vector<std::function<void()>> nodes_; | |
| 81 | + bool enabled_ = true; | |
| 82 | +}; | |
| 83 | + | |
| 84 | +// RAII scope that disables tape recording (inference / generation). | |
| 85 | +struct NoGrad { | |
| 86 | + NoGrad() : prev_(Tape::get().enabled()) { Tape::get().set_enabled(false); } | |
| 87 | + ~NoGrad() { Tape::get().set_enabled(prev_); } | |
| 88 | + NoGrad(const NoGrad&) = delete; | |
| 89 | + NoGrad& operator=(const NoGrad&) = delete; | |
| 90 | + | |
| 91 | +private: | |
| 92 | + bool prev_; | |
| 93 | +}; | |
| 94 | + | |
| 95 | +} // namespace forge | |
added
src/core/device.cpp
+114 −0
@@ -0,0 +1,114 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#include "core/device.h" | |
| 3 | + | |
| 4 | +#include <Foundation/Foundation.hpp> | |
| 5 | +#include <Metal/Metal.hpp> | |
| 6 | + | |
| 7 | +#include <cstdio> | |
| 8 | +#include <cstdlib> | |
| 9 | +#include <filesystem> | |
| 10 | + | |
| 11 | +#include <mach-o/dyld.h> | |
| 12 | + | |
| 13 | +namespace forge { | |
| 14 | + | |
| 15 | +namespace { | |
| 16 | + | |
| 17 | +std::string executable_dir() { | |
| 18 | + char buf[4096]; | |
| 19 | + uint32_t size = sizeof(buf); | |
| 20 | + if (_NSGetExecutablePath(buf, &size) != 0) return {}; | |
| 21 | + std::error_code ec; | |
| 22 | + auto canonical = std::filesystem::canonical(buf, ec); | |
| 23 | + if (ec) return {}; | |
| 24 | + return canonical.parent_path().string(); | |
| 25 | +} | |
| 26 | + | |
| 27 | +[[noreturn]] void die(const char* msg, NS::Error* err) { | |
| 28 | + std::fprintf(stderr, "forge: %s", msg); | |
| 29 | + if (err && err->localizedDescription()) { | |
| 30 | + std::fprintf(stderr, ": %s", err->localizedDescription()->utf8String()); | |
| 31 | + } | |
| 32 | + std::fprintf(stderr, "\n"); | |
| 33 | + std::abort(); | |
| 34 | +} | |
| 35 | + | |
| 36 | +} // namespace | |
| 37 | + | |
| 38 | +Device& Device::get() { | |
| 39 | + static Device instance; | |
| 40 | + return instance; | |
| 41 | +} | |
| 42 | + | |
| 43 | +Device::Device() { | |
| 44 | + device_ = MTL::CreateSystemDefaultDevice(); | |
| 45 | + if (!device_) die("no Metal device found", nullptr); | |
| 46 | + queue_ = device_->newCommandQueue(); | |
| 47 | + if (!queue_) die("failed to create command queue", nullptr); | |
| 48 | + allocator_ = std::make_unique<Allocator>(device_); | |
| 49 | + load_library(); | |
| 50 | +} | |
| 51 | + | |
| 52 | +Device::~Device() { | |
| 53 | + for (auto& [key, pso] : pipelines_) pso->release(); | |
| 54 | + if (library_) library_->release(); | |
| 55 | + allocator_.reset(); // must drop pooled buffers before the device | |
| 56 | + if (queue_) queue_->release(); | |
| 57 | + if (device_) device_->release(); | |
| 58 | +} | |
| 59 | + | |
| 60 | +void Device::load_library() { | |
| 61 | + std::vector<std::string> candidates; | |
| 62 | + if (const char* env = std::getenv("FORGE_METALLIB")) candidates.push_back(env); | |
| 63 | + if (auto dir = executable_dir(); !dir.empty()) candidates.push_back(dir + "/forge.metallib"); | |
| 64 | + candidates.push_back("forge.metallib"); | |
| 65 | + | |
| 66 | + for (const auto& path : candidates) { | |
| 67 | + if (!std::filesystem::exists(path)) continue; | |
| 68 | + NS::Error* err = nullptr; | |
| 69 | + NS::String* nspath = NS::String::string(path.c_str(), NS::UTF8StringEncoding); | |
| 70 | + library_ = device_->newLibrary(nspath, &err); | |
| 71 | + if (library_) return; | |
| 72 | + die("failed to load forge.metallib", err); | |
| 73 | + } | |
| 74 | + die("forge.metallib not found (looked next to the executable, in cwd, and at $FORGE_METALLIB)", | |
| 75 | + nullptr); | |
| 76 | +} | |
| 77 | + | |
| 78 | +MTL::ComputePipelineState* Device::pipeline(const std::string& kernel_name) { | |
| 79 | + return pipeline(kernel_name, nullptr, ""); | |
| 80 | +} | |
| 81 | + | |
| 82 | +MTL::ComputePipelineState* Device::pipeline(const std::string& kernel_name, | |
| 83 | + const MTL::FunctionConstantValues* constants, | |
| 84 | + const std::string& constants_key) { | |
| 85 | + const std::string key = | |
| 86 | + constants_key.empty() ? kernel_name : kernel_name + "/" + constants_key; | |
| 87 | + | |
| 88 | + std::lock_guard<std::mutex> lock(pipeline_mutex_); | |
| 89 | + if (auto it = pipelines_.find(key); it != pipelines_.end()) return it->second; | |
| 90 | + | |
| 91 | + NS::Error* err = nullptr; | |
| 92 | + NS::String* nsname = NS::String::string(kernel_name.c_str(), NS::UTF8StringEncoding); | |
| 93 | + MTL::Function* fn = constants | |
| 94 | + ? library_->newFunction(nsname, constants, &err) | |
| 95 | + : library_->newFunction(nsname); | |
| 96 | + if (!fn) die(("kernel not found: " + kernel_name).c_str(), err); | |
| 97 | + | |
| 98 | + MTL::ComputePipelineState* pso = device_->newComputePipelineState(fn, &err); | |
| 99 | + fn->release(); | |
| 100 | + if (!pso) die(("pipeline creation failed: " + key).c_str(), err); | |
| 101 | + | |
| 102 | + pipelines_.emplace(key, pso); | |
| 103 | + return pso; | |
| 104 | +} | |
| 105 | + | |
| 106 | +std::string Device::name() const { | |
| 107 | + return device_->name()->utf8String(); | |
| 108 | +} | |
| 109 | + | |
| 110 | +size_t Device::recommended_working_set() const { | |
| 111 | + return device_->recommendedMaxWorkingSetSize(); | |
| 112 | +} | |
| 113 | + | |
| 114 | +} // namespace forge | |
added
src/core/device.h
+66 −0
@@ -0,0 +1,66 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#pragma once | |
| 3 | + | |
| 4 | +#include "core/allocator.h" | |
| 5 | + | |
| 6 | +#include <memory> | |
| 7 | +#include <mutex> | |
| 8 | +#include <string> | |
| 9 | +#include <unordered_map> | |
| 10 | + | |
| 11 | +namespace MTL { | |
| 12 | +class Device; | |
| 13 | +class CommandQueue; | |
| 14 | +class Library; | |
| 15 | +class ComputePipelineState; | |
| 16 | +class FunctionConstantValues; | |
| 17 | +} | |
| 18 | + | |
| 19 | +namespace forge { | |
| 20 | + | |
| 21 | +// Owns the Metal device, the single command queue, the kernel library and | |
| 22 | +// the pipeline cache. One instance per process (Device::get()). | |
| 23 | +// | |
| 24 | +// All pipelines are created lazily on first use and cached by | |
| 25 | +// "kernel_name" or "kernel_name/constants_key" so steady-state training | |
| 26 | +// never touches pipeline creation. | |
| 27 | +class Device { | |
| 28 | +public: | |
| 29 | + static Device& get(); | |
| 30 | + | |
| 31 | + MTL::Device* mtl() const { return device_; } | |
| 32 | + MTL::CommandQueue* queue() const { return queue_; } | |
| 33 | + Allocator& allocator() { return *allocator_; } | |
| 34 | + | |
| 35 | + // Plain kernel, no function constants. | |
| 36 | + MTL::ComputePipelineState* pipeline(const std::string& kernel_name); | |
| 37 | + | |
| 38 | + // Specialized kernel. constants_key must uniquely identify the constant | |
| 39 | + // values (e.g. "am1_an0_ak1"); it is only used as a cache key. | |
| 40 | + MTL::ComputePipelineState* pipeline(const std::string& kernel_name, | |
| 41 | + const MTL::FunctionConstantValues* constants, | |
| 42 | + const std::string& constants_key); | |
| 43 | + | |
| 44 | + std::string name() const; | |
| 45 | + size_t recommended_working_set() const; | |
| 46 | + | |
| 47 | + Device(const Device&) = delete; | |
| 48 | + Device& operator=(const Device&) = delete; | |
| 49 | + | |
| 50 | +private: | |
| 51 | + Device(); | |
| 52 | + ~Device(); | |
| 53 | + | |
| 54 | + // Looks for forge.metallib next to the executable, in the current | |
| 55 | + // directory, or at $FORGE_METALLIB. | |
| 56 | + void load_library(); | |
| 57 | + | |
| 58 | + MTL::Device* device_ = nullptr; | |
| 59 | + MTL::CommandQueue* queue_ = nullptr; | |
| 60 | + MTL::Library* library_ = nullptr; | |
| 61 | + std::unique_ptr<Allocator> allocator_; | |
| 62 | + std::mutex pipeline_mutex_; | |
| 63 | + std::unordered_map<std::string, MTL::ComputePipelineState*> pipelines_; | |
| 64 | +}; | |
| 65 | + | |
| 66 | +} // namespace forge | |
added
src/core/dtype.h
+62 −0
@@ -0,0 +1,62 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#pragma once | |
| 3 | + | |
| 4 | +#include <cstddef> | |
| 5 | +#include <cstdint> | |
| 6 | +#include <cstring> | |
| 7 | + | |
| 8 | +namespace forge { | |
| 9 | + | |
| 10 | +// f16 uses the native ARM half type; bf16 is stored as raw uint16 with | |
| 11 | +// explicit conversions (clang's __bf16 arithmetic support is uneven and | |
| 12 | +// bf16 math always goes through f32 anyway). | |
| 13 | +using f16_t = _Float16; | |
| 14 | + | |
| 15 | +enum class DType : uint8_t { | |
| 16 | + F32, | |
| 17 | + F16, | |
| 18 | + BF16, | |
| 19 | + U16, | |
| 20 | + I32, | |
| 21 | +}; | |
| 22 | + | |
| 23 | +inline size_t dtype_size(DType dt) { | |
| 24 | + switch (dt) { | |
| 25 | + case DType::F32: return 4; | |
| 26 | + case DType::F16: return 2; | |
| 27 | + case DType::BF16: return 2; | |
| 28 | + case DType::U16: return 2; | |
| 29 | + case DType::I32: return 4; | |
| 30 | + } | |
| 31 | + return 0; | |
| 32 | +} | |
| 33 | + | |
| 34 | +inline const char* dtype_name(DType dt) { | |
| 35 | + switch (dt) { | |
| 36 | + case DType::F32: return "f32"; | |
| 37 | + case DType::F16: return "f16"; | |
| 38 | + case DType::BF16: return "bf16"; | |
| 39 | + case DType::U16: return "u16"; | |
| 40 | + case DType::I32: return "i32"; | |
| 41 | + } | |
| 42 | + return "?"; | |
| 43 | +} | |
| 44 | + | |
| 45 | +// bf16 = top 16 bits of f32. Round-to-nearest-even on the way down; NaN | |
| 46 | +// must keep a nonzero mantissa so it stays NaN after truncation. | |
| 47 | +inline uint16_t float_to_bf16(float x) { | |
| 48 | + uint32_t bits; | |
| 49 | + std::memcpy(&bits, &x, 4); | |
| 50 | + if ((bits & 0x7fffffff) > 0x7f800000) return uint16_t((bits >> 16) | 1); | |
| 51 | + uint32_t rounding = 0x7fff + ((bits >> 16) & 1); | |
| 52 | + return uint16_t((bits + rounding) >> 16); | |
| 53 | +} | |
| 54 | + | |
| 55 | +inline float bf16_to_float(uint16_t x) { | |
| 56 | + uint32_t bits = uint32_t(x) << 16; | |
| 57 | + float out; | |
| 58 | + std::memcpy(&out, &bits, 4); | |
| 59 | + return out; | |
| 60 | +} | |
| 61 | + | |
| 62 | +} // namespace forge | |
added
src/core/metal_impl.cpp
+11 −0
@@ -0,0 +1,11 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +// | |
| 3 | +// The one and only translation unit that instantiates metal-cpp's | |
| 4 | +// implementation (ObjC runtime glue). Every other file includes the | |
| 5 | +// metal-cpp headers without these defines. | |
| 6 | +#define NS_PRIVATE_IMPLEMENTATION | |
| 7 | +#define CA_PRIVATE_IMPLEMENTATION | |
| 8 | +#define MTL_PRIVATE_IMPLEMENTATION | |
| 9 | +#include <Foundation/Foundation.hpp> | |
| 10 | +#include <Metal/Metal.hpp> | |
| 11 | +#include <QuartzCore/QuartzCore.hpp> | |
added
src/core/tensor.cpp
+191 −0
@@ -0,0 +1,191 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#include "core/tensor.h" | |
| 3 | + | |
| 4 | +#include "core/device.h" | |
| 5 | + | |
| 6 | +#include <Metal/Metal.hpp> | |
| 7 | + | |
| 8 | +#include <cstdio> | |
| 9 | +#include <cstdlib> | |
| 10 | +#include <cstring> | |
| 11 | +#include <sstream> | |
| 12 | + | |
| 13 | +namespace forge { | |
| 14 | + | |
| 15 | +namespace { | |
| 16 | +[[noreturn]] void die(const std::string& msg) { | |
| 17 | + std::fprintf(stderr, "forge: %s\n", msg.c_str()); | |
| 18 | + std::abort(); | |
| 19 | +} | |
| 20 | +} // namespace | |
| 21 | + | |
| 22 | +// Owns one pooled MTLBuffer; returns it to the pool when the last Tensor | |
| 23 | +// view drops it. | |
| 24 | +struct Tensor::Storage { | |
| 25 | + MTL::Buffer* buffer = nullptr; | |
| 26 | + | |
| 27 | + explicit Storage(size_t nbytes) { | |
| 28 | + buffer = Device::get().allocator().acquire(nbytes); | |
| 29 | + } | |
| 30 | + ~Storage() { | |
| 31 | + Device::get().allocator().release(buffer); | |
| 32 | + } | |
| 33 | + Storage(const Storage&) = delete; | |
| 34 | + Storage& operator=(const Storage&) = delete; | |
| 35 | +}; | |
| 36 | + | |
| 37 | +std::vector<int64_t> Tensor::contiguous_strides(const std::vector<int64_t>& shape) { | |
| 38 | + std::vector<int64_t> strides(shape.size()); | |
| 39 | + int64_t acc = 1; | |
| 40 | + for (int64_t i = int64_t(shape.size()) - 1; i >= 0; --i) { | |
| 41 | + strides[size_t(i)] = acc; | |
| 42 | + acc *= shape[size_t(i)]; | |
| 43 | + } | |
| 44 | + return strides; | |
| 45 | +} | |
| 46 | + | |
| 47 | +Tensor Tensor::empty(std::vector<int64_t> shape, DType dtype) { | |
| 48 | + Tensor t; | |
| 49 | + t.dtype_ = dtype; | |
| 50 | + t.shape_ = std::move(shape); | |
| 51 | + t.strides_ = contiguous_strides(t.shape_); | |
| 52 | + int64_t n = 1; | |
| 53 | + for (int64_t d : t.shape_) { | |
| 54 | + if (d <= 0) die("Tensor::empty: non-positive dim"); | |
| 55 | + n *= d; | |
| 56 | + } | |
| 57 | + t.storage_ = std::make_shared<Storage>(size_t(n) * dtype_size(dtype)); | |
| 58 | + return t; | |
| 59 | +} | |
| 60 | + | |
| 61 | +Tensor Tensor::zeros(std::vector<int64_t> shape, DType dtype) { | |
| 62 | + Tensor t = empty(std::move(shape), dtype); | |
| 63 | + std::memset(t.raw(), 0, t.nbytes()); | |
| 64 | + return t; | |
| 65 | +} | |
| 66 | + | |
| 67 | +Tensor Tensor::full(std::vector<int64_t> shape, float value, DType dtype) { | |
| 68 | + Tensor t = empty(std::move(shape), dtype); | |
| 69 | + t.fill_(value); | |
| 70 | + return t; | |
| 71 | +} | |
| 72 | + | |
| 73 | +int64_t Tensor::size(int64_t dim) const { | |
| 74 | + if (dim < 0) dim += ndim(); | |
| 75 | + if (dim < 0 || dim >= ndim()) die("Tensor::size: dim out of range"); | |
| 76 | + return shape_[size_t(dim)]; | |
| 77 | +} | |
| 78 | + | |
| 79 | +int64_t Tensor::numel() const { | |
| 80 | + int64_t n = 1; | |
| 81 | + for (int64_t d : shape_) n *= d; | |
| 82 | + return n; | |
| 83 | +} | |
| 84 | + | |
| 85 | +bool Tensor::is_contiguous() const { | |
| 86 | + return strides_ == contiguous_strides(shape_); | |
| 87 | +} | |
| 88 | + | |
| 89 | +void* Tensor::raw() const { | |
| 90 | + if (!storage_) die("Tensor::raw: undefined tensor"); | |
| 91 | + return static_cast<char*>(storage_->buffer->contents()) + size_t(offset_) * itemsize(); | |
| 92 | +} | |
| 93 | + | |
| 94 | +MTL::Buffer* Tensor::buffer() const { | |
| 95 | + if (!storage_) die("Tensor::buffer: undefined tensor"); | |
| 96 | + return storage_->buffer; | |
| 97 | +} | |
| 98 | + | |
| 99 | +size_t Tensor::buffer_offset() const { | |
| 100 | + return size_t(offset_) * itemsize(); | |
| 101 | +} | |
| 102 | + | |
| 103 | +Tensor Tensor::view(std::vector<int64_t> new_shape) const { | |
| 104 | + if (!is_contiguous()) die("Tensor::view: tensor not contiguous"); | |
| 105 | + int64_t n = 1; | |
| 106 | + for (int64_t d : new_shape) n *= d; | |
| 107 | + if (n != numel()) die("Tensor::view: numel mismatch"); | |
| 108 | + Tensor t = *this; | |
| 109 | + t.shape_ = std::move(new_shape); | |
| 110 | + t.strides_ = contiguous_strides(t.shape_); | |
| 111 | + return t; | |
| 112 | +} | |
| 113 | + | |
| 114 | +Tensor Tensor::slice0(int64_t start, int64_t len) const { | |
| 115 | + if (!is_contiguous()) die("Tensor::slice0: tensor not contiguous"); | |
| 116 | + if (ndim() < 1 || start < 0 || len <= 0 || start + len > shape_[0]) | |
| 117 | + die("Tensor::slice0: range out of bounds"); | |
| 118 | + Tensor t = *this; | |
| 119 | + t.shape_[0] = len; | |
| 120 | + t.offset_ = offset_ + start * strides_[0]; | |
| 121 | + return t; | |
| 122 | +} | |
| 123 | + | |
| 124 | +void Tensor::fill_(float value) { | |
| 125 | + const int64_t n = numel(); | |
| 126 | + switch (dtype_) { | |
| 127 | + case DType::F32: { | |
| 128 | + float* p = data<float>(); | |
| 129 | + for (int64_t i = 0; i < n; ++i) p[i] = value; | |
| 130 | + break; | |
| 131 | + } | |
| 132 | + case DType::F16: { | |
| 133 | + f16_t* p = data<f16_t>(); | |
| 134 | + const f16_t v = f16_t(value); | |
| 135 | + for (int64_t i = 0; i < n; ++i) p[i] = v; | |
| 136 | + break; | |
| 137 | + } | |
| 138 | + case DType::BF16: { | |
| 139 | + uint16_t* p = data<uint16_t>(); | |
| 140 | + const uint16_t v = float_to_bf16(value); | |
| 141 | + for (int64_t i = 0; i < n; ++i) p[i] = v; | |
| 142 | + break; | |
| 143 | + } | |
| 144 | + case DType::U16: { | |
| 145 | + uint16_t* p = data<uint16_t>(); | |
| 146 | + const uint16_t v = uint16_t(value); | |
| 147 | + for (int64_t i = 0; i < n; ++i) p[i] = v; | |
| 148 | + break; | |
| 149 | + } | |
| 150 | + case DType::I32: { | |
| 151 | + int32_t* p = data<int32_t>(); | |
| 152 | + const int32_t v = int32_t(value); | |
| 153 | + for (int64_t i = 0; i < n; ++i) p[i] = v; | |
| 154 | + break; | |
| 155 | + } | |
| 156 | + } | |
| 157 | +} | |
| 158 | + | |
| 159 | +float Tensor::item_at(int64_t i) const { | |
| 160 | + switch (dtype_) { | |
| 161 | + case DType::F32: return data<float>()[i]; | |
| 162 | + case DType::F16: return float(data<f16_t>()[i]); | |
| 163 | + case DType::BF16: return bf16_to_float(data<uint16_t>()[i]); | |
| 164 | + case DType::U16: return float(data<uint16_t>()[i]); | |
| 165 | + case DType::I32: return float(data<int32_t>()[i]); | |
| 166 | + } | |
| 167 | + return 0.0f; | |
| 168 | +} | |
| 169 | + | |
| 170 | +void Tensor::set_item(int64_t i, float value) { | |
| 171 | + switch (dtype_) { | |
| 172 | + case DType::F32: data<float>()[i] = value; break; | |
| 173 | + case DType::F16: data<f16_t>()[i] = f16_t(value); break; | |
| 174 | + case DType::BF16: data<uint16_t>()[i] = float_to_bf16(value); break; | |
| 175 | + case DType::U16: data<uint16_t>()[i] = uint16_t(value); break; | |
| 176 | + case DType::I32: data<int32_t>()[i] = int32_t(value); break; | |
| 177 | + } | |
| 178 | +} | |
| 179 | + | |
| 180 | +std::string Tensor::describe() const { | |
| 181 | + std::ostringstream os; | |
| 182 | + os << "Tensor(" << dtype_name(dtype_) << ", ["; | |
| 183 | + for (size_t i = 0; i < shape_.size(); ++i) { | |
| 184 | + if (i) os << ", "; | |
| 185 | + os << shape_[i]; | |
| 186 | + } | |
| 187 | + os << "])"; | |
| 188 | + return os.str(); | |
| 189 | +} | |
| 190 | + | |
| 191 | +} // namespace forge | |
added
src/core/tensor.h
+76 −0
@@ -0,0 +1,76 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#pragma once | |
| 3 | + | |
| 4 | +#include "core/dtype.h" | |
| 5 | + | |
| 6 | +#include <cstdint> | |
| 7 | +#include <memory> | |
| 8 | +#include <string> | |
| 9 | +#include <vector> | |
| 10 | + | |
| 11 | +namespace MTL { | |
| 12 | +class Buffer; | |
| 13 | +} | |
| 14 | + | |
| 15 | +namespace forge { | |
| 16 | + | |
| 17 | +// Refcounted view over a pooled MTLBuffer (MTLStorageModeShared), so the | |
| 18 | +// same memory is addressable from CPU code and GPU kernels with zero copies. | |
| 19 | +// Row-major, element strides. Views (reshape/view/slice) share storage. | |
| 20 | +class Tensor { | |
| 21 | +public: | |
| 22 | + Tensor() = default; | |
| 23 | + | |
| 24 | + static Tensor empty(std::vector<int64_t> shape, DType dtype = DType::F32); | |
| 25 | + static Tensor zeros(std::vector<int64_t> shape, DType dtype = DType::F32); | |
| 26 | + static Tensor full(std::vector<int64_t> shape, float value, DType dtype = DType::F32); | |
| 27 | + | |
| 28 | + bool defined() const { return storage_ != nullptr; } | |
| 29 | + DType dtype() const { return dtype_; } | |
| 30 | + int64_t ndim() const { return int64_t(shape_.size()); } | |
| 31 | + const std::vector<int64_t>& shape() const { return shape_; } | |
| 32 | + const std::vector<int64_t>& strides() const { return strides_; } | |
| 33 | + int64_t size(int64_t dim) const; | |
| 34 | + int64_t numel() const; | |
| 35 | + size_t itemsize() const { return dtype_size(dtype_); } | |
| 36 | + size_t nbytes() const { return size_t(numel()) * itemsize(); } | |
| 37 | + bool is_contiguous() const; | |
| 38 | + | |
| 39 | + // CPU access. Valid at all times (unified memory) — but only touch it | |
| 40 | + // when no in-flight command buffer may write the same storage. | |
| 41 | + void* raw() const; | |
| 42 | + template <typename T> | |
| 43 | + T* data() const { return static_cast<T*>(raw()); } | |
| 44 | + | |
| 45 | + // GPU access. | |
| 46 | + MTL::Buffer* buffer() const; | |
| 47 | + size_t buffer_offset() const; // bytes from the start of the MTLBuffer | |
| 48 | + | |
| 49 | + // Views (no copy). Both require contiguous layout. | |
| 50 | + Tensor view(std::vector<int64_t> new_shape) const; | |
| 51 | + Tensor reshape(std::vector<int64_t> new_shape) const { return view(std::move(new_shape)); } | |
| 52 | + // Slice along dim 0: rows [start, start+len). Contiguous only. | |
| 53 | + Tensor slice0(int64_t start, int64_t len) const; | |
| 54 | + | |
| 55 | + void fill_(float value); | |
| 56 | + void zero_() { fill_(0.0f); } | |
| 57 | + | |
| 58 | + // Read/write a single element as float, whatever the dtype (test/debug). | |
| 59 | + float item_at(int64_t linear_index) const; | |
| 60 | + void set_item(int64_t linear_index, float value); | |
| 61 | + | |
| 62 | + std::string describe() const; | |
| 63 | + | |
| 64 | +private: | |
| 65 | + struct Storage; | |
| 66 | + | |
| 67 | + std::shared_ptr<Storage> storage_; | |
| 68 | + std::vector<int64_t> shape_; | |
| 69 | + std::vector<int64_t> strides_; // in elements | |
| 70 | + int64_t offset_ = 0; // in elements | |
| 71 | + DType dtype_ = DType::F32; | |
| 72 | + | |
| 73 | + static std::vector<int64_t> contiguous_strides(const std::vector<int64_t>& shape); | |
| 74 | +}; | |
| 75 | + | |
| 76 | +} // namespace forge | |
added
src/kernels/adamw.metal
+75 −0
@@ -0,0 +1,75 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +// | |
| 3 | +// Fused AdamW update (llm.c formulation): one thread per element; the host | |
| 4 | +// precomputes the bias corrections and folds the gradient-clip scale into | |
| 5 | +// grad_scale, so the kernel reads the raw gradient once and writes w/m/v. | |
| 6 | +// eps sits OUTSIDE the sqrt; weight decay is decoupled (never through m/v) | |
| 7 | +// and the host sets wd = 0 for dim<2 tensors. | |
| 8 | +// | |
| 9 | +// sumsq_f32: grid-stride sum of squares of one tensor into partials[tg] — | |
| 10 | +// used for the global grad-norm (host sums the partials at the sync point). | |
| 11 | +#include <metal_stdlib> | |
| 12 | +using namespace metal; | |
| 13 | + | |
| 14 | +struct AdamWParams { | |
| 15 | + float lr, beta1, beta2, bc1, bc2, eps, wd, grad_scale; | |
| 16 | +}; | |
| 17 | + | |
| 18 | +kernel void adamw_f32(device float* w [[buffer(0)]], | |
| 19 | + device const float* g [[buffer(1)]], | |
| 20 | + device float* m [[buffer(2)]], | |
| 21 | + device float* v [[buffer(3)]], | |
| 22 | + constant AdamWParams& p [[buffer(4)]], | |
| 23 | + uint gid [[thread_position_in_grid]]) { | |
| 24 | + const float grad = g[gid] * p.grad_scale; | |
| 25 | + const float mi = p.beta1 * m[gid] + (1.0f - p.beta1) * grad; | |
| 26 | + const float vi = p.beta2 * v[gid] + (1.0f - p.beta2) * grad * grad; | |
| 27 | + m[gid] = mi; | |
| 28 | + v[gid] = vi; | |
| 29 | + const float mhat = mi / p.bc1; | |
| 30 | + const float vhat = vi / p.bc2; | |
| 31 | + w[gid] -= p.lr * (mhat / (sqrt(vhat) + p.eps) + p.wd * w[gid]); | |
| 32 | +} | |
| 33 | + | |
| 34 | +kernel void sumsq_f32(device const float* x [[buffer(0)]], | |
| 35 | + device float* partials [[buffer(1)]], | |
| 36 | + constant uint& n [[buffer(2)]], | |
| 37 | + uint gid [[thread_position_in_grid]], | |
| 38 | + uint grid_sz [[threads_per_grid]], | |
| 39 | + uint tg_id [[threadgroup_position_in_grid]], | |
| 40 | + uint lane [[thread_index_in_simdgroup]], | |
| 41 | + uint simd_idx [[simdgroup_index_in_threadgroup]], | |
| 42 | + uint n_simds [[simdgroups_per_threadgroup]]) { | |
| 43 | + float acc = 0.0f; | |
| 44 | + for (uint i = gid; i < n; i += grid_sz) acc = fma(x[i], x[i], acc); | |
| 45 | + | |
| 46 | + threadgroup float scratch[32]; | |
| 47 | + const float s = simd_sum(acc); | |
| 48 | + if (simd_is_first()) scratch[simd_idx] = s; | |
| 49 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 50 | + const float mine = (lane < n_simds) ? scratch[lane] : 0.0f; | |
| 51 | + const float total = simd_sum(mine); | |
| 52 | + if (simd_idx == 0 && lane == 0) partials[tg_id] = total; | |
| 53 | +} | |
| 54 | + | |
| 55 | +// Single-threadgroup final sum (also reduces CE per-row losses). | |
| 56 | +kernel void sum_f32(device const float* x [[buffer(0)]], | |
| 57 | + device float* out [[buffer(1)]], | |
| 58 | + constant uint& n [[buffer(2)]], | |
| 59 | + constant float& mul [[buffer(3)]], | |
| 60 | + uint lid [[thread_index_in_threadgroup]], | |
| 61 | + uint tg_size [[threads_per_threadgroup]], | |
| 62 | + uint lane [[thread_index_in_simdgroup]], | |
| 63 | + uint simd_idx [[simdgroup_index_in_threadgroup]], | |
| 64 | + uint n_simds [[simdgroups_per_threadgroup]]) { | |
| 65 | + float acc = 0.0f; | |
| 66 | + for (uint i = lid; i < n; i += tg_size) acc += x[i]; | |
| 67 | + | |
| 68 | + threadgroup float scratch[32]; | |
| 69 | + const float s = simd_sum(acc); | |
| 70 | + if (simd_is_first()) scratch[simd_idx] = s; | |
| 71 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 72 | + const float mine = (lane < n_simds) ? scratch[lane] : 0.0f; | |
| 73 | + const float total = simd_sum(mine); | |
| 74 | + if (simd_idx == 0 && lane == 0) out[0] = total * mul; | |
| 75 | +} | |
added
src/kernels/attention.metal
+177 −0
@@ -0,0 +1,177 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +// | |
| 3 | +// Unfused-but-GPU-resident causal attention with GQA (M4 correctness path; | |
| 4 | +// the flash-style fused kernel is M5). Probabilities P [B,H,T,T] are | |
| 5 | +// materialized in f32 for the backward pass — exactly mirroring the CPU | |
| 6 | +// reference so parity is bit-for-bit meaningful. | |
| 7 | +// | |
| 8 | +// Parallelization: one THREAD per query row (b,h,i) in fwd/dq (grid | |
| 9 | +// B*H*T), and one thread per KV row (b,h,j) in dkv. dkv iterates the | |
| 10 | +// q-heads sharing its kv-head, so dk/dv accumulate privately — no atomics, | |
| 11 | +// deterministic. | |
| 12 | +// | |
| 13 | +// Layouts: q/o [B,T,H*hd], k/v [B,T,Hkv*hd], P [B,H,T,T] row-major. | |
| 14 | +#include <metal_stdlib> | |
| 15 | +using namespace metal; | |
| 16 | + | |
| 17 | +struct AttnParams { | |
| 18 | + uint B, T, H, HKV, HD; | |
| 19 | + float scale; | |
| 20 | + uint causal; | |
| 21 | +}; | |
| 22 | + | |
| 23 | +kernel void attention_fwd_f32(device const float* Q [[buffer(0)]], | |
| 24 | + device const float* K [[buffer(1)]], | |
| 25 | + device const float* V [[buffer(2)]], | |
| 26 | + device float* O [[buffer(3)]], | |
| 27 | + device float* P [[buffer(4)]], | |
| 28 | + constant AttnParams& p [[buffer(5)]], | |
| 29 | + uint gid [[thread_position_in_grid]]) { | |
| 30 | + const uint i = gid % p.T; | |
| 31 | + const uint h = (gid / p.T) % p.H; | |
| 32 | + const uint b = gid / (p.T * p.H); | |
| 33 | + if (b >= p.B) return; | |
| 34 | + | |
| 35 | + const uint rep = p.H / p.HKV; | |
| 36 | + const uint hkv = h / rep; | |
| 37 | + const uint Cq = p.H * p.HD; | |
| 38 | + const uint Ckv = p.HKV * p.HD; | |
| 39 | + | |
| 40 | + device const float* qi = Q + (ulong(b) * p.T + i) * Cq + h * p.HD; | |
| 41 | + device float* prow_base = P + ((ulong(b) * p.H + h) * p.T + i) * p.T; | |
| 42 | + const uint jmax = p.causal ? i : p.T - 1; | |
| 43 | + | |
| 44 | + float m = -FLT_MAX; | |
| 45 | + for (uint j = 0; j <= jmax; ++j) { | |
| 46 | + device const float* kj = K + (ulong(b) * p.T + j) * Ckv + hkv * p.HD; | |
| 47 | + float s = 0.0f; | |
| 48 | + for (uint d = 0; d < p.HD; ++d) s = fma(qi[d], kj[d], s); | |
| 49 | + s *= p.scale; | |
| 50 | + prow_base[j] = s; | |
| 51 | + m = max(m, s); | |
| 52 | + } | |
| 53 | + float sum = 0.0f; | |
| 54 | + for (uint j = 0; j <= jmax; ++j) { | |
| 55 | + const float e = exp(prow_base[j] - m); | |
| 56 | + prow_base[j] = e; | |
| 57 | + sum += e; | |
| 58 | + } | |
| 59 | + const float inv = 1.0f / sum; | |
| 60 | + for (uint j = 0; j <= jmax; ++j) prow_base[j] *= inv; | |
| 61 | + for (uint j = jmax + 1; j < p.T; ++j) prow_base[j] = 0.0f; | |
| 62 | + | |
| 63 | + device float* oi = O + (ulong(b) * p.T + i) * Cq + h * p.HD; | |
| 64 | + for (uint d = 0; d < p.HD; ++d) oi[d] = 0.0f; | |
| 65 | + for (uint j = 0; j <= jmax; ++j) { | |
| 66 | + const float prob = prow_base[j]; | |
| 67 | + if (prob == 0.0f) continue; | |
| 68 | + device const float* vj = V + (ulong(b) * p.T + j) * Ckv + hkv * p.HD; | |
| 69 | + for (uint d = 0; d < p.HD; ++d) oi[d] = fma(prob, vj[d], oi[d]); | |
| 70 | + } | |
| 71 | +} | |
| 72 | + | |
| 73 | +// D[b,h,i] = dO_i · O_i. This is the FlashAttention-2 preprocessing term and | |
| 74 | +// it equals rowsum(dP ∘ P) exactly: | |
| 75 | +// Σ_j P_ij (dO_i · V_j) = dO_i · (Σ_j P_ij V_j) = dO_i · O_i | |
| 76 | +// Computing it once per query row instead of per (i,j) pair takes the dk/dv | |
| 77 | +// kernel from O(T³·hd) to O(T²·hd). | |
| 78 | +kernel void attention_bwd_d_f32(device const float* O [[buffer(0)]], | |
| 79 | + device const float* dO [[buffer(1)]], | |
| 80 | + device float* D [[buffer(2)]], | |
| 81 | + constant AttnParams& p [[buffer(3)]], | |
| 82 | + uint gid [[thread_position_in_grid]]) { | |
| 83 | + const uint i = gid % p.T; | |
| 84 | + const uint h = (gid / p.T) % p.H; | |
| 85 | + const uint b = gid / (p.T * p.H); | |
| 86 | + if (b >= p.B) return; | |
| 87 | + | |
| 88 | + const uint Cq = p.H * p.HD; | |
| 89 | + device const float* oi = O + (ulong(b) * p.T + i) * Cq + h * p.HD; | |
| 90 | + device const float* doi = dO + (ulong(b) * p.T + i) * Cq + h * p.HD; | |
| 91 | + float acc = 0.0f; | |
| 92 | + for (uint d = 0; d < p.HD; ++d) acc = fma(doi[d], oi[d], acc); | |
| 93 | + D[(ulong(b) * p.H + h) * p.T + i] = acc; | |
| 94 | +} | |
| 95 | + | |
| 96 | +// dq for one query row (b,h,i): dq_i = Σ_j dS_ij * K_j * scale, where | |
| 97 | +// dS = P ∘ (dP − D_i) and dP_ij = dO_i · V_j. | |
| 98 | +kernel void attention_bwd_dq_f32(device const float* Q [[buffer(0)]], | |
| 99 | + device const float* K [[buffer(1)]], | |
| 100 | + device const float* V [[buffer(2)]], | |
| 101 | + device const float* P [[buffer(3)]], | |
| 102 | + device const float* dO [[buffer(4)]], | |
| 103 | + device const float* D [[buffer(5)]], | |
| 104 | + device float* dQ [[buffer(6)]], | |
| 105 | + constant AttnParams& p [[buffer(7)]], | |
| 106 | + uint gid [[thread_position_in_grid]]) { | |
| 107 | + const uint i = gid % p.T; | |
| 108 | + const uint h = (gid / p.T) % p.H; | |
| 109 | + const uint b = gid / (p.T * p.H); | |
| 110 | + if (b >= p.B) return; | |
| 111 | + | |
| 112 | + const uint rep = p.H / p.HKV; | |
| 113 | + const uint hkv = h / rep; | |
| 114 | + const uint Cq = p.H * p.HD; | |
| 115 | + const uint Ckv = p.HKV * p.HD; | |
| 116 | + | |
| 117 | + device const float* prow = P + ((ulong(b) * p.H + h) * p.T + i) * p.T; | |
| 118 | + device const float* doi = dO + (ulong(b) * p.T + i) * Cq + h * p.HD; | |
| 119 | + device float* dqi = dQ + (ulong(b) * p.T + i) * Cq + h * p.HD; | |
| 120 | + | |
| 121 | + const float row_dot = D[(ulong(b) * p.H + h) * p.T + i]; | |
| 122 | + for (uint j = 0; j < p.T; ++j) { | |
| 123 | + const float prob = prow[j]; | |
| 124 | + if (prob == 0.0f) continue; | |
| 125 | + device const float* vj = V + (ulong(b) * p.T + j) * Ckv + hkv * p.HD; | |
| 126 | + device const float* kj = K + (ulong(b) * p.T + j) * Ckv + hkv * p.HD; | |
| 127 | + float dp = 0.0f; | |
| 128 | + for (uint d = 0; d < p.HD; ++d) dp = fma(doi[d], vj[d], dp); | |
| 129 | + const float ds = prob * (dp - row_dot) * p.scale; | |
| 130 | + for (uint d = 0; d < p.HD; ++d) dqi[d] = fma(ds, kj[d], dqi[d]); | |
| 131 | + } | |
| 132 | +} | |
| 133 | + | |
| 134 | +// dk/dv for one KV row (b,hkv,j): sums over the q-heads sharing this | |
| 135 | +// kv-head and all query rows i (P_ij = 0 above the diagonal already). | |
| 136 | +kernel void attention_bwd_dkv_f32(device const float* Q [[buffer(0)]], | |
| 137 | + device const float* K [[buffer(1)]], | |
| 138 | + device const float* V [[buffer(2)]], | |
| 139 | + device const float* P [[buffer(3)]], | |
| 140 | + device const float* dO [[buffer(4)]], | |
| 141 | + device const float* D [[buffer(5)]], | |
| 142 | + device float* dK [[buffer(6)]], | |
| 143 | + device float* dV [[buffer(7)]], | |
| 144 | + constant AttnParams& p [[buffer(8)]], | |
| 145 | + uint gid [[thread_position_in_grid]]) { | |
| 146 | + const uint j = gid % p.T; | |
| 147 | + const uint hkv = (gid / p.T) % p.HKV; | |
| 148 | + const uint b = gid / (p.T * p.HKV); | |
| 149 | + if (b >= p.B) return; | |
| 150 | + | |
| 151 | + const uint rep = p.H / p.HKV; | |
| 152 | + const uint Cq = p.H * p.HD; | |
| 153 | + const uint Ckv = p.HKV * p.HD; | |
| 154 | + | |
| 155 | + device const float* vj = V + (ulong(b) * p.T + j) * Ckv + hkv * p.HD; | |
| 156 | + device float* dkj = dK + (ulong(b) * p.T + j) * Ckv + hkv * p.HD; | |
| 157 | + device float* dvj = dV + (ulong(b) * p.T + j) * Ckv + hkv * p.HD; | |
| 158 | + | |
| 159 | + for (uint r = 0; r < rep; ++r) { | |
| 160 | + const uint h = hkv * rep + r; | |
| 161 | + for (uint i = 0; i < p.T; ++i) { | |
| 162 | + const float prob = P[((ulong(b) * p.H + h) * p.T + i) * p.T + j]; | |
| 163 | + if (prob == 0.0f) continue; | |
| 164 | + device const float* qi = Q + (ulong(b) * p.T + i) * Cq + h * p.HD; | |
| 165 | + device const float* doi = dO + (ulong(b) * p.T + i) * Cq + h * p.HD; | |
| 166 | + | |
| 167 | + const float row_dot = D[(ulong(b) * p.H + h) * p.T + i]; | |
| 168 | + float dp = 0.0f; | |
| 169 | + for (uint d = 0; d < p.HD; ++d) dp = fma(doi[d], vj[d], dp); | |
| 170 | + const float ds = prob * (dp - row_dot) * p.scale; | |
| 171 | + for (uint d = 0; d < p.HD; ++d) { | |
| 172 | + dvj[d] = fma(prob, doi[d], dvj[d]); | |
| 173 | + dkj[d] = fma(ds, qi[d], dkj[d]); | |
| 174 | + } | |
| 175 | + } | |
| 176 | + } | |
| 177 | +} | |
added
src/kernels/cross_entropy.metal
+82 −0
@@ -0,0 +1,82 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +// | |
| 3 | +// Fused softmax cross-entropy (llm.c fused_classifier pattern): one | |
| 4 | +// threadgroup per row computes online (max, sumexp), the loss in | |
| 5 | +// logsumexp form, and — in the same kernel — the logit gradient | |
| 6 | +// (softmax − onehot) · inv_n, accumulated into dlogits. The full softmax | |
| 7 | +// is never materialized. targets use ignore_index = -1 (loss 0, grad 0). | |
| 8 | +// | |
| 9 | +// losses[row] receives the per-row loss; the host reduces (sum_f32) and | |
| 10 | +// scales by 1/n_valid. | |
| 11 | +#include <metal_stdlib> | |
| 12 | +using namespace metal; | |
| 13 | + | |
| 14 | +struct CEParams { | |
| 15 | + uint V; | |
| 16 | + float inv_n; // 1 / n_valid | |
| 17 | + uint want_grad; // 0: loss only | |
| 18 | +}; | |
| 19 | + | |
| 20 | +kernel void cross_entropy_f32(device const float* logits [[buffer(0)]], | |
| 21 | + device const int* targets [[buffer(1)]], | |
| 22 | + device float* losses [[buffer(2)]], | |
| 23 | + device float* dlogits [[buffer(3)]], | |
| 24 | + constant CEParams& p [[buffer(4)]], | |
| 25 | + uint row [[threadgroup_position_in_grid]], | |
| 26 | + uint lid [[thread_index_in_threadgroup]], | |
| 27 | + uint tg_size [[threads_per_threadgroup]], | |
| 28 | + uint lane [[thread_index_in_simdgroup]], | |
| 29 | + uint simd_idx [[simdgroup_index_in_threadgroup]], | |
| 30 | + uint n_simds [[simdgroups_per_threadgroup]]) { | |
| 31 | + const int tgt = targets[row]; | |
| 32 | + device const float* x = logits + ulong(row) * p.V; | |
| 33 | + | |
| 34 | + if (tgt < 0) { | |
| 35 | + if (lid == 0) losses[row] = 0.0f; | |
| 36 | + return; | |
| 37 | + } | |
| 38 | + | |
| 39 | + // online (m, l) as in softmax.metal | |
| 40 | + float m = -FLT_MAX; | |
| 41 | + float l = 0.0f; | |
| 42 | + for (uint j = lid; j < p.V; j += tg_size) { | |
| 43 | + const float v = x[j]; | |
| 44 | + const float m_new = max(m, v); | |
| 45 | + l = l * exp(m - m_new) + exp(v - m_new); | |
| 46 | + m = m_new; | |
| 47 | + } | |
| 48 | + float m_simd = simd_max(m); | |
| 49 | + float l_simd = simd_sum(l * exp(m - m_simd)); | |
| 50 | + | |
| 51 | + threadgroup float tg_m[32]; | |
| 52 | + threadgroup float tg_l[32]; | |
| 53 | + if (simd_is_first()) { | |
| 54 | + tg_m[simd_idx] = m_simd; | |
| 55 | + tg_l[simd_idx] = l_simd; | |
| 56 | + } | |
| 57 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 58 | + float m_row, l_row; | |
| 59 | + { | |
| 60 | + const uint i = min(lane, n_simds - 1); | |
| 61 | + const float mi = tg_m[i]; | |
| 62 | + const float li = tg_l[i]; | |
| 63 | + m_row = simd_max(mi); | |
| 64 | + const float contrib = (lane < n_simds) ? li * exp(mi - m_row) : 0.0f; | |
| 65 | + l_row = simd_sum(contrib); | |
| 66 | + } | |
| 67 | + | |
| 68 | + if (lid == 0) { | |
| 69 | + // loss = logsumexp − logit[target] | |
| 70 | + losses[row] = m_row + log(l_row) - x[uint(tgt)]; | |
| 71 | + } | |
| 72 | + | |
| 73 | + if (p.want_grad) { | |
| 74 | + const float inv_sum = 1.0f / l_row; | |
| 75 | + device float* drow = dlogits + ulong(row) * p.V; | |
| 76 | + for (uint j = lid; j < p.V; j += tg_size) { | |
| 77 | + const float prob = exp(x[j] - m_row) * inv_sum; | |
| 78 | + const float ind = (j == uint(tgt)) ? 1.0f : 0.0f; | |
| 79 | + drow[j] += (prob - ind) * p.inv_n; | |
| 80 | + } | |
| 81 | + } | |
| 82 | +} | |
added
src/kernels/elementwise.metal
+142 −0
@@ -0,0 +1,142 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +// | |
| 3 | +// Elementwise kernels. Flat 1-D dispatch via dispatchThreads (non-uniform | |
| 4 | +// threadgroups — no bounds-check tail), threadgroup size a multiple of 32. | |
| 5 | +// Purely bandwidth-bound; float4-vectorized variants come with the M5 | |
| 6 | +// fusion pass. | |
| 7 | +#include <metal_stdlib> | |
| 8 | +using namespace metal; | |
| 9 | + | |
| 10 | +kernel void add_f32(device const float* a [[buffer(0)]], | |
| 11 | + device const float* b [[buffer(1)]], | |
| 12 | + device float* out [[buffer(2)]], | |
| 13 | + uint gid [[thread_position_in_grid]]) { | |
| 14 | + out[gid] = a[gid] + b[gid]; | |
| 15 | +} | |
| 16 | + | |
| 17 | +kernel void mul_f32(device const float* a [[buffer(0)]], | |
| 18 | + device const float* b [[buffer(1)]], | |
| 19 | + device float* out [[buffer(2)]], | |
| 20 | + uint gid [[thread_position_in_grid]]) { | |
| 21 | + out[gid] = a[gid] * b[gid]; | |
| 22 | +} | |
| 23 | + | |
| 24 | +kernel void scale_f32(device const float* a [[buffer(0)]], | |
| 25 | + device float* out [[buffer(1)]], | |
| 26 | + constant float& s [[buffer(2)]], | |
| 27 | + uint gid [[thread_position_in_grid]]) { | |
| 28 | + out[gid] = a[gid] * s; | |
| 29 | +} | |
| 30 | + | |
| 31 | +// out[i] = x[i] + bias[i % C] — row-broadcast bias | |
| 32 | +kernel void add_bias_f32(device const float* x [[buffer(0)]], | |
| 33 | + device const float* bias [[buffer(1)]], | |
| 34 | + device float* out [[buffer(2)]], | |
| 35 | + constant uint& C [[buffer(3)]], | |
| 36 | + uint gid [[thread_position_in_grid]]) { | |
| 37 | + out[gid] = x[gid] + bias[gid % C]; | |
| 38 | +} | |
| 39 | + | |
| 40 | +kernel void silu_f32(device const float* x [[buffer(0)]], | |
| 41 | + device float* out [[buffer(1)]], | |
| 42 | + uint gid [[thread_position_in_grid]]) { | |
| 43 | + const float v = x[gid]; | |
| 44 | + out[gid] = v / (1.0f + exp(-v)); | |
| 45 | +} | |
| 46 | + | |
| 47 | +kernel void gelu_f32(device const float* x [[buffer(0)]], | |
| 48 | + device float* out [[buffer(1)]], | |
| 49 | + uint gid [[thread_position_in_grid]]) { | |
| 50 | + const float v = x[gid]; | |
| 51 | + const float k = 0.7978845608028654f; // sqrt(2/pi) | |
| 52 | + out[gid] = 0.5f * v * (1.0f + precise::tanh(k * (v + 0.044715f * v * v * v))); | |
| 53 | +} | |
| 54 | + | |
| 55 | +// ---- backward / accumulation kernels (all ACCUMULATE into their outputs) ---- | |
| 56 | + | |
| 57 | +kernel void accum_f32(device float* dst [[buffer(0)]], | |
| 58 | + device const float* src [[buffer(1)]], | |
| 59 | + uint gid [[thread_position_in_grid]]) { | |
| 60 | + dst[gid] += src[gid]; | |
| 61 | +} | |
| 62 | + | |
| 63 | +// dst += src * s ; s in a 1-element buffer so it can be produced on-GPU | |
| 64 | +// (e.g. d(loss) scaling) without a sync. | |
| 65 | +kernel void axpy_f32(device float* dst [[buffer(0)]], | |
| 66 | + device const float* src [[buffer(1)]], | |
| 67 | + device const float* s [[buffer(2)]], | |
| 68 | + uint gid [[thread_position_in_grid]]) { | |
| 69 | + dst[gid] = fma(src[gid], s[0], dst[gid]); | |
| 70 | +} | |
| 71 | + | |
| 72 | +kernel void silu_bwd_f32(device const float* x [[buffer(0)]], | |
| 73 | + device const float* dout [[buffer(1)]], | |
| 74 | + device float* dx [[buffer(2)]], | |
| 75 | + uint gid [[thread_position_in_grid]]) { | |
| 76 | + const float v = x[gid]; | |
| 77 | + const float sig = 1.0f / (1.0f + exp(-v)); | |
| 78 | + dx[gid] = fma(dout[gid], sig * (1.0f + v * (1.0f - sig)), dx[gid]); | |
| 79 | +} | |
| 80 | + | |
| 81 | +kernel void gelu_bwd_f32(device const float* x [[buffer(0)]], | |
| 82 | + device const float* dout [[buffer(1)]], | |
| 83 | + device float* dx [[buffer(2)]], | |
| 84 | + uint gid [[thread_position_in_grid]]) { | |
| 85 | + const float v = x[gid]; | |
| 86 | + const float k = 0.7978845608028654f; | |
| 87 | + const float u = k * (v + 0.044715f * v * v * v); | |
| 88 | + const float t = precise::tanh(u); | |
| 89 | + const float du = k * (1.0f + 3.0f * 0.044715f * v * v); | |
| 90 | + dx[gid] = fma(dout[gid], 0.5f * (1.0f + t) + 0.5f * v * (1.0f - t * t) * du, dx[gid]); | |
| 91 | +} | |
| 92 | + | |
| 93 | +// dbias[j] += sum_rows dout[i,j] — one thread per column, strided rows. | |
| 94 | +// Column-major walk is uncoalesced but this kernel is a tiny fraction of a | |
| 95 | +// step; revisit in the M5 fusion pass if it ever shows in a trace. | |
| 96 | +kernel void add_bias_bwd_f32(device const float* dout [[buffer(0)]], | |
| 97 | + device float* dbias [[buffer(1)]], | |
| 98 | + constant uint2& nc [[buffer(2)]], // (N, C) | |
| 99 | + uint j [[thread_position_in_grid]]) { | |
| 100 | + float acc = 0.0f; | |
| 101 | + for (uint i = 0; i < nc.x; ++i) acc += dout[ulong(i) * nc.y + j]; | |
| 102 | + dbias[j] += acc; | |
| 103 | +} | |
| 104 | + | |
| 105 | +// ---- RoPE (interleaved pairs; INV=true applies the inverse rotation and | |
| 106 | +// accumulates — the backward pass) ---- | |
| 107 | + | |
| 108 | +constant bool ROPE_INV [[function_constant(2)]]; | |
| 109 | + | |
| 110 | +struct RopeParams { | |
| 111 | + uint T, H, HD; | |
| 112 | + float theta; | |
| 113 | + uint pos_offset; | |
| 114 | +}; | |
| 115 | + | |
| 116 | +kernel void rope_f32(device const float* x [[buffer(0)]], | |
| 117 | + device float* out [[buffer(1)]], | |
| 118 | + constant RopeParams& p [[buffer(2)]], | |
| 119 | + uint gid [[thread_position_in_grid]]) { | |
| 120 | + // gid indexes (bt, h, k) pairs: one thread per rotated pair | |
| 121 | + const uint pairs_per_row = p.H * (p.HD / 2); | |
| 122 | + const uint bt = gid / pairs_per_row; | |
| 123 | + const uint rem = gid % pairs_per_row; | |
| 124 | + const uint h = rem / (p.HD / 2); | |
| 125 | + const uint k = rem % (p.HD / 2); | |
| 126 | + | |
| 127 | + const float pos = float(bt % p.T + p.pos_offset); | |
| 128 | + const float freq = pow(p.theta, -2.0f * float(k) / float(p.HD)); | |
| 129 | + const float angle = pos * freq; | |
| 130 | + const float c = cos(angle); | |
| 131 | + const float s = ROPE_INV ? -sin(angle) : sin(angle); | |
| 132 | + | |
| 133 | + const ulong i0 = ulong(bt) * (p.H * p.HD) + h * p.HD + 2 * k; | |
| 134 | + const float x0 = x[i0], x1 = x[i0 + 1]; | |
| 135 | + if (ROPE_INV) { | |
| 136 | + out[i0] += x0 * c - x1 * s; | |
| 137 | + out[i0 + 1] += x0 * s + x1 * c; | |
| 138 | + } else { | |
| 139 | + out[i0] = x0 * c - x1 * s; | |
| 140 | + out[i0 + 1] = x0 * s + x1 * c; | |
| 141 | + } | |
| 142 | +} | |
added
src/kernels/embedding.metal
+28 −0
@@ -0,0 +1,28 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +// | |
| 3 | +// Embedding gather + scatter-grad. ids are i32. Forward: one thread per | |
| 4 | +// (token, channel). Backward: one thread per (row, channel) accumulating | |
| 5 | +// with device atomic_float — repeated tokens collide, so order is | |
| 6 | +// nondeterministic (sum of floats); deterministic mode routes embedding | |
| 7 | +// backward through the CPU reference instead. | |
| 8 | +#include <metal_stdlib> | |
| 9 | +using namespace metal; | |
| 10 | + | |
| 11 | +kernel void embedding_fwd_f32(device const float* W [[buffer(0)]], | |
| 12 | + device const int* ids [[buffer(1)]], | |
| 13 | + device float* out [[buffer(2)]], | |
| 14 | + constant uint& C [[buffer(3)]], | |
| 15 | + uint2 gid [[thread_position_in_grid]]) { | |
| 16 | + // gid.x = channel, gid.y = token index | |
| 17 | + out[ulong(gid.y) * C + gid.x] = W[ulong(ids[gid.y]) * C + gid.x]; | |
| 18 | +} | |
| 19 | + | |
| 20 | +kernel void embedding_bwd_f32(device const int* ids [[buffer(0)]], | |
| 21 | + device const float* dout [[buffer(1)]], | |
| 22 | + device atomic_float* dW [[buffer(2)]], | |
| 23 | + constant uint& C [[buffer(3)]], | |
| 24 | + uint2 gid [[thread_position_in_grid]]) { | |
| 25 | + const float g = dout[ulong(gid.y) * C + gid.x]; | |
| 26 | + atomic_fetch_add_explicit(&dW[ulong(ids[gid.y]) * C + gid.x], g, | |
| 27 | + memory_order_relaxed); | |
| 28 | +} | |
added
src/kernels/flash_attention.metal
+329 −0
@@ -0,0 +1,329 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +// | |
| 3 | +// Fused (flash-style) causal attention with GQA — forward + backward, f32. | |
| 4 | +// Nothing of size T² is ever written: the forward keeps the online-softmax | |
| 5 | +// state in registers and stores only L = m + log(l), one float per query | |
| 6 | +// row, which the backward uses to recompute P tile-by-tile. That is what | |
| 7 | +// lets context length scale — the unfused path in attention.metal needs | |
| 8 | +// B·H·T² floats of probabilities per layer (2.1 GB for batch 64 × T 1024 × | |
| 9 | +// 8 heads), this needs B·H·T. | |
| 10 | +// | |
| 11 | +// Parallelization: one THREAD per output row — | |
| 12 | +// forward / dQ : one thread per (b, h, i) grid B·H·T | |
| 13 | +// dK,dV : one thread per (b, hkv, j) grid B·HKV·T | |
| 14 | +// so dK/dV accumulate privately per KV row (no atomics, deterministic) and | |
| 15 | +// each simdgroup's 32 threads are consecutive query rows of the same head, | |
| 16 | +// which makes their K/V reads a broadcast that the cache serves once. | |
| 17 | +// | |
| 18 | +// head_dim is a TEMPLATE parameter, not a runtime value: the per-thread | |
| 19 | +// q/o/accumulator arrays must be compile-time sized and fully unrolled or | |
| 20 | +// they land in thread-local (device-backed) memory instead of registers. | |
| 21 | +// Instantiated per supported head_dim below; the host falls back to the | |
| 22 | +// unfused kernel for other sizes. | |
| 23 | +// | |
| 24 | +// Online softmax (FlashAttention-2, RESEARCH.md §6): | |
| 25 | +// m_new = max(m, s) ; corr = exp(m − m_new) ; e = exp(s − m_new) | |
| 26 | +// l = l·corr + e ; o = o·corr + e·v ; divide by l once at the end | |
| 27 | +// m starts at −FLT_MAX, never −INFINITY: exp(−inf − (−inf)) is NaN. | |
| 28 | +#include <metal_stdlib> | |
| 29 | +using namespace metal; | |
| 30 | + | |
| 31 | +struct FlashParams { | |
| 32 | + uint B, T, H, HKV; | |
| 33 | + float scale; | |
| 34 | + uint causal; | |
| 35 | +}; | |
| 36 | + | |
| 37 | +// ---------------------------------------------------------------- forward | |
| 38 | + | |
| 39 | +// Threadgroup = TGQ consecutive query rows of one (b, h). K/V arrive through | |
| 40 | +// threadgroup memory in blocks of BKV rows, staged cooperatively, so each | |
| 41 | +// K/V element is fetched from device once per threadgroup instead of once | |
| 42 | +// per thread. Causal blocks entirely above the diagonal are skipped outright | |
| 43 | +// (kb_lim), and only the diagonal block pays the per-element mask test. | |
| 44 | +enum : uint { TGQ = 64, BKV = 16 }; | |
| 45 | + | |
| 46 | +template <uint HD> | |
| 47 | +kernel void flash_attn_fwd(device const float* Q [[buffer(0)]], | |
| 48 | + device const float* K [[buffer(1)]], | |
| 49 | + device const float* V [[buffer(2)]], | |
| 50 | + device float* O [[buffer(3)]], | |
| 51 | + device float* L [[buffer(4)]], | |
| 52 | + constant FlashParams& p [[buffer(5)]], | |
| 53 | + uint tgid [[threadgroup_position_in_grid]], | |
| 54 | + uint tid [[thread_index_in_threadgroup]]) { | |
| 55 | + threadgroup float Ks[BKV * HD]; | |
| 56 | + threadgroup float Vs[BKV * HD]; | |
| 57 | + | |
| 58 | + const uint q_blocks = (p.T + TGQ - 1) / TGQ; | |
| 59 | + const uint qb = tgid % q_blocks; | |
| 60 | + const uint h = (tgid / q_blocks) % p.H; | |
| 61 | + const uint b = tgid / (q_blocks * p.H); | |
| 62 | + | |
| 63 | + const uint i = qb * TGQ + tid; | |
| 64 | + const bool active = (i < p.T) && (b < p.B); | |
| 65 | + | |
| 66 | + const uint hkv = h / (p.H / p.HKV); | |
| 67 | + const uint Cq = p.H * HD; | |
| 68 | + const uint Ckv = p.HKV * HD; | |
| 69 | + | |
| 70 | + float q[HD], o[HD]; | |
| 71 | + if (active) { | |
| 72 | + device const float* qi = Q + (ulong(b) * p.T + i) * Cq + h * HD; | |
| 73 | +#pragma clang loop unroll(full) | |
| 74 | + for (uint d = 0; d < HD; ++d) { | |
| 75 | + q[d] = qi[d]; | |
| 76 | + o[d] = 0.0f; | |
| 77 | + } | |
| 78 | + } else { | |
| 79 | +#pragma clang loop unroll(full) | |
| 80 | + for (uint d = 0; d < HD; ++d) { | |
| 81 | + q[d] = 0.0f; | |
| 82 | + o[d] = 0.0f; | |
| 83 | + } | |
| 84 | + } | |
| 85 | + | |
| 86 | + float m = -FLT_MAX; | |
| 87 | + float l = 0.0f; | |
| 88 | + | |
| 89 | + // Causal: this block's largest query index bounds the KV blocks we touch. | |
| 90 | + const uint row_max = min(qb * TGQ + TGQ - 1, p.T - 1); | |
| 91 | + const uint j_end = p.causal ? (row_max + 1) : p.T; | |
| 92 | + const uint diag_start = p.causal ? (qb * TGQ) : p.T; // blocks below need no mask | |
| 93 | + | |
| 94 | + for (uint jb = 0; jb < j_end; jb += BKV) { | |
| 95 | + const uint block_len = min(BKV, j_end - jb); | |
| 96 | + | |
| 97 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 98 | + for (uint e = tid; e < block_len * HD; e += TGQ) { | |
| 99 | + const uint jr = e / HD; | |
| 100 | + const uint d = e % HD; | |
| 101 | + const ulong src = (ulong(b) * p.T + jb + jr) * Ckv + hkv * HD + d; | |
| 102 | + Ks[jr * HD + d] = K[src]; | |
| 103 | + Vs[jr * HD + d] = V[src]; | |
| 104 | + } | |
| 105 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 106 | + | |
| 107 | + if (!active) continue; | |
| 108 | + const bool needs_mask = p.causal && (jb + BKV > diag_start); | |
| 109 | + for (uint jr = 0; jr < block_len; ++jr) { | |
| 110 | + const uint j = jb + jr; | |
| 111 | + if (needs_mask && j > i) break; // rest of the block is masked too | |
| 112 | + | |
| 113 | + threadgroup const float* kj = Ks + jr * HD; | |
| 114 | + float s = 0.0f; | |
| 115 | +#pragma clang loop unroll(full) | |
| 116 | + for (uint d = 0; d < HD; ++d) s = fma(q[d], kj[d], s); | |
| 117 | + s *= p.scale; | |
| 118 | + | |
| 119 | + const float m_new = max(m, s); | |
| 120 | + const float corr = exp(m - m_new); // 0 on the first iteration | |
| 121 | + const float e = exp(s - m_new); | |
| 122 | + l = l * corr + e; | |
| 123 | + | |
| 124 | + threadgroup const float* vj = Vs + jr * HD; | |
| 125 | +#pragma clang loop unroll(full) | |
| 126 | + for (uint d = 0; d < HD; ++d) o[d] = fma(o[d], corr, e * vj[d]); | |
| 127 | + m = m_new; | |
| 128 | + } | |
| 129 | + } | |
| 130 | + | |
| 131 | + if (!active) return; | |
| 132 | + const float inv = (l > 0.0f) ? (1.0f / l) : 0.0f; | |
| 133 | + device float* oi = O + (ulong(b) * p.T + i) * Cq + h * HD; | |
| 134 | +#pragma clang loop unroll(full) | |
| 135 | + for (uint d = 0; d < HD; ++d) oi[d] = o[d] * inv; | |
| 136 | + // logsumexp; a fully-masked row would give -inf, guarded like the divide | |
| 137 | + L[(ulong(b) * p.H + h) * p.T + i] = (l > 0.0f) ? (m + log(l)) : 0.0f; | |
| 138 | +} | |
| 139 | + | |
| 140 | +// --------------------------------------------------------------- backward | |
| 141 | +// D[b,h,i] = dO_i · O_i == rowsum(dP ∘ P) (FA2 identity), computed by | |
| 142 | +// attention_bwd_d_f32 in attention.metal and passed in here. | |
| 143 | + | |
| 144 | +template <uint HD> | |
| 145 | +kernel void flash_attn_bwd_dq(device const float* Q [[buffer(0)]], | |
| 146 | + device const float* K [[buffer(1)]], | |
| 147 | + device const float* V [[buffer(2)]], | |
| 148 | + device const float* dO [[buffer(3)]], | |
| 149 | + device const float* L [[buffer(4)]], | |
| 150 | + device const float* D [[buffer(5)]], | |
| 151 | + device float* dQ [[buffer(6)]], | |
| 152 | + constant FlashParams& p [[buffer(7)]], | |
| 153 | + uint gid [[thread_position_in_grid]]) { | |
| 154 | + const uint i = gid % p.T; | |
| 155 | + const uint h = (gid / p.T) % p.H; | |
| 156 | + const uint b = gid / (p.T * p.H); | |
| 157 | + if (b >= p.B) return; | |
| 158 | + | |
| 159 | + const uint hkv = h / (p.H / p.HKV); | |
| 160 | + const uint Cq = p.H * HD; | |
| 161 | + const uint Ckv = p.HKV * HD; | |
| 162 | + | |
| 163 | + device const float* qi = Q + (ulong(b) * p.T + i) * Cq + h * HD; | |
| 164 | + device const float* doi = dO + (ulong(b) * p.T + i) * Cq + h * HD; | |
| 165 | + | |
| 166 | + float q[HD], dq[HD], go[HD]; | |
| 167 | +#pragma clang loop unroll(full) | |
| 168 | + for (uint d = 0; d < HD; ++d) { | |
| 169 | + q[d] = qi[d]; | |
| 170 | + go[d] = doi[d]; | |
| 171 | + dq[d] = 0.0f; | |
| 172 | + } | |
| 173 | + | |
| 174 | + const float li = L[(ulong(b) * p.H + h) * p.T + i]; | |
| 175 | + const float di = D[(ulong(b) * p.H + h) * p.T + i]; | |
| 176 | + const uint jmax = p.causal ? i : (p.T - 1); | |
| 177 | + | |
| 178 | + for (uint j = 0; j <= jmax; ++j) { | |
| 179 | + device const float* kj = K + (ulong(b) * p.T + j) * Ckv + hkv * HD; | |
| 180 | + device const float* vj = V + (ulong(b) * p.T + j) * Ckv + hkv * HD; | |
| 181 | + | |
| 182 | + float s = 0.0f, dp = 0.0f; | |
| 183 | +#pragma clang loop unroll(full) | |
| 184 | + for (uint d = 0; d < HD; ++d) { | |
| 185 | + s = fma(q[d], kj[d], s); | |
| 186 | + dp = fma(go[d], vj[d], dp); | |
| 187 | + } | |
| 188 | + const float prob = exp(s * p.scale - li); // recomputed, never stored | |
| 189 | + const float ds = prob * (dp - di) * p.scale; | |
| 190 | +#pragma clang loop unroll(full) | |
| 191 | + for (uint d = 0; d < HD; ++d) dq[d] = fma(ds, kj[d], dq[d]); | |
| 192 | + } | |
| 193 | + | |
| 194 | + device float* dqi = dQ + (ulong(b) * p.T + i) * Cq + h * HD; | |
| 195 | +#pragma clang loop unroll(full) | |
| 196 | + for (uint d = 0; d < HD; ++d) dqi[d] += dq[d]; | |
| 197 | +} | |
| 198 | + | |
| 199 | +// dK and dV are separate kernels on purpose. Combined, one thread holds | |
| 200 | +// dk[HD] + dv[HD] and spills: measured 150 ms -> 118 ms for gpt-10m just by | |
| 201 | +// moving the read-only k/v out of registers, so the accumulators matter too. | |
| 202 | +// Split, each thread carries a single HD-sized accumulator; the price is | |
| 203 | +// recomputing the q·k dot in both kernels, which is cheaper than the spill. | |
| 204 | + | |
| 205 | +template <uint HD> | |
| 206 | +kernel void flash_attn_bwd_dv(device const float* Q [[buffer(0)]], | |
| 207 | + device const float* K [[buffer(1)]], | |
| 208 | + device const float* dO [[buffer(2)]], | |
| 209 | + device const float* L [[buffer(3)]], | |
| 210 | + device float* dV [[buffer(4)]], | |
| 211 | + constant FlashParams& p [[buffer(5)]], | |
| 212 | + uint gid [[thread_position_in_grid]]) { | |
| 213 | + const uint j = gid % p.T; | |
| 214 | + const uint hkv = (gid / p.T) % p.HKV; | |
| 215 | + const uint b = gid / (p.T * p.HKV); | |
| 216 | + if (b >= p.B) return; | |
| 217 | + | |
| 218 | + const uint rep = p.H / p.HKV; | |
| 219 | + const uint Cq = p.H * HD; | |
| 220 | + const uint Ckv = p.HKV * HD; | |
| 221 | + device const float* kj = K + (ulong(b) * p.T + j) * Ckv + hkv * HD; | |
| 222 | + | |
| 223 | + float dv[HD]; | |
| 224 | +#pragma clang loop unroll(full) | |
| 225 | + for (uint d = 0; d < HD; ++d) dv[d] = 0.0f; | |
| 226 | + | |
| 227 | + const uint imin = p.causal ? j : 0; | |
| 228 | + for (uint r = 0; r < rep; ++r) { | |
| 229 | + const uint h = hkv * rep + r; | |
| 230 | + device const float* Lh = L + (ulong(b) * p.H + h) * p.T; | |
| 231 | + for (uint i = imin; i < p.T; ++i) { | |
| 232 | + device const float* qi = Q + (ulong(b) * p.T + i) * Cq + h * HD; | |
| 233 | + device const float* doi = dO + (ulong(b) * p.T + i) * Cq + h * HD; | |
| 234 | + float s = 0.0f; | |
| 235 | +#pragma clang loop unroll(full) | |
| 236 | + for (uint d = 0; d < HD; ++d) s = fma(qi[d], kj[d], s); | |
| 237 | + const float prob = exp(s * p.scale - Lh[i]); | |
| 238 | +#pragma clang loop unroll(full) | |
| 239 | + for (uint d = 0; d < HD; ++d) dv[d] = fma(prob, doi[d], dv[d]); | |
| 240 | + } | |
| 241 | + } | |
| 242 | + | |
| 243 | + device float* dvj = dV + (ulong(b) * p.T + j) * Ckv + hkv * HD; | |
| 244 | +#pragma clang loop unroll(full) | |
| 245 | + for (uint d = 0; d < HD; ++d) dvj[d] += dv[d]; | |
| 246 | +} | |
| 247 | + | |
| 248 | +template <uint HD> | |
| 249 | +kernel void flash_attn_bwd_dk(device const float* Q [[buffer(0)]], | |
| 250 | + device const float* K [[buffer(1)]], | |
| 251 | + device const float* V [[buffer(2)]], | |
| 252 | + device const float* dO [[buffer(3)]], | |
| 253 | + device const float* L [[buffer(4)]], | |
| 254 | + device const float* D [[buffer(5)]], | |
| 255 | + device float* dK [[buffer(6)]], | |
| 256 | + constant FlashParams& p [[buffer(7)]], | |
| 257 | + uint gid [[thread_position_in_grid]]) { | |
| 258 | + const uint j = gid % p.T; | |
| 259 | + const uint hkv = (gid / p.T) % p.HKV; | |
| 260 | + const uint b = gid / (p.T * p.HKV); | |
| 261 | + if (b >= p.B) return; | |
| 262 | + | |
| 263 | + const uint rep = p.H / p.HKV; | |
| 264 | + const uint Cq = p.H * HD; | |
| 265 | + const uint Ckv = p.HKV * HD; | |
| 266 | + device const float* kj = K + (ulong(b) * p.T + j) * Ckv + hkv * HD; | |
| 267 | + device const float* vj = V + (ulong(b) * p.T + j) * Ckv + hkv * HD; | |
| 268 | + | |
| 269 | + float dk[HD]; | |
| 270 | +#pragma clang loop unroll(full) | |
| 271 | + for (uint d = 0; d < HD; ++d) dk[d] = 0.0f; | |
| 272 | + | |
| 273 | + const uint imin = p.causal ? j : 0; | |
| 274 | + for (uint r = 0; r < rep; ++r) { | |
| 275 | + const uint h = hkv * rep + r; | |
| 276 | + device const float* Lh = L + (ulong(b) * p.H + h) * p.T; | |
| 277 | + device const float* Dh = D + (ulong(b) * p.H + h) * p.T; | |
| 278 | + for (uint i = imin; i < p.T; ++i) { | |
| 279 | + device const float* qi = Q + (ulong(b) * p.T + i) * Cq + h * HD; | |
| 280 | + device const float* doi = dO + (ulong(b) * p.T + i) * Cq + h * HD; | |
| 281 | + float s = 0.0f, dp = 0.0f; | |
| 282 | +#pragma clang loop unroll(full) | |
| 283 | + for (uint d = 0; d < HD; ++d) { | |
| 284 | + s = fma(qi[d], kj[d], s); | |
| 285 | + dp = fma(doi[d], vj[d], dp); | |
| 286 | + } | |
| 287 | + const float prob = exp(s * p.scale - Lh[i]); | |
| 288 | + const float ds = prob * (dp - Dh[i]) * p.scale; | |
| 289 | +#pragma clang loop unroll(full) | |
| 290 | + for (uint d = 0; d < HD; ++d) dk[d] = fma(ds, qi[d], dk[d]); | |
| 291 | + } | |
| 292 | + } | |
| 293 | + | |
| 294 | + device float* dkj = dK + (ulong(b) * p.T + j) * Ckv + hkv * HD; | |
| 295 | +#pragma clang loop unroll(full) | |
| 296 | + for (uint d = 0; d < HD; ++d) dkj[d] += dk[d]; | |
| 297 | +} | |
| 298 | + | |
| 299 | +// ---------------------------------------------------------- instantiations | |
| 300 | +// Every config in configs/ uses head_dim 64 (d_model / n_heads); the others | |
| 301 | +// are here so the fused path covers common variants. Unsupported sizes fall | |
| 302 | +// back to the unfused kernels on the host side. | |
| 303 | +#define INSTANTIATE_FLASH(HD) \ | |
| 304 | + template [[host_name("flash_attn_fwd_f32_hd" #HD)]] kernel void \ | |
| 305 | + flash_attn_fwd<HD>(device const float*, device const float*, device const float*, \ | |
| 306 | + device float*, device float*, constant FlashParams&, uint, \ | |
| 307 | + uint); \ | |
| 308 | + template [[host_name("flash_attn_bwd_dq_f32_hd" #HD)]] kernel void \ | |
| 309 | + flash_attn_bwd_dq<HD>(device const float*, device const float*, \ | |
| 310 | + device const float*, device const float*, \ | |
| 311 | + device const float*, device const float*, device float*, \ | |
| 312 | + constant FlashParams&, uint); \ | |
| 313 | + template [[host_name("flash_attn_bwd_dv_f32_hd" #HD)]] kernel void \ | |
| 314 | + flash_attn_bwd_dv<HD>(device const float*, device const float*, \ | |
| 315 | + device const float*, device const float*, device float*, \ | |
| 316 | + constant FlashParams&, uint); \ | |
| 317 | + template [[host_name("flash_attn_bwd_dk_f32_hd" #HD)]] kernel void \ | |
| 318 | + flash_attn_bwd_dk<HD>(device const float*, device const float*, \ | |
| 319 | + device const float*, device const float*, \ | |
| 320 | + device const float*, device const float*, device float*, \ | |
| 321 | + constant FlashParams&, uint); | |
| 322 | + | |
| 323 | +INSTANTIATE_FLASH(16) | |
| 324 | +INSTANTIATE_FLASH(32) | |
| 325 | +INSTANTIATE_FLASH(48) | |
| 326 | +INSTANTIATE_FLASH(64) | |
| 327 | +INSTANTIATE_FLASH(80) | |
| 328 | +INSTANTIATE_FLASH(96) | |
| 329 | +INSTANTIATE_FLASH(128) | |
added
src/kernels/flash_attention_mma.metal
+720 −0
@@ -0,0 +1,720 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +// | |
| 3 | +// Tiled flash attention forward using simdgroup_matrix 8x8 fragments. | |
| 4 | +// Same algorithm and same outputs as flash_attn_fwd in flash_attention.metal | |
| 5 | +// (online softmax, stores only L = m + log(l)); the difference is that QK^T | |
| 6 | +// and PV are matrix multiplies over threadgroup-staged tiles instead of | |
| 7 | +// per-thread scalar dot products, which removes the serial FMA dependency | |
| 8 | +// chain that capped the scalar kernel at ~1.6 TFLOPs. | |
| 9 | +// | |
| 10 | +// Layout (BQ=32, BK=16, 4 simdgroups = 128 threads), split-Q per FA2: | |
| 11 | +// simdgroup s owns query rows [8s, 8s+8) — exactly one fragment row — and | |
| 12 | +// holds Q and the O accumulator in registers for the whole kernel | |
| 13 | +// (2 * HD/8 fragments = 32 floats/lane at HD=64). K/V tiles pass through | |
| 14 | +// threadgroup memory; the S/P tile round-trips through it so the softmax | |
| 15 | +// reductions can run on plain threads. | |
| 16 | +// | |
| 17 | +// THE PER-ROW RESCALE. Online softmax needs O <- diag(corr) * O every KV | |
| 18 | +// block, but MSL leaves the element->lane mapping of a simdgroup_matrix | |
| 19 | +// unspecified, so a lane cannot know which row its registers belong to. | |
| 20 | +// MLX reverse-engineers the mapping; instead this builds an 8x8 diagonal | |
| 21 | +// matrix holding corr in threadgroup memory and applies it with an MMA. | |
| 22 | +// That is spec-clean, costs HD/8 extra MMAs per block (~25% more MMA work, | |
| 23 | +// measured cheaper than the alternatives), and keeps O in registers — the | |
| 24 | +// point of the exercise, since staging O in threadgroup memory would add | |
| 25 | +// 8 KB and halve residency. | |
| 26 | +// | |
| 27 | +// Threadgroup memory ~12.7 KB at HD=64: K and V tiles (padded +4 floats per | |
| 28 | +// row against bank conflicts), the S/P tile, per-row softmax state, and one | |
| 29 | +// 8x8 diagonal scratch per simdgroup. The K/V tiles are reused as the output | |
| 30 | +// staging buffer at the end, once they are dead. | |
| 31 | +#include <metal_stdlib> | |
| 32 | +#include <metal_simdgroup_matrix> | |
| 33 | +using namespace metal; | |
| 34 | + | |
| 35 | +struct FlashParams { | |
| 36 | + uint B, T, H, HKV; | |
| 37 | + float scale; | |
| 38 | + uint causal; | |
| 39 | +}; | |
| 40 | + | |
| 41 | +// Enumerators, never `constant constexpr`: see RESEARCH.md 7a — the latter is | |
| 42 | +// a constant-address-space variable, which blocks unrolling and spills every | |
| 43 | +// fragment to the stack. | |
| 44 | +enum : uint { | |
| 45 | + BQ = 32, // query rows per threadgroup | |
| 46 | + BK = 16, // KV rows per iteration | |
| 47 | + NSG = BQ / 8, // one 8-row fragment per simdgroup | |
| 48 | + MMA_THREADS = NSG * 32, | |
| 49 | + KF = BK / 8, // S fragments along the KV axis | |
| 50 | + PADF = 4, // 16 bytes | |
| 51 | + LDS = BK + PADF, // S/P tile row stride | |
| 52 | +}; | |
| 53 | + | |
| 54 | +template <uint HD> | |
| 55 | +kernel void flash_attn_fwd_mma(device const float* Q [[buffer(0)]], | |
| 56 | + device const float* K [[buffer(1)]], | |
| 57 | + device const float* V [[buffer(2)]], | |
| 58 | + device float* O [[buffer(3)]], | |
| 59 | + device float* L [[buffer(4)]], | |
| 60 | + constant FlashParams& p [[buffer(5)]], | |
| 61 | + uint tgid [[threadgroup_position_in_grid]], | |
| 62 | + uint tid [[thread_index_in_threadgroup]], | |
| 63 | + uint lane [[thread_index_in_simdgroup]], | |
| 64 | + uint sgid [[simdgroup_index_in_threadgroup]]) { | |
| 65 | + constexpr uint DF = HD / 8; // fragments along head_dim | |
| 66 | + constexpr uint LDKV = HD + PADF; | |
| 67 | + | |
| 68 | + // K and V tiles, reused as output staging after the KV loop. | |
| 69 | + threadgroup float kv[2 * BK * LDKV]; | |
| 70 | + threadgroup float* Ks = kv; | |
| 71 | + threadgroup float* Vs = kv + BK * LDKV; | |
| 72 | + threadgroup float Sbuf[BQ * LDS]; | |
| 73 | + threadgroup float row_m[BQ], row_l[BQ], row_corr[BQ]; | |
| 74 | + threadgroup float diag[NSG * 64]; | |
| 75 | + | |
| 76 | + const uint q_blocks = (p.T + BQ - 1) / BQ; | |
| 77 | + const uint qb = tgid % q_blocks; | |
| 78 | + const uint h = (tgid / q_blocks) % p.H; | |
| 79 | + const uint b = tgid / (q_blocks * p.H); | |
| 80 | + | |
| 81 | + const uint hkv = h / (p.H / p.HKV); | |
| 82 | + const uint Cq = p.H * HD; | |
| 83 | + const uint Ckv = p.HKV * HD; | |
| 84 | + const uint q0 = qb * BQ; // first query row of this threadgroup | |
| 85 | + const uint sg_row = sgid * 8; // first query row of this simdgroup | |
| 86 | + | |
| 87 | + if (tid < BQ) { | |
| 88 | + row_m[tid] = -FLT_MAX; | |
| 89 | + row_l[tid] = 0.0f; | |
| 90 | + } | |
| 91 | + | |
| 92 | + // Q stays in registers for the whole kernel. It is staged through the | |
| 93 | + // (not yet used) K/V tile first so a query block straddling the end of | |
| 94 | + // the sequence gets zero-filled rows instead of an out-of-bounds read — | |
| 95 | + // loading fragments straight from device would need the block to be a | |
| 96 | + // full 8 rows. | |
| 97 | + simdgroup_float8x8 Qf[DF], Of[DF]; | |
| 98 | + { | |
| 99 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 100 | + for (uint e = tid; e < BQ * HD; e += MMA_THREADS) { | |
| 101 | + const uint r = e / HD; | |
| 102 | + const uint d = e % HD; | |
| 103 | + const uint i = q0 + r; | |
| 104 | + kv[r * HD + d] = | |
| 105 | + (i < p.T) ? Q[(ulong(b) * p.T + i) * Cq + h * HD + d] : 0.0f; | |
| 106 | + } | |
| 107 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 108 | +#pragma clang loop unroll(full) | |
| 109 | + for (uint d = 0; d < DF; ++d) { | |
| 110 | + simdgroup_load(Qf[d], kv + sg_row * HD + d * 8, HD); | |
| 111 | + Of[d] = make_filled_simdgroup_matrix<float, 8, 8>(0.0f); | |
| 112 | + } | |
| 113 | + } | |
| 114 | + | |
| 115 | + // Causal: only KV blocks up to this threadgroup's last query row matter. | |
| 116 | + const uint row_max = min(q0 + BQ - 1, p.T - 1); | |
| 117 | + const uint j_end = p.causal ? (row_max + 1) : p.T; | |
| 118 | + | |
| 119 | + for (uint jb = 0; jb < j_end; jb += BK) { | |
| 120 | + const uint block_len = min(BK, j_end - jb); | |
| 121 | + | |
| 122 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 123 | + // Stage K/V; rows past the block length are zeroed so masked columns | |
| 124 | + // can never contribute a NaN through 0 * garbage. | |
| 125 | + for (uint e = tid; e < BK * HD; e += MMA_THREADS) { | |
| 126 | + const uint jr = e / HD; | |
| 127 | + const uint d = e % HD; | |
| 128 | + const bool ok = jr < block_len; | |
| 129 | + const ulong src = (ulong(b) * p.T + jb + jr) * Ckv + hkv * HD + d; | |
| 130 | + Ks[jr * LDKV + d] = ok ? K[src] : 0.0f; | |
| 131 | + Vs[jr * LDKV + d] = ok ? V[src] : 0.0f; | |
| 132 | + } | |
| 133 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 134 | + | |
| 135 | + // S = Q @ K^T, accumulated over head_dim fragments. | |
| 136 | + simdgroup_float8x8 Sf[KF]; | |
| 137 | +#pragma clang loop unroll(full) | |
| 138 | + for (uint j = 0; j < KF; ++j) Sf[j] = make_filled_simdgroup_matrix<float, 8, 8>(0.0f); | |
| 139 | +#pragma clang loop unroll(full) | |
| 140 | + for (uint d = 0; d < DF; ++d) { | |
| 141 | +#pragma clang loop unroll(full) | |
| 142 | + for (uint j = 0; j < KF; ++j) { | |
| 143 | + simdgroup_float8x8 KTf; | |
| 144 | + // transpose=true turns the K tile into K^T[d-block][j-block] | |
| 145 | + simdgroup_load(KTf, Ks + (j * 8) * LDKV + d * 8, LDKV, 0, true); | |
| 146 | + simdgroup_multiply_accumulate(Sf[j], Qf[d], KTf, Sf[j]); | |
| 147 | + } | |
| 148 | + } | |
| 149 | +#pragma clang loop unroll(full) | |
| 150 | + for (uint j = 0; j < KF; ++j) | |
| 151 | + simdgroup_store(Sf[j], Sbuf + sg_row * LDS + j * 8, LDS); | |
| 152 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 153 | + | |
| 154 | + // Softmax on plain threads, one row each. This is O(BQ*BK) against | |
| 155 | + // O(BQ*BK*HD) of MMA work, so leaving 96 of 128 threads idle costs | |
| 156 | + // ~1% and keeps the reduction free of any lane-mapping assumption. | |
| 157 | + if (tid < BQ) { | |
| 158 | + const uint i = q0 + tid; | |
| 159 | + threadgroup float* srow = Sbuf + tid * LDS; | |
| 160 | + const uint valid = | |
| 161 | + (i >= p.T) ? 0u | |
| 162 | + : (p.causal ? min(block_len, (i >= jb) ? (i - jb + 1) : 0u) | |
| 163 | + : block_len); | |
| 164 | + float bmax = -FLT_MAX; | |
| 165 | + for (uint j = 0; j < valid; ++j) bmax = max(bmax, srow[j] * p.scale); | |
| 166 | + | |
| 167 | + const float m_old = row_m[tid]; | |
| 168 | + const float m_new = max(m_old, bmax); | |
| 169 | + const float corr = (m_old == -FLT_MAX) ? 0.0f : exp(m_old - m_new); | |
| 170 | + float bsum = 0.0f; | |
| 171 | + for (uint j = 0; j < BK; ++j) { | |
| 172 | + if (j < valid) { | |
| 173 | + const float e = exp(srow[j] * p.scale - m_new); | |
| 174 | + srow[j] = e; | |
| 175 | + bsum += e; | |
| 176 | + } else { | |
| 177 | + srow[j] = 0.0f; // masked / out of range | |
| 178 | + } | |
| 179 | + } | |
| 180 | + row_m[tid] = (valid > 0) ? m_new : m_old; | |
| 181 | + row_l[tid] = row_l[tid] * ((valid > 0) ? corr : 1.0f) + bsum; | |
| 182 | + row_corr[tid] = (valid > 0) ? corr : 1.0f; | |
| 183 | + } | |
| 184 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 185 | + | |
| 186 | + // O <- diag(corr) @ O, then O += P @ V. | |
| 187 | + threadgroup float* dg = diag + sgid * 64; | |
| 188 | + for (uint e = lane; e < 64; e += 32) dg[e] = 0.0f; | |
| 189 | + simdgroup_barrier(mem_flags::mem_threadgroup); | |
| 190 | + if (lane < 8) dg[lane * 8 + lane] = row_corr[sg_row + lane]; | |
| 191 | + simdgroup_barrier(mem_flags::mem_threadgroup); | |
| 192 | + | |
| 193 | + simdgroup_float8x8 Dg; | |
| 194 | + simdgroup_load(Dg, dg, 8); | |
| 195 | +#pragma clang loop unroll(full) | |
| 196 | + for (uint d = 0; d < DF; ++d) { | |
| 197 | + simdgroup_float8x8 scaled; | |
| 198 | + simdgroup_multiply(scaled, Dg, Of[d]); | |
| 199 | + Of[d] = scaled; | |
| 200 | + } | |
| 201 | +#pragma clang loop unroll(full) | |
| 202 | + for (uint j = 0; j < KF; ++j) { | |
| 203 | + simdgroup_float8x8 Pf; | |
| 204 | + simdgroup_load(Pf, Sbuf + sg_row * LDS + j * 8, LDS); | |
| 205 | +#pragma clang loop unroll(full) | |
| 206 | + for (uint d = 0; d < DF; ++d) { | |
| 207 | + simdgroup_float8x8 Vf; | |
| 208 | + simdgroup_load(Vf, Vs + (j * 8) * LDKV + d * 8, LDKV); | |
| 209 | + simdgroup_multiply_accumulate(Of[d], Pf, Vf, Of[d]); | |
| 210 | + } | |
| 211 | + } | |
| 212 | + } | |
| 213 | + | |
| 214 | + // Divide by l and write out. K/V are dead now, so their tile doubles as | |
| 215 | + // the [BQ x HD] staging buffer (2 * BK * LDKV >= BQ * HD). | |
| 216 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 217 | + threadgroup float* Ostage = kv; | |
| 218 | +#pragma clang loop unroll(full) | |
| 219 | + for (uint d = 0; d < DF; ++d) | |
| 220 | + simdgroup_store(Of[d], Ostage + sg_row * HD + d * 8, HD); | |
| 221 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 222 | + | |
| 223 | + for (uint e = tid; e < BQ * HD; e += MMA_THREADS) { | |
| 224 | + const uint r = e / HD; | |
| 225 | + const uint d = e % HD; | |
| 226 | + const uint i = q0 + r; | |
| 227 | + if (i >= p.T) continue; | |
| 228 | + const float l = row_l[r]; | |
| 229 | + O[(ulong(b) * p.T + i) * Cq + h * HD + d] = | |
| 230 | + (l > 0.0f) ? (Ostage[r * HD + d] / l) : 0.0f; | |
| 231 | + } | |
| 232 | + if (tid < BQ && (q0 + tid) < p.T) { | |
| 233 | + const float l = row_l[tid]; | |
| 234 | + L[(ulong(b) * p.H + h) * p.T + q0 + tid] = | |
| 235 | + (l > 0.0f) ? (row_m[tid] + log(l)) : 0.0f; | |
| 236 | + } | |
| 237 | +} | |
| 238 | + | |
| 239 | +#define INSTANTIATE_FLASH_MMA(HD) \ | |
| 240 | + template [[host_name("flash_attn_fwd_mma_f32_hd" #HD)]] kernel void \ | |
| 241 | + flash_attn_fwd_mma<HD>(device const float*, device const float*, \ | |
| 242 | + device const float*, device float*, device float*, \ | |
| 243 | + constant FlashParams&, uint, uint, uint, uint); | |
| 244 | + | |
| 245 | +INSTANTIATE_FLASH_MMA(16) | |
| 246 | +INSTANTIATE_FLASH_MMA(32) | |
| 247 | +INSTANTIATE_FLASH_MMA(48) | |
| 248 | +INSTANTIATE_FLASH_MMA(64) | |
| 249 | +INSTANTIATE_FLASH_MMA(80) | |
| 250 | +INSTANTIATE_FLASH_MMA(96) | |
| 251 | +INSTANTIATE_FLASH_MMA(128) | |
| 252 | + | |
| 253 | +// ============================================================ backward (MMA) | |
| 254 | +// | |
| 255 | +// Backward needs no online rescale — L from the forward already fixes the | |
| 256 | +// softmax normalisation — so there is no diagonal-matrix trick here, just | |
| 257 | +// three matmuls per block with an elementwise step between them. | |
| 258 | +// | |
| 259 | +// dQ kernel (parallel over queries, one threadgroup per BQ query rows): | |
| 260 | +// S = Q @ K^T ; P = exp(S*scale - L_i) | |
| 261 | +// dP = dO @ V^T ; dS = P * (dP - D_i) * scale | |
| 262 | +// dQ += dS @ K | |
| 263 | +// dKV kernel (parallel over KV rows, one threadgroup per BKV rows): | |
| 264 | +// everything transposed, obtained by swapping the operand roles: | |
| 265 | +// S^T = K @ Q^T and dP^T = V @ dO^T, so | |
| 266 | +// dV += P^T @ dO and dK += dS^T @ Q. | |
| 267 | +// | |
| 268 | +// Splitting this way keeps dK/dV accumulation private to a threadgroup — no | |
| 269 | +// atomics, and the result is deterministic. Fragments cut register pressure | |
| 270 | +// by 4x versus the scalar kernels: an 8x8 fragment is 2 floats per lane, so | |
| 271 | +// holding K, V, dK, dV as fragments is 64 floats/lane at hd=64 where the | |
| 272 | +// scalar version needed 256 and spilled. | |
| 273 | + | |
| 274 | +enum : uint { | |
| 275 | + BQB = 16, // query block consumed per iteration by dKV | |
| 276 | + LDQB = BQB + PADF, | |
| 277 | +}; | |
| 278 | + | |
| 279 | +template <uint HD> | |
| 280 | +kernel void flash_attn_bwd_dq_mma(device const float* Q [[buffer(0)]], | |
| 281 | + device const float* K [[buffer(1)]], | |
| 282 | + device const float* V [[buffer(2)]], | |
| 283 | + device const float* dO [[buffer(3)]], | |
| 284 | + device const float* L [[buffer(4)]], | |
| 285 | + device const float* D [[buffer(5)]], | |
| 286 | + device float* dQ [[buffer(6)]], | |
| 287 | + constant FlashParams& p [[buffer(7)]], | |
| 288 | + uint tgid [[threadgroup_position_in_grid]], | |
| 289 | + uint tid [[thread_index_in_threadgroup]], | |
| 290 | + uint sgid [[simdgroup_index_in_threadgroup]]) { | |
| 291 | + constexpr uint DF = HD / 8; | |
| 292 | + constexpr uint LDKV = HD + PADF; | |
| 293 | + | |
| 294 | + threadgroup float kv[2 * BK * LDKV]; | |
| 295 | + threadgroup float* Ks = kv; | |
| 296 | + threadgroup float* Vs = kv + BK * LDKV; | |
| 297 | + threadgroup float Sbuf[BQ * LDS]; | |
| 298 | + threadgroup float Pbuf[BQ * LDS]; | |
| 299 | + threadgroup float row_L[BQ], row_D[BQ]; | |
| 300 | + | |
| 301 | + const uint q_blocks = (p.T + BQ - 1) / BQ; | |
| 302 | + const uint qb = tgid % q_blocks; | |
| 303 | + const uint h = (tgid / q_blocks) % p.H; | |
| 304 | + const uint b = tgid / (q_blocks * p.H); | |
| 305 | + | |
| 306 | + const uint hkv = h / (p.H / p.HKV); | |
| 307 | + const uint Cq = p.H * HD; | |
| 308 | + const uint Ckv = p.HKV * HD; | |
| 309 | + const uint q0 = qb * BQ; | |
| 310 | + const uint sg_row = sgid * 8; | |
| 311 | + | |
| 312 | + if (tid < BQ) { | |
| 313 | + const uint i = q0 + tid; | |
| 314 | + row_L[tid] = (i < p.T) ? L[(ulong(b) * p.H + h) * p.T + i] : 0.0f; | |
| 315 | + row_D[tid] = (i < p.T) ? D[(ulong(b) * p.H + h) * p.T + i] : 0.0f; | |
| 316 | + } | |
| 317 | + | |
| 318 | + // Q and dO into registers (staged so ragged tails zero-fill). | |
| 319 | + simdgroup_float8x8 Qf[DF], dOf[DF], dQf[DF]; | |
| 320 | + for (uint pass = 0; pass < 2; ++pass) { | |
| 321 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 322 | + device const float* src = (pass == 0) ? Q : dO; | |
| 323 | + for (uint e = tid; e < BQ * HD; e += MMA_THREADS) { | |
| 324 | + const uint r = e / HD, d = e % HD; | |
| 325 | + const uint i = q0 + r; | |
| 326 | + kv[r * HD + d] = | |
| 327 | + (i < p.T) ? src[(ulong(b) * p.T + i) * Cq + h * HD + d] : 0.0f; | |
| 328 | + } | |
| 329 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 330 | +#pragma clang loop unroll(full) | |
| 331 | + for (uint d = 0; d < DF; ++d) { | |
| 332 | + if (pass == 0) simdgroup_load(Qf[d], kv + sg_row * HD + d * 8, HD); | |
| 333 | + else simdgroup_load(dOf[d], kv + sg_row * HD + d * 8, HD); | |
| 334 | + } | |
| 335 | + } | |
| 336 | +#pragma clang loop unroll(full) | |
| 337 | + for (uint d = 0; d < DF; ++d) dQf[d] = make_filled_simdgroup_matrix<float, 8, 8>(0.0f); | |
| 338 | + | |
| 339 | + const uint row_max = min(q0 + BQ - 1, p.T - 1); | |
| 340 | + const uint j_end = p.causal ? (row_max + 1) : p.T; | |
| 341 | + | |
| 342 | + for (uint jb = 0; jb < j_end; jb += BK) { | |
| 343 | + const uint block_len = min(BK, j_end - jb); | |
| 344 | + | |
| 345 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 346 | + for (uint e = tid; e < BK * HD; e += MMA_THREADS) { | |
| 347 | + const uint jr = e / HD, d = e % HD; | |
| 348 | + const bool ok = jr < block_len; | |
| 349 | + const ulong src = (ulong(b) * p.T + jb + jr) * Ckv + hkv * HD + d; | |
| 350 | + Ks[jr * LDKV + d] = ok ? K[src] : 0.0f; | |
| 351 | + Vs[jr * LDKV + d] = ok ? V[src] : 0.0f; | |
| 352 | + } | |
| 353 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 354 | + | |
| 355 | + // S = Q @ K^T and dP = dO @ V^T | |
| 356 | + simdgroup_float8x8 Sf[KF], Pf[KF]; | |
| 357 | +#pragma clang loop unroll(full) | |
| 358 | + for (uint j = 0; j < KF; ++j) { | |
| 359 | + Sf[j] = make_filled_simdgroup_matrix<float, 8, 8>(0.0f); | |
| 360 | + Pf[j] = make_filled_simdgroup_matrix<float, 8, 8>(0.0f); | |
| 361 | + } | |
| 362 | +#pragma clang loop unroll(full) | |
| 363 | + for (uint d = 0; d < DF; ++d) { | |
| 364 | +#pragma clang loop unroll(full) | |
| 365 | + for (uint j = 0; j < KF; ++j) { | |
| 366 | + simdgroup_float8x8 KTf, VTf; | |
| 367 | + simdgroup_load(KTf, Ks + (j * 8) * LDKV + d * 8, LDKV, 0, true); | |
| 368 | + simdgroup_load(VTf, Vs + (j * 8) * LDKV + d * 8, LDKV, 0, true); | |
| 369 | + simdgroup_multiply_accumulate(Sf[j], Qf[d], KTf, Sf[j]); | |
| 370 | + simdgroup_multiply_accumulate(Pf[j], dOf[d], VTf, Pf[j]); | |
| 371 | + } | |
| 372 | + } | |
| 373 | +#pragma clang loop unroll(full) | |
| 374 | + for (uint j = 0; j < KF; ++j) { | |
| 375 | + simdgroup_store(Sf[j], Sbuf + sg_row * LDS + j * 8, LDS); | |
| 376 | + simdgroup_store(Pf[j], Pbuf + sg_row * LDS + j * 8, LDS); | |
| 377 | + } | |
| 378 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 379 | + | |
| 380 | + // dS = P * (dP - D_i) * scale, written back over Sbuf | |
| 381 | + if (tid < BQ) { | |
| 382 | + const uint i = q0 + tid; | |
| 383 | + threadgroup float* srow = Sbuf + tid * LDS; | |
| 384 | + threadgroup const float* prow = Pbuf + tid * LDS; | |
| 385 | + const uint valid = | |
| 386 | + (i >= p.T) ? 0u | |
| 387 | + : (p.causal ? min(block_len, (i >= jb) ? (i - jb + 1) : 0u) | |
| 388 | + : block_len); | |
| 389 | + const float li = row_L[tid], di = row_D[tid]; | |
| 390 | + for (uint j = 0; j < BK; ++j) { | |
| 391 | + if (j < valid) { | |
| 392 | + const float prob = exp(srow[j] * p.scale - li); | |
| 393 | + srow[j] = prob * (prow[j] - di) * p.scale; | |
| 394 | + } else { | |
| 395 | + srow[j] = 0.0f; | |
| 396 | + } | |
| 397 | + } | |
| 398 | + } | |
| 399 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 400 | + | |
| 401 | + // dQ += dS @ K | |
| 402 | +#pragma clang loop unroll(full) | |
| 403 | + for (uint j = 0; j < KF; ++j) { | |
| 404 | + simdgroup_float8x8 dSf; | |
| 405 | + simdgroup_load(dSf, Sbuf + sg_row * LDS + j * 8, LDS); | |
| 406 | +#pragma clang loop unroll(full) | |
| 407 | + for (uint d = 0; d < DF; ++d) { | |
| 408 | + simdgroup_float8x8 Kf; | |
| 409 | + simdgroup_load(Kf, Ks + (j * 8) * LDKV + d * 8, LDKV); | |
| 410 | + simdgroup_multiply_accumulate(dQf[d], dSf, Kf, dQf[d]); | |
| 411 | + } | |
| 412 | + } | |
| 413 | + } | |
| 414 | + | |
| 415 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 416 | +#pragma clang loop unroll(full) | |
| 417 | + for (uint d = 0; d < DF; ++d) | |
| 418 | + simdgroup_store(dQf[d], kv + sg_row * HD + d * 8, HD); | |
| 419 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 420 | + for (uint e = tid; e < BQ * HD; e += MMA_THREADS) { | |
| 421 | + const uint r = e / HD, d = e % HD; | |
| 422 | + const uint i = q0 + r; | |
| 423 | + if (i >= p.T) continue; | |
| 424 | + dQ[(ulong(b) * p.T + i) * Cq + h * HD + d] += kv[r * HD + d]; | |
| 425 | + } | |
| 426 | +} | |
| 427 | + | |
| 428 | +// dK and dV are separate kernels. Combined, one thread holds K, V, dK and dV | |
| 429 | +// as fragments and the compiler spills: gpudebug reported 4352 spilled bytes | |
| 430 | +// and 111 temp registers for the fused version, which made it the single most | |
| 431 | +// expensive backward kernel. Split, dV carries K+dV and dK carries K+V+dK, and | |
| 432 | +// both fit. The cost is recomputing S^T = K @ Q^T in each. | |
| 433 | + | |
| 434 | +template <uint HD> | |
| 435 | +kernel void flash_attn_bwd_dv_mma(device const float* Q [[buffer(0)]], | |
| 436 | + device const float* K [[buffer(1)]], | |
| 437 | + device const float* dO [[buffer(2)]], | |
| 438 | + device const float* L [[buffer(3)]], | |
| 439 | + device float* dV [[buffer(4)]], | |
| 440 | + constant FlashParams& p [[buffer(5)]], | |
| 441 | + uint tgid [[threadgroup_position_in_grid]], | |
| 442 | + uint tid [[thread_index_in_threadgroup]], | |
| 443 | + uint sgid [[simdgroup_index_in_threadgroup]]) { | |
| 444 | + constexpr uint DF = HD / 8; | |
| 445 | + constexpr uint LDQ = HD + PADF; | |
| 446 | + constexpr uint IF = BQB / 8; | |
| 447 | + | |
| 448 | + threadgroup float qo[2 * BQB * LDQ]; | |
| 449 | + threadgroup float* Qs = qo; | |
| 450 | + threadgroup float* dOs = qo + BQB * LDQ; | |
| 451 | + threadgroup float PTbuf[BQ * LDQB]; | |
| 452 | + threadgroup float col_L[BQB]; | |
| 453 | + | |
| 454 | + const uint kv_blocks = (p.T + BQ - 1) / BQ; | |
| 455 | + const uint jbk = tgid % kv_blocks; | |
| 456 | + const uint hkv = (tgid / kv_blocks) % p.HKV; | |
| 457 | + const uint b = tgid / (kv_blocks * p.HKV); | |
| 458 | + | |
| 459 | + const uint rep = p.H / p.HKV; | |
| 460 | + const uint Cq = p.H * HD; | |
| 461 | + const uint Ckv = p.HKV * HD; | |
| 462 | + const uint j0 = jbk * BQ; | |
| 463 | + const uint sg_row = sgid * 8; | |
| 464 | + | |
| 465 | + simdgroup_float8x8 Kf[DF], dVf[DF]; | |
| 466 | + { | |
| 467 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 468 | + for (uint e = tid; e < BQ * HD; e += MMA_THREADS) { | |
| 469 | + const uint r = e / HD, d = e % HD; | |
| 470 | + const uint j = j0 + r; | |
| 471 | + qo[r * HD + d] = | |
| 472 | + (j < p.T) ? K[(ulong(b) * p.T + j) * Ckv + hkv * HD + d] : 0.0f; | |
| 473 | + } | |
| 474 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 475 | +#pragma clang loop unroll(full) | |
| 476 | + for (uint d = 0; d < DF; ++d) { | |
| 477 | + simdgroup_load(Kf[d], qo + sg_row * HD + d * 8, HD); | |
| 478 | + dVf[d] = make_filled_simdgroup_matrix<float, 8, 8>(0.0f); | |
| 479 | + } | |
| 480 | + } | |
| 481 | + | |
| 482 | + const uint i_start = p.causal ? (j0 / BQB) * BQB : 0; | |
| 483 | + for (uint r = 0; r < rep; ++r) { | |
| 484 | + const uint h = hkv * rep + r; | |
| 485 | + device const float* Lh = L + (ulong(b) * p.H + h) * p.T; | |
| 486 | + for (uint ib = i_start; ib < p.T; ib += BQB) { | |
| 487 | + const uint blk = min(BQB, p.T - ib); | |
| 488 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 489 | + for (uint e = tid; e < BQB * HD; e += MMA_THREADS) { | |
| 490 | + const uint ir = e / HD, d = e % HD; | |
| 491 | + const bool ok = ir < blk; | |
| 492 | + const ulong src = (ulong(b) * p.T + ib + ir) * Cq + h * HD + d; | |
| 493 | + Qs[ir * LDQ + d] = ok ? Q[src] : 0.0f; | |
| 494 | + dOs[ir * LDQ + d] = ok ? dO[src] : 0.0f; | |
| 495 | + } | |
| 496 | + if (tid < BQB) col_L[tid] = (tid < blk) ? Lh[ib + tid] : 0.0f; | |
| 497 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 498 | + | |
| 499 | + simdgroup_float8x8 STf[IF]; | |
| 500 | +#pragma clang loop unroll(full) | |
| 501 | + for (uint i = 0; i < IF; ++i) | |
| 502 | + STf[i] = make_filled_simdgroup_matrix<float, 8, 8>(0.0f); | |
| 503 | +#pragma clang loop unroll(full) | |
| 504 | + for (uint d = 0; d < DF; ++d) { | |
| 505 | +#pragma clang loop unroll(full) | |
| 506 | + for (uint i = 0; i < IF; ++i) { | |
| 507 | + simdgroup_float8x8 QTf; | |
| 508 | + simdgroup_load(QTf, Qs + (i * 8) * LDQ + d * 8, LDQ, 0, true); | |
| 509 | + simdgroup_multiply_accumulate(STf[i], Kf[d], QTf, STf[i]); | |
| 510 | + } | |
| 511 | + } | |
| 512 | +#pragma clang loop unroll(full) | |
| 513 | + for (uint i = 0; i < IF; ++i) | |
| 514 | + simdgroup_store(STf[i], PTbuf + sg_row * LDQB + i * 8, LDQB); | |
| 515 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 516 | + | |
| 517 | + if (tid < BQ) { | |
| 518 | + const uint j = j0 + tid; | |
| 519 | + threadgroup float* prow = PTbuf + tid * LDQB; | |
| 520 | + for (uint i = 0; i < BQB; ++i) { | |
| 521 | + const uint iq = ib + i; | |
| 522 | + const bool ok = (i < blk) && (j < p.T) && (!p.causal || iq >= j); | |
| 523 | + prow[i] = ok ? exp(prow[i] * p.scale - col_L[i]) : 0.0f; | |
| 524 | + } | |
| 525 | + } | |
| 526 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 527 | + | |
| 528 | +#pragma clang loop unroll(full) | |
| 529 | + for (uint i = 0; i < IF; ++i) { | |
| 530 | + simdgroup_float8x8 PTx; | |
| 531 | + simdgroup_load(PTx, PTbuf + sg_row * LDQB + i * 8, LDQB); | |
| 532 | +#pragma clang loop unroll(full) | |
| 533 | + for (uint d = 0; d < DF; ++d) { | |
| 534 | + simdgroup_float8x8 dOf; | |
| 535 | + simdgroup_load(dOf, dOs + (i * 8) * LDQ + d * 8, LDQ); | |
| 536 | + simdgroup_multiply_accumulate(dVf[d], PTx, dOf, dVf[d]); | |
| 537 | + } | |
| 538 | + } | |
| 539 | + } | |
| 540 | + } | |
| 541 | + | |
| 542 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 543 | +#pragma clang loop unroll(full) | |
| 544 | + for (uint d = 0; d < DF; ++d) | |
| 545 | + simdgroup_store(dVf[d], qo + sg_row * HD + d * 8, HD); | |
| 546 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 547 | + for (uint e = tid; e < BQ * HD; e += MMA_THREADS) { | |
| 548 | + const uint r = e / HD, d = e % HD; | |
| 549 | + const uint j = j0 + r; | |
| 550 | + if (j >= p.T) continue; | |
| 551 | + dV[(ulong(b) * p.T + j) * Ckv + hkv * HD + d] += qo[r * HD + d]; | |
| 552 | + } | |
| 553 | +} | |
| 554 | + | |
| 555 | +template <uint HD> | |
| 556 | +kernel void flash_attn_bwd_dk_mma(device const float* Q [[buffer(0)]], | |
| 557 | + device const float* K [[buffer(1)]], | |
| 558 | + device const float* V [[buffer(2)]], | |
| 559 | + device const float* dO [[buffer(3)]], | |
| 560 | + device const float* L [[buffer(4)]], | |
| 561 | + device const float* D [[buffer(5)]], | |
| 562 | + device float* dK [[buffer(6)]], | |
| 563 | + constant FlashParams& p [[buffer(7)]], | |
| 564 | + uint tgid [[threadgroup_position_in_grid]], | |
| 565 | + uint tid [[thread_index_in_threadgroup]], | |
| 566 | + uint sgid [[simdgroup_index_in_threadgroup]]) { | |
| 567 | + constexpr uint DF = HD / 8; | |
| 568 | + constexpr uint LDQ = HD + PADF; | |
| 569 | + constexpr uint IF = BQB / 8; | |
| 570 | + | |
| 571 | + threadgroup float qo[2 * BQB * LDQ]; | |
| 572 | + threadgroup float* Qs = qo; | |
| 573 | + threadgroup float* dOs = qo + BQB * LDQ; | |
| 574 | + threadgroup float STbuf[BQ * LDQB]; | |
| 575 | + threadgroup float PTbuf[BQ * LDQB]; | |
| 576 | + threadgroup float col_L[BQB], col_D[BQB]; | |
| 577 | + | |
| 578 | + const uint kv_blocks = (p.T + BQ - 1) / BQ; | |
| 579 | + const uint jbk = tgid % kv_blocks; | |
| 580 | + const uint hkv = (tgid / kv_blocks) % p.HKV; | |
| 581 | + const uint b = tgid / (kv_blocks * p.HKV); | |
| 582 | + | |
| 583 | + const uint rep = p.H / p.HKV; | |
| 584 | + const uint Cq = p.H * HD; | |
| 585 | + const uint Ckv = p.HKV * HD; | |
| 586 | + const uint j0 = jbk * BQ; | |
| 587 | + const uint sg_row = sgid * 8; | |
| 588 | + | |
| 589 | + simdgroup_float8x8 Kf[DF], Vf[DF], dKf[DF]; | |
| 590 | + for (uint pass = 0; pass < 2; ++pass) { | |
| 591 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 592 | + device const float* src = (pass == 0) ? K : V; | |
| 593 | + for (uint e = tid; e < BQ * HD; e += MMA_THREADS) { | |
| 594 | + const uint r = e / HD, d = e % HD; | |
| 595 | + const uint j = j0 + r; | |
| 596 | + qo[r * HD + d] = | |
| 597 | + (j < p.T) ? src[(ulong(b) * p.T + j) * Ckv + hkv * HD + d] : 0.0f; | |
| 598 | + } | |
| 599 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 600 | +#pragma clang loop unroll(full) | |
| 601 | + for (uint d = 0; d < DF; ++d) { | |
| 602 | + if (pass == 0) simdgroup_load(Kf[d], qo + sg_row * HD + d * 8, HD); | |
| 603 | + else simdgroup_load(Vf[d], qo + sg_row * HD + d * 8, HD); | |
| 604 | + } | |
| 605 | + } | |
| 606 | +#pragma clang loop unroll(full) | |
| 607 | + for (uint d = 0; d < DF; ++d) dKf[d] = make_filled_simdgroup_matrix<float, 8, 8>(0.0f); | |
| 608 | + | |
| 609 | + const uint i_start = p.causal ? (j0 / BQB) * BQB : 0; | |
| 610 | + for (uint r = 0; r < rep; ++r) { | |
| 611 | + const uint h = hkv * rep + r; | |
| 612 | + device const float* Lh = L + (ulong(b) * p.H + h) * p.T; | |
| 613 | + device const float* Dh = D + (ulong(b) * p.H + h) * p.T; | |
| 614 | + for (uint ib = i_start; ib < p.T; ib += BQB) { | |
| 615 | + const uint blk = min(BQB, p.T - ib); | |
| 616 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 617 | + for (uint e = tid; e < BQB * HD; e += MMA_THREADS) { | |
| 618 | + const uint ir = e / HD, d = e % HD; | |
| 619 | + const bool ok = ir < blk; | |
| 620 | + const ulong src = (ulong(b) * p.T + ib + ir) * Cq + h * HD + d; | |
| 621 | + Qs[ir * LDQ + d] = ok ? Q[src] : 0.0f; | |
| 622 | + dOs[ir * LDQ + d] = ok ? dO[src] : 0.0f; | |
| 623 | + } | |
| 624 | + if (tid < BQB) { | |
| 625 | + col_L[tid] = (tid < blk) ? Lh[ib + tid] : 0.0f; | |
| 626 | + col_D[tid] = (tid < blk) ? Dh[ib + tid] : 0.0f; | |
| 627 | + } | |
| 628 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 629 | + | |
| 630 | + simdgroup_float8x8 STf[IF], PTf[IF]; | |
| 631 | +#pragma clang loop unroll(full) | |
| 632 | + for (uint i = 0; i < IF; ++i) { | |
| 633 | + STf[i] = make_filled_simdgroup_matrix<float, 8, 8>(0.0f); | |
| 634 | + PTf[i] = make_filled_simdgroup_matrix<float, 8, 8>(0.0f); | |
| 635 | + } | |
| 636 | +#pragma clang loop unroll(full) | |
| 637 | + for (uint d = 0; d < DF; ++d) { | |
| 638 | +#pragma clang loop unroll(full) | |
| 639 | + for (uint i = 0; i < IF; ++i) { | |
| 640 | + simdgroup_float8x8 QTf, dOTf; | |
| 641 | + simdgroup_load(QTf, Qs + (i * 8) * LDQ + d * 8, LDQ, 0, true); | |
| 642 | + simdgroup_load(dOTf, dOs + (i * 8) * LDQ + d * 8, LDQ, 0, true); | |
| 643 | + simdgroup_multiply_accumulate(STf[i], Kf[d], QTf, STf[i]); | |
| 644 | + simdgroup_multiply_accumulate(PTf[i], Vf[d], dOTf, PTf[i]); | |
| 645 | + } | |
| 646 | + } | |
| 647 | +#pragma clang loop unroll(full) | |
| 648 | + for (uint i = 0; i < IF; ++i) { | |
| 649 | + simdgroup_store(STf[i], STbuf + sg_row * LDQB + i * 8, LDQB); | |
| 650 | + simdgroup_store(PTf[i], PTbuf + sg_row * LDQB + i * 8, LDQB); | |
| 651 | + } | |
| 652 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 653 | + | |
| 654 | + if (tid < BQ) { | |
| 655 | + const uint j = j0 + tid; | |
| 656 | + threadgroup float* srow = STbuf + tid * LDQB; | |
| 657 | + threadgroup const float* prow = PTbuf + tid * LDQB; | |
| 658 | + for (uint i = 0; i < BQB; ++i) { | |
| 659 | + const uint iq = ib + i; | |
| 660 | + const bool ok = (i < blk) && (j < p.T) && (!p.causal || iq >= j); | |
| 661 | + srow[i] = ok ? exp(srow[i] * p.scale - col_L[i]) * | |
| 662 | + (prow[i] - col_D[i]) * p.scale | |
| 663 | + : 0.0f; | |
| 664 | + } | |
| 665 | + } | |
| 666 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 667 | + | |
| 668 | +#pragma clang loop unroll(full) | |
| 669 | + for (uint i = 0; i < IF; ++i) { | |
| 670 | + simdgroup_float8x8 STx; | |
| 671 | + simdgroup_load(STx, STbuf + sg_row * LDQB + i * 8, LDQB); | |
| 672 | +#pragma clang loop unroll(full) | |
| 673 | + for (uint d = 0; d < DF; ++d) { | |
| 674 | + simdgroup_float8x8 Qf2; | |
| 675 | + simdgroup_load(Qf2, Qs + (i * 8) * LDQ + d * 8, LDQ); | |
| 676 | + simdgroup_multiply_accumulate(dKf[d], STx, Qf2, dKf[d]); | |
| 677 | + } | |
| 678 | + } | |
| 679 | + } | |
| 680 | + } | |
| 681 | + | |
| 682 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 683 | +#pragma clang loop unroll(full) | |
| 684 | + for (uint d = 0; d < DF; ++d) | |
| 685 | + simdgroup_store(dKf[d], qo + sg_row * HD + d * 8, HD); | |
| 686 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 687 | + for (uint e = tid; e < BQ * HD; e += MMA_THREADS) { | |
| 688 | + const uint r = e / HD, d = e % HD; | |
| 689 | + const uint j = j0 + r; | |
| 690 | + if (j >= p.T) continue; | |
| 691 | + dK[(ulong(b) * p.T + j) * Ckv + hkv * HD + d] += qo[r * HD + d]; | |
| 692 | + } | |
| 693 | +} | |
| 694 | + | |
| 695 | +#define INSTANTIATE_FLASH_MMA_BWD(HD) \ | |
| 696 | + template [[host_name("flash_attn_bwd_dq_mma_f32_hd" #HD)]] kernel void \ | |
| 697 | + flash_attn_bwd_dq_mma<HD>(device const float*, device const float*, \ | |
| 698 | + device const float*, device const float*, \ | |
| 699 | + device const float*, device const float*, \ | |
| 700 | + device float*, constant FlashParams&, uint, uint, \ | |
| 701 | + uint); \ | |
| 702 | + template [[host_name("flash_attn_bwd_dv_mma_f32_hd" #HD)]] kernel void \ | |
| 703 | + flash_attn_bwd_dv_mma<HD>(device const float*, device const float*, \ | |
| 704 | + device const float*, device const float*, \ | |
| 705 | + device float*, constant FlashParams&, uint, uint, \ | |
| 706 | + uint); \ | |
| 707 | + template [[host_name("flash_attn_bwd_dk_mma_f32_hd" #HD)]] kernel void \ | |
| 708 | + flash_attn_bwd_dk_mma<HD>(device const float*, device const float*, \ | |
| 709 | + device const float*, device const float*, \ | |
| 710 | + device const float*, device const float*, \ | |
| 711 | + device float*, constant FlashParams&, uint, uint, \ | |
| 712 | + uint); | |
| 713 | + | |
| 714 | +INSTANTIATE_FLASH_MMA_BWD(16) | |
| 715 | +INSTANTIATE_FLASH_MMA_BWD(32) | |
| 716 | +INSTANTIATE_FLASH_MMA_BWD(48) | |
| 717 | +INSTANTIATE_FLASH_MMA_BWD(64) | |
| 718 | +INSTANTIATE_FLASH_MMA_BWD(80) | |
| 719 | +INSTANTIATE_FLASH_MMA_BWD(96) | |
| 720 | +INSTANTIATE_FLASH_MMA_BWD(128) | |
added
src/kernels/layernorm.metal
+210 −0
@@ -0,0 +1,210 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +// | |
| 3 | +// Row-wise RMSNorm / LayerNorm. One threadgroup per row; simd reductions | |
| 4 | +// first (register shuffles beat threadgroup memory 2:1 on Apple GPUs), | |
| 5 | +// tiny threadgroup scratch only to cross simdgroups. LayerNorm is two-pass | |
| 6 | +// (mean, then variance) — the E[x²]−E[x]² shortcut loses precision and | |
| 7 | +// these kernels are held to 1e-4 parity vs the two-pass CPU reference. | |
| 8 | +#include <metal_stdlib> | |
| 9 | +using namespace metal; | |
| 10 | + | |
| 11 | +namespace { | |
| 12 | + | |
| 13 | +// Threadgroup-wide sum of one per-thread value; returns the same total to | |
| 14 | +// every thread. Scratch must hold 32 floats. | |
| 15 | +inline float tg_sum(float v, uint lane, uint simd_idx, uint n_simds, | |
| 16 | + threadgroup float* scratch) { | |
| 17 | + const float s = simd_sum(v); | |
| 18 | + if (simd_is_first()) scratch[simd_idx] = s; | |
| 19 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 20 | + const float mine = (lane < n_simds) ? scratch[lane] : 0.0f; | |
| 21 | + return simd_sum(mine); | |
| 22 | +} | |
| 23 | + | |
| 24 | +} // namespace | |
| 25 | + | |
| 26 | +struct NormParams { | |
| 27 | + uint C; | |
| 28 | + float eps; | |
| 29 | +}; | |
| 30 | + | |
| 31 | +kernel void rmsnorm_f32(device const float* X [[buffer(0)]], | |
| 32 | + device const float* W [[buffer(1)]], | |
| 33 | + device float* OUT [[buffer(2)]], | |
| 34 | + constant NormParams& p [[buffer(3)]], | |
| 35 | + uint row [[threadgroup_position_in_grid]], | |
| 36 | + uint lid [[thread_index_in_threadgroup]], | |
| 37 | + uint tg_size [[threads_per_threadgroup]], | |
| 38 | + uint lane [[thread_index_in_simdgroup]], | |
| 39 | + uint simd_idx [[simdgroup_index_in_threadgroup]], | |
| 40 | + uint n_simds [[simdgroups_per_threadgroup]]) { | |
| 41 | + device const float* x = X + ulong(row) * p.C; | |
| 42 | + device float* out = OUT + ulong(row) * p.C; | |
| 43 | + threadgroup float scratch[32]; | |
| 44 | + | |
| 45 | + float ss = 0.0f; | |
| 46 | + for (uint j = lid; j < p.C; j += tg_size) ss = fma(x[j], x[j], ss); | |
| 47 | + ss = tg_sum(ss, lane, simd_idx, n_simds, scratch); | |
| 48 | + | |
| 49 | + const float inv_rms = rsqrt(ss / float(p.C) + p.eps); | |
| 50 | + for (uint j = lid; j < p.C; j += tg_size) out[j] = W[j] * x[j] * inv_rms; | |
| 51 | +} | |
| 52 | + | |
| 53 | +kernel void layernorm_f32(device const float* X [[buffer(0)]], | |
| 54 | + device const float* W [[buffer(1)]], | |
| 55 | + device const float* B [[buffer(2)]], | |
| 56 | + device float* OUT [[buffer(3)]], | |
| 57 | + constant NormParams& p [[buffer(4)]], | |
| 58 | + uint row [[threadgroup_position_in_grid]], | |
| 59 | + uint lid [[thread_index_in_threadgroup]], | |
| 60 | + uint tg_size [[threads_per_threadgroup]], | |
| 61 | + uint lane [[thread_index_in_simdgroup]], | |
| 62 | + uint simd_idx [[simdgroup_index_in_threadgroup]], | |
| 63 | + uint n_simds [[simdgroups_per_threadgroup]]) { | |
| 64 | + device const float* x = X + ulong(row) * p.C; | |
| 65 | + device float* out = OUT + ulong(row) * p.C; | |
| 66 | + threadgroup float scratch[32]; | |
| 67 | + | |
| 68 | + float s = 0.0f; | |
| 69 | + for (uint j = lid; j < p.C; j += tg_size) s += x[j]; | |
| 70 | + const float mean = tg_sum(s, lane, simd_idx, n_simds, scratch) / float(p.C); | |
| 71 | + threadgroup_barrier(mem_flags::mem_threadgroup); // scratch reuse | |
| 72 | + | |
| 73 | + float var = 0.0f; | |
| 74 | + for (uint j = lid; j < p.C; j += tg_size) { | |
| 75 | + const float d = x[j] - mean; | |
| 76 | + var = fma(d, d, var); | |
| 77 | + } | |
| 78 | + var = tg_sum(var, lane, simd_idx, n_simds, scratch) / float(p.C); | |
| 79 | + | |
| 80 | + const float inv_std = rsqrt(var + p.eps); | |
| 81 | + for (uint j = lid; j < p.C; j += tg_size) | |
| 82 | + out[j] = fma(W[j], (x[j] - mean) * inv_std, B[j]); | |
| 83 | +} | |
| 84 | + | |
| 85 | +// ---- backwards ------------------------------------------------------------- | |
| 86 | +// dx is row-parallel (one threadgroup per row) and saves the row statistics | |
| 87 | +// (inv_rms, or mean+inv_std) it computes anyway; dw/db are column-parallel | |
| 88 | +// (one thread per column) reading those saved stats — deterministic, no | |
| 89 | +// atomics, no per-column recomputation. | |
| 90 | + | |
| 91 | +kernel void rmsnorm_bwd_dx_f32(device const float* X [[buffer(0)]], | |
| 92 | + device const float* W [[buffer(1)]], | |
| 93 | + device const float* dOUT [[buffer(2)]], | |
| 94 | + device float* dX [[buffer(3)]], | |
| 95 | + device float* inv_rms [[buffer(4)]], // [rows] | |
| 96 | + constant NormParams& p [[buffer(5)]], | |
| 97 | + uint row [[threadgroup_position_in_grid]], | |
| 98 | + uint lid [[thread_index_in_threadgroup]], | |
| 99 | + uint tg_size [[threads_per_threadgroup]], | |
| 100 | + uint lane [[thread_index_in_simdgroup]], | |
| 101 | + uint simd_idx [[simdgroup_index_in_threadgroup]], | |
| 102 | + uint n_simds [[simdgroups_per_threadgroup]]) { | |
| 103 | + device const float* x = X + ulong(row) * p.C; | |
| 104 | + device const float* d = dOUT + ulong(row) * p.C; | |
| 105 | + device float* dx = dX + ulong(row) * p.C; | |
| 106 | + threadgroup float scratch[32]; | |
| 107 | + | |
| 108 | + float ss = 0.0f; | |
| 109 | + for (uint j = lid; j < p.C; j += tg_size) ss = fma(x[j], x[j], ss); | |
| 110 | + ss = tg_sum(ss, lane, simd_idx, n_simds, scratch); | |
| 111 | + const float ir = rsqrt(ss / float(p.C) + p.eps); | |
| 112 | + if (lid == 0) inv_rms[row] = ir; | |
| 113 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 114 | + | |
| 115 | + float dot = 0.0f; // sum_j g_j w_j x_j | |
| 116 | + for (uint j = lid; j < p.C; j += tg_size) dot = fma(d[j] * W[j], x[j], dot); | |
| 117 | + dot = tg_sum(dot, lane, simd_idx, n_simds, scratch); | |
| 118 | + | |
| 119 | + const float coef = dot * ir * ir * ir / float(p.C); | |
| 120 | + for (uint j = lid; j < p.C; j += tg_size) | |
| 121 | + dx[j] += d[j] * W[j] * ir - x[j] * coef; | |
| 122 | +} | |
| 123 | + | |
| 124 | +// dw[j] += sum_i g[i,j] * x[i,j] * inv_rms[i] | |
| 125 | +kernel void rmsnorm_bwd_dw_f32(device const float* X [[buffer(0)]], | |
| 126 | + device const float* dOUT [[buffer(1)]], | |
| 127 | + device const float* inv_rms [[buffer(2)]], | |
| 128 | + device float* dW [[buffer(3)]], | |
| 129 | + constant NormParams& p [[buffer(4)]], | |
| 130 | + constant uint& rows [[buffer(5)]], | |
| 131 | + uint j [[thread_position_in_grid]]) { | |
| 132 | + float acc = 0.0f; | |
| 133 | + for (uint i = 0; i < rows; ++i) | |
| 134 | + acc = fma(dOUT[ulong(i) * p.C + j] * X[ulong(i) * p.C + j], inv_rms[i], acc); | |
| 135 | + dW[j] += acc; | |
| 136 | +} | |
| 137 | + | |
| 138 | +kernel void layernorm_bwd_dx_f32(device const float* X [[buffer(0)]], | |
| 139 | + device const float* W [[buffer(1)]], | |
| 140 | + device const float* dOUT [[buffer(2)]], | |
| 141 | + device float* dX [[buffer(3)]], | |
| 142 | + device float* mean_out [[buffer(4)]], // [rows] | |
| 143 | + device float* istd_out [[buffer(5)]], // [rows] | |
| 144 | + constant NormParams& p [[buffer(6)]], | |
| 145 | + uint row [[threadgroup_position_in_grid]], | |
| 146 | + uint lid [[thread_index_in_threadgroup]], | |
| 147 | + uint tg_size [[threads_per_threadgroup]], | |
| 148 | + uint lane [[thread_index_in_simdgroup]], | |
| 149 | + uint simd_idx [[simdgroup_index_in_threadgroup]], | |
| 150 | + uint n_simds [[simdgroups_per_threadgroup]]) { | |
| 151 | + device const float* x = X + ulong(row) * p.C; | |
| 152 | + device const float* d = dOUT + ulong(row) * p.C; | |
| 153 | + device float* dx = dX + ulong(row) * p.C; | |
| 154 | + threadgroup float scratch[32]; | |
| 155 | + | |
| 156 | + float s = 0.0f; | |
| 157 | + for (uint j = lid; j < p.C; j += tg_size) s += x[j]; | |
| 158 | + const float mean = tg_sum(s, lane, simd_idx, n_simds, scratch) / float(p.C); | |
| 159 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 160 | + | |
| 161 | + float var = 0.0f; | |
| 162 | + for (uint j = lid; j < p.C; j += tg_size) { | |
| 163 | + const float dv = x[j] - mean; | |
| 164 | + var = fma(dv, dv, var); | |
| 165 | + } | |
| 166 | + var = tg_sum(var, lane, simd_idx, n_simds, scratch) / float(p.C); | |
| 167 | + const float inv_std = rsqrt(var + p.eps); | |
| 168 | + if (lid == 0) { | |
| 169 | + mean_out[row] = mean; | |
| 170 | + istd_out[row] = inv_std; | |
| 171 | + } | |
| 172 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 173 | + | |
| 174 | + float m_dxhat = 0.0f, m_dxhat_xhat = 0.0f; | |
| 175 | + for (uint j = lid; j < p.C; j += tg_size) { | |
| 176 | + const float xhat = (x[j] - mean) * inv_std; | |
| 177 | + const float dxhat = d[j] * W[j]; | |
| 178 | + m_dxhat += dxhat; | |
| 179 | + m_dxhat_xhat = fma(dxhat, xhat, m_dxhat_xhat); | |
| 180 | + } | |
| 181 | + m_dxhat = tg_sum(m_dxhat, lane, simd_idx, n_simds, scratch) / float(p.C); | |
| 182 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 183 | + m_dxhat_xhat = tg_sum(m_dxhat_xhat, lane, simd_idx, n_simds, scratch) / float(p.C); | |
| 184 | + | |
| 185 | + for (uint j = lid; j < p.C; j += tg_size) { | |
| 186 | + const float xhat = (x[j] - mean) * inv_std; | |
| 187 | + dx[j] += inv_std * (d[j] * W[j] - m_dxhat - xhat * m_dxhat_xhat); | |
| 188 | + } | |
| 189 | +} | |
| 190 | + | |
| 191 | +// dw[j] += sum_i g[i,j] * xhat[i,j] ; db[j] += sum_i g[i,j] | |
| 192 | +kernel void layernorm_bwd_dwdb_f32(device const float* X [[buffer(0)]], | |
| 193 | + device const float* dOUT [[buffer(1)]], | |
| 194 | + device const float* mean [[buffer(2)]], | |
| 195 | + device const float* istd [[buffer(3)]], | |
| 196 | + device float* dW [[buffer(4)]], | |
| 197 | + device float* dB [[buffer(5)]], | |
| 198 | + constant NormParams& p [[buffer(6)]], | |
| 199 | + constant uint& rows [[buffer(7)]], | |
| 200 | + uint j [[thread_position_in_grid]]) { | |
| 201 | + float aw = 0.0f, ab = 0.0f; | |
| 202 | + for (uint i = 0; i < rows; ++i) { | |
| 203 | + const float g = dOUT[ulong(i) * p.C + j]; | |
| 204 | + const float xhat = (X[ulong(i) * p.C + j] - mean[i]) * istd[i]; | |
| 205 | + aw = fma(g, xhat, aw); | |
| 206 | + ab += g; | |
| 207 | + } | |
| 208 | + dW[j] += aw; | |
| 209 | + dB[j] += ab; | |
| 210 | +} | |
added
src/kernels/matmul.metal
+76 −0
@@ -0,0 +1,76 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +// | |
| 3 | +// f32 GEMM, correctness-first (kernel roadmap steps 1–2; the simdgroup_matrix | |
| 4 | +// version is M5). Transpose variants are compiled via function constants | |
| 5 | +// TA/TB — four specialized pipelines from one source, no runtime branching | |
| 6 | +// on the hot path beyond what the compiler folds. | |
| 7 | +// | |
| 8 | +// matmul_naive_f32: one thread per output element. Grid (N, M), any | |
| 9 | +// threadgroup shape. | |
| 10 | +// matmul_tiled_f32: 16x16 output tile per threadgroup, 16x16 threads, | |
| 11 | +// K consumed in 16-wide steps through padded threadgroup tiles (+1 column | |
| 12 | +// to break bank conflicts). Edge tiles zero-fill. Grid = ceil(N/16), | |
| 13 | +// ceil(M/16) threadgroups of 16x16. | |
| 14 | +#include <metal_stdlib> | |
| 15 | +using namespace metal; | |
| 16 | + | |
| 17 | +constant bool TA [[function_constant(0)]]; | |
| 18 | +constant bool TB [[function_constant(1)]]; | |
| 19 | + | |
| 20 | +struct MatmulParams { | |
| 21 | + uint M, N, K; | |
| 22 | + uint lda, ldb; // leading dims of A and B as stored | |
| 23 | + uint accumulate; // 1: C += A·B (backward-pass gradients) | |
| 24 | +}; | |
| 25 | + | |
| 26 | +inline float load_a(device const float* A, uint i, uint k, uint lda) { | |
| 27 | + return TA ? A[k * lda + i] : A[i * lda + k]; | |
| 28 | +} | |
| 29 | +inline float load_b(device const float* B, uint k, uint j, uint ldb) { | |
| 30 | + return TB ? B[j * ldb + k] : B[k * ldb + j]; | |
| 31 | +} | |
| 32 | + | |
| 33 | +kernel void matmul_naive_f32(device const float* A [[buffer(0)]], | |
| 34 | + device const float* B [[buffer(1)]], | |
| 35 | + device float* C [[buffer(2)]], | |
| 36 | + constant MatmulParams& p [[buffer(3)]], | |
| 37 | + uint2 gid [[thread_position_in_grid]]) { | |
| 38 | + if (gid.x >= p.N || gid.y >= p.M) return; | |
| 39 | + float acc = 0.0f; | |
| 40 | + for (uint k = 0; k < p.K; ++k) | |
| 41 | + acc += load_a(A, gid.y, k, p.lda) * load_b(B, k, gid.x, p.ldb); | |
| 42 | + const uint idx = gid.y * p.N + gid.x; | |
| 43 | + C[idx] = p.accumulate ? (C[idx] + acc) : acc; | |
| 44 | +} | |
| 45 | + | |
| 46 | +constant constexpr uint TILE = 16; | |
| 47 | + | |
| 48 | +kernel void matmul_tiled_f32(device const float* A [[buffer(0)]], | |
| 49 | + device const float* B [[buffer(1)]], | |
| 50 | + device float* C [[buffer(2)]], | |
| 51 | + constant MatmulParams& p [[buffer(3)]], | |
| 52 | + uint2 tgid [[threadgroup_position_in_grid]], | |
| 53 | + uint2 tid [[thread_position_in_threadgroup]]) { | |
| 54 | + threadgroup float As[TILE][TILE + 1]; | |
| 55 | + threadgroup float Bs[TILE][TILE + 1]; | |
| 56 | + | |
| 57 | + const uint row = tgid.y * TILE + tid.y; | |
| 58 | + const uint col = tgid.x * TILE + tid.x; | |
| 59 | + | |
| 60 | + float acc = 0.0f; | |
| 61 | + for (uint kt = 0; kt < p.K; kt += TILE) { | |
| 62 | + const uint ka = kt + tid.x; | |
| 63 | + const uint kb = kt + tid.y; | |
| 64 | + As[tid.y][tid.x] = (row < p.M && ka < p.K) ? load_a(A, row, ka, p.lda) : 0.0f; | |
| 65 | + Bs[tid.y][tid.x] = (kb < p.K && col < p.N) ? load_b(B, kb, col, p.ldb) : 0.0f; | |
| 66 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 67 | + for (uint kk = 0; kk < TILE; ++kk) | |
| 68 | + acc = fma(As[tid.y][kk], Bs[kk][tid.x], acc); | |
| 69 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 70 | + } | |
| 71 | + | |
| 72 | + if (row < p.M && col < p.N) { | |
| 73 | + const uint idx = row * p.N + col; | |
| 74 | + C[idx] = p.accumulate ? (C[idx] + acc) : acc; | |
| 75 | + } | |
| 76 | +} | |
added
src/kernels/matmul_mpp.metal
+86 −0
@@ -0,0 +1,86 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +// | |
| 3 | +// GEMM via Metal Performance Primitives cooperative tensors | |
| 4 | +// (mpp::tensor_ops::matmul2d). This is the Metal 4 path that targets the | |
| 5 | +// per-core neural accelerators on M5-class hardware, and the same one | |
| 6 | +// llama.cpp uses behind GGML_METAL_HAS_TENSOR. Whether it beats the | |
| 7 | +// hand-written simdgroup_matrix kernel is a hardware question, so it exists | |
| 8 | +// here to be benchmarked (tests/bench_precision.cpp) rather than assumed: | |
| 9 | +// the "Rigel" paper measured matmul2d on an M4 Max still executing on the | |
| 10 | +// shader cores, with a hand-fused GEMM winning. | |
| 11 | +// | |
| 12 | +// Tiling: 64x32 output tile per threadgroup, 4 simdgroups (128 threads), K | |
| 13 | +// as a dynamic extent so one pipeline serves every K. Shapes must be exact | |
| 14 | +// multiples of the tile; the caller falls back otherwise. | |
| 15 | +#include <metal_stdlib> | |
| 16 | +#include <metal_tensor> | |
| 17 | +#include <MetalPerformancePrimitives/MetalPerformancePrimitives.h> | |
| 18 | + | |
| 19 | +using namespace metal; | |
| 20 | +using namespace mpp::tensor_ops; | |
| 21 | + | |
| 22 | +enum : int { TILE_M = 64, TILE_N = 32 }; | |
| 23 | + | |
| 24 | +// Operands arrive as plain device pointers and become tensors in-kernel via | |
| 25 | +// the `tensor_inline` descriptor, so the host binds ordinary buffers and needs | |
| 26 | +// no MTLTensor plumbing (the default `tensor_handle` descriptor wraps an | |
| 27 | +// opaque handle that only a host-side MTLTensor can supply). | |
| 28 | +struct MPPParams { uint32_t M, N, K; }; | |
| 29 | + | |
| 30 | +// Transposes are template parameters because the descriptor is a constexpr | |
| 31 | +// template argument. Training needs all of nn (dX = dY.W), nt (fwd X.W^T) and | |
| 32 | +// tn (dW = dY^T.X), so all three are instantiated. | |
| 33 | +template <typename T, bool TA, bool TB> | |
| 34 | +kernel void matmul_mpp(device T* Ap [[buffer(0)]], | |
| 35 | + device T* Bp [[buffer(1)]], | |
| 36 | + device float* Cp [[buffer(2)]], | |
| 37 | + constant MPPParams& p [[buffer(3)]], | |
| 38 | + uint2 tgid [[threadgroup_position_in_grid]]) { | |
| 39 | + const int32_t M = int32_t(p.M), N = int32_t(p.N), K = int32_t(p.K); | |
| 40 | + // extents are (columns, rows): A is MxK, B is KxN, C is MxN | |
| 41 | + // Element type must be non-const: MPP static_asserts it is exactly one of | |
| 42 | + // uint8_t/int8_t/uint4b/int4b/float/half/bfloat. | |
| 43 | + tensor<device T, dextents<int32_t, 2>, tensor_inline> A( | |
| 44 | + Ap, TA ? dextents<int32_t, 2>(M, K) : dextents<int32_t, 2>(K, M)); | |
| 45 | + tensor<device T, dextents<int32_t, 2>, tensor_inline> B( | |
| 46 | + Bp, TB ? dextents<int32_t, 2>(K, N) : dextents<int32_t, 2>(N, K)); | |
| 47 | + tensor<device float, dextents<int32_t, 2>, tensor_inline> C( | |
| 48 | + Cp, dextents<int32_t, 2>(N, M)); | |
| 49 | + | |
| 50 | + // relaxed_precision=false keeps f32 accumulation semantics. | |
| 51 | + constexpr auto desc = matmul2d_descriptor( | |
| 52 | + TILE_M, TILE_N, static_cast<int>(dynamic_extent), | |
| 53 | + /*transpose_left=*/TA, /*transpose_right=*/TB, | |
| 54 | + /*relaxed_precision=*/false, matmul2d_descriptor::mode::multiply); | |
| 55 | + | |
| 56 | + matmul2d<desc, execution_simdgroups<4>> op; | |
| 57 | + | |
| 58 | + // slice() is (column, row); a transposed operand is indexed the other way. | |
| 59 | + auto mA = TA ? A.slice(int(tgid.y) * TILE_M, 0) : A.slice(0, int(tgid.y) * TILE_M); | |
| 60 | + auto mB = TB ? B.slice(0, int(tgid.x) * TILE_N) : B.slice(int(tgid.x) * TILE_N, 0); | |
| 61 | + auto mC = C.slice(int(tgid.x) * TILE_N, int(tgid.y) * TILE_M); | |
| 62 | + | |
| 63 | + // The cooperative tensor's element->lane distribution is implementation | |
| 64 | + // defined and not every slot a thread holds is valid, hence the | |
| 65 | + // is_valid_element guard. (The header's own example still says get_mask, | |
| 66 | + // which does not exist in this SDK.) | |
| 67 | + auto cT = op.template get_destination_cooperative_tensor<decltype(mA), decltype(mB), float>(); | |
| 68 | +#pragma clang loop unroll(full) | |
| 69 | + for (uint16_t i = 0; i < cT.get_capacity(); ++i) | |
| 70 | + if (cT.is_valid_element(i)) cT[i] = 0.0f; | |
| 71 | + | |
| 72 | + op.run(mA, mB, cT); | |
| 73 | + cT.store(mC); | |
| 74 | +} | |
| 75 | + | |
| 76 | +#define INST_MPP(SUF, T, TA, TB) \ | |
| 77 | + template [[host_name("matmul_mpp_" #SUF)]] kernel void \ | |
| 78 | + matmul_mpp<T, TA, TB>(device T*, device T*, device float*, \ | |
| 79 | + constant MPPParams&, uint2); | |
| 80 | + | |
| 81 | +INST_MPP(f32, float, false, false) | |
| 82 | +INST_MPP(f32_nt, float, false, true) | |
| 83 | +INST_MPP(f32_tn, float, true, false) | |
| 84 | +INST_MPP(f16, half, false, false) | |
| 85 | +INST_MPP(f16_nt, half, false, true) | |
| 86 | +INST_MPP(f16_tn, half, true, false) | |
added
src/kernels/matmul_simd.metal
+187 −0
@@ -0,0 +1,187 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +// | |
| 3 | +// simdgroup_matrix GEMM — the perf-critical path (kernel roadmap step 3). | |
| 4 | +// Structure follows MLX's STEEL kernel (RESEARCH.md §5): device→threadgroup | |
| 5 | +// staged tiles with +16-byte row padding against bank conflicts, 8x8 | |
| 6 | +// simdgroup fragments, f32 accumulators, transposes handled by function | |
| 7 | +// constants rather than data movement. | |
| 8 | +// | |
| 9 | +// Threadgroup layout (BM=64, BN=64, BK=16, WM=2, WN=2): | |
| 10 | +// 4 simdgroups = 128 threads. Simdgroup (wm, wn) owns a 32x32 output | |
| 11 | +// quadrant = 4x4 grid of 8x8 fragments (TM=TN=4 -> 16 f32 accumulators, | |
| 12 | +// 32 registers/thread). | |
| 13 | +// Threadgroup memory: As 64x(16+4) + Bs 16x(64+4) + a 64-float epilogue | |
| 14 | +// scratch per simdgroup = 10.5 KB, so ~3 threadgroups stay resident per | |
| 15 | +// core. (An earlier version staged the whole 64x64 output block: 27 KB, | |
| 16 | +// one threadgroup per core, 3x slower than the 16x16 tiled kernel.) | |
| 17 | +// | |
| 18 | +// Staging is transpose-aware: the thread->address map always walks the | |
| 19 | +// operand's CONTIGUOUS axis, so all four of nn/nt/tn/tt coalesce. Mapping | |
| 20 | +// idx by the fixed logical axis instead costs ~3x on the nt case that the | |
| 21 | +// forward pass (X·Wᵀ) issues most. | |
| 22 | +// | |
| 23 | +// Tile constants are ENUMERATORS, not `constant constexpr uint`: in MSL | |
| 24 | +// `constant` is an address-space qualifier (and program-scope variables must | |
| 25 | +// live there), so `constant constexpr uint TM = 4` declares a VARIABLE and loop | |
| 26 | +// bounds built from it are not compile-time known. The fragment loops then don't unroll, `acc[i][j]` | |
| 27 | +// becomes dynamic indexing into an opaque simdgroup_matrix array, and the | |
| 28 | +// compiler allocas all 16 accumulators (256 B each) to the stack — every MMA | |
| 29 | +// pays ~1 KB of stack traffic. That cost 3x: 0.8 -> 2.4+ TFLOPs on f32. | |
| 30 | +// Verify with: metal -S -emit-llvm, then check for `alloca` and count | |
| 31 | +// air.simdgroup_matrix_8x8_multiply_accumulate calls (want TM*TN*BK/8, not 2). | |
| 32 | +// | |
| 33 | +// Edge handling: ALIGNED is a function constant, so fully-tiled matrices get | |
| 34 | +// the branch-free path with fragments stored straight to device memory, and | |
| 35 | +// ragged ones get predicated loads plus a per-fragment predicated epilogue. | |
| 36 | +// The K remainder is handled by zero-filling the staged tile. | |
| 37 | +#include <metal_stdlib> | |
| 38 | +#include <metal_simdgroup_matrix> | |
| 39 | +using namespace metal; | |
| 40 | + | |
| 41 | +constant bool TA [[function_constant(0)]]; | |
| 42 | +constant bool TB [[function_constant(1)]]; | |
| 43 | +constant bool ALIGNED [[function_constant(3)]]; | |
| 44 | + | |
| 45 | +struct MatmulParams { | |
| 46 | + uint M, N, K; | |
| 47 | + uint lda, ldb; | |
| 48 | + uint accumulate; | |
| 49 | +}; | |
| 50 | + | |
| 51 | +// Tile geometry as ENUMERATORS, not `constant` variables — see the note above. | |
| 52 | +enum : uint { | |
| 53 | + BM = 64, | |
| 54 | + BN = 64, | |
| 55 | + BK = 16, | |
| 56 | + WM = 2, // simdgroups along M | |
| 57 | + WN = 2, // simdgroups along N | |
| 58 | + NSG = WM * WN, | |
| 59 | + THREADS = NSG * 32, | |
| 60 | + TM = BM / (8 * WM), // 8x8 fragments per simdgroup along M | |
| 61 | + TN = BN / (8 * WN), | |
| 62 | + PAD = 4, // 16 bytes / sizeof(float) | |
| 63 | + LDA_S = BK + PAD, // staged A row stride | |
| 64 | + LDB_S = BN + PAD, // staged B row stride | |
| 65 | +}; | |
| 66 | + | |
| 67 | +inline void stage_a(threadgroup float* As, device const float* A, | |
| 68 | + uint row0, uint k0, constant MatmulParams& p, uint tid) { | |
| 69 | + for (uint idx = tid; idx < BM * BK; idx += THREADS) { | |
| 70 | + // walk A's contiguous axis with consecutive threads | |
| 71 | + const uint i = TA ? (idx % BM) : (idx / BK); | |
| 72 | + const uint k = TA ? (idx / BM) : (idx % BK); | |
| 73 | + const uint gr = row0 + i; | |
| 74 | + const uint gk = k0 + k; | |
| 75 | + float v = 0.0f; | |
| 76 | + if (ALIGNED || (gr < p.M && gk < p.K)) | |
| 77 | + v = TA ? A[ulong(gk) * p.lda + gr] : A[ulong(gr) * p.lda + gk]; | |
| 78 | + As[i * LDA_S + k] = v; | |
| 79 | + } | |
| 80 | +} | |
| 81 | + | |
| 82 | +inline void stage_b(threadgroup float* Bs, device const float* B, | |
| 83 | + uint k0, uint col0, constant MatmulParams& p, uint tid) { | |
| 84 | + for (uint idx = tid; idx < BK * BN; idx += THREADS) { | |
| 85 | + const uint k = TB ? (idx % BK) : (idx / BN); | |
| 86 | + const uint j = TB ? (idx / BK) : (idx % BN); | |
| 87 | + const uint gk = k0 + k; | |
| 88 | + const uint gc = col0 + j; | |
| 89 | + float v = 0.0f; | |
| 90 | + if (ALIGNED || (gk < p.K && gc < p.N)) | |
| 91 | + v = TB ? B[ulong(gc) * p.ldb + gk] : B[ulong(gk) * p.ldb + gc]; | |
| 92 | + Bs[k * LDB_S + j] = v; | |
| 93 | + } | |
| 94 | +} | |
| 95 | + | |
| 96 | +kernel void matmul_simd_f32(device const float* A [[buffer(0)]], | |
| 97 | + device const float* B [[buffer(1)]], | |
| 98 | + device float* C [[buffer(2)]], | |
| 99 | + constant MatmulParams& p [[buffer(3)]], | |
| 100 | + uint2 tgid [[threadgroup_position_in_grid]], | |
| 101 | + uint tid [[thread_index_in_threadgroup]], | |
| 102 | + uint lane [[thread_index_in_simdgroup]], | |
| 103 | + uint sgid [[simdgroup_index_in_threadgroup]]) { | |
| 104 | + threadgroup float As[BM * LDA_S]; | |
| 105 | + threadgroup float Bs[BK * LDB_S]; | |
| 106 | + threadgroup float frag_scratch[NSG * 64]; // epilogue only, 1 KB total | |
| 107 | + | |
| 108 | + const uint row0 = tgid.y * BM; | |
| 109 | + const uint col0 = tgid.x * BN; | |
| 110 | + | |
| 111 | + // this simdgroup's 32x32 quadrant of the 64x64 block | |
| 112 | + const uint sg_row = (sgid / WN) * (TM * 8); | |
| 113 | + const uint sg_col = (sgid % WN) * (TN * 8); | |
| 114 | + | |
| 115 | + simdgroup_float8x8 acc[TM][TN]; | |
| 116 | +#pragma clang loop unroll(full) | |
| 117 | + for (uint i = 0; i < TM; ++i) | |
| 118 | +#pragma clang loop unroll(full) | |
| 119 | + for (uint j = 0; j < TN; ++j) | |
| 120 | + acc[i][j] = make_filled_simdgroup_matrix<float, 8, 8>(0.0f); | |
| 121 | + | |
| 122 | + for (uint k0 = 0; k0 < p.K; k0 += BK) { | |
| 123 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 124 | + stage_a(As, A, row0, k0, p, tid); | |
| 125 | + stage_b(Bs, B, k0, col0, p, tid); | |
| 126 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 127 | + | |
| 128 | +#pragma clang loop unroll(full) | |
| 129 | + for (uint kk = 0; kk < BK; kk += 8) { | |
| 130 | + simdgroup_float8x8 afrag[TM]; | |
| 131 | + simdgroup_float8x8 bfrag[TN]; | |
| 132 | + simdgroup_barrier(mem_flags::mem_none); | |
| 133 | +#pragma clang loop unroll(full) | |
| 134 | + for (uint i = 0; i < TM; ++i) | |
| 135 | + simdgroup_load(afrag[i], As + (sg_row + i * 8) * LDA_S + kk, LDA_S); | |
| 136 | + simdgroup_barrier(mem_flags::mem_none); | |
| 137 | +#pragma clang loop unroll(full) | |
| 138 | + for (uint j = 0; j < TN; ++j) | |
| 139 | + simdgroup_load(bfrag[j], Bs + kk * LDB_S + sg_col + j * 8, LDB_S); | |
| 140 | + simdgroup_barrier(mem_flags::mem_none); | |
| 141 | +#pragma clang loop unroll(full) | |
| 142 | + for (uint i = 0; i < TM; ++i) { | |
| 143 | + // serpentine N order keeps the last-touched b fragment hot | |
| 144 | +#pragma clang loop unroll(full) | |
| 145 | + for (uint jj = 0; jj < TN; ++jj) { | |
| 146 | + const uint j = (i & 1) ? (TN - 1 - jj) : jj; | |
| 147 | + simdgroup_multiply_accumulate(acc[i][j], afrag[i], bfrag[j], acc[i][j]); | |
| 148 | + } | |
| 149 | + } | |
| 150 | + } | |
| 151 | + } | |
| 152 | + | |
| 153 | + const uint out_row = row0 + sg_row; | |
| 154 | + const uint out_col = col0 + sg_col; | |
| 155 | + | |
| 156 | + // Fast path: whole block in range and overwriting -> fragments go | |
| 157 | + // straight to device memory. | |
| 158 | + if (ALIGNED && !p.accumulate) { | |
| 159 | +#pragma clang loop unroll(full) | |
| 160 | + for (uint i = 0; i < TM; ++i) | |
| 161 | +#pragma clang loop unroll(full) | |
| 162 | + for (uint j = 0; j < TN; ++j) | |
| 163 | + simdgroup_store(acc[i][j], | |
| 164 | + C + ulong(out_row + i * 8) * p.N + out_col + j * 8, p.N); | |
| 165 | + return; | |
| 166 | + } | |
| 167 | + | |
| 168 | + // Slow path: one 8x8 fragment at a time through simdgroup-local scratch | |
| 169 | + // (no threadgroup barrier needed — only this simdgroup touches it), then | |
| 170 | + // 2 predicated element writes per lane. | |
| 171 | + threadgroup float* scratch = frag_scratch + sgid * 64; | |
| 172 | + for (uint i = 0; i < TM; ++i) { | |
| 173 | + for (uint j = 0; j < TN; ++j) { | |
| 174 | + simdgroup_barrier(mem_flags::mem_threadgroup); | |
| 175 | + simdgroup_store(acc[i][j], scratch, 8); | |
| 176 | + simdgroup_barrier(mem_flags::mem_threadgroup); | |
| 177 | + for (uint e = lane; e < 64; e += 32) { | |
| 178 | + const uint gr = out_row + i * 8 + e / 8; | |
| 179 | + const uint gc = out_col + j * 8 + e % 8; | |
| 180 | + if (!ALIGNED && (gr >= p.M || gc >= p.N)) continue; | |
| 181 | + const ulong o = ulong(gr) * p.N + gc; | |
| 182 | + const float v = scratch[e]; | |
| 183 | + C[o] = p.accumulate ? (C[o] + v) : v; | |
| 184 | + } | |
| 185 | + } | |
| 186 | + } | |
| 187 | +} | |
added
src/kernels/precision_probe.metal
+76 −0
@@ -0,0 +1,76 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +// Micro-benchmark: does simdgroup_matrix run faster with f16/bf16 operands | |
| 3 | +// than with f32 on this hardware? Same 64x64x16 tiling as the real GEMM; | |
| 4 | +// only the operand type of the staged tiles and the MMA changes. The | |
| 5 | +// accumulator stays f32 in every variant (required for training). | |
| 6 | +#include <metal_stdlib> | |
| 7 | +#include <metal_simdgroup_matrix> | |
| 8 | +using namespace metal; | |
| 9 | + | |
| 10 | +enum : uint { BM = 64, BN = 64, BK = 16, WM = 2, WN = 2, | |
| 11 | + NSG = WM*WN, THREADS = NSG*32, TM = BM/(8*WM), TN = BN/(8*WN), | |
| 12 | + PAD = 8, LDA_S = BK+PAD, LDB_S = BN+PAD }; | |
| 13 | + | |
| 14 | +struct P { uint M, N, K; }; | |
| 15 | + | |
| 16 | +// T = operand precision, ACC fragment always f32. | |
| 17 | +template <typename T, typename FRAG> | |
| 18 | +kernel void gemm_prec(device const T* A [[buffer(0)]], | |
| 19 | + device const T* B [[buffer(1)]], | |
| 20 | + device float* C [[buffer(2)]], | |
| 21 | + constant P& p [[buffer(3)]], | |
| 22 | + uint2 tgid [[threadgroup_position_in_grid]], | |
| 23 | + uint tid [[thread_index_in_threadgroup]], | |
| 24 | + uint sgid [[simdgroup_index_in_threadgroup]]) { | |
| 25 | + threadgroup T As[BM * LDA_S]; | |
| 26 | + threadgroup T Bs[BK * LDB_S]; | |
| 27 | + const uint row0 = tgid.y * BM, col0 = tgid.x * BN; | |
| 28 | + const uint sr = (sgid / WN) * (TM * 8), sc = (sgid % WN) * (TN * 8); | |
| 29 | + | |
| 30 | + simdgroup_float8x8 acc[TM][TN]; | |
| 31 | +#pragma clang loop unroll(full) | |
| 32 | + for (uint i = 0; i < TM; ++i) | |
| 33 | +#pragma clang loop unroll(full) | |
| 34 | + for (uint j = 0; j < TN; ++j) acc[i][j] = make_filled_simdgroup_matrix<float,8,8>(0.0f); | |
| 35 | + | |
| 36 | + for (uint k0 = 0; k0 < p.K; k0 += BK) { | |
| 37 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 38 | + for (uint e = tid; e < BM*BK; e += THREADS) { | |
| 39 | + const uint i = e / BK, k = e % BK; | |
| 40 | + As[i*LDA_S + k] = A[(row0+i)*p.K + k0+k]; | |
| 41 | + } | |
| 42 | + for (uint e = tid; e < BK*BN; e += THREADS) { | |
| 43 | + const uint k = e / BN, j = e % BN; | |
| 44 | + Bs[k*LDB_S + j] = B[(k0+k)*p.N + col0+j]; | |
| 45 | + } | |
| 46 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 47 | +#pragma clang loop unroll(full) | |
| 48 | + for (uint kk = 0; kk < BK; kk += 8) { | |
| 49 | + FRAG af[TM], bf[TN]; | |
| 50 | +#pragma clang loop unroll(full) | |
| 51 | + for (uint i = 0; i < TM; ++i) simdgroup_load(af[i], As + (sr+i*8)*LDA_S + kk, LDA_S); | |
| 52 | +#pragma clang loop unroll(full) | |
| 53 | + for (uint j = 0; j < TN; ++j) simdgroup_load(bf[j], Bs + kk*LDB_S + sc + j*8, LDB_S); | |
| 54 | +#pragma clang loop unroll(full) | |
| 55 | + for (uint i = 0; i < TM; ++i) | |
| 56 | +#pragma clang loop unroll(full) | |
| 57 | + for (uint j = 0; j < TN; ++j) | |
| 58 | + simdgroup_multiply_accumulate(acc[i][j], af[i], bf[j], acc[i][j]); | |
| 59 | + } | |
| 60 | + } | |
| 61 | +#pragma clang loop unroll(full) | |
| 62 | + for (uint i = 0; i < TM; ++i) | |
| 63 | +#pragma clang loop unroll(full) | |
| 64 | + for (uint j = 0; j < TN; ++j) | |
| 65 | + simdgroup_store(acc[i][j], C + (row0+sr+i*8)*p.N + col0+sc+j*8, p.N); | |
| 66 | +} | |
| 67 | + | |
| 68 | +template [[host_name("gemm_f32")]] kernel void | |
| 69 | +gemm_prec<float, simdgroup_float8x8>(device const float*, device const float*, | |
| 70 | + device float*, constant P&, uint2, uint, uint); | |
| 71 | +template [[host_name("gemm_f16")]] kernel void | |
| 72 | +gemm_prec<half, simdgroup_half8x8>(device const half*, device const half*, | |
| 73 | + device float*, constant P&, uint2, uint, uint); | |
| 74 | +template [[host_name("gemm_bf16")]] kernel void | |
| 75 | +gemm_prec<bfloat, simdgroup_bfloat8x8>(device const bfloat*, device const bfloat*, | |
| 76 | + device float*, constant P&, uint2, uint, uint); | |
added
src/kernels/softmax.metal
+66 −0
@@ -0,0 +1,66 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +// | |
| 3 | +// Fused online softmax over the last dim. One threadgroup per row; each | |
| 4 | +// thread keeps a running (max, sum) over a strided slice of the row, lanes | |
| 5 | +// combine with simd reductions, simdgroups combine through a small | |
| 6 | +// threadgroup scratch (<= 32 simdgroups, so one simd-reduce finishes it). | |
| 7 | +// Row layout: X[row * C + j]. | |
| 8 | +// | |
| 9 | +// Masked/padded values are the caller's problem; -FLT_MAX sentinels are safe | |
| 10 | +// here (never -INFINITY: fast-math exp(-inf) is undefined — RESEARCH.md §8). | |
| 11 | +#include <metal_stdlib> | |
| 12 | +using namespace metal; | |
| 13 | + | |
| 14 | +kernel void softmax_f32(device const float* X [[buffer(0)]], | |
| 15 | + device float* OUT [[buffer(1)]], | |
| 16 | + constant uint& C [[buffer(2)]], | |
| 17 | + uint row [[threadgroup_position_in_grid]], | |
| 18 | + uint lid [[thread_index_in_threadgroup]], | |
| 19 | + uint tg_size [[threads_per_threadgroup]], | |
| 20 | + uint lane [[thread_index_in_simdgroup]], | |
| 21 | + uint simd_idx [[simdgroup_index_in_threadgroup]], | |
| 22 | + uint n_simds [[simdgroups_per_threadgroup]]) { | |
| 23 | + device const float* x = X + ulong(row) * C; | |
| 24 | + device float* out = OUT + ulong(row) * C; | |
| 25 | + | |
| 26 | + // per-thread online (m, l) | |
| 27 | + float m = -FLT_MAX; | |
| 28 | + float l = 0.0f; | |
| 29 | + for (uint j = lid; j < C; j += tg_size) { | |
| 30 | + const float v = x[j]; | |
| 31 | + const float m_new = max(m, v); | |
| 32 | + l = l * exp(m - m_new) + exp(v - m_new); | |
| 33 | + m = m_new; | |
| 34 | + } | |
| 35 | + | |
| 36 | + // combine across the simdgroup | |
| 37 | + float m_simd = simd_max(m); | |
| 38 | + float l_simd = simd_sum(l * exp(m - m_simd)); | |
| 39 | + | |
| 40 | + // combine across simdgroups | |
| 41 | + threadgroup float tg_m[32]; | |
| 42 | + threadgroup float tg_l[32]; | |
| 43 | + if (simd_is_first()) { | |
| 44 | + tg_m[simd_idx] = m_simd; | |
| 45 | + tg_l[simd_idx] = l_simd; | |
| 46 | + } | |
| 47 | + threadgroup_barrier(mem_flags::mem_threadgroup); | |
| 48 | + | |
| 49 | + float m_row, l_row; | |
| 50 | + { | |
| 51 | + // Indexed by lane-within-simdgroup so EVERY simdgroup runs the same | |
| 52 | + // reduction over the same scratch and lands on identical row stats. | |
| 53 | + const uint i = min(lane, n_simds - 1); // lanes >= n_simds mirror the last entry | |
| 54 | + const float mi = tg_m[i]; | |
| 55 | + const float li = tg_l[i]; | |
| 56 | + m_row = simd_max(mi); | |
| 57 | + // lanes beyond n_simds would double-count: zero their contribution | |
| 58 | + const float contrib = (lane < n_simds) ? li * exp(mi - m_row) : 0.0f; | |
| 59 | + l_row = simd_sum(contrib); | |
| 60 | + } | |
| 61 | + | |
| 62 | + const float inv_l = 1.0f / l_row; | |
| 63 | + for (uint j = lid; j < C; j += tg_size) { | |
| 64 | + out[j] = exp(x[j] - m_row) * inv_l; | |
| 65 | + } | |
| 66 | +} | |
added
src/main.cpp
+254 −0
@@ -0,0 +1,254 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +// | |
| 3 | +// forge CLI: | |
| 4 | +// forge train --config configs/gpt-25m.json --data data/tinystories --out runs/exp1 | |
| 5 | +// [--resume runs/exp1/ckpt_latest.bin] [--backend metal|cpu] | |
| 6 | +// forge generate --checkpoint runs/exp1/ckpt_latest.bin --tokenizer data/tinystories/tok4096.model | |
| 7 | +// --prompt "Once upon a time" [--temp 0.8] [--top-k 40] [--max-tokens 256] | |
| 8 | +// forge eval --checkpoint runs/exp1/ckpt_latest.bin --data data/tinystories/val.bin | |
| 9 | +// [--batches 50] | |
| 10 | +// forge info [--config <json>] | |
| 11 | +#include <Foundation/Foundation.hpp> | |
| 12 | +#include <Metal/Metal.hpp> | |
| 13 | + | |
| 14 | +#include "core/device.h" | |
| 15 | +#include "nn/config.h" | |
| 16 | +#include "nn/transformer.h" | |
| 17 | +#include "ops/metal/metal_ops.h" | |
| 18 | +#include "ops/ops.h" | |
| 19 | +#include "tokenizer/bpe.h" | |
| 20 | +#include "train/checkpoint.h" | |
| 21 | +#include "train/dataloader.h" | |
| 22 | +#include "train/trainer.h" | |
| 23 | + | |
| 24 | +#include <cmath> | |
| 25 | +#include <cstdio> | |
| 26 | +#include <fstream> | |
| 27 | +#include <map> | |
| 28 | +#include <random> | |
| 29 | +#include <sstream> | |
| 30 | +#include <string> | |
| 31 | +#include <vector> | |
| 32 | + | |
| 33 | +namespace { | |
| 34 | + | |
| 35 | +void print_usage() { | |
| 36 | + std::printf( | |
| 37 | + "usage: forge <command> [options]\n" | |
| 38 | + "\n" | |
| 39 | + "commands:\n" | |
| 40 | + " train --config <json> --data <dir> --out <dir> [--resume <ckpt>] [--backend metal|cpu]\n" | |
| 41 | + " generate --checkpoint <ckpt> --tokenizer <model> --prompt <text>\n" | |
| 42 | + " [--temp t] [--top-k k] [--max-tokens n] [--seed s]\n" | |
| 43 | + " eval --checkpoint <ckpt> --data <val.bin> [--batches n]\n" | |
| 44 | + " info [--config <json>]\n"); | |
| 45 | +} | |
| 46 | + | |
| 47 | +std::map<std::string, std::string> parse_flags(int argc, char** argv, int start) { | |
| 48 | + std::map<std::string, std::string> flags; | |
| 49 | + for (int i = start; i < argc; ++i) { | |
| 50 | + std::string arg = argv[i]; | |
| 51 | + if (arg.rfind("--", 0) != 0) continue; | |
| 52 | + std::string key = arg.substr(2); | |
| 53 | + if (i + 1 < argc && std::string(argv[i + 1]).rfind("--", 0) != 0) { | |
| 54 | + flags[key] = argv[++i]; | |
| 55 | + } else { | |
| 56 | + flags[key] = "true"; | |
| 57 | + } | |
| 58 | + } | |
| 59 | + return flags; | |
| 60 | +} | |
| 61 | + | |
| 62 | +std::string flag(const std::map<std::string, std::string>& flags, const std::string& key, | |
| 63 | + const std::string& fallback = "") { | |
| 64 | + auto it = flags.find(key); | |
| 65 | + return it == flags.end() ? fallback : it->second; | |
| 66 | +} | |
| 67 | + | |
| 68 | +void select_backend(const std::string& name) { | |
| 69 | + if (name == "cpu") { | |
| 70 | + forge::ops::set_backend(forge::ops::Backend::CPU); | |
| 71 | + } else { | |
| 72 | + forge::ops::set_backend(forge::ops::Backend::Metal); | |
| 73 | + forge::Device::get(); // fail fast if Metal init is broken | |
| 74 | + } | |
| 75 | +} | |
| 76 | + | |
| 77 | +int cmd_info(const std::map<std::string, std::string>& flags) { | |
| 78 | + forge::Device& dev = forge::Device::get(); | |
| 79 | + std::printf("device: %s\n", dev.name().c_str()); | |
| 80 | + std::printf("recommended working set: %.1f GB\n", | |
| 81 | + double(dev.recommended_working_set()) / (1024.0 * 1024.0 * 1024.0)); | |
| 82 | + if (auto path = flag(flags, "config"); !path.empty()) { | |
| 83 | + forge::Config cfg = forge::load_config(path); | |
| 84 | + const auto& m = cfg.model; | |
| 85 | + std::printf("\nmodel: %s\n", m.name.c_str()); | |
| 86 | + std::printf(" layers=%lld d_model=%lld heads=%lld kv_heads=%lld head_dim=%lld\n", | |
| 87 | + m.n_layers, m.d_model, m.n_heads, m.n_kv_heads, m.head_dim()); | |
| 88 | + std::printf(" d_ff=%lld vocab=%lld context=%lld\n", m.d_ff, m.vocab_size, | |
| 89 | + m.context_length); | |
| 90 | + std::printf(" norm=%s act=%s rope=%d tied=%d\n", m.norm.c_str(), | |
| 91 | + m.activation.c_str(), m.use_rope, m.tied_embeddings); | |
| 92 | + std::printf(" parameters: %.2fM\n", double(m.num_params()) / 1e6); | |
| 93 | + } | |
| 94 | + return 0; | |
| 95 | +} | |
| 96 | + | |
| 97 | +int cmd_train(const std::map<std::string, std::string>& flags) { | |
| 98 | + const std::string config_path = flag(flags, "config"); | |
| 99 | + const std::string data_dir = flag(flags, "data"); | |
| 100 | + const std::string out_dir = flag(flags, "out"); | |
| 101 | + if (config_path.empty() || data_dir.empty() || out_dir.empty()) { | |
| 102 | + print_usage(); | |
| 103 | + return 1; | |
| 104 | + } | |
| 105 | + select_backend(flag(flags, "backend", "metal")); | |
| 106 | + | |
| 107 | + std::ifstream in(config_path); | |
| 108 | + std::stringstream ss; | |
| 109 | + ss << in.rdbuf(); | |
| 110 | + const std::string config_json = ss.str(); | |
| 111 | + | |
| 112 | + forge::Config cfg = forge::load_config(config_path); | |
| 113 | + forge::train::Trainer trainer(cfg, data_dir, out_dir, config_json); | |
| 114 | + trainer.train(flag(flags, "resume")); | |
| 115 | + return 0; | |
| 116 | +} | |
| 117 | + | |
| 118 | +int cmd_generate(const std::map<std::string, std::string>& flags) { | |
| 119 | + const std::string ckpt = flag(flags, "checkpoint"); | |
| 120 | + const std::string tok_path = flag(flags, "tokenizer"); | |
| 121 | + if (ckpt.empty() || tok_path.empty()) { | |
| 122 | + print_usage(); | |
| 123 | + return 1; | |
| 124 | + } | |
| 125 | + select_backend(flag(flags, "backend", "metal")); | |
| 126 | + const float temp = std::stof(flag(flags, "temp", "0.8")); | |
| 127 | + const int64_t top_k = std::stoll(flag(flags, "top-k", "40")); | |
| 128 | + const int64_t max_tokens = std::stoll(flag(flags, "max-tokens", "256")); | |
| 129 | + const uint64_t seed = std::stoull(flag(flags, "seed", "1234")); | |
| 130 | + | |
| 131 | + // model config comes from the checkpoint itself | |
| 132 | + nlohmann::json j = nlohmann::json::parse(forge::train::read_checkpoint_config(ckpt)); | |
| 133 | + forge::ModelConfig mc; | |
| 134 | + j.at("model").get_to(mc); | |
| 135 | + forge::nn::Transformer model(mc, 0); | |
| 136 | + forge::train::load_checkpoint(ckpt, model.named_parameters(), nullptr); | |
| 137 | + | |
| 138 | + forge::tok::BPETokenizer tokenizer; | |
| 139 | + tokenizer.load(tok_path); | |
| 140 | + | |
| 141 | + std::vector<int32_t> ctx = tokenizer.encode(flag(flags, "prompt", "Once upon a time")); | |
| 142 | + if (ctx.empty()) ctx.push_back(0); | |
| 143 | + std::printf("%s", tokenizer.decode(ctx).c_str()); | |
| 144 | + std::fflush(stdout); | |
| 145 | + | |
| 146 | + std::mt19937_64 rng(seed); | |
| 147 | + forge::NoGrad ng; | |
| 148 | + for (int64_t n = 0; n < max_tokens; ++n) { | |
| 149 | + // full-context recompute each token (KV cache lands with M5) | |
| 150 | + const int64_t T = | |
| 151 | + std::min<int64_t>(int64_t(ctx.size()), mc.context_length); | |
| 152 | + forge::Tensor ids = forge::Tensor::empty({1, T}, forge::DType::I32); | |
| 153 | + for (int64_t t = 0; t < T; ++t) | |
| 154 | + ids.data<int32_t>()[t] = ctx[ctx.size() - size_t(T) + size_t(t)]; | |
| 155 | + | |
| 156 | + forge::Var logits = model.forward(ids); // [T, V] | |
| 157 | + if (forge::ops::backend() == forge::ops::Backend::Metal) forge::metal::sync(); | |
| 158 | + | |
| 159 | + const float* row = logits.value().data<float>() + (T - 1) * mc.vocab_size; | |
| 160 | + std::vector<std::pair<float, int32_t>> cand(size_t(mc.vocab_size)); | |
| 161 | + for (int64_t v = 0; v < mc.vocab_size; ++v) cand[size_t(v)] = {row[v], int32_t(v)}; | |
| 162 | + const size_t k = size_t(std::min<int64_t>(top_k > 0 ? top_k : mc.vocab_size, | |
| 163 | + mc.vocab_size)); | |
| 164 | + std::partial_sort(cand.begin(), cand.begin() + long(k), cand.end(), | |
| 165 | + [](auto& a, auto& b) { return a.first > b.first; }); | |
| 166 | + | |
| 167 | + // temperature softmax over the top-k | |
| 168 | + float m = cand[0].first; | |
| 169 | + double sum = 0.0; | |
| 170 | + std::vector<double> probs(k); | |
| 171 | + const float tinv = temp > 0.0f ? 1.0f / temp : 1.0f; | |
| 172 | + for (size_t i = 0; i < k; ++i) { | |
| 173 | + probs[i] = std::exp(double((cand[i].first - m) * tinv)); | |
| 174 | + sum += probs[i]; | |
| 175 | + } | |
| 176 | + double r = std::uniform_real_distribution<double>(0.0, sum)(rng); | |
| 177 | + int32_t next = cand[0].second; | |
| 178 | + for (size_t i = 0; i < k; ++i) { | |
| 179 | + r -= probs[i]; | |
| 180 | + if (r <= 0.0) { | |
| 181 | + next = cand[i].second; | |
| 182 | + break; | |
| 183 | + } | |
| 184 | + } | |
| 185 | + ctx.push_back(next); | |
| 186 | + std::printf("%s", tokenizer.decode({next}).c_str()); | |
| 187 | + std::fflush(stdout); | |
| 188 | + } | |
| 189 | + std::printf("\n"); | |
| 190 | + return 0; | |
| 191 | +} | |
| 192 | + | |
| 193 | +int cmd_eval(const std::map<std::string, std::string>& flags) { | |
| 194 | + const std::string ckpt = flag(flags, "checkpoint"); | |
| 195 | + const std::string data = flag(flags, "data"); | |
| 196 | + if (ckpt.empty() || data.empty()) { | |
| 197 | + print_usage(); | |
| 198 | + return 1; | |
| 199 | + } | |
| 200 | + select_backend(flag(flags, "backend", "metal")); | |
| 201 | + const int64_t batches = std::stoll(flag(flags, "batches", "50")); | |
| 202 | + | |
| 203 | + nlohmann::json j = nlohmann::json::parse(forge::train::read_checkpoint_config(ckpt)); | |
| 204 | + forge::Config cfg; | |
| 205 | + if (j.contains("model")) j.at("model").get_to(cfg.model); | |
| 206 | + if (j.contains("train")) j.at("train").get_to(cfg.train); | |
| 207 | + forge::nn::Transformer model(cfg.model, 0); | |
| 208 | + forge::train::load_checkpoint(ckpt, model.named_parameters(), nullptr); | |
| 209 | + | |
| 210 | + forge::train::DataLoader loader(data, cfg.model.context_length, 0); | |
| 211 | + const int64_t B = cfg.train.batch_size, T = cfg.model.context_length; | |
| 212 | + | |
| 213 | + forge::NoGrad ng; | |
| 214 | + double total = 0.0; | |
| 215 | + for (int64_t i = 0; i < batches; ++i) { | |
| 216 | + forge::Tensor ids = forge::Tensor::empty({B, T}, forge::DType::I32); | |
| 217 | + forge::Tensor targets = forge::Tensor::empty({B * T}, forge::DType::I32); | |
| 218 | + loader.seq_batch(i, ids, targets); | |
| 219 | + forge::Var loss = model.loss(ids, targets); | |
| 220 | + if (forge::ops::backend() == forge::ops::Backend::Metal) forge::metal::sync(); | |
| 221 | + total += double(loss.value().data<float>()[0]); | |
| 222 | + } | |
| 223 | + const double mean = total / double(batches); | |
| 224 | + std::printf("val loss: %.4f | ppl: %.2f (%lld batches of %lldx%lld)\n", mean, | |
| 225 | + std::exp(mean), static_cast<long long>(batches), | |
| 226 | + static_cast<long long>(B), static_cast<long long>(T)); | |
| 227 | + return 0; | |
| 228 | +} | |
| 229 | + | |
| 230 | +} // namespace | |
| 231 | + | |
| 232 | +int main(int argc, char** argv) { | |
| 233 | + NS::AutoreleasePool* pool = NS::AutoreleasePool::alloc()->init(); | |
| 234 | + | |
| 235 | + int rc = 0; | |
| 236 | + if (argc < 2) { | |
| 237 | + print_usage(); | |
| 238 | + rc = 1; | |
| 239 | + } else { | |
| 240 | + const std::string command = argv[1]; | |
| 241 | + const auto flags = parse_flags(argc, argv, 2); | |
| 242 | + if (command == "info") rc = cmd_info(flags); | |
| 243 | + else if (command == "train") rc = cmd_train(flags); | |
| 244 | + else if (command == "generate") rc = cmd_generate(flags); | |
| 245 | + else if (command == "eval") rc = cmd_eval(flags); | |
| 246 | + else { | |
| 247 | + print_usage(); | |
| 248 | + rc = 1; | |
| 249 | + } | |
| 250 | + } | |
| 251 | + | |
| 252 | + pool->drain(); | |
| 253 | + return rc; | |
| 254 | +} | |
added
src/nn/attention.h
+71 −0
@@ -0,0 +1,71 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#pragma once | |
| 3 | + | |
| 4 | +#include "nn/config.h" | |
| 5 | +#include "nn/linear.h" | |
| 6 | +#include "ops/ops.h" | |
| 7 | + | |
| 8 | +#include <cmath> | |
| 9 | +#include <memory> | |
| 10 | + | |
| 11 | +namespace forge::nn { | |
| 12 | + | |
| 13 | +// Interface so linear-attention / sliding-window variants can slot in | |
| 14 | +// without touching TransformerBlock. | |
| 15 | +class AttentionBase : public Module { | |
| 16 | +public: | |
| 17 | + virtual ~AttentionBase() = default; | |
| 18 | + // x: [B, T, C] → [B, T, C] | |
| 19 | + virtual Var forward(const Var& x) const = 0; | |
| 20 | +}; | |
| 21 | + | |
| 22 | +// Causal multi-head self-attention with GQA and (optional) RoPE. | |
| 23 | +// No biases (llama convention). | |
| 24 | +class CausalSelfAttention : public AttentionBase { | |
| 25 | +public: | |
| 26 | + CausalSelfAttention(const ModelConfig& cfg, float proj_std, std::mt19937_64& rng) | |
| 27 | + : n_heads_(cfg.n_heads), | |
| 28 | + n_kv_heads_(cfg.n_kv_heads), | |
| 29 | + head_dim_(cfg.head_dim()), | |
| 30 | + use_rope_(cfg.use_rope), | |
| 31 | + rope_theta_(cfg.rope_theta) { | |
| 32 | + const float std = 0.02f; | |
| 33 | + const int64_t C = cfg.d_model; | |
| 34 | + const int64_t Ckv = cfg.n_kv_heads * cfg.head_dim(); | |
| 35 | + wq_ = std::make_unique<Linear>(C, C, false, std, rng); | |
| 36 | + wk_ = std::make_unique<Linear>(C, Ckv, false, std, rng); | |
| 37 | + wv_ = std::make_unique<Linear>(C, Ckv, false, std, rng); | |
| 38 | + wo_ = std::make_unique<Linear>(C, C, false, proj_std, rng); // residual projection | |
| 39 | + absorb("wq", *wq_); | |
| 40 | + absorb("wk", *wk_); | |
| 41 | + absorb("wv", *wv_); | |
| 42 | + absorb("wo", *wo_); | |
| 43 | + } | |
| 44 | + | |
| 45 | + Var forward(const Var& x) const override { | |
| 46 | + const int64_t B = x.value().size(0), T = x.value().size(1), C = x.value().size(2); | |
| 47 | + const int64_t Ckv = n_kv_heads_ * head_dim_; | |
| 48 | + | |
| 49 | + Var x2d = x.reshaped({B * T, C}); | |
| 50 | + Var q = wq_->forward(x2d).reshaped({B, T, C}); | |
| 51 | + Var k = wk_->forward(x2d).reshaped({B, T, Ckv}); | |
| 52 | + Var v = wv_->forward(x2d).reshaped({B, T, Ckv}); | |
| 53 | + | |
| 54 | + if (use_rope_) { | |
| 55 | + q = ops::rope(q, n_heads_, rope_theta_, 0); | |
| 56 | + k = ops::rope(k, n_kv_heads_, rope_theta_, 0); | |
| 57 | + } | |
| 58 | + | |
| 59 | + const float scale = 1.0f / std::sqrt(float(head_dim_)); | |
| 60 | + Var o = ops::attention(q, k, v, n_heads_, n_kv_heads_, /*causal=*/true, scale); | |
| 61 | + return wo_->forward(o.reshaped({B * T, C})).reshaped({B, T, C}); | |
| 62 | + } | |
| 63 | + | |
| 64 | +private: | |
| 65 | + int64_t n_heads_, n_kv_heads_, head_dim_; | |
| 66 | + bool use_rope_; | |
| 67 | + float rope_theta_; | |
| 68 | + std::unique_ptr<Linear> wq_, wk_, wv_, wo_; | |
| 69 | +}; | |
| 70 | + | |
| 71 | +} // namespace forge::nn | |
added
src/nn/config.h
+133 −0
@@ -0,0 +1,133 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#pragma once | |
| 3 | + | |
| 4 | +#include <cstdint> | |
| 5 | +#include <fstream> | |
| 6 | +#include <stdexcept> | |
| 7 | +#include <string> | |
| 8 | + | |
| 9 | +#include <nlohmann/json.hpp> | |
| 10 | + | |
| 11 | +namespace forge { | |
| 12 | + | |
| 13 | +// The entire architecture comes from here — no hardcoded model sizes | |
| 14 | +// anywhere else in the codebase. | |
| 15 | +struct ModelConfig { | |
| 16 | + std::string name = "model"; | |
| 17 | + int64_t n_layers = 6; | |
| 18 | + int64_t d_model = 384; | |
| 19 | + int64_t n_heads = 6; | |
| 20 | + int64_t n_kv_heads = 6; // < n_heads => GQA | |
| 21 | + int64_t d_ff = 1024; | |
| 22 | + int64_t vocab_size = 4096; | |
| 23 | + int64_t context_length = 512; | |
| 24 | + bool tied_embeddings = true; | |
| 25 | + bool use_rope = true; // false => learned positional embeddings | |
| 26 | + float rope_theta = 10000.0f; | |
| 27 | + std::string norm = "rmsnorm"; // "rmsnorm" | "layernorm" | |
| 28 | + float norm_eps = 1e-6f; | |
| 29 | + std::string activation = "swiglu"; // "swiglu" | "gelu" | |
| 30 | + float dropout = 0.0f; | |
| 31 | + | |
| 32 | + int64_t head_dim() const { return d_model / n_heads; } | |
| 33 | + | |
| 34 | + // Total parameter count (SwiGLU MLP has three mats, GELU has two). | |
| 35 | + int64_t num_params() const { | |
| 36 | + const int64_t hd = head_dim(); | |
| 37 | + const int64_t attn = d_model * d_model // wq | |
| 38 | + + 2 * d_model * n_kv_heads * hd // wk, wv | |
| 39 | + + d_model * d_model; // wo | |
| 40 | + const int64_t mlp = (activation == "swiglu") | |
| 41 | + ? 3 * d_model * d_ff | |
| 42 | + : 2 * d_model * d_ff; | |
| 43 | + const int64_t norms = (norm == "layernorm" ? 2 : 1) * d_model * (2 * n_layers + 1); | |
| 44 | + int64_t total = n_layers * (attn + mlp) + norms + vocab_size * d_model; | |
| 45 | + if (!tied_embeddings) total += vocab_size * d_model; | |
| 46 | + if (!use_rope) total += context_length * d_model; | |
| 47 | + return total; | |
| 48 | + } | |
| 49 | +}; | |
| 50 | + | |
| 51 | +struct TrainConfig { | |
| 52 | + float lr = 6e-4f; | |
| 53 | + float min_lr_ratio = 0.1f; // min_lr = lr * ratio | |
| 54 | + int64_t warmup_steps = 2000; | |
| 55 | + int64_t max_steps = 100000; | |
| 56 | + float beta1 = 0.9f; | |
| 57 | + float beta2 = 0.95f; | |
| 58 | + float eps = 1e-8f; | |
| 59 | + float weight_decay = 0.1f; // dim>=2 params only | |
| 60 | + float grad_clip = 1.0f; // 0 disables | |
| 61 | + int64_t batch_size = 32; // sequences per micro-batch | |
| 62 | + int64_t grad_accum_steps = 1; | |
| 63 | + std::string precision = "f32"; // "f32" | "f16" | "bf16" (compute dtype) | |
| 64 | + int64_t checkpoint_every = 1000; | |
| 65 | + int64_t eval_every = 500; | |
| 66 | + int64_t eval_batches = 20; | |
| 67 | + uint64_t seed = 1337; | |
| 68 | + bool deterministic = false; | |
| 69 | +}; | |
| 70 | + | |
| 71 | +struct Config { | |
| 72 | + ModelConfig model; | |
| 73 | + TrainConfig train; | |
| 74 | +}; | |
| 75 | + | |
| 76 | +inline void from_json(const nlohmann::json& j, ModelConfig& c) { | |
| 77 | + c.name = j.value("name", c.name); | |
| 78 | + c.n_layers = j.value("n_layers", c.n_layers); | |
| 79 | + c.d_model = j.value("d_model", c.d_model); | |
| 80 | + c.n_heads = j.value("n_heads", c.n_heads); | |
| 81 | + c.n_kv_heads = j.value("n_kv_heads", c.n_heads); | |
| 82 | + c.d_ff = j.value("d_ff", c.d_ff); | |
| 83 | + c.vocab_size = j.value("vocab_size", c.vocab_size); | |
| 84 | + c.context_length = j.value("context_length", c.context_length); | |
| 85 | + c.tied_embeddings = j.value("tied_embeddings", c.tied_embeddings); | |
| 86 | + c.use_rope = j.value("use_rope", c.use_rope); | |
| 87 | + c.rope_theta = j.value("rope_theta", c.rope_theta); | |
| 88 | + c.norm = j.value("norm", c.norm); | |
| 89 | + c.norm_eps = j.value("norm_eps", c.norm_eps); | |
| 90 | + c.activation = j.value("activation", c.activation); | |
| 91 | + c.dropout = j.value("dropout", c.dropout); | |
| 92 | + | |
| 93 | + if (c.d_model % c.n_heads != 0) | |
| 94 | + throw std::runtime_error("config: d_model must be divisible by n_heads"); | |
| 95 | + if (c.n_heads % c.n_kv_heads != 0) | |
| 96 | + throw std::runtime_error("config: n_heads must be divisible by n_kv_heads"); | |
| 97 | + if (c.norm != "rmsnorm" && c.norm != "layernorm") | |
| 98 | + throw std::runtime_error("config: norm must be rmsnorm or layernorm"); | |
| 99 | + if (c.activation != "swiglu" && c.activation != "gelu") | |
| 100 | + throw std::runtime_error("config: activation must be swiglu or gelu"); | |
| 101 | +} | |
| 102 | + | |
| 103 | +inline void from_json(const nlohmann::json& j, TrainConfig& c) { | |
| 104 | + c.lr = j.value("lr", c.lr); | |
| 105 | + c.min_lr_ratio = j.value("min_lr_ratio", c.min_lr_ratio); | |
| 106 | + c.warmup_steps = j.value("warmup_steps", c.warmup_steps); | |
| 107 | + c.max_steps = j.value("max_steps", c.max_steps); | |
| 108 | + c.beta1 = j.value("beta1", c.beta1); | |
| 109 | + c.beta2 = j.value("beta2", c.beta2); | |
| 110 | + c.eps = j.value("eps", c.eps); | |
| 111 | + c.weight_decay = j.value("weight_decay", c.weight_decay); | |
| 112 | + c.grad_clip = j.value("grad_clip", c.grad_clip); | |
| 113 | + c.batch_size = j.value("batch_size", c.batch_size); | |
| 114 | + c.grad_accum_steps = j.value("grad_accum_steps", c.grad_accum_steps); | |
| 115 | + c.precision = j.value("precision", c.precision); | |
| 116 | + c.checkpoint_every = j.value("checkpoint_every", c.checkpoint_every); | |
| 117 | + c.eval_every = j.value("eval_every", c.eval_every); | |
| 118 | + c.eval_batches = j.value("eval_batches", c.eval_batches); | |
| 119 | + c.seed = j.value("seed", c.seed); | |
| 120 | + c.deterministic = j.value("deterministic", c.deterministic); | |
| 121 | +} | |
| 122 | + | |
| 123 | +inline Config load_config(const std::string& path) { | |
| 124 | + std::ifstream in(path); | |
| 125 | + if (!in) throw std::runtime_error("config: cannot open " + path); | |
| 126 | + nlohmann::json j = nlohmann::json::parse(in); | |
| 127 | + Config cfg; | |
| 128 | + if (j.contains("model")) j.at("model").get_to(cfg.model); | |
| 129 | + if (j.contains("train")) j.at("train").get_to(cfg.train); | |
| 130 | + return cfg; | |
| 131 | +} | |
| 132 | + | |
| 133 | +} // namespace forge | |
added
src/nn/embedding.h
+25 −0
@@ -0,0 +1,25 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#pragma once | |
| 3 | + | |
| 4 | +#include "nn/module.h" | |
| 5 | +#include "ops/ops.h" | |
| 6 | + | |
| 7 | +namespace forge::nn { | |
| 8 | + | |
| 9 | +// Token (or positional) embedding table [V, C]. | |
| 10 | +class Embedding : public Module { | |
| 11 | +public: | |
| 12 | + Embedding(int64_t num_embeddings, int64_t dim, float weight_std, std::mt19937_64& rng) { | |
| 13 | + weight_ = register_param("weight", normal_init({num_embeddings, dim}, weight_std, rng)); | |
| 14 | + } | |
| 15 | + | |
| 16 | + // ids: [B, T] u16/i32 → [B, T, C] | |
| 17 | + Var forward(const Tensor& ids) const { return ops::embedding(weight_, ids); } | |
| 18 | + | |
| 19 | + const Var& weight() const { return weight_; } | |
| 20 | + | |
| 21 | +private: | |
| 22 | + Var weight_; | |
| 23 | +}; | |
| 24 | + | |
| 25 | +} // namespace forge::nn | |
added
src/nn/linear.cpp
+15 −0
@@ -0,0 +1,15 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#include "nn/linear.h" | |
| 3 | + | |
| 4 | +#include "ops/ops.h" | |
| 5 | + | |
| 6 | +namespace forge::nn { | |
| 7 | + | |
| 8 | +Var Linear::forward(const Var& x) const { | |
| 9 | + Var w = mask_.defined() ? ops::mul(weight_, mask_) : weight_; | |
| 10 | + Var y = ops::matmul(x, w, /*transpose_a=*/false, /*transpose_b=*/true); | |
| 11 | + if (bias_.defined()) y = ops::add_bias(y, bias_); | |
| 12 | + return y; | |
| 13 | +} | |
| 14 | + | |
| 15 | +} // namespace forge::nn | |
added
src/nn/linear.h
+43 −0
@@ -0,0 +1,43 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#pragma once | |
| 3 | + | |
| 4 | +#include "nn/module.h" | |
| 5 | + | |
| 6 | +namespace forge::nn { | |
| 7 | + | |
| 8 | +// y = x ⋅ Wᵀ (+ b). Weight stored [out, in] (PyTorch convention), x is | |
| 9 | +// [N, in] 2-D — callers flatten [B,T,C] with Var::reshaped. | |
| 10 | +// | |
| 11 | +// Research seams (see CLAUDE.md): | |
| 12 | +// - set_mask(): optional binary mask multiplied into the weight each | |
| 13 | +// forward (lottery-ticket / pruning). Gradients of masked weights are | |
| 14 | +// zeroed by the same multiply. | |
| 15 | +// - forward() is virtual so quantized-weight variants (BitNet-style) can | |
| 16 | +// override without touching call sites. | |
| 17 | +class Linear : public Module { | |
| 18 | +public: | |
| 19 | + Linear(int64_t in_features, int64_t out_features, bool has_bias, float weight_std, | |
| 20 | + std::mt19937_64& rng) { | |
| 21 | + weight_ = register_param("weight", | |
| 22 | + normal_init({out_features, in_features}, weight_std, rng)); | |
| 23 | + if (has_bias) bias_ = register_param("bias", Tensor::zeros({out_features})); | |
| 24 | + } | |
| 25 | + | |
| 26 | + virtual ~Linear() = default; | |
| 27 | + | |
| 28 | + virtual Var forward(const Var& x) const; | |
| 29 | + | |
| 30 | + // mask: [out, in], 0/1 f32. Undefined tensor clears the mask. | |
| 31 | + void set_mask(Tensor mask) { | |
| 32 | + mask_ = mask.defined() ? Var(std::move(mask), /*requires_grad=*/false) : Var(); | |
| 33 | + } | |
| 34 | + | |
| 35 | + const Var& weight() const { return weight_; } | |
| 36 | + | |
| 37 | +protected: | |
| 38 | + Var weight_; | |
| 39 | + Var bias_; // undefined if bias disabled | |
| 40 | + Var mask_; // undefined if no mask | |
| 41 | +}; | |
| 42 | + | |
| 43 | +} // namespace forge::nn | |
added
src/nn/mlp.h
+44 −0
@@ -0,0 +1,44 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#pragma once | |
| 3 | + | |
| 4 | +#include "nn/config.h" | |
| 5 | +#include "nn/linear.h" | |
| 6 | +#include "ops/ops.h" | |
| 7 | + | |
| 8 | +#include <memory> | |
| 9 | + | |
| 10 | +namespace forge::nn { | |
| 11 | + | |
| 12 | +// SwiGLU: w2( silu(x w1) ⊙ (x w3) ) | GELU: proj( gelu(fc(x)) ) | |
| 13 | +class MLP : public Module { | |
| 14 | +public: | |
| 15 | + MLP(const ModelConfig& cfg, float proj_std, std::mt19937_64& rng) | |
| 16 | + : swiglu_(cfg.activation == "swiglu") { | |
| 17 | + const float std = 0.02f; | |
| 18 | + const int64_t C = cfg.d_model, F = cfg.d_ff; | |
| 19 | + w1_ = std::make_unique<Linear>(C, F, false, std, rng); | |
| 20 | + if (swiglu_) { | |
| 21 | + w3_ = std::make_unique<Linear>(C, F, false, std, rng); | |
| 22 | + } | |
| 23 | + w2_ = std::make_unique<Linear>(F, C, false, proj_std, rng); // residual projection | |
| 24 | + absorb("w1", *w1_); | |
| 25 | + if (w3_) absorb("w3", *w3_); | |
| 26 | + absorb("w2", *w2_); | |
| 27 | + } | |
| 28 | + | |
| 29 | + // x: [N, C] 2-D | |
| 30 | + Var forward(const Var& x) const { | |
| 31 | + if (swiglu_) { | |
| 32 | + Var gate = ops::silu(w1_->forward(x)); | |
| 33 | + Var up = w3_->forward(x); | |
| 34 | + return w2_->forward(ops::mul(gate, up)); | |
| 35 | + } | |
| 36 | + return w2_->forward(ops::gelu(w1_->forward(x))); | |
| 37 | + } | |
| 38 | + | |
| 39 | +private: | |
| 40 | + bool swiglu_; | |
| 41 | + std::unique_ptr<Linear> w1_, w2_, w3_; | |
| 42 | +}; | |
| 43 | + | |
| 44 | +} // namespace forge::nn | |
added
src/nn/module.h
+58 −0
@@ -0,0 +1,58 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#pragma once | |
| 3 | + | |
| 4 | +#include "core/autograd.h" | |
| 5 | +#include "ops/cpu/cpu_ops.h" | |
| 6 | + | |
| 7 | +#include <random> | |
| 8 | +#include <string> | |
| 9 | +#include <utility> | |
| 10 | +#include <vector> | |
| 11 | + | |
| 12 | +namespace forge::nn { | |
| 13 | + | |
| 14 | +// Base module: owns nothing but the parameter registry. Children register | |
| 15 | +// their parameters at construction; parents absorb them with a name prefix. | |
| 16 | +// Tied parameters may appear under several names — consumers that update | |
| 17 | +// parameters (optimizer, checkpoint) must dedupe by Var::id(). | |
| 18 | +class Module { | |
| 19 | +public: | |
| 20 | + virtual ~Module() = default; | |
| 21 | + | |
| 22 | + const std::vector<std::pair<std::string, Var>>& named_parameters() const { | |
| 23 | + return params_; | |
| 24 | + } | |
| 25 | + | |
| 26 | + void zero_grad() const { | |
| 27 | + for (const auto& [name, p] : params_) p.zero_grad(); | |
| 28 | + } | |
| 29 | + | |
| 30 | +protected: | |
| 31 | + Var register_param(const std::string& name, Tensor init) { | |
| 32 | + Var p(std::move(init), /*requires_grad=*/true); | |
| 33 | + params_.emplace_back(name, p); | |
| 34 | + return p; | |
| 35 | + } | |
| 36 | + | |
| 37 | + // Register an existing Var under a new name (weight tying). | |
| 38 | + Var register_param(const std::string& name, const Var& shared) { | |
| 39 | + params_.emplace_back(name, shared); | |
| 40 | + return shared; | |
| 41 | + } | |
| 42 | + | |
| 43 | + void absorb(const std::string& prefix, const Module& child) { | |
| 44 | + for (const auto& [name, p] : child.named_parameters()) | |
| 45 | + params_.emplace_back(prefix + "." + name, p); | |
| 46 | + } | |
| 47 | + | |
| 48 | + std::vector<std::pair<std::string, Var>> params_; | |
| 49 | +}; | |
| 50 | + | |
| 51 | +// init helper shared by all modules | |
| 52 | +inline Tensor normal_init(std::vector<int64_t> shape, float stddev, std::mt19937_64& rng) { | |
| 53 | + Tensor t = Tensor::empty(std::move(shape)); | |
| 54 | + cpu::fill_normal(t, 0.0f, stddev, rng); | |
| 55 | + return t; | |
| 56 | +} | |
| 57 | + | |
| 58 | +} // namespace forge::nn | |
added
src/nn/transformer.h
+130 −0
@@ -0,0 +1,130 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#pragma once | |
| 3 | + | |
| 4 | +#include "nn/attention.h" | |
| 5 | +#include "nn/config.h" | |
| 6 | +#include "nn/embedding.h" | |
| 7 | +#include "nn/mlp.h" | |
| 8 | + | |
| 9 | +#include <cmath> | |
| 10 | +#include <memory> | |
| 11 | +#include <vector> | |
| 12 | + | |
| 13 | +namespace forge::nn { | |
| 14 | + | |
| 15 | +// Norm wrapper so blocks don't branch on the norm flavour. | |
| 16 | +class Norm : public Module { | |
| 17 | +public: | |
| 18 | + Norm(const ModelConfig& cfg) : layernorm_(cfg.norm == "layernorm"), eps_(cfg.norm_eps) { | |
| 19 | + w_ = register_param("weight", Tensor::full({cfg.d_model}, 1.0f)); | |
| 20 | + if (layernorm_) b_ = register_param("bias", Tensor::zeros({cfg.d_model})); | |
| 21 | + } | |
| 22 | + | |
| 23 | + Var forward(const Var& x) const { | |
| 24 | + return layernorm_ ? ops::layernorm(x, w_, b_, eps_) : ops::rmsnorm(x, w_, eps_); | |
| 25 | + } | |
| 26 | + | |
| 27 | +private: | |
| 28 | + bool layernorm_; | |
| 29 | + float eps_; | |
| 30 | + Var w_, b_; | |
| 31 | +}; | |
| 32 | + | |
| 33 | +// Pre-norm block: x += attn(norm1(x)); x += mlp(norm2(x)) | |
| 34 | +class TransformerBlock : public Module { | |
| 35 | +public: | |
| 36 | + TransformerBlock(const ModelConfig& cfg, float proj_std, std::mt19937_64& rng) | |
| 37 | + : norm1_(std::make_unique<Norm>(cfg)), | |
| 38 | + attn_(std::make_unique<CausalSelfAttention>(cfg, proj_std, rng)), | |
| 39 | + norm2_(std::make_unique<Norm>(cfg)), | |
| 40 | + mlp_(std::make_unique<MLP>(cfg, proj_std, rng)) { | |
| 41 | + absorb("norm1", *norm1_); | |
| 42 | + absorb("attn", *attn_); | |
| 43 | + absorb("norm2", *norm2_); | |
| 44 | + absorb("mlp", *mlp_); | |
| 45 | + } | |
| 46 | + | |
| 47 | + Var forward(const Var& x) const { | |
| 48 | + const int64_t B = x.value().size(0), T = x.value().size(1), C = x.value().size(2); | |
| 49 | + Var h = ops::add(x, attn_->forward(norm1_->forward(x))); | |
| 50 | + Var m = mlp_->forward(norm2_->forward(h).reshaped({B * T, C})); | |
| 51 | + return ops::add(h, m.reshaped({B, T, C})); | |
| 52 | + } | |
| 53 | + | |
| 54 | +private: | |
| 55 | + std::unique_ptr<Norm> norm1_; | |
| 56 | + std::unique_ptr<AttentionBase> attn_; | |
| 57 | + std::unique_ptr<Norm> norm2_; | |
| 58 | + std::unique_ptr<MLP> mlp_; | |
| 59 | +}; | |
| 60 | + | |
| 61 | +// Decoder-only transformer, entirely shaped by ModelConfig. | |
| 62 | +class Transformer : public Module { | |
| 63 | +public: | |
| 64 | + Transformer(const ModelConfig& cfg, uint64_t seed) : cfg_(cfg) { | |
| 65 | + std::mt19937_64 rng(seed); | |
| 66 | + const float std = 0.02f; | |
| 67 | + // Residual projections scaled by 1/sqrt(2L): one attn + one mlp | |
| 68 | + // residual add per layer (nanoGPT). | |
| 69 | + const float proj_std = std / std::sqrt(2.0f * float(cfg.n_layers)); | |
| 70 | + | |
| 71 | + tok_emb_ = std::make_unique<Embedding>(cfg.vocab_size, cfg.d_model, std, rng); | |
| 72 | + absorb("tok_emb", *tok_emb_); | |
| 73 | + if (!cfg.use_rope) { | |
| 74 | + pos_emb_ = std::make_unique<Embedding>(cfg.context_length, cfg.d_model, std, rng); | |
| 75 | + absorb("pos_emb", *pos_emb_); | |
| 76 | + } | |
| 77 | + for (int64_t i = 0; i < cfg.n_layers; ++i) { | |
| 78 | + blocks_.push_back(std::make_unique<TransformerBlock>(cfg, proj_std, rng)); | |
| 79 | + absorb("blocks." + std::to_string(i), *blocks_.back()); | |
| 80 | + } | |
| 81 | + final_norm_ = std::make_unique<Norm>(cfg); | |
| 82 | + absorb("final_norm", *final_norm_); | |
| 83 | + | |
| 84 | + if (cfg.tied_embeddings) { | |
| 85 | + lm_head_weight_ = register_param("lm_head.weight", tok_emb_->weight()); | |
| 86 | + } else { | |
| 87 | + lm_head_weight_ = register_param( | |
| 88 | + "lm_head.weight", normal_init({cfg.vocab_size, cfg.d_model}, std, rng)); | |
| 89 | + } | |
| 90 | + } | |
| 91 | + | |
| 92 | + // ids: [B, T] u16/i32 → logits [B*T, V] | |
| 93 | + Var forward(const Tensor& ids) const { | |
| 94 | + const int64_t B = ids.shape()[0], T = ids.shape()[1]; | |
| 95 | + Var x = tok_emb_->forward(ids); | |
| 96 | + if (pos_emb_) { | |
| 97 | + Tensor pos = Tensor::empty({1, T}, DType::I32); | |
| 98 | + for (int64_t t = 0; t < T; ++t) pos.data<int32_t>()[t] = int32_t(t); | |
| 99 | + Var p = pos_emb_->forward(pos); // [1, T, C] | |
| 100 | + // Broadcast over batch: viewed as [B, T*C] rows + a [T*C] "bias", | |
| 101 | + // add_bias sums the positional grad over the batch — exactly right. | |
| 102 | + Var xb = x.reshaped({B, T * cfg_.d_model}); | |
| 103 | + Var pflat = p.reshaped({T * cfg_.d_model}); | |
| 104 | + x = ops::add_bias(xb, pflat).reshaped({B, T, cfg_.d_model}); | |
| 105 | + } | |
| 106 | + for (const auto& blk : blocks_) x = blk->forward(x); | |
| 107 | + x = final_norm_->forward(x); | |
| 108 | + Var x2d = x.reshaped({B * T, cfg_.d_model}); | |
| 109 | + return ops::matmul(x2d, lm_head_weight_, false, true); // [B*T, V] | |
| 110 | + } | |
| 111 | + | |
| 112 | + // targets: [B, T] with ignore_index=-1 → scalar mean CE loss | |
| 113 | + Var loss(const Tensor& ids, const Tensor& targets) const { | |
| 114 | + Var logits = forward(ids); | |
| 115 | + Tensor tflat = targets; | |
| 116 | + return ops::cross_entropy(logits, tflat.view({targets.numel()})); | |
| 117 | + } | |
| 118 | + | |
| 119 | + const ModelConfig& config() const { return cfg_; } | |
| 120 | + | |
| 121 | +private: | |
| 122 | + ModelConfig cfg_; | |
| 123 | + std::unique_ptr<Embedding> tok_emb_; | |
| 124 | + std::unique_ptr<Embedding> pos_emb_; // null when RoPE | |
| 125 | + std::vector<std::unique_ptr<TransformerBlock>> blocks_; | |
| 126 | + std::unique_ptr<Norm> final_norm_; | |
| 127 | + Var lm_head_weight_; | |
| 128 | +}; | |
| 129 | + | |
| 130 | +} // namespace forge::nn | |
added
src/ops/cpu/cpu_ops.cpp
+555 −0
@@ -0,0 +1,555 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#include "ops/cpu/cpu_ops.h" | |
| 3 | + | |
| 4 | +#include <dispatch/dispatch.h> | |
| 5 | + | |
| 6 | +#include <cassert> | |
| 7 | +#include <cmath> | |
| 8 | +#include <cstdio> | |
| 9 | +#include <cstdlib> | |
| 10 | + | |
| 11 | +namespace forge::cpu { | |
| 12 | + | |
| 13 | +namespace { | |
| 14 | + | |
| 15 | +void check(bool cond, const char* msg) { | |
| 16 | + if (!cond) { | |
| 17 | + std::fprintf(stderr, "forge/cpu: %s\n", msg); | |
| 18 | + std::abort(); | |
| 19 | + } | |
| 20 | +} | |
| 21 | + | |
| 22 | +// Row-parallel helper (GCD). Serial below the threshold so tiny test | |
| 23 | +// tensors don't pay dispatch overhead. | |
| 24 | +template <typename F> | |
| 25 | +void parallel_rows(int64_t n, F&& body) { | |
| 26 | + if (n < 32) { | |
| 27 | + for (int64_t i = 0; i < n; ++i) body(i); | |
| 28 | + } else { | |
| 29 | + dispatch_apply(size_t(n), DISPATCH_APPLY_AUTO, | |
| 30 | + ^(size_t i) { body(int64_t(i)); }); | |
| 31 | + } | |
| 32 | +} | |
| 33 | + | |
| 34 | +int32_t token_at(const Tensor& ids, int64_t i) { | |
| 35 | + return ids.dtype() == DType::I32 ? ids.data<int32_t>()[i] | |
| 36 | + : int32_t(ids.data<uint16_t>()[i]); | |
| 37 | +} | |
| 38 | + | |
| 39 | +} // namespace | |
| 40 | + | |
| 41 | +// ---- init ------------------------------------------------------------------- | |
| 42 | + | |
| 43 | +void fill_normal(Tensor& t, float mean, float stddev, std::mt19937_64& rng) { | |
| 44 | + std::normal_distribution<float> dist(mean, stddev); | |
| 45 | + float* p = t.data<float>(); | |
| 46 | + for (int64_t i = 0; i < t.numel(); ++i) p[i] = dist(rng); | |
| 47 | +} | |
| 48 | + | |
| 49 | +void fill_uniform_int(Tensor& t, int64_t low, int64_t high, std::mt19937_64& rng) { | |
| 50 | + std::uniform_int_distribution<int64_t> dist(low, high - 1); | |
| 51 | + for (int64_t i = 0; i < t.numel(); ++i) t.set_item(i, float(dist(rng))); | |
| 52 | +} | |
| 53 | + | |
| 54 | +// ---- matmul ----------------------------------------------------------------- | |
| 55 | + | |
| 56 | +void matmul(const Tensor& a, const Tensor& b, Tensor& c, | |
| 57 | + bool transpose_a, bool transpose_b, bool accumulate) { | |
| 58 | + check(a.dtype() == DType::F32 && b.dtype() == DType::F32 && c.dtype() == DType::F32, | |
| 59 | + "matmul: f32 only"); | |
| 60 | + check(a.ndim() == 2 && b.ndim() == 2 && c.ndim() == 2, "matmul: 2-D only"); | |
| 61 | + check(a.is_contiguous() && b.is_contiguous() && c.is_contiguous(), | |
| 62 | + "matmul: contiguous only"); | |
| 63 | + | |
| 64 | + const int64_t M = transpose_a ? a.size(1) : a.size(0); | |
| 65 | + const int64_t K = transpose_a ? a.size(0) : a.size(1); | |
| 66 | + const int64_t Kb = transpose_b ? b.size(1) : b.size(0); | |
| 67 | + const int64_t N = transpose_b ? b.size(0) : b.size(1); | |
| 68 | + check(K == Kb, "matmul: inner dims mismatch"); | |
| 69 | + check(c.size(0) == M && c.size(1) == N, "matmul: output shape mismatch"); | |
| 70 | + | |
| 71 | + const float* A = a.data<float>(); | |
| 72 | + const float* B = b.data<float>(); | |
| 73 | + float* C = c.data<float>(); | |
| 74 | + const int64_t lda = a.size(1); | |
| 75 | + const int64_t ldb = b.size(1); | |
| 76 | + | |
| 77 | + // i-k-j order streams contiguous rows of B and C in the common case. | |
| 78 | + parallel_rows(M, [&](int64_t i) { | |
| 79 | + float* crow = C + i * N; | |
| 80 | + if (!accumulate) | |
| 81 | + for (int64_t j = 0; j < N; ++j) crow[j] = 0.0f; | |
| 82 | + for (int64_t k = 0; k < K; ++k) { | |
| 83 | + const float aik = transpose_a ? A[k * lda + i] : A[i * lda + k]; | |
| 84 | + if (!transpose_b) { | |
| 85 | + const float* brow = B + k * ldb; | |
| 86 | + for (int64_t j = 0; j < N; ++j) crow[j] += aik * brow[j]; | |
| 87 | + } else { | |
| 88 | + for (int64_t j = 0; j < N; ++j) crow[j] += aik * B[j * ldb + k]; | |
| 89 | + } | |
| 90 | + } | |
| 91 | + }); | |
| 92 | +} | |
| 93 | + | |
| 94 | +// ---- elementwise ------------------------------------------------------------ | |
| 95 | + | |
| 96 | +void add(const Tensor& a, const Tensor& b, Tensor& out) { | |
| 97 | + check(a.numel() == b.numel() && a.numel() == out.numel(), "add: numel mismatch"); | |
| 98 | + const float* pa = a.data<float>(); | |
| 99 | + const float* pb = b.data<float>(); | |
| 100 | + float* po = out.data<float>(); | |
| 101 | + for (int64_t i = 0; i < a.numel(); ++i) po[i] = pa[i] + pb[i]; | |
| 102 | +} | |
| 103 | + | |
| 104 | +void mul(const Tensor& a, const Tensor& b, Tensor& out) { | |
| 105 | + check(a.numel() == b.numel() && a.numel() == out.numel(), "mul: numel mismatch"); | |
| 106 | + const float* pa = a.data<float>(); | |
| 107 | + const float* pb = b.data<float>(); | |
| 108 | + float* po = out.data<float>(); | |
| 109 | + for (int64_t i = 0; i < a.numel(); ++i) po[i] = pa[i] * pb[i]; | |
| 110 | +} | |
| 111 | + | |
| 112 | +void scale(const Tensor& a, float s, Tensor& out) { | |
| 113 | + check(a.numel() == out.numel(), "scale: numel mismatch"); | |
| 114 | + const float* pa = a.data<float>(); | |
| 115 | + float* po = out.data<float>(); | |
| 116 | + for (int64_t i = 0; i < a.numel(); ++i) po[i] = pa[i] * s; | |
| 117 | +} | |
| 118 | + | |
| 119 | +void add_bias(const Tensor& x, const Tensor& bias, Tensor& out) { | |
| 120 | + const int64_t C = bias.numel(); | |
| 121 | + const int64_t N = x.numel() / C; | |
| 122 | + const float* px = x.data<float>(); | |
| 123 | + const float* pb = bias.data<float>(); | |
| 124 | + float* po = out.data<float>(); | |
| 125 | + for (int64_t i = 0; i < N; ++i) | |
| 126 | + for (int64_t j = 0; j < C; ++j) po[i * C + j] = px[i * C + j] + pb[j]; | |
| 127 | +} | |
| 128 | + | |
| 129 | +void add_bias_backward(const Tensor& dout, Tensor& dbias) { | |
| 130 | + const int64_t C = dbias.numel(); | |
| 131 | + const int64_t N = dout.numel() / C; | |
| 132 | + const float* pd = dout.data<float>(); | |
| 133 | + float* pb = dbias.data<float>(); | |
| 134 | + for (int64_t i = 0; i < N; ++i) | |
| 135 | + for (int64_t j = 0; j < C; ++j) pb[j] += pd[i * C + j]; | |
| 136 | +} | |
| 137 | + | |
| 138 | +void silu(const Tensor& x, Tensor& out) { | |
| 139 | + const float* px = x.data<float>(); | |
| 140 | + float* po = out.data<float>(); | |
| 141 | + for (int64_t i = 0; i < x.numel(); ++i) { | |
| 142 | + const float v = px[i]; | |
| 143 | + po[i] = v / (1.0f + std::exp(-v)); | |
| 144 | + } | |
| 145 | +} | |
| 146 | + | |
| 147 | +void silu_backward(const Tensor& x, const Tensor& dout, Tensor& dx) { | |
| 148 | + const float* px = x.data<float>(); | |
| 149 | + const float* pd = dout.data<float>(); | |
| 150 | + float* pdx = dx.data<float>(); | |
| 151 | + for (int64_t i = 0; i < x.numel(); ++i) { | |
| 152 | + const float v = px[i]; | |
| 153 | + const float sig = 1.0f / (1.0f + std::exp(-v)); | |
| 154 | + pdx[i] += pd[i] * sig * (1.0f + v * (1.0f - sig)); | |
| 155 | + } | |
| 156 | +} | |
| 157 | + | |
| 158 | +void gelu(const Tensor& x, Tensor& out) { | |
| 159 | + constexpr float k = 0.7978845608028654f; // sqrt(2/pi) | |
| 160 | + const float* px = x.data<float>(); | |
| 161 | + float* po = out.data<float>(); | |
| 162 | + for (int64_t i = 0; i < x.numel(); ++i) { | |
| 163 | + const float v = px[i]; | |
| 164 | + po[i] = 0.5f * v * (1.0f + std::tanh(k * (v + 0.044715f * v * v * v))); | |
| 165 | + } | |
| 166 | +} | |
| 167 | + | |
| 168 | +void gelu_backward(const Tensor& x, const Tensor& dout, Tensor& dx) { | |
| 169 | + constexpr float k = 0.7978845608028654f; | |
| 170 | + const float* px = x.data<float>(); | |
| 171 | + const float* pd = dout.data<float>(); | |
| 172 | + float* pdx = dx.data<float>(); | |
| 173 | + for (int64_t i = 0; i < x.numel(); ++i) { | |
| 174 | + const float v = px[i]; | |
| 175 | + const float u = k * (v + 0.044715f * v * v * v); | |
| 176 | + const float t = std::tanh(u); | |
| 177 | + const float du = k * (1.0f + 3.0f * 0.044715f * v * v); | |
| 178 | + pdx[i] += pd[i] * (0.5f * (1.0f + t) + 0.5f * v * (1.0f - t * t) * du); | |
| 179 | + } | |
| 180 | +} | |
| 181 | + | |
| 182 | +// ---- norms -------------------------------------------------------------------- | |
| 183 | + | |
| 184 | +void rmsnorm(const Tensor& x, const Tensor& w, float eps, Tensor& out) { | |
| 185 | + const int64_t C = w.numel(); | |
| 186 | + const int64_t N = x.numel() / C; | |
| 187 | + const float* pw = w.data<float>(); | |
| 188 | + const float* px = x.data<float>(); | |
| 189 | + float* po = out.data<float>(); | |
| 190 | + parallel_rows(N, [&](int64_t i) { | |
| 191 | + const float* row = px + i * C; | |
| 192 | + float ss = 0.0f; | |
| 193 | + for (int64_t j = 0; j < C; ++j) ss += row[j] * row[j]; | |
| 194 | + const float inv_rms = 1.0f / std::sqrt(ss / float(C) + eps); | |
| 195 | + for (int64_t j = 0; j < C; ++j) po[i * C + j] = pw[j] * row[j] * inv_rms; | |
| 196 | + }); | |
| 197 | +} | |
| 198 | + | |
| 199 | +void rmsnorm_backward(const Tensor& x, const Tensor& w, float eps, | |
| 200 | + const Tensor& dout, Tensor& dx, Tensor& dw) { | |
| 201 | + const int64_t C = w.numel(); | |
| 202 | + const int64_t N = x.numel() / C; | |
| 203 | + const float* px = x.data<float>(); | |
| 204 | + const float* pw = w.data<float>(); | |
| 205 | + const float* pd = dout.data<float>(); | |
| 206 | + float* pdx = dx.data<float>(); | |
| 207 | + float* pdw = dw.data<float>(); | |
| 208 | + // dw is a cross-row reduction: keep it serial for determinism. | |
| 209 | + for (int64_t i = 0; i < N; ++i) { | |
| 210 | + const float* row = px + i * C; | |
| 211 | + const float* drow = pd + i * C; | |
| 212 | + float ss = 0.0f; | |
| 213 | + for (int64_t j = 0; j < C; ++j) ss += row[j] * row[j]; | |
| 214 | + const float inv_rms = 1.0f / std::sqrt(ss / float(C) + eps); | |
| 215 | + float dot = 0.0f; // sum_j g_j w_j x_j | |
| 216 | + for (int64_t j = 0; j < C; ++j) dot += drow[j] * pw[j] * row[j]; | |
| 217 | + const float coef = dot * inv_rms * inv_rms * inv_rms / float(C); | |
| 218 | + for (int64_t j = 0; j < C; ++j) { | |
| 219 | + pdx[i * C + j] += drow[j] * pw[j] * inv_rms - row[j] * coef; | |
| 220 | + pdw[j] += drow[j] * row[j] * inv_rms; | |
| 221 | + } | |
| 222 | + } | |
| 223 | +} | |
| 224 | + | |
| 225 | +void layernorm(const Tensor& x, const Tensor& w, const Tensor& b, float eps, Tensor& out) { | |
| 226 | + const int64_t C = w.numel(); | |
| 227 | + const int64_t N = x.numel() / C; | |
| 228 | + const float* px = x.data<float>(); | |
| 229 | + const float* pw = w.data<float>(); | |
| 230 | + const float* pb = b.data<float>(); | |
| 231 | + float* po = out.data<float>(); | |
| 232 | + parallel_rows(N, [&](int64_t i) { | |
| 233 | + const float* row = px + i * C; | |
| 234 | + float mean = 0.0f; | |
| 235 | + for (int64_t j = 0; j < C; ++j) mean += row[j]; | |
| 236 | + mean /= float(C); | |
| 237 | + float var = 0.0f; | |
| 238 | + for (int64_t j = 0; j < C; ++j) var += (row[j] - mean) * (row[j] - mean); | |
| 239 | + var /= float(C); | |
| 240 | + const float inv_std = 1.0f / std::sqrt(var + eps); | |
| 241 | + for (int64_t j = 0; j < C; ++j) | |
| 242 | + po[i * C + j] = pw[j] * (row[j] - mean) * inv_std + pb[j]; | |
| 243 | + }); | |
| 244 | +} | |
| 245 | + | |
| 246 | +void layernorm_backward(const Tensor& x, const Tensor& w, float eps, | |
| 247 | + const Tensor& dout, Tensor& dx, Tensor& dw, Tensor& db) { | |
| 248 | + const int64_t C = w.numel(); | |
| 249 | + const int64_t N = x.numel() / C; | |
| 250 | + const float* px = x.data<float>(); | |
| 251 | + const float* pw = w.data<float>(); | |
| 252 | + const float* pd = dout.data<float>(); | |
| 253 | + float* pdx = dx.data<float>(); | |
| 254 | + float* pdw = dw.data<float>(); | |
| 255 | + float* pdb = db.data<float>(); | |
| 256 | + for (int64_t i = 0; i < N; ++i) { | |
| 257 | + const float* row = px + i * C; | |
| 258 | + const float* drow = pd + i * C; | |
| 259 | + float mean = 0.0f; | |
| 260 | + for (int64_t j = 0; j < C; ++j) mean += row[j]; | |
| 261 | + mean /= float(C); | |
| 262 | + float var = 0.0f; | |
| 263 | + for (int64_t j = 0; j < C; ++j) var += (row[j] - mean) * (row[j] - mean); | |
| 264 | + var /= float(C); | |
| 265 | + const float inv_std = 1.0f / std::sqrt(var + eps); | |
| 266 | + // dxhat = g*w ; dx = inv_std * (dxhat − mean(dxhat) − xhat*mean(dxhat∘xhat)) | |
| 267 | + float mean_dxhat = 0.0f, mean_dxhat_xhat = 0.0f; | |
| 268 | + for (int64_t j = 0; j < C; ++j) { | |
| 269 | + const float xhat = (row[j] - mean) * inv_std; | |
| 270 | + const float dxhat = drow[j] * pw[j]; | |
| 271 | + mean_dxhat += dxhat; | |
| 272 | + mean_dxhat_xhat += dxhat * xhat; | |
| 273 | + } | |
| 274 | + mean_dxhat /= float(C); | |
| 275 | + mean_dxhat_xhat /= float(C); | |
| 276 | + for (int64_t j = 0; j < C; ++j) { | |
| 277 | + const float xhat = (row[j] - mean) * inv_std; | |
| 278 | + pdx[i * C + j] += inv_std * (drow[j] * pw[j] - mean_dxhat - xhat * mean_dxhat_xhat); | |
| 279 | + pdw[j] += drow[j] * xhat; | |
| 280 | + pdb[j] += drow[j]; | |
| 281 | + } | |
| 282 | + } | |
| 283 | +} | |
| 284 | + | |
| 285 | +// ---- softmax ------------------------------------------------------------------ | |
| 286 | + | |
| 287 | +void softmax(const Tensor& x, Tensor& out) { | |
| 288 | + const int64_t C = x.shape().back(); | |
| 289 | + const int64_t N = x.numel() / C; | |
| 290 | + const float* px = x.data<float>(); | |
| 291 | + float* po = out.data<float>(); | |
| 292 | + parallel_rows(N, [&](int64_t i) { | |
| 293 | + const float* row = px + i * C; | |
| 294 | + float m = row[0]; | |
| 295 | + for (int64_t j = 1; j < C; ++j) m = std::max(m, row[j]); | |
| 296 | + float sum = 0.0f; | |
| 297 | + for (int64_t j = 0; j < C; ++j) { | |
| 298 | + const float e = std::exp(row[j] - m); | |
| 299 | + po[i * C + j] = e; | |
| 300 | + sum += e; | |
| 301 | + } | |
| 302 | + const float inv = 1.0f / sum; | |
| 303 | + for (int64_t j = 0; j < C; ++j) po[i * C + j] *= inv; | |
| 304 | + }); | |
| 305 | +} | |
| 306 | + | |
| 307 | +void softmax_backward(const Tensor& p, const Tensor& dout, Tensor& dx) { | |
| 308 | + const int64_t C = p.shape().back(); | |
| 309 | + const int64_t N = p.numel() / C; | |
| 310 | + const float* pp = p.data<float>(); | |
| 311 | + const float* pd = dout.data<float>(); | |
| 312 | + float* pdx = dx.data<float>(); | |
| 313 | + for (int64_t i = 0; i < N; ++i) { | |
| 314 | + const float* prow = pp + i * C; | |
| 315 | + const float* drow = pd + i * C; | |
| 316 | + float dot = 0.0f; | |
| 317 | + for (int64_t j = 0; j < C; ++j) dot += drow[j] * prow[j]; | |
| 318 | + for (int64_t j = 0; j < C; ++j) pdx[i * C + j] += prow[j] * (drow[j] - dot); | |
| 319 | + } | |
| 320 | +} | |
| 321 | + | |
| 322 | +// ---- embedding ------------------------------------------------------------------ | |
| 323 | + | |
| 324 | +void embedding(const Tensor& weight, const Tensor& ids, Tensor& out) { | |
| 325 | + const int64_t C = weight.size(1); | |
| 326 | + const int64_t N = ids.numel(); | |
| 327 | + const float* pw = weight.data<float>(); | |
| 328 | + float* po = out.data<float>(); | |
| 329 | + for (int64_t i = 0; i < N; ++i) { | |
| 330 | + const int32_t tok = token_at(ids, i); | |
| 331 | + const float* src = pw + int64_t(tok) * C; | |
| 332 | + float* dst = po + i * C; | |
| 333 | + for (int64_t j = 0; j < C; ++j) dst[j] = src[j]; | |
| 334 | + } | |
| 335 | +} | |
| 336 | + | |
| 337 | +void embedding_backward(const Tensor& ids, const Tensor& dout, Tensor& dweight) { | |
| 338 | + const int64_t C = dweight.size(1); | |
| 339 | + const int64_t N = ids.numel(); | |
| 340 | + const float* pd = dout.data<float>(); | |
| 341 | + float* pw = dweight.data<float>(); | |
| 342 | + for (int64_t i = 0; i < N; ++i) { | |
| 343 | + const int32_t tok = token_at(ids, i); | |
| 344 | + float* dst = pw + int64_t(tok) * C; | |
| 345 | + const float* src = pd + i * C; | |
| 346 | + for (int64_t j = 0; j < C; ++j) dst[j] += src[j]; | |
| 347 | + } | |
| 348 | +} | |
| 349 | + | |
| 350 | +// ---- RoPE ---------------------------------------------------------------------- | |
| 351 | + | |
| 352 | +namespace { | |
| 353 | +void rope_impl(const float* in, float* out, int64_t B, int64_t T, int64_t H, int64_t hd, | |
| 354 | + float theta, int64_t pos_offset, bool inverse, bool accumulate) { | |
| 355 | + const int64_t C = H * hd; | |
| 356 | + parallel_rows(B * T, [&](int64_t bt) { | |
| 357 | + const int64_t t = bt % T; | |
| 358 | + const float pos = float(t + pos_offset); | |
| 359 | + const float* src = in + bt * C; | |
| 360 | + float* dst = out + bt * C; | |
| 361 | + for (int64_t h = 0; h < H; ++h) { | |
| 362 | + for (int64_t k = 0; k < hd / 2; ++k) { | |
| 363 | + const float freq = std::pow(theta, -2.0f * float(k) / float(hd)); | |
| 364 | + const float angle = pos * freq; | |
| 365 | + const float c = std::cos(angle); | |
| 366 | + const float s = inverse ? -std::sin(angle) : std::sin(angle); | |
| 367 | + const int64_t i0 = h * hd + 2 * k; | |
| 368 | + const float x0 = src[i0], x1 = src[i0 + 1]; | |
| 369 | + const float y0 = x0 * c - x1 * s; | |
| 370 | + const float y1 = x0 * s + x1 * c; | |
| 371 | + if (accumulate) { | |
| 372 | + dst[i0] += y0; | |
| 373 | + dst[i0 + 1] += y1; | |
| 374 | + } else { | |
| 375 | + dst[i0] = y0; | |
| 376 | + dst[i0 + 1] = y1; | |
| 377 | + } | |
| 378 | + } | |
| 379 | + } | |
| 380 | + }); | |
| 381 | +} | |
| 382 | +} // namespace | |
| 383 | + | |
| 384 | +void rope(const Tensor& x, int64_t n_heads, float theta, int64_t pos_offset, Tensor& out) { | |
| 385 | + check(x.ndim() == 3, "rope: expected [B,T,C]"); | |
| 386 | + const int64_t hd = x.size(2) / n_heads; | |
| 387 | + check(hd % 2 == 0, "rope: head_dim must be even"); | |
| 388 | + rope_impl(x.data<float>(), out.data<float>(), x.size(0), x.size(1), n_heads, hd, | |
| 389 | + theta, pos_offset, /*inverse=*/false, /*accumulate=*/false); | |
| 390 | +} | |
| 391 | + | |
| 392 | +void rope_backward(const Tensor& dout, int64_t n_heads, float theta, int64_t pos_offset, | |
| 393 | + Tensor& dx) { | |
| 394 | + const int64_t hd = dout.size(2) / n_heads; | |
| 395 | + rope_impl(dout.data<float>(), dx.data<float>(), dout.size(0), dout.size(1), n_heads, | |
| 396 | + hd, theta, pos_offset, /*inverse=*/true, /*accumulate=*/true); | |
| 397 | +} | |
| 398 | + | |
| 399 | +// ---- attention ------------------------------------------------------------------- | |
| 400 | + | |
| 401 | +void attention(const Tensor& q, const Tensor& k, const Tensor& v, | |
| 402 | + int64_t n_heads, int64_t n_kv_heads, bool causal, float scale, | |
| 403 | + Tensor& out, Tensor* probs_out) { | |
| 404 | + check(q.ndim() == 3 && k.ndim() == 3 && v.ndim() == 3, "attention: expected [B,T,C]"); | |
| 405 | + const int64_t B = q.size(0), T = q.size(1); | |
| 406 | + const int64_t hd = q.size(2) / n_heads; | |
| 407 | + const int64_t rep = n_heads / n_kv_heads; | |
| 408 | + const int64_t Cq = n_heads * hd, Ckv = n_kv_heads * hd; | |
| 409 | + check(probs_out != nullptr, "attention: probs_out required (reference impl)"); | |
| 410 | + check(probs_out->numel() == B * n_heads * T * T, "attention: probs shape"); | |
| 411 | + | |
| 412 | + const float* pq = q.data<float>(); | |
| 413 | + const float* pk = k.data<float>(); | |
| 414 | + const float* pv = v.data<float>(); | |
| 415 | + float* po = out.data<float>(); | |
| 416 | + float* pp = probs_out->data<float>(); | |
| 417 | + | |
| 418 | + parallel_rows(B * n_heads, [&](int64_t bh) { | |
| 419 | + const int64_t b = bh / n_heads; | |
| 420 | + const int64_t h = bh % n_heads; | |
| 421 | + const int64_t hkv = h / rep; | |
| 422 | + float* P = pp + bh * T * T; | |
| 423 | + | |
| 424 | + for (int64_t i = 0; i < T; ++i) { | |
| 425 | + const float* qi = pq + (b * T + i) * Cq + h * hd; | |
| 426 | + const int64_t jmax = causal ? i : T - 1; | |
| 427 | + // scores (masked positions never written; treated as prob 0) | |
| 428 | + float m = -INFINITY; | |
| 429 | + for (int64_t j = 0; j <= jmax; ++j) { | |
| 430 | + const float* kj = pk + (b * T + j) * Ckv + hkv * hd; | |
| 431 | + float s = 0.0f; | |
| 432 | + for (int64_t d = 0; d < hd; ++d) s += qi[d] * kj[d]; | |
| 433 | + s *= scale; | |
| 434 | + P[i * T + j] = s; | |
| 435 | + m = std::max(m, s); | |
| 436 | + } | |
| 437 | + float sum = 0.0f; | |
| 438 | + for (int64_t j = 0; j <= jmax; ++j) { | |
| 439 | + const float e = std::exp(P[i * T + j] - m); | |
| 440 | + P[i * T + j] = e; | |
| 441 | + sum += e; | |
| 442 | + } | |
| 443 | + const float inv = 1.0f / sum; | |
| 444 | + for (int64_t j = 0; j <= jmax; ++j) P[i * T + j] *= inv; | |
| 445 | + for (int64_t j = jmax + 1; j < T; ++j) P[i * T + j] = 0.0f; | |
| 446 | + | |
| 447 | + float* oi = po + (b * T + i) * Cq + h * hd; | |
| 448 | + for (int64_t d = 0; d < hd; ++d) oi[d] = 0.0f; | |
| 449 | + for (int64_t j = 0; j <= jmax; ++j) { | |
| 450 | + const float p = P[i * T + j]; | |
| 451 | + const float* vj = pv + (b * T + j) * Ckv + hkv * hd; | |
| 452 | + for (int64_t d = 0; d < hd; ++d) oi[d] += p * vj[d]; | |
| 453 | + } | |
| 454 | + } | |
| 455 | + }); | |
| 456 | +} | |
| 457 | + | |
| 458 | +void attention_backward(const Tensor& q, const Tensor& k, const Tensor& v, | |
| 459 | + const Tensor& probs, const Tensor& out, const Tensor& dout, | |
| 460 | + int64_t n_heads, int64_t n_kv_heads, float scale, | |
| 461 | + Tensor& dq, Tensor& dk, Tensor& dv) { | |
| 462 | + const int64_t B = q.size(0), T = q.size(1); | |
| 463 | + const int64_t hd = q.size(2) / n_heads; | |
| 464 | + const int64_t rep = n_heads / n_kv_heads; | |
| 465 | + const int64_t Cq = n_heads * hd, Ckv = n_kv_heads * hd; | |
| 466 | + | |
| 467 | + const float* pq = q.data<float>(); | |
| 468 | + const float* pk = k.data<float>(); | |
| 469 | + const float* pv = v.data<float>(); | |
| 470 | + const float* pp = probs.data<float>(); | |
| 471 | + const float* po = out.data<float>(); | |
| 472 | + const float* pd = dout.data<float>(); | |
| 473 | + float* pdq = dq.data<float>(); | |
| 474 | + float* pdk = dk.data<float>(); | |
| 475 | + float* pdv = dv.data<float>(); | |
| 476 | + | |
| 477 | + // Serial over heads: dk/dv rows are shared across q-heads under GQA. | |
| 478 | + for (int64_t bh = 0; bh < B * n_heads; ++bh) { | |
| 479 | + const int64_t b = bh / n_heads; | |
| 480 | + const int64_t h = bh % n_heads; | |
| 481 | + const int64_t hkv = h / rep; | |
| 482 | + const float* P = pp + bh * T * T; | |
| 483 | + | |
| 484 | + for (int64_t i = 0; i < T; ++i) { | |
| 485 | + const float* qi = pq + (b * T + i) * Cq + h * hd; | |
| 486 | + const float* doi = pd + (b * T + i) * Cq + h * hd; | |
| 487 | + float* dqi = pdq + (b * T + i) * Cq + h * hd; | |
| 488 | + | |
| 489 | + // dP_ij = dO_i · V_j ; dS = P ∘ (dP − D_i), and the FA2 identity | |
| 490 | + // gives D_i = Σ_j dP_ij P_ij = dO_i · O_i in one pass. | |
| 491 | + const float* oi = po + (b * T + i) * Cq + h * hd; | |
| 492 | + float row_dot = 0.0f; | |
| 493 | + for (int64_t d = 0; d < hd; ++d) row_dot += doi[d] * oi[d]; | |
| 494 | + for (int64_t j = 0; j < T; ++j) { | |
| 495 | + const float p = P[i * T + j]; | |
| 496 | + if (p == 0.0f) continue; | |
| 497 | + const float* vj = pv + (b * T + j) * Ckv + hkv * hd; | |
| 498 | + const float* kj = pk + (b * T + j) * Ckv + hkv * hd; | |
| 499 | + float* dvj = pdv + (b * T + j) * Ckv + hkv * hd; | |
| 500 | + float* dkj = pdk + (b * T + j) * Ckv + hkv * hd; | |
| 501 | + | |
| 502 | + float dp = 0.0f; | |
| 503 | + for (int64_t d = 0; d < hd; ++d) dp += doi[d] * vj[d]; | |
| 504 | + const float ds = p * (dp - row_dot) * scale; | |
| 505 | + | |
| 506 | + for (int64_t d = 0; d < hd; ++d) { | |
| 507 | + dvj[d] += p * doi[d]; | |
| 508 | + dqi[d] += ds * kj[d]; | |
| 509 | + dkj[d] += ds * qi[d]; | |
| 510 | + } | |
| 511 | + } | |
| 512 | + } | |
| 513 | + } | |
| 514 | +} | |
| 515 | + | |
| 516 | +// ---- cross entropy ------------------------------------------------------------ | |
| 517 | + | |
| 518 | +float cross_entropy(const Tensor& logits, const Tensor& targets, Tensor* dlogits) { | |
| 519 | + const int64_t V = logits.size(1); | |
| 520 | + const int64_t N = logits.size(0); | |
| 521 | + const float* pl = logits.data<float>(); | |
| 522 | + | |
| 523 | + int64_t n_valid = 0; | |
| 524 | + for (int64_t i = 0; i < N; ++i) | |
| 525 | + if (token_at(targets, i) >= 0) ++n_valid; | |
| 526 | + if (n_valid == 0) return 0.0f; | |
| 527 | + | |
| 528 | + double total = 0.0; | |
| 529 | + const float inv_n = 1.0f / float(n_valid); | |
| 530 | + float* pd = dlogits ? dlogits->data<float>() : nullptr; | |
| 531 | + | |
| 532 | + for (int64_t i = 0; i < N; ++i) { | |
| 533 | + const int32_t tgt = token_at(targets, i); | |
| 534 | + if (tgt < 0) continue; | |
| 535 | + const float* row = pl + i * V; | |
| 536 | + float m = row[0]; | |
| 537 | + for (int64_t j = 1; j < V; ++j) m = std::max(m, row[j]); | |
| 538 | + double sum = 0.0; | |
| 539 | + for (int64_t j = 0; j < V; ++j) sum += std::exp(double(row[j] - m)); | |
| 540 | + const double lse = double(m) + std::log(sum); | |
| 541 | + total += lse - double(row[tgt]); | |
| 542 | + | |
| 543 | + if (pd) { | |
| 544 | + const float inv_sum = float(1.0 / sum); | |
| 545 | + float* drow = pd + i * V; | |
| 546 | + for (int64_t j = 0; j < V; ++j) { | |
| 547 | + const float p = std::exp(row[j] - m) * inv_sum; | |
| 548 | + drow[j] += (p - (j == tgt ? 1.0f : 0.0f)) * inv_n; | |
| 549 | + } | |
| 550 | + } | |
| 551 | + } | |
| 552 | + return float(total / double(n_valid)); | |
| 553 | +} | |
| 554 | + | |
| 555 | +} // namespace forge::cpu | |
added
src/ops/cpu/cpu_ops.h
+87 −0
@@ -0,0 +1,87 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#pragma once | |
| 3 | + | |
| 4 | +#include "core/tensor.h" | |
| 5 | + | |
| 6 | +#include <cstdint> | |
| 7 | +#include <random> | |
| 8 | + | |
| 9 | +// CPU reference implementations (forward + backward). These are the ground | |
| 10 | +// truth every Metal kernel is validated against — clarity beats speed here. | |
| 11 | +// All tensors f32 and contiguous unless stated otherwise. Backward | |
| 12 | +// functions ACCUMULATE into their d* outputs (autograd semantics). | |
| 13 | +namespace forge::cpu { | |
| 14 | + | |
| 15 | +// ---- init ----------------------------------------------------------------- | |
| 16 | +void fill_normal(Tensor& t, float mean, float stddev, std::mt19937_64& rng); | |
| 17 | +void fill_uniform_int(Tensor& t, int64_t low, int64_t high, std::mt19937_64& rng); | |
| 18 | + | |
| 19 | +// ---- matmul --------------------------------------------------------------- | |
| 20 | +// C[M,N] = A ⋅ B with optional transposes (A stored [M,K] or [K,M], B stored | |
| 21 | +// [K,N] or [N,K]). accumulate=true adds into C instead of overwriting | |
| 22 | +// (backward passes accumulate into gradients). | |
| 23 | +void matmul(const Tensor& a, const Tensor& b, Tensor& c, | |
| 24 | + bool transpose_a = false, bool transpose_b = false, | |
| 25 | + bool accumulate = false); | |
| 26 | + | |
| 27 | +// ---- elementwise ---------------------------------------------------------- | |
| 28 | +void add(const Tensor& a, const Tensor& b, Tensor& out); | |
| 29 | +void mul(const Tensor& a, const Tensor& b, Tensor& out); | |
| 30 | +void scale(const Tensor& a, float s, Tensor& out); | |
| 31 | +// x: [N, C], bias: [C], out[i,j] = x[i,j] + bias[j] | |
| 32 | +void add_bias(const Tensor& x, const Tensor& bias, Tensor& out); | |
| 33 | +void add_bias_backward(const Tensor& dout, Tensor& dbias); | |
| 34 | + | |
| 35 | +void silu(const Tensor& x, Tensor& out); | |
| 36 | +void silu_backward(const Tensor& x, const Tensor& dout, Tensor& dx); | |
| 37 | +void gelu(const Tensor& x, Tensor& out); // tanh approximation (GPT-2) | |
| 38 | +void gelu_backward(const Tensor& x, const Tensor& dout, Tensor& dx); | |
| 39 | + | |
| 40 | +// ---- norms (row-wise over the last dim) ----------------------------------- | |
| 41 | +// x: [N, C], w: [C] (b: [C] for layernorm), out: [N, C] | |
| 42 | +void rmsnorm(const Tensor& x, const Tensor& w, float eps, Tensor& out); | |
| 43 | +void rmsnorm_backward(const Tensor& x, const Tensor& w, float eps, | |
| 44 | + const Tensor& dout, Tensor& dx, Tensor& dw); | |
| 45 | +void layernorm(const Tensor& x, const Tensor& w, const Tensor& b, float eps, Tensor& out); | |
| 46 | +void layernorm_backward(const Tensor& x, const Tensor& w, float eps, | |
| 47 | + const Tensor& dout, Tensor& dx, Tensor& dw, Tensor& db); | |
| 48 | + | |
| 49 | +// ---- softmax (row-wise over the last dim, max-subtracted) ------------------ | |
| 50 | +void softmax(const Tensor& x, Tensor& out); | |
| 51 | +// dx = P ∘ (dout − rowsum(dout ∘ P)) | |
| 52 | +void softmax_backward(const Tensor& p, const Tensor& dout, Tensor& dx); | |
| 53 | + | |
| 54 | +// ---- embedding ------------------------------------------------------------- | |
| 55 | +// weight: [V, C] f32; ids: [N] i32; out: [N, C] | |
| 56 | +void embedding(const Tensor& weight, const Tensor& ids, Tensor& out); | |
| 57 | +void embedding_backward(const Tensor& ids, const Tensor& dout, Tensor& dweight); | |
| 58 | + | |
| 59 | +// ---- RoPE (interleaved-pairs / GPT-J convention; see RESEARCH.md §7) ------- | |
| 60 | +// x: [B, T, H*hd]; rotates pairs (2k, 2k+1) inside each head. pos_offset | |
| 61 | +// shifts absolute positions (KV-cache generation). | |
| 62 | +void rope(const Tensor& x, int64_t n_heads, float theta, int64_t pos_offset, Tensor& out); | |
| 63 | +// Backward of a rotation is the inverse rotation applied to dout. | |
| 64 | +void rope_backward(const Tensor& dout, int64_t n_heads, float theta, int64_t pos_offset, | |
| 65 | + Tensor& dx); | |
| 66 | + | |
| 67 | +// ---- attention (composed reference: scores → mask → softmax → PV) ---------- | |
| 68 | +// q: [B, T, H*hd], k/v: [B, T, Hkv*hd], out: [B, T, H*hd]. GQA via | |
| 69 | +// kv_head = h / (H / Hkv). If probs_out is non-null it receives the softmax | |
| 70 | +// probabilities [B, H, T, T] (needed for backward). | |
| 71 | +void attention(const Tensor& q, const Tensor& k, const Tensor& v, | |
| 72 | + int64_t n_heads, int64_t n_kv_heads, bool causal, float scale, | |
| 73 | + Tensor& out, Tensor* probs_out); | |
| 74 | +// Uses the FlashAttention-2 identity rowsum(dP ∘ P) == dO_i · O_i, so `out` | |
| 75 | +// (the forward result) is required. | |
| 76 | +void attention_backward(const Tensor& q, const Tensor& k, const Tensor& v, | |
| 77 | + const Tensor& probs, const Tensor& out, const Tensor& dout, | |
| 78 | + int64_t n_heads, int64_t n_kv_heads, float scale, | |
| 79 | + Tensor& dq, Tensor& dk, Tensor& dv); | |
| 80 | + | |
| 81 | +// ---- fused softmax cross-entropy ------------------------------------------- | |
| 82 | +// logits: [N, V]; targets: [N] i32 (ignore_index = -1). Returns mean loss | |
| 83 | +// over valid rows. dlogits (if non-null) receives (softmax − onehot)/n_valid | |
| 84 | +// — the gradient for dloss = 1, ACCUMULATED. | |
| 85 | +float cross_entropy(const Tensor& logits, const Tensor& targets, Tensor* dlogits); | |
| 86 | + | |
| 87 | +} // namespace forge::cpu | |
added
src/ops/metal/metal_ops.cpp
+806 −0
@@ -0,0 +1,806 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#include "ops/metal/metal_ops.h" | |
| 3 | + | |
| 4 | +#include "core/device.h" | |
| 5 | + | |
| 6 | +#include <Metal/Metal.hpp> | |
| 7 | + | |
| 8 | +#include <algorithm> | |
| 9 | +#include <cstdio> | |
| 10 | +#include <cstdlib> | |
| 11 | +#include <string> | |
| 12 | + | |
| 13 | +namespace forge::metal { | |
| 14 | + | |
| 15 | +namespace { | |
| 16 | + | |
| 17 | +void check(bool cond, const char* msg) { | |
| 18 | + if (!cond) { | |
| 19 | + std::fprintf(stderr, "forge/metal: %s\n", msg); | |
| 20 | + std::abort(); | |
| 21 | + } | |
| 22 | +} | |
| 23 | + | |
| 24 | +struct MatmulParams { | |
| 25 | + uint32_t M, N, K; | |
| 26 | + uint32_t lda, ldb; | |
| 27 | + uint32_t accumulate; | |
| 28 | +}; | |
| 29 | + | |
| 30 | +struct NormParams { | |
| 31 | + uint32_t C; | |
| 32 | + float eps; | |
| 33 | +}; | |
| 34 | + | |
| 35 | +// Reduction-style kernels: one threadgroup per row, strided row walk. | |
| 36 | +MTL::Size row_threadgroup(int64_t C, MTL::ComputePipelineState* pso) { | |
| 37 | + NS::UInteger tg = 256; | |
| 38 | + tg = std::min<NS::UInteger>(tg, pso->maxTotalThreadsPerThreadgroup()); | |
| 39 | + tg = std::min<NS::UInteger>(tg, NS::UInteger((C + 31) / 32) * 32); | |
| 40 | + return MTL::Size(std::max<NS::UInteger>(tg, 32), 1, 1); | |
| 41 | +} | |
| 42 | + | |
| 43 | +void encode_flat(const char* kernel, std::initializer_list<const Tensor*> tensors, | |
| 44 | + const void* params, size_t params_len, int64_t n) { | |
| 45 | + Device& dev = Device::get(); | |
| 46 | + MTL::ComputePipelineState* pso = dev.pipeline(kernel); | |
| 47 | + MTL::ComputeCommandEncoder* enc = Stream::get().encoder(); | |
| 48 | + enc->setComputePipelineState(pso); | |
| 49 | + int idx = 0; | |
| 50 | + for (const Tensor* t : tensors) | |
| 51 | + enc->setBuffer(t->buffer(), t->buffer_offset(), idx++); | |
| 52 | + if (params) enc->setBytes(params, params_len, idx); | |
| 53 | + const NS::UInteger tg = | |
| 54 | + std::min<NS::UInteger>(256, pso->maxTotalThreadsPerThreadgroup()); | |
| 55 | + enc->dispatchThreads(MTL::Size(NS::UInteger(n), 1, 1), MTL::Size(tg, 1, 1)); | |
| 56 | +} | |
| 57 | + | |
| 58 | +} // namespace | |
| 59 | + | |
| 60 | +// ---- Stream ----------------------------------------------------------------- | |
| 61 | + | |
| 62 | +Stream& Stream::get() { | |
| 63 | + static Stream stream; | |
| 64 | + return stream; | |
| 65 | +} | |
| 66 | + | |
| 67 | +MTL::ComputeCommandEncoder* Stream::encoder() { | |
| 68 | + return encoder_of(concurrent_ ? MTL::DispatchTypeConcurrent | |
| 69 | + : MTL::DispatchTypeSerial); | |
| 70 | +} | |
| 71 | + | |
| 72 | +void Stream::set_concurrent(bool on) { concurrent_ = on; } | |
| 73 | + | |
| 74 | +MTL::ComputeCommandEncoder* Stream::concurrent_encoder() { | |
| 75 | + return encoder_of(MTL::DispatchTypeConcurrent); | |
| 76 | +} | |
| 77 | + | |
| 78 | +MTL::ComputeCommandEncoder* Stream::encoder_of(int dispatch_type) { | |
| 79 | + if (enc_ && dispatch_type_ == dispatch_type) return enc_; | |
| 80 | + if (enc_) { | |
| 81 | + // Close the current encoder; the boundary orders tracked resources. | |
| 82 | + enc_->endEncoding(); | |
| 83 | + enc_->release(); | |
| 84 | + enc_ = nullptr; | |
| 85 | + } | |
| 86 | + if (!cmd_) { | |
| 87 | + Device::get().allocator().set_defer(true); // park releases until sync | |
| 88 | + cmd_ = Device::get().queue()->commandBuffer(); | |
| 89 | + check(cmd_ != nullptr, "commandBuffer creation failed"); | |
| 90 | + cmd_->retain(); // survives autorelease-pool drains between ops | |
| 91 | + } | |
| 92 | + enc_ = cmd_->computeCommandEncoder(MTL::DispatchType(dispatch_type)); | |
| 93 | + check(enc_ != nullptr, "encoder creation failed"); | |
| 94 | + enc_->retain(); | |
| 95 | + dispatch_type_ = dispatch_type; | |
| 96 | + return enc_; | |
| 97 | +} | |
| 98 | + | |
| 99 | +void Stream::sync() { | |
| 100 | + if (!cmd_) return; | |
| 101 | + if (enc_) enc_->endEncoding(); | |
| 102 | + cmd_->commit(); | |
| 103 | + cmd_->waitUntilCompleted(); | |
| 104 | + if (cmd_->status() == MTL::CommandBufferStatusError) { | |
| 105 | + NS::Error* err = cmd_->error(); | |
| 106 | + std::fprintf(stderr, "forge/metal: command buffer failed: %s\n", | |
| 107 | + err ? err->localizedDescription()->utf8String() : "?"); | |
| 108 | + std::abort(); | |
| 109 | + } | |
| 110 | + gpu_seconds_ += cmd_->GPUEndTime() - cmd_->GPUStartTime(); | |
| 111 | + if (enc_) enc_->release(); | |
| 112 | + cmd_->release(); | |
| 113 | + enc_ = nullptr; | |
| 114 | + cmd_ = nullptr; | |
| 115 | + dispatch_type_ = -1; | |
| 116 | + Allocator& alloc = Device::get().allocator(); | |
| 117 | + alloc.flush_retired(); // GPU is idle: parked buffers may recycle now | |
| 118 | + alloc.set_defer(false); | |
| 119 | +} | |
| 120 | + | |
| 121 | +// ---- matmul ------------------------------------------------------------------- | |
| 122 | + | |
| 123 | +void matmul(const Tensor& a, const Tensor& b, Tensor& c, | |
| 124 | + bool ta, bool tb, bool accumulate, MatmulKernel kernel) { | |
| 125 | + check(a.dtype() == DType::F32 && b.dtype() == DType::F32 && c.dtype() == DType::F32, | |
| 126 | + "matmul: f32 only"); | |
| 127 | + const int64_t M = ta ? a.size(1) : a.size(0); | |
| 128 | + const int64_t K = ta ? a.size(0) : a.size(1); | |
| 129 | + const int64_t N = tb ? b.size(0) : b.size(1); | |
| 130 | + check((tb ? b.size(1) : b.size(0)) == K, "matmul: inner dims mismatch"); | |
| 131 | + check(c.size(0) == M && c.size(1) == N, "matmul: output shape mismatch"); | |
| 132 | + | |
| 133 | + // Auto: simdgroup kernel unless the problem is too small to fill even one | |
| 134 | + // 64x64 block, where the 16x16-tiled kernel's finer granularity wins. | |
| 135 | + if (kernel == MatmulKernel::Auto) | |
| 136 | + kernel = (M >= 64 && N >= 64 && K >= 16) ? MatmulKernel::Simdgroup | |
| 137 | + : MatmulKernel::Tiled; | |
| 138 | + | |
| 139 | + Device& dev = Device::get(); | |
| 140 | + const bool simd = kernel == MatmulKernel::Simdgroup; | |
| 141 | + const bool aligned = simd && (M % 64 == 0) && (N % 64 == 0) && (K % 16 == 0); | |
| 142 | + | |
| 143 | + // Function constants: 0/1 = transposes, 3 = alignment fast path. | |
| 144 | + MTL::FunctionConstantValues* constants = MTL::FunctionConstantValues::alloc()->init(); | |
| 145 | + constants->setConstantValue(&ta, MTL::DataTypeBool, NS::UInteger(0)); | |
| 146 | + constants->setConstantValue(&tb, MTL::DataTypeBool, NS::UInteger(1)); | |
| 147 | + std::string key = std::string(ta ? "t" : "n") + (tb ? "t" : "n"); | |
| 148 | + if (simd) { | |
| 149 | + constants->setConstantValue(&aligned, MTL::DataTypeBool, NS::UInteger(3)); | |
| 150 | + key += aligned ? "/a" : "/u"; | |
| 151 | + } | |
| 152 | + const char* name = simd ? "matmul_simd_f32" | |
| 153 | + : (kernel == MatmulKernel::Naive ? "matmul_naive_f32" | |
| 154 | + : "matmul_tiled_f32"); | |
| 155 | + MTL::ComputePipelineState* pso = dev.pipeline(name, constants, key); | |
| 156 | + constants->release(); | |
| 157 | + | |
| 158 | + MatmulParams p{uint32_t(M), uint32_t(N), uint32_t(K), | |
| 159 | + uint32_t(a.size(1)), uint32_t(b.size(1)), accumulate ? 1u : 0u}; | |
| 160 | + | |
| 161 | + MTL::ComputeCommandEncoder* enc = Stream::get().encoder(); | |
| 162 | + enc->setComputePipelineState(pso); | |
| 163 | + enc->setBuffer(a.buffer(), a.buffer_offset(), 0); | |
| 164 | + enc->setBuffer(b.buffer(), b.buffer_offset(), 1); | |
| 165 | + enc->setBuffer(c.buffer(), c.buffer_offset(), 2); | |
| 166 | + enc->setBytes(&p, sizeof(p), 3); | |
| 167 | + | |
| 168 | + switch (kernel) { | |
| 169 | + case MatmulKernel::Naive: | |
| 170 | + enc->dispatchThreads(MTL::Size(NS::UInteger(N), NS::UInteger(M), 1), | |
| 171 | + MTL::Size(16, 16, 1)); | |
| 172 | + break; | |
| 173 | + case MatmulKernel::Simdgroup: { | |
| 174 | + constexpr NS::UInteger BM = 64, BN = 64, THREADS = 128; | |
| 175 | + enc->dispatchThreadgroups( | |
| 176 | + MTL::Size((NS::UInteger(N) + BN - 1) / BN, | |
| 177 | + (NS::UInteger(M) + BM - 1) / BM, 1), | |
| 178 | + MTL::Size(THREADS, 1, 1)); | |
| 179 | + break; | |
| 180 | + } | |
| 181 | + default: { | |
| 182 | + constexpr NS::UInteger TILE = 16; | |
| 183 | + enc->dispatchThreadgroups( | |
| 184 | + MTL::Size((NS::UInteger(N) + TILE - 1) / TILE, | |
| 185 | + (NS::UInteger(M) + TILE - 1) / TILE, 1), | |
| 186 | + MTL::Size(TILE, TILE, 1)); | |
| 187 | + break; | |
| 188 | + } | |
| 189 | + } | |
| 190 | +} | |
| 191 | + | |
| 192 | +// ---- elementwise --------------------------------------------------------------- | |
| 193 | + | |
| 194 | +void add(const Tensor& a, const Tensor& b, Tensor& out) { | |
| 195 | + check(a.numel() == b.numel() && a.numel() == out.numel(), "add: numel mismatch"); | |
| 196 | + encode_flat("add_f32", {&a, &b, &out}, nullptr, 0, a.numel()); | |
| 197 | +} | |
| 198 | + | |
| 199 | +void mul(const Tensor& a, const Tensor& b, Tensor& out) { | |
| 200 | + check(a.numel() == b.numel() && a.numel() == out.numel(), "mul: numel mismatch"); | |
| 201 | + encode_flat("mul_f32", {&a, &b, &out}, nullptr, 0, a.numel()); | |
| 202 | +} | |
| 203 | + | |
| 204 | +void scale(const Tensor& a, float s, Tensor& out) { | |
| 205 | + encode_flat("scale_f32", {&a, &out}, &s, sizeof(s), a.numel()); | |
| 206 | +} | |
| 207 | + | |
| 208 | +void add_bias(const Tensor& x, const Tensor& bias, Tensor& out) { | |
| 209 | + const uint32_t C = uint32_t(bias.numel()); | |
| 210 | + encode_flat("add_bias_f32", {&x, &bias, &out}, &C, sizeof(C), x.numel()); | |
| 211 | +} | |
| 212 | + | |
| 213 | +void silu(const Tensor& x, Tensor& out) { | |
| 214 | + encode_flat("silu_f32", {&x, &out}, nullptr, 0, x.numel()); | |
| 215 | +} | |
| 216 | + | |
| 217 | +void gelu(const Tensor& x, Tensor& out) { | |
| 218 | + encode_flat("gelu_f32", {&x, &out}, nullptr, 0, x.numel()); | |
| 219 | +} | |
| 220 | + | |
| 221 | +// ---- row reductions -------------------------------------------------------------- | |
| 222 | + | |
| 223 | +void softmax(const Tensor& x, Tensor& out) { | |
| 224 | + const int64_t C = x.shape().back(); | |
| 225 | + const int64_t rows = x.numel() / C; | |
| 226 | + Device& dev = Device::get(); | |
| 227 | + MTL::ComputePipelineState* pso = dev.pipeline("softmax_f32"); | |
| 228 | + MTL::ComputeCommandEncoder* enc = Stream::get().encoder(); | |
| 229 | + enc->setComputePipelineState(pso); | |
| 230 | + enc->setBuffer(x.buffer(), x.buffer_offset(), 0); | |
| 231 | + enc->setBuffer(out.buffer(), out.buffer_offset(), 1); | |
| 232 | + const uint32_t c32 = uint32_t(C); | |
| 233 | + enc->setBytes(&c32, sizeof(c32), 2); | |
| 234 | + enc->dispatchThreadgroups(MTL::Size(NS::UInteger(rows), 1, 1), | |
| 235 | + row_threadgroup(C, pso)); | |
| 236 | +} | |
| 237 | + | |
| 238 | +void rmsnorm(const Tensor& x, const Tensor& w, float eps, Tensor& out) { | |
| 239 | + const int64_t C = w.numel(); | |
| 240 | + const int64_t rows = x.numel() / C; | |
| 241 | + Device& dev = Device::get(); | |
| 242 | + MTL::ComputePipelineState* pso = dev.pipeline("rmsnorm_f32"); | |
| 243 | + MTL::ComputeCommandEncoder* enc = Stream::get().encoder(); | |
| 244 | + enc->setComputePipelineState(pso); | |
| 245 | + enc->setBuffer(x.buffer(), x.buffer_offset(), 0); | |
| 246 | + enc->setBuffer(w.buffer(), w.buffer_offset(), 1); | |
| 247 | + enc->setBuffer(out.buffer(), out.buffer_offset(), 2); | |
| 248 | + NormParams p{uint32_t(C), eps}; | |
| 249 | + enc->setBytes(&p, sizeof(p), 3); | |
| 250 | + enc->dispatchThreadgroups(MTL::Size(NS::UInteger(rows), 1, 1), | |
| 251 | + row_threadgroup(C, pso)); | |
| 252 | +} | |
| 253 | + | |
| 254 | +void layernorm(const Tensor& x, const Tensor& w, const Tensor& b, float eps, Tensor& out) { | |
| 255 | + const int64_t C = w.numel(); | |
| 256 | + const int64_t rows = x.numel() / C; | |
| 257 | + Device& dev = Device::get(); | |
| 258 | + MTL::ComputePipelineState* pso = dev.pipeline("layernorm_f32"); | |
| 259 | + MTL::ComputeCommandEncoder* enc = Stream::get().encoder(); | |
| 260 | + enc->setComputePipelineState(pso); | |
| 261 | + enc->setBuffer(x.buffer(), x.buffer_offset(), 0); | |
| 262 | + enc->setBuffer(w.buffer(), w.buffer_offset(), 1); | |
| 263 | + enc->setBuffer(b.buffer(), b.buffer_offset(), 2); | |
| 264 | + enc->setBuffer(out.buffer(), out.buffer_offset(), 3); | |
| 265 | + NormParams p{uint32_t(C), eps}; | |
| 266 | + enc->setBytes(&p, sizeof(p), 4); | |
| 267 | + enc->dispatchThreadgroups(MTL::Size(NS::UInteger(rows), 1, 1), | |
| 268 | + row_threadgroup(C, pso)); | |
| 269 | +} | |
| 270 | + | |
| 271 | + | |
| 272 | +// ---- backward / training ops --------------------------------------------------- | |
| 273 | + | |
| 274 | +void accumulate(Tensor& dst, const Tensor& src) { | |
| 275 | + encode_flat("accum_f32", {&dst, &src}, nullptr, 0, dst.numel()); | |
| 276 | +} | |
| 277 | + | |
| 278 | +void axpy(Tensor& dst, const Tensor& src, const Tensor& s) { | |
| 279 | + encode_flat("axpy_f32", {&dst, &src, &s}, nullptr, 0, dst.numel()); | |
| 280 | +} | |
| 281 | + | |
| 282 | +void silu_backward(const Tensor& x, const Tensor& dout, Tensor& dx) { | |
| 283 | + encode_flat("silu_bwd_f32", {&x, &dout, &dx}, nullptr, 0, x.numel()); | |
| 284 | +} | |
| 285 | + | |
| 286 | +void gelu_backward(const Tensor& x, const Tensor& dout, Tensor& dx) { | |
| 287 | + encode_flat("gelu_bwd_f32", {&x, &dout, &dx}, nullptr, 0, x.numel()); | |
| 288 | +} | |
| 289 | + | |
| 290 | +void add_bias_backward(const Tensor& dout, Tensor& dbias) { | |
| 291 | + const uint32_t C = uint32_t(dbias.numel()); | |
| 292 | + const uint32_t N = uint32_t(dout.numel() / C); | |
| 293 | + const uint32_t nc[2] = {N, C}; | |
| 294 | + encode_flat("add_bias_bwd_f32", {&dout, &dbias}, nc, sizeof(nc), C); | |
| 295 | +} | |
| 296 | + | |
| 297 | +void rmsnorm_backward(const Tensor& x, const Tensor& w, float eps, | |
| 298 | + const Tensor& dout, Tensor& dx, Tensor& dw) { | |
| 299 | + const int64_t C = w.numel(); | |
| 300 | + const int64_t rows = x.numel() / C; | |
| 301 | + Tensor inv_rms = Tensor::empty({rows}); | |
| 302 | + Device& dev = Device::get(); | |
| 303 | + | |
| 304 | + { | |
| 305 | + MTL::ComputePipelineState* pso = dev.pipeline("rmsnorm_bwd_dx_f32"); | |
| 306 | + MTL::ComputeCommandEncoder* enc = Stream::get().encoder(); | |
| 307 | + enc->setComputePipelineState(pso); | |
| 308 | + enc->setBuffer(x.buffer(), x.buffer_offset(), 0); | |
| 309 | + enc->setBuffer(w.buffer(), w.buffer_offset(), 1); | |
| 310 | + enc->setBuffer(dout.buffer(), dout.buffer_offset(), 2); | |
| 311 | + enc->setBuffer(dx.buffer(), dx.buffer_offset(), 3); | |
| 312 | + enc->setBuffer(inv_rms.buffer(), inv_rms.buffer_offset(), 4); | |
| 313 | + NormParams p{uint32_t(C), eps}; | |
| 314 | + enc->setBytes(&p, sizeof(p), 5); | |
| 315 | + enc->dispatchThreadgroups(MTL::Size(NS::UInteger(rows), 1, 1), | |
| 316 | + row_threadgroup(C, pso)); | |
| 317 | + } | |
| 318 | + { | |
| 319 | + MTL::ComputePipelineState* pso = dev.pipeline("rmsnorm_bwd_dw_f32"); | |
| 320 | + MTL::ComputeCommandEncoder* enc = Stream::get().encoder(); | |
| 321 | + enc->setComputePipelineState(pso); | |
| 322 | + enc->setBuffer(x.buffer(), x.buffer_offset(), 0); | |
| 323 | + enc->setBuffer(dout.buffer(), dout.buffer_offset(), 1); | |
| 324 | + enc->setBuffer(inv_rms.buffer(), inv_rms.buffer_offset(), 2); | |
| 325 | + enc->setBuffer(dw.buffer(), dw.buffer_offset(), 3); | |
| 326 | + NormParams p{uint32_t(C), eps}; | |
| 327 | + enc->setBytes(&p, sizeof(p), 4); | |
| 328 | + const uint32_t r32 = uint32_t(rows); | |
| 329 | + enc->setBytes(&r32, sizeof(r32), 5); | |
| 330 | + enc->dispatchThreads(MTL::Size(NS::UInteger(C), 1, 1), MTL::Size(64, 1, 1)); | |
| 331 | + } | |
| 332 | +} | |
| 333 | + | |
| 334 | +void layernorm_backward(const Tensor& x, const Tensor& w, float eps, | |
| 335 | + const Tensor& dout, Tensor& dx, Tensor& dw, Tensor& db) { | |
| 336 | + const int64_t C = w.numel(); | |
| 337 | + const int64_t rows = x.numel() / C; | |
| 338 | + Tensor mean = Tensor::empty({rows}); | |
| 339 | + Tensor istd = Tensor::empty({rows}); | |
| 340 | + Device& dev = Device::get(); | |
| 341 | + | |
| 342 | + { | |
| 343 | + MTL::ComputePipelineState* pso = dev.pipeline("layernorm_bwd_dx_f32"); | |
| 344 | + MTL::ComputeCommandEncoder* enc = Stream::get().encoder(); | |
| 345 | + enc->setComputePipelineState(pso); | |
| 346 | + enc->setBuffer(x.buffer(), x.buffer_offset(), 0); | |
| 347 | + enc->setBuffer(w.buffer(), w.buffer_offset(), 1); | |
| 348 | + enc->setBuffer(dout.buffer(), dout.buffer_offset(), 2); | |
| 349 | + enc->setBuffer(dx.buffer(), dx.buffer_offset(), 3); | |
| 350 | + enc->setBuffer(mean.buffer(), mean.buffer_offset(), 4); | |
| 351 | + enc->setBuffer(istd.buffer(), istd.buffer_offset(), 5); | |
| 352 | + NormParams p{uint32_t(C), eps}; | |
| 353 | + enc->setBytes(&p, sizeof(p), 6); | |
| 354 | + enc->dispatchThreadgroups(MTL::Size(NS::UInteger(rows), 1, 1), | |
| 355 | + row_threadgroup(C, pso)); | |
| 356 | + } | |
| 357 | + { | |
| 358 | + MTL::ComputePipelineState* pso = dev.pipeline("layernorm_bwd_dwdb_f32"); | |
| 359 | + MTL::ComputeCommandEncoder* enc = Stream::get().encoder(); | |
| 360 | + enc->setComputePipelineState(pso); | |
| 361 | + enc->setBuffer(x.buffer(), x.buffer_offset(), 0); | |
| 362 | + enc->setBuffer(dout.buffer(), dout.buffer_offset(), 1); | |
| 363 | + enc->setBuffer(mean.buffer(), mean.buffer_offset(), 2); | |
| 364 | + enc->setBuffer(istd.buffer(), istd.buffer_offset(), 3); | |
| 365 | + enc->setBuffer(dw.buffer(), dw.buffer_offset(), 4); | |
| 366 | + enc->setBuffer(db.buffer(), db.buffer_offset(), 5); | |
| 367 | + NormParams p{uint32_t(C), eps}; | |
| 368 | + enc->setBytes(&p, sizeof(p), 6); | |
| 369 | + const uint32_t r32 = uint32_t(rows); | |
| 370 | + enc->setBytes(&r32, sizeof(r32), 7); | |
| 371 | + enc->dispatchThreads(MTL::Size(NS::UInteger(C), 1, 1), MTL::Size(64, 1, 1)); | |
| 372 | + } | |
| 373 | +} | |
| 374 | + | |
| 375 | +// ---- rope ----------------------------------------------------------------------- | |
| 376 | + | |
| 377 | +namespace { | |
| 378 | +struct RopeParams { | |
| 379 | + uint32_t T, H, HD; | |
| 380 | + float theta; | |
| 381 | + uint32_t pos_offset; | |
| 382 | +}; | |
| 383 | + | |
| 384 | +void rope_encode(const Tensor& in, Tensor& out, int64_t n_heads, float theta, | |
| 385 | + int64_t pos_offset, bool inverse) { | |
| 386 | + const int64_t B = in.size(0), T = in.size(1), C = in.size(2); | |
| 387 | + const int64_t hd = C / n_heads; | |
| 388 | + Device& dev = Device::get(); | |
| 389 | + MTL::FunctionConstantValues* constants = MTL::FunctionConstantValues::alloc()->init(); | |
| 390 | + // slots 0/1 belong to matmul TA/TB; RoPE uses slot 2 | |
| 391 | + constants->setConstantValue(&inverse, MTL::DataTypeBool, NS::UInteger(2)); | |
| 392 | + MTL::ComputePipelineState* pso = | |
| 393 | + dev.pipeline("rope_f32", constants, inverse ? "inv" : "fwd"); | |
| 394 | + constants->release(); | |
| 395 | + | |
| 396 | + MTL::ComputeCommandEncoder* enc = Stream::get().encoder(); | |
| 397 | + enc->setComputePipelineState(pso); | |
| 398 | + enc->setBuffer(in.buffer(), in.buffer_offset(), 0); | |
| 399 | + enc->setBuffer(out.buffer(), out.buffer_offset(), 1); | |
| 400 | + RopeParams p{uint32_t(T), uint32_t(n_heads), uint32_t(hd), theta, | |
| 401 | + uint32_t(pos_offset)}; | |
| 402 | + enc->setBytes(&p, sizeof(p), 2); | |
| 403 | + const int64_t pairs = B * T * n_heads * (hd / 2); | |
| 404 | + enc->dispatchThreads(MTL::Size(NS::UInteger(pairs), 1, 1), MTL::Size(256, 1, 1)); | |
| 405 | +} | |
| 406 | +} // namespace | |
| 407 | + | |
| 408 | +void rope(const Tensor& x, int64_t n_heads, float theta, int64_t pos_offset, Tensor& out) { | |
| 409 | + rope_encode(x, out, n_heads, theta, pos_offset, /*inverse=*/false); | |
| 410 | +} | |
| 411 | + | |
| 412 | +void rope_backward(const Tensor& dout, int64_t n_heads, float theta, int64_t pos_offset, | |
| 413 | + Tensor& dx) { | |
| 414 | + rope_encode(dout, dx, n_heads, theta, pos_offset, /*inverse=*/true); | |
| 415 | +} | |
| 416 | + | |
| 417 | +// ---- embedding ------------------------------------------------------------------- | |
| 418 | + | |
| 419 | +void embedding(const Tensor& weight, const Tensor& ids, Tensor& out) { | |
| 420 | + check(ids.dtype() == DType::I32, "embedding: ids must be i32 on GPU"); | |
| 421 | + const int64_t C = weight.size(1); | |
| 422 | + const int64_t N = ids.numel(); | |
| 423 | + Device& dev = Device::get(); | |
| 424 | + MTL::ComputePipelineState* pso = dev.pipeline("embedding_fwd_f32"); | |
| 425 | + MTL::ComputeCommandEncoder* enc = Stream::get().encoder(); | |
| 426 | + enc->setComputePipelineState(pso); | |
| 427 | + enc->setBuffer(weight.buffer(), weight.buffer_offset(), 0); | |
| 428 | + enc->setBuffer(ids.buffer(), ids.buffer_offset(), 1); | |
| 429 | + enc->setBuffer(out.buffer(), out.buffer_offset(), 2); | |
| 430 | + const uint32_t c32 = uint32_t(C); | |
| 431 | + enc->setBytes(&c32, sizeof(c32), 3); | |
| 432 | + enc->dispatchThreads(MTL::Size(NS::UInteger(C), NS::UInteger(N), 1), | |
| 433 | + MTL::Size(std::min<NS::UInteger>(NS::UInteger(C), 64), 4, 1)); | |
| 434 | +} | |
| 435 | + | |
| 436 | +void embedding_backward(const Tensor& ids, const Tensor& dout, Tensor& dweight) { | |
| 437 | + check(ids.dtype() == DType::I32, "embedding_backward: ids must be i32 on GPU"); | |
| 438 | + const int64_t C = dweight.size(1); | |
| 439 | + const int64_t N = ids.numel(); | |
| 440 | + Device& dev = Device::get(); | |
| 441 | + MTL::ComputePipelineState* pso = dev.pipeline("embedding_bwd_f32"); | |
| 442 | + MTL::ComputeCommandEncoder* enc = Stream::get().encoder(); | |
| 443 | + enc->setComputePipelineState(pso); | |
| 444 | + enc->setBuffer(ids.buffer(), ids.buffer_offset(), 0); | |
| 445 | + enc->setBuffer(dout.buffer(), dout.buffer_offset(), 1); | |
| 446 | + enc->setBuffer(dweight.buffer(), dweight.buffer_offset(), 2); | |
| 447 | + const uint32_t c32 = uint32_t(C); | |
| 448 | + enc->setBytes(&c32, sizeof(c32), 3); | |
| 449 | + enc->dispatchThreads(MTL::Size(NS::UInteger(C), NS::UInteger(N), 1), | |
| 450 | + MTL::Size(std::min<NS::UInteger>(NS::UInteger(C), 64), 4, 1)); | |
| 451 | +} | |
| 452 | + | |
| 453 | +// ---- attention ------------------------------------------------------------------- | |
| 454 | + | |
| 455 | +namespace { | |
| 456 | +struct AttnParams { | |
| 457 | + uint32_t B, T, H, HKV, HD; | |
| 458 | + float scale; | |
| 459 | + uint32_t causal; | |
| 460 | +}; | |
| 461 | +} // namespace | |
| 462 | + | |
| 463 | +void attention(const Tensor& q, const Tensor& k, const Tensor& v, | |
| 464 | + int64_t n_heads, int64_t n_kv_heads, bool causal, float scale, | |
| 465 | + Tensor& out, Tensor* probs_out) { | |
| 466 | + check(probs_out != nullptr, "attention: probs_out required (unfused path)"); | |
| 467 | + const int64_t B = q.size(0), T = q.size(1); | |
| 468 | + const int64_t hd = q.size(2) / n_heads; | |
| 469 | + Device& dev = Device::get(); | |
| 470 | + MTL::ComputePipelineState* pso = dev.pipeline("attention_fwd_f32"); | |
| 471 | + MTL::ComputeCommandEncoder* enc = Stream::get().encoder(); | |
| 472 | + enc->setComputePipelineState(pso); | |
| 473 | + enc->setBuffer(q.buffer(), q.buffer_offset(), 0); | |
| 474 | + enc->setBuffer(k.buffer(), k.buffer_offset(), 1); | |
| 475 | + enc->setBuffer(v.buffer(), v.buffer_offset(), 2); | |
| 476 | + enc->setBuffer(out.buffer(), out.buffer_offset(), 3); | |
| 477 | + enc->setBuffer(probs_out->buffer(), probs_out->buffer_offset(), 4); | |
| 478 | + AttnParams p{uint32_t(B), uint32_t(T), uint32_t(n_heads), uint32_t(n_kv_heads), | |
| 479 | + uint32_t(hd), scale, causal ? 1u : 0u}; | |
| 480 | + enc->setBytes(&p, sizeof(p), 5); | |
| 481 | + enc->dispatchThreads(MTL::Size(NS::UInteger(B * n_heads * T), 1, 1), | |
| 482 | + MTL::Size(64, 1, 1)); | |
| 483 | +} | |
| 484 | + | |
| 485 | +void attention_backward(const Tensor& q, const Tensor& k, const Tensor& v, | |
| 486 | + const Tensor& probs, const Tensor& out, const Tensor& dout, | |
| 487 | + int64_t n_heads, int64_t n_kv_heads, float scale, | |
| 488 | + Tensor& dq, Tensor& dk, Tensor& dv) { | |
| 489 | + const int64_t B = q.size(0), T = q.size(1); | |
| 490 | + const int64_t hd = q.size(2) / n_heads; | |
| 491 | + Device& dev = Device::get(); | |
| 492 | + AttnParams p{uint32_t(B), uint32_t(T), uint32_t(n_heads), uint32_t(n_kv_heads), | |
| 493 | + uint32_t(hd), scale, 1u}; | |
| 494 | + | |
| 495 | + // D[b,h,i] = dO_i · O_i == rowsum(dP ∘ P): computed once per query row so | |
| 496 | + // neither backward kernel needs the O(T) inner recompute. | |
| 497 | + Tensor d_term = Tensor::empty({B, n_heads, T}); | |
| 498 | + { | |
| 499 | + MTL::ComputePipelineState* pso = dev.pipeline("attention_bwd_d_f32"); | |
| 500 | + MTL::ComputeCommandEncoder* enc = Stream::get().encoder(); | |
| 501 | + enc->setComputePipelineState(pso); | |
| 502 | + enc->setBuffer(out.buffer(), out.buffer_offset(), 0); | |
| 503 | + enc->setBuffer(dout.buffer(), dout.buffer_offset(), 1); | |
| 504 | + enc->setBuffer(d_term.buffer(), d_term.buffer_offset(), 2); | |
| 505 | + enc->setBytes(&p, sizeof(p), 3); | |
| 506 | + enc->dispatchThreads(MTL::Size(NS::UInteger(B * n_heads * T), 1, 1), | |
| 507 | + MTL::Size(64, 1, 1)); | |
| 508 | + } | |
| 509 | + { | |
| 510 | + MTL::ComputePipelineState* pso = dev.pipeline("attention_bwd_dq_f32"); | |
| 511 | + MTL::ComputeCommandEncoder* enc = Stream::get().encoder(); | |
| 512 | + enc->setComputePipelineState(pso); | |
| 513 | + enc->setBuffer(q.buffer(), q.buffer_offset(), 0); | |
| 514 | + enc->setBuffer(k.buffer(), k.buffer_offset(), 1); | |
| 515 | + enc->setBuffer(v.buffer(), v.buffer_offset(), 2); | |
| 516 | + enc->setBuffer(probs.buffer(), probs.buffer_offset(), 3); | |
| 517 | + enc->setBuffer(dout.buffer(), dout.buffer_offset(), 4); | |
| 518 | + enc->setBuffer(d_term.buffer(), d_term.buffer_offset(), 5); | |
| 519 | + enc->setBuffer(dq.buffer(), dq.buffer_offset(), 6); | |
| 520 | + enc->setBytes(&p, sizeof(p), 7); | |
| 521 | + enc->dispatchThreads(MTL::Size(NS::UInteger(B * n_heads * T), 1, 1), | |
| 522 | + MTL::Size(64, 1, 1)); | |
| 523 | + } | |
| 524 | + { | |
| 525 | + MTL::ComputePipelineState* pso = dev.pipeline("attention_bwd_dkv_f32"); | |
| 526 | + MTL::ComputeCommandEncoder* enc = Stream::get().encoder(); | |
| 527 | + enc->setComputePipelineState(pso); | |
| 528 | + enc->setBuffer(q.buffer(), q.buffer_offset(), 0); | |
| 529 | + enc->setBuffer(k.buffer(), k.buffer_offset(), 1); | |
| 530 | + enc->setBuffer(v.buffer(), v.buffer_offset(), 2); | |
| 531 | + enc->setBuffer(probs.buffer(), probs.buffer_offset(), 3); | |
| 532 | + enc->setBuffer(dout.buffer(), dout.buffer_offset(), 4); | |
| 533 | + enc->setBuffer(d_term.buffer(), d_term.buffer_offset(), 5); | |
| 534 | + enc->setBuffer(dk.buffer(), dk.buffer_offset(), 6); | |
| 535 | + enc->setBuffer(dv.buffer(), dv.buffer_offset(), 7); | |
| 536 | + enc->setBytes(&p, sizeof(p), 8); | |
| 537 | + enc->dispatchThreads(MTL::Size(NS::UInteger(B * n_kv_heads * T), 1, 1), | |
| 538 | + MTL::Size(64, 1, 1)); | |
| 539 | + } | |
| 540 | +} | |
| 541 | + | |
| 542 | +// ---- cross entropy ----------------------------------------------------------------- | |
| 543 | + | |
| 544 | +void cross_entropy(const Tensor& logits, const Tensor& targets, int64_t n_valid, | |
| 545 | + Tensor& losses, Tensor& loss_out, Tensor* dlogits) { | |
| 546 | + check(targets.dtype() == DType::I32, "cross_entropy: targets must be i32 on GPU"); | |
| 547 | + const int64_t V = logits.size(1); | |
| 548 | + const int64_t N = logits.size(0); | |
| 549 | + Device& dev = Device::get(); | |
| 550 | + | |
| 551 | + struct CEParams { | |
| 552 | + uint32_t V; | |
| 553 | + float inv_n; | |
| 554 | + uint32_t want_grad; | |
| 555 | + } p{uint32_t(V), n_valid > 0 ? 1.0f / float(n_valid) : 0.0f, | |
| 556 | + dlogits ? 1u : 0u}; | |
| 557 | + | |
| 558 | + MTL::ComputePipelineState* pso = dev.pipeline("cross_entropy_f32"); | |
| 559 | + MTL::ComputeCommandEncoder* enc = Stream::get().encoder(); | |
| 560 | + enc->setComputePipelineState(pso); | |
| 561 | + enc->setBuffer(logits.buffer(), logits.buffer_offset(), 0); | |
| 562 | + enc->setBuffer(targets.buffer(), targets.buffer_offset(), 1); | |
| 563 | + enc->setBuffer(losses.buffer(), losses.buffer_offset(), 2); | |
| 564 | + // dlogits slot must be bound even when unused | |
| 565 | + const Tensor& dl = dlogits ? *dlogits : losses; | |
| 566 | + enc->setBuffer(dl.buffer(), dl.buffer_offset(), 3); | |
| 567 | + enc->setBytes(&p, sizeof(p), 4); | |
| 568 | + enc->dispatchThreadgroups(MTL::Size(NS::UInteger(N), 1, 1), row_threadgroup(V, pso)); | |
| 569 | + | |
| 570 | + sum(losses, loss_out, n_valid > 0 ? 1.0f / float(n_valid) : 0.0f); | |
| 571 | +} | |
| 572 | + | |
| 573 | +// ---- optimizer / reductions ---------------------------------------------------------- | |
| 574 | + | |
| 575 | +void adamw_step(Tensor& w, const Tensor& g, Tensor& m, Tensor& v, | |
| 576 | + float lr, float beta1, float beta2, int64_t t, float eps, float wd, | |
| 577 | + float grad_scale) { | |
| 578 | + struct AdamWParams { | |
| 579 | + float lr, beta1, beta2, bc1, bc2, eps, wd, grad_scale; | |
| 580 | + } p{lr, beta1, beta2, 1.0f - std::pow(beta1, float(t)), | |
| 581 | + 1.0f - std::pow(beta2, float(t)), eps, wd, grad_scale}; | |
| 582 | + encode_flat("adamw_f32", {&w, &g, &m, &v}, &p, sizeof(p), w.numel()); | |
| 583 | +} | |
| 584 | + | |
| 585 | +void sumsq(const Tensor& x, Tensor& out) { | |
| 586 | + Device& dev = Device::get(); | |
| 587 | + MTL::ComputePipelineState* pso = dev.pipeline("sumsq_f32"); | |
| 588 | + MTL::ComputeCommandEncoder* enc = Stream::get().encoder(); | |
| 589 | + enc->setComputePipelineState(pso); | |
| 590 | + enc->setBuffer(x.buffer(), x.buffer_offset(), 0); | |
| 591 | + enc->setBuffer(out.buffer(), out.buffer_offset(), 1); | |
| 592 | + const uint32_t n = uint32_t(x.numel()); | |
| 593 | + enc->setBytes(&n, sizeof(n), 2); | |
| 594 | + // single threadgroup: partials[0] is the total | |
| 595 | + enc->dispatchThreadgroups(MTL::Size(1, 1, 1), MTL::Size(1024, 1, 1)); | |
| 596 | +} | |
| 597 | + | |
| 598 | +void sum(const Tensor& x, Tensor& out, float mul) { | |
| 599 | + Device& dev = Device::get(); | |
| 600 | + MTL::ComputePipelineState* pso = dev.pipeline("sum_f32"); | |
| 601 | + MTL::ComputeCommandEncoder* enc = Stream::get().encoder(); | |
| 602 | + enc->setComputePipelineState(pso); | |
| 603 | + enc->setBuffer(x.buffer(), x.buffer_offset(), 0); | |
| 604 | + enc->setBuffer(out.buffer(), out.buffer_offset(), 1); | |
| 605 | + const uint32_t n = uint32_t(x.numel()); | |
| 606 | + enc->setBytes(&n, sizeof(n), 2); | |
| 607 | + enc->setBytes(&mul, sizeof(mul), 3); | |
| 608 | + enc->dispatchThreadgroups(MTL::Size(1, 1, 1), MTL::Size(1024, 1, 1)); | |
| 609 | +} | |
| 610 | + | |
| 611 | + | |
| 612 | + | |
| 613 | +// ---- fused (flash) attention ------------------------------------------------- | |
| 614 | + | |
| 615 | +namespace { | |
| 616 | + | |
| 617 | +struct FlashParams { | |
| 618 | + uint32_t B, T, H, HKV; | |
| 619 | + float scale; | |
| 620 | + uint32_t causal; | |
| 621 | +}; | |
| 622 | + | |
| 623 | +// Must match the INSTANTIATE_FLASH list in flash_attention.metal. | |
| 624 | +constexpr int64_t kFlashHeadDims[] = {16, 32, 48, 64, 80, 96, 128}; | |
| 625 | + | |
| 626 | +std::string flash_kernel(const char* stem, int64_t head_dim) { | |
| 627 | + return std::string(stem) + "_hd" + std::to_string(head_dim); | |
| 628 | +} | |
| 629 | + | |
| 630 | +} // namespace | |
| 631 | + | |
| 632 | +bool flash_supported(int64_t head_dim) { | |
| 633 | + for (int64_t hd : kFlashHeadDims) | |
| 634 | + if (hd == head_dim) return true; | |
| 635 | + return false; | |
| 636 | +} | |
| 637 | + | |
| 638 | +void flash_attention(const Tensor& q, const Tensor& k, const Tensor& v, | |
| 639 | + int64_t n_heads, int64_t n_kv_heads, bool causal, float scale, | |
| 640 | + Tensor& out, Tensor& lse, FlashKernel kernel) { | |
| 641 | + const int64_t B = q.size(0), T = q.size(1); | |
| 642 | + const int64_t hd = q.size(2) / n_heads; | |
| 643 | + check(flash_supported(hd), "flash_attention: unsupported head_dim"); | |
| 644 | + | |
| 645 | + if (kernel == FlashKernel::Auto) kernel = FlashKernel::MMA; | |
| 646 | + const bool mma = kernel == FlashKernel::MMA; | |
| 647 | + | |
| 648 | + Device& dev = Device::get(); | |
| 649 | + MTL::ComputePipelineState* pso = dev.pipeline( | |
| 650 | + flash_kernel(mma ? "flash_attn_fwd_mma_f32" : "flash_attn_fwd_f32", hd)); | |
| 651 | + MTL::ComputeCommandEncoder* enc = Stream::get().encoder(); | |
| 652 | + enc->setComputePipelineState(pso); | |
| 653 | + enc->setBuffer(q.buffer(), q.buffer_offset(), 0); | |
| 654 | + enc->setBuffer(k.buffer(), k.buffer_offset(), 1); | |
| 655 | + enc->setBuffer(v.buffer(), v.buffer_offset(), 2); | |
| 656 | + enc->setBuffer(out.buffer(), out.buffer_offset(), 3); | |
| 657 | + enc->setBuffer(lse.buffer(), lse.buffer_offset(), 4); | |
| 658 | + FlashParams p{uint32_t(B), uint32_t(T), uint32_t(n_heads), uint32_t(n_kv_heads), | |
| 659 | + scale, causal ? 1u : 0u}; | |
| 660 | + enc->setBytes(&p, sizeof(p), 5); | |
| 661 | + // One threadgroup per (query block, head, batch). Block size and thread | |
| 662 | + // count must match the kernel's enums: the scalar kernel is one thread | |
| 663 | + // per query row (TGQ=64), the MMA kernel is BQ=32 rows across 4 | |
| 664 | + // simdgroups (128 threads). | |
| 665 | + const NS::UInteger block_q = mma ? 32 : 64; | |
| 666 | + const NS::UInteger threads = mma ? 128 : 64; | |
| 667 | + check(pso->maxTotalThreadsPerThreadgroup() >= threads, | |
| 668 | + "flash_attention: pipeline cannot host the required threadgroup size"); | |
| 669 | + const NS::UInteger q_blocks = (NS::UInteger(T) + block_q - 1) / block_q; | |
| 670 | + enc->dispatchThreadgroups( | |
| 671 | + MTL::Size(q_blocks * NS::UInteger(n_heads) * NS::UInteger(B), 1, 1), | |
| 672 | + MTL::Size(threads, 1, 1)); | |
| 673 | +} | |
| 674 | + | |
| 675 | +void flash_attention_backward(const Tensor& q, const Tensor& k, const Tensor& v, | |
| 676 | + const Tensor& out, const Tensor& lse, const Tensor& dout, | |
| 677 | + int64_t n_heads, int64_t n_kv_heads, bool causal, | |
| 678 | + float scale, Tensor& dq, Tensor& dk, Tensor& dv, | |
| 679 | + FlashKernel kernel) { | |
| 680 | + const int64_t B = q.size(0), T = q.size(1); | |
| 681 | + const int64_t hd = q.size(2) / n_heads; | |
| 682 | + check(flash_supported(hd), "flash_attention_backward: unsupported head_dim"); | |
| 683 | + if (kernel == FlashKernel::Auto) kernel = FlashKernel::MMA; | |
| 684 | + const bool mma = kernel == FlashKernel::MMA; | |
| 685 | + | |
| 686 | + Device& dev = Device::get(); | |
| 687 | + FlashParams p{uint32_t(B), uint32_t(T), uint32_t(n_heads), uint32_t(n_kv_heads), | |
| 688 | + scale, causal ? 1u : 0u}; | |
| 689 | + | |
| 690 | + // D[b,h,i] = dO_i . O_i (shared with the unfused path). | |
| 691 | + Tensor d_term = Tensor::empty({B, n_heads, T}); | |
| 692 | + { | |
| 693 | + AttnParams ap{uint32_t(B), uint32_t(T), uint32_t(n_heads), | |
| 694 | + uint32_t(n_kv_heads), uint32_t(hd), scale, causal ? 1u : 0u}; | |
| 695 | + MTL::ComputePipelineState* pso = dev.pipeline("attention_bwd_d_f32"); | |
| 696 | + MTL::ComputeCommandEncoder* enc = Stream::get().encoder(); | |
| 697 | + enc->setComputePipelineState(pso); | |
| 698 | + enc->setBuffer(out.buffer(), out.buffer_offset(), 0); | |
| 699 | + enc->setBuffer(dout.buffer(), dout.buffer_offset(), 1); | |
| 700 | + enc->setBuffer(d_term.buffer(), d_term.buffer_offset(), 2); | |
| 701 | + enc->setBytes(&ap, sizeof(ap), 3); | |
| 702 | + enc->dispatchThreads(MTL::Size(NS::UInteger(B * n_heads * T), 1, 1), | |
| 703 | + MTL::Size(64, 1, 1)); | |
| 704 | + } | |
| 705 | + { | |
| 706 | + MTL::ComputePipelineState* pso = dev.pipeline(flash_kernel( | |
| 707 | + mma ? "flash_attn_bwd_dq_mma_f32" : "flash_attn_bwd_dq_f32", hd)); | |
| 708 | + MTL::ComputeCommandEncoder* enc = Stream::get().encoder(); | |
| 709 | + enc->setComputePipelineState(pso); | |
| 710 | + enc->setBuffer(q.buffer(), q.buffer_offset(), 0); | |
| 711 | + enc->setBuffer(k.buffer(), k.buffer_offset(), 1); | |
| 712 | + enc->setBuffer(v.buffer(), v.buffer_offset(), 2); | |
| 713 | + enc->setBuffer(dout.buffer(), dout.buffer_offset(), 3); | |
| 714 | + enc->setBuffer(lse.buffer(), lse.buffer_offset(), 4); | |
| 715 | + enc->setBuffer(d_term.buffer(), d_term.buffer_offset(), 5); | |
| 716 | + enc->setBuffer(dq.buffer(), dq.buffer_offset(), 6); | |
| 717 | + enc->setBytes(&p, sizeof(p), 7); | |
| 718 | + if (mma) { | |
| 719 | + const NS::UInteger q_blocks = (NS::UInteger(T) + 32 - 1) / 32; | |
| 720 | + enc->dispatchThreadgroups( | |
| 721 | + MTL::Size(q_blocks * NS::UInteger(n_heads) * NS::UInteger(B), 1, 1), | |
| 722 | + MTL::Size(128, 1, 1)); | |
| 723 | + } else { | |
| 724 | + const NS::UInteger tg = | |
| 725 | + std::min<NS::UInteger>(64, pso->maxTotalThreadsPerThreadgroup()); | |
| 726 | + enc->dispatchThreads(MTL::Size(NS::UInteger(B * n_heads * T), 1, 1), | |
| 727 | + MTL::Size(tg, 1, 1)); | |
| 728 | + } | |
| 729 | + } | |
| 730 | + if (mma) { | |
| 731 | + // dV then dK as separate kernels: fused, the thread held K, V, dK and | |
| 732 | + // dV as fragments and spilled 4352 bytes (measured with gpudebug). | |
| 733 | + { | |
| 734 | + MTL::ComputePipelineState* pso = | |
| 735 | + dev.pipeline(flash_kernel("flash_attn_bwd_dv_mma_f32", hd)); | |
| 736 | + MTL::ComputeCommandEncoder* enc = Stream::get().encoder(); | |
| 737 | + enc->setComputePipelineState(pso); | |
| 738 | + enc->setBuffer(q.buffer(), q.buffer_offset(), 0); | |
| 739 | + enc->setBuffer(k.buffer(), k.buffer_offset(), 1); | |
| 740 | + enc->setBuffer(dout.buffer(), dout.buffer_offset(), 2); | |
| 741 | + enc->setBuffer(lse.buffer(), lse.buffer_offset(), 3); | |
| 742 | + enc->setBuffer(dv.buffer(), dv.buffer_offset(), 4); | |
| 743 | + enc->setBytes(&p, sizeof(p), 5); | |
| 744 | + const NS::UInteger kv_blocks = (NS::UInteger(T) + 32 - 1) / 32; | |
| 745 | + enc->dispatchThreadgroups( | |
| 746 | + MTL::Size(kv_blocks * NS::UInteger(n_kv_heads) * NS::UInteger(B), 1, 1), | |
| 747 | + MTL::Size(128, 1, 1)); | |
| 748 | + } | |
| 749 | + { | |
| 750 | + MTL::ComputePipelineState* pso = | |
| 751 | + dev.pipeline(flash_kernel("flash_attn_bwd_dk_mma_f32", hd)); | |
| 752 | + MTL::ComputeCommandEncoder* enc = Stream::get().encoder(); | |
| 753 | + enc->setComputePipelineState(pso); | |
| 754 | + enc->setBuffer(q.buffer(), q.buffer_offset(), 0); | |
| 755 | + enc->setBuffer(k.buffer(), k.buffer_offset(), 1); | |
| 756 | + enc->setBuffer(v.buffer(), v.buffer_offset(), 2); | |
| 757 | + enc->setBuffer(dout.buffer(), dout.buffer_offset(), 3); | |
| 758 | + enc->setBuffer(lse.buffer(), lse.buffer_offset(), 4); | |
| 759 | + enc->setBuffer(d_term.buffer(), d_term.buffer_offset(), 5); | |
| 760 | + enc->setBuffer(dk.buffer(), dk.buffer_offset(), 6); | |
| 761 | + enc->setBytes(&p, sizeof(p), 7); | |
| 762 | + const NS::UInteger kv_blocks = (NS::UInteger(T) + 32 - 1) / 32; | |
| 763 | + enc->dispatchThreadgroups( | |
| 764 | + MTL::Size(kv_blocks * NS::UInteger(n_kv_heads) * NS::UInteger(B), 1, 1), | |
| 765 | + MTL::Size(128, 1, 1)); | |
| 766 | + } | |
| 767 | + } else { | |
| 768 | + { | |
| 769 | + MTL::ComputePipelineState* pso = | |
| 770 | + dev.pipeline(flash_kernel("flash_attn_bwd_dv_f32", hd)); | |
| 771 | + MTL::ComputeCommandEncoder* enc = Stream::get().encoder(); | |
| 772 | + enc->setComputePipelineState(pso); | |
| 773 | + enc->setBuffer(q.buffer(), q.buffer_offset(), 0); | |
| 774 | + enc->setBuffer(k.buffer(), k.buffer_offset(), 1); | |
| 775 | + enc->setBuffer(dout.buffer(), dout.buffer_offset(), 2); | |
| 776 | + enc->setBuffer(lse.buffer(), lse.buffer_offset(), 3); | |
| 777 | + enc->setBuffer(dv.buffer(), dv.buffer_offset(), 4); | |
| 778 | + enc->setBytes(&p, sizeof(p), 5); | |
| 779 | + const NS::UInteger tg = | |
| 780 | + std::min<NS::UInteger>(64, pso->maxTotalThreadsPerThreadgroup()); | |
| 781 | + enc->dispatchThreads(MTL::Size(NS::UInteger(B * n_kv_heads * T), 1, 1), | |
| 782 | + MTL::Size(tg, 1, 1)); | |
| 783 | + } | |
| 784 | + { | |
| 785 | + MTL::ComputePipelineState* pso = | |
| 786 | + dev.pipeline(flash_kernel("flash_attn_bwd_dk_f32", hd)); | |
| 787 | + MTL::ComputeCommandEncoder* enc = Stream::get().encoder(); | |
| 788 | + enc->setComputePipelineState(pso); | |
| 789 | + enc->setBuffer(q.buffer(), q.buffer_offset(), 0); | |
| 790 | + enc->setBuffer(k.buffer(), k.buffer_offset(), 1); | |
| 791 | + enc->setBuffer(v.buffer(), v.buffer_offset(), 2); | |
| 792 | + enc->setBuffer(dout.buffer(), dout.buffer_offset(), 3); | |
| 793 | + enc->setBuffer(lse.buffer(), lse.buffer_offset(), 4); | |
| 794 | + enc->setBuffer(d_term.buffer(), d_term.buffer_offset(), 5); | |
| 795 | + enc->setBuffer(dk.buffer(), dk.buffer_offset(), 6); | |
| 796 | + enc->setBytes(&p, sizeof(p), 7); | |
| 797 | + const NS::UInteger tg = | |
| 798 | + std::min<NS::UInteger>(64, pso->maxTotalThreadsPerThreadgroup()); | |
| 799 | + enc->dispatchThreads(MTL::Size(NS::UInteger(B * n_kv_heads * T), 1, 1), | |
| 800 | + MTL::Size(tg, 1, 1)); | |
| 801 | + } | |
| 802 | + } | |
| 803 | +} | |
| 804 | + | |
| 805 | + | |
| 806 | +} // namespace forge::metal | |
added
src/ops/metal/metal_ops.h
+152 −0
@@ -0,0 +1,152 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#pragma once | |
| 3 | + | |
| 4 | +#include "core/tensor.h" | |
| 5 | + | |
| 6 | +#include <cstdint> | |
| 7 | + | |
| 8 | +namespace MTL { | |
| 9 | +class CommandBuffer; | |
| 10 | +class ComputeCommandEncoder; | |
| 11 | +} | |
| 12 | + | |
| 13 | +// Metal dispatch wrappers over the batched execution model (RESEARCH.md §4): | |
| 14 | +// ops encode into one long-lived serial compute encoder inside one command | |
| 15 | +// buffer; nothing runs until the caller reaches a readback boundary and | |
| 16 | +// calls sync(). Serial dispatch type means dispatch N+1 sees dispatch N's | |
| 17 | +// writes — no barriers, no per-op waits. | |
| 18 | +// | |
| 19 | +// Contract: tensors passed to these functions must stay alive until sync() | |
| 20 | +// returns (the training loop and tests naturally satisfy this; the | |
| 21 | +// allocator-level retire list lands with M4's trainer). | |
| 22 | +namespace forge::metal { | |
| 23 | + | |
| 24 | +class Stream { | |
| 25 | +public: | |
| 26 | + static Stream& get(); | |
| 27 | + | |
| 28 | + // Current encoder (lazily opens a command buffer + serial encoder). | |
| 29 | + MTL::ComputeCommandEncoder* encoder(); | |
| 30 | + | |
| 31 | + // Encoder for a run of MUTUALLY INDEPENDENT dispatches. Metal's default | |
| 32 | + // serial encoder orders every dispatch against the previous one, which | |
| 33 | + // wastes the GPU when the work is genuinely parallel — measured 15x on a | |
| 34 | + // batch of small independent dispatches (RESEARCH.md 4b). Switching | |
| 35 | + // dispatch type ends the current encoder and opens a new one in the same | |
| 36 | + // command buffer; Metal orders tracked resources across that boundary, so | |
| 37 | + // the switch itself acts as the barrier. | |
| 38 | + // | |
| 39 | + // Caller's contract: everything encoded between two switches must be | |
| 40 | + // free of read-after-write dependencies on each other. | |
| 41 | + MTL::ComputeCommandEncoder* concurrent_encoder(); | |
| 42 | + | |
| 43 | + // Makes encoder() hand back a concurrent encoder; see ConcurrentRegion. | |
| 44 | + void set_concurrent(bool on); | |
| 45 | + | |
| 46 | + // Readback boundary: end encoding, commit, wait. Returns immediately if | |
| 47 | + // nothing is pending. GPU time of the completed buffer is accumulated | |
| 48 | + // into gpu_seconds(). | |
| 49 | + void sync(); | |
| 50 | + | |
| 51 | + double gpu_seconds() const { return gpu_seconds_; } | |
| 52 | + | |
| 53 | +private: | |
| 54 | + Stream() = default; | |
| 55 | + MTL::ComputeCommandEncoder* encoder_of(int dispatch_type); | |
| 56 | + | |
| 57 | + MTL::CommandBuffer* cmd_ = nullptr; | |
| 58 | + MTL::ComputeCommandEncoder* enc_ = nullptr; | |
| 59 | + int dispatch_type_ = -1; | |
| 60 | + bool concurrent_ = false; | |
| 61 | + double gpu_seconds_ = 0.0; | |
| 62 | +}; | |
| 63 | + | |
| 64 | +inline void sync() { Stream::get().sync(); } | |
| 65 | + | |
| 66 | +// RAII: inside this scope, ops encode into a CONCURRENT compute encoder. | |
| 67 | +// Everything encoded in the region must be mutually independent (no | |
| 68 | +// read-after-write between them); dependencies across the region boundary are | |
| 69 | +// fine because switching encoder kind ends the encoder, and Metal orders | |
| 70 | +// tracked resources across encoders in a command buffer. | |
| 71 | +struct ConcurrentRegion { | |
| 72 | + ConcurrentRegion() { Stream::get().set_concurrent(true); } | |
| 73 | + ~ConcurrentRegion() { Stream::get().set_concurrent(false); } | |
| 74 | + ConcurrentRegion(const ConcurrentRegion&) = delete; | |
| 75 | + ConcurrentRegion& operator=(const ConcurrentRegion&) = delete; | |
| 76 | +}; | |
| 77 | + | |
| 78 | +// ---- f32 forward ops (parity-tested vs forge::cpu) ------------------------- | |
| 79 | +enum class MatmulKernel { Auto, Naive, Tiled, Simdgroup }; | |
| 80 | + | |
| 81 | +void matmul(const Tensor& a, const Tensor& b, Tensor& c, | |
| 82 | + bool transpose_a = false, bool transpose_b = false, | |
| 83 | + bool accumulate = false, MatmulKernel kernel = MatmulKernel::Auto); | |
| 84 | + | |
| 85 | +void add(const Tensor& a, const Tensor& b, Tensor& out); | |
| 86 | +void mul(const Tensor& a, const Tensor& b, Tensor& out); | |
| 87 | +void scale(const Tensor& a, float s, Tensor& out); | |
| 88 | +void add_bias(const Tensor& x, const Tensor& bias, Tensor& out); | |
| 89 | +void silu(const Tensor& x, Tensor& out); | |
| 90 | +void gelu(const Tensor& x, Tensor& out); | |
| 91 | + | |
| 92 | +void softmax(const Tensor& x, Tensor& out); // rows = last dim | |
| 93 | +void rmsnorm(const Tensor& x, const Tensor& w, float eps, Tensor& out); | |
| 94 | +void layernorm(const Tensor& x, const Tensor& w, const Tensor& b, float eps, Tensor& out); | |
| 95 | + | |
| 96 | +// ---- f32 backward / training ops (ACCUMULATE into d* outputs) -------------- | |
| 97 | +void accumulate(Tensor& dst, const Tensor& src); // dst += src | |
| 98 | +void axpy(Tensor& dst, const Tensor& src, const Tensor& s); // dst += src * s[0] | |
| 99 | +void silu_backward(const Tensor& x, const Tensor& dout, Tensor& dx); | |
| 100 | +void gelu_backward(const Tensor& x, const Tensor& dout, Tensor& dx); | |
| 101 | +void add_bias_backward(const Tensor& dout, Tensor& dbias); | |
| 102 | +void rmsnorm_backward(const Tensor& x, const Tensor& w, float eps, | |
| 103 | + const Tensor& dout, Tensor& dx, Tensor& dw); | |
| 104 | +void layernorm_backward(const Tensor& x, const Tensor& w, float eps, | |
| 105 | + const Tensor& dout, Tensor& dx, Tensor& dw, Tensor& db); | |
| 106 | + | |
| 107 | +void rope(const Tensor& x, int64_t n_heads, float theta, int64_t pos_offset, Tensor& out); | |
| 108 | +void rope_backward(const Tensor& dout, int64_t n_heads, float theta, int64_t pos_offset, | |
| 109 | + Tensor& dx); | |
| 110 | + | |
| 111 | +void embedding(const Tensor& weight, const Tensor& ids, Tensor& out); // ids i32 | |
| 112 | +void embedding_backward(const Tensor& ids, const Tensor& dout, Tensor& dweight); | |
| 113 | + | |
| 114 | +void attention(const Tensor& q, const Tensor& k, const Tensor& v, | |
| 115 | + int64_t n_heads, int64_t n_kv_heads, bool causal, float scale, | |
| 116 | + Tensor& out, Tensor* probs_out); | |
| 117 | +void attention_backward(const Tensor& q, const Tensor& k, const Tensor& v, | |
| 118 | + const Tensor& probs, const Tensor& out, const Tensor& dout, | |
| 119 | + int64_t n_heads, int64_t n_kv_heads, float scale, | |
| 120 | + Tensor& dq, Tensor& dk, Tensor& dv); | |
| 121 | + | |
| 122 | +// Fused (flash) attention: stores no T x T probabilities, only the per-row | |
| 123 | +// logsumexp `lse` [B, n_heads, T] that the backward re-expands. Supported for | |
| 124 | +// a fixed set of head_dims — flash_supported() reports which. | |
| 125 | +bool flash_supported(int64_t head_dim); | |
| 126 | +// Scalar: one thread per query row. MMA: simdgroup_matrix 8x8 tiles. Same | |
| 127 | +// outputs; Auto picks MMA where supported. | |
| 128 | +enum class FlashKernel { Auto, Scalar, MMA }; | |
| 129 | +void flash_attention(const Tensor& q, const Tensor& k, const Tensor& v, | |
| 130 | + int64_t n_heads, int64_t n_kv_heads, bool causal, float scale, | |
| 131 | + Tensor& out, Tensor& lse, FlashKernel kernel = FlashKernel::Auto); | |
| 132 | +void flash_attention_backward(const Tensor& q, const Tensor& k, const Tensor& v, | |
| 133 | + const Tensor& out, const Tensor& lse, const Tensor& dout, | |
| 134 | + int64_t n_heads, int64_t n_kv_heads, bool causal, | |
| 135 | + float scale, Tensor& dq, Tensor& dk, Tensor& dv, | |
| 136 | + FlashKernel kernel = FlashKernel::Auto); | |
| 137 | + | |
| 138 | +// losses: [N] per-row buffer; loss_out: [1] mean over n_valid. dlogits | |
| 139 | +// optional (accumulated). ids must be i32. | |
| 140 | +void cross_entropy(const Tensor& logits, const Tensor& targets, int64_t n_valid, | |
| 141 | + Tensor& losses, Tensor& loss_out, Tensor* dlogits); | |
| 142 | + | |
| 143 | +// Fused optimizer update for one tensor; all state f32. | |
| 144 | +void adamw_step(Tensor& w, const Tensor& g, Tensor& m, Tensor& v, | |
| 145 | + float lr, float beta1, float beta2, int64_t t, float eps, float wd, | |
| 146 | + float grad_scale); | |
| 147 | +// out[0] = sum(x^2) — single-threadgroup reduce (fine for M4). | |
| 148 | +void sumsq(const Tensor& x, Tensor& out); | |
| 149 | +// out[0] = sum(x) * mul | |
| 150 | +void sum(const Tensor& x, Tensor& out, float mul); | |
| 151 | + | |
| 152 | +} // namespace forge::metal | |
added
src/ops/ops.cpp
+403 −0
@@ -0,0 +1,403 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#include "ops/ops.h" | |
| 3 | + | |
| 4 | +#include "ops/cpu/cpu_ops.h" | |
| 5 | +#include "ops/metal/metal_ops.h" | |
| 6 | + | |
| 7 | +namespace forge::ops { | |
| 8 | + | |
| 9 | +namespace { | |
| 10 | + | |
| 11 | +Backend g_backend = Backend::CPU; | |
| 12 | + | |
| 13 | +bool gpu() { return g_backend == Backend::Metal; } | |
| 14 | + | |
| 15 | +bool grad_needed(std::initializer_list<const Var*> inputs) { | |
| 16 | + if (!Tape::get().enabled()) return false; | |
| 17 | + for (const Var* v : inputs) | |
| 18 | + if (v->requires_grad()) return true; | |
| 19 | + return false; | |
| 20 | +} | |
| 21 | + | |
| 22 | +// dst += src (backend-routed) | |
| 23 | +void accumulate(Tensor& dst, const Tensor& src) { | |
| 24 | + if (gpu()) { | |
| 25 | + metal::accumulate(dst, src); | |
| 26 | + return; | |
| 27 | + } | |
| 28 | + float* pd = dst.data<float>(); | |
| 29 | + const float* ps = src.data<float>(); | |
| 30 | + for (int64_t i = 0; i < dst.numel(); ++i) pd[i] += ps[i]; | |
| 31 | +} | |
| 32 | + | |
| 33 | +// dst += src * s, s known on CPU | |
| 34 | +void axpy_const(Tensor& dst, const Tensor& src, float s) { | |
| 35 | + if (gpu()) { | |
| 36 | + Tensor tmp = Tensor::empty(src.shape()); | |
| 37 | + metal::scale(src, s, tmp); | |
| 38 | + metal::accumulate(dst, tmp); | |
| 39 | + return; | |
| 40 | + } | |
| 41 | + float* pd = dst.data<float>(); | |
| 42 | + const float* ps = src.data<float>(); | |
| 43 | + for (int64_t i = 0; i < dst.numel(); ++i) pd[i] += ps[i] * s; | |
| 44 | +} | |
| 45 | + | |
| 46 | +// dst += src * s[0], s produced on-GPU (never read on CPU) | |
| 47 | +void axpy_tensor(Tensor& dst, const Tensor& src, const Tensor& s) { | |
| 48 | + if (gpu()) { | |
| 49 | + metal::axpy(dst, src, s); | |
| 50 | + return; | |
| 51 | + } | |
| 52 | + const float sv = s.data<float>()[0]; | |
| 53 | + float* pd = dst.data<float>(); | |
| 54 | + const float* ps = src.data<float>(); | |
| 55 | + for (int64_t i = 0; i < dst.numel(); ++i) pd[i] += ps[i] * sv; | |
| 56 | +} | |
| 57 | + | |
| 58 | +} // namespace | |
| 59 | + | |
| 60 | +void set_backend(Backend b) { g_backend = b; } | |
| 61 | +Backend backend() { return g_backend; } | |
| 62 | + | |
| 63 | +Var matmul(const Var& a, const Var& b, bool ta, bool tb) { | |
| 64 | + const int64_t M = ta ? a.value().size(1) : a.value().size(0); | |
| 65 | + const int64_t N = tb ? b.value().size(0) : b.value().size(1); | |
| 66 | + Tensor out = Tensor::empty({M, N}); | |
| 67 | + if (gpu()) metal::matmul(a.value(), b.value(), out, ta, tb); | |
| 68 | + else cpu::matmul(a.value(), b.value(), out, ta, tb); | |
| 69 | + | |
| 70 | + const bool needs = grad_needed({&a, &b}); | |
| 71 | + Var result(std::move(out), needs); | |
| 72 | + if (needs) { | |
| 73 | + Tape::get().record([a, b, result, ta, tb]() { | |
| 74 | + const Tensor& dc = result.grad(); | |
| 75 | + if (a.requires_grad()) { | |
| 76 | + Tensor& da = a.grad(); | |
| 77 | + if (gpu()) { | |
| 78 | + if (!ta) metal::matmul(dc, b.value(), da, false, !tb, true); | |
| 79 | + else metal::matmul(b.value(), dc, da, tb, true, true); | |
| 80 | + } else { | |
| 81 | + if (!ta) cpu::matmul(dc, b.value(), da, false, !tb, true); | |
| 82 | + else cpu::matmul(b.value(), dc, da, tb, true, true); | |
| 83 | + } | |
| 84 | + } | |
| 85 | + if (b.requires_grad()) { | |
| 86 | + Tensor& db = b.grad(); | |
| 87 | + if (gpu()) { | |
| 88 | + if (!tb) metal::matmul(a.value(), dc, db, !ta, false, true); | |
| 89 | + else metal::matmul(dc, a.value(), db, true, ta, true); | |
| 90 | + } else { | |
| 91 | + if (!tb) cpu::matmul(a.value(), dc, db, !ta, false, true); | |
| 92 | + else cpu::matmul(dc, a.value(), db, true, ta, true); | |
| 93 | + } | |
| 94 | + } | |
| 95 | + }); | |
| 96 | + } | |
| 97 | + return result; | |
| 98 | +} | |
| 99 | + | |
| 100 | +Var add(const Var& a, const Var& b) { | |
| 101 | + Tensor out = Tensor::empty(a.value().shape()); | |
| 102 | + if (gpu()) metal::add(a.value(), b.value(), out); | |
| 103 | + else cpu::add(a.value(), b.value(), out); | |
| 104 | + | |
| 105 | + const bool needs = grad_needed({&a, &b}); | |
| 106 | + Var result(std::move(out), needs); | |
| 107 | + if (needs) { | |
| 108 | + Tape::get().record([a, b, result]() { | |
| 109 | + if (a.requires_grad()) accumulate(a.grad(), result.grad()); | |
| 110 | + if (b.requires_grad()) accumulate(b.grad(), result.grad()); | |
| 111 | + }); | |
| 112 | + } | |
| 113 | + return result; | |
| 114 | +} | |
| 115 | + | |
| 116 | +Var add_bias(const Var& x, const Var& bias) { | |
| 117 | + Tensor out = Tensor::empty(x.value().shape()); | |
| 118 | + if (gpu()) metal::add_bias(x.value(), bias.value(), out); | |
| 119 | + else cpu::add_bias(x.value(), bias.value(), out); | |
| 120 | + | |
| 121 | + const bool needs = grad_needed({&x, &bias}); | |
| 122 | + Var result(std::move(out), needs); | |
| 123 | + if (needs) { | |
| 124 | + Tape::get().record([x, bias, result]() { | |
| 125 | + if (x.requires_grad()) accumulate(x.grad(), result.grad()); | |
| 126 | + if (bias.requires_grad()) { | |
| 127 | + if (gpu()) metal::add_bias_backward(result.grad(), bias.grad()); | |
| 128 | + else cpu::add_bias_backward(result.grad(), bias.grad()); | |
| 129 | + } | |
| 130 | + }); | |
| 131 | + } | |
| 132 | + return result; | |
| 133 | +} | |
| 134 | + | |
| 135 | +Var mul(const Var& a, const Var& b) { | |
| 136 | + Tensor out = Tensor::empty(a.value().shape()); | |
| 137 | + if (gpu()) metal::mul(a.value(), b.value(), out); | |
| 138 | + else cpu::mul(a.value(), b.value(), out); | |
| 139 | + | |
| 140 | + const bool needs = grad_needed({&a, &b}); | |
| 141 | + Var result(std::move(out), needs); | |
| 142 | + if (needs) { | |
| 143 | + Tape::get().record([a, b, result]() { | |
| 144 | + const Tensor& dout = result.grad(); | |
| 145 | + if (a.requires_grad()) { | |
| 146 | + Tensor tmp = Tensor::empty(dout.shape()); | |
| 147 | + if (gpu()) metal::mul(dout, b.value(), tmp); | |
| 148 | + else cpu::mul(dout, b.value(), tmp); | |
| 149 | + accumulate(a.grad(), tmp); | |
| 150 | + } | |
| 151 | + if (b.requires_grad()) { | |
| 152 | + Tensor tmp = Tensor::empty(dout.shape()); | |
| 153 | + if (gpu()) metal::mul(dout, a.value(), tmp); | |
| 154 | + else cpu::mul(dout, a.value(), tmp); | |
| 155 | + accumulate(b.grad(), tmp); | |
| 156 | + } | |
| 157 | + }); | |
| 158 | + } | |
| 159 | + return result; | |
| 160 | +} | |
| 161 | + | |
| 162 | +Var scale(const Var& a, float s) { | |
| 163 | + Tensor out = Tensor::empty(a.value().shape()); | |
| 164 | + if (gpu()) metal::scale(a.value(), s, out); | |
| 165 | + else cpu::scale(a.value(), s, out); | |
| 166 | + | |
| 167 | + const bool needs = grad_needed({&a}); | |
| 168 | + Var result(std::move(out), needs); | |
| 169 | + if (needs) { | |
| 170 | + Tape::get().record([a, result, s]() { | |
| 171 | + if (a.requires_grad()) axpy_const(a.grad(), result.grad(), s); | |
| 172 | + }); | |
| 173 | + } | |
| 174 | + return result; | |
| 175 | +} | |
| 176 | + | |
| 177 | +Var silu(const Var& x) { | |
| 178 | + Tensor out = Tensor::empty(x.value().shape()); | |
| 179 | + if (gpu()) metal::silu(x.value(), out); | |
| 180 | + else cpu::silu(x.value(), out); | |
| 181 | + | |
| 182 | + const bool needs = grad_needed({&x}); | |
| 183 | + Var result(std::move(out), needs); | |
| 184 | + if (needs) { | |
| 185 | + Tape::get().record([x, result]() { | |
| 186 | + if (!x.requires_grad()) return; | |
| 187 | + if (gpu()) metal::silu_backward(x.value(), result.grad(), x.grad()); | |
| 188 | + else cpu::silu_backward(x.value(), result.grad(), x.grad()); | |
| 189 | + }); | |
| 190 | + } | |
| 191 | + return result; | |
| 192 | +} | |
| 193 | + | |
| 194 | +Var gelu(const Var& x) { | |
| 195 | + Tensor out = Tensor::empty(x.value().shape()); | |
| 196 | + if (gpu()) metal::gelu(x.value(), out); | |
| 197 | + else cpu::gelu(x.value(), out); | |
| 198 | + | |
| 199 | + const bool needs = grad_needed({&x}); | |
| 200 | + Var result(std::move(out), needs); | |
| 201 | + if (needs) { | |
| 202 | + Tape::get().record([x, result]() { | |
| 203 | + if (!x.requires_grad()) return; | |
| 204 | + if (gpu()) metal::gelu_backward(x.value(), result.grad(), x.grad()); | |
| 205 | + else cpu::gelu_backward(x.value(), result.grad(), x.grad()); | |
| 206 | + }); | |
| 207 | + } | |
| 208 | + return result; | |
| 209 | +} | |
| 210 | + | |
| 211 | +Var rmsnorm(const Var& x, const Var& w, float eps) { | |
| 212 | + Tensor out = Tensor::empty(x.value().shape()); | |
| 213 | + if (gpu()) metal::rmsnorm(x.value(), w.value(), eps, out); | |
| 214 | + else cpu::rmsnorm(x.value(), w.value(), eps, out); | |
| 215 | + | |
| 216 | + const bool needs = grad_needed({&x, &w}); | |
| 217 | + Var result(std::move(out), needs); | |
| 218 | + if (needs) { | |
| 219 | + Tape::get().record([x, w, eps, result]() { | |
| 220 | + Tensor dx_scratch, dw_scratch; | |
| 221 | + Tensor& dx = x.requires_grad() ? x.grad() | |
| 222 | + : (dx_scratch = Tensor::zeros(x.value().shape())); | |
| 223 | + Tensor& dw = w.requires_grad() ? w.grad() | |
| 224 | + : (dw_scratch = Tensor::zeros(w.value().shape())); | |
| 225 | + if (gpu()) metal::rmsnorm_backward(x.value(), w.value(), eps, result.grad(), dx, dw); | |
| 226 | + else cpu::rmsnorm_backward(x.value(), w.value(), eps, result.grad(), dx, dw); | |
| 227 | + }); | |
| 228 | + } | |
| 229 | + return result; | |
| 230 | +} | |
| 231 | + | |
| 232 | +Var layernorm(const Var& x, const Var& w, const Var& b, float eps) { | |
| 233 | + Tensor out = Tensor::empty(x.value().shape()); | |
| 234 | + if (gpu()) metal::layernorm(x.value(), w.value(), b.value(), eps, out); | |
| 235 | + else cpu::layernorm(x.value(), w.value(), b.value(), eps, out); | |
| 236 | + | |
| 237 | + const bool needs = grad_needed({&x, &w, &b}); | |
| 238 | + Var result(std::move(out), needs); | |
| 239 | + if (needs) { | |
| 240 | + Tape::get().record([x, w, b, eps, result]() { | |
| 241 | + Tensor dx_scratch, dw_scratch, db_scratch; | |
| 242 | + Tensor& dx = x.requires_grad() ? x.grad() | |
| 243 | + : (dx_scratch = Tensor::zeros(x.value().shape())); | |
| 244 | + Tensor& dw = w.requires_grad() ? w.grad() | |
| 245 | + : (dw_scratch = Tensor::zeros(w.value().shape())); | |
| 246 | + Tensor& db = b.requires_grad() ? b.grad() | |
| 247 | + : (db_scratch = Tensor::zeros(b.value().shape())); | |
| 248 | + if (gpu()) | |
| 249 | + metal::layernorm_backward(x.value(), w.value(), eps, result.grad(), dx, dw, db); | |
| 250 | + else | |
| 251 | + cpu::layernorm_backward(x.value(), w.value(), eps, result.grad(), dx, dw, db); | |
| 252 | + }); | |
| 253 | + } | |
| 254 | + return result; | |
| 255 | +} | |
| 256 | + | |
| 257 | +Var embedding(const Var& weight, const Tensor& ids) { | |
| 258 | + const int64_t C = weight.value().size(1); | |
| 259 | + std::vector<int64_t> out_shape = ids.shape(); | |
| 260 | + out_shape.push_back(C); | |
| 261 | + Tensor out = Tensor::empty(std::move(out_shape)); | |
| 262 | + if (gpu()) metal::embedding(weight.value(), ids, out); | |
| 263 | + else cpu::embedding(weight.value(), ids, out); | |
| 264 | + | |
| 265 | + const bool needs = grad_needed({&weight}); | |
| 266 | + Var result(std::move(out), needs); | |
| 267 | + if (needs) { | |
| 268 | + Tape::get().record([weight, ids, result]() { | |
| 269 | + if (!weight.requires_grad()) return; | |
| 270 | + if (gpu()) metal::embedding_backward(ids, result.grad(), weight.grad()); | |
| 271 | + else cpu::embedding_backward(ids, result.grad(), weight.grad()); | |
| 272 | + }); | |
| 273 | + } | |
| 274 | + return result; | |
| 275 | +} | |
| 276 | + | |
| 277 | +Var rope(const Var& x, int64_t n_heads, float theta, int64_t pos_offset) { | |
| 278 | + Tensor out = Tensor::empty(x.value().shape()); | |
| 279 | + if (gpu()) metal::rope(x.value(), n_heads, theta, pos_offset, out); | |
| 280 | + else cpu::rope(x.value(), n_heads, theta, pos_offset, out); | |
| 281 | + | |
| 282 | + const bool needs = grad_needed({&x}); | |
| 283 | + Var result(std::move(out), needs); | |
| 284 | + if (needs) { | |
| 285 | + Tape::get().record([x, n_heads, theta, pos_offset, result]() { | |
| 286 | + if (!x.requires_grad()) return; | |
| 287 | + if (gpu()) metal::rope_backward(result.grad(), n_heads, theta, pos_offset, x.grad()); | |
| 288 | + else cpu::rope_backward(result.grad(), n_heads, theta, pos_offset, x.grad()); | |
| 289 | + }); | |
| 290 | + } | |
| 291 | + return result; | |
| 292 | +} | |
| 293 | + | |
| 294 | +Var attention(const Var& q, const Var& k, const Var& v, | |
| 295 | + int64_t n_heads, int64_t n_kv_heads, bool causal, float scale) { | |
| 296 | + const int64_t B = q.value().size(0), T = q.value().size(1); | |
| 297 | + const int64_t head_dim = q.value().size(2) / n_heads; | |
| 298 | + Tensor out = Tensor::empty(q.value().shape()); | |
| 299 | + const bool needs = grad_needed({&q, &k, &v}); | |
| 300 | + | |
| 301 | + // Fused path: keeps only the per-row logsumexp instead of a [B,H,T,T] | |
| 302 | + // probability tensor, so memory is linear in T rather than quadratic. | |
| 303 | + if (gpu() && metal::flash_supported(head_dim)) { | |
| 304 | + Tensor lse = Tensor::empty({B, n_heads, T}); | |
| 305 | + metal::flash_attention(q.value(), k.value(), v.value(), n_heads, n_kv_heads, | |
| 306 | + causal, scale, out, lse); | |
| 307 | + Var result(std::move(out), needs); | |
| 308 | + if (needs) { | |
| 309 | + Tape::get().record( | |
| 310 | + [q, k, v, lse, n_heads, n_kv_heads, causal, scale, result]() { | |
| 311 | + Tensor dq_scratch, dk_scratch, dv_scratch; | |
| 312 | + Tensor& dq = q.requires_grad() ? q.grad() | |
| 313 | + : (dq_scratch = Tensor::zeros(q.value().shape())); | |
| 314 | + Tensor& dk = k.requires_grad() ? k.grad() | |
| 315 | + : (dk_scratch = Tensor::zeros(k.value().shape())); | |
| 316 | + Tensor& dv = v.requires_grad() ? v.grad() | |
| 317 | + : (dv_scratch = Tensor::zeros(v.value().shape())); | |
| 318 | + metal::flash_attention_backward(q.value(), k.value(), v.value(), | |
| 319 | + result.value(), lse, result.grad(), | |
| 320 | + n_heads, n_kv_heads, causal, scale, | |
| 321 | + dq, dk, dv); | |
| 322 | + }); | |
| 323 | + } | |
| 324 | + return result; | |
| 325 | + } | |
| 326 | + | |
| 327 | + Tensor probs = Tensor::empty({B, n_heads, T, T}); | |
| 328 | + if (gpu()) | |
| 329 | + metal::attention(q.value(), k.value(), v.value(), n_heads, n_kv_heads, causal, | |
| 330 | + scale, out, &probs); | |
| 331 | + else | |
| 332 | + cpu::attention(q.value(), k.value(), v.value(), n_heads, n_kv_heads, causal, | |
| 333 | + scale, out, &probs); | |
| 334 | + | |
| 335 | + Var result(std::move(out), needs); | |
| 336 | + if (needs) { | |
| 337 | + Tape::get().record([q, k, v, probs, n_heads, n_kv_heads, scale, result]() { | |
| 338 | + Tensor dq_scratch, dk_scratch, dv_scratch; | |
| 339 | + Tensor& dq = q.requires_grad() ? q.grad() | |
| 340 | + : (dq_scratch = Tensor::zeros(q.value().shape())); | |
| 341 | + Tensor& dk = k.requires_grad() ? k.grad() | |
| 342 | + : (dk_scratch = Tensor::zeros(k.value().shape())); | |
| 343 | + Tensor& dv = v.requires_grad() ? v.grad() | |
| 344 | + : (dv_scratch = Tensor::zeros(v.value().shape())); | |
| 345 | + if (gpu()) | |
| 346 | + metal::attention_backward(q.value(), k.value(), v.value(), probs, | |
| 347 | + result.value(), result.grad(), n_heads, | |
| 348 | + n_kv_heads, scale, dq, dk, dv); | |
| 349 | + else | |
| 350 | + cpu::attention_backward(q.value(), k.value(), v.value(), probs, | |
| 351 | + result.value(), result.grad(), n_heads, | |
| 352 | + n_kv_heads, scale, dq, dk, dv); | |
| 353 | + }); | |
| 354 | + } | |
| 355 | + return result; | |
| 356 | +} | |
| 357 | + | |
| 358 | +Var cross_entropy(const Var& logits, const Tensor& targets) { | |
| 359 | + const bool needs = grad_needed({&logits}); | |
| 360 | + const int64_t N = logits.value().size(0); | |
| 361 | + | |
| 362 | + // n_valid comes from the CPU-resident targets (written by the data | |
| 363 | + // loader, never touched by the GPU). | |
| 364 | + int64_t n_valid = 0; | |
| 365 | + for (int64_t i = 0; i < N; ++i) { | |
| 366 | + const int32_t t = targets.dtype() == DType::I32 | |
| 367 | + ? targets.data<int32_t>()[i] | |
| 368 | + : int32_t(targets.data<uint16_t>()[i]); | |
| 369 | + if (t >= 0) ++n_valid; | |
| 370 | + } | |
| 371 | + | |
| 372 | + // llm.c pattern: the logit gradient is a byproduct of the forward pass; | |
| 373 | + // save it and scale by d(loss) at backward time. | |
| 374 | + Tensor dlogits; | |
| 375 | + if (needs) dlogits = Tensor::zeros(logits.value().shape()); | |
| 376 | + | |
| 377 | + Tensor loss_out; | |
| 378 | + if (gpu()) { | |
| 379 | + Tensor losses = Tensor::empty({N}); | |
| 380 | + loss_out = Tensor::empty({1}); | |
| 381 | + metal::cross_entropy(logits.value(), targets, n_valid, losses, loss_out, | |
| 382 | + needs ? &dlogits : nullptr); | |
| 383 | + } else { | |
| 384 | + const float loss = | |
| 385 | + cpu::cross_entropy(logits.value(), targets, needs ? &dlogits : nullptr); | |
| 386 | + loss_out = Tensor::full({1}, loss); | |
| 387 | + } | |
| 388 | + | |
| 389 | + Var result(std::move(loss_out), needs); | |
| 390 | + if (needs) { | |
| 391 | + Tape::get().record([logits, dlogits, result]() { | |
| 392 | + if (logits.requires_grad()) | |
| 393 | + axpy_tensor(logits.grad(), dlogits, result.grad()); | |
| 394 | + }); | |
| 395 | + } | |
| 396 | + return result; | |
| 397 | +} | |
| 398 | + | |
| 399 | +Var reshape(const Var& x, std::vector<int64_t> shape) { | |
| 400 | + return x.reshaped(std::move(shape)); | |
| 401 | +} | |
| 402 | + | |
| 403 | +} // namespace forge::ops | |
added
src/ops/ops.h
+54 −0
@@ -0,0 +1,54 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#pragma once | |
| 3 | + | |
| 4 | +#include "core/autograd.h" | |
| 5 | +#include "core/tensor.h" | |
| 6 | + | |
| 7 | +#include <cstdint> | |
| 8 | + | |
| 9 | +// Autograd-aware ops. Forward runs on the active backend (CPU reference or | |
| 10 | +// Metal stream) and, when the tape is enabled and an input requires grad, | |
| 11 | +// records a backward lambda that accumulates into input .grad tensors on | |
| 12 | +// the same backend. | |
| 13 | +// | |
| 14 | +// Metal-mode contract: values live in unified memory but are produced | |
| 15 | +// asynchronously — CPU code must not READ op outputs (incl. the loss | |
| 16 | +// scalar) before metal::sync(), and must not WRITE tensors that pending | |
| 17 | +// dispatches read. The trainer's step structure guarantees both. | |
| 18 | +namespace forge::ops { | |
| 19 | + | |
| 20 | +enum class Backend { CPU, Metal }; | |
| 21 | +void set_backend(Backend b); | |
| 22 | +Backend backend(); | |
| 23 | + | |
| 24 | +// c = a ⋅ b (2-D, optional transposes on the stored operands) | |
| 25 | +Var matmul(const Var& a, const Var& b, bool transpose_a = false, bool transpose_b = false); | |
| 26 | + | |
| 27 | +Var add(const Var& a, const Var& b); // same shape | |
| 28 | +Var add_bias(const Var& x, const Var& bias); // x [N,C] + bias [C] | |
| 29 | +Var mul(const Var& a, const Var& b); // elementwise | |
| 30 | +Var scale(const Var& a, float s); | |
| 31 | + | |
| 32 | +Var silu(const Var& x); | |
| 33 | +Var gelu(const Var& x); | |
| 34 | + | |
| 35 | +Var rmsnorm(const Var& x, const Var& w, float eps); | |
| 36 | +Var layernorm(const Var& x, const Var& w, const Var& b, float eps); | |
| 37 | + | |
| 38 | +// weight [V,C], ids [B,T] (u16/i32) → [B,T,C] | |
| 39 | +Var embedding(const Var& weight, const Tensor& ids); | |
| 40 | + | |
| 41 | +// x [B,T,H*hd], interleaved-pairs RoPE per head | |
| 42 | +Var rope(const Var& x, int64_t n_heads, float theta, int64_t pos_offset = 0); | |
| 43 | + | |
| 44 | +// q [B,T,H*hd], k/v [B,T,Hkv*hd] → [B,T,H*hd]; causal + GQA | |
| 45 | +Var attention(const Var& q, const Var& k, const Var& v, | |
| 46 | + int64_t n_heads, int64_t n_kv_heads, bool causal, float scale); | |
| 47 | + | |
| 48 | +// logits [N,V], targets [N] (i32/u16, ignore_index=-1) → scalar mean loss | |
| 49 | +Var cross_entropy(const Var& logits, const Tensor& targets); | |
| 50 | + | |
| 51 | +// Shape helper: shares storage, no tape node needed (grad shapes follow value shapes). | |
| 52 | +Var reshape(const Var& x, std::vector<int64_t> shape); | |
| 53 | + | |
| 54 | +} // namespace forge::ops | |
added
src/tokenizer/bpe.cpp
+82 −0
@@ -0,0 +1,82 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#include "tokenizer/bpe.h" | |
| 3 | + | |
| 4 | +#include <cstdio> | |
| 5 | +#include <cstdlib> | |
| 6 | +#include <fstream> | |
| 7 | +#include <limits> | |
| 8 | +#include <sstream> | |
| 9 | + | |
| 10 | +namespace forge::tok { | |
| 11 | + | |
| 12 | +namespace { | |
| 13 | +[[noreturn]] void die(const std::string& msg) { | |
| 14 | + std::fprintf(stderr, "forge/tokenizer: %s\n", msg.c_str()); | |
| 15 | + std::abort(); | |
| 16 | +} | |
| 17 | +} // namespace | |
| 18 | + | |
| 19 | +void BPETokenizer::load(const std::string& model_path) { | |
| 20 | + std::ifstream in(model_path); | |
| 21 | + if (!in) die("cannot open " + model_path); | |
| 22 | + std::string header; | |
| 23 | + std::getline(in, header); | |
| 24 | + if (header != "forgebpe v1") die("bad model header in " + model_path); | |
| 25 | + int64_t vocab_size = 0; | |
| 26 | + in >> vocab_size; | |
| 27 | + | |
| 28 | + vocab_.resize(size_t(vocab_size)); | |
| 29 | + for (int i = 0; i < 256 && i < vocab_size; ++i) vocab_[size_t(i)] = std::string(1, char(i)); | |
| 30 | + | |
| 31 | + int32_t id, left, right; | |
| 32 | + while (in >> id >> left >> right) { | |
| 33 | + merges_[{left, right}] = id; | |
| 34 | + vocab_[size_t(id)] = vocab_[size_t(left)] + vocab_[size_t(right)]; | |
| 35 | + } | |
| 36 | + if (int64_t(merges_.size()) != vocab_size - 256) | |
| 37 | + die("merge count does not match vocab size in " + model_path); | |
| 38 | +} | |
| 39 | + | |
| 40 | +std::vector<int32_t> BPETokenizer::encode(const std::string& text) const { | |
| 41 | + std::vector<int32_t> ids; | |
| 42 | + ids.reserve(text.size()); | |
| 43 | + for (unsigned char c : text) ids.push_back(int32_t(c)); | |
| 44 | + | |
| 45 | + // greedy: repeatedly apply the LOWEST-id (earliest-learned) merge present | |
| 46 | + while (ids.size() >= 2) { | |
| 47 | + int32_t best_id = std::numeric_limits<int32_t>::max(); | |
| 48 | + std::pair<int32_t, int32_t> best_pair{-1, -1}; | |
| 49 | + for (size_t i = 0; i + 1 < ids.size(); ++i) { | |
| 50 | + auto it = merges_.find({ids[i], ids[i + 1]}); | |
| 51 | + if (it != merges_.end() && it->second < best_id) { | |
| 52 | + best_id = it->second; | |
| 53 | + best_pair = it->first; | |
| 54 | + } | |
| 55 | + } | |
| 56 | + if (best_pair.first < 0) break; | |
| 57 | + std::vector<int32_t> next; | |
| 58 | + next.reserve(ids.size()); | |
| 59 | + for (size_t i = 0; i < ids.size();) { | |
| 60 | + if (i + 1 < ids.size() && ids[i] == best_pair.first && | |
| 61 | + ids[i + 1] == best_pair.second) { | |
| 62 | + next.push_back(best_id); | |
| 63 | + i += 2; | |
| 64 | + } else { | |
| 65 | + next.push_back(ids[i]); | |
| 66 | + i += 1; | |
| 67 | + } | |
| 68 | + } | |
| 69 | + ids = std::move(next); | |
| 70 | + } | |
| 71 | + return ids; | |
| 72 | +} | |
| 73 | + | |
| 74 | +std::string BPETokenizer::decode(const std::vector<int32_t>& ids) const { | |
| 75 | + std::string out; | |
| 76 | + for (int32_t id : ids) { | |
| 77 | + if (id >= 0 && size_t(id) < vocab_.size()) out += vocab_[size_t(id)]; | |
| 78 | + } | |
| 79 | + return out; | |
| 80 | +} | |
| 81 | + | |
| 82 | +} // namespace forge::tok | |
added
src/tokenizer/bpe.h
+33 −0
@@ -0,0 +1,33 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#pragma once | |
| 3 | + | |
| 4 | +#include <cstdint> | |
| 5 | +#include <map> | |
| 6 | +#include <string> | |
| 7 | +#include <vector> | |
| 8 | + | |
| 9 | +namespace forge::tok { | |
| 10 | + | |
| 11 | +// Byte-level BPE (minbpe BasicTokenizer semantics: no regex pre-split — | |
| 12 | +// fine for small domain vocabs, see RESEARCH.md §7). The .model file is | |
| 13 | +// produced by tools/train_tokenizer.py, whose encoder this must match | |
| 14 | +// exactly: | |
| 15 | +// forgebpe v1\n | |
| 16 | +// <vocab_size>\n | |
| 17 | +// <id> <left> <right>\n (one line per merge, ids from 256 upward) | |
| 18 | +class BPETokenizer { | |
| 19 | +public: | |
| 20 | + void load(const std::string& model_path); | |
| 21 | + | |
| 22 | + std::vector<int32_t> encode(const std::string& text) const; | |
| 23 | + std::string decode(const std::vector<int32_t>& ids) const; | |
| 24 | + | |
| 25 | + int64_t vocab_size() const { return int64_t(vocab_.size()); } | |
| 26 | + | |
| 27 | +private: | |
| 28 | + // merge ranks: (left, right) -> merged id; rank order == id order | |
| 29 | + std::map<std::pair<int32_t, int32_t>, int32_t> merges_; | |
| 30 | + std::vector<std::string> vocab_; // id -> bytes | |
| 31 | +}; | |
| 32 | + | |
| 33 | +} // namespace forge::tok | |
added
src/train/checkpoint.cpp
+158 −0
@@ -0,0 +1,158 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#include "train/checkpoint.h" | |
| 3 | + | |
| 4 | +#include <cstdio> | |
| 5 | +#include <cstdlib> | |
| 6 | +#include <cstring> | |
| 7 | +#include <unordered_map> | |
| 8 | +#include <unordered_set> | |
| 9 | + | |
| 10 | +namespace forge::train { | |
| 11 | + | |
| 12 | +namespace { | |
| 13 | + | |
| 14 | +constexpr uint32_t kMagic = 0x45475246; // "FRGE" | |
| 15 | +constexpr uint32_t kVersion = 1; | |
| 16 | + | |
| 17 | +[[noreturn]] void die(const std::string& msg) { | |
| 18 | + std::fprintf(stderr, "forge/checkpoint: %s\n", msg.c_str()); | |
| 19 | + std::abort(); | |
| 20 | +} | |
| 21 | + | |
| 22 | +void write_bytes(FILE* f, const void* p, size_t n) { | |
| 23 | + if (std::fwrite(p, 1, n, f) != n) die("write failed"); | |
| 24 | +} | |
| 25 | +void read_bytes(FILE* f, void* p, size_t n) { | |
| 26 | + if (std::fread(p, 1, n, f) != n) die("read failed (truncated checkpoint?)"); | |
| 27 | +} | |
| 28 | +template <typename T> | |
| 29 | +void write_pod(FILE* f, T v) { write_bytes(f, &v, sizeof(T)); } | |
| 30 | +template <typename T> | |
| 31 | +T read_pod(FILE* f) { | |
| 32 | + T v; | |
| 33 | + read_bytes(f, &v, sizeof(T)); | |
| 34 | + return v; | |
| 35 | +} | |
| 36 | +void write_str(FILE* f, const std::string& s) { | |
| 37 | + write_pod<uint32_t>(f, uint32_t(s.size())); | |
| 38 | + write_bytes(f, s.data(), s.size()); | |
| 39 | +} | |
| 40 | +std::string read_str(FILE* f) { | |
| 41 | + const uint32_t n = read_pod<uint32_t>(f); | |
| 42 | + std::string s(n, '\0'); | |
| 43 | + read_bytes(f, s.data(), n); | |
| 44 | + return s; | |
| 45 | +} | |
| 46 | + | |
| 47 | +// Tied params appear under several names; store each storage once (first | |
| 48 | +// name wins) and load by name into whichever Var carries it. | |
| 49 | +std::vector<std::pair<std::string, Var>> dedupe( | |
| 50 | + const std::vector<std::pair<std::string, Var>>& named) { | |
| 51 | + std::vector<std::pair<std::string, Var>> out; | |
| 52 | + std::unordered_set<const void*> seen; | |
| 53 | + for (const auto& [name, p] : named) | |
| 54 | + if (seen.insert(p.id()).second) out.emplace_back(name, p); | |
| 55 | + return out; | |
| 56 | +} | |
| 57 | + | |
| 58 | +} // namespace | |
| 59 | + | |
| 60 | +void save_checkpoint(const std::string& path, | |
| 61 | + const std::vector<std::pair<std::string, Var>>& named_params, | |
| 62 | + AdamW* opt, const CheckpointData& meta) { | |
| 63 | + const std::string tmp = path + ".tmp"; | |
| 64 | + FILE* f = std::fopen(tmp.c_str(), "wb"); | |
| 65 | + if (!f) die("cannot open " + tmp); | |
| 66 | + | |
| 67 | + write_pod<uint32_t>(f, kMagic); | |
| 68 | + write_pod<uint32_t>(f, kVersion); | |
| 69 | + write_pod<int64_t>(f, meta.step); | |
| 70 | + write_pod<int64_t>(f, opt ? opt->t() : 0); | |
| 71 | + write_pod<uint64_t>(f, meta.rng_state); | |
| 72 | + write_str(f, meta.config_json); | |
| 73 | + | |
| 74 | + const auto params = dedupe(named_params); | |
| 75 | + write_pod<uint32_t>(f, uint32_t(params.size())); | |
| 76 | + for (const auto& [name, p] : params) { | |
| 77 | + write_str(f, name); | |
| 78 | + const auto& shape = p.value().shape(); | |
| 79 | + write_pod<uint32_t>(f, uint32_t(shape.size())); | |
| 80 | + for (int64_t d : shape) write_pod<int64_t>(f, d); | |
| 81 | + write_bytes(f, p.value().raw(), p.value().nbytes()); | |
| 82 | + } | |
| 83 | + | |
| 84 | + const bool has_opt = opt && !opt->m().empty(); | |
| 85 | + write_pod<uint32_t>(f, has_opt ? uint32_t(opt->m().size()) : 0u); | |
| 86 | + if (has_opt) { | |
| 87 | + for (size_t i = 0; i < opt->m().size(); ++i) { | |
| 88 | + write_bytes(f, opt->m()[i].raw(), opt->m()[i].nbytes()); | |
| 89 | + write_bytes(f, opt->v()[i].raw(), opt->v()[i].nbytes()); | |
| 90 | + } | |
| 91 | + } | |
| 92 | + | |
| 93 | + std::fclose(f); | |
| 94 | + if (std::rename(tmp.c_str(), path.c_str()) != 0) die("rename failed: " + path); | |
| 95 | +} | |
| 96 | + | |
| 97 | +CheckpointData load_checkpoint(const std::string& path, | |
| 98 | + const std::vector<std::pair<std::string, Var>>& named_params, | |
| 99 | + AdamW* opt) { | |
| 100 | + FILE* f = std::fopen(path.c_str(), "rb"); | |
| 101 | + if (!f) die("cannot open " + path); | |
| 102 | + if (read_pod<uint32_t>(f) != kMagic) die("bad magic: " + path); | |
| 103 | + if (read_pod<uint32_t>(f) != kVersion) die("unsupported version: " + path); | |
| 104 | + | |
| 105 | + CheckpointData meta; | |
| 106 | + meta.step = read_pod<int64_t>(f); | |
| 107 | + const int64_t opt_t = read_pod<int64_t>(f); | |
| 108 | + meta.rng_state = read_pod<uint64_t>(f); | |
| 109 | + meta.config_json = read_str(f); | |
| 110 | + | |
| 111 | + std::unordered_map<std::string, Var> by_name; | |
| 112 | + for (const auto& [name, p] : dedupe(named_params)) by_name.emplace(name, p); | |
| 113 | + | |
| 114 | + const uint32_t n_params = read_pod<uint32_t>(f); | |
| 115 | + if (n_params != by_name.size()) die("parameter count mismatch"); | |
| 116 | + for (uint32_t i = 0; i < n_params; ++i) { | |
| 117 | + const std::string name = read_str(f); | |
| 118 | + const uint32_t ndim = read_pod<uint32_t>(f); | |
| 119 | + std::vector<int64_t> shape(ndim); | |
| 120 | + for (auto& d : shape) d = read_pod<int64_t>(f); | |
| 121 | + auto it = by_name.find(name); | |
| 122 | + if (it == by_name.end()) die("unknown parameter in checkpoint: " + name); | |
| 123 | + if (it->second.value().shape() != shape) die("shape mismatch: " + name); | |
| 124 | + read_bytes(f, it->second.value().raw(), it->second.value().nbytes()); | |
| 125 | + } | |
| 126 | + | |
| 127 | + const uint32_t n_opt = read_pod<uint32_t>(f); | |
| 128 | + if (n_opt > 0 && opt) { | |
| 129 | + opt->ensure_state(); | |
| 130 | + if (opt->m().size() != n_opt) die("optimizer state count mismatch"); | |
| 131 | + for (uint32_t i = 0; i < n_opt; ++i) { | |
| 132 | + read_bytes(f, opt->m()[i].raw(), opt->m()[i].nbytes()); | |
| 133 | + read_bytes(f, opt->v()[i].raw(), opt->v()[i].nbytes()); | |
| 134 | + } | |
| 135 | + opt->set_t(opt_t); | |
| 136 | + } else if (n_opt > 0) { | |
| 137 | + // skip optimizer state | |
| 138 | + std::fseek(f, 0, SEEK_END); | |
| 139 | + } | |
| 140 | + | |
| 141 | + std::fclose(f); | |
| 142 | + return meta; | |
| 143 | +} | |
| 144 | + | |
| 145 | +std::string read_checkpoint_config(const std::string& path) { | |
| 146 | + FILE* f = std::fopen(path.c_str(), "rb"); | |
| 147 | + if (!f) die("cannot open " + path); | |
| 148 | + if (read_pod<uint32_t>(f) != kMagic) die("bad magic: " + path); | |
| 149 | + if (read_pod<uint32_t>(f) != kVersion) die("unsupported version: " + path); | |
| 150 | + read_pod<int64_t>(f); | |
| 151 | + read_pod<int64_t>(f); | |
| 152 | + read_pod<uint64_t>(f); | |
| 153 | + const std::string cfg = read_str(f); | |
| 154 | + std::fclose(f); | |
| 155 | + return cfg; | |
| 156 | +} | |
| 157 | + | |
| 158 | +} // namespace forge::train | |
added
src/train/checkpoint.h
+41 −0
@@ -0,0 +1,41 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#pragma once | |
| 3 | + | |
| 4 | +#include "core/autograd.h" | |
| 5 | +#include "train/optimizer.h" | |
| 6 | + | |
| 7 | +#include <cstdint> | |
| 8 | +#include <string> | |
| 9 | +#include <utility> | |
| 10 | +#include <vector> | |
| 11 | + | |
| 12 | +namespace forge::train { | |
| 13 | + | |
| 14 | +// Binary checkpoint: model params (by name), optimizer moments + t, step, | |
| 15 | +// RNG state, and the config JSON — everything needed for exact resume | |
| 16 | +// (CLAUDE.md training-loop requirements). Format: | |
| 17 | +// magic "FRGE" u32 | version u32 | step i64 | opt_t i64 | rng u64 | |
| 18 | +// | config_json (u32 len + bytes) | |
| 19 | +// | n_params u32 | per param: name (u32+bytes), ndim u32, dims i64[], | |
| 20 | +// f32 data | |
| 21 | +// | n_opt u32 | per tracked param: m f32[], v f32[] (0 if never stepped) | |
| 22 | +struct CheckpointData { | |
| 23 | + int64_t step = 0; | |
| 24 | + uint64_t rng_state = 0; | |
| 25 | + std::string config_json; | |
| 26 | +}; | |
| 27 | + | |
| 28 | +void save_checkpoint(const std::string& path, | |
| 29 | + const std::vector<std::pair<std::string, Var>>& named_params, | |
| 30 | + AdamW* opt, const CheckpointData& meta); | |
| 31 | + | |
| 32 | +// Loads params by NAME into an already-constructed model (shape-checked). | |
| 33 | +// opt may be null (inference). Returns the stored metadata. | |
| 34 | +CheckpointData load_checkpoint(const std::string& path, | |
| 35 | + const std::vector<std::pair<std::string, Var>>& named_params, | |
| 36 | + AdamW* opt); | |
| 37 | + | |
| 38 | +// Reads just the config JSON (to construct the model before loading). | |
| 39 | +std::string read_checkpoint_config(const std::string& path); | |
| 40 | + | |
| 41 | +} // namespace forge::train | |
added
src/train/dataloader.cpp
+80 −0
@@ -0,0 +1,80 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#include "train/dataloader.h" | |
| 3 | + | |
| 4 | +#include <fcntl.h> | |
| 5 | +#include <sys/mman.h> | |
| 6 | +#include <sys/stat.h> | |
| 7 | +#include <unistd.h> | |
| 8 | + | |
| 9 | +#include <cstdio> | |
| 10 | +#include <cstdlib> | |
| 11 | +#include <cstring> | |
| 12 | + | |
| 13 | +namespace forge::train { | |
| 14 | + | |
| 15 | +namespace { | |
| 16 | +constexpr int32_t kMagic = 20240520; | |
| 17 | +constexpr size_t kHeaderInts = 256; | |
| 18 | + | |
| 19 | +[[noreturn]] void die(const std::string& msg) { | |
| 20 | + std::fprintf(stderr, "forge/data: %s\n", msg.c_str()); | |
| 21 | + std::abort(); | |
| 22 | +} | |
| 23 | +} // namespace | |
| 24 | + | |
| 25 | +DataLoader::DataLoader(const std::string& bin_path, int64_t context_length, uint64_t seed) | |
| 26 | + : context_(context_length), rng_(seed) { | |
| 27 | + const int fd = ::open(bin_path.c_str(), O_RDONLY); | |
| 28 | + if (fd < 0) die("cannot open " + bin_path); | |
| 29 | + struct stat st{}; | |
| 30 | + if (fstat(fd, &st) != 0) die("fstat failed on " + bin_path); | |
| 31 | + map_len_ = size_t(st.st_size); | |
| 32 | + map_ = mmap(nullptr, map_len_, PROT_READ, MAP_PRIVATE, fd, 0); | |
| 33 | + ::close(fd); | |
| 34 | + if (map_ == MAP_FAILED) die("mmap failed on " + bin_path); | |
| 35 | + madvise(map_, map_len_, MADV_RANDOM); | |
| 36 | + | |
| 37 | + const int32_t* header = static_cast<const int32_t*>(map_); | |
| 38 | + if (map_len_ >= kHeaderInts * 4 && header[0] == kMagic && header[1] == 1) { | |
| 39 | + num_tokens_ = header[2]; | |
| 40 | + tokens_ = reinterpret_cast<const uint16_t*>(header + kHeaderInts); | |
| 41 | + if (size_t(num_tokens_) * 2 + kHeaderInts * 4 > map_len_) | |
| 42 | + die("header token count exceeds file size: " + bin_path); | |
| 43 | + } else { | |
| 44 | + // headerless: the whole file is uint16 tokens | |
| 45 | + num_tokens_ = int64_t(map_len_ / 2); | |
| 46 | + tokens_ = static_cast<const uint16_t*>(map_); | |
| 47 | + } | |
| 48 | + if (num_tokens_ < context_ + 1) | |
| 49 | + die("dataset smaller than one context window: " + bin_path); | |
| 50 | +} | |
| 51 | + | |
| 52 | +DataLoader::~DataLoader() { | |
| 53 | + if (map_ && map_ != MAP_FAILED) munmap(map_, map_len_); | |
| 54 | +} | |
| 55 | + | |
| 56 | +void DataLoader::fill(int64_t start, int64_t T, int32_t* ids_row, int32_t* tgt_row) const { | |
| 57 | + for (int64_t t = 0; t < T; ++t) { | |
| 58 | + ids_row[t] = int32_t(tokens_[start + t]); | |
| 59 | + tgt_row[t] = int32_t(tokens_[start + t + 1]); | |
| 60 | + } | |
| 61 | +} | |
| 62 | + | |
| 63 | +void DataLoader::next_batch(Tensor& ids, Tensor& targets) { | |
| 64 | + const int64_t B = ids.size(0), T = ids.size(1); | |
| 65 | + std::uniform_int_distribution<int64_t> dist(0, num_tokens_ - T - 1); | |
| 66 | + for (int64_t b = 0; b < B; ++b) { | |
| 67 | + fill(dist(rng_), T, ids.data<int32_t>() + b * T, targets.data<int32_t>() + b * T); | |
| 68 | + } | |
| 69 | +} | |
| 70 | + | |
| 71 | +void DataLoader::seq_batch(int64_t index, Tensor& ids, Tensor& targets) const { | |
| 72 | + const int64_t B = ids.size(0), T = ids.size(1); | |
| 73 | + const int64_t span = num_tokens_ - T - 1; | |
| 74 | + for (int64_t b = 0; b < B; ++b) { | |
| 75 | + const int64_t start = ((index * B + b) * T) % span; | |
| 76 | + fill(start, T, ids.data<int32_t>() + b * T, targets.data<int32_t>() + b * T); | |
| 77 | + } | |
| 78 | +} | |
| 79 | + | |
| 80 | +} // namespace forge::train | |
added
src/train/dataloader.h
+42 −0
@@ -0,0 +1,42 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#pragma once | |
| 3 | + | |
| 4 | +#include "core/tensor.h" | |
| 5 | + | |
| 6 | +#include <cstdint> | |
| 7 | +#include <random> | |
| 8 | +#include <string> | |
| 9 | + | |
| 10 | +namespace forge::train { | |
| 11 | + | |
| 12 | +// mmap'd uint16 token stream (llm.c format: 256 int32 header {20240520, 1, | |
| 13 | +// num_tokens}; headerless nanoGPT-style .bin also accepted). Batches are | |
| 14 | +// random contiguous windows of context_length+1; ids/targets are written | |
| 15 | +// into caller tensors as i32 (what the GPU kernels take). | |
| 16 | +class DataLoader { | |
| 17 | +public: | |
| 18 | + DataLoader(const std::string& bin_path, int64_t context_length, uint64_t seed); | |
| 19 | + ~DataLoader(); | |
| 20 | + | |
| 21 | + DataLoader(const DataLoader&) = delete; | |
| 22 | + DataLoader& operator=(const DataLoader&) = delete; | |
| 23 | + | |
| 24 | + // ids: [B, T] i32, targets: [B*T] i32 (targets = ids shifted by one) | |
| 25 | + void next_batch(Tensor& ids, Tensor& targets); | |
| 26 | + // Deterministic sequential window (evaluation); wraps around. | |
| 27 | + void seq_batch(int64_t index, Tensor& ids, Tensor& targets) const; | |
| 28 | + | |
| 29 | + int64_t num_tokens() const { return num_tokens_; } | |
| 30 | + | |
| 31 | +private: | |
| 32 | + void fill(int64_t start, int64_t T, int32_t* ids_row, int32_t* tgt_row) const; | |
| 33 | + | |
| 34 | + const uint16_t* tokens_ = nullptr; // into the mmap | |
| 35 | + void* map_ = nullptr; | |
| 36 | + size_t map_len_ = 0; | |
| 37 | + int64_t num_tokens_ = 0; | |
| 38 | + int64_t context_ = 0; | |
| 39 | + std::mt19937_64 rng_; | |
| 40 | +}; | |
| 41 | + | |
| 42 | +} // namespace forge::train | |
added
src/train/optimizer.cpp
+124 −0
@@ -0,0 +1,124 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#include "train/optimizer.h" | |
| 3 | + | |
| 4 | +#include "ops/metal/metal_ops.h" | |
| 5 | +#include "ops/ops.h" | |
| 6 | + | |
| 7 | +#include <cmath> | |
| 8 | +#include <unordered_set> | |
| 9 | + | |
| 10 | +namespace forge::train { | |
| 11 | + | |
| 12 | +AdamW::AdamW(const std::vector<std::pair<std::string, Var>>& named_params, Options opts) | |
| 13 | + : opts_(opts) { | |
| 14 | + std::unordered_set<const void*> seen; | |
| 15 | + for (const auto& [name, p] : named_params) { | |
| 16 | + if (!p.defined() || !p.requires_grad()) continue; | |
| 17 | + if (!seen.insert(p.id()).second) continue; // tied param, already tracked | |
| 18 | + params_.push_back(p); | |
| 19 | + decay_.push_back(p.value().ndim() >= 2); | |
| 20 | + } | |
| 21 | +} | |
| 22 | + | |
| 23 | +float AdamW::clip_global_norm(float max_norm) { | |
| 24 | + double sq = 0.0; | |
| 25 | + for (const Var& p : params_) { | |
| 26 | + if (!p.has_grad()) continue; | |
| 27 | + const float* g = p.grad().data<float>(); | |
| 28 | + for (int64_t i = 0; i < p.grad().numel(); ++i) sq += double(g[i]) * double(g[i]); | |
| 29 | + } | |
| 30 | + const float norm = float(std::sqrt(sq)); | |
| 31 | + if (max_norm > 0.0f && norm > max_norm) { | |
| 32 | + const float s = max_norm / norm; | |
| 33 | + for (const Var& p : params_) { | |
| 34 | + if (!p.has_grad()) continue; | |
| 35 | + float* g = p.grad().data<float>(); | |
| 36 | + for (int64_t i = 0; i < p.grad().numel(); ++i) g[i] *= s; | |
| 37 | + } | |
| 38 | + } | |
| 39 | + return norm; | |
| 40 | +} | |
| 41 | + | |
| 42 | +void AdamW::ensure_state() { | |
| 43 | + if (!m_.empty()) return; | |
| 44 | + m_.reserve(params_.size()); | |
| 45 | + v_.reserve(params_.size()); | |
| 46 | + for (const Var& p : params_) { | |
| 47 | + m_.push_back(Tensor::zeros(p.value().shape())); | |
| 48 | + v_.push_back(Tensor::zeros(p.value().shape())); | |
| 49 | + } | |
| 50 | +} | |
| 51 | + | |
| 52 | +float AdamW::step_with_clip(float lr, float max_norm) { | |
| 53 | + if (ops::backend() == ops::Backend::CPU) { | |
| 54 | + const float norm = clip_global_norm(max_norm); | |
| 55 | + step(lr); | |
| 56 | + return norm; | |
| 57 | + } | |
| 58 | + | |
| 59 | + // Metal path. Encode per-tensor sum-of-squares into one partials buffer, | |
| 60 | + // then sync — this is also the boundary where the step's loss becomes | |
| 61 | + // readable. | |
| 62 | + // These are hundreds of independent single-threadgroup reductions (one per | |
| 63 | + // parameter tensor), which a serial encoder pointlessly serializes. | |
| 64 | + Tensor partials = Tensor::empty({int64_t(params_.size())}); | |
| 65 | + { | |
| 66 | + metal::ConcurrentRegion region; | |
| 67 | + for (size_t i = 0; i < params_.size(); ++i) { | |
| 68 | + Tensor slot = partials.slice0(int64_t(i), 1); | |
| 69 | + metal::sumsq(params_[i].grad(), slot); | |
| 70 | + } | |
| 71 | + } | |
| 72 | + metal::sync(); | |
| 73 | + | |
| 74 | + double sq = 0.0; | |
| 75 | + for (size_t i = 0; i < params_.size(); ++i) sq += double(partials.data<float>()[i]); | |
| 76 | + const float norm = float(std::sqrt(sq)); | |
| 77 | + const float grad_scale = | |
| 78 | + (max_norm > 0.0f && norm > max_norm) ? max_norm / norm : 1.0f; | |
| 79 | + | |
| 80 | + ensure_state(); // CPU-side zeros are safe here: the stream is closed | |
| 81 | + ++t_; | |
| 82 | + { | |
| 83 | + // Each parameter's update touches only its own w/g/m/v — independent. | |
| 84 | + metal::ConcurrentRegion region; | |
| 85 | + for (size_t i = 0; i < params_.size(); ++i) { | |
| 86 | + const float wd = decay_[i] ? opts_.weight_decay : 0.0f; | |
| 87 | + metal::adamw_step(params_[i].value(), params_[i].grad(), m_[i], v_[i], lr, | |
| 88 | + opts_.beta1, opts_.beta2, t_, opts_.eps, wd, grad_scale); | |
| 89 | + } | |
| 90 | + } | |
| 91 | + metal::sync(); // weights final before the next step's CPU-side zero_grad | |
| 92 | + return norm; | |
| 93 | +} | |
| 94 | + | |
| 95 | +void AdamW::step(float lr) { | |
| 96 | + ensure_state(); | |
| 97 | + ++t_; // 1-based: bias correction divides by (1 - beta^t) | |
| 98 | + const float bc1 = 1.0f - std::pow(opts_.beta1, float(t_)); | |
| 99 | + const float bc2 = 1.0f - std::pow(opts_.beta2, float(t_)); | |
| 100 | + | |
| 101 | + for (size_t pi = 0; pi < params_.size(); ++pi) { | |
| 102 | + Var& p = params_[pi]; | |
| 103 | + if (!p.has_grad()) continue; | |
| 104 | + const float wd = decay_[pi] ? opts_.weight_decay : 0.0f; | |
| 105 | + float* w = p.value().data<float>(); | |
| 106 | + const float* g = p.grad().data<float>(); | |
| 107 | + float* m = m_[pi].data<float>(); | |
| 108 | + float* v = v_[pi].data<float>(); | |
| 109 | + const int64_t n = p.value().numel(); | |
| 110 | + for (int64_t i = 0; i < n; ++i) { | |
| 111 | + m[i] = opts_.beta1 * m[i] + (1.0f - opts_.beta1) * g[i]; | |
| 112 | + v[i] = opts_.beta2 * v[i] + (1.0f - opts_.beta2) * g[i] * g[i]; | |
| 113 | + const float mhat = m[i] / bc1; | |
| 114 | + const float vhat = v[i] / bc2; | |
| 115 | + w[i] -= lr * (mhat / (std::sqrt(vhat) + opts_.eps) + wd * w[i]); | |
| 116 | + } | |
| 117 | + } | |
| 118 | +} | |
| 119 | + | |
| 120 | +void AdamW::zero_grad() { | |
| 121 | + for (const Var& p : params_) p.zero_grad(); | |
| 122 | +} | |
| 123 | + | |
| 124 | +} // namespace forge::train | |
added
src/train/optimizer.h
+58 −0
@@ -0,0 +1,58 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#pragma once | |
| 3 | + | |
| 4 | +#include "core/autograd.h" | |
| 5 | + | |
| 6 | +#include <string> | |
| 7 | +#include <utility> | |
| 8 | +#include <vector> | |
| 9 | + | |
| 10 | +namespace forge::train { | |
| 11 | + | |
| 12 | +// AdamW with decoupled weight decay (llm.c/PyTorch convention: eps outside | |
| 13 | +// sqrt, wd only on dim>=2 params, decay folded into the same update). | |
| 14 | +// Deduplicates tied parameters by Var::id(). | |
| 15 | +class AdamW { | |
| 16 | +public: | |
| 17 | + struct Options { | |
| 18 | + float beta1 = 0.9f; | |
| 19 | + float beta2 = 0.95f; | |
| 20 | + float eps = 1e-8f; | |
| 21 | + float weight_decay = 0.1f; | |
| 22 | + }; | |
| 23 | + | |
| 24 | + AdamW(const std::vector<std::pair<std::string, Var>>& named_params, Options opts); | |
| 25 | + | |
| 26 | + // Global-norm gradient clip: returns the pre-clip norm and scales all | |
| 27 | + // grads by min(1, max_norm/norm). No-op when max_norm <= 0. (CPU path.) | |
| 28 | + float clip_global_norm(float max_norm); | |
| 29 | + | |
| 30 | + void step(float lr); | |
| 31 | + void zero_grad(); | |
| 32 | + | |
| 33 | + // Backend-routed step: clip + AdamW in one call, returns the pre-clip | |
| 34 | + // grad norm. On Metal it expects backward() already encoded: it encodes | |
| 35 | + // per-tensor sumsq, syncs (this is the step's loss-readback boundary), | |
| 36 | + // folds the clip factor into the fused adamw kernel's grad_scale, and | |
| 37 | + // syncs again after the update. | |
| 38 | + float step_with_clip(float lr, float max_norm); | |
| 39 | + | |
| 40 | + // Allocate m/v now (checkpoint loading needs the buffers to exist). | |
| 41 | + void ensure_state(); | |
| 42 | + | |
| 43 | + int64_t t() const { return t_; } | |
| 44 | + // Checkpoint access (M4): moments in parameter order. | |
| 45 | + std::vector<Tensor>& m() { return m_; } | |
| 46 | + std::vector<Tensor>& v() { return v_; } | |
| 47 | + void set_t(int64_t t) { t_ = t; } | |
| 48 | + const std::vector<Var>& params() const { return params_; } | |
| 49 | + | |
| 50 | +private: | |
| 51 | + Options opts_; | |
| 52 | + std::vector<Var> params_; | |
| 53 | + std::vector<bool> decay_; // dim >= 2 | |
| 54 | + std::vector<Tensor> m_, v_; // f32, allocated lazily at first step | |
| 55 | + int64_t t_ = 0; | |
| 56 | +}; | |
| 57 | + | |
| 58 | +} // namespace forge::train | |
added
src/train/scheduler.h
+20 −0
@@ -0,0 +1,20 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#pragma once | |
| 3 | + | |
| 4 | +#include <cmath> | |
| 5 | +#include <cstdint> | |
| 6 | + | |
| 7 | +namespace forge::train { | |
| 8 | + | |
| 9 | +// Linear warmup → cosine decay to min_lr (nanoGPT schedule; (it+1)/(warmup+1) | |
| 10 | +// avoids a zero-lr first step). | |
| 11 | +inline float lr_at(int64_t step, float max_lr, float min_lr, int64_t warmup_steps, | |
| 12 | + int64_t decay_steps) { | |
| 13 | + if (step < warmup_steps) return max_lr * float(step + 1) / float(warmup_steps + 1); | |
| 14 | + if (step >= decay_steps) return min_lr; | |
| 15 | + const float ratio = float(step - warmup_steps) / float(decay_steps - warmup_steps); | |
| 16 | + const float coeff = 0.5f * (1.0f + std::cos(float(M_PI) * ratio)); | |
| 17 | + return min_lr + coeff * (max_lr - min_lr); | |
| 18 | +} | |
| 19 | + | |
| 20 | +} // namespace forge::train | |
added
src/train/trainer.cpp
+157 −0
@@ -0,0 +1,157 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#include "train/trainer.h" | |
| 3 | + | |
| 4 | +#include "ops/metal/metal_ops.h" | |
| 5 | +#include "ops/ops.h" | |
| 6 | +#include "train/checkpoint.h" | |
| 7 | +#include "train/scheduler.h" | |
| 8 | + | |
| 9 | +#include <Foundation/Foundation.hpp> | |
| 10 | + | |
| 11 | +#include <chrono> | |
| 12 | +#include <cstdio> | |
| 13 | +#include <filesystem> | |
| 14 | + | |
| 15 | +namespace forge::train { | |
| 16 | + | |
| 17 | +Trainer::Trainer(Config cfg, const std::string& data_dir, const std::string& out_dir, | |
| 18 | + const std::string& config_json) | |
| 19 | + : cfg_(cfg), out_dir_(out_dir), config_json_(config_json) { | |
| 20 | + std::filesystem::create_directories(out_dir_); | |
| 21 | + model_ = std::make_unique<nn::Transformer>(cfg_.model, cfg_.train.seed); | |
| 22 | + | |
| 23 | + AdamW::Options opts; | |
| 24 | + opts.beta1 = cfg_.train.beta1; | |
| 25 | + opts.beta2 = cfg_.train.beta2; | |
| 26 | + opts.eps = cfg_.train.eps; | |
| 27 | + opts.weight_decay = cfg_.train.weight_decay; | |
| 28 | + opt_ = std::make_unique<AdamW>(model_->named_parameters(), opts); | |
| 29 | + | |
| 30 | + train_data_ = std::make_unique<DataLoader>(data_dir + "/train.bin", | |
| 31 | + cfg_.model.context_length, | |
| 32 | + cfg_.train.seed); | |
| 33 | + const std::string val_path = data_dir + "/val.bin"; | |
| 34 | + if (std::filesystem::exists(val_path)) { | |
| 35 | + val_data_ = std::make_unique<DataLoader>(val_path, cfg_.model.context_length, | |
| 36 | + cfg_.train.seed + 1); | |
| 37 | + } | |
| 38 | +} | |
| 39 | + | |
| 40 | +float Trainer::eval_loss() { | |
| 41 | + if (!val_data_) return -1.0f; | |
| 42 | + NoGrad ng; | |
| 43 | + const int64_t B = cfg_.train.batch_size, T = cfg_.model.context_length; | |
| 44 | + double total = 0.0; | |
| 45 | + for (int64_t i = 0; i < cfg_.train.eval_batches; ++i) { | |
| 46 | + Tensor ids = Tensor::empty({B, T}, DType::I32); | |
| 47 | + Tensor targets = Tensor::empty({B * T}, DType::I32); | |
| 48 | + val_data_->seq_batch(i, ids, targets); | |
| 49 | + Var loss = model_->loss(ids, targets); | |
| 50 | + if (ops::backend() == ops::Backend::Metal) metal::sync(); | |
| 51 | + total += double(loss.value().data<float>()[0]); | |
| 52 | + } | |
| 53 | + return float(total / double(cfg_.train.eval_batches)); | |
| 54 | +} | |
| 55 | + | |
| 56 | +void Trainer::save(int64_t step) { | |
| 57 | + CheckpointData meta; | |
| 58 | + meta.step = step; | |
| 59 | + meta.rng_state = uint64_t(step); // dataloader reseeded from step on resume | |
| 60 | + meta.config_json = config_json_; | |
| 61 | + char name[64]; | |
| 62 | + std::snprintf(name, sizeof(name), "/ckpt_%06lld.bin", static_cast<long long>(step)); | |
| 63 | + save_checkpoint(out_dir_ + name, model_->named_parameters(), opt_.get(), meta); | |
| 64 | + save_checkpoint(out_dir_ + "/ckpt_latest.bin", model_->named_parameters(), opt_.get(), | |
| 65 | + meta); | |
| 66 | + std::printf("checkpoint saved: %s\n", (out_dir_ + name).c_str()); | |
| 67 | +} | |
| 68 | + | |
| 69 | +void Trainer::train(const std::string& resume_from) { | |
| 70 | + const auto& tc = cfg_.train; | |
| 71 | + const int64_t B = tc.batch_size, T = cfg_.model.context_length; | |
| 72 | + | |
| 73 | + int64_t start_step = 0; | |
| 74 | + if (!resume_from.empty()) { | |
| 75 | + CheckpointData meta = | |
| 76 | + load_checkpoint(resume_from, model_->named_parameters(), opt_.get()); | |
| 77 | + start_step = meta.step; | |
| 78 | + std::printf("resumed from %s at step %lld\n", resume_from.c_str(), | |
| 79 | + static_cast<long long>(start_step)); | |
| 80 | + } | |
| 81 | + | |
| 82 | + FILE* csv = std::fopen((out_dir_ + "/log.csv").c_str(), | |
| 83 | + start_step > 0 ? "ab" : "wb"); | |
| 84 | + if (csv && start_step == 0) | |
| 85 | + std::fprintf(csv, "step,loss,lr,grad_norm,tokens_per_sec,val_loss\n"); | |
| 86 | + | |
| 87 | + const int64_t min_lr_steps = tc.max_steps; | |
| 88 | + const float min_lr = tc.lr * tc.min_lr_ratio; | |
| 89 | + const int64_t tokens_per_step = B * T * tc.grad_accum_steps; | |
| 90 | + | |
| 91 | + std::printf("training %s: %lld params, %lld steps, %lld tokens/step, backend=%s\n", | |
| 92 | + cfg_.model.name.c_str(), | |
| 93 | + static_cast<long long>(cfg_.model.num_params()), | |
| 94 | + static_cast<long long>(tc.max_steps), | |
| 95 | + static_cast<long long>(tokens_per_step), | |
| 96 | + ops::backend() == ops::Backend::Metal ? "metal" : "cpu"); | |
| 97 | + | |
| 98 | + for (int64_t step = start_step; step < tc.max_steps; ++step) { | |
| 99 | + NS::AutoreleasePool* pool = NS::AutoreleasePool::alloc()->init(); | |
| 100 | + const auto t0 = std::chrono::steady_clock::now(); | |
| 101 | + const float lr = lr_at(step, tc.lr, min_lr, tc.warmup_steps, min_lr_steps); | |
| 102 | + | |
| 103 | + opt_->zero_grad(); | |
| 104 | + const bool on_gpu = ops::backend() == ops::Backend::Metal; | |
| 105 | + double loss_val = 0.0; | |
| 106 | + for (int64_t micro = 0; micro < tc.grad_accum_steps; ++micro) { | |
| 107 | + Tensor ids = Tensor::empty({B, T}, DType::I32); | |
| 108 | + Tensor targets = Tensor::empty({B * T}, DType::I32); | |
| 109 | + train_data_->next_batch(ids, targets); | |
| 110 | + Var loss = model_->loss(ids, targets); | |
| 111 | + // scale so accumulated grads average over micro-batches | |
| 112 | + Var scaled = ops::scale(loss, 1.0f / float(tc.grad_accum_steps)); | |
| 113 | + Tape::get().backward(scaled); | |
| 114 | + // Sync per micro-batch, not per step: pooled buffers freed while a | |
| 115 | + // command buffer is open sit on the allocator's retire list until | |
| 116 | + // the next sync, so without this every micro-batch's activations | |
| 117 | + // stay resident and peak memory scales with grad_accum_steps. | |
| 118 | + if (on_gpu) metal::sync(); | |
| 119 | + loss_val += double(loss.value().data<float>()[0]) / | |
| 120 | + double(tc.grad_accum_steps); | |
| 121 | + } | |
| 122 | + | |
| 123 | + const float grad_norm = opt_->step_with_clip(lr, tc.grad_clip); | |
| 124 | + | |
| 125 | + const auto t1 = std::chrono::steady_clock::now(); | |
| 126 | + const double dt = std::chrono::duration<double>(t1 - t0).count(); | |
| 127 | + const double tps = double(tokens_per_step) / dt; | |
| 128 | + | |
| 129 | + float val = -1.0f; | |
| 130 | + if (tc.eval_every > 0 && (step + 1) % tc.eval_every == 0) val = eval_loss(); | |
| 131 | + | |
| 132 | + if (step < 10 || step % 10 == 0 || val >= 0.0f) { | |
| 133 | + std::printf("step %6lld | loss %.4f | lr %.2e | gnorm %.3f | %.0f tok/s", | |
| 134 | + static_cast<long long>(step), loss_val, double(lr), | |
| 135 | + double(grad_norm), tps); | |
| 136 | + if (val >= 0.0f) std::printf(" | val %.4f", double(val)); | |
| 137 | + std::printf("\n"); | |
| 138 | + std::fflush(stdout); | |
| 139 | + } | |
| 140 | + if (csv) { | |
| 141 | + std::fprintf(csv, "%lld,%.6f,%.6e,%.6f,%.1f,%.6f\n", | |
| 142 | + static_cast<long long>(step), loss_val, double(lr), | |
| 143 | + double(grad_norm), tps, double(val)); | |
| 144 | + std::fflush(csv); | |
| 145 | + } | |
| 146 | + | |
| 147 | + if (tc.checkpoint_every > 0 && (step + 1) % tc.checkpoint_every == 0) | |
| 148 | + save(step + 1); | |
| 149 | + | |
| 150 | + pool->drain(); | |
| 151 | + } | |
| 152 | + | |
| 153 | + save(tc.max_steps); | |
| 154 | + if (csv) std::fclose(csv); | |
| 155 | +} | |
| 156 | + | |
| 157 | +} // namespace forge::train | |
added
src/train/trainer.h
+38 −0
@@ -0,0 +1,38 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#pragma once | |
| 3 | + | |
| 4 | +#include "nn/config.h" | |
| 5 | +#include "nn/transformer.h" | |
| 6 | +#include "train/dataloader.h" | |
| 7 | +#include "train/optimizer.h" | |
| 8 | + | |
| 9 | +#include <memory> | |
| 10 | +#include <string> | |
| 11 | + | |
| 12 | +namespace forge::train { | |
| 13 | + | |
| 14 | +// Training loop: gradient accumulation, warmup+cosine LR, global-norm clip, | |
| 15 | +// periodic eval + resumable checkpoints, stdout + CSV logging (step, loss, | |
| 16 | +// tokens/sec, lr, grad norm — CLAUDE.md training-loop requirements). | |
| 17 | +class Trainer { | |
| 18 | +public: | |
| 19 | + Trainer(Config cfg, const std::string& data_dir, const std::string& out_dir, | |
| 20 | + const std::string& config_json); | |
| 21 | + | |
| 22 | + // resume_from: checkpoint path or empty. | |
| 23 | + void train(const std::string& resume_from); | |
| 24 | + | |
| 25 | +private: | |
| 26 | + float eval_loss(); | |
| 27 | + void save(int64_t step); | |
| 28 | + | |
| 29 | + Config cfg_; | |
| 30 | + std::string out_dir_; | |
| 31 | + std::string config_json_; | |
| 32 | + std::unique_ptr<nn::Transformer> model_; | |
| 33 | + std::unique_ptr<AdamW> opt_; | |
| 34 | + std::unique_ptr<DataLoader> train_data_; | |
| 35 | + std::unique_ptr<DataLoader> val_data_; | |
| 36 | +}; | |
| 37 | + | |
| 38 | +} // namespace forge::train | |
added
tests/bench_attention.cpp
+96 −0
@@ -0,0 +1,96 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +// | |
| 3 | +// Per-kernel attention timing at real training shapes, so optimization work | |
| 4 | +// targets whatever actually dominates a step instead of what looks slow. | |
| 5 | +// Causal attention does T(T+1)/2 of the T^2 pairs; each pair costs 2*hd MACs | |
| 6 | +// in the forward (QK then PV), so the reported TFLOPs use 4*hd flops/pair. | |
| 7 | +#include <Foundation/Foundation.hpp> | |
| 8 | +#include <Metal/Metal.hpp> | |
| 9 | + | |
| 10 | +#include "core/device.h" | |
| 11 | +#include "core/tensor.h" | |
| 12 | +#include "ops/metal/metal_ops.h" | |
| 13 | + | |
| 14 | +#include <cstdio> | |
| 15 | +#include <random> | |
| 16 | + | |
| 17 | +namespace { | |
| 18 | + | |
| 19 | +double now_gpu() { return forge::metal::Stream::get().gpu_seconds(); } | |
| 20 | + | |
| 21 | +void fill(forge::Tensor& t, std::mt19937& rng) { | |
| 22 | + std::uniform_real_distribution<float> d(-1.0f, 1.0f); | |
| 23 | + for (int64_t i = 0; i < t.numel(); ++i) t.data<float>()[i] = d(rng); | |
| 24 | +} | |
| 25 | + | |
| 26 | +} // namespace | |
| 27 | + | |
| 28 | +int main() { | |
| 29 | + NS::AutoreleasePool* pool = NS::AutoreleasePool::alloc()->init(); | |
| 30 | + std::printf("device: %s\n\n", forge::Device::get().name().c_str()); | |
| 31 | + | |
| 32 | + struct Case { int64_t B, T, H, HKV, HD; const char* tag; }; | |
| 33 | + const Case cases[] = { | |
| 34 | + {64, 512, 6, 6, 64, "gpt-10m B64 T512 H6"}, | |
| 35 | + {32, 128, 4, 2, 64, "gpt-smoke B32 T128 H4"}, | |
| 36 | + {64, 1024, 8, 8, 64, "gpt-25m B64 T1024 H8"}, | |
| 37 | + }; | |
| 38 | + | |
| 39 | + std::printf("%-24s %9s %9s %9s %9s %9s\n", "case", "scalar", "mma", "bwd ms", | |
| 40 | + "scalarTF", "mmaTF"); | |
| 41 | + for (const Case& c : cases) { | |
| 42 | + const int64_t Cq = c.H * c.HD, Ckv = c.HKV * c.HD; | |
| 43 | + std::mt19937 rng(1); | |
| 44 | + forge::Tensor q = forge::Tensor::empty({c.B, c.T, Cq}); | |
| 45 | + forge::Tensor k = forge::Tensor::empty({c.B, c.T, Ckv}); | |
| 46 | + forge::Tensor v = forge::Tensor::empty({c.B, c.T, Ckv}); | |
| 47 | + forge::Tensor o = forge::Tensor::empty({c.B, c.T, Cq}); | |
| 48 | + forge::Tensor dO = forge::Tensor::empty({c.B, c.T, Cq}); | |
| 49 | + forge::Tensor lse = forge::Tensor::empty({c.B, c.H, c.T}); | |
| 50 | + forge::Tensor dq = forge::Tensor::zeros({c.B, c.T, Cq}); | |
| 51 | + forge::Tensor dk = forge::Tensor::zeros({c.B, c.T, Ckv}); | |
| 52 | + forge::Tensor dv = forge::Tensor::zeros({c.B, c.T, Ckv}); | |
| 53 | + fill(q, rng); fill(k, rng); fill(v, rng); fill(dO, rng); | |
| 54 | + const float scale = 1.0f / std::sqrt(float(c.HD)); | |
| 55 | + const int iters = 3; | |
| 56 | + | |
| 57 | + using FK = forge::metal::FlashKernel; | |
| 58 | + double fwd_by_kernel[2]; | |
| 59 | + const FK kernels[2] = {FK::Scalar, FK::MMA}; | |
| 60 | + for (int ki = 0; ki < 2; ++ki) { | |
| 61 | + forge::metal::flash_attention(q, k, v, c.H, c.HKV, true, scale, o, lse, | |
| 62 | + kernels[ki]); | |
| 63 | + forge::metal::sync(); // warm up + build pipeline | |
| 64 | + const double t = now_gpu(); | |
| 65 | + for (int i = 0; i < iters; ++i) | |
| 66 | + forge::metal::flash_attention(q, k, v, c.H, c.HKV, true, scale, o, lse, | |
| 67 | + kernels[ki]); | |
| 68 | + forge::metal::sync(); | |
| 69 | + fwd_by_kernel[ki] = (now_gpu() - t) / iters; | |
| 70 | + } | |
| 71 | + const double fwd = fwd_by_kernel[1]; | |
| 72 | + double t0; | |
| 73 | + | |
| 74 | + // The backward wrapper runs D + dq + dkv; time the whole thing, then | |
| 75 | + // the D+dq part alone, and take dkv as the difference. | |
| 76 | + forge::metal::flash_attention_backward(q, k, v, o, lse, dO, c.H, c.HKV, true, | |
| 77 | + scale, dq, dk, dv); | |
| 78 | + forge::metal::sync(); | |
| 79 | + t0 = now_gpu(); | |
| 80 | + for (int i = 0; i < iters; ++i) | |
| 81 | + forge::metal::flash_attention_backward(q, k, v, o, lse, dO, c.H, c.HKV, | |
| 82 | + true, scale, dq, dk, dv); | |
| 83 | + forge::metal::sync(); | |
| 84 | + const double bwd = (now_gpu() - t0) / iters; | |
| 85 | + | |
| 86 | + const double pairs = double(c.B) * c.H * double(c.T) * (c.T + 1) / 2.0; | |
| 87 | + const double flops = pairs * 4.0 * double(c.HD); | |
| 88 | + std::printf("%-24s %9.2f %9.2f %9.2f %8.2fT %8.2fT\n", c.tag, | |
| 89 | + fwd_by_kernel[0] * 1e3, fwd_by_kernel[1] * 1e3, bwd * 1e3, | |
| 90 | + flops / fwd_by_kernel[0] / 1e12, flops / fwd_by_kernel[1] / 1e12); | |
| 91 | + std::fflush(stdout); | |
| 92 | + } | |
| 93 | + | |
| 94 | + pool->drain(); | |
| 95 | + return 0; | |
| 96 | +} | |
added
tests/bench_matmul.cpp
+85 −0
@@ -0,0 +1,85 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +// | |
| 3 | +// GEMM throughput benchmark. Times each kernel over the shapes a transformer | |
| 4 | +// step actually issues (fwd X·Wᵀ, dX = dY·W, dW = dYᵀ·X) plus square shapes | |
| 5 | +// for a clean TFLOPs number. GPU time comes from command-buffer | |
| 6 | +// GPUStartTime/GPUEndTime, so it excludes CPU encode overhead. | |
| 7 | +#include <Foundation/Foundation.hpp> | |
| 8 | +#include <Metal/Metal.hpp> | |
| 9 | + | |
| 10 | +#include "core/device.h" | |
| 11 | +#include "core/tensor.h" | |
| 12 | +#include "ops/metal/metal_ops.h" | |
| 13 | + | |
| 14 | +#include <cstdio> | |
| 15 | +#include <random> | |
| 16 | +#include <vector> | |
| 17 | + | |
| 18 | +using forge::metal::MatmulKernel; | |
| 19 | + | |
| 20 | +namespace { | |
| 21 | + | |
| 22 | +struct Case { | |
| 23 | + int64_t M, K, N; | |
| 24 | + bool ta, tb; | |
| 25 | + const char* label; | |
| 26 | +}; | |
| 27 | + | |
| 28 | +double bench(const Case& c, MatmulKernel kernel, int iters) { | |
| 29 | + forge::Tensor a = c.ta ? forge::Tensor::empty({c.K, c.M}) | |
| 30 | + : forge::Tensor::empty({c.M, c.K}); | |
| 31 | + forge::Tensor b = c.tb ? forge::Tensor::empty({c.N, c.K}) | |
| 32 | + : forge::Tensor::empty({c.K, c.N}); | |
| 33 | + forge::Tensor out = forge::Tensor::empty({c.M, c.N}); | |
| 34 | + std::mt19937 rng(7); | |
| 35 | + std::uniform_real_distribution<float> dist(-1.0f, 1.0f); | |
| 36 | + for (int64_t i = 0; i < a.numel(); ++i) a.data<float>()[i] = dist(rng); | |
| 37 | + for (int64_t i = 0; i < b.numel(); ++i) b.data<float>()[i] = dist(rng); | |
| 38 | + | |
| 39 | + // warmup (also builds the pipeline) | |
| 40 | + forge::metal::matmul(a, b, out, c.ta, c.tb, false, kernel); | |
| 41 | + forge::metal::sync(); | |
| 42 | + | |
| 43 | + const double t0 = forge::metal::Stream::get().gpu_seconds(); | |
| 44 | + for (int i = 0; i < iters; ++i) | |
| 45 | + forge::metal::matmul(a, b, out, c.ta, c.tb, false, kernel); | |
| 46 | + forge::metal::sync(); | |
| 47 | + const double elapsed = forge::metal::Stream::get().gpu_seconds() - t0; | |
| 48 | + | |
| 49 | + const double flops = 2.0 * double(c.M) * double(c.N) * double(c.K) * iters; | |
| 50 | + return flops / elapsed / 1e12; // TFLOP/s | |
| 51 | +} | |
| 52 | + | |
| 53 | +} // namespace | |
| 54 | + | |
| 55 | +int main() { | |
| 56 | + NS::AutoreleasePool* pool = NS::AutoreleasePool::alloc()->init(); | |
| 57 | + std::printf("device: %s\n\n", forge::Device::get().name().c_str()); | |
| 58 | + | |
| 59 | + // gpt-25m-ish shapes: batch*seq = 64*1024 rows, d_model 512, d_ff 1408 | |
| 60 | + const std::vector<Case> cases = { | |
| 61 | + {4096, 4096, 4096, false, false, "square 4096"}, | |
| 62 | + {2048, 2048, 2048, false, false, "square 2048"}, | |
| 63 | + {1024, 1024, 1024, false, false, "square 1024"}, | |
| 64 | + {65536, 512, 1408, false, true, "fwd mlp X·W1ᵀ"}, | |
| 65 | + {65536, 1408, 512, false, true, "fwd mlp H·W2ᵀ"}, | |
| 66 | + {65536, 512, 512, false, true, "fwd attn X·Wqᵀ"}, | |
| 67 | + {65536, 512, 1408, false, false, "bwd dX = dY·W"}, | |
| 68 | + {512, 65536, 1408, true, false, "bwd dW = dYᵀ·X"}, | |
| 69 | + {65536, 512, 4096, false, true, "lm head (vocab 4096)"}, | |
| 70 | + }; | |
| 71 | + | |
| 72 | + std::printf("%-24s %10s %10s %10s\n", "case", "naive", "tiled", "simd"); | |
| 73 | + for (const Case& c : cases) { | |
| 74 | + const double gflop = 2.0 * double(c.M) * double(c.N) * double(c.K) / 1e9; | |
| 75 | + const int iters = gflop > 50.0 ? 3 : 10; | |
| 76 | + const double n = bench(c, MatmulKernel::Naive, iters); | |
| 77 | + const double t = bench(c, MatmulKernel::Tiled, iters); | |
| 78 | + const double s = bench(c, MatmulKernel::Simdgroup, iters); | |
| 79 | + std::printf("%-24s %9.2fT %9.2fT %9.2fT\n", c.label, n, t, s); | |
| 80 | + std::fflush(stdout); | |
| 81 | + } | |
| 82 | + | |
| 83 | + pool->drain(); | |
| 84 | + return 0; | |
| 85 | +} | |
added
tests/bench_precision.cpp
+144 −0
@@ -0,0 +1,144 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +// | |
| 3 | +// Does simdgroup_matrix actually run faster with f16/bf16 operands than with | |
| 4 | +// f32 on this hardware? The literature disagrees: metal-benchmarks measures | |
| 5 | +// f16 and f32 FMA at the SAME rate on Apple GPUs (the win coming from | |
| 6 | +// registers/bandwidth), while Apple's M3 material claims up to 2x ALU for | |
| 7 | +// family 9 via FP16/FP32/INT co-issue. Since mixed precision is a large | |
| 8 | +// amount of work to plumb through a training framework, measure before | |
| 9 | +// committing. | |
| 10 | +// | |
| 11 | +// All three variants use the identical 64x64x16 tiling from matmul_simd and | |
| 12 | +// an f32 accumulator (required for training); only the staged-tile operand | |
| 13 | +// type and the MMA fragment type differ. | |
| 14 | +#include <Foundation/Foundation.hpp> | |
| 15 | +#include <Metal/Metal.hpp> | |
| 16 | + | |
| 17 | +#include "core/device.h" | |
| 18 | +#include "core/tensor.h" | |
| 19 | +#include "ops/metal/metal_ops.h" | |
| 20 | + | |
| 21 | +#include <cstdio> | |
| 22 | +#include <random> | |
| 23 | +#include <string> | |
| 24 | + | |
| 25 | +using namespace forge; | |
| 26 | + | |
| 27 | +namespace { | |
| 28 | + | |
| 29 | +struct Params { uint32_t M, N, K; }; | |
| 30 | + | |
| 31 | +double run(const char* kernel, DType dt, int64_t M, int64_t N, int64_t K, int iters) { | |
| 32 | + Tensor a = Tensor::empty({M, K}, dt); | |
| 33 | + Tensor b = Tensor::empty({K, N}, dt); | |
| 34 | + Tensor c = Tensor::empty({M, N}, DType::F32); | |
| 35 | + std::mt19937 rng(3); | |
| 36 | + std::uniform_real_distribution<float> dist(-1.0f, 1.0f); | |
| 37 | + for (int64_t i = 0; i < a.numel(); ++i) a.set_item(i, dist(rng)); | |
| 38 | + for (int64_t i = 0; i < b.numel(); ++i) b.set_item(i, dist(rng)); | |
| 39 | + | |
| 40 | + Device& dev = Device::get(); | |
| 41 | + MTL::ComputePipelineState* pso = dev.pipeline(kernel); | |
| 42 | + Params p{uint32_t(M), uint32_t(N), uint32_t(K)}; | |
| 43 | + | |
| 44 | + auto encode = [&]() { | |
| 45 | + MTL::ComputeCommandEncoder* enc = metal::Stream::get().encoder(); | |
| 46 | + enc->setComputePipelineState(pso); | |
| 47 | + enc->setBuffer(a.buffer(), a.buffer_offset(), 0); | |
| 48 | + enc->setBuffer(b.buffer(), b.buffer_offset(), 1); | |
| 49 | + enc->setBuffer(c.buffer(), c.buffer_offset(), 2); | |
| 50 | + enc->setBytes(&p, sizeof(p), 3); | |
| 51 | + enc->dispatchThreadgroups( | |
| 52 | + MTL::Size(NS::UInteger(N) / 64, NS::UInteger(M) / 64, 1), | |
| 53 | + MTL::Size(128, 1, 1)); | |
| 54 | + }; | |
| 55 | + | |
| 56 | + encode(); | |
| 57 | + metal::sync(); // warm up, build pipeline | |
| 58 | + const double t0 = metal::Stream::get().gpu_seconds(); | |
| 59 | + for (int i = 0; i < iters; ++i) encode(); | |
| 60 | + metal::sync(); | |
| 61 | + const double dt_s = metal::Stream::get().gpu_seconds() - t0; | |
| 62 | + return 2.0 * double(M) * double(N) * double(K) * iters / dt_s / 1e12; | |
| 63 | +} | |
| 64 | + | |
| 65 | +double run_mpp(const char* kernel, DType dt, int64_t M, int64_t N, int64_t K, | |
| 66 | + int iters) { | |
| 67 | + Tensor a = Tensor::empty({M, K}, dt); | |
| 68 | + Tensor b = Tensor::empty({K, N}, dt); | |
| 69 | + Tensor c = Tensor::empty({M, N}, DType::F32); | |
| 70 | + std::mt19937 rng(3); | |
| 71 | + std::uniform_real_distribution<float> dist(-1.0f, 1.0f); | |
| 72 | + for (int64_t i = 0; i < a.numel(); ++i) a.set_item(i, dist(rng)); | |
| 73 | + for (int64_t i = 0; i < b.numel(); ++i) b.set_item(i, dist(rng)); | |
| 74 | + | |
| 75 | + Device& dev = Device::get(); | |
| 76 | + MTL::ComputePipelineState* pso = dev.pipeline(kernel); | |
| 77 | + Params p{uint32_t(M), uint32_t(N), uint32_t(K)}; | |
| 78 | + | |
| 79 | + auto encode = [&]() { | |
| 80 | + MTL::ComputeCommandEncoder* enc = metal::Stream::get().encoder(); | |
| 81 | + enc->setComputePipelineState(pso); | |
| 82 | + enc->setBuffer(a.buffer(), a.buffer_offset(), 0); | |
| 83 | + enc->setBuffer(b.buffer(), b.buffer_offset(), 1); | |
| 84 | + enc->setBuffer(c.buffer(), c.buffer_offset(), 2); | |
| 85 | + enc->setBytes(&p, sizeof(p), 3); | |
| 86 | + enc->dispatchThreadgroups( | |
| 87 | + MTL::Size(NS::UInteger(N) / 32, NS::UInteger(M) / 64, 1), | |
| 88 | + MTL::Size(128, 1, 1)); | |
| 89 | + }; | |
| 90 | + | |
| 91 | + encode(); | |
| 92 | + metal::sync(); | |
| 93 | + const double t0 = metal::Stream::get().gpu_seconds(); | |
| 94 | + for (int i = 0; i < iters; ++i) encode(); | |
| 95 | + metal::sync(); | |
| 96 | + const double dt_s = metal::Stream::get().gpu_seconds() - t0; | |
| 97 | + return 2.0 * double(M) * double(N) * double(K) * iters / dt_s / 1e12; | |
| 98 | +} | |
| 99 | + | |
| 100 | +} // namespace | |
| 101 | + | |
| 102 | +int main() { | |
| 103 | + NS::AutoreleasePool* pool = NS::AutoreleasePool::alloc()->init(); | |
| 104 | + std::printf("device: %s\n\n", Device::get().name().c_str()); | |
| 105 | + std::printf("simdgroup_matrix throughput by operand precision " | |
| 106 | + "(f32 accumulator throughout)\n"); | |
| 107 | + std::printf("%-16s %10s %10s %10s\n", "shape", "f32", "f16", "bf16"); | |
| 108 | + | |
| 109 | + struct Shape { int64_t M, N, K; }; | |
| 110 | + const Shape shapes[] = {{2048, 2048, 2048}, {4096, 4096, 4096}, | |
| 111 | + {65536, 512, 1408}}; | |
| 112 | + for (const Shape& s : shapes) { | |
| 113 | + const double gflop = 2.0 * double(s.M) * double(s.N) * double(s.K) / 1e9; | |
| 114 | + const int iters = gflop > 50.0 ? 3 : 10; | |
| 115 | + const double f32 = run("gemm_f32", DType::F32, s.M, s.N, s.K, iters); | |
| 116 | + const double f16 = run("gemm_f16", DType::F16, s.M, s.N, s.K, iters); | |
| 117 | + const double bf16 = run("gemm_bf16", DType::BF16, s.M, s.N, s.K, iters); | |
| 118 | + char tag[32]; | |
| 119 | + std::snprintf(tag, sizeof(tag), "%lldx%lldx%lld", (long long)s.M, | |
| 120 | + (long long)s.N, (long long)s.K); | |
| 121 | + std::printf("%-16s %9.2fT %9.2fT %9.2fT\n", tag, f32, f16, bf16); | |
| 122 | + std::fflush(stdout); | |
| 123 | + } | |
| 124 | + | |
| 125 | + // Metal Performance Primitives cooperative-tensor matmul2d — the Metal 4 | |
| 126 | + // path that targets M5 neural accelerators. 64x32 tile, 4 simdgroups. | |
| 127 | + std::printf("\nMPP matmul2d (cooperative tensors, Metal 4)\n"); | |
| 128 | + std::printf("%-16s %10s %10s\n", "shape", "f32", "f16"); | |
| 129 | + for (const Shape& s : shapes) { | |
| 130 | + if (s.M % 64 != 0 || s.N % 32 != 0) continue; | |
| 131 | + const double gflop = 2.0 * double(s.M) * double(s.N) * double(s.K) / 1e9; | |
| 132 | + const int iters = gflop > 50.0 ? 3 : 10; | |
| 133 | + const double f32 = run_mpp("matmul_mpp_f32", DType::F32, s.M, s.N, s.K, iters); | |
| 134 | + const double f16 = run_mpp("matmul_mpp_f16", DType::F16, s.M, s.N, s.K, iters); | |
| 135 | + char tag[32]; | |
| 136 | + std::snprintf(tag, sizeof(tag), "%lldx%lldx%lld", (long long)s.M, | |
| 137 | + (long long)s.N, (long long)s.K); | |
| 138 | + std::printf("%-16s %9.2fT %9.2fT\n", tag, f32, f16); | |
| 139 | + std::fflush(stdout); | |
| 140 | + } | |
| 141 | + | |
| 142 | + pool->drain(); | |
| 143 | + return 0; | |
| 144 | +} | |
added
tests/mppcheck.cpp
+118 −0
@@ -0,0 +1,118 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +#include <Foundation/Foundation.hpp> | |
| 3 | +#include <Metal/Metal.hpp> | |
| 4 | +#include "core/device.h" | |
| 5 | +#include "core/tensor.h" | |
| 6 | +#include "ops/cpu/cpu_ops.h" | |
| 7 | +#include "ops/metal/metal_ops.h" | |
| 8 | +#include <cmath> | |
| 9 | +#include <cstdio> | |
| 10 | +#include <random> | |
| 11 | +#include <string> | |
| 12 | +using namespace forge; | |
| 13 | +struct Params { uint32_t M, N, K; }; | |
| 14 | +int main() { | |
| 15 | + NS::AutoreleasePool* pool = NS::AutoreleasePool::alloc()->init(); | |
| 16 | + const int64_t M = 128, N = 64, K = 96; // exact multiples of the 64x32 tile | |
| 17 | + Tensor a = Tensor::empty({M, K}), b = Tensor::empty({K, N}); | |
| 18 | + Tensor c = Tensor::zeros({M, N}), ref = Tensor::empty({M, N}); | |
| 19 | + std::mt19937 rng(5); | |
| 20 | + std::uniform_real_distribution<float> d(-1.f, 1.f); | |
| 21 | + for (int64_t i = 0; i < a.numel(); ++i) a.data<float>()[i] = d(rng); | |
| 22 | + for (int64_t i = 0; i < b.numel(); ++i) b.data<float>()[i] = d(rng); | |
| 23 | + cpu::matmul(a, b, ref, false, false); | |
| 24 | + | |
| 25 | + Device& dev = Device::get(); | |
| 26 | + MTL::ComputePipelineState* pso = dev.pipeline("matmul_mpp_f32"); | |
| 27 | + Params p{uint32_t(M), uint32_t(N), uint32_t(K)}; | |
| 28 | + MTL::ComputeCommandEncoder* enc = metal::Stream::get().encoder(); | |
| 29 | + enc->setComputePipelineState(pso); | |
| 30 | + enc->setBuffer(a.buffer(), a.buffer_offset(), 0); | |
| 31 | + enc->setBuffer(b.buffer(), b.buffer_offset(), 1); | |
| 32 | + enc->setBuffer(c.buffer(), c.buffer_offset(), 2); | |
| 33 | + enc->setBytes(&p, sizeof(p), 3); | |
| 34 | + enc->dispatchThreadgroups(MTL::Size(N/32, M/64, 1), MTL::Size(128,1,1)); | |
| 35 | + metal::sync(); | |
| 36 | + | |
| 37 | + float worst = 0.f; int64_t nz = 0; | |
| 38 | + for (int64_t i = 0; i < c.numel(); ++i) { | |
| 39 | + worst = std::max(worst, std::fabs(c.data<float>()[i] - ref.data<float>()[i])); | |
| 40 | + if (c.data<float>()[i] != 0.f) ++nz; | |
| 41 | + } | |
| 42 | + std::printf("MPP matmul2d f32 vs CPU: max abs err %.3e (nonzero outputs %lld/%lld)\n", | |
| 43 | + double(worst), (long long)nz, (long long)c.numel()); | |
| 44 | + const bool ok32 = worst <= 1e-4f && nz == c.numel(); | |
| 45 | + std::printf("%s\n", ok32 ? "CORRECT" : "WRONG"); | |
| 46 | + | |
| 47 | + // f16 operands, f32 accumulator: the headline throughput number, so check | |
| 48 | + // it computes the same product within f16 input precision. | |
| 49 | + Tensor ah = Tensor::empty({M, K}, DType::F16); | |
| 50 | + Tensor bh = Tensor::empty({K, N}, DType::F16); | |
| 51 | + Tensor ch = Tensor::zeros({M, N}); | |
| 52 | + Tensor refh = Tensor::empty({M, N}); | |
| 53 | + for (int64_t i = 0; i < a.numel(); ++i) ah.set_item(i, a.data<float>()[i]); | |
| 54 | + for (int64_t i = 0; i < b.numel(); ++i) bh.set_item(i, b.data<float>()[i]); | |
| 55 | + // reference from the ROUNDED f16 values, so we compare arithmetic not rounding | |
| 56 | + Tensor ar = Tensor::empty({M, K}), br = Tensor::empty({K, N}); | |
| 57 | + for (int64_t i = 0; i < a.numel(); ++i) ar.data<float>()[i] = ah.item_at(i); | |
| 58 | + for (int64_t i = 0; i < b.numel(); ++i) br.data<float>()[i] = bh.item_at(i); | |
| 59 | + cpu::matmul(ar, br, refh, false, false); | |
| 60 | + | |
| 61 | + MTL::ComputePipelineState* pso16 = dev.pipeline("matmul_mpp_f16"); | |
| 62 | + MTL::ComputeCommandEncoder* e2 = metal::Stream::get().encoder(); | |
| 63 | + e2->setComputePipelineState(pso16); | |
| 64 | + e2->setBuffer(ah.buffer(), ah.buffer_offset(), 0); | |
| 65 | + e2->setBuffer(bh.buffer(), bh.buffer_offset(), 1); | |
| 66 | + e2->setBuffer(ch.buffer(), ch.buffer_offset(), 2); | |
| 67 | + e2->setBytes(&p, sizeof(p), 3); | |
| 68 | + e2->dispatchThreadgroups(MTL::Size(N/32, M/64, 1), MTL::Size(128,1,1)); | |
| 69 | + metal::sync(); | |
| 70 | + | |
| 71 | + float w16 = 0.f; int64_t nz16 = 0; | |
| 72 | + for (int64_t i = 0; i < ch.numel(); ++i) { | |
| 73 | + w16 = std::max(w16, std::fabs(ch.data<float>()[i] - refh.data<float>()[i])); | |
| 74 | + if (ch.data<float>()[i] != 0.f) ++nz16; | |
| 75 | + } | |
| 76 | + const bool ok16 = w16 <= 1e-3f && nz16 == ch.numel(); | |
| 77 | + std::printf("MPP matmul2d f16 vs CPU(f16 inputs): max abs err %.3e " | |
| 78 | + "(nonzero %lld/%lld)\n%s\n", double(w16), (long long)nz16, | |
| 79 | + (long long)ch.numel(), ok16 ? "CORRECT" : "WRONG"); | |
| 80 | + // Transposed variants — training needs nt (forward X.W^T) and tn (dW = dY^T.X). | |
| 81 | + bool okT = true; | |
| 82 | + struct TCase { const char* suffix; bool ta, tb; const char* tag; }; | |
| 83 | + const TCase tcases[] = {{"_nt", false, true, "nt (fwd X.Wt)"}, | |
| 84 | + {"_tn", true, false, "tn (dW = dYt.X)"}}; | |
| 85 | + for (const TCase& tc : tcases) { | |
| 86 | + Tensor at = tc.ta ? Tensor::empty({K, M}) : Tensor::empty({M, K}); | |
| 87 | + Tensor bt = tc.tb ? Tensor::empty({N, K}) : Tensor::empty({K, N}); | |
| 88 | + Tensor ct = Tensor::zeros({M, N}), rt = Tensor::empty({M, N}); | |
| 89 | + for (int64_t i = 0; i < at.numel(); ++i) at.data<float>()[i] = d(rng); | |
| 90 | + for (int64_t i = 0; i < bt.numel(); ++i) bt.data<float>()[i] = d(rng); | |
| 91 | + cpu::matmul(at, bt, rt, tc.ta, tc.tb); | |
| 92 | + | |
| 93 | + std::string kn = std::string("matmul_mpp_f32") + tc.suffix; | |
| 94 | + MTL::ComputePipelineState* ps = dev.pipeline(kn); | |
| 95 | + MTL::ComputeCommandEncoder* e3 = metal::Stream::get().encoder(); | |
| 96 | + e3->setComputePipelineState(ps); | |
| 97 | + e3->setBuffer(at.buffer(), at.buffer_offset(), 0); | |
| 98 | + e3->setBuffer(bt.buffer(), bt.buffer_offset(), 1); | |
| 99 | + e3->setBuffer(ct.buffer(), ct.buffer_offset(), 2); | |
| 100 | + e3->setBytes(&p, sizeof(p), 3); | |
| 101 | + e3->dispatchThreadgroups(MTL::Size(N/32, M/64, 1), MTL::Size(128,1,1)); | |
| 102 | + metal::sync(); | |
| 103 | + | |
| 104 | + float w = 0.f; int64_t nzt = 0; | |
| 105 | + for (int64_t i = 0; i < ct.numel(); ++i) { | |
| 106 | + w = std::max(w, std::fabs(ct.data<float>()[i] - rt.data<float>()[i])); | |
| 107 | + if (ct.data<float>()[i] != 0.f) ++nzt; | |
| 108 | + } | |
| 109 | + const bool ok = w <= 1e-4f && nzt == ct.numel(); | |
| 110 | + okT = okT && ok; | |
| 111 | + std::printf("MPP matmul2d f32 %-16s max abs err %.3e (nonzero %lld/%lld) %s\n", | |
| 112 | + tc.tag, double(w), (long long)nzt, (long long)ct.numel(), | |
| 113 | + ok ? "CORRECT" : "WRONG"); | |
| 114 | + } | |
| 115 | + | |
| 116 | + pool->drain(); | |
| 117 | + return (ok32 && ok16 && okT) ? 0 : 1; | |
| 118 | +} | |
added
tests/test_gradcheck.cpp
+237 −0
@@ -0,0 +1,237 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +// | |
| 3 | +// Numerical gradient checking (central differences) for every op with | |
| 4 | +// parameters, plus a full tiny-model loss. Forward math is f32, so the | |
| 5 | +// difference quotient carries ~1e-4-relative noise; eps is chosen per the | |
| 6 | +// usual sqrt(machine-eps) rule for O(1) inputs and tolerances follow | |
| 7 | +// CLAUDE.md (rel error <= 1e-3 with an absolute floor). | |
| 8 | +#include "core/autograd.h" | |
| 9 | +#include "nn/transformer.h" | |
| 10 | +#include "ops/ops.h" | |
| 11 | + | |
| 12 | +#include <cmath> | |
| 13 | +#include <cstdio> | |
| 14 | +#include <functional> | |
| 15 | +#include <random> | |
| 16 | +#include <vector> | |
| 17 | + | |
| 18 | +using namespace forge; | |
| 19 | + | |
| 20 | +namespace { | |
| 21 | + | |
| 22 | +int g_failures = 0; | |
| 23 | +constexpr float kEps = 1e-2f; | |
| 24 | +constexpr float kRelTol = 1e-3f; | |
| 25 | +constexpr float kAbsTol = 1e-4f; | |
| 26 | + | |
| 27 | +// loss_fn must run forward and return the scalar loss Var. Gradcheck | |
| 28 | +// perturbs every element of every checked Var and compares the analytic | |
| 29 | +// gradient against (f(x+e) - f(x-e)) / 2e. | |
| 30 | +void gradcheck(const char* name, const std::function<Var()>& loss_fn, | |
| 31 | + const std::vector<Var>& checked, float eps = kEps, float tol = kRelTol) { | |
| 32 | + // analytic | |
| 33 | + for (const Var& p : checked) p.zero_grad(); | |
| 34 | + Tape::get().clear(); | |
| 35 | + Var loss = loss_fn(); | |
| 36 | + Tape::get().backward(loss); | |
| 37 | + | |
| 38 | + float worst = 0.0f; | |
| 39 | + for (const Var& p : checked) { | |
| 40 | + const int64_t n = p.value().numel(); | |
| 41 | + float* x = p.value().data<float>(); | |
| 42 | + const float* g = p.grad().data<float>(); | |
| 43 | + for (int64_t i = 0; i < n; ++i) { | |
| 44 | + const float saved = x[i]; | |
| 45 | + float fp, fm; | |
| 46 | + { | |
| 47 | + NoGrad ng; | |
| 48 | + x[i] = saved + eps; | |
| 49 | + fp = loss_fn().value().data<float>()[0]; | |
| 50 | + x[i] = saved - eps; | |
| 51 | + fm = loss_fn().value().data<float>()[0]; | |
| 52 | + } | |
| 53 | + x[i] = saved; | |
| 54 | + const float num = (fp - fm) / (2.0f * eps); | |
| 55 | + const float ana = g[i]; | |
| 56 | + const float err = std::fabs(num - ana); | |
| 57 | + const float rel = err / std::max(kAbsTol / kRelTol, std::fabs(num) + std::fabs(ana)); | |
| 58 | + worst = std::max(worst, rel); | |
| 59 | + } | |
| 60 | + } | |
| 61 | + if (worst <= tol) { | |
| 62 | + std::printf(" ok: %s (max rel err %.2e)\n", name, double(worst)); | |
| 63 | + } else { | |
| 64 | + std::printf(" FAIL: %s (max rel err %.2e > %.0e)\n", name, double(worst), | |
| 65 | + double(tol)); | |
| 66 | + ++g_failures; | |
| 67 | + } | |
| 68 | +} | |
| 69 | + | |
| 70 | +Var make_var(std::vector<int64_t> shape, std::mt19937_64& rng, float std = 1.0f) { | |
| 71 | + Tensor t = Tensor::empty(std::move(shape)); | |
| 72 | + cpu::fill_normal(t, 0.0f, std, rng); | |
| 73 | + return Var(std::move(t), /*requires_grad=*/true); | |
| 74 | +} | |
| 75 | + | |
| 76 | +// Reduce an op output to a scalar via a fixed random projection so every | |
| 77 | +// output element influences the loss. | |
| 78 | +Var project(const Var& y, const Tensor& r) { | |
| 79 | + Var rv(r, /*requires_grad=*/false); | |
| 80 | + const int64_t n = y.value().numel(); | |
| 81 | + Var y2 = y.reshaped({1, n}); | |
| 82 | + Var r2 = rv.reshaped({1, n}); | |
| 83 | + return ops::matmul(y2, r2, false, true); // [1,1] | |
| 84 | +} | |
| 85 | + | |
| 86 | +Tensor random_proj(int64_t n, std::mt19937_64& rng) { | |
| 87 | + Tensor r = Tensor::empty({n}); | |
| 88 | + cpu::fill_normal(r, 0.0f, 1.0f, rng); | |
| 89 | + return r; | |
| 90 | +} | |
| 91 | + | |
| 92 | +} // namespace | |
| 93 | + | |
| 94 | +int main() { | |
| 95 | + std::mt19937_64 rng(42); | |
| 96 | + | |
| 97 | + // matmul, all transpose variants | |
| 98 | + for (int ta = 0; ta <= 1; ++ta) { | |
| 99 | + for (int tb = 0; tb <= 1; ++tb) { | |
| 100 | + const int64_t M = 3, K = 4, N = 5; | |
| 101 | + Var a = ta ? make_var({K, M}, rng) : make_var({M, K}, rng); | |
| 102 | + Var b = tb ? make_var({N, K}, rng) : make_var({K, N}, rng); | |
| 103 | + Tensor r = random_proj(M * N, rng); | |
| 104 | + char name[64]; | |
| 105 | + std::snprintf(name, sizeof(name), "matmul ta=%d tb=%d", ta, tb); | |
| 106 | + gradcheck(name, [&]() { return project(ops::matmul(a, b, ta, tb), r); }, | |
| 107 | + {a, b}); | |
| 108 | + } | |
| 109 | + } | |
| 110 | + | |
| 111 | + // add_bias | |
| 112 | + { | |
| 113 | + Var x = make_var({4, 6}, rng); | |
| 114 | + Var b = make_var({6}, rng); | |
| 115 | + Tensor r = random_proj(24, rng); | |
| 116 | + gradcheck("add_bias", [&]() { return project(ops::add_bias(x, b), r); }, {x, b}); | |
| 117 | + } | |
| 118 | + | |
| 119 | + // mul | |
| 120 | + { | |
| 121 | + Var a = make_var({3, 5}, rng); | |
| 122 | + Var b = make_var({3, 5}, rng); | |
| 123 | + Tensor r = random_proj(15, rng); | |
| 124 | + gradcheck("mul", [&]() { return project(ops::mul(a, b), r); }, {a, b}); | |
| 125 | + } | |
| 126 | + | |
| 127 | + // silu / gelu | |
| 128 | + { | |
| 129 | + Var x = make_var({4, 5}, rng); | |
| 130 | + Tensor r = random_proj(20, rng); | |
| 131 | + gradcheck("silu", [&]() { return project(ops::silu(x), r); }, {x}); | |
| 132 | + gradcheck("gelu", [&]() { return project(ops::gelu(x), r); }, {x}); | |
| 133 | + } | |
| 134 | + | |
| 135 | + // rmsnorm / layernorm | |
| 136 | + { | |
| 137 | + Var x = make_var({3, 8}, rng); | |
| 138 | + Var w = make_var({8}, rng); | |
| 139 | + Var b = make_var({8}, rng); | |
| 140 | + Tensor r = random_proj(24, rng); | |
| 141 | + gradcheck("rmsnorm", [&]() { return project(ops::rmsnorm(x, w, 1e-6f), r); }, {x, w}); | |
| 142 | + gradcheck("layernorm", | |
| 143 | + [&]() { return project(ops::layernorm(x, w, b, 1e-6f), r); }, {x, w, b}); | |
| 144 | + } | |
| 145 | + | |
| 146 | + // embedding | |
| 147 | + { | |
| 148 | + Var w = make_var({7, 4}, rng); | |
| 149 | + Tensor ids = Tensor::empty({2, 3}, DType::I32); | |
| 150 | + std::mt19937_64 idrng(7); | |
| 151 | + cpu::fill_uniform_int(ids, 0, 7, idrng); | |
| 152 | + Tensor r = random_proj(2 * 3 * 4, rng); | |
| 153 | + gradcheck("embedding", [&]() { return project(ops::embedding(w, ids), r); }, {w}); | |
| 154 | + } | |
| 155 | + | |
| 156 | + // rope | |
| 157 | + { | |
| 158 | + Var x = make_var({2, 5, 2 * 6}, rng); // B=2, T=5, H=2, hd=6 | |
| 159 | + Tensor r = random_proj(2 * 5 * 12, rng); | |
| 160 | + gradcheck("rope", [&]() { return project(ops::rope(x, 2, 10000.0f, 0), r); }, {x}); | |
| 161 | + } | |
| 162 | + | |
| 163 | + // attention: MHA causal, then GQA | |
| 164 | + { | |
| 165 | + const int64_t B = 2, T = 4, H = 2, hd = 4; | |
| 166 | + Var q = make_var({B, T, H * hd}, rng, 0.5f); | |
| 167 | + Var k = make_var({B, T, H * hd}, rng, 0.5f); | |
| 168 | + Var v = make_var({B, T, H * hd}, rng, 0.5f); | |
| 169 | + Tensor r = random_proj(B * T * H * hd, rng); | |
| 170 | + const float scale = 1.0f / std::sqrt(float(hd)); | |
| 171 | + gradcheck("attention causal MHA", | |
| 172 | + [&]() { return project(ops::attention(q, k, v, H, H, true, scale), r); }, | |
| 173 | + {q, k, v}); | |
| 174 | + | |
| 175 | + Var kg = make_var({B, T, 1 * hd}, rng, 0.5f); | |
| 176 | + Var vg = make_var({B, T, 1 * hd}, rng, 0.5f); | |
| 177 | + gradcheck("attention causal GQA (2q/1kv)", | |
| 178 | + [&]() { return project(ops::attention(q, kg, vg, H, 1, true, scale), r); }, | |
| 179 | + {q, kg, vg}); | |
| 180 | + } | |
| 181 | + | |
| 182 | + // cross entropy (with an ignored index) | |
| 183 | + { | |
| 184 | + Var logits = make_var({6, 9}, rng); | |
| 185 | + Tensor targets = Tensor::empty({6}, DType::I32); | |
| 186 | + std::mt19937_64 idrng(11); | |
| 187 | + cpu::fill_uniform_int(targets, 0, 9, idrng); | |
| 188 | + targets.data<int32_t>()[3] = -1; // ignore_index | |
| 189 | + gradcheck("cross_entropy", [&]() { return ops::cross_entropy(logits, targets); }, | |
| 190 | + {logits}); | |
| 191 | + } | |
| 192 | + | |
| 193 | + // full tiny model: every parameter of a 2-layer transformer | |
| 194 | + { | |
| 195 | + ModelConfig cfg; | |
| 196 | + cfg.n_layers = 2; | |
| 197 | + cfg.d_model = 16; | |
| 198 | + cfg.n_heads = 2; | |
| 199 | + cfg.n_kv_heads = 1; // exercise GQA | |
| 200 | + cfg.d_ff = 24; | |
| 201 | + cfg.vocab_size = 11; | |
| 202 | + cfg.context_length = 8; | |
| 203 | + cfg.tied_embeddings = true; | |
| 204 | + nn::Transformer model(cfg, 123); | |
| 205 | + | |
| 206 | + Tensor ids = Tensor::empty({2, 5}, DType::I32); | |
| 207 | + Tensor targets = Tensor::empty({2 * 5}, DType::I32); | |
| 208 | + std::mt19937_64 idrng(13); | |
| 209 | + cpu::fill_uniform_int(ids, 0, cfg.vocab_size, idrng); | |
| 210 | + cpu::fill_uniform_int(targets, 0, cfg.vocab_size, idrng); | |
| 211 | + | |
| 212 | + std::vector<Var> params; | |
| 213 | + std::unordered_map<const void*, bool> seen; | |
| 214 | + for (const auto& [name, p] : model.named_parameters()) { | |
| 215 | + if (!seen.emplace(p.id(), true).second) continue; | |
| 216 | + params.push_back(p); | |
| 217 | + } | |
| 218 | + // Composite check: the analytic gradient is exact per the per-op | |
| 219 | + // checks above; the f32 forward pass through ~100 ops limits the | |
| 220 | + // central-difference quotient itself (error scales as eps^2 — | |
| 221 | + // verified 2.6e-2 @ eps=1e-2 -> 2.5e-3 @ eps=3e-3), so this runs | |
| 222 | + // with a documented noise-bound tolerance, not the per-op 1e-3. | |
| 223 | + gradcheck("tiny transformer (all params)", | |
| 224 | + [&]() { | |
| 225 | + Var logits = model.forward(ids); | |
| 226 | + return ops::cross_entropy(logits, targets); | |
| 227 | + }, | |
| 228 | + params, /*eps=*/3e-3f, /*tol=*/5e-3f); | |
| 229 | + } | |
| 230 | + | |
| 231 | + if (g_failures) { | |
| 232 | + std::printf("\n%d gradcheck(s) FAILED\n", g_failures); | |
| 233 | + return 1; | |
| 234 | + } | |
| 235 | + std::printf("\nall gradchecks passed\n"); | |
| 236 | + return 0; | |
| 237 | +} | |
added
tests/test_ops.cpp
+517 −0
@@ -0,0 +1,517 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +// | |
| 3 | +// CPU-vs-Metal parity for every GPU kernel (CLAUDE.md protocol #1: max abs | |
| 4 | +// error <= 1e-4 for f32), plus tensor mechanics and the CPU matmul oracle | |
| 5 | +// check. GPU ops encode into the batched Stream; comparisons happen after | |
| 6 | +// metal::sync() — the readback boundary. | |
| 7 | +#include <Foundation/Foundation.hpp> | |
| 8 | +#include <Metal/Metal.hpp> | |
| 9 | + | |
| 10 | +#include "core/tensor.h" | |
| 11 | +#include "nn/transformer.h" | |
| 12 | +#include "ops/cpu/cpu_ops.h" | |
| 13 | +#include "ops/metal/metal_ops.h" | |
| 14 | +#include "ops/ops.h" | |
| 15 | + | |
| 16 | +#include <cmath> | |
| 17 | +#include <cstdio> | |
| 18 | +#include <cstdlib> | |
| 19 | +#include <random> | |
| 20 | +#include <vector> | |
| 21 | + | |
| 22 | +namespace { | |
| 23 | + | |
| 24 | +int g_failures = 0; | |
| 25 | +constexpr float kTol = 1e-4f; | |
| 26 | + | |
| 27 | +void expect(bool cond, const char* what) { | |
| 28 | + if (cond) { | |
| 29 | + std::printf(" ok: %s\n", what); | |
| 30 | + } else { | |
| 31 | + std::printf(" FAIL: %s\n", what); | |
| 32 | + ++g_failures; | |
| 33 | + } | |
| 34 | +} | |
| 35 | + | |
| 36 | +void expect_close(const forge::Tensor& got, const forge::Tensor& want, const char* what, | |
| 37 | + float tol = kTol) { | |
| 38 | + const float* pg = got.data<float>(); | |
| 39 | + const float* pw = want.data<float>(); | |
| 40 | + float m = 0.0f; | |
| 41 | + for (int64_t i = 0; i < got.numel(); ++i) m = std::max(m, std::fabs(pg[i] - pw[i])); | |
| 42 | + if (m <= tol) { | |
| 43 | + std::printf(" ok: %s (max abs err %.2e)\n", what, double(m)); | |
| 44 | + } else { | |
| 45 | + std::printf(" FAIL: %s (max abs err %.2e > %.0e)\n", what, double(m), double(tol)); | |
| 46 | + ++g_failures; | |
| 47 | + } | |
| 48 | +} | |
| 49 | + | |
| 50 | +void fill_random(forge::Tensor& t, std::mt19937& rng, float lo = -1.0f, float hi = 1.0f) { | |
| 51 | + std::uniform_real_distribution<float> dist(lo, hi); | |
| 52 | + float* p = t.data<float>(); | |
| 53 | + for (int64_t i = 0; i < t.numel(); ++i) p[i] = dist(rng); | |
| 54 | +} | |
| 55 | + | |
| 56 | +// Deliberately dumb oracle: index-arithmetic triple loop with double acc. | |
| 57 | +void matmul_oracle(const forge::Tensor& a, const forge::Tensor& b, forge::Tensor& c, | |
| 58 | + bool ta, bool tb) { | |
| 59 | + const int64_t M = c.size(0), N = c.size(1); | |
| 60 | + const int64_t K = ta ? a.size(0) : a.size(1); | |
| 61 | + const int64_t lda = a.size(1), ldb = b.size(1); | |
| 62 | + const float* A = a.data<float>(); | |
| 63 | + const float* B = b.data<float>(); | |
| 64 | + float* C = c.data<float>(); | |
| 65 | + for (int64_t i = 0; i < M; ++i) | |
| 66 | + for (int64_t j = 0; j < N; ++j) { | |
| 67 | + double acc = 0.0; | |
| 68 | + for (int64_t k = 0; k < K; ++k) { | |
| 69 | + const float av = ta ? A[k * lda + i] : A[i * lda + k]; | |
| 70 | + const float bv = tb ? B[j * ldb + k] : B[k * ldb + j]; | |
| 71 | + acc += double(av) * double(bv); | |
| 72 | + } | |
| 73 | + C[i * N + j] = float(acc); | |
| 74 | + } | |
| 75 | +} | |
| 76 | + | |
| 77 | +void test_tensor_basics() { | |
| 78 | + std::printf("tensor basics\n"); | |
| 79 | + forge::Tensor t = forge::Tensor::zeros({4, 8}); | |
| 80 | + expect(t.numel() == 32, "numel"); | |
| 81 | + expect(t.is_contiguous(), "contiguous"); | |
| 82 | + expect(t.strides()[0] == 8 && t.strides()[1] == 1, "row-major strides"); | |
| 83 | + | |
| 84 | + forge::Tensor v = t.view({8, 4}); | |
| 85 | + v.set_item(0, 42.0f); | |
| 86 | + expect(t.item_at(0) == 42.0f, "view shares storage"); | |
| 87 | + | |
| 88 | + forge::Tensor s = t.slice0(1, 2); | |
| 89 | + s.set_item(0, 7.0f); | |
| 90 | + expect(t.item_at(8) == 7.0f, "slice0 shares storage at offset"); | |
| 91 | + | |
| 92 | + forge::Tensor h = forge::Tensor::full({3}, 1.5f, forge::DType::F16); | |
| 93 | + expect(std::fabs(h.item_at(2) - 1.5f) < 1e-6f, "f16 roundtrip"); | |
| 94 | + forge::Tensor bf = forge::Tensor::full({3}, 1.5f, forge::DType::BF16); | |
| 95 | + expect(std::fabs(bf.item_at(1) - 1.5f) < 1e-6f, "bf16 roundtrip"); | |
| 96 | +} | |
| 97 | + | |
| 98 | +void test_cpu_matmul() { | |
| 99 | + std::printf("cpu matmul vs oracle (all transpose variants)\n"); | |
| 100 | + std::mt19937 rng(1234); | |
| 101 | + const int64_t M = 17, K = 23, N = 13; | |
| 102 | + for (int ta = 0; ta <= 1; ++ta) | |
| 103 | + for (int tb = 0; tb <= 1; ++tb) { | |
| 104 | + forge::Tensor a = ta ? forge::Tensor::empty({K, M}) : forge::Tensor::empty({M, K}); | |
| 105 | + forge::Tensor b = tb ? forge::Tensor::empty({N, K}) : forge::Tensor::empty({K, N}); | |
| 106 | + forge::Tensor c = forge::Tensor::empty({M, N}); | |
| 107 | + forge::Tensor ref = forge::Tensor::empty({M, N}); | |
| 108 | + fill_random(a, rng); | |
| 109 | + fill_random(b, rng); | |
| 110 | + forge::cpu::matmul(a, b, c, ta, tb); | |
| 111 | + matmul_oracle(a, b, ref, ta, tb); | |
| 112 | + char label[64]; | |
| 113 | + std::snprintf(label, sizeof(label), "cpu matmul ta=%d tb=%d", ta, tb); | |
| 114 | + expect_close(c, ref, label, 1e-5f); | |
| 115 | + } | |
| 116 | +} | |
| 117 | + | |
| 118 | +void test_metal_matmul() { | |
| 119 | + std::printf("metal matmul parity (all kernels, all transpose variants)\n"); | |
| 120 | + std::mt19937 rng(4321); | |
| 121 | + using K_t = forge::metal::MatmulKernel; | |
| 122 | + struct KernelCase { K_t kernel; const char* name; }; | |
| 123 | + const KernelCase kernels[] = {{K_t::Naive, "naive"}, | |
| 124 | + {K_t::Tiled, "tiled"}, | |
| 125 | + {K_t::Simdgroup, "simd"}}; | |
| 126 | + // Ragged (non-multiple of any tile dim) and exactly-tiled shapes, so the | |
| 127 | + // simdgroup kernel's predicated and ALIGNED fast paths both get covered. | |
| 128 | + struct Shape { int64_t M, K, N; const char* tag; }; | |
| 129 | + const Shape shapes[] = {{67, 129, 45, "ragged"}, {128, 64, 192, "aligned"}}; | |
| 130 | + | |
| 131 | + for (const auto& kc : kernels) | |
| 132 | + for (const auto& sh : shapes) | |
| 133 | + for (int ta = 0; ta <= 1; ++ta) | |
| 134 | + for (int tb = 0; tb <= 1; ++tb) { | |
| 135 | + forge::Tensor a = ta ? forge::Tensor::empty({sh.K, sh.M}) | |
| 136 | + : forge::Tensor::empty({sh.M, sh.K}); | |
| 137 | + forge::Tensor b = tb ? forge::Tensor::empty({sh.N, sh.K}) | |
| 138 | + : forge::Tensor::empty({sh.K, sh.N}); | |
| 139 | + forge::Tensor gpu = forge::Tensor::empty({sh.M, sh.N}); | |
| 140 | + forge::Tensor ref = forge::Tensor::empty({sh.M, sh.N}); | |
| 141 | + fill_random(a, rng); | |
| 142 | + fill_random(b, rng); | |
| 143 | + forge::cpu::matmul(a, b, ref, ta, tb); | |
| 144 | + forge::metal::matmul(a, b, gpu, ta, tb, false, kc.kernel); | |
| 145 | + forge::metal::sync(); | |
| 146 | + char label[80]; | |
| 147 | + std::snprintf(label, sizeof(label), "%s matmul %s ta=%d tb=%d", | |
| 148 | + kc.name, sh.tag, ta, tb); | |
| 149 | + expect_close(gpu, ref, label); | |
| 150 | + } | |
| 151 | + | |
| 152 | + // accumulate flag (ragged shape: both epilogue paths get predication) | |
| 153 | + const int64_t M = 67, K = 129, N = 45; | |
| 154 | + forge::Tensor a = forge::Tensor::empty({M, K}); | |
| 155 | + forge::Tensor b = forge::Tensor::empty({K, N}); | |
| 156 | + forge::Tensor acc_gpu = forge::Tensor::empty({M, N}); | |
| 157 | + forge::Tensor acc_ref = forge::Tensor::empty({M, N}); | |
| 158 | + fill_random(a, rng); | |
| 159 | + fill_random(b, rng); | |
| 160 | + fill_random(acc_gpu, rng); | |
| 161 | + std::memcpy(acc_ref.raw(), acc_gpu.raw(), acc_gpu.nbytes()); | |
| 162 | + forge::cpu::matmul(a, b, acc_ref, false, false, true); | |
| 163 | + forge::metal::matmul(a, b, acc_gpu, false, false, true, K_t::Tiled); | |
| 164 | + forge::metal::sync(); | |
| 165 | + expect_close(acc_gpu, acc_ref, "tiled matmul accumulate=true"); | |
| 166 | + | |
| 167 | + // simdgroup accumulate takes the staged (non-fast) epilogue path | |
| 168 | + forge::Tensor sacc_gpu = forge::Tensor::empty({M, N}); | |
| 169 | + forge::Tensor sacc_ref = forge::Tensor::empty({M, N}); | |
| 170 | + fill_random(sacc_gpu, rng); | |
| 171 | + std::memcpy(sacc_ref.raw(), sacc_gpu.raw(), sacc_gpu.nbytes()); | |
| 172 | + forge::cpu::matmul(a, b, sacc_ref, false, false, true); | |
| 173 | + forge::metal::matmul(a, b, sacc_gpu, false, false, true, K_t::Simdgroup); | |
| 174 | + forge::metal::sync(); | |
| 175 | + expect_close(sacc_gpu, sacc_ref, "simd matmul accumulate=true"); | |
| 176 | +} | |
| 177 | + | |
| 178 | +void test_metal_elementwise() { | |
| 179 | + std::printf("metal elementwise parity\n"); | |
| 180 | + std::mt19937 rng(777); | |
| 181 | + const int64_t n = 65537; // non-multiple of the threadgroup size | |
| 182 | + forge::Tensor x = forge::Tensor::empty({n}); | |
| 183 | + forge::Tensor y = forge::Tensor::empty({n}); | |
| 184 | + fill_random(x, rng, -4.0f, 4.0f); | |
| 185 | + fill_random(y, rng, -4.0f, 4.0f); | |
| 186 | + | |
| 187 | + forge::Tensor gpu = forge::Tensor::empty({n}); | |
| 188 | + forge::Tensor ref = forge::Tensor::empty({n}); | |
| 189 | + | |
| 190 | + forge::cpu::add(x, y, ref); | |
| 191 | + forge::metal::add(x, y, gpu); | |
| 192 | + forge::metal::sync(); | |
| 193 | + expect_close(gpu, ref, "add"); | |
| 194 | + | |
| 195 | + forge::cpu::mul(x, y, ref); | |
| 196 | + forge::metal::mul(x, y, gpu); | |
| 197 | + forge::metal::sync(); | |
| 198 | + expect_close(gpu, ref, "mul"); | |
| 199 | + | |
| 200 | + forge::cpu::scale(x, 0.37f, ref); | |
| 201 | + forge::metal::scale(x, 0.37f, gpu); | |
| 202 | + forge::metal::sync(); | |
| 203 | + expect_close(gpu, ref, "scale"); | |
| 204 | + | |
| 205 | + forge::cpu::silu(x, ref); | |
| 206 | + forge::metal::silu(x, gpu); | |
| 207 | + forge::metal::sync(); | |
| 208 | + expect_close(gpu, ref, "silu"); | |
| 209 | + | |
| 210 | + forge::cpu::gelu(x, ref); | |
| 211 | + forge::metal::gelu(x, gpu); | |
| 212 | + forge::metal::sync(); | |
| 213 | + expect_close(gpu, ref, "gelu"); | |
| 214 | + | |
| 215 | + // add_bias: [N, C] + [C] | |
| 216 | + const int64_t rows = 513, C = 127; | |
| 217 | + forge::Tensor xb = forge::Tensor::empty({rows, C}); | |
| 218 | + forge::Tensor bias = forge::Tensor::empty({C}); | |
| 219 | + fill_random(xb, rng); | |
| 220 | + fill_random(bias, rng); | |
| 221 | + forge::Tensor gpub = forge::Tensor::empty({rows, C}); | |
| 222 | + forge::Tensor refb = forge::Tensor::empty({rows, C}); | |
| 223 | + forge::cpu::add_bias(xb, bias, refb); | |
| 224 | + forge::metal::add_bias(xb, bias, gpub); | |
| 225 | + forge::metal::sync(); | |
| 226 | + expect_close(gpub, refb, "add_bias"); | |
| 227 | +} | |
| 228 | + | |
| 229 | +void test_metal_rowops() { | |
| 230 | + std::printf("metal softmax/norm parity\n"); | |
| 231 | + std::mt19937 rng(31337); | |
| 232 | + // Row lengths: tiny (< one simdgroup), odd, large (> threadgroup size, | |
| 233 | + // vocab-like). | |
| 234 | + for (int64_t C : {7LL, 63LL, 384LL, 4099LL}) { | |
| 235 | + const int64_t rows = 129; | |
| 236 | + forge::Tensor x = forge::Tensor::empty({rows, C}); | |
| 237 | + fill_random(x, rng, -8.0f, 8.0f); | |
| 238 | + forge::Tensor gpu = forge::Tensor::empty({rows, C}); | |
| 239 | + forge::Tensor ref = forge::Tensor::empty({rows, C}); | |
| 240 | + | |
| 241 | + char label[64]; | |
| 242 | + forge::cpu::softmax(x, ref); | |
| 243 | + forge::metal::softmax(x, gpu); | |
| 244 | + forge::metal::sync(); | |
| 245 | + std::snprintf(label, sizeof(label), "softmax C=%lld", C); | |
| 246 | + expect_close(gpu, ref, label); | |
| 247 | + | |
| 248 | + forge::Tensor w = forge::Tensor::empty({C}); | |
| 249 | + forge::Tensor b = forge::Tensor::empty({C}); | |
| 250 | + fill_random(w, rng); | |
| 251 | + fill_random(b, rng); | |
| 252 | + | |
| 253 | + forge::cpu::rmsnorm(x, w, 1e-6f, ref); | |
| 254 | + forge::metal::rmsnorm(x, w, 1e-6f, gpu); | |
| 255 | + forge::metal::sync(); | |
| 256 | + std::snprintf(label, sizeof(label), "rmsnorm C=%lld", C); | |
| 257 | + expect_close(gpu, ref, label); | |
| 258 | + | |
| 259 | + forge::cpu::layernorm(x, w, b, 1e-6f, ref); | |
| 260 | + forge::metal::layernorm(x, w, b, 1e-6f, gpu); | |
| 261 | + forge::metal::sync(); | |
| 262 | + std::snprintf(label, sizeof(label), "layernorm C=%lld", C); | |
| 263 | + expect_close(gpu, ref, label); | |
| 264 | + } | |
| 265 | +} | |
| 266 | + | |
| 267 | +void test_batched_encoding() { | |
| 268 | + std::printf("batched encoding (many dispatches, one sync)\n"); | |
| 269 | + std::mt19937 rng(55); | |
| 270 | + const int64_t n = 4096; | |
| 271 | + forge::Tensor x = forge::Tensor::empty({n}); | |
| 272 | + fill_random(x, rng); | |
| 273 | + // chain of 20 dependent ops in ONE command buffer: out = ((x+x)*0.5) etc. | |
| 274 | + forge::Tensor cur = forge::Tensor::empty({n}); | |
| 275 | + forge::metal::add(x, x, cur); // cur = 2x | |
| 276 | + for (int i = 0; i < 19; ++i) { | |
| 277 | + forge::Tensor next = forge::Tensor::empty({n}); | |
| 278 | + forge::metal::scale(cur, 0.9f, next); | |
| 279 | + cur = next; | |
| 280 | + } | |
| 281 | + forge::metal::sync(); | |
| 282 | + // expected: 2 * 0.9^19 * x | |
| 283 | + const float k = 2.0f * std::pow(0.9f, 19.0f); | |
| 284 | + forge::Tensor ref = forge::Tensor::empty({n}); | |
| 285 | + forge::cpu::scale(x, k, ref); | |
| 286 | + expect_close(cur, ref, "20-dispatch serial chain", 1e-5f); | |
| 287 | +} | |
| 288 | + | |
| 289 | +// Full-model forward+backward on CPU vs Metal: one test that exercises | |
| 290 | +// every training kernel (embedding fwd/bwd, rope, attention fwd/dq/dkv, | |
| 291 | +// matmul all variants, norms fwd/bwd, activations fwd/bwd, CE fwd+grad, | |
| 292 | +// accumulate/axpy) through the real autograd tape. | |
| 293 | +void test_backend_parity_model(const char* label, const forge::ModelConfig& cfg) { | |
| 294 | + std::printf("backend parity: %s\n", label); | |
| 295 | + forge::nn::Transformer model(cfg, 7); | |
| 296 | + | |
| 297 | + std::mt19937_64 rng(21); | |
| 298 | + forge::Tensor ids = forge::Tensor::empty({2, 6}, forge::DType::I32); | |
| 299 | + forge::Tensor targets = forge::Tensor::empty({2 * 6}, forge::DType::I32); | |
| 300 | + forge::cpu::fill_uniform_int(ids, 0, cfg.vocab_size, rng); | |
| 301 | + forge::cpu::fill_uniform_int(targets, 0, cfg.vocab_size, rng); | |
| 302 | + targets.data<int32_t>()[5] = -1; // exercise ignore_index | |
| 303 | + | |
| 304 | + struct Saved { std::string name; forge::Tensor grad; }; | |
| 305 | + std::vector<Saved> cpu_grads; | |
| 306 | + float cpu_loss = 0.0f; | |
| 307 | + | |
| 308 | + { | |
| 309 | + forge::ops::set_backend(forge::ops::Backend::CPU); | |
| 310 | + model.zero_grad(); | |
| 311 | + forge::Var loss = model.loss(ids, targets); | |
| 312 | + forge::Tape::get().backward(loss); | |
| 313 | + cpu_loss = loss.value().data<float>()[0]; | |
| 314 | + std::unordered_map<const void*, bool> seen; | |
| 315 | + for (const auto& [name, p] : model.named_parameters()) { | |
| 316 | + if (!seen.emplace(p.id(), true).second) continue; | |
| 317 | + forge::Tensor copy = forge::Tensor::empty(p.grad().shape()); | |
| 318 | + std::memcpy(copy.raw(), p.grad().raw(), copy.nbytes()); | |
| 319 | + cpu_grads.push_back({name, copy}); | |
| 320 | + } | |
| 321 | + } | |
| 322 | + | |
| 323 | + float metal_loss = 0.0f; | |
| 324 | + { | |
| 325 | + forge::ops::set_backend(forge::ops::Backend::Metal); | |
| 326 | + model.zero_grad(); | |
| 327 | + forge::Var loss = model.loss(ids, targets); | |
| 328 | + forge::Tape::get().backward(loss); | |
| 329 | + forge::metal::sync(); | |
| 330 | + metal_loss = loss.value().data<float>()[0]; | |
| 331 | + forge::ops::set_backend(forge::ops::Backend::CPU); | |
| 332 | + } | |
| 333 | + | |
| 334 | + char lbl[96]; | |
| 335 | + std::snprintf(lbl, sizeof(lbl), "%s loss (cpu %.5f vs gpu %.5f)", label, | |
| 336 | + double(cpu_loss), double(metal_loss)); | |
| 337 | + expect(std::fabs(cpu_loss - metal_loss) <= 1e-4f, lbl); | |
| 338 | + | |
| 339 | + size_t idx = 0; | |
| 340 | + float worst = 0.0f; | |
| 341 | + std::string worst_name; | |
| 342 | + std::unordered_map<const void*, bool> seen; | |
| 343 | + for (const auto& [name, p] : model.named_parameters()) { | |
| 344 | + if (!seen.emplace(p.id(), true).second) continue; | |
| 345 | + const forge::Tensor& want = cpu_grads[idx++].grad; | |
| 346 | + const float* pg = p.grad().data<float>(); | |
| 347 | + const float* pw = want.data<float>(); | |
| 348 | + for (int64_t i = 0; i < want.numel(); ++i) { | |
| 349 | + const float e = std::fabs(pg[i] - pw[i]); | |
| 350 | + if (e > worst) { worst = e; worst_name = name; } | |
| 351 | + } | |
| 352 | + } | |
| 353 | + std::snprintf(lbl, sizeof(lbl), "%s all grads (worst %.2e @ %s)", label, | |
| 354 | + double(worst), worst_name.c_str()); | |
| 355 | + expect(worst <= kTol, lbl); | |
| 356 | +} | |
| 357 | + | |
| 358 | +// Fused flash attention vs the CPU reference: forward output, the saved | |
| 359 | +// logsumexp, and all three input gradients. Covers MHA and GQA, causal and | |
| 360 | +// non-causal, and a head_dim on the supported list (the model-level parity | |
| 361 | +// tests above use head_dim 8, which deliberately falls back to the unfused | |
| 362 | +// kernel — so without this the fused path would go untested). | |
| 363 | +void test_flash_attention() { | |
| 364 | + std::printf("flash attention parity (fused vs cpu reference)\n"); | |
| 365 | + std::mt19937 rng(2024); | |
| 366 | + | |
| 367 | + struct Case { int64_t B, T, H, HKV, HD; bool causal; const char* tag; }; | |
| 368 | + const Case cases[] = { | |
| 369 | + {2, 37, 4, 4, 64, true, "MHA causal hd64 (ragged T)"}, | |
| 370 | + {2, 64, 4, 2, 64, true, "GQA causal hd64 (2q/1kv)"}, | |
| 371 | + {1, 33, 2, 2, 32, true, "MHA causal hd32"}, | |
| 372 | + {2, 24, 2, 2, 64, false, "MHA non-causal hd64"}, | |
| 373 | + {1, 40, 1, 1, 128, true, "single head hd128"}, | |
| 374 | + }; | |
| 375 | + | |
| 376 | + for (const auto& c : cases) { | |
| 377 | + const int64_t Cq = c.H * c.HD, Ckv = c.HKV * c.HD; | |
| 378 | + forge::Tensor q = forge::Tensor::empty({c.B, c.T, Cq}); | |
| 379 | + forge::Tensor k = forge::Tensor::empty({c.B, c.T, Ckv}); | |
| 380 | + forge::Tensor v = forge::Tensor::empty({c.B, c.T, Ckv}); | |
| 381 | + fill_random(q, rng, -1.5f, 1.5f); | |
| 382 | + fill_random(k, rng, -1.5f, 1.5f); | |
| 383 | + fill_random(v, rng, -1.5f, 1.5f); | |
| 384 | + const float scale = 1.0f / std::sqrt(float(c.HD)); | |
| 385 | + | |
| 386 | + forge::Tensor ref_out = forge::Tensor::empty({c.B, c.T, Cq}); | |
| 387 | + forge::Tensor probs = forge::Tensor::empty({c.B, c.H, c.T, c.T}); | |
| 388 | + forge::cpu::attention(q, k, v, c.H, c.HKV, c.causal, scale, ref_out, &probs); | |
| 389 | + | |
| 390 | + forge::Tensor gpu_out = forge::Tensor::empty({c.B, c.T, Cq}); | |
| 391 | + forge::Tensor lse = forge::Tensor::empty({c.B, c.H, c.T}); | |
| 392 | + forge::metal::flash_attention(q, k, v, c.H, c.HKV, c.causal, scale, gpu_out, lse); | |
| 393 | + forge::metal::sync(); | |
| 394 | + | |
| 395 | + char label[96]; | |
| 396 | + std::snprintf(label, sizeof(label), "%s fwd", c.tag); | |
| 397 | + expect_close(gpu_out, ref_out, label); | |
| 398 | + | |
| 399 | + // backward: same upstream gradient into both paths | |
| 400 | + forge::Tensor dout = forge::Tensor::empty({c.B, c.T, Cq}); | |
| 401 | + fill_random(dout, rng); | |
| 402 | + | |
| 403 | + forge::Tensor rdq = forge::Tensor::zeros({c.B, c.T, Cq}); | |
| 404 | + forge::Tensor rdk = forge::Tensor::zeros({c.B, c.T, Ckv}); | |
| 405 | + forge::Tensor rdv = forge::Tensor::zeros({c.B, c.T, Ckv}); | |
| 406 | + forge::cpu::attention_backward(q, k, v, probs, ref_out, dout, c.H, c.HKV, scale, | |
| 407 | + rdq, rdk, rdv); | |
| 408 | + | |
| 409 | + forge::Tensor gdq = forge::Tensor::zeros({c.B, c.T, Cq}); | |
| 410 | + forge::Tensor gdk = forge::Tensor::zeros({c.B, c.T, Ckv}); | |
| 411 | + forge::Tensor gdv = forge::Tensor::zeros({c.B, c.T, Ckv}); | |
| 412 | + forge::metal::flash_attention_backward(q, k, v, gpu_out, lse, dout, c.H, c.HKV, | |
| 413 | + c.causal, scale, gdq, gdk, gdv); | |
| 414 | + forge::metal::sync(); | |
| 415 | + | |
| 416 | + std::snprintf(label, sizeof(label), "%s dq", c.tag); | |
| 417 | + expect_close(gdq, rdq, label); | |
| 418 | + std::snprintf(label, sizeof(label), "%s dk", c.tag); | |
| 419 | + expect_close(gdk, rdk, label); | |
| 420 | + std::snprintf(label, sizeof(label), "%s dv", c.tag); | |
| 421 | + expect_close(gdv, rdv, label); | |
| 422 | + } | |
| 423 | +} | |
| 424 | + | |
| 425 | +void test_adamw_kernel() { | |
| 426 | + std::printf("adamw kernel parity\n"); | |
| 427 | + std::mt19937 rng(99); | |
| 428 | + const int64_t n = 4097; | |
| 429 | + forge::Tensor w_gpu = forge::Tensor::empty({n}); | |
| 430 | + forge::Tensor g = forge::Tensor::empty({n}); | |
| 431 | + fill_random(w_gpu, rng); | |
| 432 | + fill_random(g, rng); | |
| 433 | + forge::Tensor w_cpu = forge::Tensor::empty({n}); | |
| 434 | + std::memcpy(w_cpu.raw(), w_gpu.raw(), w_gpu.nbytes()); | |
| 435 | + forge::Tensor m_gpu = forge::Tensor::zeros({n}); | |
| 436 | + forge::Tensor v_gpu = forge::Tensor::zeros({n}); | |
| 437 | + forge::Tensor m_cpu = forge::Tensor::zeros({n}); | |
| 438 | + forge::Tensor v_cpu = forge::Tensor::zeros({n}); | |
| 439 | + | |
| 440 | + const float lr = 1e-3f, b1 = 0.9f, b2 = 0.95f, eps = 1e-8f, wd = 0.1f, gs = 0.7f; | |
| 441 | + for (int64_t t = 1; t <= 3; ++t) { | |
| 442 | + forge::metal::adamw_step(w_gpu, g, m_gpu, v_gpu, lr, b1, b2, t, eps, wd, gs); | |
| 443 | + forge::metal::sync(); | |
| 444 | + const float bc1 = 1.0f - std::pow(b1, float(t)); | |
| 445 | + const float bc2 = 1.0f - std::pow(b2, float(t)); | |
| 446 | + float* w = w_cpu.data<float>(); | |
| 447 | + float* m = m_cpu.data<float>(); | |
| 448 | + float* v = v_cpu.data<float>(); | |
| 449 | + const float* gp = g.data<float>(); | |
| 450 | + for (int64_t i = 0; i < n; ++i) { | |
| 451 | + const float grad = gp[i] * gs; | |
| 452 | + m[i] = b1 * m[i] + (1 - b1) * grad; | |
| 453 | + v[i] = b2 * v[i] + (1 - b2) * grad * grad; | |
| 454 | + w[i] -= lr * ((m[i] / bc1) / (std::sqrt(v[i] / bc2) + eps) + wd * w[i]); | |
| 455 | + } | |
| 456 | + } | |
| 457 | + expect_close(w_gpu, w_cpu, "adamw 3 steps", 1e-5f); | |
| 458 | + | |
| 459 | + // sumsq + sum reductions | |
| 460 | + forge::Tensor out = forge::Tensor::empty({1}); | |
| 461 | + forge::metal::sumsq(g, out); | |
| 462 | + forge::metal::sync(); | |
| 463 | + double want = 0.0; | |
| 464 | + for (int64_t i = 0; i < n; ++i) { | |
| 465 | + const double x = double(g.data<float>()[i]); | |
| 466 | + want += x * x; | |
| 467 | + } | |
| 468 | + expect(std::fabs(out.data<float>()[0] - float(want)) <= 1e-2f, "sumsq"); | |
| 469 | + forge::metal::sum(g, out, 0.5f); | |
| 470 | + forge::metal::sync(); | |
| 471 | + double s = 0.0; | |
| 472 | + for (int64_t i = 0; i < n; ++i) s += double(g.data<float>()[i]); | |
| 473 | + expect(std::fabs(out.data<float>()[0] - float(s * 0.5)) <= 1e-2f, "sum"); | |
| 474 | +} | |
| 475 | + | |
| 476 | +} // namespace | |
| 477 | + | |
| 478 | +int main() { | |
| 479 | + NS::AutoreleasePool* pool = NS::AutoreleasePool::alloc()->init(); | |
| 480 | + | |
| 481 | + test_tensor_basics(); | |
| 482 | + test_cpu_matmul(); | |
| 483 | + test_metal_matmul(); | |
| 484 | + test_metal_elementwise(); | |
| 485 | + test_metal_rowops(); | |
| 486 | + test_batched_encoding(); | |
| 487 | + | |
| 488 | + { | |
| 489 | + forge::ModelConfig cfg; | |
| 490 | + cfg.n_layers = 2; cfg.d_model = 16; cfg.n_heads = 2; cfg.n_kv_heads = 1; | |
| 491 | + cfg.d_ff = 24; cfg.vocab_size = 11; cfg.context_length = 8; | |
| 492 | + cfg.tied_embeddings = true; | |
| 493 | + test_backend_parity_model("rmsnorm/swiglu/rope/gqa/tied [unfused hd8]", cfg); | |
| 494 | + | |
| 495 | + // head_dim 64 -> the whole model runs through the fused attention path | |
| 496 | + forge::ModelConfig fcfg; | |
| 497 | + fcfg.n_layers = 2; fcfg.d_model = 128; fcfg.n_heads = 2; fcfg.n_kv_heads = 1; | |
| 498 | + fcfg.d_ff = 96; fcfg.vocab_size = 11; fcfg.context_length = 8; | |
| 499 | + fcfg.tied_embeddings = true; | |
| 500 | + test_backend_parity_model("full model via fused attention [hd64]", fcfg); | |
| 501 | + | |
| 502 | + cfg.norm = "layernorm"; cfg.activation = "gelu"; cfg.use_rope = false; | |
| 503 | + cfg.n_kv_heads = 2; cfg.tied_embeddings = false; | |
| 504 | + test_backend_parity_model("layernorm/gelu/pos-emb/mha", cfg); | |
| 505 | + } | |
| 506 | + test_flash_attention(); | |
| 507 | + test_adamw_kernel(); | |
| 508 | + | |
| 509 | + pool->drain(); | |
| 510 | + | |
| 511 | + if (g_failures) { | |
| 512 | + std::printf("\n%d test(s) FAILED\n", g_failures); | |
| 513 | + return 1; | |
| 514 | + } | |
| 515 | + std::printf("\nall tests passed\n"); | |
| 516 | + return 0; | |
| 517 | +} | |
added
tests/test_overfit.cpp
+69 −0
@@ -0,0 +1,69 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +// | |
| 3 | +// End-to-end sanity (CLAUDE.md testing protocol #3): overfit a single batch | |
| 4 | +// of 64 sequences to loss < 0.05 within 500 steps. Tiny model so the CPU | |
| 5 | +// reference path finishes in reasonable time; the same test re-runs on the | |
| 6 | +// GPU path at M4. | |
| 7 | +#include "nn/transformer.h" | |
| 8 | +#include "ops/ops.h" | |
| 9 | +#include "train/optimizer.h" | |
| 10 | + | |
| 11 | +#include <cstdio> | |
| 12 | +#include <random> | |
| 13 | + | |
| 14 | +using namespace forge; | |
| 15 | + | |
| 16 | +int main() { | |
| 17 | + ModelConfig cfg; | |
| 18 | + cfg.n_layers = 2; | |
| 19 | + cfg.d_model = 64; | |
| 20 | + cfg.n_heads = 2; | |
| 21 | + cfg.n_kv_heads = 2; | |
| 22 | + cfg.d_ff = 172; // ~8/3 * d | |
| 23 | + cfg.vocab_size = 128; | |
| 24 | + cfg.context_length = 32; | |
| 25 | + cfg.tied_embeddings = true; | |
| 26 | + | |
| 27 | + const int64_t B = 64, T = 32; | |
| 28 | + | |
| 29 | + nn::Transformer model(cfg, 1337); | |
| 30 | + | |
| 31 | + std::mt19937_64 rng(999); | |
| 32 | + Tensor ids = Tensor::empty({B, T}, DType::I32); | |
| 33 | + cpu::fill_uniform_int(ids, 0, cfg.vocab_size, rng); | |
| 34 | + // next-token targets within the fixed batch | |
| 35 | + Tensor targets = Tensor::empty({B * T}, DType::I32); | |
| 36 | + for (int64_t b = 0; b < B; ++b) { | |
| 37 | + for (int64_t t = 0; t < T; ++t) { | |
| 38 | + targets.data<int32_t>()[b * T + t] = | |
| 39 | + (t + 1 < T) ? ids.data<int32_t>()[b * T + t + 1] : -1; | |
| 40 | + } | |
| 41 | + } | |
| 42 | + | |
| 43 | + train::AdamW::Options opts; | |
| 44 | + opts.weight_decay = 0.0f; // pure memorization task | |
| 45 | + train::AdamW opt(model.named_parameters(), opts); | |
| 46 | + | |
| 47 | + const float lr = 3e-3f; | |
| 48 | + float loss_val = -1.0f; | |
| 49 | + for (int step = 0; step < 500; ++step) { | |
| 50 | + opt.zero_grad(); | |
| 51 | + Var loss = model.loss(ids, targets); | |
| 52 | + Tape::get().backward(loss); | |
| 53 | + opt.clip_global_norm(1.0f); | |
| 54 | + opt.step(lr); | |
| 55 | + loss_val = loss.value().data<float>()[0]; | |
| 56 | + if (step % 50 == 0) std::printf("step %3d loss %.4f\n", step, double(loss_val)); | |
| 57 | + if (loss_val < 0.05f) { | |
| 58 | + std::printf("step %3d loss %.4f — target reached\n", step, double(loss_val)); | |
| 59 | + break; | |
| 60 | + } | |
| 61 | + } | |
| 62 | + | |
| 63 | + if (loss_val < 0.05f) { | |
| 64 | + std::printf("\noverfit ok (final loss %.4f)\n", double(loss_val)); | |
| 65 | + return 0; | |
| 66 | + } | |
| 67 | + std::printf("\noverfit FAILED (final loss %.4f >= 0.05)\n", double(loss_val)); | |
| 68 | + return 1; | |
| 69 | +} | |
added
tests/test_tokenizer.cpp
+108 −0
@@ -0,0 +1,108 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +// | |
| 3 | +// Tokenizer tests: decode(encode(x)) == x for byte-exact round-tripping, | |
| 4 | +// and — when tools/ has produced token ids for the same text — that the C++ | |
| 5 | +// greedy lowest-id encoder agrees with the vectorized Python encoder used to | |
| 6 | +// build the .bin files. The test writes its own tiny .model so it needs no | |
| 7 | +// data prep; the Python-agreement half runs only if FORGE_TOK_MODEL and | |
| 8 | +// FORGE_TOK_IDS are set (wired up by tests/tokenizer_agreement.py). | |
| 9 | +#include "tokenizer/bpe.h" | |
| 10 | + | |
| 11 | +#include <cstdio> | |
| 12 | +#include <cstdlib> | |
| 13 | +#include <fstream> | |
| 14 | +#include <sstream> | |
| 15 | +#include <string> | |
| 16 | +#include <vector> | |
| 17 | + | |
| 18 | +namespace { | |
| 19 | + | |
| 20 | +int g_failures = 0; | |
| 21 | + | |
| 22 | +void expect(bool cond, const char* what) { | |
| 23 | + if (cond) { | |
| 24 | + std::printf(" ok: %s\n", what); | |
| 25 | + } else { | |
| 26 | + std::printf(" FAIL: %s\n", what); | |
| 27 | + ++g_failures; | |
| 28 | + } | |
| 29 | +} | |
| 30 | + | |
| 31 | +// Minimal hand-built vocab: merges chosen so encoding "aaabdaaabac" exercises | |
| 32 | +// repeated-pair overlap handling (minbpe's worked example). | |
| 33 | +void write_test_model(const std::string& path) { | |
| 34 | + std::ofstream out(path); | |
| 35 | + out << "forgebpe v1\n"; | |
| 36 | + out << 259 << "\n"; | |
| 37 | + out << "256 97 97\n"; // 'aa' | |
| 38 | + out << "257 256 97\n"; // 'aaa' | |
| 39 | + out << "258 257 98\n"; // 'aaab' | |
| 40 | +} | |
| 41 | + | |
| 42 | +} // namespace | |
| 43 | + | |
| 44 | +int main() { | |
| 45 | + const std::string model_path = "test_tok.model"; | |
| 46 | + write_test_model(model_path); | |
| 47 | + | |
| 48 | + forge::tok::BPETokenizer tok; | |
| 49 | + tok.load(model_path); | |
| 50 | + expect(tok.vocab_size() == 259, "vocab size"); | |
| 51 | + | |
| 52 | + // "aaabdaaabac": 'aaab' merges twice, leaving d/a/c as raw bytes | |
| 53 | + const std::string text = "aaabdaaabac"; | |
| 54 | + std::vector<int32_t> ids = tok.encode(text); | |
| 55 | + expect(ids == std::vector<int32_t>({258, 100, 258, 97, 99}), "merge order"); | |
| 56 | + expect(tok.decode(ids) == text, "round-trip (merged)"); | |
| 57 | + | |
| 58 | + // Overlap: "aaaa" applies merge 256 ('aa') to BOTH non-overlapping pairs | |
| 59 | + // in one round, so 'aaa' (id 257) never forms — greedy is lowest-id-first, | |
| 60 | + // not longest-match. "aaa" does reach 257 (256 then 256+97). | |
| 61 | + expect(tok.decode(tok.encode("aaaa")) == "aaaa", "round-trip (overlap)"); | |
| 62 | + expect(tok.encode("aaaa") == std::vector<int32_t>({256, 256}), "overlap merge"); | |
| 63 | + expect(tok.encode("aaa") == std::vector<int32_t>({257}), "chained merge"); | |
| 64 | + | |
| 65 | + // Bytes with no merges, and full 0-255 range including UTF-8 and NUL | |
| 66 | + std::string bytes; | |
| 67 | + for (int i = 1; i < 256; ++i) bytes += char(i); | |
| 68 | + expect(tok.decode(tok.encode(bytes)) == bytes, "round-trip (all byte values)"); | |
| 69 | + const std::string utf8 = "héllo wörld — ünïcode ✓"; | |
| 70 | + expect(tok.decode(tok.encode(utf8)) == utf8, "round-trip (utf-8)"); | |
| 71 | + expect(tok.encode("").empty(), "empty input"); | |
| 72 | + | |
| 73 | + std::remove(model_path.c_str()); | |
| 74 | + | |
| 75 | + // Optional: agreement with the Python encoder on real text. | |
| 76 | + const char* py_model = std::getenv("FORGE_TOK_MODEL"); | |
| 77 | + const char* py_ids = std::getenv("FORGE_TOK_IDS"); | |
| 78 | + const char* py_text = std::getenv("FORGE_TOK_TEXT"); | |
| 79 | + if (py_model && py_ids && py_text) { | |
| 80 | + forge::tok::BPETokenizer real; | |
| 81 | + real.load(py_model); | |
| 82 | + std::ifstream tf(py_text, std::ios::binary); | |
| 83 | + std::stringstream ts; | |
| 84 | + ts << tf.rdbuf(); | |
| 85 | + std::vector<int32_t> mine = real.encode(ts.str()); | |
| 86 | + | |
| 87 | + std::ifstream idf(py_ids); | |
| 88 | + std::vector<int32_t> theirs; | |
| 89 | + int32_t v; | |
| 90 | + while (idf >> v) theirs.push_back(v); | |
| 91 | + | |
| 92 | + char label[128]; | |
| 93 | + std::snprintf(label, sizeof(label), | |
| 94 | + "python encoder agreement (%zu vs %zu tokens)", mine.size(), | |
| 95 | + theirs.size()); | |
| 96 | + expect(mine == theirs, label); | |
| 97 | + expect(real.decode(mine) == ts.str(), "round-trip (real vocab, real text)"); | |
| 98 | + } else { | |
| 99 | + std::printf(" skip: python agreement (set FORGE_TOK_MODEL/IDS/TEXT)\n"); | |
| 100 | + } | |
| 101 | + | |
| 102 | + if (g_failures) { | |
| 103 | + std::printf("\n%d tokenizer test(s) FAILED\n", g_failures); | |
| 104 | + return 1; | |
| 105 | + } | |
| 106 | + std::printf("\nall tokenizer tests passed\n"); | |
| 107 | + return 0; | |
| 108 | +} | |
added
tests/tokenizer_agreement.py
+61 −0
@@ -0,0 +1,61 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +"""Cross-check the Python (vectorized) and C++ (greedy lowest-id) BPE | |
| 3 | +encoders on real text. | |
| 4 | + | |
| 5 | +Encodes a text sample with tools/prepare_data.py's encoder, dumps the ids, | |
| 6 | +then runs test_tokenizer with FORGE_TOK_* pointing at them so the C++ side | |
| 7 | +compares. Exits nonzero on disagreement. | |
| 8 | + | |
| 9 | +Usage: | |
| 10 | + python3 tests/tokenizer_agreement.py --model data/tinystories/tok4096.model \ | |
| 11 | + --text data/tinystories/TinyStoriesV2-GPT4-valid.txt --bytes 200000 \ | |
| 12 | + --binary build/test_tokenizer | |
| 13 | +""" | |
| 14 | +import argparse | |
| 15 | +import os | |
| 16 | +import subprocess | |
| 17 | +import sys | |
| 18 | +import tempfile | |
| 19 | + | |
| 20 | +sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "tools")) | |
| 21 | +import prepare_data as pd # noqa: E402 | |
| 22 | + | |
| 23 | + | |
| 24 | +def main(): | |
| 25 | + ap = argparse.ArgumentParser() | |
| 26 | + ap.add_argument("--model", required=True) | |
| 27 | + ap.add_argument("--text", required=True) | |
| 28 | + ap.add_argument("--bytes", type=int, default=200000) | |
| 29 | + ap.add_argument("--binary", default="build/test_tokenizer") | |
| 30 | + args = ap.parse_args() | |
| 31 | + | |
| 32 | + with open(args.text, "rb") as f: | |
| 33 | + sample = f.read(args.bytes) | |
| 34 | + # cut at a story boundary so the comparison matches how .bin is built | |
| 35 | + sep = b"<|endoftext|>" | |
| 36 | + if sep in sample: | |
| 37 | + sample = sample[: sample.rindex(sep)] | |
| 38 | + | |
| 39 | + pd._MERGES = pd.load_merges(args.model) | |
| 40 | + ids = pd.encode_block(sample) | |
| 41 | + print(f"python: {len(sample)} bytes -> {len(ids)} tokens") | |
| 42 | + | |
| 43 | + with tempfile.TemporaryDirectory() as td: | |
| 44 | + text_path = os.path.join(td, "text.bin") | |
| 45 | + ids_path = os.path.join(td, "ids.txt") | |
| 46 | + with open(text_path, "wb") as f: | |
| 47 | + f.write(sample) | |
| 48 | + with open(ids_path, "w") as f: | |
| 49 | + f.write("\n".join(str(int(i)) for i in ids)) | |
| 50 | + | |
| 51 | + env = dict(os.environ) | |
| 52 | + env["FORGE_TOK_MODEL"] = os.path.abspath(args.model) | |
| 53 | + env["FORGE_TOK_TEXT"] = text_path | |
| 54 | + env["FORGE_TOK_IDS"] = ids_path | |
| 55 | + rc = subprocess.call([os.path.abspath(args.binary)], env=env, | |
| 56 | + cwd=os.path.dirname(os.path.abspath(args.binary))) | |
| 57 | + sys.exit(rc) | |
| 58 | + | |
| 59 | + | |
| 60 | +if __name__ == "__main__": | |
| 61 | + main() | |
added
third_party/metal-cpp/Foundation/Foundation.hpp
+47 −0
@@ -0,0 +1,47 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Foundation/Foundation.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2024 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 24 | + | |
| 25 | +#include "NSArray.hpp" | |
| 26 | +#include "NSAutoreleasePool.hpp" | |
| 27 | +#include "NSBundle.hpp" | |
| 28 | +#include "NSData.hpp" | |
| 29 | +#include "NSDate.hpp" | |
| 30 | +#include "NSDefines.hpp" | |
| 31 | +#include "NSDictionary.hpp" | |
| 32 | +#include "NSEnumerator.hpp" | |
| 33 | +#include "NSError.hpp" | |
| 34 | +#include "NSLock.hpp" | |
| 35 | +#include "NSNotification.hpp" | |
| 36 | +#include "NSNumber.hpp" | |
| 37 | +#include "NSObject.hpp" | |
| 38 | +#include "NSPrivate.hpp" | |
| 39 | +#include "NSProcessInfo.hpp" | |
| 40 | +#include "NSRange.hpp" | |
| 41 | +#include "NSSet.hpp" | |
| 42 | +#include "NSSharedPtr.hpp" | |
| 43 | +#include "NSString.hpp" | |
| 44 | +#include "NSTypes.hpp" | |
| 45 | +#include "NSURL.hpp" | |
| 46 | + | |
| 47 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
added
third_party/metal-cpp/Foundation/NSArray.hpp
+124 −0
@@ -0,0 +1,124 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Foundation/NSArray.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2024 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 24 | + | |
| 25 | +#include "NSObject.hpp" | |
| 26 | +#include "NSTypes.hpp" | |
| 27 | +#include "NSEnumerator.hpp" | |
| 28 | + | |
| 29 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 30 | + | |
| 31 | +namespace NS | |
| 32 | +{ | |
| 33 | +class Array : public Copying<Array> | |
| 34 | +{ | |
| 35 | +public: | |
| 36 | + static Array* array(); | |
| 37 | + static Array* array(const Object* pObject); | |
| 38 | + static Array* array(const Object* const* pObjects, UInteger count); | |
| 39 | + | |
| 40 | + static Array* alloc(); | |
| 41 | + | |
| 42 | + Array* init(); | |
| 43 | + Array* init(const Object* const* pObjects, UInteger count); | |
| 44 | + Array* init(const class Coder* pCoder); | |
| 45 | + | |
| 46 | + template <class _Object = Object> | |
| 47 | + _Object* object(UInteger index) const; | |
| 48 | + UInteger count() const; | |
| 49 | + Enumerator<Object>* objectEnumerator() const; | |
| 50 | +}; | |
| 51 | +} | |
| 52 | + | |
| 53 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 54 | + | |
| 55 | +_NS_INLINE NS::Array* NS::Array::array() | |
| 56 | +{ | |
| 57 | + return Object::sendMessage<Array*>(_NS_PRIVATE_CLS(NSArray), _NS_PRIVATE_SEL(array)); | |
| 58 | +} | |
| 59 | + | |
| 60 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 61 | + | |
| 62 | +_NS_INLINE NS::Array* NS::Array::array(const Object* pObject) | |
| 63 | +{ | |
| 64 | + return Object::sendMessage<Array*>(_NS_PRIVATE_CLS(NSArray), _NS_PRIVATE_SEL(arrayWithObject_), pObject); | |
| 65 | +} | |
| 66 | + | |
| 67 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 68 | + | |
| 69 | +_NS_INLINE NS::Array* NS::Array::array(const Object* const* pObjects, UInteger count) | |
| 70 | +{ | |
| 71 | + return Object::sendMessage<Array*>(_NS_PRIVATE_CLS(NSArray), _NS_PRIVATE_SEL(arrayWithObjects_count_), pObjects, count); | |
| 72 | +} | |
| 73 | + | |
| 74 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 75 | + | |
| 76 | +_NS_INLINE NS::Array* NS::Array::alloc() | |
| 77 | +{ | |
| 78 | + return NS::Object::alloc<Array>(_NS_PRIVATE_CLS(NSArray)); | |
| 79 | +} | |
| 80 | + | |
| 81 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 82 | + | |
| 83 | +_NS_INLINE NS::Array* NS::Array::init() | |
| 84 | +{ | |
| 85 | + return NS::Object::init<Array>(); | |
| 86 | +} | |
| 87 | + | |
| 88 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 89 | + | |
| 90 | +_NS_INLINE NS::Array* NS::Array::init(const Object* const* pObjects, UInteger count) | |
| 91 | +{ | |
| 92 | + return Object::sendMessage<Array*>(this, _NS_PRIVATE_SEL(initWithObjects_count_), pObjects, count); | |
| 93 | +} | |
| 94 | + | |
| 95 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 96 | + | |
| 97 | +_NS_INLINE NS::Array* NS::Array::init(const class Coder* pCoder) | |
| 98 | +{ | |
| 99 | + return Object::sendMessage<Array*>(this, _NS_PRIVATE_SEL(initWithCoder_), pCoder); | |
| 100 | +} | |
| 101 | + | |
| 102 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 103 | + | |
| 104 | +_NS_INLINE NS::UInteger NS::Array::count() const | |
| 105 | +{ | |
| 106 | + return Object::sendMessage<UInteger>(this, _NS_PRIVATE_SEL(count)); | |
| 107 | +} | |
| 108 | + | |
| 109 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 110 | + | |
| 111 | +template <class _Object> | |
| 112 | +_NS_INLINE _Object* NS::Array::object(UInteger index) const | |
| 113 | +{ | |
| 114 | + return Object::sendMessage<_Object*>(this, _NS_PRIVATE_SEL(objectAtIndex_), index); | |
| 115 | +} | |
| 116 | + | |
| 117 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 118 | + | |
| 119 | +_NS_INLINE NS::Enumerator<NS::Object>* NS::Array::objectEnumerator() const | |
| 120 | +{ | |
| 121 | + return NS::Object::sendMessage<Enumerator<NS::Object>*>(this, _NS_PRIVATE_SEL(objectEnumerator)); | |
| 122 | +} | |
| 123 | + | |
| 124 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
added
third_party/metal-cpp/Foundation/NSAutoreleasePool.hpp
+83 −0
@@ -0,0 +1,83 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Foundation/NSAutoreleasePool.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2024 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 24 | + | |
| 25 | +#include "NSDefines.hpp" | |
| 26 | +#include "NSObject.hpp" | |
| 27 | +#include "NSPrivate.hpp" | |
| 28 | +#include "NSTypes.hpp" | |
| 29 | + | |
| 30 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 31 | + | |
| 32 | +namespace NS | |
| 33 | +{ | |
| 34 | +class AutoreleasePool : public Object | |
| 35 | +{ | |
| 36 | +public: | |
| 37 | + static AutoreleasePool* alloc(); | |
| 38 | + AutoreleasePool* init(); | |
| 39 | + | |
| 40 | + void drain(); | |
| 41 | + | |
| 42 | + void addObject(Object* pObject); | |
| 43 | + | |
| 44 | + static void showPools(); | |
| 45 | +}; | |
| 46 | +} | |
| 47 | + | |
| 48 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 49 | + | |
| 50 | +_NS_INLINE NS::AutoreleasePool* NS::AutoreleasePool::alloc() | |
| 51 | +{ | |
| 52 | + return NS::Object::alloc<AutoreleasePool>(_NS_PRIVATE_CLS(NSAutoreleasePool)); | |
| 53 | +} | |
| 54 | + | |
| 55 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 56 | + | |
| 57 | +_NS_INLINE NS::AutoreleasePool* NS::AutoreleasePool::init() | |
| 58 | +{ | |
| 59 | + return NS::Object::init<AutoreleasePool>(); | |
| 60 | +} | |
| 61 | + | |
| 62 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 63 | + | |
| 64 | +_NS_INLINE void NS::AutoreleasePool::drain() | |
| 65 | +{ | |
| 66 | + Object::sendMessage<void>(this, _NS_PRIVATE_SEL(drain)); | |
| 67 | +} | |
| 68 | + | |
| 69 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 70 | + | |
| 71 | +_NS_INLINE void NS::AutoreleasePool::addObject(Object* pObject) | |
| 72 | +{ | |
| 73 | + Object::sendMessage<void>(this, _NS_PRIVATE_SEL(addObject_), pObject); | |
| 74 | +} | |
| 75 | + | |
| 76 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 77 | + | |
| 78 | +_NS_INLINE void NS::AutoreleasePool::showPools() | |
| 79 | +{ | |
| 80 | + Object::sendMessage<void>(_NS_PRIVATE_CLS(NSAutoreleasePool), _NS_PRIVATE_SEL(showPools)); | |
| 81 | +} | |
| 82 | + | |
| 83 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
added
third_party/metal-cpp/Foundation/NSBundle.hpp
+374 −0
@@ -0,0 +1,374 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Foundation/NSBundle.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2024 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 24 | + | |
| 25 | +#include "NSDefines.hpp" | |
| 26 | +#include "NSNotification.hpp" | |
| 27 | +#include "NSObject.hpp" | |
| 28 | +#include "NSTypes.hpp" | |
| 29 | + | |
| 30 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 31 | + | |
| 32 | +namespace NS | |
| 33 | +{ | |
| 34 | +_NS_CONST(NotificationName, BundleDidLoadNotification); | |
| 35 | +_NS_CONST(NotificationName, BundleResourceRequestLowDiskSpaceNotification); | |
| 36 | + | |
| 37 | +class String* LocalizedString(const String* pKey, const String*); | |
| 38 | +class String* LocalizedStringFromTable(const String* pKey, const String* pTbl, const String*); | |
| 39 | +class String* LocalizedStringFromTableInBundle(const String* pKey, const String* pTbl, const class Bundle* pBdle, const String*); | |
| 40 | +class String* LocalizedStringWithDefaultValue(const String* pKey, const String* pTbl, const class Bundle* pBdle, const String* pVal, const String*); | |
| 41 | + | |
| 42 | +class Bundle : public Referencing<Bundle> | |
| 43 | +{ | |
| 44 | +public: | |
| 45 | + static Bundle* mainBundle(); | |
| 46 | + | |
| 47 | + static Bundle* bundle(const class String* pPath); | |
| 48 | + static Bundle* bundle(const class URL* pURL); | |
| 49 | + | |
| 50 | + static class Array* allBundles(); | |
| 51 | + static class Array* allFrameworks(); | |
| 52 | + | |
| 53 | + static Bundle* alloc(); | |
| 54 | + | |
| 55 | + Bundle* init(const class String* pPath); | |
| 56 | + Bundle* init(const class URL* pURL); | |
| 57 | + | |
| 58 | + bool load(); | |
| 59 | + bool unload(); | |
| 60 | + | |
| 61 | + bool isLoaded() const; | |
| 62 | + | |
| 63 | + bool preflightAndReturnError(class Error** pError) const; | |
| 64 | + bool loadAndReturnError(class Error** pError); | |
| 65 | + | |
| 66 | + class URL* bundleURL() const; | |
| 67 | + class URL* resourceURL() const; | |
| 68 | + class URL* executableURL() const; | |
| 69 | + class URL* URLForAuxiliaryExecutable(const class String* pExecutableName) const; | |
| 70 | + | |
| 71 | + class URL* privateFrameworksURL() const; | |
| 72 | + class URL* sharedFrameworksURL() const; | |
| 73 | + class URL* sharedSupportURL() const; | |
| 74 | + class URL* builtInPlugInsURL() const; | |
| 75 | + class URL* appStoreReceiptURL() const; | |
| 76 | + | |
| 77 | + class String* bundlePath() const; | |
| 78 | + class String* resourcePath() const; | |
| 79 | + class String* executablePath() const; | |
| 80 | + class String* pathForAuxiliaryExecutable(const class String* pExecutableName) const; | |
| 81 | + | |
| 82 | + class String* privateFrameworksPath() const; | |
| 83 | + class String* sharedFrameworksPath() const; | |
| 84 | + class String* sharedSupportPath() const; | |
| 85 | + class String* builtInPlugInsPath() const; | |
| 86 | + | |
| 87 | + class String* bundleIdentifier() const; | |
| 88 | + class Dictionary* infoDictionary() const; | |
| 89 | + class Dictionary* localizedInfoDictionary() const; | |
| 90 | + class Object* objectForInfoDictionaryKey(const class String* pKey); | |
| 91 | + | |
| 92 | + class String* localizedString(const class String* pKey, const class String* pValue = nullptr, const class String* pTableName = nullptr) const; | |
| 93 | +}; | |
| 94 | +} | |
| 95 | + | |
| 96 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 97 | + | |
| 98 | +_NS_PRIVATE_DEF_CONST(NS::NotificationName, BundleDidLoadNotification); | |
| 99 | +_NS_PRIVATE_DEF_CONST(NS::NotificationName, BundleResourceRequestLowDiskSpaceNotification); | |
| 100 | + | |
| 101 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 102 | + | |
| 103 | +_NS_INLINE NS::String* NS::LocalizedString(const String* pKey, const String*) | |
| 104 | +{ | |
| 105 | + return Bundle::mainBundle()->localizedString(pKey, nullptr, nullptr); | |
| 106 | +} | |
| 107 | + | |
| 108 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 109 | + | |
| 110 | +_NS_INLINE NS::String* NS::LocalizedStringFromTable(const String* pKey, const String* pTbl, const String*) | |
| 111 | +{ | |
| 112 | + return Bundle::mainBundle()->localizedString(pKey, nullptr, pTbl); | |
| 113 | +} | |
| 114 | + | |
| 115 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 116 | + | |
| 117 | +_NS_INLINE NS::String* NS::LocalizedStringFromTableInBundle(const String* pKey, const String* pTbl, const Bundle* pBdl, const String*) | |
| 118 | +{ | |
| 119 | + return pBdl->localizedString(pKey, nullptr, pTbl); | |
| 120 | +} | |
| 121 | + | |
| 122 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 123 | + | |
| 124 | +_NS_INLINE NS::String* NS::LocalizedStringWithDefaultValue(const String* pKey, const String* pTbl, const Bundle* pBdl, const String* pVal, const String*) | |
| 125 | +{ | |
| 126 | + return pBdl->localizedString(pKey, pVal, pTbl); | |
| 127 | +} | |
| 128 | + | |
| 129 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 130 | + | |
| 131 | +_NS_INLINE NS::Bundle* NS::Bundle::mainBundle() | |
| 132 | +{ | |
| 133 | + return Object::sendMessage<Bundle*>(_NS_PRIVATE_CLS(NSBundle), _NS_PRIVATE_SEL(mainBundle)); | |
| 134 | +} | |
| 135 | + | |
| 136 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 137 | + | |
| 138 | +_NS_INLINE NS::Bundle* NS::Bundle::bundle(const class String* pPath) | |
| 139 | +{ | |
| 140 | + return Object::sendMessage<Bundle*>(_NS_PRIVATE_CLS(NSBundle), _NS_PRIVATE_SEL(bundleWithPath_), pPath); | |
| 141 | +} | |
| 142 | + | |
| 143 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 144 | + | |
| 145 | +_NS_INLINE NS::Bundle* NS::Bundle::bundle(const class URL* pURL) | |
| 146 | +{ | |
| 147 | + return Object::sendMessage<Bundle*>(_NS_PRIVATE_CLS(NSBundle), _NS_PRIVATE_SEL(bundleWithURL_), pURL); | |
| 148 | +} | |
| 149 | + | |
| 150 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 151 | + | |
| 152 | +_NS_INLINE NS::Array* NS::Bundle::allBundles() | |
| 153 | +{ | |
| 154 | + return Object::sendMessage<Array*>(_NS_PRIVATE_CLS(NSBundle), _NS_PRIVATE_SEL(allBundles)); | |
| 155 | +} | |
| 156 | + | |
| 157 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 158 | + | |
| 159 | +_NS_INLINE NS::Array* NS::Bundle::allFrameworks() | |
| 160 | +{ | |
| 161 | + return Object::sendMessage<Array*>(_NS_PRIVATE_CLS(NSBundle), _NS_PRIVATE_SEL(allFrameworks)); | |
| 162 | +} | |
| 163 | + | |
| 164 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 165 | + | |
| 166 | +_NS_INLINE NS::Bundle* NS::Bundle::alloc() | |
| 167 | +{ | |
| 168 | + return Object::sendMessage<Bundle*>(_NS_PRIVATE_CLS(NSBundle), _NS_PRIVATE_SEL(alloc)); | |
| 169 | +} | |
| 170 | + | |
| 171 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 172 | + | |
| 173 | +_NS_INLINE NS::Bundle* NS::Bundle::init(const String* pPath) | |
| 174 | +{ | |
| 175 | + return Object::sendMessage<Bundle*>(this, _NS_PRIVATE_SEL(initWithPath_), pPath); | |
| 176 | +} | |
| 177 | + | |
| 178 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 179 | + | |
| 180 | +_NS_INLINE NS::Bundle* NS::Bundle::init(const URL* pURL) | |
| 181 | +{ | |
| 182 | + return Object::sendMessage<Bundle*>(this, _NS_PRIVATE_SEL(initWithURL_), pURL); | |
| 183 | +} | |
| 184 | + | |
| 185 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 186 | + | |
| 187 | +_NS_INLINE bool NS::Bundle::load() | |
| 188 | +{ | |
| 189 | + return Object::sendMessage<bool>(this, _NS_PRIVATE_SEL(load)); | |
| 190 | +} | |
| 191 | + | |
| 192 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 193 | + | |
| 194 | +_NS_INLINE bool NS::Bundle::unload() | |
| 195 | +{ | |
| 196 | + return Object::sendMessage<bool>(this, _NS_PRIVATE_SEL(unload)); | |
| 197 | +} | |
| 198 | + | |
| 199 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 200 | + | |
| 201 | +_NS_INLINE bool NS::Bundle::isLoaded() const | |
| 202 | +{ | |
| 203 | + return Object::sendMessage<bool>(this, _NS_PRIVATE_SEL(isLoaded)); | |
| 204 | +} | |
| 205 | + | |
| 206 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 207 | + | |
| 208 | +_NS_INLINE bool NS::Bundle::preflightAndReturnError(Error** pError) const | |
| 209 | +{ | |
| 210 | + return Object::sendMessage<bool>(this, _NS_PRIVATE_SEL(preflightAndReturnError_), pError); | |
| 211 | +} | |
| 212 | + | |
| 213 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 214 | + | |
| 215 | +_NS_INLINE bool NS::Bundle::loadAndReturnError(Error** pError) | |
| 216 | +{ | |
| 217 | + return Object::sendMessage<bool>(this, _NS_PRIVATE_SEL(loadAndReturnError_), pError); | |
| 218 | +} | |
| 219 | + | |
| 220 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 221 | + | |
| 222 | +_NS_INLINE NS::URL* NS::Bundle::bundleURL() const | |
| 223 | +{ | |
| 224 | + return Object::sendMessage<URL*>(this, _NS_PRIVATE_SEL(bundleURL)); | |
| 225 | +} | |
| 226 | + | |
| 227 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 228 | + | |
| 229 | +_NS_INLINE NS::URL* NS::Bundle::resourceURL() const | |
| 230 | +{ | |
| 231 | + return Object::sendMessage<URL*>(this, _NS_PRIVATE_SEL(resourceURL)); | |
| 232 | +} | |
| 233 | + | |
| 234 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 235 | + | |
| 236 | +_NS_INLINE NS::URL* NS::Bundle::executableURL() const | |
| 237 | +{ | |
| 238 | + return Object::sendMessage<URL*>(this, _NS_PRIVATE_SEL(executableURL)); | |
| 239 | +} | |
| 240 | + | |
| 241 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 242 | + | |
| 243 | +_NS_INLINE NS::URL* NS::Bundle::URLForAuxiliaryExecutable(const String* pExecutableName) const | |
| 244 | +{ | |
| 245 | + return Object::sendMessage<URL*>(this, _NS_PRIVATE_SEL(URLForAuxiliaryExecutable_), pExecutableName); | |
| 246 | +} | |
| 247 | + | |
| 248 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 249 | + | |
| 250 | +_NS_INLINE NS::URL* NS::Bundle::privateFrameworksURL() const | |
| 251 | +{ | |
| 252 | + return Object::sendMessage<URL*>(this, _NS_PRIVATE_SEL(privateFrameworksURL)); | |
| 253 | +} | |
| 254 | + | |
| 255 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 256 | + | |
| 257 | +_NS_INLINE NS::URL* NS::Bundle::sharedFrameworksURL() const | |
| 258 | +{ | |
| 259 | + return Object::sendMessage<URL*>(this, _NS_PRIVATE_SEL(sharedFrameworksURL)); | |
| 260 | +} | |
| 261 | + | |
| 262 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 263 | + | |
| 264 | +_NS_INLINE NS::URL* NS::Bundle::sharedSupportURL() const | |
| 265 | +{ | |
| 266 | + return Object::sendMessage<URL*>(this, _NS_PRIVATE_SEL(sharedSupportURL)); | |
| 267 | +} | |
| 268 | + | |
| 269 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 270 | + | |
| 271 | +_NS_INLINE NS::URL* NS::Bundle::builtInPlugInsURL() const | |
| 272 | +{ | |
| 273 | + return Object::sendMessage<URL*>(this, _NS_PRIVATE_SEL(builtInPlugInsURL)); | |
| 274 | +} | |
| 275 | + | |
| 276 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 277 | + | |
| 278 | +_NS_INLINE NS::URL* NS::Bundle::appStoreReceiptURL() const | |
| 279 | +{ | |
| 280 | + return Object::sendMessage<URL*>(this, _NS_PRIVATE_SEL(appStoreReceiptURL)); | |
| 281 | +} | |
| 282 | + | |
| 283 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 284 | + | |
| 285 | +_NS_INLINE NS::String* NS::Bundle::bundlePath() const | |
| 286 | +{ | |
| 287 | + return Object::sendMessage<String*>(this, _NS_PRIVATE_SEL(bundlePath)); | |
| 288 | +} | |
| 289 | + | |
| 290 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 291 | + | |
| 292 | +_NS_INLINE NS::String* NS::Bundle::resourcePath() const | |
| 293 | +{ | |
| 294 | + return Object::sendMessage<String*>(this, _NS_PRIVATE_SEL(resourcePath)); | |
| 295 | +} | |
| 296 | + | |
| 297 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 298 | + | |
| 299 | +_NS_INLINE NS::String* NS::Bundle::executablePath() const | |
| 300 | +{ | |
| 301 | + return Object::sendMessage<String*>(this, _NS_PRIVATE_SEL(executablePath)); | |
| 302 | +} | |
| 303 | + | |
| 304 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 305 | + | |
| 306 | +_NS_INLINE NS::String* NS::Bundle::pathForAuxiliaryExecutable(const String* pExecutableName) const | |
| 307 | +{ | |
| 308 | + return Object::sendMessage<String*>(this, _NS_PRIVATE_SEL(pathForAuxiliaryExecutable_), pExecutableName); | |
| 309 | +} | |
| 310 | + | |
| 311 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 312 | + | |
| 313 | +_NS_INLINE NS::String* NS::Bundle::privateFrameworksPath() const | |
| 314 | +{ | |
| 315 | + return Object::sendMessage<String*>(this, _NS_PRIVATE_SEL(privateFrameworksPath)); | |
| 316 | +} | |
| 317 | + | |
| 318 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 319 | + | |
| 320 | +_NS_INLINE NS::String* NS::Bundle::sharedFrameworksPath() const | |
| 321 | +{ | |
| 322 | + return Object::sendMessage<String*>(this, _NS_PRIVATE_SEL(sharedFrameworksPath)); | |
| 323 | +} | |
| 324 | + | |
| 325 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 326 | + | |
| 327 | +_NS_INLINE NS::String* NS::Bundle::sharedSupportPath() const | |
| 328 | +{ | |
| 329 | + return Object::sendMessage<String*>(this, _NS_PRIVATE_SEL(sharedSupportPath)); | |
| 330 | +} | |
| 331 | + | |
| 332 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 333 | + | |
| 334 | +_NS_INLINE NS::String* NS::Bundle::builtInPlugInsPath() const | |
| 335 | +{ | |
| 336 | + return Object::sendMessage<String*>(this, _NS_PRIVATE_SEL(builtInPlugInsPath)); | |
| 337 | +} | |
| 338 | + | |
| 339 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 340 | + | |
| 341 | +_NS_INLINE NS::String* NS::Bundle::bundleIdentifier() const | |
| 342 | +{ | |
| 343 | + return Object::sendMessage<String*>(this, _NS_PRIVATE_SEL(bundleIdentifier)); | |
| 344 | +} | |
| 345 | + | |
| 346 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 347 | + | |
| 348 | +_NS_INLINE NS::Dictionary* NS::Bundle::infoDictionary() const | |
| 349 | +{ | |
| 350 | + return Object::sendMessage<Dictionary*>(this, _NS_PRIVATE_SEL(infoDictionary)); | |
| 351 | +} | |
| 352 | + | |
| 353 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 354 | + | |
| 355 | +_NS_INLINE NS::Dictionary* NS::Bundle::localizedInfoDictionary() const | |
| 356 | +{ | |
| 357 | + return Object::sendMessage<Dictionary*>(this, _NS_PRIVATE_SEL(localizedInfoDictionary)); | |
| 358 | +} | |
| 359 | + | |
| 360 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 361 | + | |
| 362 | +_NS_INLINE NS::Object* NS::Bundle::objectForInfoDictionaryKey(const String* pKey) | |
| 363 | +{ | |
| 364 | + return Object::sendMessage<Object*>(this, _NS_PRIVATE_SEL(objectForInfoDictionaryKey_), pKey); | |
| 365 | +} | |
| 366 | + | |
| 367 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 368 | + | |
| 369 | +_NS_INLINE NS::String* NS::Bundle::localizedString(const String* pKey, const String* pValue /* = nullptr */, const String* pTableName /* = nullptr */) const | |
| 370 | +{ | |
| 371 | + return Object::sendMessage<String*>(this, _NS_PRIVATE_SEL(localizedStringForKey_value_table_), pKey, pValue, pTableName); | |
| 372 | +} | |
| 373 | + | |
| 374 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
added
third_party/metal-cpp/Foundation/NSData.hpp
+54 −0
@@ -0,0 +1,54 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Foundation/NSData.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2024 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 24 | + | |
| 25 | +#include "NSObject.hpp" | |
| 26 | +#include "NSTypes.hpp" | |
| 27 | + | |
| 28 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 29 | + | |
| 30 | +namespace NS | |
| 31 | +{ | |
| 32 | +class Data : public Copying<Data> | |
| 33 | +{ | |
| 34 | +public: | |
| 35 | + const void* bytes() const; | |
| 36 | + UInteger length() const; | |
| 37 | +}; | |
| 38 | +} | |
| 39 | + | |
| 40 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 41 | + | |
| 42 | +_NS_INLINE const void* NS::Data::bytes() const | |
| 43 | +{ | |
| 44 | + return Object::sendMessage<void*>(this, _NS_PRIVATE_SEL(bytes)); | |
| 45 | +} | |
| 46 | + | |
| 47 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 48 | + | |
| 49 | +_NS_INLINE NS::UInteger NS::Data::length() const | |
| 50 | +{ | |
| 51 | + return Object::sendMessage<UInteger>(this, _NS_PRIVATE_SEL(length)); | |
| 52 | +} | |
| 53 | + | |
| 54 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
added
third_party/metal-cpp/Foundation/NSDate.hpp
+53 −0
@@ -0,0 +1,53 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Foundation/NSDate.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2024 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | + | |
| 22 | +#pragma once | |
| 23 | + | |
| 24 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 25 | + | |
| 26 | +#include "NSDefines.hpp" | |
| 27 | +#include "NSObject.hpp" | |
| 28 | +#include "NSPrivate.hpp" | |
| 29 | +#include "NSTypes.hpp" | |
| 30 | + | |
| 31 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 32 | + | |
| 33 | +namespace NS | |
| 34 | +{ | |
| 35 | + | |
| 36 | +using TimeInterval = double; | |
| 37 | + | |
| 38 | +class Date : public Copying<Date> | |
| 39 | +{ | |
| 40 | +public: | |
| 41 | + static Date* dateWithTimeIntervalSinceNow(TimeInterval secs); | |
| 42 | +}; | |
| 43 | + | |
| 44 | +} // NS | |
| 45 | + | |
| 46 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 47 | + | |
| 48 | +_NS_INLINE NS::Date* NS::Date::dateWithTimeIntervalSinceNow(NS::TimeInterval secs) | |
| 49 | +{ | |
| 50 | + return NS::Object::sendMessage<NS::Date*>(_NS_PRIVATE_CLS(NSDate), _NS_PRIVATE_SEL(dateWithTimeIntervalSinceNow_), secs); | |
| 51 | +} | |
| 52 | + | |
| 53 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| \ No newline at end of file | ||
added
third_party/metal-cpp/Foundation/NSDefines.hpp
+45 −0
@@ -0,0 +1,45 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Foundation/NSDefines.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2024 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 24 | + | |
| 25 | +#define _NS_WEAK_IMPORT __attribute__((weak_import)) | |
| 26 | +#ifdef METALCPP_SYMBOL_VISIBILITY_HIDDEN | |
| 27 | +#define _NS_EXPORT __attribute__((visibility("hidden"))) | |
| 28 | +#else | |
| 29 | +#define _NS_EXPORT __attribute__((visibility("default"))) | |
| 30 | +#endif // METALCPP_SYMBOL_VISIBILITY_HIDDEN | |
| 31 | +#define _NS_EXTERN extern "C" _NS_EXPORT | |
| 32 | +#define _NS_INLINE inline __attribute__((always_inline)) | |
| 33 | +#define _NS_PACKED __attribute__((packed)) | |
| 34 | + | |
| 35 | +#define _NS_CONST(type, name) _NS_EXTERN type const name | |
| 36 | +#define _NS_ENUM(type, name) enum name : type | |
| 37 | +#define _NS_OPTIONS(type, name) \ | |
| 38 | + using name = type; \ | |
| 39 | + enum : name | |
| 40 | + | |
| 41 | +#define _NS_CAST_TO_UINT(value) static_cast<NS::UInteger>(value) | |
| 42 | +#define _NS_VALIDATE_SIZE(ns, name) static_assert(sizeof(ns::name) == sizeof(ns##name), "size mismatch " #ns "::" #name) | |
| 43 | +#define _NS_VALIDATE_ENUM(ns, name) static_assert(_NS_CAST_TO_UINT(ns::name) == _NS_CAST_TO_UINT(ns##name), "value mismatch " #ns "::" #name) | |
| 44 | + | |
| 45 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
added
third_party/metal-cpp/Foundation/NSDictionary.hpp
+128 −0
@@ -0,0 +1,128 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Foundation/NSDictionary.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2024 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 24 | + | |
| 25 | +#include "NSEnumerator.hpp" | |
| 26 | +#include "NSObject.hpp" | |
| 27 | +#include "NSTypes.hpp" | |
| 28 | + | |
| 29 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 30 | + | |
| 31 | +namespace NS | |
| 32 | +{ | |
| 33 | +class Dictionary : public NS::Copying<Dictionary> | |
| 34 | +{ | |
| 35 | +public: | |
| 36 | + static Dictionary* dictionary(); | |
| 37 | + static Dictionary* dictionary(const Object* pObject, const Object* pKey); | |
| 38 | + static Dictionary* dictionary(const Object* const* pObjects, const Object* const* pKeys, UInteger count); | |
| 39 | + | |
| 40 | + static Dictionary* alloc(); | |
| 41 | + | |
| 42 | + Dictionary* init(); | |
| 43 | + Dictionary* init(const Object* const* pObjects, const Object* const* pKeys, UInteger count); | |
| 44 | + Dictionary* init(const class Coder* pCoder); | |
| 45 | + | |
| 46 | + template <class _KeyType = Object> | |
| 47 | + Enumerator<_KeyType>* keyEnumerator() const; | |
| 48 | + | |
| 49 | + template <class _Object = Object> | |
| 50 | + _Object* object(const Object* pKey) const; | |
| 51 | + UInteger count() const; | |
| 52 | +}; | |
| 53 | +} | |
| 54 | + | |
| 55 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 56 | + | |
| 57 | +_NS_INLINE NS::Dictionary* NS::Dictionary::dictionary() | |
| 58 | +{ | |
| 59 | + return Object::sendMessage<Dictionary*>(_NS_PRIVATE_CLS(NSDictionary), _NS_PRIVATE_SEL(dictionary)); | |
| 60 | +} | |
| 61 | + | |
| 62 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 63 | + | |
| 64 | +_NS_INLINE NS::Dictionary* NS::Dictionary::dictionary(const Object* pObject, const Object* pKey) | |
| 65 | +{ | |
| 66 | + return Object::sendMessage<Dictionary*>(_NS_PRIVATE_CLS(NSDictionary), _NS_PRIVATE_SEL(dictionaryWithObject_forKey_), pObject, pKey); | |
| 67 | +} | |
| 68 | + | |
| 69 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 70 | + | |
| 71 | +_NS_INLINE NS::Dictionary* NS::Dictionary::dictionary(const Object* const* pObjects, const Object* const* pKeys, UInteger count) | |
| 72 | +{ | |
| 73 | + return Object::sendMessage<Dictionary*>(_NS_PRIVATE_CLS(NSDictionary), _NS_PRIVATE_SEL(dictionaryWithObjects_forKeys_count_), | |
| 74 | + pObjects, pKeys, count); | |
| 75 | +} | |
| 76 | + | |
| 77 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 78 | + | |
| 79 | +_NS_INLINE NS::Dictionary* NS::Dictionary::alloc() | |
| 80 | +{ | |
| 81 | + return NS::Object::alloc<Dictionary>(_NS_PRIVATE_CLS(NSDictionary)); | |
| 82 | +} | |
| 83 | + | |
| 84 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 85 | + | |
| 86 | +_NS_INLINE NS::Dictionary* NS::Dictionary::init() | |
| 87 | +{ | |
| 88 | + return NS::Object::init<Dictionary>(); | |
| 89 | +} | |
| 90 | + | |
| 91 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 92 | + | |
| 93 | +_NS_INLINE NS::Dictionary* NS::Dictionary::init(const Object* const* pObjects, const Object* const* pKeys, UInteger count) | |
| 94 | +{ | |
| 95 | + return Object::sendMessage<Dictionary*>(this, _NS_PRIVATE_SEL(initWithObjects_forKeys_count_), pObjects, pKeys, count); | |
| 96 | +} | |
| 97 | + | |
| 98 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 99 | + | |
| 100 | +_NS_INLINE NS::Dictionary* NS::Dictionary::init(const class Coder* pCoder) | |
| 101 | +{ | |
| 102 | + return Object::sendMessage<Dictionary*>(this, _NS_PRIVATE_SEL(initWithCoder_), pCoder); | |
| 103 | +} | |
| 104 | + | |
| 105 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 106 | + | |
| 107 | +template <class _KeyType> | |
| 108 | +_NS_INLINE NS::Enumerator<_KeyType>* NS::Dictionary::keyEnumerator() const | |
| 109 | +{ | |
| 110 | + return Object::sendMessage<Enumerator<_KeyType>*>(this, _NS_PRIVATE_SEL(keyEnumerator)); | |
| 111 | +} | |
| 112 | + | |
| 113 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 114 | + | |
| 115 | +template <class _Object> | |
| 116 | +_NS_INLINE _Object* NS::Dictionary::object(const Object* pKey) const | |
| 117 | +{ | |
| 118 | + return Object::sendMessage<_Object*>(this, _NS_PRIVATE_SEL(objectForKey_), pKey); | |
| 119 | +} | |
| 120 | + | |
| 121 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 122 | + | |
| 123 | +_NS_INLINE NS::UInteger NS::Dictionary::count() const | |
| 124 | +{ | |
| 125 | + return Object::sendMessage<UInteger>(this, _NS_PRIVATE_SEL(count)); | |
| 126 | +} | |
| 127 | + | |
| 128 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
added
third_party/metal-cpp/Foundation/NSEnumerator.hpp
+78 −0
@@ -0,0 +1,78 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Foundation/NSEnumerator.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2024 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 24 | + | |
| 25 | +#include "NSObject.hpp" | |
| 26 | +#include "NSTypes.hpp" | |
| 27 | + | |
| 28 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 29 | + | |
| 30 | +namespace NS | |
| 31 | +{ | |
| 32 | +struct FastEnumerationState | |
| 33 | +{ | |
| 34 | + unsigned long state; | |
| 35 | + Object** itemsPtr; | |
| 36 | + unsigned long* mutationsPtr; | |
| 37 | + unsigned long extra[5]; | |
| 38 | +} _NS_PACKED; | |
| 39 | + | |
| 40 | +class FastEnumeration : public Referencing<FastEnumeration> | |
| 41 | +{ | |
| 42 | +public: | |
| 43 | + NS::UInteger countByEnumerating(FastEnumerationState* pState, Object** pBuffer, NS::UInteger len); | |
| 44 | +}; | |
| 45 | + | |
| 46 | +template <class _ObjectType> | |
| 47 | +class Enumerator : public Referencing<Enumerator<_ObjectType>, FastEnumeration> | |
| 48 | +{ | |
| 49 | +public: | |
| 50 | + _ObjectType* nextObject(); | |
| 51 | + class Array* allObjects(); | |
| 52 | +}; | |
| 53 | +} | |
| 54 | + | |
| 55 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 56 | + | |
| 57 | +_NS_INLINE NS::UInteger NS::FastEnumeration::countByEnumerating(FastEnumerationState* pState, Object** pBuffer, NS::UInteger len) | |
| 58 | +{ | |
| 59 | + return Object::sendMessage<UInteger>(this, _NS_PRIVATE_SEL(countByEnumeratingWithState_objects_count_), pState, pBuffer, len); | |
| 60 | +} | |
| 61 | + | |
| 62 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 63 | + | |
| 64 | +template <class _ObjectType> | |
| 65 | +_NS_INLINE _ObjectType* NS::Enumerator<_ObjectType>::nextObject() | |
| 66 | +{ | |
| 67 | + return Object::sendMessage<_ObjectType*>(this, _NS_PRIVATE_SEL(nextObject)); | |
| 68 | +} | |
| 69 | + | |
| 70 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 71 | + | |
| 72 | +template <class _ObjectType> | |
| 73 | +_NS_INLINE NS::Array* NS::Enumerator<_ObjectType>::allObjects() | |
| 74 | +{ | |
| 75 | + return Object::sendMessage<Array*>(this, _NS_PRIVATE_SEL(allObjects)); | |
| 76 | +} | |
| 77 | + | |
| 78 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
added
third_party/metal-cpp/Foundation/NSError.hpp
+173 −0
@@ -0,0 +1,173 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Foundation/NSError.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2024 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 24 | + | |
| 25 | +#include "NSDefines.hpp" | |
| 26 | +#include "NSObject.hpp" | |
| 27 | +#include "NSPrivate.hpp" | |
| 28 | +#include "NSTypes.hpp" | |
| 29 | + | |
| 30 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 31 | + | |
| 32 | +namespace NS | |
| 33 | +{ | |
| 34 | +using ErrorDomain = class String*; | |
| 35 | + | |
| 36 | +_NS_CONST(ErrorDomain, CocoaErrorDomain); | |
| 37 | +_NS_CONST(ErrorDomain, POSIXErrorDomain); | |
| 38 | +_NS_CONST(ErrorDomain, OSStatusErrorDomain); | |
| 39 | +_NS_CONST(ErrorDomain, MachErrorDomain); | |
| 40 | + | |
| 41 | +using ErrorUserInfoKey = class String*; | |
| 42 | + | |
| 43 | +_NS_CONST(ErrorUserInfoKey, UnderlyingErrorKey); | |
| 44 | +_NS_CONST(ErrorUserInfoKey, LocalizedDescriptionKey); | |
| 45 | +_NS_CONST(ErrorUserInfoKey, LocalizedFailureReasonErrorKey); | |
| 46 | +_NS_CONST(ErrorUserInfoKey, LocalizedRecoverySuggestionErrorKey); | |
| 47 | +_NS_CONST(ErrorUserInfoKey, LocalizedRecoveryOptionsErrorKey); | |
| 48 | +_NS_CONST(ErrorUserInfoKey, RecoveryAttempterErrorKey); | |
| 49 | +_NS_CONST(ErrorUserInfoKey, HelpAnchorErrorKey); | |
| 50 | +_NS_CONST(ErrorUserInfoKey, DebugDescriptionErrorKey); | |
| 51 | +_NS_CONST(ErrorUserInfoKey, LocalizedFailureErrorKey); | |
| 52 | +_NS_CONST(ErrorUserInfoKey, StringEncodingErrorKey); | |
| 53 | +_NS_CONST(ErrorUserInfoKey, URLErrorKey); | |
| 54 | +_NS_CONST(ErrorUserInfoKey, FilePathErrorKey); | |
| 55 | + | |
| 56 | +class Error : public Copying<Error> | |
| 57 | +{ | |
| 58 | +public: | |
| 59 | + static Error* error(ErrorDomain domain, Integer code, class Dictionary* pDictionary); | |
| 60 | + | |
| 61 | + static Error* alloc(); | |
| 62 | + Error* init(); | |
| 63 | + Error* init(ErrorDomain domain, Integer code, class Dictionary* pDictionary); | |
| 64 | + | |
| 65 | + Integer code() const; | |
| 66 | + ErrorDomain domain() const; | |
| 67 | + class Dictionary* userInfo() const; | |
| 68 | + | |
| 69 | + class String* localizedDescription() const; | |
| 70 | + class Array* localizedRecoveryOptions() const; | |
| 71 | + class String* localizedRecoverySuggestion() const; | |
| 72 | + class String* localizedFailureReason() const; | |
| 73 | +}; | |
| 74 | +} | |
| 75 | + | |
| 76 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 77 | + | |
| 78 | +_NS_PRIVATE_DEF_CONST(NS::ErrorDomain, CocoaErrorDomain); | |
| 79 | +_NS_PRIVATE_DEF_CONST(NS::ErrorDomain, POSIXErrorDomain); | |
| 80 | +_NS_PRIVATE_DEF_CONST(NS::ErrorDomain, OSStatusErrorDomain); | |
| 81 | +_NS_PRIVATE_DEF_CONST(NS::ErrorDomain, MachErrorDomain); | |
| 82 | + | |
| 83 | +_NS_PRIVATE_DEF_CONST(NS::ErrorUserInfoKey, UnderlyingErrorKey); | |
| 84 | +_NS_PRIVATE_DEF_CONST(NS::ErrorUserInfoKey, LocalizedDescriptionKey); | |
| 85 | +_NS_PRIVATE_DEF_CONST(NS::ErrorUserInfoKey, LocalizedFailureReasonErrorKey); | |
| 86 | +_NS_PRIVATE_DEF_CONST(NS::ErrorUserInfoKey, LocalizedRecoverySuggestionErrorKey); | |
| 87 | +_NS_PRIVATE_DEF_CONST(NS::ErrorUserInfoKey, LocalizedRecoveryOptionsErrorKey); | |
| 88 | +_NS_PRIVATE_DEF_CONST(NS::ErrorUserInfoKey, RecoveryAttempterErrorKey); | |
| 89 | +_NS_PRIVATE_DEF_CONST(NS::ErrorUserInfoKey, HelpAnchorErrorKey); | |
| 90 | +_NS_PRIVATE_DEF_CONST(NS::ErrorUserInfoKey, DebugDescriptionErrorKey); | |
| 91 | +_NS_PRIVATE_DEF_CONST(NS::ErrorUserInfoKey, LocalizedFailureErrorKey); | |
| 92 | +_NS_PRIVATE_DEF_CONST(NS::ErrorUserInfoKey, StringEncodingErrorKey); | |
| 93 | +_NS_PRIVATE_DEF_CONST(NS::ErrorUserInfoKey, URLErrorKey); | |
| 94 | +_NS_PRIVATE_DEF_CONST(NS::ErrorUserInfoKey, FilePathErrorKey); | |
| 95 | + | |
| 96 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 97 | + | |
| 98 | +_NS_INLINE NS::Error* NS::Error::error(ErrorDomain domain, Integer code, class Dictionary* pDictionary) | |
| 99 | +{ | |
| 100 | + return Object::sendMessage<Error*>(_NS_PRIVATE_CLS(NSError), _NS_PRIVATE_SEL(errorWithDomain_code_userInfo_), domain, code, pDictionary); | |
| 101 | +} | |
| 102 | + | |
| 103 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 104 | + | |
| 105 | +_NS_INLINE NS::Error* NS::Error::alloc() | |
| 106 | +{ | |
| 107 | + return Object::alloc<Error>(_NS_PRIVATE_CLS(NSError)); | |
| 108 | +} | |
| 109 | + | |
| 110 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 111 | + | |
| 112 | +_NS_INLINE NS::Error* NS::Error::init() | |
| 113 | +{ | |
| 114 | + return Object::init<Error>(); | |
| 115 | +} | |
| 116 | + | |
| 117 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 118 | + | |
| 119 | +_NS_INLINE NS::Error* NS::Error::init(ErrorDomain domain, Integer code, class Dictionary* pDictionary) | |
| 120 | +{ | |
| 121 | + return Object::sendMessage<Error*>(this, _NS_PRIVATE_SEL(initWithDomain_code_userInfo_), domain, code, pDictionary); | |
| 122 | +} | |
| 123 | + | |
| 124 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 125 | + | |
| 126 | +_NS_INLINE NS::Integer NS::Error::code() const | |
| 127 | +{ | |
| 128 | + return Object::sendMessage<Integer>(this, _NS_PRIVATE_SEL(code)); | |
| 129 | +} | |
| 130 | + | |
| 131 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 132 | + | |
| 133 | +_NS_INLINE NS::ErrorDomain NS::Error::domain() const | |
| 134 | +{ | |
| 135 | + return Object::sendMessage<ErrorDomain>(this, _NS_PRIVATE_SEL(domain)); | |
| 136 | +} | |
| 137 | + | |
| 138 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 139 | + | |
| 140 | +_NS_INLINE NS::Dictionary* NS::Error::userInfo() const | |
| 141 | +{ | |
| 142 | + return Object::sendMessage<Dictionary*>(this, _NS_PRIVATE_SEL(userInfo)); | |
| 143 | +} | |
| 144 | + | |
| 145 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 146 | + | |
| 147 | +_NS_INLINE NS::String* NS::Error::localizedDescription() const | |
| 148 | +{ | |
| 149 | + return Object::sendMessage<String*>(this, _NS_PRIVATE_SEL(localizedDescription)); | |
| 150 | +} | |
| 151 | + | |
| 152 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 153 | + | |
| 154 | +_NS_INLINE NS::Array* NS::Error::localizedRecoveryOptions() const | |
| 155 | +{ | |
| 156 | + return Object::sendMessage<Array*>(this, _NS_PRIVATE_SEL(localizedRecoveryOptions)); | |
| 157 | +} | |
| 158 | + | |
| 159 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 160 | + | |
| 161 | +_NS_INLINE NS::String* NS::Error::localizedRecoverySuggestion() const | |
| 162 | +{ | |
| 163 | + return Object::sendMessage<String*>(this, _NS_PRIVATE_SEL(localizedRecoverySuggestion)); | |
| 164 | +} | |
| 165 | + | |
| 166 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 167 | + | |
| 168 | +_NS_INLINE NS::String* NS::Error::localizedFailureReason() const | |
| 169 | +{ | |
| 170 | + return Object::sendMessage<String*>(this, _NS_PRIVATE_SEL(localizedFailureReason)); | |
| 171 | +} | |
| 172 | + | |
| 173 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
added
third_party/metal-cpp/Foundation/NSLock.hpp
+118 −0
@@ -0,0 +1,118 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Foundation/NSLock.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2024 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | + | |
| 22 | +#pragma once | |
| 23 | + | |
| 24 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 25 | + | |
| 26 | +#include "NSDefines.hpp" | |
| 27 | +#include "NSObject.hpp" | |
| 28 | +#include "NSPrivate.hpp" | |
| 29 | +#include "NSTypes.hpp" | |
| 30 | +#include "NSDate.hpp" | |
| 31 | + | |
| 32 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 33 | + | |
| 34 | +namespace NS | |
| 35 | +{ | |
| 36 | + | |
| 37 | +template <class _Class, class _Base = class Object> | |
| 38 | +class Locking : public _Base | |
| 39 | +{ | |
| 40 | +public: | |
| 41 | + void lock(); | |
| 42 | + void unlock(); | |
| 43 | +}; | |
| 44 | + | |
| 45 | +class Condition : public Locking<Condition> | |
| 46 | +{ | |
| 47 | +public: | |
| 48 | + static Condition* alloc(); | |
| 49 | + | |
| 50 | + Condition* init(); | |
| 51 | + | |
| 52 | + void wait(); | |
| 53 | + bool waitUntilDate(Date* pLimit); | |
| 54 | + void signal(); | |
| 55 | + void broadcast(); | |
| 56 | +}; | |
| 57 | + | |
| 58 | +} // NS | |
| 59 | + | |
| 60 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 61 | + | |
| 62 | +template<class _Class, class _Base /* = NS::Object */> | |
| 63 | +_NS_INLINE void NS::Locking<_Class, _Base>::lock() | |
| 64 | +{ | |
| 65 | + NS::Object::sendMessage<void>(this, _NS_PRIVATE_SEL(lock)); | |
| 66 | +} | |
| 67 | + | |
| 68 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 69 | + | |
| 70 | +template<class _Class, class _Base /* = NS::Object */> | |
| 71 | +_NS_INLINE void NS::Locking<_Class, _Base>::unlock() | |
| 72 | +{ | |
| 73 | + NS::Object::sendMessage<void>(this, _NS_PRIVATE_SEL(unlock)); | |
| 74 | +} | |
| 75 | + | |
| 76 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 77 | + | |
| 78 | +_NS_INLINE NS::Condition* NS::Condition::alloc() | |
| 79 | +{ | |
| 80 | + return NS::Object::alloc<NS::Condition>(_NS_PRIVATE_CLS(NSCondition)); | |
| 81 | +} | |
| 82 | + | |
| 83 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 84 | + | |
| 85 | +_NS_INLINE NS::Condition* NS::Condition::init() | |
| 86 | +{ | |
| 87 | + return NS::Object::init<NS::Condition>(); | |
| 88 | +} | |
| 89 | + | |
| 90 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 91 | + | |
| 92 | +_NS_INLINE void NS::Condition::wait() | |
| 93 | +{ | |
| 94 | + NS::Object::sendMessage<void>(this, _NS_PRIVATE_SEL(wait)); | |
| 95 | +} | |
| 96 | + | |
| 97 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 98 | + | |
| 99 | +_NS_INLINE bool NS::Condition::waitUntilDate(NS::Date* pLimit) | |
| 100 | +{ | |
| 101 | + return NS::Object::sendMessage<bool>(this, _NS_PRIVATE_SEL(waitUntilDate_), pLimit); | |
| 102 | +} | |
| 103 | + | |
| 104 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 105 | + | |
| 106 | +_NS_INLINE void NS::Condition::signal() | |
| 107 | +{ | |
| 108 | + NS::Object::sendMessage<void>(this, _NS_PRIVATE_SEL(signal)); | |
| 109 | +} | |
| 110 | + | |
| 111 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 112 | + | |
| 113 | +_NS_INLINE void NS::Condition::broadcast() | |
| 114 | +{ | |
| 115 | + NS::Object::sendMessage<void>(this, _NS_PRIVATE_SEL(broadcast)); | |
| 116 | +} | |
| 117 | + | |
| 118 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| \ No newline at end of file | ||
added
third_party/metal-cpp/Foundation/NSNotification.hpp
+110 −0
@@ -0,0 +1,110 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Foundation/NSNotification.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2024 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 24 | + | |
| 25 | +#include "NSDefines.hpp" | |
| 26 | +#include "NSDictionary.hpp" | |
| 27 | +#include "NSObject.hpp" | |
| 28 | +#include "NSString.hpp" | |
| 29 | +#include "NSTypes.hpp" | |
| 30 | +#include <functional> | |
| 31 | + | |
| 32 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 33 | + | |
| 34 | +namespace NS | |
| 35 | +{ | |
| 36 | +using NotificationName = class String*; | |
| 37 | + | |
| 38 | +class Notification : public NS::Referencing<Notification> | |
| 39 | +{ | |
| 40 | +public: | |
| 41 | + NS::String* name() const; | |
| 42 | + NS::Object* object() const; | |
| 43 | + NS::Dictionary* userInfo() const; | |
| 44 | +}; | |
| 45 | + | |
| 46 | +using ObserverBlock = void(^)(Notification*); | |
| 47 | +using ObserverFunction = std::function<void(Notification*)>; | |
| 48 | + | |
| 49 | +class NotificationCenter : public NS::Referencing<NotificationCenter> | |
| 50 | +{ | |
| 51 | + public: | |
| 52 | + static class NotificationCenter* defaultCenter(); | |
| 53 | + Object* addObserver(NotificationName name, Object* pObj, void* pQueue, ObserverBlock block); | |
| 54 | + Object* addObserver(NotificationName name, Object* pObj, void* pQueue, ObserverFunction &handler); | |
| 55 | + void removeObserver(Object* pObserver); | |
| 56 | + | |
| 57 | +}; | |
| 58 | +} | |
| 59 | + | |
| 60 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 61 | + | |
| 62 | +_NS_INLINE NS::String* NS::Notification::name() const | |
| 63 | +{ | |
| 64 | + return Object::sendMessage<NS::String*>(this, _NS_PRIVATE_SEL(name)); | |
| 65 | +} | |
| 66 | + | |
| 67 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 68 | + | |
| 69 | +_NS_INLINE NS::Object* NS::Notification::object() const | |
| 70 | +{ | |
| 71 | + return Object::sendMessage<NS::Object*>(this, _NS_PRIVATE_SEL(object)); | |
| 72 | +} | |
| 73 | + | |
| 74 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 75 | + | |
| 76 | +_NS_INLINE NS::Dictionary* NS::Notification::userInfo() const | |
| 77 | +{ | |
| 78 | + return Object::sendMessage<NS::Dictionary*>(this, _NS_PRIVATE_SEL(userInfo)); | |
| 79 | +} | |
| 80 | + | |
| 81 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 82 | + | |
| 83 | +_NS_INLINE NS::NotificationCenter* NS::NotificationCenter::defaultCenter() | |
| 84 | +{ | |
| 85 | + return NS::Object::sendMessage<NS::NotificationCenter*>(_NS_PRIVATE_CLS(NSNotificationCenter), _NS_PRIVATE_SEL(defaultCenter)); | |
| 86 | +} | |
| 87 | + | |
| 88 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 89 | + | |
| 90 | +_NS_INLINE NS::Object* NS::NotificationCenter::addObserver(NS::NotificationName name, Object* pObj, void* pQueue, NS::ObserverBlock block) | |
| 91 | +{ | |
| 92 | + return NS::Object::sendMessage<Object*>(this, _NS_PRIVATE_SEL(addObserverName_object_queue_block_), name, pObj, pQueue, block); | |
| 93 | +} | |
| 94 | + | |
| 95 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 96 | + | |
| 97 | +_NS_INLINE NS::Object* NS::NotificationCenter::addObserver(NS::NotificationName name, Object* pObj, void* pQueue, NS::ObserverFunction &handler) | |
| 98 | +{ | |
| 99 | + __block ObserverFunction blockFunction = handler; | |
| 100 | + | |
| 101 | + return addObserver(name, pObj, pQueue, ^(NS::Notification* pNotif) {blockFunction(pNotif);}); | |
| 102 | +} | |
| 103 | + | |
| 104 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 105 | + | |
| 106 | +_NS_INLINE void NS::NotificationCenter::removeObserver(Object* pObserver) | |
| 107 | +{ | |
| 108 | + return NS::Object::sendMessage<void>(this, _NS_PRIVATE_SEL(removeObserver_), pObserver); | |
| 109 | +} | |
| 110 | + | |
added
third_party/metal-cpp/Foundation/NSNumber.hpp
+501 −0
@@ -0,0 +1,501 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Foundation/NSNumber.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2024 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 24 | + | |
| 25 | +#include "NSObjCRuntime.hpp" | |
| 26 | +#include "NSObject.hpp" | |
| 27 | +#include "NSTypes.hpp" | |
| 28 | + | |
| 29 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 30 | + | |
| 31 | +namespace NS | |
| 32 | +{ | |
| 33 | +class Value : public Copying<Value> | |
| 34 | +{ | |
| 35 | +public: | |
| 36 | + static Value* value(const void* pValue, const char* pType); | |
| 37 | + static Value* value(const void* pPointer); | |
| 38 | + | |
| 39 | + static Value* alloc(); | |
| 40 | + | |
| 41 | + Value* init(const void* pValue, const char* pType); | |
| 42 | + Value* init(const class Coder* pCoder); | |
| 43 | + | |
| 44 | + void getValue(void* pValue, UInteger size) const; | |
| 45 | + const char* objCType() const; | |
| 46 | + | |
| 47 | + bool isEqualToValue(Value* pValue) const; | |
| 48 | + void* pointerValue() const; | |
| 49 | +}; | |
| 50 | + | |
| 51 | +class Number : public Copying<Number, Value> | |
| 52 | +{ | |
| 53 | +public: | |
| 54 | + static Number* number(char value); | |
| 55 | + static Number* number(unsigned char value); | |
| 56 | + static Number* number(short value); | |
| 57 | + static Number* number(unsigned short value); | |
| 58 | + static Number* number(int value); | |
| 59 | + static Number* number(unsigned int value); | |
| 60 | + static Number* number(long value); | |
| 61 | + static Number* number(unsigned long value); | |
| 62 | + static Number* number(long long value); | |
| 63 | + static Number* number(unsigned long long value); | |
| 64 | + static Number* number(float value); | |
| 65 | + static Number* number(double value); | |
| 66 | + static Number* number(bool value); | |
| 67 | + | |
| 68 | + static Number* alloc(); | |
| 69 | + | |
| 70 | + Number* init(const class Coder* pCoder); | |
| 71 | + Number* init(char value); | |
| 72 | + Number* init(unsigned char value); | |
| 73 | + Number* init(short value); | |
| 74 | + Number* init(unsigned short value); | |
| 75 | + Number* init(int value); | |
| 76 | + Number* init(unsigned int value); | |
| 77 | + Number* init(long value); | |
| 78 | + Number* init(unsigned long value); | |
| 79 | + Number* init(long long value); | |
| 80 | + Number* init(unsigned long long value); | |
| 81 | + Number* init(float value); | |
| 82 | + Number* init(double value); | |
| 83 | + Number* init(bool value); | |
| 84 | + | |
| 85 | + char charValue() const; | |
| 86 | + unsigned char unsignedCharValue() const; | |
| 87 | + short shortValue() const; | |
| 88 | + unsigned short unsignedShortValue() const; | |
| 89 | + int intValue() const; | |
| 90 | + unsigned int unsignedIntValue() const; | |
| 91 | + long longValue() const; | |
| 92 | + unsigned long unsignedLongValue() const; | |
| 93 | + long long longLongValue() const; | |
| 94 | + unsigned long long unsignedLongLongValue() const; | |
| 95 | + float floatValue() const; | |
| 96 | + double doubleValue() const; | |
| 97 | + bool boolValue() const; | |
| 98 | + Integer integerValue() const; | |
| 99 | + UInteger unsignedIntegerValue() const; | |
| 100 | + class String* stringValue() const; | |
| 101 | + | |
| 102 | + ComparisonResult compare(const Number* pOtherNumber) const; | |
| 103 | + bool isEqualToNumber(const Number* pNumber) const; | |
| 104 | + | |
| 105 | + class String* descriptionWithLocale(const Object* pLocale) const; | |
| 106 | +}; | |
| 107 | +} | |
| 108 | + | |
| 109 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 110 | + | |
| 111 | +_NS_INLINE NS::Value* NS::Value::value(const void* pValue, const char* pType) | |
| 112 | +{ | |
| 113 | + return Object::sendMessage<Value*>(_NS_PRIVATE_CLS(NSValue), _NS_PRIVATE_SEL(valueWithBytes_objCType_), pValue, pType); | |
| 114 | +} | |
| 115 | + | |
| 116 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 117 | + | |
| 118 | +_NS_INLINE NS::Value* NS::Value::value(const void* pPointer) | |
| 119 | +{ | |
| 120 | + return Object::sendMessage<Value*>(_NS_PRIVATE_CLS(NSValue), _NS_PRIVATE_SEL(valueWithPointer_), pPointer); | |
| 121 | +} | |
| 122 | + | |
| 123 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 124 | + | |
| 125 | +_NS_INLINE NS::Value* NS::Value::alloc() | |
| 126 | +{ | |
| 127 | + return NS::Object::alloc<Value>(_NS_PRIVATE_CLS(NSValue)); | |
| 128 | +} | |
| 129 | + | |
| 130 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 131 | + | |
| 132 | +_NS_INLINE NS::Value* NS::Value::init(const void* pValue, const char* pType) | |
| 133 | +{ | |
| 134 | + return Object::sendMessage<Value*>(this, _NS_PRIVATE_SEL(initWithBytes_objCType_), pValue, pType); | |
| 135 | +} | |
| 136 | + | |
| 137 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 138 | + | |
| 139 | +_NS_INLINE NS::Value* NS::Value::init(const class Coder* pCoder) | |
| 140 | +{ | |
| 141 | + return Object::sendMessage<Value*>(this, _NS_PRIVATE_SEL(initWithCoder_), pCoder); | |
| 142 | +} | |
| 143 | + | |
| 144 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 145 | + | |
| 146 | +_NS_INLINE void NS::Value::getValue(void* pValue, UInteger size) const | |
| 147 | +{ | |
| 148 | + Object::sendMessage<void>(this, _NS_PRIVATE_SEL(getValue_size_), pValue, size); | |
| 149 | +} | |
| 150 | + | |
| 151 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 152 | + | |
| 153 | +_NS_INLINE const char* NS::Value::objCType() const | |
| 154 | +{ | |
| 155 | + return Object::sendMessage<const char*>(this, _NS_PRIVATE_SEL(objCType)); | |
| 156 | +} | |
| 157 | + | |
| 158 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 159 | + | |
| 160 | +_NS_INLINE bool NS::Value::isEqualToValue(Value* pValue) const | |
| 161 | +{ | |
| 162 | + return Object::sendMessage<bool>(this, _NS_PRIVATE_SEL(isEqualToValue_), pValue); | |
| 163 | +} | |
| 164 | + | |
| 165 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 166 | + | |
| 167 | +_NS_INLINE void* NS::Value::pointerValue() const | |
| 168 | +{ | |
| 169 | + return Object::sendMessage<void*>(this, _NS_PRIVATE_SEL(pointerValue)); | |
| 170 | +} | |
| 171 | + | |
| 172 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 173 | + | |
| 174 | +_NS_INLINE NS::Number* NS::Number::number(char value) | |
| 175 | +{ | |
| 176 | + return Object::sendMessage<Number*>(_NS_PRIVATE_CLS(NSNumber), _NS_PRIVATE_SEL(numberWithChar_), value); | |
| 177 | +} | |
| 178 | + | |
| 179 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 180 | + | |
| 181 | +_NS_INLINE NS::Number* NS::Number::number(unsigned char value) | |
| 182 | +{ | |
| 183 | + return Object::sendMessage<Number*>(_NS_PRIVATE_CLS(NSNumber), _NS_PRIVATE_SEL(numberWithUnsignedChar_), value); | |
| 184 | +} | |
| 185 | + | |
| 186 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 187 | + | |
| 188 | +_NS_INLINE NS::Number* NS::Number::number(short value) | |
| 189 | +{ | |
| 190 | + return Object::sendMessage<Number*>(_NS_PRIVATE_CLS(NSNumber), _NS_PRIVATE_SEL(numberWithShort_), value); | |
| 191 | +} | |
| 192 | + | |
| 193 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 194 | + | |
| 195 | +_NS_INLINE NS::Number* NS::Number::number(unsigned short value) | |
| 196 | +{ | |
| 197 | + return Object::sendMessage<Number*>(_NS_PRIVATE_CLS(NSNumber), _NS_PRIVATE_SEL(numberWithUnsignedShort_), value); | |
| 198 | +} | |
| 199 | + | |
| 200 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 201 | + | |
| 202 | +_NS_INLINE NS::Number* NS::Number::number(int value) | |
| 203 | +{ | |
| 204 | + return Object::sendMessage<Number*>(_NS_PRIVATE_CLS(NSNumber), _NS_PRIVATE_SEL(numberWithInt_), value); | |
| 205 | +} | |
| 206 | + | |
| 207 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 208 | + | |
| 209 | +_NS_INLINE NS::Number* NS::Number::number(unsigned int value) | |
| 210 | +{ | |
| 211 | + return Object::sendMessage<Number*>(_NS_PRIVATE_CLS(NSNumber), _NS_PRIVATE_SEL(numberWithUnsignedInt_), value); | |
| 212 | +} | |
| 213 | + | |
| 214 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 215 | + | |
| 216 | +_NS_INLINE NS::Number* NS::Number::number(long value) | |
| 217 | +{ | |
| 218 | + return Object::sendMessage<Number*>(_NS_PRIVATE_CLS(NSNumber), _NS_PRIVATE_SEL(numberWithLong_), value); | |
| 219 | +} | |
| 220 | + | |
| 221 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 222 | + | |
| 223 | +_NS_INLINE NS::Number* NS::Number::number(unsigned long value) | |
| 224 | +{ | |
| 225 | + return Object::sendMessage<Number*>(_NS_PRIVATE_CLS(NSNumber), _NS_PRIVATE_SEL(numberWithUnsignedLong_), value); | |
| 226 | +} | |
| 227 | + | |
| 228 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 229 | + | |
| 230 | +_NS_INLINE NS::Number* NS::Number::number(long long value) | |
| 231 | +{ | |
| 232 | + return Object::sendMessage<Number*>(_NS_PRIVATE_CLS(NSNumber), _NS_PRIVATE_SEL(numberWithLongLong_), value); | |
| 233 | +} | |
| 234 | + | |
| 235 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 236 | + | |
| 237 | +_NS_INLINE NS::Number* NS::Number::number(unsigned long long value) | |
| 238 | +{ | |
| 239 | + return Object::sendMessage<Number*>(_NS_PRIVATE_CLS(NSNumber), _NS_PRIVATE_SEL(numberWithUnsignedLongLong_), value); | |
| 240 | +} | |
| 241 | + | |
| 242 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 243 | + | |
| 244 | +_NS_INLINE NS::Number* NS::Number::number(float value) | |
| 245 | +{ | |
| 246 | + return Object::sendMessage<Number*>(_NS_PRIVATE_CLS(NSNumber), _NS_PRIVATE_SEL(numberWithFloat_), value); | |
| 247 | +} | |
| 248 | + | |
| 249 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 250 | + | |
| 251 | +_NS_INLINE NS::Number* NS::Number::number(double value) | |
| 252 | +{ | |
| 253 | + return Object::sendMessage<Number*>(_NS_PRIVATE_CLS(NSNumber), _NS_PRIVATE_SEL(numberWithDouble_), value); | |
| 254 | +} | |
| 255 | + | |
| 256 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 257 | + | |
| 258 | +_NS_INLINE NS::Number* NS::Number::number(bool value) | |
| 259 | +{ | |
| 260 | + return Object::sendMessage<Number*>(_NS_PRIVATE_CLS(NSNumber), _NS_PRIVATE_SEL(numberWithBool_), value); | |
| 261 | +} | |
| 262 | + | |
| 263 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 264 | + | |
| 265 | +_NS_INLINE NS::Number* NS::Number::alloc() | |
| 266 | +{ | |
| 267 | + return NS::Object::alloc<Number>(_NS_PRIVATE_CLS(NSNumber)); | |
| 268 | +} | |
| 269 | + | |
| 270 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 271 | + | |
| 272 | +_NS_INLINE NS::Number* NS::Number::init(const Coder* pCoder) | |
| 273 | +{ | |
| 274 | + return Object::sendMessage<Number*>(this, _NS_PRIVATE_SEL(initWithCoder_), pCoder); | |
| 275 | +} | |
| 276 | + | |
| 277 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 278 | + | |
| 279 | +_NS_INLINE NS::Number* NS::Number::init(char value) | |
| 280 | +{ | |
| 281 | + return Object::sendMessage<Number*>(this, _NS_PRIVATE_SEL(initWithChar_), value); | |
| 282 | +} | |
| 283 | + | |
| 284 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 285 | + | |
| 286 | +_NS_INLINE NS::Number* NS::Number::init(unsigned char value) | |
| 287 | +{ | |
| 288 | + return Object::sendMessage<Number*>(this, _NS_PRIVATE_SEL(initWithUnsignedChar_), value); | |
| 289 | +} | |
| 290 | + | |
| 291 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 292 | + | |
| 293 | +_NS_INLINE NS::Number* NS::Number::init(short value) | |
| 294 | +{ | |
| 295 | + return Object::sendMessage<Number*>(this, _NS_PRIVATE_SEL(initWithShort_), value); | |
| 296 | +} | |
| 297 | + | |
| 298 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 299 | + | |
| 300 | +_NS_INLINE NS::Number* NS::Number::init(unsigned short value) | |
| 301 | +{ | |
| 302 | + return Object::sendMessage<Number*>(this, _NS_PRIVATE_SEL(initWithUnsignedShort_), value); | |
| 303 | +} | |
| 304 | + | |
| 305 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 306 | + | |
| 307 | +_NS_INLINE NS::Number* NS::Number::init(int value) | |
| 308 | +{ | |
| 309 | + return Object::sendMessage<Number*>(this, _NS_PRIVATE_SEL(initWithInt_), value); | |
| 310 | +} | |
| 311 | + | |
| 312 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 313 | + | |
| 314 | +_NS_INLINE NS::Number* NS::Number::init(unsigned int value) | |
| 315 | +{ | |
| 316 | + return Object::sendMessage<Number*>(this, _NS_PRIVATE_SEL(initWithUnsignedInt_), value); | |
| 317 | +} | |
| 318 | + | |
| 319 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 320 | + | |
| 321 | +_NS_INLINE NS::Number* NS::Number::init(long value) | |
| 322 | +{ | |
| 323 | + return Object::sendMessage<Number*>(this, _NS_PRIVATE_SEL(initWithLong_), value); | |
| 324 | +} | |
| 325 | + | |
| 326 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 327 | + | |
| 328 | +_NS_INLINE NS::Number* NS::Number::init(unsigned long value) | |
| 329 | +{ | |
| 330 | + return Object::sendMessage<Number*>(this, _NS_PRIVATE_SEL(initWithUnsignedLong_), value); | |
| 331 | +} | |
| 332 | + | |
| 333 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 334 | + | |
| 335 | +_NS_INLINE NS::Number* NS::Number::init(long long value) | |
| 336 | +{ | |
| 337 | + return Object::sendMessage<Number*>(this, _NS_PRIVATE_SEL(initWithLongLong_), value); | |
| 338 | +} | |
| 339 | + | |
| 340 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 341 | + | |
| 342 | +_NS_INLINE NS::Number* NS::Number::init(unsigned long long value) | |
| 343 | +{ | |
| 344 | + return Object::sendMessage<Number*>(this, _NS_PRIVATE_SEL(initWithUnsignedLongLong_), value); | |
| 345 | +} | |
| 346 | + | |
| 347 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 348 | + | |
| 349 | +_NS_INLINE NS::Number* NS::Number::init(float value) | |
| 350 | +{ | |
| 351 | + return Object::sendMessage<Number*>(this, _NS_PRIVATE_SEL(initWithFloat_), value); | |
| 352 | +} | |
| 353 | + | |
| 354 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 355 | + | |
| 356 | +_NS_INLINE NS::Number* NS::Number::init(double value) | |
| 357 | +{ | |
| 358 | + return Object::sendMessage<Number*>(this, _NS_PRIVATE_SEL(initWithDouble_), value); | |
| 359 | +} | |
| 360 | + | |
| 361 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 362 | + | |
| 363 | +_NS_INLINE NS::Number* NS::Number::init(bool value) | |
| 364 | +{ | |
| 365 | + return Object::sendMessage<Number*>(this, _NS_PRIVATE_SEL(initWithBool_), value); | |
| 366 | +} | |
| 367 | + | |
| 368 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 369 | + | |
| 370 | +_NS_INLINE char NS::Number::charValue() const | |
| 371 | +{ | |
| 372 | + return Object::sendMessage<char>(this, _NS_PRIVATE_SEL(charValue)); | |
| 373 | +} | |
| 374 | + | |
| 375 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 376 | + | |
| 377 | +_NS_INLINE unsigned char NS::Number::unsignedCharValue() const | |
| 378 | +{ | |
| 379 | + return Object::sendMessage<unsigned char>(this, _NS_PRIVATE_SEL(unsignedCharValue)); | |
| 380 | +} | |
| 381 | + | |
| 382 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 383 | + | |
| 384 | +_NS_INLINE short NS::Number::shortValue() const | |
| 385 | +{ | |
| 386 | + return Object::sendMessage<short>(this, _NS_PRIVATE_SEL(shortValue)); | |
| 387 | +} | |
| 388 | + | |
| 389 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 390 | + | |
| 391 | +_NS_INLINE unsigned short NS::Number::unsignedShortValue() const | |
| 392 | +{ | |
| 393 | + return Object::sendMessage<unsigned short>(this, _NS_PRIVATE_SEL(unsignedShortValue)); | |
| 394 | +} | |
| 395 | + | |
| 396 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 397 | + | |
| 398 | +_NS_INLINE int NS::Number::intValue() const | |
| 399 | +{ | |
| 400 | + return Object::sendMessage<int>(this, _NS_PRIVATE_SEL(intValue)); | |
| 401 | +} | |
| 402 | + | |
| 403 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 404 | + | |
| 405 | +_NS_INLINE unsigned int NS::Number::unsignedIntValue() const | |
| 406 | +{ | |
| 407 | + return Object::sendMessage<unsigned int>(this, _NS_PRIVATE_SEL(unsignedIntValue)); | |
| 408 | +} | |
| 409 | + | |
| 410 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 411 | + | |
| 412 | +_NS_INLINE long NS::Number::longValue() const | |
| 413 | +{ | |
| 414 | + return Object::sendMessage<long>(this, _NS_PRIVATE_SEL(longValue)); | |
| 415 | +} | |
| 416 | + | |
| 417 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 418 | + | |
| 419 | +_NS_INLINE unsigned long NS::Number::unsignedLongValue() const | |
| 420 | +{ | |
| 421 | + return Object::sendMessage<unsigned long>(this, _NS_PRIVATE_SEL(unsignedLongValue)); | |
| 422 | +} | |
| 423 | + | |
| 424 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 425 | + | |
| 426 | +_NS_INLINE long long NS::Number::longLongValue() const | |
| 427 | +{ | |
| 428 | + return Object::sendMessage<long long>(this, _NS_PRIVATE_SEL(longLongValue)); | |
| 429 | +} | |
| 430 | + | |
| 431 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 432 | + | |
| 433 | +_NS_INLINE unsigned long long NS::Number::unsignedLongLongValue() const | |
| 434 | +{ | |
| 435 | + return Object::sendMessage<unsigned long long>(this, _NS_PRIVATE_SEL(unsignedLongLongValue)); | |
| 436 | +} | |
| 437 | + | |
| 438 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 439 | + | |
| 440 | +_NS_INLINE float NS::Number::floatValue() const | |
| 441 | +{ | |
| 442 | + return Object::sendMessage<float>(this, _NS_PRIVATE_SEL(floatValue)); | |
| 443 | +} | |
| 444 | + | |
| 445 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 446 | + | |
| 447 | +_NS_INLINE double NS::Number::doubleValue() const | |
| 448 | +{ | |
| 449 | + return Object::sendMessage<double>(this, _NS_PRIVATE_SEL(doubleValue)); | |
| 450 | +} | |
| 451 | + | |
| 452 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 453 | + | |
| 454 | +_NS_INLINE bool NS::Number::boolValue() const | |
| 455 | +{ | |
| 456 | + return Object::sendMessage<bool>(this, _NS_PRIVATE_SEL(boolValue)); | |
| 457 | +} | |
| 458 | + | |
| 459 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 460 | + | |
| 461 | +_NS_INLINE NS::Integer NS::Number::integerValue() const | |
| 462 | +{ | |
| 463 | + return Object::sendMessage<Integer>(this, _NS_PRIVATE_SEL(integerValue)); | |
| 464 | +} | |
| 465 | + | |
| 466 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 467 | + | |
| 468 | +_NS_INLINE NS::UInteger NS::Number::unsignedIntegerValue() const | |
| 469 | +{ | |
| 470 | + return Object::sendMessage<UInteger>(this, _NS_PRIVATE_SEL(unsignedIntegerValue)); | |
| 471 | +} | |
| 472 | + | |
| 473 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 474 | + | |
| 475 | +_NS_INLINE NS::String* NS::Number::stringValue() const | |
| 476 | +{ | |
| 477 | + return Object::sendMessage<String*>(this, _NS_PRIVATE_SEL(stringValue)); | |
| 478 | +} | |
| 479 | + | |
| 480 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 481 | + | |
| 482 | +_NS_INLINE NS::ComparisonResult NS::Number::compare(const Number* pOtherNumber) const | |
| 483 | +{ | |
| 484 | + return Object::sendMessage<ComparisonResult>(this, _NS_PRIVATE_SEL(compare_), pOtherNumber); | |
| 485 | +} | |
| 486 | + | |
| 487 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 488 | + | |
| 489 | +_NS_INLINE bool NS::Number::isEqualToNumber(const Number* pNumber) const | |
| 490 | +{ | |
| 491 | + return Object::sendMessage<bool>(this, _NS_PRIVATE_SEL(isEqualToNumber_), pNumber); | |
| 492 | +} | |
| 493 | + | |
| 494 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 495 | + | |
| 496 | +_NS_INLINE NS::String* NS::Number::descriptionWithLocale(const Object* pLocale) const | |
| 497 | +{ | |
| 498 | + return Object::sendMessage<String*>(this, _NS_PRIVATE_SEL(descriptionWithLocale_), pLocale); | |
| 499 | +} | |
| 500 | + | |
| 501 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
added
third_party/metal-cpp/Foundation/NSObjCRuntime.hpp
+43 −0
@@ -0,0 +1,43 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Foundation/NSObjCRuntime.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2024 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 24 | + | |
| 25 | +#include "NSDefines.hpp" | |
| 26 | +#include "NSTypes.hpp" | |
| 27 | + | |
| 28 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 29 | + | |
| 30 | +namespace NS | |
| 31 | +{ | |
| 32 | + | |
| 33 | +_NS_ENUM(Integer, ComparisonResult) { | |
| 34 | + OrderedAscending = -1L, | |
| 35 | + OrderedSame, | |
| 36 | + OrderedDescending | |
| 37 | +}; | |
| 38 | + | |
| 39 | +const Integer NotFound = IntegerMax; | |
| 40 | + | |
| 41 | +} | |
| 42 | + | |
| 43 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
added
third_party/metal-cpp/Foundation/NSObject.hpp
+302 −0
@@ -0,0 +1,302 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Foundation/NSObject.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2024 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 24 | + | |
| 25 | +#include "NSDefines.hpp" | |
| 26 | +#include "NSPrivate.hpp" | |
| 27 | +#include "NSTypes.hpp" | |
| 28 | + | |
| 29 | +#include <objc/message.h> | |
| 30 | +#include <objc/runtime.h> | |
| 31 | + | |
| 32 | +#include <type_traits> | |
| 33 | + | |
| 34 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 35 | + | |
| 36 | +namespace NS | |
| 37 | +{ | |
| 38 | +template <class _Class, class _Base = class Object> | |
| 39 | +class _NS_EXPORT Referencing : public _Base | |
| 40 | +{ | |
| 41 | +public: | |
| 42 | + _Class* retain(); | |
| 43 | + void release(); | |
| 44 | + | |
| 45 | + _Class* autorelease(); | |
| 46 | + | |
| 47 | + UInteger retainCount() const; | |
| 48 | +}; | |
| 49 | + | |
| 50 | +template <class _Class, class _Base = class Object> | |
| 51 | +class Copying : public Referencing<_Class, _Base> | |
| 52 | +{ | |
| 53 | +public: | |
| 54 | + _Class* copy() const; | |
| 55 | +}; | |
| 56 | + | |
| 57 | +template <class _Class, class _Base = class Object> | |
| 58 | +class SecureCoding : public Referencing<_Class, _Base> | |
| 59 | +{ | |
| 60 | +}; | |
| 61 | + | |
| 62 | +class Object : public Referencing<Object, objc_object> | |
| 63 | +{ | |
| 64 | +public: | |
| 65 | + UInteger hash() const; | |
| 66 | + bool isEqual(const Object* pObject) const; | |
| 67 | + | |
| 68 | + class String* description() const; | |
| 69 | + class String* debugDescription() const; | |
| 70 | + | |
| 71 | +protected: | |
| 72 | + friend class Referencing<Object, objc_object>; | |
| 73 | + | |
| 74 | + template <class _Class> | |
| 75 | + static _Class* alloc(const char* pClassName); | |
| 76 | + template <class _Class> | |
| 77 | + static _Class* alloc(const void* pClass); | |
| 78 | + template <class _Class> | |
| 79 | + _Class* init(); | |
| 80 | + | |
| 81 | + template <class _Dst> | |
| 82 | + static _Dst bridgingCast(const void* pObj); | |
| 83 | + static class MethodSignature* methodSignatureForSelector(const void* pObj, SEL selector); | |
| 84 | + static bool respondsToSelector(const void* pObj, SEL selector); | |
| 85 | + template <typename _Type> | |
| 86 | + static constexpr bool doesRequireMsgSendStret(); | |
| 87 | + template <typename _Ret, typename... _Args> | |
| 88 | + static _Ret sendMessage(const void* pObj, SEL selector, _Args... args); | |
| 89 | + template <typename _Ret, typename... _Args> | |
| 90 | + static _Ret sendMessageSafe(const void* pObj, SEL selector, _Args... args); | |
| 91 | + | |
| 92 | +private: | |
| 93 | + Object() = delete; | |
| 94 | + Object(const Object&) = delete; | |
| 95 | + ~Object() = delete; | |
| 96 | + | |
| 97 | + Object& operator=(const Object&) = delete; | |
| 98 | +}; | |
| 99 | +} | |
| 100 | + | |
| 101 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 102 | + | |
| 103 | +template <class _Class, class _Base /* = Object */> | |
| 104 | +_NS_INLINE _Class* NS::Referencing<_Class, _Base>::retain() | |
| 105 | +{ | |
| 106 | + return Object::sendMessage<_Class*>(this, _NS_PRIVATE_SEL(retain)); | |
| 107 | +} | |
| 108 | + | |
| 109 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 110 | + | |
| 111 | +template <class _Class, class _Base /* = Object */> | |
| 112 | +_NS_INLINE void NS::Referencing<_Class, _Base>::release() | |
| 113 | +{ | |
| 114 | + Object::sendMessage<void>(this, _NS_PRIVATE_SEL(release)); | |
| 115 | +} | |
| 116 | + | |
| 117 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 118 | + | |
| 119 | +template <class _Class, class _Base /* = Object */> | |
| 120 | +_NS_INLINE _Class* NS::Referencing<_Class, _Base>::autorelease() | |
| 121 | +{ | |
| 122 | + return Object::sendMessage<_Class*>(this, _NS_PRIVATE_SEL(autorelease)); | |
| 123 | +} | |
| 124 | + | |
| 125 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 126 | + | |
| 127 | +template <class _Class, class _Base /* = Object */> | |
| 128 | +_NS_INLINE NS::UInteger NS::Referencing<_Class, _Base>::retainCount() const | |
| 129 | +{ | |
| 130 | + return Object::sendMessage<UInteger>(this, _NS_PRIVATE_SEL(retainCount)); | |
| 131 | +} | |
| 132 | + | |
| 133 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 134 | + | |
| 135 | +template <class _Class, class _Base /* = Object */> | |
| 136 | +_NS_INLINE _Class* NS::Copying<_Class, _Base>::copy() const | |
| 137 | +{ | |
| 138 | + return Object::sendMessage<_Class*>(this, _NS_PRIVATE_SEL(copy)); | |
| 139 | +} | |
| 140 | + | |
| 141 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 142 | + | |
| 143 | +template <class _Dst> | |
| 144 | +_NS_INLINE _Dst NS::Object::bridgingCast(const void* pObj) | |
| 145 | +{ | |
| 146 | +#ifdef __OBJC__ | |
| 147 | + return (__bridge _Dst)pObj; | |
| 148 | +#else | |
| 149 | + return (_Dst)pObj; | |
| 150 | +#endif // __OBJC__ | |
| 151 | +} | |
| 152 | + | |
| 153 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 154 | + | |
| 155 | +template <typename _Type> | |
| 156 | +_NS_INLINE constexpr bool NS::Object::doesRequireMsgSendStret() | |
| 157 | +{ | |
| 158 | +#if (defined(__i386__) || defined(__x86_64__)) | |
| 159 | + constexpr size_t kStructLimit = (sizeof(std::uintptr_t) << 1); | |
| 160 | + | |
| 161 | + return sizeof(_Type) > kStructLimit; | |
| 162 | +#elif defined(__arm64__) | |
| 163 | + return false; | |
| 164 | +#elif defined(__arm__) | |
| 165 | + constexpr size_t kStructLimit = sizeof(std::uintptr_t); | |
| 166 | + | |
| 167 | + return std::is_class_v<_Type> && (sizeof(_Type) > kStructLimit); | |
| 168 | +#else | |
| 169 | +#error "Unsupported architecture!" | |
| 170 | +#endif | |
| 171 | +} | |
| 172 | + | |
| 173 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 174 | + | |
| 175 | +template <> | |
| 176 | +_NS_INLINE constexpr bool NS::Object::doesRequireMsgSendStret<void>() | |
| 177 | +{ | |
| 178 | + return false; | |
| 179 | +} | |
| 180 | + | |
| 181 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 182 | + | |
| 183 | +template <typename _Ret, typename... _Args> | |
| 184 | +_NS_INLINE _Ret NS::Object::sendMessage(const void* pObj, SEL selector, _Args... args) | |
| 185 | +{ | |
| 186 | +#if (defined(__i386__) || defined(__x86_64__)) | |
| 187 | + if constexpr (std::is_floating_point<_Ret>()) | |
| 188 | + { | |
| 189 | + using SendMessageProcFpret = _Ret (*)(const void*, SEL, _Args...); | |
| 190 | + | |
| 191 | + const SendMessageProcFpret pProc = reinterpret_cast<SendMessageProcFpret>(&objc_msgSend_fpret); | |
| 192 | + | |
| 193 | + return (*pProc)(pObj, selector, args...); | |
| 194 | + } | |
| 195 | + else | |
| 196 | +#endif // ( defined( __i386__ ) || defined( __x86_64__ ) ) | |
| 197 | +#if !defined(__arm64__) | |
| 198 | + if constexpr (doesRequireMsgSendStret<_Ret>()) | |
| 199 | + { | |
| 200 | + using SendMessageProcStret = void (*)(_Ret*, const void*, SEL, _Args...); | |
| 201 | + | |
| 202 | + const SendMessageProcStret pProc = reinterpret_cast<SendMessageProcStret>(&objc_msgSend_stret); | |
| 203 | + _Ret ret; | |
| 204 | + | |
| 205 | + (*pProc)(&ret, pObj, selector, args...); | |
| 206 | + | |
| 207 | + return ret; | |
| 208 | + } | |
| 209 | + else | |
| 210 | +#endif // !defined( __arm64__ ) | |
| 211 | + { | |
| 212 | + using SendMessageProc = _Ret (*)(const void*, SEL, _Args...); | |
| 213 | + | |
| 214 | + const SendMessageProc pProc = reinterpret_cast<SendMessageProc>(&objc_msgSend); | |
| 215 | + | |
| 216 | + return (*pProc)(pObj, selector, args...); | |
| 217 | + } | |
| 218 | +} | |
| 219 | + | |
| 220 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 221 | + | |
| 222 | +_NS_INLINE NS::MethodSignature* NS::Object::methodSignatureForSelector(const void* pObj, SEL selector) | |
| 223 | +{ | |
| 224 | + return sendMessage<MethodSignature*>(pObj, _NS_PRIVATE_SEL(methodSignatureForSelector_), selector); | |
| 225 | +} | |
| 226 | + | |
| 227 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 228 | + | |
| 229 | +_NS_INLINE bool NS::Object::respondsToSelector(const void* pObj, SEL selector) | |
| 230 | +{ | |
| 231 | + return sendMessage<bool>(pObj, _NS_PRIVATE_SEL(respondsToSelector_), selector); | |
| 232 | +} | |
| 233 | + | |
| 234 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 235 | + | |
| 236 | +template <typename _Ret, typename... _Args> | |
| 237 | +_NS_INLINE _Ret NS::Object::sendMessageSafe(const void* pObj, SEL selector, _Args... args) | |
| 238 | +{ | |
| 239 | + if ((respondsToSelector(pObj, selector)) || (nullptr != methodSignatureForSelector(pObj, selector))) | |
| 240 | + { | |
| 241 | + return sendMessage<_Ret>(pObj, selector, args...); | |
| 242 | + } | |
| 243 | + | |
| 244 | + if constexpr (!std::is_void<_Ret>::value) | |
| 245 | + { | |
| 246 | + return _Ret(0); | |
| 247 | + } | |
| 248 | +} | |
| 249 | + | |
| 250 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 251 | + | |
| 252 | +template <class _Class> | |
| 253 | +_NS_INLINE _Class* NS::Object::alloc(const char* pClassName) | |
| 254 | +{ | |
| 255 | + return sendMessage<_Class*>(objc_lookUpClass(pClassName), _NS_PRIVATE_SEL(alloc)); | |
| 256 | +} | |
| 257 | + | |
| 258 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 259 | + | |
| 260 | +template <class _Class> | |
| 261 | +_NS_INLINE _Class* NS::Object::alloc(const void* pClass) | |
| 262 | +{ | |
| 263 | + return sendMessage<_Class*>(pClass, _NS_PRIVATE_SEL(alloc)); | |
| 264 | +} | |
| 265 | + | |
| 266 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 267 | + | |
| 268 | +template <class _Class> | |
| 269 | +_NS_INLINE _Class* NS::Object::init() | |
| 270 | +{ | |
| 271 | + return sendMessage<_Class*>(this, _NS_PRIVATE_SEL(init)); | |
| 272 | +} | |
| 273 | + | |
| 274 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 275 | + | |
| 276 | +_NS_INLINE NS::UInteger NS::Object::hash() const | |
| 277 | +{ | |
| 278 | + return sendMessage<UInteger>(this, _NS_PRIVATE_SEL(hash)); | |
| 279 | +} | |
| 280 | + | |
| 281 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 282 | + | |
| 283 | +_NS_INLINE bool NS::Object::isEqual(const Object* pObject) const | |
| 284 | +{ | |
| 285 | + return sendMessage<bool>(this, _NS_PRIVATE_SEL(isEqual_), pObject); | |
| 286 | +} | |
| 287 | + | |
| 288 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 289 | + | |
| 290 | +_NS_INLINE NS::String* NS::Object::description() const | |
| 291 | +{ | |
| 292 | + return sendMessage<String*>(this, _NS_PRIVATE_SEL(description)); | |
| 293 | +} | |
| 294 | + | |
| 295 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 296 | + | |
| 297 | +_NS_INLINE NS::String* NS::Object::debugDescription() const | |
| 298 | +{ | |
| 299 | + return sendMessageSafe<String*>(this, _NS_PRIVATE_SEL(debugDescription)); | |
| 300 | +} | |
| 301 | + | |
| 302 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
added
third_party/metal-cpp/Foundation/NSPrivate.hpp
+531 −0
@@ -0,0 +1,531 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Foundation/NSPrivate.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2024 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 24 | + | |
| 25 | +#include <objc/runtime.h> | |
| 26 | + | |
| 27 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 28 | + | |
| 29 | +#define _NS_PRIVATE_CLS(symbol) (Private::Class::s_k##symbol) | |
| 30 | +#define _NS_PRIVATE_SEL(accessor) (Private::Selector::s_k##accessor) | |
| 31 | + | |
| 32 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 33 | + | |
| 34 | +#if defined(NS_PRIVATE_IMPLEMENTATION) | |
| 35 | + | |
| 36 | +#include <dlfcn.h> | |
| 37 | + | |
| 38 | +namespace NS::Private | |
| 39 | +{ | |
| 40 | + template <typename _Type> | |
| 41 | + inline _Type const LoadSymbol(const char* pSymbol) | |
| 42 | + { | |
| 43 | + const _Type* pAddress = static_cast<_Type*>(dlsym(RTLD_DEFAULT, pSymbol)); | |
| 44 | + | |
| 45 | + return pAddress ? *pAddress : _Type(); | |
| 46 | + } | |
| 47 | +} // NS::Private | |
| 48 | + | |
| 49 | +#ifdef METALCPP_SYMBOL_VISIBILITY_HIDDEN | |
| 50 | +#define _NS_PRIVATE_VISIBILITY __attribute__((visibility("hidden"))) | |
| 51 | +#else | |
| 52 | +#define _NS_PRIVATE_VISIBILITY __attribute__((visibility("default"))) | |
| 53 | +#endif // METALCPP_SYMBOL_VISIBILITY_HIDDEN | |
| 54 | + | |
| 55 | +#define _NS_PRIVATE_IMPORT __attribute__((weak_import)) | |
| 56 | + | |
| 57 | +#ifdef __OBJC__ | |
| 58 | +#define _NS_PRIVATE_OBJC_LOOKUP_CLASS(symbol) ((__bridge void*)objc_lookUpClass(#symbol)) | |
| 59 | +#define _NS_PRIVATE_OBJC_GET_PROTOCOL(symbol) ((__bridge void*)objc_getProtocol(#symbol)) | |
| 60 | +#else | |
| 61 | +#define _NS_PRIVATE_OBJC_LOOKUP_CLASS(symbol) objc_lookUpClass(#symbol) | |
| 62 | +#define _NS_PRIVATE_OBJC_GET_PROTOCOL(symbol) objc_getProtocol(#symbol) | |
| 63 | +#endif // __OBJC__ | |
| 64 | + | |
| 65 | +#define _NS_PRIVATE_DEF_CLS(symbol) void* s_k##symbol _NS_PRIVATE_VISIBILITY = _NS_PRIVATE_OBJC_LOOKUP_CLASS(symbol) | |
| 66 | +#define _NS_PRIVATE_DEF_PRO(symbol) void* s_k##symbol _NS_PRIVATE_VISIBILITY = _NS_PRIVATE_OBJC_GET_PROTOCOL(symbol) | |
| 67 | +#define _NS_PRIVATE_DEF_SEL(accessor, symbol) SEL s_k##accessor _NS_PRIVATE_VISIBILITY = sel_registerName(symbol) | |
| 68 | + | |
| 69 | +#if defined(__MAC_26_0) || defined(__IPHONE_26_0) || defined(__TVOS_26_0) | |
| 70 | +#define _NS_PRIVATE_DEF_CONST(type, symbol) \ | |
| 71 | + _NS_EXTERN type const NS##symbol _NS_PRIVATE_IMPORT; \ | |
| 72 | + type const NS::symbol = (nullptr != &NS##symbol) ? NS##symbol : type() | |
| 73 | +#else | |
| 74 | +#define _NS_PRIVATE_DEF_CONST(type, symbol) \ | |
| 75 | + _NS_EXTERN type const MTL##symbol _NS_PRIVATE_IMPORT; \ | |
| 76 | + type const NS::symbol = Private::LoadSymbol<type>("NS" #symbol) | |
| 77 | +#endif | |
| 78 | + | |
| 79 | +#else | |
| 80 | + | |
| 81 | +#define _NS_PRIVATE_DEF_CLS(symbol) extern void* s_k##symbol | |
| 82 | +#define _NS_PRIVATE_DEF_PRO(symbol) extern void* s_k##symbol | |
| 83 | +#define _NS_PRIVATE_DEF_SEL(accessor, symbol) extern SEL s_k##accessor | |
| 84 | +#define _NS_PRIVATE_DEF_CONST(type, symbol) extern type const NS::symbol | |
| 85 | + | |
| 86 | +#endif // NS_PRIVATE_IMPLEMENTATION | |
| 87 | + | |
| 88 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 89 | + | |
| 90 | +namespace NS | |
| 91 | +{ | |
| 92 | +namespace Private | |
| 93 | +{ | |
| 94 | + namespace Class | |
| 95 | + { | |
| 96 | + | |
| 97 | + _NS_PRIVATE_DEF_CLS(NSArray); | |
| 98 | + _NS_PRIVATE_DEF_CLS(NSAutoreleasePool); | |
| 99 | + _NS_PRIVATE_DEF_CLS(NSBundle); | |
| 100 | + _NS_PRIVATE_DEF_CLS(NSCondition); | |
| 101 | + _NS_PRIVATE_DEF_CLS(NSDate); | |
| 102 | + _NS_PRIVATE_DEF_CLS(NSDictionary); | |
| 103 | + _NS_PRIVATE_DEF_CLS(NSError); | |
| 104 | + _NS_PRIVATE_DEF_CLS(NSNotificationCenter); | |
| 105 | + _NS_PRIVATE_DEF_CLS(NSNumber); | |
| 106 | + _NS_PRIVATE_DEF_CLS(NSObject); | |
| 107 | + _NS_PRIVATE_DEF_CLS(NSProcessInfo); | |
| 108 | + _NS_PRIVATE_DEF_CLS(NSSet); | |
| 109 | + _NS_PRIVATE_DEF_CLS(NSString); | |
| 110 | + _NS_PRIVATE_DEF_CLS(NSURL); | |
| 111 | + _NS_PRIVATE_DEF_CLS(NSValue); | |
| 112 | + | |
| 113 | + } // Class | |
| 114 | +} // Private | |
| 115 | +} // MTL | |
| 116 | + | |
| 117 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 118 | + | |
| 119 | +namespace NS | |
| 120 | +{ | |
| 121 | +namespace Private | |
| 122 | +{ | |
| 123 | + namespace Protocol | |
| 124 | + { | |
| 125 | + | |
| 126 | + } // Protocol | |
| 127 | +} // Private | |
| 128 | +} // NS | |
| 129 | + | |
| 130 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 131 | + | |
| 132 | +namespace NS | |
| 133 | +{ | |
| 134 | +namespace Private | |
| 135 | +{ | |
| 136 | + namespace Selector | |
| 137 | + { | |
| 138 | + | |
| 139 | + _NS_PRIVATE_DEF_SEL(addObject_, | |
| 140 | + "addObject:"); | |
| 141 | + _NS_PRIVATE_DEF_SEL(addObserverName_object_queue_block_, | |
| 142 | + "addObserverForName:object:queue:usingBlock:"); | |
| 143 | + _NS_PRIVATE_DEF_SEL(activeProcessorCount, | |
| 144 | + "activeProcessorCount"); | |
| 145 | + _NS_PRIVATE_DEF_SEL(allBundles, | |
| 146 | + "allBundles"); | |
| 147 | + _NS_PRIVATE_DEF_SEL(allFrameworks, | |
| 148 | + "allFrameworks"); | |
| 149 | + _NS_PRIVATE_DEF_SEL(allObjects, | |
| 150 | + "allObjects"); | |
| 151 | + _NS_PRIVATE_DEF_SEL(alloc, | |
| 152 | + "alloc"); | |
| 153 | + _NS_PRIVATE_DEF_SEL(appStoreReceiptURL, | |
| 154 | + "appStoreReceiptURL"); | |
| 155 | + _NS_PRIVATE_DEF_SEL(arguments, | |
| 156 | + "arguments"); | |
| 157 | + _NS_PRIVATE_DEF_SEL(array, | |
| 158 | + "array"); | |
| 159 | + _NS_PRIVATE_DEF_SEL(arrayWithObject_, | |
| 160 | + "arrayWithObject:"); | |
| 161 | + _NS_PRIVATE_DEF_SEL(arrayWithObjects_count_, | |
| 162 | + "arrayWithObjects:count:"); | |
| 163 | + _NS_PRIVATE_DEF_SEL(automaticTerminationSupportEnabled, | |
| 164 | + "automaticTerminationSupportEnabled"); | |
| 165 | + _NS_PRIVATE_DEF_SEL(autorelease, | |
| 166 | + "autorelease"); | |
| 167 | + _NS_PRIVATE_DEF_SEL(beginActivityWithOptions_reason_, | |
| 168 | + "beginActivityWithOptions:reason:"); | |
| 169 | + _NS_PRIVATE_DEF_SEL(boolValue, | |
| 170 | + "boolValue"); | |
| 171 | + _NS_PRIVATE_DEF_SEL(broadcast, | |
| 172 | + "broadcast"); | |
| 173 | + _NS_PRIVATE_DEF_SEL(builtInPlugInsPath, | |
| 174 | + "builtInPlugInsPath"); | |
| 175 | + _NS_PRIVATE_DEF_SEL(builtInPlugInsURL, | |
| 176 | + "builtInPlugInsURL"); | |
| 177 | + _NS_PRIVATE_DEF_SEL(bundleIdentifier, | |
| 178 | + "bundleIdentifier"); | |
| 179 | + _NS_PRIVATE_DEF_SEL(bundlePath, | |
| 180 | + "bundlePath"); | |
| 181 | + _NS_PRIVATE_DEF_SEL(bundleURL, | |
| 182 | + "bundleURL"); | |
| 183 | + _NS_PRIVATE_DEF_SEL(bundleWithPath_, | |
| 184 | + "bundleWithPath:"); | |
| 185 | + _NS_PRIVATE_DEF_SEL(bundleWithURL_, | |
| 186 | + "bundleWithURL:"); | |
| 187 | + _NS_PRIVATE_DEF_SEL(bytes, | |
| 188 | + "bytes"); | |
| 189 | + _NS_PRIVATE_DEF_SEL(caseInsensitiveCompare_, | |
| 190 | + "caseInsensitiveCompare:"); | |
| 191 | + _NS_PRIVATE_DEF_SEL(characterAtIndex_, | |
| 192 | + "characterAtIndex:"); | |
| 193 | + _NS_PRIVATE_DEF_SEL(charValue, | |
| 194 | + "charValue"); | |
| 195 | + _NS_PRIVATE_DEF_SEL(countByEnumeratingWithState_objects_count_, | |
| 196 | + "countByEnumeratingWithState:objects:count:"); | |
| 197 | + _NS_PRIVATE_DEF_SEL(cStringUsingEncoding_, | |
| 198 | + "cStringUsingEncoding:"); | |
| 199 | + _NS_PRIVATE_DEF_SEL(code, | |
| 200 | + "code"); | |
| 201 | + _NS_PRIVATE_DEF_SEL(compare_, | |
| 202 | + "compare:"); | |
| 203 | + _NS_PRIVATE_DEF_SEL(copy, | |
| 204 | + "copy"); | |
| 205 | + _NS_PRIVATE_DEF_SEL(count, | |
| 206 | + "count"); | |
| 207 | + _NS_PRIVATE_DEF_SEL(dateWithTimeIntervalSinceNow_, | |
| 208 | + "dateWithTimeIntervalSinceNow:"); | |
| 209 | + _NS_PRIVATE_DEF_SEL(defaultCenter, | |
| 210 | + "defaultCenter"); | |
| 211 | + _NS_PRIVATE_DEF_SEL(descriptionWithLocale_, | |
| 212 | + "descriptionWithLocale:"); | |
| 213 | + _NS_PRIVATE_DEF_SEL(disableAutomaticTermination_, | |
| 214 | + "disableAutomaticTermination:"); | |
| 215 | + _NS_PRIVATE_DEF_SEL(disableSuddenTermination, | |
| 216 | + "disableSuddenTermination"); | |
| 217 | + _NS_PRIVATE_DEF_SEL(debugDescription, | |
| 218 | + "debugDescription"); | |
| 219 | + _NS_PRIVATE_DEF_SEL(description, | |
| 220 | + "description"); | |
| 221 | + _NS_PRIVATE_DEF_SEL(dictionary, | |
| 222 | + "dictionary"); | |
| 223 | + _NS_PRIVATE_DEF_SEL(dictionaryWithObject_forKey_, | |
| 224 | + "dictionaryWithObject:forKey:"); | |
| 225 | + _NS_PRIVATE_DEF_SEL(dictionaryWithObjects_forKeys_count_, | |
| 226 | + "dictionaryWithObjects:forKeys:count:"); | |
| 227 | + _NS_PRIVATE_DEF_SEL(domain, | |
| 228 | + "domain"); | |
| 229 | + _NS_PRIVATE_DEF_SEL(doubleValue, | |
| 230 | + "doubleValue"); | |
| 231 | + _NS_PRIVATE_DEF_SEL(drain, | |
| 232 | + "drain"); | |
| 233 | + _NS_PRIVATE_DEF_SEL(enableAutomaticTermination_, | |
| 234 | + "enableAutomaticTermination:"); | |
| 235 | + _NS_PRIVATE_DEF_SEL(enableSuddenTermination, | |
| 236 | + "enableSuddenTermination"); | |
| 237 | + _NS_PRIVATE_DEF_SEL(endActivity_, | |
| 238 | + "endActivity:"); | |
| 239 | + _NS_PRIVATE_DEF_SEL(environment, | |
| 240 | + "environment"); | |
| 241 | + _NS_PRIVATE_DEF_SEL(errorWithDomain_code_userInfo_, | |
| 242 | + "errorWithDomain:code:userInfo:"); | |
| 243 | + _NS_PRIVATE_DEF_SEL(executablePath, | |
| 244 | + "executablePath"); | |
| 245 | + _NS_PRIVATE_DEF_SEL(executableURL, | |
| 246 | + "executableURL"); | |
| 247 | + _NS_PRIVATE_DEF_SEL(fileSystemRepresentation, | |
| 248 | + "fileSystemRepresentation"); | |
| 249 | + _NS_PRIVATE_DEF_SEL(fileURLWithPath_, | |
| 250 | + "fileURLWithPath:"); | |
| 251 | + _NS_PRIVATE_DEF_SEL(floatValue, | |
| 252 | + "floatValue"); | |
| 253 | + _NS_PRIVATE_DEF_SEL(fullUserName, | |
| 254 | + "fullUserName"); | |
| 255 | + _NS_PRIVATE_DEF_SEL(getValue_size_, | |
| 256 | + "getValue:size:"); | |
| 257 | + _NS_PRIVATE_DEF_SEL(globallyUniqueString, | |
| 258 | + "globallyUniqueString"); | |
| 259 | + _NS_PRIVATE_DEF_SEL(hash, | |
| 260 | + "hash"); | |
| 261 | + _NS_PRIVATE_DEF_SEL(hasPerformanceProfile_, | |
| 262 | + "hasPerformanceProfile:"); | |
| 263 | + _NS_PRIVATE_DEF_SEL(hostName, | |
| 264 | + "hostName"); | |
| 265 | + _NS_PRIVATE_DEF_SEL(infoDictionary, | |
| 266 | + "infoDictionary"); | |
| 267 | + _NS_PRIVATE_DEF_SEL(init, | |
| 268 | + "init"); | |
| 269 | + _NS_PRIVATE_DEF_SEL(initFileURLWithPath_, | |
| 270 | + "initFileURLWithPath:"); | |
| 271 | + _NS_PRIVATE_DEF_SEL(initWithBool_, | |
| 272 | + "initWithBool:"); | |
| 273 | + _NS_PRIVATE_DEF_SEL(initWithBytes_objCType_, | |
| 274 | + "initWithBytes:objCType:"); | |
| 275 | + _NS_PRIVATE_DEF_SEL(initWithBytesNoCopy_length_encoding_freeWhenDone_, | |
| 276 | + "initWithBytesNoCopy:length:encoding:freeWhenDone:"); | |
| 277 | + _NS_PRIVATE_DEF_SEL(initWithChar_, | |
| 278 | + "initWithChar:"); | |
| 279 | + _NS_PRIVATE_DEF_SEL(initWithCoder_, | |
| 280 | + "initWithCoder:"); | |
| 281 | + _NS_PRIVATE_DEF_SEL(initWithCString_encoding_, | |
| 282 | + "initWithCString:encoding:"); | |
| 283 | + _NS_PRIVATE_DEF_SEL(initWithDomain_code_userInfo_, | |
| 284 | + "initWithDomain:code:userInfo:"); | |
| 285 | + _NS_PRIVATE_DEF_SEL(initWithDouble_, | |
| 286 | + "initWithDouble:"); | |
| 287 | + _NS_PRIVATE_DEF_SEL(initWithFloat_, | |
| 288 | + "initWithFloat:"); | |
| 289 | + _NS_PRIVATE_DEF_SEL(initWithInt_, | |
| 290 | + "initWithInt:"); | |
| 291 | + _NS_PRIVATE_DEF_SEL(initWithLong_, | |
| 292 | + "initWithLong:"); | |
| 293 | + _NS_PRIVATE_DEF_SEL(initWithLongLong_, | |
| 294 | + "initWithLongLong:"); | |
| 295 | + _NS_PRIVATE_DEF_SEL(initWithObjects_count_, | |
| 296 | + "initWithObjects:count:"); | |
| 297 | + _NS_PRIVATE_DEF_SEL(initWithObjects_forKeys_count_, | |
| 298 | + "initWithObjects:forKeys:count:"); | |
| 299 | + _NS_PRIVATE_DEF_SEL(initWithPath_, | |
| 300 | + "initWithPath:"); | |
| 301 | + _NS_PRIVATE_DEF_SEL(initWithShort_, | |
| 302 | + "initWithShort:"); | |
| 303 | + _NS_PRIVATE_DEF_SEL(initWithString_, | |
| 304 | + "initWithString:"); | |
| 305 | + _NS_PRIVATE_DEF_SEL(initWithUnsignedChar_, | |
| 306 | + "initWithUnsignedChar:"); | |
| 307 | + _NS_PRIVATE_DEF_SEL(initWithUnsignedInt_, | |
| 308 | + "initWithUnsignedInt:"); | |
| 309 | + _NS_PRIVATE_DEF_SEL(initWithUnsignedLong_, | |
| 310 | + "initWithUnsignedLong:"); | |
| 311 | + _NS_PRIVATE_DEF_SEL(initWithUnsignedLongLong_, | |
| 312 | + "initWithUnsignedLongLong:"); | |
| 313 | + _NS_PRIVATE_DEF_SEL(initWithUnsignedShort_, | |
| 314 | + "initWithUnsignedShort:"); | |
| 315 | + _NS_PRIVATE_DEF_SEL(initWithURL_, | |
| 316 | + "initWithURL:"); | |
| 317 | + _NS_PRIVATE_DEF_SEL(integerValue, | |
| 318 | + "integerValue"); | |
| 319 | + _NS_PRIVATE_DEF_SEL(intValue, | |
| 320 | + "intValue"); | |
| 321 | + _NS_PRIVATE_DEF_SEL(isDeviceCertified_, | |
| 322 | + "isDeviceCertifiedFor:"); | |
| 323 | + _NS_PRIVATE_DEF_SEL(isEqual_, | |
| 324 | + "isEqual:"); | |
| 325 | + _NS_PRIVATE_DEF_SEL(isEqualToNumber_, | |
| 326 | + "isEqualToNumber:"); | |
| 327 | + _NS_PRIVATE_DEF_SEL(isEqualToString_, | |
| 328 | + "isEqualToString:"); | |
| 329 | + _NS_PRIVATE_DEF_SEL(isEqualToValue_, | |
| 330 | + "isEqualToValue:"); | |
| 331 | + _NS_PRIVATE_DEF_SEL(isiOSAppOnMac, | |
| 332 | + "isiOSAppOnMac"); | |
| 333 | + _NS_PRIVATE_DEF_SEL(isLoaded, | |
| 334 | + "isLoaded"); | |
| 335 | + _NS_PRIVATE_DEF_SEL(isLowPowerModeEnabled, | |
| 336 | + "isLowPowerModeEnabled"); | |
| 337 | + _NS_PRIVATE_DEF_SEL(isMacCatalystApp, | |
| 338 | + "isMacCatalystApp"); | |
| 339 | + _NS_PRIVATE_DEF_SEL(isOperatingSystemAtLeastVersion_, | |
| 340 | + "isOperatingSystemAtLeastVersion:"); | |
| 341 | + _NS_PRIVATE_DEF_SEL(keyEnumerator, | |
| 342 | + "keyEnumerator"); | |
| 343 | + _NS_PRIVATE_DEF_SEL(length, | |
| 344 | + "length"); | |
| 345 | + _NS_PRIVATE_DEF_SEL(lengthOfBytesUsingEncoding_, | |
| 346 | + "lengthOfBytesUsingEncoding:"); | |
| 347 | + _NS_PRIVATE_DEF_SEL(load, | |
| 348 | + "load"); | |
| 349 | + _NS_PRIVATE_DEF_SEL(loadAndReturnError_, | |
| 350 | + "loadAndReturnError:"); | |
| 351 | + _NS_PRIVATE_DEF_SEL(localizedDescription, | |
| 352 | + "localizedDescription"); | |
| 353 | + _NS_PRIVATE_DEF_SEL(localizedFailureReason, | |
| 354 | + "localizedFailureReason"); | |
| 355 | + _NS_PRIVATE_DEF_SEL(localizedInfoDictionary, | |
| 356 | + "localizedInfoDictionary"); | |
| 357 | + _NS_PRIVATE_DEF_SEL(localizedRecoveryOptions, | |
| 358 | + "localizedRecoveryOptions"); | |
| 359 | + _NS_PRIVATE_DEF_SEL(localizedRecoverySuggestion, | |
| 360 | + "localizedRecoverySuggestion"); | |
| 361 | + _NS_PRIVATE_DEF_SEL(localizedStringForKey_value_table_, | |
| 362 | + "localizedStringForKey:value:table:"); | |
| 363 | + _NS_PRIVATE_DEF_SEL(lock, | |
| 364 | + "lock"); | |
| 365 | + _NS_PRIVATE_DEF_SEL(longValue, | |
| 366 | + "longValue"); | |
| 367 | + _NS_PRIVATE_DEF_SEL(longLongValue, | |
| 368 | + "longLongValue"); | |
| 369 | + _NS_PRIVATE_DEF_SEL(mainBundle, | |
| 370 | + "mainBundle"); | |
| 371 | + _NS_PRIVATE_DEF_SEL(maximumLengthOfBytesUsingEncoding_, | |
| 372 | + "maximumLengthOfBytesUsingEncoding:"); | |
| 373 | + _NS_PRIVATE_DEF_SEL(methodSignatureForSelector_, | |
| 374 | + "methodSignatureForSelector:"); | |
| 375 | + _NS_PRIVATE_DEF_SEL(name, | |
| 376 | + "name"); | |
| 377 | + _NS_PRIVATE_DEF_SEL(nextObject, | |
| 378 | + "nextObject"); | |
| 379 | + _NS_PRIVATE_DEF_SEL(numberWithBool_, | |
| 380 | + "numberWithBool:"); | |
| 381 | + _NS_PRIVATE_DEF_SEL(numberWithChar_, | |
| 382 | + "numberWithChar:"); | |
| 383 | + _NS_PRIVATE_DEF_SEL(numberWithDouble_, | |
| 384 | + "numberWithDouble:"); | |
| 385 | + _NS_PRIVATE_DEF_SEL(numberWithFloat_, | |
| 386 | + "numberWithFloat:"); | |
| 387 | + _NS_PRIVATE_DEF_SEL(numberWithInt_, | |
| 388 | + "numberWithInt:"); | |
| 389 | + _NS_PRIVATE_DEF_SEL(numberWithLong_, | |
| 390 | + "numberWithLong:"); | |
| 391 | + _NS_PRIVATE_DEF_SEL(numberWithLongLong_, | |
| 392 | + "numberWithLongLong:"); | |
| 393 | + _NS_PRIVATE_DEF_SEL(numberWithShort_, | |
| 394 | + "numberWithShort:"); | |
| 395 | + _NS_PRIVATE_DEF_SEL(numberWithUnsignedChar_, | |
| 396 | + "numberWithUnsignedChar:"); | |
| 397 | + _NS_PRIVATE_DEF_SEL(numberWithUnsignedInt_, | |
| 398 | + "numberWithUnsignedInt:"); | |
| 399 | + _NS_PRIVATE_DEF_SEL(numberWithUnsignedLong_, | |
| 400 | + "numberWithUnsignedLong:"); | |
| 401 | + _NS_PRIVATE_DEF_SEL(numberWithUnsignedLongLong_, | |
| 402 | + "numberWithUnsignedLongLong:"); | |
| 403 | + _NS_PRIVATE_DEF_SEL(numberWithUnsignedShort_, | |
| 404 | + "numberWithUnsignedShort:"); | |
| 405 | + _NS_PRIVATE_DEF_SEL(objCType, | |
| 406 | + "objCType"); | |
| 407 | + _NS_PRIVATE_DEF_SEL(object, | |
| 408 | + "object"); | |
| 409 | + _NS_PRIVATE_DEF_SEL(objectAtIndex_, | |
| 410 | + "objectAtIndex:"); | |
| 411 | + _NS_PRIVATE_DEF_SEL(objectEnumerator, | |
| 412 | + "objectEnumerator"); | |
| 413 | + _NS_PRIVATE_DEF_SEL(objectForInfoDictionaryKey_, | |
| 414 | + "objectForInfoDictionaryKey:"); | |
| 415 | + _NS_PRIVATE_DEF_SEL(objectForKey_, | |
| 416 | + "objectForKey:"); | |
| 417 | + _NS_PRIVATE_DEF_SEL(operatingSystem, | |
| 418 | + "operatingSystem"); | |
| 419 | + _NS_PRIVATE_DEF_SEL(operatingSystemVersion, | |
| 420 | + "operatingSystemVersion"); | |
| 421 | + _NS_PRIVATE_DEF_SEL(operatingSystemVersionString, | |
| 422 | + "operatingSystemVersionString"); | |
| 423 | + _NS_PRIVATE_DEF_SEL(pathForAuxiliaryExecutable_, | |
| 424 | + "pathForAuxiliaryExecutable:"); | |
| 425 | + _NS_PRIVATE_DEF_SEL(performActivityWithOptions_reason_usingBlock_, | |
| 426 | + "performActivityWithOptions:reason:usingBlock:"); | |
| 427 | + _NS_PRIVATE_DEF_SEL(performExpiringActivityWithReason_usingBlock_, | |
| 428 | + "performExpiringActivityWithReason:usingBlock:"); | |
| 429 | + _NS_PRIVATE_DEF_SEL(physicalMemory, | |
| 430 | + "physicalMemory"); | |
| 431 | + _NS_PRIVATE_DEF_SEL(pointerValue, | |
| 432 | + "pointerValue"); | |
| 433 | + _NS_PRIVATE_DEF_SEL(preflightAndReturnError_, | |
| 434 | + "preflightAndReturnError:"); | |
| 435 | + _NS_PRIVATE_DEF_SEL(privateFrameworksPath, | |
| 436 | + "privateFrameworksPath"); | |
| 437 | + _NS_PRIVATE_DEF_SEL(privateFrameworksURL, | |
| 438 | + "privateFrameworksURL"); | |
| 439 | + _NS_PRIVATE_DEF_SEL(processIdentifier, | |
| 440 | + "processIdentifier"); | |
| 441 | + _NS_PRIVATE_DEF_SEL(processInfo, | |
| 442 | + "processInfo"); | |
| 443 | + _NS_PRIVATE_DEF_SEL(processName, | |
| 444 | + "processName"); | |
| 445 | + _NS_PRIVATE_DEF_SEL(processorCount, | |
| 446 | + "processorCount"); | |
| 447 | + _NS_PRIVATE_DEF_SEL(rangeOfString_options_, | |
| 448 | + "rangeOfString:options:"); | |
| 449 | + _NS_PRIVATE_DEF_SEL(release, | |
| 450 | + "release"); | |
| 451 | + _NS_PRIVATE_DEF_SEL(removeObserver_, | |
| 452 | + "removeObserver:"); | |
| 453 | + _NS_PRIVATE_DEF_SEL(resourcePath, | |
| 454 | + "resourcePath"); | |
| 455 | + _NS_PRIVATE_DEF_SEL(resourceURL, | |
| 456 | + "resourceURL"); | |
| 457 | + _NS_PRIVATE_DEF_SEL(respondsToSelector_, | |
| 458 | + "respondsToSelector:"); | |
| 459 | + _NS_PRIVATE_DEF_SEL(retain, | |
| 460 | + "retain"); | |
| 461 | + _NS_PRIVATE_DEF_SEL(retainCount, | |
| 462 | + "retainCount"); | |
| 463 | + _NS_PRIVATE_DEF_SEL(setAutomaticTerminationSupportEnabled_, | |
| 464 | + "setAutomaticTerminationSupportEnabled:"); | |
| 465 | + _NS_PRIVATE_DEF_SEL(setProcessName_, | |
| 466 | + "setProcessName:"); | |
| 467 | + _NS_PRIVATE_DEF_SEL(sharedFrameworksPath, | |
| 468 | + "sharedFrameworksPath"); | |
| 469 | + _NS_PRIVATE_DEF_SEL(sharedFrameworksURL, | |
| 470 | + "sharedFrameworksURL"); | |
| 471 | + _NS_PRIVATE_DEF_SEL(sharedSupportPath, | |
| 472 | + "sharedSupportPath"); | |
| 473 | + _NS_PRIVATE_DEF_SEL(sharedSupportURL, | |
| 474 | + "sharedSupportURL"); | |
| 475 | + _NS_PRIVATE_DEF_SEL(shortValue, | |
| 476 | + "shortValue"); | |
| 477 | + _NS_PRIVATE_DEF_SEL(showPools, | |
| 478 | + "showPools"); | |
| 479 | + _NS_PRIVATE_DEF_SEL(signal, | |
| 480 | + "signal"); | |
| 481 | + _NS_PRIVATE_DEF_SEL(string, | |
| 482 | + "string"); | |
| 483 | + _NS_PRIVATE_DEF_SEL(stringValue, | |
| 484 | + "stringValue"); | |
| 485 | + _NS_PRIVATE_DEF_SEL(stringWithString_, | |
| 486 | + "stringWithString:"); | |
| 487 | + _NS_PRIVATE_DEF_SEL(stringWithCString_encoding_, | |
| 488 | + "stringWithCString:encoding:"); | |
| 489 | + _NS_PRIVATE_DEF_SEL(stringByAppendingString_, | |
| 490 | + "stringByAppendingString:"); | |
| 491 | + _NS_PRIVATE_DEF_SEL(systemUptime, | |
| 492 | + "systemUptime"); | |
| 493 | + _NS_PRIVATE_DEF_SEL(thermalState, | |
| 494 | + "thermalState"); | |
| 495 | + _NS_PRIVATE_DEF_SEL(unload, | |
| 496 | + "unload"); | |
| 497 | + _NS_PRIVATE_DEF_SEL(unlock, | |
| 498 | + "unlock"); | |
| 499 | + _NS_PRIVATE_DEF_SEL(unsignedCharValue, | |
| 500 | + "unsignedCharValue"); | |
| 501 | + _NS_PRIVATE_DEF_SEL(unsignedIntegerValue, | |
| 502 | + "unsignedIntegerValue"); | |
| 503 | + _NS_PRIVATE_DEF_SEL(unsignedIntValue, | |
| 504 | + "unsignedIntValue"); | |
| 505 | + _NS_PRIVATE_DEF_SEL(unsignedLongValue, | |
| 506 | + "unsignedLongValue"); | |
| 507 | + _NS_PRIVATE_DEF_SEL(unsignedLongLongValue, | |
| 508 | + "unsignedLongLongValue"); | |
| 509 | + _NS_PRIVATE_DEF_SEL(unsignedShortValue, | |
| 510 | + "unsignedShortValue"); | |
| 511 | + _NS_PRIVATE_DEF_SEL(URLForAuxiliaryExecutable_, | |
| 512 | + "URLForAuxiliaryExecutable:"); | |
| 513 | + _NS_PRIVATE_DEF_SEL(userInfo, | |
| 514 | + "userInfo"); | |
| 515 | + _NS_PRIVATE_DEF_SEL(userName, | |
| 516 | + "userName"); | |
| 517 | + _NS_PRIVATE_DEF_SEL(UTF8String, | |
| 518 | + "UTF8String"); | |
| 519 | + _NS_PRIVATE_DEF_SEL(valueWithBytes_objCType_, | |
| 520 | + "valueWithBytes:objCType:"); | |
| 521 | + _NS_PRIVATE_DEF_SEL(valueWithPointer_, | |
| 522 | + "valueWithPointer:"); | |
| 523 | + _NS_PRIVATE_DEF_SEL(wait, | |
| 524 | + "wait"); | |
| 525 | + _NS_PRIVATE_DEF_SEL(waitUntilDate_, | |
| 526 | + "waitUntilDate:"); | |
| 527 | + } // Class | |
| 528 | +} // Private | |
| 529 | +} // MTL | |
| 530 | + | |
| 531 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
added
third_party/metal-cpp/Foundation/NSProcessInfo.hpp
+386 −0
@@ -0,0 +1,386 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Foundation/NSProcessInfo.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2024 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 24 | + | |
| 25 | +#include "NSDefines.hpp" | |
| 26 | +#include "NSNotification.hpp" | |
| 27 | +#include "NSObject.hpp" | |
| 28 | +#include "NSPrivate.hpp" | |
| 29 | +#include "NSTypes.hpp" | |
| 30 | + | |
| 31 | +#include <functional> | |
| 32 | + | |
| 33 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 34 | + | |
| 35 | +namespace NS | |
| 36 | +{ | |
| 37 | +_NS_CONST(NotificationName, ProcessInfoThermalStateDidChangeNotification); | |
| 38 | +_NS_CONST(NotificationName, ProcessInfoPowerStateDidChangeNotification); | |
| 39 | +_NS_CONST(NotificationName, ProcessInfoPerformanceProfileDidChangeNotification); | |
| 40 | + | |
| 41 | +_NS_ENUM(NS::Integer, ProcessInfoThermalState) { | |
| 42 | + ProcessInfoThermalStateNominal = 0, | |
| 43 | + ProcessInfoThermalStateFair = 1, | |
| 44 | + ProcessInfoThermalStateSerious = 2, | |
| 45 | + ProcessInfoThermalStateCritical = 3 | |
| 46 | +}; | |
| 47 | + | |
| 48 | +_NS_OPTIONS(std::uint64_t, ActivityOptions) { | |
| 49 | + ActivityIdleDisplaySleepDisabled = (1ULL << 40), | |
| 50 | + ActivityIdleSystemSleepDisabled = (1ULL << 20), | |
| 51 | + ActivitySuddenTerminationDisabled = (1ULL << 14), | |
| 52 | + ActivityAutomaticTerminationDisabled = (1ULL << 15), | |
| 53 | + ActivityUserInitiated = (0x00FFFFFFULL | ActivityIdleSystemSleepDisabled), | |
| 54 | + ActivityUserInitiatedAllowingIdleSystemSleep = (ActivityUserInitiated & ~ActivityIdleSystemSleepDisabled), | |
| 55 | + ActivityBackground = 0x000000FFULL, | |
| 56 | + ActivityLatencyCritical = 0xFF00000000ULL, | |
| 57 | +}; | |
| 58 | + | |
| 59 | +typedef NS::Integer DeviceCertification; | |
| 60 | +_NS_CONST(DeviceCertification, DeviceCertificationiPhonePerformanceGaming); | |
| 61 | + | |
| 62 | +typedef NS::Integer ProcessPerformanceProfile; | |
| 63 | +_NS_CONST(ProcessPerformanceProfile, ProcessPerformanceProfileDefault); | |
| 64 | +_NS_CONST(ProcessPerformanceProfile, ProcessPerformanceProfileSustained); | |
| 65 | + | |
| 66 | +class ProcessInfo : public Referencing<ProcessInfo> | |
| 67 | +{ | |
| 68 | +public: | |
| 69 | + static ProcessInfo* processInfo(); | |
| 70 | + | |
| 71 | + class Array* arguments() const; | |
| 72 | + class Dictionary* environment() const; | |
| 73 | + class String* hostName() const; | |
| 74 | + class String* processName() const; | |
| 75 | + void setProcessName(const String* pString); | |
| 76 | + int processIdentifier() const; | |
| 77 | + class String* globallyUniqueString() const; | |
| 78 | + | |
| 79 | + class String* userName() const; | |
| 80 | + class String* fullUserName() const; | |
| 81 | + | |
| 82 | + UInteger operatingSystem() const; | |
| 83 | + OperatingSystemVersion operatingSystemVersion() const; | |
| 84 | + class String* operatingSystemVersionString() const; | |
| 85 | + bool isOperatingSystemAtLeastVersion(OperatingSystemVersion version) const; | |
| 86 | + | |
| 87 | + UInteger processorCount() const; | |
| 88 | + UInteger activeProcessorCount() const; | |
| 89 | + unsigned long long physicalMemory() const; | |
| 90 | + TimeInterval systemUptime() const; | |
| 91 | + | |
| 92 | + void disableSuddenTermination(); | |
| 93 | + void enableSuddenTermination(); | |
| 94 | + | |
| 95 | + void disableAutomaticTermination(const class String* pReason); | |
| 96 | + void enableAutomaticTermination(const class String* pReason); | |
| 97 | + bool automaticTerminationSupportEnabled() const; | |
| 98 | + void setAutomaticTerminationSupportEnabled(bool enabled); | |
| 99 | + | |
| 100 | + class Object* beginActivity(ActivityOptions options, const class String* pReason); | |
| 101 | + void endActivity(class Object* pActivity); | |
| 102 | + void performActivity(ActivityOptions options, const class String* pReason, void (^block)(void)); | |
| 103 | + void performActivity(ActivityOptions options, const class String* pReason, const std::function<void()>& func); | |
| 104 | + void performExpiringActivity(const class String* pReason, void (^block)(bool expired)); | |
| 105 | + void performExpiringActivity(const class String* pReason, const std::function<void(bool expired)>& func); | |
| 106 | + | |
| 107 | + ProcessInfoThermalState thermalState() const; | |
| 108 | + bool isLowPowerModeEnabled() const; | |
| 109 | + | |
| 110 | + bool isiOSAppOnMac() const; | |
| 111 | + bool isMacCatalystApp() const; | |
| 112 | + | |
| 113 | + bool isDeviceCertified(DeviceCertification performanceTier) const; | |
| 114 | + bool hasPerformanceProfile(ProcessPerformanceProfile performanceProfile) const; | |
| 115 | + | |
| 116 | +}; | |
| 117 | +} | |
| 118 | + | |
| 119 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 120 | + | |
| 121 | +_NS_PRIVATE_DEF_CONST(NS::NotificationName, ProcessInfoThermalStateDidChangeNotification); | |
| 122 | +_NS_PRIVATE_DEF_CONST(NS::NotificationName, ProcessInfoPowerStateDidChangeNotification); | |
| 123 | + | |
| 124 | +// The linker searches for these symbols in the Metal framework, be sure to link it in as well: | |
| 125 | +_NS_PRIVATE_DEF_CONST(NS::NotificationName, ProcessInfoPerformanceProfileDidChangeNotification); | |
| 126 | +_NS_PRIVATE_DEF_CONST(NS::DeviceCertification, DeviceCertificationiPhonePerformanceGaming); | |
| 127 | +_NS_PRIVATE_DEF_CONST(NS::ProcessPerformanceProfile, ProcessPerformanceProfileDefault); | |
| 128 | +_NS_PRIVATE_DEF_CONST(NS::ProcessPerformanceProfile, ProcessPerformanceProfileSustained); | |
| 129 | + | |
| 130 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 131 | + | |
| 132 | +_NS_INLINE NS::ProcessInfo* NS::ProcessInfo::processInfo() | |
| 133 | +{ | |
| 134 | + return Object::sendMessage<ProcessInfo*>(_NS_PRIVATE_CLS(NSProcessInfo), _NS_PRIVATE_SEL(processInfo)); | |
| 135 | +} | |
| 136 | + | |
| 137 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 138 | + | |
| 139 | +_NS_INLINE NS::Array* NS::ProcessInfo::arguments() const | |
| 140 | +{ | |
| 141 | + return Object::sendMessage<Array*>(this, _NS_PRIVATE_SEL(arguments)); | |
| 142 | +} | |
| 143 | + | |
| 144 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 145 | + | |
| 146 | +_NS_INLINE NS::Dictionary* NS::ProcessInfo::environment() const | |
| 147 | +{ | |
| 148 | + return Object::sendMessage<Dictionary*>(this, _NS_PRIVATE_SEL(environment)); | |
| 149 | +} | |
| 150 | + | |
| 151 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 152 | + | |
| 153 | +_NS_INLINE NS::String* NS::ProcessInfo::hostName() const | |
| 154 | +{ | |
| 155 | + return Object::sendMessage<String*>(this, _NS_PRIVATE_SEL(hostName)); | |
| 156 | +} | |
| 157 | + | |
| 158 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 159 | + | |
| 160 | +_NS_INLINE NS::String* NS::ProcessInfo::processName() const | |
| 161 | +{ | |
| 162 | + return Object::sendMessage<String*>(this, _NS_PRIVATE_SEL(processName)); | |
| 163 | +} | |
| 164 | + | |
| 165 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 166 | + | |
| 167 | +_NS_INLINE void NS::ProcessInfo::setProcessName(const String* pString) | |
| 168 | +{ | |
| 169 | + Object::sendMessage<void>(this, _NS_PRIVATE_SEL(setProcessName_), pString); | |
| 170 | +} | |
| 171 | + | |
| 172 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 173 | + | |
| 174 | +_NS_INLINE int NS::ProcessInfo::processIdentifier() const | |
| 175 | +{ | |
| 176 | + return Object::sendMessage<int>(this, _NS_PRIVATE_SEL(processIdentifier)); | |
| 177 | +} | |
| 178 | + | |
| 179 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 180 | + | |
| 181 | +_NS_INLINE NS::String* NS::ProcessInfo::globallyUniqueString() const | |
| 182 | +{ | |
| 183 | + return Object::sendMessage<String*>(this, _NS_PRIVATE_SEL(globallyUniqueString)); | |
| 184 | +} | |
| 185 | + | |
| 186 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 187 | + | |
| 188 | +_NS_INLINE NS::String* NS::ProcessInfo::userName() const | |
| 189 | +{ | |
| 190 | + return Object::sendMessageSafe<String*>(this, _NS_PRIVATE_SEL(userName)); | |
| 191 | +} | |
| 192 | + | |
| 193 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 194 | + | |
| 195 | +_NS_INLINE NS::String* NS::ProcessInfo::fullUserName() const | |
| 196 | +{ | |
| 197 | + return Object::sendMessageSafe<String*>(this, _NS_PRIVATE_SEL(fullUserName)); | |
| 198 | +} | |
| 199 | + | |
| 200 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 201 | + | |
| 202 | +_NS_INLINE NS::UInteger NS::ProcessInfo::operatingSystem() const | |
| 203 | +{ | |
| 204 | + return Object::sendMessage<UInteger>(this, _NS_PRIVATE_SEL(operatingSystem)); | |
| 205 | +} | |
| 206 | + | |
| 207 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 208 | + | |
| 209 | +_NS_INLINE NS::OperatingSystemVersion NS::ProcessInfo::operatingSystemVersion() const | |
| 210 | +{ | |
| 211 | + return Object::sendMessage<OperatingSystemVersion>(this, _NS_PRIVATE_SEL(operatingSystemVersion)); | |
| 212 | +} | |
| 213 | + | |
| 214 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 215 | + | |
| 216 | +_NS_INLINE NS::String* NS::ProcessInfo::operatingSystemVersionString() const | |
| 217 | +{ | |
| 218 | + return Object::sendMessage<String*>(this, _NS_PRIVATE_SEL(operatingSystemVersionString)); | |
| 219 | +} | |
| 220 | + | |
| 221 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 222 | + | |
| 223 | +_NS_INLINE bool NS::ProcessInfo::isOperatingSystemAtLeastVersion(OperatingSystemVersion version) const | |
| 224 | +{ | |
| 225 | + return Object::sendMessage<bool>(this, _NS_PRIVATE_SEL(isOperatingSystemAtLeastVersion_), version); | |
| 226 | +} | |
| 227 | + | |
| 228 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 229 | + | |
| 230 | +_NS_INLINE NS::UInteger NS::ProcessInfo::processorCount() const | |
| 231 | +{ | |
| 232 | + return Object::sendMessage<UInteger>(this, _NS_PRIVATE_SEL(processorCount)); | |
| 233 | +} | |
| 234 | + | |
| 235 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 236 | + | |
| 237 | +_NS_INLINE NS::UInteger NS::ProcessInfo::activeProcessorCount() const | |
| 238 | +{ | |
| 239 | + return Object::sendMessage<UInteger>(this, _NS_PRIVATE_SEL(activeProcessorCount)); | |
| 240 | +} | |
| 241 | + | |
| 242 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 243 | + | |
| 244 | +_NS_INLINE unsigned long long NS::ProcessInfo::physicalMemory() const | |
| 245 | +{ | |
| 246 | + return Object::sendMessage<unsigned long long>(this, _NS_PRIVATE_SEL(physicalMemory)); | |
| 247 | +} | |
| 248 | + | |
| 249 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 250 | + | |
| 251 | +_NS_INLINE NS::TimeInterval NS::ProcessInfo::systemUptime() const | |
| 252 | +{ | |
| 253 | + return Object::sendMessage<TimeInterval>(this, _NS_PRIVATE_SEL(systemUptime)); | |
| 254 | +} | |
| 255 | + | |
| 256 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 257 | + | |
| 258 | +_NS_INLINE void NS::ProcessInfo::disableSuddenTermination() | |
| 259 | +{ | |
| 260 | + Object::sendMessageSafe<void>(this, _NS_PRIVATE_SEL(disableSuddenTermination)); | |
| 261 | +} | |
| 262 | + | |
| 263 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 264 | + | |
| 265 | +_NS_INLINE void NS::ProcessInfo::enableSuddenTermination() | |
| 266 | +{ | |
| 267 | + Object::sendMessageSafe<void>(this, _NS_PRIVATE_SEL(enableSuddenTermination)); | |
| 268 | +} | |
| 269 | + | |
| 270 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 271 | + | |
| 272 | +_NS_INLINE void NS::ProcessInfo::disableAutomaticTermination(const String* pReason) | |
| 273 | +{ | |
| 274 | + Object::sendMessageSafe<void>(this, _NS_PRIVATE_SEL(disableAutomaticTermination_), pReason); | |
| 275 | +} | |
| 276 | + | |
| 277 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 278 | + | |
| 279 | +_NS_INLINE void NS::ProcessInfo::enableAutomaticTermination(const String* pReason) | |
| 280 | +{ | |
| 281 | + Object::sendMessageSafe<void>(this, _NS_PRIVATE_SEL(enableAutomaticTermination_), pReason); | |
| 282 | +} | |
| 283 | + | |
| 284 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 285 | + | |
| 286 | +_NS_INLINE bool NS::ProcessInfo::automaticTerminationSupportEnabled() const | |
| 287 | +{ | |
| 288 | + return Object::sendMessageSafe<bool>(this, _NS_PRIVATE_SEL(automaticTerminationSupportEnabled)); | |
| 289 | +} | |
| 290 | + | |
| 291 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 292 | + | |
| 293 | +_NS_INLINE void NS::ProcessInfo::setAutomaticTerminationSupportEnabled(bool enabled) | |
| 294 | +{ | |
| 295 | + Object::sendMessageSafe<void>(this, _NS_PRIVATE_SEL(setAutomaticTerminationSupportEnabled_), enabled); | |
| 296 | +} | |
| 297 | + | |
| 298 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 299 | + | |
| 300 | +_NS_INLINE NS::Object* NS::ProcessInfo::beginActivity(ActivityOptions options, const String* pReason) | |
| 301 | +{ | |
| 302 | + return Object::sendMessage<Object*>(this, _NS_PRIVATE_SEL(beginActivityWithOptions_reason_), options, pReason); | |
| 303 | +} | |
| 304 | + | |
| 305 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 306 | + | |
| 307 | +_NS_INLINE void NS::ProcessInfo::endActivity(Object* pActivity) | |
| 308 | +{ | |
| 309 | + Object::sendMessage<void>(this, _NS_PRIVATE_SEL(endActivity_), pActivity); | |
| 310 | +} | |
| 311 | + | |
| 312 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 313 | + | |
| 314 | +_NS_INLINE void NS::ProcessInfo::performActivity(ActivityOptions options, const String* pReason, void (^block)(void)) | |
| 315 | +{ | |
| 316 | + Object::sendMessage<void>(this, _NS_PRIVATE_SEL(performActivityWithOptions_reason_usingBlock_), options, pReason, block); | |
| 317 | +} | |
| 318 | + | |
| 319 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 320 | + | |
| 321 | +_NS_INLINE void NS::ProcessInfo::performActivity(ActivityOptions options, const String* pReason, const std::function<void()>& function) | |
| 322 | +{ | |
| 323 | + __block std::function<void()> blockFunction = function; | |
| 324 | + | |
| 325 | + performActivity(options, pReason, ^() { blockFunction(); }); | |
| 326 | +} | |
| 327 | + | |
| 328 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 329 | + | |
| 330 | +_NS_INLINE void NS::ProcessInfo::performExpiringActivity(const String* pReason, void (^block)(bool expired)) | |
| 331 | +{ | |
| 332 | + Object::sendMessageSafe<void>(this, _NS_PRIVATE_SEL(performExpiringActivityWithReason_usingBlock_), pReason, block); | |
| 333 | +} | |
| 334 | + | |
| 335 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 336 | + | |
| 337 | +_NS_INLINE void NS::ProcessInfo::performExpiringActivity(const String* pReason, const std::function<void(bool expired)>& function) | |
| 338 | +{ | |
| 339 | + __block std::function<void(bool expired)> blockFunction = function; | |
| 340 | + | |
| 341 | + performExpiringActivity(pReason, ^(bool expired) { blockFunction(expired); }); | |
| 342 | +} | |
| 343 | + | |
| 344 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 345 | + | |
| 346 | +_NS_INLINE NS::ProcessInfoThermalState NS::ProcessInfo::thermalState() const | |
| 347 | +{ | |
| 348 | + return Object::sendMessage<ProcessInfoThermalState>(this, _NS_PRIVATE_SEL(thermalState)); | |
| 349 | +} | |
| 350 | + | |
| 351 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 352 | + | |
| 353 | +_NS_INLINE bool NS::ProcessInfo::isLowPowerModeEnabled() const | |
| 354 | +{ | |
| 355 | + return Object::sendMessageSafe<bool>(this, _NS_PRIVATE_SEL(isLowPowerModeEnabled)); | |
| 356 | +} | |
| 357 | + | |
| 358 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 359 | + | |
| 360 | +_NS_INLINE bool NS::ProcessInfo::isiOSAppOnMac() const | |
| 361 | +{ | |
| 362 | + return Object::sendMessageSafe<bool>(this, _NS_PRIVATE_SEL(isiOSAppOnMac)); | |
| 363 | +} | |
| 364 | + | |
| 365 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 366 | + | |
| 367 | +_NS_INLINE bool NS::ProcessInfo::isMacCatalystApp() const | |
| 368 | +{ | |
| 369 | + return Object::sendMessageSafe<bool>(this, _NS_PRIVATE_SEL(isMacCatalystApp)); | |
| 370 | +} | |
| 371 | + | |
| 372 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 373 | + | |
| 374 | +_NS_INLINE bool NS::ProcessInfo::isDeviceCertified(DeviceCertification performanceTier) const | |
| 375 | +{ | |
| 376 | + return Object::sendMessageSafe<bool>(this, _NS_PRIVATE_SEL(isDeviceCertified_), performanceTier); | |
| 377 | +} | |
| 378 | + | |
| 379 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 380 | + | |
| 381 | +_NS_INLINE bool NS::ProcessInfo::hasPerformanceProfile(ProcessPerformanceProfile performanceProfile) const | |
| 382 | +{ | |
| 383 | + return Object::sendMessageSafe<bool>(this, _NS_PRIVATE_SEL(hasPerformanceProfile_), performanceProfile); | |
| 384 | +} | |
| 385 | + | |
| 386 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
added
third_party/metal-cpp/Foundation/NSRange.hpp
+83 −0
@@ -0,0 +1,83 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Foundation/NSRange.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2024 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 24 | + | |
| 25 | +#include "NSDefines.hpp" | |
| 26 | +#include "NSTypes.hpp" | |
| 27 | + | |
| 28 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 29 | + | |
| 30 | +namespace NS | |
| 31 | +{ | |
| 32 | +struct Range | |
| 33 | +{ | |
| 34 | + static Range Make(UInteger loc, UInteger len); | |
| 35 | + | |
| 36 | + Range(UInteger loc, UInteger len); | |
| 37 | + | |
| 38 | + bool Equal(const Range& range) const; | |
| 39 | + bool LocationInRange(UInteger loc) const; | |
| 40 | + UInteger Max() const; | |
| 41 | + | |
| 42 | + UInteger location; | |
| 43 | + UInteger length; | |
| 44 | +} _NS_PACKED; | |
| 45 | +} | |
| 46 | + | |
| 47 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 48 | + | |
| 49 | +_NS_INLINE NS::Range::Range(UInteger loc, UInteger len) | |
| 50 | + : location(loc) | |
| 51 | + , length(len) | |
| 52 | +{ | |
| 53 | +} | |
| 54 | + | |
| 55 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 56 | + | |
| 57 | +_NS_INLINE NS::Range NS::Range::Make(UInteger loc, UInteger len) | |
| 58 | +{ | |
| 59 | + return Range(loc, len); | |
| 60 | +} | |
| 61 | + | |
| 62 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 63 | + | |
| 64 | +_NS_INLINE bool NS::Range::Equal(const Range& range) const | |
| 65 | +{ | |
| 66 | + return (location == range.location) && (length == range.length); | |
| 67 | +} | |
| 68 | + | |
| 69 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 70 | + | |
| 71 | +_NS_INLINE bool NS::Range::LocationInRange(UInteger loc) const | |
| 72 | +{ | |
| 73 | + return (!(loc < location)) && ((loc - location) < length); | |
| 74 | +} | |
| 75 | + | |
| 76 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 77 | + | |
| 78 | +_NS_INLINE NS::UInteger NS::Range::Max() const | |
| 79 | +{ | |
| 80 | + return location + length; | |
| 81 | +} | |
| 82 | + | |
| 83 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
added
third_party/metal-cpp/Foundation/NSSet.hpp
+87 −0
@@ -0,0 +1,87 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Foundation/NSSet.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2024 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 24 | + | |
| 25 | +#include "NSObject.hpp" | |
| 26 | +#include "NSEnumerator.hpp" | |
| 27 | + | |
| 28 | +/*****Immutable Set*******/ | |
| 29 | + | |
| 30 | +namespace NS | |
| 31 | +{ | |
| 32 | + class Set : public NS::Copying <Set> | |
| 33 | + { | |
| 34 | + public: | |
| 35 | + UInteger count() const; | |
| 36 | + Enumerator<Object>* objectEnumerator() const; | |
| 37 | + | |
| 38 | + static Set* alloc(); | |
| 39 | + | |
| 40 | + Set* init(); | |
| 41 | + Set* init(const Object* const* pObjects, UInteger count); | |
| 42 | + Set* init(const class Coder* pCoder); | |
| 43 | + | |
| 44 | + }; | |
| 45 | +} | |
| 46 | + | |
| 47 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 48 | + | |
| 49 | +_NS_INLINE NS::UInteger NS::Set::count() const | |
| 50 | +{ | |
| 51 | + return NS::Object::sendMessage<NS::UInteger>(this, _NS_PRIVATE_SEL(count)); | |
| 52 | +} | |
| 53 | + | |
| 54 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 55 | + | |
| 56 | +_NS_INLINE NS::Enumerator<NS::Object>* NS::Set::objectEnumerator() const | |
| 57 | +{ | |
| 58 | + return NS::Object::sendMessage<Enumerator<NS::Object>*>(this, _NS_PRIVATE_SEL(objectEnumerator)); | |
| 59 | +} | |
| 60 | + | |
| 61 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 62 | + | |
| 63 | +_NS_INLINE NS::Set* NS::Set::alloc() | |
| 64 | +{ | |
| 65 | + return NS::Object::alloc<Set>(_NS_PRIVATE_CLS(NSSet)); | |
| 66 | +} | |
| 67 | + | |
| 68 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 69 | + | |
| 70 | +_NS_INLINE NS::Set* NS::Set::init() | |
| 71 | +{ | |
| 72 | + return NS::Object::init<Set>(); | |
| 73 | +} | |
| 74 | + | |
| 75 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 76 | + | |
| 77 | +_NS_INLINE NS::Set* NS::Set::init(const Object* const* pObjects, NS::UInteger count) | |
| 78 | +{ | |
| 79 | + return NS::Object::sendMessage<Set*>(this, _NS_PRIVATE_SEL(initWithObjects_count_), pObjects, count); | |
| 80 | +} | |
| 81 | + | |
| 82 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 83 | + | |
| 84 | +_NS_INLINE NS::Set* NS::Set::init(const class Coder* pCoder) | |
| 85 | +{ | |
| 86 | + return Object::sendMessage<Set*>(this, _NS_PRIVATE_SEL(initWithCoder_), pCoder); | |
| 87 | +} | |
added
third_party/metal-cpp/Foundation/NSSharedPtr.hpp
+324 −0
@@ -0,0 +1,324 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Foundation/NSSharedPtr.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2024 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include <cstddef> | |
| 24 | +#include "NSDefines.hpp" | |
| 25 | + | |
| 26 | +namespace NS | |
| 27 | +{ | |
| 28 | +template <class _Class> | |
| 29 | +class SharedPtr | |
| 30 | +{ | |
| 31 | +public: | |
| 32 | + /** | |
| 33 | + * Create a new null pointer. | |
| 34 | + */ | |
| 35 | + SharedPtr(); | |
| 36 | + | |
| 37 | + /** | |
| 38 | + * Destroy this SharedPtr, decreasing the reference count. | |
| 39 | + */ | |
| 40 | + ~SharedPtr(); | |
| 41 | + | |
| 42 | + /** | |
| 43 | + * Create a new null pointer. | |
| 44 | + */ | |
| 45 | + SharedPtr(std::nullptr_t) noexcept; | |
| 46 | + | |
| 47 | + /** | |
| 48 | + * SharedPtr copy constructor. | |
| 49 | + */ | |
| 50 | + SharedPtr(const SharedPtr<_Class>& other) noexcept; | |
| 51 | + | |
| 52 | + /** | |
| 53 | + * Construction from another pointee type. | |
| 54 | + */ | |
| 55 | + template <class _OtherClass> | |
| 56 | + SharedPtr(const SharedPtr<_OtherClass>& other, typename std::enable_if_t<std::is_convertible_v<_OtherClass *, _Class *>> * = nullptr) noexcept; | |
| 57 | + | |
| 58 | + /** | |
| 59 | + * SharedPtr move constructor. | |
| 60 | + */ | |
| 61 | + SharedPtr(SharedPtr<_Class>&& other) noexcept; | |
| 62 | + | |
| 63 | + /** | |
| 64 | + * Move from another pointee type. | |
| 65 | + */ | |
| 66 | + template <class _OtherClass> | |
| 67 | + SharedPtr(SharedPtr<_OtherClass>&& other, typename std::enable_if_t<std::is_convertible_v<_OtherClass *, _Class *>> * = nullptr) noexcept; | |
| 68 | + | |
| 69 | + /** | |
| 70 | + * Copy assignment operator. | |
| 71 | + * Copying increases reference count. Only releases previous pointee if objects are different. | |
| 72 | + */ | |
| 73 | + SharedPtr& operator=(const SharedPtr<_Class>& other); | |
| 74 | + | |
| 75 | + /** | |
| 76 | + * Copy-assignment from different pointee. | |
| 77 | + * Copying increases reference count. Only releases previous pointee if objects are different. | |
| 78 | + */ | |
| 79 | + template <class _OtherClass> | |
| 80 | + typename std::enable_if_t<std::is_convertible_v<_OtherClass *, _Class *>, SharedPtr &> | |
| 81 | + operator=(const SharedPtr<_OtherClass>& other); | |
| 82 | + | |
| 83 | + /** | |
| 84 | + * Move assignment operator. | |
| 85 | + * Move without affecting reference counts, unless pointees are equal. Moved-from object is reset to nullptr. | |
| 86 | + */ | |
| 87 | + SharedPtr& operator=(SharedPtr<_Class>&& other); | |
| 88 | + | |
| 89 | + /** | |
| 90 | + * Move-asignment from different pointee. | |
| 91 | + * Move without affecting reference counts, unless pointees are equal. Moved-from object is reset to nullptr. | |
| 92 | + */ | |
| 93 | + template <class _OtherClass> | |
| 94 | + typename std::enable_if_t<std::is_convertible_v<_OtherClass *, _Class *>, SharedPtr &> | |
| 95 | + operator=(SharedPtr<_OtherClass>&& other); | |
| 96 | + | |
| 97 | + /** | |
| 98 | + * Access raw pointee. | |
| 99 | + * @warning Avoid wrapping the returned value again, as it may lead double frees unless this object becomes detached. | |
| 100 | + */ | |
| 101 | + _Class* get() const; | |
| 102 | + | |
| 103 | + /** | |
| 104 | + * Call operations directly on the pointee. | |
| 105 | + */ | |
| 106 | + _Class* operator->() const; | |
| 107 | + | |
| 108 | + /** | |
| 109 | + * Implicit cast to bool. | |
| 110 | + */ | |
| 111 | + explicit operator bool() const; | |
| 112 | + | |
| 113 | + /** | |
| 114 | + * Reset this SharedPtr to null, decreasing the reference count. | |
| 115 | + */ | |
| 116 | + void reset(); | |
| 117 | + | |
| 118 | + /** | |
| 119 | + * Detach the SharedPtr from the pointee, without decreasing the reference count. | |
| 120 | + */ | |
| 121 | + void detach(); | |
| 122 | + | |
| 123 | + template <class _OtherClass> | |
| 124 | + friend SharedPtr<_OtherClass> RetainPtr(_OtherClass* ptr); | |
| 125 | + | |
| 126 | + template <class _OtherClass> | |
| 127 | + friend SharedPtr<_OtherClass> TransferPtr(_OtherClass* ptr); | |
| 128 | + | |
| 129 | +private: | |
| 130 | + _Class* m_pObject; | |
| 131 | +}; | |
| 132 | + | |
| 133 | +/** | |
| 134 | + * Create a SharedPtr by retaining an existing raw pointer. | |
| 135 | + * Increases the reference count of the passed-in object. | |
| 136 | + * If the passed-in object was in an AutoreleasePool, it will be removed from it. | |
| 137 | + */ | |
| 138 | +template <class _Class> | |
| 139 | +_NS_INLINE NS::SharedPtr<_Class> RetainPtr(_Class* pObject) | |
| 140 | +{ | |
| 141 | + NS::SharedPtr<_Class> ret; | |
| 142 | + ret.m_pObject = pObject->retain(); | |
| 143 | + return ret; | |
| 144 | +} | |
| 145 | + | |
| 146 | +/* | |
| 147 | + * Create a SharedPtr by transfering the ownership of an existing raw pointer to SharedPtr. | |
| 148 | + * Does not increase the reference count of the passed-in pointer, it is assumed to be >= 1. | |
| 149 | + * This method does not remove objects from an AutoreleasePool. | |
| 150 | +*/ | |
| 151 | +template <class _Class> | |
| 152 | +_NS_INLINE NS::SharedPtr<_Class> TransferPtr(_Class* pObject) | |
| 153 | +{ | |
| 154 | + NS::SharedPtr<_Class> ret; | |
| 155 | + ret.m_pObject = pObject; | |
| 156 | + return ret; | |
| 157 | +} | |
| 158 | + | |
| 159 | +} | |
| 160 | + | |
| 161 | +template <class _Class> | |
| 162 | +_NS_INLINE NS::SharedPtr<_Class>::SharedPtr() | |
| 163 | + : m_pObject(nullptr) | |
| 164 | +{ | |
| 165 | +} | |
| 166 | + | |
| 167 | +template <class _Class> | |
| 168 | +_NS_INLINE NS::SharedPtr<_Class>::~SharedPtr<_Class>() __attribute__((no_sanitize("undefined"))) | |
| 169 | +{ | |
| 170 | + m_pObject->release(); | |
| 171 | +} | |
| 172 | + | |
| 173 | +template <class _Class> | |
| 174 | +_NS_INLINE NS::SharedPtr<_Class>::SharedPtr(std::nullptr_t) noexcept | |
| 175 | + : m_pObject(nullptr) | |
| 176 | +{ | |
| 177 | +} | |
| 178 | + | |
| 179 | +template <class _Class> | |
| 180 | +_NS_INLINE NS::SharedPtr<_Class>::SharedPtr(const SharedPtr<_Class>& other) noexcept | |
| 181 | + : m_pObject(other.m_pObject->retain()) | |
| 182 | +{ | |
| 183 | +} | |
| 184 | + | |
| 185 | +template <class _Class> | |
| 186 | +template <class _OtherClass> | |
| 187 | +_NS_INLINE NS::SharedPtr<_Class>::SharedPtr(const SharedPtr<_OtherClass>& other, typename std::enable_if_t<std::is_convertible_v<_OtherClass *, _Class *>> *) noexcept | |
| 188 | + : m_pObject(reinterpret_cast<_Class*>(other.get()->retain())) | |
| 189 | +{ | |
| 190 | +} | |
| 191 | + | |
| 192 | +template <class _Class> | |
| 193 | +_NS_INLINE NS::SharedPtr<_Class>::SharedPtr(SharedPtr<_Class>&& other) noexcept | |
| 194 | + : m_pObject(other.m_pObject) | |
| 195 | +{ | |
| 196 | + other.m_pObject = nullptr; | |
| 197 | +} | |
| 198 | + | |
| 199 | +template <class _Class> | |
| 200 | +template <class _OtherClass> | |
| 201 | +_NS_INLINE NS::SharedPtr<_Class>::SharedPtr(SharedPtr<_OtherClass>&& other, typename std::enable_if_t<std::is_convertible_v<_OtherClass *, _Class *>> *) noexcept | |
| 202 | + : m_pObject(reinterpret_cast<_Class*>(other.get())) | |
| 203 | +{ | |
| 204 | + other.detach(); | |
| 205 | +} | |
| 206 | + | |
| 207 | +template <class _Class> | |
| 208 | +_NS_INLINE _Class* NS::SharedPtr<_Class>::get() const | |
| 209 | +{ | |
| 210 | + return m_pObject; | |
| 211 | +} | |
| 212 | + | |
| 213 | +template <class _Class> | |
| 214 | +_NS_INLINE _Class* NS::SharedPtr<_Class>::operator->() const | |
| 215 | +{ | |
| 216 | + return m_pObject; | |
| 217 | +} | |
| 218 | + | |
| 219 | +template <class _Class> | |
| 220 | +_NS_INLINE NS::SharedPtr<_Class>::operator bool() const | |
| 221 | +{ | |
| 222 | + return nullptr != m_pObject; | |
| 223 | +} | |
| 224 | + | |
| 225 | +template <class _Class> | |
| 226 | +_NS_INLINE void NS::SharedPtr<_Class>::reset() __attribute__((no_sanitize("undefined"))) | |
| 227 | +{ | |
| 228 | + m_pObject->release(); | |
| 229 | + m_pObject = nullptr; | |
| 230 | +} | |
| 231 | + | |
| 232 | +template <class _Class> | |
| 233 | +_NS_INLINE void NS::SharedPtr<_Class>::detach() | |
| 234 | +{ | |
| 235 | + m_pObject = nullptr; | |
| 236 | +} | |
| 237 | + | |
| 238 | +template <class _Class> | |
| 239 | +_NS_INLINE NS::SharedPtr<_Class>& NS::SharedPtr<_Class>::operator=(const SharedPtr<_Class>& other) __attribute__((no_sanitize("undefined"))) | |
| 240 | +{ | |
| 241 | + _Class* pOldObject = m_pObject; | |
| 242 | + | |
| 243 | + m_pObject = other.m_pObject->retain(); | |
| 244 | + | |
| 245 | + pOldObject->release(); | |
| 246 | + | |
| 247 | + return *this; | |
| 248 | +} | |
| 249 | + | |
| 250 | +template <class _Class> | |
| 251 | +template <class _OtherClass> | |
| 252 | +typename std::enable_if_t<std::is_convertible_v<_OtherClass *, _Class *>, NS::SharedPtr<_Class> &> | |
| 253 | +_NS_INLINE NS::SharedPtr<_Class>::operator=(const SharedPtr<_OtherClass>& other) __attribute__((no_sanitize("undefined"))) | |
| 254 | +{ | |
| 255 | + _Class* pOldObject = m_pObject; | |
| 256 | + | |
| 257 | + m_pObject = reinterpret_cast<_Class*>(other.get()->retain()); | |
| 258 | + | |
| 259 | + pOldObject->release(); | |
| 260 | + | |
| 261 | + return *this; | |
| 262 | +} | |
| 263 | + | |
| 264 | +template <class _Class> | |
| 265 | +_NS_INLINE NS::SharedPtr<_Class>& NS::SharedPtr<_Class>::operator=(SharedPtr<_Class>&& other) __attribute__((no_sanitize("undefined"))) | |
| 266 | +{ | |
| 267 | + if (m_pObject != other.m_pObject) | |
| 268 | + { | |
| 269 | + m_pObject->release(); | |
| 270 | + m_pObject = other.m_pObject; | |
| 271 | + } | |
| 272 | + else | |
| 273 | + { | |
| 274 | + m_pObject = other.m_pObject; | |
| 275 | + other.m_pObject->release(); | |
| 276 | + } | |
| 277 | + other.m_pObject = nullptr; | |
| 278 | + return *this; | |
| 279 | +} | |
| 280 | + | |
| 281 | +template <class _Class> | |
| 282 | +template <class _OtherClass> | |
| 283 | +typename std::enable_if_t<std::is_convertible_v<_OtherClass *, _Class *>, NS::SharedPtr<_Class> &> | |
| 284 | +_NS_INLINE NS::SharedPtr<_Class>::operator=(SharedPtr<_OtherClass>&& other) __attribute__((no_sanitize("undefined"))) | |
| 285 | +{ | |
| 286 | + if (m_pObject != other.get()) | |
| 287 | + { | |
| 288 | + m_pObject->release(); | |
| 289 | + m_pObject = reinterpret_cast<_Class*>(other.get()); | |
| 290 | + other.detach(); | |
| 291 | + } | |
| 292 | + else | |
| 293 | + { | |
| 294 | + m_pObject = other.get(); | |
| 295 | + other.reset(); | |
| 296 | + } | |
| 297 | + return *this; | |
| 298 | +} | |
| 299 | + | |
| 300 | +template <class _ClassLhs, class _ClassRhs> | |
| 301 | +_NS_INLINE bool operator==(const NS::SharedPtr<_ClassLhs>& lhs, const NS::SharedPtr<_ClassRhs>& rhs) | |
| 302 | +{ | |
| 303 | + return lhs.get() == rhs.get(); | |
| 304 | +} | |
| 305 | + | |
| 306 | +template <class _ClassLhs, class _ClassRhs> | |
| 307 | +_NS_INLINE bool operator!=(const NS::SharedPtr<_ClassLhs>& lhs, const NS::SharedPtr<_ClassRhs>& rhs) | |
| 308 | +{ | |
| 309 | + return lhs.get() != rhs.get(); | |
| 310 | +} | |
| 311 | + | |
| 312 | +namespace std | |
| 313 | +{ | |
| 314 | + | |
| 315 | +template <class T> | |
| 316 | +struct hash<NS::SharedPtr<T>> | |
| 317 | +{ | |
| 318 | + size_t operator()(const NS::SharedPtr<T>& p) const | |
| 319 | + { | |
| 320 | + return std::hash<T*>{}(p.get()); | |
| 321 | + } | |
| 322 | +}; | |
| 323 | + | |
| 324 | +} // namespace std | |
added
third_party/metal-cpp/Foundation/NSString.hpp
+255 −0
@@ -0,0 +1,255 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Foundation/NSString.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2024 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 24 | + | |
| 25 | +#include "NSDefines.hpp" | |
| 26 | +#include "NSObjCRuntime.hpp" | |
| 27 | +#include "NSObject.hpp" | |
| 28 | +#include "NSPrivate.hpp" | |
| 29 | +#include "NSRange.hpp" | |
| 30 | +#include "NSTypes.hpp" | |
| 31 | + | |
| 32 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 33 | + | |
| 34 | +namespace NS | |
| 35 | +{ | |
| 36 | +_NS_ENUM(NS::UInteger, StringEncoding) { | |
| 37 | + ASCIIStringEncoding = 1, | |
| 38 | + NEXTSTEPStringEncoding = 2, | |
| 39 | + JapaneseEUCStringEncoding = 3, | |
| 40 | + UTF8StringEncoding = 4, | |
| 41 | + ISOLatin1StringEncoding = 5, | |
| 42 | + SymbolStringEncoding = 6, | |
| 43 | + NonLossyASCIIStringEncoding = 7, | |
| 44 | + ShiftJISStringEncoding = 8, | |
| 45 | + ISOLatin2StringEncoding = 9, | |
| 46 | + UnicodeStringEncoding = 10, | |
| 47 | + WindowsCP1251StringEncoding = 11, | |
| 48 | + WindowsCP1252StringEncoding = 12, | |
| 49 | + WindowsCP1253StringEncoding = 13, | |
| 50 | + WindowsCP1254StringEncoding = 14, | |
| 51 | + WindowsCP1250StringEncoding = 15, | |
| 52 | + ISO2022JPStringEncoding = 21, | |
| 53 | + MacOSRomanStringEncoding = 30, | |
| 54 | + | |
| 55 | + UTF16StringEncoding = UnicodeStringEncoding, | |
| 56 | + | |
| 57 | + UTF16BigEndianStringEncoding = 0x90000100, | |
| 58 | + UTF16LittleEndianStringEncoding = 0x94000100, | |
| 59 | + | |
| 60 | + UTF32StringEncoding = 0x8c000100, | |
| 61 | + UTF32BigEndianStringEncoding = 0x98000100, | |
| 62 | + UTF32LittleEndianStringEncoding = 0x9c000100 | |
| 63 | +}; | |
| 64 | + | |
| 65 | +_NS_OPTIONS(NS::UInteger, StringCompareOptions) { | |
| 66 | + CaseInsensitiveSearch = 1, | |
| 67 | + LiteralSearch = 2, | |
| 68 | + BackwardsSearch = 4, | |
| 69 | + AnchoredSearch = 8, | |
| 70 | + NumericSearch = 64, | |
| 71 | + DiacriticInsensitiveSearch = 128, | |
| 72 | + WidthInsensitiveSearch = 256, | |
| 73 | + ForcedOrderingSearch = 512, | |
| 74 | + RegularExpressionSearch = 1024 | |
| 75 | +}; | |
| 76 | + | |
| 77 | +using unichar = unsigned short; | |
| 78 | + | |
| 79 | +class String : public Copying<String> | |
| 80 | +{ | |
| 81 | +public: | |
| 82 | + static String* string(); | |
| 83 | + static String* string(const String* pString); | |
| 84 | + static String* string(const char* pString, StringEncoding encoding); | |
| 85 | + | |
| 86 | + static String* alloc(); | |
| 87 | + String* init(); | |
| 88 | + String* init(const String* pString); | |
| 89 | + String* init(const char* pString, StringEncoding encoding); | |
| 90 | + String* init(void* pBytes, UInteger len, StringEncoding encoding, bool freeBuffer); | |
| 91 | + | |
| 92 | + unichar character(UInteger index) const; | |
| 93 | + UInteger length() const; | |
| 94 | + | |
| 95 | + const char* cString(StringEncoding encoding) const; | |
| 96 | + const char* utf8String() const; | |
| 97 | + UInteger maximumLengthOfBytes(StringEncoding encoding) const; | |
| 98 | + UInteger lengthOfBytes(StringEncoding encoding) const; | |
| 99 | + | |
| 100 | + bool isEqualToString(const String* pString) const; | |
| 101 | + Range rangeOfString(const String* pString, StringCompareOptions options) const; | |
| 102 | + | |
| 103 | + const char* fileSystemRepresentation() const; | |
| 104 | + | |
| 105 | + String* stringByAppendingString(const String* pString) const; | |
| 106 | + ComparisonResult caseInsensitiveCompare(const String* pString) const; | |
| 107 | +}; | |
| 108 | + | |
| 109 | +/// Create an NS::String* from a string literal. | |
| 110 | +#define MTLSTR(literal) (NS::String*)__builtin___CFStringMakeConstantString("" literal "") | |
| 111 | + | |
| 112 | +template <std::size_t _StringLen> | |
| 113 | +[[deprecated("please use MTLSTR(str)")]] constexpr const String* MakeConstantString(const char (&str)[_StringLen]) | |
| 114 | +{ | |
| 115 | + return reinterpret_cast<const String*>(__CFStringMakeConstantString(str)); | |
| 116 | +} | |
| 117 | + | |
| 118 | +} | |
| 119 | + | |
| 120 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 121 | + | |
| 122 | +_NS_INLINE NS::String* NS::String::string() | |
| 123 | +{ | |
| 124 | + return Object::sendMessage<String*>(_NS_PRIVATE_CLS(NSString), _NS_PRIVATE_SEL(string)); | |
| 125 | +} | |
| 126 | + | |
| 127 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 128 | + | |
| 129 | +_NS_INLINE NS::String* NS::String::string(const String* pString) | |
| 130 | +{ | |
| 131 | + return Object::sendMessage<String*>(_NS_PRIVATE_CLS(NSString), _NS_PRIVATE_SEL(stringWithString_), pString); | |
| 132 | +} | |
| 133 | + | |
| 134 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 135 | + | |
| 136 | +_NS_INLINE NS::String* NS::String::string(const char* pString, StringEncoding encoding) | |
| 137 | +{ | |
| 138 | + return Object::sendMessage<String*>(_NS_PRIVATE_CLS(NSString), _NS_PRIVATE_SEL(stringWithCString_encoding_), pString, encoding); | |
| 139 | +} | |
| 140 | + | |
| 141 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 142 | + | |
| 143 | +_NS_INLINE NS::String* NS::String::alloc() | |
| 144 | +{ | |
| 145 | + return Object::alloc<String>(_NS_PRIVATE_CLS(NSString)); | |
| 146 | +} | |
| 147 | + | |
| 148 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 149 | + | |
| 150 | +_NS_INLINE NS::String* NS::String::init() | |
| 151 | +{ | |
| 152 | + return Object::init<String>(); | |
| 153 | +} | |
| 154 | + | |
| 155 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 156 | + | |
| 157 | +_NS_INLINE NS::String* NS::String::init(const String* pString) | |
| 158 | +{ | |
| 159 | + return Object::sendMessage<String*>(this, _NS_PRIVATE_SEL(initWithString_), pString); | |
| 160 | +} | |
| 161 | + | |
| 162 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 163 | + | |
| 164 | +_NS_INLINE NS::String* NS::String::init(const char* pString, StringEncoding encoding) | |
| 165 | +{ | |
| 166 | + return Object::sendMessage<String*>(this, _NS_PRIVATE_SEL(initWithCString_encoding_), pString, encoding); | |
| 167 | +} | |
| 168 | + | |
| 169 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 170 | + | |
| 171 | +_NS_INLINE NS::String* NS::String::init(void* pBytes, UInteger len, StringEncoding encoding, bool freeBuffer) | |
| 172 | +{ | |
| 173 | + return Object::sendMessage<String*>(this, _NS_PRIVATE_SEL(initWithBytesNoCopy_length_encoding_freeWhenDone_), pBytes, len, encoding, freeBuffer); | |
| 174 | +} | |
| 175 | + | |
| 176 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 177 | + | |
| 178 | +_NS_INLINE NS::unichar NS::String::character(UInteger index) const | |
| 179 | +{ | |
| 180 | + return Object::sendMessage<unichar>(this, _NS_PRIVATE_SEL(characterAtIndex_), index); | |
| 181 | +} | |
| 182 | + | |
| 183 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 184 | + | |
| 185 | +_NS_INLINE NS::UInteger NS::String::length() const | |
| 186 | +{ | |
| 187 | + return Object::sendMessage<UInteger>(this, _NS_PRIVATE_SEL(length)); | |
| 188 | +} | |
| 189 | + | |
| 190 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 191 | + | |
| 192 | +_NS_INLINE const char* NS::String::cString(StringEncoding encoding) const | |
| 193 | +{ | |
| 194 | + return Object::sendMessage<const char*>(this, _NS_PRIVATE_SEL(cStringUsingEncoding_), encoding); | |
| 195 | +} | |
| 196 | + | |
| 197 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 198 | + | |
| 199 | +_NS_INLINE const char* NS::String::utf8String() const | |
| 200 | +{ | |
| 201 | + return Object::sendMessage<const char*>(this, _NS_PRIVATE_SEL(UTF8String)); | |
| 202 | +} | |
| 203 | + | |
| 204 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 205 | + | |
| 206 | +_NS_INLINE NS::UInteger NS::String::maximumLengthOfBytes(StringEncoding encoding) const | |
| 207 | +{ | |
| 208 | + return Object::sendMessage<UInteger>(this, _NS_PRIVATE_SEL(maximumLengthOfBytesUsingEncoding_), encoding); | |
| 209 | +} | |
| 210 | + | |
| 211 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 212 | + | |
| 213 | +_NS_INLINE NS::UInteger NS::String::lengthOfBytes(StringEncoding encoding) const | |
| 214 | +{ | |
| 215 | + return Object::sendMessage<UInteger>(this, _NS_PRIVATE_SEL(lengthOfBytesUsingEncoding_), encoding); | |
| 216 | +} | |
| 217 | + | |
| 218 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 219 | + | |
| 220 | +_NS_INLINE bool NS::String::isEqualToString(const NS::String* pString) const | |
| 221 | +{ | |
| 222 | + return Object::sendMessage<bool>(this, _NS_PRIVATE_SEL(isEqualToString_), pString); | |
| 223 | +} | |
| 224 | + | |
| 225 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 226 | + | |
| 227 | +_NS_INLINE NS::Range NS::String::rangeOfString(const NS::String* pString, NS::StringCompareOptions options) const | |
| 228 | +{ | |
| 229 | + return Object::sendMessage<Range>(this, _NS_PRIVATE_SEL(rangeOfString_options_), pString, options); | |
| 230 | +} | |
| 231 | + | |
| 232 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 233 | + | |
| 234 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 235 | + | |
| 236 | +_NS_INLINE const char* NS::String::fileSystemRepresentation() const | |
| 237 | +{ | |
| 238 | + return Object::sendMessage<const char*>(this, _NS_PRIVATE_SEL(fileSystemRepresentation)); | |
| 239 | +} | |
| 240 | + | |
| 241 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 242 | + | |
| 243 | +_NS_INLINE NS::String* NS::String::stringByAppendingString(const String* pString) const | |
| 244 | +{ | |
| 245 | + return Object::sendMessage<NS::String*>(this, _NS_PRIVATE_SEL(stringByAppendingString_), pString); | |
| 246 | +} | |
| 247 | + | |
| 248 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 249 | + | |
| 250 | +_NS_INLINE NS::ComparisonResult NS::String::caseInsensitiveCompare(const String* pString) const | |
| 251 | +{ | |
| 252 | + return Object::sendMessage<NS::ComparisonResult>(this, _NS_PRIVATE_SEL(caseInsensitiveCompare_), pString); | |
| 253 | +} | |
| 254 | + | |
| 255 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
added
third_party/metal-cpp/Foundation/NSTypes.hpp
+51 −0
@@ -0,0 +1,51 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Foundation/NSTypes.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2024 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 24 | + | |
| 25 | +#include "NSDefines.hpp" | |
| 26 | + | |
| 27 | +#include <CoreFoundation/CoreFoundation.h> | |
| 28 | +#include <cstdint> | |
| 29 | + | |
| 30 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 31 | + | |
| 32 | +namespace NS | |
| 33 | +{ | |
| 34 | +using TimeInterval = double; | |
| 35 | + | |
| 36 | +using Integer = std::intptr_t; | |
| 37 | +using UInteger = std::uintptr_t; | |
| 38 | + | |
| 39 | +const Integer IntegerMax = INTPTR_MAX; | |
| 40 | +const Integer IntegerMin = INTPTR_MIN; | |
| 41 | +const UInteger UIntegerMax = UINTPTR_MAX; | |
| 42 | + | |
| 43 | +struct OperatingSystemVersion | |
| 44 | +{ | |
| 45 | + Integer majorVersion; | |
| 46 | + Integer minorVersion; | |
| 47 | + Integer patchVersion; | |
| 48 | +} _NS_PACKED; | |
| 49 | +} | |
| 50 | + | |
| 51 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
added
third_party/metal-cpp/Foundation/NSURL.hpp
+90 −0
@@ -0,0 +1,90 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Foundation/NSURL.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2024 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 24 | + | |
| 25 | +#include "NSDefines.hpp" | |
| 26 | +#include "NSObject.hpp" | |
| 27 | +#include "NSPrivate.hpp" | |
| 28 | +#include "NSTypes.hpp" | |
| 29 | + | |
| 30 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 31 | + | |
| 32 | +namespace NS | |
| 33 | +{ | |
| 34 | +class URL : public Copying<URL> | |
| 35 | +{ | |
| 36 | +public: | |
| 37 | + static URL* fileURLWithPath(const class String* pPath); | |
| 38 | + | |
| 39 | + static URL* alloc(); | |
| 40 | + URL* init(); | |
| 41 | + URL* init(const class String* pString); | |
| 42 | + URL* initFileURLWithPath(const class String* pPath); | |
| 43 | + | |
| 44 | + const char* fileSystemRepresentation() const; | |
| 45 | +}; | |
| 46 | +} | |
| 47 | + | |
| 48 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 49 | + | |
| 50 | +_NS_INLINE NS::URL* NS::URL::fileURLWithPath(const String* pPath) | |
| 51 | +{ | |
| 52 | + return Object::sendMessage<URL*>(_NS_PRIVATE_CLS(NSURL), _NS_PRIVATE_SEL(fileURLWithPath_), pPath); | |
| 53 | +} | |
| 54 | + | |
| 55 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 56 | + | |
| 57 | +_NS_INLINE NS::URL* NS::URL::alloc() | |
| 58 | +{ | |
| 59 | + return Object::alloc<URL>(_NS_PRIVATE_CLS(NSURL)); | |
| 60 | +} | |
| 61 | + | |
| 62 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 63 | + | |
| 64 | +_NS_INLINE NS::URL* NS::URL::init() | |
| 65 | +{ | |
| 66 | + return Object::init<URL>(); | |
| 67 | +} | |
| 68 | + | |
| 69 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 70 | + | |
| 71 | +_NS_INLINE NS::URL* NS::URL::init(const String* pString) | |
| 72 | +{ | |
| 73 | + return Object::sendMessage<URL*>(this, _NS_PRIVATE_SEL(initWithString_), pString); | |
| 74 | +} | |
| 75 | + | |
| 76 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 77 | + | |
| 78 | +_NS_INLINE NS::URL* NS::URL::initFileURLWithPath(const String* pPath) | |
| 79 | +{ | |
| 80 | + return Object::sendMessage<URL*>(this, _NS_PRIVATE_SEL(initFileURLWithPath_), pPath); | |
| 81 | +} | |
| 82 | + | |
| 83 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 84 | + | |
| 85 | +_NS_INLINE const char* NS::URL::fileSystemRepresentation() const | |
| 86 | +{ | |
| 87 | + return Object::sendMessage<const char*>(this, _NS_PRIVATE_SEL(fileSystemRepresentation)); | |
| 88 | +} | |
| 89 | + | |
| 90 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
added
third_party/metal-cpp/LICENSE.txt
+202 −0
@@ -0,0 +1,202 @@ | ||
| 1 | + | |
| 2 | + Apache License | |
| 3 | + Version 2.0, January 2004 | |
| 4 | + http://www.apache.org/licenses/ | |
| 5 | + | |
| 6 | + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION | |
| 7 | + | |
| 8 | + 1. Definitions. | |
| 9 | + | |
| 10 | + "License" shall mean the terms and conditions for use, reproduction, | |
| 11 | + and distribution as defined by Sections 1 through 9 of this document. | |
| 12 | + | |
| 13 | + "Licensor" shall mean the copyright owner or entity authorized by | |
| 14 | + the copyright owner that is granting the License. | |
| 15 | + | |
| 16 | + "Legal Entity" shall mean the union of the acting entity and all | |
| 17 | + other entities that control, are controlled by, or are under common | |
| 18 | + control with that entity. For the purposes of this definition, | |
| 19 | + "control" means (i) the power, direct or indirect, to cause the | |
| 20 | + direction or management of such entity, whether by contract or | |
| 21 | + otherwise, or (ii) ownership of fifty percent (50%) or more of the | |
| 22 | + outstanding shares, or (iii) beneficial ownership of such entity. | |
| 23 | + | |
| 24 | + "You" (or "Your") shall mean an individual or Legal Entity | |
| 25 | + exercising permissions granted by this License. | |
| 26 | + | |
| 27 | + "Source" form shall mean the preferred form for making modifications, | |
| 28 | + including but not limited to software source code, documentation | |
| 29 | + source, and configuration files. | |
| 30 | + | |
| 31 | + "Object" form shall mean any form resulting from mechanical | |
| 32 | + transformation or translation of a Source form, including but | |
| 33 | + not limited to compiled object code, generated documentation, | |
| 34 | + and conversions to other media types. | |
| 35 | + | |
| 36 | + "Work" shall mean the work of authorship, whether in Source or | |
| 37 | + Object form, made available under the License, as indicated by a | |
| 38 | + copyright notice that is included in or attached to the work | |
| 39 | + (an example is provided in the Appendix below). | |
| 40 | + | |
| 41 | + "Derivative Works" shall mean any work, whether in Source or Object | |
| 42 | + form, that is based on (or derived from) the Work and for which the | |
| 43 | + editorial revisions, annotations, elaborations, or other modifications | |
| 44 | + represent, as a whole, an original work of authorship. For the purposes | |
| 45 | + of this License, Derivative Works shall not include works that remain | |
| 46 | + separable from, or merely link (or bind by name) to the interfaces of, | |
| 47 | + the Work and Derivative Works thereof. | |
| 48 | + | |
| 49 | + "Contribution" shall mean any work of authorship, including | |
| 50 | + the original version of the Work and any modifications or additions | |
| 51 | + to that Work or Derivative Works thereof, that is intentionally | |
| 52 | + submitted to Licensor for inclusion in the Work by the copyright owner | |
| 53 | + or by an individual or Legal Entity authorized to submit on behalf of | |
| 54 | + the copyright owner. For the purposes of this definition, "submitted" | |
| 55 | + means any form of electronic, verbal, or written communication sent | |
| 56 | + to the Licensor or its representatives, including but not limited to | |
| 57 | + communication on electronic mailing lists, source code control systems, | |
| 58 | + and issue tracking systems that are managed by, or on behalf of, the | |
| 59 | + Licensor for the purpose of discussing and improving the Work, but | |
| 60 | + excluding communication that is conspicuously marked or otherwise | |
| 61 | + designated in writing by the copyright owner as "Not a Contribution." | |
| 62 | + | |
| 63 | + "Contributor" shall mean Licensor and any individual or Legal Entity | |
| 64 | + on behalf of whom a Contribution has been received by Licensor and | |
| 65 | + subsequently incorporated within the Work. | |
| 66 | + | |
| 67 | + 2. Grant of Copyright License. Subject to the terms and conditions of | |
| 68 | + this License, each Contributor hereby grants to You a perpetual, | |
| 69 | + worldwide, non-exclusive, no-charge, royalty-free, irrevocable | |
| 70 | + copyright license to reproduce, prepare Derivative Works of, | |
| 71 | + publicly display, publicly perform, sublicense, and distribute the | |
| 72 | + Work and such Derivative Works in Source or Object form. | |
| 73 | + | |
| 74 | + 3. Grant of Patent License. Subject to the terms and conditions of | |
| 75 | + this License, each Contributor hereby grants to You a perpetual, | |
| 76 | + worldwide, non-exclusive, no-charge, royalty-free, irrevocable | |
| 77 | + (except as stated in this section) patent license to make, have made, | |
| 78 | + use, offer to sell, sell, import, and otherwise transfer the Work, | |
| 79 | + where such license applies only to those patent claims licensable | |
| 80 | + by such Contributor that are necessarily infringed by their | |
| 81 | + Contribution(s) alone or by combination of their Contribution(s) | |
| 82 | + with the Work to which such Contribution(s) was submitted. If You | |
| 83 | + institute patent litigation against any entity (including a | |
| 84 | + cross-claim or counterclaim in a lawsuit) alleging that the Work | |
| 85 | + or a Contribution incorporated within the Work constitutes direct | |
| 86 | + or contributory patent infringement, then any patent licenses | |
| 87 | + granted to You under this License for that Work shall terminate | |
| 88 | + as of the date such litigation is filed. | |
| 89 | + | |
| 90 | + 4. Redistribution. You may reproduce and distribute copies of the | |
| 91 | + Work or Derivative Works thereof in any medium, with or without | |
| 92 | + modifications, and in Source or Object form, provided that You | |
| 93 | + meet the following conditions: | |
| 94 | + | |
| 95 | + (a) You must give any other recipients of the Work or | |
| 96 | + Derivative Works a copy of this License; and | |
| 97 | + | |
| 98 | + (b) You must cause any modified files to carry prominent notices | |
| 99 | + stating that You changed the files; and | |
| 100 | + | |
| 101 | + (c) You must retain, in the Source form of any Derivative Works | |
| 102 | + that You distribute, all copyright, patent, trademark, and | |
| 103 | + attribution notices from the Source form of the Work, | |
| 104 | + excluding those notices that do not pertain to any part of | |
| 105 | + the Derivative Works; and | |
| 106 | + | |
| 107 | + (d) If the Work includes a "NOTICE" text file as part of its | |
| 108 | + distribution, then any Derivative Works that You distribute must | |
| 109 | + include a readable copy of the attribution notices contained | |
| 110 | + within such NOTICE file, excluding those notices that do not | |
| 111 | + pertain to any part of the Derivative Works, in at least one | |
| 112 | + of the following places: within a NOTICE text file distributed | |
| 113 | + as part of the Derivative Works; within the Source form or | |
| 114 | + documentation, if provided along with the Derivative Works; or, | |
| 115 | + within a display generated by the Derivative Works, if and | |
| 116 | + wherever such third-party notices normally appear. The contents | |
| 117 | + of the NOTICE file are for informational purposes only and | |
| 118 | + do not modify the License. You may add Your own attribution | |
| 119 | + notices within Derivative Works that You distribute, alongside | |
| 120 | + or as an addendum to the NOTICE text from the Work, provided | |
| 121 | + that such additional attribution notices cannot be construed | |
| 122 | + as modifying the License. | |
| 123 | + | |
| 124 | + You may add Your own copyright statement to Your modifications and | |
| 125 | + may provide additional or different license terms and conditions | |
| 126 | + for use, reproduction, or distribution of Your modifications, or | |
| 127 | + for any such Derivative Works as a whole, provided Your use, | |
| 128 | + reproduction, and distribution of the Work otherwise complies with | |
| 129 | + the conditions stated in this License. | |
| 130 | + | |
| 131 | + 5. Submission of Contributions. Unless You explicitly state otherwise, | |
| 132 | + any Contribution intentionally submitted for inclusion in the Work | |
| 133 | + by You to the Licensor shall be under the terms and conditions of | |
| 134 | + this License, without any additional terms or conditions. | |
| 135 | + Notwithstanding the above, nothing herein shall supersede or modify | |
| 136 | + the terms of any separate license agreement you may have executed | |
| 137 | + with Licensor regarding such Contributions. | |
| 138 | + | |
| 139 | + 6. Trademarks. This License does not grant permission to use the trade | |
| 140 | + names, trademarks, service marks, or product names of the Licensor, | |
| 141 | + except as required for reasonable and customary use in describing the | |
| 142 | + origin of the Work and reproducing the content of the NOTICE file. | |
| 143 | + | |
| 144 | + 7. Disclaimer of Warranty. Unless required by applicable law or | |
| 145 | + agreed to in writing, Licensor provides the Work (and each | |
| 146 | + Contributor provides its Contributions) on an "AS IS" BASIS, | |
| 147 | + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or | |
| 148 | + implied, including, without limitation, any warranties or conditions | |
| 149 | + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A | |
| 150 | + PARTICULAR PURPOSE. You are solely responsible for determining the | |
| 151 | + appropriateness of using or redistributing the Work and assume any | |
| 152 | + risks associated with Your exercise of permissions under this License. | |
| 153 | + | |
| 154 | + 8. Limitation of Liability. In no event and under no legal theory, | |
| 155 | + whether in tort (including negligence), contract, or otherwise, | |
| 156 | + unless required by applicable law (such as deliberate and grossly | |
| 157 | + negligent acts) or agreed to in writing, shall any Contributor be | |
| 158 | + liable to You for damages, including any direct, indirect, special, | |
| 159 | + incidental, or consequential damages of any character arising as a | |
| 160 | + result of this License or out of the use or inability to use the | |
| 161 | + Work (including but not limited to damages for loss of goodwill, | |
| 162 | + work stoppage, computer failure or malfunction, or any and all | |
| 163 | + other commercial damages or losses), even if such Contributor | |
| 164 | + has been advised of the possibility of such damages. | |
| 165 | + | |
| 166 | + 9. Accepting Warranty or Additional Liability. While redistributing | |
| 167 | + the Work or Derivative Works thereof, You may choose to offer, | |
| 168 | + and charge a fee for, acceptance of support, warranty, indemnity, | |
| 169 | + or other liability obligations and/or rights consistent with this | |
| 170 | + License. However, in accepting such obligations, You may act only | |
| 171 | + on Your own behalf and on Your sole responsibility, not on behalf | |
| 172 | + of any other Contributor, and only if You agree to indemnify, | |
| 173 | + defend, and hold each Contributor harmless for any liability | |
| 174 | + incurred by, or claims asserted against, such Contributor by reason | |
| 175 | + of your accepting any such warranty or additional liability. | |
| 176 | + | |
| 177 | + END OF TERMS AND CONDITIONS | |
| 178 | + | |
| 179 | + APPENDIX: How to apply the Apache License to your work. | |
| 180 | + | |
| 181 | + To apply the Apache License to your work, attach the following | |
| 182 | + boilerplate notice, with the fields enclosed by brackets "[]" | |
| 183 | + replaced with your own identifying information. (Don't include | |
| 184 | + the brackets!) The text should be enclosed in the appropriate | |
| 185 | + comment syntax for the file format. We also recommend that a | |
| 186 | + file or class name and description of purpose be included on the | |
| 187 | + same "printed page" as the copyright notice for easier | |
| 188 | + identification within third-party archives. | |
| 189 | + | |
| 190 | + Copyright © 2024 Apple Inc. | |
| 191 | + | |
| 192 | + Licensed under the Apache License, Version 2.0 (the "License"); | |
| 193 | + you may not use this file except in compliance with the License. | |
| 194 | + You may obtain a copy of the License at | |
| 195 | + | |
| 196 | + http://www.apache.org/licenses/LICENSE-2.0 | |
| 197 | + | |
| 198 | + Unless required by applicable law or agreed to in writing, software | |
| 199 | + distributed under the License is distributed on an "AS IS" BASIS, | |
| 200 | + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 201 | + See the License for the specific language governing permissions and | |
| 202 | + limitations under the License. | |
added
third_party/metal-cpp/Metal/MTL4AccelerationStructure.hpp
+1395 −0
@@ -0,0 +1,1395 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Metal/MTL4AccelerationStructure.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2025 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include "../Foundation/Foundation.hpp" | |
| 24 | +#include "MTLAccelerationStructure.hpp" | |
| 25 | +#include "MTLAccelerationStructureTypes.hpp" | |
| 26 | +#include "MTLArgument.hpp" | |
| 27 | +#include "MTLDefines.hpp" | |
| 28 | +#include "MTLHeaderBridge.hpp" | |
| 29 | +#include "MTLPrivate.hpp" | |
| 30 | +#include "MTLStageInputOutputDescriptor.hpp" | |
| 31 | + | |
| 32 | +namespace MTL4 | |
| 33 | +{ | |
| 34 | +class AccelerationStructureBoundingBoxGeometryDescriptor; | |
| 35 | +class AccelerationStructureCurveGeometryDescriptor; | |
| 36 | +class AccelerationStructureDescriptor; | |
| 37 | +class AccelerationStructureGeometryDescriptor; | |
| 38 | +class AccelerationStructureMotionBoundingBoxGeometryDescriptor; | |
| 39 | +class AccelerationStructureMotionCurveGeometryDescriptor; | |
| 40 | +class AccelerationStructureMotionTriangleGeometryDescriptor; | |
| 41 | +class AccelerationStructureTriangleGeometryDescriptor; | |
| 42 | +class IndirectInstanceAccelerationStructureDescriptor; | |
| 43 | +class InstanceAccelerationStructureDescriptor; | |
| 44 | +class PrimitiveAccelerationStructureDescriptor; | |
| 45 | + | |
| 46 | +class AccelerationStructureDescriptor : public NS::Copying<AccelerationStructureDescriptor, MTL::AccelerationStructureDescriptor> | |
| 47 | +{ | |
| 48 | +public: | |
| 49 | + static AccelerationStructureDescriptor* alloc(); | |
| 50 | + | |
| 51 | + AccelerationStructureDescriptor* init(); | |
| 52 | +}; | |
| 53 | +class AccelerationStructureGeometryDescriptor : public NS::Copying<AccelerationStructureGeometryDescriptor> | |
| 54 | +{ | |
| 55 | +public: | |
| 56 | + static AccelerationStructureGeometryDescriptor* alloc(); | |
| 57 | + | |
| 58 | + bool allowDuplicateIntersectionFunctionInvocation() const; | |
| 59 | + | |
| 60 | + AccelerationStructureGeometryDescriptor* init(); | |
| 61 | + | |
| 62 | + NS::UInteger intersectionFunctionTableOffset() const; | |
| 63 | + | |
| 64 | + NS::String* label() const; | |
| 65 | + | |
| 66 | + bool opaque() const; | |
| 67 | + | |
| 68 | + BufferRange primitiveDataBuffer() const; | |
| 69 | + | |
| 70 | + NS::UInteger primitiveDataElementSize() const; | |
| 71 | + | |
| 72 | + NS::UInteger primitiveDataStride() const; | |
| 73 | + | |
| 74 | + void setAllowDuplicateIntersectionFunctionInvocation(bool allowDuplicateIntersectionFunctionInvocation); | |
| 75 | + | |
| 76 | + void setIntersectionFunctionTableOffset(NS::UInteger intersectionFunctionTableOffset); | |
| 77 | + | |
| 78 | + void setLabel(const NS::String* label); | |
| 79 | + | |
| 80 | + void setOpaque(bool opaque); | |
| 81 | + | |
| 82 | + void setPrimitiveDataBuffer(const MTL4::BufferRange primitiveDataBuffer); | |
| 83 | + | |
| 84 | + void setPrimitiveDataElementSize(NS::UInteger primitiveDataElementSize); | |
| 85 | + | |
| 86 | + void setPrimitiveDataStride(NS::UInteger primitiveDataStride); | |
| 87 | +}; | |
| 88 | +class PrimitiveAccelerationStructureDescriptor : public NS::Copying<PrimitiveAccelerationStructureDescriptor, AccelerationStructureDescriptor> | |
| 89 | +{ | |
| 90 | +public: | |
| 91 | + static PrimitiveAccelerationStructureDescriptor* alloc(); | |
| 92 | + | |
| 93 | + NS::Array* geometryDescriptors() const; | |
| 94 | + | |
| 95 | + PrimitiveAccelerationStructureDescriptor* init(); | |
| 96 | + | |
| 97 | + MTL::MotionBorderMode motionEndBorderMode() const; | |
| 98 | + | |
| 99 | + float motionEndTime() const; | |
| 100 | + | |
| 101 | + NS::UInteger motionKeyframeCount() const; | |
| 102 | + | |
| 103 | + MTL::MotionBorderMode motionStartBorderMode() const; | |
| 104 | + | |
| 105 | + float motionStartTime() const; | |
| 106 | + | |
| 107 | + void setGeometryDescriptors(const NS::Array* geometryDescriptors); | |
| 108 | + | |
| 109 | + void setMotionEndBorderMode(MTL::MotionBorderMode motionEndBorderMode); | |
| 110 | + | |
| 111 | + void setMotionEndTime(float motionEndTime); | |
| 112 | + | |
| 113 | + void setMotionKeyframeCount(NS::UInteger motionKeyframeCount); | |
| 114 | + | |
| 115 | + void setMotionStartBorderMode(MTL::MotionBorderMode motionStartBorderMode); | |
| 116 | + | |
| 117 | + void setMotionStartTime(float motionStartTime); | |
| 118 | +}; | |
| 119 | +class AccelerationStructureTriangleGeometryDescriptor : public NS::Copying<AccelerationStructureTriangleGeometryDescriptor, AccelerationStructureGeometryDescriptor> | |
| 120 | +{ | |
| 121 | +public: | |
| 122 | + static AccelerationStructureTriangleGeometryDescriptor* alloc(); | |
| 123 | + | |
| 124 | + BufferRange indexBuffer() const; | |
| 125 | + | |
| 126 | + MTL::IndexType indexType() const; | |
| 127 | + | |
| 128 | + AccelerationStructureTriangleGeometryDescriptor* init(); | |
| 129 | + | |
| 130 | + void setIndexBuffer(const MTL4::BufferRange indexBuffer); | |
| 131 | + | |
| 132 | + void setIndexType(MTL::IndexType indexType); | |
| 133 | + | |
| 134 | + void setTransformationMatrixBuffer(const MTL4::BufferRange transformationMatrixBuffer); | |
| 135 | + | |
| 136 | + void setTransformationMatrixLayout(MTL::MatrixLayout transformationMatrixLayout); | |
| 137 | + | |
| 138 | + void setTriangleCount(NS::UInteger triangleCount); | |
| 139 | + | |
| 140 | + void setVertexBuffer(const MTL4::BufferRange vertexBuffer); | |
| 141 | + | |
| 142 | + void setVertexFormat(MTL::AttributeFormat vertexFormat); | |
| 143 | + | |
| 144 | + void setVertexStride(NS::UInteger vertexStride); | |
| 145 | + | |
| 146 | + BufferRange transformationMatrixBuffer() const; | |
| 147 | + | |
| 148 | + MTL::MatrixLayout transformationMatrixLayout() const; | |
| 149 | + | |
| 150 | + NS::UInteger triangleCount() const; | |
| 151 | + | |
| 152 | + BufferRange vertexBuffer() const; | |
| 153 | + | |
| 154 | + MTL::AttributeFormat vertexFormat() const; | |
| 155 | + | |
| 156 | + NS::UInteger vertexStride() const; | |
| 157 | +}; | |
| 158 | +class AccelerationStructureBoundingBoxGeometryDescriptor : public NS::Copying<AccelerationStructureBoundingBoxGeometryDescriptor, AccelerationStructureGeometryDescriptor> | |
| 159 | +{ | |
| 160 | +public: | |
| 161 | + static AccelerationStructureBoundingBoxGeometryDescriptor* alloc(); | |
| 162 | + | |
| 163 | + BufferRange boundingBoxBuffer() const; | |
| 164 | + | |
| 165 | + NS::UInteger boundingBoxCount() const; | |
| 166 | + | |
| 167 | + NS::UInteger boundingBoxStride() const; | |
| 168 | + | |
| 169 | + AccelerationStructureBoundingBoxGeometryDescriptor* init(); | |
| 170 | + | |
| 171 | + void setBoundingBoxBuffer(const MTL4::BufferRange boundingBoxBuffer); | |
| 172 | + | |
| 173 | + void setBoundingBoxCount(NS::UInteger boundingBoxCount); | |
| 174 | + | |
| 175 | + void setBoundingBoxStride(NS::UInteger boundingBoxStride); | |
| 176 | +}; | |
| 177 | +class AccelerationStructureMotionTriangleGeometryDescriptor : public NS::Copying<AccelerationStructureMotionTriangleGeometryDescriptor, AccelerationStructureGeometryDescriptor> | |
| 178 | +{ | |
| 179 | +public: | |
| 180 | + static AccelerationStructureMotionTriangleGeometryDescriptor* alloc(); | |
| 181 | + | |
| 182 | + BufferRange indexBuffer() const; | |
| 183 | + | |
| 184 | + MTL::IndexType indexType() const; | |
| 185 | + | |
| 186 | + AccelerationStructureMotionTriangleGeometryDescriptor* init(); | |
| 187 | + | |
| 188 | + void setIndexBuffer(const MTL4::BufferRange indexBuffer); | |
| 189 | + | |
| 190 | + void setIndexType(MTL::IndexType indexType); | |
| 191 | + | |
| 192 | + void setTransformationMatrixBuffer(const MTL4::BufferRange transformationMatrixBuffer); | |
| 193 | + | |
| 194 | + void setTransformationMatrixLayout(MTL::MatrixLayout transformationMatrixLayout); | |
| 195 | + | |
| 196 | + void setTriangleCount(NS::UInteger triangleCount); | |
| 197 | + | |
| 198 | + void setVertexBuffers(const MTL4::BufferRange vertexBuffers); | |
| 199 | + | |
| 200 | + void setVertexFormat(MTL::AttributeFormat vertexFormat); | |
| 201 | + | |
| 202 | + void setVertexStride(NS::UInteger vertexStride); | |
| 203 | + | |
| 204 | + BufferRange transformationMatrixBuffer() const; | |
| 205 | + | |
| 206 | + MTL::MatrixLayout transformationMatrixLayout() const; | |
| 207 | + | |
| 208 | + NS::UInteger triangleCount() const; | |
| 209 | + | |
| 210 | + BufferRange vertexBuffers() const; | |
| 211 | + | |
| 212 | + MTL::AttributeFormat vertexFormat() const; | |
| 213 | + | |
| 214 | + NS::UInteger vertexStride() const; | |
| 215 | +}; | |
| 216 | +class AccelerationStructureMotionBoundingBoxGeometryDescriptor : public NS::Copying<AccelerationStructureMotionBoundingBoxGeometryDescriptor, AccelerationStructureGeometryDescriptor> | |
| 217 | +{ | |
| 218 | +public: | |
| 219 | + static AccelerationStructureMotionBoundingBoxGeometryDescriptor* alloc(); | |
| 220 | + | |
| 221 | + BufferRange boundingBoxBuffers() const; | |
| 222 | + | |
| 223 | + NS::UInteger boundingBoxCount() const; | |
| 224 | + | |
| 225 | + NS::UInteger boundingBoxStride() const; | |
| 226 | + | |
| 227 | + AccelerationStructureMotionBoundingBoxGeometryDescriptor* init(); | |
| 228 | + | |
| 229 | + void setBoundingBoxBuffers(const MTL4::BufferRange boundingBoxBuffers); | |
| 230 | + | |
| 231 | + void setBoundingBoxCount(NS::UInteger boundingBoxCount); | |
| 232 | + | |
| 233 | + void setBoundingBoxStride(NS::UInteger boundingBoxStride); | |
| 234 | +}; | |
| 235 | +class AccelerationStructureCurveGeometryDescriptor : public NS::Copying<AccelerationStructureCurveGeometryDescriptor, AccelerationStructureGeometryDescriptor> | |
| 236 | +{ | |
| 237 | +public: | |
| 238 | + static AccelerationStructureCurveGeometryDescriptor* alloc(); | |
| 239 | + | |
| 240 | + BufferRange controlPointBuffer() const; | |
| 241 | + | |
| 242 | + NS::UInteger controlPointCount() const; | |
| 243 | + | |
| 244 | + MTL::AttributeFormat controlPointFormat() const; | |
| 245 | + | |
| 246 | + NS::UInteger controlPointStride() const; | |
| 247 | + | |
| 248 | + MTL::CurveBasis curveBasis() const; | |
| 249 | + | |
| 250 | + MTL::CurveEndCaps curveEndCaps() const; | |
| 251 | + | |
| 252 | + MTL::CurveType curveType() const; | |
| 253 | + | |
| 254 | + BufferRange indexBuffer() const; | |
| 255 | + | |
| 256 | + MTL::IndexType indexType() const; | |
| 257 | + | |
| 258 | + AccelerationStructureCurveGeometryDescriptor* init(); | |
| 259 | + | |
| 260 | + BufferRange radiusBuffer() const; | |
| 261 | + | |
| 262 | + MTL::AttributeFormat radiusFormat() const; | |
| 263 | + | |
| 264 | + NS::UInteger radiusStride() const; | |
| 265 | + | |
| 266 | + NS::UInteger segmentControlPointCount() const; | |
| 267 | + | |
| 268 | + NS::UInteger segmentCount() const; | |
| 269 | + | |
| 270 | + void setControlPointBuffer(const MTL4::BufferRange controlPointBuffer); | |
| 271 | + | |
| 272 | + void setControlPointCount(NS::UInteger controlPointCount); | |
| 273 | + | |
| 274 | + void setControlPointFormat(MTL::AttributeFormat controlPointFormat); | |
| 275 | + | |
| 276 | + void setControlPointStride(NS::UInteger controlPointStride); | |
| 277 | + | |
| 278 | + void setCurveBasis(MTL::CurveBasis curveBasis); | |
| 279 | + | |
| 280 | + void setCurveEndCaps(MTL::CurveEndCaps curveEndCaps); | |
| 281 | + | |
| 282 | + void setCurveType(MTL::CurveType curveType); | |
| 283 | + | |
| 284 | + void setIndexBuffer(const MTL4::BufferRange indexBuffer); | |
| 285 | + | |
| 286 | + void setIndexType(MTL::IndexType indexType); | |
| 287 | + | |
| 288 | + void setRadiusBuffer(const MTL4::BufferRange radiusBuffer); | |
| 289 | + | |
| 290 | + void setRadiusFormat(MTL::AttributeFormat radiusFormat); | |
| 291 | + | |
| 292 | + void setRadiusStride(NS::UInteger radiusStride); | |
| 293 | + | |
| 294 | + void setSegmentControlPointCount(NS::UInteger segmentControlPointCount); | |
| 295 | + | |
| 296 | + void setSegmentCount(NS::UInteger segmentCount); | |
| 297 | +}; | |
| 298 | +class AccelerationStructureMotionCurveGeometryDescriptor : public NS::Copying<AccelerationStructureMotionCurveGeometryDescriptor, AccelerationStructureGeometryDescriptor> | |
| 299 | +{ | |
| 300 | +public: | |
| 301 | + static AccelerationStructureMotionCurveGeometryDescriptor* alloc(); | |
| 302 | + | |
| 303 | + BufferRange controlPointBuffers() const; | |
| 304 | + | |
| 305 | + NS::UInteger controlPointCount() const; | |
| 306 | + | |
| 307 | + MTL::AttributeFormat controlPointFormat() const; | |
| 308 | + | |
| 309 | + NS::UInteger controlPointStride() const; | |
| 310 | + | |
| 311 | + MTL::CurveBasis curveBasis() const; | |
| 312 | + | |
| 313 | + MTL::CurveEndCaps curveEndCaps() const; | |
| 314 | + | |
| 315 | + MTL::CurveType curveType() const; | |
| 316 | + | |
| 317 | + BufferRange indexBuffer() const; | |
| 318 | + | |
| 319 | + MTL::IndexType indexType() const; | |
| 320 | + | |
| 321 | + AccelerationStructureMotionCurveGeometryDescriptor* init(); | |
| 322 | + | |
| 323 | + BufferRange radiusBuffers() const; | |
| 324 | + | |
| 325 | + MTL::AttributeFormat radiusFormat() const; | |
| 326 | + | |
| 327 | + NS::UInteger radiusStride() const; | |
| 328 | + | |
| 329 | + NS::UInteger segmentControlPointCount() const; | |
| 330 | + | |
| 331 | + NS::UInteger segmentCount() const; | |
| 332 | + | |
| 333 | + void setControlPointBuffers(const MTL4::BufferRange controlPointBuffers); | |
| 334 | + | |
| 335 | + void setControlPointCount(NS::UInteger controlPointCount); | |
| 336 | + | |
| 337 | + void setControlPointFormat(MTL::AttributeFormat controlPointFormat); | |
| 338 | + | |
| 339 | + void setControlPointStride(NS::UInteger controlPointStride); | |
| 340 | + | |
| 341 | + void setCurveBasis(MTL::CurveBasis curveBasis); | |
| 342 | + | |
| 343 | + void setCurveEndCaps(MTL::CurveEndCaps curveEndCaps); | |
| 344 | + | |
| 345 | + void setCurveType(MTL::CurveType curveType); | |
| 346 | + | |
| 347 | + void setIndexBuffer(const MTL4::BufferRange indexBuffer); | |
| 348 | + | |
| 349 | + void setIndexType(MTL::IndexType indexType); | |
| 350 | + | |
| 351 | + void setRadiusBuffers(const MTL4::BufferRange radiusBuffers); | |
| 352 | + | |
| 353 | + void setRadiusFormat(MTL::AttributeFormat radiusFormat); | |
| 354 | + | |
| 355 | + void setRadiusStride(NS::UInteger radiusStride); | |
| 356 | + | |
| 357 | + void setSegmentControlPointCount(NS::UInteger segmentControlPointCount); | |
| 358 | + | |
| 359 | + void setSegmentCount(NS::UInteger segmentCount); | |
| 360 | +}; | |
| 361 | +class InstanceAccelerationStructureDescriptor : public NS::Copying<InstanceAccelerationStructureDescriptor, AccelerationStructureDescriptor> | |
| 362 | +{ | |
| 363 | +public: | |
| 364 | + static InstanceAccelerationStructureDescriptor* alloc(); | |
| 365 | + | |
| 366 | + InstanceAccelerationStructureDescriptor* init(); | |
| 367 | + | |
| 368 | + NS::UInteger instanceCount() const; | |
| 369 | + | |
| 370 | + BufferRange instanceDescriptorBuffer() const; | |
| 371 | + | |
| 372 | + NS::UInteger instanceDescriptorStride() const; | |
| 373 | + | |
| 374 | + MTL::AccelerationStructureInstanceDescriptorType instanceDescriptorType() const; | |
| 375 | + | |
| 376 | + MTL::MatrixLayout instanceTransformationMatrixLayout() const; | |
| 377 | + | |
| 378 | + BufferRange motionTransformBuffer() const; | |
| 379 | + | |
| 380 | + NS::UInteger motionTransformCount() const; | |
| 381 | + | |
| 382 | + NS::UInteger motionTransformStride() const; | |
| 383 | + | |
| 384 | + MTL::TransformType motionTransformType() const; | |
| 385 | + | |
| 386 | + void setInstanceCount(NS::UInteger instanceCount); | |
| 387 | + | |
| 388 | + void setInstanceDescriptorBuffer(const MTL4::BufferRange instanceDescriptorBuffer); | |
| 389 | + | |
| 390 | + void setInstanceDescriptorStride(NS::UInteger instanceDescriptorStride); | |
| 391 | + | |
| 392 | + void setInstanceDescriptorType(MTL::AccelerationStructureInstanceDescriptorType instanceDescriptorType); | |
| 393 | + | |
| 394 | + void setInstanceTransformationMatrixLayout(MTL::MatrixLayout instanceTransformationMatrixLayout); | |
| 395 | + | |
| 396 | + void setMotionTransformBuffer(const MTL4::BufferRange motionTransformBuffer); | |
| 397 | + | |
| 398 | + void setMotionTransformCount(NS::UInteger motionTransformCount); | |
| 399 | + | |
| 400 | + void setMotionTransformStride(NS::UInteger motionTransformStride); | |
| 401 | + | |
| 402 | + void setMotionTransformType(MTL::TransformType motionTransformType); | |
| 403 | +}; | |
| 404 | +class IndirectInstanceAccelerationStructureDescriptor : public NS::Copying<IndirectInstanceAccelerationStructureDescriptor, AccelerationStructureDescriptor> | |
| 405 | +{ | |
| 406 | +public: | |
| 407 | + static IndirectInstanceAccelerationStructureDescriptor* alloc(); | |
| 408 | + | |
| 409 | + IndirectInstanceAccelerationStructureDescriptor* init(); | |
| 410 | + | |
| 411 | + BufferRange instanceCountBuffer() const; | |
| 412 | + | |
| 413 | + BufferRange instanceDescriptorBuffer() const; | |
| 414 | + | |
| 415 | + NS::UInteger instanceDescriptorStride() const; | |
| 416 | + | |
| 417 | + MTL::AccelerationStructureInstanceDescriptorType instanceDescriptorType() const; | |
| 418 | + | |
| 419 | + MTL::MatrixLayout instanceTransformationMatrixLayout() const; | |
| 420 | + | |
| 421 | + NS::UInteger maxInstanceCount() const; | |
| 422 | + | |
| 423 | + NS::UInteger maxMotionTransformCount() const; | |
| 424 | + | |
| 425 | + BufferRange motionTransformBuffer() const; | |
| 426 | + | |
| 427 | + BufferRange motionTransformCountBuffer() const; | |
| 428 | + | |
| 429 | + NS::UInteger motionTransformStride() const; | |
| 430 | + | |
| 431 | + MTL::TransformType motionTransformType() const; | |
| 432 | + | |
| 433 | + void setInstanceCountBuffer(const MTL4::BufferRange instanceCountBuffer); | |
| 434 | + | |
| 435 | + void setInstanceDescriptorBuffer(const MTL4::BufferRange instanceDescriptorBuffer); | |
| 436 | + | |
| 437 | + void setInstanceDescriptorStride(NS::UInteger instanceDescriptorStride); | |
| 438 | + | |
| 439 | + void setInstanceDescriptorType(MTL::AccelerationStructureInstanceDescriptorType instanceDescriptorType); | |
| 440 | + | |
| 441 | + void setInstanceTransformationMatrixLayout(MTL::MatrixLayout instanceTransformationMatrixLayout); | |
| 442 | + | |
| 443 | + void setMaxInstanceCount(NS::UInteger maxInstanceCount); | |
| 444 | + | |
| 445 | + void setMaxMotionTransformCount(NS::UInteger maxMotionTransformCount); | |
| 446 | + | |
| 447 | + void setMotionTransformBuffer(const MTL4::BufferRange motionTransformBuffer); | |
| 448 | + | |
| 449 | + void setMotionTransformCountBuffer(const MTL4::BufferRange motionTransformCountBuffer); | |
| 450 | + | |
| 451 | + void setMotionTransformStride(NS::UInteger motionTransformStride); | |
| 452 | + | |
| 453 | + void setMotionTransformType(MTL::TransformType motionTransformType); | |
| 454 | +}; | |
| 455 | + | |
| 456 | +} | |
| 457 | +_MTL_INLINE MTL4::AccelerationStructureDescriptor* MTL4::AccelerationStructureDescriptor::alloc() | |
| 458 | +{ | |
| 459 | + return NS::Object::alloc<MTL4::AccelerationStructureDescriptor>(_MTL_PRIVATE_CLS(MTL4AccelerationStructureDescriptor)); | |
| 460 | +} | |
| 461 | + | |
| 462 | +_MTL_INLINE MTL4::AccelerationStructureDescriptor* MTL4::AccelerationStructureDescriptor::init() | |
| 463 | +{ | |
| 464 | + return NS::Object::init<MTL4::AccelerationStructureDescriptor>(); | |
| 465 | +} | |
| 466 | + | |
| 467 | +_MTL_INLINE MTL4::AccelerationStructureGeometryDescriptor* MTL4::AccelerationStructureGeometryDescriptor::alloc() | |
| 468 | +{ | |
| 469 | + return NS::Object::alloc<MTL4::AccelerationStructureGeometryDescriptor>(_MTL_PRIVATE_CLS(MTL4AccelerationStructureGeometryDescriptor)); | |
| 470 | +} | |
| 471 | + | |
| 472 | +_MTL_INLINE bool MTL4::AccelerationStructureGeometryDescriptor::allowDuplicateIntersectionFunctionInvocation() const | |
| 473 | +{ | |
| 474 | + return Object::sendMessage<bool>(this, _MTL_PRIVATE_SEL(allowDuplicateIntersectionFunctionInvocation)); | |
| 475 | +} | |
| 476 | + | |
| 477 | +_MTL_INLINE MTL4::AccelerationStructureGeometryDescriptor* MTL4::AccelerationStructureGeometryDescriptor::init() | |
| 478 | +{ | |
| 479 | + return NS::Object::init<MTL4::AccelerationStructureGeometryDescriptor>(); | |
| 480 | +} | |
| 481 | + | |
| 482 | +_MTL_INLINE NS::UInteger MTL4::AccelerationStructureGeometryDescriptor::intersectionFunctionTableOffset() const | |
| 483 | +{ | |
| 484 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(intersectionFunctionTableOffset)); | |
| 485 | +} | |
| 486 | + | |
| 487 | +_MTL_INLINE NS::String* MTL4::AccelerationStructureGeometryDescriptor::label() const | |
| 488 | +{ | |
| 489 | + return Object::sendMessage<NS::String*>(this, _MTL_PRIVATE_SEL(label)); | |
| 490 | +} | |
| 491 | + | |
| 492 | +_MTL_INLINE bool MTL4::AccelerationStructureGeometryDescriptor::opaque() const | |
| 493 | +{ | |
| 494 | + return Object::sendMessage<bool>(this, _MTL_PRIVATE_SEL(opaque)); | |
| 495 | +} | |
| 496 | + | |
| 497 | +_MTL_INLINE MTL4::BufferRange MTL4::AccelerationStructureGeometryDescriptor::primitiveDataBuffer() const | |
| 498 | +{ | |
| 499 | + return Object::sendMessage<MTL4::BufferRange>(this, _MTL_PRIVATE_SEL(primitiveDataBuffer)); | |
| 500 | +} | |
| 501 | + | |
| 502 | +_MTL_INLINE NS::UInteger MTL4::AccelerationStructureGeometryDescriptor::primitiveDataElementSize() const | |
| 503 | +{ | |
| 504 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(primitiveDataElementSize)); | |
| 505 | +} | |
| 506 | + | |
| 507 | +_MTL_INLINE NS::UInteger MTL4::AccelerationStructureGeometryDescriptor::primitiveDataStride() const | |
| 508 | +{ | |
| 509 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(primitiveDataStride)); | |
| 510 | +} | |
| 511 | + | |
| 512 | +_MTL_INLINE void MTL4::AccelerationStructureGeometryDescriptor::setAllowDuplicateIntersectionFunctionInvocation(bool allowDuplicateIntersectionFunctionInvocation) | |
| 513 | +{ | |
| 514 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setAllowDuplicateIntersectionFunctionInvocation_), allowDuplicateIntersectionFunctionInvocation); | |
| 515 | +} | |
| 516 | + | |
| 517 | +_MTL_INLINE void MTL4::AccelerationStructureGeometryDescriptor::setIntersectionFunctionTableOffset(NS::UInteger intersectionFunctionTableOffset) | |
| 518 | +{ | |
| 519 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setIntersectionFunctionTableOffset_), intersectionFunctionTableOffset); | |
| 520 | +} | |
| 521 | + | |
| 522 | +_MTL_INLINE void MTL4::AccelerationStructureGeometryDescriptor::setLabel(const NS::String* label) | |
| 523 | +{ | |
| 524 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setLabel_), label); | |
| 525 | +} | |
| 526 | + | |
| 527 | +_MTL_INLINE void MTL4::AccelerationStructureGeometryDescriptor::setOpaque(bool opaque) | |
| 528 | +{ | |
| 529 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setOpaque_), opaque); | |
| 530 | +} | |
| 531 | + | |
| 532 | +_MTL_INLINE void MTL4::AccelerationStructureGeometryDescriptor::setPrimitiveDataBuffer(const MTL4::BufferRange primitiveDataBuffer) | |
| 533 | +{ | |
| 534 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setPrimitiveDataBuffer_), primitiveDataBuffer); | |
| 535 | +} | |
| 536 | + | |
| 537 | +_MTL_INLINE void MTL4::AccelerationStructureGeometryDescriptor::setPrimitiveDataElementSize(NS::UInteger primitiveDataElementSize) | |
| 538 | +{ | |
| 539 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setPrimitiveDataElementSize_), primitiveDataElementSize); | |
| 540 | +} | |
| 541 | + | |
| 542 | +_MTL_INLINE void MTL4::AccelerationStructureGeometryDescriptor::setPrimitiveDataStride(NS::UInteger primitiveDataStride) | |
| 543 | +{ | |
| 544 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setPrimitiveDataStride_), primitiveDataStride); | |
| 545 | +} | |
| 546 | + | |
| 547 | +_MTL_INLINE MTL4::PrimitiveAccelerationStructureDescriptor* MTL4::PrimitiveAccelerationStructureDescriptor::alloc() | |
| 548 | +{ | |
| 549 | + return NS::Object::alloc<MTL4::PrimitiveAccelerationStructureDescriptor>(_MTL_PRIVATE_CLS(MTL4PrimitiveAccelerationStructureDescriptor)); | |
| 550 | +} | |
| 551 | + | |
| 552 | +_MTL_INLINE NS::Array* MTL4::PrimitiveAccelerationStructureDescriptor::geometryDescriptors() const | |
| 553 | +{ | |
| 554 | + return Object::sendMessage<NS::Array*>(this, _MTL_PRIVATE_SEL(geometryDescriptors)); | |
| 555 | +} | |
| 556 | + | |
| 557 | +_MTL_INLINE MTL4::PrimitiveAccelerationStructureDescriptor* MTL4::PrimitiveAccelerationStructureDescriptor::init() | |
| 558 | +{ | |
| 559 | + return NS::Object::init<MTL4::PrimitiveAccelerationStructureDescriptor>(); | |
| 560 | +} | |
| 561 | + | |
| 562 | +_MTL_INLINE MTL::MotionBorderMode MTL4::PrimitiveAccelerationStructureDescriptor::motionEndBorderMode() const | |
| 563 | +{ | |
| 564 | + return Object::sendMessage<MTL::MotionBorderMode>(this, _MTL_PRIVATE_SEL(motionEndBorderMode)); | |
| 565 | +} | |
| 566 | + | |
| 567 | +_MTL_INLINE float MTL4::PrimitiveAccelerationStructureDescriptor::motionEndTime() const | |
| 568 | +{ | |
| 569 | + return Object::sendMessage<float>(this, _MTL_PRIVATE_SEL(motionEndTime)); | |
| 570 | +} | |
| 571 | + | |
| 572 | +_MTL_INLINE NS::UInteger MTL4::PrimitiveAccelerationStructureDescriptor::motionKeyframeCount() const | |
| 573 | +{ | |
| 574 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(motionKeyframeCount)); | |
| 575 | +} | |
| 576 | + | |
| 577 | +_MTL_INLINE MTL::MotionBorderMode MTL4::PrimitiveAccelerationStructureDescriptor::motionStartBorderMode() const | |
| 578 | +{ | |
| 579 | + return Object::sendMessage<MTL::MotionBorderMode>(this, _MTL_PRIVATE_SEL(motionStartBorderMode)); | |
| 580 | +} | |
| 581 | + | |
| 582 | +_MTL_INLINE float MTL4::PrimitiveAccelerationStructureDescriptor::motionStartTime() const | |
| 583 | +{ | |
| 584 | + return Object::sendMessage<float>(this, _MTL_PRIVATE_SEL(motionStartTime)); | |
| 585 | +} | |
| 586 | + | |
| 587 | +_MTL_INLINE void MTL4::PrimitiveAccelerationStructureDescriptor::setGeometryDescriptors(const NS::Array* geometryDescriptors) | |
| 588 | +{ | |
| 589 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setGeometryDescriptors_), geometryDescriptors); | |
| 590 | +} | |
| 591 | + | |
| 592 | +_MTL_INLINE void MTL4::PrimitiveAccelerationStructureDescriptor::setMotionEndBorderMode(MTL::MotionBorderMode motionEndBorderMode) | |
| 593 | +{ | |
| 594 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMotionEndBorderMode_), motionEndBorderMode); | |
| 595 | +} | |
| 596 | + | |
| 597 | +_MTL_INLINE void MTL4::PrimitiveAccelerationStructureDescriptor::setMotionEndTime(float motionEndTime) | |
| 598 | +{ | |
| 599 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMotionEndTime_), motionEndTime); | |
| 600 | +} | |
| 601 | + | |
| 602 | +_MTL_INLINE void MTL4::PrimitiveAccelerationStructureDescriptor::setMotionKeyframeCount(NS::UInteger motionKeyframeCount) | |
| 603 | +{ | |
| 604 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMotionKeyframeCount_), motionKeyframeCount); | |
| 605 | +} | |
| 606 | + | |
| 607 | +_MTL_INLINE void MTL4::PrimitiveAccelerationStructureDescriptor::setMotionStartBorderMode(MTL::MotionBorderMode motionStartBorderMode) | |
| 608 | +{ | |
| 609 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMotionStartBorderMode_), motionStartBorderMode); | |
| 610 | +} | |
| 611 | + | |
| 612 | +_MTL_INLINE void MTL4::PrimitiveAccelerationStructureDescriptor::setMotionStartTime(float motionStartTime) | |
| 613 | +{ | |
| 614 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMotionStartTime_), motionStartTime); | |
| 615 | +} | |
| 616 | + | |
| 617 | +_MTL_INLINE MTL4::AccelerationStructureTriangleGeometryDescriptor* MTL4::AccelerationStructureTriangleGeometryDescriptor::alloc() | |
| 618 | +{ | |
| 619 | + return NS::Object::alloc<MTL4::AccelerationStructureTriangleGeometryDescriptor>(_MTL_PRIVATE_CLS(MTL4AccelerationStructureTriangleGeometryDescriptor)); | |
| 620 | +} | |
| 621 | + | |
| 622 | +_MTL_INLINE MTL4::BufferRange MTL4::AccelerationStructureTriangleGeometryDescriptor::indexBuffer() const | |
| 623 | +{ | |
| 624 | + return Object::sendMessage<MTL4::BufferRange>(this, _MTL_PRIVATE_SEL(indexBuffer)); | |
| 625 | +} | |
| 626 | + | |
| 627 | +_MTL_INLINE MTL::IndexType MTL4::AccelerationStructureTriangleGeometryDescriptor::indexType() const | |
| 628 | +{ | |
| 629 | + return Object::sendMessage<MTL::IndexType>(this, _MTL_PRIVATE_SEL(indexType)); | |
| 630 | +} | |
| 631 | + | |
| 632 | +_MTL_INLINE MTL4::AccelerationStructureTriangleGeometryDescriptor* MTL4::AccelerationStructureTriangleGeometryDescriptor::init() | |
| 633 | +{ | |
| 634 | + return NS::Object::init<MTL4::AccelerationStructureTriangleGeometryDescriptor>(); | |
| 635 | +} | |
| 636 | + | |
| 637 | +_MTL_INLINE void MTL4::AccelerationStructureTriangleGeometryDescriptor::setIndexBuffer(const MTL4::BufferRange indexBuffer) | |
| 638 | +{ | |
| 639 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setIndexBuffer_), indexBuffer); | |
| 640 | +} | |
| 641 | + | |
| 642 | +_MTL_INLINE void MTL4::AccelerationStructureTriangleGeometryDescriptor::setIndexType(MTL::IndexType indexType) | |
| 643 | +{ | |
| 644 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setIndexType_), indexType); | |
| 645 | +} | |
| 646 | + | |
| 647 | +_MTL_INLINE void MTL4::AccelerationStructureTriangleGeometryDescriptor::setTransformationMatrixBuffer(const MTL4::BufferRange transformationMatrixBuffer) | |
| 648 | +{ | |
| 649 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setTransformationMatrixBuffer_), transformationMatrixBuffer); | |
| 650 | +} | |
| 651 | + | |
| 652 | +_MTL_INLINE void MTL4::AccelerationStructureTriangleGeometryDescriptor::setTransformationMatrixLayout(MTL::MatrixLayout transformationMatrixLayout) | |
| 653 | +{ | |
| 654 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setTransformationMatrixLayout_), transformationMatrixLayout); | |
| 655 | +} | |
| 656 | + | |
| 657 | +_MTL_INLINE void MTL4::AccelerationStructureTriangleGeometryDescriptor::setTriangleCount(NS::UInteger triangleCount) | |
| 658 | +{ | |
| 659 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setTriangleCount_), triangleCount); | |
| 660 | +} | |
| 661 | + | |
| 662 | +_MTL_INLINE void MTL4::AccelerationStructureTriangleGeometryDescriptor::setVertexBuffer(const MTL4::BufferRange vertexBuffer) | |
| 663 | +{ | |
| 664 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setVertexBuffer_), vertexBuffer); | |
| 665 | +} | |
| 666 | + | |
| 667 | +_MTL_INLINE void MTL4::AccelerationStructureTriangleGeometryDescriptor::setVertexFormat(MTL::AttributeFormat vertexFormat) | |
| 668 | +{ | |
| 669 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setVertexFormat_), vertexFormat); | |
| 670 | +} | |
| 671 | + | |
| 672 | +_MTL_INLINE void MTL4::AccelerationStructureTriangleGeometryDescriptor::setVertexStride(NS::UInteger vertexStride) | |
| 673 | +{ | |
| 674 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setVertexStride_), vertexStride); | |
| 675 | +} | |
| 676 | + | |
| 677 | +_MTL_INLINE MTL4::BufferRange MTL4::AccelerationStructureTriangleGeometryDescriptor::transformationMatrixBuffer() const | |
| 678 | +{ | |
| 679 | + return Object::sendMessage<MTL4::BufferRange>(this, _MTL_PRIVATE_SEL(transformationMatrixBuffer)); | |
| 680 | +} | |
| 681 | + | |
| 682 | +_MTL_INLINE MTL::MatrixLayout MTL4::AccelerationStructureTriangleGeometryDescriptor::transformationMatrixLayout() const | |
| 683 | +{ | |
| 684 | + return Object::sendMessage<MTL::MatrixLayout>(this, _MTL_PRIVATE_SEL(transformationMatrixLayout)); | |
| 685 | +} | |
| 686 | + | |
| 687 | +_MTL_INLINE NS::UInteger MTL4::AccelerationStructureTriangleGeometryDescriptor::triangleCount() const | |
| 688 | +{ | |
| 689 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(triangleCount)); | |
| 690 | +} | |
| 691 | + | |
| 692 | +_MTL_INLINE MTL4::BufferRange MTL4::AccelerationStructureTriangleGeometryDescriptor::vertexBuffer() const | |
| 693 | +{ | |
| 694 | + return Object::sendMessage<MTL4::BufferRange>(this, _MTL_PRIVATE_SEL(vertexBuffer)); | |
| 695 | +} | |
| 696 | + | |
| 697 | +_MTL_INLINE MTL::AttributeFormat MTL4::AccelerationStructureTriangleGeometryDescriptor::vertexFormat() const | |
| 698 | +{ | |
| 699 | + return Object::sendMessage<MTL::AttributeFormat>(this, _MTL_PRIVATE_SEL(vertexFormat)); | |
| 700 | +} | |
| 701 | + | |
| 702 | +_MTL_INLINE NS::UInteger MTL4::AccelerationStructureTriangleGeometryDescriptor::vertexStride() const | |
| 703 | +{ | |
| 704 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(vertexStride)); | |
| 705 | +} | |
| 706 | + | |
| 707 | +_MTL_INLINE MTL4::AccelerationStructureBoundingBoxGeometryDescriptor* MTL4::AccelerationStructureBoundingBoxGeometryDescriptor::alloc() | |
| 708 | +{ | |
| 709 | + return NS::Object::alloc<MTL4::AccelerationStructureBoundingBoxGeometryDescriptor>(_MTL_PRIVATE_CLS(MTL4AccelerationStructureBoundingBoxGeometryDescriptor)); | |
| 710 | +} | |
| 711 | + | |
| 712 | +_MTL_INLINE MTL4::BufferRange MTL4::AccelerationStructureBoundingBoxGeometryDescriptor::boundingBoxBuffer() const | |
| 713 | +{ | |
| 714 | + return Object::sendMessage<MTL4::BufferRange>(this, _MTL_PRIVATE_SEL(boundingBoxBuffer)); | |
| 715 | +} | |
| 716 | + | |
| 717 | +_MTL_INLINE NS::UInteger MTL4::AccelerationStructureBoundingBoxGeometryDescriptor::boundingBoxCount() const | |
| 718 | +{ | |
| 719 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(boundingBoxCount)); | |
| 720 | +} | |
| 721 | + | |
| 722 | +_MTL_INLINE NS::UInteger MTL4::AccelerationStructureBoundingBoxGeometryDescriptor::boundingBoxStride() const | |
| 723 | +{ | |
| 724 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(boundingBoxStride)); | |
| 725 | +} | |
| 726 | + | |
| 727 | +_MTL_INLINE MTL4::AccelerationStructureBoundingBoxGeometryDescriptor* MTL4::AccelerationStructureBoundingBoxGeometryDescriptor::init() | |
| 728 | +{ | |
| 729 | + return NS::Object::init<MTL4::AccelerationStructureBoundingBoxGeometryDescriptor>(); | |
| 730 | +} | |
| 731 | + | |
| 732 | +_MTL_INLINE void MTL4::AccelerationStructureBoundingBoxGeometryDescriptor::setBoundingBoxBuffer(const MTL4::BufferRange boundingBoxBuffer) | |
| 733 | +{ | |
| 734 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setBoundingBoxBuffer_), boundingBoxBuffer); | |
| 735 | +} | |
| 736 | + | |
| 737 | +_MTL_INLINE void MTL4::AccelerationStructureBoundingBoxGeometryDescriptor::setBoundingBoxCount(NS::UInteger boundingBoxCount) | |
| 738 | +{ | |
| 739 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setBoundingBoxCount_), boundingBoxCount); | |
| 740 | +} | |
| 741 | + | |
| 742 | +_MTL_INLINE void MTL4::AccelerationStructureBoundingBoxGeometryDescriptor::setBoundingBoxStride(NS::UInteger boundingBoxStride) | |
| 743 | +{ | |
| 744 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setBoundingBoxStride_), boundingBoxStride); | |
| 745 | +} | |
| 746 | + | |
| 747 | +_MTL_INLINE MTL4::AccelerationStructureMotionTriangleGeometryDescriptor* MTL4::AccelerationStructureMotionTriangleGeometryDescriptor::alloc() | |
| 748 | +{ | |
| 749 | + return NS::Object::alloc<MTL4::AccelerationStructureMotionTriangleGeometryDescriptor>(_MTL_PRIVATE_CLS(MTL4AccelerationStructureMotionTriangleGeometryDescriptor)); | |
| 750 | +} | |
| 751 | + | |
| 752 | +_MTL_INLINE MTL4::BufferRange MTL4::AccelerationStructureMotionTriangleGeometryDescriptor::indexBuffer() const | |
| 753 | +{ | |
| 754 | + return Object::sendMessage<MTL4::BufferRange>(this, _MTL_PRIVATE_SEL(indexBuffer)); | |
| 755 | +} | |
| 756 | + | |
| 757 | +_MTL_INLINE MTL::IndexType MTL4::AccelerationStructureMotionTriangleGeometryDescriptor::indexType() const | |
| 758 | +{ | |
| 759 | + return Object::sendMessage<MTL::IndexType>(this, _MTL_PRIVATE_SEL(indexType)); | |
| 760 | +} | |
| 761 | + | |
| 762 | +_MTL_INLINE MTL4::AccelerationStructureMotionTriangleGeometryDescriptor* MTL4::AccelerationStructureMotionTriangleGeometryDescriptor::init() | |
| 763 | +{ | |
| 764 | + return NS::Object::init<MTL4::AccelerationStructureMotionTriangleGeometryDescriptor>(); | |
| 765 | +} | |
| 766 | + | |
| 767 | +_MTL_INLINE void MTL4::AccelerationStructureMotionTriangleGeometryDescriptor::setIndexBuffer(const MTL4::BufferRange indexBuffer) | |
| 768 | +{ | |
| 769 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setIndexBuffer_), indexBuffer); | |
| 770 | +} | |
| 771 | + | |
| 772 | +_MTL_INLINE void MTL4::AccelerationStructureMotionTriangleGeometryDescriptor::setIndexType(MTL::IndexType indexType) | |
| 773 | +{ | |
| 774 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setIndexType_), indexType); | |
| 775 | +} | |
| 776 | + | |
| 777 | +_MTL_INLINE void MTL4::AccelerationStructureMotionTriangleGeometryDescriptor::setTransformationMatrixBuffer(const MTL4::BufferRange transformationMatrixBuffer) | |
| 778 | +{ | |
| 779 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setTransformationMatrixBuffer_), transformationMatrixBuffer); | |
| 780 | +} | |
| 781 | + | |
| 782 | +_MTL_INLINE void MTL4::AccelerationStructureMotionTriangleGeometryDescriptor::setTransformationMatrixLayout(MTL::MatrixLayout transformationMatrixLayout) | |
| 783 | +{ | |
| 784 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setTransformationMatrixLayout_), transformationMatrixLayout); | |
| 785 | +} | |
| 786 | + | |
| 787 | +_MTL_INLINE void MTL4::AccelerationStructureMotionTriangleGeometryDescriptor::setTriangleCount(NS::UInteger triangleCount) | |
| 788 | +{ | |
| 789 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setTriangleCount_), triangleCount); | |
| 790 | +} | |
| 791 | + | |
| 792 | +_MTL_INLINE void MTL4::AccelerationStructureMotionTriangleGeometryDescriptor::setVertexBuffers(const MTL4::BufferRange vertexBuffers) | |
| 793 | +{ | |
| 794 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setVertexBuffers_), vertexBuffers); | |
| 795 | +} | |
| 796 | + | |
| 797 | +_MTL_INLINE void MTL4::AccelerationStructureMotionTriangleGeometryDescriptor::setVertexFormat(MTL::AttributeFormat vertexFormat) | |
| 798 | +{ | |
| 799 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setVertexFormat_), vertexFormat); | |
| 800 | +} | |
| 801 | + | |
| 802 | +_MTL_INLINE void MTL4::AccelerationStructureMotionTriangleGeometryDescriptor::setVertexStride(NS::UInteger vertexStride) | |
| 803 | +{ | |
| 804 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setVertexStride_), vertexStride); | |
| 805 | +} | |
| 806 | + | |
| 807 | +_MTL_INLINE MTL4::BufferRange MTL4::AccelerationStructureMotionTriangleGeometryDescriptor::transformationMatrixBuffer() const | |
| 808 | +{ | |
| 809 | + return Object::sendMessage<MTL4::BufferRange>(this, _MTL_PRIVATE_SEL(transformationMatrixBuffer)); | |
| 810 | +} | |
| 811 | + | |
| 812 | +_MTL_INLINE MTL::MatrixLayout MTL4::AccelerationStructureMotionTriangleGeometryDescriptor::transformationMatrixLayout() const | |
| 813 | +{ | |
| 814 | + return Object::sendMessage<MTL::MatrixLayout>(this, _MTL_PRIVATE_SEL(transformationMatrixLayout)); | |
| 815 | +} | |
| 816 | + | |
| 817 | +_MTL_INLINE NS::UInteger MTL4::AccelerationStructureMotionTriangleGeometryDescriptor::triangleCount() const | |
| 818 | +{ | |
| 819 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(triangleCount)); | |
| 820 | +} | |
| 821 | + | |
| 822 | +_MTL_INLINE MTL4::BufferRange MTL4::AccelerationStructureMotionTriangleGeometryDescriptor::vertexBuffers() const | |
| 823 | +{ | |
| 824 | + return Object::sendMessage<MTL4::BufferRange>(this, _MTL_PRIVATE_SEL(vertexBuffers)); | |
| 825 | +} | |
| 826 | + | |
| 827 | +_MTL_INLINE MTL::AttributeFormat MTL4::AccelerationStructureMotionTriangleGeometryDescriptor::vertexFormat() const | |
| 828 | +{ | |
| 829 | + return Object::sendMessage<MTL::AttributeFormat>(this, _MTL_PRIVATE_SEL(vertexFormat)); | |
| 830 | +} | |
| 831 | + | |
| 832 | +_MTL_INLINE NS::UInteger MTL4::AccelerationStructureMotionTriangleGeometryDescriptor::vertexStride() const | |
| 833 | +{ | |
| 834 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(vertexStride)); | |
| 835 | +} | |
| 836 | + | |
| 837 | +_MTL_INLINE MTL4::AccelerationStructureMotionBoundingBoxGeometryDescriptor* MTL4::AccelerationStructureMotionBoundingBoxGeometryDescriptor::alloc() | |
| 838 | +{ | |
| 839 | + return NS::Object::alloc<MTL4::AccelerationStructureMotionBoundingBoxGeometryDescriptor>(_MTL_PRIVATE_CLS(MTL4AccelerationStructureMotionBoundingBoxGeometryDescriptor)); | |
| 840 | +} | |
| 841 | + | |
| 842 | +_MTL_INLINE MTL4::BufferRange MTL4::AccelerationStructureMotionBoundingBoxGeometryDescriptor::boundingBoxBuffers() const | |
| 843 | +{ | |
| 844 | + return Object::sendMessage<MTL4::BufferRange>(this, _MTL_PRIVATE_SEL(boundingBoxBuffers)); | |
| 845 | +} | |
| 846 | + | |
| 847 | +_MTL_INLINE NS::UInteger MTL4::AccelerationStructureMotionBoundingBoxGeometryDescriptor::boundingBoxCount() const | |
| 848 | +{ | |
| 849 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(boundingBoxCount)); | |
| 850 | +} | |
| 851 | + | |
| 852 | +_MTL_INLINE NS::UInteger MTL4::AccelerationStructureMotionBoundingBoxGeometryDescriptor::boundingBoxStride() const | |
| 853 | +{ | |
| 854 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(boundingBoxStride)); | |
| 855 | +} | |
| 856 | + | |
| 857 | +_MTL_INLINE MTL4::AccelerationStructureMotionBoundingBoxGeometryDescriptor* MTL4::AccelerationStructureMotionBoundingBoxGeometryDescriptor::init() | |
| 858 | +{ | |
| 859 | + return NS::Object::init<MTL4::AccelerationStructureMotionBoundingBoxGeometryDescriptor>(); | |
| 860 | +} | |
| 861 | + | |
| 862 | +_MTL_INLINE void MTL4::AccelerationStructureMotionBoundingBoxGeometryDescriptor::setBoundingBoxBuffers(const MTL4::BufferRange boundingBoxBuffers) | |
| 863 | +{ | |
| 864 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setBoundingBoxBuffers_), boundingBoxBuffers); | |
| 865 | +} | |
| 866 | + | |
| 867 | +_MTL_INLINE void MTL4::AccelerationStructureMotionBoundingBoxGeometryDescriptor::setBoundingBoxCount(NS::UInteger boundingBoxCount) | |
| 868 | +{ | |
| 869 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setBoundingBoxCount_), boundingBoxCount); | |
| 870 | +} | |
| 871 | + | |
| 872 | +_MTL_INLINE void MTL4::AccelerationStructureMotionBoundingBoxGeometryDescriptor::setBoundingBoxStride(NS::UInteger boundingBoxStride) | |
| 873 | +{ | |
| 874 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setBoundingBoxStride_), boundingBoxStride); | |
| 875 | +} | |
| 876 | + | |
| 877 | +_MTL_INLINE MTL4::AccelerationStructureCurveGeometryDescriptor* MTL4::AccelerationStructureCurveGeometryDescriptor::alloc() | |
| 878 | +{ | |
| 879 | + return NS::Object::alloc<MTL4::AccelerationStructureCurveGeometryDescriptor>(_MTL_PRIVATE_CLS(MTL4AccelerationStructureCurveGeometryDescriptor)); | |
| 880 | +} | |
| 881 | + | |
| 882 | +_MTL_INLINE MTL4::BufferRange MTL4::AccelerationStructureCurveGeometryDescriptor::controlPointBuffer() const | |
| 883 | +{ | |
| 884 | + return Object::sendMessage<MTL4::BufferRange>(this, _MTL_PRIVATE_SEL(controlPointBuffer)); | |
| 885 | +} | |
| 886 | + | |
| 887 | +_MTL_INLINE NS::UInteger MTL4::AccelerationStructureCurveGeometryDescriptor::controlPointCount() const | |
| 888 | +{ | |
| 889 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(controlPointCount)); | |
| 890 | +} | |
| 891 | + | |
| 892 | +_MTL_INLINE MTL::AttributeFormat MTL4::AccelerationStructureCurveGeometryDescriptor::controlPointFormat() const | |
| 893 | +{ | |
| 894 | + return Object::sendMessage<MTL::AttributeFormat>(this, _MTL_PRIVATE_SEL(controlPointFormat)); | |
| 895 | +} | |
| 896 | + | |
| 897 | +_MTL_INLINE NS::UInteger MTL4::AccelerationStructureCurveGeometryDescriptor::controlPointStride() const | |
| 898 | +{ | |
| 899 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(controlPointStride)); | |
| 900 | +} | |
| 901 | + | |
| 902 | +_MTL_INLINE MTL::CurveBasis MTL4::AccelerationStructureCurveGeometryDescriptor::curveBasis() const | |
| 903 | +{ | |
| 904 | + return Object::sendMessage<MTL::CurveBasis>(this, _MTL_PRIVATE_SEL(curveBasis)); | |
| 905 | +} | |
| 906 | + | |
| 907 | +_MTL_INLINE MTL::CurveEndCaps MTL4::AccelerationStructureCurveGeometryDescriptor::curveEndCaps() const | |
| 908 | +{ | |
| 909 | + return Object::sendMessage<MTL::CurveEndCaps>(this, _MTL_PRIVATE_SEL(curveEndCaps)); | |
| 910 | +} | |
| 911 | + | |
| 912 | +_MTL_INLINE MTL::CurveType MTL4::AccelerationStructureCurveGeometryDescriptor::curveType() const | |
| 913 | +{ | |
| 914 | + return Object::sendMessage<MTL::CurveType>(this, _MTL_PRIVATE_SEL(curveType)); | |
| 915 | +} | |
| 916 | + | |
| 917 | +_MTL_INLINE MTL4::BufferRange MTL4::AccelerationStructureCurveGeometryDescriptor::indexBuffer() const | |
| 918 | +{ | |
| 919 | + return Object::sendMessage<MTL4::BufferRange>(this, _MTL_PRIVATE_SEL(indexBuffer)); | |
| 920 | +} | |
| 921 | + | |
| 922 | +_MTL_INLINE MTL::IndexType MTL4::AccelerationStructureCurveGeometryDescriptor::indexType() const | |
| 923 | +{ | |
| 924 | + return Object::sendMessage<MTL::IndexType>(this, _MTL_PRIVATE_SEL(indexType)); | |
| 925 | +} | |
| 926 | + | |
| 927 | +_MTL_INLINE MTL4::AccelerationStructureCurveGeometryDescriptor* MTL4::AccelerationStructureCurveGeometryDescriptor::init() | |
| 928 | +{ | |
| 929 | + return NS::Object::init<MTL4::AccelerationStructureCurveGeometryDescriptor>(); | |
| 930 | +} | |
| 931 | + | |
| 932 | +_MTL_INLINE MTL4::BufferRange MTL4::AccelerationStructureCurveGeometryDescriptor::radiusBuffer() const | |
| 933 | +{ | |
| 934 | + return Object::sendMessage<MTL4::BufferRange>(this, _MTL_PRIVATE_SEL(radiusBuffer)); | |
| 935 | +} | |
| 936 | + | |
| 937 | +_MTL_INLINE MTL::AttributeFormat MTL4::AccelerationStructureCurveGeometryDescriptor::radiusFormat() const | |
| 938 | +{ | |
| 939 | + return Object::sendMessage<MTL::AttributeFormat>(this, _MTL_PRIVATE_SEL(radiusFormat)); | |
| 940 | +} | |
| 941 | + | |
| 942 | +_MTL_INLINE NS::UInteger MTL4::AccelerationStructureCurveGeometryDescriptor::radiusStride() const | |
| 943 | +{ | |
| 944 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(radiusStride)); | |
| 945 | +} | |
| 946 | + | |
| 947 | +_MTL_INLINE NS::UInteger MTL4::AccelerationStructureCurveGeometryDescriptor::segmentControlPointCount() const | |
| 948 | +{ | |
| 949 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(segmentControlPointCount)); | |
| 950 | +} | |
| 951 | + | |
| 952 | +_MTL_INLINE NS::UInteger MTL4::AccelerationStructureCurveGeometryDescriptor::segmentCount() const | |
| 953 | +{ | |
| 954 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(segmentCount)); | |
| 955 | +} | |
| 956 | + | |
| 957 | +_MTL_INLINE void MTL4::AccelerationStructureCurveGeometryDescriptor::setControlPointBuffer(const MTL4::BufferRange controlPointBuffer) | |
| 958 | +{ | |
| 959 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setControlPointBuffer_), controlPointBuffer); | |
| 960 | +} | |
| 961 | + | |
| 962 | +_MTL_INLINE void MTL4::AccelerationStructureCurveGeometryDescriptor::setControlPointCount(NS::UInteger controlPointCount) | |
| 963 | +{ | |
| 964 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setControlPointCount_), controlPointCount); | |
| 965 | +} | |
| 966 | + | |
| 967 | +_MTL_INLINE void MTL4::AccelerationStructureCurveGeometryDescriptor::setControlPointFormat(MTL::AttributeFormat controlPointFormat) | |
| 968 | +{ | |
| 969 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setControlPointFormat_), controlPointFormat); | |
| 970 | +} | |
| 971 | + | |
| 972 | +_MTL_INLINE void MTL4::AccelerationStructureCurveGeometryDescriptor::setControlPointStride(NS::UInteger controlPointStride) | |
| 973 | +{ | |
| 974 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setControlPointStride_), controlPointStride); | |
| 975 | +} | |
| 976 | + | |
| 977 | +_MTL_INLINE void MTL4::AccelerationStructureCurveGeometryDescriptor::setCurveBasis(MTL::CurveBasis curveBasis) | |
| 978 | +{ | |
| 979 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setCurveBasis_), curveBasis); | |
| 980 | +} | |
| 981 | + | |
| 982 | +_MTL_INLINE void MTL4::AccelerationStructureCurveGeometryDescriptor::setCurveEndCaps(MTL::CurveEndCaps curveEndCaps) | |
| 983 | +{ | |
| 984 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setCurveEndCaps_), curveEndCaps); | |
| 985 | +} | |
| 986 | + | |
| 987 | +_MTL_INLINE void MTL4::AccelerationStructureCurveGeometryDescriptor::setCurveType(MTL::CurveType curveType) | |
| 988 | +{ | |
| 989 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setCurveType_), curveType); | |
| 990 | +} | |
| 991 | + | |
| 992 | +_MTL_INLINE void MTL4::AccelerationStructureCurveGeometryDescriptor::setIndexBuffer(const MTL4::BufferRange indexBuffer) | |
| 993 | +{ | |
| 994 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setIndexBuffer_), indexBuffer); | |
| 995 | +} | |
| 996 | + | |
| 997 | +_MTL_INLINE void MTL4::AccelerationStructureCurveGeometryDescriptor::setIndexType(MTL::IndexType indexType) | |
| 998 | +{ | |
| 999 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setIndexType_), indexType); | |
| 1000 | +} | |
| 1001 | + | |
| 1002 | +_MTL_INLINE void MTL4::AccelerationStructureCurveGeometryDescriptor::setRadiusBuffer(const MTL4::BufferRange radiusBuffer) | |
| 1003 | +{ | |
| 1004 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setRadiusBuffer_), radiusBuffer); | |
| 1005 | +} | |
| 1006 | + | |
| 1007 | +_MTL_INLINE void MTL4::AccelerationStructureCurveGeometryDescriptor::setRadiusFormat(MTL::AttributeFormat radiusFormat) | |
| 1008 | +{ | |
| 1009 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setRadiusFormat_), radiusFormat); | |
| 1010 | +} | |
| 1011 | + | |
| 1012 | +_MTL_INLINE void MTL4::AccelerationStructureCurveGeometryDescriptor::setRadiusStride(NS::UInteger radiusStride) | |
| 1013 | +{ | |
| 1014 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setRadiusStride_), radiusStride); | |
| 1015 | +} | |
| 1016 | + | |
| 1017 | +_MTL_INLINE void MTL4::AccelerationStructureCurveGeometryDescriptor::setSegmentControlPointCount(NS::UInteger segmentControlPointCount) | |
| 1018 | +{ | |
| 1019 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setSegmentControlPointCount_), segmentControlPointCount); | |
| 1020 | +} | |
| 1021 | + | |
| 1022 | +_MTL_INLINE void MTL4::AccelerationStructureCurveGeometryDescriptor::setSegmentCount(NS::UInteger segmentCount) | |
| 1023 | +{ | |
| 1024 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setSegmentCount_), segmentCount); | |
| 1025 | +} | |
| 1026 | + | |
| 1027 | +_MTL_INLINE MTL4::AccelerationStructureMotionCurveGeometryDescriptor* MTL4::AccelerationStructureMotionCurveGeometryDescriptor::alloc() | |
| 1028 | +{ | |
| 1029 | + return NS::Object::alloc<MTL4::AccelerationStructureMotionCurveGeometryDescriptor>(_MTL_PRIVATE_CLS(MTL4AccelerationStructureMotionCurveGeometryDescriptor)); | |
| 1030 | +} | |
| 1031 | + | |
| 1032 | +_MTL_INLINE MTL4::BufferRange MTL4::AccelerationStructureMotionCurveGeometryDescriptor::controlPointBuffers() const | |
| 1033 | +{ | |
| 1034 | + return Object::sendMessage<MTL4::BufferRange>(this, _MTL_PRIVATE_SEL(controlPointBuffers)); | |
| 1035 | +} | |
| 1036 | + | |
| 1037 | +_MTL_INLINE NS::UInteger MTL4::AccelerationStructureMotionCurveGeometryDescriptor::controlPointCount() const | |
| 1038 | +{ | |
| 1039 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(controlPointCount)); | |
| 1040 | +} | |
| 1041 | + | |
| 1042 | +_MTL_INLINE MTL::AttributeFormat MTL4::AccelerationStructureMotionCurveGeometryDescriptor::controlPointFormat() const | |
| 1043 | +{ | |
| 1044 | + return Object::sendMessage<MTL::AttributeFormat>(this, _MTL_PRIVATE_SEL(controlPointFormat)); | |
| 1045 | +} | |
| 1046 | + | |
| 1047 | +_MTL_INLINE NS::UInteger MTL4::AccelerationStructureMotionCurveGeometryDescriptor::controlPointStride() const | |
| 1048 | +{ | |
| 1049 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(controlPointStride)); | |
| 1050 | +} | |
| 1051 | + | |
| 1052 | +_MTL_INLINE MTL::CurveBasis MTL4::AccelerationStructureMotionCurveGeometryDescriptor::curveBasis() const | |
| 1053 | +{ | |
| 1054 | + return Object::sendMessage<MTL::CurveBasis>(this, _MTL_PRIVATE_SEL(curveBasis)); | |
| 1055 | +} | |
| 1056 | + | |
| 1057 | +_MTL_INLINE MTL::CurveEndCaps MTL4::AccelerationStructureMotionCurveGeometryDescriptor::curveEndCaps() const | |
| 1058 | +{ | |
| 1059 | + return Object::sendMessage<MTL::CurveEndCaps>(this, _MTL_PRIVATE_SEL(curveEndCaps)); | |
| 1060 | +} | |
| 1061 | + | |
| 1062 | +_MTL_INLINE MTL::CurveType MTL4::AccelerationStructureMotionCurveGeometryDescriptor::curveType() const | |
| 1063 | +{ | |
| 1064 | + return Object::sendMessage<MTL::CurveType>(this, _MTL_PRIVATE_SEL(curveType)); | |
| 1065 | +} | |
| 1066 | + | |
| 1067 | +_MTL_INLINE MTL4::BufferRange MTL4::AccelerationStructureMotionCurveGeometryDescriptor::indexBuffer() const | |
| 1068 | +{ | |
| 1069 | + return Object::sendMessage<MTL4::BufferRange>(this, _MTL_PRIVATE_SEL(indexBuffer)); | |
| 1070 | +} | |
| 1071 | + | |
| 1072 | +_MTL_INLINE MTL::IndexType MTL4::AccelerationStructureMotionCurveGeometryDescriptor::indexType() const | |
| 1073 | +{ | |
| 1074 | + return Object::sendMessage<MTL::IndexType>(this, _MTL_PRIVATE_SEL(indexType)); | |
| 1075 | +} | |
| 1076 | + | |
| 1077 | +_MTL_INLINE MTL4::AccelerationStructureMotionCurveGeometryDescriptor* MTL4::AccelerationStructureMotionCurveGeometryDescriptor::init() | |
| 1078 | +{ | |
| 1079 | + return NS::Object::init<MTL4::AccelerationStructureMotionCurveGeometryDescriptor>(); | |
| 1080 | +} | |
| 1081 | + | |
| 1082 | +_MTL_INLINE MTL4::BufferRange MTL4::AccelerationStructureMotionCurveGeometryDescriptor::radiusBuffers() const | |
| 1083 | +{ | |
| 1084 | + return Object::sendMessage<MTL4::BufferRange>(this, _MTL_PRIVATE_SEL(radiusBuffers)); | |
| 1085 | +} | |
| 1086 | + | |
| 1087 | +_MTL_INLINE MTL::AttributeFormat MTL4::AccelerationStructureMotionCurveGeometryDescriptor::radiusFormat() const | |
| 1088 | +{ | |
| 1089 | + return Object::sendMessage<MTL::AttributeFormat>(this, _MTL_PRIVATE_SEL(radiusFormat)); | |
| 1090 | +} | |
| 1091 | + | |
| 1092 | +_MTL_INLINE NS::UInteger MTL4::AccelerationStructureMotionCurveGeometryDescriptor::radiusStride() const | |
| 1093 | +{ | |
| 1094 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(radiusStride)); | |
| 1095 | +} | |
| 1096 | + | |
| 1097 | +_MTL_INLINE NS::UInteger MTL4::AccelerationStructureMotionCurveGeometryDescriptor::segmentControlPointCount() const | |
| 1098 | +{ | |
| 1099 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(segmentControlPointCount)); | |
| 1100 | +} | |
| 1101 | + | |
| 1102 | +_MTL_INLINE NS::UInteger MTL4::AccelerationStructureMotionCurveGeometryDescriptor::segmentCount() const | |
| 1103 | +{ | |
| 1104 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(segmentCount)); | |
| 1105 | +} | |
| 1106 | + | |
| 1107 | +_MTL_INLINE void MTL4::AccelerationStructureMotionCurveGeometryDescriptor::setControlPointBuffers(const MTL4::BufferRange controlPointBuffers) | |
| 1108 | +{ | |
| 1109 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setControlPointBuffers_), controlPointBuffers); | |
| 1110 | +} | |
| 1111 | + | |
| 1112 | +_MTL_INLINE void MTL4::AccelerationStructureMotionCurveGeometryDescriptor::setControlPointCount(NS::UInteger controlPointCount) | |
| 1113 | +{ | |
| 1114 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setControlPointCount_), controlPointCount); | |
| 1115 | +} | |
| 1116 | + | |
| 1117 | +_MTL_INLINE void MTL4::AccelerationStructureMotionCurveGeometryDescriptor::setControlPointFormat(MTL::AttributeFormat controlPointFormat) | |
| 1118 | +{ | |
| 1119 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setControlPointFormat_), controlPointFormat); | |
| 1120 | +} | |
| 1121 | + | |
| 1122 | +_MTL_INLINE void MTL4::AccelerationStructureMotionCurveGeometryDescriptor::setControlPointStride(NS::UInteger controlPointStride) | |
| 1123 | +{ | |
| 1124 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setControlPointStride_), controlPointStride); | |
| 1125 | +} | |
| 1126 | + | |
| 1127 | +_MTL_INLINE void MTL4::AccelerationStructureMotionCurveGeometryDescriptor::setCurveBasis(MTL::CurveBasis curveBasis) | |
| 1128 | +{ | |
| 1129 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setCurveBasis_), curveBasis); | |
| 1130 | +} | |
| 1131 | + | |
| 1132 | +_MTL_INLINE void MTL4::AccelerationStructureMotionCurveGeometryDescriptor::setCurveEndCaps(MTL::CurveEndCaps curveEndCaps) | |
| 1133 | +{ | |
| 1134 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setCurveEndCaps_), curveEndCaps); | |
| 1135 | +} | |
| 1136 | + | |
| 1137 | +_MTL_INLINE void MTL4::AccelerationStructureMotionCurveGeometryDescriptor::setCurveType(MTL::CurveType curveType) | |
| 1138 | +{ | |
| 1139 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setCurveType_), curveType); | |
| 1140 | +} | |
| 1141 | + | |
| 1142 | +_MTL_INLINE void MTL4::AccelerationStructureMotionCurveGeometryDescriptor::setIndexBuffer(const MTL4::BufferRange indexBuffer) | |
| 1143 | +{ | |
| 1144 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setIndexBuffer_), indexBuffer); | |
| 1145 | +} | |
| 1146 | + | |
| 1147 | +_MTL_INLINE void MTL4::AccelerationStructureMotionCurveGeometryDescriptor::setIndexType(MTL::IndexType indexType) | |
| 1148 | +{ | |
| 1149 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setIndexType_), indexType); | |
| 1150 | +} | |
| 1151 | + | |
| 1152 | +_MTL_INLINE void MTL4::AccelerationStructureMotionCurveGeometryDescriptor::setRadiusBuffers(const MTL4::BufferRange radiusBuffers) | |
| 1153 | +{ | |
| 1154 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setRadiusBuffers_), radiusBuffers); | |
| 1155 | +} | |
| 1156 | + | |
| 1157 | +_MTL_INLINE void MTL4::AccelerationStructureMotionCurveGeometryDescriptor::setRadiusFormat(MTL::AttributeFormat radiusFormat) | |
| 1158 | +{ | |
| 1159 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setRadiusFormat_), radiusFormat); | |
| 1160 | +} | |
| 1161 | + | |
| 1162 | +_MTL_INLINE void MTL4::AccelerationStructureMotionCurveGeometryDescriptor::setRadiusStride(NS::UInteger radiusStride) | |
| 1163 | +{ | |
| 1164 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setRadiusStride_), radiusStride); | |
| 1165 | +} | |
| 1166 | + | |
| 1167 | +_MTL_INLINE void MTL4::AccelerationStructureMotionCurveGeometryDescriptor::setSegmentControlPointCount(NS::UInteger segmentControlPointCount) | |
| 1168 | +{ | |
| 1169 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setSegmentControlPointCount_), segmentControlPointCount); | |
| 1170 | +} | |
| 1171 | + | |
| 1172 | +_MTL_INLINE void MTL4::AccelerationStructureMotionCurveGeometryDescriptor::setSegmentCount(NS::UInteger segmentCount) | |
| 1173 | +{ | |
| 1174 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setSegmentCount_), segmentCount); | |
| 1175 | +} | |
| 1176 | + | |
| 1177 | +_MTL_INLINE MTL4::InstanceAccelerationStructureDescriptor* MTL4::InstanceAccelerationStructureDescriptor::alloc() | |
| 1178 | +{ | |
| 1179 | + return NS::Object::alloc<MTL4::InstanceAccelerationStructureDescriptor>(_MTL_PRIVATE_CLS(MTL4InstanceAccelerationStructureDescriptor)); | |
| 1180 | +} | |
| 1181 | + | |
| 1182 | +_MTL_INLINE MTL4::InstanceAccelerationStructureDescriptor* MTL4::InstanceAccelerationStructureDescriptor::init() | |
| 1183 | +{ | |
| 1184 | + return NS::Object::init<MTL4::InstanceAccelerationStructureDescriptor>(); | |
| 1185 | +} | |
| 1186 | + | |
| 1187 | +_MTL_INLINE NS::UInteger MTL4::InstanceAccelerationStructureDescriptor::instanceCount() const | |
| 1188 | +{ | |
| 1189 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(instanceCount)); | |
| 1190 | +} | |
| 1191 | + | |
| 1192 | +_MTL_INLINE MTL4::BufferRange MTL4::InstanceAccelerationStructureDescriptor::instanceDescriptorBuffer() const | |
| 1193 | +{ | |
| 1194 | + return Object::sendMessage<MTL4::BufferRange>(this, _MTL_PRIVATE_SEL(instanceDescriptorBuffer)); | |
| 1195 | +} | |
| 1196 | + | |
| 1197 | +_MTL_INLINE NS::UInteger MTL4::InstanceAccelerationStructureDescriptor::instanceDescriptorStride() const | |
| 1198 | +{ | |
| 1199 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(instanceDescriptorStride)); | |
| 1200 | +} | |
| 1201 | + | |
| 1202 | +_MTL_INLINE MTL::AccelerationStructureInstanceDescriptorType MTL4::InstanceAccelerationStructureDescriptor::instanceDescriptorType() const | |
| 1203 | +{ | |
| 1204 | + return Object::sendMessage<MTL::AccelerationStructureInstanceDescriptorType>(this, _MTL_PRIVATE_SEL(instanceDescriptorType)); | |
| 1205 | +} | |
| 1206 | + | |
| 1207 | +_MTL_INLINE MTL::MatrixLayout MTL4::InstanceAccelerationStructureDescriptor::instanceTransformationMatrixLayout() const | |
| 1208 | +{ | |
| 1209 | + return Object::sendMessage<MTL::MatrixLayout>(this, _MTL_PRIVATE_SEL(instanceTransformationMatrixLayout)); | |
| 1210 | +} | |
| 1211 | + | |
| 1212 | +_MTL_INLINE MTL4::BufferRange MTL4::InstanceAccelerationStructureDescriptor::motionTransformBuffer() const | |
| 1213 | +{ | |
| 1214 | + return Object::sendMessage<MTL4::BufferRange>(this, _MTL_PRIVATE_SEL(motionTransformBuffer)); | |
| 1215 | +} | |
| 1216 | + | |
| 1217 | +_MTL_INLINE NS::UInteger MTL4::InstanceAccelerationStructureDescriptor::motionTransformCount() const | |
| 1218 | +{ | |
| 1219 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(motionTransformCount)); | |
| 1220 | +} | |
| 1221 | + | |
| 1222 | +_MTL_INLINE NS::UInteger MTL4::InstanceAccelerationStructureDescriptor::motionTransformStride() const | |
| 1223 | +{ | |
| 1224 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(motionTransformStride)); | |
| 1225 | +} | |
| 1226 | + | |
| 1227 | +_MTL_INLINE MTL::TransformType MTL4::InstanceAccelerationStructureDescriptor::motionTransformType() const | |
| 1228 | +{ | |
| 1229 | + return Object::sendMessage<MTL::TransformType>(this, _MTL_PRIVATE_SEL(motionTransformType)); | |
| 1230 | +} | |
| 1231 | + | |
| 1232 | +_MTL_INLINE void MTL4::InstanceAccelerationStructureDescriptor::setInstanceCount(NS::UInteger instanceCount) | |
| 1233 | +{ | |
| 1234 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setInstanceCount_), instanceCount); | |
| 1235 | +} | |
| 1236 | + | |
| 1237 | +_MTL_INLINE void MTL4::InstanceAccelerationStructureDescriptor::setInstanceDescriptorBuffer(const MTL4::BufferRange instanceDescriptorBuffer) | |
| 1238 | +{ | |
| 1239 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setInstanceDescriptorBuffer_), instanceDescriptorBuffer); | |
| 1240 | +} | |
| 1241 | + | |
| 1242 | +_MTL_INLINE void MTL4::InstanceAccelerationStructureDescriptor::setInstanceDescriptorStride(NS::UInteger instanceDescriptorStride) | |
| 1243 | +{ | |
| 1244 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setInstanceDescriptorStride_), instanceDescriptorStride); | |
| 1245 | +} | |
| 1246 | + | |
| 1247 | +_MTL_INLINE void MTL4::InstanceAccelerationStructureDescriptor::setInstanceDescriptorType(MTL::AccelerationStructureInstanceDescriptorType instanceDescriptorType) | |
| 1248 | +{ | |
| 1249 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setInstanceDescriptorType_), instanceDescriptorType); | |
| 1250 | +} | |
| 1251 | + | |
| 1252 | +_MTL_INLINE void MTL4::InstanceAccelerationStructureDescriptor::setInstanceTransformationMatrixLayout(MTL::MatrixLayout instanceTransformationMatrixLayout) | |
| 1253 | +{ | |
| 1254 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setInstanceTransformationMatrixLayout_), instanceTransformationMatrixLayout); | |
| 1255 | +} | |
| 1256 | + | |
| 1257 | +_MTL_INLINE void MTL4::InstanceAccelerationStructureDescriptor::setMotionTransformBuffer(const MTL4::BufferRange motionTransformBuffer) | |
| 1258 | +{ | |
| 1259 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMotionTransformBuffer_), motionTransformBuffer); | |
| 1260 | +} | |
| 1261 | + | |
| 1262 | +_MTL_INLINE void MTL4::InstanceAccelerationStructureDescriptor::setMotionTransformCount(NS::UInteger motionTransformCount) | |
| 1263 | +{ | |
| 1264 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMotionTransformCount_), motionTransformCount); | |
| 1265 | +} | |
| 1266 | + | |
| 1267 | +_MTL_INLINE void MTL4::InstanceAccelerationStructureDescriptor::setMotionTransformStride(NS::UInteger motionTransformStride) | |
| 1268 | +{ | |
| 1269 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMotionTransformStride_), motionTransformStride); | |
| 1270 | +} | |
| 1271 | + | |
| 1272 | +_MTL_INLINE void MTL4::InstanceAccelerationStructureDescriptor::setMotionTransformType(MTL::TransformType motionTransformType) | |
| 1273 | +{ | |
| 1274 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMotionTransformType_), motionTransformType); | |
| 1275 | +} | |
| 1276 | + | |
| 1277 | +_MTL_INLINE MTL4::IndirectInstanceAccelerationStructureDescriptor* MTL4::IndirectInstanceAccelerationStructureDescriptor::alloc() | |
| 1278 | +{ | |
| 1279 | + return NS::Object::alloc<MTL4::IndirectInstanceAccelerationStructureDescriptor>(_MTL_PRIVATE_CLS(MTL4IndirectInstanceAccelerationStructureDescriptor)); | |
| 1280 | +} | |
| 1281 | + | |
| 1282 | +_MTL_INLINE MTL4::IndirectInstanceAccelerationStructureDescriptor* MTL4::IndirectInstanceAccelerationStructureDescriptor::init() | |
| 1283 | +{ | |
| 1284 | + return NS::Object::init<MTL4::IndirectInstanceAccelerationStructureDescriptor>(); | |
| 1285 | +} | |
| 1286 | + | |
| 1287 | +_MTL_INLINE MTL4::BufferRange MTL4::IndirectInstanceAccelerationStructureDescriptor::instanceCountBuffer() const | |
| 1288 | +{ | |
| 1289 | + return Object::sendMessage<MTL4::BufferRange>(this, _MTL_PRIVATE_SEL(instanceCountBuffer)); | |
| 1290 | +} | |
| 1291 | + | |
| 1292 | +_MTL_INLINE MTL4::BufferRange MTL4::IndirectInstanceAccelerationStructureDescriptor::instanceDescriptorBuffer() const | |
| 1293 | +{ | |
| 1294 | + return Object::sendMessage<MTL4::BufferRange>(this, _MTL_PRIVATE_SEL(instanceDescriptorBuffer)); | |
| 1295 | +} | |
| 1296 | + | |
| 1297 | +_MTL_INLINE NS::UInteger MTL4::IndirectInstanceAccelerationStructureDescriptor::instanceDescriptorStride() const | |
| 1298 | +{ | |
| 1299 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(instanceDescriptorStride)); | |
| 1300 | +} | |
| 1301 | + | |
| 1302 | +_MTL_INLINE MTL::AccelerationStructureInstanceDescriptorType MTL4::IndirectInstanceAccelerationStructureDescriptor::instanceDescriptorType() const | |
| 1303 | +{ | |
| 1304 | + return Object::sendMessage<MTL::AccelerationStructureInstanceDescriptorType>(this, _MTL_PRIVATE_SEL(instanceDescriptorType)); | |
| 1305 | +} | |
| 1306 | + | |
| 1307 | +_MTL_INLINE MTL::MatrixLayout MTL4::IndirectInstanceAccelerationStructureDescriptor::instanceTransformationMatrixLayout() const | |
| 1308 | +{ | |
| 1309 | + return Object::sendMessage<MTL::MatrixLayout>(this, _MTL_PRIVATE_SEL(instanceTransformationMatrixLayout)); | |
| 1310 | +} | |
| 1311 | + | |
| 1312 | +_MTL_INLINE NS::UInteger MTL4::IndirectInstanceAccelerationStructureDescriptor::maxInstanceCount() const | |
| 1313 | +{ | |
| 1314 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(maxInstanceCount)); | |
| 1315 | +} | |
| 1316 | + | |
| 1317 | +_MTL_INLINE NS::UInteger MTL4::IndirectInstanceAccelerationStructureDescriptor::maxMotionTransformCount() const | |
| 1318 | +{ | |
| 1319 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(maxMotionTransformCount)); | |
| 1320 | +} | |
| 1321 | + | |
| 1322 | +_MTL_INLINE MTL4::BufferRange MTL4::IndirectInstanceAccelerationStructureDescriptor::motionTransformBuffer() const | |
| 1323 | +{ | |
| 1324 | + return Object::sendMessage<MTL4::BufferRange>(this, _MTL_PRIVATE_SEL(motionTransformBuffer)); | |
| 1325 | +} | |
| 1326 | + | |
| 1327 | +_MTL_INLINE MTL4::BufferRange MTL4::IndirectInstanceAccelerationStructureDescriptor::motionTransformCountBuffer() const | |
| 1328 | +{ | |
| 1329 | + return Object::sendMessage<MTL4::BufferRange>(this, _MTL_PRIVATE_SEL(motionTransformCountBuffer)); | |
| 1330 | +} | |
| 1331 | + | |
| 1332 | +_MTL_INLINE NS::UInteger MTL4::IndirectInstanceAccelerationStructureDescriptor::motionTransformStride() const | |
| 1333 | +{ | |
| 1334 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(motionTransformStride)); | |
| 1335 | +} | |
| 1336 | + | |
| 1337 | +_MTL_INLINE MTL::TransformType MTL4::IndirectInstanceAccelerationStructureDescriptor::motionTransformType() const | |
| 1338 | +{ | |
| 1339 | + return Object::sendMessage<MTL::TransformType>(this, _MTL_PRIVATE_SEL(motionTransformType)); | |
| 1340 | +} | |
| 1341 | + | |
| 1342 | +_MTL_INLINE void MTL4::IndirectInstanceAccelerationStructureDescriptor::setInstanceCountBuffer(const MTL4::BufferRange instanceCountBuffer) | |
| 1343 | +{ | |
| 1344 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setInstanceCountBuffer_), instanceCountBuffer); | |
| 1345 | +} | |
| 1346 | + | |
| 1347 | +_MTL_INLINE void MTL4::IndirectInstanceAccelerationStructureDescriptor::setInstanceDescriptorBuffer(const MTL4::BufferRange instanceDescriptorBuffer) | |
| 1348 | +{ | |
| 1349 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setInstanceDescriptorBuffer_), instanceDescriptorBuffer); | |
| 1350 | +} | |
| 1351 | + | |
| 1352 | +_MTL_INLINE void MTL4::IndirectInstanceAccelerationStructureDescriptor::setInstanceDescriptorStride(NS::UInteger instanceDescriptorStride) | |
| 1353 | +{ | |
| 1354 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setInstanceDescriptorStride_), instanceDescriptorStride); | |
| 1355 | +} | |
| 1356 | + | |
| 1357 | +_MTL_INLINE void MTL4::IndirectInstanceAccelerationStructureDescriptor::setInstanceDescriptorType(MTL::AccelerationStructureInstanceDescriptorType instanceDescriptorType) | |
| 1358 | +{ | |
| 1359 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setInstanceDescriptorType_), instanceDescriptorType); | |
| 1360 | +} | |
| 1361 | + | |
| 1362 | +_MTL_INLINE void MTL4::IndirectInstanceAccelerationStructureDescriptor::setInstanceTransformationMatrixLayout(MTL::MatrixLayout instanceTransformationMatrixLayout) | |
| 1363 | +{ | |
| 1364 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setInstanceTransformationMatrixLayout_), instanceTransformationMatrixLayout); | |
| 1365 | +} | |
| 1366 | + | |
| 1367 | +_MTL_INLINE void MTL4::IndirectInstanceAccelerationStructureDescriptor::setMaxInstanceCount(NS::UInteger maxInstanceCount) | |
| 1368 | +{ | |
| 1369 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMaxInstanceCount_), maxInstanceCount); | |
| 1370 | +} | |
| 1371 | + | |
| 1372 | +_MTL_INLINE void MTL4::IndirectInstanceAccelerationStructureDescriptor::setMaxMotionTransformCount(NS::UInteger maxMotionTransformCount) | |
| 1373 | +{ | |
| 1374 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMaxMotionTransformCount_), maxMotionTransformCount); | |
| 1375 | +} | |
| 1376 | + | |
| 1377 | +_MTL_INLINE void MTL4::IndirectInstanceAccelerationStructureDescriptor::setMotionTransformBuffer(const MTL4::BufferRange motionTransformBuffer) | |
| 1378 | +{ | |
| 1379 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMotionTransformBuffer_), motionTransformBuffer); | |
| 1380 | +} | |
| 1381 | + | |
| 1382 | +_MTL_INLINE void MTL4::IndirectInstanceAccelerationStructureDescriptor::setMotionTransformCountBuffer(const MTL4::BufferRange motionTransformCountBuffer) | |
| 1383 | +{ | |
| 1384 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMotionTransformCountBuffer_), motionTransformCountBuffer); | |
| 1385 | +} | |
| 1386 | + | |
| 1387 | +_MTL_INLINE void MTL4::IndirectInstanceAccelerationStructureDescriptor::setMotionTransformStride(NS::UInteger motionTransformStride) | |
| 1388 | +{ | |
| 1389 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMotionTransformStride_), motionTransformStride); | |
| 1390 | +} | |
| 1391 | + | |
| 1392 | +_MTL_INLINE void MTL4::IndirectInstanceAccelerationStructureDescriptor::setMotionTransformType(MTL::TransformType motionTransformType) | |
| 1393 | +{ | |
| 1394 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMotionTransformType_), motionTransformType); | |
| 1395 | +} | |
added
third_party/metal-cpp/Metal/MTL4Archive.hpp
+93 −0
@@ -0,0 +1,93 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Metal/MTL4Archive.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2025 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include "../Foundation/Foundation.hpp" | |
| 24 | +#include "MTLDefines.hpp" | |
| 25 | +#include "MTLHeaderBridge.hpp" | |
| 26 | +#include "MTLPrivate.hpp" | |
| 27 | + | |
| 28 | +namespace MTL | |
| 29 | +{ | |
| 30 | +class ComputePipelineState; | |
| 31 | +class RenderPipelineState; | |
| 32 | +} | |
| 33 | + | |
| 34 | +namespace MTL4 | |
| 35 | +{ | |
| 36 | +class BinaryFunction; | |
| 37 | +class BinaryFunctionDescriptor; | |
| 38 | +class ComputePipelineDescriptor; | |
| 39 | +class PipelineDescriptor; | |
| 40 | +class PipelineStageDynamicLinkingDescriptor; | |
| 41 | +class RenderPipelineDynamicLinkingDescriptor; | |
| 42 | + | |
| 43 | +class Archive : public NS::Referencing<Archive> | |
| 44 | +{ | |
| 45 | +public: | |
| 46 | + NS::String* label() const; | |
| 47 | + | |
| 48 | + BinaryFunction* newBinaryFunction(const MTL4::BinaryFunctionDescriptor* descriptor, NS::Error** error); | |
| 49 | + | |
| 50 | + MTL::ComputePipelineState* newComputePipelineState(const MTL4::ComputePipelineDescriptor* descriptor, NS::Error** error); | |
| 51 | + MTL::ComputePipelineState* newComputePipelineState(const MTL4::ComputePipelineDescriptor* descriptor, const MTL4::PipelineStageDynamicLinkingDescriptor* dynamicLinkingDescriptor, NS::Error** error); | |
| 52 | + | |
| 53 | + MTL::RenderPipelineState* newRenderPipelineState(const MTL4::PipelineDescriptor* descriptor, NS::Error** error); | |
| 54 | + MTL::RenderPipelineState* newRenderPipelineState(const MTL4::PipelineDescriptor* descriptor, const MTL4::RenderPipelineDynamicLinkingDescriptor* dynamicLinkingDescriptor, NS::Error** error); | |
| 55 | + | |
| 56 | + void setLabel(const NS::String* label); | |
| 57 | +}; | |
| 58 | + | |
| 59 | +} | |
| 60 | +_MTL_INLINE NS::String* MTL4::Archive::label() const | |
| 61 | +{ | |
| 62 | + return Object::sendMessage<NS::String*>(this, _MTL_PRIVATE_SEL(label)); | |
| 63 | +} | |
| 64 | + | |
| 65 | +_MTL_INLINE MTL4::BinaryFunction* MTL4::Archive::newBinaryFunction(const MTL4::BinaryFunctionDescriptor* descriptor, NS::Error** error) | |
| 66 | +{ | |
| 67 | + return Object::sendMessage<MTL4::BinaryFunction*>(this, _MTL_PRIVATE_SEL(newBinaryFunctionWithDescriptor_error_), descriptor, error); | |
| 68 | +} | |
| 69 | + | |
| 70 | +_MTL_INLINE MTL::ComputePipelineState* MTL4::Archive::newComputePipelineState(const MTL4::ComputePipelineDescriptor* descriptor, NS::Error** error) | |
| 71 | +{ | |
| 72 | + return Object::sendMessage<MTL::ComputePipelineState*>(this, _MTL_PRIVATE_SEL(newComputePipelineStateWithDescriptor_error_), descriptor, error); | |
| 73 | +} | |
| 74 | + | |
| 75 | +_MTL_INLINE MTL::ComputePipelineState* MTL4::Archive::newComputePipelineState(const MTL4::ComputePipelineDescriptor* descriptor, const MTL4::PipelineStageDynamicLinkingDescriptor* dynamicLinkingDescriptor, NS::Error** error) | |
| 76 | +{ | |
| 77 | + return Object::sendMessage<MTL::ComputePipelineState*>(this, _MTL_PRIVATE_SEL(newComputePipelineStateWithDescriptor_dynamicLinkingDescriptor_error_), descriptor, dynamicLinkingDescriptor, error); | |
| 78 | +} | |
| 79 | + | |
| 80 | +_MTL_INLINE MTL::RenderPipelineState* MTL4::Archive::newRenderPipelineState(const MTL4::PipelineDescriptor* descriptor, NS::Error** error) | |
| 81 | +{ | |
| 82 | + return Object::sendMessage<MTL::RenderPipelineState*>(this, _MTL_PRIVATE_SEL(newRenderPipelineStateWithDescriptor_error_), descriptor, error); | |
| 83 | +} | |
| 84 | + | |
| 85 | +_MTL_INLINE MTL::RenderPipelineState* MTL4::Archive::newRenderPipelineState(const MTL4::PipelineDescriptor* descriptor, const MTL4::RenderPipelineDynamicLinkingDescriptor* dynamicLinkingDescriptor, NS::Error** error) | |
| 86 | +{ | |
| 87 | + return Object::sendMessage<MTL::RenderPipelineState*>(this, _MTL_PRIVATE_SEL(newRenderPipelineStateWithDescriptor_dynamicLinkingDescriptor_error_), descriptor, dynamicLinkingDescriptor, error); | |
| 88 | +} | |
| 89 | + | |
| 90 | +_MTL_INLINE void MTL4::Archive::setLabel(const NS::String* label) | |
| 91 | +{ | |
| 92 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setLabel_), label); | |
| 93 | +} | |
added
third_party/metal-cpp/Metal/MTL4ArgumentTable.hpp
+187 −0
@@ -0,0 +1,187 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Metal/MTL4ArgumentTable.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2025 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include "../Foundation/Foundation.hpp" | |
| 24 | +#include "MTLDefines.hpp" | |
| 25 | +#include "MTLGPUAddress.hpp" | |
| 26 | +#include "MTLHeaderBridge.hpp" | |
| 27 | +#include "MTLPrivate.hpp" | |
| 28 | +#include "MTLTypes.hpp" | |
| 29 | + | |
| 30 | +namespace MTL | |
| 31 | +{ | |
| 32 | +class Device; | |
| 33 | +} | |
| 34 | + | |
| 35 | +namespace MTL4 | |
| 36 | +{ | |
| 37 | +class ArgumentTableDescriptor : public NS::Copying<ArgumentTableDescriptor> | |
| 38 | +{ | |
| 39 | +public: | |
| 40 | + static ArgumentTableDescriptor* alloc(); | |
| 41 | + | |
| 42 | + ArgumentTableDescriptor* init(); | |
| 43 | + bool initializeBindings() const; | |
| 44 | + | |
| 45 | + NS::String* label() const; | |
| 46 | + | |
| 47 | + NS::UInteger maxBufferBindCount() const; | |
| 48 | + | |
| 49 | + NS::UInteger maxSamplerStateBindCount() const; | |
| 50 | + | |
| 51 | + NS::UInteger maxTextureBindCount() const; | |
| 52 | + | |
| 53 | + void setInitializeBindings(bool initializeBindings); | |
| 54 | + | |
| 55 | + void setLabel(const NS::String* label); | |
| 56 | + | |
| 57 | + void setMaxBufferBindCount(NS::UInteger maxBufferBindCount); | |
| 58 | + | |
| 59 | + void setMaxSamplerStateBindCount(NS::UInteger maxSamplerStateBindCount); | |
| 60 | + | |
| 61 | + void setMaxTextureBindCount(NS::UInteger maxTextureBindCount); | |
| 62 | + | |
| 63 | + void setSupportAttributeStrides(bool supportAttributeStrides); | |
| 64 | + bool supportAttributeStrides() const; | |
| 65 | +}; | |
| 66 | +class ArgumentTable : public NS::Referencing<ArgumentTable> | |
| 67 | +{ | |
| 68 | +public: | |
| 69 | + MTL::Device* device() const; | |
| 70 | + | |
| 71 | + NS::String* label() const; | |
| 72 | + | |
| 73 | + void setAddress(MTL::GPUAddress gpuAddress, NS::UInteger bindingIndex); | |
| 74 | + void setAddress(MTL::GPUAddress gpuAddress, NS::UInteger stride, NS::UInteger bindingIndex); | |
| 75 | + | |
| 76 | + void setResource(MTL::ResourceID resourceID, NS::UInteger bindingIndex); | |
| 77 | + | |
| 78 | + void setSamplerState(MTL::ResourceID resourceID, NS::UInteger bindingIndex); | |
| 79 | + | |
| 80 | + void setTexture(MTL::ResourceID resourceID, NS::UInteger bindingIndex); | |
| 81 | +}; | |
| 82 | + | |
| 83 | +} | |
| 84 | +_MTL_INLINE MTL4::ArgumentTableDescriptor* MTL4::ArgumentTableDescriptor::alloc() | |
| 85 | +{ | |
| 86 | + return NS::Object::alloc<MTL4::ArgumentTableDescriptor>(_MTL_PRIVATE_CLS(MTL4ArgumentTableDescriptor)); | |
| 87 | +} | |
| 88 | + | |
| 89 | +_MTL_INLINE MTL4::ArgumentTableDescriptor* MTL4::ArgumentTableDescriptor::init() | |
| 90 | +{ | |
| 91 | + return NS::Object::init<MTL4::ArgumentTableDescriptor>(); | |
| 92 | +} | |
| 93 | + | |
| 94 | +_MTL_INLINE bool MTL4::ArgumentTableDescriptor::initializeBindings() const | |
| 95 | +{ | |
| 96 | + return Object::sendMessage<bool>(this, _MTL_PRIVATE_SEL(initializeBindings)); | |
| 97 | +} | |
| 98 | + | |
| 99 | +_MTL_INLINE NS::String* MTL4::ArgumentTableDescriptor::label() const | |
| 100 | +{ | |
| 101 | + return Object::sendMessage<NS::String*>(this, _MTL_PRIVATE_SEL(label)); | |
| 102 | +} | |
| 103 | + | |
| 104 | +_MTL_INLINE NS::UInteger MTL4::ArgumentTableDescriptor::maxBufferBindCount() const | |
| 105 | +{ | |
| 106 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(maxBufferBindCount)); | |
| 107 | +} | |
| 108 | + | |
| 109 | +_MTL_INLINE NS::UInteger MTL4::ArgumentTableDescriptor::maxSamplerStateBindCount() const | |
| 110 | +{ | |
| 111 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(maxSamplerStateBindCount)); | |
| 112 | +} | |
| 113 | + | |
| 114 | +_MTL_INLINE NS::UInteger MTL4::ArgumentTableDescriptor::maxTextureBindCount() const | |
| 115 | +{ | |
| 116 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(maxTextureBindCount)); | |
| 117 | +} | |
| 118 | + | |
| 119 | +_MTL_INLINE void MTL4::ArgumentTableDescriptor::setInitializeBindings(bool initializeBindings) | |
| 120 | +{ | |
| 121 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setInitializeBindings_), initializeBindings); | |
| 122 | +} | |
| 123 | + | |
| 124 | +_MTL_INLINE void MTL4::ArgumentTableDescriptor::setLabel(const NS::String* label) | |
| 125 | +{ | |
| 126 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setLabel_), label); | |
| 127 | +} | |
| 128 | + | |
| 129 | +_MTL_INLINE void MTL4::ArgumentTableDescriptor::setMaxBufferBindCount(NS::UInteger maxBufferBindCount) | |
| 130 | +{ | |
| 131 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMaxBufferBindCount_), maxBufferBindCount); | |
| 132 | +} | |
| 133 | + | |
| 134 | +_MTL_INLINE void MTL4::ArgumentTableDescriptor::setMaxSamplerStateBindCount(NS::UInteger maxSamplerStateBindCount) | |
| 135 | +{ | |
| 136 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMaxSamplerStateBindCount_), maxSamplerStateBindCount); | |
| 137 | +} | |
| 138 | + | |
| 139 | +_MTL_INLINE void MTL4::ArgumentTableDescriptor::setMaxTextureBindCount(NS::UInteger maxTextureBindCount) | |
| 140 | +{ | |
| 141 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMaxTextureBindCount_), maxTextureBindCount); | |
| 142 | +} | |
| 143 | + | |
| 144 | +_MTL_INLINE void MTL4::ArgumentTableDescriptor::setSupportAttributeStrides(bool supportAttributeStrides) | |
| 145 | +{ | |
| 146 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setSupportAttributeStrides_), supportAttributeStrides); | |
| 147 | +} | |
| 148 | + | |
| 149 | +_MTL_INLINE bool MTL4::ArgumentTableDescriptor::supportAttributeStrides() const | |
| 150 | +{ | |
| 151 | + return Object::sendMessageSafe<bool>(this, _MTL_PRIVATE_SEL(supportAttributeStrides)); | |
| 152 | +} | |
| 153 | + | |
| 154 | +_MTL_INLINE MTL::Device* MTL4::ArgumentTable::device() const | |
| 155 | +{ | |
| 156 | + return Object::sendMessage<MTL::Device*>(this, _MTL_PRIVATE_SEL(device)); | |
| 157 | +} | |
| 158 | + | |
| 159 | +_MTL_INLINE NS::String* MTL4::ArgumentTable::label() const | |
| 160 | +{ | |
| 161 | + return Object::sendMessage<NS::String*>(this, _MTL_PRIVATE_SEL(label)); | |
| 162 | +} | |
| 163 | + | |
| 164 | +_MTL_INLINE void MTL4::ArgumentTable::setAddress(MTL::GPUAddress gpuAddress, NS::UInteger bindingIndex) | |
| 165 | +{ | |
| 166 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setAddress_atIndex_), gpuAddress, bindingIndex); | |
| 167 | +} | |
| 168 | + | |
| 169 | +_MTL_INLINE void MTL4::ArgumentTable::setAddress(MTL::GPUAddress gpuAddress, NS::UInteger stride, NS::UInteger bindingIndex) | |
| 170 | +{ | |
| 171 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setAddress_attributeStride_atIndex_), gpuAddress, stride, bindingIndex); | |
| 172 | +} | |
| 173 | + | |
| 174 | +_MTL_INLINE void MTL4::ArgumentTable::setResource(MTL::ResourceID resourceID, NS::UInteger bindingIndex) | |
| 175 | +{ | |
| 176 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setResource_atBufferIndex_), resourceID, bindingIndex); | |
| 177 | +} | |
| 178 | + | |
| 179 | +_MTL_INLINE void MTL4::ArgumentTable::setSamplerState(MTL::ResourceID resourceID, NS::UInteger bindingIndex) | |
| 180 | +{ | |
| 181 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setSamplerState_atIndex_), resourceID, bindingIndex); | |
| 182 | +} | |
| 183 | + | |
| 184 | +_MTL_INLINE void MTL4::ArgumentTable::setTexture(MTL::ResourceID resourceID, NS::UInteger bindingIndex) | |
| 185 | +{ | |
| 186 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setTexture_atIndex_), resourceID, bindingIndex); | |
| 187 | +} | |
added
third_party/metal-cpp/Metal/MTL4BinaryFunction.hpp
+50 −0
@@ -0,0 +1,50 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Metal/MTL4BinaryFunction.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2025 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include "../Foundation/Foundation.hpp" | |
| 24 | +#include "MTLDefines.hpp" | |
| 25 | +#include "MTLHeaderBridge.hpp" | |
| 26 | +#include "MTLLibrary.hpp" | |
| 27 | +#include "MTLPrivate.hpp" | |
| 28 | + | |
| 29 | +namespace MTL4 | |
| 30 | +{ | |
| 31 | + | |
| 32 | +class BinaryFunction : public NS::Referencing<BinaryFunction> | |
| 33 | +{ | |
| 34 | +public: | |
| 35 | + MTL::FunctionType functionType() const; | |
| 36 | + | |
| 37 | + NS::String* name() const; | |
| 38 | +}; | |
| 39 | + | |
| 40 | +} | |
| 41 | + | |
| 42 | +_MTL_INLINE MTL::FunctionType MTL4::BinaryFunction::functionType() const | |
| 43 | +{ | |
| 44 | + return Object::sendMessage<MTL::FunctionType>(this, _MTL_PRIVATE_SEL(functionType)); | |
| 45 | +} | |
| 46 | + | |
| 47 | +_MTL_INLINE NS::String* MTL4::BinaryFunction::name() const | |
| 48 | +{ | |
| 49 | + return Object::sendMessage<NS::String*>(this, _MTL_PRIVATE_SEL(name)); | |
| 50 | +} | |
added
third_party/metal-cpp/Metal/MTL4BinaryFunctionDescriptor.hpp
+97 −0
@@ -0,0 +1,97 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Metal/MTL4BinaryFunctionDescriptor.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2025 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include "../Foundation/Foundation.hpp" | |
| 24 | +#include "MTLDefines.hpp" | |
| 25 | +#include "MTLHeaderBridge.hpp" | |
| 26 | +#include "MTLPrivate.hpp" | |
| 27 | + | |
| 28 | +namespace MTL4 | |
| 29 | +{ | |
| 30 | +class BinaryFunctionDescriptor; | |
| 31 | +class FunctionDescriptor; | |
| 32 | + | |
| 33 | +_MTL_OPTIONS(NS::UInteger, BinaryFunctionOptions) { | |
| 34 | + BinaryFunctionOptionNone = 0, | |
| 35 | + BinaryFunctionOptionPipelineIndependent = 1 << 1, | |
| 36 | +}; | |
| 37 | + | |
| 38 | +class BinaryFunctionDescriptor : public NS::Copying<BinaryFunctionDescriptor> | |
| 39 | +{ | |
| 40 | +public: | |
| 41 | + static BinaryFunctionDescriptor* alloc(); | |
| 42 | + | |
| 43 | + FunctionDescriptor* functionDescriptor() const; | |
| 44 | + | |
| 45 | + BinaryFunctionDescriptor* init(); | |
| 46 | + | |
| 47 | + NS::String* name() const; | |
| 48 | + | |
| 49 | + BinaryFunctionOptions options() const; | |
| 50 | + | |
| 51 | + void setFunctionDescriptor(const MTL4::FunctionDescriptor* functionDescriptor); | |
| 52 | + | |
| 53 | + void setName(const NS::String* name); | |
| 54 | + | |
| 55 | + void setOptions(MTL4::BinaryFunctionOptions options); | |
| 56 | +}; | |
| 57 | + | |
| 58 | +} | |
| 59 | +_MTL_INLINE MTL4::BinaryFunctionDescriptor* MTL4::BinaryFunctionDescriptor::alloc() | |
| 60 | +{ | |
| 61 | + return NS::Object::alloc<MTL4::BinaryFunctionDescriptor>(_MTL_PRIVATE_CLS(MTL4BinaryFunctionDescriptor)); | |
| 62 | +} | |
| 63 | + | |
| 64 | +_MTL_INLINE MTL4::FunctionDescriptor* MTL4::BinaryFunctionDescriptor::functionDescriptor() const | |
| 65 | +{ | |
| 66 | + return Object::sendMessage<MTL4::FunctionDescriptor*>(this, _MTL_PRIVATE_SEL(functionDescriptor)); | |
| 67 | +} | |
| 68 | + | |
| 69 | +_MTL_INLINE MTL4::BinaryFunctionDescriptor* MTL4::BinaryFunctionDescriptor::init() | |
| 70 | +{ | |
| 71 | + return NS::Object::init<MTL4::BinaryFunctionDescriptor>(); | |
| 72 | +} | |
| 73 | + | |
| 74 | +_MTL_INLINE NS::String* MTL4::BinaryFunctionDescriptor::name() const | |
| 75 | +{ | |
| 76 | + return Object::sendMessage<NS::String*>(this, _MTL_PRIVATE_SEL(name)); | |
| 77 | +} | |
| 78 | + | |
| 79 | +_MTL_INLINE MTL4::BinaryFunctionOptions MTL4::BinaryFunctionDescriptor::options() const | |
| 80 | +{ | |
| 81 | + return Object::sendMessage<MTL4::BinaryFunctionOptions>(this, _MTL_PRIVATE_SEL(options)); | |
| 82 | +} | |
| 83 | + | |
| 84 | +_MTL_INLINE void MTL4::BinaryFunctionDescriptor::setFunctionDescriptor(const MTL4::FunctionDescriptor* functionDescriptor) | |
| 85 | +{ | |
| 86 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setFunctionDescriptor_), functionDescriptor); | |
| 87 | +} | |
| 88 | + | |
| 89 | +_MTL_INLINE void MTL4::BinaryFunctionDescriptor::setName(const NS::String* name) | |
| 90 | +{ | |
| 91 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setName_), name); | |
| 92 | +} | |
| 93 | + | |
| 94 | +_MTL_INLINE void MTL4::BinaryFunctionDescriptor::setOptions(MTL4::BinaryFunctionOptions options) | |
| 95 | +{ | |
| 96 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setOptions_), options); | |
| 97 | +} | |
added
third_party/metal-cpp/Metal/MTL4CommandAllocator.hpp
+100 −0
@@ -0,0 +1,100 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Metal/MTL4CommandAllocator.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2025 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include "../Foundation/Foundation.hpp" | |
| 24 | +#include "MTLDefines.hpp" | |
| 25 | +#include "MTLHeaderBridge.hpp" | |
| 26 | +#include "MTLPrivate.hpp" | |
| 27 | +#include <cstdint> | |
| 28 | + | |
| 29 | +namespace MTL | |
| 30 | +{ | |
| 31 | +class Device; | |
| 32 | +} | |
| 33 | + | |
| 34 | +namespace MTL4 | |
| 35 | +{ | |
| 36 | + | |
| 37 | +class CommandAllocatorDescriptor : public NS::Copying<CommandAllocatorDescriptor> | |
| 38 | +{ | |
| 39 | +public: | |
| 40 | + static CommandAllocatorDescriptor* alloc(); | |
| 41 | + | |
| 42 | + CommandAllocatorDescriptor* init(); | |
| 43 | + | |
| 44 | + NS::String* label() const; | |
| 45 | + void setLabel(const NS::String* label); | |
| 46 | +}; | |
| 47 | + | |
| 48 | +class CommandAllocator : public NS::Referencing<CommandAllocator> | |
| 49 | +{ | |
| 50 | +public: | |
| 51 | + uint64_t allocatedSize(); | |
| 52 | + | |
| 53 | + MTL::Device* device() const; | |
| 54 | + | |
| 55 | + NS::String* label() const; | |
| 56 | + | |
| 57 | + void reset(); | |
| 58 | +}; | |
| 59 | + | |
| 60 | +} | |
| 61 | + | |
| 62 | +_MTL_INLINE MTL4::CommandAllocatorDescriptor* MTL4::CommandAllocatorDescriptor::alloc() | |
| 63 | +{ | |
| 64 | + return NS::Object::alloc<MTL4::CommandAllocatorDescriptor>(_MTL_PRIVATE_CLS(MTL4CommandAllocatorDescriptor)); | |
| 65 | +} | |
| 66 | + | |
| 67 | +_MTL_INLINE MTL4::CommandAllocatorDescriptor* MTL4::CommandAllocatorDescriptor::init() | |
| 68 | +{ | |
| 69 | + return NS::Object::init<MTL4::CommandAllocatorDescriptor>(); | |
| 70 | +} | |
| 71 | + | |
| 72 | +_MTL_INLINE NS::String* MTL4::CommandAllocatorDescriptor::label() const | |
| 73 | +{ | |
| 74 | + return Object::sendMessage<NS::String*>(this, _MTL_PRIVATE_SEL(label)); | |
| 75 | +} | |
| 76 | + | |
| 77 | +_MTL_INLINE void MTL4::CommandAllocatorDescriptor::setLabel(const NS::String* label) | |
| 78 | +{ | |
| 79 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setLabel_), label); | |
| 80 | +} | |
| 81 | + | |
| 82 | +_MTL_INLINE uint64_t MTL4::CommandAllocator::allocatedSize() | |
| 83 | +{ | |
| 84 | + return Object::sendMessage<uint64_t>(this, _MTL_PRIVATE_SEL(allocatedSize)); | |
| 85 | +} | |
| 86 | + | |
| 87 | +_MTL_INLINE MTL::Device* MTL4::CommandAllocator::device() const | |
| 88 | +{ | |
| 89 | + return Object::sendMessage<MTL::Device*>(this, _MTL_PRIVATE_SEL(device)); | |
| 90 | +} | |
| 91 | + | |
| 92 | +_MTL_INLINE NS::String* MTL4::CommandAllocator::label() const | |
| 93 | +{ | |
| 94 | + return Object::sendMessage<NS::String*>(this, _MTL_PRIVATE_SEL(label)); | |
| 95 | +} | |
| 96 | + | |
| 97 | +_MTL_INLINE void MTL4::CommandAllocator::reset() | |
| 98 | +{ | |
| 99 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(reset)); | |
| 100 | +} | |
added
third_party/metal-cpp/Metal/MTL4CommandBuffer.hpp
+193 −0
@@ -0,0 +1,193 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Metal/MTL4CommandBuffer.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2025 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include "../Foundation/Foundation.hpp" | |
| 24 | +#include "MTL4RenderCommandEncoder.hpp" | |
| 25 | +#include "MTLAccelerationStructureTypes.hpp" | |
| 26 | +#include "MTLDefines.hpp" | |
| 27 | +#include "MTLHeaderBridge.hpp" | |
| 28 | +#include "MTLPrivate.hpp" | |
| 29 | + | |
| 30 | +namespace MTL4 | |
| 31 | +{ | |
| 32 | +class CommandAllocator; | |
| 33 | +class CommandBufferOptions; | |
| 34 | +class ComputeCommandEncoder; | |
| 35 | +class CounterHeap; | |
| 36 | +class MachineLearningCommandEncoder; | |
| 37 | +class RenderCommandEncoder; | |
| 38 | +class RenderPassDescriptor; | |
| 39 | +} | |
| 40 | + | |
| 41 | +namespace MTL | |
| 42 | +{ | |
| 43 | +class Device; | |
| 44 | +class Fence; | |
| 45 | +class LogState; | |
| 46 | +class ResidencySet; | |
| 47 | +} | |
| 48 | + | |
| 49 | +namespace MTL4 | |
| 50 | +{ | |
| 51 | +class CommandBufferOptions : public NS::Copying<CommandBufferOptions> | |
| 52 | +{ | |
| 53 | +public: | |
| 54 | + static CommandBufferOptions* alloc(); | |
| 55 | + | |
| 56 | + CommandBufferOptions* init(); | |
| 57 | + | |
| 58 | + MTL::LogState* logState() const; | |
| 59 | + void setLogState(const MTL::LogState* logState); | |
| 60 | +}; | |
| 61 | +class CommandBuffer : public NS::Referencing<CommandBuffer> | |
| 62 | +{ | |
| 63 | +public: | |
| 64 | + void beginCommandBuffer(const MTL4::CommandAllocator* allocator); | |
| 65 | + void beginCommandBuffer(const MTL4::CommandAllocator* allocator, const MTL4::CommandBufferOptions* options); | |
| 66 | + | |
| 67 | + ComputeCommandEncoder* computeCommandEncoder(); | |
| 68 | + | |
| 69 | + MTL::Device* device() const; | |
| 70 | + | |
| 71 | + void endCommandBuffer(); | |
| 72 | + | |
| 73 | + NS::String* label() const; | |
| 74 | + | |
| 75 | + MachineLearningCommandEncoder* machineLearningCommandEncoder(); | |
| 76 | + | |
| 77 | + void popDebugGroup(); | |
| 78 | + | |
| 79 | + void pushDebugGroup(const NS::String* string); | |
| 80 | + | |
| 81 | + RenderCommandEncoder* renderCommandEncoder(const MTL4::RenderPassDescriptor* descriptor); | |
| 82 | + RenderCommandEncoder* renderCommandEncoder(const MTL4::RenderPassDescriptor* descriptor, MTL4::RenderEncoderOptions options); | |
| 83 | + | |
| 84 | + void resolveCounterHeap(const MTL4::CounterHeap* counterHeap, NS::Range range, const MTL4::BufferRange bufferRange, const MTL::Fence* fenceToWait, const MTL::Fence* fenceToUpdate); | |
| 85 | + | |
| 86 | + void setLabel(const NS::String* label); | |
| 87 | + | |
| 88 | + void useResidencySet(const MTL::ResidencySet* residencySet); | |
| 89 | + void useResidencySets(const MTL::ResidencySet* const residencySets[], NS::UInteger count); | |
| 90 | + | |
| 91 | + void writeTimestampIntoHeap(const MTL4::CounterHeap* counterHeap, NS::UInteger index); | |
| 92 | +}; | |
| 93 | + | |
| 94 | +} | |
| 95 | +_MTL_INLINE MTL4::CommandBufferOptions* MTL4::CommandBufferOptions::alloc() | |
| 96 | +{ | |
| 97 | + return NS::Object::alloc<MTL4::CommandBufferOptions>(_MTL_PRIVATE_CLS(MTL4CommandBufferOptions)); | |
| 98 | +} | |
| 99 | + | |
| 100 | +_MTL_INLINE MTL4::CommandBufferOptions* MTL4::CommandBufferOptions::init() | |
| 101 | +{ | |
| 102 | + return NS::Object::init<MTL4::CommandBufferOptions>(); | |
| 103 | +} | |
| 104 | + | |
| 105 | +_MTL_INLINE MTL::LogState* MTL4::CommandBufferOptions::logState() const | |
| 106 | +{ | |
| 107 | + return Object::sendMessage<MTL::LogState*>(this, _MTL_PRIVATE_SEL(logState)); | |
| 108 | +} | |
| 109 | + | |
| 110 | +_MTL_INLINE void MTL4::CommandBufferOptions::setLogState(const MTL::LogState* logState) | |
| 111 | +{ | |
| 112 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setLogState_), logState); | |
| 113 | +} | |
| 114 | + | |
| 115 | +_MTL_INLINE void MTL4::CommandBuffer::beginCommandBuffer(const MTL4::CommandAllocator* allocator) | |
| 116 | +{ | |
| 117 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(beginCommandBufferWithAllocator_), allocator); | |
| 118 | +} | |
| 119 | + | |
| 120 | +_MTL_INLINE void MTL4::CommandBuffer::beginCommandBuffer(const MTL4::CommandAllocator* allocator, const MTL4::CommandBufferOptions* options) | |
| 121 | +{ | |
| 122 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(beginCommandBufferWithAllocator_options_), allocator, options); | |
| 123 | +} | |
| 124 | + | |
| 125 | +_MTL_INLINE MTL4::ComputeCommandEncoder* MTL4::CommandBuffer::computeCommandEncoder() | |
| 126 | +{ | |
| 127 | + return Object::sendMessage<MTL4::ComputeCommandEncoder*>(this, _MTL_PRIVATE_SEL(computeCommandEncoder)); | |
| 128 | +} | |
| 129 | + | |
| 130 | +_MTL_INLINE MTL::Device* MTL4::CommandBuffer::device() const | |
| 131 | +{ | |
| 132 | + return Object::sendMessage<MTL::Device*>(this, _MTL_PRIVATE_SEL(device)); | |
| 133 | +} | |
| 134 | + | |
| 135 | +_MTL_INLINE void MTL4::CommandBuffer::endCommandBuffer() | |
| 136 | +{ | |
| 137 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(endCommandBuffer)); | |
| 138 | +} | |
| 139 | + | |
| 140 | +_MTL_INLINE NS::String* MTL4::CommandBuffer::label() const | |
| 141 | +{ | |
| 142 | + return Object::sendMessage<NS::String*>(this, _MTL_PRIVATE_SEL(label)); | |
| 143 | +} | |
| 144 | + | |
| 145 | +_MTL_INLINE MTL4::MachineLearningCommandEncoder* MTL4::CommandBuffer::machineLearningCommandEncoder() | |
| 146 | +{ | |
| 147 | + return Object::sendMessage<MTL4::MachineLearningCommandEncoder*>(this, _MTL_PRIVATE_SEL(machineLearningCommandEncoder)); | |
| 148 | +} | |
| 149 | + | |
| 150 | +_MTL_INLINE void MTL4::CommandBuffer::popDebugGroup() | |
| 151 | +{ | |
| 152 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(popDebugGroup)); | |
| 153 | +} | |
| 154 | + | |
| 155 | +_MTL_INLINE void MTL4::CommandBuffer::pushDebugGroup(const NS::String* string) | |
| 156 | +{ | |
| 157 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(pushDebugGroup_), string); | |
| 158 | +} | |
| 159 | + | |
| 160 | +_MTL_INLINE MTL4::RenderCommandEncoder* MTL4::CommandBuffer::renderCommandEncoder(const MTL4::RenderPassDescriptor* descriptor) | |
| 161 | +{ | |
| 162 | + return Object::sendMessage<MTL4::RenderCommandEncoder*>(this, _MTL_PRIVATE_SEL(renderCommandEncoderWithDescriptor_), descriptor); | |
| 163 | +} | |
| 164 | + | |
| 165 | +_MTL_INLINE MTL4::RenderCommandEncoder* MTL4::CommandBuffer::renderCommandEncoder(const MTL4::RenderPassDescriptor* descriptor, MTL4::RenderEncoderOptions options) | |
| 166 | +{ | |
| 167 | + return Object::sendMessage<MTL4::RenderCommandEncoder*>(this, _MTL_PRIVATE_SEL(renderCommandEncoderWithDescriptor_options_), descriptor, options); | |
| 168 | +} | |
| 169 | + | |
| 170 | +_MTL_INLINE void MTL4::CommandBuffer::resolveCounterHeap(const MTL4::CounterHeap* counterHeap, NS::Range range, const MTL4::BufferRange bufferRange, const MTL::Fence* fenceToWait, const MTL::Fence* fenceToUpdate) | |
| 171 | +{ | |
| 172 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(resolveCounterHeap_withRange_intoBuffer_waitFence_updateFence_), counterHeap, range, bufferRange, fenceToWait, fenceToUpdate); | |
| 173 | +} | |
| 174 | + | |
| 175 | +_MTL_INLINE void MTL4::CommandBuffer::setLabel(const NS::String* label) | |
| 176 | +{ | |
| 177 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setLabel_), label); | |
| 178 | +} | |
| 179 | + | |
| 180 | +_MTL_INLINE void MTL4::CommandBuffer::useResidencySet(const MTL::ResidencySet* residencySet) | |
| 181 | +{ | |
| 182 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(useResidencySet_), residencySet); | |
| 183 | +} | |
| 184 | + | |
| 185 | +_MTL_INLINE void MTL4::CommandBuffer::useResidencySets(const MTL::ResidencySet* const residencySets[], NS::UInteger count) | |
| 186 | +{ | |
| 187 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(useResidencySets_count_), residencySets, count); | |
| 188 | +} | |
| 189 | + | |
| 190 | +_MTL_INLINE void MTL4::CommandBuffer::writeTimestampIntoHeap(const MTL4::CounterHeap* counterHeap, NS::UInteger index) | |
| 191 | +{ | |
| 192 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(writeTimestampIntoHeap_atIndex_), counterHeap, index); | |
| 193 | +} | |
added
third_party/metal-cpp/Metal/MTL4CommandEncoder.hpp
+134 −0
@@ -0,0 +1,134 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Metal/MTL4CommandEncoder.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2025 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include "../Foundation/Foundation.hpp" | |
| 24 | +#include "MTLCommandEncoder.hpp" | |
| 25 | +#include "MTLDefines.hpp" | |
| 26 | +#include "MTLHeaderBridge.hpp" | |
| 27 | +#include "MTLPrivate.hpp" | |
| 28 | + | |
| 29 | +namespace MTL4 | |
| 30 | +{ | |
| 31 | +class CommandBuffer; | |
| 32 | +} | |
| 33 | + | |
| 34 | +namespace MTL | |
| 35 | +{ | |
| 36 | +class Fence; | |
| 37 | +} | |
| 38 | + | |
| 39 | +namespace MTL4 | |
| 40 | +{ | |
| 41 | +_MTL_OPTIONS(NS::UInteger, VisibilityOptions) { | |
| 42 | + VisibilityOptionNone = 0, | |
| 43 | + VisibilityOptionDevice = 1, | |
| 44 | + VisibilityOptionResourceAlias = 1 << 1, | |
| 45 | +}; | |
| 46 | + | |
| 47 | +class CommandEncoder : public NS::Referencing<CommandEncoder> | |
| 48 | +{ | |
| 49 | +public: | |
| 50 | + void barrierAfterEncoderStages(MTL::Stages afterEncoderStages, MTL::Stages beforeEncoderStages, MTL4::VisibilityOptions visibilityOptions); | |
| 51 | + | |
| 52 | + void barrierAfterQueueStages(MTL::Stages afterQueueStages, MTL::Stages beforeStages, MTL4::VisibilityOptions visibilityOptions); | |
| 53 | + | |
| 54 | + void barrierAfterStages(MTL::Stages afterStages, MTL::Stages beforeQueueStages, MTL4::VisibilityOptions visibilityOptions); | |
| 55 | + | |
| 56 | + CommandBuffer* commandBuffer() const; | |
| 57 | + | |
| 58 | + void endEncoding(); | |
| 59 | + | |
| 60 | + void insertDebugSignpost(const NS::String* string); | |
| 61 | + | |
| 62 | + NS::String* label() const; | |
| 63 | + | |
| 64 | + void popDebugGroup(); | |
| 65 | + | |
| 66 | + void pushDebugGroup(const NS::String* string); | |
| 67 | + | |
| 68 | + void setLabel(const NS::String* label); | |
| 69 | + | |
| 70 | + void updateFence(const MTL::Fence* fence, MTL::Stages afterEncoderStages); | |
| 71 | + | |
| 72 | + void waitForFence(const MTL::Fence* fence, MTL::Stages beforeEncoderStages); | |
| 73 | +}; | |
| 74 | + | |
| 75 | +} | |
| 76 | +_MTL_INLINE void MTL4::CommandEncoder::barrierAfterEncoderStages(MTL::Stages afterEncoderStages, MTL::Stages beforeEncoderStages, MTL4::VisibilityOptions visibilityOptions) | |
| 77 | +{ | |
| 78 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(barrierAfterEncoderStages_beforeEncoderStages_visibilityOptions_), afterEncoderStages, beforeEncoderStages, visibilityOptions); | |
| 79 | +} | |
| 80 | + | |
| 81 | +_MTL_INLINE void MTL4::CommandEncoder::barrierAfterQueueStages(MTL::Stages afterQueueStages, MTL::Stages beforeStages, MTL4::VisibilityOptions visibilityOptions) | |
| 82 | +{ | |
| 83 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(barrierAfterQueueStages_beforeStages_visibilityOptions_), afterQueueStages, beforeStages, visibilityOptions); | |
| 84 | +} | |
| 85 | + | |
| 86 | +_MTL_INLINE void MTL4::CommandEncoder::barrierAfterStages(MTL::Stages afterStages, MTL::Stages beforeQueueStages, MTL4::VisibilityOptions visibilityOptions) | |
| 87 | +{ | |
| 88 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(barrierAfterStages_beforeQueueStages_visibilityOptions_), afterStages, beforeQueueStages, visibilityOptions); | |
| 89 | +} | |
| 90 | + | |
| 91 | +_MTL_INLINE MTL4::CommandBuffer* MTL4::CommandEncoder::commandBuffer() const | |
| 92 | +{ | |
| 93 | + return Object::sendMessage<MTL4::CommandBuffer*>(this, _MTL_PRIVATE_SEL(commandBuffer)); | |
| 94 | +} | |
| 95 | + | |
| 96 | +_MTL_INLINE void MTL4::CommandEncoder::endEncoding() | |
| 97 | +{ | |
| 98 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(endEncoding)); | |
| 99 | +} | |
| 100 | + | |
| 101 | +_MTL_INLINE void MTL4::CommandEncoder::insertDebugSignpost(const NS::String* string) | |
| 102 | +{ | |
| 103 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(insertDebugSignpost_), string); | |
| 104 | +} | |
| 105 | + | |
| 106 | +_MTL_INLINE NS::String* MTL4::CommandEncoder::label() const | |
| 107 | +{ | |
| 108 | + return Object::sendMessage<NS::String*>(this, _MTL_PRIVATE_SEL(label)); | |
| 109 | +} | |
| 110 | + | |
| 111 | +_MTL_INLINE void MTL4::CommandEncoder::popDebugGroup() | |
| 112 | +{ | |
| 113 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(popDebugGroup)); | |
| 114 | +} | |
| 115 | + | |
| 116 | +_MTL_INLINE void MTL4::CommandEncoder::pushDebugGroup(const NS::String* string) | |
| 117 | +{ | |
| 118 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(pushDebugGroup_), string); | |
| 119 | +} | |
| 120 | + | |
| 121 | +_MTL_INLINE void MTL4::CommandEncoder::setLabel(const NS::String* label) | |
| 122 | +{ | |
| 123 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setLabel_), label); | |
| 124 | +} | |
| 125 | + | |
| 126 | +_MTL_INLINE void MTL4::CommandEncoder::updateFence(const MTL::Fence* fence, MTL::Stages afterEncoderStages) | |
| 127 | +{ | |
| 128 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(updateFence_afterEncoderStages_), fence, afterEncoderStages); | |
| 129 | +} | |
| 130 | + | |
| 131 | +_MTL_INLINE void MTL4::CommandEncoder::waitForFence(const MTL::Fence* fence, MTL::Stages beforeEncoderStages) | |
| 132 | +{ | |
| 133 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(waitForFence_beforeEncoderStages_), fence, beforeEncoderStages); | |
| 134 | +} | |
added
third_party/metal-cpp/Metal/MTL4CommandQueue.hpp
+283 −0
@@ -0,0 +1,283 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Metal/MTL4CommandQueue.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2025 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include "../Foundation/Foundation.hpp" | |
| 24 | +#include "MTL4CommitFeedback.hpp" | |
| 25 | +#include "MTLDefines.hpp" | |
| 26 | +#include "MTLHeaderBridge.hpp" | |
| 27 | +#include "MTLPrivate.hpp" | |
| 28 | +#include "MTLResourceStateCommandEncoder.hpp" | |
| 29 | +#include "MTLTypes.hpp" | |
| 30 | +#include <cstdint> | |
| 31 | +#include <dispatch/dispatch.h> | |
| 32 | + | |
| 33 | +namespace MTL | |
| 34 | +{ | |
| 35 | +class Buffer; | |
| 36 | +class Device; | |
| 37 | +class Drawable; | |
| 38 | +class Event; | |
| 39 | +class Heap; | |
| 40 | +class ResidencySet; | |
| 41 | +class Texture; | |
| 42 | +} | |
| 43 | + | |
| 44 | +namespace MTL4 | |
| 45 | +{ | |
| 46 | +class CommandBuffer; | |
| 47 | +class CommandQueueDescriptor; | |
| 48 | +class CommitOptions; | |
| 49 | +struct CopySparseBufferMappingOperation; | |
| 50 | +struct CopySparseTextureMappingOperation; | |
| 51 | +struct UpdateSparseBufferMappingOperation; | |
| 52 | +struct UpdateSparseTextureMappingOperation; | |
| 53 | +_MTL_ENUM(NS::Integer, CommandQueueError) { | |
| 54 | + CommandQueueErrorNone = 0, | |
| 55 | + CommandQueueErrorTimeout = 1, | |
| 56 | + CommandQueueErrorNotPermitted = 2, | |
| 57 | + CommandQueueErrorOutOfMemory = 3, | |
| 58 | + CommandQueueErrorDeviceRemoved = 4, | |
| 59 | + CommandQueueErrorAccessRevoked = 5, | |
| 60 | + CommandQueueErrorInternal = 6, | |
| 61 | +}; | |
| 62 | + | |
| 63 | +struct UpdateSparseTextureMappingOperation | |
| 64 | +{ | |
| 65 | + MTL::SparseTextureMappingMode mode; | |
| 66 | + MTL::Region textureRegion; | |
| 67 | + NS::UInteger textureLevel; | |
| 68 | + NS::UInteger textureSlice; | |
| 69 | + NS::UInteger heapOffset; | |
| 70 | +} _MTL_PACKED; | |
| 71 | + | |
| 72 | +struct CopySparseTextureMappingOperation | |
| 73 | +{ | |
| 74 | + MTL::Region sourceRegion; | |
| 75 | + NS::UInteger sourceLevel; | |
| 76 | + NS::UInteger sourceSlice; | |
| 77 | + MTL::Origin destinationOrigin; | |
| 78 | + NS::UInteger destinationLevel; | |
| 79 | + NS::UInteger destinationSlice; | |
| 80 | +} _MTL_PACKED; | |
| 81 | + | |
| 82 | +struct UpdateSparseBufferMappingOperation | |
| 83 | +{ | |
| 84 | + MTL::SparseTextureMappingMode mode; | |
| 85 | + NS::Range bufferRange; | |
| 86 | + NS::UInteger heapOffset; | |
| 87 | +} _MTL_PACKED; | |
| 88 | + | |
| 89 | +struct CopySparseBufferMappingOperation | |
| 90 | +{ | |
| 91 | + NS::Range sourceRange; | |
| 92 | + NS::UInteger destinationOffset; | |
| 93 | +} _MTL_PACKED; | |
| 94 | + | |
| 95 | +class CommitOptions : public NS::Referencing<CommitOptions> | |
| 96 | +{ | |
| 97 | +public: | |
| 98 | + void addFeedbackHandler(const MTL4::CommitFeedbackHandler block); | |
| 99 | + void addFeedbackHandler(const MTL4::CommitFeedbackHandlerFunction& function); | |
| 100 | + | |
| 101 | + static CommitOptions* alloc(); | |
| 102 | + | |
| 103 | + CommitOptions* init(); | |
| 104 | +}; | |
| 105 | +class CommandQueueDescriptor : public NS::Copying<CommandQueueDescriptor> | |
| 106 | +{ | |
| 107 | +public: | |
| 108 | + static CommandQueueDescriptor* alloc(); | |
| 109 | + | |
| 110 | + dispatch_queue_t feedbackQueue() const; | |
| 111 | + | |
| 112 | + CommandQueueDescriptor* init(); | |
| 113 | + | |
| 114 | + NS::String* label() const; | |
| 115 | + | |
| 116 | + void setFeedbackQueue(const dispatch_queue_t feedbackQueue); | |
| 117 | + | |
| 118 | + void setLabel(const NS::String* label); | |
| 119 | +}; | |
| 120 | +class CommandQueue : public NS::Referencing<CommandQueue> | |
| 121 | +{ | |
| 122 | +public: | |
| 123 | + void addResidencySet(const MTL::ResidencySet* residencySet); | |
| 124 | + void addResidencySets(const MTL::ResidencySet* const residencySets[], NS::UInteger count); | |
| 125 | + | |
| 126 | + void commit(const MTL4::CommandBuffer* const commandBuffers[], NS::UInteger count); | |
| 127 | + void commit(const MTL4::CommandBuffer* const commandBuffers[], NS::UInteger count, const MTL4::CommitOptions* options); | |
| 128 | + | |
| 129 | + void copyBufferMappingsFromBuffer(const MTL::Buffer* sourceBuffer, const MTL::Buffer* destinationBuffer, const MTL4::CopySparseBufferMappingOperation* operations, NS::UInteger count); | |
| 130 | + | |
| 131 | + void copyTextureMappingsFromTexture(const MTL::Texture* sourceTexture, const MTL::Texture* destinationTexture, const MTL4::CopySparseTextureMappingOperation* operations, NS::UInteger count); | |
| 132 | + | |
| 133 | + MTL::Device* device() const; | |
| 134 | + | |
| 135 | + NS::String* label() const; | |
| 136 | + | |
| 137 | + void removeResidencySet(const MTL::ResidencySet* residencySet); | |
| 138 | + void removeResidencySets(const MTL::ResidencySet* const residencySets[], NS::UInteger count); | |
| 139 | + | |
| 140 | + void signalDrawable(const MTL::Drawable* drawable); | |
| 141 | + | |
| 142 | + void signalEvent(const MTL::Event* event, uint64_t value); | |
| 143 | + | |
| 144 | + void updateBufferMappings(const MTL::Buffer* buffer, const MTL::Heap* heap, const MTL4::UpdateSparseBufferMappingOperation* operations, NS::UInteger count); | |
| 145 | + | |
| 146 | + void updateTextureMappings(const MTL::Texture* texture, const MTL::Heap* heap, const MTL4::UpdateSparseTextureMappingOperation* operations, NS::UInteger count); | |
| 147 | + | |
| 148 | + void wait(const MTL::Event* event, uint64_t value); | |
| 149 | + void wait(const MTL::Drawable* drawable); | |
| 150 | +}; | |
| 151 | + | |
| 152 | +} | |
| 153 | + | |
| 154 | +_MTL_INLINE void MTL4::CommitOptions::addFeedbackHandler(const MTL4::CommitFeedbackHandler block) | |
| 155 | +{ | |
| 156 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(addFeedbackHandler_), block); | |
| 157 | +} | |
| 158 | + | |
| 159 | +_MTL_INLINE void MTL4::CommitOptions::addFeedbackHandler(const MTL4::CommitFeedbackHandlerFunction& function) | |
| 160 | +{ | |
| 161 | + __block MTL4::CommitFeedbackHandlerFunction blockFunction = function; | |
| 162 | + addFeedbackHandler(^(MTL4::CommitFeedback* pFeedback) { blockFunction(pFeedback); }); | |
| 163 | +} | |
| 164 | + | |
| 165 | +_MTL_INLINE MTL4::CommitOptions* MTL4::CommitOptions::alloc() | |
| 166 | +{ | |
| 167 | + return NS::Object::alloc<MTL4::CommitOptions>(_MTL_PRIVATE_CLS(MTL4CommitOptions)); | |
| 168 | +} | |
| 169 | + | |
| 170 | +_MTL_INLINE MTL4::CommitOptions* MTL4::CommitOptions::init() | |
| 171 | +{ | |
| 172 | + return NS::Object::init<MTL4::CommitOptions>(); | |
| 173 | +} | |
| 174 | + | |
| 175 | +_MTL_INLINE MTL4::CommandQueueDescriptor* MTL4::CommandQueueDescriptor::alloc() | |
| 176 | +{ | |
| 177 | + return NS::Object::alloc<MTL4::CommandQueueDescriptor>(_MTL_PRIVATE_CLS(MTL4CommandQueueDescriptor)); | |
| 178 | +} | |
| 179 | + | |
| 180 | +_MTL_INLINE dispatch_queue_t MTL4::CommandQueueDescriptor::feedbackQueue() const | |
| 181 | +{ | |
| 182 | + return Object::sendMessage<dispatch_queue_t>(this, _MTL_PRIVATE_SEL(feedbackQueue)); | |
| 183 | +} | |
| 184 | + | |
| 185 | +_MTL_INLINE MTL4::CommandQueueDescriptor* MTL4::CommandQueueDescriptor::init() | |
| 186 | +{ | |
| 187 | + return NS::Object::init<MTL4::CommandQueueDescriptor>(); | |
| 188 | +} | |
| 189 | + | |
| 190 | +_MTL_INLINE NS::String* MTL4::CommandQueueDescriptor::label() const | |
| 191 | +{ | |
| 192 | + return Object::sendMessage<NS::String*>(this, _MTL_PRIVATE_SEL(label)); | |
| 193 | +} | |
| 194 | + | |
| 195 | +_MTL_INLINE void MTL4::CommandQueueDescriptor::setFeedbackQueue(const dispatch_queue_t feedbackQueue) | |
| 196 | +{ | |
| 197 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setFeedbackQueue_), feedbackQueue); | |
| 198 | +} | |
| 199 | + | |
| 200 | +_MTL_INLINE void MTL4::CommandQueueDescriptor::setLabel(const NS::String* label) | |
| 201 | +{ | |
| 202 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setLabel_), label); | |
| 203 | +} | |
| 204 | + | |
| 205 | +_MTL_INLINE void MTL4::CommandQueue::addResidencySet(const MTL::ResidencySet* residencySet) | |
| 206 | +{ | |
| 207 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(addResidencySet_), residencySet); | |
| 208 | +} | |
| 209 | + | |
| 210 | +_MTL_INLINE void MTL4::CommandQueue::addResidencySets(const MTL::ResidencySet* const residencySets[], NS::UInteger count) | |
| 211 | +{ | |
| 212 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(addResidencySets_count_), residencySets, count); | |
| 213 | +} | |
| 214 | + | |
| 215 | +_MTL_INLINE void MTL4::CommandQueue::commit(const MTL4::CommandBuffer* const commandBuffers[], NS::UInteger count) | |
| 216 | +{ | |
| 217 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(commit_count_), commandBuffers, count); | |
| 218 | +} | |
| 219 | + | |
| 220 | +_MTL_INLINE void MTL4::CommandQueue::commit(const MTL4::CommandBuffer* const commandBuffers[], NS::UInteger count, const MTL4::CommitOptions* options) | |
| 221 | +{ | |
| 222 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(commit_count_options_), commandBuffers, count, options); | |
| 223 | +} | |
| 224 | + | |
| 225 | +_MTL_INLINE void MTL4::CommandQueue::copyBufferMappingsFromBuffer(const MTL::Buffer* sourceBuffer, const MTL::Buffer* destinationBuffer, const MTL4::CopySparseBufferMappingOperation* operations, NS::UInteger count) | |
| 226 | +{ | |
| 227 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(copyBufferMappingsFromBuffer_toBuffer_operations_count_), sourceBuffer, destinationBuffer, operations, count); | |
| 228 | +} | |
| 229 | + | |
| 230 | +_MTL_INLINE void MTL4::CommandQueue::copyTextureMappingsFromTexture(const MTL::Texture* sourceTexture, const MTL::Texture* destinationTexture, const MTL4::CopySparseTextureMappingOperation* operations, NS::UInteger count) | |
| 231 | +{ | |
| 232 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(copyTextureMappingsFromTexture_toTexture_operations_count_), sourceTexture, destinationTexture, operations, count); | |
| 233 | +} | |
| 234 | + | |
| 235 | +_MTL_INLINE MTL::Device* MTL4::CommandQueue::device() const | |
| 236 | +{ | |
| 237 | + return Object::sendMessage<MTL::Device*>(this, _MTL_PRIVATE_SEL(device)); | |
| 238 | +} | |
| 239 | + | |
| 240 | +_MTL_INLINE NS::String* MTL4::CommandQueue::label() const | |
| 241 | +{ | |
| 242 | + return Object::sendMessage<NS::String*>(this, _MTL_PRIVATE_SEL(label)); | |
| 243 | +} | |
| 244 | + | |
| 245 | +_MTL_INLINE void MTL4::CommandQueue::removeResidencySet(const MTL::ResidencySet* residencySet) | |
| 246 | +{ | |
| 247 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(removeResidencySet_), residencySet); | |
| 248 | +} | |
| 249 | + | |
| 250 | +_MTL_INLINE void MTL4::CommandQueue::removeResidencySets(const MTL::ResidencySet* const residencySets[], NS::UInteger count) | |
| 251 | +{ | |
| 252 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(removeResidencySets_count_), residencySets, count); | |
| 253 | +} | |
| 254 | + | |
| 255 | +_MTL_INLINE void MTL4::CommandQueue::signalDrawable(const MTL::Drawable* drawable) | |
| 256 | +{ | |
| 257 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(signalDrawable_), drawable); | |
| 258 | +} | |
| 259 | + | |
| 260 | +_MTL_INLINE void MTL4::CommandQueue::signalEvent(const MTL::Event* event, uint64_t value) | |
| 261 | +{ | |
| 262 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(signalEvent_value_), event, value); | |
| 263 | +} | |
| 264 | + | |
| 265 | +_MTL_INLINE void MTL4::CommandQueue::updateBufferMappings(const MTL::Buffer* buffer, const MTL::Heap* heap, const MTL4::UpdateSparseBufferMappingOperation* operations, NS::UInteger count) | |
| 266 | +{ | |
| 267 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(updateBufferMappings_heap_operations_count_), buffer, heap, operations, count); | |
| 268 | +} | |
| 269 | + | |
| 270 | +_MTL_INLINE void MTL4::CommandQueue::updateTextureMappings(const MTL::Texture* texture, const MTL::Heap* heap, const MTL4::UpdateSparseTextureMappingOperation* operations, NS::UInteger count) | |
| 271 | +{ | |
| 272 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(updateTextureMappings_heap_operations_count_), texture, heap, operations, count); | |
| 273 | +} | |
| 274 | + | |
| 275 | +_MTL_INLINE void MTL4::CommandQueue::wait(const MTL::Event* event, uint64_t value) | |
| 276 | +{ | |
| 277 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(waitForEvent_value_), event, value); | |
| 278 | +} | |
| 279 | + | |
| 280 | +_MTL_INLINE void MTL4::CommandQueue::wait(const MTL::Drawable* drawable) | |
| 281 | +{ | |
| 282 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(waitForDrawable_), drawable); | |
| 283 | +} | |
added
third_party/metal-cpp/Metal/MTL4CommitFeedback.hpp
+62 −0
@@ -0,0 +1,62 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Metal/MTL4CommitFeedback.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2025 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include "../Foundation/Foundation.hpp" | |
| 24 | +#include "MTLDefines.hpp" | |
| 25 | +#include "MTLHeaderBridge.hpp" | |
| 26 | +#include "MTLPrivate.hpp" | |
| 27 | +#include <CoreFoundation/CoreFoundation.h> | |
| 28 | + | |
| 29 | +#include <functional> | |
| 30 | + | |
| 31 | +namespace MTL4 | |
| 32 | +{ | |
| 33 | +class CommitFeedback; | |
| 34 | + | |
| 35 | +using CommitFeedbackHandler = void (^)(MTL4::CommitFeedback*); | |
| 36 | +using CommitFeedbackHandlerFunction = std::function<void(MTL4::CommitFeedback*)>; | |
| 37 | + | |
| 38 | +class CommitFeedback : public NS::Referencing<CommitFeedback> | |
| 39 | +{ | |
| 40 | +public: | |
| 41 | + CFTimeInterval GPUEndTime() const; | |
| 42 | + | |
| 43 | + CFTimeInterval GPUStartTime() const; | |
| 44 | + | |
| 45 | + NS::Error* error() const; | |
| 46 | +}; | |
| 47 | + | |
| 48 | +} | |
| 49 | +_MTL_INLINE CFTimeInterval MTL4::CommitFeedback::GPUEndTime() const | |
| 50 | +{ | |
| 51 | + return Object::sendMessage<CFTimeInterval>(this, _MTL_PRIVATE_SEL(GPUEndTime)); | |
| 52 | +} | |
| 53 | + | |
| 54 | +_MTL_INLINE CFTimeInterval MTL4::CommitFeedback::GPUStartTime() const | |
| 55 | +{ | |
| 56 | + return Object::sendMessage<CFTimeInterval>(this, _MTL_PRIVATE_SEL(GPUStartTime)); | |
| 57 | +} | |
| 58 | + | |
| 59 | +_MTL_INLINE NS::Error* MTL4::CommitFeedback::error() const | |
| 60 | +{ | |
| 61 | + return Object::sendMessage<NS::Error*>(this, _MTL_PRIVATE_SEL(error)); | |
| 62 | +} | |
added
third_party/metal-cpp/Metal/MTL4Compiler.hpp
+345 −0
@@ -0,0 +1,345 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Metal/MTL4Compiler.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2025 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include "../Foundation/Foundation.hpp" | |
| 24 | +#include "MTLDefines.hpp" | |
| 25 | +#include "MTLDevice.hpp" | |
| 26 | +#include "MTLHeaderBridge.hpp" | |
| 27 | +#include "MTLPrivate.hpp" | |
| 28 | + | |
| 29 | +#include <functional> | |
| 30 | + | |
| 31 | +namespace MTL4 | |
| 32 | +{ | |
| 33 | +class BinaryFunction; | |
| 34 | +class BinaryFunctionDescriptor; | |
| 35 | +class CompilerDescriptor; | |
| 36 | +class CompilerTask; | |
| 37 | +class CompilerTaskOptions; | |
| 38 | +class ComputePipelineDescriptor; | |
| 39 | +class LibraryDescriptor; | |
| 40 | +class MachineLearningPipelineDescriptor; | |
| 41 | +class MachineLearningPipelineState; | |
| 42 | +class PipelineDataSetSerializer; | |
| 43 | +class PipelineDescriptor; | |
| 44 | +class PipelineStageDynamicLinkingDescriptor; | |
| 45 | +class RenderPipelineDynamicLinkingDescriptor; | |
| 46 | +} | |
| 47 | + | |
| 48 | +namespace MTL | |
| 49 | +{ | |
| 50 | +class ComputePipelineState; | |
| 51 | +class Device; | |
| 52 | +class DynamicLibrary; | |
| 53 | +class Library; | |
| 54 | +class RenderPipelineState; | |
| 55 | + | |
| 56 | +using NewDynamicLibraryCompletionHandler = void (^)(MTL::DynamicLibrary*, NS::Error*); | |
| 57 | +using NewDynamicLibraryCompletionHandlerFunction = std::function<void(MTL::DynamicLibrary*, NS::Error*)>; | |
| 58 | +} | |
| 59 | + | |
| 60 | +namespace MTL4 | |
| 61 | +{ | |
| 62 | +using NewComputePipelineStateCompletionHandler = void (^)(MTL::ComputePipelineState*, NS::Error*); | |
| 63 | +using NewComputePipelineStateCompletionHandlerFunction = std::function<void(MTL::ComputePipelineState*, NS::Error*)>; | |
| 64 | +using NewRenderPipelineStateCompletionHandler = void (^)(MTL::RenderPipelineState*, NS::Error*); | |
| 65 | +using NewRenderPipelineStateCompletionHandlerFunction = std::function<void(MTL::RenderPipelineState*, NS::Error*)>; | |
| 66 | +using NewBinaryFunctionCompletionHandler = void (^)(MTL4::BinaryFunction*, NS::Error*); | |
| 67 | +using NewBinaryFunctionCompletionHandlerFunction = std::function<void(MTL4::BinaryFunction*, NS::Error*)>; | |
| 68 | +using NewMachineLearningPipelineStateCompletionHandler = void (^)(MTL4::MachineLearningPipelineState*, NS::Error*); | |
| 69 | +using NewMachineLearningPipelineStateCompletionHandlerFunction = std::function<void(MTL4::MachineLearningPipelineState*, NS::Error*)>; | |
| 70 | + | |
| 71 | +class CompilerDescriptor : public NS::Copying<CompilerDescriptor> | |
| 72 | +{ | |
| 73 | +public: | |
| 74 | + static CompilerDescriptor* alloc(); | |
| 75 | + | |
| 76 | + CompilerDescriptor* init(); | |
| 77 | + | |
| 78 | + NS::String* label() const; | |
| 79 | + | |
| 80 | + PipelineDataSetSerializer* pipelineDataSetSerializer() const; | |
| 81 | + | |
| 82 | + void setLabel(const NS::String* label); | |
| 83 | + | |
| 84 | + void setPipelineDataSetSerializer(const MTL4::PipelineDataSetSerializer* pipelineDataSetSerializer); | |
| 85 | +}; | |
| 86 | +class CompilerTaskOptions : public NS::Copying<CompilerTaskOptions> | |
| 87 | +{ | |
| 88 | +public: | |
| 89 | + static CompilerTaskOptions* alloc(); | |
| 90 | + | |
| 91 | + CompilerTaskOptions* init(); | |
| 92 | + | |
| 93 | + NS::Array* lookupArchives() const; | |
| 94 | + void setLookupArchives(const NS::Array* lookupArchives); | |
| 95 | +}; | |
| 96 | +class Compiler : public NS::Referencing<Compiler> | |
| 97 | +{ | |
| 98 | +public: | |
| 99 | + MTL::Device* device() const; | |
| 100 | + | |
| 101 | + NS::String* label() const; | |
| 102 | + | |
| 103 | + BinaryFunction* newBinaryFunction(const MTL4::BinaryFunctionDescriptor* descriptor, const MTL4::CompilerTaskOptions* compilerTaskOptions, NS::Error** error); | |
| 104 | + CompilerTask* newBinaryFunction(const MTL4::BinaryFunctionDescriptor* descriptor, const MTL4::CompilerTaskOptions* compilerTaskOptions, const MTL4::NewBinaryFunctionCompletionHandler completionHandler); | |
| 105 | + | |
| 106 | + MTL::ComputePipelineState* newComputePipelineState(const MTL4::ComputePipelineDescriptor* descriptor, const MTL4::CompilerTaskOptions* compilerTaskOptions, NS::Error** error); | |
| 107 | + MTL::ComputePipelineState* newComputePipelineState(const MTL4::ComputePipelineDescriptor* descriptor, const MTL4::PipelineStageDynamicLinkingDescriptor* dynamicLinkingDescriptor, const MTL4::CompilerTaskOptions* compilerTaskOptions, NS::Error** error); | |
| 108 | + CompilerTask* newComputePipelineState(const MTL4::ComputePipelineDescriptor* descriptor, const MTL4::CompilerTaskOptions* compilerTaskOptions, const MTL::NewComputePipelineStateCompletionHandler completionHandler); | |
| 109 | + CompilerTask* newComputePipelineState(const MTL4::ComputePipelineDescriptor* descriptor, const MTL4::PipelineStageDynamicLinkingDescriptor* dynamicLinkingDescriptor, const MTL4::CompilerTaskOptions* compilerTaskOptions, const MTL::NewComputePipelineStateCompletionHandler completionHandler); | |
| 110 | + CompilerTask* newComputePipelineState(const MTL4::ComputePipelineDescriptor* pDescriptor, const MTL4::CompilerTaskOptions* options, const MTL4::NewComputePipelineStateCompletionHandlerFunction& function); | |
| 111 | + | |
| 112 | + MTL::DynamicLibrary* newDynamicLibrary(const MTL::Library* library, NS::Error** error); | |
| 113 | + MTL::DynamicLibrary* newDynamicLibrary(const NS::URL* url, NS::Error** error); | |
| 114 | + CompilerTask* newDynamicLibrary(const MTL::Library* library, const MTL::NewDynamicLibraryCompletionHandler completionHandler); | |
| 115 | + CompilerTask* newDynamicLibrary(const NS::URL* url, const MTL::NewDynamicLibraryCompletionHandler completionHandler); | |
| 116 | + CompilerTask* newDynamicLibrary(const MTL::Library* pLibrary, const MTL::NewDynamicLibraryCompletionHandlerFunction& function); | |
| 117 | + CompilerTask* newDynamicLibrary(const NS::URL* pURL, const MTL::NewDynamicLibraryCompletionHandlerFunction& function); | |
| 118 | + | |
| 119 | + MTL::Library* newLibrary(const MTL4::LibraryDescriptor* descriptor, NS::Error** error); | |
| 120 | + CompilerTask* newLibrary(const MTL4::LibraryDescriptor* descriptor, const MTL::NewLibraryCompletionHandler completionHandler); | |
| 121 | + CompilerTask* newLibrary(const MTL4::LibraryDescriptor* pDescriptor, const MTL::NewLibraryCompletionHandlerFunction& function); | |
| 122 | + | |
| 123 | + MachineLearningPipelineState* newMachineLearningPipelineState(const MTL4::MachineLearningPipelineDescriptor* descriptor, NS::Error** error); | |
| 124 | + CompilerTask* newMachineLearningPipelineState(const MTL4::MachineLearningPipelineDescriptor* descriptor, const MTL4::NewMachineLearningPipelineStateCompletionHandler completionHandler); | |
| 125 | + CompilerTask* newMachineLearningPipelineState(const MTL4::MachineLearningPipelineDescriptor* pDescriptor, const MTL4::NewMachineLearningPipelineStateCompletionHandlerFunction& function); | |
| 126 | + | |
| 127 | + MTL::RenderPipelineState* newRenderPipelineState(const MTL4::PipelineDescriptor* descriptor, const MTL4::CompilerTaskOptions* compilerTaskOptions, NS::Error** error); | |
| 128 | + MTL::RenderPipelineState* newRenderPipelineState(const MTL4::PipelineDescriptor* descriptor, const MTL4::RenderPipelineDynamicLinkingDescriptor* dynamicLinkingDescriptor, const MTL4::CompilerTaskOptions* compilerTaskOptions, NS::Error** error); | |
| 129 | + CompilerTask* newRenderPipelineState(const MTL4::PipelineDescriptor* descriptor, const MTL4::CompilerTaskOptions* compilerTaskOptions, const MTL::NewRenderPipelineStateCompletionHandler completionHandler); | |
| 130 | + CompilerTask* newRenderPipelineState(const MTL4::PipelineDescriptor* descriptor, const MTL4::RenderPipelineDynamicLinkingDescriptor* dynamicLinkingDescriptor, const MTL4::CompilerTaskOptions* compilerTaskOptions, const MTL::NewRenderPipelineStateCompletionHandler completionHandler); | |
| 131 | + CompilerTask* newRenderPipelineState(const MTL4::PipelineDescriptor* pDescriptor, const MTL4::CompilerTaskOptions* options, const MTL4::NewRenderPipelineStateCompletionHandlerFunction& function); | |
| 132 | + MTL::RenderPipelineState* newRenderPipelineStateBySpecialization(const MTL4::PipelineDescriptor* descriptor, const MTL::RenderPipelineState* pipeline, NS::Error** error); | |
| 133 | + CompilerTask* newRenderPipelineStateBySpecialization(const MTL4::PipelineDescriptor* descriptor, const MTL::RenderPipelineState* pipeline, const MTL::NewRenderPipelineStateCompletionHandler completionHandler); | |
| 134 | + CompilerTask* newRenderPipelineStateBySpecialization(const MTL4::PipelineDescriptor* pDescriptor, const MTL::RenderPipelineState* pPipeline, const MTL4::NewRenderPipelineStateCompletionHandlerFunction& function); | |
| 135 | + | |
| 136 | + PipelineDataSetSerializer* pipelineDataSetSerializer() const; | |
| 137 | +}; | |
| 138 | + | |
| 139 | +} | |
| 140 | +_MTL_INLINE MTL4::CompilerDescriptor* MTL4::CompilerDescriptor::alloc() | |
| 141 | +{ | |
| 142 | + return NS::Object::alloc<MTL4::CompilerDescriptor>(_MTL_PRIVATE_CLS(MTL4CompilerDescriptor)); | |
| 143 | +} | |
| 144 | + | |
| 145 | +_MTL_INLINE MTL4::CompilerDescriptor* MTL4::CompilerDescriptor::init() | |
| 146 | +{ | |
| 147 | + return NS::Object::init<MTL4::CompilerDescriptor>(); | |
| 148 | +} | |
| 149 | + | |
| 150 | +_MTL_INLINE NS::String* MTL4::CompilerDescriptor::label() const | |
| 151 | +{ | |
| 152 | + return Object::sendMessage<NS::String*>(this, _MTL_PRIVATE_SEL(label)); | |
| 153 | +} | |
| 154 | + | |
| 155 | +_MTL_INLINE MTL4::PipelineDataSetSerializer* MTL4::CompilerDescriptor::pipelineDataSetSerializer() const | |
| 156 | +{ | |
| 157 | + return Object::sendMessage<MTL4::PipelineDataSetSerializer*>(this, _MTL_PRIVATE_SEL(pipelineDataSetSerializer)); | |
| 158 | +} | |
| 159 | + | |
| 160 | +_MTL_INLINE void MTL4::CompilerDescriptor::setLabel(const NS::String* label) | |
| 161 | +{ | |
| 162 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setLabel_), label); | |
| 163 | +} | |
| 164 | + | |
| 165 | +_MTL_INLINE void MTL4::CompilerDescriptor::setPipelineDataSetSerializer(const MTL4::PipelineDataSetSerializer* pipelineDataSetSerializer) | |
| 166 | +{ | |
| 167 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setPipelineDataSetSerializer_), pipelineDataSetSerializer); | |
| 168 | +} | |
| 169 | + | |
| 170 | +_MTL_INLINE MTL4::CompilerTaskOptions* MTL4::CompilerTaskOptions::alloc() | |
| 171 | +{ | |
| 172 | + return NS::Object::alloc<MTL4::CompilerTaskOptions>(_MTL_PRIVATE_CLS(MTL4CompilerTaskOptions)); | |
| 173 | +} | |
| 174 | + | |
| 175 | +_MTL_INLINE MTL4::CompilerTaskOptions* MTL4::CompilerTaskOptions::init() | |
| 176 | +{ | |
| 177 | + return NS::Object::init<MTL4::CompilerTaskOptions>(); | |
| 178 | +} | |
| 179 | + | |
| 180 | +_MTL_INLINE NS::Array* MTL4::CompilerTaskOptions::lookupArchives() const | |
| 181 | +{ | |
| 182 | + return Object::sendMessage<NS::Array*>(this, _MTL_PRIVATE_SEL(lookupArchives)); | |
| 183 | +} | |
| 184 | + | |
| 185 | +_MTL_INLINE void MTL4::CompilerTaskOptions::setLookupArchives(const NS::Array* lookupArchives) | |
| 186 | +{ | |
| 187 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setLookupArchives_), lookupArchives); | |
| 188 | +} | |
| 189 | + | |
| 190 | +_MTL_INLINE MTL::Device* MTL4::Compiler::device() const | |
| 191 | +{ | |
| 192 | + return Object::sendMessage<MTL::Device*>(this, _MTL_PRIVATE_SEL(device)); | |
| 193 | +} | |
| 194 | + | |
| 195 | +_MTL_INLINE NS::String* MTL4::Compiler::label() const | |
| 196 | +{ | |
| 197 | + return Object::sendMessage<NS::String*>(this, _MTL_PRIVATE_SEL(label)); | |
| 198 | +} | |
| 199 | + | |
| 200 | +_MTL_INLINE MTL4::BinaryFunction* MTL4::Compiler::newBinaryFunction(const MTL4::BinaryFunctionDescriptor* descriptor, const MTL4::CompilerTaskOptions* compilerTaskOptions, NS::Error** error) | |
| 201 | +{ | |
| 202 | + return Object::sendMessage<MTL4::BinaryFunction*>(this, _MTL_PRIVATE_SEL(newBinaryFunctionWithDescriptor_compilerTaskOptions_error_), descriptor, compilerTaskOptions, error); | |
| 203 | +} | |
| 204 | + | |
| 205 | +_MTL_INLINE MTL4::CompilerTask* MTL4::Compiler::newBinaryFunction(const MTL4::BinaryFunctionDescriptor* descriptor, const MTL4::CompilerTaskOptions* compilerTaskOptions, const MTL4::NewBinaryFunctionCompletionHandler completionHandler) | |
| 206 | +{ | |
| 207 | + return Object::sendMessage<MTL4::CompilerTask*>(this, _MTL_PRIVATE_SEL(newBinaryFunctionWithDescriptor_compilerTaskOptions_completionHandler_), descriptor, compilerTaskOptions, completionHandler); | |
| 208 | +} | |
| 209 | + | |
| 210 | +_MTL_INLINE MTL::ComputePipelineState* MTL4::Compiler::newComputePipelineState(const MTL4::ComputePipelineDescriptor* descriptor, const MTL4::CompilerTaskOptions* compilerTaskOptions, NS::Error** error) | |
| 211 | +{ | |
| 212 | + return Object::sendMessage<MTL::ComputePipelineState*>(this, _MTL_PRIVATE_SEL(newComputePipelineStateWithDescriptor_compilerTaskOptions_error_), descriptor, compilerTaskOptions, error); | |
| 213 | +} | |
| 214 | + | |
| 215 | +_MTL_INLINE MTL::ComputePipelineState* MTL4::Compiler::newComputePipelineState(const MTL4::ComputePipelineDescriptor* descriptor, const MTL4::PipelineStageDynamicLinkingDescriptor* dynamicLinkingDescriptor, const MTL4::CompilerTaskOptions* compilerTaskOptions, NS::Error** error) | |
| 216 | +{ | |
| 217 | + return Object::sendMessage<MTL::ComputePipelineState*>(this, _MTL_PRIVATE_SEL(newComputePipelineStateWithDescriptor_dynamicLinkingDescriptor_compilerTaskOptions_error_), descriptor, dynamicLinkingDescriptor, compilerTaskOptions, error); | |
| 218 | +} | |
| 219 | + | |
| 220 | +_MTL_INLINE MTL4::CompilerTask* MTL4::Compiler::newComputePipelineState(const MTL4::ComputePipelineDescriptor* descriptor, const MTL4::CompilerTaskOptions* compilerTaskOptions, const MTL::NewComputePipelineStateCompletionHandler completionHandler) | |
| 221 | +{ | |
| 222 | + return Object::sendMessage<MTL4::CompilerTask*>(this, _MTL_PRIVATE_SEL(newComputePipelineStateWithDescriptor_compilerTaskOptions_completionHandler_), descriptor, compilerTaskOptions, completionHandler); | |
| 223 | +} | |
| 224 | + | |
| 225 | +_MTL_INLINE MTL4::CompilerTask* MTL4::Compiler::newComputePipelineState(const MTL4::ComputePipelineDescriptor* descriptor, const MTL4::PipelineStageDynamicLinkingDescriptor* dynamicLinkingDescriptor, const MTL4::CompilerTaskOptions* compilerTaskOptions, const MTL::NewComputePipelineStateCompletionHandler completionHandler) | |
| 226 | +{ | |
| 227 | + return Object::sendMessage<MTL4::CompilerTask*>(this, _MTL_PRIVATE_SEL(newComputePipelineStateWithDescriptor_dynamicLinkingDescriptor_compilerTaskOptions_completionHandler_), descriptor, dynamicLinkingDescriptor, compilerTaskOptions, completionHandler); | |
| 228 | +} | |
| 229 | + | |
| 230 | +_MTL_INLINE MTL4::CompilerTask* MTL4::Compiler::newComputePipelineState(const MTL4::ComputePipelineDescriptor* pDescriptor, const MTL4::CompilerTaskOptions* options, const MTL4::NewComputePipelineStateCompletionHandlerFunction& function) | |
| 231 | +{ | |
| 232 | + __block MTL4::NewComputePipelineStateCompletionHandlerFunction blockFunction = function; | |
| 233 | + return newComputePipelineState(pDescriptor, options, ^(MTL::ComputePipelineState* pPipeline, NS::Error* pError) { blockFunction(pPipeline, pError); }); | |
| 234 | +} | |
| 235 | + | |
| 236 | +_MTL_INLINE MTL::DynamicLibrary* MTL4::Compiler::newDynamicLibrary(const MTL::Library* library, NS::Error** error) | |
| 237 | +{ | |
| 238 | + return Object::sendMessage<MTL::DynamicLibrary*>(this, _MTL_PRIVATE_SEL(newDynamicLibrary_error_), library, error); | |
| 239 | +} | |
| 240 | + | |
| 241 | +_MTL_INLINE MTL::DynamicLibrary* MTL4::Compiler::newDynamicLibrary(const NS::URL* url, NS::Error** error) | |
| 242 | +{ | |
| 243 | + return Object::sendMessage<MTL::DynamicLibrary*>(this, _MTL_PRIVATE_SEL(newDynamicLibraryWithURL_error_), url, error); | |
| 244 | +} | |
| 245 | + | |
| 246 | +_MTL_INLINE MTL4::CompilerTask* MTL4::Compiler::newDynamicLibrary(const MTL::Library* library, const MTL::NewDynamicLibraryCompletionHandler completionHandler) | |
| 247 | +{ | |
| 248 | + return Object::sendMessage<MTL4::CompilerTask*>(this, _MTL_PRIVATE_SEL(newDynamicLibrary_completionHandler_), library, completionHandler); | |
| 249 | +} | |
| 250 | + | |
| 251 | +_MTL_INLINE MTL4::CompilerTask* MTL4::Compiler::newDynamicLibrary(const NS::URL* url, const MTL::NewDynamicLibraryCompletionHandler completionHandler) | |
| 252 | +{ | |
| 253 | + return Object::sendMessage<MTL4::CompilerTask*>(this, _MTL_PRIVATE_SEL(newDynamicLibraryWithURL_completionHandler_), url, completionHandler); | |
| 254 | +} | |
| 255 | + | |
| 256 | +_MTL_INLINE MTL4::CompilerTask* MTL4::Compiler::newDynamicLibrary(const MTL::Library* pLibrary, const MTL::NewDynamicLibraryCompletionHandlerFunction& function) | |
| 257 | +{ | |
| 258 | + __block MTL::NewDynamicLibraryCompletionHandlerFunction blockFunction = function; | |
| 259 | + return newDynamicLibrary(pLibrary, ^(MTL::DynamicLibrary* pLibraryRef, NS::Error* pError) { blockFunction(pLibraryRef, pError); }); | |
| 260 | +} | |
| 261 | + | |
| 262 | +_MTL_INLINE MTL4::CompilerTask* MTL4::Compiler::newDynamicLibrary(const NS::URL* pURL, const MTL::NewDynamicLibraryCompletionHandlerFunction& function) | |
| 263 | +{ | |
| 264 | + __block MTL::NewDynamicLibraryCompletionHandlerFunction blockFunction = function; | |
| 265 | + return newDynamicLibrary(pURL, ^(MTL::DynamicLibrary* pLibrary, NS::Error* pError) { blockFunction(pLibrary, pError); }); | |
| 266 | +} | |
| 267 | + | |
| 268 | +_MTL_INLINE MTL::Library* MTL4::Compiler::newLibrary(const MTL4::LibraryDescriptor* descriptor, NS::Error** error) | |
| 269 | +{ | |
| 270 | + return Object::sendMessage<MTL::Library*>(this, _MTL_PRIVATE_SEL(newLibraryWithDescriptor_error_), descriptor, error); | |
| 271 | +} | |
| 272 | + | |
| 273 | +_MTL_INLINE MTL4::CompilerTask* MTL4::Compiler::newLibrary(const MTL4::LibraryDescriptor* descriptor, const MTL::NewLibraryCompletionHandler completionHandler) | |
| 274 | +{ | |
| 275 | + return Object::sendMessage<MTL4::CompilerTask*>(this, _MTL_PRIVATE_SEL(newLibraryWithDescriptor_completionHandler_), descriptor, completionHandler); | |
| 276 | +} | |
| 277 | + | |
| 278 | +_MTL_INLINE MTL4::CompilerTask* MTL4::Compiler::newLibrary(const MTL4::LibraryDescriptor* pDescriptor, const MTL::NewLibraryCompletionHandlerFunction& function) | |
| 279 | +{ | |
| 280 | + __block MTL::NewLibraryCompletionHandlerFunction blockFunction = function; | |
| 281 | + return newLibrary(pDescriptor, ^(MTL::Library* pLibrary, NS::Error* pError) { blockFunction(pLibrary, pError); }); | |
| 282 | +} | |
| 283 | + | |
| 284 | +_MTL_INLINE MTL4::MachineLearningPipelineState* MTL4::Compiler::newMachineLearningPipelineState(const MTL4::MachineLearningPipelineDescriptor* descriptor, NS::Error** error) | |
| 285 | +{ | |
| 286 | + return Object::sendMessage<MTL4::MachineLearningPipelineState*>(this, _MTL_PRIVATE_SEL(newMachineLearningPipelineStateWithDescriptor_error_), descriptor, error); | |
| 287 | +} | |
| 288 | + | |
| 289 | +_MTL_INLINE MTL4::CompilerTask* MTL4::Compiler::newMachineLearningPipelineState(const MTL4::MachineLearningPipelineDescriptor* descriptor, const MTL4::NewMachineLearningPipelineStateCompletionHandler completionHandler) | |
| 290 | +{ | |
| 291 | + return Object::sendMessage<MTL4::CompilerTask*>(this, _MTL_PRIVATE_SEL(newMachineLearningPipelineStateWithDescriptor_completionHandler_), descriptor, completionHandler); | |
| 292 | +} | |
| 293 | + | |
| 294 | +_MTL_INLINE MTL4::CompilerTask* MTL4::Compiler::newMachineLearningPipelineState(const MTL4::MachineLearningPipelineDescriptor* pDescriptor, const MTL4::NewMachineLearningPipelineStateCompletionHandlerFunction& function) | |
| 295 | +{ | |
| 296 | + __block MTL4::NewMachineLearningPipelineStateCompletionHandlerFunction blockFunction = function; | |
| 297 | + return newMachineLearningPipelineState(pDescriptor, ^(MTL4::MachineLearningPipelineState* pPipeline, NS::Error* pError) { blockFunction(pPipeline, pError); }); | |
| 298 | +} | |
| 299 | + | |
| 300 | +_MTL_INLINE MTL::RenderPipelineState* MTL4::Compiler::newRenderPipelineState(const MTL4::PipelineDescriptor* descriptor, const MTL4::CompilerTaskOptions* compilerTaskOptions, NS::Error** error) | |
| 301 | +{ | |
| 302 | + return Object::sendMessage<MTL::RenderPipelineState*>(this, _MTL_PRIVATE_SEL(newRenderPipelineStateWithDescriptor_compilerTaskOptions_error_), descriptor, compilerTaskOptions, error); | |
| 303 | +} | |
| 304 | + | |
| 305 | +_MTL_INLINE MTL::RenderPipelineState* MTL4::Compiler::newRenderPipelineState(const MTL4::PipelineDescriptor* descriptor, const MTL4::RenderPipelineDynamicLinkingDescriptor* dynamicLinkingDescriptor, const MTL4::CompilerTaskOptions* compilerTaskOptions, NS::Error** error) | |
| 306 | +{ | |
| 307 | + return Object::sendMessage<MTL::RenderPipelineState*>(this, _MTL_PRIVATE_SEL(newRenderPipelineStateWithDescriptor_dynamicLinkingDescriptor_compilerTaskOptions_error_), descriptor, dynamicLinkingDescriptor, compilerTaskOptions, error); | |
| 308 | +} | |
| 309 | + | |
| 310 | +_MTL_INLINE MTL4::CompilerTask* MTL4::Compiler::newRenderPipelineState(const MTL4::PipelineDescriptor* descriptor, const MTL4::CompilerTaskOptions* compilerTaskOptions, const MTL::NewRenderPipelineStateCompletionHandler completionHandler) | |
| 311 | +{ | |
| 312 | + return Object::sendMessage<MTL4::CompilerTask*>(this, _MTL_PRIVATE_SEL(newRenderPipelineStateWithDescriptor_compilerTaskOptions_completionHandler_), descriptor, compilerTaskOptions, completionHandler); | |
| 313 | +} | |
| 314 | + | |
| 315 | +_MTL_INLINE MTL4::CompilerTask* MTL4::Compiler::newRenderPipelineState(const MTL4::PipelineDescriptor* descriptor, const MTL4::RenderPipelineDynamicLinkingDescriptor* dynamicLinkingDescriptor, const MTL4::CompilerTaskOptions* compilerTaskOptions, const MTL::NewRenderPipelineStateCompletionHandler completionHandler) | |
| 316 | +{ | |
| 317 | + return Object::sendMessage<MTL4::CompilerTask*>(this, _MTL_PRIVATE_SEL(newRenderPipelineStateWithDescriptor_dynamicLinkingDescriptor_compilerTaskOptions_completionHandler_), descriptor, dynamicLinkingDescriptor, compilerTaskOptions, completionHandler); | |
| 318 | +} | |
| 319 | + | |
| 320 | +_MTL_INLINE MTL4::CompilerTask* MTL4::Compiler::newRenderPipelineState(const MTL4::PipelineDescriptor* pDescriptor, const MTL4::CompilerTaskOptions* options, const MTL4::NewRenderPipelineStateCompletionHandlerFunction& function) | |
| 321 | +{ | |
| 322 | + __block MTL4::NewRenderPipelineStateCompletionHandlerFunction blockFunction = function; | |
| 323 | + return newRenderPipelineState(pDescriptor, options, ^(MTL::RenderPipelineState* pPipeline, NS::Error* pError) { blockFunction(pPipeline, pError); }); | |
| 324 | +} | |
| 325 | + | |
| 326 | +_MTL_INLINE MTL::RenderPipelineState* MTL4::Compiler::newRenderPipelineStateBySpecialization(const MTL4::PipelineDescriptor* descriptor, const MTL::RenderPipelineState* pipeline, NS::Error** error) | |
| 327 | +{ | |
| 328 | + return Object::sendMessage<MTL::RenderPipelineState*>(this, _MTL_PRIVATE_SEL(newRenderPipelineStateBySpecializationWithDescriptor_pipeline_error_), descriptor, pipeline, error); | |
| 329 | +} | |
| 330 | + | |
| 331 | +_MTL_INLINE MTL4::CompilerTask* MTL4::Compiler::newRenderPipelineStateBySpecialization(const MTL4::PipelineDescriptor* descriptor, const MTL::RenderPipelineState* pipeline, const MTL::NewRenderPipelineStateCompletionHandler completionHandler) | |
| 332 | +{ | |
| 333 | + return Object::sendMessage<MTL4::CompilerTask*>(this, _MTL_PRIVATE_SEL(newRenderPipelineStateBySpecializationWithDescriptor_pipeline_completionHandler_), descriptor, pipeline, completionHandler); | |
| 334 | +} | |
| 335 | + | |
| 336 | +_MTL_INLINE MTL4::CompilerTask* MTL4::Compiler::newRenderPipelineStateBySpecialization(const MTL4::PipelineDescriptor* pDescriptor, const MTL::RenderPipelineState* pPipeline, const MTL4::NewRenderPipelineStateCompletionHandlerFunction& function) | |
| 337 | +{ | |
| 338 | + __block MTL4::NewRenderPipelineStateCompletionHandlerFunction blockFunction = function; | |
| 339 | + return newRenderPipelineStateBySpecialization(pDescriptor, pPipeline, ^(MTL::RenderPipelineState* pPipelineRef, NS::Error* pError) { blockFunction(pPipelineRef, pError); }); | |
| 340 | +} | |
| 341 | + | |
| 342 | +_MTL_INLINE MTL4::PipelineDataSetSerializer* MTL4::Compiler::pipelineDataSetSerializer() const | |
| 343 | +{ | |
| 344 | + return Object::sendMessage<MTL4::PipelineDataSetSerializer*>(this, _MTL_PRIVATE_SEL(pipelineDataSetSerializer)); | |
| 345 | +} | |
added
third_party/metal-cpp/Metal/MTL4CompilerTask.hpp
+63 −0
@@ -0,0 +1,63 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Metal/MTL4CompilerTask.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2025 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include "../Foundation/Foundation.hpp" | |
| 24 | +#include "MTLDefines.hpp" | |
| 25 | +#include "MTLHeaderBridge.hpp" | |
| 26 | +#include "MTLPrivate.hpp" | |
| 27 | + | |
| 28 | +namespace MTL4 | |
| 29 | +{ | |
| 30 | +class Compiler; | |
| 31 | +_MTL_ENUM(NS::Integer, CompilerTaskStatus) { | |
| 32 | + CompilerTaskStatusNone = 0, | |
| 33 | + CompilerTaskStatusScheduled = 1, | |
| 34 | + CompilerTaskStatusCompiling = 2, | |
| 35 | + CompilerTaskStatusFinished = 3, | |
| 36 | +}; | |
| 37 | + | |
| 38 | +class CompilerTask : public NS::Referencing<CompilerTask> | |
| 39 | +{ | |
| 40 | +public: | |
| 41 | + Compiler* compiler() const; | |
| 42 | + | |
| 43 | + CompilerTaskStatus status() const; | |
| 44 | + | |
| 45 | + void waitUntilCompleted(); | |
| 46 | +}; | |
| 47 | + | |
| 48 | +} | |
| 49 | + | |
| 50 | +_MTL_INLINE MTL4::Compiler* MTL4::CompilerTask::compiler() const | |
| 51 | +{ | |
| 52 | + return Object::sendMessage<MTL4::Compiler*>(this, _MTL_PRIVATE_SEL(compiler)); | |
| 53 | +} | |
| 54 | + | |
| 55 | +_MTL_INLINE MTL4::CompilerTaskStatus MTL4::CompilerTask::status() const | |
| 56 | +{ | |
| 57 | + return Object::sendMessage<MTL4::CompilerTaskStatus>(this, _MTL_PRIVATE_SEL(status)); | |
| 58 | +} | |
| 59 | + | |
| 60 | +_MTL_INLINE void MTL4::CompilerTask::waitUntilCompleted() | |
| 61 | +{ | |
| 62 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(waitUntilCompleted)); | |
| 63 | +} | |
added
third_party/metal-cpp/Metal/MTL4ComputeCommandEncoder.hpp
+307 −0
@@ -0,0 +1,307 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Metal/MTL4ComputeCommandEncoder.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2025 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include "../Foundation/Foundation.hpp" | |
| 24 | +#include "MTL4CommandEncoder.hpp" | |
| 25 | +#include "MTL4Counters.hpp" | |
| 26 | +#include "MTLAccelerationStructure.hpp" | |
| 27 | +#include "MTLAccelerationStructureTypes.hpp" | |
| 28 | +#include "MTLBlitCommandEncoder.hpp" | |
| 29 | +#include "MTLCommandEncoder.hpp" | |
| 30 | +#include "MTLDefines.hpp" | |
| 31 | +#include "MTLGPUAddress.hpp" | |
| 32 | +#include "MTLHeaderBridge.hpp" | |
| 33 | +#include "MTLPrivate.hpp" | |
| 34 | +#include "MTLTensor.hpp" | |
| 35 | +#include "MTLTypes.hpp" | |
| 36 | +#include <cstdint> | |
| 37 | + | |
| 38 | +namespace MTL4 | |
| 39 | +{ | |
| 40 | +class AccelerationStructureDescriptor; | |
| 41 | +class ArgumentTable; | |
| 42 | +class CounterHeap; | |
| 43 | +} | |
| 44 | + | |
| 45 | +namespace MTL | |
| 46 | +{ | |
| 47 | +class AccelerationStructure; | |
| 48 | +class Buffer; | |
| 49 | +class ComputePipelineState; | |
| 50 | +class IndirectCommandBuffer; | |
| 51 | +class Tensor; | |
| 52 | +class TensorExtents; | |
| 53 | +class Texture; | |
| 54 | +} | |
| 55 | + | |
| 56 | +namespace MTL4 | |
| 57 | +{ | |
| 58 | +class ComputeCommandEncoder : public NS::Referencing<ComputeCommandEncoder, CommandEncoder> | |
| 59 | +{ | |
| 60 | +public: | |
| 61 | + void buildAccelerationStructure(const MTL::AccelerationStructure* accelerationStructure, const MTL4::AccelerationStructureDescriptor* descriptor, const MTL4::BufferRange scratchBuffer); | |
| 62 | + | |
| 63 | + void copyAccelerationStructure(const MTL::AccelerationStructure* sourceAccelerationStructure, const MTL::AccelerationStructure* destinationAccelerationStructure); | |
| 64 | + | |
| 65 | + void copyAndCompactAccelerationStructure(const MTL::AccelerationStructure* sourceAccelerationStructure, const MTL::AccelerationStructure* destinationAccelerationStructure); | |
| 66 | + | |
| 67 | + void copyFromBuffer(const MTL::Buffer* sourceBuffer, NS::UInteger sourceOffset, const MTL::Buffer* destinationBuffer, NS::UInteger destinationOffset, NS::UInteger size); | |
| 68 | + void copyFromBuffer(const MTL::Buffer* sourceBuffer, NS::UInteger sourceOffset, NS::UInteger sourceBytesPerRow, NS::UInteger sourceBytesPerImage, MTL::Size sourceSize, const MTL::Texture* destinationTexture, NS::UInteger destinationSlice, NS::UInteger destinationLevel, MTL::Origin destinationOrigin); | |
| 69 | + void copyFromBuffer(const MTL::Buffer* sourceBuffer, NS::UInteger sourceOffset, NS::UInteger sourceBytesPerRow, NS::UInteger sourceBytesPerImage, MTL::Size sourceSize, const MTL::Texture* destinationTexture, NS::UInteger destinationSlice, NS::UInteger destinationLevel, MTL::Origin destinationOrigin, MTL::BlitOption options); | |
| 70 | + | |
| 71 | + void copyFromTensor(const MTL::Tensor* sourceTensor, const MTL::TensorExtents* sourceOrigin, const MTL::TensorExtents* sourceDimensions, const MTL::Tensor* destinationTensor, const MTL::TensorExtents* destinationOrigin, const MTL::TensorExtents* destinationDimensions); | |
| 72 | + void copyFromTensor(const MTL::Tensor* sourceTensor, const MTL::TensorExtents* sourceOrigin, const MTL::TensorExtents* sourceDimensions, MTL::TensorPlaneType sourcePlane, const MTL::Tensor* destinationTensor, const MTL::TensorExtents* destinationOrigin, const MTL::TensorExtents* destinationDimensions, MTL::TensorPlaneType destinationPlane); | |
| 73 | + | |
| 74 | + void copyFromTexture(const MTL::Texture* sourceTexture, const MTL::Texture* destinationTexture); | |
| 75 | + void copyFromTexture(const MTL::Texture* sourceTexture, NS::UInteger sourceSlice, NS::UInteger sourceLevel, const MTL::Texture* destinationTexture, NS::UInteger destinationSlice, NS::UInteger destinationLevel, NS::UInteger sliceCount, NS::UInteger levelCount); | |
| 76 | + void copyFromTexture(const MTL::Texture* sourceTexture, NS::UInteger sourceSlice, NS::UInteger sourceLevel, MTL::Origin sourceOrigin, MTL::Size sourceSize, const MTL::Texture* destinationTexture, NS::UInteger destinationSlice, NS::UInteger destinationLevel, MTL::Origin destinationOrigin); | |
| 77 | + void copyFromTexture(const MTL::Texture* sourceTexture, NS::UInteger sourceSlice, NS::UInteger sourceLevel, MTL::Origin sourceOrigin, MTL::Size sourceSize, const MTL::Buffer* destinationBuffer, NS::UInteger destinationOffset, NS::UInteger destinationBytesPerRow, NS::UInteger destinationBytesPerImage); | |
| 78 | + void copyFromTexture(const MTL::Texture* sourceTexture, NS::UInteger sourceSlice, NS::UInteger sourceLevel, MTL::Origin sourceOrigin, MTL::Size sourceSize, const MTL::Buffer* destinationBuffer, NS::UInteger destinationOffset, NS::UInteger destinationBytesPerRow, NS::UInteger destinationBytesPerImage, MTL::BlitOption options); | |
| 79 | + | |
| 80 | + void copyIndirectCommandBuffer(const MTL::IndirectCommandBuffer* source, NS::Range sourceRange, const MTL::IndirectCommandBuffer* destination, NS::UInteger destinationIndex); | |
| 81 | + | |
| 82 | + void dispatchThreadgroups(MTL::Size threadgroupsPerGrid, MTL::Size threadsPerThreadgroup); | |
| 83 | + void dispatchThreadgroups(MTL::GPUAddress indirectBuffer, MTL::Size threadsPerThreadgroup); | |
| 84 | + | |
| 85 | + void dispatchThreads(MTL::Size threadsPerGrid, MTL::Size threadsPerThreadgroup); | |
| 86 | + void dispatchThreads(MTL::GPUAddress indirectBuffer); | |
| 87 | + | |
| 88 | + void executeCommandsInBuffer(const MTL::IndirectCommandBuffer* indirectCommandBuffer, NS::Range executionRange); | |
| 89 | + void executeCommandsInBuffer(const MTL::IndirectCommandBuffer* indirectCommandbuffer, MTL::GPUAddress indirectRangeBuffer); | |
| 90 | + | |
| 91 | + void fillBuffer(const MTL::Buffer* buffer, NS::Range range, uint8_t value); | |
| 92 | + | |
| 93 | + void generateMipmaps(const MTL::Texture* texture); | |
| 94 | + | |
| 95 | + void optimizeContentsForCPUAccess(const MTL::Texture* texture); | |
| 96 | + void optimizeContentsForCPUAccess(const MTL::Texture* texture, NS::UInteger slice, NS::UInteger level); | |
| 97 | + | |
| 98 | + void optimizeContentsForGPUAccess(const MTL::Texture* texture); | |
| 99 | + void optimizeContentsForGPUAccess(const MTL::Texture* texture, NS::UInteger slice, NS::UInteger level); | |
| 100 | + | |
| 101 | + void optimizeIndirectCommandBuffer(const MTL::IndirectCommandBuffer* indirectCommandBuffer, NS::Range range); | |
| 102 | + | |
| 103 | + void refitAccelerationStructure(const MTL::AccelerationStructure* sourceAccelerationStructure, const MTL4::AccelerationStructureDescriptor* descriptor, const MTL::AccelerationStructure* destinationAccelerationStructure, const MTL4::BufferRange scratchBuffer); | |
| 104 | + void refitAccelerationStructure(const MTL::AccelerationStructure* sourceAccelerationStructure, const MTL4::AccelerationStructureDescriptor* descriptor, const MTL::AccelerationStructure* destinationAccelerationStructure, const MTL4::BufferRange scratchBuffer, MTL::AccelerationStructureRefitOptions options); | |
| 105 | + | |
| 106 | + void resetCommandsInBuffer(const MTL::IndirectCommandBuffer* buffer, NS::Range range); | |
| 107 | + | |
| 108 | + void setArgumentTable(const MTL4::ArgumentTable* argumentTable); | |
| 109 | + | |
| 110 | + void setComputePipelineState(const MTL::ComputePipelineState* state); | |
| 111 | + | |
| 112 | + void setImageblockWidth(NS::UInteger width, NS::UInteger height); | |
| 113 | + | |
| 114 | + void setThreadgroupMemoryLength(NS::UInteger length, NS::UInteger index); | |
| 115 | + | |
| 116 | + MTL::Stages stages(); | |
| 117 | + | |
| 118 | + void writeCompactedAccelerationStructureSize(const MTL::AccelerationStructure* accelerationStructure, const MTL4::BufferRange buffer); | |
| 119 | + | |
| 120 | + void writeTimestamp(MTL4::TimestampGranularity granularity, const MTL4::CounterHeap* counterHeap, NS::UInteger index); | |
| 121 | +}; | |
| 122 | + | |
| 123 | +} | |
| 124 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::buildAccelerationStructure(const MTL::AccelerationStructure* accelerationStructure, const MTL4::AccelerationStructureDescriptor* descriptor, const MTL4::BufferRange scratchBuffer) | |
| 125 | +{ | |
| 126 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(buildAccelerationStructure_descriptor_scratchBuffer_), accelerationStructure, descriptor, scratchBuffer); | |
| 127 | +} | |
| 128 | + | |
| 129 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::copyAccelerationStructure(const MTL::AccelerationStructure* sourceAccelerationStructure, const MTL::AccelerationStructure* destinationAccelerationStructure) | |
| 130 | +{ | |
| 131 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(copyAccelerationStructure_toAccelerationStructure_), sourceAccelerationStructure, destinationAccelerationStructure); | |
| 132 | +} | |
| 133 | + | |
| 134 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::copyAndCompactAccelerationStructure(const MTL::AccelerationStructure* sourceAccelerationStructure, const MTL::AccelerationStructure* destinationAccelerationStructure) | |
| 135 | +{ | |
| 136 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(copyAndCompactAccelerationStructure_toAccelerationStructure_), sourceAccelerationStructure, destinationAccelerationStructure); | |
| 137 | +} | |
| 138 | + | |
| 139 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::copyFromBuffer(const MTL::Buffer* sourceBuffer, NS::UInteger sourceOffset, const MTL::Buffer* destinationBuffer, NS::UInteger destinationOffset, NS::UInteger size) | |
| 140 | +{ | |
| 141 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(copyFromBuffer_sourceOffset_toBuffer_destinationOffset_size_), sourceBuffer, sourceOffset, destinationBuffer, destinationOffset, size); | |
| 142 | +} | |
| 143 | + | |
| 144 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::copyFromBuffer(const MTL::Buffer* sourceBuffer, NS::UInteger sourceOffset, NS::UInteger sourceBytesPerRow, NS::UInteger sourceBytesPerImage, MTL::Size sourceSize, const MTL::Texture* destinationTexture, NS::UInteger destinationSlice, NS::UInteger destinationLevel, MTL::Origin destinationOrigin) | |
| 145 | +{ | |
| 146 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(copyFromBuffer_sourceOffset_sourceBytesPerRow_sourceBytesPerImage_sourceSize_toTexture_destinationSlice_destinationLevel_destinationOrigin_), sourceBuffer, sourceOffset, sourceBytesPerRow, sourceBytesPerImage, sourceSize, destinationTexture, destinationSlice, destinationLevel, destinationOrigin); | |
| 147 | +} | |
| 148 | + | |
| 149 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::copyFromBuffer(const MTL::Buffer* sourceBuffer, NS::UInteger sourceOffset, NS::UInteger sourceBytesPerRow, NS::UInteger sourceBytesPerImage, MTL::Size sourceSize, const MTL::Texture* destinationTexture, NS::UInteger destinationSlice, NS::UInteger destinationLevel, MTL::Origin destinationOrigin, MTL::BlitOption options) | |
| 150 | +{ | |
| 151 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(copyFromBuffer_sourceOffset_sourceBytesPerRow_sourceBytesPerImage_sourceSize_toTexture_destinationSlice_destinationLevel_destinationOrigin_options_), sourceBuffer, sourceOffset, sourceBytesPerRow, sourceBytesPerImage, sourceSize, destinationTexture, destinationSlice, destinationLevel, destinationOrigin, options); | |
| 152 | +} | |
| 153 | + | |
| 154 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::copyFromTensor(const MTL::Tensor* sourceTensor, const MTL::TensorExtents* sourceOrigin, const MTL::TensorExtents* sourceDimensions, const MTL::Tensor* destinationTensor, const MTL::TensorExtents* destinationOrigin, const MTL::TensorExtents* destinationDimensions) | |
| 155 | +{ | |
| 156 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(copyFromTensor_sourceOrigin_sourceDimensions_toTensor_destinationOrigin_destinationDimensions_), sourceTensor, sourceOrigin, sourceDimensions, destinationTensor, destinationOrigin, destinationDimensions); | |
| 157 | +} | |
| 158 | + | |
| 159 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::copyFromTensor(const MTL::Tensor* sourceTensor, const MTL::TensorExtents* sourceOrigin, const MTL::TensorExtents* sourceDimensions, MTL::TensorPlaneType sourcePlane, const MTL::Tensor* destinationTensor, const MTL::TensorExtents* destinationOrigin, const MTL::TensorExtents* destinationDimensions, MTL::TensorPlaneType destinationPlane) | |
| 160 | +{ | |
| 161 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(copyFromTensor_sourceOrigin_sourceDimensions_sourcePlane_toTensor_destinationOrigin_destinationDimensions_destinationPlane_), sourceTensor, sourceOrigin, sourceDimensions, sourcePlane, destinationTensor, destinationOrigin, destinationDimensions, destinationPlane); | |
| 162 | +} | |
| 163 | + | |
| 164 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::copyFromTexture(const MTL::Texture* sourceTexture, const MTL::Texture* destinationTexture) | |
| 165 | +{ | |
| 166 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(copyFromTexture_toTexture_), sourceTexture, destinationTexture); | |
| 167 | +} | |
| 168 | + | |
| 169 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::copyFromTexture(const MTL::Texture* sourceTexture, NS::UInteger sourceSlice, NS::UInteger sourceLevel, const MTL::Texture* destinationTexture, NS::UInteger destinationSlice, NS::UInteger destinationLevel, NS::UInteger sliceCount, NS::UInteger levelCount) | |
| 170 | +{ | |
| 171 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(copyFromTexture_sourceSlice_sourceLevel_toTexture_destinationSlice_destinationLevel_sliceCount_levelCount_), sourceTexture, sourceSlice, sourceLevel, destinationTexture, destinationSlice, destinationLevel, sliceCount, levelCount); | |
| 172 | +} | |
| 173 | + | |
| 174 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::copyFromTexture(const MTL::Texture* sourceTexture, NS::UInteger sourceSlice, NS::UInteger sourceLevel, MTL::Origin sourceOrigin, MTL::Size sourceSize, const MTL::Texture* destinationTexture, NS::UInteger destinationSlice, NS::UInteger destinationLevel, MTL::Origin destinationOrigin) | |
| 175 | +{ | |
| 176 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(copyFromTexture_sourceSlice_sourceLevel_sourceOrigin_sourceSize_toTexture_destinationSlice_destinationLevel_destinationOrigin_), sourceTexture, sourceSlice, sourceLevel, sourceOrigin, sourceSize, destinationTexture, destinationSlice, destinationLevel, destinationOrigin); | |
| 177 | +} | |
| 178 | + | |
| 179 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::copyFromTexture(const MTL::Texture* sourceTexture, NS::UInteger sourceSlice, NS::UInteger sourceLevel, MTL::Origin sourceOrigin, MTL::Size sourceSize, const MTL::Buffer* destinationBuffer, NS::UInteger destinationOffset, NS::UInteger destinationBytesPerRow, NS::UInteger destinationBytesPerImage) | |
| 180 | +{ | |
| 181 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(copyFromTexture_sourceSlice_sourceLevel_sourceOrigin_sourceSize_toBuffer_destinationOffset_destinationBytesPerRow_destinationBytesPerImage_), sourceTexture, sourceSlice, sourceLevel, sourceOrigin, sourceSize, destinationBuffer, destinationOffset, destinationBytesPerRow, destinationBytesPerImage); | |
| 182 | +} | |
| 183 | + | |
| 184 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::copyFromTexture(const MTL::Texture* sourceTexture, NS::UInteger sourceSlice, NS::UInteger sourceLevel, MTL::Origin sourceOrigin, MTL::Size sourceSize, const MTL::Buffer* destinationBuffer, NS::UInteger destinationOffset, NS::UInteger destinationBytesPerRow, NS::UInteger destinationBytesPerImage, MTL::BlitOption options) | |
| 185 | +{ | |
| 186 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(copyFromTexture_sourceSlice_sourceLevel_sourceOrigin_sourceSize_toBuffer_destinationOffset_destinationBytesPerRow_destinationBytesPerImage_options_), sourceTexture, sourceSlice, sourceLevel, sourceOrigin, sourceSize, destinationBuffer, destinationOffset, destinationBytesPerRow, destinationBytesPerImage, options); | |
| 187 | +} | |
| 188 | + | |
| 189 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::copyIndirectCommandBuffer(const MTL::IndirectCommandBuffer* source, NS::Range sourceRange, const MTL::IndirectCommandBuffer* destination, NS::UInteger destinationIndex) | |
| 190 | +{ | |
| 191 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(copyIndirectCommandBuffer_sourceRange_destination_destinationIndex_), source, sourceRange, destination, destinationIndex); | |
| 192 | +} | |
| 193 | + | |
| 194 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::dispatchThreadgroups(MTL::Size threadgroupsPerGrid, MTL::Size threadsPerThreadgroup) | |
| 195 | +{ | |
| 196 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(dispatchThreadgroups_threadsPerThreadgroup_), threadgroupsPerGrid, threadsPerThreadgroup); | |
| 197 | +} | |
| 198 | + | |
| 199 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::dispatchThreadgroups(MTL::GPUAddress indirectBuffer, MTL::Size threadsPerThreadgroup) | |
| 200 | +{ | |
| 201 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(dispatchThreadgroupsWithIndirectBuffer_threadsPerThreadgroup_), indirectBuffer, threadsPerThreadgroup); | |
| 202 | +} | |
| 203 | + | |
| 204 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::dispatchThreads(MTL::Size threadsPerGrid, MTL::Size threadsPerThreadgroup) | |
| 205 | +{ | |
| 206 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(dispatchThreads_threadsPerThreadgroup_), threadsPerGrid, threadsPerThreadgroup); | |
| 207 | +} | |
| 208 | + | |
| 209 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::dispatchThreads(MTL::GPUAddress indirectBuffer) | |
| 210 | +{ | |
| 211 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(dispatchThreadsWithIndirectBuffer_), indirectBuffer); | |
| 212 | +} | |
| 213 | + | |
| 214 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::executeCommandsInBuffer(const MTL::IndirectCommandBuffer* indirectCommandBuffer, NS::Range executionRange) | |
| 215 | +{ | |
| 216 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(executeCommandsInBuffer_withRange_), indirectCommandBuffer, executionRange); | |
| 217 | +} | |
| 218 | + | |
| 219 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::executeCommandsInBuffer(const MTL::IndirectCommandBuffer* indirectCommandbuffer, MTL::GPUAddress indirectRangeBuffer) | |
| 220 | +{ | |
| 221 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(executeCommandsInBuffer_indirectBuffer_), indirectCommandbuffer, indirectRangeBuffer); | |
| 222 | +} | |
| 223 | + | |
| 224 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::fillBuffer(const MTL::Buffer* buffer, NS::Range range, uint8_t value) | |
| 225 | +{ | |
| 226 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(fillBuffer_range_value_), buffer, range, value); | |
| 227 | +} | |
| 228 | + | |
| 229 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::generateMipmaps(const MTL::Texture* texture) | |
| 230 | +{ | |
| 231 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(generateMipmapsForTexture_), texture); | |
| 232 | +} | |
| 233 | + | |
| 234 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::optimizeContentsForCPUAccess(const MTL::Texture* texture) | |
| 235 | +{ | |
| 236 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(optimizeContentsForCPUAccess_), texture); | |
| 237 | +} | |
| 238 | + | |
| 239 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::optimizeContentsForCPUAccess(const MTL::Texture* texture, NS::UInteger slice, NS::UInteger level) | |
| 240 | +{ | |
| 241 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(optimizeContentsForCPUAccess_slice_level_), texture, slice, level); | |
| 242 | +} | |
| 243 | + | |
| 244 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::optimizeContentsForGPUAccess(const MTL::Texture* texture) | |
| 245 | +{ | |
| 246 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(optimizeContentsForGPUAccess_), texture); | |
| 247 | +} | |
| 248 | + | |
| 249 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::optimizeContentsForGPUAccess(const MTL::Texture* texture, NS::UInteger slice, NS::UInteger level) | |
| 250 | +{ | |
| 251 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(optimizeContentsForGPUAccess_slice_level_), texture, slice, level); | |
| 252 | +} | |
| 253 | + | |
| 254 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::optimizeIndirectCommandBuffer(const MTL::IndirectCommandBuffer* indirectCommandBuffer, NS::Range range) | |
| 255 | +{ | |
| 256 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(optimizeIndirectCommandBuffer_withRange_), indirectCommandBuffer, range); | |
| 257 | +} | |
| 258 | + | |
| 259 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::refitAccelerationStructure(const MTL::AccelerationStructure* sourceAccelerationStructure, const MTL4::AccelerationStructureDescriptor* descriptor, const MTL::AccelerationStructure* destinationAccelerationStructure, const MTL4::BufferRange scratchBuffer) | |
| 260 | +{ | |
| 261 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(refitAccelerationStructure_descriptor_destination_scratchBuffer_), sourceAccelerationStructure, descriptor, destinationAccelerationStructure, scratchBuffer); | |
| 262 | +} | |
| 263 | + | |
| 264 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::refitAccelerationStructure(const MTL::AccelerationStructure* sourceAccelerationStructure, const MTL4::AccelerationStructureDescriptor* descriptor, const MTL::AccelerationStructure* destinationAccelerationStructure, const MTL4::BufferRange scratchBuffer, MTL::AccelerationStructureRefitOptions options) | |
| 265 | +{ | |
| 266 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(refitAccelerationStructure_descriptor_destination_scratchBuffer_options_), sourceAccelerationStructure, descriptor, destinationAccelerationStructure, scratchBuffer, options); | |
| 267 | +} | |
| 268 | + | |
| 269 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::resetCommandsInBuffer(const MTL::IndirectCommandBuffer* buffer, NS::Range range) | |
| 270 | +{ | |
| 271 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(resetCommandsInBuffer_withRange_), buffer, range); | |
| 272 | +} | |
| 273 | + | |
| 274 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::setArgumentTable(const MTL4::ArgumentTable* argumentTable) | |
| 275 | +{ | |
| 276 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setArgumentTable_), argumentTable); | |
| 277 | +} | |
| 278 | + | |
| 279 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::setComputePipelineState(const MTL::ComputePipelineState* state) | |
| 280 | +{ | |
| 281 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setComputePipelineState_), state); | |
| 282 | +} | |
| 283 | + | |
| 284 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::setImageblockWidth(NS::UInteger width, NS::UInteger height) | |
| 285 | +{ | |
| 286 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setImageblockWidth_height_), width, height); | |
| 287 | +} | |
| 288 | + | |
| 289 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::setThreadgroupMemoryLength(NS::UInteger length, NS::UInteger index) | |
| 290 | +{ | |
| 291 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setThreadgroupMemoryLength_atIndex_), length, index); | |
| 292 | +} | |
| 293 | + | |
| 294 | +_MTL_INLINE MTL::Stages MTL4::ComputeCommandEncoder::stages() | |
| 295 | +{ | |
| 296 | + return Object::sendMessage<MTL::Stages>(this, _MTL_PRIVATE_SEL(stages)); | |
| 297 | +} | |
| 298 | + | |
| 299 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::writeCompactedAccelerationStructureSize(const MTL::AccelerationStructure* accelerationStructure, const MTL4::BufferRange buffer) | |
| 300 | +{ | |
| 301 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(writeCompactedAccelerationStructureSize_toBuffer_), accelerationStructure, buffer); | |
| 302 | +} | |
| 303 | + | |
| 304 | +_MTL_INLINE void MTL4::ComputeCommandEncoder::writeTimestamp(MTL4::TimestampGranularity granularity, const MTL4::CounterHeap* counterHeap, NS::UInteger index) | |
| 305 | +{ | |
| 306 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(writeTimestampWithGranularity_intoHeap_atIndex_), granularity, counterHeap, index); | |
| 307 | +} | |
added
third_party/metal-cpp/Metal/MTL4ComputePipeline.hpp
+158 −0
@@ -0,0 +1,158 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Metal/MTL4ComputePipeline.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2025 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include "../Foundation/Foundation.hpp" | |
| 24 | +#include "MTL4PipelineState.hpp" | |
| 25 | +#include "MTLDefines.hpp" | |
| 26 | +#include "MTLHeaderBridge.hpp" | |
| 27 | +#include "MTLPrivate.hpp" | |
| 28 | +#include "MTLTypes.hpp" | |
| 29 | + | |
| 30 | +namespace MTL4 | |
| 31 | +{ | |
| 32 | +class ComputePipelineDescriptor; | |
| 33 | +class FunctionDescriptor; | |
| 34 | +class StaticLinkingDescriptor; | |
| 35 | + | |
| 36 | +class ComputePipelineDescriptor : public NS::Copying<ComputePipelineDescriptor, PipelineDescriptor> | |
| 37 | +{ | |
| 38 | +public: | |
| 39 | + static ComputePipelineDescriptor* alloc(); | |
| 40 | + | |
| 41 | + FunctionDescriptor* computeFunctionDescriptor() const; | |
| 42 | + | |
| 43 | + ComputePipelineDescriptor* init(); | |
| 44 | + | |
| 45 | + NS::UInteger maxTotalThreadsPerThreadgroup() const; | |
| 46 | + | |
| 47 | + MTL::Size requiredThreadsPerThreadgroup() const; | |
| 48 | + | |
| 49 | + void reset(); | |
| 50 | + | |
| 51 | + void setComputeFunctionDescriptor(const MTL4::FunctionDescriptor* computeFunctionDescriptor); | |
| 52 | + | |
| 53 | + void setMaxTotalThreadsPerThreadgroup(NS::UInteger maxTotalThreadsPerThreadgroup); | |
| 54 | + | |
| 55 | + void setRequiredThreadsPerThreadgroup(MTL::Size requiredThreadsPerThreadgroup); | |
| 56 | + | |
| 57 | + void setStaticLinkingDescriptor(const MTL4::StaticLinkingDescriptor* staticLinkingDescriptor); | |
| 58 | + | |
| 59 | + void setSupportBinaryLinking(bool supportBinaryLinking); | |
| 60 | + | |
| 61 | + void setSupportIndirectCommandBuffers(MTL4::IndirectCommandBufferSupportState supportIndirectCommandBuffers); | |
| 62 | + | |
| 63 | + void setThreadGroupSizeIsMultipleOfThreadExecutionWidth(bool threadGroupSizeIsMultipleOfThreadExecutionWidth); | |
| 64 | + | |
| 65 | + StaticLinkingDescriptor* staticLinkingDescriptor() const; | |
| 66 | + | |
| 67 | + bool supportBinaryLinking() const; | |
| 68 | + | |
| 69 | + IndirectCommandBufferSupportState supportIndirectCommandBuffers() const; | |
| 70 | + | |
| 71 | + bool threadGroupSizeIsMultipleOfThreadExecutionWidth() const; | |
| 72 | +}; | |
| 73 | + | |
| 74 | +} | |
| 75 | +_MTL_INLINE MTL4::ComputePipelineDescriptor* MTL4::ComputePipelineDescriptor::alloc() | |
| 76 | +{ | |
| 77 | + return NS::Object::alloc<MTL4::ComputePipelineDescriptor>(_MTL_PRIVATE_CLS(MTL4ComputePipelineDescriptor)); | |
| 78 | +} | |
| 79 | + | |
| 80 | +_MTL_INLINE MTL4::FunctionDescriptor* MTL4::ComputePipelineDescriptor::computeFunctionDescriptor() const | |
| 81 | +{ | |
| 82 | + return Object::sendMessage<MTL4::FunctionDescriptor*>(this, _MTL_PRIVATE_SEL(computeFunctionDescriptor)); | |
| 83 | +} | |
| 84 | + | |
| 85 | +_MTL_INLINE MTL4::ComputePipelineDescriptor* MTL4::ComputePipelineDescriptor::init() | |
| 86 | +{ | |
| 87 | + return NS::Object::init<MTL4::ComputePipelineDescriptor>(); | |
| 88 | +} | |
| 89 | + | |
| 90 | +_MTL_INLINE NS::UInteger MTL4::ComputePipelineDescriptor::maxTotalThreadsPerThreadgroup() const | |
| 91 | +{ | |
| 92 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(maxTotalThreadsPerThreadgroup)); | |
| 93 | +} | |
| 94 | + | |
| 95 | +_MTL_INLINE MTL::Size MTL4::ComputePipelineDescriptor::requiredThreadsPerThreadgroup() const | |
| 96 | +{ | |
| 97 | + return Object::sendMessage<MTL::Size>(this, _MTL_PRIVATE_SEL(requiredThreadsPerThreadgroup)); | |
| 98 | +} | |
| 99 | + | |
| 100 | +_MTL_INLINE void MTL4::ComputePipelineDescriptor::reset() | |
| 101 | +{ | |
| 102 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(reset)); | |
| 103 | +} | |
| 104 | + | |
| 105 | +_MTL_INLINE void MTL4::ComputePipelineDescriptor::setComputeFunctionDescriptor(const MTL4::FunctionDescriptor* computeFunctionDescriptor) | |
| 106 | +{ | |
| 107 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setComputeFunctionDescriptor_), computeFunctionDescriptor); | |
| 108 | +} | |
| 109 | + | |
| 110 | +_MTL_INLINE void MTL4::ComputePipelineDescriptor::setMaxTotalThreadsPerThreadgroup(NS::UInteger maxTotalThreadsPerThreadgroup) | |
| 111 | +{ | |
| 112 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMaxTotalThreadsPerThreadgroup_), maxTotalThreadsPerThreadgroup); | |
| 113 | +} | |
| 114 | + | |
| 115 | +_MTL_INLINE void MTL4::ComputePipelineDescriptor::setRequiredThreadsPerThreadgroup(MTL::Size requiredThreadsPerThreadgroup) | |
| 116 | +{ | |
| 117 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setRequiredThreadsPerThreadgroup_), requiredThreadsPerThreadgroup); | |
| 118 | +} | |
| 119 | + | |
| 120 | +_MTL_INLINE void MTL4::ComputePipelineDescriptor::setStaticLinkingDescriptor(const MTL4::StaticLinkingDescriptor* staticLinkingDescriptor) | |
| 121 | +{ | |
| 122 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setStaticLinkingDescriptor_), staticLinkingDescriptor); | |
| 123 | +} | |
| 124 | + | |
| 125 | +_MTL_INLINE void MTL4::ComputePipelineDescriptor::setSupportBinaryLinking(bool supportBinaryLinking) | |
| 126 | +{ | |
| 127 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setSupportBinaryLinking_), supportBinaryLinking); | |
| 128 | +} | |
| 129 | + | |
| 130 | +_MTL_INLINE void MTL4::ComputePipelineDescriptor::setSupportIndirectCommandBuffers(MTL4::IndirectCommandBufferSupportState supportIndirectCommandBuffers) | |
| 131 | +{ | |
| 132 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setSupportIndirectCommandBuffers_), supportIndirectCommandBuffers); | |
| 133 | +} | |
| 134 | + | |
| 135 | +_MTL_INLINE void MTL4::ComputePipelineDescriptor::setThreadGroupSizeIsMultipleOfThreadExecutionWidth(bool threadGroupSizeIsMultipleOfThreadExecutionWidth) | |
| 136 | +{ | |
| 137 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setThreadGroupSizeIsMultipleOfThreadExecutionWidth_), threadGroupSizeIsMultipleOfThreadExecutionWidth); | |
| 138 | +} | |
| 139 | + | |
| 140 | +_MTL_INLINE MTL4::StaticLinkingDescriptor* MTL4::ComputePipelineDescriptor::staticLinkingDescriptor() const | |
| 141 | +{ | |
| 142 | + return Object::sendMessage<MTL4::StaticLinkingDescriptor*>(this, _MTL_PRIVATE_SEL(staticLinkingDescriptor)); | |
| 143 | +} | |
| 144 | + | |
| 145 | +_MTL_INLINE bool MTL4::ComputePipelineDescriptor::supportBinaryLinking() const | |
| 146 | +{ | |
| 147 | + return Object::sendMessageSafe<bool>(this, _MTL_PRIVATE_SEL(supportBinaryLinking)); | |
| 148 | +} | |
| 149 | + | |
| 150 | +_MTL_INLINE MTL4::IndirectCommandBufferSupportState MTL4::ComputePipelineDescriptor::supportIndirectCommandBuffers() const | |
| 151 | +{ | |
| 152 | + return Object::sendMessage<MTL4::IndirectCommandBufferSupportState>(this, _MTL_PRIVATE_SEL(supportIndirectCommandBuffers)); | |
| 153 | +} | |
| 154 | + | |
| 155 | +_MTL_INLINE bool MTL4::ComputePipelineDescriptor::threadGroupSizeIsMultipleOfThreadExecutionWidth() const | |
| 156 | +{ | |
| 157 | + return Object::sendMessage<bool>(this, _MTL_PRIVATE_SEL(threadGroupSizeIsMultipleOfThreadExecutionWidth)); | |
| 158 | +} | |
added
third_party/metal-cpp/Metal/MTL4Counters.hpp
+138 −0
@@ -0,0 +1,138 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Metal/MTL4Counters.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2025 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include "../Foundation/Foundation.hpp" | |
| 24 | +#include "MTLDefines.hpp" | |
| 25 | +#include "MTLHeaderBridge.hpp" | |
| 26 | +#include "MTLPrivate.hpp" | |
| 27 | +#include <cstdint> | |
| 28 | + | |
| 29 | +#include <cstdint> | |
| 30 | + | |
| 31 | +namespace MTL4 | |
| 32 | +{ | |
| 33 | +class CounterHeapDescriptor; | |
| 34 | +_MTL_ENUM(NS::Integer, CounterHeapType) { | |
| 35 | + CounterHeapTypeInvalid, | |
| 36 | + CounterHeapTypeTimestamp, | |
| 37 | +}; | |
| 38 | + | |
| 39 | +_MTL_ENUM(NS::Integer, TimestampGranularity) { | |
| 40 | + TimestampGranularityRelaxed = 0, | |
| 41 | + TimestampGranularityPrecise = 1, | |
| 42 | +}; | |
| 43 | + | |
| 44 | +struct TimestampHeapEntry | |
| 45 | +{ | |
| 46 | + uint64_t timestamp; | |
| 47 | +} _MTL_PACKED; | |
| 48 | + | |
| 49 | +class CounterHeapDescriptor : public NS::Copying<CounterHeapDescriptor> | |
| 50 | +{ | |
| 51 | +public: | |
| 52 | + static CounterHeapDescriptor* alloc(); | |
| 53 | + | |
| 54 | + NS::UInteger count() const; | |
| 55 | + | |
| 56 | + CounterHeapDescriptor* init(); | |
| 57 | + | |
| 58 | + void setCount(NS::UInteger count); | |
| 59 | + | |
| 60 | + void setType(MTL4::CounterHeapType type); | |
| 61 | + CounterHeapType type() const; | |
| 62 | +}; | |
| 63 | +class CounterHeap : public NS::Referencing<CounterHeap> | |
| 64 | +{ | |
| 65 | +public: | |
| 66 | + NS::UInteger count() const; | |
| 67 | + void invalidateCounterRange(NS::Range range); | |
| 68 | + | |
| 69 | + NS::String* label() const; | |
| 70 | + | |
| 71 | + NS::Data* resolveCounterRange(NS::Range range); | |
| 72 | + | |
| 73 | + void setLabel(const NS::String* label); | |
| 74 | + | |
| 75 | + CounterHeapType type() const; | |
| 76 | +}; | |
| 77 | + | |
| 78 | +} | |
| 79 | + | |
| 80 | +_MTL_INLINE MTL4::CounterHeapDescriptor* MTL4::CounterHeapDescriptor::alloc() | |
| 81 | +{ | |
| 82 | + return NS::Object::alloc<MTL4::CounterHeapDescriptor>(_MTL_PRIVATE_CLS(MTL4CounterHeapDescriptor)); | |
| 83 | +} | |
| 84 | + | |
| 85 | +_MTL_INLINE NS::UInteger MTL4::CounterHeapDescriptor::count() const | |
| 86 | +{ | |
| 87 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(count)); | |
| 88 | +} | |
| 89 | + | |
| 90 | +_MTL_INLINE MTL4::CounterHeapDescriptor* MTL4::CounterHeapDescriptor::init() | |
| 91 | +{ | |
| 92 | + return NS::Object::init<MTL4::CounterHeapDescriptor>(); | |
| 93 | +} | |
| 94 | + | |
| 95 | +_MTL_INLINE void MTL4::CounterHeapDescriptor::setCount(NS::UInteger count) | |
| 96 | +{ | |
| 97 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setCount_), count); | |
| 98 | +} | |
| 99 | + | |
| 100 | +_MTL_INLINE void MTL4::CounterHeapDescriptor::setType(MTL4::CounterHeapType type) | |
| 101 | +{ | |
| 102 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setType_), type); | |
| 103 | +} | |
| 104 | + | |
| 105 | +_MTL_INLINE MTL4::CounterHeapType MTL4::CounterHeapDescriptor::type() const | |
| 106 | +{ | |
| 107 | + return Object::sendMessage<MTL4::CounterHeapType>(this, _MTL_PRIVATE_SEL(type)); | |
| 108 | +} | |
| 109 | + | |
| 110 | +_MTL_INLINE NS::UInteger MTL4::CounterHeap::count() const | |
| 111 | +{ | |
| 112 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(count)); | |
| 113 | +} | |
| 114 | + | |
| 115 | +_MTL_INLINE void MTL4::CounterHeap::invalidateCounterRange(NS::Range range) | |
| 116 | +{ | |
| 117 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(invalidateCounterRange_), range); | |
| 118 | +} | |
| 119 | + | |
| 120 | +_MTL_INLINE NS::String* MTL4::CounterHeap::label() const | |
| 121 | +{ | |
| 122 | + return Object::sendMessage<NS::String*>(this, _MTL_PRIVATE_SEL(label)); | |
| 123 | +} | |
| 124 | + | |
| 125 | +_MTL_INLINE NS::Data* MTL4::CounterHeap::resolveCounterRange(NS::Range range) | |
| 126 | +{ | |
| 127 | + return Object::sendMessage<NS::Data*>(this, _MTL_PRIVATE_SEL(resolveCounterRange_), range); | |
| 128 | +} | |
| 129 | + | |
| 130 | +_MTL_INLINE void MTL4::CounterHeap::setLabel(const NS::String* label) | |
| 131 | +{ | |
| 132 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setLabel_), label); | |
| 133 | +} | |
| 134 | + | |
| 135 | +_MTL_INLINE MTL4::CounterHeapType MTL4::CounterHeap::type() const | |
| 136 | +{ | |
| 137 | + return Object::sendMessage<MTL4::CounterHeapType>(this, _MTL_PRIVATE_SEL(type)); | |
| 138 | +} | |
added
third_party/metal-cpp/Metal/MTL4FunctionDescriptor.hpp
+49 −0
@@ -0,0 +1,49 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Metal//MTL4FunctionDescriptor.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2025 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include "../Foundation/Foundation.hpp" | |
| 24 | +#include "MTLDefines.hpp" | |
| 25 | +#include "MTLHeaderBridge.hpp" | |
| 26 | +#include "MTLPrivate.hpp" | |
| 27 | + | |
| 28 | +namespace MTL4 | |
| 29 | +{ | |
| 30 | +class FunctionDescriptor; | |
| 31 | + | |
| 32 | +class FunctionDescriptor : public NS::Copying<FunctionDescriptor> | |
| 33 | +{ | |
| 34 | +public: | |
| 35 | + static FunctionDescriptor* alloc(); | |
| 36 | + | |
| 37 | + FunctionDescriptor* init(); | |
| 38 | +}; | |
| 39 | + | |
| 40 | +} | |
| 41 | +_MTL_INLINE MTL4::FunctionDescriptor* MTL4::FunctionDescriptor::alloc() | |
| 42 | +{ | |
| 43 | + return NS::Object::alloc<MTL4::FunctionDescriptor>(_MTL_PRIVATE_CLS(MTL4FunctionDescriptor)); | |
| 44 | +} | |
| 45 | + | |
| 46 | +_MTL_INLINE MTL4::FunctionDescriptor* MTL4::FunctionDescriptor::init() | |
| 47 | +{ | |
| 48 | + return NS::Object::init<MTL4::FunctionDescriptor>(); | |
| 49 | +} | |
added
third_party/metal-cpp/Metal/MTL4LibraryDescriptor.hpp
+98 −0
@@ -0,0 +1,98 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Metal/MTL4LibraryDescriptor.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2025 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include "../Foundation/Foundation.hpp" | |
| 24 | +#include "MTLDefines.hpp" | |
| 25 | +#include "MTLHeaderBridge.hpp" | |
| 26 | +#include "MTLPrivate.hpp" | |
| 27 | + | |
| 28 | +namespace MTL4 | |
| 29 | +{ | |
| 30 | +class LibraryDescriptor; | |
| 31 | +} | |
| 32 | + | |
| 33 | +namespace MTL | |
| 34 | +{ | |
| 35 | +class CompileOptions; | |
| 36 | +} | |
| 37 | + | |
| 38 | +namespace MTL4 | |
| 39 | +{ | |
| 40 | +class LibraryDescriptor : public NS::Copying<LibraryDescriptor> | |
| 41 | +{ | |
| 42 | +public: | |
| 43 | + static LibraryDescriptor* alloc(); | |
| 44 | + | |
| 45 | + LibraryDescriptor* init(); | |
| 46 | + | |
| 47 | + NS::String* name() const; | |
| 48 | + | |
| 49 | + MTL::CompileOptions* options() const; | |
| 50 | + | |
| 51 | + void setName(const NS::String* name); | |
| 52 | + | |
| 53 | + void setOptions(const MTL::CompileOptions* options); | |
| 54 | + | |
| 55 | + void setSource(const NS::String* source); | |
| 56 | + NS::String* source() const; | |
| 57 | +}; | |
| 58 | + | |
| 59 | +} | |
| 60 | +_MTL_INLINE MTL4::LibraryDescriptor* MTL4::LibraryDescriptor::alloc() | |
| 61 | +{ | |
| 62 | + return NS::Object::alloc<MTL4::LibraryDescriptor>(_MTL_PRIVATE_CLS(MTL4LibraryDescriptor)); | |
| 63 | +} | |
| 64 | + | |
| 65 | +_MTL_INLINE MTL4::LibraryDescriptor* MTL4::LibraryDescriptor::init() | |
| 66 | +{ | |
| 67 | + return NS::Object::init<MTL4::LibraryDescriptor>(); | |
| 68 | +} | |
| 69 | + | |
| 70 | +_MTL_INLINE NS::String* MTL4::LibraryDescriptor::name() const | |
| 71 | +{ | |
| 72 | + return Object::sendMessage<NS::String*>(this, _MTL_PRIVATE_SEL(name)); | |
| 73 | +} | |
| 74 | + | |
| 75 | +_MTL_INLINE MTL::CompileOptions* MTL4::LibraryDescriptor::options() const | |
| 76 | +{ | |
| 77 | + return Object::sendMessage<MTL::CompileOptions*>(this, _MTL_PRIVATE_SEL(options)); | |
| 78 | +} | |
| 79 | + | |
| 80 | +_MTL_INLINE void MTL4::LibraryDescriptor::setName(const NS::String* name) | |
| 81 | +{ | |
| 82 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setName_), name); | |
| 83 | +} | |
| 84 | + | |
| 85 | +_MTL_INLINE void MTL4::LibraryDescriptor::setOptions(const MTL::CompileOptions* options) | |
| 86 | +{ | |
| 87 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setOptions_), options); | |
| 88 | +} | |
| 89 | + | |
| 90 | +_MTL_INLINE void MTL4::LibraryDescriptor::setSource(const NS::String* source) | |
| 91 | +{ | |
| 92 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setSource_), source); | |
| 93 | +} | |
| 94 | + | |
| 95 | +_MTL_INLINE NS::String* MTL4::LibraryDescriptor::source() const | |
| 96 | +{ | |
| 97 | + return Object::sendMessage<NS::String*>(this, _MTL_PRIVATE_SEL(source)); | |
| 98 | +} | |
added
third_party/metal-cpp/Metal/MTL4LibraryFunctionDescriptor.hpp
+86 −0
@@ -0,0 +1,86 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Metal/MTL4LibraryFunctionDescriptor.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2025 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include "../Foundation/Foundation.hpp" | |
| 24 | +#include "MTL4FunctionDescriptor.hpp" | |
| 25 | +#include "MTLDefines.hpp" | |
| 26 | +#include "MTLHeaderBridge.hpp" | |
| 27 | +#include "MTLPrivate.hpp" | |
| 28 | + | |
| 29 | +namespace MTL4 | |
| 30 | +{ | |
| 31 | +class LibraryFunctionDescriptor; | |
| 32 | +} | |
| 33 | + | |
| 34 | +namespace MTL | |
| 35 | +{ | |
| 36 | +class Library; | |
| 37 | +} | |
| 38 | + | |
| 39 | +namespace MTL4 | |
| 40 | +{ | |
| 41 | +class LibraryFunctionDescriptor : public NS::Copying<LibraryFunctionDescriptor, FunctionDescriptor> | |
| 42 | +{ | |
| 43 | +public: | |
| 44 | + static LibraryFunctionDescriptor* alloc(); | |
| 45 | + | |
| 46 | + LibraryFunctionDescriptor* init(); | |
| 47 | + | |
| 48 | + MTL::Library* library() const; | |
| 49 | + | |
| 50 | + NS::String* name() const; | |
| 51 | + | |
| 52 | + void setLibrary(const MTL::Library* library); | |
| 53 | + | |
| 54 | + void setName(const NS::String* name); | |
| 55 | +}; | |
| 56 | + | |
| 57 | +} | |
| 58 | +_MTL_INLINE MTL4::LibraryFunctionDescriptor* MTL4::LibraryFunctionDescriptor::alloc() | |
| 59 | +{ | |
| 60 | + return NS::Object::alloc<MTL4::LibraryFunctionDescriptor>(_MTL_PRIVATE_CLS(MTL4LibraryFunctionDescriptor)); | |
| 61 | +} | |
| 62 | + | |
| 63 | +_MTL_INLINE MTL4::LibraryFunctionDescriptor* MTL4::LibraryFunctionDescriptor::init() | |
| 64 | +{ | |
| 65 | + return NS::Object::init<MTL4::LibraryFunctionDescriptor>(); | |
| 66 | +} | |
| 67 | + | |
| 68 | +_MTL_INLINE MTL::Library* MTL4::LibraryFunctionDescriptor::library() const | |
| 69 | +{ | |
| 70 | + return Object::sendMessage<MTL::Library*>(this, _MTL_PRIVATE_SEL(library)); | |
| 71 | +} | |
| 72 | + | |
| 73 | +_MTL_INLINE NS::String* MTL4::LibraryFunctionDescriptor::name() const | |
| 74 | +{ | |
| 75 | + return Object::sendMessage<NS::String*>(this, _MTL_PRIVATE_SEL(name)); | |
| 76 | +} | |
| 77 | + | |
| 78 | +_MTL_INLINE void MTL4::LibraryFunctionDescriptor::setLibrary(const MTL::Library* library) | |
| 79 | +{ | |
| 80 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setLibrary_), library); | |
| 81 | +} | |
| 82 | + | |
| 83 | +_MTL_INLINE void MTL4::LibraryFunctionDescriptor::setName(const NS::String* name) | |
| 84 | +{ | |
| 85 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setName_), name); | |
| 86 | +} | |
added
third_party/metal-cpp/Metal/MTL4LinkingDescriptor.hpp
+204 −0
@@ -0,0 +1,204 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Metal/MTL4LinkingDescriptor.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2025 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include "../Foundation/Foundation.hpp" | |
| 24 | +#include "MTLDefines.hpp" | |
| 25 | +#include "MTLHeaderBridge.hpp" | |
| 26 | +#include "MTLPrivate.hpp" | |
| 27 | + | |
| 28 | +namespace MTL4 | |
| 29 | +{ | |
| 30 | +class PipelineStageDynamicLinkingDescriptor; | |
| 31 | +class RenderPipelineDynamicLinkingDescriptor; | |
| 32 | +class StaticLinkingDescriptor; | |
| 33 | + | |
| 34 | +class StaticLinkingDescriptor : public NS::Copying<StaticLinkingDescriptor> | |
| 35 | +{ | |
| 36 | +public: | |
| 37 | + static StaticLinkingDescriptor* alloc(); | |
| 38 | + | |
| 39 | + NS::Array* functionDescriptors() const; | |
| 40 | + | |
| 41 | + NS::Dictionary* groups() const; | |
| 42 | + | |
| 43 | + StaticLinkingDescriptor* init(); | |
| 44 | + | |
| 45 | + NS::Array* privateFunctionDescriptors() const; | |
| 46 | + | |
| 47 | + void setFunctionDescriptors(const NS::Array* functionDescriptors); | |
| 48 | + | |
| 49 | + void setGroups(const NS::Dictionary* groups); | |
| 50 | + | |
| 51 | + void setPrivateFunctionDescriptors(const NS::Array* privateFunctionDescriptors); | |
| 52 | +}; | |
| 53 | +class PipelineStageDynamicLinkingDescriptor : public NS::Copying<PipelineStageDynamicLinkingDescriptor> | |
| 54 | +{ | |
| 55 | +public: | |
| 56 | + static PipelineStageDynamicLinkingDescriptor* alloc(); | |
| 57 | + | |
| 58 | + NS::Array* binaryLinkedFunctions() const; | |
| 59 | + | |
| 60 | + PipelineStageDynamicLinkingDescriptor* init(); | |
| 61 | + | |
| 62 | + NS::UInteger maxCallStackDepth() const; | |
| 63 | + | |
| 64 | + NS::Array* preloadedLibraries() const; | |
| 65 | + | |
| 66 | + void setBinaryLinkedFunctions(const NS::Array* binaryLinkedFunctions); | |
| 67 | + | |
| 68 | + void setMaxCallStackDepth(NS::UInteger maxCallStackDepth); | |
| 69 | + | |
| 70 | + void setPreloadedLibraries(const NS::Array* preloadedLibraries); | |
| 71 | +}; | |
| 72 | +class RenderPipelineDynamicLinkingDescriptor : public NS::Copying<RenderPipelineDynamicLinkingDescriptor> | |
| 73 | +{ | |
| 74 | +public: | |
| 75 | + static RenderPipelineDynamicLinkingDescriptor* alloc(); | |
| 76 | + | |
| 77 | + PipelineStageDynamicLinkingDescriptor* fragmentLinkingDescriptor() const; | |
| 78 | + | |
| 79 | + RenderPipelineDynamicLinkingDescriptor* init(); | |
| 80 | + | |
| 81 | + PipelineStageDynamicLinkingDescriptor* meshLinkingDescriptor() const; | |
| 82 | + | |
| 83 | + PipelineStageDynamicLinkingDescriptor* objectLinkingDescriptor() const; | |
| 84 | + | |
| 85 | + PipelineStageDynamicLinkingDescriptor* tileLinkingDescriptor() const; | |
| 86 | + | |
| 87 | + PipelineStageDynamicLinkingDescriptor* vertexLinkingDescriptor() const; | |
| 88 | +}; | |
| 89 | + | |
| 90 | +} | |
| 91 | +_MTL_INLINE MTL4::StaticLinkingDescriptor* MTL4::StaticLinkingDescriptor::alloc() | |
| 92 | +{ | |
| 93 | + return NS::Object::alloc<MTL4::StaticLinkingDescriptor>(_MTL_PRIVATE_CLS(MTL4StaticLinkingDescriptor)); | |
| 94 | +} | |
| 95 | + | |
| 96 | +_MTL_INLINE NS::Array* MTL4::StaticLinkingDescriptor::functionDescriptors() const | |
| 97 | +{ | |
| 98 | + return Object::sendMessage<NS::Array*>(this, _MTL_PRIVATE_SEL(functionDescriptors)); | |
| 99 | +} | |
| 100 | + | |
| 101 | +_MTL_INLINE NS::Dictionary* MTL4::StaticLinkingDescriptor::groups() const | |
| 102 | +{ | |
| 103 | + return Object::sendMessage<NS::Dictionary*>(this, _MTL_PRIVATE_SEL(groups)); | |
| 104 | +} | |
| 105 | + | |
| 106 | +_MTL_INLINE MTL4::StaticLinkingDescriptor* MTL4::StaticLinkingDescriptor::init() | |
| 107 | +{ | |
| 108 | + return NS::Object::init<MTL4::StaticLinkingDescriptor>(); | |
| 109 | +} | |
| 110 | + | |
| 111 | +_MTL_INLINE NS::Array* MTL4::StaticLinkingDescriptor::privateFunctionDescriptors() const | |
| 112 | +{ | |
| 113 | + return Object::sendMessage<NS::Array*>(this, _MTL_PRIVATE_SEL(privateFunctionDescriptors)); | |
| 114 | +} | |
| 115 | + | |
| 116 | +_MTL_INLINE void MTL4::StaticLinkingDescriptor::setFunctionDescriptors(const NS::Array* functionDescriptors) | |
| 117 | +{ | |
| 118 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setFunctionDescriptors_), functionDescriptors); | |
| 119 | +} | |
| 120 | + | |
| 121 | +_MTL_INLINE void MTL4::StaticLinkingDescriptor::setGroups(const NS::Dictionary* groups) | |
| 122 | +{ | |
| 123 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setGroups_), groups); | |
| 124 | +} | |
| 125 | + | |
| 126 | +_MTL_INLINE void MTL4::StaticLinkingDescriptor::setPrivateFunctionDescriptors(const NS::Array* privateFunctionDescriptors) | |
| 127 | +{ | |
| 128 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setPrivateFunctionDescriptors_), privateFunctionDescriptors); | |
| 129 | +} | |
| 130 | + | |
| 131 | +_MTL_INLINE MTL4::PipelineStageDynamicLinkingDescriptor* MTL4::PipelineStageDynamicLinkingDescriptor::alloc() | |
| 132 | +{ | |
| 133 | + return NS::Object::alloc<MTL4::PipelineStageDynamicLinkingDescriptor>(_MTL_PRIVATE_CLS(MTL4PipelineStageDynamicLinkingDescriptor)); | |
| 134 | +} | |
| 135 | + | |
| 136 | +_MTL_INLINE NS::Array* MTL4::PipelineStageDynamicLinkingDescriptor::binaryLinkedFunctions() const | |
| 137 | +{ | |
| 138 | + return Object::sendMessage<NS::Array*>(this, _MTL_PRIVATE_SEL(binaryLinkedFunctions)); | |
| 139 | +} | |
| 140 | + | |
| 141 | +_MTL_INLINE MTL4::PipelineStageDynamicLinkingDescriptor* MTL4::PipelineStageDynamicLinkingDescriptor::init() | |
| 142 | +{ | |
| 143 | + return NS::Object::init<MTL4::PipelineStageDynamicLinkingDescriptor>(); | |
| 144 | +} | |
| 145 | + | |
| 146 | +_MTL_INLINE NS::UInteger MTL4::PipelineStageDynamicLinkingDescriptor::maxCallStackDepth() const | |
| 147 | +{ | |
| 148 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(maxCallStackDepth)); | |
| 149 | +} | |
| 150 | + | |
| 151 | +_MTL_INLINE NS::Array* MTL4::PipelineStageDynamicLinkingDescriptor::preloadedLibraries() const | |
| 152 | +{ | |
| 153 | + return Object::sendMessage<NS::Array*>(this, _MTL_PRIVATE_SEL(preloadedLibraries)); | |
| 154 | +} | |
| 155 | + | |
| 156 | +_MTL_INLINE void MTL4::PipelineStageDynamicLinkingDescriptor::setBinaryLinkedFunctions(const NS::Array* binaryLinkedFunctions) | |
| 157 | +{ | |
| 158 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setBinaryLinkedFunctions_), binaryLinkedFunctions); | |
| 159 | +} | |
| 160 | + | |
| 161 | +_MTL_INLINE void MTL4::PipelineStageDynamicLinkingDescriptor::setMaxCallStackDepth(NS::UInteger maxCallStackDepth) | |
| 162 | +{ | |
| 163 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMaxCallStackDepth_), maxCallStackDepth); | |
| 164 | +} | |
| 165 | + | |
| 166 | +_MTL_INLINE void MTL4::PipelineStageDynamicLinkingDescriptor::setPreloadedLibraries(const NS::Array* preloadedLibraries) | |
| 167 | +{ | |
| 168 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setPreloadedLibraries_), preloadedLibraries); | |
| 169 | +} | |
| 170 | + | |
| 171 | +_MTL_INLINE MTL4::RenderPipelineDynamicLinkingDescriptor* MTL4::RenderPipelineDynamicLinkingDescriptor::alloc() | |
| 172 | +{ | |
| 173 | + return NS::Object::alloc<MTL4::RenderPipelineDynamicLinkingDescriptor>(_MTL_PRIVATE_CLS(MTL4RenderPipelineDynamicLinkingDescriptor)); | |
| 174 | +} | |
| 175 | + | |
| 176 | +_MTL_INLINE MTL4::PipelineStageDynamicLinkingDescriptor* MTL4::RenderPipelineDynamicLinkingDescriptor::fragmentLinkingDescriptor() const | |
| 177 | +{ | |
| 178 | + return Object::sendMessage<MTL4::PipelineStageDynamicLinkingDescriptor*>(this, _MTL_PRIVATE_SEL(fragmentLinkingDescriptor)); | |
| 179 | +} | |
| 180 | + | |
| 181 | +_MTL_INLINE MTL4::RenderPipelineDynamicLinkingDescriptor* MTL4::RenderPipelineDynamicLinkingDescriptor::init() | |
| 182 | +{ | |
| 183 | + return NS::Object::init<MTL4::RenderPipelineDynamicLinkingDescriptor>(); | |
| 184 | +} | |
| 185 | + | |
| 186 | +_MTL_INLINE MTL4::PipelineStageDynamicLinkingDescriptor* MTL4::RenderPipelineDynamicLinkingDescriptor::meshLinkingDescriptor() const | |
| 187 | +{ | |
| 188 | + return Object::sendMessage<MTL4::PipelineStageDynamicLinkingDescriptor*>(this, _MTL_PRIVATE_SEL(meshLinkingDescriptor)); | |
| 189 | +} | |
| 190 | + | |
| 191 | +_MTL_INLINE MTL4::PipelineStageDynamicLinkingDescriptor* MTL4::RenderPipelineDynamicLinkingDescriptor::objectLinkingDescriptor() const | |
| 192 | +{ | |
| 193 | + return Object::sendMessage<MTL4::PipelineStageDynamicLinkingDescriptor*>(this, _MTL_PRIVATE_SEL(objectLinkingDescriptor)); | |
| 194 | +} | |
| 195 | + | |
| 196 | +_MTL_INLINE MTL4::PipelineStageDynamicLinkingDescriptor* MTL4::RenderPipelineDynamicLinkingDescriptor::tileLinkingDescriptor() const | |
| 197 | +{ | |
| 198 | + return Object::sendMessage<MTL4::PipelineStageDynamicLinkingDescriptor*>(this, _MTL_PRIVATE_SEL(tileLinkingDescriptor)); | |
| 199 | +} | |
| 200 | + | |
| 201 | +_MTL_INLINE MTL4::PipelineStageDynamicLinkingDescriptor* MTL4::RenderPipelineDynamicLinkingDescriptor::vertexLinkingDescriptor() const | |
| 202 | +{ | |
| 203 | + return Object::sendMessage<MTL4::PipelineStageDynamicLinkingDescriptor*>(this, _MTL_PRIVATE_SEL(vertexLinkingDescriptor)); | |
| 204 | +} | |
added
third_party/metal-cpp/Metal/MTL4MachineLearningCommandEncoder.hpp
+66 −0
@@ -0,0 +1,66 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Metal/MTL4MachineLearningCommandEncoder.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2025 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include "../Foundation/Foundation.hpp" | |
| 24 | +#include "MTL4CommandEncoder.hpp" | |
| 25 | +#include "MTLDefines.hpp" | |
| 26 | +#include "MTLHeaderBridge.hpp" | |
| 27 | +#include "MTLPrivate.hpp" | |
| 28 | + | |
| 29 | +namespace MTL4 | |
| 30 | +{ | |
| 31 | +class ArgumentTable; | |
| 32 | +class MachineLearningPipelineState; | |
| 33 | +} | |
| 34 | + | |
| 35 | +namespace MTL | |
| 36 | +{ | |
| 37 | +class Heap; | |
| 38 | +} | |
| 39 | + | |
| 40 | +namespace MTL4 | |
| 41 | +{ | |
| 42 | +class MachineLearningCommandEncoder : public NS::Referencing<MachineLearningCommandEncoder, CommandEncoder> | |
| 43 | +{ | |
| 44 | +public: | |
| 45 | + void dispatchNetwork(const MTL::Heap* heap); | |
| 46 | + | |
| 47 | + void setArgumentTable(const MTL4::ArgumentTable* argumentTable); | |
| 48 | + | |
| 49 | + void setPipelineState(const MTL4::MachineLearningPipelineState* pipelineState); | |
| 50 | +}; | |
| 51 | + | |
| 52 | +} | |
| 53 | +_MTL_INLINE void MTL4::MachineLearningCommandEncoder::dispatchNetwork(const MTL::Heap* heap) | |
| 54 | +{ | |
| 55 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(dispatchNetworkWithIntermediatesHeap_), heap); | |
| 56 | +} | |
| 57 | + | |
| 58 | +_MTL_INLINE void MTL4::MachineLearningCommandEncoder::setArgumentTable(const MTL4::ArgumentTable* argumentTable) | |
| 59 | +{ | |
| 60 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setArgumentTable_), argumentTable); | |
| 61 | +} | |
| 62 | + | |
| 63 | +_MTL_INLINE void MTL4::MachineLearningCommandEncoder::setPipelineState(const MTL4::MachineLearningPipelineState* pipelineState) | |
| 64 | +{ | |
| 65 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setPipelineState_), pipelineState); | |
| 66 | +} | |
added
third_party/metal-cpp/Metal/MTL4MachineLearningPipeline.hpp
+172 −0
@@ -0,0 +1,172 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Metal/MTL4MachineLearningPipeline.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2025 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include "../Foundation/Foundation.hpp" | |
| 24 | +#include "MTL4PipelineState.hpp" | |
| 25 | +#include "MTLAllocation.hpp" | |
| 26 | +#include "MTLDefines.hpp" | |
| 27 | +#include "MTLHeaderBridge.hpp" | |
| 28 | +#include "MTLPrivate.hpp" | |
| 29 | + | |
| 30 | +namespace MTL4 | |
| 31 | +{ | |
| 32 | +class FunctionDescriptor; | |
| 33 | +class MachineLearningPipelineDescriptor; | |
| 34 | +class MachineLearningPipelineReflection; | |
| 35 | +} | |
| 36 | + | |
| 37 | +namespace MTL | |
| 38 | +{ | |
| 39 | +class Device; | |
| 40 | +class TensorExtents; | |
| 41 | +} | |
| 42 | + | |
| 43 | +namespace MTL4 | |
| 44 | +{ | |
| 45 | +class MachineLearningPipelineDescriptor : public NS::Copying<MachineLearningPipelineDescriptor, PipelineDescriptor> | |
| 46 | +{ | |
| 47 | +public: | |
| 48 | + static MachineLearningPipelineDescriptor* alloc(); | |
| 49 | + | |
| 50 | + MachineLearningPipelineDescriptor* init(); | |
| 51 | + | |
| 52 | + MTL::TensorExtents* inputDimensionsAtBufferIndex(NS::Integer bufferIndex); | |
| 53 | + | |
| 54 | + NS::String* label() const; | |
| 55 | + | |
| 56 | + FunctionDescriptor* machineLearningFunctionDescriptor() const; | |
| 57 | + | |
| 58 | + void reset(); | |
| 59 | + | |
| 60 | + void setInputDimensions(const MTL::TensorExtents* dimensions, NS::Integer bufferIndex); | |
| 61 | + void setInputDimensions(const NS::Array* dimensions, NS::Range range); | |
| 62 | + | |
| 63 | + void setLabel(const NS::String* label); | |
| 64 | + | |
| 65 | + void setMachineLearningFunctionDescriptor(const MTL4::FunctionDescriptor* machineLearningFunctionDescriptor); | |
| 66 | +}; | |
| 67 | +class MachineLearningPipelineReflection : public NS::Referencing<MachineLearningPipelineReflection> | |
| 68 | +{ | |
| 69 | +public: | |
| 70 | + static MachineLearningPipelineReflection* alloc(); | |
| 71 | + | |
| 72 | + NS::Array* bindings() const; | |
| 73 | + | |
| 74 | + MachineLearningPipelineReflection* init(); | |
| 75 | +}; | |
| 76 | +class MachineLearningPipelineState : public NS::Referencing<MachineLearningPipelineState, MTL::Allocation> | |
| 77 | +{ | |
| 78 | +public: | |
| 79 | + MTL::Device* device() const; | |
| 80 | + | |
| 81 | + NS::UInteger intermediatesHeapSize() const; | |
| 82 | + | |
| 83 | + NS::String* label() const; | |
| 84 | + | |
| 85 | + MachineLearningPipelineReflection* reflection() const; | |
| 86 | +}; | |
| 87 | + | |
| 88 | +} | |
| 89 | +_MTL_INLINE MTL4::MachineLearningPipelineDescriptor* MTL4::MachineLearningPipelineDescriptor::alloc() | |
| 90 | +{ | |
| 91 | + return NS::Object::alloc<MTL4::MachineLearningPipelineDescriptor>(_MTL_PRIVATE_CLS(MTL4MachineLearningPipelineDescriptor)); | |
| 92 | +} | |
| 93 | + | |
| 94 | +_MTL_INLINE MTL4::MachineLearningPipelineDescriptor* MTL4::MachineLearningPipelineDescriptor::init() | |
| 95 | +{ | |
| 96 | + return NS::Object::init<MTL4::MachineLearningPipelineDescriptor>(); | |
| 97 | +} | |
| 98 | + | |
| 99 | +_MTL_INLINE MTL::TensorExtents* MTL4::MachineLearningPipelineDescriptor::inputDimensionsAtBufferIndex(NS::Integer bufferIndex) | |
| 100 | +{ | |
| 101 | + return Object::sendMessage<MTL::TensorExtents*>(this, _MTL_PRIVATE_SEL(inputDimensionsAtBufferIndex_), bufferIndex); | |
| 102 | +} | |
| 103 | + | |
| 104 | +_MTL_INLINE NS::String* MTL4::MachineLearningPipelineDescriptor::label() const | |
| 105 | +{ | |
| 106 | + return Object::sendMessage<NS::String*>(this, _MTL_PRIVATE_SEL(label)); | |
| 107 | +} | |
| 108 | + | |
| 109 | +_MTL_INLINE MTL4::FunctionDescriptor* MTL4::MachineLearningPipelineDescriptor::machineLearningFunctionDescriptor() const | |
| 110 | +{ | |
| 111 | + return Object::sendMessage<MTL4::FunctionDescriptor*>(this, _MTL_PRIVATE_SEL(machineLearningFunctionDescriptor)); | |
| 112 | +} | |
| 113 | + | |
| 114 | +_MTL_INLINE void MTL4::MachineLearningPipelineDescriptor::reset() | |
| 115 | +{ | |
| 116 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(reset)); | |
| 117 | +} | |
| 118 | + | |
| 119 | +_MTL_INLINE void MTL4::MachineLearningPipelineDescriptor::setInputDimensions(const MTL::TensorExtents* dimensions, NS::Integer bufferIndex) | |
| 120 | +{ | |
| 121 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setInputDimensions_atBufferIndex_), dimensions, bufferIndex); | |
| 122 | +} | |
| 123 | + | |
| 124 | +_MTL_INLINE void MTL4::MachineLearningPipelineDescriptor::setInputDimensions(const NS::Array* dimensions, NS::Range range) | |
| 125 | +{ | |
| 126 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setInputDimensions_withRange_), dimensions, range); | |
| 127 | +} | |
| 128 | + | |
| 129 | +_MTL_INLINE void MTL4::MachineLearningPipelineDescriptor::setLabel(const NS::String* label) | |
| 130 | +{ | |
| 131 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setLabel_), label); | |
| 132 | +} | |
| 133 | + | |
| 134 | +_MTL_INLINE void MTL4::MachineLearningPipelineDescriptor::setMachineLearningFunctionDescriptor(const MTL4::FunctionDescriptor* machineLearningFunctionDescriptor) | |
| 135 | +{ | |
| 136 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMachineLearningFunctionDescriptor_), machineLearningFunctionDescriptor); | |
| 137 | +} | |
| 138 | + | |
| 139 | +_MTL_INLINE MTL4::MachineLearningPipelineReflection* MTL4::MachineLearningPipelineReflection::alloc() | |
| 140 | +{ | |
| 141 | + return NS::Object::alloc<MTL4::MachineLearningPipelineReflection>(_MTL_PRIVATE_CLS(MTL4MachineLearningPipelineReflection)); | |
| 142 | +} | |
| 143 | + | |
| 144 | +_MTL_INLINE NS::Array* MTL4::MachineLearningPipelineReflection::bindings() const | |
| 145 | +{ | |
| 146 | + return Object::sendMessage<NS::Array*>(this, _MTL_PRIVATE_SEL(bindings)); | |
| 147 | +} | |
| 148 | + | |
| 149 | +_MTL_INLINE MTL4::MachineLearningPipelineReflection* MTL4::MachineLearningPipelineReflection::init() | |
| 150 | +{ | |
| 151 | + return NS::Object::init<MTL4::MachineLearningPipelineReflection>(); | |
| 152 | +} | |
| 153 | + | |
| 154 | +_MTL_INLINE MTL::Device* MTL4::MachineLearningPipelineState::device() const | |
| 155 | +{ | |
| 156 | + return Object::sendMessage<MTL::Device*>(this, _MTL_PRIVATE_SEL(device)); | |
| 157 | +} | |
| 158 | + | |
| 159 | +_MTL_INLINE NS::UInteger MTL4::MachineLearningPipelineState::intermediatesHeapSize() const | |
| 160 | +{ | |
| 161 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(intermediatesHeapSize)); | |
| 162 | +} | |
| 163 | + | |
| 164 | +_MTL_INLINE NS::String* MTL4::MachineLearningPipelineState::label() const | |
| 165 | +{ | |
| 166 | + return Object::sendMessage<NS::String*>(this, _MTL_PRIVATE_SEL(label)); | |
| 167 | +} | |
| 168 | + | |
| 169 | +_MTL_INLINE MTL4::MachineLearningPipelineReflection* MTL4::MachineLearningPipelineState::reflection() const | |
| 170 | +{ | |
| 171 | + return Object::sendMessage<MTL4::MachineLearningPipelineReflection*>(this, _MTL_PRIVATE_SEL(reflection)); | |
| 172 | +} | |
added
third_party/metal-cpp/Metal/MTL4MeshRenderPipeline.hpp
+413 −0
@@ -0,0 +1,413 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Metal/MTL4MeshRenderPipeline.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2025 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include "../Foundation/Foundation.hpp" | |
| 24 | +#include "MTL4PipelineState.hpp" | |
| 25 | +#include "MTL4RenderPipeline.hpp" | |
| 26 | +#include "MTLDefines.hpp" | |
| 27 | +#include "MTLHeaderBridge.hpp" | |
| 28 | +#include "MTLPrivate.hpp" | |
| 29 | +#include "MTLTypes.hpp" | |
| 30 | + | |
| 31 | +namespace MTL4 | |
| 32 | +{ | |
| 33 | +class FunctionDescriptor; | |
| 34 | +class MeshRenderPipelineDescriptor; | |
| 35 | +class RenderPipelineColorAttachmentDescriptorArray; | |
| 36 | +class StaticLinkingDescriptor; | |
| 37 | + | |
| 38 | +class MeshRenderPipelineDescriptor : public NS::Copying<MeshRenderPipelineDescriptor, PipelineDescriptor> | |
| 39 | +{ | |
| 40 | +public: | |
| 41 | + static MeshRenderPipelineDescriptor* alloc(); | |
| 42 | + | |
| 43 | + AlphaToCoverageState alphaToCoverageState() const; | |
| 44 | + | |
| 45 | + AlphaToOneState alphaToOneState() const; | |
| 46 | + | |
| 47 | + LogicalToPhysicalColorAttachmentMappingState colorAttachmentMappingState() const; | |
| 48 | + | |
| 49 | + RenderPipelineColorAttachmentDescriptorArray* colorAttachments() const; | |
| 50 | + | |
| 51 | + FunctionDescriptor* fragmentFunctionDescriptor() const; | |
| 52 | + | |
| 53 | + StaticLinkingDescriptor* fragmentStaticLinkingDescriptor() const; | |
| 54 | + | |
| 55 | + MeshRenderPipelineDescriptor* init(); | |
| 56 | + | |
| 57 | + bool isRasterizationEnabled() const; | |
| 58 | + | |
| 59 | + NS::UInteger maxTotalThreadgroupsPerMeshGrid() const; | |
| 60 | + | |
| 61 | + NS::UInteger maxTotalThreadsPerMeshThreadgroup() const; | |
| 62 | + | |
| 63 | + NS::UInteger maxTotalThreadsPerObjectThreadgroup() const; | |
| 64 | + | |
| 65 | + NS::UInteger maxVertexAmplificationCount() const; | |
| 66 | + | |
| 67 | + FunctionDescriptor* meshFunctionDescriptor() const; | |
| 68 | + | |
| 69 | + StaticLinkingDescriptor* meshStaticLinkingDescriptor() const; | |
| 70 | + | |
| 71 | + bool meshThreadgroupSizeIsMultipleOfThreadExecutionWidth() const; | |
| 72 | + | |
| 73 | + FunctionDescriptor* objectFunctionDescriptor() const; | |
| 74 | + | |
| 75 | + StaticLinkingDescriptor* objectStaticLinkingDescriptor() const; | |
| 76 | + | |
| 77 | + bool objectThreadgroupSizeIsMultipleOfThreadExecutionWidth() const; | |
| 78 | + | |
| 79 | + NS::UInteger payloadMemoryLength() const; | |
| 80 | + | |
| 81 | + NS::UInteger rasterSampleCount() const; | |
| 82 | + | |
| 83 | + [[deprecated("please use isRasterizationEnabled instead")]] | |
| 84 | + bool rasterizationEnabled() const; | |
| 85 | + | |
| 86 | + MTL::Size requiredThreadsPerMeshThreadgroup() const; | |
| 87 | + | |
| 88 | + MTL::Size requiredThreadsPerObjectThreadgroup() const; | |
| 89 | + | |
| 90 | + void reset(); | |
| 91 | + | |
| 92 | + void setAlphaToCoverageState(MTL4::AlphaToCoverageState alphaToCoverageState); | |
| 93 | + | |
| 94 | + void setAlphaToOneState(MTL4::AlphaToOneState alphaToOneState); | |
| 95 | + | |
| 96 | + void setColorAttachmentMappingState(MTL4::LogicalToPhysicalColorAttachmentMappingState colorAttachmentMappingState); | |
| 97 | + | |
| 98 | + void setFragmentFunctionDescriptor(const MTL4::FunctionDescriptor* fragmentFunctionDescriptor); | |
| 99 | + | |
| 100 | + void setFragmentStaticLinkingDescriptor(const MTL4::StaticLinkingDescriptor* fragmentStaticLinkingDescriptor); | |
| 101 | + | |
| 102 | + void setMaxTotalThreadgroupsPerMeshGrid(NS::UInteger maxTotalThreadgroupsPerMeshGrid); | |
| 103 | + | |
| 104 | + void setMaxTotalThreadsPerMeshThreadgroup(NS::UInteger maxTotalThreadsPerMeshThreadgroup); | |
| 105 | + | |
| 106 | + void setMaxTotalThreadsPerObjectThreadgroup(NS::UInteger maxTotalThreadsPerObjectThreadgroup); | |
| 107 | + | |
| 108 | + void setMaxVertexAmplificationCount(NS::UInteger maxVertexAmplificationCount); | |
| 109 | + | |
| 110 | + void setMeshFunctionDescriptor(const MTL4::FunctionDescriptor* meshFunctionDescriptor); | |
| 111 | + | |
| 112 | + void setMeshStaticLinkingDescriptor(const MTL4::StaticLinkingDescriptor* meshStaticLinkingDescriptor); | |
| 113 | + | |
| 114 | + void setMeshThreadgroupSizeIsMultipleOfThreadExecutionWidth(bool meshThreadgroupSizeIsMultipleOfThreadExecutionWidth); | |
| 115 | + | |
| 116 | + void setObjectFunctionDescriptor(const MTL4::FunctionDescriptor* objectFunctionDescriptor); | |
| 117 | + | |
| 118 | + void setObjectStaticLinkingDescriptor(const MTL4::StaticLinkingDescriptor* objectStaticLinkingDescriptor); | |
| 119 | + | |
| 120 | + void setObjectThreadgroupSizeIsMultipleOfThreadExecutionWidth(bool objectThreadgroupSizeIsMultipleOfThreadExecutionWidth); | |
| 121 | + | |
| 122 | + void setPayloadMemoryLength(NS::UInteger payloadMemoryLength); | |
| 123 | + | |
| 124 | + void setRasterSampleCount(NS::UInteger rasterSampleCount); | |
| 125 | + | |
| 126 | + void setRasterizationEnabled(bool rasterizationEnabled); | |
| 127 | + | |
| 128 | + void setRequiredThreadsPerMeshThreadgroup(MTL::Size requiredThreadsPerMeshThreadgroup); | |
| 129 | + | |
| 130 | + void setRequiredThreadsPerObjectThreadgroup(MTL::Size requiredThreadsPerObjectThreadgroup); | |
| 131 | + | |
| 132 | + void setSupportFragmentBinaryLinking(bool supportFragmentBinaryLinking); | |
| 133 | + | |
| 134 | + void setSupportIndirectCommandBuffers(MTL4::IndirectCommandBufferSupportState supportIndirectCommandBuffers); | |
| 135 | + | |
| 136 | + void setSupportMeshBinaryLinking(bool supportMeshBinaryLinking); | |
| 137 | + | |
| 138 | + void setSupportObjectBinaryLinking(bool supportObjectBinaryLinking); | |
| 139 | + | |
| 140 | + bool supportFragmentBinaryLinking() const; | |
| 141 | + | |
| 142 | + IndirectCommandBufferSupportState supportIndirectCommandBuffers() const; | |
| 143 | + | |
| 144 | + bool supportMeshBinaryLinking() const; | |
| 145 | + | |
| 146 | + bool supportObjectBinaryLinking() const; | |
| 147 | +}; | |
| 148 | + | |
| 149 | +} | |
| 150 | +_MTL_INLINE MTL4::MeshRenderPipelineDescriptor* MTL4::MeshRenderPipelineDescriptor::alloc() | |
| 151 | +{ | |
| 152 | + return NS::Object::alloc<MTL4::MeshRenderPipelineDescriptor>(_MTL_PRIVATE_CLS(MTL4MeshRenderPipelineDescriptor)); | |
| 153 | +} | |
| 154 | + | |
| 155 | +_MTL_INLINE MTL4::AlphaToCoverageState MTL4::MeshRenderPipelineDescriptor::alphaToCoverageState() const | |
| 156 | +{ | |
| 157 | + return Object::sendMessage<MTL4::AlphaToCoverageState>(this, _MTL_PRIVATE_SEL(alphaToCoverageState)); | |
| 158 | +} | |
| 159 | + | |
| 160 | +_MTL_INLINE MTL4::AlphaToOneState MTL4::MeshRenderPipelineDescriptor::alphaToOneState() const | |
| 161 | +{ | |
| 162 | + return Object::sendMessage<MTL4::AlphaToOneState>(this, _MTL_PRIVATE_SEL(alphaToOneState)); | |
| 163 | +} | |
| 164 | + | |
| 165 | +_MTL_INLINE MTL4::LogicalToPhysicalColorAttachmentMappingState MTL4::MeshRenderPipelineDescriptor::colorAttachmentMappingState() const | |
| 166 | +{ | |
| 167 | + return Object::sendMessage<MTL4::LogicalToPhysicalColorAttachmentMappingState>(this, _MTL_PRIVATE_SEL(colorAttachmentMappingState)); | |
| 168 | +} | |
| 169 | + | |
| 170 | +_MTL_INLINE MTL4::RenderPipelineColorAttachmentDescriptorArray* MTL4::MeshRenderPipelineDescriptor::colorAttachments() const | |
| 171 | +{ | |
| 172 | + return Object::sendMessage<MTL4::RenderPipelineColorAttachmentDescriptorArray*>(this, _MTL_PRIVATE_SEL(colorAttachments)); | |
| 173 | +} | |
| 174 | + | |
| 175 | +_MTL_INLINE MTL4::FunctionDescriptor* MTL4::MeshRenderPipelineDescriptor::fragmentFunctionDescriptor() const | |
| 176 | +{ | |
| 177 | + return Object::sendMessage<MTL4::FunctionDescriptor*>(this, _MTL_PRIVATE_SEL(fragmentFunctionDescriptor)); | |
| 178 | +} | |
| 179 | + | |
| 180 | +_MTL_INLINE MTL4::StaticLinkingDescriptor* MTL4::MeshRenderPipelineDescriptor::fragmentStaticLinkingDescriptor() const | |
| 181 | +{ | |
| 182 | + return Object::sendMessage<MTL4::StaticLinkingDescriptor*>(this, _MTL_PRIVATE_SEL(fragmentStaticLinkingDescriptor)); | |
| 183 | +} | |
| 184 | + | |
| 185 | +_MTL_INLINE MTL4::MeshRenderPipelineDescriptor* MTL4::MeshRenderPipelineDescriptor::init() | |
| 186 | +{ | |
| 187 | + return NS::Object::init<MTL4::MeshRenderPipelineDescriptor>(); | |
| 188 | +} | |
| 189 | + | |
| 190 | +_MTL_INLINE bool MTL4::MeshRenderPipelineDescriptor::isRasterizationEnabled() const | |
| 191 | +{ | |
| 192 | + return Object::sendMessage<bool>(this, _MTL_PRIVATE_SEL(isRasterizationEnabled)); | |
| 193 | +} | |
| 194 | + | |
| 195 | +_MTL_INLINE NS::UInteger MTL4::MeshRenderPipelineDescriptor::maxTotalThreadgroupsPerMeshGrid() const | |
| 196 | +{ | |
| 197 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(maxTotalThreadgroupsPerMeshGrid)); | |
| 198 | +} | |
| 199 | + | |
| 200 | +_MTL_INLINE NS::UInteger MTL4::MeshRenderPipelineDescriptor::maxTotalThreadsPerMeshThreadgroup() const | |
| 201 | +{ | |
| 202 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(maxTotalThreadsPerMeshThreadgroup)); | |
| 203 | +} | |
| 204 | + | |
| 205 | +_MTL_INLINE NS::UInteger MTL4::MeshRenderPipelineDescriptor::maxTotalThreadsPerObjectThreadgroup() const | |
| 206 | +{ | |
| 207 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(maxTotalThreadsPerObjectThreadgroup)); | |
| 208 | +} | |
| 209 | + | |
| 210 | +_MTL_INLINE NS::UInteger MTL4::MeshRenderPipelineDescriptor::maxVertexAmplificationCount() const | |
| 211 | +{ | |
| 212 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(maxVertexAmplificationCount)); | |
| 213 | +} | |
| 214 | + | |
| 215 | +_MTL_INLINE MTL4::FunctionDescriptor* MTL4::MeshRenderPipelineDescriptor::meshFunctionDescriptor() const | |
| 216 | +{ | |
| 217 | + return Object::sendMessage<MTL4::FunctionDescriptor*>(this, _MTL_PRIVATE_SEL(meshFunctionDescriptor)); | |
| 218 | +} | |
| 219 | + | |
| 220 | +_MTL_INLINE MTL4::StaticLinkingDescriptor* MTL4::MeshRenderPipelineDescriptor::meshStaticLinkingDescriptor() const | |
| 221 | +{ | |
| 222 | + return Object::sendMessage<MTL4::StaticLinkingDescriptor*>(this, _MTL_PRIVATE_SEL(meshStaticLinkingDescriptor)); | |
| 223 | +} | |
| 224 | + | |
| 225 | +_MTL_INLINE bool MTL4::MeshRenderPipelineDescriptor::meshThreadgroupSizeIsMultipleOfThreadExecutionWidth() const | |
| 226 | +{ | |
| 227 | + return Object::sendMessage<bool>(this, _MTL_PRIVATE_SEL(meshThreadgroupSizeIsMultipleOfThreadExecutionWidth)); | |
| 228 | +} | |
| 229 | + | |
| 230 | +_MTL_INLINE MTL4::FunctionDescriptor* MTL4::MeshRenderPipelineDescriptor::objectFunctionDescriptor() const | |
| 231 | +{ | |
| 232 | + return Object::sendMessage<MTL4::FunctionDescriptor*>(this, _MTL_PRIVATE_SEL(objectFunctionDescriptor)); | |
| 233 | +} | |
| 234 | + | |
| 235 | +_MTL_INLINE MTL4::StaticLinkingDescriptor* MTL4::MeshRenderPipelineDescriptor::objectStaticLinkingDescriptor() const | |
| 236 | +{ | |
| 237 | + return Object::sendMessage<MTL4::StaticLinkingDescriptor*>(this, _MTL_PRIVATE_SEL(objectStaticLinkingDescriptor)); | |
| 238 | +} | |
| 239 | + | |
| 240 | +_MTL_INLINE bool MTL4::MeshRenderPipelineDescriptor::objectThreadgroupSizeIsMultipleOfThreadExecutionWidth() const | |
| 241 | +{ | |
| 242 | + return Object::sendMessage<bool>(this, _MTL_PRIVATE_SEL(objectThreadgroupSizeIsMultipleOfThreadExecutionWidth)); | |
| 243 | +} | |
| 244 | + | |
| 245 | +_MTL_INLINE NS::UInteger MTL4::MeshRenderPipelineDescriptor::payloadMemoryLength() const | |
| 246 | +{ | |
| 247 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(payloadMemoryLength)); | |
| 248 | +} | |
| 249 | + | |
| 250 | +_MTL_INLINE NS::UInteger MTL4::MeshRenderPipelineDescriptor::rasterSampleCount() const | |
| 251 | +{ | |
| 252 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(rasterSampleCount)); | |
| 253 | +} | |
| 254 | + | |
| 255 | +_MTL_INLINE bool MTL4::MeshRenderPipelineDescriptor::rasterizationEnabled() const | |
| 256 | +{ | |
| 257 | + return Object::sendMessage<bool>(this, _MTL_PRIVATE_SEL(isRasterizationEnabled)); | |
| 258 | +} | |
| 259 | + | |
| 260 | +_MTL_INLINE MTL::Size MTL4::MeshRenderPipelineDescriptor::requiredThreadsPerMeshThreadgroup() const | |
| 261 | +{ | |
| 262 | + return Object::sendMessage<MTL::Size>(this, _MTL_PRIVATE_SEL(requiredThreadsPerMeshThreadgroup)); | |
| 263 | +} | |
| 264 | + | |
| 265 | +_MTL_INLINE MTL::Size MTL4::MeshRenderPipelineDescriptor::requiredThreadsPerObjectThreadgroup() const | |
| 266 | +{ | |
| 267 | + return Object::sendMessage<MTL::Size>(this, _MTL_PRIVATE_SEL(requiredThreadsPerObjectThreadgroup)); | |
| 268 | +} | |
| 269 | + | |
| 270 | +_MTL_INLINE void MTL4::MeshRenderPipelineDescriptor::reset() | |
| 271 | +{ | |
| 272 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(reset)); | |
| 273 | +} | |
| 274 | + | |
| 275 | +_MTL_INLINE void MTL4::MeshRenderPipelineDescriptor::setAlphaToCoverageState(MTL4::AlphaToCoverageState alphaToCoverageState) | |
| 276 | +{ | |
| 277 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setAlphaToCoverageState_), alphaToCoverageState); | |
| 278 | +} | |
| 279 | + | |
| 280 | +_MTL_INLINE void MTL4::MeshRenderPipelineDescriptor::setAlphaToOneState(MTL4::AlphaToOneState alphaToOneState) | |
| 281 | +{ | |
| 282 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setAlphaToOneState_), alphaToOneState); | |
| 283 | +} | |
| 284 | + | |
| 285 | +_MTL_INLINE void MTL4::MeshRenderPipelineDescriptor::setColorAttachmentMappingState(MTL4::LogicalToPhysicalColorAttachmentMappingState colorAttachmentMappingState) | |
| 286 | +{ | |
| 287 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setColorAttachmentMappingState_), colorAttachmentMappingState); | |
| 288 | +} | |
| 289 | + | |
| 290 | +_MTL_INLINE void MTL4::MeshRenderPipelineDescriptor::setFragmentFunctionDescriptor(const MTL4::FunctionDescriptor* fragmentFunctionDescriptor) | |
| 291 | +{ | |
| 292 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setFragmentFunctionDescriptor_), fragmentFunctionDescriptor); | |
| 293 | +} | |
| 294 | + | |
| 295 | +_MTL_INLINE void MTL4::MeshRenderPipelineDescriptor::setFragmentStaticLinkingDescriptor(const MTL4::StaticLinkingDescriptor* fragmentStaticLinkingDescriptor) | |
| 296 | +{ | |
| 297 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setFragmentStaticLinkingDescriptor_), fragmentStaticLinkingDescriptor); | |
| 298 | +} | |
| 299 | + | |
| 300 | +_MTL_INLINE void MTL4::MeshRenderPipelineDescriptor::setMaxTotalThreadgroupsPerMeshGrid(NS::UInteger maxTotalThreadgroupsPerMeshGrid) | |
| 301 | +{ | |
| 302 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMaxTotalThreadgroupsPerMeshGrid_), maxTotalThreadgroupsPerMeshGrid); | |
| 303 | +} | |
| 304 | + | |
| 305 | +_MTL_INLINE void MTL4::MeshRenderPipelineDescriptor::setMaxTotalThreadsPerMeshThreadgroup(NS::UInteger maxTotalThreadsPerMeshThreadgroup) | |
| 306 | +{ | |
| 307 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMaxTotalThreadsPerMeshThreadgroup_), maxTotalThreadsPerMeshThreadgroup); | |
| 308 | +} | |
| 309 | + | |
| 310 | +_MTL_INLINE void MTL4::MeshRenderPipelineDescriptor::setMaxTotalThreadsPerObjectThreadgroup(NS::UInteger maxTotalThreadsPerObjectThreadgroup) | |
| 311 | +{ | |
| 312 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMaxTotalThreadsPerObjectThreadgroup_), maxTotalThreadsPerObjectThreadgroup); | |
| 313 | +} | |
| 314 | + | |
| 315 | +_MTL_INLINE void MTL4::MeshRenderPipelineDescriptor::setMaxVertexAmplificationCount(NS::UInteger maxVertexAmplificationCount) | |
| 316 | +{ | |
| 317 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMaxVertexAmplificationCount_), maxVertexAmplificationCount); | |
| 318 | +} | |
| 319 | + | |
| 320 | +_MTL_INLINE void MTL4::MeshRenderPipelineDescriptor::setMeshFunctionDescriptor(const MTL4::FunctionDescriptor* meshFunctionDescriptor) | |
| 321 | +{ | |
| 322 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMeshFunctionDescriptor_), meshFunctionDescriptor); | |
| 323 | +} | |
| 324 | + | |
| 325 | +_MTL_INLINE void MTL4::MeshRenderPipelineDescriptor::setMeshStaticLinkingDescriptor(const MTL4::StaticLinkingDescriptor* meshStaticLinkingDescriptor) | |
| 326 | +{ | |
| 327 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMeshStaticLinkingDescriptor_), meshStaticLinkingDescriptor); | |
| 328 | +} | |
| 329 | + | |
| 330 | +_MTL_INLINE void MTL4::MeshRenderPipelineDescriptor::setMeshThreadgroupSizeIsMultipleOfThreadExecutionWidth(bool meshThreadgroupSizeIsMultipleOfThreadExecutionWidth) | |
| 331 | +{ | |
| 332 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setMeshThreadgroupSizeIsMultipleOfThreadExecutionWidth_), meshThreadgroupSizeIsMultipleOfThreadExecutionWidth); | |
| 333 | +} | |
| 334 | + | |
| 335 | +_MTL_INLINE void MTL4::MeshRenderPipelineDescriptor::setObjectFunctionDescriptor(const MTL4::FunctionDescriptor* objectFunctionDescriptor) | |
| 336 | +{ | |
| 337 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setObjectFunctionDescriptor_), objectFunctionDescriptor); | |
| 338 | +} | |
| 339 | + | |
| 340 | +_MTL_INLINE void MTL4::MeshRenderPipelineDescriptor::setObjectStaticLinkingDescriptor(const MTL4::StaticLinkingDescriptor* objectStaticLinkingDescriptor) | |
| 341 | +{ | |
| 342 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setObjectStaticLinkingDescriptor_), objectStaticLinkingDescriptor); | |
| 343 | +} | |
| 344 | + | |
| 345 | +_MTL_INLINE void MTL4::MeshRenderPipelineDescriptor::setObjectThreadgroupSizeIsMultipleOfThreadExecutionWidth(bool objectThreadgroupSizeIsMultipleOfThreadExecutionWidth) | |
| 346 | +{ | |
| 347 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setObjectThreadgroupSizeIsMultipleOfThreadExecutionWidth_), objectThreadgroupSizeIsMultipleOfThreadExecutionWidth); | |
| 348 | +} | |
| 349 | + | |
| 350 | +_MTL_INLINE void MTL4::MeshRenderPipelineDescriptor::setPayloadMemoryLength(NS::UInteger payloadMemoryLength) | |
| 351 | +{ | |
| 352 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setPayloadMemoryLength_), payloadMemoryLength); | |
| 353 | +} | |
| 354 | + | |
| 355 | +_MTL_INLINE void MTL4::MeshRenderPipelineDescriptor::setRasterSampleCount(NS::UInteger rasterSampleCount) | |
| 356 | +{ | |
| 357 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setRasterSampleCount_), rasterSampleCount); | |
| 358 | +} | |
| 359 | + | |
| 360 | +_MTL_INLINE void MTL4::MeshRenderPipelineDescriptor::setRasterizationEnabled(bool rasterizationEnabled) | |
| 361 | +{ | |
| 362 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setRasterizationEnabled_), rasterizationEnabled); | |
| 363 | +} | |
| 364 | + | |
| 365 | +_MTL_INLINE void MTL4::MeshRenderPipelineDescriptor::setRequiredThreadsPerMeshThreadgroup(MTL::Size requiredThreadsPerMeshThreadgroup) | |
| 366 | +{ | |
| 367 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setRequiredThreadsPerMeshThreadgroup_), requiredThreadsPerMeshThreadgroup); | |
| 368 | +} | |
| 369 | + | |
| 370 | +_MTL_INLINE void MTL4::MeshRenderPipelineDescriptor::setRequiredThreadsPerObjectThreadgroup(MTL::Size requiredThreadsPerObjectThreadgroup) | |
| 371 | +{ | |
| 372 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setRequiredThreadsPerObjectThreadgroup_), requiredThreadsPerObjectThreadgroup); | |
| 373 | +} | |
| 374 | + | |
| 375 | +_MTL_INLINE void MTL4::MeshRenderPipelineDescriptor::setSupportFragmentBinaryLinking(bool supportFragmentBinaryLinking) | |
| 376 | +{ | |
| 377 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setSupportFragmentBinaryLinking_), supportFragmentBinaryLinking); | |
| 378 | +} | |
| 379 | + | |
| 380 | +_MTL_INLINE void MTL4::MeshRenderPipelineDescriptor::setSupportIndirectCommandBuffers(MTL4::IndirectCommandBufferSupportState supportIndirectCommandBuffers) | |
| 381 | +{ | |
| 382 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setSupportIndirectCommandBuffers_), supportIndirectCommandBuffers); | |
| 383 | +} | |
| 384 | + | |
| 385 | +_MTL_INLINE void MTL4::MeshRenderPipelineDescriptor::setSupportMeshBinaryLinking(bool supportMeshBinaryLinking) | |
| 386 | +{ | |
| 387 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setSupportMeshBinaryLinking_), supportMeshBinaryLinking); | |
| 388 | +} | |
| 389 | + | |
| 390 | +_MTL_INLINE void MTL4::MeshRenderPipelineDescriptor::setSupportObjectBinaryLinking(bool supportObjectBinaryLinking) | |
| 391 | +{ | |
| 392 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setSupportObjectBinaryLinking_), supportObjectBinaryLinking); | |
| 393 | +} | |
| 394 | + | |
| 395 | +_MTL_INLINE bool MTL4::MeshRenderPipelineDescriptor::supportFragmentBinaryLinking() const | |
| 396 | +{ | |
| 397 | + return Object::sendMessageSafe<bool>(this, _MTL_PRIVATE_SEL(supportFragmentBinaryLinking)); | |
| 398 | +} | |
| 399 | + | |
| 400 | +_MTL_INLINE MTL4::IndirectCommandBufferSupportState MTL4::MeshRenderPipelineDescriptor::supportIndirectCommandBuffers() const | |
| 401 | +{ | |
| 402 | + return Object::sendMessage<MTL4::IndirectCommandBufferSupportState>(this, _MTL_PRIVATE_SEL(supportIndirectCommandBuffers)); | |
| 403 | +} | |
| 404 | + | |
| 405 | +_MTL_INLINE bool MTL4::MeshRenderPipelineDescriptor::supportMeshBinaryLinking() const | |
| 406 | +{ | |
| 407 | + return Object::sendMessageSafe<bool>(this, _MTL_PRIVATE_SEL(supportMeshBinaryLinking)); | |
| 408 | +} | |
| 409 | + | |
| 410 | +_MTL_INLINE bool MTL4::MeshRenderPipelineDescriptor::supportObjectBinaryLinking() const | |
| 411 | +{ | |
| 412 | + return Object::sendMessageSafe<bool>(this, _MTL_PRIVATE_SEL(supportObjectBinaryLinking)); | |
| 413 | +} | |
added
third_party/metal-cpp/Metal/MTL4PipelineDataSetSerializer.hpp
+85 −0
@@ -0,0 +1,85 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Metal/MTL4PipelineDataSetSerializer.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2025 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include "../Foundation/Foundation.hpp" | |
| 24 | +#include "MTLDefines.hpp" | |
| 25 | +#include "MTLHeaderBridge.hpp" | |
| 26 | +#include "MTLPrivate.hpp" | |
| 27 | + | |
| 28 | +namespace MTL4 | |
| 29 | +{ | |
| 30 | +class PipelineDataSetSerializerDescriptor; | |
| 31 | + | |
| 32 | +_MTL_OPTIONS(NS::UInteger, PipelineDataSetSerializerConfiguration) { | |
| 33 | + PipelineDataSetSerializerConfigurationCaptureDescriptors = 1, | |
| 34 | + PipelineDataSetSerializerConfigurationCaptureBinaries = 1 << 1, | |
| 35 | +}; | |
| 36 | + | |
| 37 | +class PipelineDataSetSerializerDescriptor : public NS::Copying<PipelineDataSetSerializerDescriptor> | |
| 38 | +{ | |
| 39 | +public: | |
| 40 | + static PipelineDataSetSerializerDescriptor* alloc(); | |
| 41 | + | |
| 42 | + PipelineDataSetSerializerConfiguration configuration() const; | |
| 43 | + | |
| 44 | + PipelineDataSetSerializerDescriptor* init(); | |
| 45 | + | |
| 46 | + void setConfiguration(MTL4::PipelineDataSetSerializerConfiguration configuration); | |
| 47 | +}; | |
| 48 | +class PipelineDataSetSerializer : public NS::Referencing<PipelineDataSetSerializer> | |
| 49 | +{ | |
| 50 | +public: | |
| 51 | + bool serializeAsArchiveAndFlushToURL(const NS::URL* url, NS::Error** error); | |
| 52 | + | |
| 53 | + NS::Data* serializeAsPipelinesScript(NS::Error** error); | |
| 54 | +}; | |
| 55 | + | |
| 56 | +} | |
| 57 | +_MTL_INLINE MTL4::PipelineDataSetSerializerDescriptor* MTL4::PipelineDataSetSerializerDescriptor::alloc() | |
| 58 | +{ | |
| 59 | + return NS::Object::alloc<MTL4::PipelineDataSetSerializerDescriptor>(_MTL_PRIVATE_CLS(MTL4PipelineDataSetSerializerDescriptor)); | |
| 60 | +} | |
| 61 | + | |
| 62 | +_MTL_INLINE MTL4::PipelineDataSetSerializerConfiguration MTL4::PipelineDataSetSerializerDescriptor::configuration() const | |
| 63 | +{ | |
| 64 | + return Object::sendMessage<MTL4::PipelineDataSetSerializerConfiguration>(this, _MTL_PRIVATE_SEL(configuration)); | |
| 65 | +} | |
| 66 | + | |
| 67 | +_MTL_INLINE MTL4::PipelineDataSetSerializerDescriptor* MTL4::PipelineDataSetSerializerDescriptor::init() | |
| 68 | +{ | |
| 69 | + return NS::Object::init<MTL4::PipelineDataSetSerializerDescriptor>(); | |
| 70 | +} | |
| 71 | + | |
| 72 | +_MTL_INLINE void MTL4::PipelineDataSetSerializerDescriptor::setConfiguration(MTL4::PipelineDataSetSerializerConfiguration configuration) | |
| 73 | +{ | |
| 74 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setConfiguration_), configuration); | |
| 75 | +} | |
| 76 | + | |
| 77 | +_MTL_INLINE bool MTL4::PipelineDataSetSerializer::serializeAsArchiveAndFlushToURL(const NS::URL* url, NS::Error** error) | |
| 78 | +{ | |
| 79 | + return Object::sendMessage<bool>(this, _MTL_PRIVATE_SEL(serializeAsArchiveAndFlushToURL_error_), url, error); | |
| 80 | +} | |
| 81 | + | |
| 82 | +_MTL_INLINE NS::Data* MTL4::PipelineDataSetSerializer::serializeAsPipelinesScript(NS::Error** error) | |
| 83 | +{ | |
| 84 | + return Object::sendMessage<NS::Data*>(this, _MTL_PRIVATE_SEL(serializeAsPipelinesScriptWithError_), error); | |
| 85 | +} | |
added
third_party/metal-cpp/Metal/MTL4PipelineState.hpp
+150 −0
@@ -0,0 +1,150 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Metal/MTL4PipelineState.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2025 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include "../Foundation/Foundation.hpp" | |
| 24 | +#include "MTLDefines.hpp" | |
| 25 | +#include "MTLHeaderBridge.hpp" | |
| 26 | +#include "MTLPipeline.hpp" | |
| 27 | +#include "MTLPrivate.hpp" | |
| 28 | + | |
| 29 | +namespace MTL4 | |
| 30 | +{ | |
| 31 | +class PipelineDescriptor; | |
| 32 | +class PipelineOptions; | |
| 33 | +_MTL_ENUM(NS::Integer, AlphaToOneState) { | |
| 34 | + AlphaToOneStateDisabled = 0, | |
| 35 | + AlphaToOneStateEnabled = 1, | |
| 36 | +}; | |
| 37 | + | |
| 38 | +_MTL_ENUM(NS::Integer, AlphaToCoverageState) { | |
| 39 | + AlphaToCoverageStateDisabled = 0, | |
| 40 | + AlphaToCoverageStateEnabled = 1, | |
| 41 | +}; | |
| 42 | + | |
| 43 | +_MTL_ENUM(NS::Integer, BlendState) { | |
| 44 | + BlendStateDisabled = 0, | |
| 45 | + BlendStateEnabled = 1, | |
| 46 | + BlendStateUnspecialized = 2, | |
| 47 | +}; | |
| 48 | + | |
| 49 | +_MTL_ENUM(NS::Integer, IndirectCommandBufferSupportState) { | |
| 50 | + IndirectCommandBufferSupportStateDisabled = 0, | |
| 51 | + IndirectCommandBufferSupportStateEnabled = 1, | |
| 52 | +}; | |
| 53 | + | |
| 54 | +_MTL_OPTIONS(NS::UInteger, ShaderReflection) { | |
| 55 | + ShaderReflectionNone = 0, | |
| 56 | + ShaderReflectionBindingInfo = 1, | |
| 57 | + ShaderReflectionBufferTypeInfo = 1 << 1, | |
| 58 | +}; | |
| 59 | + | |
| 60 | +class PipelineOptions : public NS::Copying<PipelineOptions> | |
| 61 | +{ | |
| 62 | +public: | |
| 63 | + static PipelineOptions* alloc(); | |
| 64 | + | |
| 65 | + PipelineOptions* init(); | |
| 66 | + | |
| 67 | + void setShaderReflection(MTL4::ShaderReflection shaderReflection); | |
| 68 | + | |
| 69 | + void setShaderValidation(MTL::ShaderValidation shaderValidation); | |
| 70 | + | |
| 71 | + ShaderReflection shaderReflection() const; | |
| 72 | + | |
| 73 | + MTL::ShaderValidation shaderValidation() const; | |
| 74 | +}; | |
| 75 | +class PipelineDescriptor : public NS::Copying<PipelineDescriptor> | |
| 76 | +{ | |
| 77 | +public: | |
| 78 | + static PipelineDescriptor* alloc(); | |
| 79 | + | |
| 80 | + PipelineDescriptor* init(); | |
| 81 | + | |
| 82 | + NS::String* label() const; | |
| 83 | + | |
| 84 | + PipelineOptions* options() const; | |
| 85 | + | |
| 86 | + void setLabel(const NS::String* label); | |
| 87 | + | |
| 88 | + void setOptions(const MTL4::PipelineOptions* options); | |
| 89 | +}; | |
| 90 | + | |
| 91 | +} | |
| 92 | +_MTL_INLINE MTL4::PipelineOptions* MTL4::PipelineOptions::alloc() | |
| 93 | +{ | |
| 94 | + return NS::Object::alloc<MTL4::PipelineOptions>(_MTL_PRIVATE_CLS(MTL4PipelineOptions)); | |
| 95 | +} | |
| 96 | + | |
| 97 | +_MTL_INLINE MTL4::PipelineOptions* MTL4::PipelineOptions::init() | |
| 98 | +{ | |
| 99 | + return NS::Object::init<MTL4::PipelineOptions>(); | |
| 100 | +} | |
| 101 | + | |
| 102 | +_MTL_INLINE void MTL4::PipelineOptions::setShaderReflection(MTL4::ShaderReflection shaderReflection) | |
| 103 | +{ | |
| 104 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setShaderReflection_), shaderReflection); | |
| 105 | +} | |
| 106 | + | |
| 107 | +_MTL_INLINE void MTL4::PipelineOptions::setShaderValidation(MTL::ShaderValidation shaderValidation) | |
| 108 | +{ | |
| 109 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setShaderValidation_), shaderValidation); | |
| 110 | +} | |
| 111 | + | |
| 112 | +_MTL_INLINE MTL4::ShaderReflection MTL4::PipelineOptions::shaderReflection() const | |
| 113 | +{ | |
| 114 | + return Object::sendMessage<MTL4::ShaderReflection>(this, _MTL_PRIVATE_SEL(shaderReflection)); | |
| 115 | +} | |
| 116 | + | |
| 117 | +_MTL_INLINE MTL::ShaderValidation MTL4::PipelineOptions::shaderValidation() const | |
| 118 | +{ | |
| 119 | + return Object::sendMessage<MTL::ShaderValidation>(this, _MTL_PRIVATE_SEL(shaderValidation)); | |
| 120 | +} | |
| 121 | + | |
| 122 | +_MTL_INLINE MTL4::PipelineDescriptor* MTL4::PipelineDescriptor::alloc() | |
| 123 | +{ | |
| 124 | + return NS::Object::alloc<MTL4::PipelineDescriptor>(_MTL_PRIVATE_CLS(MTL4PipelineDescriptor)); | |
| 125 | +} | |
| 126 | + | |
| 127 | +_MTL_INLINE MTL4::PipelineDescriptor* MTL4::PipelineDescriptor::init() | |
| 128 | +{ | |
| 129 | + return NS::Object::init<MTL4::PipelineDescriptor>(); | |
| 130 | +} | |
| 131 | + | |
| 132 | +_MTL_INLINE NS::String* MTL4::PipelineDescriptor::label() const | |
| 133 | +{ | |
| 134 | + return Object::sendMessage<NS::String*>(this, _MTL_PRIVATE_SEL(label)); | |
| 135 | +} | |
| 136 | + | |
| 137 | +_MTL_INLINE MTL4::PipelineOptions* MTL4::PipelineDescriptor::options() const | |
| 138 | +{ | |
| 139 | + return Object::sendMessage<MTL4::PipelineOptions*>(this, _MTL_PRIVATE_SEL(options)); | |
| 140 | +} | |
| 141 | + | |
| 142 | +_MTL_INLINE void MTL4::PipelineDescriptor::setLabel(const NS::String* label) | |
| 143 | +{ | |
| 144 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setLabel_), label); | |
| 145 | +} | |
| 146 | + | |
| 147 | +_MTL_INLINE void MTL4::PipelineDescriptor::setOptions(const MTL4::PipelineOptions* options) | |
| 148 | +{ | |
| 149 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setOptions_), options); | |
| 150 | +} | |
added
third_party/metal-cpp/Metal/MTL4RenderCommandEncoder.hpp
+340 −0
@@ -0,0 +1,340 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Metal/MTL4RenderCommandEncoder.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2025 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include "../Foundation/Foundation.hpp" | |
| 24 | +#include "MTL4CommandEncoder.hpp" | |
| 25 | +#include "MTL4Counters.hpp" | |
| 26 | +#include "MTLArgument.hpp" | |
| 27 | +#include "MTLDefines.hpp" | |
| 28 | +#include "MTLGPUAddress.hpp" | |
| 29 | +#include "MTLHeaderBridge.hpp" | |
| 30 | +#include "MTLPrivate.hpp" | |
| 31 | +#include "MTLRenderCommandEncoder.hpp" | |
| 32 | +#include "MTLRenderPass.hpp" | |
| 33 | +#include "MTLTypes.hpp" | |
| 34 | +#include <cstdint> | |
| 35 | + | |
| 36 | +namespace MTL4 | |
| 37 | +{ | |
| 38 | +class ArgumentTable; | |
| 39 | +class CounterHeap; | |
| 40 | +} | |
| 41 | + | |
| 42 | +namespace MTL | |
| 43 | +{ | |
| 44 | +class DepthStencilState; | |
| 45 | +class IndirectCommandBuffer; | |
| 46 | +class LogicalToPhysicalColorAttachmentMap; | |
| 47 | +class RenderPipelineState; | |
| 48 | +struct ScissorRect; | |
| 49 | +struct VertexAmplificationViewMapping; | |
| 50 | +struct Viewport; | |
| 51 | + | |
| 52 | +} | |
| 53 | +namespace MTL4 | |
| 54 | +{ | |
| 55 | +_MTL_OPTIONS(NS::UInteger, RenderEncoderOptions) { | |
| 56 | + RenderEncoderOptionNone = 0, | |
| 57 | + RenderEncoderOptionSuspending = 1, | |
| 58 | + RenderEncoderOptionResuming = 1 << 1, | |
| 59 | +}; | |
| 60 | + | |
| 61 | +class RenderCommandEncoder : public NS::Referencing<RenderCommandEncoder, CommandEncoder> | |
| 62 | +{ | |
| 63 | +public: | |
| 64 | + void dispatchThreadsPerTile(MTL::Size threadsPerTile); | |
| 65 | + | |
| 66 | + void drawIndexedPrimitives(MTL::PrimitiveType primitiveType, NS::UInteger indexCount, MTL::IndexType indexType, MTL::GPUAddress indexBuffer, NS::UInteger indexBufferLength); | |
| 67 | + void drawIndexedPrimitives(MTL::PrimitiveType primitiveType, NS::UInteger indexCount, MTL::IndexType indexType, MTL::GPUAddress indexBuffer, NS::UInteger indexBufferLength, NS::UInteger instanceCount); | |
| 68 | + void drawIndexedPrimitives(MTL::PrimitiveType primitiveType, NS::UInteger indexCount, MTL::IndexType indexType, MTL::GPUAddress indexBuffer, NS::UInteger indexBufferLength, NS::UInteger instanceCount, NS::Integer baseVertex, NS::UInteger baseInstance); | |
| 69 | + void drawIndexedPrimitives(MTL::PrimitiveType primitiveType, MTL::IndexType indexType, MTL::GPUAddress indexBuffer, NS::UInteger indexBufferLength, MTL::GPUAddress indirectBuffer); | |
| 70 | + | |
| 71 | + void drawMeshThreadgroups(MTL::Size threadgroupsPerGrid, MTL::Size threadsPerObjectThreadgroup, MTL::Size threadsPerMeshThreadgroup); | |
| 72 | + void drawMeshThreadgroups(MTL::GPUAddress indirectBuffer, MTL::Size threadsPerObjectThreadgroup, MTL::Size threadsPerMeshThreadgroup); | |
| 73 | + | |
| 74 | + void drawMeshThreads(MTL::Size threadsPerGrid, MTL::Size threadsPerObjectThreadgroup, MTL::Size threadsPerMeshThreadgroup); | |
| 75 | + | |
| 76 | + void drawPrimitives(MTL::PrimitiveType primitiveType, NS::UInteger vertexStart, NS::UInteger vertexCount); | |
| 77 | + void drawPrimitives(MTL::PrimitiveType primitiveType, NS::UInteger vertexStart, NS::UInteger vertexCount, NS::UInteger instanceCount); | |
| 78 | + void drawPrimitives(MTL::PrimitiveType primitiveType, NS::UInteger vertexStart, NS::UInteger vertexCount, NS::UInteger instanceCount, NS::UInteger baseInstance); | |
| 79 | + void drawPrimitives(MTL::PrimitiveType primitiveType, MTL::GPUAddress indirectBuffer); | |
| 80 | + | |
| 81 | + void executeCommandsInBuffer(const MTL::IndirectCommandBuffer* indirectCommandBuffer, NS::Range executionRange); | |
| 82 | + void executeCommandsInBuffer(const MTL::IndirectCommandBuffer* indirectCommandBuffer, MTL::GPUAddress indirectRangeBuffer); | |
| 83 | + | |
| 84 | + void setArgumentTable(const MTL4::ArgumentTable* argumentTable, MTL::RenderStages stages); | |
| 85 | + | |
| 86 | + void setBlendColor(float red, float green, float blue, float alpha); | |
| 87 | + | |
| 88 | + void setColorAttachmentMap(const MTL::LogicalToPhysicalColorAttachmentMap* mapping); | |
| 89 | + | |
| 90 | + void setColorStoreAction(MTL::StoreAction storeAction, NS::UInteger colorAttachmentIndex); | |
| 91 | + | |
| 92 | + void setCullMode(MTL::CullMode cullMode); | |
| 93 | + | |
| 94 | + void setDepthBias(float depthBias, float slopeScale, float clamp); | |
| 95 | + | |
| 96 | + void setDepthClipMode(MTL::DepthClipMode depthClipMode); | |
| 97 | + | |
| 98 | + void setDepthStencilState(const MTL::DepthStencilState* depthStencilState); | |
| 99 | + | |
| 100 | + void setDepthStoreAction(MTL::StoreAction storeAction); | |
| 101 | + | |
| 102 | + void setDepthTestBounds(float minBound, float maxBound); | |
| 103 | + | |
| 104 | + void setFrontFacingWinding(MTL::Winding frontFacingWinding); | |
| 105 | + | |
| 106 | + void setObjectThreadgroupMemoryLength(NS::UInteger length, NS::UInteger index); | |
| 107 | + | |
| 108 | + void setRenderPipelineState(const MTL::RenderPipelineState* pipelineState); | |
| 109 | + | |
| 110 | + void setScissorRect(MTL::ScissorRect rect); | |
| 111 | + void setScissorRects(const MTL::ScissorRect* scissorRects, NS::UInteger count); | |
| 112 | + | |
| 113 | + void setStencilReferenceValue(uint32_t referenceValue); | |
| 114 | + void setStencilReferenceValues(uint32_t frontReferenceValue, uint32_t backReferenceValue); | |
| 115 | + | |
| 116 | + void setStencilStoreAction(MTL::StoreAction storeAction); | |
| 117 | + | |
| 118 | + void setThreadgroupMemoryLength(NS::UInteger length, NS::UInteger offset, NS::UInteger index); | |
| 119 | + | |
| 120 | + void setTriangleFillMode(MTL::TriangleFillMode fillMode); | |
| 121 | + | |
| 122 | + void setVertexAmplificationCount(NS::UInteger count, const MTL::VertexAmplificationViewMapping* viewMappings); | |
| 123 | + | |
| 124 | + void setViewport(MTL::Viewport viewport); | |
| 125 | + void setViewports(const MTL::Viewport* viewports, NS::UInteger count); | |
| 126 | + | |
| 127 | + void setVisibilityResultMode(MTL::VisibilityResultMode mode, NS::UInteger offset); | |
| 128 | + | |
| 129 | + NS::UInteger tileHeight() const; | |
| 130 | + | |
| 131 | + NS::UInteger tileWidth() const; | |
| 132 | + | |
| 133 | + void writeTimestamp(MTL4::TimestampGranularity granularity, MTL::RenderStages stage, const MTL4::CounterHeap* counterHeap, NS::UInteger index); | |
| 134 | +}; | |
| 135 | + | |
| 136 | +} | |
| 137 | +_MTL_INLINE void MTL4::RenderCommandEncoder::dispatchThreadsPerTile(MTL::Size threadsPerTile) | |
| 138 | +{ | |
| 139 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(dispatchThreadsPerTile_), threadsPerTile); | |
| 140 | +} | |
| 141 | + | |
| 142 | +_MTL_INLINE void MTL4::RenderCommandEncoder::drawIndexedPrimitives(MTL::PrimitiveType primitiveType, NS::UInteger indexCount, MTL::IndexType indexType, MTL::GPUAddress indexBuffer, NS::UInteger indexBufferLength) | |
| 143 | +{ | |
| 144 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(drawIndexedPrimitives_indexCount_indexType_indexBuffer_indexBufferLength_), primitiveType, indexCount, indexType, indexBuffer, indexBufferLength); | |
| 145 | +} | |
| 146 | + | |
| 147 | +_MTL_INLINE void MTL4::RenderCommandEncoder::drawIndexedPrimitives(MTL::PrimitiveType primitiveType, NS::UInteger indexCount, MTL::IndexType indexType, MTL::GPUAddress indexBuffer, NS::UInteger indexBufferLength, NS::UInteger instanceCount) | |
| 148 | +{ | |
| 149 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(drawIndexedPrimitives_indexCount_indexType_indexBuffer_indexBufferLength_instanceCount_), primitiveType, indexCount, indexType, indexBuffer, indexBufferLength, instanceCount); | |
| 150 | +} | |
| 151 | + | |
| 152 | +_MTL_INLINE void MTL4::RenderCommandEncoder::drawIndexedPrimitives(MTL::PrimitiveType primitiveType, NS::UInteger indexCount, MTL::IndexType indexType, MTL::GPUAddress indexBuffer, NS::UInteger indexBufferLength, NS::UInteger instanceCount, NS::Integer baseVertex, NS::UInteger baseInstance) | |
| 153 | +{ | |
| 154 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(drawIndexedPrimitives_indexCount_indexType_indexBuffer_indexBufferLength_instanceCount_baseVertex_baseInstance_), primitiveType, indexCount, indexType, indexBuffer, indexBufferLength, instanceCount, baseVertex, baseInstance); | |
| 155 | +} | |
| 156 | + | |
| 157 | +_MTL_INLINE void MTL4::RenderCommandEncoder::drawIndexedPrimitives(MTL::PrimitiveType primitiveType, MTL::IndexType indexType, MTL::GPUAddress indexBuffer, NS::UInteger indexBufferLength, MTL::GPUAddress indirectBuffer) | |
| 158 | +{ | |
| 159 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(drawIndexedPrimitives_indexType_indexBuffer_indexBufferLength_indirectBuffer_), primitiveType, indexType, indexBuffer, indexBufferLength, indirectBuffer); | |
| 160 | +} | |
| 161 | + | |
| 162 | +_MTL_INLINE void MTL4::RenderCommandEncoder::drawMeshThreadgroups(MTL::Size threadgroupsPerGrid, MTL::Size threadsPerObjectThreadgroup, MTL::Size threadsPerMeshThreadgroup) | |
| 163 | +{ | |
| 164 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(drawMeshThreadgroups_threadsPerObjectThreadgroup_threadsPerMeshThreadgroup_), threadgroupsPerGrid, threadsPerObjectThreadgroup, threadsPerMeshThreadgroup); | |
| 165 | +} | |
| 166 | + | |
| 167 | +_MTL_INLINE void MTL4::RenderCommandEncoder::drawMeshThreadgroups(MTL::GPUAddress indirectBuffer, MTL::Size threadsPerObjectThreadgroup, MTL::Size threadsPerMeshThreadgroup) | |
| 168 | +{ | |
| 169 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(drawMeshThreadgroupsWithIndirectBuffer_threadsPerObjectThreadgroup_threadsPerMeshThreadgroup_), indirectBuffer, threadsPerObjectThreadgroup, threadsPerMeshThreadgroup); | |
| 170 | +} | |
| 171 | + | |
| 172 | +_MTL_INLINE void MTL4::RenderCommandEncoder::drawMeshThreads(MTL::Size threadsPerGrid, MTL::Size threadsPerObjectThreadgroup, MTL::Size threadsPerMeshThreadgroup) | |
| 173 | +{ | |
| 174 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(drawMeshThreads_threadsPerObjectThreadgroup_threadsPerMeshThreadgroup_), threadsPerGrid, threadsPerObjectThreadgroup, threadsPerMeshThreadgroup); | |
| 175 | +} | |
| 176 | + | |
| 177 | +_MTL_INLINE void MTL4::RenderCommandEncoder::drawPrimitives(MTL::PrimitiveType primitiveType, NS::UInteger vertexStart, NS::UInteger vertexCount) | |
| 178 | +{ | |
| 179 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(drawPrimitives_vertexStart_vertexCount_), primitiveType, vertexStart, vertexCount); | |
| 180 | +} | |
| 181 | + | |
| 182 | +_MTL_INLINE void MTL4::RenderCommandEncoder::drawPrimitives(MTL::PrimitiveType primitiveType, NS::UInteger vertexStart, NS::UInteger vertexCount, NS::UInteger instanceCount) | |
| 183 | +{ | |
| 184 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(drawPrimitives_vertexStart_vertexCount_instanceCount_), primitiveType, vertexStart, vertexCount, instanceCount); | |
| 185 | +} | |
| 186 | + | |
| 187 | +_MTL_INLINE void MTL4::RenderCommandEncoder::drawPrimitives(MTL::PrimitiveType primitiveType, NS::UInteger vertexStart, NS::UInteger vertexCount, NS::UInteger instanceCount, NS::UInteger baseInstance) | |
| 188 | +{ | |
| 189 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(drawPrimitives_vertexStart_vertexCount_instanceCount_baseInstance_), primitiveType, vertexStart, vertexCount, instanceCount, baseInstance); | |
| 190 | +} | |
| 191 | + | |
| 192 | +_MTL_INLINE void MTL4::RenderCommandEncoder::drawPrimitives(MTL::PrimitiveType primitiveType, MTL::GPUAddress indirectBuffer) | |
| 193 | +{ | |
| 194 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(drawPrimitives_indirectBuffer_), primitiveType, indirectBuffer); | |
| 195 | +} | |
| 196 | + | |
| 197 | +_MTL_INLINE void MTL4::RenderCommandEncoder::executeCommandsInBuffer(const MTL::IndirectCommandBuffer* indirectCommandBuffer, NS::Range executionRange) | |
| 198 | +{ | |
| 199 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(executeCommandsInBuffer_withRange_), indirectCommandBuffer, executionRange); | |
| 200 | +} | |
| 201 | + | |
| 202 | +_MTL_INLINE void MTL4::RenderCommandEncoder::executeCommandsInBuffer(const MTL::IndirectCommandBuffer* indirectCommandBuffer, MTL::GPUAddress indirectRangeBuffer) | |
| 203 | +{ | |
| 204 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(executeCommandsInBuffer_indirectBuffer_), indirectCommandBuffer, indirectRangeBuffer); | |
| 205 | +} | |
| 206 | + | |
| 207 | +_MTL_INLINE void MTL4::RenderCommandEncoder::setArgumentTable(const MTL4::ArgumentTable* argumentTable, MTL::RenderStages stages) | |
| 208 | +{ | |
| 209 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setArgumentTable_atStages_), argumentTable, stages); | |
| 210 | +} | |
| 211 | + | |
| 212 | +_MTL_INLINE void MTL4::RenderCommandEncoder::setBlendColor(float red, float green, float blue, float alpha) | |
| 213 | +{ | |
| 214 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setBlendColorRed_green_blue_alpha_), red, green, blue, alpha); | |
| 215 | +} | |
| 216 | + | |
| 217 | +_MTL_INLINE void MTL4::RenderCommandEncoder::setColorAttachmentMap(const MTL::LogicalToPhysicalColorAttachmentMap* mapping) | |
| 218 | +{ | |
| 219 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setColorAttachmentMap_), mapping); | |
| 220 | +} | |
| 221 | + | |
| 222 | +_MTL_INLINE void MTL4::RenderCommandEncoder::setColorStoreAction(MTL::StoreAction storeAction, NS::UInteger colorAttachmentIndex) | |
| 223 | +{ | |
| 224 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setColorStoreAction_atIndex_), storeAction, colorAttachmentIndex); | |
| 225 | +} | |
| 226 | + | |
| 227 | +_MTL_INLINE void MTL4::RenderCommandEncoder::setCullMode(MTL::CullMode cullMode) | |
| 228 | +{ | |
| 229 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setCullMode_), cullMode); | |
| 230 | +} | |
| 231 | + | |
| 232 | +_MTL_INLINE void MTL4::RenderCommandEncoder::setDepthBias(float depthBias, float slopeScale, float clamp) | |
| 233 | +{ | |
| 234 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setDepthBias_slopeScale_clamp_), depthBias, slopeScale, clamp); | |
| 235 | +} | |
| 236 | + | |
| 237 | +_MTL_INLINE void MTL4::RenderCommandEncoder::setDepthClipMode(MTL::DepthClipMode depthClipMode) | |
| 238 | +{ | |
| 239 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setDepthClipMode_), depthClipMode); | |
| 240 | +} | |
| 241 | + | |
| 242 | +_MTL_INLINE void MTL4::RenderCommandEncoder::setDepthStencilState(const MTL::DepthStencilState* depthStencilState) | |
| 243 | +{ | |
| 244 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setDepthStencilState_), depthStencilState); | |
| 245 | +} | |
| 246 | + | |
| 247 | +_MTL_INLINE void MTL4::RenderCommandEncoder::setDepthStoreAction(MTL::StoreAction storeAction) | |
| 248 | +{ | |
| 249 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setDepthStoreAction_), storeAction); | |
| 250 | +} | |
| 251 | + | |
| 252 | +_MTL_INLINE void MTL4::RenderCommandEncoder::setDepthTestBounds(float minBound, float maxBound) | |
| 253 | +{ | |
| 254 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setDepthTestMinBound_maxBound_), minBound, maxBound); | |
| 255 | +} | |
| 256 | + | |
| 257 | +_MTL_INLINE void MTL4::RenderCommandEncoder::setFrontFacingWinding(MTL::Winding frontFacingWinding) | |
| 258 | +{ | |
| 259 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setFrontFacingWinding_), frontFacingWinding); | |
| 260 | +} | |
| 261 | + | |
| 262 | +_MTL_INLINE void MTL4::RenderCommandEncoder::setObjectThreadgroupMemoryLength(NS::UInteger length, NS::UInteger index) | |
| 263 | +{ | |
| 264 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setObjectThreadgroupMemoryLength_atIndex_), length, index); | |
| 265 | +} | |
| 266 | + | |
| 267 | +_MTL_INLINE void MTL4::RenderCommandEncoder::setRenderPipelineState(const MTL::RenderPipelineState* pipelineState) | |
| 268 | +{ | |
| 269 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setRenderPipelineState_), pipelineState); | |
| 270 | +} | |
| 271 | + | |
| 272 | +_MTL_INLINE void MTL4::RenderCommandEncoder::setScissorRect(MTL::ScissorRect rect) | |
| 273 | +{ | |
| 274 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setScissorRect_), rect); | |
| 275 | +} | |
| 276 | + | |
| 277 | +_MTL_INLINE void MTL4::RenderCommandEncoder::setScissorRects(const MTL::ScissorRect* scissorRects, NS::UInteger count) | |
| 278 | +{ | |
| 279 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setScissorRects_count_), scissorRects, count); | |
| 280 | +} | |
| 281 | + | |
| 282 | +_MTL_INLINE void MTL4::RenderCommandEncoder::setStencilReferenceValue(uint32_t referenceValue) | |
| 283 | +{ | |
| 284 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setStencilReferenceValue_), referenceValue); | |
| 285 | +} | |
| 286 | + | |
| 287 | +_MTL_INLINE void MTL4::RenderCommandEncoder::setStencilReferenceValues(uint32_t frontReferenceValue, uint32_t backReferenceValue) | |
| 288 | +{ | |
| 289 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setStencilFrontReferenceValue_backReferenceValue_), frontReferenceValue, backReferenceValue); | |
| 290 | +} | |
| 291 | + | |
| 292 | +_MTL_INLINE void MTL4::RenderCommandEncoder::setStencilStoreAction(MTL::StoreAction storeAction) | |
| 293 | +{ | |
| 294 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setStencilStoreAction_), storeAction); | |
| 295 | +} | |
| 296 | + | |
| 297 | +_MTL_INLINE void MTL4::RenderCommandEncoder::setThreadgroupMemoryLength(NS::UInteger length, NS::UInteger offset, NS::UInteger index) | |
| 298 | +{ | |
| 299 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setThreadgroupMemoryLength_offset_atIndex_), length, offset, index); | |
| 300 | +} | |
| 301 | + | |
| 302 | +_MTL_INLINE void MTL4::RenderCommandEncoder::setTriangleFillMode(MTL::TriangleFillMode fillMode) | |
| 303 | +{ | |
| 304 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setTriangleFillMode_), fillMode); | |
| 305 | +} | |
| 306 | + | |
| 307 | +_MTL_INLINE void MTL4::RenderCommandEncoder::setVertexAmplificationCount(NS::UInteger count, const MTL::VertexAmplificationViewMapping* viewMappings) | |
| 308 | +{ | |
| 309 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setVertexAmplificationCount_viewMappings_), count, viewMappings); | |
| 310 | +} | |
| 311 | + | |
| 312 | +_MTL_INLINE void MTL4::RenderCommandEncoder::setViewport(MTL::Viewport viewport) | |
| 313 | +{ | |
| 314 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setViewport_), viewport); | |
| 315 | +} | |
| 316 | + | |
| 317 | +_MTL_INLINE void MTL4::RenderCommandEncoder::setViewports(const MTL::Viewport* viewports, NS::UInteger count) | |
| 318 | +{ | |
| 319 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setViewports_count_), viewports, count); | |
| 320 | +} | |
| 321 | + | |
| 322 | +_MTL_INLINE void MTL4::RenderCommandEncoder::setVisibilityResultMode(MTL::VisibilityResultMode mode, NS::UInteger offset) | |
| 323 | +{ | |
| 324 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setVisibilityResultMode_offset_), mode, offset); | |
| 325 | +} | |
| 326 | + | |
| 327 | +_MTL_INLINE NS::UInteger MTL4::RenderCommandEncoder::tileHeight() const | |
| 328 | +{ | |
| 329 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(tileHeight)); | |
| 330 | +} | |
| 331 | + | |
| 332 | +_MTL_INLINE NS::UInteger MTL4::RenderCommandEncoder::tileWidth() const | |
| 333 | +{ | |
| 334 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(tileWidth)); | |
| 335 | +} | |
| 336 | + | |
| 337 | +_MTL_INLINE void MTL4::RenderCommandEncoder::writeTimestamp(MTL4::TimestampGranularity granularity, MTL::RenderStages stage, const MTL4::CounterHeap* counterHeap, NS::UInteger index) | |
| 338 | +{ | |
| 339 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(writeTimestampWithGranularity_afterStage_intoHeap_atIndex_), granularity, stage, counterHeap, index); | |
| 340 | +} | |
added
third_party/metal-cpp/Metal/MTL4RenderPass.hpp
+280 −0
@@ -0,0 +1,280 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Metal/MTL4RenderPass.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2025 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include "../Foundation/Foundation.hpp" | |
| 24 | +#include "MTLDefines.hpp" | |
| 25 | +#include "MTLHeaderBridge.hpp" | |
| 26 | +#include "MTLPrivate.hpp" | |
| 27 | +#include "MTLRenderPass.hpp" | |
| 28 | + | |
| 29 | +namespace MTL4 | |
| 30 | +{ | |
| 31 | +class RenderPassDescriptor; | |
| 32 | +} | |
| 33 | + | |
| 34 | +namespace MTL | |
| 35 | +{ | |
| 36 | +class Buffer; | |
| 37 | +class RasterizationRateMap; | |
| 38 | +class RenderPassColorAttachmentDescriptorArray; | |
| 39 | +class RenderPassDepthAttachmentDescriptor; | |
| 40 | +class RenderPassStencilAttachmentDescriptor; | |
| 41 | +struct SamplePosition; | |
| 42 | +} | |
| 43 | + | |
| 44 | +namespace MTL4 | |
| 45 | +{ | |
| 46 | +class RenderPassDescriptor : public NS::Copying<RenderPassDescriptor> | |
| 47 | +{ | |
| 48 | +public: | |
| 49 | + static RenderPassDescriptor* alloc(); | |
| 50 | + | |
| 51 | + MTL::RenderPassColorAttachmentDescriptorArray* colorAttachments() const; | |
| 52 | + | |
| 53 | + NS::UInteger defaultRasterSampleCount() const; | |
| 54 | + | |
| 55 | + MTL::RenderPassDepthAttachmentDescriptor* depthAttachment() const; | |
| 56 | + | |
| 57 | + NS::UInteger getSamplePositions(MTL::SamplePosition* positions, NS::UInteger count); | |
| 58 | + | |
| 59 | + NS::UInteger imageblockSampleLength() const; | |
| 60 | + | |
| 61 | + RenderPassDescriptor* init(); | |
| 62 | + | |
| 63 | + MTL::RasterizationRateMap* rasterizationRateMap() const; | |
| 64 | + | |
| 65 | + NS::UInteger renderTargetArrayLength() const; | |
| 66 | + | |
| 67 | + NS::UInteger renderTargetHeight() const; | |
| 68 | + | |
| 69 | + NS::UInteger renderTargetWidth() const; | |
| 70 | + | |
| 71 | + void setDefaultRasterSampleCount(NS::UInteger defaultRasterSampleCount); | |
| 72 | + | |
| 73 | + void setDepthAttachment(const MTL::RenderPassDepthAttachmentDescriptor* depthAttachment); | |
| 74 | + | |
| 75 | + void setImageblockSampleLength(NS::UInteger imageblockSampleLength); | |
| 76 | + | |
| 77 | + void setRasterizationRateMap(const MTL::RasterizationRateMap* rasterizationRateMap); | |
| 78 | + | |
| 79 | + void setRenderTargetArrayLength(NS::UInteger renderTargetArrayLength); | |
| 80 | + | |
| 81 | + void setRenderTargetHeight(NS::UInteger renderTargetHeight); | |
| 82 | + | |
| 83 | + void setRenderTargetWidth(NS::UInteger renderTargetWidth); | |
| 84 | + | |
| 85 | + void setSamplePositions(const MTL::SamplePosition* positions, NS::UInteger count); | |
| 86 | + | |
| 87 | + void setStencilAttachment(const MTL::RenderPassStencilAttachmentDescriptor* stencilAttachment); | |
| 88 | + | |
| 89 | + void setSupportColorAttachmentMapping(bool supportColorAttachmentMapping); | |
| 90 | + | |
| 91 | + void setThreadgroupMemoryLength(NS::UInteger threadgroupMemoryLength); | |
| 92 | + | |
| 93 | + void setTileHeight(NS::UInteger tileHeight); | |
| 94 | + | |
| 95 | + void setTileWidth(NS::UInteger tileWidth); | |
| 96 | + | |
| 97 | + void setVisibilityResultBuffer(const MTL::Buffer* visibilityResultBuffer); | |
| 98 | + | |
| 99 | + void setVisibilityResultType(MTL::VisibilityResultType visibilityResultType); | |
| 100 | + | |
| 101 | + MTL::RenderPassStencilAttachmentDescriptor* stencilAttachment() const; | |
| 102 | + | |
| 103 | + bool supportColorAttachmentMapping() const; | |
| 104 | + | |
| 105 | + NS::UInteger threadgroupMemoryLength() const; | |
| 106 | + | |
| 107 | + NS::UInteger tileHeight() const; | |
| 108 | + | |
| 109 | + NS::UInteger tileWidth() const; | |
| 110 | + | |
| 111 | + MTL::Buffer* visibilityResultBuffer() const; | |
| 112 | + | |
| 113 | + MTL::VisibilityResultType visibilityResultType() const; | |
| 114 | +}; | |
| 115 | + | |
| 116 | +} | |
| 117 | +_MTL_INLINE MTL4::RenderPassDescriptor* MTL4::RenderPassDescriptor::alloc() | |
| 118 | +{ | |
| 119 | + return NS::Object::alloc<MTL4::RenderPassDescriptor>(_MTL_PRIVATE_CLS(MTL4RenderPassDescriptor)); | |
| 120 | +} | |
| 121 | + | |
| 122 | +_MTL_INLINE MTL::RenderPassColorAttachmentDescriptorArray* MTL4::RenderPassDescriptor::colorAttachments() const | |
| 123 | +{ | |
| 124 | + return Object::sendMessage<MTL::RenderPassColorAttachmentDescriptorArray*>(this, _MTL_PRIVATE_SEL(colorAttachments)); | |
| 125 | +} | |
| 126 | + | |
| 127 | +_MTL_INLINE NS::UInteger MTL4::RenderPassDescriptor::defaultRasterSampleCount() const | |
| 128 | +{ | |
| 129 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(defaultRasterSampleCount)); | |
| 130 | +} | |
| 131 | + | |
| 132 | +_MTL_INLINE MTL::RenderPassDepthAttachmentDescriptor* MTL4::RenderPassDescriptor::depthAttachment() const | |
| 133 | +{ | |
| 134 | + return Object::sendMessage<MTL::RenderPassDepthAttachmentDescriptor*>(this, _MTL_PRIVATE_SEL(depthAttachment)); | |
| 135 | +} | |
| 136 | + | |
| 137 | +_MTL_INLINE NS::UInteger MTL4::RenderPassDescriptor::getSamplePositions(MTL::SamplePosition* positions, NS::UInteger count) | |
| 138 | +{ | |
| 139 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(getSamplePositions_count_), positions, count); | |
| 140 | +} | |
| 141 | + | |
| 142 | +_MTL_INLINE NS::UInteger MTL4::RenderPassDescriptor::imageblockSampleLength() const | |
| 143 | +{ | |
| 144 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(imageblockSampleLength)); | |
| 145 | +} | |
| 146 | + | |
| 147 | +_MTL_INLINE MTL4::RenderPassDescriptor* MTL4::RenderPassDescriptor::init() | |
| 148 | +{ | |
| 149 | + return NS::Object::init<MTL4::RenderPassDescriptor>(); | |
| 150 | +} | |
| 151 | + | |
| 152 | +_MTL_INLINE MTL::RasterizationRateMap* MTL4::RenderPassDescriptor::rasterizationRateMap() const | |
| 153 | +{ | |
| 154 | + return Object::sendMessage<MTL::RasterizationRateMap*>(this, _MTL_PRIVATE_SEL(rasterizationRateMap)); | |
| 155 | +} | |
| 156 | + | |
| 157 | +_MTL_INLINE NS::UInteger MTL4::RenderPassDescriptor::renderTargetArrayLength() const | |
| 158 | +{ | |
| 159 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(renderTargetArrayLength)); | |
| 160 | +} | |
| 161 | + | |
| 162 | +_MTL_INLINE NS::UInteger MTL4::RenderPassDescriptor::renderTargetHeight() const | |
| 163 | +{ | |
| 164 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(renderTargetHeight)); | |
| 165 | +} | |
| 166 | + | |
| 167 | +_MTL_INLINE NS::UInteger MTL4::RenderPassDescriptor::renderTargetWidth() const | |
| 168 | +{ | |
| 169 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(renderTargetWidth)); | |
| 170 | +} | |
| 171 | + | |
| 172 | +_MTL_INLINE void MTL4::RenderPassDescriptor::setDefaultRasterSampleCount(NS::UInteger defaultRasterSampleCount) | |
| 173 | +{ | |
| 174 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setDefaultRasterSampleCount_), defaultRasterSampleCount); | |
| 175 | +} | |
| 176 | + | |
| 177 | +_MTL_INLINE void MTL4::RenderPassDescriptor::setDepthAttachment(const MTL::RenderPassDepthAttachmentDescriptor* depthAttachment) | |
| 178 | +{ | |
| 179 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setDepthAttachment_), depthAttachment); | |
| 180 | +} | |
| 181 | + | |
| 182 | +_MTL_INLINE void MTL4::RenderPassDescriptor::setImageblockSampleLength(NS::UInteger imageblockSampleLength) | |
| 183 | +{ | |
| 184 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setImageblockSampleLength_), imageblockSampleLength); | |
| 185 | +} | |
| 186 | + | |
| 187 | +_MTL_INLINE void MTL4::RenderPassDescriptor::setRasterizationRateMap(const MTL::RasterizationRateMap* rasterizationRateMap) | |
| 188 | +{ | |
| 189 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setRasterizationRateMap_), rasterizationRateMap); | |
| 190 | +} | |
| 191 | + | |
| 192 | +_MTL_INLINE void MTL4::RenderPassDescriptor::setRenderTargetArrayLength(NS::UInteger renderTargetArrayLength) | |
| 193 | +{ | |
| 194 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setRenderTargetArrayLength_), renderTargetArrayLength); | |
| 195 | +} | |
| 196 | + | |
| 197 | +_MTL_INLINE void MTL4::RenderPassDescriptor::setRenderTargetHeight(NS::UInteger renderTargetHeight) | |
| 198 | +{ | |
| 199 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setRenderTargetHeight_), renderTargetHeight); | |
| 200 | +} | |
| 201 | + | |
| 202 | +_MTL_INLINE void MTL4::RenderPassDescriptor::setRenderTargetWidth(NS::UInteger renderTargetWidth) | |
| 203 | +{ | |
| 204 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setRenderTargetWidth_), renderTargetWidth); | |
| 205 | +} | |
| 206 | + | |
| 207 | +_MTL_INLINE void MTL4::RenderPassDescriptor::setSamplePositions(const MTL::SamplePosition* positions, NS::UInteger count) | |
| 208 | +{ | |
| 209 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setSamplePositions_count_), positions, count); | |
| 210 | +} | |
| 211 | + | |
| 212 | +_MTL_INLINE void MTL4::RenderPassDescriptor::setStencilAttachment(const MTL::RenderPassStencilAttachmentDescriptor* stencilAttachment) | |
| 213 | +{ | |
| 214 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setStencilAttachment_), stencilAttachment); | |
| 215 | +} | |
| 216 | + | |
| 217 | +_MTL_INLINE void MTL4::RenderPassDescriptor::setSupportColorAttachmentMapping(bool supportColorAttachmentMapping) | |
| 218 | +{ | |
| 219 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setSupportColorAttachmentMapping_), supportColorAttachmentMapping); | |
| 220 | +} | |
| 221 | + | |
| 222 | +_MTL_INLINE void MTL4::RenderPassDescriptor::setThreadgroupMemoryLength(NS::UInteger threadgroupMemoryLength) | |
| 223 | +{ | |
| 224 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setThreadgroupMemoryLength_), threadgroupMemoryLength); | |
| 225 | +} | |
| 226 | + | |
| 227 | +_MTL_INLINE void MTL4::RenderPassDescriptor::setTileHeight(NS::UInteger tileHeight) | |
| 228 | +{ | |
| 229 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setTileHeight_), tileHeight); | |
| 230 | +} | |
| 231 | + | |
| 232 | +_MTL_INLINE void MTL4::RenderPassDescriptor::setTileWidth(NS::UInteger tileWidth) | |
| 233 | +{ | |
| 234 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setTileWidth_), tileWidth); | |
| 235 | +} | |
| 236 | + | |
| 237 | +_MTL_INLINE void MTL4::RenderPassDescriptor::setVisibilityResultBuffer(const MTL::Buffer* visibilityResultBuffer) | |
| 238 | +{ | |
| 239 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setVisibilityResultBuffer_), visibilityResultBuffer); | |
| 240 | +} | |
| 241 | + | |
| 242 | +_MTL_INLINE void MTL4::RenderPassDescriptor::setVisibilityResultType(MTL::VisibilityResultType visibilityResultType) | |
| 243 | +{ | |
| 244 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setVisibilityResultType_), visibilityResultType); | |
| 245 | +} | |
| 246 | + | |
| 247 | +_MTL_INLINE MTL::RenderPassStencilAttachmentDescriptor* MTL4::RenderPassDescriptor::stencilAttachment() const | |
| 248 | +{ | |
| 249 | + return Object::sendMessage<MTL::RenderPassStencilAttachmentDescriptor*>(this, _MTL_PRIVATE_SEL(stencilAttachment)); | |
| 250 | +} | |
| 251 | + | |
| 252 | +_MTL_INLINE bool MTL4::RenderPassDescriptor::supportColorAttachmentMapping() const | |
| 253 | +{ | |
| 254 | + return Object::sendMessageSafe<bool>(this, _MTL_PRIVATE_SEL(supportColorAttachmentMapping)); | |
| 255 | +} | |
| 256 | + | |
| 257 | +_MTL_INLINE NS::UInteger MTL4::RenderPassDescriptor::threadgroupMemoryLength() const | |
| 258 | +{ | |
| 259 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(threadgroupMemoryLength)); | |
| 260 | +} | |
| 261 | + | |
| 262 | +_MTL_INLINE NS::UInteger MTL4::RenderPassDescriptor::tileHeight() const | |
| 263 | +{ | |
| 264 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(tileHeight)); | |
| 265 | +} | |
| 266 | + | |
| 267 | +_MTL_INLINE NS::UInteger MTL4::RenderPassDescriptor::tileWidth() const | |
| 268 | +{ | |
| 269 | + return Object::sendMessage<NS::UInteger>(this, _MTL_PRIVATE_SEL(tileWidth)); | |
| 270 | +} | |
| 271 | + | |
| 272 | +_MTL_INLINE MTL::Buffer* MTL4::RenderPassDescriptor::visibilityResultBuffer() const | |
| 273 | +{ | |
| 274 | + return Object::sendMessage<MTL::Buffer*>(this, _MTL_PRIVATE_SEL(visibilityResultBuffer)); | |
| 275 | +} | |
| 276 | + | |
| 277 | +_MTL_INLINE MTL::VisibilityResultType MTL4::RenderPassDescriptor::visibilityResultType() const | |
| 278 | +{ | |
| 279 | + return Object::sendMessage<MTL::VisibilityResultType>(this, _MTL_PRIVATE_SEL(visibilityResultType)); | |
| 280 | +} | |
added
third_party/metal-cpp/Metal/MTL4RenderPipeline.hpp
+326 −0
@@ -0,0 +1,587 @@ | ||
| 1 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 2 | +// | |
| 3 | +// Metal/MTL4RenderPipeline.hpp | |
| 4 | +// | |
| 5 | +// Copyright 2020-2025 Apple Inc. | |
| 6 | +// | |
| 7 | +// Licensed under the Apache License, Version 2.0 (the "License"); | |
| 8 | +// you may not use this file except in compliance with the License. | |
| 9 | +// You may obtain a copy of the License at | |
| 10 | +// | |
| 11 | +// http://www.apache.org/licenses/LICENSE-2.0 | |
| 12 | +// | |
| 13 | +// Unless required by applicable law or agreed to in writing, software | |
| 14 | +// distributed under the License is distributed on an "AS IS" BASIS, | |
| 15 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 16 | +// See the License for the specific language governing permissions and | |
| 17 | +// limitations under the License. | |
| 18 | +// | |
| 19 | +//------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 20 | + | |
| 21 | +#pragma once | |
| 22 | + | |
| 23 | +#include "../Foundation/Foundation.hpp" | |
| 24 | +#include "MTL4PipelineState.hpp" | |
| 25 | +#include "MTLDefines.hpp" | |
| 26 | +#include "MTLHeaderBridge.hpp" | |
| 27 | +#include "MTLPixelFormat.hpp" | |
| 28 | +#include "MTLPrivate.hpp" | |
| 29 | +#include "MTLRenderPipeline.hpp" | |
| 30 | + | |
| 31 | +namespace MTL4 | |
| 32 | +{ | |
| 33 | +class FunctionDescriptor; | |
| 34 | +class RenderPipelineBinaryFunctionsDescriptor; | |
| 35 | +class RenderPipelineColorAttachmentDescriptor; | |
| 36 | +class RenderPipelineColorAttachmentDescriptorArray; | |
| 37 | +class RenderPipelineDescriptor; | |
| 38 | +class StaticLinkingDescriptor; | |
| 39 | +} | |
| 40 | + | |
| 41 | +namespace MTL | |
| 42 | +{ | |
| 43 | +class VertexDescriptor; | |
| 44 | +} | |
| 45 | + | |
| 46 | +namespace MTL4 | |
| 47 | +{ | |
| 48 | +_MTL_ENUM(NS::Integer, LogicalToPhysicalColorAttachmentMappingState) { | |
| 49 | + LogicalToPhysicalColorAttachmentMappingStateIdentity = 0, | |
| 50 | + LogicalToPhysicalColorAttachmentMappingStateInherited = 1, | |
| 51 | +}; | |
| 52 | + | |
| 53 | +class RenderPipelineColorAttachmentDescriptor : public NS::Copying<RenderPipelineColorAttachmentDescriptor> | |
| 54 | +{ | |
| 55 | +public: | |
| 56 | + static RenderPipelineColorAttachmentDescriptor* alloc(); | |
| 57 | + | |
| 58 | + MTL::BlendOperation alphaBlendOperation() const; | |
| 59 | + | |
| 60 | + BlendState blendingState() const; | |
| 61 | + | |
| 62 | + MTL::BlendFactor destinationAlphaBlendFactor() const; | |
| 63 | + | |
| 64 | + MTL::BlendFactor destinationRGBBlendFactor() const; | |
| 65 | + | |
| 66 | + RenderPipelineColorAttachmentDescriptor* init(); | |
| 67 | + | |
| 68 | + MTL::PixelFormat pixelFormat() const; | |
| 69 | + | |
| 70 | + void reset(); | |
| 71 | + | |
| 72 | + MTL::BlendOperation rgbBlendOperation() const; | |
| 73 | + | |
| 74 | + void setAlphaBlendOperation(MTL::BlendOperation alphaBlendOperation); | |
| 75 | + | |
| 76 | + void setBlendingState(MTL4::BlendState blendingState); | |
| 77 | + | |
| 78 | + void setDestinationAlphaBlendFactor(MTL::BlendFactor destinationAlphaBlendFactor); | |
| 79 | + | |
| 80 | + void setDestinationRGBBlendFactor(MTL::BlendFactor destinationRGBBlendFactor); | |
| 81 | + | |
| 82 | + void setPixelFormat(MTL::PixelFormat pixelFormat); | |
| 83 | + | |
| 84 | + void setRgbBlendOperation(MTL::BlendOperation rgbBlendOperation); | |
| 85 | + | |
| 86 | + void setSourceAlphaBlendFactor(MTL::BlendFactor sourceAlphaBlendFactor); | |
| 87 | + | |
| 88 | + void setSourceRGBBlendFactor(MTL::BlendFactor sourceRGBBlendFactor); | |
| 89 | + | |
| 90 | + void setWriteMask(MTL::ColorWriteMask writeMask); | |
| 91 | + | |
| 92 | + MTL::BlendFactor sourceAlphaBlendFactor() const; | |
| 93 | + | |
| 94 | + MTL::BlendFactor sourceRGBBlendFactor() const; | |
| 95 | + | |
| 96 | + MTL::ColorWriteMask writeMask() const; | |
| 97 | +}; | |
| 98 | + | |
| 99 | +class RenderPipelineColorAttachmentDescriptorArray : public NS::Copying<RenderPipelineColorAttachmentDescriptorArray> | |
| 100 | +{ | |
| 101 | +public: | |
| 102 | + static RenderPipelineColorAttachmentDescriptorArray* alloc(); | |
| 103 | + | |
| 104 | + RenderPipelineColorAttachmentDescriptorArray* init(); | |
| 105 | + | |
| 106 | + RenderPipelineColorAttachmentDescriptor* object(NS::UInteger attachmentIndex); | |
| 107 | + | |
| 108 | + void reset(); | |
| 109 | + | |
| 110 | + void setObject(const MTL4::RenderPipelineColorAttachmentDescriptor* attachment, NS::UInteger attachmentIndex); | |
| 111 | +}; | |
| 112 | + | |
| 113 | +class RenderPipelineBinaryFunctionsDescriptor : public NS::Copying<RenderPipelineBinaryFunctionsDescriptor> | |
| 114 | +{ | |
| 115 | +public: | |
| 116 | + static RenderPipelineBinaryFunctionsDescriptor* alloc(); | |
| 117 | + | |
| 118 | + NS::Array* fragmentAdditionalBinaryFunctions() const; | |
| 119 | + | |
| 120 | + RenderPipelineBinaryFunctionsDescriptor* init(); | |
| 121 | + | |
| 122 | + NS::Array* meshAdditionalBinaryFunctions() const; | |
| 123 | + | |
| 124 | + NS::Array* objectAdditionalBinaryFunctions() const; | |
| 125 | + | |
| 126 | + void reset(); | |
| 127 | + | |
| 128 | + void setFragmentAdditionalBinaryFunctions(const NS::Array* fragmentAdditionalBinaryFunctions); | |
| 129 | + | |
| 130 | + void setMeshAdditionalBinaryFunctions(const NS::Array* meshAdditionalBinaryFunctions); | |
| 131 | + | |
| 132 | + void setObjectAdditionalBinaryFunctions(const NS::Array* objectAdditionalBinaryFunctions); | |
| 133 | + | |
| 134 | + void setTileAdditionalBinaryFunctions(const NS::Array* tileAdditionalBinaryFunctions); | |
| 135 | + | |
| 136 | + void setVertexAdditionalBinaryFunctions(const NS::Array* vertexAdditionalBinaryFunctions); | |
| 137 | + | |
| 138 | + NS::Array* tileAdditionalBinaryFunctions() const; | |
| 139 | + | |
| 140 | + NS::Array* vertexAdditionalBinaryFunctions() const; | |
| 141 | +}; | |
| 142 | + | |
| 143 | +class RenderPipelineDescriptor : public NS::Copying<RenderPipelineDescriptor, PipelineDescriptor> | |
| 144 | +{ | |
| 145 | +public: | |
| 146 | + static RenderPipelineDescriptor* alloc(); | |
| 147 | + | |
| 148 | + AlphaToCoverageState alphaToCoverageState() const; | |
| 149 | + | |
| 150 | + AlphaToOneState alphaToOneState() const; | |
| 151 | + | |
| 152 | + LogicalToPhysicalColorAttachmentMappingState colorAttachmentMappingState() const; | |
| 153 | + | |
| 154 | + RenderPipelineColorAttachmentDescriptorArray* colorAttachments() const; | |
| 155 | + | |
| 156 | + FunctionDescriptor* fragmentFunctionDescriptor() const; | |
| 157 | + | |
| 158 | + StaticLinkingDescriptor* fragmentStaticLinkingDescriptor() const; | |
| 159 | + | |
| 160 | + RenderPipelineDescriptor* init(); | |
| 161 | + | |
| 162 | + MTL::PrimitiveTopologyClass inputPrimitiveTopology() const; | |
| 163 | + | |
| 164 | + bool isRasterizationEnabled() const; | |
| 165 | + | |
| 166 | + NS::UInteger maxVertexAmplificationCount() const; | |
| 167 | + | |
| 168 | + NS::UInteger rasterSampleCount() const; | |
| 169 | + | |
| 170 | + [[deprecated("please use isRasterizationEnabled instead")]] | |
| 171 | + bool rasterizationEnabled() const; | |
| 172 | + | |
| 173 | + void reset(); | |
| 174 | + | |
| 175 | + void setAlphaToCoverageState(MTL4::AlphaToCoverageState alphaToCoverageState); | |
| 176 | + | |
| 177 | + void setAlphaToOneState(MTL4::AlphaToOneState alphaToOneState); | |
| 178 | + | |
| 179 | + void setColorAttachmentMappingState(MTL4::LogicalToPhysicalColorAttachmentMappingState colorAttachmentMappingState); | |
| 180 | + | |
| 181 | + void setFragmentFunctionDescriptor(const MTL4::FunctionDescriptor* fragmentFunctionDescriptor); | |
| 182 | + | |
| 183 | + void setFragmentStaticLinkingDescriptor(const MTL4::StaticLinkingDescriptor* fragmentStaticLinkingDescriptor); | |
| 184 | + | |
| 185 | + void setInputPrimitiveTopology(MTL::PrimitiveTopologyClass inputPrimitiveTopology); | |
| 186 | + | |
| 187 | + void setMaxVertexAmplificationCount(NS::UInteger maxVertexAmplificationCount); | |
| 188 | + | |
| 189 | + void setRasterSampleCount(NS::UInteger rasterSampleCount); | |
| 190 | + | |
| 191 | + void setRasterizationEnabled(bool rasterizationEnabled); | |
| 192 | + | |
| 193 | + void setSupportFragmentBinaryLinking(bool supportFragmentBinaryLinking); | |
| 194 | + | |
| 195 | + void setSupportIndirectCommandBuffers(MTL4::IndirectCommandBufferSupportState supportIndirectCommandBuffers); | |
| 196 | + | |
| 197 | + void setSupportVertexBinaryLinking(bool supportVertexBinaryLinking); | |
| 198 | + | |
| 199 | + void setVertexDescriptor(const MTL::VertexDescriptor* vertexDescriptor); | |
| 200 | + | |
| 201 | + void setVertexFunctionDescriptor(const MTL4::FunctionDescriptor* vertexFunctionDescriptor); | |
| 202 | + | |
| 203 | + void setVertexStaticLinkingDescriptor(const MTL4::StaticLinkingDescriptor* vertexStaticLinkingDescriptor); | |
| 204 | + | |
| 205 | + bool supportFragmentBinaryLinking() const; | |
| 206 | + | |
| 207 | + IndirectCommandBufferSupportState supportIndirectCommandBuffers() const; | |
| 208 | + | |
| 209 | + bool supportVertexBinaryLinking() const; | |
| 210 | + | |
| 211 | + MTL::VertexDescriptor* vertexDescriptor() const; | |
| 212 | + | |
| 213 | + FunctionDescriptor* vertexFunctionDescriptor() const; | |
| 214 | + | |
| 215 | + StaticLinkingDescriptor* vertexStaticLinkingDescriptor() const; | |
| 216 | +}; | |
| 217 | + | |
| 218 | +} | |
| 219 | +_MTL_INLINE MTL4::RenderPipelineColorAttachmentDescriptor* MTL4::RenderPipelineColorAttachmentDescriptor::alloc() | |
| 220 | +{ | |
| 221 | + return NS::Object::alloc<MTL4::RenderPipelineColorAttachmentDescriptor>(_MTL_PRIVATE_CLS(MTL4RenderPipelineColorAttachmentDescriptor)); | |
| 222 | +} | |
| 223 | + | |
| 224 | +_MTL_INLINE MTL::BlendOperation MTL4::RenderPipelineColorAttachmentDescriptor::alphaBlendOperation() const | |
| 225 | +{ | |
| 226 | + return Object::sendMessage<MTL::BlendOperation>(this, _MTL_PRIVATE_SEL(alphaBlendOperation)); | |
| 227 | +} | |
| 228 | + | |
| 229 | +_MTL_INLINE MTL4::BlendState MTL4::RenderPipelineColorAttachmentDescriptor::blendingState() const | |
| 230 | +{ | |
| 231 | + return Object::sendMessage<MTL4::BlendState>(this, _MTL_PRIVATE_SEL(blendingState)); | |
| 232 | +} | |
| 233 | + | |
| 234 | +_MTL_INLINE MTL::BlendFactor MTL4::RenderPipelineColorAttachmentDescriptor::destinationAlphaBlendFactor() const | |
| 235 | +{ | |
| 236 | + return Object::sendMessage<MTL::BlendFactor>(this, _MTL_PRIVATE_SEL(destinationAlphaBlendFactor)); | |
| 237 | +} | |
| 238 | + | |
| 239 | +_MTL_INLINE MTL::BlendFactor MTL4::RenderPipelineColorAttachmentDescriptor::destinationRGBBlendFactor() const | |
| 240 | +{ | |
| 241 | + return Object::sendMessage<MTL::BlendFactor>(this, _MTL_PRIVATE_SEL(destinationRGBBlendFactor)); | |
| 242 | +} | |
| 243 | + | |
| 244 | +_MTL_INLINE MTL4::RenderPipelineColorAttachmentDescriptor* MTL4::RenderPipelineColorAttachmentDescriptor::init() | |
| 245 | +{ | |
| 246 | + return NS::Object::init<MTL4::RenderPipelineColorAttachmentDescriptor>(); | |
| 247 | +} | |
| 248 | + | |
| 249 | +_MTL_INLINE MTL::PixelFormat MTL4::RenderPipelineColorAttachmentDescriptor::pixelFormat() const | |
| 250 | +{ | |
| 251 | + return Object::sendMessage<MTL::PixelFormat>(this, _MTL_PRIVATE_SEL(pixelFormat)); | |
| 252 | +} | |
| 253 | + | |
| 254 | +_MTL_INLINE void MTL4::RenderPipelineColorAttachmentDescriptor::reset() | |
| 255 | +{ | |
| 256 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(reset)); | |
| 257 | +} | |
| 258 | + | |
| 259 | +_MTL_INLINE MTL::BlendOperation MTL4::RenderPipelineColorAttachmentDescriptor::rgbBlendOperation() const | |
| 260 | +{ | |
| 261 | + return Object::sendMessage<MTL::BlendOperation>(this, _MTL_PRIVATE_SEL(rgbBlendOperation)); | |
| 262 | +} | |
| 263 | + | |
| 264 | +_MTL_INLINE void MTL4::RenderPipelineColorAttachmentDescriptor::setAlphaBlendOperation(MTL::BlendOperation alphaBlendOperation) | |
| 265 | +{ | |
| 266 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setAlphaBlendOperation_), alphaBlendOperation); | |
| 267 | +} | |
| 268 | + | |
| 269 | +_MTL_INLINE void MTL4::RenderPipelineColorAttachmentDescriptor::setBlendingState(MTL4::BlendState blendingState) | |
| 270 | +{ | |
| 271 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setBlendingState_), blendingState); | |
| 272 | +} | |
| 273 | + | |
| 274 | +_MTL_INLINE void MTL4::RenderPipelineColorAttachmentDescriptor::setDestinationAlphaBlendFactor(MTL::BlendFactor destinationAlphaBlendFactor) | |
| 275 | +{ | |
| 276 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setDestinationAlphaBlendFactor_), destinationAlphaBlendFactor); | |
| 277 | +} | |
| 278 | + | |
| 279 | +_MTL_INLINE void MTL4::RenderPipelineColorAttachmentDescriptor::setDestinationRGBBlendFactor(MTL::BlendFactor destinationRGBBlendFactor) | |
| 280 | +{ | |
| 281 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setDestinationRGBBlendFactor_), destinationRGBBlendFactor); | |
| 282 | +} | |
| 283 | + | |
| 284 | +_MTL_INLINE void MTL4::RenderPipelineColorAttachmentDescriptor::setPixelFormat(MTL::PixelFormat pixelFormat) | |
| 285 | +{ | |
| 286 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setPixelFormat_), pixelFormat); | |
| 287 | +} | |
| 288 | + | |
| 289 | +_MTL_INLINE void MTL4::RenderPipelineColorAttachmentDescriptor::setRgbBlendOperation(MTL::BlendOperation rgbBlendOperation) | |
| 290 | +{ | |
| 291 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setRgbBlendOperation_), rgbBlendOperation); | |
| 292 | +} | |
| 293 | + | |
| 294 | +_MTL_INLINE void MTL4::RenderPipelineColorAttachmentDescriptor::setSourceAlphaBlendFactor(MTL::BlendFactor sourceAlphaBlendFactor) | |
| 295 | +{ | |
| 296 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setSourceAlphaBlendFactor_), sourceAlphaBlendFactor); | |
| 297 | +} | |
| 298 | + | |
| 299 | +_MTL_INLINE void MTL4::RenderPipelineColorAttachmentDescriptor::setSourceRGBBlendFactor(MTL::BlendFactor sourceRGBBlendFactor) | |
| 300 | +{ | |
| 301 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setSourceRGBBlendFactor_), sourceRGBBlendFactor); | |
| 302 | +} | |
| 303 | + | |
| 304 | +_MTL_INLINE void MTL4::RenderPipelineColorAttachmentDescriptor::setWriteMask(MTL::ColorWriteMask writeMask) | |
| 305 | +{ | |
| 306 | + Object::sendMessage<void>(this, _MTL_PRIVATE_SEL(setWriteMask_), writeMask); | |
| 307 | +} | |
| 308 | + | |
| 309 | +_MTL_INLINE MTL::BlendFactor MTL4::RenderPipelineColorAttachmentDescriptor::sourceAlphaBlendFactor() const | |
| 310 | +{ | |
| 311 | + return Object::sendMessage<MTL::BlendFactor>(this, _MTL_PRIVATE_SEL(sourceAlphaBlendFactor)); | |
| 312 | +} | |
| 313 | + | |
| 314 | +_MTL_INLINE MTL::BlendFactor MTL4::RenderPipelineColorAttachmentDescriptor::sourceRGBBlendFactor() const | |
| 315 | +{ | |
| 316 | + return Object::sendMessage<MTL::BlendFactor>(this, _MTL_PRIVATE_SEL(sourceRGBBlendFactor)); | |
| 317 | +} | |
| 318 | + | |
| 319 | +_MTL_INLINE MTL::ColorWriteMask MTL4::RenderPipelineColorAttachmentDescriptor::writeMask() const | |
| 320 | +{ | |
| 321 | + return Object::sendMessage<MTL::ColorWriteMask>(this, _MTL_PRIVATE_SEL(writeMask)); | |
| 322 | +} | |
| 323 | + | |
| 324 | +_MTL_INLINE MTL4::RenderPipelineColorAttachmentDescriptorArray* MTL4::RenderPipelineColorAttachmentDescriptorArray::alloc() | |
| 325 | +{ | |
| 326 | + return NS::Object::alloc<MTL4::RenderPipelineColorAttachmentDescriptorArray>(_MTL_PRIVATE_CLS(MTL4RenderPipelineColorAttachmentDescriptorArray)); | |
Diff truncated — file too large.