Generated-Computers/Locals — Parallel AI agent orchestration for macOS
Run multiple AI assistants simultaneously on your Mac with significantly improved throughput via localized GPU-accelerated serving.
Performance
Practical reporting on AI systems, tools, and engineering execution.
Git repos
h2oai/h2o4gpu — GPU solvers built on scikit-learn, maintenance status unclear
H2O4GPU provides GPU-backed scikit-learn-compatible solvers in Python and R, but targets Python 3.6 and CUDA 8–10, suggesting it predates active maintenance.
awesome-llm-token-optimization — curated playbook and links for cutting LLM token costs
A curated repository of tactics, vendor docs, and papers focused on reducing LLM token spend (prompt caching, model routing, batching, compression) with vendor-specific notes you should validate before production.
scikit-learn-intelex (sklearnex): a drop‑in accelerator for scikit‑learn on CPUs and GPUs
Drop-in extension that patches scikit‑learn to use optimized CPU/GPU implementations (sklearnex); README claims an average 8.5× speedup and multi-GPU support — test on your workload before production use.
Agent Router — Envoy-based OpenAI-compatible control plane for unified LLM and tool access
Agent Router provides an OpenAI-compatible API and Envoy-powered control plane to centralize routing, credentials, quotas, and failover across hosted and self-hosted LLM providers.
Torch‑TensorRT: compile PyTorch models to TensorRT for lower inference latency on NVIDIA GPUs
Torch‑TensorRT compiles PyTorch/TorchScript/FX models to TensorRT for NVIDIA GPUs, offering quick in‑process compilation via torch.compile and an export path for ahead‑of‑time C++ deployment.
LichtFeld Studio — Integrated workstation for 3D Gaussian Splatting (train, inspect, edit, export)
A native desktop + headless application that combines training, interactive inspection, gaussian editing, Python plugins, automation, and export for 3D Gaussian Splatting workflows.
Spirula Studio — Cross‑vendor Vulkan trainer for 3D Gaussian Splatting (video → splat → mesh)
A self-contained 3D Gaussian Splatting trainer that converts photos or video into splats and textured meshes using Vulkan across NVIDIA, AMD, Intel, and Apple GPUs, with built-in SfM and viewer.
google/doubleml-pipeline — Automated Double Machine Learning + FLAML for causal marketing measurement
A Python pipeline that operationalizes Double Machine Learning with FLAML AutoML for causal effect estimation, sensitivity checks, publication plots, and priors extraction — designed for marketing ROI and incremental-impact analysis.
LMCache — a persistent, engine‑independent KV cache layer to cut LLM time‑to‑first‑token
LMCache is an engine‑independent daemon that persists and manages LLM KV caches to lower time‑to‑first‑token and improve throughput for long‑context, multi‑turn, and agentic workloads.