Buy for the workload you actually have先确定要完成的任务，再选机器

You want a private coding assistant that can read a project, answer questions about documents and stay available on your desk. Both machines promise large unified memory. Yet the purchase can fail in two quite different ways: the model fits but answers too slowly, or the hardware is capable but the required software does not run on it.你想要一个留在桌面的私人助手，能阅读项目、回答文档问题，并随时待命。两台机器都提供大容量统一内存，但买错设备可能有两种完全不同的结果：模型装得下，回答却太慢；硬件足够，工作所需的软件却跑不起来。

DGX SparkDGX Spark

Choose it for CUDA development. Linux, NVIDIA containers, custom GPU operators and a path toward NVIDIA server deployment are the reasons to pay for it. Treat 128 GB as a fixed memory envelope to validate.以 CUDA 研发为主，优先考虑它。 Linux、NVIDIA 容器、自定义 GPU 算子，以及向 NVIDIA 服务器迁移的工作流，是购买理由。128 GB 是需要验证的固定内存边界。

Mac Studio UltraMac Studio Ultra

Choose it for personal inference and capacity. If MLX or Metal supports your model, high memory bandwidth and larger memory options make it the stronger starting point for a single-user desktop assistant. It also remains your everyday Mac.以个人推理和内存容量为主，优先考虑它。 模型能在 MLX 或 Metal 上运行时，更高的内存带宽与更大的内存选项，使它更适合作为单用户桌面助手的起点，也能兼任日常电脑。

For a first 7B–14B model, try an existing computer before buying either. For a paid multi-user service, compare a dedicated GPU server as well. This is a hardware and deployment analysis, not a head-to-head benchmark: the specifications below are sourced, the budgets are calculated, and no token-rate or power measurement on either machine is claimed.如果只是初次尝试 7B–14B 模型，先用已有电脑，两台都未必需要买。如果要做面向多人的付费服务，还应比较独立 GPU 服务器。本文属于硬件与部署分析：规格来自官方，预算来自计算，未声称在两台真机上完成速度或功耗对测。

Compare the current machines, not their launch headlines先对齐代际、配置与报价

Checked on 19 October 2026: Apple’s current Ultra model is M5 Ultra. The 819 GB/s often quoted for M3 Ultra belongs to the previous generation. Here, “Spark” means NVIDIA’s 128 GB / 4 TB system; the announced 64 GB OEM version is a different configuration. Apple specifications · NVIDIA specifications.核对日期为 2026 年 10 月 19 日：Apple 当前 Ultra 机型为 M5 Ultra，网上常见的 819 GB/s 是上一代 M3 Ultra 的数据。本文的 Spark 指 NVIDIA 的 128 GB／4 TB 整机，已公布的 64 GB OEM 版本是另一种配置。Apple 官方规格 · NVIDIA 官方规格。

Item项目DGX SparkDGX SparkMac Studio · M5 UltraMac Studio · M5 Ultra
Compute platform计算平台 DGX SparkDGX SparkGB10 Grace Blackwell; 20-core Arm CPU; Blackwell GPUGB10 Grace Blackwell；20 核 Arm CPU；Blackwell GPU Mac Studio · M5 UltraMac Studio · M5 Ultra30-core CPU / 64-core GPU; optional 36 / 8030 核 CPU／64 核 GPU；可选 36／80 核
Unified memory统一内存 DGX SparkDGX Spark128 GB LPDDR5X128 GB LPDDR5X Mac Studio · M5 UltraMac Studio · M5 Ultra96 GB base; 256 GB configurable; announced 512 GB requires 36 / 80基础 96 GB；可选 256 GB；已公布的 512 GB 要求 36／80 核
Peak memory bandwidth标称内存带宽 DGX SparkDGX Spark273 GB/s273 GB/s Mac Studio · M5 UltraMac Studio · M5 Ultra1,200 GB/s1,200 GB/s
Primary local AI stack主要本地 AI 栈 DGX SparkDGX SparkDGX OS / Linux Arm64; CUDA, NVIDIA containers, vLLM, TensorRT-LLM, llama.cppDGX OS／Linux Arm64；CUDA、NVIDIA 容器、vLLM、TensorRT-LLM、llama.cpp Mac Studio · M5 UltraMac Studio · M5 UltramacOS / Apple silicon; MLX, Metal, llama.cpp; MPS where the application supports itmacOS／Apple silicon；MLX、Metal、llama.cpp；应用支持时使用 MPS
Storage in the compared offer所列配置的存储 DGX SparkDGX Spark4 TB NVMe SSD4 TB NVMe SSD Mac Studio · M5 UltraMac Studio · M5 Ultra1 TB base SSD; capacity upgrades cost extra基础 SSD 为 1 TB；扩容需另计费用
Expansion path扩展路径 DGX SparkDGX Spark10 GbE plus ConnectX-7 for supported distributed workloads10 GbE 与 ConnectX-7；分布式任务需软件配合 Mac Studio · M5 UltraMac Studio · M5 Ultra10 GbE and Thunderbolt 5; supported distributed stacks can use linked Macs10 GbE 与雷雳 5；支持的分布式软件可使用多机互联

Price anchors are US$6,950 for the NVIDIA 128 GB / 4 TB offer and US$5,499 starting for the 96 GB / 1 TB M5 Ultra Mac Studio. Apple’s China starting price is RMB 46,999. The US figures still buy different memory and storage configurations, not equal systems. Quote the exact RAM/SSD configuration, tax, shipping and local warranty together. As of this check, the 512 GB Mac option is announced for late October; do not treat it as an immediately deliverable purchase. NVIDIA store · Apple US announcement · Apple China announcement.价格参照为 NVIDIA 128 GB／4 TB 配置的 6,950 美元，以及 96 GB／1 TB M5 Ultra Mac Studio 的 5,499 美元起；Apple 中国大陆起价为 46,999 元。即使使用两个美国标价，内存与存储也并不相同，不能当成等配置报价。询价时把内存、SSD、税费、运费与本地保修放在一起核算。截至本次核对，512 GB Mac 选项计划于 10 月下旬推出，不应当成现在就能交付的现货。NVIDIA 商店 · Apple 美国发布说明 · Apple 中国发布说明。

Unified memory gives the CPU and GPU access to a common memory pool. It removes a separate small VRAM capacity barrier, but does not turn the entire installed RAM into freely allocatable model memory. The operating system, driver limits, other applications and inference buffers still matter. Likewise, Spark’s “up to 1 PFLOP FP4” is a theoretical FP4 peak using sparsity; it is not a measured chat speed and cannot be divided by an Apple TOPS figure to choose a winner. NVIDIA’s datasheet footnote specifies the sparse FP4 basis.统一内存让 CPU 与 GPU 访问共同的内存池，消除了独立显存容量较小这一类限制，但整机内存并不会全部变成模型可自由使用的空间。操作系统、驱动分配限制、其他应用和推理缓冲仍要占位置。同理，Spark“最高 1 PFLOP FP4”是采用稀疏性的理论 FP4 峰值，不是聊天实测速率，也不能拿它与 Apple 的 TOPS 相除来决定输赢。NVIDIA 规格书脚注明确了稀疏 FP4 口径。

Size the whole working set内存要算整个工作集

Start with raw weight storage. P is the number of stored parameters, and q is the number of bits per parameter. At exactly four bits, 8B needs 4 GB, 32B needs 16 GB and 70B needs 35 GB before scales, unquantized tensors and runtime allocations. A “4-bit model” file is not guaranteed to equal that arithmetic minimum.先算权重的原始存储量。P 是需要存储的参数数目，q 是每个参数的位数。按严格 4 bit 计算，8B 为 4 GB，32B 为 16 GB，70B 为 35 GB；这里还没放入量化缩放因子、保留高精度的张量与运行时分配。“4 bit 模型”文件不一定恰好等于这个算术最小值。

The second large allocation is the KV cache: each attention layer retains keys and values for tokens already processed. For a conventional full-context decoder with equal-length sequences, its uncompressed size is approximately the expression below. L is the layer count, Hkv the number of key/value heads, d the head dimension, T the cached tokens per sequence, B the number of resident sequences and s the bytes per element. T includes the prompt and generated tokens retained so far.另一项大分配是 KV cache：注意力层保存已经处理过的 token 的键和值。对于常规全上下文解码器，若各路序列等长，未压缩 KV 的大小可按下式估算。L 是层数，Hkv 是键／值头数，d 是头维度，T 是每路缓存的 token 数，B 是驻留序列数，s 是每个元素的字节数。T 包含仍被保留的输入与已生成内容。

Grouped-query attention reduces Hkv; sliding windows, shared prefixes and compressed caches change this budget. Quantizing the weights does not automatically quantize the KV cache. Transformers’ cache documentation describes the separate cache strategies and their trade-offs.分组查询注意力会减少 Hkv；滑动窗口、共享前缀和压缩缓存会改变预算。权重量化不等于 KV 也被量化。Transformers 的缓存文档分别说明了这些策略及其取舍。

Use one explicit example: a dense 70B model, four-bit raw weights, 80 layers, 8 KV heads, head dimension 128 and FP16 KV elements. Assume quantization metadata adds 10% to raw weights, and reserve 16 GiB for the OS, applications and runtime. These last two numbers are planning assumptions, not measurements or universal overhead ratios. The KV allocation is 320 KiB per cached token per sequence.用一个参数明确的算例：70B 稠密模型、4 bit 原始权重、80 层、8 个 KV 头、头维度 128，KV 元素采用 FP16。再假定量化元数据增加原始权重的 10%，为系统、应用与运行时预留 16 GiB。后两项是规划假设，不是实测，也不是通用开销比例。这个结构每路序列的每个缓存 token 需要 320 KiB。

A 70B illustrative memory budget: one 8,192-token sequence totals 54.4 GiB; one 32,768-token sequence or four 8,192-token sequences totals 61.9 GiB; four 32,768-token sequences totals 91.9 GiB. Weights and reserve are fixed; KV grows. 70B 内存预算算例：一路8192 token共54.4 GiB；一路32768或四路8192均为61.9 GiB；四路32768增至91.9 GiB。权重和预留固定，KV随工作负载增大。
Figure 1 · The same model can need 54.4 or 91.9 GiB. Columns use binary GiB (2³⁰ bytes); raw weight examples use decimal GB (10⁹ bytes). No prefix sharing, KV quantization or offloading is assumed. These are total-system budgets, not GPU allocation guarantees.图 1 · 同一个模型，预算可以从 54.4 增到 91.9 GiB。条形图采用二进制 GiB（2³⁰ 字节），前文原始权重采用十进制 GB（10⁹ 字节）。未假定共享前缀、KV 量化或卸载；这属于整机预算，不保证 GPU 能分配同样的空间。

This is why 96 GB can be sensible for a bounded single-user workload and still be a poor choice for four long conversations. Check the actual checkpoint’s loaded memory, prefill peak and the platform’s allocation limit. For the single-sequence 70B Q4 / 32K case, I would first validate the 96 GB Mac. Consider 256 GB for multiple resident models, four long conversations, or measured peaks approaching the allocation limit. Spend on memory when that is the constraint; extra SSD space and swap do not remove it. On Spark, reduce context/concurrency or change model format when the 128 GB envelope is insufficient.因此，96 GB 可以适合边界明确的单用户任务，却不一定适合四路长对话。 需要核对具体权重的加载占用、预填充峰值与平台的分配上限。对于单路 70B Q4／32K 任务，我会先验证 Mac 的 96 GB 档位。需要多模型同时驻留、四路长对话，或实测峰值逼近分配上限时，再考虑 256 GB。内存确实成为约束时再为它花钱；增加 SSD 或交换空间解决不了同一问题。Spark 的 128 GB 不够时，需要缩短上下文、降低并发，或更换模型格式。

For a mixture-of-experts model, distinguish active parameters per token from total stored parameters. A 235B model with only a small subset active still has about 117.5 GB of raw four-bit weights if all experts stay resident. That is already too close to a 128 GB system for a casual “it fits” claim. Streaming experts from storage can change capacity requirements, but also changes latency and the deployment problem.对于混合专家模型（MoE），要区分每个 token 的激活参数与全部存储参数。235B 模型即使每次只激活一小部分专家，全部专家驻留时，4 bit 原始权重仍约为 117.5 GB。对 128 GB 整机而言，这已经不能轻率地说“装得下”。从存储中按需调入专家可以改变容量需求，但也改变了延迟与部署问题。

Separate first-token latency from generation speed首字等待与逐字生成是两种负载

An autoregressive model first processes the prompt (prefill), then produces new tokens one step at a time (decode). A long document can take time before the first visible answer even if subsequent tokens arrive quickly. Conversely, a short prompt can start quickly and then crawl. Measure time to first token, inter-token latency and aggregate output throughput separately; include queueing when testing a service.自回归模型先处理输入（prefill，预填充），再逐步生成新 token（decode，解码）。长文档可能让首字等待很久，即使后续输出很快；短问题也可能很快开始，之后却一个字一个字地挪。首 token 延迟、相邻 token 延迟与总输出吞吐要分开测，服务验收还要把排队时间算进去。

Prefill can reuse weights across many prompt tokens and exploit large matrix operations. Single-sequence dense decode often has much less reuse: it streams a large fraction of the weights to produce one token. This makes memory bandwidth a useful first constraint. Long contexts add KV traffic, while batching changes the reuse pattern. NVIDIA’s attention analysis explains why prefill and decode need different attention designs.预填充能在多个输入 token 之间复用权重，利用较大的矩阵运算。单路稠密模型解码的复用机会通常小得多：为了产生一个 token，需要读取大量权重。因此，内存带宽是一个有用的初始约束。长上下文还会增加 KV 读取，并批处理则改变复用方式。NVIDIA 对注意力的分析说明了为什么两个阶段需要不同的优化。

For the narrow model in which every decode step reads all 35 GB of raw weights once, advertised bandwidth gives 273 / 35 = 7.8 and 1,200 / 35 ≈ 34.3 token/s. These are ideal weight-streaming ceilings within those assumptions. They omit quantization metadata, dequantization, attention/KV traffic, synchronization and other memory users. They are not measured speeds, expected speeds or a universal bound on all inference algorithms.如果严格假定每个解码步都完整读取一次 35 GB 原始权重，代入标称带宽，得到 273／35 = 7.8，以及 1,200／35 ≈ 34.3 token/s。这是在上述假设内的理想权重读取上界，省略了量化元数据、反量化、注意力与 KV 读取、同步以及其他内存使用者。它们不是实测速率、预期速率，也不是所有推理算法的普遍上界。

Ideal dense single-stream weight-reading ceilings for 35 GB per token: 273 GB/s gives 7.8 token/s and 1,200 GB/s gives 34.3. No measured machine performance is shown. 每个token读取35 GB权重的理想单路稠密解码上界：273 GB/s对应7.8 token/s，1200 GB/s对应34.3。图中不是机器实测性能。
Figure 2 · A reason to take Mac memory bandwidth seriously, not a claim that it is 4.4 times faster in every task. MoE, batching and speculative decoding do not obey the same weight-per-output-token assumption. The remaining work can lower performance under this model.图 2 · 这解释了为什么 Mac 的内存带宽值得重视，却不能证明它在所有任务中都快 4.4 倍。MoE、并批处理与推测解码不满足同样的“每个输出 token 读取权重”假设；模型中省略的工作会进一步消耗时间。

For a person reading one answer, per-user latency usually matters most. For four agents or users, a serving engine can batch their work and raise total throughput while making each individual stream slower. A well-supported CUDA kernel can also change prefill performance dramatically. Therefore my Mac recommendation is for a compatible personal inference workload, not a benchmark verdict over every model and serving configuration.一个人阅读一个回答时，更关心自己的延迟。四个 agent 或用户同时访问时，服务引擎可以并批提高总吞吐，但每一路未必更快；适配良好的 CUDA 内核也可能显著改变预填充性能。因此，前文推荐 Mac 的前提是适配的个人推理任务，并不是所有模型与服务配置的跑分裁决。

Choose the backend before downloading the weights先选后端，再下载权重

Route路线Use it for适合任务Check before committing先核对什么
Ollama / LM StudioOllama／LM Studio Use it for适合任务A quick local chat interface and model management; useful before a hardware purchase先把本地聊天与模型管理用起来，适合购买硬件前验证需求 Check before committing先核对什么Exact model support, GPU offload and logs. A familiar interface does not guarantee the same backend or speed on both platforms.具体模型是否支持、是否真正卸载到 GPU，以及运行日志；相同界面不意味着两台机器使用相同后端或速度。
MLX-LM · MacMLX-LM · Mac Use it for适合任务Native Apple-silicon inference and supported adapter fine-tuningApple 芯片原生推理，以及支持的适配器微调 Check before committing先核对什么Use an MLX-compatible checkpoint. Metal/MLX is not CUDA; a CUDA-only Python extension does not become portable because both CPUs are Arm.选择兼容 MLX 的权重。Metal／MLX 不是 CUDA；两边 CPU 都是 Arm，也不意味着 CUDA 专用 Python 扩展可直接移植。
vLLM / TensorRT-LLM · SparkvLLM／TensorRT-LLM · Spark Use it for适合任务NVIDIA development and supported multi-request servingNVIDIA 研发与受支持的多请求服务 Check before committing先核对什么Use the Spark support matrix and Arm64 / Blackwell-compatible builds. Model architecture, quantization and kernels must all match.按 Spark 支持矩阵选择兼容 Arm64／Blackwell 的版本，模型结构、量化方式与内核三者都要匹配。
llama.cpp · bothllama.cpp · 两者 Use it for适合任务GGUF inference, a small local server and a controlled cross-platform baselineGGUF 推理、小型本地服务，以及可控的跨平台比较基线 Check before committing先核对什么CUDA on Spark; Metal on Mac. Record the same GGUF hash and source revision when comparing.Spark 使用 CUDA，Mac 使用 Metal；对比时记录相同 GGUF 文件哈希与源码版本。

“Four-bit” specifies a numerical representation, not a portable file format. GGUF quantization, MLX groupwise weights, AWQ/GPTQ, MXFP4 and NVFP4 have different layouts and support requirements. A backend must implement both the model architecture and the selected representation. Start with a small supported model, verify the API and memory behavior, then scale it up. MLX-LM · llama.cpp · Spark vLLM playbook.“4 bit”描述数值表示，并不是通用文件格式。GGUF 量化、MLX 分组量化权重、AWQ／GPTQ、MXFP4 与 NVFP4 的布局和支持条件各不相同。后端必须同时支持模型结构与所选表示。先用一个明确受支持的小模型跑通 API、检查内存行为，再增加规模。MLX-LM · llama.cpp · Spark vLLM 指南。

A quick first run with Ollama先用 Ollama 验证需求

For a first local chat, install Ollama from its official distribution, start the application, and try the local model below. On Mac, run it natively: Docker Desktop does not give this container Metal GPU acceleration. Use ollama ps to inspect placement. Choose a local model tag rather than a cloud model; the FAQ documents how to disable cloud features when required. Ollama FAQ · local model entry.如果只想先体验本地对话，从官方渠道安装并启动 Ollama，再试下面的本地模型。Mac 上原生运行：Docker Desktop 不会为该容器提供 Metal GPU 加速。用 ollama ps 查看模型放在哪里运行。选择本地模型标签，而不是云端模型；需要禁用云功能时，按 FAQ 的设置操作。Ollama FAQ · 本地模型条目。

ollama pull qwen3:8b
ollama run qwen3:8b
# In another terminal while the model is loaded:
ollama ps

Deploy a local API on MacMac：用 MLX 启动本地 API

Run this natively in macOS with Python 3.11 or newer, as required by the current MLX-LM release. The example uses a community-converted Qwen3 8B four-bit checkpoint as a small startup model, not as the best available model or a benchmark match for the Spark example. Confirm the model card, license and package requirements before downloading. The commands follow the project interface; they have not been executed on an M5 Ultra for this article.在 macOS 中原生运行，当前 MLX-LM 要求 Python 3.11 或更高版本。这里用社区转换的 Qwen3 8B 四位权重做小规模启动示例，不宣称它是当前最优模型，也不把它与下文 Spark 的模型当成对测。下载前核对模型卡、许可证和依赖要求。命令依据项目接口编写，本文没有在 M5 Ultra 真机上执行。

python3 -m venv .venv-mlx
source .venv-mlx/bin/activate
python -m pip install --upgrade pip mlx-lm

mlx_lm.server \
  --model mlx-community/Qwen3-8B-4bit \
  --host 127.0.0.1 --port 8080

In another terminal, query the local service. The request disables Qwen3 thinking for a short startup check. The first run downloads weights; allow loading to finish. The server is intended for local development, not an unprotected public endpoint. MLX-LM server source · checkpoint model card.另开终端调用本地服务。请求关闭 Qwen3 的思考模式，便于做简短的启动检查。首次运行需要下载权重，并等待加载完成。该服务用于本地开发，不应直接作为无保护的公网接口。MLX-LM 服务源码 · 权重模型卡。

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "mlx-community/Qwen3-8B-4bit",
    "chat_template_kwargs": {"enable_thinking": false},
    "messages": [{"role": "user", "content": "Explain KV cache in three sentences."}],
    "max_tokens": 512
  }'

python -m pip freeze > requirements-mlx-lock.txt

Watch memory pressure and swap, not just the “free RAM” number. Unified memory removes the CPU–GPU copy boundary for many operations, but it does not make disk paging fast. If loading or generation becomes unstable, reduce context, close other large applications or use a smaller checkpoint before adjusting system memory limits. Keep the working package versions and model revision with the project.关注内存压力与交换量，不要只看“空闲内存”数。统一内存可以省去许多 CPU—GPU 之间的数据搬运，但不能让磁盘换页变快。加载或生成不稳定时，先减小上下文、关闭其他大应用或更换较小权重，再考虑系统内存限制。把跑通时的依赖版本与模型修订号一起保留下来。

Deploy a CUDA service on SparkSpark：用受支持的 CUDA 容器起步

First complete the DGX OS update and verify the driver, Docker and NVIDIA Container Toolkit. Use a container explicitly supported for Spark’s Arm64 CPU and Blackwell GPU. An x86 CUDA image is not a substitute. The following development example adapts NVIDIA’s vLLM playbook: it pins the documented example image and uses the supported gpt-oss-20b checkpoint. Recheck the playbook and release notes when installing; this recipe is source-checked, not Spark-hardware-tested here.先完成 DGX OS 更新，检查驱动、Docker 与 NVIDIA Container Toolkit。容器必须明确支持 Spark 的 Arm64 CPU 与 Blackwell GPU，不能拿 x86 CUDA 镜像直接替代。下面的开发示例改写自 NVIDIA 的 vLLM 指南，固定其示例镜像版本，并使用支持列表中的 gpt-oss-20b。安装时重新核对指南与发布说明；本文核对了来源，没有在 Spark 真机上运行。

nvidia-smi

mkdir -p "$PWD/.cache-huggingface"
docker run --rm --gpus all \
  -p 127.0.0.1:8000:8000 \
  --shm-size=8g \
  -v "$PWD/.cache-huggingface:/root/.cache/huggingface" \
  nvcr.io/nvidia/vllm:26.05.post1-py3 \
  vllm serve openai/gpt-oss-20b \
    --host 0.0.0.0 --port 8000 \
    --max-model-len 8192 \
    --max-num-seqs 1 \
    --gpu-memory-utilization 0.70

The process listens inside the container on 0.0.0.0, but Docker publishes it only on the host’s 127.0.0.1. The one-sequence, 8,192-token settings deliberately keep the first deployment bounded; 70% is a starting allocation setting, not a throughput optimum. A gated model additionally requires accepting its terms and passing the appropriate access credential; do not bake credentials into images. NVIDIA deployment instructions.容器内进程监听 0.0.0.0，但 Docker 只把端口发布到宿主机的 127.0.0.1。一路序列、8,192 token 是为了限制第一次部署的工作集；70% 是起始分配设置，不是吞吐最优参数。访问受限模型还需同意相应条款并传入访问凭据，不要把凭据写进镜像。NVIDIA 部署说明。

After the server is ready, open a second terminal on Spark for the following requests. The first terminal keeps the container running.等待服务就绪后，在 Spark 上另开终端发送下面的请求，保留第一个终端运行容器。

curl -f http://127.0.0.1:8000/health
curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-oss-20b",
    "reasoning_effort": "low",
    "messages": [{"role": "user", "content": "Explain KV cache in three sentences."}],
    "max_tokens": 1024
  }'

The reasoning model can spend output tokens before a final answer, so a successful health check is not proof that a short response budget will finish the answer. An OpenAI-compatible API lets an editor or chat interface point at this local base URL; it does not make tool calling, reasoning fields or multimodal inputs identical across models. Test the exact client workflow. For remote use from your own laptop, an SSH tunnel keeps the service off the public network, for example ssh -L 8000:127.0.0.1:8000 user@spark-host.推理模型可能在最终回答前消耗输出 token，因此健康检查成功不代表较短的输出预算一定能完成回答。兼容 OpenAI 的 API 可以让编辑器或聊天界面改用本地 base URL，但工具调用、推理字段与多模态输入并不会因此在所有模型上完全一致，需要验证具体客户端流程。如果从自己的笔记本远程使用，可以通过 SSH 隧道访问，例如 ssh -L 8000:127.0.0.1:8000 user@spark-host，不必把服务端口开放到公网。

Make the application local, not only the model模型在本地，应用链路也要在本地

  1. Question问题输入The local client submits a question about your documents.本地客户端提交与文档有关的问题。
  2. Retrieval检索上下文A local embedding model and index select relevant passages.本地嵌入模型与索引选出相关片段。
  3. Generation生成回答The inference API uses the question and passages to generate an answer.推理 API 结合问题和片段生成回答。

For a private document assistant, begin with retrieval-augmented generation (RAG): parse and index local documents, retrieve relevant passages, and pass only those passages to the model with source references. Keep embeddings, reranking and the vector index local as well. A local language model attached to a cloud embedding API or an unrestricted web tool is not an offline application. Download models and dependencies first, then test the workflow with network access disabled.私人文档助手可以先从检索增强生成（RAG）开始：解析并索引本地文档，检索相关片段，把这些片段及其出处交给模型。嵌入、重排序与向量索引也要留在本地。模型在本机，但嵌入调用云端 API、工具可以随意联网，这仍不是离线应用。先下载模型和依赖，再断网测试完整流程。

Use adapter fine-tuning when the goal is a repeatable style, output format or task behavior; use retrieval for documents that change and must be cited. Both MLX-LM and the NVIDIA ecosystem offer supported adapter-training routes. CUDA dependence makes Spark the more natural development choice, but Mac is not “inference only.” Conversely, a vendor’s “fine-tune 70B” statement must not be read as full-parameter Adam training of a 70B model. Gradients, optimizer states and activations can dwarf quantized inference memory.需要稳定的文风、输出格式或任务行为时，可以考虑适配器微调；文档会更新、回答需要出处时，优先使用检索。MLX-LM 与 NVIDIA 生态都有受支持的适配器训练路线。依赖 CUDA 的项目更适合以 Spark 起步，但 Mac 并非“只能推理”。反过来，厂商所说“可微调 70B”，也不能理解为能用 Adam 完整训练 70B 的全部参数：梯度、优化器状态和激活量可能远大于量化推理的内存。

Multi-machine execution is a separate engineering project on either platform. ConnectX networking on Spark and supported Thunderbolt/RDMA workflows on Mac can connect devices, but a cable does not turn two computers into one transparent memory pool. The runtime must partition work, communicate tensors and tolerate the added latency. Validate a single node first, then justify a cluster with a workload that needs it.两种平台的多机运行都属于另一项工程。Spark 的 ConnectX 网络与 Mac 上受支持的雷雳／RDMA 工作流可以连接设备，但插上电缆，不会把两台电脑变成一个透明的大内存池。运行时需要切分任务、交换张量，并承担额外延迟。先验证单机，再用实际任务证明集群的必要性。

Ask for an acceptance test, not a headline score采购前，按自己的任务验收

There are two useful comparisons. For a controlled hardware/backend comparison, use the same GGUF file, llama.cpp revision, context, KV type and GPU offload policy, with CUDA on Spark and Metal on Mac. For the system you would actually use, test each platform’s best supported stack with comparable model quality and the same task set. Label the two comparisons separately; forcing one common backend can hide native advantages, while unmatched native quantizations can change answer quality.有两种有价值的比较。第一种是控制变量：使用相同 GGUF、llama.cpp 版本、上下文、KV 类型与 GPU 卸载策略，Spark 用 CUDA，Mac 用 Metal。第二种是比较实际要用的系统：各自采用适配良好的软件栈，控制模型质量，跑同一组任务。两种结果分别标注；强制同一后端可能掩盖原生优势，不同原生量化又可能改变回答质量。

With a local GGUF already downloaded and a matching GPU-enabled llama.cpp build, the following records a model-inference baseline. Set MODEL to your real file. The pp and tg tests are separate; this command does not measure end-to-end service latency or guarantee decode speed after an 8K prompt. Use combined -pg tests and a real request harness for that. llama-bench options.下载好本地 GGUF，并用对应 GPU 后端编译 llama.cpp 后，下面的命令可记录一个推理引擎基线。把 MODEL 改成真实文件。pp 与 tg 是独立测试；这条命令不测完整服务延迟，也不能证明 8K 输入后的解码速度。后者还应使用组合 -pg 测试与真实请求驱动器。llama-bench 参数说明。

MODEL=/absolute/path/to/model.gguf
./build/bin/llama-bench \
  -m "$MODEL" -ngl 99 \
  -p 512,8192 -n 128 -r 5 \
  -ctk f16 -ctv f16 -o json > llama-bench.json
Test验收项Hold constant固定条件Record记录结果
Short chat短对话 Hold constant固定条件512 input tokens, 256 output tokens, one request; model already loaded512 输入 token、256 输出 token、单路请求；模型已加载 Record记录结果Time to first token, per-stream token latency and task correctness首 token 延迟、单路 token 间隔与任务正确率
Read a project阅读项目 Hold constant固定条件8K and 32K inputs; total context reserves room for output; identical files and questions8K 与 32K 输入；总上下文额外留出输出空间；相同文件与问题 Record记录结果First-token delay, peak memory, swap/page faults and answer grounding首字等待、峰值内存、交换／缺页以及回答是否有依据
Shared service共享服务 Hold constant固定条件1 and 4 simultaneous requests; same prompt/output distributions1 路与 4 路同时请求；相同输入／输出长度分布 Record记录结果Aggregate output tokens/s plus p50/p95 end-to-end latency; count failures总输出 token/s 与端到端 p50／p95 延迟，同时记录失败率
Sustained load持续负载 Hold constant固定条件Repeat the target mix for at least 30 minutes after warmup; log the cold start separately预热后持续重复目标任务至少 30 分钟；冷启动另测 Record记录结果Late-run latency, memory growth, wall power, noise and thermal behavior后半程延迟、内存增长、插座功率、噪声与温度行为

Set pass/fail requirements before asking a seller to run the test. An example for an interactive assistant is warm first-token latency under 3 seconds for the short prompt, at least 15 output token/s per stream, no out-of-memory errors and acceptable correctness on your own questions. Those are example requirements, not claimed results; a document-analysis job may accept a slower first token. Preserve raw request timings, model hashes, runtime versions and power logs.请商家测试前，先写明通过标准。交互助手可以例如要求：短输入的热启动首字延迟小于 3 秒，每路至少 15 输出 token/s，没有内存不足，并在自己的问题集上达到可接受的正确率。这些是示例需求，不是本文的实测结果；文档分析任务可以接受更长的首字等待。保留原始请求时间、模型哈希、运行时版本与功率记录。

Spend on the constraint you cannot work around把预算花在绕不开的约束上

Your actual need实际需求My starting choice我的起点建议What changes the choice什么条件会改变选择
Learning; 7B–14B chat入门；7B–14B 聊天 My starting choice我的起点建议Neither, initially. Try the current computer or a smaller machine.先不买这两台。 用现有电脑或较小的机器验证。 What changes the choice什么条件会改变选择Buy only after a specific memory or latency limit is demonstrated.证明确实遇到容量或延迟限制后，再升级。
One user; compatible 32B–70B inference单用户；兼容的32B–70B推理 My starting choice我的起点建议Mac Studio Ultra. First validate 96 GB for one 70B Q4 / 32K sequence. Consider 256 GB for multiple resident models, four long sequences or insufficient measured headroom.Mac Studio Ultra。 单路70B Q4／32K先验证96 GB；多模型驻留、四路长对话或实测余量不足，再考虑256 GB。 What changes the choice什么条件会改变选择A required CUDA extension, unsupported architecture or failed latency test points elsewhere.必须使用CUDA扩展、模型结构不支持，或延迟验收不通过时，改选。
CUDA research; NVIDIA deployment pathCUDA研发；后续部署到NVIDIA服务器 My starting choice我的起点建议DGX Spark. Buy software compatibility and a local development workflow.DGX Spark。 购买的是软件兼容性与本地研发工作流。 What changes the choice什么条件会改变选择If the real working set exceeds 128 GB or a target kernel is unsupported, compare a different GPU system.实际工作集超过128 GB，或目标内核不支持时，比较其他GPU系统。
Working sets exceeding Spark’s usable memory工作集超出Spark的可用内存 My starting choice我的起点建议Mac Studio 256 GB if the measured budget fits and the model is supported; evaluate 512 GB only when deliverable.预算确实装得下且模型受支持时选Mac Studio 256 GB；512 GB档位要等可交付后再评估。 What changes the choice什么条件会改变选择Capacity alone cannot fix a decode-latency failure. A server or a smaller/better model may be preferable.容量解决不了解码过慢；服务器或更小、更合适的模型可能更好。
Multi-user production; full large-model training多人生产服务；大模型全参数训练 My starting choice我的起点建议Compare dedicated GPU servers first. A desktop is still useful for prototyping.先比较独立GPU服务器。 桌面设备仍适合原型验证。 What changes the choice什么条件会改变选择Choose either desktop only after throughput, tail latency, failure recovery and workload-specific memory all pass.只有吞吐、尾延迟、故障恢复和实际内存占用都通过，才选桌面设备承载。

Total cost is the configured machine, tax and warranty, storage/backup, electricity and maintenance time. SSD space holds checkpoints; it does not replace inference memory or memory bandwidth. Estimate electricity from measured average wall power and hours of use, not the power-supply rating. A server that is busy only occasionally may be cheaper to rent, while a private machine used every day can justify ownership without winning a peak-FLOPS comparison.总成本包含具体配置、税费与保修、存储与备份、电费和维护时间。SSD 用来保存权重，不能替代推理内存与内存带宽。电费应按实际平均插座功率与使用时长估算，而不是用电源额定功率代替。偶尔使用的服务器可能租用更省；每天使用的私人机器，也可能无需赢得峰值 FLOPS 对比就值得拥有。

My default recommendation: Mac Studio Ultra for a compatible personal inference workstation; DGX Spark for a CUDA development workstation. First prove that the model and context fit, then require the target latency, then price the exact configuration. The better machine is the one that passes those three checks for the job you will run tomorrow.我的默认建议：兼容的个人推理工作站选 Mac Studio Ultra，CUDA 研发工作站选 DGX Spark。 先证明模型和上下文装得下，再要求达到目标延迟，最后给具体配置询价。更好的机器，应当能通过这三关，完成你明天真正要运行的任务。

Calculation files: memory and bandwidth data. The figures can be reproduced with scripts/render-local-llm-budget.py in the website repository. No hardware benchmark is generated by that script.计算数据：内存与带宽预算。图示可用网站仓库中的 scripts/render-local-llm-budget.py 重现；该脚本不产生硬件实测数据。