Inkling's Open Weights Still Require Cluster-Scale Memory

Thinking Machines released Inkling on July 15, 2026, as its first open-weights model. Inkling accepts text, images, and audio and produces text; its weights carry an Apache-2.0 license. The release gives developers unusually broad access to a large multimodal model, but it does not turn that model into an ordinary local deployment.

The deployment numbers create the practical stake. Inkling has 975 billion total parameters and 41 billion active parameters. The Hugging Face guide says serving the BF16 checkpoint requires 2 TB of VRAM, while the NVFP4 checkpoint requires 600 GB, before KV-cache and other runtime headroom. For a team deciding how to evaluate Inkling, those figures point to three paths: use a hosted service to test capability, fund a cluster evaluation to test self-hosting, or choose a smaller model when local operation is the requirement that cannot move.

Open weights grant control while infrastructure remains your responsibility

The official announcement says Thinking Machines released the full weights, and the public model card lists Apache-2.0. In practical terms, a team can download the checkpoint, inspect how it is packaged, run it through supported inference software, and modify or fine-tune it within the license terms. That is materially different from relying only on a vendor-controlled API.

A team that self-hosts Inkling must supply the accelerator memory, interconnects, storage, serving software, monitoring, and engineering needed to operate the selected configuration. Open weights remove an access restriction while leaving the physical requirements of the checkpoint intact. This distinction matters most with a sparse model such as Inkling, because its per-token compute and its deployment footprint describe different parts of the system.

Thinking Machines also says Inkling supports a context window of up to one million tokens and was pretrained on 45 trillion tokens spanning text, images, audio, and video. Those are vendor statements about the model and its training. They do not establish how every hosted endpoint or self-managed configuration will perform at the longest context length.

The 41B active figure describes compute; the checkpoint still holds 975B parameters

A mixture-of-experts model, usually shortened to MoE, divides parts of the network into many specialized groups of weights called experts. A router chooses a subset of those experts for each token instead of sending every token through every expert. Inkling is a 66-layer sparse MoE with 256 routed experts and two shared experts; for each token, it uses six routed experts plus the two shared experts, according to the model card.

“Active parameters” counts the parameters used along a token's routed path. Inkling's 41B figure is therefore a rough architectural proxy for per-token model work, not a measurement of latency, throughput, or hardware efficiency. The serving system still needs access to all 975B parameters because later tokens may be routed to different experts.

Keep the two parameter counts separate. Active parameters describe the selected computation path. Total parameters determine the complete weight set that must be stored, loaded, and distributed across the serving system. Neither count, by itself, states the checkpoint's file size, the VRAM required by a particular numeric format, or the additional memory needed at runtime.

The official BF16 and NVFP4 configurations require 2 TB and 600 GB of VRAM

BF16, or bfloat16, represents numerical values with 16 bits. NVFP4 is NVIDIA's 4-bit floating-point format, used here for a quantized version of Inkling. The numeric format changes the checkpoint's storage footprint and the memory needed to hold its weights; it does not change Inkling's 975B total-parameter or 41B active-parameter counts. The Hugging Face guide pairs BF16 with Hopper or newer GPUs and NVFP4 with NVIDIA Blackwell GPUs.

VRAM is the high-speed memory attached to a GPU or other accelerator. The Hugging Face launch and inference guide says the BF16 configuration requires 2 TB of VRAM and the NVFP4 configuration requires 600 GB. These are GPU-memory requirements for the loaded official configurations, not on-disk file-size measurements or complete serving-memory budgets.

Runtime serving needs additional memory. The key-value cache, or KV cache, stores attention data for tokens the model has already processed and grows with context length and generated output. The Hugging Face guide advises leaving KV-cache headroom or capping the maximum model length when memory limits appear. It provides SGLang and vLLM examples and says realistic deployment requires multiple nodes and a distribution system such as Slurm.

Local recipes still matter. The model card lists support through SGLang, vLLM, TokenSpeed, Unsloth, and Hugging Face Transformers; the Hugging Face guide separately documents llama.cpp. “Local” in this context means running outside a hosted model API, including on infrastructure a team controls. At 600 GB or 2 TB of checkpoint VRAM, the supported reference configurations are not evidence of a normal workstation deployment. More aggressive third-party quantization may lower the memory requirement further, but it creates a different model configuration whose quality, modality support, speed, and stability need their own evaluation.

Hosted evaluation answers whether Inkling fits the workload

The lowest-commitment path is hosted access. The announcement says Inkling is available in the Tinker Playground and through Together AI, Fireworks, Modal, Databricks, and Baseten; Hugging Face also documents remote inference through its provider router. These options let a team test real inputs without first assembling a cluster, but modality support must be verified for the selected endpoint. At release, the Hugging Face guide said audio support through Inference Providers was still in progress.

A hosted evaluation can establish whether Inkling follows the task, handles the required modalities, produces useful outputs, and merits deeper testing. It can also expose prompt and preprocessing requirements. Run the same representative requests, context shapes, and repeated-input patterns used for other candidates; our cache-aware model comparison guide explains why those workload details should stay consistent across providers.

Hosted results cannot prove that a self-managed cluster will match the provider's latency, throughput, reliability, or cost. They also inherit the endpoint's limits. The official product page shows 64K and 256K context options on Tinker, below the model's announced one-million-token maximum, and modality support can vary by provider. A hosted test should be reported with the provider, endpoint, context setting, and reasoning effort attached to the result.

Cluster evaluation answers whether self-hosting is operable for your team

A cluster evaluation is justified when self-hosting is already tied to a real requirement, such as a deployment environment, data-handling constraint, customization plan, or service dependency the team is prepared to own. It needs an approved hardware budget or access to suitable rented infrastructure, plus engineering time for model distribution, serving, observability, failure recovery, and workload tests. The Hugging Face guide's multi-node direction makes this an infrastructure project, even though supported launch commands are available.

This path can measure the selected BF16 or NVFP4 configuration under the team's own conditions. It can test end-to-end latency, concurrent throughput, accelerator utilization, context limits, image and audio preprocessing, restart behavior, and the operational effect of distributing weights across nodes. If one-million-token context is a requirement, the cluster evaluation is also where the team can test that limit directly rather than inferring it from the architecture claim.

A bounded cluster run still cannot establish full production economics by itself. Total cost depends on sustained utilization, concurrency, power or cloud rates, staffing, redundancy, storage, network behavior, and the amount of memory reserved for context. Fine-tuning through Tinker is available, and the Tinker cookbook includes sampling, supervised fine-tuning, reinforcement learning, distillation, and multimodal examples, but recipe availability does not demonstrate a return on fine-tuning for a particular workload.

A smaller model is the direct path when local operation is the hard constraint

Some evaluations begin with a model and ask what hardware it needs. When the fixed requirement is a workstation, a small server, low latency at modest traffic, or operation without a cluster, model size needs to enter the decision earlier. A released smaller model may offer less capability than Inkling, yet be more useful if the team can deploy, measure, and maintain it on the infrastructure it actually has.

A smaller-model evaluation can measure task quality, latency, and memory use on the hardware the team actually has. Those results must come from the smaller model itself; Inkling's vendor-reported results do not transfer to another model, especially for multimodal reasoning or long-context tasks. Our analysis of Qwen 3.5's compact models covers the commodity-hardware side of this decision; the relevant comparison here is the smallest model that clears the workload's acceptance threshold, not a parameter-count contest.

Thinking Machines previewed an Inkling-Small model with 12B active parameters in the announcement, but said its full weights were still being tested before release. As of July 15, 2026, Inkling-Small was a preview rather than an available open-weights substitute for Inkling. Teams choosing the smaller-model path need to evaluate a model they can actually obtain and operate.

Each access path answers a different evaluation question

Scroll sideways to see all 4 columns.

Access path	Choose it when	What it can measure in the tested setup	What it does not establish
Hosted evaluation	Capability and multimodal task fit are still uncertain	Output usefulness, modality handling, prompt shape, provider-level latency, and whether deeper work is warranted	Self-host operability, cluster economics, or one-million-token behavior when the endpoint exposes less context
Funded cluster evaluation	Self-hosting is a real requirement and cluster resources are available	Memory fit for a selected precision, latency, throughput, context behavior, failure recovery, and operating effort in the tested configuration	Production total cost, long-run reliability, or fine-tuning return without representative scale and duration
Smaller-model evaluation	Local hardware, latency, or operational simplicity is the fixed boundary	Whether a deployable model clears the workload's quality threshold on available infrastructure	Inkling-level capability or a direct transfer of Inkling benchmark results

The table is also a sequence for teams that do not yet have a fixed hosting requirement. Start hosted when the main uncertainty is whether Inkling improves the workload at all. Move to a funded cluster only when the hosted result is strong enough and self-hosting has a concrete benefit that justifies multi-node operations. Evaluate a smaller model when the cluster requirement would defeat the reason for running locally.

Keep the resulting deployment facts tied to the workload and date. Provider limits, model versions, context settings, precision, and inference software can change; the same discipline used to keep model-selection facts current applies here. A result without its configuration can make a hosted success look like proof of local operability when it is not.

The current evidence supports evaluation; production remains unproven

Thinking Machines says plainly that Inkling is not the strongest overall model available, open or closed. The model card reports broad benchmark and safety results, but those results are vendor-reported and sensitive to the evaluation setup. The Tinker cookbook itself notes that system prompts, token limits, temperature, and timeouts can shift benchmark scores.

The five sources listed below document the release, model properties, supported inference paths, Tinker availability, and cookbook examples. They do not provide independent measurements of production deployments, total cost, sustained throughput, end-to-end latency, returns from fine-tuning, or behavior at the one-million-token context limit.

That source gap does not predict how Inkling will perform in those conditions. It limits what the current public evidence can establish.

Inkling's open weights expand what a team is permitted to inspect, customize, and operate. Its sparse routing reduces the work done for each token, while the 975B-parameter checkpoint keeps deployment at cluster scale in the official BF16 and NVFP4 configurations. Choose hosted access to decide whether the model deserves investment, a cluster evaluation to decide whether your team can operate it under a real self-hosting requirement, or a smaller model when the deployment must fit the hardware already within reach.

If that choice is still open, bring one real multimodal workload to BaristaLabs' AI consulting team. We can compare the hosted, cluster, and smaller-model paths against the workload's actual inputs, context, latency, privacy, and operating requirements without assuming which path should win.

Sources

• Thinking Machines Lab: Introducing Inkling
• Thinking Machines Lab: Inkling product page
• Hugging Face: Inkling model card
• Hugging Face: Welcome Inkling by Thinking Machines
• Thinking Machines Lab: Tinker Cookbook