How to Run 70B & 120B Local LLMs on the MINISFORUM MS-S1 MAX-P495

Can you run a 70B or 120B large language model locally without building a workstation around multiple expensive graphics cards? The MINISFORUM MS-S1 MAX-P495 is designed to make that kind of local AI workflow more accessible. With an AMD Ryzen™ AI Max+ PRO 495 processor, 192GB of LPDDR5X-8533 unified memory, and up to 160GB of allocatable VRAM, it gives developers room to run large quantized models on a compact workstation.

The practical steps matter as much as the specifications. This guide explains how to plan memory, allocate shared graphics memory in BIOS, and get started with local inference on the Windows 11 system supplied with the P495.

1. Can the MINISFORUM MS-S1 MAX-P495 Run 70B and 120B Parameter LLMs Locally?

Yes. The MINISFORUM MS-S1 MAX-P495 can run suitable quantized 70B and 120B-class language models locally. Its 192GB unified memory and up to 160GB allocatable VRAM provide substantial capacity for model weights, KV Cache, and inference buffers. Real-world performance depends on the model, quantization, context length, and inference software.

Calculate memory requirements before downloading a model

Start with a simple estimate for the model weights:

Approximate weight memory = parameter count × bits per parameter ÷ 8

At an idealized four bits per parameter, a 70-billion-parameter model needs about 35GB for its weights alone. A 120B model at the same nominal precision needs about 60GB. Actual model files differ because quantization formats include metadata, scales, and sometimes higher-precision tensors.

Plan additional memory for:

• KV Cache, which stores attention data for the active context window.
• Inference buffers and temporary tensors used during generation.
• Windows 11, background applications, and other system processes.
• Longer prompts or multiple concurrent requests, which can increase memory use.

A model file fitting into the available memory is a starting point, not the whole calculation. Context length and runtime overhead must also be considered.

Compare the memory headroom of different workstations

This table compares approximate capacity and likely model fit. Q4 and Q8 refer to common quantization levels; exact file sizes and runtime memory use depend on the model and implementation.

Workstation configuration	Allocatable / physical memory	70B Q4	70B Q8	120B Q4
MS-S1 MAX (128GB Memory)	Up to 96GB allocatable VRAM	Generally feasible	Often feasible	Model and context dependent
MS-S1 MAX-P495 (192GB Memory)	Up to 160GB allocatable VRAM	Generally feasible	Generally feasible	Generally feasible

 

A dual-GPU workstation does not automatically behave like a single 48GB GPU. Model placement and multi-GPU support depend on the inference engine and system configuration. The P495’s advantage is its large unified memory pool, which gives a single system more room for large quantized model weights and long-context workloads.

What about 284B MoE models?

Mixture-of-Experts (MoE) models activate only part of their parameter set for each token, which can reduce computation compared with a similarly sized dense model. The full model weights still need to be available to the inference system.

The MINISFORUM MS-S1 MAX-P495 has also been tested with DeepSeek V4 Flash 284B in Q4 quantization, using approximately 144GB of model memory and achieving a measured generation speed of 12.35 tokens per second (tok/s). This is a practical example of how quantization and unified memory can make a very large MoE model usable on a single compact workstation.

Benchmarks should always be interpreted alongside the exact model build, quantization, context length, inference engine, and measurement method. These details help readers reproduce results and compare like with like.

A useful question before choosing a model: do you need the largest possible parameter count, or do you need a specific balance of reasoning quality, context length, and response speed? The best model for a workflow is the one that meets its requirements—not necessarily the one with the largest headline number.

2. How Do You Allocate 160GB of VRAM on the MINISFORUM MS-S1 MAX-P495?

The MS-S1 MAX-P495 uses unified memory architecture (UMA). Its CPU and integrated Radeon graphics share system memory, and up to 160GB can be allocated as graphics memory. For local LLM inference, the BIOS memory setting is an important part of preparing the system.

Step 1: Set the UMA memory allocation in BIOS

Use the following path on AMD-based MINISFORUM models such as the P495:

1. Restart the workstation and press Delete to enter BIOS Setup.
2. Open Setup → Advanced → AMD CBS → NBIO Common Options → GFX Configuration.
3. Select iGPU Configuration and set it to UMA Specified.
4. Set UMA Frame Buffer Size to the capacity required for your workload, such as 160GB.
5. Save the changes and restart the system.

After Windows starts, check that your graphics and inference software can access the expected memory allocation. For a 70B or 120B model, leave enough headroom for the operating system, runtime buffers, and KV Cache. Choose the allocation based on the model and context you intend to run.

Step 2: Confirm your Windows 11 environment

The P495 ships with Windows 11, so you can begin without replacing the operating system or changing Linux kernel settings. Before installing an inference engine:

• Install available Windows updates and the supported AMD graphics driver for the system.
• Keep sufficient free space on the SSD for model downloads; large models can occupy tens or more than one hundred gigabytes.
• Close unnecessary memory-intensive applications before loading a large model.
• Restart after changing BIOS settings, then launch the inference software and verify that the GPU backend is available.

For this guide, the recommended first step is Ollama on Windows. Users who want more direct control can also use a Windows-compatible llama.cpp build.

3. What Is the Best Inference Engine for AMD Ryzen AI Max+ PRO 495: Ollama or llama.cpp?

For a Windows 11 desktop workflow, Ollama is a straightforward way to download and run supported models. llama.cpp is a good alternative for users who want more control over GGUF model files, GPU layer offloading, and command-line parameters. The right choice depends on the model format and the backend supported by the installed build.

Engine	Best fit	Why choose it
Ollama for Windows	Quick local setup and model management	Simple model commands and a local API
llama.cpp for Windows	GGUF models and detailed inference tuning	Direct control over context, GPU offloading, and runtime options

 

Step 1: Install Ollama and run a 70B or 120B model

Install the Windows version of Ollama from its official website. Once installation is complete, open PowerShell or Windows Terminal and run a model command for a model available in the Ollama library.

ollama run llama3.3:70b

For a larger model, the following example launches the 120B-class gpt-oss model when the corresponding library entry is available:

ollama run gpt-oss:120b

The first launch downloads the model, so allow time for the download to complete. Check the available disk space beforehand. Model tags and quantization options can change, so confirm the current library entry before starting a large download.

Step 2: Set the context length with a Modelfile

Ollama supports the num_ctx parameter in a Modelfile. The example below creates a local configuration with an 8,192-token context. Adjust the value to match the model and workload you want to run.

FROM gpt-oss:120b

PARAMETER num_ctx 8192

Save this text as a file named Modelfile in a working folder. Then run these commands in that folder:

ollama create gpt-oss-120b-local -f Modelfile

ollama run gpt-oss-120b-local

A larger context can improve the amount of text the model can consider at once, but it also increases KV Cache memory use. Start with a context suitable for your task, then tune it for longer documents or extended conversations.

Step 3: Use llama.cpp for direct control

If you prefer GGUF files and more direct runtime control, install a Windows-compatible llama.cpp build with a supported GPU backend. Download builds from the project’s official releases and follow the instructions for the specific build you choose.

A typical command-line pattern for a compatible GGUF model is:

llama-cli.exe -m "D:\Models\model.gguf" -ngl 999 -c 8192 -p "Explain how local LLM inference works."

The -ngl 999 option requests offloading as many model layers as the backend can place on the GPU. The actual placement is shown in the startup output and depends on the model and build. Replace the example file path with the location of your own GGUF model.

For a fair comparison, keep the model file, quantization, prompt, context length, and generation settings consistent. Record prompt-processing speed and generation speed separately; they measure different parts of the inference workload.

Step 4: Understand the role of the NPU

The AMD Ryzen™ AI Max+ PRO 495 includes a dedicated NPU rated at up to 55 TOPS. This is a separate specification from the processor’s total AI compute rating. NPU acceleration requires compatible software and supported operations; ordinary Ollama or llama.cpp LLM inference does not automatically move to the NPU.

The NPU can complement the main inference workload in supported applications, such as local speech processing or other compatible AI tasks. For a reliable setup, verify software support for the exact workload rather than assuming every AI application will use the NPU automatically.

4. How Can You Scale the MS-S1 MAX-P495 for 300B+ and 671B Parameter Models?

For models with hundreds of billions of parameters, multi-node inference can distribute model work across several systems. A 671B model stored at an idealized four bits per parameter represents approximately 335.5GB of raw weight data before metadata, cache, and runtime overhead. Distributing the workload across multiple systems can provide additional memory capacity.

The MS-S1 MAX-P495 includes dual 10GbE networking and USB4 V2 connectivity. Multi-node inference software, including distributed features available in llama.cpp, can be used to build a larger local AI setup. The exact model placement and performance depend on the inference framework, network configuration, and model format.

Plan and measure the cluster

Before expanding to a 300B+ model, define the intended workload and benchmark a smaller model across the planned nodes. Measure:

• Model loading time and peak memory use on each node.
• Prompt-processing throughput and token generation speed.
• Network utilization and communication latency.
• Response latency under the expected number of simultaneous requests.

These measurements help identify whether memory capacity, compute, or inter-node communication is limiting performance. They also provide a reproducible basis for scaling the setup to a larger model.

5. FAQs

1. Does memory bandwidth affect local LLM token speed?

Yes. During token generation, the system repeatedly accesses model weights, so memory bandwidth can strongly influence workloads that are memory-bound. The P495’s approximately 273GB/s memory bandwidth is an important hardware specification, although real token speed also depends on quantization, model architecture, context length, and inference software.

2. How does the P495’s power use compare with a multi-GPU server?

The P495 is a compact system designed to run local AI workloads without the footprint of a large multi-GPU workstation. MINISFORUM specifies 120W sustained-load power and 160W peak power for this configuration. Actual wall power varies by workload and system settings, so compare measurements taken under equivalent conditions.

3. Can I expand the P495 with a discrete GPU or additional SSD storage?

The P495 is not marketed as supporting discrete GPU expansion. For storage expansion, the system provides M.2 SSD slots; check the official product specifications for the supported slot configuration and drive compatibility. Choosing a fast, sufficiently large SSD is useful when storing multiple large model files.

4. Can the P495 run a 122B model with a 131K context window?

Yes. The P495 has been demonstrated running a 122B model with a 131K-token context window. This is a useful example of how the system’s 192GB unified memory can support large-model workloads with extended context. Actual performance will vary with the specific model, quantization, inference engine, and test configuration.

5. Can the P495 run DeepSeek and other large MoE models?

Yes. In the reported tests, DeepSeek-V4-Flash-0731-UD-Q4_K_XL achieved 12.35 tok/s, while Qwen3.8-Flash-Next-GUF-UD_Q4_K_XL achieved 26.48 tok/s. These figures illustrate that different model architectures and implementations can produce different generation speeds on the same workstation. For reproducible comparisons, publish the test context, software version, and measurement method alongside the tok/s result.

Build Your Local AI Workflow with the MINISFORUM MS-S1 MAX-P495

Running a large language model locally is a combination of memory planning, software selection, and repeatable testing. The MINISFORUM MS-S1 MAX-P495 brings an AMD Ryzen™ AI Max+ PRO 495 processor, 192GB of unified memory, and up to 160GB of allocatable VRAM into a compact workstation built for demanding local AI workloads.

Start by configuring UMA memory in BIOS, then install an inference engine on the preinstalled Windows 11 system. Choose a quantized model that matches your workload, set an appropriate context length, and record generation speed under repeatable conditions. From 70B models to tested 284B MoE workloads, this step-by-step approach helps turn memory capacity into a practical local AI workflow.

Ready to get started? Visit the official MINISFORUM MS-S1 MAX-P495 product page to review the specifications and configuration details for your local AI workstation.