Products / PCIe Inference Card
Server-class LLM / VLA inference accelerator
IPROVF1 PCIe Inference Card
100 LLM cores on a single die beneath a 3D-stacked 64 GB memory system delivering 48 TB/s, on a standard PCIe Gen5 ×16 add-in card that drops into any server.
Pre-silicon parameters are design targets.
Key specifications
The numbers that matter for serving.
The card is balanced at about 21 FLOP per byte of memory bandwidth. Decode reaches peak compute at a batch of roughly 21, not several hundred.
Detailed engineering documentation is shared with customers under NDA through the data room.
Request the datasheet| Compute | 1000 TFLOPS FP8-E4M3 (native W8A8) / 500 TFLOPS BF16, dense |
| Organization | Single die, 100 LLM cores, 250,000 dual-mode MAC processing elements |
| Quantization | Native dynamic W8A8: per-channel weight × per-token activation scale; official FP8-dynamic checkpoints load directly |
| Attention | Native attention engine with hardware FlashAttention (online softmax, on-chip score tile) |
| Device memory | 64 GB 3D-stacked DRAM, 48.0 TB/s |
| On-chip SRAM | 128 MB, double-buffered weight and KV streaming |
| KV cache | Hardware paged KV (16-token blocks), 32k+ token context |
| Host interface | PCIe Gen5 ×16, 128 GB/s bidirectional |
| Scale-out | 1.0 TB/s die-to-die link; two devices form IPROVF2 (2000 TFLOPS FP8, 128 GB, 96 TB/s) |
| Management | SMBus; MCTP over PCIe VDM |
| Software | PyTorch / Hugging Face / vLLM integration; MediaChipset SDK: graph compiler, kernel library, runtime, HAL and kernel driver |
| Compliance | FCC Part 15 Class A; CE; BSMI; RoHS; REACH |
Positioning
Against NVIDIA H200.
Where a single IPROVF1 trails, on peak compute and capacity, two devices close the gap. One card covers models up to ≈50B and two cover larger ones, in both cases with roughly an order of magnitude more memory bandwidth than the part they replace.
| Metric | IPROVF1 | NVIDIA H200 SXM¹ | Advantage |
|---|---|---|---|
| Dense FP8 compute | 1000 TFLOPS | ≈1980 TFLOPS | IPROVF2 matches (2000) |
| Dense BF16 compute | 500 TFLOPS | ≈990 TFLOPS | IPROVF2 matches (1000) |
| Memory capacity | 64 GB | 141 GB | IPROVF2 close (128 GB) |
| Memory bandwidth | 48.0 TB/s | 4.8 TB/s | 10× |
| Compute per byte of bandwidth | 20.8 FLOP/B | ≈410 FLOP/B | 20× better balanced for decode |
| Batch-1 decode, 13B FP8 | ≈3,700 tok/s | ≈370 tok/s | 10× |
| Batch needed to reach peak | ≈21 | ≈410 | 20× smaller |
| Form factor | PCIe add-in card | SXM module | Standard server |
| Design focus | Inference | Training + inference | — |
¹ NVIDIA-published dense figures for H200 SXM. Sparsity-enabled headline numbers are 2× these and do not apply to dense inference. IPROVF1 figures are pre-silicon design targets.
Architecture highlights
Bandwidth, delivered.
A memory system built for streaming
Weights and KV cache stream from DRAM rather than living on chip, so model size is bounded by memory capacity, not on-chip storage. SRAM is spent on double-buffering, which turns bandwidth into delivered throughput.
Attention as a first-class instruction
A dedicated engine replaces the usual matrix-multiply / softmax / matrix-multiply sequence, keeping the score matrix (the largest tensor in a long-sequence layer) on die instead of writing and re-reading it several times.
Runtime-provisionable compute
The array can be allocated in 10–100 % steps at runtime for capacity partitioning or multi-tenancy, with no effect on model accuracy.
Software & validation
Runs the stack you already use.
PyTorch, Hugging Face and vLLM on top; the MediaChipset SDK (graph compiler, kernel library, runtime, HAL and kernel driver) underneath. Validated against the reference GPU implementation on LLM, vision-language and VLA models, in BF16 and FP8 W8A8.
See the software stack| Result | Measured |
|---|---|
| BF16 answer-token agreement with the GPU reference, 9 LLMs | 54/54 (100 %) |
| Vision-language-action model on a real robot image | 7/7 action tokens, action Δ = 0.0 |
| End-to-end latency vs. the GPU reference, 10 models | 1.00–1.10× |
| Prefill / decode throughput vs. the GPU reference (median) | ≈1.00× / ≈0.97× |
| FP8 GEMM speed-up over BF16, compute-bound | 1.92× |
Measured results are from the IPROVF-Sim validation stack.
Documentation
Engineering detail lives in the data room.
Customers under NDA download datasheets and engineering documentation from the data room as they are released.
Scale beyond one card
Two cards form IPROVF2 over the 1.0 TB/s die-to-die link: 2000 TFLOPS FP8, 128 GB, 96 TB/s. Or deploy them integrated in a MediaChipset AI Inference Server.