Products / AI Inference Server
Integrated inference system
MediaChipset AI Inference Server
IPROVF1 cards integrated, validated and shipped with the serving stack installed. Rack it, load your checkpoints and serve, with ten times the memory bandwidth per accelerator of training-class GPUs.
Pre-silicon: system figures are aggregates of IPROVF1 design targets.
Configurations
Four cards or eight.
Every accelerator gets its own PCIe Gen5 ×16 link. Run each card as an independent model endpoint, or pair cards as IPROVF2 for 50B–100B models.
| 4-card configuration | 8-card configuration | |
|---|---|---|
| Accelerators | 4 × IPROVF1 | 8 × IPROVF1 |
| FP8 dense compute | 4 PFLOPS | 8 PFLOPS |
| BF16 dense compute | 2 PFLOPS | 4 PFLOPS |
| Device memory | 256 GB | 512 GB |
| Aggregate memory bandwidth | 192 TB/s | 384 TB/s |
| On-chip SRAM | 512 MB | 1 GB |
| Host links | 4 × PCIe Gen5 ×16 | 8 × PCIe Gen5 ×16 |
| Independent 3B–20B endpoints | up to 4 | up to 8 |
| 50B–100B models on IPROVF2 pairs | up to 2 | up to 4 |
| Batch-1 decode, 13B FP8, one model per card | ≈14,800 tok/s | ≈29,600 tok/s |
Aggregates are per-card IPROVF1 design targets multiplied by the card count. Decode throughput is the bandwidth-bound ceiling. Host processor, system memory, storage, networking and power options are configured to order; see the server documentation in the data room.
Workloads
Made for interactive serving.
Each card reaches peak compute at a batch of about 21, so the server stays efficient without queueing hundreds of requests.
Chat & copilots
Low time-per-token for each user, with no batching penalty on latency.
Multi-step agents
Sequential tool calls and reasoning chains where every step waits on decode.
Vision-language
Image-and-text models are part of the validated model set.
Robotics control
A vision-language-action model validated to 7/7 action tokens on a real robot image, action Δ = 0.0.
Shipped ready to serve
Software installed, models validated.
The server ships with the IPROVF kernel driver, HAL and runtime installed: the MediaChipset SDK alongside PyTorch, Hugging Face and vLLM serving. Bring your FP8-dynamic or BF16 checkpoints.
- MediaChipset SDK
- Graph compiler
- Kernel library
- PyTorch
- vLLM
- Hugging Face
Multi-tenant by design
Partition each card's compute in 10–100 % steps at runtime to host several tenants or models, with no effect on accuracy.
Fleet management
Accelerators expose SMBus and MCTP over PCIe VDM for out-of-band monitoring through the server's management controller.
Compliance
Accelerator cards target FCC Part 15 Class A, CE, BSMI, RoHS and REACH.
Next steps
Size a system for your models.
Tell us which models, context lengths and concurrency you serve. We'll come back with a configuration and an evaluation plan.