Buying more GPUs is no longer a complete AI infrastructure strategy.
多買 GPU,已經不再是一套完整的 AI 基礎設施策略。
Three announcements from the past 24 hours point in the same direction. NVIDIA open-sourced NVIDIA Snapshot, which restores initialized GPU inference workloads in 3.6 to 42.7 seconds, 8x to 36x faster than a cold start. In NVIDIA’s tests, the largest MoE models went from 6–19 minute cold starts to under 43 seconds.
過去 24 小時的三則消息,都指向同一個方向。NVIDIA 開源了 NVIDIA Snapshot,能在 3.6 到 42.7 秒內恢復已初始化的 GPU 推論工作負載,比冷啟動快 8 到 36 倍。在 NVIDIA 的測試中,最大的 MoE 模型從 6 到 19 分鐘的冷啟動,縮短到 43 秒以內。
AMD made its 6th-gen EPYC SP7, with up to 256 cores, generally available and is rolling out Helion racks for agentic AI. Agents need plenty of CPUs for orchestration, tool calls, and sandboxes, not just GPUs for inference. Here in Taipei, MSI also announced SP7 platforms, including a 6U liquid-cooled server supporting up to 8 GPUs with 400G networking.
AMD 第六代 EPYC SP7(最多 256 核心)已正式供貨,並陸續推出專為代理式 AI 打造的 Helion 機櫃。AI 代理除了需要 GPU 做推論,也需要大量 CPU 來處理流程調度、工具呼叫和沙盒。在台北,MSI 也發表了 SP7 平台,包括一款支援最多 8 張 GPU、搭配 400G 網路的 6U 液冷伺服器。
The next edge isn’t who owns the most GPUs, but who keeps them busy. GPUs waiting on cold starts, underpowered host CPUs, or cooling limits quietly burn budget.
下一個優勢不在誰擁有最多 GPU,而在誰能讓 GPU 持續有效運作。等待冷啟動的 GPU、效能不足的主機 CPU,或散熱限制,都在默默燒掉預算。
At NexCore.tw Corporation, we start from utilization, then size CPU, memory bandwidth, networking, and cooling around it.
在 NexCore.tw Corporation,我們從利用率出發,再依此配置 CPU、記憶體頻寬、網路和散熱。