Reports
AI-generated structured vendor updates
NVIDIA Shifts AI Infrastructure Metric from FLOPS to Cost Per Token
NVIDIA advocates for "cost per token" as the primary economic metric for AI infrastructure, replacing "FLOPS per dollar." This shift moves the focus from computational inputs to business outputs, requiring full-stack optimization across hardware, software, and networking to lower enterprise AI inference TCO.
Google Launches Gemma 4 Open Models, Targeting Edge Inference and AI Agent Architecture
Google introduces the Gemma 4 open model family, with four sizes from 2B to 31B parameters, emphasizing breakthrough intelligence-per-parameter and native support for agentic workflows, multimodality, and long context. The small models are engineered for edge devices, aiming to bring frontier reasoning to mobile and IoT scenarios.
Google Launches Gemma 4 Open Model Family
Google introduces Gemma 4 open model family with four size variants, optimized for edge and mobile devices. The series supports multimodal processing, long context windows and 140+ languages under Apache 2.0 license.
AMD Announces Breakthrough MLPerf Inference 6.0 Results, Showcasing Multinode Scaling and Multimodal Capabilities
AMD's MLPerf Inference 6.0 submission, powered by Instinct MI355X GPUs, surpassed 1 million tokens per second for the first time on models like Llama 2 70B and GPT-OSS-120B. The results highlight efficient multinode scaling, rapid enablement of new workloads (e.g., text-to-video model Wan-2.2-t2v), and reproducible performance across a broad partner ecosystem.
NVIDIA IGX Thor: 8x Edge AI Compute with ConnectX-7 Network Lock-In
NVIDIA launches IGX Thor edge AI platform with Blackwell GPU, up to 5,581 FP4 TFLOPS, dual 200GbE RDMA via ConnectX-7, and ISO 26262 safety. Pin-compatible with Jetson Thor and 10-year lifecycle enable seamless migration, but create vendor lock-in through proprietary networking and GPU dependencies.
Meta Accelerates Custom AI Chip Roadmap with Focus on Inference Optimization
Meta plans to launch four generations of MTIA AI chips in two years, adopting an 'inference-first' design strategy optimized for generative AI tasks. Built on PyTorch and open standards, the chips enable seamless data center deployment, targeting improved compute efficiency and cost control.
NVIDIA Jetson Advances Localized Deployment of Open-Source AI Models at Edge
NVIDIA's Jetson edge AI platform enables localized deployment of open-source generative AI models like Qwen3 4B and Mistral 3 on edge devices. The platform offers a complete hardware range from Jetson Orin Nano to Thor, integrating compute and memory in SoM for simplified design. Key performance shows Jetson Thor achieves 52 tokens/sec for Mistral 3 inference.
Trend Micro Report Highlights AI Supply Chain Risks and Model Attack Surfaces
Trend Micro's 'Fault Lines in the AI Ecosystem' report systematically analyzes security risks in the AI supply chain, including training data poisoning, third-party plugin vulnerabilities, and model theft attacks. It indicates that enterprise AI security boundaries have expanded from traditional IT infrastructure to the model layer and data pipelines.
SGLang 0.5.13 Delivers 25x MoE Inference Speedup via Predictive Routing and Sparse KV Cache
SGLang 0.5.13 introduces two-stage MoE routing prediction and sparse KV cache, achieving a 25x inference speedup on NVIDIA GB300 NVL72. Benchmarks on A100 show 65% throughput gain, 40% latency reduction, and 62% lower routing overhead. This optimization directly attacks the core bottleneck of MoE inference, potentially reshaping AI inference economics.