Benchmarking Qwen 3.8 Flash Next on Strix Halo: Engines, Quants, and Agentic Performance

This episode focuses on running the new open-weight model Qwen 3.8 Flash Next locally on a Strix Halo system. It covers how the model's N-gram embedding table works, compares multiple inference engines and quantization recipes, and checks whether the speed gains translate into usable agentic coding performance. A companion test on a dual Radeon RX 9700 setup is planned for a future video.

■ Model Architecture
- The N-gram embedding table and how it supplements attention with learned lookups
- The two-layer placement strategy and the gating mechanism that controls enrichment
- Trade-offs between matrix multiplication and search-based lookups, including RAM/disk handling

■ Engine and Quantization Benchmarks
- llama.cpp, the Vulkan performance fork, the Ngram Halo fork, and Halogen Flash Server
- Unsloth dynamic UD-IQ3_XS and UD-IQ4_XS quants versus Halogen's own quantization recipe
- Speed measurements of prefill and decode across multiple context depths
- Availability of multi-token prediction (NTP) across engines

■ Quality and Agent Testing
- Moving from SWE-bench mini to a new Terminal Bench Mini subset covering broader agentic workflows
- How task completion is measured and why a full setup run takes roughly 24 hours
- Current recommendations, plus planned tests including the ROCm FPX fork

■ Tools and Ecosystem
- The AI Toolbox Cockpit for downloading tested quants and launching servers
- The Framework article, updated with the latest engine recommendations
- Upcoming coverage of dual RX 9700 results and GLM 5.3 Fast

This is aimed at people running large language models locally who want a practical, benchmark-driven basis for choosing engines and quantization levels. Viewers come away with a clearer picture of how to evaluate inference setups, where to download tested configurations, and which resources to follow for ongoing updates.