Donato Capitella Video Summaries

This discussion examines the reported US restriction on DeepSeek AI models, which appears to cover not only cloud-hosted APIs but also model weights run locally. As a cybersecurity professional who actively uses DeepSeek models, the presenter breaks down how AI models can actually be compromised, and weighs the plausibility of hidden backdoors against the practical risks involved.

■ Understanding the Threat Surface
- Risks of sending data through cloud APIs hosted in China
- Possibility of request interception and redirection to malicious models

■ Local Model Weight Security
- Historical risks of code execution during deserialization of binary weight formats
- Why the industry shifted to SafeTensors and GGML
- Remaining parser vulnerabilities in modern formats

■ Inference-Time Backdoor Vulnerabilities
- How behavioral backdoors could be trained into model weights
- Triggers tied to keywords, URLs, or specific contexts
- Why deliberately planted backdoors would be an unlikely long-term strategy

■ Practical Guidance for Users
- Distinguishing official weights and quantizations from unverified community uploads on Hugging Face
- Avoiding unauthorized tool access, credentials, and server exposure for AI agents
- Planning an isolated workspace environment with least-privilege principles

If you work with local LLMs, use AI coding agents, or download models from Hugging Face, this gives you a concrete framework for evaluating supply-chain risks and running models more safely. It is also useful context for anyone trying to understand what the DeepSeek ban does and does not actually cover.

This video introduces the newly released DeepSeek V4.1 Flash model and walks through how to set it up and run it locally. Since the update only changes model weights rather than architecture, the existing setup can be reused without modification. The presenter also shares benchmark observations and performance measurements from running the model on local hardware.

■ Model Update and Quantizations
- What changed in the new V4.1 Flash release and how it compares to the previous version
- Custom quantizations (including a low-bit version) available for download, and where to get them

■ Setup and Operation
- How to download the model weights via the AI Toolbox Cockpit interface
- Server mode configuration, context length options, and SWA-limited responses
- Enabling speculative decoding and vision features
- Running the model standalone on one device or in a coordinator/worker setup across two devices

■ Performance and Benchmarks
- Prefill and token generation speeds measured on one and two connected devices
- Early Terminal Bench Mini results comparing token usage and completion time with the previous version
- Where full benchmark results will be published

This video is aimed at users interested in running large language models locally, particularly those with compatible local hardware or an existing setup. Viewers can learn what the new release changes, how to get it running quickly, and what kinds of performance differences to expect in real-world tasks.

An updated look at running local AI models on a dual AMD Radeon AI Pro 9700 setup, covering the latest recommended models, inference engines, and performance configurations as of October 2026. The video benchmarks multiple engines tailored to specific GPU architectures and model formats, comparing token generation and prompt processing speeds across single and dual GPU configurations.

■ Recommended Models & Inference Engines
- Qwen 3.8 27B and Flash Next model families
- GGZ14 engine with speculative decoding (D Flash 2)
- R9V, Rob's VLM build, and Radiance engines
- llama.cpp with ROCm/Vulkan backends as baseline

■ Benchmark Methodology & Results
- OpenAI-compatible endpoint testing simulating agent workloads
- Single-GPU vs. dual-GPU comparisons at 32K–64K context depths
- Prompt processing and token generation speed measurements
- Multi-request concurrency testing

■ Setup & Configuration
- Server build overview (Ryzen 9 9900X3D, 64GB RAM)
- AI Toolbox Cockpit for engine installation and parameter tuning
- Quantization formats: MX FP4, FP8, dynamic quantization

Ideal for users running or considering a local AI workstation. Viewers will gain a practical understanding of which model and engine combinations deliver the best results on AMD Radeon AI Pro hardware, and how to configure them via AI Toolbox Cockpit.

2-bit quantization of DeepSeek models consistently draws skepticism, so this session breaks down why heavily quantized models can still perform well. Recorded during a walk in Greenwich, it explains the mechanics behind modern quantization recipes and reports on ongoing experiments comparing the original and quantized models on agentic tasks.

■ Why Quantization Works Better Than Expected
- Why not every weight is compressed to 2 bits: attention projections, shared experts and router gates kept at 8-bit or 16-bit
- The role of model scale and built-in redundancy
- Quantization-aware training in DeepSeek V4 and V4 1.1

■ Inside the Quantization Recipe
- Grouped quantization: how small weight groups avoid needing to cover the full BF16 range
- Magnitude pattern tables (IQ2 XXS / IQ2_XXS): mapping eight weights to a shared magnitude value plus sign storage
- Origin of the magnitude patterns and their approximation role

■ Importance Matrix and Practical Testing
- Building an importance matrix with a calibration dataset to prioritize which weights to preserve
- Difficulties of isolating quantization differences on Terminal Bench agent tasks
- Notes on sampling behavior at inference time

If you work with local LLMs, run open-weight models, or want a clearer mental model of what happens when a 16-bit model becomes a 2-bit one, this gives you the vocabulary and intuition to judge quantization claims. It also shows what is involved in benchmarking the real-world cost of quantization on agent-style workloads.

أستعرض فيه تجربتي في تطبيق دعم نموذج DeepSeek V 4.1 flash الجديد على محرك Dwarf Star، بعد أن أصبح متاحًا كحاوية جاهزة يمكن تثبيتها بسهولة عبر AI Toolbox Cockpit. أتناول طبيعة النموذج الضخم وخيارات تجهيزه للتشغيل التفاعلي على وحدة معالجة متسارعة واحدة، وما يحتاجه من إعداد موزع.

■ Streak Sailor وواجهة التشغيل
- تعريف معالج Ryzen AI Max Plus 395 من AMD بوصفه وحدة معالجة متسارعة (APU)
- الأجهزة التي يستخدم هذا المعالج، مثل Framework Desktop و AMD Ryzen AI Halo
- حجم نموذج DeepSeek V 4.1 flash ومتطلبات تشغيله التفاعلي

■ خيارات تشغيل النموذج محليًا
- بناء إعداد موزع بجهازين للوصول إلى التشغيل التفاعلي
- تشغيل النموذج على مضيف واحد باستخدام بث SSD
- تضمين النموذج في صندوق أدوات Dwarf Star وتثبيته عبر AI Toolbox Cockpit

هذه التجربة موجهة للمهتمين بالتشغيل المحلي للنماذج اللغوية الضخمة على عتاد مكتبي، والذين يبحثون عن خطوات عملية لتجهيز النموذج واستدعائه من بيئة عملهم. معها ستحصل على صورة واضحة عن متطلبات العتاد والطرق المتاحة لإشغال نموذج بهذا الحجم محليًا دون الرجوع إلى خدمات سحابية.

This episode covers adding DeepSeek V4.1 Flash support to the Dwarf Star engine, which is now available as a ready-to-run container through AI Toolbox Cockpit. I walk through how to run this ~552B-parameter mixture-of-experts model locally on one or two Strix Halo devices, and explain what changed architecturally compared to previous DeepSeek releases.

■ Architecture & Design
- Why the model name understates the change: a causal encoder / decoder split that halves prefill compute
- Attention memory reductions, including shared global keys/values and 4-bit FP4 cache storage
- N-gram tables: longer n-grams, larger tables, and where the table information is injected

■ Setup & Performance
- Quantization format used, total file size, and why a single 128 GB device needs SSD streaming
- Single-host versus two-node cluster configuration in AI Toolbox Cockpit
- Long prefill and decode speed figures for both configurations, plus Terminal Bench Mini results

■ What's Next
- Upcoming Strix Halo refresh with 192 GB unified memory and expected speculative decoding support

If you run local models, want to explore agentic and coding workloads on a single machine, or are curious about efficient large-model inference, this gives you the concrete setup steps, measured numbers, and the reasoning behind the model's design choices so you can try it yourself.

Five days of testing the new Quant 3.8 Flash Next model on the AMD Ryzen AI Halo platform based on Strix Halo are summarized here, with a focus on how the new Halogen Flash server handles the workload. The video reports throughput figures for pre-processing and generation at a long context length and checks whether that speed holds up in quality terms. It also runs an agentic coding workflow benchmark and shows where the model lands on the author's leaderboard.

■ Performance and Throughput
- Pre-processing and token generation speeds measured with the new Halogen Flash server
- Behavior at a 32K-token context depth

■ Benchmark and Workflow Testing
- Agentic coding workflow evaluation across a set of tasks
- First-attempt results and overall task completion
- Placement on the author's personal leaderboard

■ Hardware Context
- Running the model on the AMD Ryzen AI Halo (Strix Halo) processor

If you are evaluating local or edge-side LLM inference, comparing serving stacks, or following agentic coding benchmarks, this gives you a concrete reference point for speed-versus-quality tradeoffs on this hardware. Watch to the end for the full walkthrough and setup details on the author's channel.

This episode focuses on running the new open-weight model Qwen 3.8 Flash Next locally on a Strix Halo system. It covers how the model's N-gram embedding table works, compares multiple inference engines and quantization recipes, and checks whether the speed gains translate into usable agentic coding performance. A companion test on a dual Radeon RX 9700 setup is planned for a future video.

■ Model Architecture
- The N-gram embedding table and how it supplements attention with learned lookups
- The two-layer placement strategy and the gating mechanism that controls enrichment
- Trade-offs between matrix multiplication and search-based lookups, including RAM/disk handling

■ Engine and Quantization Benchmarks
- llama.cpp, the Vulkan performance fork, the Ngram Halo fork, and Halogen Flash Server
- Unsloth dynamic UD-IQ3_XS and UD-IQ4_XS quants versus Halogen's own quantization recipe
- Speed measurements of prefill and decode across multiple context depths
- Availability of multi-token prediction (NTP) across engines

■ Quality and Agent Testing
- Moving from SWE-bench mini to a new Terminal Bench Mini subset covering broader agentic workflows
- How task completion is measured and why a full setup run takes roughly 24 hours
- Current recommendations, plus planned tests including the ROCm FPX fork

■ Tools and Ecosystem
- The AI Toolbox Cockpit for downloading tested quants and launching servers
- The Framework article, updated with the latest engine recommendations
- Upcoming coverage of dual RX 9700 results and GLM 5.3 Fast

This is aimed at people running large language models locally who want a practical, benchmark-driven basis for choosing engines and quantization levels. Viewers come away with a clearer picture of how to evaluate inference setups, where to download tested configurations, and which resources to follow for ongoing updates.

Two large open-weight models were released this week: Qwen 3.8 Flash Next and GLM 5.3 Flash. This quick summary covers their architectures, quantization and hardware requirements, and the current state of llama.cpp support, ahead of a dedicated follow-up video.

■ Qwen 3.8 Flash Next
- 125B parameters in total: 6B active plus a separate 51B n-gram table
- 2-bit quantization fits 128GB; running on 64GB depends on keeping the n-gram table on disk
- llama.cpp support was merged only days ago and is still being optimized

■ GLM 5.3 Flash
- MoE model with 321B total parameters, using 18B per token
- 2-bit quantization runs on 128GB desktops, with a 3-bit option to allocate less space to context
- llama.cpp support was still in progress at the time of recording

■ Upcoming content
- A detailed video comparing both models against DeepSeek v4 flash and 1.3.8 27b

If you run language models locally, this summary lets you quickly check hardware requirements and software readiness for these releases. Use it as a reference before the full performance and quality comparison is published.

An update on running local models on 32GB and 64GB RAM systems, with a new agent-focused benchmark methodology built around API-based workflows instead of older harnesses. The video also covers recent small-model releases, toolchain updates across multiple hardware platforms, and an upcoming project that ties everything together in one interface.

■ Local LLM Benchmarks & Model Landscape
- Why 32GB and 64GB configurations get less attention than 128GB, and what can be done with them
- Recent releases: Meta's MoE 30B, upcoming Qwen 3.8 sizes, and quantized comparisons
- New benchmark methodology using API-level agent workflows with tool calls and long-context measurement
- Quantization quality testing using a 20-task subset of Terminal Bench (terminal, sysadmin, and software engineering tasks)

■ Hardware & Tooling Updates
- ROCm 7.14 updates across Strix Halo, R9700, Intel B70, and HP's GB10 system
- vLLM tensor-parallel testing on R9700, plus llama.cpp, VLM, and ComfyUI tool updates (including MiniMax H3 video generation)
- AI Toolbox Cockpit: a unified interface for platforms, engines, model downloads, and benchmark results

■ Community
- Taking over the Strix Halo Home Lab Discord and plans to highlight community projects

Recommended for people running local models on limited RAM who want realistic agent-workflow benchmarks, tooling updates, and a view of how quantization choices hold up on long-horizon tasks. Viewers can use it to decide which models, engines, and hardware setups to test on their own systems.