How I slashed 90% of API costs using this one AI model

You can probably tell from this blog post's cover image which AI model I'm talking about. And yes, DeepSeek v4 Pro is ~90% cheaper than the other frontier models. How does DeepSeek achieve this?

Here's what you need to know:

1. Mixture-of-Experts (MoE)

DeepSeek uses Mixture-of-Experts to balance model capacity with computational efficiency.

• Specialized Experts: Each feedforward block is divided into multiple "experts" (e.g., V3: 256 experts). Instead of every token passing through all parameters, only a few top-ranked experts (2–4 per token) are activated by a routing mechanism that scores token embeddings against expert embeddings.
• Routing Mechanism: A sigmoid gating function with learnable expert biases dynamically selects which experts are engaged for each token. This resolves load-imbalance problems common in MoE architectures without adding auxiliary-loss terms.
• Compute Savings: MoE allows DeepSeek to behave like a 671B parameter dense model on training while activating only ~5.5% (37B) of parameters during inference. This reduces both GPU memory usage and runtime costs, enabling long-context capabilities without prohibitive expenditure.
• Shared Experts & Redundancy: Certain popular experts can be duplicated across GPUs in deployment to avoid bottlenecks. This ensures throughput remains high under uneven token distribution.

2. KV Cache and Multi-Head Latent Attention (MLA)

Transformer models rely on storing Key (K) and Value (V) tensors for each token to compute attention efficiently over long contexts—this is the KV cache. For extremely long sequences (e.g., 128K tokens in DeepSeek), naive caching would cause memory usage to skyrocket.

MLA tackles this:

• Low-Rank Compression: Instead of caching all K/V tensors at full precision, DeepSeek projects them into a smaller latent space (c_t), drastically reducing storage (sometimes >90% memory savings).
• Query Compression: Although queries are ephemeral, MLA also compresses them slightly to maintain consistency without burdening cache memory.
• Separation of Content and Position: Rotary Positional Embeddings (RoPE) are applied independent of content representations. Keys are split into a compressed content component and a RoPE-aware positional component, ensuring positional info is preserved while keeping most KV tensors compressed.
• Decompression-on-Demand: During attention computation, latent representations are projected back to full dimensions through a lightweight reconstruction, effectively providing the same representational power at a fraction of the memory cost.
• Inference Impact: MLA enables long-context inference while keeping GPU memory requirements practical. In V3, KV cache memory was reduced by over 93% without degrading quality.

3. Cache Optimization Strategies

DeepSeek’s architectural approach not only compresses KV tensors but complements this with efficient caching strategies:

Absorb vs Naive Modes:

• Naive: Stores all expanded K/V tensors. Highly memory-intensive.
• Absorb: Stores only compressed latent representations, expanding them dynamically when needed. Reduces cache usage by nearly 98–99% in practice.
• Dynamic Context Scaling: For sequences exceeding base context lengths, attention scaling factors and positional embeddings are adjusted to preserve numerical stability while maximizing effective receptive field.

4. Synergy Between MoE and MLA

• Memory-Efficient Activation: Since only a subset of experts is active for every token (MoE), the memory saved by MLA complements this sparse computation, enabling model deployments with both enormous capacity and manageable resource usage.
• Preservation of Model Quality: Unlike grouping heads arbitrarily (e.g., GQA) or quantizing caches aggressively, MLA provides low-rank compression while still allowing attention heads to access distinct representations for diverse heads.
• Scaling to 1T Parameters (V4): With upcoming V4 models, MoE and MLA together permit ultra-high parameter counts (1T) while reducing active parameters per token (32B) and keeping memory and compute costs feasible.

5. Practical Implications

• Fine-Tuning: LoRA-like adapters can be inserted post-training into attention layers to adapt MoE and MLA without retraining the full model, balancing capacity, efficiency, and accuracy.
• High-Context Applications: For tasks like multi-document reasoning or long-session chat, MLA ensures long token histories remain manageable in GPU memory, while MoE delivers computational efficiency.
• Inference Cost: The combination allows DeepSeek to offer API pricing that is significantly lower than dense models of comparable capability while maintaining competitive performance.

These innovations make DeepSeek AI suitable for research, enterprise applications, and long-form reasoning tasks where both speed and context depth are paramount.

What can you realistically expect?

I wanted to build a project that would use enough tokens to realistically benchmark typical usage for a project. As I work for a warehouse and logistics company I decided to build a project that allow users to manage typical equipment found in a warehouse.

I called it EquipIt and you can visit the project on GitHub: (https://github.com/techno-yeti/equipit).

For EquipIt I used less than £2 of tokens, I believe this is due to the above mentioned reasons but mostly hitting the cache.

I wanted to show some screenshots for the EquipIt app to showcase the style and detail the AI came up with. I believe that DeepSeek v4 Pro is usable for commercial use (see project screenshots below):

Planned Requests Dashboard: shows the current requests planned for that day (the day can be changed with the calendar select widget).

Listing All Requests: shows all requests stored in the app (good for planning days and weeks ahead).

Printable Delivery Advice Notes: shows printable delivery advice note for the driver to give to the security gate on exiting the yard, this can only be used once).

Limiting Email Addresses: to stop other companies using the app there is an email restriction function.

There are other features that are not shown here: trailer templating, listing equipment types, user management and the help center.

Conclusion

DeepSeek’s architectural elegance lies in orchestrating sparsity (MoE) and efficient attention caching (MLA) to enable high-parameter, long-context LLMs that remain economically and operationally feasible. By compressing what needs to be stored and sparsifying what needs to be computed, DeepSeek achieves a remarkable balance between scale, speed, and accuracy, representing a state-of-the-art approach to large language model design.