Filter Posts
Deep Learning LLM AI Research
Attention Residuals: How Kimi Rethinks Depth-Wise Information Flow in LLMs
Deep Learning
Attention Residuals: How Kimi Rethinks Depth-Wise Information Flow in LLMs

Kimi's Attention Residuals paper proposes replacing fixed residual connections with learned softmax attention over depth, a simple but powerful idea that yields 1.25x …

March 20, 2026 7min
Attention Mechanisms Compared: Standard, Linear, and Flash
Deep Learning
Attention Mechanisms Compared: Standard, Linear, and Flash

A deep dive comparing standard softmax attention, linear attention, and Flash Attention: their math, complexity, trade-offs, and when to use each.

March 11, 2026 5min
Gemini 3 Pro sets new AI benchmarks
LLM
Gemini 3 Pro sets new AI benchmarks

Google DeepMind unveils Gemini 3 Pro, their most intelligent model to date, outperforming GPT-5.1 and Claude 4.5 across key benchmarks with breakthrough multimodal …

November 18, 2025 9min
DeepSeek-R1 reaches Nature with pure reinforcement learning
AI Research
DeepSeek-R1 reaches Nature with pure reinforcement learning

An in-depth analysis of DeepSeek-R1's groundbreaking Nature publication: achieving GPT-4 level performance with pure reinforcement learning at just $294K training cost

September 21, 2025 6min
ChatGPT vs Claude: different users, different uses
LLM
ChatGPT vs Claude: different users, different uses

Studies from OpenAI and Anthropic reveal differences in how users interact with ChatGPT and Claude, suggesting complementary rather than competing AI ecosystems.

September 17, 2025 6min
How people really use ChatGPT
LLM
How people really use ChatGPT

OpenAI reveals fascinating insights into how millions of users interact with ChatGPT, showing a shift toward personal and exploratory use cases rather than work-focused …

September 16, 2025 6min
Jiufeng Li
Jiufeng Li

ML/AI Engineer & Data Scientist sharing insights on autonomous AI agents, LLM deployment, and enterprise ML systems

Learn More
Recent Posts
Attention Residuals: How Kimi Rethinks Depth-Wise Information Flow in LLMs
Mar 20, 2026
Attention Mechanisms Compared: Standard, Linear, and Flash
Mar 11, 2026
Gemini 3 Pro sets new AI benchmarks
Nov 18, 2025
DeepSeek-R1 reaches Nature with pure reinforcement learning
Sep 21, 2025
ChatGPT vs Claude: different users, different uses
Sep 17, 2025
Popular Tags
ai-breakthrough (2) ai-comparison (1) ai-usage (1) anthropic (1) attention (2) benchmarks (1) chatgpt (2) claude (1) consumer-ai (1) deep-learning (2) deepmind (1) deepseek (1) determinism (1) flash-attention (1) gemini (1) gemini 3 pro (1) google (1) gpu (1) kimi (1) linear-attention (1) llm (3) machine-learning (1) moonshot-ai (1) multimodal (1) nature (1) openai (2) optimization (1) productivity (1) reasoning (1) reinforcement-learning (1) reproducibility (1) research (1) residual-connections (1) transformer (2) usage-patterns (1)