DualPath: Breaking the Storage Bandwidth Bottleneck in Agentic LLM Inference
Paper
âą 2602.21548 âą Published
âą 28
AI & ML interests
None defined yet.
Recent Activity
View all activity
victorÂ
submitted a paper to Daily Papers 3 days ago
nielsrÂ
submitted a paper to Daily Papers 6 days ago
mariagranduryÂ
authored 2 papers 10 days ago
nielsrÂ
submitted a paper to Daily Papers 11 days ago
julien-cÂ
submitted a paper to Daily Papers about 1 month ago
Post
977
Interesting article: use Claude Code to help open models write CUDA kernels (for eg) by turning CC traces into Skills. They made a library out of it đ
https://huggingface.co/blog/upskill
https://huggingface.co/blog/upskill
nielsrÂ
submitted a paper to Daily Papers about 1 month ago
IlyasMoutawwakilÂ
posted an
update about 1 month ago
Post
3051
Transformers v5 just landed! đ
It significantly unifies and reduces modeling code across architectures, while opening the door to a whole new class of performance optimizations.
My favorite new feature? đ€
The new dynamic weight loader + converter. Hereâs why đ
Over the last few months, the core Transformers maintainers built an incredibly fast weight loader, capable of converting tensors on the fly while loading them in parallel threads. This means weâre no longer constrained by how parameters are laid out inside the safetensors weight files.
In practice, this unlocks two big things:
- Much more modular modeling code. You can now clearly see how architectures build on top of each other (DeepSeek v2 â v3, Qwen v2 â v3 â MoE, etc.). This makes shared bottlenecks obvious and lets us optimize the right building blocks once, for all model families.
- Performance optimizations beyond what torch.compile can do alone. torch.compile operates on the computation graph, but it canât change parameter layouts. With the new loader, we can restructure weights at load time: fusing MoE expert projections, merging attention QKV projections, and enabling more compute-dense kernels that simply werenât possible before.
Personally, I'm honored to have contributed in this direction, including the work on optimizing MoE implementations and making modeling code more torch-exportable, so these optimizations can be ported cleanly across runtimes.
Overall, Transformers v5 is a strong signal of where the community and industry are converging: Modularity and Performance, without sacrificing Flexibility.
Transformers v5 makes its signature from_pretrained an entrypoint where you can mix and match:
- Parallelism
- Quantization
- Custom kernels
- Flash/Paged attention
- Continuous batching
- ...
Kudos to everyone involved! I highly recommend the:
Release notes: https://github.com/huggingface/transformers/releases/tag/v5.0.0
Blog post: https://huggingface.co/blog/transformers-v5
It significantly unifies and reduces modeling code across architectures, while opening the door to a whole new class of performance optimizations.
My favorite new feature? đ€
The new dynamic weight loader + converter. Hereâs why đ
Over the last few months, the core Transformers maintainers built an incredibly fast weight loader, capable of converting tensors on the fly while loading them in parallel threads. This means weâre no longer constrained by how parameters are laid out inside the safetensors weight files.
In practice, this unlocks two big things:
- Much more modular modeling code. You can now clearly see how architectures build on top of each other (DeepSeek v2 â v3, Qwen v2 â v3 â MoE, etc.). This makes shared bottlenecks obvious and lets us optimize the right building blocks once, for all model families.
- Performance optimizations beyond what torch.compile can do alone. torch.compile operates on the computation graph, but it canât change parameter layouts. With the new loader, we can restructure weights at load time: fusing MoE expert projections, merging attention QKV projections, and enabling more compute-dense kernels that simply werenât possible before.
Personally, I'm honored to have contributed in this direction, including the work on optimizing MoE implementations and making modeling code more torch-exportable, so these optimizations can be ported cleanly across runtimes.
Overall, Transformers v5 is a strong signal of where the community and industry are converging: Modularity and Performance, without sacrificing Flexibility.
Transformers v5 makes its signature from_pretrained an entrypoint where you can mix and match:
- Parallelism
- Quantization
- Custom kernels
- Flash/Paged attention
- Continuous batching
- ...
Kudos to everyone involved! I highly recommend the:
Release notes: https://github.com/huggingface/transformers/releases/tag/v5.0.0
Blog post: https://huggingface.co/blog/transformers-v5
-
- 3 replies
IlyasMoutawwakilÂ
posted an
update about 1 month ago
Post
2384
After 2 months of refinement, I'm happy to announce that a lot of Transformers' modeling code is now significantly more torch-compile & export-friendly đ„
Why it had to be done đ
PyTorch's Dynamo compiler is increasingly becoming the default interoperability layer for ML systems. Anything that relies on torch.export or torch.compile, from model optimization to cross-framework integrations, benefits directly when models can be captured as a single dynamo-traced graph !
Transformers models are now easier to:
âïž Compile end-to-end with torch.compile backends
đŠ Export reliably via torch.export and torch.onnx.export
đ Deploy to ONNX / ONNX Runtime, Intel Corporation's OpenVINO, NVIDIA AutoDeploy (TRT-LLM), AMD's Quark, Meta's Executorch and more hardware-specific runtimes.
This work aims at unblocking entire TorchDynamo-based toolchains that rely on exporting Transformers across runtimes and accelerators.
We are doubling down on Transformers commitment to be a first-class citizen of the PyTorch ecosystem, more exportable, more optimizable, and easier to deploy everywhere.
There are definitely some edge-cases that we still haven't addressed so don't hesitate to try compiling / exporting your favorite transformers and to open issues / PRs.
PR in the comments ! More updates coming coming soon !
Why it had to be done đ
PyTorch's Dynamo compiler is increasingly becoming the default interoperability layer for ML systems. Anything that relies on torch.export or torch.compile, from model optimization to cross-framework integrations, benefits directly when models can be captured as a single dynamo-traced graph !
Transformers models are now easier to:
âïž Compile end-to-end with torch.compile backends
đŠ Export reliably via torch.export and torch.onnx.export
đ Deploy to ONNX / ONNX Runtime, Intel Corporation's OpenVINO, NVIDIA AutoDeploy (TRT-LLM), AMD's Quark, Meta's Executorch and more hardware-specific runtimes.
This work aims at unblocking entire TorchDynamo-based toolchains that rely on exporting Transformers across runtimes and accelerators.
We are doubling down on Transformers commitment to be a first-class citizen of the PyTorch ecosystem, more exportable, more optimizable, and easier to deploy everywhere.
There are definitely some edge-cases that we still haven't addressed so don't hesitate to try compiling / exporting your favorite transformers and to open issues / PRs.
PR in the comments ! More updates coming coming soon !
-
- 1 reply
nielsrÂ
submitted a paper to Daily Papers about 2 months ago
Post
2275
Genuine recommendation: You should really use this AutoHotKey macro. Save the file as
In my experience the past few months,
macros.ahk and run it. Before sending a prompt to your coding agent, press Ctrl + Alt + 1 and paste your prompt to any regular chatbot. Then send the output to the agent. This is the actual, boring, real way to "10x your prompting". Use the other number keys to avoid repeating yourself over and over again. I use this macro prolly 100-200 times per day. AutoHotKey isn't as new or hype as a lot of other workflows, but there's a reason it's still widely used after 17 years. Don't overcomplicate it.; Requires AutoHotkey v1.1+
; All macros are `Ctrl + Alt + <variable>`
^!1::
Send, Please help me more clearly articulate what I mean with this message (write the message in a code block):
return
^!2::
Send, Please make the following changes:
return
^!3::
Send, It seems you got cut off by the maximum response limit. Please continue by picking up where you left off.
returnIn my experience the past few months,
Ctrl + Alt + 1 works best with Instruct models (non-thinking). Reasoning causes some models to ramble and miss the point. I've just been using GPT-5.x for this. nielsrÂ
submitted a paper to Daily Papers 2 months ago
Post
2632
đš New tool for the Nymbo/Tools MCP server: The new
How it works: The tool exposes the standard discover/info/resources/validate actions. Skills live in
I've included a
Caveat: On HF Spaces,
Try it out ~ https://www.nymbo.net/nymbot
Agent_Skills tool provides full support for Agent Skills (Claude Skills but open-source).How it works: The tool exposes the standard discover/info/resources/validate actions. Skills live in
/Skills under the same File_System root, and any bundled scripts run through Shell_Command, no new infrastructure required.Agent_Skills(action="discover") # List all available skills
Agent_Skills(action="info", skill_name="music-downloader") # Full SKILL.md
Agent_Skills(action="resources", skill_name="music-downloader") # Scripts, refs, assetsI've included a
music-downloader skill as a working demo, it wraps yt-dlp for YouTube/SoundCloud audio extraction.Caveat: On HF Spaces,
Shell_Command works for most tasks, but some operations (like YouTube downloads) are restricted due to the container environment. For full functionality, run the server locally on your machine.Try it out ~ https://www.nymbo.net/nymbot
Post
3435
Nvidia is on a roll lately. Nemotron 3 Nano is my new fav local model, but here's the real flex: they published the entire evaluation setup. Configs, prompts, logs, all of it. This is how you do open models đ„
https://huggingface.co/blog/nvidia/nemotron-3-nano-evaluation-recipe
https://huggingface.co/blog/nvidia/nemotron-3-nano-evaluation-recipe
Post
3069
Muon vs MuonClip vs Muon+Adamw
Muon has gone from an experiment to a mainstream optimizer, but does it hold up for fineâtuning? We ran headâtoâhead tests on Qwen3â4B (10k+ highâquality instruction rows) to find out.
Short story: Pure Muon converged fastest at the start, but its gradientânorm spikes made training unstable. MuonClip (Kimi K2âs clipping) stabilizes long pretraining runs, yet in our smallâscale fineâtune it underperformed, lower token accuracy and slower convergence. The winner was the hybrid: Muon for 2D layers + AdamW for 1D layers. It delivered the best balance of stability and final performance and even beat vanilla AdamW.
Takeaway: for small-scale fine-tuning, hybrid = practical and reliable.
Next Step: scale to larger models/datasets to see if Muonâs spikes become catastrophic or if clipping wins out.
Full Blog Link: https://huggingface.co/blog/KingNish/optimizer-part1
Muon has gone from an experiment to a mainstream optimizer, but does it hold up for fineâtuning? We ran headâtoâhead tests on Qwen3â4B (10k+ highâquality instruction rows) to find out.
Short story: Pure Muon converged fastest at the start, but its gradientânorm spikes made training unstable. MuonClip (Kimi K2âs clipping) stabilizes long pretraining runs, yet in our smallâscale fineâtune it underperformed, lower token accuracy and slower convergence. The winner was the hybrid: Muon for 2D layers + AdamW for 1D layers. It delivered the best balance of stability and final performance and even beat vanilla AdamW.
Takeaway: for small-scale fine-tuning, hybrid = practical and reliable.
Next Step: scale to larger models/datasets to see if Muonâs spikes become catastrophic or if clipping wins out.
Full Blog Link: https://huggingface.co/blog/KingNish/optimizer-part1
Post
2677
I tested Muon vs MuonClip vs Muon+AdamW for fine-tuning LLMs
Just published a blog on that, Read here đ https://huggingface.co/blog/KingNish/optimizer-part1
Just published a blog on that, Read here đ https://huggingface.co/blog/KingNish/optimizer-part1
- 1 reply
mrfakenameÂ
posted an
update 3 months ago
Post
16353
Excited to share that I've joined the Hugging Face Fellows program! đ€
Looking forward to contributing to & working more closely with the open-source ecosystem - huge thanks to everyone who's supported me on this journey! đ
Looking forward to contributing to & working more closely with the open-source ecosystem - huge thanks to everyone who's supported me on this journey! đ
christopherÂ
authored a paper 3 months ago
reach-vbÂ
authored a paper 3 months ago