Skip to content

optimize ace_step: completely remove torch runtime dependency + unified conversion pipeline#577

Merged
lucasnewman merged 2 commits into
Blaizzy:pc/add-acefrom
mm65x:pc/enhance-ace-step
Mar 21, 2026
Merged

optimize ace_step: completely remove torch runtime dependency + unified conversion pipeline#577
lucasnewman merged 2 commits into
Blaizzy:pc/add-acefrom
mm65x:pc/enhance-ace-step

Conversation

@mm65x

@mm65x mm65x commented Mar 15, 2026

Copy link
Copy Markdown
Contributor

Description

this PR builds directly on the massive native MLX implementation from #499, optimizing the ace_step implementation by completely removing the need for PyTorch at runtime, and adding a robust unified conversion script.

i noticed two small things missing from the branch that this PR addresses:

  1. the ace_step generation module still fell back to import torch just to load the silence_latent.pt tensor, which technically means users still need a massive PyTorch installation just to run inference.
  2. there wasn't a standard convert.py script for users to pull the raw HuggingFace weights and quantize/package them into a neat safetensors directory format locally.

Changes in the codebase

  • created a robust convert.py that hooks directly into the upstream HuggingFace repository to extract the DiT, the VAE, and the Text Encoder.
  • the conversion script intercepts the PyTorch meta tensors and automatically converts the silence_latent.pt into a pure silence_latent.npy numpy array.
  • removed the import torch runtime dependency in ace_step.py completely, hooking it up to read the npy file via numpy instead!

this makes the generation pipeline 100% native PyTorch-free at runtime!

@mm65x mm65x changed the title optimize ace_step: completely remove torch runtime dependency + unified conversion pipeline optimize ace_step: remove torch runtime dependency + add conversion pipeline Mar 15, 2026
@mm65x mm65x changed the title optimize ace_step: remove torch runtime dependency + add conversion pipeline optimize ace_step: completely remove torch runtime dependency + unified conversion pipeline Mar 15, 2026
@lucasnewman

Copy link
Copy Markdown
Collaborator

@mm65x Any change you can put the converted model in mlx-community on the HF hub? It would be great if we have a standard repo id that "just works" when you use it.

Comment thread mlx_audio/tts/models/ace_step/convert.py Outdated
@mm65x
mm65x force-pushed the pc/enhance-ace-step branch from d081433 to 1386580 Compare March 20, 2026 10:14
@mm65x

mm65x commented Mar 21, 2026

Copy link
Copy Markdown
Contributor Author

done

@mm65x

mm65x commented Mar 21, 2026

Copy link
Copy Markdown
Contributor Author

uploaded the converted model here: https://huggingface.co/mlx-community/ACE-Step-v1.5-turbo-mlx

repo id is mlx-community/ACE-Step-v1.5-turbo-mlx if you want to point docs/examples at a standard hosted MLX repo.

@Blaizzy

Blaizzy commented Mar 21, 2026

Copy link
Copy Markdown
Owner

Please name it by dtype or quant type

mlx-community/ACE-Step-v1.5-turbo-bf16

Or

...-4bit

The convert script already adds tags that identify the model as MLX 👌🏽

@mm65x

mm65x commented Mar 21, 2026

Copy link
Copy Markdown
Contributor Author

done

@lucasnewman lucasnewman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@lucasnewman
lucasnewman merged commit 903d66e into Blaizzy:pc/add-ace Mar 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants