Add Plug-in Support: Proposal for CLAP Support

[pljones]

What is the current behaviour and why should it be changed?

Currently there is no ability for the Jamulus user to add audio processing, other than the built-in Reverb in the client and Jam Recorder in the server. There are several other proposals to extend audio processing, including additional effects and Jam Streaming.

So there is a choice of whether each of these feature should be added and maintained inside Jamulus, duplicating functionality available elsewhere or whether to adopt a means of adding these features to Jamulus on a plug-in basis.

Describe possible approaches

A number of plug-in APIs are available. However, many are either platform-specific or have restrictive licences.

The proposal here is to adopt CLAP both on the Client and the Server so that features can be added:

• In the Client
  • To audio input prior to sending to the server
  • To audio received from the server before output
• In the Server
  • To receive a pre-mixer stream from each connected client during a session (opening the server's configured max clients stereo pairs)
  • To receive a post-mixer single stereo pair (potentially increasing the number of stereo pairs by one)

https://cleveraudio.org/developers-getting-started/
https://github.com/free-audio/clap-host?tab=readme-ov-file#minimal-clap-host

Has this feature been discussed and generally agreed?

See https://github.com/orgs/jamulussoftware/discussions/3382#discussioncomment-10743921

[ann0see]

Thanks. I have looked at how the reverb is implemented in Jamulus and it would be great if we could have a thin plugin layer between CLAP and Jamulus similar to the reverb functionality.

The question is also:

1. How the user would add plugins
2. How the user would control plugins (Menu entry?)
3. If it's possible to rewrite the recorder as CLAP plugin - or if that's needed.

[ann0see]

Kind of a odd comment, I know but this guy on YouTube: https://m.youtube.com/watch?v=U1P85VzO6Dk may be worth collaborating with if we use CLAP. He seems to be interested in pushing CLAP forward.

[pljones]

How the user would add plugins

The normal way is some kind of "scan" of well known locations (these are documented on the linked site), plus user-configurable paths, plus manual addition.

How the user would control plugins (Menu entry?)

Plugins are expected to provide their own controls. In the client, I'd envisage replacing the existing reverb fader with some kind of "rack" control with a button to open the UI to add, remove or control each plug in.

If it's possible to rewrite the recorder as CLAP plugin - or if that's needed.

I've mentally noted to do this as a first "proof of concept" on the server side. I'd also like to get @dingodoppelt's Jam Streamer working the same way.