0.14.0

.devcontainer/devcontainer.json

@@ -30,7 +30,9 @@
 				"dbaeumer.vscode-eslint",
 				"unifiedjs.vscode-mdx",
 				"bradlc.vscode-tailwindcss",
-				"TakumiI.markdowntable"
+				"TakumiI.markdowntable",
+				"YoavBls.pretty-ts-errors",
+				"bowlerr.sqlite-intelliview-vscode"
 			]
 		}
 	},

.github/workflows/testAndSanity.yml

@@ -18,11 +18,16 @@ jobs:
     name: Tests / Build / Sanity Run
     runs-on: ubuntu-latest
     steps:
-      - name: Check out the repo
+      - name: Check out the repo (gha)
+        if: ${{ !env.ACT }}
         uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd #v6.0.2
         with:
           ref: ${{ inputs.ref }}
           persist-credentials: false
+
+      - name: Check out the repo (act)
+        if: ${{ env.ACT }}
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd #v6.0.2
       - name: Use Node.js
         uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f #6.3.0
         with:

.gitignore

@@ -151,7 +151,10 @@ tmp-*
 
 *storybook.log
 storybook-static
-lib
+
+*.db
+*.db.*
+*.db-*lib
 *.db
 *.db.*
 *.db-*lib

.vscode/extensions.json

@@ -0,0 +1,6 @@
+{
+    "recommendations": [
+        "sanjulaganepola.github-local-actions",
+        "github.vscode-github-actions"
+    ]
+}

docsite/docs/configuration/configuration.mdx

@@ -245,11 +245,8 @@ WARN : [App] [Sources] [spotify Secrets] Matched: None | Unmatched: SPOTIFY_SECR
 
 </DetailsAdmo>
 
-## Application Options
 
-These options affect multi-scrobbler's behavior and are not specific to any source/client.
-
-### Base URL
+## Base URL
 
 Defines the URL that is used to generate default redirect URLs for authentication on [spotify](/configuration/sources/spotify) and [lastfm](/configuration/clients/lastfm) -- as well as some logging hints.
 
@@ -280,197 +277,96 @@ Useful when running with [docker](../installation/installation.mdx#docker) so th
 
 </details>
 
-### Caching
-
-Multi-scrobbler caches some activities to persist important data across restarts, reduce external API calls, and make some actions faster.
-
-All of the activities below are **always** cached **in-memory** with an optional, configurable [**secondary** store](#secondary-caching-configuration) for persistence.
-
-<Tabs groupId="cachedThings" queryString>
-<TabItem value="scrobbles" label="Scrobbles">
-
-**Queued** and **Failed** Scrobbles are cached so that any un-scrobbled data you have is persisted across restarts of multi-scrobbler.
-
-:::tip
-
-By default, this data use a [Secondary](#secondary-caching-configuration) [File](./?cacheType=file#secondary-caching-configuration) store, configured for you automatically.
-
-If you have configured a [persisted volume/bind mount](/installation?dockerSetting=storage#recommended-settings) for configuration (`/config` is mounted in [docker compose](/quickstart#create-docker-compose-file)) then you are already done. If you are not persisting this directory then you should consider setting up [Valkey Cache](./?cacheType=valkey#secondary-caching-configuration) for this.
-
-:::
-
-##### Configuration
-
-Use any [Secondary Cache](#secondary-caching-configuration), the config examples below show the default values:
-
-<Tabs>
-<TabItem value="env" label="ENV">
-
-| Environmental Variable | Required? | Default   | Description           |
-| :--------------------- | --------- | --------- | --------------------- |
-| `CACHE_SCROBBLE`       | No        | `file`    | The cache type to use |
-| `CACHE_SCROBBLE_CONN`  | No        | `/config` |                       |
+## Caching
 
-</TabItem>
+Multi-scrobbler implements caching to persist important data across restarts, reduce external API calls, and make some actions faster.
 
-<TabItem value="aio" label="AIO">
+A default **in-memory** cache store is used so that you always benefit from some caching. An optional, [**secondary** store](#secondary-caching) can be configured for greater caching capabilities.
 
-```json5 title="config.json"
-{
-    "cache": {
-        "scrobble": {
-            "provider": "file",
-            "connection": "/config"
-        }
-    },
-    // ...
-}
-```
-</TabItem>
-</Tabs>
+<DetailsAdmo type="note" summary="What is Cached?">
 
-</TabItem>
+<Tabs groupId="cachedThings">
 <TabItem value="auth" label="Auth Data">
-
 Authentication sessions/tokens/etc... are cached for quicker requests and for persistence across restarts.
-
-:::tip
-
-By default, this data use a [Secondary](#secondary-caching-configuration) [File](./?cacheType=file#secondary-caching-configuration) store, configured for you automatically.
-
-If you have configured a [persisted volume/bind mount](/installation?dockerSetting=storage#recommended-settings) for configuration (`/config` is mounted in [docker compose](/quickstart#create-docker-compose-file)) then you are already done. If you are not persisting this directory then you should consider setting up [Valkey Cache](./?cacheType=valkey#secondary-caching-configuration) for this.
-
-:::
-
-##### Configuration
-
-Use any [Secondary Cache](#secondary-caching-configuration), the config examples below show the default values:
-
-<Tabs>
-<TabItem value="env" label="ENV">
-
-| Environmental Variable | Required? | Default   | Description           |
-| :--------------------- | --------- | --------- | --------------------- |
-| `CACHE_AUTH`           | No        | `file`    | The cache type to use |
-| `CACHE_AUTH_CONN`      | No        | `/config` |                       |
-
 </TabItem>
-
-<TabItem value="aio" label="AIO">
-
-```json5 title="config.json"
-{
-    "cache": {
-        "auth": {
-            "provider": "file",
-            "connection": "/config"
-        }
-    },
-    // ...
-}
-```
-</TabItem>
-</Tabs>
-
+<TabItem value="transform" label="Scrobble Transforms">
+The results of [transform rules](/configuration/transforms) are cached so that if a scrobble with identical data (track/artists/album) is identified and it has the same set of transform rules then the cached transform results can be applied.
 </TabItem>
 <TabItem value="metadata" label="Transform API Calls">
-
 API Calls to external (metadata) services used to [Enhance Scrobbles](/configuration/transforms), like calls to [Musicbrainz](/configuration/transforms/musicbrainz), can be cached to avoid duplicate calls and speed up scrobble transformations.
-
-By default, these calls are only cached in memory. If you wish for cached calls to be persisted across restarts then setup [Valkey Cache](./?cacheType=valkey#secondary-caching-configuration).
-
-##### Configuration
-
-Use any [Secondary Cache](#secondary-caching-configuration), the config examples below show the default values:
-
-<Tabs>
-<TabItem value="env" label="ENV">
-
-| Environmental Variable | Required? | Default | Description           |
-| :--------------------- | --------- | ------- | --------------------- |
-| `CACHE_METADATA`       | No        |         | The cache type to use |
-| `CACHE_METADATA_CONN`  | No        |         |                       |
-
-</TabItem>
-
-<TabItem value="aio" label="AIO">
-
-```json5 title="config.json"
-{
-    "cache": {
-        "metadata": {
-            "provider": "valkey",
-            "connection": "yourConnectionStringHere"
-        }
-    },
-    // ...
-}
-```
 </TabItem>
 </Tabs>
 
-</TabItem>
-</Tabs>
+</DetailsAdmo>
 
-#### Secondary Caching Configuration
+<DetailsAdmo type="important" summary="Auth Cache Configuration">
 
-The type of cache used, and its connection properties, can be configured through **ENV** or **AIO** config.
+Auth caching defaults to a **file** that is stored in the `CONFIG_DIR` directory using the pre-defined file name `ms-auth.cache`.
 
-<Tabs groupId="cacheType" queryString>
+This provides automatic persistence across restarts for long-lived auth data/credentials if you have configured a [persisted volume/bind mount](/installation?dockerSetting=storage#recommended-settings) for configuration (`/config` is mounted in [docker compose](/quickstart#create-docker-compose-file)).
 
-<TabItem value="file" label="File">
+If you wish to use the [secondary store](#secondary-caching) for caching Auth you must explicitly configure it. This is because valkey can potentially be *ephemeral* if you do not provide a volume for its data directory.
 
-**File** cache is stored in the `CONFIG_DIR` directory using the pre-defined file name `ms-[cacheName].cache`.
+To explicitly configure auth to use the secondary store:
 
 <Tabs groupId="configType" queryString>
-
 <TabItem value="env" label="ENV">
 
-Example
-
-| Environmental Variable | Required? | Default   | Description                                                  |
-| :--------------------- | --------- | --------- | ------------------------------------------------------------ |
-| `CACHE_SCROBBLE`       | No        | `file`    | The cache type to use                                        |
-| `CACHE_SCROBBLE_CONN`  | No        | `/config` | The directory, within the container, to store the cache file |
+```yaml title="compose.yaml"
+services:
+  multi-scrobbler:
+  # ...
+  environment:
+  // highlight-start
+    - CACHE_AUTH=valkey
+  // highlight-end
+    # ...
+```
 
 </TabItem>
 
 <TabItem value="aio" label="AIO">
 
-Example
-
 ```json5 title="config.json"
 {
     "cache": {
-        "scrobble": {
-            "provider": "file",
-            "connection": "/config"
+        "auth": {
+            "provider": "valkey"
         }
     },
     // ...
 }
 ```
-
 </TabItem>
-
 </Tabs>
-</TabItem>
 
-<TabItem value="valkey" label="Valkey">
+</DetailsAdmo>
+
+### Secondary Caching Store {#secondary-caching}
+
+Using a secondary store enables:
 
-[**Valkey**](https://valkey.io/) is an open-source fork of Redis.
+* persistence of cached data across restarts
+* a larger store (more data is saved)
+* a longer time-to-live in the store (cached data is fetchable for a longer period)
+
+These benefits are particularly beneficial when using [transforms](/configuration/transforms) like Musicbrainz and it is **strongly recommended** for these scenarios.
+
+Currently, Multi-scrobbler only supports [**Valkey**](https://valkey.io/), an open-source fork of Redis, as a secondary store.
 
 <details>
 
 <summary>Adding Valkey container to multi-scrobbler</summary>
 
 A valkey container can be added to the [multi-scrobbler docker compose stack](/installation?runType=docker-compose#docker):
 
+<Tabs>
+<TabItem value="named" label="Named Volume">
+
 ```yaml title="docker-compose.yml"
 services:
   multi-scrobbler:
   # ...
-  # adding everything below
   // highlight-start
   valkey:
     image: valkey/valkey
@@ -483,6 +379,25 @@ volumes:
   // highlight-end
 ```
 
+</TabItem>
+<TabItem value="bind" label="Bind Mount">
+
+```yaml title="docker-compose.yml"
+services:
+  multi-scrobbler:
+  # ...
+  // highlight-start
+  valkey:
+    image: valkey/valkey
+    volumes:
+      - ./valkeyData:/data
+  // highlight-end
+```
+
+</TabItem>
+
+</Tabs>
+
 Use `redis://valkey:6379` as the connection string in the configurations below.
 
 </details>
@@ -497,12 +412,16 @@ redis://HOST_IP:HOST_PORT
 
 <TabItem value="env" label="ENV">
 
-Example
-
-| Environmental Variable | Required? | Default  | Description                                                         |
-| :--------------------- | --------- | -------- | ------------------------------------------------------------------- |
-| `CACHE_METADATA`       | Yes       | `valkey` | The cache type to use                                               |
-| `CACHE_METADATA_CONN`  | Yes       |          | The host/IP and port to connect to EX: `redis://192.168.0.120:6379` |
+```yaml title="compose.yaml"
+services:
+  multi-scrobbler:
+  # ...
+  environment:
+  // highlight-start
+    - CACHE_VALKEY=redis://192.168.0.120:6379
+  // highlight-end
+    # ...
+```
 
 </TabItem>
 
@@ -513,23 +432,17 @@ Example
 ```json5 title="config.json"
 {
     "cache": {
-        "metadata": {
-            "provider": "valkey",
-            "connection": "redis://192.168.0.120:6379"
-        }
+        "valkey": "redis://192.168.0.120:6379"
     },
     // ...
 }
 ```
 
 </TabItem>
 
-</Tabs>
-</TabItem>
-
 </Tabs>
 
-### Debug Mode
+## Debug Mode
 
 Turning on Debug Mode will
 
@@ -550,7 +463,7 @@ To set debug mode either add it to [AIO `config.json`](./?configType=aio#configu
 
 or set the [ENV](./?configType=env#configuration-types) `DEBUG_MODE=true`
 
-### Disable Web
+## Disable Web
 
 If you do not need the dashboard and/or ingress sources, or have security concerns about ingress and cannot control your hosting environment, the web server and API can be disabled.
 

docsite/docs/configuration/sources/lastfm-endpoint.mdx

@@ -41,6 +41,6 @@ http://localhost:9078/api/lastfm/mySlug
 <Config config="LastFMEndpointSourceConfig" fileContent={LastfmEndpointConfig} name="endpointlfm">
         | Environmental Variable | Required? | Default | Description                                                                                                        |
         | :--------------------- | :-------- | ------- | ------------------------------------------------------------------------------------------------------------------ |
-        | `LFMENDPOINT_ENABLE`   | No        |         | Use LFM Endpoint as a Source without any other configuration. Only required if slug/token are not provided as ENVs |
+        | `LFM_ENABLE`           | No        |         | Use LFM Endpoint as a Source without any other configuration. Only required if slug/token are not provided as ENVs |
         | `LFM_SLUG`             | No        |         | (Optional) The URL suffix to use for accepting LFM scrobbles                                                       |
 </Config>