Skip to content

Latest commit

 

History

History
1300 lines (949 loc) · 35.9 KB

IMPLEMENTATION.md

File metadata and controls

1300 lines (949 loc) · 35.9 KB

Implementation Notes

This document provides detailed technical information about key systems in the Audio Program Generator.

Table of Contents


TTS Cache System

Overview

The Audio Program Generator uses an intelligent caching system to avoid redundant API calls and speed up audio generation. The cache is stored in your browser's IndexedDB and persists across sessions.

Cache Storage

Location

  • Storage Type: Browser IndexedDB (not filesystem)
  • Database Name: tts-cache
  • Object Store: audio-snippets
  • Size Limit: 100 MB (automatically managed) file:///docs/IMPLEMENTATION.md#api-retry-mechanism

Viewing the Cache

You can inspect the cache contents using your browser's DevTools:

  1. Open DevTools (F12 or Cmd+Option+I)
  2. Navigate to the Application tab (Chrome) or Storage tab (Firefox)
  3. Expand IndexedDB in the left sidebar
  4. Look for the tts-cache database
  5. Click on audio-snippets object store

Each cache entry contains:

  • key: The unique cache identifier (e.g., tts_1a2b3c4d)
  • blob: The audio data (Blob object)
  • timestamp: When the entry was cached
  • size: Size in bytes
  • text: The original phrase text (for reference)

Cache Key Generation

How Keys Are Generated

Cache keys are generated using a hash of three components:

engine:text:options

Components:

  1. Engine: The TTS engine name (openai, google-cloud, gtts)
  2. Text: The exact phrase text
  3. Options: TTS-specific options (voice, speed, model, etc.), sorted alphabetically

Example:

openai:Stand upright and relax:{"format":"wav","model":"tts-1","speed":1,"voice":"nova"}

This is hashed to produce a cache key like: tts_1a2b3c4d

Key Format

tts_<hash>

Where <hash> is a base-36 encoded hash of the combined string.

When Cache Entries Are Reused

The cache system is smart about when to reuse existing audio. Here's what affects cache hits:

✅ Cache WILL Be Reused When:

Change Cache Behavior Reason
Phrase order changed ✅ Reused Each phrase is cached independently
Phrase durations changed ✅ Reused Duration only affects silence, not speech
Background music changed ✅ Reused Music is mixed separately, not cached
Export format changed (WAV/MP3) ✅ Reused Export format doesn't affect TTS generation
Same phrase appears multiple times ✅ Reused Same phrase = same cache key

❌ Cache MUST Be Regenerated When:

Change Cache Behavior Reason
Phrase text changed ❌ New generation Text is part of cache key
TTS engine changed ❌ New generation Engine is part of cache key
Voice changed ❌ New generation Voice is in options (cache key)
Speed/rate changed ❌ New generation Speed is in options (cache key)
Model changed (OpenAI) ❌ New generation Model is in options (cache key)
Pitch changed (Google Cloud) ❌ New generation Pitch is in options (cache key)
Audio format changed ❌ New generation Format is in options (cache key)

Practical Examples

Example 1: Reordering Phrases

Original file:

Welcome; 3
Relax; 5
Breathe; 2

Reordered file:

Breathe; 2
Welcome; 3
Relax; 5

Result: ✅ All three phrases use cached audio. No API calls needed.


Example 2: Changing Durations

Original file:

Welcome; 3
Relax; 5

Changed durations:

Welcome; 10
Relax; 2

Result: ✅ Cached audio reused. Only the silence duration changes (no API calls).


Example 3: Changing Voice

Original settings:

  • Voice: Nova
  • Phrases: "Welcome", "Relax"

Changed settings:

  • Voice: Shimmer
  • Phrases: "Welcome", "Relax"

Result: ❌ Both phrases regenerated with new voice. Two API calls needed.


Example 4: Mixing Cached and New

Cached phrases:

Welcome; 3
Relax; 5

New file:

Welcome; 3
Breathe; 2
Relax; 5

Result:

  • ✅ "Welcome" - cached
  • ❌ "Breathe" - new API call
  • ✅ "Relax" - cached

Only one API call needed for the new phrase.

Cache Management

Automatic Management

The cache automatically manages itself:

  • Size limit: 100 MB maximum
  • Eviction: Oldest entries removed when limit reached (LRU - Least Recently Used)
  • Persistence: Survives browser restarts

Manual Management

Clear All Cache

Click the Clear Cache button in the Output section of the app to remove all cached audio.

Clear Specific Entries

To regenerate specific phrases:

  1. Clear the entire cache
  2. Regenerate your audio program

Or modify the phrase text slightly (e.g., add a space), generate, then change it back.

View Cache Statistics

Cache statistics are shown in the browser console when the app loads:

📦 TTS Cache: 48 snippets, 2.17 MB

Cache Efficiency Tips

Maximize Cache Hits

  1. Keep phrase text consistent: Even small changes (punctuation, capitalization) create new cache entries
  2. Reuse common phrases: Phrases like "Relax", "Breathe", etc. are cached once and reused
  3. Experiment with durations first: Change timing without regenerating audio
  4. Test background music: Try different music without affecting cached speech

When to Clear Cache

Clear the cache when:

  • Switching to a different voice permanently
  • Changing TTS engines
  • Testing new audio quality settings
  • Cache grows too large (approaching 100 MB)
  • Audio sounds corrupted or outdated

Cache Size Considerations

Typical sizes:

  • Short phrase (1-3 words): ~30-50 KB
  • Medium phrase (5-10 words): ~60-100 KB
  • Long phrase (15+ words): ~120-200 KB

Example calculation:

  • 50 unique phrases × 80 KB average = ~4 MB
  • 100 unique phrases × 80 KB average = ~8 MB

The 100 MB limit allows for approximately 1,000-1,500 unique phrases.

Technical Details

Cache Key Algorithm

function generateKey(text, engine, options) {
  // Sort options for consistent keys
  const optionsStr = JSON.stringify(options, Object.keys(options).sort());
  const combined = `${engine}:${text}:${optionsStr}`;
  
  // Simple hash function
  let hash = 0;
  for (let i = 0; i < combined.length; i++) {
    const char = combined.charCodeAt(i);
    hash = ((hash << 5) - hash) + char;
    hash = hash & hash; // Convert to 32-bit integer
  }
  
  return `tts_${Math.abs(hash).toString(36)}`;
}

Cache Entry Structure

{
  key: "tts_1a2b3c4d",           // Unique identifier
  blob: Blob,                     // Audio data
  timestamp: 1699380000000,       // Unix timestamp (ms)
  size: 67244,                    // Size in bytes
  text: "Stand upright and relax" // Original phrase (for debugging)
}

IndexedDB Schema

Database: tts-cache (version 1)
  ObjectStore: audio-snippets
    KeyPath: key
    Indexes:
      - timestamp (non-unique)

Troubleshooting

Cache Not Working

Symptoms: Every phrase regenerates, even unchanged ones

Solutions:

  1. Check browser console for cache errors
  2. Verify IndexedDB is enabled in browser settings
  3. Check if in private/incognito mode (cache may not persist)
  4. Clear browser data and restart

Cache Growing Too Large

Symptoms: Browser storage warnings, slow performance

Solutions:

  1. Click "Clear Cache" button
  2. Reduce number of unique phrases
  3. Reuse common phrases across programs

Corrupted Cache Entries

Symptoms: Audio decode errors, playback failures

Solutions:

  1. Clear the cache completely
  2. Regenerate affected phrases
  3. Check browser console for specific errors

Best Practices

  1. Design for reusability: Create a library of common phrases
  2. Version your phrase files: Keep track of which phrases are cached
  3. Monitor cache size: Check console output periodically
  4. Clear strategically: Only clear when necessary (voice/engine changes)
  5. Test incrementally: Add new phrases gradually to maximize cache hits

Related Files

  • Implementation: /scripts/services/TTSCacheService.js
  • Usage: /scripts/controllers/AppController.js
  • Tests: /tests/unit/TTSCacheService.test.js

API Retry Mechanism

Overview

The Audio Program Generator includes a robust retry mechanism to handle intermittent API failures, particularly for the OpenAI TTS API. This system automatically detects and recovers from empty responses, network issues, and temporary API instability.

How It Works

When the OpenAI TTS API returns an empty response (0 bytes) despite a successful HTTP 200 status, the retry mechanism automatically:

  1. Detects the issue: Checks if the response buffer is empty
  2. Logs the problem: Records which phrase failed and response details
  3. Waits with backoff: Pauses before retrying (exponential backoff)
  4. Retries the request: Makes a new API call with identical parameters
  5. Validates the retry: Checks if the new response is valid
  6. Repeats if needed: Continues up to maximum retry attempts
  7. Caches on success: Stores successful result to avoid future retries

Retry Strategy

Exponential Backoff

The system uses exponential backoff to avoid overwhelming the API:

Attempt Delay Total Wait Time
Initial 0ms 0ms
Retry 1 1000ms (1s) 1s
Retry 2 2000ms (2s) 3s
Retry 3 3000ms (3s) 6s

Maximum retries: 3 attempts (4 total tries including initial)

Why Exponential Backoff?

  • Reduces API load: Gives the API time to recover
  • Avoids rate limiting: Prevents triggering rate limit protections
  • Increases success rate: Temporary issues often resolve within seconds
  • User-friendly: Total wait time (6s max) is acceptable for users

Console Output

The retry mechanism provides detailed console logging to help diagnose issues:

Successful Initial Request

🔄 Making OpenAI TTS API request: { text: "Welcome", model: "tts-1", voice: "nova", format: "wav" }
📡 API Response: { status: 200, statusText: "", ok: true, contentType: "audio/wav", contentLength: null, headers: [...] }
📥 Reading response as array buffer...
📥 Array buffer read complete: 67244 bytes
📦 Created blob: { size: 67244, type: "audio/wav" }

Empty Response with Successful Retry

🔄 Making OpenAI TTS API request: { text: "Relax", model: "tts-1", voice: "nova", format: "wav" }
📡 API Response: { status: 200, statusText: "", ok: true, contentType: "audio/wav", contentLength: "0", headers: [...] }
📥 Reading response as array buffer...
📥 Array buffer read complete: 0 bytes
❌ OpenAI returned empty audio for: "Relax"
Response headers: [["content-type", "audio/wav"], ["content-length", "0"], ...]
⏳ Waiting 1000ms before retry 1/3...
🔄 Retry attempt 1/3...
📥 Retry 1 buffer size: 64244 bytes
✅ Retry 1 succeeded!

Multiple Retries Needed

❌ OpenAI returned empty audio for: "Breathe deeply"
⏳ Waiting 1000ms before retry 1/3...
🔄 Retry attempt 1/3...
📥 Retry 1 buffer size: 0 bytes
⏳ Waiting 2000ms before retry 2/3...
🔄 Retry attempt 2/3...
📥 Retry 2 buffer size: 57044 bytes
✅ Retry 2 succeeded!

All Retries Failed

❌ OpenAI returned empty audio for: "Problematic phrase"
⏳ Waiting 1000ms before retry 1/3...
🔄 Retry attempt 1/3...
📥 Retry 1 buffer size: 0 bytes
⏳ Waiting 2000ms before retry 2/3...
🔄 Retry attempt 2/3...
📥 Retry 2 buffer size: 0 bytes
⏳ Waiting 3000ms before retry 3/3...
🔄 Retry attempt 3/3...
📥 Retry 3 buffer size: 0 bytes
❌ All retries failed for phrase: "Problematic phrase"
Error: OpenAI API returned empty audio for phrase: "Problematic phrase" even after 3 retries.

When Retries Are Triggered

✅ Retry Conditions

Retries are triggered when:

  • Response status is 200 OK but buffer is empty (0 bytes)
  • Response contains invalid audio data
  • Network timeout or connection error

❌ No Retry Conditions

Retries are NOT triggered for:

  • HTTP error responses (4xx, 5xx) - these fail immediately
  • Invalid API key errors
  • Rate limit errors (429) - these should be handled differently
  • Valid audio responses (even if small)

Error Handling

Recoverable Errors (with retry)

  • Empty response (0 bytes)
  • Network timeouts
  • Temporary API instability

Non-Recoverable Errors (immediate failure)

  • Invalid API key (401)
  • Rate limit exceeded (429)
  • Invalid request parameters (400)
  • Server errors (500, 503)

Performance Impact

Best Case (No Retries Needed)

  • Time: Normal API response time (~1-3 seconds per phrase)
  • API Calls: 1 per phrase
  • User Experience: Seamless, no delays

Typical Case (1 Retry)

  • Time: +1 second delay for first retry
  • API Calls: 2 per affected phrase
  • User Experience: Minor delay, usually unnoticed

Worst Case (3 Retries)

  • Time: +6 seconds total delay (1s + 2s + 3s)
  • API Calls: 4 per affected phrase
  • User Experience: Noticeable but acceptable delay

Integration with Caching

The retry mechanism works seamlessly with the cache system:

  1. Check cache first: Before any API call
  2. API call with retry: If cache miss
  3. Cache on success: Store successful result (even from retry)
  4. Future requests: Use cached version (no retry needed)

Example flow:

Phrase: "Relax"
├─ Check cache → MISS
├─ API call → Empty response (0 bytes)
├─ Retry 1 (after 1s) → Success (64KB)
├─ Cache result → Stored
└─ Future requests → Cache HIT (no API call)

Configuration

Current configuration (in OpenAITTSAdapter.js):

const maxRetries = 3;
const retryDelays = [1000, 2000, 3000]; // milliseconds

These values are optimized for:

  • User experience: Total wait time acceptable
  • API stability: Enough time for recovery
  • Success rate: High probability of recovery within 3 retries

Real-World Performance

Based on testing with intermittent API issues:

Metric Value
Success rate (no retry) ~85-90%
Success rate (with 1 retry) ~95-98%
Success rate (with 2-3 retries) ~99%+
Average retry rate ~10-15% of requests
Average delay per phrase ~0.1-0.2 seconds

Troubleshooting

High Retry Rate

Symptoms: Many phrases requiring retries (>30%)

Possible causes:

  • OpenAI API experiencing widespread issues
  • Network connectivity problems
  • Rate limiting (too many requests too fast)

Solutions:

  1. Check OpenAI Status Page
  2. Reduce concurrent requests (generate fewer phrases at once)
  3. Check your internet connection
  4. Wait a few minutes and try again

Retries Always Fail

Symptoms: All retries return empty responses

Possible causes:

  • Specific phrase causing API issues
  • API key quota exhausted
  • API service degradation

Solutions:

  1. Try a different phrase to isolate the issue
  2. Check your OpenAI account usage/quota
  3. Try a different voice or model
  4. Contact OpenAI support if persistent

No Retry Logs

Symptoms: Errors occur but no retry logs appear

Possible causes:

  • HTTP error response (not 200 OK)
  • Request failing before response received

Solutions:

  1. Check for API key errors in console
  2. Verify network connectivity
  3. Check browser console for other errors

Best Practices

  1. Monitor console logs: Watch for retry patterns
  2. Clear cache if issues persist: Force fresh generation
  3. Report persistent failures: Note specific phrases that always fail
  4. Be patient: Allow retries to complete (up to 6 seconds)
  5. Check API status: Verify OpenAI service health during issues

Related Files

  • Implementation: /scripts/services/OpenAITTSAdapter.js (lines 200-250)
  • Error handling: /scripts/services/AudioService.js
  • Tests: /tests/unit/OpenAITTSAdapter.test.js

MP3 Export System

Overview

The Audio Program Generator includes MP3 export functionality using the lamejs library, providing 7-10x file size reduction compared to WAV format while maintaining excellent audio quality.

Implementation

Core Components

Files involved:

  • index.html - Export format selector and bitrate controls
  • scripts/services/AudioService.js - MP3 encoding logic
  • scripts/controllers/AppController.js - Export workflow

MP3 Encoding Process

The system uses the lamejs library (loaded from CDN) to encode audio:

  1. Check library availability: Verify lamejs is loaded
  2. Convert audio format: Float32 samples → Int16 PCM
  3. Encode in chunks: Process 1152 samples per MP3 frame
  4. Report progress: Update UI every 10 blocks
  5. Yield to UI: Prevent browser freezing
  6. Create blob: Package encoded data

Features

Format Selection

Users can choose between:

  • WAV (uncompressed, large file) - For archival and editing
  • MP3 (compressed, smaller file) - For distribution and playback

Quality Options

Bitrate Quality File Size Use Case
128 kbps Good ~1 MB/min Podcasts, voice-only
192 kbps Very Good ~1.4 MB/min Recommended default
256 kbps Excellent ~1.9 MB/min Music with speech
320 kbps Near-Perfect ~2.4 MB/min Audiophile quality

Compression Ratios

For a 60-second meditation program with background music:

  • WAV: 10.3 MB (uncompressed)
  • MP3 @ 128 kbps: 960 KB (10.7x smaller)
  • MP3 @ 192 kbps: 1.4 MB (7.4x smaller) ← Default
  • MP3 @ 256 kbps: 1.9 MB (5.4x smaller)
  • MP3 @ 320 kbps: 2.4 MB (4.3x smaller)

User Experience

Workflow

  1. Generate audio program (any TTS engine)
  2. Select export format (WAV or MP3)
  3. If MP3: Select quality (128/192/256/320 kbps)
  4. Click "Download Audio"
  5. See progress bar during encoding
  6. File downloads automatically

Progress Indication

  • Real-time progress bar during MP3 encoding
  • Shows percentage complete (0-100%)
  • Updates every ~10 blocks for smooth UI
  • Yields to UI thread periodically to prevent freezing

Network Handling

Offline Detection

The system includes an offline detection banner that:

  • Appears when CDN resources fail to load
  • Updates in real-time based on network status
  • Provides one-click reload button when connection restored
  • Works with inline CSS (no CDN dependency)

Network Scenarios

Scenario 1: Offline (library not cached)

  • Error message with WAV fallback option
  • Clear explanation of the issue

Scenario 2: Offline (library cached)

  • MP3 encoding works perfectly
  • No internet required

Scenario 3: Network drops during encoding

  • Encoding continues successfully
  • Library already loaded in memory

Scenario 4: Network restored

  • Green banner with "Reload Now" button
  • One-click to restore full functionality

Error Handling

Library Not Available

MP3 encoder not available. This may be due to:

1. No internet connection (library not cached yet)
2. Browser blocking the script
3. Network error

Please check your connection and reload the page, or use WAV format instead.

Encoding Cancelled

  • Graceful cancellation without error popup
  • Progress bar shows "Encoding cancelled"
  • No data loss

Performance

Encoding Speed

  • ~2-5 seconds for 1 minute of audio (CPU-dependent)
  • Single-threaded (JavaScript limitation)
  • Yields to UI every 10 frames to prevent freezing

Memory Usage

For 60-second stereo audio:

  • AudioBuffer (in memory): 21 MB
  • Int16 PCM conversion: 10.5 MB
  • Encoder buffers: ~5 MB
  • Peak memory usage: ~36 MB
  • Final MP3 file (192 kbps): 1.4 MB

Browser Compatibility

Browser lamejs Support Caching Performance
Chrome ✅ ✅ Excellent
Edge ✅ ✅ Excellent
Firefox ✅ ✅ Good
Safari ✅ ✅ Good

CDN Caching:

  • First load: Downloads ~50KB library
  • Subsequent loads: Instant (cached)
  • Works offline after first load

Technical Details

Float32 to Int16 Conversion

floatTo16BitPCM(input) {
  const output = new Int16Array(input.length);
  for (let i = 0; i < input.length; i++) {
    // Clamp to -1.0 to 1.0
    const s = Math.max(-1, Math.min(1, input[i]));
    // Convert to 16-bit integer (-32768 to 32767)
    output[i] = s < 0 ? s * 0x8000 : s * 0x7FFF;
  }
  return output;
}

Encoding Loop

async audioBufferToMP3(buffer, bitrate = 192, onProgress = null) {
  const mp3encoder = new lamejs.Mp3Encoder(channels, sampleRate, bitrate);
  const mp3Data = [];
  const blockSize = 1152; // MP3 frame size
  
  for (let i = 0; i < samples; i += blockSize) {
    // Check for cancellation
    if (this.encodingCancelled) {
      throw new Error('MP3 encoding cancelled by user');
    }

    // Encode chunk
    const mp3buf = mp3encoder.encodeBuffer(leftChunk, rightChunk);
    if (mp3buf.length > 0) {
      mp3Data.push(new Int8Array(mp3buf));
    }

    // Report progress
    if (onProgress) {
      const progress = Math.floor((currentBlock / totalBlocks) * 100);
      onProgress(progress);
    }

    // Yield to UI thread every 10 blocks
    if (i % (blockSize * 10) === 0) {
      await new Promise(resolve => setTimeout(resolve, 0));
    }
  }

  // Flush encoder
  const mp3buf = mp3encoder.flush();
  if (mp3buf.length > 0) {
    mp3Data.push(new Int8Array(mp3buf));
  }

  return new Blob(mp3Data, { type: 'audio/mp3' });
}

Known Limitations

  1. Encoding Speed: Slower than native FFmpeg (JavaScript vs C)
  2. Single-threaded: Can't use multiple CPU cores easily
  3. Memory Usage: Requires ~3x the final file size in RAM
  4. Browser Freeze: Very long files (>10 min) may cause brief UI freeze

Recommendations

For Users:

  • Use 192 kbps for best quality/size balance
  • Use WAV only if you need to edit the audio later
  • First load requires internet (to download library)
  • After first load, works completely offline

For Developers:

  • Consider Web Workers for files >5 minutes
  • Monitor memory usage for very long programs
  • Add file size estimate before encoding

Future Enhancements

Potential improvements:

  1. Web Workers: Move encoding to background thread
  2. AAC/OGG Support: Add more codec options
  3. Variable Bitrate (VBR): Better quality/size tradeoff
  4. Metadata: Add ID3 tags (title, artist, etc.)
  5. Batch Export: Export multiple files at once
  6. Quality Preview: Let users hear sample before full export

Related Files

  • Implementation: /scripts/services/AudioService.js (audioBufferToMP3 method)
  • UI: /index.html (export format selector)
  • Controller: /scripts/controllers/AppController.js (handleDownload method)

OpenAI TTS Integration

Overview

The Audio Program Generator supports OpenAI's TTS-1 and TTS-1-HD models as a text-to-speech engine option. OpenAI TTS provides high-quality voices with simple setup and pay-as-you-go pricing.

Features

Voice Options

  • Nova (Female - warm and friendly) - Recommended for meditation
  • Shimmer (Female - soft and gentle) - Great for relaxation
  • Alloy (Neutral - balanced)
  • Echo (Male)
  • Fable (Male - British)
  • Onyx (Male - deep)

Model Options

  • TTS-1 (Standard): $15 per 1M characters
  • TTS-1-HD (High Definition): $30 per 1M characters

Parameters

  • Speed: 0.25x to 4.0x (adjustable via slider)
  • Format: WAV (for compatibility with mixing)

Capabilities

  • ✅ Full audio export support
  • ✅ Background music mixing
  • ✅ Fade in/out effects
  • ✅ API key validation
  • ✅ Secure local storage of API key
  • ✅ Pay-as-you-go pricing (no monthly fees)

Architecture

Adapter Pattern

The OpenAI TTS implementation follows the adapter pattern:

  • Extends TTSEngineAdapter base class
  • Implements required methods: getName(), getCapabilities(), generateSpeech(), validateApiKey()
  • Handles silence generation for phrase pauses
  • Converts audio to WAV format for consistency

API Integration

  • Uses OpenAI's /v1/audio/speech endpoint
  • Bearer token authentication
  • Supports multiple audio formats (defaults to WAV)
  • Proper error handling with descriptive messages

Storage

  • API keys stored in localStorage: openai-tts-api-key
  • Separate from Google Cloud API key
  • Auto-loaded on page refresh

Pricing Comparison

For 100K characters (typical usage):

Provider Cost Setup Complexity
OpenAI TTS $1.50 (standard)
$3.00 (HD)
⭐⭐⭐⭐⭐ Simple
Google Cloud $0.00 (free tier)
$1.60 (after)
⭐⭐ Complex
Web Speech API $0.00 ⭐⭐⭐⭐⭐ None (limited)

User Benefits

  1. Simple Setup: No billing account complexities like Google Cloud
  2. Predictable Costs: Pay only for what you use, no monthly fees
  3. High Quality: Excellent voice quality comparable to Google Cloud
  4. Flexibility: 6 voices to choose from, speed control
  5. Full Features: Export, mixing, and all advanced features supported

Implementation Details

Files Created

/scripts/services/OpenAITTSAdapter.js

  • Implements the TTSEngineAdapter interface
  • Supports TTS-1 (standard) and TTS-1-HD (high quality) models
  • 6 preset voices with descriptions
  • Speed control (0.25x - 4.0x)
  • Returns WAV audio blobs for mixing and export

Files Modified

/scripts/services/TTSService.js

  • Added OpenAI adapter to the adapters registry
  • Enhanced loadApiKey() to load OpenAI API key from localStorage
  • Enhanced saveApiKey() to support multiple engines

/scripts/controllers/AppController.js

  • Added OpenAI settings UI visibility toggle
  • Enhanced handleSaveApiKey() to support both Google Cloud and OpenAI
  • Added OpenAI API key loading
  • Added OpenAI TTS generation logic
  • Added OpenAI speed slider to value display setup

/index.html

  • Added "OpenAI TTS" option to engine dropdown
  • Created OpenAI settings section with:
    • API key input field with save button
    • Voice selector (6 voices with descriptions)
    • Model quality selector (tts-1 vs tts-1-hd)
    • Speed slider (0.25x - 4.0x)

API Details

Request Format

POST https://api.openai.com/v1/audio/speech
Headers:
  Authorization: Bearer sk-...
  Content-Type: application/json

Body:
{
  "model": "tts-1",
  "input": "Welcome to your audio program",
  "voice": "nova",
  "response_format": "wav",
  "speed": 1.0
}

Response

  • Binary audio data (WAV format)
  • Content-Type: audio/wav
  • Typical size: 50-150 KB per phrase

Error Handling

Invalid API Key

OpenAI API error: Incorrect API key provided

Quota Exceeded

OpenAI API error: You exceeded your current quota

Network Error

Failed to generate speech: Network error

Best Practices

  1. Set usage limits: Configure monthly budget in OpenAI dashboard
  2. Use standard model: TTS-1 is sufficient for most use cases
  3. Cache aggressively: Leverage the caching system to minimize API calls
  4. Monitor costs: Check OpenAI dashboard regularly

Future Enhancements

When gpt-4o-mini-tts is officially released:

  • Add support for voice_instructions parameter (the "vibe" feature)
  • Add emotion/tone controls
  • Add accent customization
  • Implement advanced voice customization UI

Potential improvements:

  • Add usage tracking/cost estimation
  • Add voice preview samples
  • Add batch generation optimization

Notes

  • OpenAI API is separate from ChatGPT Plus subscription
  • API keys are stored locally (never sent to our servers)
  • Users should set usage limits in OpenAI dashboard
  • Standard model (TTS-1) is recommended for most use cases
  • HD model (TTS-1-HD) is only needed for highest quality requirements

Related Files

  • Implementation: /scripts/services/OpenAITTSAdapter.js
  • Service: /scripts/services/TTSService.js
  • Controller: /scripts/controllers/AppController.js
  • Tests: /tests/unit/OpenAITTSAdapter.test.js

Project Cache System

Overview

The Project Cache System automatically saves your audio projects (phrase files, background music, and all settings) so you can quickly restore them when you return to the app. This eliminates the need to re-upload files and reconfigure settings for projects you work on frequently.

How It Works

When you successfully generate audio, the system automatically saves:

  • Phrase file content - The text of all your phrases
  • Background music - The audio file (if used)
  • TTS settings - Engine, voice, speed, pitch, etc.
  • Export settings - Format (WAV/MP3) and bitrate

The next time you visit the app, your recent projects appear at the top of the page, ready to restore with one click.

Storage Details

Location: Browser IndexedDB

  • Database name: apg-projects
  • Object store: projects
  • Max projects: 10 (automatically prunes oldest)

What's stored per project:

{
  id: "project_1699380000000_abc123",
  name: "morning-meditation.txt",
  phraseFileContent: "Welcome; 3\nRelax; 5\n...",
  backgroundMusic: Blob,              // Audio file
  backgroundMusicName: "calm-music.mp3",
  ttsEngine: "openai",
  ttsOptions: { voice: "nova", model: "tts-1", speed: 1.0 },
  exportSettings: { format: "mp3", bitrate: 192 },
  timestamp: 1699380000000
}

User Experience

Recent Projects Section

When you have saved projects, a "Recent Projects" section appears at the top of the page showing:

  • Project name (from phrase file name)
  • Time saved ("Just now", "2 hours ago", "3 days ago", etc.)
  • TTS engine used
  • Background music status
  • Restore button - One-click to load all settings
  • Delete button (×) - Remove individual projects

Automatic Saving

Projects are automatically saved after successful audio generation. No user action required!

Restoring a Project

Click the "Restore" button on any project card:

  1. Phrase file is loaded
  2. Background music is loaded (if used)
  3. TTS engine is selected
  4. All voice/speed/pitch settings are restored
  5. Export format and bitrate are restored
  6. Click "Generate Audio" to recreate the project

Key benefit: TTS snippets are already cached, so regeneration is fast (2-5 seconds instead of 30+ seconds)!

Storage Management

Automatic Pruning

  • System keeps the 10 most recent projects
  • Older projects are automatically deleted
  • Pruning happens after each save

Manual Clearing

  • Clear All button: Removes all saved projects
  • Individual delete (×): Remove specific projects
  • Confirmation dialog prevents accidental deletion

Storage Size

Typical project sizes:

  • Phrase file: ~1-5 KB (text)
  • Background music: ~3-10 MB (audio file)
  • Settings: ~1 KB (JSON)
  • Total per project: ~3-10 MB

10 projects: ~30-100 MB total

Benefits

Time Savings

Without Project Cache:

  1. Upload phrase file
  2. Upload background music
  3. Select TTS engine
  4. Configure voice/speed/pitch
  5. Set export format
  6. Generate audio (30+ seconds)

With Project Cache:

  1. Click "Restore"
  2. Click "Generate Audio" (2-5 seconds with TTS cache)

Time saved: ~90% for repeat projects

Use Cases

  1. Iterative refinement: Tweak phrases, regenerate quickly
  2. Multiple versions: Try different voices/speeds
  3. Daily routines: Morning meditation, evening relaxation
  4. Template projects: Reuse structure with new content
  5. A/B testing: Compare different background music

Integration with TTS Cache

The Project Cache works seamlessly with the TTS Cache:

First generation (no caches):

  • TTS API calls: 10 phrases = 10 API calls
  • Time: ~30 seconds
  • Project saved automatically

Restore and regenerate (TTS cached):

  • TTS API calls: 0 (all cached!)
  • Time: ~2-5 seconds (just mixing)
  • Fast iteration on settings

Change one phrase:

  • TTS API calls: 1 (only the changed phrase)
  • Time: ~5 seconds
  • Other phrases use cache

Technical Details

Project ID Generation

generateId() {
  return `project_${Date.now()}_${Math.random().toString(36).substr(2, 9)}`;
}
// Example: "project_1699380000000_k3j2h5g9"

Timestamp Formatting

formatTimestamp(timestamp) {
  const diffMins = Math.floor((now - timestamp) / 60000);
  
  if (diffMins < 1) return 'Just now';
  if (diffMins < 60) return `${diffMins} min ago`;
  if (diffHours < 24) return `${diffHours} hour(s) ago`;
  if (diffDays < 7) return `${diffDays} day(s) ago`;
  return date.toLocaleDateString();
}

File Restoration

Uses the DataTransfer API to programmatically set file inputs:

const dataTransfer = new DataTransfer();
dataTransfer.items.add(phraseFile);
phraseInput.files = dataTransfer.files;

Console Output

On Page Load

💼 Projects: 3 saved, 25.4 MB

When Saving

💾 Saved project: "morning-meditation.txt"

When Restoring

🔄 Restoring project: "morning-meditation.txt"

When Pruning

🧹 Pruning 1 old project(s)...
🗑️ Deleted project: project_1699380000000_abc123

Browser Compatibility

Browser IndexedDB Support DataTransfer API Status
Chrome ✅ ✅ Full support
Edge ✅ ✅ Full support
Firefox ✅ ✅ Full support
Safari ✅ ✅ Full support

All modern browsers fully support the required APIs.

Limitations

  1. Browser-specific: Projects are stored per browser, not synced across devices
  2. Origin-specific: localhost and deployed site have separate storage
  3. No cloud sync: Projects are local only
  4. 10 project limit: Older projects are automatically deleted
  5. No export/import: Can't share projects between browsers (yet)

Privacy & Security

  • Local storage only: Projects never leave your browser
  • No server upload: All data stays on your device
  • User control: Easy to delete individual or all projects
  • No tracking: System doesn't track what you create

Future Enhancements

Potential improvements:

  1. Export/Import: Save projects as files, share between devices
  2. Cloud sync: Optional sync across browsers/devices
  3. Project folders: Organize projects into categories
  4. Search/filter: Find projects by name or date
  5. Project notes: Add descriptions or tags
  6. Unlimited storage: Remove 10-project limit with pagination
  7. Duplicate project: Clone and modify existing projects
  8. Project templates: Pre-made structures for common use cases

Troubleshooting

Projects Not Appearing

Symptoms: No "Recent Projects" section visible

Possible causes:

  • No projects saved yet (generate audio first)
  • Browser in private/incognito mode
  • IndexedDB disabled in browser settings

Solutions:

  1. Generate an audio project first
  2. Check browser console for errors
  3. Verify IndexedDB is enabled

Restore Not Working

Symptoms: Click "Restore" but files don't load

Possible causes:

  • Browser security restrictions
  • Corrupted project data

Solutions:

  1. Try deleting and recreating the project
  2. Check browser console for errors
  3. Clear all projects and start fresh

Storage Full

Symptoms: Warning about storage limits

Solutions:

  1. Click "Clear All" to remove all projects
  2. Delete individual old projects
  3. Use smaller background music files

Best Practices

  1. Name files clearly: Use descriptive phrase file names (they become project names)
  2. Regular cleanup: Delete projects you no longer need
  3. Test restore: Verify projects restore correctly before relying on them
  4. Backup important projects: Download the final audio for safekeeping
  5. Use TTS cache: Let both caches work together for maximum speed

Related Files

  • Implementation: /scripts/services/ProjectCacheService.js
  • Controller: /scripts/controllers/AppController.js (project methods)
  • UI: /index.html (Recent Projects section)