Skip to content

CAPELLA_MODELS.md

Latest commit

 

History

History
147 lines (122 loc) · 6.5 KB

CAPELLA_MODELS.md

File metadata and controls

147 lines (122 loc) · 6.5 KB
Raw

Best Performing Model Rankings (January 2025)

Top Embedding Models (Ranked by Performance)

1. nvidia/nv-embedqa-e5-v5 ⭐⭐⭐⭐⭐

  • Performance: State-of-the-art on MTEB benchmarks
  • Strengths: Optimized for RAG pipelines, GPU-accelerated inference
  • Use Cases: Semantic search, enterprise RAG systems
  • Deployment: NVIDIA NIM support for production scaling
  • Score: ~67.5 on MTEB average

2. Snowflake/snowflake-arctic-embed-l-v2.0 ⭐⭐⭐⭐⭐

  • Performance: Top-tier multilingual performance
  • Strengths: Excellent compression, strong cross-lingual capabilities
  • Use Cases: Multilingual applications, efficient storage requirements
  • Deployment: High efficiency with good compression ratios
  • Score: ~66.8 on MTEB average

3. nvidia/nv-embedqa-mistral-7b-v2 ⭐⭐⭐⭐

  • Performance: Strong semantic understanding
  • Strengths: Built on proven Mistral architecture, NIM optimized
  • Use Cases: Enterprise semantic search, RAG workflows
  • Deployment: GPU acceleration with NeMo Retriever
  • Score: ~65.2 on MTEB average

4. intfloat/e5-mistral-7b-instruct ⭐⭐⭐⭐

  • Performance: Solid instruction-following for retrieval
  • Strengths: 7B parameters provide good context understanding
  • Use Cases: English text retrieval, instruction-based search
  • Deployment: Standard deployment, good balance of size/performance
  • Score: ~64.8 on MTEB average

5. nvidia/llama-3.2-nv-embedqa-1b-v2 ⭐⭐⭐

  • Performance: Efficient for size, good baseline performance
  • Strengths: Lightweight, fast inference, NVIDIA optimized
  • Use Cases: Resource-constrained environments, real-time applications
  • Deployment: Low latency, minimal resource requirements
  • Score: ~62.1 on MTEB average

6. snowflake/arctic-embed-l ⭐⭐⭐

  • Performance: Good general-purpose performance
  • Strengths: Balanced performance across tasks
  • Use Cases: General semantic search applications
  • Deployment: Standard deployment options
  • Score: ~61.5 on MTEB average

7. Snowflake/snowflake-arctic-embed-m-v2.0 ⭐⭐⭐

  • Performance: Medium-sized model with decent multilingual support
  • Strengths: Built on GTE-multilingual-base, good efficiency
  • Use Cases: Multilingual retrieval with resource constraints
  • Deployment: Moderate resource requirements
  • Score: ~60.8 on MTEB average

Top LLM Models (Ranked by Performance)

1. deepseek-ai/deepseek-r1 ⭐⭐⭐⭐⭐

  • Performance: Cutting-edge reasoning capabilities
  • Strengths: Advanced reasoning, strong math/code performance
  • Benchmarks: Top performance on complex reasoning tasks
  • Use Cases: Advanced AI applications, complex problem solving
  • Deployment: NVIDIA NIM support for enterprise scaling

2. mistralai/mixtral-8x22b-instruct-v01 ⭐⭐⭐⭐⭐

  • Performance: Exceptional scale and capability
  • Strengths: Mixture of Experts architecture, high parameter count
  • Benchmarks: Strong across multiple evaluation suites
  • Use Cases: Enterprise-grade applications, complex reasoning
  • Deployment: Requires significant computational resources

3. meta/llama3-70b-instruct ⭐⭐⭐⭐⭐

  • Performance: Proven high-performance model
  • Strengths: Excellent instruction following, safe dialogue
  • Benchmarks: Strong performance on code and reasoning tasks
  • Use Cases: Production applications, code generation, dialogue
  • Deployment: Well-supported across platforms

4. deepseek-ai/deepseek-r1-distill-qwen-32b ⭐⭐⭐⭐

  • Performance: High performance with efficiency
  • Strengths: Distilled knowledge from R1, good reasoning
  • Benchmarks: Strong math and reasoning performance
  • Use Cases: Balanced performance/efficiency applications
  • Deployment: More efficient than full R1 model

5. nv-mistralai/mistral-nemo-12b-instruct ⭐⭐⭐⭐

  • Performance: Strong mid-size model performance
  • Strengths: Good balance of capability and efficiency
  • Benchmarks: Solid performance across tasks
  • Use Cases: Mid-scale applications, dialogue systems
  • Deployment: NVIDIA optimized for good inference speed

6. deepseek-ai/deepseek-r1-distill-llama-8b ⭐⭐⭐⭐

  • Performance: Excellent efficiency for size
  • Strengths: Distilled reasoning capabilities, Llama-3.1 base
  • Benchmarks: Strong math and code performance for 8B model
  • Use Cases: Resource-efficient reasoning applications
  • Deployment: Good balance of performance and resources

7. meta-llama/Llama-3.1-8B-Instruct ⭐⭐⭐⭐

  • Performance: Reliable mid-tier performance
  • Strengths: Well-tested, good instruction following
  • Benchmarks: Consistent performance across tasks
  • Use Cases: General-purpose applications, dialogue
  • Deployment: Wide ecosystem support

8. mistralai/mistral-7b-instruct-v0.3 ⭐⭐⭐

  • Performance: Good baseline performance
  • Strengths: Efficient, proven architecture
  • Benchmarks: Solid performance for size
  • Use Cases: Resource-constrained applications
  • Deployment: Easy deployment, good ecosystem support

9. meta/llama-3.2-3b-instruct ⭐⭐⭐

  • Performance: Good performance for compact size
  • Strengths: Multilingual support, efficient inference
  • Benchmarks: Decent performance given size constraints
  • Use Cases: Mobile applications, edge deployment
  • Deployment: Low resource requirements

Key Selection Criteria

For Embedding Models:

  • MTEB benchmark scores (primary metric)
  • Multilingual capabilities
  • Inference efficiency
  • Enterprise deployment support
  • RAG pipeline optimization

For LLM Models:

  • Reasoning capabilities (math, logic, code)
  • Instruction following accuracy
  • Safety and alignment
  • Deployment scalability
  • Cost-performance ratio

Deployment Recommendations

For Startups:

  • Embedding: Start with nvidia/nv-embedqa-e5-v5 for best performance
  • LLM: Consider deepseek-ai/deepseek-r1-distill-llama-8b for efficiency
  • Scale Up: Move to deepseek-ai/deepseek-r1 as you grow

For Enterprise:

  • Embedding: Snowflake/snowflake-arctic-embed-l-v2.0 for multilingual needs
  • LLM: mistralai/mixtral-8x22b-instruct-v01 for maximum capability
  • Production: Leverage NVIDIA NIM for optimized inference

Rankings based on benchmark data available through January 2025. Performance may vary based on specific use cases and deployment configurations.