Model Selection Guide: Choosing the Right Translation Model

MAITO offers several translation models. This comprehensive guide helps you choose the right one for your hardware and needs.

Quick Decision Tree

Do you have a dedicated GPU (NVIDIA/AMD)?
├─ YES → Try Rosetta 4B DML first
│         └─ If slower than expected → Try Rosetta 4B CPU
└─ NO → Use Rosetta 4B CPU

Are you curious about alternatives?
└─ YES → Download Phi-3 variants and compare
          Keep whichever performs best for you

Available Models Overview

Rosetta Models (Recommended)

Rosetta 4B CPU-INT4

• Size: 2.5GB
• Optimization: CPU inference
• Best For: CPU-only systems, laptops without dedicated GPU
• Performance: 8-15 t/s on modern CPUs
• Compatibility: Universal - works on any x64 Windows

Rosetta 4B DML-INT4

• Size: 2.5GB
• Optimization: DirectML GPU acceleration
• Best For: Systems with dedicated GPU or strong integrated graphics
• Performance: 15-25 t/s with compatible GPU
• Compatibility: Requires DirectML-compatible GPU (most modern GPUs)

Phi Models (Alternative)

Phi-3-4B CPU-INT4

• Size: ~2.3GB
• Optimization: CPU inference
• Best For: Users wanting to compare with general-purpose model
• Performance: 7-14 t/s on modern CPUs
• Compatibility: Universal

Phi-3-4B DML-INT4

• Size: ~2.3GB
• Optimization: DirectML GPU acceleration
• Best For: GPU systems, comparison with Rosetta DML
• Performance: 14-23 t/s with compatible GPU
• Compatibility: Requires DirectML-compatible GPU

Detailed Model Comparison

Translation Quality

Quality Notes:

• CPU vs DML variants have identical quality (same model, different inference path)
• Rosetta optimized specifically for translation → slightly better for this task
• Phi-3 is more general-purpose → excellent but not translation-specialized
• Both families provide very good translation quality

Performance by Hardware Type

High-End Desktop (i7/Ryzen 7, 16GB+, RTX/RX GPU)

Recommended: Rosetta 4B DML

Benchmark Results (typical):

• Rosetta 4B DML: 20-25 t/s ⭐ Best
• Phi-3 4B DML: 18-23 t/s
• Rosetta 4B CPU: 12-15 t/s
• Phi-3 4B CPU: 11-14 t/s

Why DML: Your GPU can handle AI workloads well, use it!

Mid-Range Laptop (i5/Ryzen 5, 8-12GB, Integrated Graphics)

Recommended: Rosetta 4B CPU

Benchmark Results (typical):

• Rosetta 4B CPU: 9-12 t/s ⭐ Best
• Phi-3 4B CPU: 8-11 t/s
• Rosetta 4B DML: 7-10 t/s (integrated GPU not strong enough)
• Phi-3 4B DML: 7-10 t/s

Why CPU: Integrated GPUs often slower than modern CPUs for these workloads

Entry-Level System (i3, 4-6GB RAM, No Dedicated GPU)

Recommended: Rosetta 4B CPU

Benchmark Results (typical):

• Rosetta 4B CPU: 4-7 t/s ⭐ Best
• Phi-3 4B CPU: 3-6 t/s
• DML variants: Not recommended (insufficient GPU, limited RAM)

Why Rosetta CPU: Most efficient for limited resources, auto memory management

CPU vs DML (GPU): When to Use Each

Use CPU Models When:

✅ No Dedicated GPU

• Integrated graphics (Intel UHD, Iris Xe, AMD Vega)
• Laptop without discrete GPU
• Desktop without graphics card

✅ CPU Is Very Strong

• Latest generation CPUs (Intel 12th gen+, Ryzen 5000+)
• High core count (8+ cores)
• Sometimes faster than mid-range GPUs

✅ GPU Is Busy

• Gaming, video editing, or other GPU-intensive tasks running
• GPU resources needed elsewhere
• CPU inference doesn’t block GPU

✅ Compatibility Concerns

• Older GPU with questionable DirectML support
• Driver issues with DML
• CPU is guaranteed to work

Use DML Models When:

✅ Dedicated GPU Present

• NVIDIA GTX 10-series or newer
• AMD RX 5000-series or newer
• Recent discrete GPU

✅ GPU Has Good VRAM

• 4GB+ VRAM recommended
• More VRAM = better performance potential

✅ Seeking Maximum Speed

• GPU offloads work from CPU
• Typically 2-3x faster than CPU
• Best for frequent translation

✅ Benchmark Confirms

• Test both CPU and DML
• Keep whichever is actually faster
• Don’t assume DML is always faster

Rosetta vs Phi: Making the Choice

Choose Rosetta If:

✅ Translation is primary use case

• Rosetta specifically trained for translation
• Optimized for this exact task

✅ You want best quality-speed balance

• Efficient inference
• Excellent translation quality

✅ You trust recommendations

• MAITO recommends Rosetta for a reason
• Default choice for most users

Consider Phi If:

✅ You’re curious and want to compare

• Valid reason to try alternatives
• Benchmark and decide empirically

✅ You use the model for other tasks

• Phi is general-purpose
• Can be used outside MAITO (with ONNX Runtime)

✅ Specific language pair performs better

• Possible that Phi excels in certain pairs
• Test with your actual language combinations

Most Users Should:

1. Start with Rosetta (MAITO’s recommendation)
2. Use it for a week or two
3. If curious, download Phi and compare
4. Keep whichever you prefer

Reality: Most users stick with Rosetta after trying both.

Real-World Scenarios

Scenario 1: Developer on MacBook Pro Alternative (Windows Laptop)

Hardware: HP Spectre x360, Intel i7-1260P, 16GB RAM, Intel Iris Xe

Testing:

• Rosetta 4B CPU: 11.2 t/s
• Rosetta 4B DML: 8.7 t/s (Iris Xe slower than CPU!)
• Phi-3 4B CPU: 10.1 t/s

Choice: Rosetta 4B CPU ⭐ Reason: CPU faster than integrated GPU, Rosetta best quality

Scenario 2: Gamer Desktop

Hardware: Custom build, Ryzen 9 5900X, 32GB RAM, RTX 3070

Testing:

• Rosetta 4B DML: 24.3 t/s ⭐
• Phi-3 4B DML: 22.1 t/s
• Rosetta 4B CPU: 14.8 t/s
• Phi-3 4B CPU: 13.9 t/s

Choice: Rosetta 4B DML ⭐ Reason: GPU acceleration provides huge boost, Rosetta still best

Scenario 3: Budget Office Laptop

Hardware: Dell Inspiron 15, Intel i3-1115G4, 4GB RAM, Intel UHD Graphics

Testing:

• Rosetta 4B CPU: 4.2 t/s ⭐
• Phi-3 4B CPU: 3.8 t/s
• DML variants: Too slow, out of memory errors

Choice: Rosetta 4B CPU ⭐ Reason: Only viable option, MAITO auto-reduces context for stability

Scenario 4: Content Creator Laptop

Hardware: ASUS ROG Zephyrus, Intel i9-12900H, 32GB RAM, RTX 3060 Laptop

Testing:

• Rosetta 4B DML: 19.7 t/s ⭐
• Rosetta 4B CPU: 13.2 t/s
• Phi-3 4B DML: 18.3 t/s

Choice: Rosetta 4B DML ⭐ Reason: Strong GPU, Rosetta slightly faster and better quality

Model Selection Flowchart

START: Need to choose a model

Step 1: Do you have a dedicated GPU?
├─ NO → Go to Step 2
└─ YES → Go to Step 3

Step 2: CPU-Only Systems
Download: Rosetta 4B CPU
Benchmark: Should get 8-15 t/s on modern CPU, 4-7 t/s on older
Acceptable?
├─ YES → Keep using, you're done!
└─ NO → Consider DeepL instead (slow system)

Step 3: GPU Systems
Download: Rosetta 4B DML
Benchmark: Should get 15-25 t/s
Is it 2x faster than CPU?
├─ YES → Keep DML, you're done!
└─ NO → Try Rosetta 4B CPU and compare
        Keep whichever is faster

Optional Step 4: Experimentation
Want to try alternatives?
└─ YES → Download Phi-3 (CPU or DML to match your best)
         Benchmark
         Compare quality on actual translations
         Keep your favorite

END: You have the best model for YOUR system!

Advanced: Multiple Model Strategy

Some power users download multiple models:

Strategy 1: Dual Models for Different Tasks

Rosetta 4B DML (primary):

• Everyday translations
• Quick lookups
• Speed priority

Phi-3 4B CPU (secondary):

• Experimental translations
• Comparing quality
• When GPU is busy

Strategy 2: Language-Specific Models

Rosetta (primary):

• Most language pairs

Phi-3 (secondary):

• Specific language pair where it performs better
• Switch as needed

Strategy 3: Backup Model

Rosetta 4B DML (primary):

• Daily driver

Rosetta 4B CPU (backup):

• When GPU is needed for other tasks (gaming, video editing)
• Power saving mode
• Thermal throttling prevention

Updating Models

When new versions are released:

1. Don’t Delete Old Version Yet:

   • Download new version alongside old
   • Benchmark new version
   • Compare quality

2. Test Before Committing:

   • Use new version for a few days
   • Verify it’s actually better
   • Check your specific language pairs

3. Clean Up:

   • Once satisfied, remove old version
   • Or keep both if you find use for each

Common Mistakes

❌ Assuming DML Is Always Better

Reality: Integrated GPUs often perform worse than modern CPUs. Always benchmark!

❌ Not Benchmarking

Reality: Your hardware is unique. Default recommendations may not be optimal for you.

❌ Choosing Based on Model Size

Reality: Smaller ≠ worse. These are all INT4 quantized models with similar capabilities.

❌ Using Too Many Models

Reality: Download 2-3 maximum. Test, choose favorite, remove others. Don’t hoard!

❌ Forgetting to Remove Old Versions

Reality: After downloading new model version, remove old to free space.

Final Recommendations

For 90% of Users:

Start Here: Rosetta 4B CPU or Rosetta 4B DML (based on GPU presence)

Why: Optimized for translation, excellent quality, MAITO’s default recommendation.

Next Steps:

1. Download recommended model
2. Benchmark (see performance)
3. Use for 1-2 weeks
4. If satisfied, done!
5. If curious, try alternatives

For Experimenters:

Try Both: Rosetta and Phi variants

Method:

1. Download all 4 variants (if disk space permits)
2. Benchmark each
3. Translate same test texts with each
4. Compare quality and speed
5. Remove 3, keep the best

For Budget Hardware:

Only Choice: Rosetta 4B CPU

Why: Most efficient, smallest footprint, auto memory management, best performance per watt.

What’s Next?

Related articles:

• Rosetta Models Deep Dive - Understanding Rosetta
• Performance Optimization - Maximizing speed
• Benchmark Performance - How to test models
• Download Models - Model management