Local Translation Architecture: How It Works

For curious users who want to understand how MAITO’s local translation works behind the scenes.

High-Level Architecture

User Input
    ↓
Text Processing
    ↓
Local Translation Engine
    ↓
AI Model (ONNX Format)
    ↓
Output (Streaming)

Core Components

1. Translation Engine

MAITO’s local translation engine provides:

Key Features:

• Streaming output (see translations as they’re generated)
• Cancellation support (stop anytime with Escape key)
• Progress tracking (percentage complete, time estimates)
• Automatic text handling for any length
• Full privacy (nothing leaves your computer)

2. AI Model Runtime

MAITO uses ONNX Runtime from Microsoft to run AI models:

What It Does:

• Loads AI model files from your computer
• Executes the neural network
• Manages memory efficiently
• Provides GPU acceleration when available
• Generates translations token by token

3. Text Processing

Before translation, MAITO intelligently processes your text:

Features:

• Automatic handling of long texts (no practical length limit)
• Smart splitting at natural boundaries (paragraphs, sentences)
• Context preservation for better translation quality
• Completely transparent to you as the user

How Translation Works

Step 1: Initialization

When you first use local translation (or switch to a model):

1. MAITO loads the AI model into memory
2. Available system memory is assessed
3. Settings are optimized for your hardware
4. The model is ready for translation

Time: 2-3 seconds (only happens once until you restart or switch models)

Step 2: Translation Process

When you click Translate:

1. Your text is prepared for the AI model
2. The AI model processes it and generates translation one piece at a time
3. You see results immediately as they stream in
4. Progress updates show you how much is complete
5. Final result appears when complete

Step 3: Streaming Output

One of MAITO’s unique features is streaming translation:

• You don’t wait for the entire translation to complete
• Text appears word-by-word or phrase-by-phrase
• You can read the translation as it’s being generated
• You can cancel anytime if you see issues
• Progress bar shows percentage complete and time remaining

Memory and Performance

Memory Requirements

What the AI Model Needs:

• Model file storage: ~2.5GB on disk
• RAM for model: ~2.5GB when loaded
• RAM for processing: ~0.5-2GB depending on text length
• Total recommended: 8GB system RAM for comfortable operation

Automatic Optimization:

• MAITO automatically adjusts settings based on your available RAM
• On systems with less memory, context window is reduced
• Prevents crashes and ensures smooth operation
• You don’t need to configure anything manually

Translation Speed

Typical Performance:

• Short texts (100 words): 5-10 seconds
• Emails (300 words): 15-30 seconds
• Articles (1,000 words): 1-2 minutes
• Long documents (5,000 words): 5-10 minutes

What Affects Speed:

• Your CPU/GPU speed
• Available system RAM
• Text complexity
• GPU acceleration (DML models are 2-3x faster)

GPU Acceleration

MAITO supports GPU acceleration through DirectML:

Benefits:

• 2-3x faster translation compared to CPU-only
• Uses your graphics card’s parallel processing power
• Works with NVIDIA, AMD, and Intel GPUs
• Automatic fallback to CPU if GPU is busy

Requirements:

• Modern graphics card (most GPUs from 2016+ work)
• Up-to-date graphics drivers
• Windows 10/11
• DML variant of the model (separate download)

Quality and Consistency

Deterministic Translation

MAITO’s local translation is configured for consistent, predictable results:

• Same input text always produces the same translation
• No randomness or variation between runs
• Optimized for accuracy over creativity
• Focused on clear, understandable translations

Error Prevention

MAITO includes safeguards against common AI issues:

Repetition Detection:

• Monitors for endless repetition loops
• Automatically stops if detected
• Shows user-friendly error message
• You can simply try again

Memory Safety:

• Pre-checks available memory before loading models
• Automatically adjusts settings for your hardware
• Prevents out-of-memory crashes
• Graceful degradation on lower-end systems

Document Translation Pipelines

The local engine translates documents as well as text. A document is never handed to the model whole - it is taken apart, translated piece by piece, and reassembled. Each format has its own pipeline.

Plain Text (.txt)

The simplest path. MAITO detects the file’s encoding (including byte-order marks, UTF-16 and legacy Windows code pages), splits the content into chunks at sentence and paragraph boundaries, translates each chunk, and writes the result back with the original line endings intact.

Word (.docx, .doc)

The Word pipeline walks the document tree and emits one translation unit per paragraph - body text, table cells, headers, footers, comments and footnotes included.

The hard part is formatting. A single paragraph can contain a dozen differently-styled fragments (“runs”): a bold word, an italic phrase, a hyperlink. To keep them attached to the right words after translation, MAITO inserts invisible boundary markers into the text it sends to the model, then validates that they came back intact and uses them to reapply the original formatting. When a model mangles the markers beyond recognition, that paragraph falls back to a single uniform style rather than producing garbled output.

Along the way the pipeline also accepts tracked changes before translating, and flips paragraph direction for right-to-left target languages.

A .doc file is converted to the modern format and saved as .docx.

PDF (hybrid)

PDF is the hardest format, because a PDF describes where ink goes on a page rather than what the document is. MAITO therefore does not edit the PDF directly. Instead:

1. Preflight - reject the cases that cannot work: encrypted files, and image-only scans with no text layer
2. Convert - turn the PDF into a Word document using a third-party component from GroupDocs
3. Translate - run that Word document through the Word pipeline above
4. Render - convert the translated Word document back to PDF

Every conversion step loses a little structure, which is why PDF results are best-effort and why annotations and form fields typically do not survive. Skipping step 4 - the Save as .docx checkbox - gives you a cleaner, editable result.

Step 2 is also the one part of local translation that touches the network: the conversion component contacts its vendor to activate. No document content is transmitted. See Privacy and Security.

Shared Machinery

All three pipelines share the same supporting parts:

• Translation lock - one translation at a time, process-wide. A running document blocks text translation, webpage translation and the global hotkey until it completes
• Segment cache - identical paragraphs within a document are translated once and reused, which pays off well on repetitive documents
• Adaptive time estimate - the first estimate comes from your device rating, then MAITO switches to your own measured throughput after a few segments, and remembers it across sessions per model, format and language pair
• Output cleanup - strips conversational preambles (“Sure, here is your translation…”) and detects paragraphs the model echoed back untranslated
• Working directory - intermediate files live in %LOCALAPPDATA%\Pangaia Software\MAITO\DocumentTranslation\, one folder per run, removed on completion and swept after seven days if something goes wrong

Advanced Settings

Document translation exposes several power-user toggles that have no Settings UI. They live in settings.json under %APPDATA%\Pangaia Software\MAITO\:

Privacy and Security

Everything Stays Local

What Never Leaves Your Computer:

• The text you translate
• The documents you translate
• Your translation history
• Language preferences
• Any sensitive content

What’s Logged On Your Device:

• Error conditions and generic events (for troubleshooting)

Offline Operation

Once models are downloaded:

• No internet connection required
• Works on planes, trains, remote locations
• No dependence on cloud services
• No monthly usage limits or quotas

User Interface Integration

Real-Time Updates

While translating, you see:

• Streaming text: Translation appears as it’s generated
• Progress bar: Visual indicator of completion
• Percentage: Exact progress (e.g., “47% complete”)
• Time estimate: Remaining time based on current speed
• Tokens/second: Real-time performance metric

Cancellation

At any time during translation:

• Press Escape key to stop
• Click the Cancel button
• Translation stops immediately
• Partial results are preserved
• No waiting for completion

Technical Foundation

Technology Stack

MAITO’s local translation is built on:

• ONNX Runtime: Microsoft’s high-performance AI inference engine
• DirectML: Windows GPU acceleration technology
• Transformer Models: Modern neural network architecture for language
• INT4 Quantization: Efficient 4-bit model weights for smaller size

Model Format

• Format: ONNX (Open Neural Network Exchange)
• Quantization: INT4 (4-bit integer weights)
• Size: 2.5GB (vs 8-10GB for full precision)
• Quality: Minimal accuracy loss vs full precision
• Performance: Faster inference, lower memory usage

Configuration Options

Available Settings

In MAITO Settings → Translation → Local Engine:

• Default Engine: Choose DeepL or Local as default
• Model Selection: Choose which AI model to use
• Max Characters: Set character limit per translation (default: 4,000)
• Model Management: Download, remove, or benchmark models

Automatic Adjustments

MAITO automatically handles:

• Memory allocation based on available RAM
• GPU vs CPU selection based on hardware
• Context window sizing for long texts
• Error recovery and retry logic

Performance Metrics

What MAITO Tracks

During and after translation:

• Tokens per second: Generation speed
• Total time: Complete translation duration
• Character count: Input and output lengths
• Memory usage: RAM consumption

Using Metrics

Performance metrics help you:

• Compare different models
• Understand hardware capabilities
• Identify potential issues
• Optimize your workflow

Common Questions

Q: Why does initialization take 2-3 seconds?

A: Loading a 2.5GB AI model into memory takes time. MAITO caches the model, so subsequent translations are faster. However, if you don’t use MAITO for a while, the model is removed to free up your device’s memory.

The faster your drive is (preferably a NVME SSD), the less time this take.

Q: Can I use local translation for very long documents?

A: Yes. MAITO automatically handles texts of any length through intelligent processing, and whole Word, PDF and text files through the document pipelines described above. No manual chunking required. The practical limit is patience rather than size: a long PDF on a modest machine can take well over half an hour, and it blocks other translation while it runs.

Q: Why is GPU (DML) version faster?

A: GPUs are designed for parallel processing, which is perfect for AI inference. The DML version utilizes your graphics card’s power for 2-3x speed improvement.

Q: Does it work offline?

A: Yes, with one documented exception. Once models are downloaded, text, Word and plain-text translation need no internet connection at all. PDF translation activates a third-party conversion component that pings its vendor; it tolerates being offline for up to seven days at a stretch.

Q: How much RAM do I really need?

A: 8GB is recommended. MAITO can work with less (down to 4-6GB) but will automatically reduce capabilities to prevent crashes.

What’s Next?

Learn more about MAITO’s local translation:

• Rosetta Models - MAITO’s specialized translation models
• Performance Optimization - Get the best performance
• Model Selection Guide - Choose the right model
• What is Local Translation? - Beginner-friendly overview

MAITO’s local translation combines modern AI technology with user-friendly design to bring powerful, private translation to your desktop. No cloud required, no subscriptions, no compromises on privacy.