Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions docs/features/grammar-check.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Grammar Check (LanguageTool)

Check spelling, grammar, punctuation, casing and style with a [LanguageTool](https://languagetool.org) server - the public API, or your own installation (for example the [docker image](https://github.com/Erikvl87/docker-languagetool)). Nothing is changed until you apply the fixes you agree with.

- **Menu:** Tools → Grammar check (LanguageTool)...

Unlike [AI Review](ai-review.md), LanguageTool is rule based: it points at an exact span of text and offers concrete replacements, so it never rewrites a line and never invents changes. It is also considerably faster, and with a self-hosted server the subtitle text never leaves your machine.

## Server

Enter the base address of the server in the **Server** box - `https://api.languagetool.org` for the public API, or something like `http://localhost:8010` for a local docker container. The address may include the `/v2` or `/v2/check` suffix; Subtitle Edit strips it.

The circular arrow button next to the box tests the connection and reloads the language list, which also happens automatically when the window opens.

> The public API is rate limited and rejects large amounts of text from one address. For a whole subtitle, a self-hosted server is the better choice.

## Language

The language list comes from the server. It defaults to the auto-detected language of the subtitle, and falls back to the server's own detection if you leave it on **Auto**. For languages with variants (English, German, Portuguese...) pick the right one - the variants disagree about spelling and punctuation.

**Picky** turns on the stricter rules LanguageTool leaves off by default: redundancy, wordiness and typography suggestions. Useful for a final polish, noisy for a first pass.

## Checking

Press **Check**. The subtitle is sent in batches and issues appear in the grid while the check runs - press **Stop** to keep what has been found so far.

Each row shows:

- **Apply** — checkbox deciding whether the fix is applied
- **Line number** and a **category** tag (spelling, grammar, punctuation, casing, style)
- **Issue** — LanguageTool's short description of the rule that fired
- **Before / After** — with the changed words highlighted

Selecting a row shows the full explanation below the grid. When LanguageTool offers more than one replacement, a drop-down appears next to the explanation - pick the one you want and the After column follows.

Filter the grid with the category chips. Press **Apply N fixes** to apply the checked rows; several fixes on the same line are applied together, and the whole run is a single undo step (Ctrl+Z reverts everything).

Spelling, grammar, punctuation and casing fixes are checked by default. Style suggestions are a matter of taste, so they start unchecked. Rules that only point at a problem without offering a replacement cannot be checked at all - correct those by hand.

## Formatting tags and line breaks

Italic and font tags, ASSA override blocks such as `{\an8}` and music symbols are sent as markup: LanguageTool ignores them instead of reading them as words, but the positions it reports still refer to the original line, so a fix lands exactly where it belongs and the tags stay untouched. On the rare occasion an issue spans a tag or a line break, it is dropped rather than applied - fixing it would break the formatting.

A sentence continuing into the next subtitle is checked as one sentence, the way it reads on screen, so agreement errors spread over two lines are found. Fixes are still applied per line, so timing and reading speed are unaffected.

## Settings

The **Settings** button holds the options that are rarely changed:

- **User name** and **API key** — only needed for a LanguageTool premium account, or a server that requires credentials
- **Disabled rules** — comma separated rule ids to ignore, e.g. `WHITESPACE_RULE,UPPERCASE_SENTENCE_START`. The rule id of a selected issue is shown in the explanation line
- **Lines per request** — how many subtitle lines go into one request (25 by default)
1 change: 1 addition & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ Subtitle Edit is a free, open-source editor for video subtitles. This is the doc

### Tools
- [AI Review](features/ai-review.md) — Proofread with a local LLM (llama.cpp, Ollama, or any OpenAI-compatible endpoint)
- [Grammar Check](features/grammar-check.md) — Spelling, grammar, punctuation and style via a LanguageTool server (public API or self-hosted)
- [Fix Common Errors](features/fix-common-errors.md) — Automatic error detection and fixing
- [Check and Fix Netflix Errors](features/netflix-errors.md) — Netflix quality checks, proposed fixes, and CSV reports
- [Batch Convert](features/batch-convert.md) — Convert multiple subtitle files
Expand Down
210 changes: 210 additions & 0 deletions src/libuilogic/Grammar/LanguageToolAnnotatedText.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,210 @@
using Nikse.SubtitleEdit.Core.Common;
using System.Text;
using System.Text.Json;
using System.Text.RegularExpressions;

namespace Nikse.SubtitleEdit.UiLogic.Grammar;

/// <summary>
/// Builds the annotated document that LanguageTool's /v2/check "data" parameter takes, and maps the
/// offsets it answers with back onto the subtitle lines that went in.
///
/// Formatting tags, music symbols and the line break inside a subtitle are sent as "markup" so
/// LanguageTool neither spell checks them nor reads them as words, while the offsets it returns stay
/// relative to the complete text - markup included - so a replacement can be applied straight to the
/// original line instead of rebuilding it from a cleaned copy.
///
/// Consecutive lines go into one document, joined with a space where the sentence continues into the
/// next line and with a blank line where it does not, so grammar is checked the way the text reads on
/// screen rather than one subtitle at a time.
/// </summary>
public class LanguageToolAnnotatedText
{
// Same shapes the AI review protocol treats as tags, plus the line break and music symbols.
private static readonly Regex MarkupRegex = new(@"<[^>]*>|\{\\[^}]*\}|\r\n|\n|[♪♫♬♩]", RegexOptions.Compiled);

private readonly List<TextSpan> _lineSpans;
private readonly List<TextSpan> _markupSpans;

/// <summary>The value for the "data" parameter.</summary>
public string Json { get; }

/// <summary>The complete text - markup included - that the returned offsets refer to.</summary>
public string Text { get; }

public bool IsEmpty => Text.Trim().Length == 0;

private LanguageToolAnnotatedText(string json, string text, List<TextSpan> lineSpans, List<TextSpan> markupSpans)
{
Json = json;
Text = text;
_lineSpans = lineSpans;
_markupSpans = markupSpans;
}

public static LanguageToolAnnotatedText Build(IReadOnlyList<string> lines)
{
var text = new StringBuilder();
var lineSpans = new List<TextSpan>(lines.Count);
var markupSpans = new List<TextSpan>();

using var stream = new MemoryStream();
using (var writer = new Utf8JsonWriter(stream))
{
writer.WriteStartObject();
writer.WriteStartArray("annotation");

for (var i = 0; i < lines.Count; i++)
{
if (i > 0)
{
// A separator of our own, so it is markup: interpreted as a space while the
// sentence runs on, as a paragraph break once it has ended.
const string separator = "\n";
WriteMarkup(writer, separator, EndsSentence(lines[i - 1]) ? "\n\n" : " ");
markupSpans.Add(new TextSpan(text.Length, separator.Length));
text.Append(separator);
}

var start = text.Length;
AppendLine(writer, text, markupSpans, lines[i] ?? string.Empty);
lineSpans.Add(new TextSpan(start, text.Length - start));
}

writer.WriteEndArray();
writer.WriteEndObject();
}

return new LanguageToolAnnotatedText(Encoding.UTF8.GetString(stream.ToArray()), text.ToString(), lineSpans, markupSpans);
}

/// <summary>
/// Maps a match onto the line it belongs to. False when the match spans more than one line or
/// touches markup - those cannot be applied without mangling a tag or a line break.
/// </summary>
public bool TryMapToLine(int offset, int length, out int lineIndex, out int lineOffset)
{
lineIndex = -1;
lineOffset = -1;
if (length <= 0 || offset < 0 || offset + length > Text.Length)
{
return false;
}

for (var i = 0; i < _lineSpans.Count; i++)
{
var span = _lineSpans[i];
if (offset < span.Start || offset + length > span.Start + span.Length)
{
continue;
}

if (OverlapsMarkup(offset, length))
{
return false;
}

lineIndex = i;
lineOffset = offset - span.Start;
return true;
}

return false;
}

private bool OverlapsMarkup(int offset, int length)
{
foreach (var span in _markupSpans)
{
if (offset < span.Start + span.Length && span.Start < offset + length)
{
return true;
}
}

return false;
}

/// <summary>True when the line finishes a sentence, i.e. the next line starts a new one.</summary>
public static bool EndsSentence(string text)
{
var s = HtmlUtil.RemoveHtmlTags(text ?? string.Empty, true).TrimEnd();
if (s.Length == 0)
{
return true;
}

// skip closing quotes/brackets after the sentence-final mark
var i = s.Length - 1;
while (i >= 0 && "\"'’”»)]".IndexOf(s[i]) >= 0)
{
i--;
}

if (i < 0)
{
return true;
}

return ".!?…".IndexOf(s[i]) >= 0;
}

private static void AppendLine(Utf8JsonWriter writer, StringBuilder text, List<TextSpan> markupSpans, string line)
{
var index = 0;
foreach (Match match in MarkupRegex.Matches(line))
{
if (match.Index > index)
{
var plain = line.Substring(index, match.Index - index);
WriteText(writer, plain);
text.Append(plain);
}

// A line break inside a subtitle is a space to the reader, tags and music symbols are nothing.
var isNewLine = match.Value == "\n" || match.Value == "\r\n";
WriteMarkup(writer, match.Value, isNewLine ? " " : null);
markupSpans.Add(new TextSpan(text.Length, match.Length));
text.Append(match.Value);
index = match.Index + match.Length;
}

if (index < line.Length)
{
var rest = line.Substring(index);
WriteText(writer, rest);
text.Append(rest);
}
}

private static void WriteText(Utf8JsonWriter writer, string value)
{
writer.WriteStartObject();
writer.WriteString("text", value);
writer.WriteEndObject();
}

private static void WriteMarkup(Utf8JsonWriter writer, string value, string? interpretAs)
{
writer.WriteStartObject();
writer.WriteString("markup", value);
if (interpretAs != null)
{
writer.WriteString("interpretAs", interpretAs);
}

writer.WriteEndObject();
}

private readonly struct TextSpan
{
public TextSpan(int start, int length)
{
Start = start;
Length = length;
}

public int Start { get; }
public int Length { get; }
}
}
Loading