Sembr: Semantic Line Break (SemBr) Paragraph Reflow

Sembr rewrites already-rendered paragraph text into one-sentence-per-line form (“semantic line breaks”): soft-wrapped lines are joined back into logical sentences, then newlines are re-inserted at sentence boundaries. CommonMark renderers collapse single newlines inside a paragraph, so the rendered output is unchanged – this is purely a source-legibility transform (sentence-granular diffs and edits).

Off by default; opt in via the sembr plugin option ([plugin.myst] sembr = true in .mdformat.toml or --sembr on the CLI); Postprocessor.postprocess_paragraph is the caller.

The splitter is deliberately conservative: a boundary requires sentence-final punctuation, a following sentence opener, and must not sit inside a code span or inline math, after an abbreviation, single-letter initial, or bare number, or where the new line would re-parse as block syntax (ordered-list interrupts). A missed split costs nothing; a wrong split could change the AST (mdformat’s validation would then reject the whole reformat).

The patterns live in a my.regex.RegexStore (wikiparse’s idiom): named, composable definitions joined by subroutine invocation, the abbreviation list condensed into a prefix-factored alternation tree, and compilation deferred to first use (lazy load), so importing this module stays cheap.

Semantic Line Break (SemBr) Paragraph Reflow.

Sembr rewrites already-rendered paragraph text into one-sentence-per-line form (“semantic line breaks”): soft-wrapped lines are joined back into logical sentences, then newlines are re-inserted at sentence boundaries. CommonMark renderers collapse single newlines inside a paragraph, so the rendered output is unchanged – this is purely a source-legibility transform (sentence-granular diffs and edits).

Off by default; opt in via the sembr plugin option ([plugin.myst] sembr = true in .mdformat.toml or --sembr on the CLI); Postprocessor.postprocess_paragraph is the caller.

The splitter is deliberately conservative: a boundary requires sentence-final punctuation, a following sentence opener, and must not sit inside a code span or inline math, after an abbreviation, single-letter initial, or bare number, or where the new line would re-parse as block syntax (ordered-list interrupts). A missed split costs nothing; a wrong split could change the AST (mdformat’s validation would then reject the whole reformat).

The patterns live in a my.regex.RegexStore (wikiparse’s idiom): named, composable definitions joined by subroutine invocation, the abbreviation list condensed into a prefix-factored alternation tree, and compilation deferred to first use (lazy load), so importing this module stays cheap.

myform.Sembr.CLOSERS = '["\'’”)\\]]'

Closing punctuation that may trail a sentence-final mark (straight/curly quotes, brackets).

myform.Sembr.OPENERS = '["\'‘“(\\[*_]'

Opening punctuation that may lead the next sentence (straight/curly quotes, brackets, emphasis).

myform.Sembr.ABBREVS = ('mr', 'mrs', 'ms', 'dr', 'prof', 'sr', 'jr', 'esq', 'hon', 'st', 'mt', 'pres', 'gov', 'sen', 'rep', 'supt', 'gen', 'col', 'maj', 'capt', 'lt', 'sgt', 'adm', 'cmdr', 'fig', 'figs', 'eq', 'eqs', 'no', 'nos', 'vol', 'vols', 'ch', 'sec', 'ver', 'rev', 'ed', 'eds', 'trans', 'impl', 'op', 'cit', 'loc', 'sc', 'para', 'pp', 'ff', 'vs', 'etc', 'al', 'cf', 'ca', 'approx', 'resp', 'viz', 'ibid', 'i\\.e', 'e\\.g', 'univ', 'dept', 'inc', 'ltd', 'co', 'corp', 'assn', 'bros', 'jan', 'feb', 'mar', 'apr', 'jun', 'jul', 'aug', 'sep', 'sept', 'oct', 'nov', 'dec')

honorifics, latinisms, common scholarly/technical abbreviations, and month names. Matched case-insensitively, so keep every entry lowercase; dotted forms (i.e.) escape their inner period.

Type:

Dotted tokens that end in a period without ending a sentence

myform.Sembr.SEMBR_RGXS = <RegexStore w/ split, block_interrupt, and 6 more... (options=['force_reinvocations', 'lazy_load', 'autostrip_spaces'])>

The splitter’s pattern store. Lists concatenate with an empty separator; _abbrev is a condensed (prefix-factored), case-insensitive alternation invoked as a subroutine by split. code_span writes its backreferences as \g<tick> so the store’s re-invocation normalization does not rewrite them into subroutine calls.

class myform.Sembr.Sembr

[STATIC] Reflow rendered paragraph text to one sentence per line.

classmethod reflow(text: str) → str

Reflow rendered paragraph text to one sentence per line.

Parameters:

text – A paragraph node’s rendered Markdown text (possibly multi-line).

Returns:

The text with soft line breaks joined and sentence boundaries newlined.