//languages/com.aardarch.aardink.languages.internal.markdown/MarkdownTokenizer
MarkdownTokenizer
object MarkdownTokenizer : RegexTokenizer
Regex-driven Markdown tokenizer. The editor doesn’t ship a “Heading” token type, so token types are mapped onto visually-distinct existing categories:
- headings → TokenType.Annotation
- bold / italic emphasis markers → TokenType.Keyword
- inline code / fenced code → TokenType.StringLiteral
- links and images → TokenType.FunctionCall
- blockquote markers, list bullets → TokenType.Punctuation
Functions
| Name | Summary |
|---|---|
| canSpanLines | open override fun canSpanLines(lineIndex: Int, tokens: List<Token>): Boolean |
| keyboardToolbarChars | open override fun keyboardToolbarChars(): List<Char> |
| tokenizeFull | open override fun tokenizeFull(text: String): List<Token> |
| tokenizeLines | open override fun tokenizeLines(text: String, dirtyRange: IntRange, previousTokens: List<Token>): List<Token> |