Community Wishlist Survey 2016/CodeMirror

The CodeMirror project (also known as wikitext editor syntax highlighting) aims to make it easier for contributors to edit wikitext by using colors and shade to make it easier to visually separate article text from the code for links, references and templates. This will help editors working on paragraphs with many references, or complicated templates where it's easy to miss a curly brace and mess up the template. We are using Extension:CodeMirror for this.

Tracked in Phabricator:
Task T259059
Tracked in Phabricator:
Task T101246 resolved

Screenshot of Syntax Highlighting in English. Screenshot of Syntax Highlighting in Hindi. Screenshot of Syntax Highlighting with the popup for first-time users.

The syntax highlighting is available in the classic wikitext editor as a button () that appears in the toolbar. It is also available in the 2017 Wikitext editor as an option that appears in the "burger menu icon" dropdown menu. The user can switch the highlighting on and off using that button. This is done via a hidden preference, so if you turn it off then it will stay off, until you turn it on again. After activating for the very first time, highlighting is set to off, so you have to switch it on manually to make it work.

The feature was primarily built by volunteer developer Pastakhov; Community Tech helped him to finish the feature, and get it deployed. On the Community Tech side, Niharika and Ryan did the bulk of the work, with help from the Editing team's Ed Sanders on integration with Visual Editor's wikitext mode.

As of September 2023, Community Tech is working on upgrading to CodeMirror 6, which will bring great enhancements such as RTL support, improved performance, accessiblity, and mobile support.

Rationale

We have decided to use Extension:CodeMirror as the syntax highlighting tool.

Here are the advantages that CodeMirror offers:

  • Much faster performance, it's been heavily optimized
  • Uses "wysiwyg"-style effects, to make it easier to see the impact of the wikitext elements -- for example, bold words are rendered in bold, italics in italics.
  • Section headings are bigger, which helps the editor navigate on the screen.
  • Links are underlined and blue, which makes them more recognizable as links. When there's displaytext that's different from the link, the link is in blue, and the displaytext after the pipe is black, to indicate that's the part that will be seen as text.
  • Uses bold colored text for template parameters, light colored text for entries.
  • Uses light background highlighting on links, and not on other elements. Too much background highlighting can make the wikitext harder to scan.
  • Waits for the editor to finish typing before it changes highlighting colors, to allow the editor to finish typing the link or the bold/italics without making everything below that line flash as different colors.

The other tools we looked at before deciding on CodeMirror were wikEd by Cacycle, and Syntax highlighter by Remember the dot.

Status Updates

March 26, 2024

CodeMirror 6 is now deployed to Test Wikipedia. We invite users to try it out and provide feedback. It will be slowly deployed to more wikis in the coming weeks.

September 13, 2023

Community Tech is working on upgrading the extension to use the latest version of the CodeMirror library. This promises to bring many enhancements such as RTL support, improved performance, accessibility, and mobile support.

March 9, 2022

Bracket matching and improved syntax highlighting colors will be implemented on all wikis (except English Wikipedia (March 16)). (more information)

June 14, 2018

Syntax highlighting has graduated from Beta Feature status and is now available for all users inside the 2010 Wikitext Editor on all left-to-right language wikis, in addition to all languages in the 2017 Wikitext Editor. The feature can be enabled during any edit by selecting the 🖋 Highlight icon in the toolbar.

We hope this feature makes it easier to edit in wikitext and debug malformed markup. We'd love to hear your feedback on the talkpage.

May 30, 2018

Syntax highlighting is now available for all users inside the 2017 Wikitext Editor on all wikis. In the 2010 wikitext editor we have decided to not fix any of the edge case defects for Internet Explorer or Edge, and we are not going to support IME editing (phab:T194102).

Community Tech is currently focusing all attention on another 6-week project so development will resume in July. We hope to have the feature released by the end of July or early August. You can track the rollout here: phab:T185030.

April 30, 2018

Syntax highlighting will be available without a beta feature for 2017 Wikitext Editor on all Wikimedia projects (LTR and RTL) this week.

April 26, 2018

We are nearly ready to graduate this feature from beta on the 2017 wikitext editor for both LTR and RTL languages and the 2010 wikitext editor for LTR languages. We're been working on fixing the last bout of bugs from different browsers+editor combination.

When graduated from beta, the feature will be available for all users inside the wikitext editor, but will still require enabling by clicking the ✍️ highlighter icon in the toolbar or ≡ menu.

March 7, 2018

"Wikitext editor syntax highlighting" has survived as a beta feature for more than six months now. The influx of bug reports we saw when it was first launched has been reduced to a trickle. There are some outstanding bugs that the team is working on fixing.

Unfortunately, work on getting RTL support is still pending. The issues with RTL lie in the underlying library for the extension, CodeMirror, and need support from the library maintainer to get fixed. The library is undergoing a rewrite after which we will potentially be able to address the RTL issues. The extension is on the Hebrew beta wiki where the RTL issues can be tested.

We're hoping to get "Wikitext editor syntax highlighting" out of beta within the next month. The feature will be available for all to use by default as a toolbar option in the 2010 and 2017 Wikitext editors. You can follow phab:T185030 to stay up-to-date with the developments on this project.

September 19, 2017

Happy to report that a couple important bugs are fixed! Ctrl-F browser search now works for both the regular edit window and the new wikitext mode, and word jumping using the option+arrow key now works on Mac browsers.

Fixing browser spellcheck is close to done; we'll be releasing that fix soon.

We still need to fix an issue with the new wikitext mode not respecting whether the beta feature is turned on or off -- you can turn it off on the Beta features page, but still see it on in the new wikitext mode. You can follow progress on this ticket: phab:T173480.

September 7, 2017

We're currently working on an important bug -- the Ctrl-F browser search only works if the text you're looking for is currently displayed in the edit window, or just below it. We've been investigating this for the last couple of weeks; here's an update on how it's going. You can follow progress on this ticket: phab:T174480.

We were looking at using the search add-on that GitHub uses in their version of CodeMirror, and it's not an acceptable solution for us. It only works when you've got the cursor in the edit window -- if you're not inside the edit window, ctrl-F still gives you the browser search. That's not very discoverable -- for some people, it would just be broken sometimes. The add-on also uses a combination of ctrl-F to start the search and ctrl-G to move from one highlighted word to another, which is also not very discoverable. So that is a dead end for us.

We're now investigating a couple other options, including loading the page outside the edit window in the background, after the text in the edit window is displayed. This might slow things down, so we're testing the performance.

We're still focused on solving this issue; I'm sorry it's not fixed yet.

August 23, 2017

Some feedback from Andy M. on Wikimedia-L to consider:

"My first impression is that it is lacking in contrast – it's far harder, now, to differentiate the various types of content. Indeed the colour pairs used (e.g. #8800CC vs. #AAAAB3) fail WCAG web accessibility guidelines for colour contrast. I realise that choice of styling colours is a "bikeshed" matter, but contrast ratio is a quantifiable and objective accessibility issue. Also, because the script does not load immediately, the larger headings cause the page to "dance" as the script kicks in."

August 2, 2017

Syntax Highlighting has now been released as a Beta feature on all LTR Wikimedia projects! We hope that people love using the new feature. Please leave feedback on the talk page!

We're still working on making it work for RTL languages; you can follow this progress on this ticket: T170001.

Temporarily turned off, see phabricator:T172458.

July 18, 2017

CodeMirror deployed to Simple English Wikipedia as a Beta Feature.

March 22, 2017

Team discussion, playing with CodeMirror on CommTechwiki. Performance is good, as advertised. We're generally happy with the highlighting choices. We found some bugs, and some elements that should be improved -- for example, ref tags aren't highlighted in the current version. Ryan took a lot of notes, and he'll be working on these issues -- either working with the developers, or making adjustments on our own version.

March 21, 2017

Ryan talked to Ed S on the Editing team -- they're interested in including the syntax highlighter in the new wikitext editor. This will require some investigation.

There's a current hacked version by Ed: https://gerrit.wikimedia.org/r/#/c/343878/

This version works by loading CodeMirror underneath the WikiText editor surface, making the text and background of the WikiText editor transparent, and syncing user input from the WikiText editor to the CodeMirror surface as it is entered. The problem is that it's sluggish and has syncing issues -- see phab:T161054 and phab:T161052 for more.

February 27, 2017

There's an existing extension -- Extension:CodeMirror -- that provides wikitext syntax highlighting on demand, adding a button at the top of the wikitext editor to turn highlighting on and off. (This uses a sticky preference so that an editor can turn it on once, and then see it on all future page loads.) The highlighting is acceptably fast, it takes less than a second even on a lengthy article like English WP's Barack Obama page.

There are two different use cases for syntax highlighting -- essentially, it could be a power editor feature, or a new editor training tool. Very active editors want syntax highlighting to help them edit; if this is the use case that we're focusing on, then we can make it a special preference, and let people decide if they want to use it. If we want to focus on helping new editors, then the feature would have to be on by default, and would require much stricter QA. The comments on the Wishlist proposal and votes indicate that people are requesting the power editor feature.

With that in mind -- we're planning to deploy CodeMirror as a special preference for the old wikitext editor. This will need some instruction on the preferences page to explain how to use the button & turn it on.

The VE team is planning to include this in the 2017 wikitext editor as well; we'll talk to Ed S. to see how we can work together. We'll have to make sure we agree on the color choices. In general, we'll be expected to explain the color choices, so we'll look at the existing CodeMirror colors and make sure we understand the rationale.

Another thing we have to figure out -- how will this work outside of en.wp?

Color and style customization

This is a partial list of the CodeMirror CSS classes. The complete list of default CodeMirror styling definitions is on Phabricator

The colors and styles can be overridden by defining your own colors and styles for the classes in the table below on your common.css page (User:Example/common.css).

If you're not familiar with editing CSS, see the section for beginners below.

You can control the appearance of the CodeMirror editing window as a whole using the .CodeMirror-lines CSS class. Use this at your own risk. Some things like changing the line-height will not work in the 2017 wikitext editor. Also be aware that rules added for any of the below CSS classes will have precedence.

CSS Class(es) Element CodeMirror styling (CSS) wikEd's styling (See InitFrameCSS) Schnark's styling Rememberthedot's styling
.cm-mw-skipformatting leading space foo bar  foo bar  foo bar
.cm-mw-list asterisk or number sign * foo bar, # foo bar * foo bar # foo bar * foo bar # foo bar * foo bar # foo bar
.cm-mw-signature,

.cm-mw-hr

signature, horizontal rule ∼∼∼∼, ---- ~~~~, ---- ~~~~, ---- ~~~~, ----
.cm-mw-indenting colons ::: foo bar ::: foo bar ::: foo bar ::: foo bar
.cm-mw-mnemonic HTML entities & & & &
.cm-mw-comment HTML comment <!-- comment --> <!-- comment --> <!-- comment --> <!-- comment -->
.cm-mw-apostrophes-bold,

.cm-mw-apostrophes-italic

bold and italic '''bold''' ''italic'' '''bold''' ''italic'' '''bold''' ''italic'' '''bold''' ''italic''
Headings
pre.cm-mw-section-1 heading (size not changed in NWE) = Heading = = Heading = = Heading = = Heading =
pre.cm-mw-section-2 heading (size not changed in NWE) == Heading == == Heading == == Heading == == Heading ==
pre.cm-mw-section-3 heading === Heading === === Heading === === Heading === === Heading ===
pre.cm-mw-section-4 heading ==== Heading ==== ==== Heading ==== ==== Heading ==== ==== Heading ====
pre.cm-mw-section-5 heading ===== Heading ===== ===== Heading ===== ===== Heading ===== ===== Heading =====
pre.cm-mw-section-6 heading ====== Heading ====== ====== Heading ====== ====== Heading ====== ====== Heading ======
.cm-mw-section-header equals signs ==
Templates
example {{cite book|title=foo bar}} {{cite book|title=foo bar}} {{cite book|title=foo bar}} {{cite book|title=foo bar}}
.cm-mw-template other template content foo bar
.cm-mw-template-name template name cite book cite book
.cm-mw-template-argument-name template parameters title= title=
.cm-mw-template-delimiter pipe | |
.cm-mw-template-bracket opening/closing brackets {{ }} {{ }}
.cm-mw-template-ground

.cm-mw-template2-ground
.cm-mw-template3-ground

whole template

(number is the level of nesting)

{{foo|{{cite book|title=foo bar}}}}
Variables
example {{{foo|bar}}} {{{foo|bar}}} {{{foo bar}}} {{{foo bar}}}
.cm-mw-templatevariable variable bar
.cm-mw-templatevariable-name triple-braced variable name foo
.cm-mw-templatevariable-bracket opening/closing brackets {{{ }}}
.cm-mw-templatevariable-delimiter pipe |
Parser functions
example {{#assessment: A | High }} {{PAGENAME}}
.cm-mw-parserfunction parser function parameters High
.cm-mw-parserfunction-name parser function name PAGENAME #invoke
.cm-mw-parserfunction-bracket opening/closing brackets {{ }}
.cm-mw-parserfunction-delimiter delimiter :
Extension tags
Note: The extensions can use own styles and rules for highlighting text inside own tags. For example the Cite extension highlights text inside <ref> tags as a wikitext.
example <poem style="padding: 1em;">string
whole line
string</poem>
<ref name="foo"> </ref> <tag>...</tag> <tag>...</tag>
pre.cm-mw-exttag whole line inside tags
whole line
.cm-mw-exttag text inside tags string
.cm-mw-exttag-name tag name poem
.cm-mw-exttag-bracket opening/closing brackets < >
.cm-mw-exttag-attribute reference attributes style="padding: 1em;"
HTML tags
example <span style="foo">foo bar</span> <span style="foo">foo bar</span> <tag style="foo">foo bar</tag> <tag style="foo">foo bar</tag>
.cm-mw-htmltag-name HTML tag span
.cm-mw-htmltag-bracket opening/closing brackets < >
.cm-mw-htmltag-attribute parameters and values style="foo"

superscript subscript

foo<sup>bar</sup> foo<sub>bar</sub>

Foo<sup>bar</sup>
Foo<sub>bar</sub>

Pre-formatted text
pre.cm-mw-tag-pre,

.cm-mw-tag-pre

pre-formatted text <pre>foo bar</pre> <pre>foo bar</pre> <tag>foo bar</tag> <tag>foo bar</tag>
pre.cm-mw-tag-nowiki,

.cm-mw-tag-nowiki

nowiki text <nowiki>foo bar</nowiki> <nowiki>foo bar</nowiki> <tag>foo bar</tag> <tag>foo bar</tag>
Internal links
example

[[foo bar]] [[foo#section|bar]]

[[foo bar]] [[foo#section|bar]]

[[foo bar]] [[foo#section|bar]]

[[foo bar]] [[foo#section|bar]]

.cm-mw-link-pagename internal link target foo bar
.cm-mw-link # character #section
.cm-mw-link-tosection characters after the # #section
.cm-mw-link-bracket opening/closing brackets [[ ]]
.cm-mw-link-text link label (no additional formatting)
.cm-mw-link-delimiter pipe |
.cm-mw-pagename internal link targets foo bar
External links
example

https://www.example.org [https://www.example.org] [https://www.example.org example.org]

https://www.example.org [https://www.example.org] [https://www.example.org example.org]

https://example.org [https://example.org] [https://example.org example.org]

https://example.org [https://example.org] [https://example.org example.org]

.cm-mw-extlink,

.cm-mw-free-extlink

external link target www.example.org
.cm-mw-extlink-protocol,

.cm-mw-free-extlink-protocol

protocol https://
.cm-mw-extlink-text link label (no additional formatting)
.cm-mw-extlink-bracket opening/closing brackets [ ]
Tables
example

{| class="wikitable" |- ! Header text !! Header text |- | Example || Example |}

{| class="wikitable" |- ! Header text !! Header text  |- | Example || Example  |}

{| class="wikitable" |- ! Header text !! Header text  |- | Example || Example  |}

{| class="wikitable"
|-
! Header text !! Header text
|-
| Example || Example
|}

.cm-mw-table-bracket opening/closing brackets and pipe {| |}
.cm-mw-table-delimiter row and column separators | |- !
.cm-mw-table-definition table classes and styles class="wikitable"
.cm-mw-table-caption header cell text !caption
Non-CodeMirror
magic words __TOC__ __TOC__ __TOC__
semi-colon (d.term) ; foo bar
(treated as extension tag) math <math>\sum_{i=0}^\infty 2^{-i}</math> <math>\sum_{i=0}^\infty 2^{-i}</math> <tag>foo bar</tag> <tag>foo bar</tag>
(treated as extension tag) syntaxhighlight (former score) <syntaxhighlight>...</syntaxhighlight> <syntaxhighlight>...</syntaxhighlight> <tag>foo bar</tag> <tag>foo bar</tag>
(treated as internal link) category [[category:foo bar]] [[category:foo bar]] [[category:foo bar]] [[category:foo bar]]
file preview (file preview at the right-hand edge)
character disambig

(nbsp thinsp ndash mdash minus etc.)

  (screenshot) -

How to do it for those who have very little understanding of css

  • Open the page [[Special:MyPage/common.css]] on the wiki where you want this to apply (use m:Special:MyPage/global.css for all wikis alltogether instead).
  • An example is shown for lists (* and #). Add a line
    .cm-mw-list {background-color: green; color: red;}
    
    at the bottom (the cm-mw-list comes from the second table line in the table above). Save. Open some highlighted edit page and try lists - * and #. Green and red can be substituted by the colours you prefer, for example rgb #123456.
  • Save and test by viewing an edit window on the same wiki where there is a list so you can see if it is working as expected. (the background of the symbol should be green and the symbol red).
  • Edit your common.css to change the colours to whatever you prefer, and save again.
  • Do the same thing for all the changes you want, using the code from the first column of the table and the appropriate colours of your choice.
  • If you need to change to a different set of colours for special purposes occasionally, you can comment out the one set of css code using regular /* ... */ block comments and add another set. To change back, switch the comment code from the one set to the other. This is a bit of a pain, but better than not having the choice.
  • If you prefer some color scheme on many wikis and another on one or two, use local file as explained on them – it will override the global one.