Hecto, Chapter 7: Syntax Highlighting

Hecto, Chapter 7: Syntax Highlighting

November 8, 2019

Previous chapter - Overview - Appendices

We are almost done with our text editor - we’re only missing some syntax highlighting.

Colorful Digits

Let’s start by just getting some color on the screen, as simply as possible. We’ll attempt to highlight numbers by coloring each digit character red.

src/row.rs CHANGED
@@ -1,5 +1,6 @@
1
1
  use crate::SearchDirection;
2
2
  use std::cmp;
3
+ use termion::color;
3
4
  use unicode_segmentation::UnicodeSegmentation;
4
5
 
5
6
  #[derive(Default)]
@@ -28,10 +29,21 @@ impl Row {
28
29
  .skip(start)
29
30
  .take(end - start)
30
31
  {
31
- if grapheme == "\t" {
32
- result.push_str(" ");
33
- } else {
34
- result.push_str(grapheme);
32
+ if let Some(c) = grapheme.chars().next() {
33
+ if c == '\t' {
34
+ result.push_str(" ");
35
+ } else if c.is_ascii_digit() {
36
+ result.push_str(
37
+ &format!(
38
+ "{}{}{}",
39
+ termion::color::Fg(color::Rgb(220, 163, 163)),
40
+ c,
41
+ color::Fg(color::Reset)
42
+ )[..],
43
+ );
44
+ } else {
45
+ result.push(c);
46
+ }
35
47
  }
36
48
  }
37
49
  result

See this step on github

We have now converted our grapheme into a character, so that we can use is_ascii_digit - which determines whether or not a character is a digit. If it is, we change the foreground color to a shade of red and reset it immediately afterwards.

Refactor syntax highlighting

Now we know how to color text, but we’re going to have to do a lot more work to actually highlight entire strings, keywords, comments, and so on. We can’t just decide what color to use based on the class of each character, like we’re doing with digits currently. What we want to do is figure out the highlighting for each row of text before we display it, and then rehighlight a line whenever it gets changed. What makes things more complicated is that the highlighting depends on characters currently out of view - for instance, if a String starts to the left of the currently visible portion of the row, we want to treat a " on screen as the end of a string, and not as the start of one. Our current strategy to look at each visible character is therefore not sufficient.

Instead, we are going to store the highlighting of each character of a row in a vector. Let’s start by adding an enum which will hold our different highlighting types as well as a vector to hold them.

src/highlighting.rs ADDED
@@ -0,0 +1,4 @@
1
+ pub enum Type {
2
+ None,
3
+ Number,
4
+ }
src/main.rs CHANGED
@@ -9,6 +9,7 @@
9
9
  )]
10
10
  mod document;
11
11
  mod editor;
12
+ mod highlighting;
12
13
  mod row;
13
14
  mod terminal;
14
15
  pub use document::Document;
src/row.rs CHANGED
@@ -1,3 +1,4 @@
1
+ use crate::highlighting;
1
2
  use crate::SearchDirection;
2
3
  use std::cmp;
3
4
  use termion::color;
@@ -6,6 +7,7 @@ use unicode_segmentation::UnicodeSegmentation;
6
7
  #[derive(Default)]
7
8
  pub struct Row {
8
9
  string: String,
10
+ highlighting: Vec<highlighting::Type>,
9
11
  len: usize,
10
12
  }
11
13
 
@@ -13,6 +15,7 @@ impl From<&str> for Row {
13
15
  fn from(slice: &str) -> Self {
14
16
  Self {
15
17
  string: String::from(slice),
18
+ highlighting: Vec::new(),
16
19
  len: slice.graphemes(true).count(),
17
20
  }
18
21
  }
@@ -112,6 +115,7 @@ impl Row {
112
115
  Self {
113
116
  string: splitted_row,
114
117
  len: splitted_length,
118
+ highlighting: Vec::new(),
115
119
  }
116
120
  }
117
121
  pub fn as_bytes(&self) -> &[u8] {

See this step on github

Highlighting will be controlled by the Document, the rows do not “highlight themselves”. The reason will become clearer as we add more code - essentially, to highlight a row, more information than what is present in the Row is needed. This means that any operation on the Row from the outside can potentially render the highlighting invalid. When developing functionality for Row, we are going to pay special attention so that the worst that can happen is a wrong highlighting (as opposed to a crash).

This is why we are not setting the highlight for example in split.

For now, we’ll focus on highlighting numbers only. So we want every character that’s part of a number to have a corresponding Type::Number value in the highlighting vector, and we want every other value in highlighting to be Type::None.

Let’s create a new highlight function in our row. This function will go through the characters of the string and highlight them by setting each value in the highlighting vector.

src/highlighting.rs CHANGED
@@ -1,4 +1,4 @@
1
1
  pub enum Type {
2
2
  None,
3
3
  Number,
4
- }
4
+ }
src/row.rs CHANGED
@@ -158,4 +158,15 @@ impl Row {
158
158
  }
159
159
  None
160
160
  }
161
+ pub fn highlight(&mut self) {
162
+ let mut highlighting = Vec::new();
163
+ for c in self.string.chars() {
164
+ if c.is_ascii_digit() {
165
+ highlighting.push(highlighting::Type::Number);
166
+ } else {
167
+ highlighting.push(highlighting::Type::None);
168
+ }
169
+ }
170
+ self.highlighting = highlighting;
171
+ }
161
172
  }

See this step on github

The code of highlight is straightforward: If a character is a digit, we push Type::Number, otherwise, we push Type::None.

Now we want to have a function which maps the Type to a color. Let’s implement that as a method of Type:

src/highlighting.rs CHANGED
@@ -1,4 +1,14 @@
1
+ use termion::color;
1
2
  pub enum Type {
2
3
  None,
3
4
  Number,
4
- }
5
+ }
6
+
7
+ impl Type {
8
+ fn to_color(&self) -> impl color::Color {
9
+ match self {
10
+ Type::Number => color::Rgb(220, 163, 163),
11
+ _ => color::Rgb(255, 255, 255),
12
+ }
13
+ }
14
+ }

See this step on github

We are returning red now for numbers and white for all other cases. Now let’s finally draw the highlighted text to the screen!

src/document.rs CHANGED
@@ -16,7 +16,9 @@ impl Document {
16
16
  let contents = fs::read_to_string(filename)?;
17
17
  let mut rows = Vec::new();
18
18
  for value in contents.lines() {
19
- rows.push(Row::from(value));
19
+ let mut row = Row::from(value);
20
+ row.highlight();
21
+ rows.push(row);
20
22
  }
21
23
  Ok(Self {
22
24
  rows,
@@ -42,7 +44,10 @@ impl Document {
42
44
  return;
43
45
  }
44
46
  #[allow(clippy::indexing_slicing)]
45
- let new_row = self.rows[at.y].split(at.x);
47
+ let current_row = &mut self.rows[at.y];
48
+ let mut new_row = current_row.split(at.x);
49
+ current_row.highlight();
50
+ new_row.highlight();
46
51
  #[allow(clippy::integer_arithmetic)]
47
52
  self.rows.insert(at.y + 1, new_row);
48
53
  }
@@ -58,11 +63,13 @@ impl Document {
58
63
  if at.y == self.rows.len() {
59
64
  let mut row = Row::default();
60
65
  row.insert(0, c);
66
+ row.highlight();
61
67
  self.rows.push(row);
62
68
  } else {
63
69
  #[allow(clippy::indexing_slicing)]
64
70
  let row = &mut self.rows[at.y];
65
71
  row.insert(at.x, c);
72
+ row.highlight();
66
73
  }
67
74
  }
68
75
  #[allow(clippy::integer_arithmetic, clippy::indexing_slicing)]
@@ -76,9 +83,11 @@ impl Document {
76
83
  let next_row = self.rows.remove(at.y + 1);
77
84
  let row = &mut self.rows[at.y];
78
85
  row.append(&next_row);
86
+ row.highlight();
79
87
  } else {
80
88
  let row = &mut self.rows[at.y];
81
89
  row.delete(at.x);
90
+ row.highlight();
82
91
  }
83
92
  }
84
93
  pub fn save(&mut self) -> Result<(), Error> {
src/highlighting.rs CHANGED
@@ -5,7 +5,7 @@ pub enum Type {
5
5
  }
6
6
 
7
7
  impl Type {
8
- fn to_color(&self) -> impl color::Color {
8
+ pub fn to_color(&self) -> impl color::Color {
9
9
  match self {
10
10
  Type::Number => color::Rgb(220, 163, 163),
11
11
  _ => color::Rgb(255, 255, 255),
src/row.rs CHANGED
@@ -27,26 +27,27 @@ impl Row {
27
27
  let start = cmp::min(start, end);
28
28
  let mut result = String::new();
29
29
  #[allow(clippy::integer_arithmetic)]
30
- for grapheme in self.string[..]
30
+ for (index, grapheme) in self.string[..]
31
31
  .graphemes(true)
32
+ .enumerate()
32
33
  .skip(start)
33
34
  .take(end - start)
34
35
  {
35
36
  if let Some(c) = grapheme.chars().next() {
37
+ let highlighting_type = self
38
+ .highlighting
39
+ .get(index)
40
+ .unwrap_or(&highlighting::Type::None);
41
+ let start_highlight =
42
+ format!("{}", termion::color::Fg(highlighting_type.to_color()));
43
+ result.push_str(&start_highlight[..]);
36
44
  if c == '\t' {
37
45
  result.push_str(" ");
38
- } else if c.is_ascii_digit() {
39
- result.push_str(
40
- &format!(
41
- "{}{}{}",
42
- termion::color::Fg(color::Rgb(220, 163, 163)),
43
- c,
44
- color::Fg(color::Reset)
45
- )[..],
46
- );
47
46
  } else {
48
47
  result.push(c);
49
48
  }
49
+ let end_highlight = format!("{}", termion::color::Fg(color::Reset));
50
+ result.push_str(&end_highlight[..]);
50
51
  }
51
52
  }
52
53
  result

See this step on github

First, we call highlight everywhere in Document where we modify a row. Then, we refactor our rendering: We get the correct highlighting for the current index (which we obtain by using enumerate). We change the color, append to the string which we return, and change the color back again.

This works, but do we really have to write out an escape sequence before every single character? In practice, most characters are going to be the same color as the previous character, so most of the escape sequences are redundant. Let’s keep track of the current text color as we loop through the characters, and only print out an escape sequence when the color changes.

src/highlighting.rs CHANGED
@@ -1,4 +1,5 @@
1
1
  use termion::color;
2
+ #[derive(PartialEq)]
2
3
  pub enum Type {
3
4
  None,
4
5
  Number,
src/row.rs CHANGED
@@ -26,6 +26,7 @@ impl Row {
26
26
  let end = cmp::min(end, self.string.len());
27
27
  let start = cmp::min(start, end);
28
28
  let mut result = String::new();
29
+ let mut current_highlighting = &highlighting::Type::None;
29
30
  #[allow(clippy::integer_arithmetic)]
30
31
  for (index, grapheme) in self.string[..]
31
32
  .graphemes(true)
@@ -38,18 +39,21 @@ impl Row {
38
39
  .highlighting
39
40
  .get(index)
40
41
  .unwrap_or(&highlighting::Type::None);
41
- let start_highlight =
42
- format!("{}", termion::color::Fg(highlighting_type.to_color()));
43
- result.push_str(&start_highlight[..]);
42
+ if highlighting_type != current_highlighting {
43
+ current_highlighting = highlighting_type;
44
+ let start_highlight =
45
+ format!("{}", termion::color::Fg(highlighting_type.to_color()));
46
+ result.push_str(&start_highlight[..]);
47
+ }
44
48
  if c == '\t' {
45
49
  result.push_str(" ");
46
50
  } else {
47
51
  result.push(c);
48
52
  }
49
- let end_highlight = format!("{}", termion::color::Fg(color::Reset));
50
- result.push_str(&end_highlight[..]);
51
53
  }
52
54
  }
55
+ let end_highlight = format!("{}", termion::color::Fg(color::Reset));
56
+ result.push_str(&end_highlight[..]);
53
57
  result
54
58
  }
55
59
  pub fn len(&self) -> usize {

See this step on github

We use current_highlighting to keep track of what we are currently rendering. When it changes, we add the color change to the render string. We have also moved the ending of the highlighting outside of the loop, so that we reset the color at the end of each line. To allow comparison between Types, we derive PartialEq again.

Colorful search results

Before we start highlighting strings and keywords and all that, let’s use our highlighting system to highlight search results. We’ll start by adding Match to the HighlightingType enum, and mapping it to the color blue in to_color.

src/highlighting.rs CHANGED
@@ -3,12 +3,14 @@ use termion::color;
3
3
  pub enum Type {
4
4
  None,
5
5
  Number,
6
+ Match,
6
7
  }
7
8
 
8
9
  impl Type {
9
10
  pub fn to_color(&self) -> impl color::Color {
10
11
  match self {
11
12
  Type::Number => color::Rgb(220, 163, 163),
13
+ Type::Match => color::Rgb(38, 139, 210),
12
14
  _ => color::Rgb(255, 255, 255),
13
15
  }
14
16
  }

See this step on github

Next, we want to change highlight so that it accepts an optional word. If no word is given, no match is highlighted.

src/document.rs CHANGED
@@ -17,7 +17,7 @@ impl Document {
17
17
  let mut rows = Vec::new();
18
18
  for value in contents.lines() {
19
19
  let mut row = Row::from(value);
20
- row.highlight();
20
+ row.highlight(None);
21
21
  rows.push(row);
22
22
  }
23
23
  Ok(Self {
@@ -46,8 +46,8 @@ impl Document {
46
46
  #[allow(clippy::indexing_slicing)]
47
47
  let current_row = &mut self.rows[at.y];
48
48
  let mut new_row = current_row.split(at.x);
49
- current_row.highlight();
50
- new_row.highlight();
49
+ current_row.highlight(None);
50
+ new_row.highlight(None);
51
51
  #[allow(clippy::integer_arithmetic)]
52
52
  self.rows.insert(at.y + 1, new_row);
53
53
  }
@@ -63,13 +63,13 @@ impl Document {
63
63
  if at.y == self.rows.len() {
64
64
  let mut row = Row::default();
65
65
  row.insert(0, c);
66
- row.highlight();
66
+ row.highlight(None);
67
67
  self.rows.push(row);
68
68
  } else {
69
69
  #[allow(clippy::indexing_slicing)]
70
70
  let row = &mut self.rows[at.y];
71
71
  row.insert(at.x, c);
72
- row.highlight();
72
+ row.highlight(None);
73
73
  }
74
74
  }
75
75
  #[allow(clippy::integer_arithmetic, clippy::indexing_slicing)]
@@ -83,11 +83,11 @@ impl Document {
83
83
  let next_row = self.rows.remove(at.y + 1);
84
84
  let row = &mut self.rows[at.y];
85
85
  row.append(&next_row);
86
- row.highlight();
86
+ row.highlight(None);
87
87
  } else {
88
88
  let row = &mut self.rows[at.y];
89
89
  row.delete(at.x);
90
- row.highlight();
90
+ row.highlight(None);
91
91
  }
92
92
  }
93
93
  pub fn save(&mut self) -> Result<(), Error> {
@@ -140,4 +140,9 @@ impl Document {
140
140
  }
141
141
  None
142
142
  }
143
+ pub fn highlight(&mut self, word: Option<&str>) {
144
+ for row in &mut self.rows {
145
+ row.highlight(word);
146
+ }
147
+ }
143
148
  }
src/editor.rs CHANGED
@@ -150,6 +150,7 @@ impl Editor {
150
150
  } else if moved {
151
151
  editor.move_cursor(Key::Left);
152
152
  }
153
+ editor.document.highlight(Some(query));
153
154
  },
154
155
  )
155
156
  .unwrap_or(None);
@@ -158,6 +159,7 @@ impl Editor {
158
159
  self.cursor_position = old_position;
159
160
  self.scroll();
160
161
  }
162
+ self.document.highlight(None);
161
163
  }
162
164
  fn process_keypress(&mut self) -> Result<(), std::io::Error> {
163
165
  let pressed_key = Terminal::read_key()?;
src/row.rs CHANGED
@@ -127,7 +127,7 @@ impl Row {
127
127
  self.string.as_bytes()
128
128
  }
129
129
  pub fn find(&self, query: &str, at: usize, direction: SearchDirection) -> Option<usize> {
130
- if at > self.len {
130
+ if at > self.len || query.is_empty() {
131
131
  return None;
132
132
  }
133
133
  let start = if direction == SearchDirection::Forward {
@@ -163,15 +163,44 @@ impl Row {
163
163
  }
164
164
  None
165
165
  }
166
- pub fn highlight(&mut self) {
166
+ pub fn highlight(&mut self, word: Option<&str>) {
167
167
  let mut highlighting = Vec::new();
168
- for c in self.string.chars() {
168
+ let chars: Vec<char> = self.string.chars().collect();
169
+ let mut matches = Vec::new();
170
+ let mut search_index = 0;
171
+
172
+ if let Some(word) = word {
173
+ while let Some(search_match) = self.find(word, search_index, SearchDirection::Forward) {
174
+ matches.push(search_match);
175
+ if let Some(next_index) = search_match.checked_add(word[..].graphemes(true).count())
176
+ {
177
+ search_index = next_index;
178
+ } else {
179
+ break;
180
+ }
181
+ }
182
+ }
183
+
184
+ let mut index = 0;
185
+ while let Some(c) = chars.get(index) {
186
+ if let Some(word) = word {
187
+ if matches.contains(&index) {
188
+ for _ in word[..].graphemes(true) {
189
+ index += 1;
190
+ highlighting.push(highlighting::Type::Match);
191
+ }
192
+ continue;
193
+ }
194
+ }
195
+
169
196
  if c.is_ascii_digit() {
170
197
  highlighting.push(highlighting::Type::Number);
171
198
  } else {
172
199
  highlighting.push(highlighting::Type::None);
173
200
  }
201
+ index += 1;
174
202
  }
203
+
175
204
  self.highlighting = highlighting;
176
205
  }
177
206
  }

See this step on github

Before we investigate the changes in highlight, let’s first focus on the other changes: We allow an optional parameter to highlight, which we provide as None everywhere we call highlight. We then add a method called highlight to the document, which triggers a highlighting of all rows in the doc.

We use that method during our search by passing the current query in, and None as soon as the search has been aborted.

Now, what about highlight?

First, we collect all the matches of the word we have in the current row. That is more efficient than performing the search on every character.

We are using two new concepts while doing so. One is while..let, which is similar to if..let, but it loops as long as the condition is satisifed. We use that to loop through our current row, advancing the starting point for our search directly behind the last match on every turn.

We are also using a new method to add 1: checked_add. This function returns an Option with the result if no overflow occured, or None otherwise. Why can’t we use saturating_add here? Well, let’s assume our match happens to be the very last part of the row, and we are also at the end of usize. Then we can’t advance next_index any further with saturading_add, it would return the same result over and over, the while condition would never be false and we would loop indefinitely.

Speaking of loops, we also changed our loop for processing characters. This allows us to consume multiple characters at a time. We use this to push multiple highlighting types at once while we are at the index of a match which we want to highlight.

Try it out, and you will see that all search results light up in your editor.

Colorful numbers

Alright, let’s start working on highlighting numbers properly.

Right now, nubers are highlighted even if they’re part of an identifier, such as the 32 in u32. To fix that, we’ll require that numbers are preceded by a separator character, which includes whitespace or punctuation characters. We can use is_ascii_punctuation and is_ascii_whitespace for that.

src/row.rs CHANGED
@@ -180,7 +180,7 @@ impl Row {
180
180
  }
181
181
  }
182
182
  }
183
-
183
+ let mut prev_is_separator = true;
184
184
  let mut index = 0;
185
185
  while let Some(c) = chars.get(index) {
186
186
  if let Some(word) = word {
@@ -192,12 +192,22 @@ impl Row {
192
192
  continue;
193
193
  }
194
194
  }
195
-
196
- if c.is_ascii_digit() {
195
+ let previous_highlight = if index > 0 {
196
+ #[allow(clippy::integer_arithmetic)]
197
+ highlighting
198
+ .get(index - 1)
199
+ .unwrap_or(&highlighting::Type::None)
200
+ } else {
201
+ &highlighting::Type::None
202
+ };
203
+ if c.is_ascii_digit()
204
+ && (prev_is_separator || previous_highlight == &highlighting::Type::Number)
205
+ {
197
206
  highlighting.push(highlighting::Type::Number);
198
207
  } else {
199
208
  highlighting.push(highlighting::Type::None);
200
209
  }
210
+ prev_is_separator = c.is_ascii_punctuation() || c.is_ascii_whitespace();
201
211
  index += 1;
202
212
  }
203
213
 

See this step on github

We’re adding a new variable prev_separator to check if the last character we saw was a valid separator, after which we want to highlight numbers properly, or any other character, after which we don’t want to highlight numbers.

We are also accessing the previous highlighting so that even if we are not behind a separator, we continue highlighting numbers as numbers - which allows us to highlight numbers with more than one digit.

At the end of the loop, we set prev_is_separator to true if the current character is either an ascii punctiation or a whitespace, otherwise it’s set to false. Then we increment index to consume the character.

Now let’s support highlighting numbers that contain decimal points.

src/row.rs CHANGED
@@ -200,8 +200,9 @@ impl Row {
200
200
  } else {
201
201
  &highlighting::Type::None
202
202
  };
203
- if c.is_ascii_digit()
204
- && (prev_is_separator || previous_highlight == &highlighting::Type::Number)
203
+ if (c.is_ascii_digit()
204
+ && (prev_is_separator || previous_highlight == &highlighting::Type::Number))
205
+ || (c == &'.' && previous_highlight == &highlighting::Type::Number)
205
206
  {
206
207
  highlighting.push(highlighting::Type::Number);
207
208
  } else {

See this step on github

A . character that comes after a character that we just highlighted as a number will now be considered part of the number.

Detect file type

Before we go on to highlight other things, we’re going to add filetype detection to our editor. This will allow us to have different rules for how to highlight different types of files. For example, text files shouldn’t have any highlighting, and Rust files should highlight numbers, strings, chars, comments and many keywords specific to Rust.

Let’s create a struct FileType which will hold our Filetype information for now.

src/document.rs CHANGED
@@ -1,3 +1,4 @@
1
+ use crate::FileType;
1
2
  use crate::Position;
2
3
  use crate::Row;
3
4
  use crate::SearchDirection;
@@ -9,6 +10,7 @@ pub struct Document {
9
10
  rows: Vec<Row>,
10
11
  pub file_name: Option<String>,
11
12
  dirty: bool,
13
+ file_type: FileType,
12
14
  }
13
15
 
14
16
  impl Document {
@@ -24,6 +26,7 @@ impl Document {
24
26
  rows,
25
27
  file_name: Some(filename.to_string()),
26
28
  dirty: false,
29
+ file_type: FileType::default(),
27
30
  })
28
31
  }
29
32
  pub fn row(&self, index: usize) -> Option<&Row> {
src/filetype.rs ADDED
@@ -0,0 +1,18 @@
1
+ pub struct FileType {
2
+ name: String,
3
+ hl_opts: HighlightingOptions,
4
+ }
5
+
6
+ #[derive(Default)]
7
+ pub struct HighlightingOptions {
8
+ pub numbers: bool,
9
+ }
10
+
11
+ impl Default for FileType {
12
+ fn default() -> Self {
13
+ Self {
14
+ name: String::from("No filetype"),
15
+ hl_opts: HighlightingOptions::default(),
16
+ }
17
+ }
18
+ }
src/main.rs CHANGED
@@ -9,6 +9,7 @@
9
9
  )]
10
10
  mod document;
11
11
  mod editor;
12
+ mod filetype;
12
13
  mod highlighting;
13
14
  mod row;
14
15
  mod terminal;
@@ -16,6 +17,7 @@ pub use document::Document;
16
17
  use editor::Editor;
17
18
  pub use editor::Position;
18
19
  pub use editor::SearchDirection;
20
+ pub use filetype::FileType;
19
21
  pub use row::Row;
20
22
  pub use terminal::Terminal;
21
23
 

See this step on github

HighlightingOptions will hold a couple of booleans which determine whether or not a certain type should be highlighted. Since it’s closely related to the file type, we keep both in the same file. For now, we only add numbers to determine whether or not numbers should be highlighted. We use #[derive(Default)] for this struct so that HighlightOptions::default returns HighlightOptions initialized with default values. Since HighlightOptions will only contain bools, and the default for bools is false, this suits us well and means that when we add a highlighting option, we only need to change it where we need it, for everything else it will just be unused.

We implement default for FileType, this time, we set the string to "No filetype" and a default HighlightingOptions object. For now, we only use the default file type, even when opening files. Let’s get the filetype displayed in the editor.

src/document.rs CHANGED
@@ -29,6 +29,9 @@ impl Document {
29
29
  file_type: FileType::default(),
30
30
  })
31
31
  }
32
+ pub fn file_type(&self) -> String {
33
+ self.file_type.name()
34
+ }
32
35
  pub fn row(&self, index: usize) -> Option<&Row> {
33
36
  self.rows.get(index)
34
37
  }
src/editor.rs CHANGED
@@ -343,7 +343,8 @@ impl Editor {
343
343
  );
344
344
 
345
345
  let line_indicator = format!(
346
- "{}/{}",
346
+ "{} | {}/{}",
347
+ self.document.file_type(),
347
348
  self.cursor_position.y.saturating_add(1),
348
349
  self.document.len()
349
350
  );
src/filetype.rs CHANGED
@@ -16,3 +16,9 @@ impl Default for FileType {
16
16
  }
17
17
  }
18
18
  }
19
+
20
+ impl FileType {
21
+ pub fn name(&self) -> String {
22
+ self.name.clone()
23
+ }
24
+ }

See this step on github

We have decided against making the name property of FileType directly editable by Document, and we have further decided to hide away the fact that FileType is a struct from Editor. As far as the editor is concerned, FileType is simply a string.

Now we need a way to detect the file type and set the correct Highlighting_Options.

src/document.rs CHANGED
@@ -26,7 +26,7 @@ impl Document {
26
26
  rows,
27
27
  file_name: Some(filename.to_string()),
28
28
  dirty: false,
29
- file_type: FileType::default(),
29
+ file_type: FileType::from(filename),
30
30
  })
31
31
  }
32
32
  pub fn file_type(&self) -> String {
@@ -103,6 +103,7 @@ impl Document {
103
103
  file.write_all(row.as_bytes())?;
104
104
  file.write_all(b"\n")?;
105
105
  }
106
+ self.file_type = FileType::from(file_name);
106
107
  self.dirty = false;
107
108
  }
108
109
  Ok(())
src/filetype.rs CHANGED
@@ -21,4 +21,13 @@ impl FileType {
21
21
  pub fn name(&self) -> String {
22
22
  self.name.clone()
23
23
  }
24
+ pub fn from(file_name: &str) -> Self {
25
+ if file_name.ends_with(".rs") {
26
+ return Self {
27
+ name: String::from("Rust"),
28
+ hl_opts: HighlightingOptions { numbers: true },
29
+ };
30
+ }
31
+ Self::default()
32
+ }
24
33
  }

See this step on github

We add from to FileType to determine the file type from its name. If we don’t know the type, we simply return the default value. We set the file type on open and on save. You are now able to open a file, verify it displays the correct file type, and confirm that the file type changes when you change the file ending on save. Very satisfying!

Now, let’s actually highlight the files. For that, we need to actually do something with the HighlightingOptions.

src/document.rs CHANGED
@@ -16,17 +16,18 @@ pub struct Document {
16
16
  impl Document {
17
17
  pub fn open(filename: &str) -> Result<Self, std::io::Error> {
18
18
  let contents = fs::read_to_string(filename)?;
19
+ let file_type = FileType::from(filename);
19
20
  let mut rows = Vec::new();
20
21
  for value in contents.lines() {
21
22
  let mut row = Row::from(value);
22
- row.highlight(None);
23
+ row.highlight(file_type.highlighting_options(), None);
23
24
  rows.push(row);
24
25
  }
25
26
  Ok(Self {
26
27
  rows,
27
28
  file_name: Some(filename.to_string()),
28
29
  dirty: false,
29
- file_type: FileType::from(filename),
30
+ file_type,
30
31
  })
31
32
  }
32
33
  pub fn file_type(&self) -> String {
@@ -52,8 +53,8 @@ impl Document {
52
53
  #[allow(clippy::indexing_slicing)]
53
54
  let current_row = &mut self.rows[at.y];
54
55
  let mut new_row = current_row.split(at.x);
55
- current_row.highlight(None);
56
- new_row.highlight(None);
56
+ current_row.highlight(self.file_type.highlighting_options(), None);
57
+ new_row.highlight(self.file_type.highlighting_options(), None);
57
58
  #[allow(clippy::integer_arithmetic)]
58
59
  self.rows.insert(at.y + 1, new_row);
59
60
  }
@@ -69,13 +70,13 @@ impl Document {
69
70
  if at.y == self.rows.len() {
70
71
  let mut row = Row::default();
71
72
  row.insert(0, c);
72
- row.highlight(None);
73
+ row.highlight(self.file_type.highlighting_options(), None);
73
74
  self.rows.push(row);
74
75
  } else {
75
76
  #[allow(clippy::indexing_slicing)]
76
77
  let row = &mut self.rows[at.y];
77
78
  row.insert(at.x, c);
78
- row.highlight(None);
79
+ row.highlight(self.file_type.highlighting_options(), None);
79
80
  }
80
81
  }
81
82
  #[allow(clippy::integer_arithmetic, clippy::indexing_slicing)]
@@ -89,11 +90,11 @@ impl Document {
89
90
  let next_row = self.rows.remove(at.y + 1);
90
91
  let row = &mut self.rows[at.y];
91
92
  row.append(&next_row);
92
- row.highlight(None);
93
+ row.highlight(self.file_type.highlighting_options(), None);
93
94
  } else {
94
95
  let row = &mut self.rows[at.y];
95
96
  row.delete(at.x);
96
- row.highlight(None);
97
+ row.highlight(self.file_type.highlighting_options(), None);
97
98
  }
98
99
  }
99
100
  pub fn save(&mut self) -> Result<(), Error> {
@@ -149,7 +150,7 @@ impl Document {
149
150
  }
150
151
  pub fn highlight(&mut self, word: Option<&str>) {
151
152
  for row in &mut self.rows {
152
- row.highlight(word);
153
+ row.highlight(self.file_type.highlighting_options(), word);
153
154
  }
154
155
  }
155
156
  }
src/filetype.rs CHANGED
@@ -3,7 +3,7 @@ pub struct FileType {
3
3
  hl_opts: HighlightingOptions,
4
4
  }
5
5
 
6
- #[derive(Default)]
6
+ #[derive(Default, Copy, Clone)]
7
7
  pub struct HighlightingOptions {
8
8
  pub numbers: bool,
9
9
  }
@@ -21,6 +21,9 @@ impl FileType {
21
21
  pub fn name(&self) -> String {
22
22
  self.name.clone()
23
23
  }
24
+ pub fn highlighting_options(&self) -> HighlightingOptions {
25
+ self.hl_opts
26
+ }
24
27
  pub fn from(file_name: &str) -> Self {
25
28
  if file_name.ends_with(".rs") {
26
29
  return Self {
src/main.rs CHANGED
@@ -18,6 +18,7 @@ use editor::Editor;
18
18
  pub use editor::Position;
19
19
  pub use editor::SearchDirection;
20
20
  pub use filetype::FileType;
21
+ pub use filetype::HighlightingOptions;
21
22
  pub use row::Row;
22
23
  pub use terminal::Terminal;
23
24
 
src/row.rs CHANGED
@@ -1,4 +1,5 @@
1
1
  use crate::highlighting;
2
+ use crate::HighlightingOptions;
2
3
  use crate::SearchDirection;
3
4
  use std::cmp;
4
5
  use termion::color;
@@ -163,7 +164,7 @@ impl Row {
163
164
  }
164
165
  None
165
166
  }
166
- pub fn highlight(&mut self, word: Option<&str>) {
167
+ pub fn highlight(&mut self, opts: HighlightingOptions, word: Option<&str>) {
167
168
  let mut highlighting = Vec::new();
168
169
  let chars: Vec<char> = self.string.chars().collect();
169
170
  let mut matches = Vec::new();
@@ -200,11 +201,15 @@ impl Row {
200
201
  } else {
201
202
  &highlighting::Type::None
202
203
  };
203
- if (c.is_ascii_digit()
204
- && (prev_is_separator || previous_highlight == &highlighting::Type::Number))
205
- || (c == &'.' && previous_highlight == &highlighting::Type::Number)
206
- {
207
- highlighting.push(highlighting::Type::Number);
204
+ if opts.numbers {
205
+ if (c.is_ascii_digit()
206
+ && (prev_is_separator || previous_highlight == &highlighting::Type::Number))
207
+ || (c == &'.' && previous_highlight == &highlighting::Type::Number)
208
+ {
209
+ highlighting.push(highlighting::Type::Number);
210
+ } else {
211
+ highlighting.push(highlighting::Type::None);
212
+ }
208
213
  } else {
209
214
  highlighting.push(highlighting::Type::None);
210
215
  }

See this step on github

The bulk of this change deals with passing around the highlighting options, so that Row has it available when its being highlighted. Then, within highlight, we simply wrap the highlighting for numbers in another if statement which checks whether or not numbers is enabled.

You can now see that Rust files are highlighted correctly, and non-rust files are not highlighted. But what is that? When you save a new file as a rust file, the highlighting starts acting all weird. Very unsatisfying!

While we’re at it, let’s also fix another minor nit-pick: We have taken great effort to ensure no one can write the file type and the highlighting options, but we keep the members of HighlightingOptions open for anyone to edit.

src/document.rs CHANGED
@@ -100,11 +100,12 @@ impl Document {
100
100
  pub fn save(&mut self) -> Result<(), Error> {
101
101
  if let Some(file_name) = &self.file_name {
102
102
  let mut file = fs::File::create(file_name)?;
103
- for row in &self.rows {
103
+ self.file_type = FileType::from(file_name);
104
+ for row in &mut self.rows {
104
105
  file.write_all(row.as_bytes())?;
105
106
  file.write_all(b"\n")?;
107
+ row.highlight(self.file_type.highlighting_options(), None)
106
108
  }
107
- self.file_type = FileType::from(file_name);
108
109
  self.dirty = false;
109
110
  }
110
111
  Ok(())
src/filetype.rs CHANGED
@@ -5,7 +5,7 @@ pub struct FileType {
5
5
 
6
6
  #[derive(Default, Copy, Clone)]
7
7
  pub struct HighlightingOptions {
8
- pub numbers: bool,
8
+ numbers: bool,
9
9
  }
10
10
 
11
11
  impl Default for FileType {
@@ -34,3 +34,9 @@ impl FileType {
34
34
  Self::default()
35
35
  }
36
36
  }
37
+
38
+ impl HighlightingOptions {
39
+ pub fn numbers(self) -> bool {
40
+ self.numbers
41
+ }
42
+ }
src/row.rs CHANGED
@@ -201,7 +201,7 @@ impl Row {
201
201
  } else {
202
202
  &highlighting::Type::None
203
203
  };
204
- if opts.numbers {
204
+ if opts.numbers() {
205
205
  if (c.is_ascii_digit()
206
206
  && (prev_is_separator || previous_highlight == &highlighting::Type::Number))
207
207
  || (c == &'.' && previous_highlight == &highlighting::Type::Number)

See this step on github

We are now re-highlighting each row as we save the file, and our number property is now finally read-only.

A quick side note: If you’re eagle-eyed, you will have noticed that the function numbers accepts self instead of &self (Notice the missing &). The distinction is not important in the scope of this tutorial, and as our struct grows, we will be reverting back to & soon. In a nutshell, for small structs, it’s more efficient to work directly with the value (without the &) than with the reference. Try adding the & and check the clippy output in case you’re interested.

Colorful strings

With all that out of the way, we can finally get to highlighting more things! Let’s start with highlighting strings.

src/filetype.rs CHANGED
@@ -6,6 +6,7 @@ pub struct FileType {
6
6
  #[derive(Default, Copy, Clone)]
7
7
  pub struct HighlightingOptions {
8
8
  numbers: bool,
9
+ strings: bool,
9
10
  }
10
11
 
11
12
  impl Default for FileType {
@@ -28,7 +29,10 @@ impl FileType {
28
29
  if file_name.ends_with(".rs") {
29
30
  return Self {
30
31
  name: String::from("Rust"),
31
- hl_opts: HighlightingOptions { numbers: true },
32
+ hl_opts: HighlightingOptions {
33
+ numbers: true,
34
+ strings: true,
35
+ },
32
36
  };
33
37
  }
34
38
  Self::default()
@@ -39,4 +43,7 @@ impl HighlightingOptions {
39
43
  pub fn numbers(self) -> bool {
40
44
  self.numbers
41
45
  }
46
+ pub fn strings(self) -> bool {
47
+ self.strings
48
+ }
42
49
  }
src/highlighting.rs CHANGED
@@ -4,6 +4,7 @@ pub enum Type {
4
4
  None,
5
5
  Number,
6
6
  Match,
7
+ String,
7
8
  }
8
9
 
9
10
  impl Type {
@@ -11,6 +12,7 @@ impl Type {
11
12
  match self {
12
13
  Type::Number => color::Rgb(220, 163, 163),
13
14
  Type::Match => color::Rgb(38, 139, 210),
15
+ Type::String => color::Rgb(211, 54, 130),
14
16
  _ => color::Rgb(255, 255, 255),
15
17
  }
16
18
  }

See this step on github

Now for the actual highlighting code. We will use an in_string variable to keep track of whether we are currently inside a string. If we are, then we’ll keep highlighting the current character as a string until we hit the closing quote.

src/row.rs CHANGED
@@ -182,6 +182,7 @@ impl Row {
182
182
  }
183
183
  }
184
184
  let mut prev_is_separator = true;
185
+ let mut in_string = false;
185
186
  let mut index = 0;
186
187
  while let Some(c) = chars.get(index) {
187
188
  if let Some(word) = word {
@@ -201,6 +202,25 @@ impl Row {
201
202
  } else {
202
203
  &highlighting::Type::None
203
204
  };
205
+ if opts.strings() {
206
+ if in_string {
207
+ highlighting.push(highlighting::Type::String);
208
+ if *c == '"' {
209
+ in_string = false;
210
+ prev_is_separator = true;
211
+ } else {
212
+ prev_is_separator = false;
213
+ }
214
+ index += 1;
215
+ continue;
216
+ } else if prev_is_separator && *c == '"' {
217
+ highlighting.push(highlighting::Type::String);
218
+ in_string = true;
219
+ prev_is_separator = true;
220
+ index += 1;
221
+ continue;
222
+ }
223
+ }
204
224
  if opts.numbers() {
205
225
  if (c.is_ascii_digit()
206
226
  && (prev_is_separator || previous_highlight == &highlighting::Type::Number))

See this step on github

Let’sa go through this change from top to bottom: If in_string is set, then we know that the current character can be highlighted as a string. Then we check if the current character is the closing quote, and if so, we reset in_string to false. Then, since we highlighted the current character, we have to consume it by incrementing index and continueing out of the current loop iteration. We also set prev_is_separator to true so that if we’re done highlighting the string, the closing quote is considered a separator.

IF we’re not currently in a string, then we have to check if we’re at the beginning of one by checking for the starting quote. If we are, we set in_string to true, highlight the quote and consume it.

We have also introduced a new concept here: The dereferencing operator, *.. The point is that c in this code is a reference to a character, and we check if it is the desired character by comparing it with a reference to another character, like c == &'"'. Alternatively, we can dereference c and directly compare it to the desired character. This makes the code a bit easier to read. Let’s apply this in other parts as well.

src/row.rs CHANGED
@@ -223,8 +223,8 @@ impl Row {
223
223
  }
224
224
  if opts.numbers() {
225
225
  if (c.is_ascii_digit()
226
- && (prev_is_separator || previous_highlight == &highlighting::Type::Number))
227
- || (c == &'.' && previous_highlight == &highlighting::Type::Number)
226
+ && (prev_is_separator || *previous_highlight == highlighting::Type::Number))
227
+ || (*c == '.' && *previous_highlight == highlighting::Type::Number)
228
228
  {
229
229
  highlighting.push(highlighting::Type::Number);
230
230
  } else {

See this step on github

We should probably take escaped quotes into account when highlighting strings and characters. If the sequence \" occurs in a string, then the escaped quote doesn’t close the string in the vast majority of languages.

src/row.rs CHANGED
@@ -205,6 +205,12 @@ impl Row {
205
205
  if opts.strings() {
206
206
  if in_string {
207
207
  highlighting.push(highlighting::Type::String);
208
+
209
+ if *c == '\\' && index < self.len().saturating_sub(1) {
210
+ highlighting.push(highlighting::Type::String);
211
+ index += 2;
212
+ continue;
213
+ }
208
214
  if *c == '"' {
209
215
  in_string = false;
210
216
  prev_is_separator = true;

See this step on github

If we’re in a string and the current character is a \, and there is at least one more character in that line that comes after the backslash, then we highligh the character that comes after the backslash with HighlightingType::String and consume it. We increment index by 2 to consume both characters at once.

Colorful characters

Now that strings work, let’s focus on characters next. We start with the basics first.

src/filetype.rs CHANGED
@@ -7,6 +7,7 @@ pub struct FileType {
7
7
  pub struct HighlightingOptions {
8
8
  numbers: bool,
9
9
  strings: bool,
10
+ characters: bool,
10
11
  }
11
12
 
12
13
  impl Default for FileType {
@@ -32,6 +33,7 @@ impl FileType {
32
33
  hl_opts: HighlightingOptions {
33
34
  numbers: true,
34
35
  strings: true,
36
+ characters: true,
35
37
  },
36
38
  };
37
39
  }
@@ -46,4 +48,7 @@ impl HighlightingOptions {
46
48
  pub fn strings(self) -> bool {
47
49
  self.strings
48
50
  }
51
+ pub fn characters(self) -> bool {
52
+ self.characters
53
+ }
49
54
  }
src/highlighting.rs CHANGED
@@ -5,6 +5,7 @@ pub enum Type {
5
5
  Number,
6
6
  Match,
7
7
  String,
8
+ Character,
8
9
  }
9
10
 
10
11
  impl Type {
@@ -13,6 +14,7 @@ impl Type {
13
14
  Type::Number => color::Rgb(220, 163, 163),
14
15
  Type::Match => color::Rgb(38, 139, 210),
15
16
  Type::String => color::Rgb(211, 54, 130),
17
+ Type::Character => color::Rgb(108, 113, 196),
16
18
  _ => color::Rgb(255, 255, 255),
17
19
  }
18
20
  }

See this step on github

You might want to do character highlighting the same way as we highlight strings. That would create two problems, though: First, we would over-eagerly highlight nonsense like 'this is definitely no character'. Second, it would not work with Lifetimes: A lifetime can be indicated with a single '. Our highlighting would not take this into account and highlight everything after the opening ' as a character.

When we highlight strings, we are also not looking for a closing character and simply end the highlighting of a string when the line ends. That’s fine, since an unclosed string is probably a typo anyways, and that typo is easier to spot if string-highlighting goes from the opening quotes to the end than with no highlighting at all.

Let’s see how we can implement character highlighting:

src/row.rs CHANGED
@@ -202,6 +202,28 @@ impl Row {
202
202
  } else {
203
203
  &highlighting::Type::None
204
204
  };
205
+ if opts.characters() && !in_string && *c == '\'' {
206
+ prev_is_separator = true;
207
+ if let Some(next_char) = chars.get(index.saturating_add(1)) {
208
+ let closing_index = if *next_char == '\\' {
209
+ index.saturating_add(3)
210
+ } else {
211
+ index.saturating_add(2)
212
+ };
213
+ if let Some(closing_char) = chars.get(closing_index) {
214
+ if *closing_char == '\'' {
215
+ for _ in 0..=closing_index.saturating_sub(index) {
216
+ highlighting.push(highlighting::Type::Character);
217
+ index += 1;
218
+ }
219
+ continue;
220
+ }
221
+ }
222
+ };
223
+ highlighting.push(highlighting::Type::None);
224
+ index += 1;
225
+ continue;
226
+ }
205
227
  if opts.strings() {
206
228
  if in_string {
207
229
  highlighting.push(highlighting::Type::String);

See this step on github

We are handling character highlighting before string highlighting, but only if we are not currently within a string. That way, we can make sure that on the one hand side, string opening quotes are not handled within a character, and on the other hand side, character opening quotes are not handled within a string.

If we are on a character opening quote, our plan is to handle three different cases:

  • We are on a ' and the next character is anything other than \. Then we look at the character after the next. If it’s a ', we highlight all three characters. In other words, we are handling '*' here.
  • We are on a ' and the next character is a \. If it is, we look one character further. If it’s a ', we highlight all four characters. In other words, we are handling escaped characters, \*', here.
  • In all other cases, we are not highlighting the current ' as a character and advance the index by one, consuming the '.

Looking at the code, we start by setting prev_is_separator to true: In all three cases, when we are done, we want the last consumed character to be treated as a separator, as it will always be a '.

Next, we are looking at the next character to determine where we expect the ' to match the opening quote. If we are looking at a \ as the next character, we expect the closing character being the 3rd character after the current character. Otherwise, we expect it to be the 2nd.

We are then looking at the character we expect to be the closing quote. If it is the closing quote, we are consuming all characters up and including the closing quote. (The = in the for..in loop means that we are going to include the last index of the provided range)

If we did not find the closing quote where we expect it to be, we are simply highlighting the current ' as None and consume it.

Colorful single-line comments

Next, let’s highlight single-line comments. (We’ll leave multi-line comments until the end, because they’re complicated).

src/filetype.rs CHANGED
@@ -8,6 +8,7 @@ pub struct HighlightingOptions {
8
8
  numbers: bool,
9
9
  strings: bool,
10
10
  characters: bool,
11
+ comments: bool,
11
12
  }
12
13
 
13
14
  impl Default for FileType {
@@ -34,6 +35,7 @@ impl FileType {
34
35
  numbers: true,
35
36
  strings: true,
36
37
  characters: true,
38
+ comments: true,
37
39
  },
38
40
  };
39
41
  }
@@ -51,4 +53,7 @@ impl HighlightingOptions {
51
53
  pub fn characters(self) -> bool {
52
54
  self.characters
53
55
  }
56
+ pub fn comments(self) -> bool {
57
+ self.comments
58
+ }
54
59
  }
src/highlighting.rs CHANGED
@@ -6,6 +6,7 @@ pub enum Type {
6
6
  Match,
7
7
  String,
8
8
  Character,
9
+ Comment,
9
10
  }
10
11
 
11
12
  impl Type {
@@ -15,6 +16,7 @@ impl Type {
15
16
  Type::Match => color::Rgb(38, 139, 210),
16
17
  Type::String => color::Rgb(211, 54, 130),
17
18
  Type::Character => color::Rgb(108, 113, 196),
19
+ Type::Comment => color::Rgb(133, 153, 0),
18
20
  _ => color::Rgb(255, 255, 255),
19
21
  }
20
22
  }

See this step on github

Single line comments are easy to detect: If we encounter a / outside of a string, all we need to do is check if it is followed by another /. If so, treat the whole rest of the line as a comment.

src/row.rs CHANGED
@@ -249,6 +249,17 @@ impl Row {
249
249
  continue;
250
250
  }
251
251
  }
252
+
253
+ if opts.comments() && *c == '/' {
254
+ if let Some(next_char) = chars.get(index.saturating_add(1)) {
255
+ if *next_char == '/' {
256
+ for _ in index..chars.len() {
257
+ highlighting.push(highlighting::Type::Comment);
258
+ }
259
+ break;
260
+ }
261
+ };
262
+ }
252
263
  if opts.numbers() {
253
264
  if (c.is_ascii_digit()
254
265
  && (prev_is_separator || *previous_highlight == highlighting::Type::Number))

See this step on github

You should now be able to confirm that highlighting single line comments works. Before we move to other things, however, we need to improve our code.

Improve code quality

If you are following along using Clippy, you might have noticed that Clippy now highlights the entirety of our highlight function. The grievance that it wants to point out is that our function is getting too long. Adding to this function is easy as we go along, but if you take a step back and try to take the whole thing in, then you will realize just how difficult it is to follow along. In one of the early chapters of the tutorial, this was the precise reason why we started taking stuff out of main: To maintain readability. We are going to do the same thing now.

Our strategy will be to split our big highlight function into separate independent functions. Each function can either return false, meaning that they did not highlight the current character, or true, indicating that they handled the character. We are also going to allow these functions to modify the pointer to the current character, to allow each function to consume more than one character at a time.

src/row.rs CHANGED
@@ -164,118 +164,148 @@ impl Row {
164
164
  }
165
165
  None
166
166
  }
167
- pub fn highlight(&mut self, opts: HighlightingOptions, word: Option<&str>) {
168
- let mut highlighting = Vec::new();
169
- let chars: Vec<char> = self.string.chars().collect();
170
- let mut matches = Vec::new();
171
- let mut search_index = 0;
172
167
 
168
+ fn highlight_match(&mut self, word: Option<&str>) {
173
169
  if let Some(word) = word {
174
- while let Some(search_match) = self.find(word, search_index, SearchDirection::Forward) {
175
- matches.push(search_match);
170
+ if word.is_empty() {
171
+ return;
172
+ }
173
+ let mut index = 0;
174
+ while let Some(search_match) = self.find(word, index, SearchDirection::Forward) {
176
175
  if let Some(next_index) = search_match.checked_add(word[..].graphemes(true).count())
177
176
  {
178
- search_index = next_index;
177
+ #[allow(clippy::indexing_slicing)]
178
+ for i in index.saturating_add(search_match)..next_index {
179
+ self.highlighting[i] = highlighting::Type::Match;
180
+ }
181
+ index = next_index;
179
182
  } else {
180
183
  break;
181
184
  }
182
185
  }
183
186
  }
184
- let mut prev_is_separator = true;
185
- let mut in_string = false;
186
- let mut index = 0;
187
- while let Some(c) = chars.get(index) {
188
- if let Some(word) = word {
189
- if matches.contains(&index) {
190
- for _ in word[..].graphemes(true) {
191
- index += 1;
192
- highlighting.push(highlighting::Type::Match);
193
- }
194
- continue;
195
- }
196
- }
197
- let previous_highlight = if index > 0 {
198
- #[allow(clippy::integer_arithmetic)]
199
- highlighting
200
- .get(index - 1)
201
- .unwrap_or(&highlighting::Type::None)
202
- } else {
203
- &highlighting::Type::None
204
- };
187
+ }
188
+
189
+ fn highlight_char(
190
+ &mut self,
191
+ index: &mut usize,
192
+ opts: HighlightingOptions,
193
+ c: char,
194
+ chars: &[char],
195
+ ) -> bool {
196
+ if opts.characters() && c == '\'' {
197
+ if let Some(next_char) = chars.get(index.saturating_add(1)) {
198
+ let closing_index = if *next_char == '\\' {
199
+ index.saturating_add(3)
200
+ } else {
201
+ index.saturating_add(2)
202
+ };
203
+ if let Some(closing_char) = chars.get(closing_index) {
204
+ if *closing_char == '\'' {
205
+ for _ in 0..=closing_index.saturating_sub(*index) {
206
+ self.highlighting.push(highlighting::Type::Character);
207
+ *index += 1;
205
- if opts.characters() && !in_string && *c == '\'' {
206
- prev_is_separator = true;
207
- if let Some(next_char) = chars.get(index.saturating_add(1)) {
208
- let closing_index = if *next_char == '\\' {
209
- index.saturating_add(3)
210
- } else {
211
- index.saturating_add(2)
212
- };
213
- if let Some(closing_char) = chars.get(closing_index) {
214
- if *closing_char == '\'' {
215
- for _ in 0..=closing_index.saturating_sub(index) {
216
- highlighting.push(highlighting::Type::Character);
217
- index += 1;
218
- }
219
- continue;
220
208
  }
209
+ return true;
221
210
  }
222
- };
211
+ }
223
- highlighting.push(highlighting::Type::None);
224
- index += 1;
225
- continue;
226
212
  }
227
- if opts.strings() {
228
- if in_string {
229
- highlighting.push(highlighting::Type::String);
213
+ }
214
+ false
215
+ }
230
216
 
231
- if *c == '\\' && index < self.len().saturating_sub(1) {
232
- highlighting.push(highlighting::Type::String);
233
- index += 2;
234
- continue;
217
+ fn highlight_comment(
218
+ &mut self,
219
+ index: &mut usize,
220
+ opts: HighlightingOptions,
221
+ c: char,
222
+ chars: &[char],
223
+ ) -> bool {
224
+ if opts.comments() && c == '/' && *index < chars.len() {
225
+ if let Some(next_char) = chars.get(index.saturating_add(1)) {
226
+ if *next_char == '/' {
227
+ for _ in *index..chars.len() {
228
+ self.highlighting.push(highlighting::Type::Comment);
229
+ *index += 1;
235
230
  }
236
- if *c == '"' {
237
- in_string = false;
238
- prev_is_separator = true;
239
- } else {
240
- prev_is_separator = false;
231
+ return true;
232
+ }
233
+ };
234
+ }
235
+ false
236
+ }
237
+
238
+ fn highlight_string(
239
+ &mut self,
240
+ index: &mut usize,
241
+ opts: HighlightingOptions,
242
+ c: char,
243
+ chars: &[char],
244
+ ) -> bool {
245
+ if opts.strings() && c == '"' {
246
+ loop {
247
+ self.highlighting.push(highlighting::Type::String);
248
+ *index += 1;
249
+ if let Some(next_char) = chars.get(*index) {
250
+ if *next_char == '"' {
251
+ break;
241
252
  }
242
- index += 1;
243
- continue;
253
+ } else {
254
+ break;
244
- } else if prev_is_separator && *c == '"' {
245
- highlighting.push(highlighting::Type::String);
246
- in_string = true;
247
- prev_is_separator = true;
248
- index += 1;
249
- continue;
250
255
  }
251
256
  }
257
+ self.highlighting.push(highlighting::Type::String);
258
+ *index += 1;
259
+ return true;
260
+ }
261
+ false
262
+ }
252
263
 
253
- if opts.comments() && *c == '/' {
254
- if let Some(next_char) = chars.get(index.saturating_add(1)) {
255
- if *next_char == '/' {
256
- for _ in index..chars.len() {
257
- highlighting.push(highlighting::Type::Comment);
258
- }
264
+ fn highlight_number(
265
+ &mut self,
266
+ index: &mut usize,
267
+ opts: HighlightingOptions,
268
+ c: char,
269
+ chars: &[char],
270
+ ) -> bool {
271
+ if opts.numbers() && c.is_ascii_digit() {
272
+ if *index > 0 {
273
+ #[allow(clippy::indexing_slicing, clippy::integer_arithmetic)]
274
+ let prev_char = chars[*index - 1];
275
+ if !prev_char.is_ascii_punctuation() && !prev_char.is_ascii_whitespace() {
276
+ return false;
277
+ }
278
+ }
279
+ loop {
280
+ self.highlighting.push(highlighting::Type::Number);
281
+ *index += 1;
282
+ if let Some(next_char) = chars.get(*index) {
283
+ if *next_char != '.' && !next_char.is_ascii_digit() {
259
284
  break;
260
285
  }
261
- };
262
- }
263
- if opts.numbers() {
264
- if (c.is_ascii_digit()
265
- && (prev_is_separator || *previous_highlight == highlighting::Type::Number))
266
- || (*c == '.' && *previous_highlight == highlighting::Type::Number)
267
- {
268
- highlighting.push(highlighting::Type::Number);
269
286
  } else {
270
- highlighting.push(highlighting::Type::None);
287
+ break;
271
288
  }
272
- } else {
273
- highlighting.push(highlighting::Type::None);
274
289
  }
275
- prev_is_separator = c.is_ascii_punctuation() || c.is_ascii_whitespace();
290
+ return true;
291
+ }
292
+ false
293
+ }
294
+ pub fn highlight(&mut self, opts: HighlightingOptions, word: Option<&str>) {
295
+ self.highlighting = Vec::new();
296
+ let chars: Vec<char> = self.string.chars().collect();
297
+ let mut index = 0;
298
+ while let Some(c) = chars.get(index) {
299
+ if self.highlight_char(&mut index, opts, *c, &chars)
300
+ || self.highlight_comment(&mut index, opts, *c, &chars)
301
+ || self.highlight_string(&mut index, opts, *c, &chars)
302
+ || self.highlight_number(&mut index, opts, *c, &chars)
303
+ {
304
+ continue;
305
+ }
306
+ self.highlighting.push(highlighting::Type::None);
276
307
  index += 1;
277
308
  }
278
-
309
+ self.highlight_match(word);
279
- self.highlighting = highlighting;
280
310
  }
281
311
  }

See this step on github

If you have trouble following this diff alone, don’t forget that the direct link below the diff lets you see the full file after the change.

Let’s go through this change by looking at highlight first. That is the function we wanted to simplify, and it has gotten so much smaller. We have removed any highlighting logic except for the highlighting of None, in case no other highlighting has matched. By the way, if-statements are evaluated from left to right, so, for instance, if highlight_comment returns true, none of the other highlight functions is being called.

We will now investigate all the changes to all five highlighters we have written so far.

highlight_char and highlight_comment are largely unchanged, they have been moved to separate functions and changed to return true in case they do some highlighting, and false otherwise.

highlight_string has changed quite a lot. Instead of having a variable which keeps track of whether or not we are in a string, we are now checking if we are at an opening quote. If so, we consume the remainder of the row until we meed the end of it or a closing quote and highlight it as a string.

highlight_number, with which we started the whole highlighting process, has also changed slightly. If invoked, it checks if the previous character was a separator (instead of having a variable keeping track of it). If so, the highlighting is done now the same as in other functions: We consume the whole number, including the dot, in one go before returning from this function.

We have now placed highlight_match below the loop and have changed the logic a bit. Previously, we were storing the indices of the matches to be used later during highlighting. Now, we are updating self.highlighting directly from within highlight_match. We need to handle this case separately as we want our match not to interfere with other highlights. For instance, if we search for con, we want the first three letters of const to be highlighted as a search result, and the remaining two letters as a keyword.

Side note on refactoring

We have now done quite a big refactoring, and I want to say a word or two about the refacotring process itself. Many tutorials present you with a polished final solution which you can happily implement. But that is not how code evolves in practice. How we did it in this tutorial is much more realistic: You start with a simple problem, in this case, highlighting a single digit, and then you expand it over time. As the features grow, your code grows, too, and the assumptions you made at the beginning on what the code should look like will soon be outdated or plain wrong.

Being able to judge when to refactor is a skill that needs training, it doesn’t come by itself. Luckily, there are tools to help you build that skill. One is clippy. We could have ignored or silenced this warning, but ultimately, clippy was right in pointing out that our code was getting too complex! A refactoring was in order, if not overdue.

Is the code now perfect? Definitely not. There are at least the following two things wrong with it:

  • The loop in highlight implicitly relies on the highlight functions to advance index. If any of these functions returns true, but does not modify index, we run into an infinite loop. This is not obvious in the code and therefore not ideal.
  • The highlighting will not work around the borders of usize. We are pushing things into the highlighting array without checking if it is safe, and we are advancing index in many cases without any kind of check. It is not easy to see and understand how our code will behave in this case. Will it crash? Will it enter an infinite loop?

We are not going to solve these issues in this tutorial, but I invite you to take a stab at them, they are not terribly hard to solve.

If you are now disappointed that even after our refactoring our code still has flaws, then consider this: These flaws have been in the code before, but now they are much, much easier to see. Also, we wouldn’t have detected them if we hadn’t done the refactoring in the first place.

In a real world scenario, I would consider the weakness around large files as not relevant - we saw earlier why we would run in to all kinds of other problems first before these flaws would really become important, at least on modern systems.

I would also either refactor or at least document the other issue, to make sure that everyone who extends the highlighting in the future (could be my own future me) knows what to do.

Colorful keywords

Now let’s turn to highlighting keywords. We’re going to allow languages to specify two types of keywords that will be highlighted in different colors. (In Rust, we’ll highlight actual keywords in one color and common type names in the other color).

Let’s add two Vectors to the HighlightingOptions, one for primary, one for secondary keywords.

src/document.rs CHANGED
@@ -20,7 +20,7 @@ impl Document {
20
20
  let mut rows = Vec::new();
21
21
  for value in contents.lines() {
22
22
  let mut row = Row::from(value);
23
- row.highlight(file_type.highlighting_options(), None);
23
+ row.highlight(&file_type.highlighting_options(), None);
24
24
  rows.push(row);
25
25
  }
26
26
  Ok(Self {
@@ -53,8 +53,8 @@ impl Document {
53
53
  #[allow(clippy::indexing_slicing)]
54
54
  let current_row = &mut self.rows[at.y];
55
55
  let mut new_row = current_row.split(at.x);
56
- current_row.highlight(self.file_type.highlighting_options(), None);
57
- new_row.highlight(self.file_type.highlighting_options(), None);
56
+ current_row.highlight(&self.file_type.highlighting_options(), None);
57
+ new_row.highlight(&self.file_type.highlighting_options(), None);
58
58
  #[allow(clippy::integer_arithmetic)]
59
59
  self.rows.insert(at.y + 1, new_row);
60
60
  }
@@ -70,13 +70,13 @@ impl Document {
70
70
  if at.y == self.rows.len() {
71
71
  let mut row = Row::default();
72
72
  row.insert(0, c);
73
- row.highlight(self.file_type.highlighting_options(), None);
73
+ row.highlight(&self.file_type.highlighting_options(), None);
74
74
  self.rows.push(row);
75
75
  } else {
76
76
  #[allow(clippy::indexing_slicing)]
77
77
  let row = &mut self.rows[at.y];
78
78
  row.insert(at.x, c);
79
- row.highlight(self.file_type.highlighting_options(), None);
79
+ row.highlight(&self.file_type.highlighting_options(), None);
80
80
  }
81
81
  }
82
82
  #[allow(clippy::integer_arithmetic, clippy::indexing_slicing)]
@@ -90,11 +90,11 @@ impl Document {
90
90
  let next_row = self.rows.remove(at.y + 1);
91
91
  let row = &mut self.rows[at.y];
92
92
  row.append(&next_row);
93
- row.highlight(self.file_type.highlighting_options(), None);
93
+ row.highlight(&self.file_type.highlighting_options(), None);
94
94
  } else {
95
95
  let row = &mut self.rows[at.y];
96
96
  row.delete(at.x);
97
- row.highlight(self.file_type.highlighting_options(), None);
97
+ row.highlight(&self.file_type.highlighting_options(), None);
98
98
  }
99
99
  }
100
100
  pub fn save(&mut self) -> Result<(), Error> {
@@ -104,7 +104,7 @@ impl Document {
104
104
  for row in &mut self.rows {
105
105
  file.write_all(row.as_bytes())?;
106
106
  file.write_all(b"\n")?;
107
- row.highlight(self.file_type.highlighting_options(), None)
107
+ row.highlight(&self.file_type.highlighting_options(), None)
108
108
  }
109
109
  self.dirty = false;
110
110
  }
@@ -151,7 +151,7 @@ impl Document {
151
151
  }
152
152
  pub fn highlight(&mut self, word: Option<&str>) {
153
153
  for row in &mut self.rows {
154
- row.highlight(self.file_type.highlighting_options(), word);
154
+ row.highlight(&self.file_type.highlighting_options(), word);
155
155
  }
156
156
  }
157
157
  }
src/filetype.rs CHANGED
@@ -3,12 +3,14 @@ pub struct FileType {
3
3
  hl_opts: HighlightingOptions,
4
4
  }
5
5
 
6
- #[derive(Default, Copy, Clone)]
6
+ #[derive(Default)]
7
7
  pub struct HighlightingOptions {
8
8
  numbers: bool,
9
9
  strings: bool,
10
10
  characters: bool,
11
11
  comments: bool,
12
+ primary_keywords: Vec<String>,
13
+ secondary_keywords: Vec<String>,
12
14
  }
13
15
 
14
16
  impl Default for FileType {
@@ -24,8 +26,8 @@ impl FileType {
24
26
  pub fn name(&self) -> String {
25
27
  self.name.clone()
26
28
  }
27
- pub fn highlighting_options(&self) -> HighlightingOptions {
28
- self.hl_opts
29
+ pub fn highlighting_options(&self) -> &HighlightingOptions {
30
+ &self.hl_opts
29
31
  }
30
32
  pub fn from(file_name: &str) -> Self {
31
33
  if file_name.ends_with(".rs") {
@@ -36,6 +38,75 @@ impl FileType {
36
38
  strings: true,
37
39
  characters: true,
38
40
  comments: true,
41
+ primary_keywords: vec![
42
+ "as".to_string(),
43
+ "break".to_string(),
44
+ "const".to_string(),
45
+ "continue".to_string(),
46
+ "crate".to_string(),
47
+ "else".to_string(),
48
+ "enum".to_string(),
49
+ "extern".to_string(),
50
+ "false".to_string(),
51
+ "fn".to_string(),
52
+ "for".to_string(),
53
+ "if".to_string(),
54
+ "impl".to_string(),
55
+ "in".to_string(),
56
+ "let".to_string(),
57
+ "loop".to_string(),
58
+ "match".to_string(),
59
+ "mod".to_string(),
60
+ "move".to_string(),
61
+ "mut".to_string(),
62
+ "pub".to_string(),
63
+ "ref".to_string(),
64
+ "return".to_string(),
65
+ "self".to_string(),
66
+ "Self".to_string(),
67
+ "static".to_string(),
68
+ "struct".to_string(),
69
+ "super".to_string(),
70
+ "trait".to_string(),
71
+ "true".to_string(),
72
+ "type".to_string(),
73
+ "unsafe".to_string(),
74
+ "use".to_string(),
75
+ "where".to_string(),
76
+ "while".to_string(),
77
+ "dyn".to_string(),
78
+ "abstract".to_string(),
79
+ "become".to_string(),
80
+ "box".to_string(),
81
+ "do".to_string(),
82
+ "final".to_string(),
83
+ "macro".to_string(),
84
+ "override".to_string(),
85
+ "priv".to_string(),
86
+ "typeof".to_string(),
87
+ "unsized".to_string(),
88
+ "virtual".to_string(),
89
+ "yield".to_string(),
90
+ "async".to_string(),
91
+ "await".to_string(),
92
+ "try".to_string(),
93
+ ],
94
+ secondary_keywords: vec![
95
+ "bool".to_string(),
96
+ "char".to_string(),
97
+ "i8".to_string(),
98
+ "i16".to_string(),
99
+ "i32".to_string(),
100
+ "i64".to_string(),
101
+ "isize".to_string(),
102
+ "u8".to_string(),
103
+ "u16".to_string(),
104
+ "u32".to_string(),
105
+ "u64".to_string(),
106
+ "usize".to_string(),
107
+ "f32".to_string(),
108
+ "f64".to_string(),
109
+ ],
39
110
  },
40
111
  };
41
112
  }
@@ -44,16 +115,16 @@ impl FileType {
44
115
  }
45
116
 
46
117
  impl HighlightingOptions {
47
- pub fn numbers(self) -> bool {
118
+ pub fn numbers(&self) -> bool {
48
119
  self.numbers
49
120
  }
50
- pub fn strings(self) -> bool {
121
+ pub fn strings(&self) -> bool {
51
122
  self.strings
52
123
  }
53
- pub fn characters(self) -> bool {
124
+ pub fn characters(&self) -> bool {
54
125
  self.characters
55
126
  }
56
- pub fn comments(self) -> bool {
127
+ pub fn comments(&self) -> bool {
57
128
  self.comments
58
129
  }
59
130
  }
src/highlighting.rs CHANGED
@@ -7,6 +7,8 @@ pub enum Type {
7
7
  String,
8
8
  Character,
9
9
  Comment,
10
+ PrimaryKeywords,
11
+ SecondaryKeywords,
10
12
  }
11
13
 
12
14
  impl Type {
@@ -17,6 +19,8 @@ impl Type {
17
19
  Type::String => color::Rgb(211, 54, 130),
18
20
  Type::Character => color::Rgb(108, 113, 196),
19
21
  Type::Comment => color::Rgb(133, 153, 0),
22
+ Type::PrimaryKeywords => color::Rgb(181, 137, 0),
23
+ Type::SecondaryKeywords => color::Rgb(42, 161, 152),
20
24
  _ => color::Rgb(255, 255, 255),
21
25
  }
22
26
  }
src/row.rs CHANGED
@@ -189,7 +189,7 @@ impl Row {
189
189
  fn highlight_char(
190
190
  &mut self,
191
191
  index: &mut usize,
192
- opts: HighlightingOptions,
192
+ opts: &HighlightingOptions,
193
193
  c: char,
194
194
  chars: &[char],
195
195
  ) -> bool {
@@ -217,7 +217,7 @@ impl Row {
217
217
  fn highlight_comment(
218
218
  &mut self,
219
219
  index: &mut usize,
220
- opts: HighlightingOptions,
220
+ opts: &HighlightingOptions,
221
221
  c: char,
222
222
  chars: &[char],
223
223
  ) -> bool {
@@ -238,7 +238,7 @@ impl Row {
238
238
  fn highlight_string(
239
239
  &mut self,
240
240
  index: &mut usize,
241
- opts: HighlightingOptions,
241
+ opts: &HighlightingOptions,
242
242
  c: char,
243
243
  chars: &[char],
244
244
  ) -> bool {
@@ -264,7 +264,7 @@ impl Row {
264
264
  fn highlight_number(
265
265
  &mut self,
266
266
  index: &mut usize,
267
- opts: HighlightingOptions,
267
+ opts: &HighlightingOptions,
268
268
  c: char,
269
269
  chars: &[char],
270
270
  ) -> bool {
@@ -291,7 +291,7 @@ impl Row {
291
291
  }
292
292
  false
293
293
  }
294
- pub fn highlight(&mut self, opts: HighlightingOptions, word: Option<&str>) {
294
+ pub fn highlight(&mut self, opts: &HighlightingOptions, word: Option<&str>) {
295
295
  self.highlighting = Vec::new();
296
296
  let chars: Vec<char> = self.string.chars().collect();
297
297
  let mut index = 0;

See this step on github

We had to jump through a surprisingly large number of loops to get this to work. So what happened?

When we added a more complex data structure like Vec<String> to HighlightingOptions, it’s no longer sensible to copy HighlightingOptions around all the time, which was the case before. Also, we can no longer let the compiler derive the Copy trait for us, so we needed to change a few things in our code to make sure that from now on, only a reference to HighlightingOptions is being passed around.

Now that we have the keywords available, let’s highlight them. We start with the primary keywords first.

src/filetype.rs CHANGED
@@ -127,4 +127,10 @@ impl HighlightingOptions {
127
127
  pub fn comments(&self) -> bool {
128
128
  self.comments
129
129
  }
130
+ pub fn primary_keywords(&self) -> &Vec<String> {
131
+ &self.primary_keywords
132
+ }
133
+ pub fn secondary_keywords(&self) -> &Vec<String> {
134
+ &self.secondary_keywords
135
+ }
130
136
  }
src/highlighting.rs CHANGED
@@ -1,5 +1,5 @@
1
1
  use termion::color;
2
- #[derive(PartialEq)]
2
+ #[derive(PartialEq, Clone, Copy)]
3
3
  pub enum Type {
4
4
  None,
5
5
  Number,
@@ -12,7 +12,7 @@ pub enum Type {
12
12
  }
13
13
 
14
14
  impl Type {
15
- pub fn to_color(&self) -> impl color::Color {
15
+ pub fn to_color(self) -> impl color::Color {
16
16
  match self {
17
17
  Type::Number => color::Rgb(220, 163, 163),
18
18
  Type::Match => color::Rgb(38, 139, 210),
src/row.rs CHANGED
@@ -186,6 +186,45 @@ impl Row {
186
186
  }
187
187
  }
188
188
 
189
+ fn highlight_str(
190
+ &mut self,
191
+ index: &mut usize,
192
+ substring: &str,
193
+ chars: &[char],
194
+ hl_type: highlighting::Type,
195
+ ) -> bool {
196
+ if substring.is_empty() {
197
+ return false;
198
+ }
199
+ for (substring_index, c) in substring.chars().enumerate() {
200
+ if let Some(next_char) = chars.get(index.saturating_add(substring_index)) {
201
+ if *next_char != c {
202
+ return false;
203
+ }
204
+ } else {
205
+ return false;
206
+ }
207
+ }
208
+ for _ in 0..substring.len() {
209
+ self.highlighting.push(hl_type);
210
+ *index += 1;
211
+ }
212
+ true
213
+ }
214
+ fn highlight_primary_keywords(
215
+ &mut self,
216
+ index: &mut usize,
217
+ opts: &HighlightingOptions,
218
+ chars: &[char],
219
+ ) -> bool {
220
+ for word in opts.primary_keywords() {
221
+ if self.highlight_str(index, word, chars, highlighting::Type::PrimaryKeywords) {
222
+ return true;
223
+ }
224
+ }
225
+ false
226
+ }
227
+
189
228
  fn highlight_char(
190
229
  &mut self,
191
230
  index: &mut usize,
@@ -298,6 +337,7 @@ impl Row {
298
337
  while let Some(c) = chars.get(index) {
299
338
  if self.highlight_char(&mut index, opts, *c, &chars)
300
339
  || self.highlight_comment(&mut index, opts, *c, &chars)
340
+ || self.highlight_primary_keywords(&mut index, &opts, &chars)
301
341
  || self.highlight_string(&mut index, opts, *c, &chars)
302
342
  || self.highlight_number(&mut index, opts, *c, &chars)
303
343
  {

See this step on github

To make keyword highlighting work, we are building a new function called highlight_str, which is responsible for highlighting a substring with a given type. It does this by comparing every character after the current character with the given string. If the whole string is matched, self.highlighting is updated with the right highlighting type. If one character is off, the whole highlighting for this string is aborted.

This means that we need to pass around highlighting::Type, but instead of worrying about references and borrowing, wee are simply deriving Copy and Clone for that type and simply copy it around. Since we have no plans for Type to grow, this is a sensible decision for now.

If you start hecto now, you will see that primary keywords are highlighted. However, there is still a bug: We don’t want the in of String to be highlighted, keywords need to be preceded and followed by a separator. Let’s fix that

src/row.rs CHANGED
@@ -217,7 +217,22 @@ impl Row {
217
217
  opts: &HighlightingOptions,
218
218
  chars: &[char],
219
219
  ) -> bool {
220
+ if *index > 0 {
221
+ #[allow(clippy::indexing_slicing, clippy::integer_arithmetic)]
222
+ let prev_char = chars[*index - 1];
223
+ if !is_separator(prev_char) {
224
+ return false;
225
+ }
226
+ }
220
227
  for word in opts.primary_keywords() {
228
+ if *index < chars.len().saturating_sub(word.len()) {
229
+ #[allow(clippy::indexing_slicing, clippy::integer_arithmetic)]
230
+ let next_char = chars[*index + word.len()];
231
+ if !is_separator(next_char) {
232
+ continue;
233
+ }
234
+ }
235
+
221
236
  if self.highlight_str(index, word, chars, highlighting::Type::PrimaryKeywords) {
222
237
  return true;
223
238
  }
@@ -299,7 +314,6 @@ impl Row {
299
314
  }
300
315
  false
301
316
  }
302
-
303
317
  fn highlight_number(
304
318
  &mut self,
305
319
  index: &mut usize,
@@ -311,7 +325,7 @@ impl Row {
311
325
  if *index > 0 {
312
326
  #[allow(clippy::indexing_slicing, clippy::integer_arithmetic)]
313
327
  let prev_char = chars[*index - 1];
314
- if !prev_char.is_ascii_punctuation() && !prev_char.is_ascii_whitespace() {
328
+ if !is_separator(prev_char) {
315
329
  return false;
316
330
  }
317
331
  }
@@ -349,3 +363,7 @@ impl Row {
349
363
  self.highlight_match(word);
350
364
  }
351
365
  }
366
+
367
+ fn is_separator(c: char) -> bool {
368
+ c.is_ascii_punctuation() || c.is_ascii_whitespace()
369
+ }

See this step on github

Now, let’s try and highlight secondary keywords as well.

src/row.rs CHANGED
@@ -211,11 +211,12 @@ impl Row {
211
211
  }
212
212
  true
213
213
  }
214
- fn highlight_primary_keywords(
214
+ fn highlight_keywords(
215
215
  &mut self,
216
216
  index: &mut usize,
217
- opts: &HighlightingOptions,
218
217
  chars: &[char],
218
+ keywords: &[String],
219
+ hl_type: highlighting::Type,
219
220
  ) -> bool {
220
221
  if *index > 0 {
221
222
  #[allow(clippy::indexing_slicing, clippy::integer_arithmetic)]
@@ -224,7 +225,7 @@ impl Row {
224
225
  return false;
225
226
  }
226
227
  }
227
- for word in opts.primary_keywords() {
228
+ for word in keywords {
228
229
  if *index < chars.len().saturating_sub(word.len()) {
229
230
  #[allow(clippy::indexing_slicing, clippy::integer_arithmetic)]
230
231
  let next_char = chars[*index + word.len()];
@@ -233,13 +234,40 @@ impl Row {
233
234
  }
234
235
  }
235
236
 
236
- if self.highlight_str(index, word, chars, highlighting::Type::PrimaryKeywords) {
237
+ if self.highlight_str(index, &word, chars, hl_type) {
237
238
  return true;
238
239
  }
239
240
  }
240
241
  false
241
242
  }
242
243
 
244
+ fn highlight_primary_keywords(
245
+ &mut self,
246
+ index: &mut usize,
247
+ opts: &HighlightingOptions,
248
+ chars: &[char],
249
+ ) -> bool {
250
+ self.highlight_keywords(
251
+ index,
252
+ chars,
253
+ opts.primary_keywords(),
254
+ highlighting::Type::PrimaryKeywords,
255
+ )
256
+ }
257
+ fn highlight_secondary_keywords(
258
+ &mut self,
259
+ index: &mut usize,
260
+ opts: &HighlightingOptions,
261
+ chars: &[char],
262
+ ) -> bool {
263
+ self.highlight_keywords(
264
+ index,
265
+ chars,
266
+ opts.secondary_keywords(),
267
+ highlighting::Type::SecondaryKeywords,
268
+ )
269
+ }
270
+
243
271
  fn highlight_char(
244
272
  &mut self,
245
273
  index: &mut usize,
@@ -352,6 +380,7 @@ impl Row {
352
380
  if self.highlight_char(&mut index, opts, *c, &chars)
353
381
  || self.highlight_comment(&mut index, opts, *c, &chars)
354
382
  || self.highlight_primary_keywords(&mut index, &opts, &chars)
383
+ || self.highlight_secondary_keywords(&mut index, &opts, &chars)
355
384
  || self.highlight_string(&mut index, opts, *c, &chars)
356
385
  || self.highlight_number(&mut index, opts, *c, &chars)
357
386
  {

See this step on github

We have now extracted the core of `highlight_primary_keywords_ into a more generic function which highlights a given word with a given type if it’s surrounded by separators.

When you open your editor now, you should be able to see primary and secondary keywords highlighted.

Colorful multi-line comments

Okay, we have one last feature to implement: multi-line comment highlighting. Let’s start by adding the appropriate entry to HighlightingOptions and Type.

src/filetype.rs CHANGED
@@ -9,6 +9,7 @@ pub struct HighlightingOptions {
9
9
  strings: bool,
10
10
  characters: bool,
11
11
  comments: bool,
12
+ multiline_comments: bool,
12
13
  primary_keywords: Vec<String>,
13
14
  secondary_keywords: Vec<String>,
14
15
  }
@@ -38,6 +39,7 @@ impl FileType {
38
39
  strings: true,
39
40
  characters: true,
40
41
  comments: true,
42
+ multiline_comments: true,
41
43
  primary_keywords: vec![
42
44
  "as".to_string(),
43
45
  "break".to_string(),
@@ -133,4 +135,7 @@ impl HighlightingOptions {
133
135
  pub fn secondary_keywords(&self) -> &Vec<String> {
134
136
  &self.secondary_keywords
135
137
  }
138
+ pub fn multiline_comments(&self) -> bool {
139
+ self.multiline_comments
140
+ }
136
141
  }
src/highlighting.rs CHANGED
@@ -7,6 +7,7 @@ pub enum Type {
7
7
  String,
8
8
  Character,
9
9
  Comment,
10
+ MultilineComment,
10
11
  PrimaryKeywords,
11
12
  SecondaryKeywords,
12
13
  }
@@ -18,7 +19,7 @@ impl Type {
18
19
  Type::Match => color::Rgb(38, 139, 210),
19
20
  Type::String => color::Rgb(211, 54, 130),
20
21
  Type::Character => color::Rgb(108, 113, 196),
21
- Type::Comment => color::Rgb(133, 153, 0),
22
+ Type::Comment | Type::MultilineComment => color::Rgb(133, 153, 0),
22
23
  Type::PrimaryKeywords => color::Rgb(181, 137, 0),
23
24
  Type::SecondaryKeywords => color::Rgb(42, 161, 152),
24
25
  _ => color::Rgb(255, 255, 255),

See this step on github

We’ll highlight multi-line comments to be the same color as single-line comments. Now let’s do the highlighting. We won’t worry about multiple lines just yet.

src/row.rs CHANGED
@@ -316,6 +316,33 @@ impl Row {
316
316
  }
317
317
  false
318
318
  }
319
+ #[allow(clippy::indexing_slicing, clippy::integer_arithmetic)]
320
+ fn highlight_multiline_comment(
321
+ &mut self,
322
+ index: &mut usize,
323
+ opts: &HighlightingOptions,
324
+ c: char,
325
+ chars: &[char],
326
+ ) -> bool {
327
+ if opts.comments() && c == '/' && *index < chars.len() {
328
+ if let Some(next_char) = chars.get(index.saturating_add(1)) {
329
+ if *next_char == '*' {
330
+ let closing_index =
331
+ if let Some(closing_index) = self.string[*index + 2..].find("*/") {
332
+ *index + closing_index + 4
333
+ } else {
334
+ chars.len()
335
+ };
336
+ for _ in *index..closing_index {
337
+ self.highlighting.push(highlighting::Type::MultilineComment);
338
+ *index += 1;
339
+ }
340
+ return true;
341
+ }
342
+ };
343
+ }
344
+ false
345
+ }
319
346
 
320
347
  fn highlight_string(
321
348
  &mut self,
@@ -379,6 +406,7 @@ impl Row {
379
406
  while let Some(c) = chars.get(index) {
380
407
  if self.highlight_char(&mut index, opts, *c, &chars)
381
408
  || self.highlight_comment(&mut index, opts, *c, &chars)
409
+ || self.highlight_multiline_comment(&mut index, &opts, *c, &chars)
382
410
  || self.highlight_primary_keywords(&mut index, &opts, &chars)
383
411
  || self.highlight_secondary_keywords(&mut index, &opts, &chars)
384
412
  || self.highlight_string(&mut index, opts, *c, &chars)

See this step on github

This is essentially a combination of highlighting strings and highlighting single-line comments. If we are on a /*, we use find to get the index of the closing */ . Since theoretically we could have multiple comments in the same line, we only search from the current index position forward in the string.

We highlight the whole comment, plus the 4 characters needed by the opening and closing characters of the comment.

Now, how do we highlight multiple rows? The strategy we will be using is that we pass the information if we ended on an unclosed comment back to document, so that document can highlight the next row differently, if needed.

src/document.rs CHANGED
@@ -20,7 +20,7 @@ impl Document {
20
20
  let mut rows = Vec::new();
21
21
  for value in contents.lines() {
22
22
  let mut row = Row::from(value);
23
- row.highlight(&file_type.highlighting_options(), None);
23
+ row.highlight(&file_type.highlighting_options(), None, false);
24
24
  rows.push(row);
25
25
  }
26
26
  Ok(Self {
@@ -53,8 +53,8 @@ impl Document {
53
53
  #[allow(clippy::indexing_slicing)]
54
54
  let current_row = &mut self.rows[at.y];
55
55
  let mut new_row = current_row.split(at.x);
56
- current_row.highlight(&self.file_type.highlighting_options(), None);
57
- new_row.highlight(&self.file_type.highlighting_options(), None);
56
+ current_row.highlight(&self.file_type.highlighting_options(), None, false);
57
+ new_row.highlight(&self.file_type.highlighting_options(), None, false);
58
58
  #[allow(clippy::integer_arithmetic)]
59
59
  self.rows.insert(at.y + 1, new_row);
60
60
  }
@@ -70,13 +70,13 @@ impl Document {
70
70
  if at.y == self.rows.len() {
71
71
  let mut row = Row::default();
72
72
  row.insert(0, c);
73
- row.highlight(&self.file_type.highlighting_options(), None);
73
+ row.highlight(&self.file_type.highlighting_options(), None, false);
74
74
  self.rows.push(row);
75
75
  } else {
76
76
  #[allow(clippy::indexing_slicing)]
77
77
  let row = &mut self.rows[at.y];
78
78
  row.insert(at.x, c);
79
- row.highlight(&self.file_type.highlighting_options(), None);
79
+ row.highlight(&self.file_type.highlighting_options(), None, false);
80
80
  }
81
81
  }
82
82
  #[allow(clippy::integer_arithmetic, clippy::indexing_slicing)]
@@ -90,21 +90,26 @@ impl Document {
90
90
  let next_row = self.rows.remove(at.y + 1);
91
91
  let row = &mut self.rows[at.y];
92
92
  row.append(&next_row);
93
- row.highlight(&self.file_type.highlighting_options(), None);
93
+ row.highlight(&self.file_type.highlighting_options(), None, false);
94
94
  } else {
95
95
  let row = &mut self.rows[at.y];
96
96
  row.delete(at.x);
97
- row.highlight(&self.file_type.highlighting_options(), None);
97
+ row.highlight(&self.file_type.highlighting_options(), None, false);
98
98
  }
99
99
  }
100
100
  pub fn save(&mut self) -> Result<(), Error> {
101
101
  if let Some(file_name) = &self.file_name {
102
102
  let mut file = fs::File::create(file_name)?;
103
103
  self.file_type = FileType::from(file_name);
104
+ let mut start_with_comment = false;
104
105
  for row in &mut self.rows {
105
106
  file.write_all(row.as_bytes())?;
106
107
  file.write_all(b"\n")?;
107
- row.highlight(&self.file_type.highlighting_options(), None)
108
+ start_with_comment = row.highlight(
109
+ &self.file_type.highlighting_options(),
110
+ None,
111
+ start_with_comment,
112
+ );
108
113
  }
109
114
  self.dirty = false;
110
115
  }
@@ -150,8 +155,13 @@ impl Document {
150
155
  None
151
156
  }
152
157
  pub fn highlight(&mut self, word: Option<&str>) {
158
+ let mut start_with_comment = false;
153
159
  for row in &mut self.rows {
154
- row.highlight(&self.file_type.highlighting_options(), word);
160
+ start_with_comment = row.highlight(
161
+ &self.file_type.highlighting_options(),
162
+ word,
163
+ start_with_comment,
164
+ );
155
165
  }
156
166
  }
157
167
  }
src/row.rs CHANGED
@@ -399,14 +399,36 @@ impl Row {
399
399
  }
400
400
  false
401
401
  }
402
- pub fn highlight(&mut self, opts: &HighlightingOptions, word: Option<&str>) {
402
+ #[allow(clippy::indexing_slicing, clippy::integer_arithmetic)]
403
+ pub fn highlight(
404
+ &mut self,
405
+ opts: &HighlightingOptions,
406
+ word: Option<&str>,
407
+ start_with_comment: bool,
408
+ ) -> bool {
403
409
  self.highlighting = Vec::new();
404
410
  let chars: Vec<char> = self.string.chars().collect();
405
411
  let mut index = 0;
412
+ let mut in_ml_comment = start_with_comment;
413
+ if in_ml_comment {
414
+ let closing_index = if let Some(closing_index) = self.string.find("*/") {
415
+ closing_index + 2
416
+ } else {
417
+ chars.len()
418
+ };
419
+ for _ in 0..closing_index {
420
+ self.highlighting.push(highlighting::Type::MultilineComment);
421
+ }
422
+ index = closing_index;
423
+ }
406
424
  while let Some(c) = chars.get(index) {
425
+ if self.highlight_multiline_comment(&mut index, &opts, *c, &chars) {
426
+ in_ml_comment = true;
427
+ continue;
428
+ }
429
+ in_ml_comment = false;
407
430
  if self.highlight_char(&mut index, opts, *c, &chars)
408
431
  || self.highlight_comment(&mut index, opts, *c, &chars)
409
- || self.highlight_multiline_comment(&mut index, &opts, *c, &chars)
410
432
  || self.highlight_primary_keywords(&mut index, &opts, &chars)
411
433
  || self.highlight_secondary_keywords(&mut index, &opts, &chars)
412
434
  || self.highlight_string(&mut index, opts, *c, &chars)
@@ -418,6 +440,10 @@ impl Row {
418
440
  index += 1;
419
441
  }
420
442
  self.highlight_match(word);
443
+ if in_ml_comment && &self.string[self.string.len().saturating_sub(2)..] != "*/" {
444
+ return true;
445
+ }
446
+ false
421
447
  }
422
448
  }
423
449
 

See this step on github

We have now changed the signature of highlight: It receives a new boolean, and it returns a boolean. The boolean it receives is true when the highlighting should start within a multiline comment. If that is the case, we look for the closing index before we enter the highlighting loop and highlight everything until then as a mutli line comment. Similar to what we are doing in highlight_multiline_comment, we go up until the end of the line if we can’t find the closing */.

During highlighting, we are now keeping track if the last thing we highlighted was a multiline comment. If we end the loop on a multiline comment, we check if the last thing we saw was the ending of a multiline comment, as we want to return false if we are out of a multi line comment, and true if the multi line comment we are in has not been closed yet.

You can try saving a file with multi line comments now, or you can enter a multi line comment and then trigger a search to see the correct highlighting. Let’s now make sure that the highlighting is also correct in all other cases.

src/document.rs CHANGED
@@ -17,10 +17,12 @@ impl Document {
17
17
  pub fn open(filename: &str) -> Result<Self, std::io::Error> {
18
18
  let contents = fs::read_to_string(filename)?;
19
19
  let file_type = FileType::from(filename);
20
+ let mut start_with_comment = false;
20
21
  let mut rows = Vec::new();
21
22
  for value in contents.lines() {
22
23
  let mut row = Row::from(value);
23
- row.highlight(&file_type.highlighting_options(), None, false);
24
+ start_with_comment =
25
+ row.highlight(&file_type.highlighting_options(), None, start_with_comment);
24
26
  rows.push(row);
25
27
  }
26
28
  Ok(Self {
@@ -52,9 +54,7 @@ impl Document {
52
54
  }
53
55
  #[allow(clippy::indexing_slicing)]
54
56
  let current_row = &mut self.rows[at.y];
55
- let mut new_row = current_row.split(at.x);
57
+ let new_row = current_row.split(at.x);
56
- current_row.highlight(&self.file_type.highlighting_options(), None, false);
57
- new_row.highlight(&self.file_type.highlighting_options(), None, false);
58
58
  #[allow(clippy::integer_arithmetic)]
59
59
  self.rows.insert(at.y + 1, new_row);
60
60
  }
@@ -65,19 +65,16 @@ impl Document {
65
65
  self.dirty = true;
66
66
  if c == '\n' {
67
67
  self.insert_newline(at);
68
- return;
68
+ } else if at.y == self.rows.len() {
69
- }
70
- if at.y == self.rows.len() {
71
69
  let mut row = Row::default();
72
70
  row.insert(0, c);
73
- row.highlight(&self.file_type.highlighting_options(), None, false);
74
71
  self.rows.push(row);
75
72
  } else {
76
73
  #[allow(clippy::indexing_slicing)]
77
74
  let row = &mut self.rows[at.y];
78
75
  row.insert(at.x, c);
79
- row.highlight(&self.file_type.highlighting_options(), None, false);
80
76
  }
77
+ self.highlight(None);
81
78
  }
82
79
  #[allow(clippy::integer_arithmetic, clippy::indexing_slicing)]
83
80
  pub fn delete(&mut self, at: &Position) {
@@ -90,12 +87,11 @@ impl Document {
90
87
  let next_row = self.rows.remove(at.y + 1);
91
88
  let row = &mut self.rows[at.y];
92
89
  row.append(&next_row);
93
- row.highlight(&self.file_type.highlighting_options(), None, false);
94
90
  } else {
95
91
  let row = &mut self.rows[at.y];
96
92
  row.delete(at.x);
97
- row.highlight(&self.file_type.highlighting_options(), None, false);
98
93
  }
94
+ self.highlight(None);
99
95
  }
100
96
  pub fn save(&mut self) -> Result<(), Error> {
101
97
  if let Some(file_name) = &self.file_name {

See this step on github

We have now restructured the code a bit so that the whole document is re-highlighted on any update or delete operation. If you check it now, you should notice two things: First, it works! Very satisfying! Second, the performance is abysmal. Not satisfying!

The reason is, of course, that we are re-highlighting the whole document all the time, on every change, to make sure the highlighting is correct everywhere and in every case. Let’s try to improve the performance by highlighting the document less often.

src/document.rs CHANGED
@@ -17,13 +17,9 @@ impl Document {
17
17
  pub fn open(filename: &str) -> Result<Self, std::io::Error> {
18
18
  let contents = fs::read_to_string(filename)?;
19
19
  let file_type = FileType::from(filename);
20
- let mut start_with_comment = false;
21
20
  let mut rows = Vec::new();
22
21
  for value in contents.lines() {
23
- let mut row = Row::from(value);
22
+ rows.push(Row::from(value));
24
- start_with_comment =
25
- row.highlight(&file_type.highlighting_options(), None, start_with_comment);
26
- rows.push(row);
27
23
  }
28
24
  Ok(Self {
29
25
  rows,
@@ -74,7 +70,6 @@ impl Document {
74
70
  let row = &mut self.rows[at.y];
75
71
  row.insert(at.x, c);
76
72
  }
77
- self.highlight(None);
78
73
  }
79
74
  #[allow(clippy::integer_arithmetic, clippy::indexing_slicing)]
80
75
  pub fn delete(&mut self, at: &Position) {
@@ -91,21 +86,14 @@ impl Document {
91
86
  let row = &mut self.rows[at.y];
92
87
  row.delete(at.x);
93
88
  }
94
- self.highlight(None);
95
89
  }
96
90
  pub fn save(&mut self) -> Result<(), Error> {
97
91
  if let Some(file_name) = &self.file_name {
98
92
  let mut file = fs::File::create(file_name)?;
99
93
  self.file_type = FileType::from(file_name);
100
- let mut start_with_comment = false;
101
94
  for row in &mut self.rows {
102
95
  file.write_all(row.as_bytes())?;
103
96
  file.write_all(b"\n")?;
104
- start_with_comment = row.highlight(
105
- &self.file_type.highlighting_options(),
106
- None,
107
- start_with_comment,
108
- );
109
97
  }
110
98
  self.dirty = false;
111
99
  }
@@ -150,9 +138,19 @@ impl Document {
150
138
  }
151
139
  None
152
140
  }
153
- pub fn highlight(&mut self, word: Option<&str>) {
141
+ pub fn highlight(&mut self, word: &Option<String>, until: Option<usize>) {
154
142
  let mut start_with_comment = false;
155
- for row in &mut self.rows {
143
+ let until = if let Some(until) = until {
144
+ if until.saturating_add(1) < self.rows.len() {
145
+ until.saturating_add(1)
146
+ } else {
147
+ self.rows.len()
148
+ }
149
+ } else {
150
+ self.rows.len()
151
+ };
152
+ #[allow(clippy::indexing_slicing)]
153
+ for row in &mut self.rows[..until] {
156
154
  start_with_comment = row.highlight(
157
155
  &self.file_type.highlighting_options(),
158
156
  word,
src/editor.rs CHANGED
@@ -45,6 +45,7 @@ pub struct Editor {
45
45
  document: Document,
46
46
  status_message: StatusMessage,
47
47
  quit_times: u8,
48
+ highlighted_word: Option<String>,
48
49
  }
49
50
 
50
51
  impl Editor {
@@ -86,16 +87,25 @@ impl Editor {
86
87
  offset: Position::default(),
87
88
  status_message: StatusMessage::from(initial_status),
88
89
  quit_times: QUIT_TIMES,
90
+ highlighted_word: None,
89
91
  }
90
92
  }
91
93
 
92
- fn refresh_screen(&self) -> Result<(), std::io::Error> {
94
+ fn refresh_screen(&mut self) -> Result<(), std::io::Error> {
93
95
  Terminal::cursor_hide();
94
96
  Terminal::cursor_position(&Position::default());
95
97
  if self.should_quit {
96
98
  Terminal::clear_screen();
97
99
  println!("Goodbye.\r");
98
100
  } else {
101
+ self.document.highlight(
102
+ &self.highlighted_word,
103
+ Some(
104
+ self.offset
105
+ .y
106
+ .saturating_add(self.terminal.size().height as usize),
107
+ ),
108
+ );
99
109
  self.draw_rows();
100
110
  self.draw_status_bar();
101
111
  self.draw_message_bar();
@@ -150,7 +160,7 @@ impl Editor {
150
160
  } else if moved {
151
161
  editor.move_cursor(Key::Left);
152
162
  }
153
- editor.document.highlight(Some(query));
163
+ editor.highlighted_word = Some(query.to_string());
154
164
  },
155
165
  )
156
166
  .unwrap_or(None);
@@ -159,7 +169,7 @@ impl Editor {
159
169
  self.cursor_position = old_position;
160
170
  self.scroll();
161
171
  }
162
- self.document.highlight(None);
172
+ self.highlighted_word = None;
163
173
  }
164
174
  fn process_keypress(&mut self) -> Result<(), std::io::Error> {
165
175
  let pressed_key = Terminal::read_key()?;
src/row.rs CHANGED
@@ -165,7 +165,7 @@ impl Row {
165
165
  None
166
166
  }
167
167
 
168
- fn highlight_match(&mut self, word: Option<&str>) {
168
+ fn highlight_match(&mut self, word: &Option<String>) {
169
169
  if let Some(word) = word {
170
170
  if word.is_empty() {
171
171
  return;
@@ -403,7 +403,7 @@ impl Row {
403
403
  pub fn highlight(
404
404
  &mut self,
405
405
  opts: &HighlightingOptions,
406
- word: Option<&str>,
406
+ word: &Option<String>,
407
407
  start_with_comment: bool,
408
408
  ) -> bool {
409
409
  self.highlighting = Vec::new();

See this step on github

We have now removed all highlighting directly within Row, as we are now doing the highlighting controlled by the editor. To do that, we have added a new paramter to highlight in row: until, which denotes the index of the last line for which the highlighting should be calculated. We know that the highlighting of everything on screen always depends on the earlier rows in the document, but not the rows below.

We are setting the highlighting in the editor’s refresh_screen method. This also means that we have to change how search results are being displayed, as refresh_screen is always called when we conduct a search, which means that we would overwrite the highlighting of a word during refresh_screen, rendering our highlighting of the match useless. We solve this by storing the currently highlighted word as part of the editor struct, setting and resetting this during the search.

You should now be able to verify that the performance is now much, much better. The higher up the user is in the file, the better the performance will be, as we will be only highlighting rows up until the end of the current screen. But if the user is at the bottom of the file, we are still doing a lot of highlighting of all the rows above.

It’s true that we need to take the rows above into account when highlighting to understand whether or not we are in a multi-line comment or not. However, there is no need to actually re-do the highlighting for all these lines.

Let’s make sure that only the line which has been edited is being highlighted, as well as all following lines (to make sure multi line comments still work). Together with the recent change, this means that we are highlighting only the lines between the current line and the last line on screen.

src/document.rs CHANGED
@@ -70,6 +70,14 @@ impl Document {
70
70
  let row = &mut self.rows[at.y];
71
71
  row.insert(at.x, c);
72
72
  }
73
+ self.unhighlight_rows(at.y);
74
+ }
75
+
76
+ fn unhighlight_rows(&mut self, start: usize) {
77
+ let start = start.saturating_sub(1);
78
+ for row in self.rows.iter_mut().skip(start) {
79
+ row.is_highlighted = false;
80
+ }
73
81
  }
74
82
  #[allow(clippy::integer_arithmetic, clippy::indexing_slicing)]
75
83
  pub fn delete(&mut self, at: &Position) {
@@ -86,6 +94,7 @@ impl Document {
86
94
  let row = &mut self.rows[at.y];
87
95
  row.delete(at.x);
88
96
  }
97
+ self.unhighlight_rows(at.y);
89
98
  }
90
99
  pub fn save(&mut self) -> Result<(), Error> {
91
100
  if let Some(file_name) = &self.file_name {
src/row.rs CHANGED
@@ -9,6 +9,7 @@ use unicode_segmentation::UnicodeSegmentation;
9
9
  pub struct Row {
10
10
  string: String,
11
11
  highlighting: Vec<highlighting::Type>,
12
+ pub is_highlighted: bool,
12
13
  len: usize,
13
14
  }
14
15
 
@@ -17,6 +18,7 @@ impl From<&str> for Row {
17
18
  Self {
18
19
  string: String::from(slice),
19
20
  highlighting: Vec::new(),
21
+ is_highlighted: false,
20
22
  len: slice.graphemes(true).count(),
21
23
  }
22
24
  }
@@ -118,9 +120,11 @@ impl Row {
118
120
 
119
121
  self.string = row;
120
122
  self.len = length;
123
+ self.is_highlighted = false;
121
124
  Self {
122
125
  string: splitted_row,
123
126
  len: splitted_length,
127
+ is_highlighted: false,
124
128
  highlighting: Vec::new(),
125
129
  }
126
130
  }
@@ -406,8 +410,19 @@ impl Row {
406
410
  word: &Option<String>,
407
411
  start_with_comment: bool,
408
412
  ) -> bool {
409
- self.highlighting = Vec::new();
410
413
  let chars: Vec<char> = self.string.chars().collect();
414
+ if self.is_highlighted && word.is_none() {
415
+ if let Some(hl_type) = self.highlighting.last() {
416
+ if *hl_type == highlighting::Type::MultilineComment
417
+ && self.string.len() > 1
418
+ && self.string[self.string.len() - 2..] == *"*/"
419
+ {
420
+ return true;
421
+ }
422
+ }
423
+ return false;
424
+ }
425
+ self.highlighting = Vec::new();
411
426
  let mut index = 0;
412
427
  let mut in_ml_comment = start_with_comment;
413
428
  if in_ml_comment {
@@ -443,6 +458,7 @@ impl Row {
443
458
  if in_ml_comment && &self.string[self.string.len().saturating_sub(2)..] != "*/" {
444
459
  return true;
445
460
  }
461
+ self.is_highlighted = true;
446
462
  false
447
463
  }
448
464
  }

See this step on github

We solve this by storing on a row whether or not it is currently properly highlighted. If highlight is called on a row which is already highlighted, we only check if this row ends with an unclosed multiline comment, to determine the return value of highlight. If it’s not highlighted, we do the highlighting as usual and then set is_highlighted to true.

We also make sure that is_highlighted is false whenever a row is modified or created.

In our document, we are then setting is_highlighted to false for every row after and including the row which has been currently edited.

In combination with the previous change, this means that all rows after the current row are marked for re-highlighting whenever we change something, but re-highlighting is only done up until the last row on screen. In combination, this means that when you are typing, at most all the rows currently on the screen are being re-highlighted, which can be done at a much better performance.

To make sure our search results are still displayed, we re-highlight a row even if it has been highlighted previously in case a word is provided to highlight.

You’re done!

That’s it! Our text editor is finished. In the appendices, you’ll find some ideas for features you might want to extend the editor with on your own.

Twitter, Facebook