In the previous post of this series we started the Language Designer and entered some high-level general properties about the language we’re going to create, which is ECMAScript. Today we’re going to use a wizard to quickly create a dynamic lexer for our language. With the features described below, we have specifically tried to make the generation of a new language as easy as possible for new customers.
Selecting a lexer type
A new language project doesn’t have a lexer defined. When we click the Lexer button in the ribbon, the Lexer pane is opened. We are presented with a Lexer Type sub-pane that allows us to choose the type of lexer to use.
For our ECMAScript language sample, we’ll use a Dynamic lexer so select Dynamic in the radio button list and click the Change Lexer Type button. This opens the New Dynamic Lexer Wizard.
The New Dynamic Lexer Wizard
The New Dynamic Lexer Wizard is the fastest way to create a new lexer for a language. It asks you a series of questions about your language and it generates a lexer for you. This takes most of the guesswork out of creating a lexer from scratch and gets you a working lexer in minutes.
For our ECMAScript language, we’ll use some specifications found on the Mozilla web site for ECMAScript (http://www.mozilla.org/js/language/E262-3.pdf).
Overview
The first page that opens in the wizard is an overview. It allows you to select whether to walk through the wizard to create a lexer or whether to just make an empty lexer instead.
For this walkthrough we’ll choose the first option.
Line terminators and identifiers
This page asks whether line terminators are significant and asks for a general guideline on identifier syntax.
For ECMAScript we’ll choose that line terminators are significant and we’ll use standard identifier syntax. Note that the output from this wizard is completely modifiable after the wizard completes.
Keywords
On this page, we paste in the list of keywords for the language and indicate their case sensitivity.
The wizard uses whitespace to delimit the keywords in the textbox. This makes it easy to paste keywords directly from a table in a web page or PDF language specification. The wizard will also auto-sort the keywords following completion.
Operators
On this page, operators for the language are pasted in. A button can be used to auto-insert the most common operators used in most languages. Also check which delimiters have special meaning in the language.
For ECMAScript, we’ve pasted in the operators listed in the language specification and marked that parenthesis, curly and square braces are all significant.
Strings (primary syntax)
On this page, we indicate the primary string syntax.
For ECMAScript, we select double quotes with backslash escapes as the primary syntax.
Strings (alternate syntax)
On this page, we indicate the alternate string syntax.
For ECMAScript, we select single quotes with backslash escapes as the alternate syntax.
Numbers
On this page we choose which number literal types to support.
For ECMAScript, we choose all integer, real, and hex numbers.
Comments
On the final wizard page, we input the syntax for comments.
For ECMAScript, we choose // to start single-line comments and /* */ to be the multi-line comment delimiters.
Next steps
When we press the Finish button, a new dynamic lexer is initialized for us based on the language profile information we entered. In the next post, we’ll examine what was created. Stay tuned!