The Actipro Blog

All the latest UI control development news from Actipro

SyntaxEditor for WPF - Adding syntactic/semantic parsing support

As we move forward on new features for SyntaxEditor for WPF, the next feature area we’ve been focusing on is adding core support for triggering syntactic/semantic parsing (commonly called “parsing”) following document text changes.  SyntaxEditor for WinForms has a very nice system for this that allows a worker thread to perform the parsing operations and then report back to the document when complete.  It’s a core item for getting advanced IntelliPrompt features such as those found in our .NET Languages Add-on working.

Parsing can take time to perform since you generally use the phase to build a tree-based model of the document via an abstract syntax tree, along with performing syntax checking. 

SyntaxEditor for WinForms parsing pipeline – the good and the bad

Our WinForms design of the SemanticParserService works great in numerous respects.  It supports the calling of our own semantic parsers built using the Grammar Designer, supports calling of any third-party parser instead (ANTLR, GOLD parser, etc.), offloads work onto a separate thread to prevent UI blocking, and has numerous methods for querying what it is doing at any given time.

The two main negative feedback items we’ve received from customers in regards to the parsing pipeline are:

1) It’s a bit convoluted to implement some of the methods in your syntax language that are required to call the SemanticParserService and handle the results.

2) There are some scenarios where having more than one worker thread in the SemanticParserService would be handy.

SyntaxEditor for WPF improvements

As we work on our next generation text/parsing framework that is in development with the SyntaxEditor for WPF product, we have tried to improve on feature areas when they get moved over.  Here is another example…

We are changing naming a bit from the WinForms design.  Since the industry standard of naming for the process of scanning/tokenizing is Lexing and syntactic/semantic parsing is Parsing, we are going with those general names in the next WPF build.  Therefore lexical parsers will now be known as Lexers and semantic parsers will now be called Parsers.  This keeps all our naming in line with all other major products (MGrammar, ANTLR, Lex, YACC, academic texts, etc.) and also keeps shorter type names too.

This past week we began work on building the parsing pipeline, and by that I mean the ability for an IParser (new interface) to be used to parse a document following text changes.  Although we ended up reusing a lot of the design of the WinForms version in this area since it has been very successful, we did make a couple very important enhancements.

Parsing pipeline simplification

The WinForms version, while flexible, was a bit overly complex and convoluted in how a request for semantic parsing was made and returned.  It could require three method implementations that weren’t very obvious without examples.

In the WPF design, it’s much easier.  Here’s how it goes…

1) If your language supports syntactic/semantic parsing, you have a class that implements IParser and register that class as a service with your language.

2) IParser has a single method called Parse that gives you information about what to parse and you return a result.  Implement that method, which could be anything from writing your own parsing code to calling a third-party parser like ANTLR.

That’s it!  If you have made available statically an IParseRequestDispatcher (which is the WPF’s version of the SemanticParserService), your parser is automatically called when appropriate via a worker thread.  Although we may still have to tweak a couple more things, it doesn’t really get any simpler than that.

Multi-threaded parsing support

As mentioned above, IParseRequestDispatcher is the WPF’s version of the WinForms SemanticParserService.  It has pretty much the same object model although it is instance-based instead of static.  We did this so that custom dispatchers can be written down the road to replace our default one if appropriate.  There is another object that stores a static instance of the current “global” dispatcher that should be used, if any.  So where in WinForms you would have started the SemanticParserService, in WPF at the start of your app you would new up an instance of our default dispatcher and set that to a property on a static object that tracks the “global” dispatcher (AmbientParseRequestDispatcherProvider).

Again most of the object model for the dispatcher is the same as WinForms however we’ve made one very key improvement here.  The new version supports multiple threads, where WinForms only supported one worker thread.  The maximum number of threads can be configured.  By default though, it is based on how many processor cores your machines have.  So if you have a 32-bit system, it will most likely still use one worker thread by default.  If you have a quad-core system, it will use up to three.

It doesn’t start all the threads immediately either.  It only creates more than one if necessary (too many requests queued up).  And it will remove old unused threads after some time.

The benefit of multiple threads is seen when there are potentially multiple requests being thrown at the dispatcher.  For instance, say your app loads a project and it has to immediately parse hundreds of files (this whole framework can be used independent of a SyntaxEditor control).  At the same time the end user is editing one or more files.  In the dispatcher queue, requests are prioritized where editor requests can be prioritized to occur before any other low-priority “project” requests.  With multiple threads, you can ensure that the editor side of things continues to be getting parse results back fast, even if another worker thread is busy tackling some large project files.

Summary

Expect to see the beginnings of this parser pipeline in the next build.  Again, our design is open-ended so any third-party parser is supported, meaning you can choose to call ANTLR or whatever you use for parsing.

This framework is being built in .NET 2.0 code too so that we can use it in future versions of SyntaxEditor for WinForms, or for other platforms.

Pingbacks and trackbacks (1)+

Comments are closed