Introduction

This is part 1 in a multi-article series on building your own IDE. The series will show how to extend the Visual Studio Isolated Shell to create an IDE. Each article in the series will build on the previous article to add new features to the environment. This article will create the My C language service in C# using the Irony Compiler Construction Kit. The language was initially created in the ManagedMyC example that is included with the Visual Studio SDK. The SDK example uses the Managed Babel System.

Using the Code in the Download

The first download supplied with this article contains the completed My C language service sample discussed in this article. It requires Visual Studio 2008 Standard edition or higher, and the Microsoft Visual Studio 2008 SDK.

In order to run the solution:

Download and unzip the source files.

Open MyCLanguageService.sln in Visual Studio 2008.

Build and run the project (F5).

When the Visual Studio experimental hive opens, open a My C source file (.myc) in the code editor. Two test files are included in the download: short.myc and thing.myc.

Background

A language service in Visual Studio provides language-specific support (syntax coloring and highlighting, statement completion, brace matching, parameter information tooltips, etc.). A language service is implemented by creating a Visual Studio Integration Package. A project template is provided when the Visual Studio SDK is installed.

The Irony Compiler Construction Kit combines the lexical and parser specification in one C# file. Also, since the file is already in C#, no code generation takes place. This creates a solution that is easier to debug and maintain.

Creating the Language Service Package

The second download is a compressed file containing a template to generate a Visual Studio Integration Package for an Irony Language Service. In order to use the template, download and unzip the contents to a temporary location. The two DLL files, Wizard.dll and IronyLanguageServiceWizard.dll, should be placed in the Global Assembly Cache (C:\Windows\assembly). The third file, IronyLanguageServicePackage.zip, should be placed in the Visual Studio user templates directory, typically "My Documents\Visual Studio 2008\Templates\ProjectTemplates". This will install a Visual Studio template that can be accessed in the New Project dialog. It will appear in the My Templates section when C# is selected.

When creating a new Irony Language Service Package, a wizard is run to gather information specific to the language that is being created.

The first page requests basic information about the language. The company name will be used as the namespace for all the classes. Also, if your language supports multiple file extensions, separate each extension with a semicolon.

Next, the wizard requests the features your language will support in Visual Studio. Leaving the options checked does not create any additional work, because the necessary integration is already included in the package.

Package Load Keys (PLK) are required for VSPackages to load successfully. The Visual Studio SDK provides a Developer Load Key (DLK) that makes it possible for VSPackages to load without a PLK during development. If the language being developed will be used on machines without the Visual Studio SDK, a PLK is a must. A PLK can be generated here.

Defining Grammar in Irony

After running the Irony Language Service Wizard, there should be a Grammar.cs file open in the editor. This is where the Irony Grammar will be defined. The LEX and YACC files included with the ManagedMyC sample are being used for the language specification.

Testing the My C Grammar

There are two options for testing the grammar: Grammar Explorer and Visual Studio Experimental Hive. The next two sections are a walkthrough in setting up the environment for either option.

Grammar Explorer

One of the many nice things about Irony is that it includes a tool for testing Irony grammars, called Grammar Explorer.

To setup the environment, right-click the language service project and select Properties. Select the Debug tab and choose Start External Program, and point it to the Irony.GrammarExplorer executable in the Resources directory.

The first time Grammar Explorer is run, the grammar will need to be added. To add a grammar into Grammar Explorer, click the button next to the drop down and click Add Grammar from the menu.

Find the language service DLL and click Open.

The grammar should appear in the list and be selected, so the only thing to do here is click OK.

Once the grammar is added, it can be selected from the drop down. Now, it will display the grammar errors.

The shift-reduce conflict on the else is in the original YACC grammar, and is caught by MPPG when the ManagedMyC example is compiled. This is a common problem with C grammars. Here is an example demonstrating the conflict:

if (x>y) if (x<z) foo(); else bar();

The compiler faces the problem of deciding whether to reduce "if (x<z) foo();" to an if_statement and then shift the "else", or if the "else" should be shifted first: both would be equally valid under the grammar as stated, but reduction would result in this:

if (x>y)
{
if (x<z)
{
foo();
}
}
else
{
bar();
}

... associating the "else" with the first "if", whereas shifting the "else" produces...

if (x>y)
{
if (x<z)
{
foo();
}
else
{
bar();
}
}

... with the "else" associated with the second "if".

Irony provides a way to resolve this issue, which is presented below. However, if left alone, the grammar will still perform correctly. This is because the default behaviour of Irony is to shift, which produces the correct result in C and similar languages.

Visual Studio Experimental Hive

To setup the environment, right-click the language service project and select Properties. Select the Debug tab and choose Start External Program, and point it to the Visual Studio 2008 executable (default: C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\devenv.exe). Also, add /ranu /rootsuffix Exp to the command line arguments.

Build and run the project in Visual Studio (F5). This will launch the experimental hive. To test the language service, open a file with the .myc extension, or create a text file and save it with the .myc extension. Two files were included in the demo project: short.myc and thing.myc.

Grammar Additions\Changes

This section covers additions or changes made to the My C sample grammar to demonstrate additional features of Irony.

Possible Non-terminal Duplication Warning

This warning is displayed anytime a non-terminal's rule contains a single non-terminal. There are several ways to resolve this issue. First, consider whether the non-terminal is really necessary. If the non-terminal should be there, the issue can be resolved by merging the two non-terminals.

The My C grammar has a program non-terminal that is causing this warning. This non-terminal was included from the original YACC file, and was probably placed there for expansion. To resolve the warning, the program and the declarations non-terminals are merged. Note that the program non-terminal must follow declarations so that the assignment is made after its rule is initialized.

Number Prefixes and Suffixes

Irony also supports number prefixes and suffixes. For example, the C language uses the prefix 0x to specify hexadecimal numbers. It also uses the f suffix to designate a value as a float. This can be defined in Irony with the following:

Registering Brace Pairs

In Irony, it is a little different. Any terminal that should be matched in Visual Studio is registered in the grammar with the function RegisterBracePair. The My C grammar registers parenthesis and curly braces.

this.RegisterBracePair("{", "}");
this.RegisterBracePair("(", ")");

Operator Precedence

Another strength of Irony is the ability to define operator precedence. The My C language gives a higher precedence to the multiply and divide operators.

Points of Interest

When I first discovered Visual Studio language services, I used the Managed Babel System. After working with a YACC grammar, I quickly found that it was difficult to debug and would be hard to maintain in the future. Irony seems to solve both of these issues, and performs just as well. Irony also uses BNF expressions, so it was a very easy transition from YACC.

Articles to Come

Part 2 - Building your First Isolated Shell

Part 3 - Building your First Custom Project System

Part 4 - Building your First Advanced Language Service

Part 5 - Building your First Microsoft Build Task

Part 6 - Building your First Custom Debug Engine

History

December 10, 2009

Updated article and download content to work with the alpha release of Irony (October 13, 2009). Changes to the source code were made by Thaddeus Ryker.

About the Author

Comments and Discussions

I am very interested in the Visual Studio Extension that you created but am having trouble getting it running in VS2013. I was able to get the extension running but have issues with null references in various places.

I have tryed to use the Wizzar bit got this error:
---------------------------
Microsoft Visual Studio
---------------------------
Error: this template attempted to load component assembly 'IronyLanguageServiceWizard, Version=1.0.0.0, Culture=Neutral, PublicKeyToken=9ee59967db6ae5cb'. For more information on this problem and how to enable this template, please see documentation on Customizing Project Templates.
---------------------------
OK
---------------------------
what is wrong?

I followed this article pretty good, I constructed my own grammar, tested it with the Grammar Explorer and it works. But when I launch the experimental hive with /ranu /rootsuffix Exp as command line switches, I can't seem to get the intellisense as shown in the picture. I can't seem to get the package registered properly. I'm using Visual Studio 2010.
Any suggestions would be great. Thanks.

This was taken from the mini-python grammer that is included with the irony source at http://irony.codeplex.com. I did not test this so I may be missing something else but this information should be on Irony's site at codeplex.

Is it also possible to have a dynamic list of tokens (such as user created function/class names ... foo, bar, etc.) like VS intellisense has? I'm guessing this is beyond the code that is generated by Irony(?) but must still be possible..!

High I'm trying to build an extensibility project using Irony how ever I'm on VS2010 and things seems to have change a bit when it comes to the SDK. Im getting a few errors. How you tryed the 2010 SDK?

Unfortunately, I haven't looked at extensibility in Visual Studio 2010 yet. In the Visual Studio 2008 SDK, there was a MyC language service sample that this article was loosly based off of. If this has been included in the VIsual Studio 2010 SDK it may contain the answers you need.

I've found an easy solution
create a standard VS Package solution (using the SDK template) and import all the files from the IronyLanguageService solution
add references to TextManager.Interop8.0 and Irony and that's basically it (a few errors will still exist but they are easy to fix since the compiler almost tells you what to do)

I'm just now digging into creating a language service with Irony, and the first big thing that strikes me about Irony is that it doesn't seem to do any error recovery, it just stops altogether on the first syntax error like many LALR parsers. I'm hoping that I'm wrong, and that it does have some error recovery abilities, and that it's just the samples I've seen which use Irony that exhibit this problem.

Does anyone know if Irony can continue parsing after encountering a syntax error in the parsed source?

I am not sure if error recovery has been implemented yet, but I know it is on the to do list. To get a better answer contact the author at the www.codeplex.com/irony[^]. He is usually quick to respond and you may be able to convince him to add this sonner rather than later.