ReSharper and Roslyn: Q&A

As you probably know based on news from BUILD Windows conference last week, Roslyn, the new .NET Compiler Platform with code analysis APIs for C# and VB.NET, has been pushed to Preview phase and open sourced. In case the event slipped your attention, here’s a nice tour of Roslyn on the C# FAQ blog at MSDN.

Immediately we at JetBrains were faced with multiple questions on the perspectives of ReSharper using Roslyn for its code analysis and how the two tools might compete. The flow of questions wouldn’t end, to the point of introducing a template to answer them:

Seriously though, it was clear that we needed to elaborate on the ReSharper vs Roslyn issue. Hence this post.

We sat down with Sergey Shkredov (@serjic), ReSharper Project Lead and .NET Tools Department Lead at JetBrains, and Alex Shvedov (@controlflow), a Senior Developer on the ReSharper team who’s responsible for ReSharper’s Generate functionality, code annotations and support for XML-based languages. The following Q&A is a summary of the conversation that we had with them.

What’s JetBrains stance towards Roslyn? Do we consider the technology and its Open Source status important and valuable?

Roslyn is definitely important and a good step forward for Microsoft in that it should help Visual Studio users take advantage of more C# and VB.NET code editing and analysis features in Visual Studio out of the box.

It should also help Visual Studio extension developers write code-centric extensions against a consistent API while having the opportunity to know how it works inside, thanks to the Open Source status of the project. This is not to mention hackers who are willing to spend their time forking the compiler and tuning it to make, say, C# the ideal language they’ve always envisioned.

We also believe that Roslyn is no less important for Microsoft itself. Faced with the burden of maintaining a plethora of Visual Studio integrated tools including code editing tools, IntelliTrace and code designers, the folks at Microsoft are interested in making these tools as flexible and easy to update as possible. Roslyn should enable updating .NET languages and experimenting with them faster than before. Apart from that, the old compiler wouldn’t let launch compilation steps in parallel, and Roslyn is expected to enable that, bringing more scalability to the table.

What’s the point of making Roslyn Open Source?

As to the act of letting Roslyn go Open Source, we don’t believe that Microsoft is expecting anyone from outside of the company to develop the compiler for them. Programming languages are entities too monolithic and complex to justify accepting external changes of any significance. Therefore, we expect Microsoft to keep the function of designing .NET languages totally to itself without depending on the community.

The true value of Roslyn going Open Source lies in enabling extension developers to look into Roslyn code that is relevant to their purposes: how it’s written and whether it’s efficient. They might debug or profile it to see if it’s the culprit of unexpected behavior in their extensions or if it introduces performance issues. This is possibly the workflow whereby meaningful pull requests might start coming in to the Roslyn repository.

As to possible endeavors to fork and modify the compiler to address application- or domain-specific tasks, this scenario appears to be a shot in the foot. Even if the default compiler in Visual Studio can be replaced with a fork, instrumental support for the fork ends as soon as you go beyond Visual Studio. In theory we can imagine a custom INotifyPropertyChanged implementation based on a Roslyn fork that can even gain certain popularity. However, we can barely imagine supporting it in ReSharper as our intention is to focus on supporting the official version of Roslyn.

Will ReSharper take advantage of Roslyn?

The short answer to this tremendously popular question is, no, ReSharper will not use Roslyn. There are at least two major reasons behind this.

The first reason is the effort it would take, in terms of rewriting, testing and stabilizing. We’ve been developing and evolving ReSharper for 10 years, and we have a very successful platform for implementing our inspections and refactorings. In many ways, Roslyn is very similar to the model we already have for ReSharper: we build abstract syntax trees of the code and create a semantic model for type resolution which we use to implement the many inspections and refactorings. Replacing that much code would take an enormous amount of time, and risk destabilizing currently working code. We’d rather concentrate on the functionality we want to add or optimize, rather than spend the next release cycle reimplementing what we’ve already got working.

The second reason is architectural. Many things that ReSharper does cannot be supported with Roslyn, as they’re too dependent on concepts in our own code model. Examples of these features include Solution-Wide Error Analysis, code inspections requiring fast lookup of inheritors, and code inspections that require having the “big picture” such as finding unused public classes. In cases where Roslyn does provide suitable core APIs, they don’t provide the benefit of having years of optimization behind them: say, finding all derived types of a given type in Roslyn implies enumerating through all classes and checking whether each of them is derived. On the ReSharper side, this functionality belongs to the core and is highly optimized.

The code model underlying ReSharper features is conceptually different from Roslyn’s code model. This is highlighted by drastically different approaches to processing and updating syntax trees. In contrast to ReSharper, Roslyn syntax trees are immutable, meaning that a new tree is built for every change.

Another core difference is that Roslyn covers exactly two languages, C# and VB.NET, whereas ReSharper architecture is multilingual, supporting cross-language references and non-trivial language mixtures such as Razor. Moreover, ReSharper provides an internal feature framework that streamlines consistent feature coverage for each new supported language. This is something that Roslyn doesn’t have by definition.

Will it be practical to use both ReSharper and Roslyn-based functionality in Visual Studio?

This is a tricky problem as it’s still uncertain whether we would be able to disable Roslyn-based features (such as refactorings or error highlighting) when integrating into new releases of Visual Studio. If we’re unable to do that, performance would take a hit. Apart from ReSharper’s own immanent memory and performance impact, Roslyn’s immutable code model would increase memory traffic, which would in turn lead to more frequent garbage collection, negatively impacting performance.

We’re hopeful that this problem would be solved in favor of letting us disable Roslyn features that ReSharper overrides, because otherwise ReSharper would have to work in a highly resource-restricted environment. Irrelevant of whether this happens though, we’ll keep doing what we can do, minimizing ReSharper’s own performance impact.

As Roslyn is now Open Source, which parts of its code are going to be of particular interest to ReSharper developers?

We’ll be sure to peek into Roslyn code and tests from time to time, to see how C# and VB.NET language features are implemented. We don’t rule out that actual code supporting them is going to emerge before formal specifications are finalized. In fact, we’ve already started.

That’s more or less the picture of living in the Roslyn world as we see it today. As time goes by, we’ll see if things turn out the way we expected them to.

Meanwhile, if you have questions that were not addressed in this post, please ask in comments and we’ll try to come up with meaningful answers.

Thanks. We’re going to use (as we’ve traditionally used) all sources of information about languages in VS that are available to us, and the list now includes Roslyn. As written above, We’ll be sure to peek into Roslyn code and tests from time to time, to see how C# and VB.NET language features are implemented.

We’re hoping there would be a choice to use ReSharper with or without Roslyn-based features, just like currently there’s a choice to disable VS native refactorings and IntelliSense or keep them enabled. A different story is the performance impact caused by concurrent usage of the two. We’ll see how significant it would be in practice as soon as VS vNext emerges.

Although I understand your reasoning it makes me disappointed. ReSharper performance has been an issue for years and despite your claims of improved performance it still takes 10+ minutes for ReSharper to do a solution wide analysis of the project I’m working on. Roslyn completes in a fraction of the time.. and in the background.. also something ReSharper could do more of, don’t lock the UI thread.

It wouldn’t be such an issue if ReSharper didn’t also frequently forget what it knows and restarts analyzing from scratch. Far too often I find it faster to rename classes or properties by hand than let ReSharper do it, again Roslyn does the same task much faster.

So at least make some features optional, I know I’d much rather rename properties with Roslyn than with ReSharper.

1. Roslyn has NO solution-wide analysis. This feature is unique for ReSharper. We are tracking dependencies between files and incrementally re-analyzing them to provide features with knowledge of all the solution, all the types defined, all the members used. If you don’t like SWA and it’s performance impact, you can simply turn it off (for example, I’m working with it’s disabled).

2. ‘Roslyn completes in a fraction of the time’ – I guess, what you talking about is indexes/caches build time. This is required step for any IDE tooling, it doesn’t involve any kind of code analysis. Even in a super-huge solution like R# solution itself (300 projects), cold caches build (solution was never opened before) takes no longer than 40-60 seconds (you may see this as ‘Processing files’ in R# status bar). This is really fast comparing with amount of work and we always working to improve this.

3. All our inspection are running out of UI thread.

4. Roslyn is faster in rename refactoring by now because it can’t rename member references from XAML, from ASPX, from Razor. It can’t rename related members (parameter names from base/derived class members) of all kinds (like WPF dependency properties represented as multiple members).

We also have plans to make our rename faster or at least give you the ability to control the amount of “smartness” you’re expecting from refactoring.

1. True. I do like SWA though, the value it provides i great. That’s why I still keep it enabled despite all it’s problems, but it’s getting increasingly worse, like false positives so I have to shut down VS, clear the cache and rebuild from scratch (which takes 10 minutes) and like I said it frequently restarts from scratch by iteself. I love it, but it’s a pain.

2. I’m talking about the time it takes to get warm enough to perform some kind of refactoring.

3. Something, or several things, in ReSharper frequently locks the UI though so VS feels laggy and non responsive. All such problems completely disappear when I suspend R#.

I also have been curious since the announcement, but I had already surmised that it would be remiss of JetBrains to attempt an excising and refactoring project of this magnitude for a decade-long development project such as Resharper where none is needed and frankly is supperior in some respects to Roslyn.

I have used Resharper since version 4.0 and I plan to keep on using it. It’s been my best development tool for many years and I don’t plan on stopping now, regardless if it uses Roslyn or not.

Just as others have pointed out, I too have performance issues with Resharper that still plague me and disabling it does seem to improve Visual Studio performance in general and I do on occasion when it’s particularly bad. However, I’m willing to take the performance hit as needed because of the productivity gain. Although in light of the admonition of the 300+ project solution, on which I’m sure Resharper is used, I find my pitiful 100+ project solution that bogs down to be highly suspect and that perhaps it’s other issues, such as virus scanners or other services running that are out of my control, that ultimately affect performance, not Resharper.

In any case, thank you JetBrains for all your hard work and dedication! I’m a very satisfied customer.

We are indeed using ReSharper when developing ReSharper, which allows us to quickly solve the first wave of issues, performance-wise or stability-wise.

However given the enormity of environments that ReSharper operates in (which are shaped by Visual Studio versions including updates, operating systems, hardware configurations, .NET framework versions, external libraries, other VS plugins and extensions, as well as countless variations of size and structure of end-user solutions), we can’t ensure optimal performance in all possible combinations upfront.

That’s why we always ask users who experience severe performance issues to profile Visual Studio and provide us with snapshots. In your particular case we’d also like to take a look at a snapshot of VS with your solution while it’s working slow. Whenever the performance problem occurs and/or intensifies, please consider taking a snapshot by going to ReSharper > Help > Profile Visual Studio and sending it our way for investigation.

If we find that ReSharper is guilty, we’ll make optimizations in the next release. If we find that it’s something else that contributes to the lag the most, we’ll at least let you know what’s going on.

Meanwhile, you might want to spend ~half an hour fine-tuning your environment in ways that could make VS and ReSharper work faster. For the list of things to check, you can refer to this guide.

Most of people was hoping that Resharper could finally been faster with Roslyn. I use ReSharper for many years, and the performance impact is huge. I tried on different machines, sent profile files to ReSharper team, etc.
Its a consensus that ReSharper slows down VS

So… you think we should drop support all the languages except C# and VB.NET, take a few years to rewrite our features to do AST transformations in a immutable way, write missing parts (currently Roslyn is less than 5% of R# by the number of LOC), drop our caches to do fast find usages (just compare it with Roslyn’s on a huge solution) and fast inheritors lookup (without iterating all of the files in solution), completely drop solution-wide analysis… and this will make as faster and a greater product?

Or you think it is possible to abstract features over immutable AST in a just two languages and a mutable one in any other? What for? Just to share syntax trees? Parsing is really really cheap comparing to semantic resolution (ask Roslyn folks).

Also, don’t underestimate perceived performance. You may have the worlds fastest parser or code analyzer but it will still feel slow if something else keeps locking the UI. R# really is the biggest performance thief I still have installed.

When you say “custom C# features”, do you mean language features implemented in unofficial forks of the compiler? If this is what you mean, then unfortunately we’re not planning to do that. However we’re surely going to support all new language features implemented in Microsoft-maintained repository.

Microsoft still owns C# and VB language specifications, and has ultimate control over the language design process (although the community input, particularly, through discussions on CodePlex, has a significant impact on it too).

But the actual implementation is up to the contributors of a particular project (such as Mono or Roslyn), and Microsoft has no responsibility for fidelity, correctness or intellectual property used in those projects. Still, we (test team) at Microsoft take much effort to test the open-source Roslyn project against the language specifications (and drafts) and file issues for all bugs and deviations found.

As I knew the answer to the question “will R# use roslyn” because I asked it in another thread already.. I’ll reiterate in different words: you are going down a lonely road. It’s doubtfull you will be able to shutdown Roslyn, even more doubtfull your customers will want you to, seeing some of the core VS experiences like edit-and-continue will rely on it.

This puts you in a bad spot, because thanks to Roslyn simple R#-like features will crop up through community extensions, and then it will start to be a decision between “use extension X Y Z with minimal perf impact” and “use the whole featureset of R# with big perf and loadtime hit”, and over time that decision will get more and more biased towards the community extensions, as their featureset will grow. Of course that’s part of the reason for your work on the SDK, and on perf improvements lately, I understand that, you saw Roslyn coming for a long time. That also means the decision was made a long time ago. Still… unfortunate.
We need a more nimble, modular, faster R#. If you invest the time to improve loadtimes, make R# load async, etc.. you could as well switch to Roslyn. Just bite the bullet.. please. 😉

1. “It’s doubtfull you will be able to shutdown Roslyn…” – maybe this is not clear from the post, but are not going to ‘shutdown Roslyn’. Yep, core VS features will rely on Roslyn – compilation, edit & continue, debugging, any other VS feature that uses C#/VB language services. What we want is the ability to replace features we can do better/faster/cross-language/etc, just like we doing with the VS features by now, overriding some menus, replacing code completion, etc.

2. We always, always investing in R# performance. We improving our caching every release (we moving our caches to leveldb), we offloading cold data from memory, we reducing memory traffic and GC pressure, trying to use new faster algorithms and data structures, inspecting JIT time (we use CLR4.5 multicore JIT to reduce startup time), dropping closures/de-functionalizing our code in a hot paths and a LOT more! We are trying to be async as we can, but a lot of communication with VS APIs should be done exclusively on the UI thread and some of APIs may be really slow (project model).

3. Yes, modularity is very important. I think we can greatly improve this part in the future

Good thread. I would agree that it would be helpful to have a R# Lite. I was having bad performance problems in VS13 recently and ruled out R# as a culprit because I suspended it and the issue remained. A colleague informed me that suspending R# was not enough. It had to be uninstalled. I uninstalled and VS was speedy again.
I love most of the features of R#, but there are only a handful that are live or die for me and they’re little things like Expand or Shrink Selection, Introduce Variable, etc. If community contributions on Roslyn start filling those holes, I’ll have a harder time justifying the R# performance impact especially given the fact that I don’t much use some of the more perf expensive features.
Thanks for the great product and the great blog post.

I have been using R# for more than a decade now, and I have to admit I’m disappointed. I do realize that Roslyn is 5% of R#, I do realize that it is nearly impossible to make the kind of business decision.

But I also feel like I’m using only 5% of R# too. I feel (and it is just a feeling right now) that in a few years community is going to write 5% of features of R# that I use 95% of the time and it is not going to be that hard of a choice. I’m afraid it’s not going to be in favor of R#.

To be clear, I’ve never used CodeRush, but from what I hear, its feature set and performance is on par with ReSharper, if not superior in certain cases. Except apparently they are willing to bite the bullet and give their users the opportunity to benefit from Roslyn.

All the luck in the world to them, first of all.
Then, they do provide a limited set of features that is superior to ReSharper (their templating engine for example is more flexible) but in terms of analyzing code, I’m afraid they’re behind. That said, they might benefit from Roslyn code analysis per se.

Pretty risky, huh? How everything is working now in VS? Current unmanaged C# language service is more memory hungry than Roslyn: 150-200Mb difference on R# solution 😉

Future evolution for free? It’s a myth Yep, Roslyn will bring new C#/VB features parsing support and semantic resolution first, but this is just a small part of a language feature ‘support’. Roslyn doesn’t solve the need of reviewing and updating tons of context actions/quick fixes, all our refactorings and other features unique for R#.

I don’t doubt that ReSharper’s code analysis is truly top notch under the hood, but isn’t there a lot of performance related issues simply related to integrating with Visual Studio? After all, there’s a lot of old fashion stuff involved, like COM interop and whatnot, and you can’t have perfect control over the IDE environment. Even a clean install of Visual Studio is far from snappy compared to other modern IDEs. And that awfully slow VS install/update process… But I digress.

I’m thinking JetBrains would be exceptionally well-positioned for creating an alternative IDE to Visual Studio, based on the successful IDEA-technology. If I could buy a .NET IDE that provided me with ReSharper’s analysis and refactorings, some WebStorm-like web development support, and all the other great stuff that IDEA-based IDEs bring along… well, I’d buy it in an instant. It could blow Visual Studio out of the water. Please consider it some day.

I have all my fingers crossed that this is what will finally get you guys to do a complete C# IDE – we all know it’s going to be awesome 😉

I too would pick “IDEA for C#” over VStudio (roslyn or not) any day of the week (keeping it on the IDEA platform, written in Java, would be both cool and quite ironic. And it would allow me to develop C# natively on my Mac, Yay!).

I am currently using both AppCode and IDEA on a daily basis and love them, but also do a fair amount of Unity development and am forced to either run Windows/VStudio parallel to OsX, or use MonoDevelop which is a royal pain in the neck.

Anything – and I mean *anything* – you could cook up in the area of a cross-platform C# IDE would be infinitely better than Mono and I would buy it instantly.

You would be very surprised. AppCode on the Mac did that to Apple’s Xcode, and IntelliJ is a much faster/better IDE to develop Java code in than Visual Studio is C# – the reason why Google has chosen it as its official IDE for Android, rebranded as Android Studio.

I think if JetBrains was given a clean slate they would be able to build a much better / faster / less monolithic IDE than Visual Studio has become over its many generations. Just look at the experience JetBrains has building IDE’s across many languages/OSes/stacks. The Visual Studio team doesn’t have nearly the expertise/experience. Sorry to say.

JETBRAINS TEAM:
What is your stance on finally building a cross-platform .NET IDE? With Rosyln open-sourced, and ASP.NET vNext officially opening the doors to Mono (http://www.hanselman.com/blog/IntroducingASPNETVNext.aspx) we could finally write code in an operating system we LOVE (OS X) while using the world’s most modern and advanced language – C#.

Today we have to choose: crappy OS/apps/environment (Windows) VS crappy coding languages (Java, PHP, etc.). I see the open-sourcing of .NET as being the key opportunity for you guys to set out on building a fantastic IDE on BOTH Windows and the Mac to leave the likes of MonoDevelop and Visual Studio in the dust. Without pre-announcing anything you might have going in stealth… what is your stance on this being a trigger for you to set out on this much-needed initiative?

Even though I want them to create IDEA for .NET developers as an option, you’re quite wrong the VS team has quite a lot of experience the issue is not the team but with VS that over the years the internal components never changed much, adding a feature required a lot of infrastructure work and speaking to old components is not fun, not to mention the huge dissonance between VS and the languages themselves and the long development cycle of VS where bug fixes took ages to finally get fixed.

We know what functional data structure is and using some of them in R# codebase. We perfectly know the benefits functional data structures provide: with immutable trees you can share parsing results between incremental parsings (and reduce allocations count), easily have snapshot functionality like you have in Roslyn, easily share trees between concurrent tasks and so on.

But, of course immutable trees have one issue – there is no .Parent node references there. You can write a lot algorighms and features without parent node reference, but having parent references is VERY handy in IDE tools development. This is why you have two immutable trees – “Green” and “Red” one. Green tree is built by parser and have no parent references. Red tree is built on-demand over existing Green tree and provide user with .Parent references.

Since most of Roslyn diagnostic functionality traverses exactly Red tree, we expecting extra memory traffic happens because of Red tree on-demand allocation. We will be happy if this will not became the issue in practice, when a lot of diagnostic analyzers will be implemented.

Talking about practice, while Roslyn still is not able to compile ReSharper solution because of bugs, it takes about ~10-12 minutes to build indexes at cold VS start (~1 minute for R#). Heap usage looks very unpredictable while index is building – it jumps from 0.9Gb to >2.0Gb between GCs every 10-20 seconds. Also index build process seems to be single-threaded (why so?).

Regarding the practice.
Solution with 320 projects. VS CTP6.
Just typed for 20 seconds in a 200-lines of code file. Etw shows there were allocated about 820 MB.
820.
Megabytes.
Although UI thread is pretty free, as a user I constantly feel this lagging.

Please improve/extend tools like “search with pattern”, and the R# SDK in order to offer a powerful alternative to Roslyn to the ones that wish to develop third-party analysis tools and code-quality-tools (dependency checker, code-design rule checkers). R# is the leader in matter of .Net code analysis, but lack of low-level customizations.

I am pretty sure that this could all change 10 or 15 times before Visual Studio 2014 actually lands in RTM, though. We should all know by now that the programming world changes every few minutes – anything can happen.

No, you won’t lose this feature. ReSharper is a design-time tool for source code (mostly), we do not integrate into any of VS debugging capabilities. So as usual, compiling your code, debugging, intellitrace and edit & continue goes through old C# or new Roslyn compilers, R# has nothing to do with it.

I’ve been a long time advocate for tools like Resharper; it’s saved me and the teams I’ve worked with countless hours performing code analysis, refactoring, and other daily challenges. Even menial stuff like discovering assemblies and making the appropriate references, is indispensable, IMO.

However, the short answer of “no” is disappointing to say the least. The times are changing, whether we like it or not, the OSS march is in full swing. Has been for well over a year now, in relevant C# .NET stacks. That’s not changing any time soon that I can surmise. The fact of life is that it is tough to keep pace when the sands are shifting so quickly, but that’s just the fact of life we must all deal with.

Hopefully you change your minds on this and make the necessary changes to keep pace.

I have been using CodeRush and ReSharper together, in my day to day work, for many years.

I stopped to pay my CodeRush licence last year, the reason is its performance is not getting better, and there isn’t much new feature coming along.

However, I recently find ReSharper’s performance for large solution is getting worse, especially the poor performance on Code Analysis. I have to disable solution-wide code analysis, and even file-level analysis sometime. The intellisence’s performance is not that good neither.