As with most things, I'm sure this concept has been tried before - I just haven't come across editors that use what I've termed 'Virtual Formatting'. The principle is that there's a floating left-margin that simulates the effect of the padding space/tab characters that are conventionally inserted by the developer or the editor itself to format the code. The editor continously parses code (even when commented-out) as you type and calculates the required indent based on the context where each line-feed is found

I'm developing this idea working specifically with an XML editor as XML has some peculiar problems with formatting characters and it tends to be heavily nested, however I believe many of the principles still hold for conventional code.

Have you experience coding with such a tool or do you have a view on whether it would help or hinder? Would it cause problems with version-control systems? (it detects and strips out all existing padding characters)

Unless you've tried it, the behavior of such a tool is hard to describe, it looks conventional until you actually start editing. I've put up a screencast video showing a prototype in action which demonstrates editing XML, changing it's hierarchy and doing drag/drop and copy and paste operations, and then how formatting is broken/fixed when invalid characters are typed.

Edit
All answers/comments have so far been negative - so to attempt to redress balance, some benefits of virtual-formatting to think about:

No more debates on formatting standards, just place line-feeds where that conforms to your chosen/mandated convention

Where space is at a premium (in a book/blog/documentation) you can word-wrap but still getting perfect indentation

Each code-block can have a 'mouse-handle' immediately adjacent to where it starts, not squeezed into the screen edge - click this to select the whole block or inner-block

Drag, drop and forget - becomes viable for the first time

No time spent reformatting other peoples code

No incorrectly formatted code (in the sense that there is none - just the rendering)

Using Backspace instead of Ctrl+Backspace keeps your fingers on the keyboard guide-keys

Consider that there are roughly 25% fewer editable characters (in a sample XSLT), doesn't that have efficiency benefits?

Edit - Conclusions so far

Developers have established tools and working methods that efficiently overcome most of the disadvantages inherent in the use of padding characters used for indentation.

There is concern that removal of formatting characters will detrimentally affect some differencing tools.

Developers want the flexibility to 'fine-tune' formatting in such a way that automated rendering could not handle.

The removal of leading spaces/tabs means that a 'code-aware' tool capable of code-formatting is needed to review such code efficiently - a plain-text editor would show no formatting.

Those that feel there may be some hypothetical benefits (to virtual-indentation), have a view that the disadvantages outweigh those potential benefits - conclusively.

Edit - Verdict

The perception of the obstacles and few (if any) benefits is such that it would be unwise for me, as a sole-developer, to pursue this space-free editing concept for general languages. For XML/XSLT, however (because of its special treatment of whitespace), there seems to be some agreement of potential at least.

Edit - Product Shipped

In spite of the generally negative sentiment found here, I went ahead and shipped the editor. I made a free version in the hope it would bring criticism in the form of more concrete issues, based on real experience. Somewhat frustratingly, there have been no complaints so far (in fact barely any feedback considering download volume). I'd like to think this was because users adjusted to the idea so well that they see this as a 'so what?' kind of feature - but there's no way of telling...

i think some Scheme editors do this kind of formatting on the fly, but inserting (and deleting) true spaces
–
JavierJun 6 '11 at 14:05

@Javier I must confess I haven't come across Scheme - I will look this up, my view is that there are some languages where formatting is more critical/hazardous than others, so Scheme is possibly one of those
–
pgfearoJun 6 '11 at 14:16

I suggest you see Firebug and how it edits HTML tree.
–
Lie RyanJun 6 '11 at 16:30

@Lie Ryan - Yes, I like the tree editor, so far as it goes. But it still feels like you're filling in a form rather than free-flow text, I guess that's because it only edits attributes, not elements (in this mode, so far as I can tell).
–
pgfearoJun 6 '11 at 16:53

HTML isn't exactly a freeflow text to begin with, but I see what you mean by feeling like filling forms (in fact that's why I like it, the restrictions make it impossible to inadvertently introduce unclosed/mismatched tags, unquoted attributes, etc; I wouldn't use a dedicated XML editor if it allowed me to write non-well-formed XML). In Firebug you can add new node by right click > edit HTML, quite inconvenient but sufficient for minor editing that web developers needed. In a more serious editor though, it's possible to have button/context menu/shortcut for inserting nodes.
–
Lie RyanJun 6 '11 at 17:19

8 Answers
8

The biggest problem would be how files would appear to other tools, most particularly version control tools. Line endings are significant to these tools. I would not want to see a merge screen where I have an entire class in one line and try and merge some part of the text at column 347.

+1 - I can't see any benefit to this, all it is going to do is mess up version control and diff tools.
–
user23157Jun 6 '11 at 14:35

4

@pgfearo The problem I have with something like this is that it would be almost great, but the <1% of changes I don't want to make that the editor goes ahead and changes anyway would be maddening. Your editor would have to be completely customizable to the nth degree to ensure that it meets everyone's needs.
–
Michael ToddJun 6 '11 at 14:58

1

@pgfearo - xml formatting, yes, this is definitely the approach to take, but I thought that was how most editors formatted xml anyway.
–
user23157Jun 6 '11 at 16:20

Why does the indentation have to be saved, should we be imposing our formatting on others? Do people still use tools that don't have XML plug-ins available, that can format the XML to their needs (not those of the author)?
–
pgfearoJun 6 '11 at 17:05

On Emacs. I've only used Emacs long enough to know that I didn't have the time (then) to learn it properly to do it justice. I suspect Emacs users would equally be hard pushed to try the 'other way'. I will look at Emacs again, but I worry about the 'mostly gets it right' - you can improve on that.
–
pgfearoJun 6 '11 at 17:18

I've seen this idea before, but I've never seen it actually work out. Most of the time, when I've seen the idea come up, it's been shot down by every developer out there - or when it has been implemented, it made version control virtually useless as merely viewing folded code would change the file and further confuse the diff tool. (An example of just how bad this is is the Witango Development Studio.)

I can think of several hurdles your editor would have to overcome in order to be useful to many developers:

Changes to a file must not create noise in the *nix diff -uw command. (No spurious diffs!)

Editing a (previously unformatted) file must not complicate merges in SCM tools.

It must play nicely with other editors used by the dev team.

Users must be able to customize formatting to their hearts' content - some people are quite passionate about how their code appears.

Edits to existing code must not change the existing formatting. It's crucial that the editor doesn't suddenly reformat a whole file for no apparent reason and doesn't change the formatting of a line for simple minor edits - that'll confuse diff tools and anger other team members.

New lines added to a file should conform to existing modelines' settings, or should conform to the local (within +/-3 lines) existing formatting.

The editor should have a settings panel that allows the user to define the team's code formatting policy that is separate from what the editor actually shows.

Needless to say, it's obvious that this will require some amount of metadata about the file's existing formatting. You'd likely want to keep it separate from the sources, but keeping it in sync with an ever-changing repository would probably be quite difficult.

As a Vim user, I'd also opine that the editor should respect Vim, Emacs, and other editors' modelines and preserve the modeline's proscribed formatting, regardless of what it ultimately displays.

In my opinion, the requirements for such software make it untenable. Too many users will expect too much of it too often to make such a project successful.

So it looks like we're stuck with our current solution, not because its the best, but because it's too hard to change. Sadly, this is true in so many areas of IT that it would be foolish/arrogant for me to feel I could fix this on my own. As I'm kind of obstinate I will still use this concept integrated as an optional tool in a much larger desktop solution where processing/analysis is the main concern. That way the developer has a choice.
–
pgfearoJun 6 '11 at 18:22

@pgfearo: No, I think we're "stuck" with our current situation, not because it's best, but because there isn't anything better. To win over Vim and Emacs users to the idea, you'll need to provide a substantial benefit over the already efficient tools they use. To win over everyone else, you'll need only to provide the features that Vim and Emacs users already enjoy to a wider audience. Wait, maybe that TextMate license is worth it.
–
greyfadeJun 6 '11 at 18:26

@pgfearo: My overall point is that I don't think you'll ever achieve something that anyone will want to use over their existing tools, except for a very small subset of the work we do. There's an obvious benefit to XML and XSLT hackers, since structure is 99% of the work, but there's far fewer obvious benefits to less strictly structured languages like C, Java, C#, Ruby, etc. This is a very hard sell to the hard-core.
–
greyfadeJun 6 '11 at 18:29

Yes, this point about efficient existing tools/methods is #1 in the edited (interim) conclusions to the question. From what's been said already, I doubt I should even attempt to win over Vim/Emacs users - if there is ever interest in this it will come from elsewhere.
–
pgfearoJun 6 '11 at 18:39

Accepted this answer because it breaks down the issues succinctly and is inline with the general sentiment of other answer/comments. Not the answer I was hoping for, but it helped persuade me, that - at least for non XML/XSLT environments - there's no point in pursuing this concept further, no need to waste further effort.
–
pgfearoJun 7 '11 at 13:23

They can freely change their formatting style without breaking anything.

Auto-formatting can be turned off to display and edit files raw.

Have a minimal team formatter configured for the team standard.

Store this formatter in team version control.

Files are saved with team formatting as a fallback for other tools.

Diffs don't suffer from formatting problems.

It's ridiculous to worry about the economy of saving files without extraneous whitespace, so you don't actually lose anything by saving the files with some formatting defined by a team standard. In the case of a solo developer, you could offer the ability to have saved formatting (for the "team" of one) mirror displayed formatting.

Useful points. I like the idea of having shared formatter configuration files. Auto-formatting is already switchable in the prototype - because its essential to isolate 'real' whitespace from the virtual stuff. Switching between views of course doesn't alter a single character. My inclination is not to provide option within the tool to save 'with formatting' but instead to provide 'adapters' that are free, cross-platform and can also run as a web-service, these could use the very same XML formatter config files you suggest.
–
pgfearoJun 6 '11 at 21:20

I'd be okay with code being autoformatted if the language wasn't format sensitive cough Python cough.

Emacs does Lisp formatting real nice, and it formatted on the fly I'd be pretty happy. I can't stand automatic insertion of parens or other delimiters: none of the autoinserters I've tried get it close to right for how I type.

Yes, I think F# also uses formatting within its syntax. Auto-insertion of delimiters is an issue. For XML this takes the form of adding quote marks etc. It isn't perfect but it compensates for touch-typists who will possibly be ahead of the race, so it checks that something that's typed hasn't already been queued for auto-insertion anyway - avoiding repetition. Without auto-insertion, the parser is left guessing how to indent - it can delay for a bit, but there would be some 'jiggling' of the margin as the parser tries to comprehend the true intent of the code being typed.
–
pgfearoJun 6 '11 at 21:33

I heart the idea in concept. And while you may succeed, I've yet to find an editor that sufficiently does everything I want it to do in terms of formatting. Here's a few roadblocks (IMO):

Whitespace-picky languages

What gets saved to output? (You would, to some degree, have to save at least some of the indenting so that it remains readable to others without the tool)

It must absolutely play nice with the existing source/versioning control products already in place

It must be highly customizable, especially for those of us (me included) that are very, very, very picky about how code gets formatted and indented (and where that comma should go in a list, but if the list is really long, how lines should be broken, etc.)

It must truly understand the language. By this I mean that it must follow the language exactly as the compiler will understand it. I've had far too many editors with colored syntax that get the syntax wrong because there is some nutty facet of the language that can't simply be handled with a RegExp. Granted, you indicated that you would be continually parsing the language, but who is to say that what you parse conforms to the compiler I'm using? (Believe me, there's a lot of standards out there, and there's a lot of funny-business that goes on in most languages, including code that looks wrong to a simple parser, but is perfectly correct to the compiler.)

It must be fast. And here's where the rubber meets the road -- you can't be doing a full compile run on my code every keystroke (especially with a large project), unless you can optimize your parser to the Nth degree. If there's even a hint of slowness in the editor, no one will use it. (I'm a perfect example of abandoning slow editors in favor of faster ones. If it lags in the least, I'm out.)

Could you create something that meets most of the criteria reasonably well? Probably. But we programmers are finicky and picky and loathe to change and will jump-ship at the first sign of trouble. Here you're trouble is going to be if the output ticks someone else off on the dev team, or if it doesn't play 100% nice with the versioning system, or if it doesn't format my code exactly so or if it misunderstands the language and mis-indents my code, or if it lags a millisecond beyond what I expect. Because, much as I heart the idea, I'll jump ship to my preferred editors in a heartbeat.

On your point on performance: responsiveness can be comparable to existing tools. Because formatting has no effect on the code text, processing can be run in the background and terminated at will. With XSLT, there's a cascade approach, formatting/colorising is performed before fine-grain validation and ultimate compiling. As all this happens on a background thread and is interrupted immediately on the next keystroke there's no perceptible performance lag. Only the visible text needs refreshing immediately, remaining text is reformatted in chunked bursts when there's a pause.
–
pgfearoJun 7 '11 at 8:16

What gets saved to output? No formatting directly, as this would impose a formatting style on others. The idea (in response to previous answers) is to provide pluggable formatters that use a standard config file declaring formatting rules and output accordingly, on demand.
–
pgfearoJun 7 '11 at 8:28

This is interesting. I don't fully understand the MPS approach yet, but the design for this editor uses an AST equivalent to maps to positions in the plain text XML. So whenever there's user interaction with the plain text, the editor knows immediately what part of the 'AST' is being edited. This helps it determine whether reparsing is needed, and also helps protect parts of the tree from inadvertent deletion, when the user performs 'press and hold' deletion, for example. This is also used to present data on the current selection in real time. I will look at MPS further.
–
pgfearoSep 15 '11 at 21:36

Now that I've published the XML editor described in my question, I'm in a better (but not ideal) position to attempt to answer this myself - so I will:

After well over 500 downloads in less than 2 weeks, the reaction to this ground-breaking new concept in dynamic code formatting has been...

NIL

No complaints, no praise. My (hopeful) interpretation of this is that people just want and expect an editor that feels normal and doesn't mess up their code. Users will barely notice that their formatting is entirely virtual and evidence suggests they care even less.

It would be a mistake though to read too much into this non-reaction, this tool is free, and unfortunately this means at least some users will be less inclined to criticise. If they didn't like it, they may just have discarded it and used something else.