Problem Statement

It is easy to change the values of compile-time vars like *warn-on-reflection* and *unchecked-math* from a particular point in a file until the end of the file with set!. You can also change the value of one of these vars for one top-level form by setting it to a desired value before the top-level form, and then setting its value back to the original after that top-level form.

However, there are cases where it would be desirable to change the value of such a var for an individual expression.

For example, you may want *unchecked-math* to be true for part of a function's definition, either a few individual subexpressions, or a larger part of it.

Another is that you want *warn-on-reflection* to be true for most of your library, but there are a few instances of reflection that you are aware of and either they cannot be eliminated, or they are rare enough in practice that they are not a performance problem. You want to mark the expressions using reflection so that they do not cause a compile-time warning, but you want such warnings enabled everywhere else in that same function in case future changes introduce a new use of reflection.

Future plan that I consider the biggest bang-for-the-buck for this idea: In Clojure core and all modular contrib libraries, make it the default to build with reflection warnings enabled. After type-hinting the ones away that can be easily and safely done so, wrap the rest of the expressions causing the warnings in (known-reflection ...) so they no longer cause warnings, but are clearly marked in the source code as causing reflection. From that point forward, every time a developer adds code that causes a new reflection warning, they will find out on the very next build attempt, and it will likely get fixed or marked with (known-reflection ...) quite soon afterwards. Such changes in Clojure core and contrib would be done in separate commits after this change was in.

Proposal A: new special form compile-time-let

This is likely the less desirable way to do it. You can skip it and read Proposal B if you want to get to the good stuff.

Introduce a new special form in Clojure called compile-time-let that takes a vector of bindings, and makes those bindings active in the textual scope of the compile-time-let expression during compile time. For example:

;; I know it is easy to eliminate reflection in this simple example.;; Imagine a case where it was impossible or undesirable to eliminate reflection.

A proof-of-concept implementation of compile-time-let for Clojure on the JVM has been added as an attachment patch1.txt to this page. Select "Attachments" under the Tools menu near the top right of the page to see it.

If you want to change *warn-on-reflection* or *unchecked-math* for an entire file, the current method of adding a line like (set! *warn-on-reflection* true) near the top of the file continues to work, and is the recommended method to do so.

Earlier I believed that for my patch metadata-patch1.txt to work, it would require first applying a patch to fix CLJ-865 (select "Attachments" under the Tools menu near the top right of this page to find metadata-patch1.txt). Further testing has shown that my patch seems to work even without fixing CLJ-865. I think fixing CLJ-865 is a good idea anyway, but it isn't a prerequisite for this feature as implemented by the current patch.

If you want to change *warn-on-reflection* or *unchecked-math* for an entire file, the current method of adding a line like (set! *warn-on-reflection* true) near the top of the file continues to work, and is the recommended method to do so.

Tradeoffs

Proposal B is better in several ways: it takes advantage of metadata annotating Clojure code that already exists in the compiler, and extends its use a little bit. It creates no new special forms. Source code with the new metadata annotations would compile without errors using older versions of Clojure, although the older Clojure compiler would silently give unintended compile-time behavior.

With Proposal A, new code that used the compile-time-let special form would not compile with older versions of Clojure, unless augmented with a definition of compile-time-let that would not give the desired behavior, but would simply be a "no op" that allows the code to compile.

Open questions

Note: The following open questions could be addressed by a later patch. The existing patches that are restricted to modifying *unchecked-math* and *warn-on-reflection* are useful without the enhancement below.

Question: Should expression-level modification of *compiler-options* also be allowed, e.g. by implementing the proper behavior for keywords :elide-meta and :disable-locals-clearing?

If so, should it be done with a key :compiler-options whose value is a map, like this?

Regardless of the choice above, I believe the proper behavior would be to pushThreadBindings() a new value of the var *compiler-options* whenever metadata like the above is seen, and the new value of *compiler-options* should be the result of merging its current value with the new key/value pairs. For example, if *compiler-options* was currently equal to {:elide-meta [:doc]}, then within the following expression:

^{:disable-locals-clearing true}( ... Clojure code here ... )

the value of *compiler-options* would be {:elide-meta [:doc] :disable-locals-clearing true}.

Background

Discussions on the clojure-dev group:

Thread with subject "Proposal for *warn-on-reflection* to be true by default" begun on Feb 25, 2012:

It seems like avoiding additional special forms is a good idea, especially considering all the work going on with alterate implementations. Why do you propose a new special form over the metadata proposal?

This may be exactly the wrong answer, but for one, I was able to figure out how to implement it relatively quickly. Comments in the thread proposing metadata for this purpose, especially Stuart Sierra's statement "This would be a pretty substantial change to the current compiler, I think" make me wonder whether anyone will exert the effort required to make it happen.

Is reflection the primary use case motivating this? I ask because there might be ways to make the problem go away entirely. E.g., have the compiler walk all classes visible to a package and automatically eliminate reflection when there is no ambiguity.

It was what motivated me personally. The current patches allow you to do per-expression changes to *unchecked-math*, too, which might be useful to some people, although I haven't solicited or heard any feedback about how useful people would find that. Although the attached patch doesn't implement it, the idea could be used to do per-subexpression assignments for the other options now in *compiler-options*, i.e. :elide-meta and :disable-locals-clearing, and such a patch should be able to automatically work for any new keys that might be added to *compiler-options* in the future.

In your scenario where the compiler automatically eliminated unambiguous reflection, would the behavior be to always warn about cases of ambiguous reflection, to always be silent about it, or to follow the current value of *warn-on-reflection*? Maybe there would be far fewer cases of ambiguous reflection, but for whatever remains, we are in the same situation as we are today.

Out of curiosity, how have you and Rich been able to notice the new uses of reflection in the June and July 2012 commits to Clojure core? By enabling *warn-on-reflection* by hand in the sections of code with changes? Enabling it globally in the changed files or in all of Clojure core and doing a diff on the before- and after-patch compiler output? Looking for them by eye and brain manually?

A small comment, then I'll try not to ask about it again for at least a couple of months

If:

(1) we did something like this enhancement in Clojure

(2) we then eliminated all reflection warnings in Clojure core, either by type hints, or marking them with (known-reflection ...). There were around 30 to 40 of them last I checked around Feb 2012, and many can easily be type-hinted away.

and

(3) we enabled *warn-on-reflection* by default in all Clojure core builds, e.g. by uncommenting this line in build.xml:

then the prescreening process would automatically detect any proposed patch that introduced new reflection warnings, because it automatically detects any compile time warnings and marks such patches as not ready yet. The only possible uses of reflection that could be introduced by a patch would be called out explicitly in the source code with (known-reflection ...) or ^{:warn-on-reflection false}.

While interesting, I don't consider this a priority, and it's a big feature to support a small benefit (warn-on-reflection true by default). There may be other ways to achieve that, and it might be practically untenable anyway due to the effect on existing codebases.