Clojure JIRAhttp://dev.clojure.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+CLJ+AND+status+%3D+Reopened+ORDER+BY+priority+DESC
An XML representation of a search requesten-us4.464925-07-2011[CLJ-1544] AOT bug involving namespaces loaded before AOT compilation startedhttp://dev.clojure.org/jira/browse/CLJ-1544
Clojure<p>If namespace "a" that is being AOT compiled requires a namespace "b" that has been loaded but not AOT compiled, the classfile for that namespace will never be emitted on disk, causing errors when compiling uberjars or in other cases.</p>
<p>A minimal reproducible case is described in the following comment: <a href="http://dev.clojure.org/jira/browse/CLJ-1544?focusedCommentId=36734&amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-36734">http://dev.clojure.org/jira/browse/CLJ-1544?focusedCommentId=36734&amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-36734</a></p>
<p>Other examples of the bug:<br/>
<a href="https://github.com/arohner/clj-aot-repro">https://github.com/arohner/clj-aot-repro</a><br/>
<a href="https://github.com/methylene/class-not-found">https://github.com/methylene/class-not-found</a></p>
<p>A real issue triggered by this bug: <a href="https://github.com/cemerick/austin/issues/23">https://github.com/cemerick/austin/issues/23</a></p>
<p>Related ticket: <a href="http://dev.clojure.org/jira/browse/CLJ-1641" title="AOT compilation fails for a kind of circular dependency after CLJ-1544 patch"><del>CLJ-1641</del></a> contains descriptions and comments about some potentially unwanted consequences of applying proposed patch 0001-<a href="http://dev.clojure.org/jira/browse/CLJ-1544" title="AOT bug involving namespaces loaded before AOT compilation started">CLJ-1544</a>-force-reloading-of-namespaces-during-AOT-co-v3.patch</p>
<p><b>Approach:</b> The approach taken by the attached patch is to force reloading of namespaces during AOT compilation if no matching classfile is found in the compile-path or in the classpath</p>
<p><b>Patch:</b> 0001-<a href="http://dev.clojure.org/jira/browse/CLJ-1544" title="AOT bug involving namespaces loaded before AOT compilation started">CLJ-1544</a>-force-reloading-of-namespaces-during-AOT-co-v3.patch</p>
<p><b>Screened by:</b> Alex Miller</p>CLJ-1544AOT bug involving namespaces loaded before AOT compilation startedDefectCriticalReopenedUnresolvedUnassignedAllen RohneraotWed, 1 Oct 2014 13:24:36 -0500Fri, 20 Feb 2015 10:11:38 -0600Release 1.878<p>Possibly related: <a href="http://dev.clojure.org/jira/browse/CLJ-1457" title="once the compiler pops the dynamic classloader from the stack, attempts to read record reader literals will fail"><del>CLJ-1457</del></a></p><p>Has anyone been able to reproduce this bug from a bare clojure repl? I have been trying to take lein out of the equation for an hour but I don't seem to be able to reproduce it &#8211; this makes me think that it's possible that this is a lein/classlojure/nrepl issue rather than a compiler/classloader bug</p><p>I was actually able to reproduce and understand this bug thanks to a minimal example reduced from a testcase for <a href="http://dev.clojure.org/jira/browse/CLJ-1413" title="A problem involving user.clj and defrecord"><del>CLJ-1413</del></a>.</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">&gt;cat error.sh
#!/bin/sh
rm -rf target &amp;&amp; mkdir target
java -cp src:clojure.jar clojure.main - &lt;&lt;EOF
(require 'myrecord)
(set! *compile-path* <span class="code-quote">"target"</span>)
(compile 'core)
EOF
java -cp target:clojure.jar clojure.main -e <span class="code-quote">"(use 'core)"</span>
&gt; cat src/core.clj
(in-ns 'core)
(clojure.core/require 'myrecord)
(clojure.core/<span class="code-keyword">import</span> myrecord.somerecord)
&gt;cat src/myrecord.clj
(in-ns 'myrecord)
(clojure.core/defrecord somerecord [])
&gt; ./error.sh
Exception in thread <span class="code-quote">"main"</span> java.lang.ExceptionInInitializerError
at java.lang.<span class="code-object">Class</span>.forName0(Native Method)
at java.lang.<span class="code-object">Class</span>.forName(<span class="code-object">Class</span>.java:344)
at clojure.lang.RT.classForName(RT.java:2113)
at clojure.lang.RT.classForName(RT.java:2122)
at clojure.lang.RT.loadClassForName(RT.java:2141)
at clojure.lang.RT.load(RT.java:430)
at clojure.lang.RT.load(RT.java:411)
at clojure.core$load$fn__5403.invoke(core.clj:5808)
at clojure.core$load.doInvoke(core.clj:5807)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invoke(core.clj:5613)
at clojure.core$load_lib$fn__5352.invoke(core.clj:5653)
at clojure.core$load_lib.doInvoke(core.clj:5652)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invoke(core.clj:628)
at clojure.core$load_libs.doInvoke(core.clj:5691)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invoke(core.clj:630)
at clojure.core$use.doInvoke(core.clj:5785)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at user$eval212.invoke(NO_SOURCE_FILE:1)
at clojure.lang.<span class="code-object">Compiler</span>.eval(<span class="code-object">Compiler</span>.java:6767)
at clojure.lang.<span class="code-object">Compiler</span>.eval(<span class="code-object">Compiler</span>.java:6730)
at clojure.core$eval.invoke(core.clj:3076)
at clojure.main$eval_opt.invoke(main.clj:288)
at clojure.main$initialize.invoke(main.clj:307)
at clojure.main$null_opt.invoke(main.clj:342)
at clojure.main$main.doInvoke(main.clj:420)
at clojure.lang.RestFn.invoke(RestFn.java:421)
at clojure.lang.Var.invoke(Var.java:383)
at clojure.lang.AFn.applyToHelper(AFn.java:156)
at clojure.lang.Var.applyTo(Var.java:700)
at clojure.main.main(main.java:37)
Caused by: java.io.FileNotFoundException: Could not locate myrecord__init.class or myrecord.clj on classpath.
at clojure.lang.RT.load(RT.java:443)
at clojure.lang.RT.load(RT.java:411)
at clojure.core$load$fn__5403.invoke(core.clj:5808)
at clojure.core$load.doInvoke(core.clj:5807)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invoke(core.clj:5613)
at clojure.core$load_lib$fn__5352.invoke(core.clj:5653)
at clojure.core$load_lib.doInvoke(core.clj:5652)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invoke(core.clj:628)
at clojure.core$load_libs.doInvoke(core.clj:5691)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invoke(core.clj:628)
at clojure.core$require.doInvoke(core.clj:5774)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at core__init.load(Unknown Source)
at core__init.&lt;clinit&gt;(Unknown Source)
... 33 more</pre>
</div></div>
<p>This bug also has also affected Austin: <a href="https://github.com/cemerick/austin/issues/23">https://github.com/cemerick/austin/issues/23</a></p>
<p>Essentially this bug manifests itself when a namespace defining a protocol or a type/record has been JIT loaded and a namespace that needs the protocol/type/record class is being AOT compiled later. Since the namespace defining the class has already been loaded the class is never emitted on disk.</p><p>I've attached a tentative patch fixing the issue in the only way I found reasonable: forcing the reloading of namespaces during AOT compilation if the compiled classfile is not found in the compile-path or in the classpath</p><p>Updated patch forces reloading of the namespace even if a classfile exists in the compile-path but the source file is newer, mimicking the logic of clojure.lang.RT/load</p><p>Further testing demonstrated that this bug is not only scoped to deftypes/defprotocols but can manifest itself in the general case of a namespace "a" requiring a namespace "b" already loaded, and AOT compiling the namespace "a"</p><p>I'm also affected by this bug. Is there some workaround I can apply in the meantime, e.g., by dictating the order in which namespaces are going to be loaded/compiled in project.clj?</p><p>Tassilo, if you don't have control over whether or not a namespace that an AOT namespace depends on has already been loaded before compilation starts, requiring those namespaces with :reload-all should be enough to work around this issue</p><p>Nicola, thanks! But in the meantime I've switched to using clojure.java.api and omit AOT-compilation. That works just fine, too.</p><p>Tassilo, that's often a good solution, another is to use a shim clojure class</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">(ns myproject.main-shim (:gen-class))
(defn -main [&amp; args]
(require 'myproject.main)
((resolve 'myproject.main) args))</pre>
</div></div>
<p>then your shim namespace is AOT-compiled but nothing else in your project is.</p><p>Thanks Michael, that's a very good suggestion. In fact, I've always used AOT only as a means to export some functions to Java-land. Basically, I did as you suggest but required the to-be-exported fn's namespace in the ns-form which then causes AOT-compilation of that namespace and its own deps recursively. So your approach seems to be as convenient from the Java side (no need to clojure.java.require `require` in order to require the namespace with the fn I wanna call <img class="emoticon" src="http://dev.clojure.org/jira/images/icons/emoticons/wink.gif" height="20" width="20" align="absmiddle" alt="" border="0"/>) while still omitting AOT. Awesome!</p><p>I'm marking this as incomplete to prevent further screening until the bug reported here: <a href="http://dev.clojure.org/jira/browse/CLJ-1620?focusedCommentId=37232&amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-37232">http://dev.clojure.org/jira/browse/CLJ-1620?focusedCommentId=37232&amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-37232</a> is figured out</p><p>Fixed the patch, I'm re marking the tickets as Vetted as it was before.</p><p>This patch is being rolled back for 1.7.0-alpha6 pending further investigation into underlying problems and possible solutions.</p><p>I'm not 100% sure, but this looks a lot like <a href="https://github.com/cursiveclojure/cursive/issues/369">Cursive issue 369</a>. It had a case that I could reproduce with JDK 7 but not JDK 8, has the same mysterious missing namespace class symptom, and involves mixed AOT/non-AOT namespaces. However it's happening at runtime, not at compile time, which doesn't seem consistent.</p><p>My error report above was incorrectly tied to this issue (see <a href="http://dev.clojure.org/jira/browse/CLJ-1636" title="SeqIterator can return incorrect results "><del>CLJ-1636</del></a>). I will delete the comment.</p><p>Since ticket <a href="http://dev.clojure.org/jira/browse/CLJ-1641" title="AOT compilation fails for a kind of circular dependency after CLJ-1544 patch"><del>CLJ-1641</del></a> has been closed, I'll repost here a comment I posted in that ticket + the patch I proposed, arguing why I think the patch I proposed for this ticket should not have been reverted:</p>
<p>Zach, I agree that having different behaviour between AOT and JIT is wrong.</p>
<p>But I also don't agree that having clojure error out on circular dependencies should be considered a bug, I would argue that the way manifold used to implement the circular dependency between manifold.stream and manifold.stream.graph was a just a hack around lack of validation in require.</p>
<p>My proposal to fix this disparity between AOT and JIT is by making require/use check for circular dependencies before checking for already-loaded namespaces.</p>
<p>This way, both under JIT and AOT code like</p>
<p>(ns foo.a (:require foo.b))<br/>
(ns foo.b)<br/>
(require 'foo.a)</p>
<p>will fail with a circular depdenency error.</p>
<p>This is what the patch I just attached (0001-<a href="http://dev.clojure.org/jira/browse/CLJ-1641" title="AOT compilation fails for a kind of circular dependency after CLJ-1544 patch"><del>CLJ-1641</del></a><del>disallow-circular-dependencies-even-if-the</del>.patch) does.</p>ApprovalIncompleteGlobal RankPatchCode[CLJ-1093] Empty PersistentCollections get incorrectly evaluated as their generic clojure counterparthttp://dev.clojure.org/jira/browse/CLJ-1093
Clojure<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">user&gt; (defrecord x [])
user.x
user&gt; #user.x[] ;; expect: #user.x{}
{}
user&gt; #user.x{} ;; expect: #user.x{}
{}
user&gt; #clojure.lang.PersistentTreeMap[]
{}
user&gt; (class *1) ;; expect: clojure.lang.PersistentTreeMap
clojure.lang.PersistentArrayMap</pre>
</div></div>
<p><b>Cause:</b> Compiler's ConstantExpr parser returns an EmptyExpr for all empty persistent collections, even if they are of types other than the core collections (for example: records, sorted collections, custom collections). EmptyExpr reports its java class as one the classes - IPersistentList/IPersistentVector/IPersistentMap/IPersistentSet rather than the original type.</p>
<p><b>Proposed:</b> If one of the Persistent* classes, then create EmptyExpr as before, otherwise retain the ConstantExpression of the original collection. <br/>
Since EmptyExpr is a compiler optimization that applies only to some concrete clojure collections, making EmptyExpr dispatch on concrete types rather than on generic interfaces makes the compiler behave as expected</p>
<p><b>Patch:</b> 0001-<a href="http://dev.clojure.org/jira/browse/CLJ-1093" title="Empty PersistentCollections get incorrectly evaluated as their generic clojure counterpart">CLJ-1093</a>-v2.patch</p>
<p><b>Screened by:</b></p>CLJ-1093Empty PersistentCollections get incorrectly evaluated as their generic clojure counterpartDefectMajorReopenedUnresolvedUnassignedNicola MomettocollectionscompilerWed, 24 Oct 2012 09:13:56 -0500Mon, 6 Oct 2014 12:29:29 -0500Release 1.4Release 1.5Release 1.854<p>Unable to reproduce this bug on latest version of master. Most likely fixed by some of the recent changes to data literal readers. </p>
<p>Marking Not-Approved. </p><p>Could not reproduce in master. </p><p>I just checked, and the problem still exists for records with no arguments:</p>
<p>Clojure 1.6.0-master-SNAPSHOT<br/>
user=&gt; (defrecord a [])<br/>
user.a<br/>
user=&gt; #user.a[]<br/>
{}</p>
<p>Admittedly it's an edge case and I see little usage for no-arguments records, but I think it should be addressed aswell since the current behaviour is not what one would expect</p><p>Got the following REPL interaction:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">% java -jar ~/.m2/repository/org/clojure/clojure/1.5.0/clojure-1.5.0.jar
user=&gt; (defrecord a [])
user.a
user=&gt; (a.)
#user.a{}
user=&gt; #user.a{}
{}
#user.a[]
{}</pre>
</div></div>
<p>This should be reopened or declined for another reason than reproducability.</p><p>I'm reopening this since the bug is still there.</p>
<p>Patch clj-1093-fix-empty-record-literal-patch-v2.txt dated Mar 13, 2013 is identical to Bronsa's patch 001-fix-empty-record-literal.patch dated Oct 24, 2012, except that it applies cleanly to latest master. I'm not sure why the older patch doesn't but git doesn't like something about it.</p><p>Patch 0001-<a href="http://dev.clojure.org/jira/browse/CLJ-1093" title="Empty PersistentCollections get incorrectly evaluated as their generic clojure counterpart">CLJ-1093</a>-fix-empty-records-literal-v2.patch solves more issues than the previous patch that was not evident to me at the time.</p>
<p>Only collections that are either PersistentList or PersistentVector or PersistentHash<span class="error">&#91;Map|Set&#93;</span> or PersistentArrayMap can now be EmptyExpr.<br/>
This is because we don't want every IPersistentCollection to be emitted as a generic one eg.</p>
<p>user=&gt; (class #clojure.lang.PersistentTreeMap[])<br/>
clojure.lang.PersistentArrayMap</p>
<p>Incidentally, this patch also solves <a href="http://dev.clojure.org/jira/browse/CLJ-1187" title="Clojure loses quoted metadata on empty literals"><del>CLJ-1187</del></a><br/>
This patch should be preferred over the one on <a href="http://dev.clojure.org/jira/browse/CLJ-1187" title="Clojure loses quoted metadata on empty literals"><del>CLJ-1187</del></a> since it's more general</p><p>Maybe this is related:</p>
<p>user=&gt; (def x `(quote ~(list 1 (clojure.lang.PersistentTreeMap/create (seq <span class="error">&#91;1 2 3 4&#93;</span>)))))<br/>
#'user/x<br/>
user=&gt; x<br/>
(quote (1 {1 2, 3 4}))<br/>
user=&gt; (class (second (second x)))<br/>
clojure.lang.PersistentTreeMap<br/>
user=&gt; (eval x)<br/>
(1 {1 2, 3 4})<br/>
user=&gt; (class (second (eval x)))<br/>
clojure.lang.PersistentArrayMap</p>
<p>Even if the collection is not evaluated, it is still converted to the generic clojure counterpart.</p><p>In the change for ObjectExpr.emitValue() where you've added PersistentArrayMap to the PersistentHashMap case, should the IPersistentVector case below that be PersistentVector instead, otherwise it would snare a custom IPersistentVector that's not a PersistentVector, right?</p>
<p>This line: "else if(form instanceof ISeq)" at the end of the Compiler diff has different leading tabs which makes the diff slightly more confusing than it could be.</p>
<p>Would be nice to add a test for the sorted map case in the description.</p>
<p>Marking incomplete to address some of these.</p><p>bump</p><p>Attached patch 0001-<a href="http://dev.clojure.org/jira/browse/CLJ-1093" title="Empty PersistentCollections get incorrectly evaluated as their generic clojure counterpart">CLJ-1093</a>-fix-empty-collection-literal-evaluation.patch which implements your suggestions.</p>
<p>replacing IPersistentVector with PersistentVector in ObjectExpr.emitValue() exposes a print-dup limitation: it expects every IPersistentCollection to have a static "create" method.</p>
<p>This required special casing for MapEntry and APersistentVector$SubVector</p><p>I updated the patch adding print-dups for APersistentVector$SubVec and other IPersistentVectors rather than special casing them in the compiler</p><p>All of the checks on concrete classes in the Compiler parts of this patch don't sit well with me. I understand how you got to this point and I don't have an alternate recommendation (yet) but all of that just feels like the wrong direction.</p>
<p>We want to be built on abstractions such that internal collections are not special; they should conform to the same expectations as an external collection and both should follow the same paths in the compiler - needing to check for those types is a flag for me that something is amiss.</p>
<p>I am marking Incomplete for now based on these thoughts. </p><p>I've been thinking for a while about this issue and I've come to the conclusion that in my later patches I've been trying to incorporate fixes for 3 different albeit related issues:</p>
<p>1- Clojure transforms all empty IPersistentCollections in their generic Clojure counterpart</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">user&gt; (defrecord x [])
user.x
user&gt; #user.x[] ;; expected: #user.x{}
{}
user&gt; #user.x{} ;; expected: #user.x{}
{}
user&gt; #clojure.lang.PersistentTreeMap[]
{}
user&gt; (class *1) ;; expected: clojure.lang.PersistentTreeMap
clojure.lang.PersistentArrayMap</pre>
</div></div>
<p>2- Clojure transforms all the literals of collections implementing the Clojure interfaces (IPersistentList, IPersistentVector ..) that are NOT defined with deftype or defrecord, to their<br/>
generic Clojure counterpart</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">user=&gt; (class (eval (sorted-map 1 1)))
clojure.lang.PersistentArrayMap ;; expected: clojure.lang.PersistentTreeMap</pre>
</div></div>
<p>3- print-dup is broken for some Clojure persistent collections</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">user=&gt; (print-dup (subvec [1] 0) *out*)
#=(clojure.lang.APersistentVector$SubVector/create [1])
user=&gt; #=(clojure.lang.APersistentVector$SubVector/create [1])
IllegalArgumentException No matching method found: create clojure.lang.Reflector.invokeMatchingMethod (Reflector.java:53)</pre>
</div></div>
<p>I'll keep this ticket regarding issue #1 and open two other tickets for issue #2 and #3</p><p>I've attached a new patch fixing only this issue, the approach is explained in the description</p><p>0001-<a href="http://dev.clojure.org/jira/browse/CLJ-1093" title="Empty PersistentCollections get incorrectly evaluated as their generic clojure counterpart">CLJ-1093</a>-v2.patch is an updated patch that correctly handles metadata evaluation semantics on empty literals and adds some tests for it</p>ApprovalVettedGlobal RankPatchCode and Test[CLJ-1172] Cross-linking between clojure.lang.Compiler and clojure.lang.RThttp://dev.clojure.org/jira/browse/CLJ-1172
Clojure<p>This is my code (an example):</p>
<div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent">
<pre>import clojure.lang.Compiler;
import clojure.lang.RT;
import clojure.lang.Var;
Compiler.load("(+ 5 %)");
Var foo = RT.var("bar", "foo");
Object result = foo.invoke(10);
assert result.toString().equals("15");
</pre>
</div></div>
<p>This is what I'm getting:</p>
<div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent">
<pre>ava.lang.ExceptionInInitializerError
at clojure.lang.Compiler.&lt;clinit&gt;(Compiler.java:47)
at foo.main(Main.java:75)
Caused by: java.lang.NullPointerException
at clojure.lang.RT.baseLoader(RT.java:2043)
at clojure.lang.RT.load(RT.java:417)
at clojure.lang.RT.load(RT.java:411)
at clojure.lang.RT.doInit(RT.java:447)
at clojure.lang.RT.&lt;clinit&gt;(RT.java:329)
... 36 more
</pre>
</div></div>
<p>The same code worked just fine with version 1.4. Looks like <tt>Compiler</tt> is using <tt>RT</tt> and <tt>RT</tt> is using <tt>Compiler</tt>, both statically.</p>version 1.5.0-RC17CLJ-1172Cross-linking between clojure.lang.Compiler and clojure.lang.RTDefectMinorReopenedUnresolvedUnassignedYegor BugayenkoThu, 28 Feb 2013 06:30:50 -0600Fri, 21 Jun 2013 10:36:56 -0500Release 1.502<p>I cross-posted this question to SO: <a href="http://stackoverflow.com/questions/15207596">http://stackoverflow.com/questions/15207596</a></p><p>calling <tt>RT.init()</tt> before <tt>Compiler.load()</tt> solves the problem</p><p>Yegor, do you consider it OK to close this ticket as not being a problem, or at least one with a reasonable workaround?</p><p>Yes, of course. Let's close it.</p><p>Ticket submitter agrees that this is not an issue, or that there is a reasonable workaround.</p><p>This issue came up again on the Clojure group. <a href="https://groups.google.com/forum/?hl=en_US&amp;fromgroups=#!topic/clojure/2xdLNMb9yyQ">https://groups.google.com/forum/?hl=en_US&amp;fromgroups=#!topic/clojure/2xdLNMb9yyQ</a></p>
<p>I did some testing, and the issue did not exist in Clojure 1.5.0-RC3 and before, and it has existed since 1.5.0-RC4. There was only one commit between those two points:</p>
<p> <a href="https://github.com/clojure/clojure/commit/9b80a552fdabeabdd93951a625b55ae49c2f8d83">https://github.com/clojure/clojure/commit/9b80a552fdabeabdd93951a625b55ae49c2f8d83</a></p>
<p>Maybe this new behavior is an intended consequence of that change. I don't know. In any case, it seems like perhaps the "No need to call RT.init() anymore" message might be outdated?</p><p>Reopening since it came up again, and there is some more info known about the issue. I'll let someone who knows more about the issue decide whether to close it.</p><p>Doing this RT.load("clojure/core"); at the top works avoids the message from RT.init()</p><p>It seems like <tt>RT.load("clojure/core")</tt> does not hide the message anymore - at least not from 1.5.1.</p>Global Rank[CLJ-1368] Document usage for case with non-readable constantshttp://dev.clojure.org/jira/browse/CLJ-1368
Clojure<h2><a name="Problem"></a>Problem</h2>
<p>It is pretty obscure how to get constant-time dispatch for e.g. Enums, even if user knows about case.</p>
<h2><a name="Proposal"></a>Proposal</h2>
<p>The possibility to dispatch to arbitrary constants with case, by wrapper macro, should be documented.</p>
<h3><a name="Wording"></a>Wording</h3>
<ul class="alternate" type="square">
<li>Should it warn against doing that with unstable values?</li>
<li>Should it mention anything else than java Enums?</li>
</ul>
<h3><a name="CaseTechniques"></a>Case Techniques </h3>
<p>Case is documented for accepting all readable forms as test-constants. However, it can also be made to use any compile-time-known constants as test-constants, by wrapping it in another macro.</p>
<p>Sometimes this is appropriate, e.g. when dispatching on a java Enum.<br/>
Other times, less so, e.g. when dispatching on objects whose hash changes when the vm is restarted (breaks AOT).</p>
<h3><a name="Implications"></a>Implications</h3>
<p>This technique is an application of a more general technique: Passing non-literals to a macro from another macro.<br/>
Are there other macros that have use cases like this?</p>
<h2><a name="References"></a>References</h2>
<p><a href="https://groups.google.com/d/topic/clojure/3yGjDO2YnjQ/discussion">https://groups.google.com/d/topic/clojure/3yGjDO2YnjQ/discussion</a></p>CLJ-1368Document usage for case with non-readable constantsEnhancementMinorReopenedUnresolvedUnassignedHerwig HochleitnerdocsinteropSun, 2 Mar 2014 10:34:59 -0600Sun, 2 Mar 2014 11:34:18 -060003<p><del>This is a duplicate of</del> <a href="http://dev.clojure.org/jira/browse/CLJ-1367">http://dev.clojure.org/jira/browse/CLJ-1367</a></p>
<p>Actually, it's an alternate solution</p>Global Rank[CLJ-1161] sources jar has bad versions.properties resourcehttp://dev.clojure.org/jira/browse/CLJ-1161
Clojure<p>The "sources" jar (at least since Clojure 1.4 and including 1.5 RC) has a bad version.properties file in it. The resource clojure/version.properties is literally:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">version=${version}</pre>
</div></div>
<p>The regular Clojure jar has the correct version string in that resource.</p>
<p>I came across a problem when I was experimenting with the sources jar (as used by IDEs). I naively added the sources jar to my classpath, and Clojure died on start up. The bad clojure/versions.properties file was found first, which led to a parse error as the clojure version was being set.</p>
<p><b>Solution:</b> patch leaves version.properties file out of sources JAR, where it causes problems for tools.</p>CLJ-1161sources jar has bad versions.properties resourceDefectMinorReopenedUnresolvedStuart HallowaySteve MinerMon, 11 Feb 2013 10:03:01 -0600Mon, 6 Oct 2014 13:10:54 -0500Release 1.4Release 1.5Release 1.801<p>Notes from the dev mailing list:</p>
<p>The "sources" JAR is generated by another Maven plugin, configured here:<br/>
<a href="https://github.com/clojure/clojure/blob/clojure-1.5.0-RC15/pom.xml#L169-L181">https://github.com/clojure/clojure/blob/clojure-1.5.0-RC15/pom.xml#L169-L181</a></p>
<p>The simplest solution might be to just exclude the file from the sources jar. It looks like maven-source-plugin has an excludes option which would do the trick:</p>
<p><a href="http://maven.apache.org/plugins/maven-source-plugin/jar-mojo.html#excludes">http://maven.apache.org/plugins/maven-source-plugin/jar-mojo.html#excludes</a></p><p>This issue is marked closed, but I'm still seeing it: the clojure-1.6.0-sources.jar, clojure-1.5.1-sources.jar, etc on Maven Central still contain the bad version.properties files. More specifically, it looks like the fix has been applied to builds in the SNAPSHOTS repository, but not to RELEASES.</p>
<p>Fix applied: <a href="https://oss.sonatype.org/content/repositories/snapshots/org/clojure/clojure/">https://oss.sonatype.org/content/repositories/snapshots/org/clojure/clojure/</a><br/>
Not fixed: <a href="https://oss.sonatype.org/content/repositories/releases/org/clojure/clojure/">https://oss.sonatype.org/content/repositories/releases/org/clojure/clojure/</a></p><p>Not sure what's needed here, but marking incomplete.</p><p>Would a fix for this update existing sources jars (1.5.1, 1.6.0, etc) on Central? Or would any fix have to wait on the next Clojure release?</p><p>For all the same reasons that mutable state is undesirable, changing an existing release jar in the central Maven repository is also undesirable. While it's not technically impossible, we will not update existing releases and this will need to wait for the next. I've looked at this problem a little and I do not yet know enough to know how to fix it or why it even varies between snapshot and release. Help welcome!</p>
<p>In which tool do you see a resulting problem from this?</p><p>Despite the way I phrased the question, I'd hoped that would be the answer. It's the right policy.</p>
<p>Unfortunately, this issue leaves the released sources jars essentially unusable from a tools standpoint. CIDER now has source code navigation from stacktraces &#8211; you can jump into both Clojure and Java function definitions from the error/trace. For the latter, the sources jar (for Clojure or any other Java library) needs to be on the classpath as a dev dependency. There's more host interop support in the works for CIDER too ("embrace the host platform"), but not being able to add a dependency on a stable Clojure sources jar presents a wrinkle.</p>
<p>Are the official Clojure releases built by Hudson? The Hudson build right before the 1.6.0 release (<a href="http://build.clojure.org/job/clojure/532/org.clojure$clojure/">#532</a>) and the one right after (<a href="http://build.clojure.org/job/clojure/534/org.clojure$clojure/">#534</a>) both show the exclusion fix, as does the git <a href="https://github.com/clojure/clojure/blob/clojure-1.6.0/pom.xml#L181">clojure-1.6.0</a> tag, which when I check out and build from source, is fine. The Hudson builds with release tags (e.g. 1.6 = <a href="http://build.clojure.org/job/clojure/533/org.clojure$clojure/">#533</a>, 1.6-RC1 = <a href="http://build.clojure.org/view/Clojure/job/clojure/512/">#512</a>, etc), though, don't show any artifacts other than a pom.xml. This would seem to make it rather hard to audit builds...am I missing something? </p>ApprovalIncompleteGlobal RankPatchCode[CLJ-1623] Support zero-depth structures for update and update-inhttp://dev.clojure.org/jira/browse/CLJ-1623
Clojure<p>Currently "update" and "update-in" assume a nested associative structure at least 1 level deep.</p>
<p>For greater generality, it would be preferable to also support the case of 0 levels deep (i.e. a nested associative structure where there is only a leaf node)</p>
<p>examples:</p>
<p>;; Zero-length paths would be supported in update-in<br/>
(update-in 1 [] inc) =&gt; 2</p>
<p>;; update would get an extra arity which simply substitutes the new value<br/>
(update :old :new) =&gt; :new</p>CLJ-1623Support zero-depth structures for update and update-inEnhancementMinorReopenedUnresolvedUnassignedMike AndersonMon, 22 Dec 2014 23:26:17 -0600Wed, 24 Dec 2014 07:55:10 -0600Release 1.700<p>Duplicate of <a href="http://dev.clojure.org/jira/browse/CLJ-373" title="update-in with empty key paths"><del>CLJ-373</del></a> which has been declined?</p><p>Rich has declined similar requests in the past.</p><p>I disagree with the reasons for rejecting the previous patch. Can we revisit this?</p>
<p>Yes, it is a (very minor) behaviour change for update-in, but only only on undefined implementation behaviour, and even then only on the error case. If people are relying on this then their code is already very broken.</p>
<p>On the plus side, is makes the behaviour more logical and consistent. There is clearly demand for the change (see the various comments in favour of improving this in <a href="http://dev.clojure.org/jira/browse/CLJ-373" title="update-in with empty key paths"><del>CLJ-373</del></a>)</p>
<p>As an aside: if you really want to keep the old behaviour of disallowing empty paths then it would be better to convert the NullPointerException into a meaningful error message e.g. "Empty key paths are not allowed"</p>
<p>Also, I am proposing a corresponding change to update which doesn't have the above concern (since it is introducing a whole new arity)</p><p>Sorry, Rich has said he's not interested.</p>Global Rank[CLJ-1591] Symbol not being bound in namespace when name clashes with clojure.corehttp://dev.clojure.org/jira/browse/CLJ-1591
Clojure<p>The following code fails (both in 1.6 and latest 1.7-alpha4):</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">user=&gt; (ns foo)
nil
foo=&gt; (def inc inc)
WARNING: inc already refers to: #'clojure.core/inc in namespace: foo, being replaced by: #'foo/inc
#'foo/inc
;; Note inc is unbound at <span class="code-keyword">this</span> point, which causes the exception below
foo=&gt; inc
#&lt;Unbound Unbound: #'foo/inc&gt;
foo=&gt; (ns bar)
nil
bar=&gt; (require ['foo :refer ['inc]])
WARNING: inc already refers to: #'clojure.core/inc in namespace: bar, being replaced by: #'foo/inc
nil
bar=&gt; (inc 8)
IllegalStateException Attempting to call unbound fn: #'foo/inc clojure.lang.Var$Unbound.throwArity (Var.java:43)</pre>
</div></div>
<p>Further investigation shows that foo/inc is unbound:</p>
<p>foo/inc<br/>
=&gt; #&lt;Unbound Unbound: #'foo/inc&gt;</p>
<p>Further investigation also shows that replacing the (def inc inc) with almost anything else, e.g. (def inc dec), (def inc clojure.core/inc), or (def inc (fn <span class="error">&#91;n&#93;</span> (+ n 1))), causes no exception (but the warnings remain).</p>
<p>I would expect:<br/>
a) foo/inc should be bound and have the same value as clojure.core/inc<br/>
b) No error when requiring foo/inc<br/>
c) bar/inc should be bound to foo/inc</p>CLJ-1591Symbol not being bound in namespace when name clashes with clojure.coreDefectMinorReopenedUnresolvedUnassignedMike AndersonFri, 14 Nov 2014 20:54:35 -0600Sun, 28 Dec 2014 11:20:44 -0600Release 1.6Release 1.723<p>The second error should be expected, the right syntax should be (require ['foo :refer <span class="error">&#91;&#39;inc&#93;</span>]) (note the leading quote before inc)</p><p>Thanks for the catch Nicola - I've edited the description. Still get the same error however (just with a slightly different message)</p><p>See comment...</p><p>@Alex what comment? Note that the error still occurs even with the right syntax....</p><p>Appears to have been closed prematurely</p><p>I can't reproduce with the correct syntax:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">Clojure 1.7.0-master-SNAPSHOT
user=&gt; (ns foo)
nil
foo=&gt; (def inc inc)
WARNING: inc already refers to: #'clojure.core/inc in namespace: foo, being replaced by: #'foo/inc
#'foo/inc
foo=&gt; (ns bar)
nil
bar=&gt; (require ['foo :refer ['inc]])
WARNING: inc already refers to: #'clojure.core/inc in namespace: bar, being replaced by: #'foo/inc
nil</pre>
</div></div><p>The problem is that the var is still unbound and causes e.g. the following error:</p>
<p>=&gt; (foo/inc 8)<br/>
IllegalStateException Attempting to call unbound fn: #'foo/inc clojure.lang.Var$Unbound.throwArity (Var.java:43)</p>
<p>I don't think that should be expected - or am I missing something?</p><p>Ah, will take a look. But not right now. <img class="emoticon" src="http://dev.clojure.org/jira/images/icons/emoticons/smile.gif" height="20" width="20" align="absmiddle" alt="" border="0"/></p><p>Updated the description with a few more details. The exception goes away if you do (def inc (fn <span class="error">&#91;n&#93;</span> (+ n 1))) instead of (def inc inc), for example. The warnings remain.</p><p>Unsure if this is the same issue (I think it might be?), but I reproduced the exact same error message with AOT compilation involved:</p>
<p>reproduced in this git repository: <a href="https://github.com/yeller/compiler_update_not_referenced_bug">https://github.com/yeller/compiler_update_not_referenced_bug</a></p>
<p>clone it, run `lein do clean, uberjar, test`, and that error message will show up every time for me</p><p>Mike, I think replacing (def inc inc) in your example with (def inc clojure.core/inc) should be considered as a reasonable workaround for this issue, unless you have some use case where you need to def inc to something that is not in clojure.core (and if so, why?)</p>
<p>The reason (def inc inc) behaves this way is, if not absolutely necessary, at least commonly used in Clojure programs to define recursive functions, e.g. (defn fib <span class="error">&#91;n&#93;</span> (if (&lt;= n 1) 1 (+ (fib (dec n)) (fib (- n 2))))), so that the occurrences of fib in the body are resolved to the fib being defined.</p><p>Moving to 1.7 until I can look at this more deeply.</p><p>Andy - yes the workaround is fine for me right now.</p>
<p>I don't think this is an urgent issue but it may be exposing a subtle complexity regarding assumptions about the state of the namespace at different times. Perhaps the semantics should be something like:</p>
<ul class="alternate" type="square">
<li>The def statement itself should be run <b>before</b> the var is interned. e.g. (def inc (inc 5)) should result in (def inc 6)</li>
<li>Anything complied / deferred to run after completion of the def statement should use the new var (i.e. the new var should be referenced by fns, lazy sequences etc.)</li>
</ul>
<p>I'm not sure what your proposal means in a case like this:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">(def inc (fn [x] (inc x)))</pre>
</div></div>
<p>Is the second inc to be interpreted/resolved before or after the new inc is created? Because it is (fn ...) it should be the after-behavior? What else besides fn should cause the after-behavior, rather than the before-behavior?</p>
<p>Even more fun (not saying that people often write code like this, but the compiler can handle it today):</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">(def inc (<span class="code-keyword">if</span> (&gt; (inc y) 5)
(fn [x] (inc x))
(fn [x] (dec x))))</pre>
</div></div>
<p>I think the current compiler behavior of 'in the body of a def, the def'd symbol always refers to the new var, not any earlier def'd vars' is fairly straightforward to explain.</p><p>Should I file the AOT issue reproduced in that thing as a new issue?</p><p>Tom: Alex Miller or another screener would be best to say whether the AOT issue should be a separate ticket, but my best guess would be "go for it". I tried to look at the link you gave but it seems not to point to anything. Could you double-check that link?</p><p>Andy,</p>
<p>Great. I'll write one up tomorrow sometime. I accidentally left that repo as private, should be visible now.</p><p>This comment is really most relevant for ticket <a href="http://dev.clojure.org/jira/browse/CLJ-1604" title="AOT&#39;ed code that defs a var with clojure.core symbol name causes IllegalStateException"><del>CLJ-1604</del></a>, where it has been copied:</p>
<p>Tom, looked at your project. Thanks for that. It appears not to have anything like (def inc inc) in it. It throws exception during test step of 'lein do clean, uberjar, test' consistently for me, too, but compiles with only warnings and passes tests with 'lein do clean, test'. I have more test results showing in which Clojure versions these results change. To summarize, the changes to Clojure that appear to make the biggest difference in the results are below (these should be added to the new ticket you create &#8211; you are welcome to do so):</p>
<p>Clojure 1.6.0, 1.7.0-alpha1, and later changes up through the commit with description "<a href="http://dev.clojure.org/jira/browse/CLJ-1378" title="Hints don&#39;t work with #() form of function"><del>CLJ-1378</del></a>: Allows FnExpr to override its reported class with a type hint": No errors or warnings for either lein command above.</p>
<p>Next commit with description "Add clojure.core/update, like update-in but takes a single key" that adds clojure.core/update: 'lein do clean, test' is fine, but 'lein do clean, uberjar' throws exception during compilation, probably due to <a href="http://dev.clojure.org/jira/browse/CLJ-1241" title="NPE when AOTing overrided clojure.core functions"><del>CLJ-1241</del></a>.</p>
<p>Next commit with description "fix <a href="http://dev.clojure.org/jira/browse/CLJ-1241" title="NPE when AOTing overrided clojure.core functions"><del>CLJ-1241</del></a>": 'lein do clean, test' and 'lein do clean, uberjar' give warnings about clojure.core/update, but no errors or exceptions. 'lein do clean, uberjar, test' throws exception during test step that is same as the one I see with Clojure 1.7.0-alpha4. Debug prints of values of clojure.core/update and int-map/update (in data.int-map and in Tom's namespace compiler-update-not-referenced-bug.core) show things look fine when printed inside data.int-map, and in Tom's namespace when not doing the uberjar, but when doing the uberjar, test, int-map/update is unbound in Tom's namespace.</p>
<p>In case it makes a difference, my testing was done with Mac OS X 10.9.5, Leiningen 2.5.0 on Java 1.7.0_45 Java HotSpot(TM) 64-Bit Server VM</p><p>Tom, I've opened a ticket with a patch fixing the AOT issue: <a href="http://dev.clojure.org/jira/browse/CLJ-1604">http://dev.clojure.org/jira/browse/CLJ-1604</a></p>ApprovalTriagedGlobal Rank[CLJ-1099] better error message when passing non-seq to seqhttp://dev.clojure.org/jira/browse/CLJ-1099
Clojure<p>Design discussion <a href="http://dev.clojure.org/display/design/Better+Error+Messages">here</a>.</p>
<p>This patch improves Clojure's error message for a single common error: passing a non-seq where a seq is neede. More importantly, it is intended as a prototype for other similar improvements in the future.</p>
<p>Error message before:</p>
<div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent">
<pre>(cons 1 2)
=&gt; IllegalArgumentException Don't know how to create ISeq from: java.lang.Long
</pre>
</div></div>
<p>Error message after:</p>
<div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent">
<pre>user=&gt; (cons 1 2)
ExceptionInfo Don't know how to create ISeq from: java.lang.Long
user=&gt; (ex-data *e)
{:instance 2}
</pre>
</div></div>
<p><b>Patch:</b> better-error-message-for-seq.patch<br/>
NOTE: This patch was reverted as it affected the inlining of RT.seqFrom().</p>CLJ-1099better error message when passing non-seq to seqEnhancementMinorReopenedUnresolvedUnassignedStuart HallowaycheckargsThu, 1 Nov 2012 10:03:14 -0500Tue, 24 Feb 2015 11:57:23 -060013<p>Wouldn't it be better to make it read "Don't know how to create ISeq from: 2 (java.lang.Long)"? How many beginners will figure<br/>
out ex-data exists and how to use it?</p><p>Hi Michael,</p>
<p>ex-info messages should not, in general, pr-str things into their bodies. This raises the question of print-length and print-level in a place where the user doesn't have good control, while the whole point of ex-info is to be in the data business, not the string business. Users can control printing from ex-data any way they like.</p>
<p>There are two possible ways to make beginners aware of ex-data: Tell them about it in one (or a few places) in docs, or in an infinite number of places saying "This would have been useful here, but we didn't use it because you might not know about it." I prefer the former.</p>
<p>That said, I think it would be great to increase the visibility of ex-info and ex-data early on in documentation for beginners, and to make sure that things like exception printing in logs are flexible enough not to lose the benefits of ex-info.</p><p>Just a comment that this fix was committed before release 1.6.0, and then reverted very shortly before release 1.6.0. I believe the reason for reverting was due to concerns that this change made performance about 5% slower in some relatively common cases, with a suspicion that it could have affected inlining of the seqFrom method.</p>
<p>Not sure whether the ticket should be reopened or not.</p>ApprovalVettedGlobal RankPatchCode[CLJ-1371] divide(Object, Object) with (NaN, 0) does not return NaNhttp://dev.clojure.org/jira/browse/CLJ-1371
Clojure<p>user=&gt; (def x Double/NaN)<br/>
#'user/x<br/>
user=&gt; (/ x 0)</p>
<p>ArithmeticException Divide by zero clojure.lang.Numbers.divide (Numbers.java:156)<br/>
user=&gt; (/ Double/NaN 0)<br/>
Double/NaN</p>CLJ-1371divide(Object, Object) with (NaN, 0) does not return NaNDefectTrivialReopenedUnresolvedUnassignedYongqian LimathFri, 7 Mar 2014 02:36:11 -0600Fri, 7 Mar 2014 09:48:40 -060001<p>As per the Java Language Specification (<a href="http://docs.oracle.com/javase/specs/jls/se7/html/jls-4.html#jls-4.2.4">http://docs.oracle.com/javase/specs/jls/se7/html/jls-4.html#jls-4.2.4</a>), </p>
<p>"All numeric operations with NaN as an operand produce NaN as a result."</p><p>But in the first example it produces an ArithmeticException.</p><p>Ah, I see the question now.</p>
<p>Here we are dividing a double by a long. In the first case, this is parsed as divide(Object, long) which then calls divide(Object, Object), which throws ArithmeticException if the second arg is 0 (regardless of the first arg).</p>
<p>In the second case it's parsed as divide(double, long) which just relies on Java to properly upcast the primitive long to a double to do the divide. </p>
<p>Note that making this call with 2 doubles does return NaN:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">user=&gt; (def x <span class="code-object">Double</span>/NaN)
#'user/x
user=&gt; (/ x 0.0)
NaN</pre>
</div></div>
<p>or type hinting x to a double works as well:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">user=&gt; (def x <span class="code-object">Double</span>/NaN)
#'user/x
user=&gt; (/ ^<span class="code-object">double</span> x 0.0)
NaN</pre>
</div></div>
<p>I think one option to "fix" this behavior would be to add checks in divide(Object, Object) to check whether x is NaN and instead return NaN.</p>Global Rank