Clojure JIRAhttp://dev.clojure.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+10040+AND+labels+%3D+timestamp
An XML representation of a search requesten-us4.464925-07-2011[CLJS-454] Instance Reader to Support Micro/Nanosecondshttp://dev.clojure.org/jira/browse/CLJS-454
ClojureScript<p>Any timestamps with a greater than millisecond precision cannot be handled by the ClouseScript reader:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">&gt; cljs.reader.read_date(<span class="code-quote">"2012-12-30T23:20:05.066980000-00:00"</span>)
&gt; Error: Assert failed: timestamp millisecond field must be in range 0..999 Failed: 0&lt;=66980000&lt;=999 (&lt;= low n high)</pre>
</div></div>
<p>Here "2012-12-30T23:20:05.066980000-00:00" is an example of an ordinary timestamp that is returned from Postgres.</p>
<p>ClojureScript reader interprets the nanosecond portion "066980000" as milliseconds and the check <a href="https://github.com/clojure/clojurescript/blob/47bc8f03f5c05aa754ebfed2e4332967692f291a/src/cljs/cljs/reader.cljs#L494">here</a> fails:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">def parse-and-validate-timestamp
...
(check 0 ms 999 <span class="code-quote">"timestamp millisecond field must be in range 0..999"</span>)</pre>
</div></div>N/ACLJS-454Instance Reader to Support Micro/NanosecondsDefectMinorClosedCompletedUnassignedAnatoly PolinskyreadertimestampFri, 4 Jan 2013 15:30:23 -0600Sat, 3 Aug 2013 17:05:45 -0500Sat, 3 Aug 2013 17:05:40 -050045<p>Is this behavior supported in Clojure?</p><p>It seems <a href="https://github.com/clojure/clojure/blob/master/src/clj/clojure/instant.clj#L61">it is</a></p><p>@David,</p>
<p> Yep, it is supported. Enhancing Jozef's response:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">user=&gt; (use 'clojure.instant)
nil
user=&gt; (read-instant-date <span class="code-quote">"2012-12-30T23:20:05.066980000-00:00"</span>)
#inst <span class="code-quote">"2012-12-30T23:20:05.066-00:00"</span></pre>
</div></div>
<p>/Anatoly</p><p>Ok, do you all have a good idea about what a patch would look like?</p><p>Since JavaScript has no support for nanoseconds in Date, I'd vote for dropping the nanoseconds. Currently the reader just blows up when reading a String with java.sql.Timestamp printed in CLJ, since that is printed in nanosecond precision.</p>
<p>While probably not the best solution, I attached a patch for cljs.reader/parse-and-validate-timestamp fn that just drops the nanoseconds from the millisecond portion. Would be easier to just strip the ms string to 3 digits but that would cause "2012-12-30T23:20:05.066980000012312312123121-00:00" to validate.</p><p>Well, Math. Just realized that this is not really correct, since 1000 is closer to 999 than it is to 100.</p><p>Sorry, for that brainfart.</p><p>Or we could return a custom ClojureScript Date type that provides the same api as js/Date but adds support for nanoseconds. </p><p>One could extend the js/Date prototype with a setNanos/getNanos method and call them accordingly. I'd offer to implement that but the cljs.reader/parse-and-validate-timestamp function scares me, any objections to rewriting that?</p>
<p>As for the js/Date extension, thats easy enough:</p>
<p>(set! (.. js/Date -prototype -setNanos) (fn <span class="error">&#91;ns&#93;</span> (this-as self (set! (.-nanos self) ns))))<br/>
(set! (.. js/Date -prototype -getNanos) (fn [] (this-as self (or (.-nanos self) 0))))</p>
<p>Not sure if its a good idea though, messing with otherwise native code might not be "stable".</p>
<p>Not convinced that keeping the nanos around is "required" since javascript cannot construct Dates with nanos, but its probably better not to lose the nanos in case of round tripping from clj -&gt; cljs -&gt; clj.</p><p>We definitely don't want to mutate Date's prototype without namespacing. I'm not sure we want to mutate Date's prototype at all. That's why I suggested a ClojureScript type with the same interface as Date just as Google has done in Closure.</p><p>This patch (<a href="http://dev.clojure.org/jira/browse/CLJS-454" title="Instance Reader to Support Micro/Nanoseconds"><del>CLJS-454</del></a>.diff) takes the "truncate to millisecs" approach which I think is correct considering that the clojure reader does the same thing (but they truncate to nanoseconds instead).</p>
<p>The patch does some refactoring of parse-and-validate-timestamp and in the process fixes <a href="http://dev.clojure.org/jira/browse/CLJS-564" title="Clojurescript reader can&#39;t read instant literals with some trailing components missing"><del>CLJS-564</del></a> as well as a subtle bug where the interpretation of the fraction part differed between clojurescript and clojure (see commit msg for details)</p>
<p>I also added tests.</p><p>There are duplicate tests in core-tests<span class="error">&#91;1&#93;</span> and reader-tests<span class="error">&#91;2&#93;</span>. This wouldn't be a big deal but each of them generates 7000+ assertions which is a bit excessive. Also note that the assertions in reader-tests doesn't do any padding so instant literals of the form #inst "2010-1-1..." are generated (which are not valid but goes unnoticed in the current implementation). This is why the <a href="http://dev.clojure.org/jira/browse/CLJS-454" title="Instance Reader to Support Micro/Nanoseconds"><del>CLJS-454</del></a>.diff patch fails to pass the tests.</p>
<p>Should I update the patch where I remove some of the tests (either in core-tests or reader-tests)?</p>
<p><span class="error">&#91;1&#93;</span> <a href="https://github.com/clojure/clojurescript/blob/master/test/cljs/cljs/core_test.cljs#L1770">https://github.com/clojure/clojurescript/blob/master/test/cljs/cljs/core_test.cljs#L1770</a><br/>
<span class="error">&#91;2&#93;</span> <a href="https://github.com/clojure/clojurescript/blob/master/test/cljs/cljs/reader_test.cljs#L57">https://github.com/clojure/clojurescript/blob/master/test/cljs/cljs/reader_test.cljs#L57</a></p><p>I note that Clojure includes these tests so probably not a good idea to remove them. Also they actually test different things right? I'd rather see the tests fixed if they need to be adjusted for the new behavior.</p><p>I can't find the tests you're referring to. Link?</p><p><a href="https://github.com/clojure/clojure/blob/master/test/clojure/test_clojure/reader.clj#L482">https://github.com/clojure/clojure/blob/master/test/clojure/test_clojure/reader.clj#L482</a></p><p>format take care of the padding</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">(format <span class="code-quote">"#inst \"</span>2010-%02d-%02dT%02d:14:15.666-06:00\"" month day hour)</pre>
</div></div>
<p>so those tests won't generate strings of the form "2010-1-1T1:14:15.666-06:00".</p>
<p>Also, the <em>clojure</em> reader can't read such literals and requires zero-padding:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">user=&gt; #inst <span class="code-quote">"2010-1-1T1:14:15.666-06:00"</span>
RuntimeException Unrecognized date/time syntax: 2010-1-1T1:14:15.666-06:00 clojure.instant/fn--6183/fn--6184 (instant.clj:118)
user=&gt; #inst <span class="code-quote">"2010-01-01T01:14:15.666-06:00"</span>
#inst <span class="code-quote">"2010-01-01T07:14:15.666-00:00"</span></pre>
</div></div>
<p>The <em>clojurescript</em> reader returns bogus dates:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">ClojureScript:cljs.user&gt; (reader/read-string <span class="code-quote">"#inst \"</span>2010-1-1T1:14:15.666-06:00\"")
#inst <span class="code-quote">"1970-01-01T00:00:00.000-00:00"</span>
ClojureScript:cljs.user&gt; (reader/read-string <span class="code-quote">"#inst \"</span>2010-01-01T01:14:15.666-06:00\"")
#inst <span class="code-quote">"2010-01-01T07:14:15.666-00:00"</span></pre>
</div></div>
<p>which is probably the reason the tests from <a href="https://github.com/clojure/clojurescript/blob/master/test/cljs/cljs/reader_test.cljs#L57">https://github.com/clojure/clojurescript/blob/master/test/cljs/cljs/reader_test.cljs#L57</a> still passes without assertion errors.</p><p>fixed, <a href="http://github.com/clojure/clojurescript/commit/a749ab6c04baa6bd4c890e860adf43b3d702932b">http://github.com/clojure/clojurescript/commit/a749ab6c04baa6bd4c890e860adf43b3d702932b</a></p>Global Rank