Details

Description

I'm using clojure.core.typed 0.2.19 with the slim classifier but I've observed the same without slim.

My suspicion is that the latent filters associated with functions grow in size exponentially with each extra optional key to a HMap (based on the output when you have a type error). I think it's generating all combinations of present and absent keys for the HMap when calculating latent filters for a function.

I've attached a tarball with a lein project with ten namespaces that all contain the same ten simple functions in the form

Ambrose Bonnaire-Sergeant
added a comment - 03/Dec/13 12:50 PM - edited Yes, the primitives are :mandatory, :absent-keys and :complete; :optional expands to be in terms of those.
Thanks for the report, I haven't done performance testing on this strategy. It will probably need to be reconsidered.

I've spend quite some time to learn core.typed on a real use case, and I have to say that I'm amazed by the power of the analysis and the potential to prevent errors that otherwise would be detected to late! Apart from that the type-annotations are valuable and precise information about the interface when doing maintenance at code you did not see for some time.
Two big reasons to use core.typed, however, as I turned more and more code into typed code the number of optional keys increased in my core data-record increase, and core.typed (check-ns) slowed down to the extreme (using version 0.2.25 (current stable)). I refactored the code to get a single line of code that triggers the issue on my core data structure a (Seqable DbMsg). Attached you find the project.clj and code (core.clj vs 8:09pm).

When I reduce the number of optional keys to 4-5 the (check-ns) runs in a few seconds. But having 9-11 optional keys kills the (check-ns) process.

I hope this one-liner helps making the issues easy to reproduce.
Currently I don't know a work-around for this issue,
so I can not use core.typed to check my project.

I guess that many people/projects should run into the same issues,
as the pattern is returning quite often in clojure:

1. start with data-set with small Hashmaps
2. the hashmaps evolve in a number steps (meaning more keys are added and some of the existing keys are removed)
I guess the workaround would be to use limited or no optional keys, and prepare a custom HMap definition for each stage of analysis. Although this would work, the amount of bookkeeping already significant for a all records are running through the same stages (linear process), and will be a showstopper if it is a non-linear process (not all records follow the same path (sequence of analysis stages)).
I will investigate whether splitting in multiple types (for different stages) will rescue my day.

Cees van Kemenade
added a comment - 26/Jan/14 1:05 PM - edited +1 For this issue.
I've spend quite some time to learn core.typed on a real use case, and I have to say that I'm amazed by the power of the analysis and the potential to prevent errors that otherwise would be detected to late! Apart from that the type-annotations are valuable and precise information about the interface when doing maintenance at code you did not see for some time.
Two big reasons to use core.typed, however, as I turned more and more code into typed code the number of optional keys increased in my core data-record increase, and core.typed (check-ns) slowed down to the extreme (using version 0.2.25 (current stable)). I refactored the code to get a single line of code that triggers the issue on my core data structure a (Seqable DbMsg). Attached you find the project.clj and code (core.clj vs 8:09pm).
When I reduce the number of optional keys to 4-5 the (check-ns) runs in a few seconds. But having 9-11 optional keys kills the (check-ns) process.
I hope this one-liner helps making the issues easy to reproduce.
Currently I don't know a work-around for this issue,
so I can not use core.typed to check my project.
I guess that many people/projects should run into the same issues,
as the pattern is returning quite often in clojure:
1. start with data-set with small Hashmaps
2. the hashmaps evolve in a number steps (meaning more keys are added and some of the existing keys are removed)
I guess the workaround would be to use limited or no optional keys, and prepare a custom HMap definition for each stage of analysis. Although this would work, the amount of bookkeeping already significant for a all records are running through the same stages (linear process), and will be a showstopper if it is a non-linear process (not all records follow the same path (sequence of analysis stages)).
I will investigate whether splitting in multiple types (for different stages) will rescue my day.

This is theoretically trivial to implement, and just involves shuffling around where optional keys are handled. However, this is spread out all over the code base, so I may not complete the patch for a week.

Ambrose Bonnaire-Sergeant
added a comment - 27/Jan/14 6:34 AM This is theoretically trivial to implement, and just involves shuffling around where optional keys are handled. However, this is spread out all over the code base, so I may not complete the patch for a week.
Great to hear core.typed is working out for you!

The code in hmap/big-options.clj (from your test-suite) still contains a workaround to make core.typed proceed without errors.
The (if (string? notifs) ... on line 107 is introduced to prevent a type-error (core.typed infers it that notifs might als be a (sequable String). However, as this is the else-branch of line 104 (if (or (seq? notifs) (vector notifs)) ... the case that notifs is a (Seqable string) can be excluded already).
Also using on line 104 the more general (if (sequential? notifs) ... does not resolve this issues.

Is this inference error already on the list, or would you recommend posting a seperate JIRA-case for this issue?

Cees van Kemenade
added a comment - 14/Feb/14 4:11 AM - edited This resolve the issue and also resolves CT-102.
Thanks!
The code in hmap/big-options.clj (from your test-suite) still contains a workaround to make core.typed proceed without errors.
The (if (string? notifs) ... on line 107 is introduced to prevent a type-error (core.typed infers it that notifs might als be a (sequable String). However, as this is the else-branch of line 104 (if (or (seq? notifs) (vector notifs)) ... the case that notifs is a (Seqable string) can be excluded already).
Also using on line 104 the more general (if (sequential? notifs) ... does not resolve this issues.
Is this inference error already on the list, or would you recommend posting a seperate JIRA-case for this issue?