ICFP 2016 Research Papers

ICFP 2016 provides a forum for researchers and developers to hear about the latest work on the design, implementations, principles, and uses of functional programming. The conference covers the entire spectrum of work, from practice to theory, including its peripheries.

Call for Papers

Scope

ICFP 2016 seeks original papers on the art and science of functional programming. Submissions are invited on all topics from principles to practice, from foundations to features, and from abstraction to application. The scope includes all languages that encourage functional programming, including both purely applicative and imperative languages, as well as languages with objects, concurrency, or parallelism. Topics of interest include (but are not limited to):

Experience Reports: short papers that provide evidence that functional programming really works or describe obstacles that have kept it from working.

If you are concerned about the appropriateness of some topic, do not hesitate to contact the program chair.

Abbreviated instructions for authors

By Wednesday, March 16 2016, 15:00 (UTC), submit a full paper of at most 12 pages (6 pages for an Experience Report), in standard SIGPLAN conference format, including figures but excluding bibliography.

The deadlines will be strictly enforced and papers exceeding the page limits will be summarily rejected.

ICFP 2016 will employ a lightweight double-blind reviewing process. To facilitate this, submitted papers must adhere to two rules:

author names and institutions must be omitted, and

references to authors’ own related work should be in the third person (e.g., not “We build on our previous work …” but rather “We build on the work of …”).

The purpose of this process is to help the PC and external reviewers come to an initial judgement about the paper without bias, not to make it impossible for them to discover the authors if they were to try. Nothing should be done in the name of anonymity that weakens the submission or makes the job of reviewing the paper more difficult (e.g., important background references should not be omitted or anonymized). In addition, authors should feel free to disseminate their ideas or draft versions of their paper as they normally would. For instance, authors may post drafts of their papers on the web or give talks on their research ideas. We have put together a document answering frequently asked questions (last updated February 8, 2016) that should address many common concerns.

Authors have the option to attach supplementary material to a submission, on the understanding that reviewers may choose not to look at it. The material should be uploaded at submission time, as a single pdf or a tarball, not via a URL. This supplementary material may or may not be anonymized; if not anonymized, it will only be revealed to reviewers after they have submitted their review of your paper and learned your identity.

Authors of resubmitted (but previously rejected) papers have the option to attach an annotated copy of the reviews of their previous submission(s), explaining how they have addressed these previous reviews in the present submission. If a reviewer identifies him/herself as a reviewer of this previous submission and wishes to see how his/her comments have been addressed, the program chair will communicate to this reviewer the annotated copy of his/her previous review. Otherwise, no reviewer will read the annotated copies of the previous reviews.

Overall, a submission will be evaluated according to its relevance, correctness, significance, originality, and clarity. It should explain its contributions in both general and technical terms, clearly identifying what has been accomplished, explaining why it is significant, and comparing it with previous work. The technical content should be accessible to a broad audience. Functional Pearls and Experience Reports are separate categories of papers that need not report original research results and must be marked as such at the time of submission. Detailed guidelines on both categories are given below.

Presentations will be videotaped and released online if the presenter consents. The proceedings will be freely available for download from the ACM Digital Library from at least one week before the start of the conference until two weeks after the conference.

Formatting: Submissions must be in PDF format printable in black and white on US Letter sized paper and interpretable by Ghostscript. Papers must adhere to the standard SIGPLAN conference format: two columns, nine-point font on a ten-point baseline, with columns 20pc (3.33in) wide and 54pc (9in) tall, with a column gutter of 2pc (0.33in). A suitable document template for LaTeX is available at http://www.sigplan.org/Resources/Author/.

Improved versions of a paper may be submitted at any point before the submission deadline using the same web interface.

Author response: Authors will have a 72-hour period, starting at 15:00 UTC on Monday, 2 May, 2016, to read reviews and respond to them.

ACM Author-Izer is a unique service that enables ACM authors to generate and post links on either their home page or institutional repository for visitors to download the definitive version of their articles from the ACM Digital Library at no charge. Downloads through Author-Izer links are captured in official ACM statistics, improving the accuracy of usage and impact measurements. Consistently linking the definitive version of ACM article should reduce user confusion over article versioning. After your article has been published and assigned to your ACM Author Profile page, please visit http://www.acm.org/publications/acm-author-izer-service to learn how to create your links for free downloads from the ACM DL.

AUTHORS TAKE NOTE: The official publication date is the date the proceedings are made available in the ACM Digital Library. This date may be up to two weeks prior to the first day of your conference. The official publication date affects the deadline for any patent filings related to published work.

Special categories of papers

In addition to research papers, ICFP solicits two kinds of papers that do not require original research contributions: Functional Pearls, which are full papers, and Experience Reports, which are limited to six pages. Authors submitting such papers may wish to consider the following advice.

Functional Pearls

A Functional Pearl is an elegant essay about something related to functional programming. Examples include, but are not limited to:

a new and thought-provoking way of looking at an old idea

an instructive example of program calculation or proof

a nifty presentation of an old or new data structure

an interesting application of functional programming techniques

a novel use or exposition of functional programming in the classroom

While pearls often demonstrate an idea through the development of a short program, there is no requirement or expectation that they do so. Thus, they encompass the notions of theoretical and educational pearls.

Functional Pearls are valued as highly and judged as rigorously as ordinary papers, but using somewhat different criteria. In particular, a pearl is not required to report original research, but, it should be concise, instructive, and entertaining. Your pearl is likely to be rejected if your readers get bored, if the material gets too complicated, if too much specialized knowledge is needed, or if the writing is inelegant. The key to writing a good pearl is polishing.

A submission you wish to have treated as a pearl must be marked as such on the submission web page, and should contain the words “Functional Pearl” somewhere in its title or subtitle. These steps will alert reviewers to use the appropriate evaluation criteria. Pearls will be combined with ordinary papers, however, for the purpose of computing the conference’s acceptance rate.

Experience Reports

The purpose of an Experience Report is to help create a body of published, refereed, citable evidence that functional programming really works — or to describe what obstacles prevent it from working.

Possible topics for an Experience Report include, but are not limited to:

insights gained from real-world projects using functional programming

comparison of functional programming with conventional programming in the context of an industrial project or a university curriculum

project-management, business, or legal issues encountered when using functional programming in a real-world project

curricular issues encountered when using functional programming in education

real-world constraints that created special challenges for an implementation of a functional language or for functional programming in general

An Experience Report is distinguished from a normal ICFP paper by its title, by its length, and by the criteria used to evaluate it.

Both in the proceedings and in any citations, the title of each accepted Experience Report must begin with the words “Experience Report” followed by a colon. The acceptance rate for Experience Reports will be computed and reported separately from the rate for ordinary papers.

An Experience Report is at most six pages long. Each accepted Experience Report will be presented at the conference, but depending on the number of Experience Reports and regular papers accepted, authors of Experience reports may be asked to give shorter talks.

Because the purpose of Experience Reports is to enable our community to accumulate a body of evidence about the efficacy of functional programming, an acceptable Experience Report need not add to the body of knowledge of the functional-programming community by presenting novel results or conclusions. It is sufficient if the Report states a clear thesis and provides supporting evidence. The thesis must be relevant to ICFP, but it need not be novel.

The program committee will accept or reject Experience Reports based on whether they judge the evidence to be convincing. Anecdotal evidence will be acceptable provided it is well argued and the author explains what efforts were made to gather as much evidence as possible. Typically, more convincing evidence is obtained from papers which show how functional programming was used than from papers which only say that functional programming was used. The most convincing evidence often includes comparisons of situations before and after the introduction or discontinuation of functional programming. Evidence drawn from a single person’s experience may be sufficient, but more weight will be given to evidence drawn from the experience of groups of people.

An Experience Report should be short and to the point: make a claim about how well functional programming worked on your project and why, and produce evidence to substantiate your claim. If functional programming worked for you in the same ways it has worked for others, you need only to summarize the results—the main part of your paper should discuss how well it worked and in what context. Most readers will not want to know all the details of your project and its implementation, but please characterize your project and its context well enough so that readers can judge to what degree your experience is relevant to their own projects. Be especially careful to highlight any unusual aspects of your project. Also keep in mind that specifics about your project are more valuable than generalities about functional programming; for example, it is more valuable to say that your team delivered its software a month ahead of schedule than it is to say that functional programming made your team more productive.

If your paper not only describes experience but also presents new technical results, or if your experience refutes cherished beliefs of the functional-programming community, you may be better off submitting it as a full paper, which will be judged by the usual criteria of novelty, originality, and relevance. If you are unsure in which category to submit, the program chair will be happy to help you decide.

Submission and Reviewing FAQ

This document is adapted (with permission and changes) from
Mike Hicks and David Walker's double-blind reviewing FAQ for POPL 2012
and 2015 (see
the POPL'12
Program Chair's Report for how the process worked out). Eijiro
Sumii is responsible for all the contents of this version.

General

A: Our goal is to give each a reviewer an unbiased
"first look" at each paper. Studies have shown that a reviewer's
attitude toward a submission may be affected, even unconsciously, by
the identity of the author (see link below to more details). We want
reviewers to be able to approach each submission without such
involuntary reactions as "Barnaby; he writes a good paper" or "Who are
these people? I have never heard of them." For this reason, we ask
that authors to omit their names from their submissions, and that they
avoid revealing their identity through citation. Note that many
systems and security conferences use double-blind reviewing and have
done so for years (e.g., SIGCOMM, OSDI, IEEE Security and Privacy,
SIGMOD). POPL and PLDI have done it for the last several years.

A key principle to keep in mind is that we intend this process to
be cooperative, not adversarial. If a reviewer does discover an
author's identity though a subtle clue or oversight the author will
not be penalized.

For those wanting more information, see the list of studies about
gender bias in other fields and links to CS-related articles that
cover this and other forms of bias below.

A: Studies of blinding with the flavor we are
using show that author identities remain unknown 53% to 79% of the
time (see Snodgrass, linked below, for details). Moreover, about
5-10% of the time (again, see Snodgrass), a reviewer is certain of the
authors, but then turns out to be at least partially mistaken. So,
while sometimes authorship can be guessed correctly, the question is,
is imperfect blinding better than no blinding at all? If author names
are not explicitly in front of the reviewer on the front page, does
that help at all even for the remaining submissions where it would be
possible to guess? Our conjecture is that on balance the answer is
"yes".

A: I have heard of this happening, and this is
indeed a serious issue. In the approach we are taking for ICFP'16,
author names are revealed to reviewers after they have submitted their
review. Therefore, a reviewer can correct their review if they indeed
have penalized the authors inappropriately. Unblinding prior to the
PC meeting also avoids abuses in which committee members end up
advancing the cause of a paper with which they have a conflict.

For authors

A: Your job is not to make your identity
undiscoverable but simply to make it possible for our reviewers to
evaluate your submission without having to know who you are. The
specific guidelines stated in the call for papers are simple: omit
authors' names from your title page (or list them as "omitted for
submission"), and when you cite your own work, refer to it in the
third person. For example, if your name is Smith and you have worked
on amphibious type systems, instead of saying "We extend our earlier
work on statically typed toads (Smith 2004)," you might say "We extend
Smith's (2004) earlier work on statically typed toads." Also, be sure
not to include any acknowledgements that would give away your
identity. If you have any questions, feel free to ask the PC
chair.

A: On the submission site there will be an option
to submit supplementary material along with your main paper. This
supplementary material may or may not be anonymized; if not
anonymized, it will only be revealed to reviewers after they have
submitted their review of your paper and learned your identity.
Reviewers are under no obligation to look at this material. The
submission itself is the object of review and so it should strive to
convince the reader of at least the plausibility of reported results;
supplemental material only serves to confirm, in more detail, the idea
argued in the paper. Of course, reviewers are free to change their
review upon viewing supplemental material (or for any other reason).
For those authors who wish to supplement, we encourage them to mention
the supplement in the body of the paper so reviewers know to look for
it, if necessary. E.g., "The proof of Lemma 1 is included in the
anonymous supplemental material submitted with this paper."

A: Yes, submit it via HotCRP. Authors have been
known to release a TR, code, etc. via an anonymous hosting service,
and to include a URL to that material in the paper. However, we
discourage authors from using such tactics except for materials that
cannot, for some reason, be uploaded to the official site (e.g., a
live demo). We emphasize that authors should strive to make their
paper as convincing as possible within the submission page limit, in
case reviewers choose not to access supplemental material. Also, see
the next question.

A In general, we discourage authors from providing
supplementary materials via links to external web sites. It is
possible to change the linked items after the submission deadline has
passed, and, to be fair to all authors, we would like to be sure
reviewers evaluate materials that have been completed prior to the
submission deadline. Having said that, it is appropriate to link to
items, such as an online demo, that can't easily be submitted.
Needless to say, attempting to discover the reviewers for your paper
by tracking visitors to such a demo site would be a breach of academic
integrity. Supplementary items such as PDFs should always be uploaded
to HotCRP.

A: No. The relationship between systems and
authors changes over time, so there will be at least some doubt about
authorship. Increasing this doubt by changing the system name would
help with anonymity, but it would compromise the research process. In
particular, changing the name requires explaining a lot about the
system again because you can't just refer to the existing papers,
which use the proper name. Not citing these papers runs the risk of
the reviewers who know about the existing system thinking you are
replicating earlier work. It is also confusing for the reviewers to
read about the paper under Name X and then have the name be changed to
Name Y. Will all the reviewers go and re-read the final version with
the correct name? If not, they have the wrong name in their heads,
which could be harmful in the long run.

A: No. But we recommend you do not use the same
title for your ICFP submission, so that it is clearly distinguished
from the prior paper. In general there is rarely a good reason to
anonymize a citation. One possibility is for work that is tightly
related to the present submission and is also under review. But such
works may often be non-anonymous. When in doubt, contact the PC
Chair.

A(last updated February 8, 2016): As far as the authors' publicity actions are
concerned, a paper under double-blind review is largely the same as a
paper under regular (single-blind) review. Double-blind reviewing
should not hinder the usual communication of results.

That said, we do ask that you not attempt to deliberately subvert
the double-blind reviewing process by announcing the names of the
authors of your paper to the potential reviewers of your paper. It is
difficult to define exactly what counts as "subversion" here,
but generally speaking please refrain from sending individual e-mail
to members of the PC or ERC about your work (unless they are
conflicted out anyway), posting mail to a major mailing list (e.g.
TYPES) announcing your paper, or posting about it on social media.
On the other hand, it is perfectly fine, for example, to visit other
institutions and give talks about your work, to present your submitted
work during job interviews, to present your work at professional
meetings (e.g. Dagstuhl), or to post your work on your web page. In
general, PC/ERC members will not be asked to recuse themselves if they
discover the (likely) identity of an author through such means. If
you're not sure about what constitutes "subversion", please consult
directly with the Program Chair.

A: Using DBR does not change the principle that
reviewers should not review papers with which they have a conflict of
interest, even if they do not immediately know who the authors are.
Quoting (with slight alteration) from
the ACM SIGPLAN
review policies document:

A conflict of interest is defined as a situation in which the
reviewer can be viewed as being able to benefit personally in the
process of reviewing a paper. For example, if a reviewer is
considering a paper written by a member of his own group, a current
student, his advisor, or a group that he is seen as being in close
competition with, then the outcome of the review process can have
direct benefit to the reviewer's own status. If a conflict of
interest exists, the potential reviewer should decline to review the
paper.

As an author, you should list PC and ERC members (and any others,
since others may be asked for outside reviewers) which you believe
have a conflict with you. While particular criteria for making this
determination may vary, please apply the following guidelines,
identifying a potential reviewer Bob as conflicted if

Bob was your co-author or collaborator at some point within the
last 2 years

Bob is an advisor or advisee of yours

Bob is a family member

Bob has a non-trivial financial stake in your work (e.g.,
invested in your startup company)

Also please identify institutions with which you are affiliated;
all employees or affiliates of these institutions will also be
considered conflicted.

If a possible reviewer does not meet the above criteria, please
do not identify him/her as conflicted. Doing so could be
viewed as an attempt to prevent a qualified, but possibly skeptical
reviewer from reviewing your paper. If you nevertheless believe that
a reviewer who does not meet the above criteria is conflicted, you may
identify the person and send a note to the PC Chair.

For reviewers

A: If at any point you feel that the authors'
actions are largely aimed at ensuring that potential reviewers know
their identity, you should contact the Program Chair. Otherwise you
should not treat double-blind reviewing differently from regular blind
reviewing. In particular, you should refrain from seeking out
information on the authors' identity, but if you discover it
accidentally this will not automatically disqualify you as a reviewer.
Use your best judgment.

A: PC and ERC members should do their own reviews,
not delegate them to someone else. If doing so is problematic for
some papers, e.g., you don't feel completely qualified, then consider
the following options. First, submit a review for your paper that is
as careful as possible, outlining areas where you think your knowledge
is lacking. Assuming we have sufficient expert reviews, that could be
the end of it: non-expert reviews are valuable too, since conference
attendees are by-and-large not experts for any given paper. Second,
if you feel like the gaps in your knowledge are substantial, submit a
first cut review, and then work with the PC chair to solicit an
external review. This is easy: after submitting your review the paper
is unblinded, so you at least know not to solicit the authors! You
will also know other reviewers of the paper that have already been
solicited. If none of these expert reviewers is acceptable to you,
just check with the PC Chair that the person you do wish to solicit is
not conflicted with the authors. In addition, the PC chair will
attempt to balance the load on external reviewers. Keep in mind that
while we would like the PC to make as informed a decision as possible
about each submitted paper, each additional review we solicit places a
burden on the community.

As a last resort, if you feel like your review would be extremely
uninformed and you'd rather not even submit a first cut, contact the
PC Chair, and another reviewer will be assigned.

A: Having students (or interns at a research lab)
participate in the review process is good for their education.
However, you should not just "offload" your reviews to your students.
Each review assigned to you is your responsibility. We recommend the
following process: If you are sure that your student's conflicts of
interest are a subset of your own, you and your student may both begin
to do your own separate reviews in parallel. (A student's review
should never simply be a substitute for your own work.) If your
student's conflicts of interest are not a subset of your own, you may
do your own first-cut review first and then unblind the authors so you
can check, or you may consult with the PC chair. Either way, once the
student has completed their review, you should check the review to
ensure the tone is professional and the content is appropriate. Then
you may merge the student's review with your own.

A: The conference review system will ask that you
identify conflicts of interest when you get an account on the
submission system. Please see the related question
applied to authors to decide how to identify conflicts. Feel free
to also identify additional authors whose papers you feel you could
not review fairly for reasons other than those given (e.g., strong
personal friendship).

A: PC submissions are allowed (except for the PC
chair) with the usual condition of a higher standard (clearly as good
as or better than other accepted papers, that is, strong support and
no significant doubt). At the physical PC meeting, they will be
discussed (possibly with external reviews) after all the other papers,
only by PC members who have not submitted a paper.

A: The scope of ICFP is broad and encompasses all
topics that pertain to functional programming. Hence, if you feel a
paper would be an excellent POPL (or PLDI or OOPSLA) paper then it
might also be an excellent ICFP paper. To be accepted at ICFP, a
paper must discuss functional programming in some way, shape or form
and it must have the potential to make a lasting impact on our
field.

A: The scope of ICFP is broad and encompasses all
topics that pertain to functional programming. However, if you
discover you have been assigned a paper that does not contribute to
the study of functional programming, please contact the program chair.
We will discuss it and may decide to reject the paper on grounds of
scope. Of course, if we decide after all that the paper is within the
scope of ICFP, you should review it like any other paper.

Here are a few studies on the potential effects of bias manifesting
in a merit review process, focusing on bias against women. (These
were collected by David Wagner.)

There's the
famous story
of gender bias in orchestra try-outs, where moving to blind
auditions seems to have increased the hiring of female musicians by
up to 33% or so. Today some orchestras even go so far as to ask
musicians to remove their shoes (or roll out thick carpets) before
auditioning, to try to prevent gender-revealing cues from the sound
of the auditioner's shoes.

One study
found bias in assessment of identical CVs but with names and genders
changed. In particular, the researchers mailed out c.v.'s for a
faculty position, but randomly swapped the gender of the name on
some of them. They found that both men and women reviewers ranked
supposedly-male job applicants higher than supposedly-female
applicants -- even though the contents of the c.v. were identical.
Presumably, none of the reviewers thought of themselves as biased,
yet their evaluations in fact exhibited gender bias. (However: in
contrast to the gender bias at hiring time, if the reviewers were
instead asked to evaluate whether a candidate should be granted
tenure, the big gender differences disappeared. For whatever that's
worth.)

The Implicit
Association Test illustrates how factors can bias our
decision-making, without us realising it. For instance, a large
fraction of the population has a tendency to associate men with
career (professional life) and women with family (home life),
without realizing it. The claim is that we have certain gender
stereotypes and schemas which unconsciously influence the way we
think. The interesting thing about the IAT is that you can take it
yourself. If you want to give it a try, select the Gender-Career
IAT or the Gender-Science IAT
from here.
There's evidence that these unconscious biases affect our behavior.
For instance, one study of recommendation letters written for 300
applicants (looking only at the ones who were eventually hired)
found that, when writing about men, letter-writers were more likely
to highlight the applicant's research and technical skills, while
when writing about women, letter-writers were more likely to mention
the applicant's teaching and interpersonal skills.

This study
reports experience from an ecology journal that switched from
non-blind to blind reviewing. After the switch, they found a
significant (~8%) increase in the acceptance rate for
female-first-authored submissions. To put it another way, they saw
a 33% increase in the fraction of published papers whose first
author is female (28% -> 37%). Keep in mind that this is not a
controlled experiment, so it proves correlation but not causation,
and there appears to be controversy in the literature about the
work. So it as at most a plausibility result that gender bias could
be present in the sciences, but far from definitive.