A few remarks on Matthew Strassler’s “Flesh and Blood with Multi-Muons” November 17, 2008

[I know, I know… I had promised that today I would issue a fourth installment of my multi-threaded post on the multi-muon analysis, and instead this morning (well, that depends where you’re sitting) I am offering you something slightly different: instead than concrete details on the analysis, here is a review of a review of the same. I trust you understand that blogs, like newspapers or magazines, have their own priority lists…]

Last evening I read with a mixture of interest and surprise the paper recently appeared on the Arxiv by Matthew Strassler, a theorist from Rutgers University, and a supporter of so-called “hidden valley” models of physics beyond the Standard Model.

The interest stems from obvious reasons: after CDF published the study on multi-muon events, any discussion of the effect, as much as any tentative explanation -be it a mundane or an exotic one- is worth my undivided attention. And, mind you, let me say from the outset that I salute professor Strassler’s thoughts and considerations as useful and stimulating, and the mechanisms he suggests promising avenues for further research on the subject.

But there’s room for surprise, and not all of it is of pleasant nature.

Some of the surprise comes from a few of the remarks contained in the 20-pages document, and some comes from the way it is written. More on the remarks below, while about the way it is written I can say off-hand that I should probably be grateful to theorists these days, since they have started to make their papers free of complicated formulas, at the expense of a rather large rate of unnecessary adjectives: Strassler’s paper has indeed a remarkable formula count of zero.

In general I feel surprised by reading in an Arxiv paper something one usually finds in a blog: a list of ideas and questions concerning a paper published by a respectable scientific collaboration. It looks like prof. Strassler does not have a blog, and so he uses the Arxiv as a dump of his train of thoughts. Incidentally, this blog is of course open to him for a guest post, if he ever wants to try this kind of arena for his ideas.

I guess my criticism on the style boils down to this: it seems less productive to write an Arxiv paper containing a list of ideas and questions -and quite a bit of criticism-, than just picking up the phone and call the authors of the analysis, as I am told many other theorists are doing these days. No, he apparently has not made the phone call yet. That is quite unfortunate, because if he had he would maybe have learned a thing or two about the CDF analysis beyond what is published, and he would have had a chance to find an answer to some of his questions. Then, his ideas might have gotten some useful input and could have been refined. In his paper, instead, they sometimes read like a laundry list (check for instance pages 18-19, where he has seven bullets of plots he asks CDF to produce).

In his preprint Strassler mentions repeatedly that the multi-muon paper is written by “a subset of the CDF collaboration“. It appears that he stresses this fact on purpose, as if it is a datum of scientific importance. Fortunately he does not go as far as to claim that his observation casts doubt on the results, but his lingering on the issue appears strange, and to me, inappropriate. Calling our publication “a paper by a subset of the CDF collaboration” is plain wrong, because the paper is by the CDF collaboration, regardless of who signs it. The collaboration is one, and it is more than a collection of individuals: it admits no subset. I know theorists are much more promiscuous in the way they associate and disperse in different author lists; but a collaboration is a collaboration, and once a member, you only get to decide whether to sign or not a paper, but the collaboration publishes, not you.

This matter is important, so maybe I need to stress it once more. Let me remind everybody that the multi-muon analysis is a CDF publication, and that the CDF collaboration stands by this paper just as much as it stands by every other one of the half thousand it has published in its long, illustrious life. Signing a CDF paper is a great privilege, and since prof. Strassler does not know personally all of the people in CDF (I, for one, never had the pleasure to meet him), nor does he know about the internal discussions that have taken place concerning the publication, he should be expected to leave this issue aside, lest he gives the impression of discussing matters he is wholly unqualified to discuss. This impression is set from the very beginning in Strassler’s preprint, and remains in the background throughout its 20 pages, resonating in a few specific spots.

Let me now go into the contents of the “flesh and blood” paper very briefly. I cannot discuss all of it here today, but I will make an attempt at showing a couple of further examples of what I do not like in it, thereby creating a biased view of my overall opinion: the parts I liked will be left out of this post. Also, in the process of showing what I object to, I will be quoting out of context: a rather reproachable conduct, I must admit, but I have no real choice if I want to make this post shorter than the paper it deals with.

So here is the very incipit of the Introduction:

“Very recently, an unknown subsetof the CDF collaboration has signed its name to one of particle physics’ most extraordinary papers”.

Well, after thanking prof. Strassler for the unnecessary, improbable adjective, one is left wondering whether he can compute the ratio of small integers, like 370/600. But, at least until we get to read about his cross section estimates, we prefer to grant that he can, and so we have to hypothesize that maybe, by “unknown subset” he means to say he does not know the 370 authors who signed the “extraordinary paper”. Paraphrasing Oscar Wilde, “To not know an experimentalist is an accident; to not know 370 is carelessness“. But Strassler does know at least two CDF members: these are two of his Rutgers colleagues, who in fact get thanked in the concluding lines of his paper. Unfortunately, they did not sign the CDF publication. From this one might be tempted to speculate that Strassler only got to hear comments and internal information biased in a particular direction…

Yet prof. Strassler is quite clear to state from the outset he is very interested in the CDF analysis:

“No one would be happier than the author of the present note if this “suggestion of evidence” were to hold up under scrutiny”.

I omit discussing whether I find acceptable or not the way he interprets as a “suggestion of evidence” the conclusions of the CDF study, but I cannot fail to explain that he should rather take a ticket and join the line of happy scientists cheering the discovery of new physics, than single out himself as the one. This is a small bit of immodesty which however, after having noted it, I think we should pardon, given that he has indeed worked on hidden valley models for a long time.

We can also pardon him for saying that the paper is “too short given its potential importance“, right in the next paragraph. On this one count, I think he really manages to stand out of the crowd head and shoulders: of all the comments I have heard about the CDF paper, none went so far as to say that the 70 pages were too few.

Then, a sentence I am still trying to decypher:

“No serious attempt is made to interpret the data. This exercise may well be helpful […] even if the specific results of [1] (and a related attempt at an intepretation by the experimentalists involved [11]) are eventually discredited.”

Does Strassler mean to say that the study in [11] (the interpretation of multi-muon events, by the original authors of the study) was unserious ? Or does he rather mean it is useful to put together interpretations of similar effects even if they end up straight in the waste bin ? That would justify the career of a lot of theorists…

After the above sentences, which are contained in the introduction, we find section II, which is called “Preliminary comments“. Here I am puzzled to find Strassler’s paper wrestling with the number of events quoted in the CDF publication, reaching odd conclusions. Strassler incorrectly quotes 75 picobarns as the cross-section for ghost events: a number which comes out of the blue, and for which my explanation is the following: he uses the number of ghost events, “153895” as he quotes (forgetting this number refers to the subset of “ghost events” passing loose SVX criteria, but of that I can pardon him, he has a thing with subsets), and he assumes this corresponds to 2.1 inverse femtobarns of data. Then, would do the trick: 150k divided by 2k inverse picobarns is indeed 75 picobarns . Is this what he computed ? Well, it is wrong, since the luminosity corresponding to the 153,895 events is 742 inverse picobarns, and not 2.1/fb. See, this is one of the many instances when one cannot help noticing that a phone call before submitting to the Arxiv would have been a good idea. Cross section estimates are best left with experimentalists, otherwise what will we do for a living ?

Also odd is his following remark:

“if the efficiency estimate were in error for a subclass of events, and the efficiency were only, say, 23.4 percent, then the number of ghost events would drop by 1/5″.

Now, please. CDF publishes a paper, it quotes an efficiency (24.4+-0.2%), and it estimates an excess. What do we get if a theorist, albeit a distinguished one, ventures to say that if the efficiency were wrong (by 5-sigma from the quoted value), the excess would be significantly different ? I miss the scientific value of that sentence. Wait, there is more: only a paragraph below he insists:

“For these two reasons, we must view the number of unexplained ghost events as highly uncertain”.

Excuse me: we own the data, we publish an estimate, we give a uncertainty. You may well question whether it is correct or not, but simply saying an estimate is “highly uncertain” without coming down to explain what mechanisms may have caused an error in the CDF determination of the efficiency, is not constructive criticism, and is rather annoying. Not to mention that the CDF publication where the ingredients for the determination of that efficiency were measured is not quoted in Strassler’s paper!

Ok, I think I have done enough commenting for today. To conclude this post, I will quote without commentary a few sentences which I find peculiar. I have to say it: while the CDF paper is not the clearest I have had the pleasure to sign, I feel the need to stand by it when I see it attacked by non-constructive criticism.

“…the paper[…] is far too short given its potential importance, and many critical plots that could support the case are absent”.

“No serious attempt is made to interpret the data”.

“It is not clear why these checks were not performed”.

“There are a number of other plots whose presence, or absence, in Appendix B of [1] is very surprising. In particular, though obviously presented so as to support the interpretation of [11], the plots in Appendix B do not actually appear to do so.”

“…the challenges that this analysis faces are useful as a springboard for discussion. Clearly, if there were a signal of this type in the data, it would indeed by quite difficult to find it, and the approach used in [1] is far from optimal.”

After this list of less-than-constructive comments, let me quote Freeman Dyson for a change:

“The professional duty of a scientist confronted with a new and exciting theory (or data) is to try to prove it wrong. That is the way science works. This is the way science stay honest. Criticism is absolutely necessary to make room for better understanding.”

Am I the only one to think Dyson meant constructive criticism ?

UPDATE:version 2 of Strassler’s paper came out on November 17th, a week after version 1. This new version makes no mention at all of the “subset” of CDF authors. I thank Matthew Strassler for realizing this correction was useful.

OK, Tommaso, this is quite amusing, but it seems that you are taking the criticism with a bit too much sensitivity, as noted already above. Mind you, I think anybody outside CDF who spends less than 2 months thinking about this before coming to any conclusions probably deserves criticism. Oh, hang on a minute, that includes me. Actually, it probably includes a fair fraction of high energy theorists – most of whom are no doubt alarmingly ignorant of the inner workings of CDF. Forgive us if we deduce from your 1% estimate for new physics that the HIDDEN CDF evidence does not radically alter the present conclusions.

Kea, please stop thinking in terms of “we” and “you” and try to understand. I am not criticizing Strassler for not understanding that a dataset is 0.7/fb rather than 2.1/fb: the paper is indeed not too clear about it. I understand that the paper is hard to digest -it is not easy for me to read, either. Strassler however was helped -for his own admission- by two CDF physicists. If one browses the author lists of the CDF publication, one sees that they did not sign it!

So, I am criticizing him for publishing a preprint where he pretends to understand what we (experimentalists) do not understand of our data, and explain it to us. What to get, otherwise, from his fiddling with the efficiency of the SVX selection (the efficiency for charged tracks to be accepted by a very complicated set of hit requirements inside the inner silicon layers, as reported and measured by CDF), hypothesizing that it is wrong by five sigmas ?

As I wrote in the post, I did not discuss the parts of the paper I liked. Indeed, the paper is valuable where it considers different potential signatures of hidden valley modes which might result in multiple muons. It is unfortunate that it casts discredit on the CDF publication with some random remarks.

well… Lubos called me a small Peter Woit once. Now your comment makes me think I’ve accomplished a perfect mutation, combining the virtues (and the faults) of those two successful bloggers. I must feel flattered!

Jokes aside, my “mistake” with Nima was to say what I thought, and I still think. I do not consider it a mistake, although I understand that the way I live my life may be regarded as uncautious, that my career may be damaged, that I do not know how to “pick my battles”, etcetera. Having a blog is valuable to me, but I would never keep it to write in a constrained manner. My limits are decency, avoiding libel causes, saying what I think, and truth. Plus, of course, enjoying the whole thing in the process.

The first time I meet Nima, I will offer to shake hands, and if he is as intelligent as he looks he will take the offer. It is not a tragedy.

With Strassler I will do the same, but please understand that there is quite a difference in the two cases. Here we are discussing a paper which I, as well as many of my colleagues, found a bit arrogant in some specific spots, bordering the offensive. I do not mind: it happens. But I responded in kinds. Game even, let’s move on.

the comment that you are another Lubos is perhaps more flattering than anything I can offer you today but let me try. ;-)

As we learned on the USLHC blogs, we would surely agree about many sociological points concerning the CDF analysis, for example that the standards of verification should be equal regardless of the “sign” of the paper, i.e. whether it confirms someone’s expectations or not.

On the other hand, I am puzzled by your irritation by Matt’s comments and thinking.

I find it completely plausible that those 70 pages just don’t look enough to Matt as a justification of analysis of a statement that is potentially as profound as the CDF statement. There are shorter papers but there are also much longer papers. I can imagine that a similar result, even if true and important, could be announced and explained on 3 pages. But I can also imagine that it is all about the technicalities and Matt is right that those 70 pages just don’t contain enough stuff for him because his interest in the details exceeds yours. It is just a fraction of a page per collaborator, after all.

Doesn’t Matt – as a theorist – have the right to decide how much information is enough for him? You know, I am all for publishing papers even if they look shocking, as long as they survived the standard scrutiny. On the other hand, I disagree that a single paper should automatically cause a paradigm shift and make all theorists instantly believe the strongest possible interpretation of the paper.

The theorists like Matt also have the right to think that some extra checks should have been done, some graphs should have been added, and that some attempts to explain the data don’t seem serious. I can’t imagine how you could possibly prevent them from thinking so. You sound like the Inquisition.

Also, the CDF may respect its internal policies what is meant by the “CDF collaboration paper” but that doesn’t mean that everyone else must pretend that additional information doesn’t exist if it does. If Matt knows that the paper was signed by a subset of the collaboration only and if he finds this fact relevant, I just don’t think he should be expected to work hard to keep some secrecies and preserve taboos. Pretending that the paper is a paper of “everyone” is just about your politics: but it is not the underlying truth.

Even more importantly, I find it crazy for you to doubt Matt’s obvious strategy in looking for the reasons of the apparent discrepancy. If this surprising result may be explained by some (a small number) of the measured numbers to be 5 sigma away from the announced central value, it is actually reasonable to think that this explanation is the most likely one. The prior probability of theories that would explain all the data as new, real physics is much smaller than the probability than one number is over 5 sigma from the announced value.

You know, experimenters are only people and statistics happens, too, right? Do I have to enumerate you dozens of big-statement experiments that have been wrong in the history? Your comment that the experimenters have the monopoly to interpret their papers and to judge the certainty of their statements is utterly absurd. Experimenters, much like theorists, have just done some work that can be judged by other people, too. Other people may find both value of it as well as problems with it.

Clearly, something “not quite standard” is needed to explain the CDF data, but what the “not quite standard thing” is remains an open question, and an imperfect piece of experimenters’ work is clearly one of the most likely scenarios that will ultimately explain it. You’re both arrogant and ludicrous if you want to deny this possibility.

[…] to the thread of the former post, where, out of the blue, Lubos starts an attempt at explaining why Strassler’s estimate of the cross section of “ghost events” in the recent CDF publication is right, and I am […]

Tommaso,
Just to let you know, most of the very highly respected phenomenologists whose opinions I have heard regarding this CDF muon anomaly basically say flat out that the result is ‘crap’. A main reason for this attitude is, among other things, that one-third of the collaboration did not sign their name.

Just why would someone in the collaboration not sign—either because they didn`t contribute to it, or else they disagreed with the conclusions, or were so unsure of them that they didn`t want to be embarrassed.
I wonder if Strassler`s two colleagues at Rutgers fit the latter description.

I have no problem with the fact that phenomenologists regard as crap the CDF study. It is their problem, not mine. They obviously cannot judge by themselves whether the excess found by CDF is a background or not, and so they sheepishly rely on hints like the fact that there has been a controversy in the publication process. I do not blame for that, and besides, as i wrote several times in this blog, it is quite likely that the excess is a background which CDF has so far been unable to track down completely.

In any case, the paper needed to be published, because it affects previous measurements in the B sector, already published by CDF.

Concerning the author list, most of those who did not sign did so because they wanted more studies -which would take of the order of six months- before publishing; the majority of the collaboration instead wanted to put this out first.
Others chose not to sign because they did not get proper answers by the authors. Others still because they did not understand some detail of the analysis. Others because they are strong enemies of the main author. One did not sign because of a failed sentimental relationship with one of the authors…. What do your respected phenomenologists make of this ? I am curious.

experimentalists may have a better shot at understanding the critical points of the CDF analysis. However, if they say the study is “crap”, they are not serious, or they have a biased judgement. The problem is that they did not read carefully enough the reasons, stated in the abstract and in the conclusions, why CDF decided to published a study which is not complete. These muons exist, and whatever background they are, they are not coming from B decays. So several measurements published by CDF in the past are affected. Is that clear enough ?

[…] of which is an excellent starting point for multi-muon model-building), which in-turn produced a response on Tommaso’s blog… which eventually turned a bit ugly in the comments section. Anyway, the best […]

it seems like maybe Matt was insinuating that his colleagues at Rutgers did not sign the paper because they disagreed with the results – that would explain the subset comment.

About writing papers with no formulae – Matt is a child of the 1990’s string revolution even if he has kept his interest in particle physics. Formulae or no formulae we stringers got used to papers with no meaningful content. This is relevant in that stringers sometimes think the standards for publication are just as low in other fields of physics and science in general – making Matt believe other people might have been sloppy.

Hmmm k, I do not want to get into more arguing about this… Time has passed (more than 1 year in fact). I exchanged email with him, he was not very happy about my blog post, asked me to write about the good points (which I mention) and not just the bad ones of his article. I gave up…

I remain convinced that he wrote the paper in an impulse, which explains the lower standards.
Cheers,
T.