CMS looking back

The CMS collaboration have kindly posted a pleasant video that reveals the moments when they “unblinded” their Higgs diphoton results within the collaboration in the run-up to the public discovery announcement in July.

I find it interesting to look back and see how these events relate to what was going on publicly on the blogs at the time. From the video we learn that the CMS collaboration were shown the first results on 15th June. The Higgs analysis group within CMS must have seen it at least a day or two before in order to prepare the plots for the talk. We can assume that ATLAS were seeing their results at about the same time. For two days the collaborations were able to walk around knowing that they knew stuff that the outside didn’t until Peter Woit blew the lid with his leak about the new results. This was a little upsetting for them at first as shown by the response from CMS blogger Michael Schmitt who later calmed down a bit. One argument they gave was that they did not want the information to pass between the ATLAS and CMS collaborations because it would spoil their independence but if the rumour can reach the outside world so quickly it is clear that it would not be kept secret at CERN and other institutions where 3000 excited physicists from each collaboration share the same social spaces.

The rumours were saying that the diphoton excess was at around 4 sigma and the video shows that it was indeed around 4.3 for CMS. In my own analysis the next day I estimated that this was based on about 3/fb of the data which turns out to be exactly right for CMS as seen in the video when the camera zooms in at 2:20. I also pointed out that when they add the full dataset the signal could easily go down and in fact it did descend to 4.1 sigma as seen in the next part of the video. I am not always right but I was this time. Subsequently the New York Times reported an email from the spokesperson for ATLAS saying that they should not believe the blogs. Now we know that this was a euphemism for “please don’t report what they are saying because it is perfectly accurate and we were hoping to keep it as a surprise for the next conference”.

Another point worth making here is that the collaborations like to make big statements about how they do their analysis blind. This is supposed to mean that they don’t look at the results until they have fixed the parameters of the analysis so that they cannot introduce any bias. From this video we can see what this really means in practice. They unblind the data as an early check then they “re-blind” it while they adjust the analysis. Then they unblind it again two weeks later with just 30% more data added. Come-on guys own up, this is not quite in the spirit of how blind analysis is meant to work. Luckily the signal is so clear that it is indisputable in any case.

Getting more up-to-date, remember that CMS have not yet published the diphoton update with 13/fb at 8 TeV. Rumours revealed that this was because the excess had diminished. At the Edinburgh Higgs symposium some more details about the situation were given. The talks are not online but Matt Strassler who was there has told us that the results have now been deemed correct. It may be understandable that when the results are not quite what they hope for they will scrutinize them more carefully, but I find it wrong that they do not then publish the results once checked. It was clear that they intended to publish these plots at HCP2012 in November and would have done so if they showed a bigger excess. By not releasing them now they are introducing a bias in what is publicly known and theorists are left to draw conclusions based on the ATLAS results only which still show an over-excess in the diphoton channel. It will all be history once the final results with about 20/fb are released soon but it would be helpful if they could keep this sort of biasing factor to a minimum.

The March meeting at Moriond is slated as the occasion for the final update but only of they are happy with the results. Their analysis has been used and refined many times in the last two years and by now they should be confident enough to say that they will publish regardless of the result they get. The data for the last proton run was available before Christmas so by now the collaborations should have completed their final analysis. The fact that we don’t have any rumours suggests that this time they have decided to confine knowledge of the results to the smaller Higgs groups within the collaborations and they may actually succeed in keeping them secret until the conference.

Like this:

LikeLoading...

Related

This entry was posted on Wednesday, January 30th, 2013 at 11:04 am and is filed under Higgs Hunting, Large Hadron Collider. You can follow any responses to this entry through the RSS 2.0 feed.
Both comments and pings are currently closed.

22 Responses to CMS looking back

As to “… The March meeting at Moriond is slated as the occasion for the final update
but only of they are happy with the results …”.

nbcnews science said on 27 January 2013:
“Davos, Switzerland …
The world should know with certainty by the middle of this year
whether a subatomic particle … is a … Higgs boson …
Rolf Heuer, director of … CERN, said he is confident that
“towards the middle of the year, we will be there.” …”.

Does the “middle of the year” instead of Moriond mean
that CERN are NOT now “happy with the results” ?

Does the fact that Heuer was speaking to the economic leaders
of the world at Davos mean that he is trying to make sure that the European/WesternWorld economic mess will not kill the expensive repair/upgrade of the LHC ?

Firstly he is being conservative and does not want to put undue pressure on the collaborations. If he says July and the answer comes in March nobody will complain.

Secondly, showing that this is a Higgs Boson and not something else has been defined to mean that its spin and parity must match 0+. This is a different analysis from the usual measure of the decay rates and is something they have not done many times. The first public results for this were given in December I think. It is possible that we will get the decay rates at Moriond and the spin-parity later. It may also be that if the spin/parity results are not strong enough then there are some other more difficult ways they can do the analysis to get a better sensitivity which would then take a bit longer.

Having said that, my top bet is that they will have all the results from CMS and ATLAS individually by March but they will need to combine results for spin and parity to be able to claim proof of 0+ . A combination of decay rates might be possible by March but combinations for spin-parity could be delicate and they wont want to rush that.

One should not forget that what matters is the minimization of the mean square error, not the bias itself, which is just one of the two ingredients. The other is variance. If, by accepting a small bias, one can reduce the variance significantly – eg by further screening unblinded results when these look suspicious – one is doing good science.

I hardly dare to propose a non SM Higgs, that nature is equipped with a double “zero mass Higgs” particle(s) with 130 Gev opposite oscillatory energy able to create by collision two gluons travelling away as dual photons.
See: http://vixra.org/pdf/1112.0065v2.pdf
In this letter I present some possible transformations by the collision of (Non- SM) Higgs within Quark- Gluon clouds into decay results such as, gg into Di-Photons, ZZ into 4 Lepton or WW into LvLv

Regarding “Come-on guys own up, this is not quite in the spirit of how blind analysis is meant to work.”: you are the first to advocate frequent public results. Those are not possible unless we unblind and reblind several times.

Or should we analyse the data only once in a completely blind way? Like Tommaso very nicely pointed out, we pay the price of possibly having a small bias in order to hack at the variance and keep the community at large appraised.

Finally, regarding the HCP 2012 Higgs results, CMS made a very comprehensive release. By ignoring the very large body of work that was made public and focussing on one single channel that we want to scrutinise more closely, you really miss out on a lot of science.

I am happy that they provide frequent updates, but here we see that they do internal updates even more frequently. Again I dont think that is necessarily wrong. If they want to refine and reduce the mean square then that is a valid method. I just think they should not say that the data was blinded when it was revealed two weeks earlier with just a little less available.

I dont claim comprehensive reporting. I have concentrated mostly on the Higgs because I have limited time and prefer to look deeply at one point than give a broad view. Most of the results presented are negative. The overall picture this gives is interesting and I have discussed it, but reporting individual negative results makes for dull reading. Other bloggers report some stuff that I dont and I have no interest in repeating what has been well said elsewhere.

Oh, but by “large body of work” I was sticking to Higgs measurements.
My comment was geared at the focus on CMS not having an updated results in the γγ channel, glossing over the updated bb, ττ, WW and ZZ ones…

OK, I don’t believe this interpretation. “Rumors” may be an unreliably sounding description but it seems implausible to me that most of the people didn’t think they “had known” what the other team had given their unavoidable interactions.

What I am told by 3 people from the other detector may be fluke, a rumor that can be wrong, but Incandela’s or Gianotti’s talks may also be just a random conglomerate of photons that emerged from random thermal noise coming from the Sun on a beach. Still, below a certain probability threshold, no one considers these doubts “real”.

To me there is a very clear distinction between what is conveyed in indirect ways (rumour, gossip, etc) and information that is directly provided (presentation, publication, etc).
The foremost difference is the possiblity to discuss what I am being told.

Tommaso Dorigo describes the CMS video under discussion here as
“… a video by the CMS collaboration, where small clips of internal meetings are shown …”.

If “small clips” of CMS “internal meetings” were shown,
that seems to me to imply that
CMS internal meetings are routinely video-recorded.
It seems likely to me that ATLAS would follow a similar policy.

Aside from bureacratic control-freak tendency to secrecy,
is there any reason
why the full video-recordings of all LHC internal meetings
relevant to analysis of Higgs data are not being made available to the public who paid for the project ?

I dont think this implies that all meetings are videoed. I would love to see all the reports too but they need to be able to risk asking stupid questions at review meetings without having them made public I think.

Complete openness to the public would defeat the reason of having two experiments. The analysis HAS to be independent to get our money’s worth. Otherwise one should sequester the two analysis teams in monasteries with vows of not looking.

I think it is sad that LHC people would be afraid
“… to risk asking stupid questions at review meetings …”
if the meetings that were being video-recorded anyway
were to be “made public”.

As a lawyer I have to ask questions in public,
in front of judges and juries and witnesses and other lawyers
(who may be serious adversaries),
in order to try to get to the truth of the case.
Sometimes my questions maybe stupid and everybody
then sees how stupid I can be,
but that is part of the price I pay for participation
in the judicial search for truth,
and
I can say from experience that the more the judicial proces
is open (and not closed for secret hearings)
the more likely it is that the goal of truth will be obtained.

If the physicists at CERN are too cowardly to ask their questions in public view, shame on them.

In addition to defeating the purpose of two experiments, the spontaneity of group work would be destroyed by the knowledge of outside observers, just because some humans would play to the cameras and others would feel self conscious.These are brainstorming sessions in reality and need the spontaneity.

A large factor also is the desire of getting the recognition of peers for the twenty years of scientific labor invested in the experiment by many, certainly the seniors. They do not want to be preempted by some rash publication by an enthusiastic outsider, more so if the tooted plots turn out to be in error after second and third look. ( Many 3 sigmas have disappeared down the drain and some 4 sigmas turn out to be calibration flukes) The noise level would be enormous.

The complexity of the experiments is enormous too.Those 3000 people signing the papers are necessary because of it. Getting events, accumulating them and analyzing them needs the coherent collaboration of all these experts. The expertise cannot be transmitted over video.

The existing system of conferences, preprints and peer reviews imo guarantees that the public is getting its money’s worth in the end.