Hi Francesco,
I was in general pointing out that the rule-of-thumb of the Chebychev inequality helps to remember what sort of deviations can be expected in a set of measurements of any given size.
If the question is, "is a single value of this large a deviation evidence of departure from the input model?" I claim that the answer is "no". You could have at least two of these deviations before suspecting a difference from the SM.
Once it gets to 5 sigma, then the answer suddenly switches to "yes" since then the Chebychev would suspect <1 measurement to fall outside this range.
I was assuming that all the measurements together are measuring the same thing, that is "does the SM work", but if we formulate instead 24 different questions, "does the SM work here, there, over there, in this corner" etc., then the 0.5% is obviously very correct.
I think what's there in the paper is fine, but wonder always if it isn't stated too strongly,
This statement would be incorrect: "we find 0.5%, which means we are 99.5% confident the SM is wrong"
What we actually have is: 23 different measurements that the SM is perfectly fine, and one measurement which has a reasonable statistical deviation, given that we measure 24 things. So we are very certain the SM is correct.
Thanks,
Rob
------------------------------------------ Robert Lambert FOM-VU-NIKHEF-Bfys LHCb Email: rob.lambert@cern.ch ------------------------------------------ Nikhef N251 Tel: +31 20 592 2131 Fax: +31 20 592 5155 ------------------------------------------ CERN, 13-1-018 Tel: +41 22 767 4024 Fax: +41 22 766 8109 ------------------------------------------
________________________________________ From: Francesco Dettori [fdettori@nikhef.nl] Sent: 02 July 2013 12:43 To: bfys-physics@nikhef.nl Cc: Rob Lambert; Diego Martinez Santos; Marcel Merk Subject: Re: [Bfys-physics] B->K*mumu paper
Hi,
I actually do not agree. The 0.5% is the correct percentage to quote computed from the correct distribution. Chebychev inequality is more referring to the opposite: you would not expect that more than 1/k^2 would fall outside k sigmas. However that does not tell you how probably that is.
On the other side the physics importance is relevant: the SM cannot be excluded if you consider those measurements as independent. However those are not independent because they are sensitive to different coefficients.
my two cents, cheers Francesco
On Tuesday 02 July 2013 10:22:28 Rob Lambert wrote: |Hi again, | |Well, for me, concentrating on the 0.5% is over-stating the evidence. The SM |is still perfectly valid if one of 24 measurements shows a 3.7 sigma |discrepancy. Perhaps an added line (footnote?) with respect to the Chebychev |inequality would be prudent in the paper ("Using the rule-of-thumb of |Chebychev's inequality, 7% of the 24 measured values could be expected to |fall outside of the 3.7 sigma range, and thus our current results show no |significant inconsistency with the Standard Model."). | |Thanks, | |Rob | |------------------------------------------ |Robert Lambert |FOM-VU-NIKHEF-Bfys LHCb |Email: rob.lambert@cern.ch |------------------------------------------ |Nikhef N251 |Tel: +31 20 592 2131 |Fax: +31 20 592 5155 |------------------------------------------ |CERN, 13-1-018 |Tel: +41 22 767 4024 |Fax: +41 22 766 8109 |------------------------------------------ | |________________________________________ |From: Francesco Dettori [fdettori@nikhef.nl] |Sent: 02 July 2013 11:58 |To: Diego Martinez Santos |Cc: bfys-physics@nikhef.nl; Rob Lambert; Marcel Merk |Subject: Re: [Bfys-physics] B->K*mumu paper | |Ciao, | |yes I perfectly agree with you, I was just trying to summarise possible Rob |points :) | | |Francesco | |On Tuesday 02 July 2013 09:56:21 Diego Martinez Santos wrote: ||Ciao || ||I'm not sure they speak about "global tension", I mean, what they say ||is different from making a chi2 with 24 DoF. From the abstract: ||"Considering the 24 measurements as independent, the probability to observe ||such a discrepancy in one of them is 0:5%." || ||i.e, I guess they just mean that in 0.5% of experiments made of 24 ||indepenendent measurements you have at least a discrepancy of 3.7 s or more ||(and seem consistent with the toy I made). || ||Cheers; || ||Diego || ||________________________________________ ||Dende: Francesco Dettori [fdettori@nikhef.nl] ||Enviado: martes, 02 de xullo de 2013 11:46 ||Ata: bfys-physics@nikhef.nl ||Cc: Rob Lambert; Diego Martinez Santos; Marcel Merk ||Asunto: Re: [Bfys-physics] B->K*mumu paper || ||Ciao Rob, || ||I don't understand completely what is your point: ||- that they should not claim any discrepancy because globally the agreement ||is fine ||- or that the final claim of 2.8 sigma global tension is wrong. || ||cheers, ||Francesco || ||On Tuesday 02 July 2013 09:33:44 Rob Lambert wrote: |||Hi Again, ||| |||Another way to think about it is to fit a Gaussian to the 24 data points, |||and analyse whether one of them is an outlier or not. This is analytic |||given a Gaussian distribution with one outlier and reduces to the Chebychev |||inequality (or at least, I thought it did). ||| |||Thanks, ||| |||Rob ||| |||------------------------------------------ |||Robert Lambert |||FOM-VU-NIKHEF-Bfys LHCb |||Email: rob.lambert@cern.ch |||------------------------------------------ |||Nikhef N251 |||Tel: +31 20 592 2131 |||Fax: +31 20 592 5155 |||------------------------------------------ |||CERN, 13-1-018 |||Tel: +41 22 767 4024 |||Fax: +41 22 766 8109 |||------------------------------------------ |||________________________________ |||From: bfys-physics-bounces@nikhef.nl [bfys-physics-bounces@nikhef.nl] on |||behalf of Rob Lambert [Rob.Lambert@cern.ch] Sent: 02 July 2013 11:30 |||To: Diego Martinez Santos; Marcel Merk |||Cc: bfys-physics@nikhef.nl |||Subject: Re: [Bfys-physics] B->K*mumu paper ||| |||(24*0.0001*2) ||| ||| ^ missed a zero .... ||| |||Rob ||| |||------------------------------------------ |||Robert Lambert |||FOM-VU-NIKHEF-Bfys LHCb |||Email: rob.lambert@cern.ch |||------------------------------------------ |||Nikhef N251 |||Tel: +31 20 592 2131 |||Fax: +31 20 592 5155 |||------------------------------------------ |||CERN, 13-1-018 |||Tel: +41 22 767 4024 |||Fax: +41 22 766 8109 |||------------------------------------------ |||________________________________ |||From: Rob Lambert |||Sent: 02 July 2013 11:29 |||To: Diego Martinez Santos; Marcel Merk |||Cc: bfys-physics@nikhef.nl |||Subject: RE: [Bfys-physics] B->K*mumu paper ||| |||Hi Diego, ||| |||Well, I'm 100% sure that's right because it's just 24*0.001*2, which is the |||integrated probability for a two-sided Gaussian test, but the question |||isn't |||"what's the probability of that happening" which is actually "1", because |||it |||did actually happen, but "how many such deviations would you need to see |||before you can invalidate the null hypothesis of no deviation", and this |||number is given by Chebychev's equality... well, I thought it was, anyway. ||| |||Thanks, ||| |||Rob ||| |||------------------------------------------ |||Robert Lambert |||FOM-VU-NIKHEF-Bfys LHCb |||Email: rob.lambert@cern.ch |||------------------------------------------ |||Nikhef N251 |||Tel: +31 20 592 2131 |||Fax: +31 20 592 5155 |||------------------------------------------ |||CERN, 13-1-018 |||Tel: +41 22 767 4024 |||Fax: +41 22 766 8109 |||------------------------------------------ |||________________________________ |||From: Diego Martinez Santos |||Sent: 02 July 2013 11:17 |||To: Rob Lambert; Marcel Merk |||Cc: bfys-physics@nikhef.nl |||Subject: RE: [Bfys-physics] B->K*mumu paper ||| ||| |||(PS: 0.566 \pm 0.033. forgot the % of course) ||| |||________________________________ |||Dende: Diego Martinez Santos |||Enviado: martes, 02 de xullo de 2013 11:16 |||Ata: Rob Lambert; Marcel Merk |||Cc: bfys-physics@nikhef.nl |||Asunto: RE: [Bfys-physics] B->K*mumu paper ||| ||| |||Hi ||| ||| ||| |||I;ve just made quick toyMC, I get that the probability(maxSigma) > 3.7 for |||experiments consisting ||| |||of 24 measurements is 0.566 \pm 0.033. ||| ||| ||| ||| ||| |||Voila the code (needs SetupUrania): ||| ||| ||| |||from scipy import random as rnd |||from Urania import * |||AccessPackage("Bs2MuMu") ||| |||from RTuple import * ||| |||def do_set_of_experiments(N): ||| l = [] ||| ||| for i in range(N): ||| l.append(abs(rnd.normal())) ||| ||| l.sort() ||| l.reverse() ||| return l[0] ||| |||def do_test(N, Nexp = 50000): ||| tup = RTuple("File_test",["maxSigma/F"]) ||| ||| for i in range(Nexp): ||| tup.fillItem("maxSigma",do_set_of_experiments(24)) ||| tup.fill() ||| ||| tup.close() ||| |||Then you just call do_test(24) and analyse the produced Ntuple. ||| ||| ||| ||| ||| ||| ||| |||Cheers; ||| ||| ||| |||Diego ||| ||| ||| ||| ||| ||| ||| ||| ||| |||________________________________ |||Dende: bfys-physics-bounces@nikhef.nl [bfys-physics-bounces@nikhef.nl] en |||nome de Rob Lambert [Rob.Lambert@cern.ch] Enviado: martes, 02 de xullo de |||2013 10:53 |||Ata: Marcel Merk |||Cc: bfys-physics@nikhef.nl |||Asunto: Re: [Bfys-physics] B->K*mumu paper ||| |||Yes. Chebychev's inequality gives an estimate for how often a deviation |||this |||large this should happen, and it seems to be perfectly within the realms of |||probability, in fact it seems to be expected, and even two such deviations |||would not be strange. ||| |||Gerhard might want to correct me here, though... ||| |||Cheers, ||| |||Rob ||| |||------------------------------------------ |||Robert Lambert |||FOM-VU-NIKHEF-Bfys LHCb |||Email: rob.lambert@cern.ch |||------------------------------------------ |||Nikhef N251 |||Tel: +31 20 592 2131 |||Fax: +31 20 592 5155 |||------------------------------------------ |||CERN, 13-1-018 |||Tel: +41 22 767 4024 |||Fax: +41 22 766 8109 |||------------------------------------------ |||________________________________ |||From: Marcel Merk [marcel.merk@nikhef.nl] |||Sent: 02 July 2013 10:51 |||To: Rob Lambert |||Cc: bfys-physics@nikhef.nl |||Subject: Re: [Bfys-physics] B->K*mumu paper ||| |||Hi Rob, |||Let me just clarify: you question the claim of 0.5%, is that correct? |||just to make sure I understand which point you comment on. |||cheers, |||- Marcel ||| ||| |||On 2 July 2013 10:06, Rob Lambert |||<Rob.Lambert@cern.chmailto:Rob.Lambert@cern.ch> wrote: Erm, just some |||statistical queries.. what about chebychev's inequality: ||| |||http://en.wikipedia.org/wiki/Chebyshev%27s_inequality ||| |||"No more than 1/k^{2} of the distribution's values can be more than k |||standard deviations away from the mean" ||| |||In this case it is 1/(3.7*3.7) which is 7%, and 24*7% is 1.75, so no more |||than 1.75 of the observables should be this far away. One deviation this |||large is perfectly expected from Chebychev's inequality. ||| |||Thanks, ||| |||Rob ||| |||------------------------------------------ |||Robert Lambert |||FOM-VU-NIKHEF-Bfys LHCb |||Email: rob.lambert@cern.chmailto:rob.lambert@cern.ch |||------------------------------------------ |||Nikhef N251 |||Tel: +31 20 592 2131tel:%2B31%2020%20592%202131 |||Fax: +31 20 592 5155tel:%2B31%2020%20592%205155 |||------------------------------------------ |||CERN, 13-1-018 |||Tel: +41 22 767 4024tel:%2B41%2022%20767%204024 |||Fax: +41 22 766 8109tel:%2B41%2022%20766%208109 |||------------------------------------------ |||________________________________ |||From: bfys-physics-bounces@nikhef.nlmailto:bfys-physics-bounces@nikhef.nl |||[bfys-physics-bounces@nikhef.nlmailto:bfys-physics-bounces@nikhef.nl] on |||behalf of Marcel Merk [marcel.merk@nikhef.nlmailto:marcel.merk@nikhef.nl] |||Sent: 01 July 2013 18:01 |||To: bfys-physics@nikhef.nlmailto:bfys-physics@nikhef.nl |||Subject: [Bfys-physics] B->K*mumu paper ||| |||Dear bfys-physics friend, |||This is to let you know that I collect comments to the paper below. |||To stimulate you to read it: we claim that the probability that the |||measurement is consistent with the Standard Model is only 0.5%. A local |||discrepancy of 3.7 sigma is observed. ||| ||| LHCb PAPER-2013-037: ||| "Measurement of form factor independent observables ||| ||| in the decay $B^0\to K^{*0}\mu^+\mu^-$" ||| ||| Link : https://cds.cern.ch/record/1557918 ||| Deadline : 09-Jul-2013 ||| |||best regards, |||- Marcel ||| |||_______________________________________________ |||Bfys-physics mailing list |||Bfys-physics@nikhef.nlmailto:Bfys-physics@nikhef.nl |||https://mailman.nikhef.nl/mailman/listinfo/bfys-physics || ||--- || || Francesco Dettori || Nikhef - VU || || +31 20 592 2230 (Nikhef) || +41 22 76 76284 (CERN) | |--- | Francesco Dettori | Nikhef - VU | | +31 20 592 2230 (Nikhef) | +41 22 76 76284 (CERN) | |_______________________________________________ |Bfys-physics mailing list |Bfys-physics@nikhef.nl |https://mailman.nikhef.nl/mailman/listinfo/bfys-physics
--- Francesco Dettori Nikhef - VU
+31 20 592 2230 (Nikhef) +41 22 76 76284 (CERN)