Yes. Chebychev's inequality gives an estimate for how often a deviation this large this should happen, and it seems to be perfectly within the realms of probability, in fact it seems to be expected, and even two such deviations would not be strange.

Gerhard might want to correct me here, though...

Cheers,

Rob

------------------------------------------ Robert Lambert FOM-VU-NIKHEF-Bfys LHCb Email: rob.lambert@cern.ch ------------------------------------------ Nikhef N251 Tel: +31 20 592 2131 Fax: +31 20 592 5155 ------------------------------------------ CERN, 13-1-018 Tel: +41 22 767 4024 Fax: +41 22 766 8109 ------------------------------------------ ________________________________ From: Marcel Merk [marcel.merk@nikhef.nl] Sent: 02 July 2013 10:51 To: Rob Lambert Cc: bfys-physics@nikhef.nl Subject: Re: [Bfys-physics] B->K*mumu paper

Hi Rob, Let me just clarify: you question the claim of 0.5%, is that correct? just to make sure I understand which point you comment on. cheers, - Marcel

On 2 July 2013 10:06, Rob Lambert <Rob.Lambert@cern.chmailto:Rob.Lambert@cern.ch> wrote: Erm, just some statistical queries.. what about chebychev's inequality:

http://en.wikipedia.org/wiki/Chebyshev%27s_inequality

"No more than 1/k^{2} of the distribution's values can be more than k standard deviations away from the mean"

In this case it is 1/(3.7*3.7) which is 7%, and 24*7% is 1.75, so no more than 1.75 of the observables should be this far away. One deviation this large is perfectly expected from Chebychev's inequality.

Thanks,

Rob

------------------------------------------ Robert Lambert FOM-VU-NIKHEF-Bfys LHCb Email: rob.lambert@cern.chmailto:rob.lambert@cern.ch ------------------------------------------ Nikhef N251 Tel: +31 20 592 2131tel:%2B31%2020%20592%202131 Fax: +31 20 592 5155tel:%2B31%2020%20592%205155 ------------------------------------------ CERN, 13-1-018 Tel: +41 22 767 4024tel:%2B41%2022%20767%204024 Fax: +41 22 766 8109tel:%2B41%2022%20766%208109 ------------------------------------------ ________________________________ From: bfys-physics-bounces@nikhef.nlmailto:bfys-physics-bounces@nikhef.nl [bfys-physics-bounces@nikhef.nlmailto:bfys-physics-bounces@nikhef.nl] on behalf of Marcel Merk [marcel.merk@nikhef.nlmailto:marcel.merk@nikhef.nl] Sent: 01 July 2013 18:01 To: bfys-physics@nikhef.nlmailto:bfys-physics@nikhef.nl Subject: [Bfys-physics] B->K*mumu paper

Dear bfys-physics friend, This is to let you know that I collect comments to the paper below. To stimulate you to read it: we claim that the probability that the measurement is consistent with the Standard Model is only 0.5%. A local discrepancy of 3.7 sigma is observed.

LHCb PAPER-2013-037: "Measurement of form factor independent observables in the decay $B^0\to K^{*0}\mu^+\mu^-$" Link : https://cds.cern.ch/record/1557918 Deadline : 09-Jul-2013

best regards, - Marcel

_______________________________________________ Bfys-physics mailing list Bfys-physics@nikhef.nlmailto:Bfys-physics@nikhef.nl https://mailman.nikhef.nl/mailman/listinfo/bfys-physics

(PS: 0.566 \pm 0.033. forgot the % of course)

________________________________ Dende: Diego Martinez Santos Enviado: martes, 02 de xullo de 2013 11:16 Ata: Rob Lambert; Marcel Merk Cc: bfys-physics@nikhef.nl Asunto: RE: [Bfys-physics] B->K*mumu paper

Hi

I;ve just made quick toyMC, I get that the probability(maxSigma) > 3.7 for experiments consisting

of 24 measurements is 0.566 \pm 0.033.

Voila the code (needs SetupUrania):

from scipy import random as rnd from Urania import * AccessPackage("Bs2MuMu")

from RTuple import *

def do_set_of_experiments(N): l = [] for i in range(N): l.append(abs(rnd.normal())) l.sort() l.reverse() return l[0]

def do_test(N, Nexp = 50000): tup = RTuple("File_test",["maxSigma/F"]) for i in range(Nexp): tup.fillItem("maxSigma",do_set_of_experiments(24)) tup.fill() tup.close()

Then you just call do_test(24) and analyse the produced Ntuple.

Cheers;

Diego

________________________________ Dende: bfys-physics-bounces@nikhef.nl [bfys-physics-bounces@nikhef.nl] en nome de Rob Lambert [Rob.Lambert@cern.ch] Enviado: martes, 02 de xullo de 2013 10:53 Ata: Marcel Merk Cc: bfys-physics@nikhef.nl Asunto: Re: [Bfys-physics] B->K*mumu paper

Yes. Chebychev's inequality gives an estimate for how often a deviation this large this should happen, and it seems to be perfectly within the realms of probability, in fact it seems to be expected, and even two such deviations would not be strange.

Gerhard might want to correct me here, though...

Cheers,

Rob

------------------------------------------ Robert Lambert FOM-VU-NIKHEF-Bfys LHCb Email: rob.lambert@cern.ch ------------------------------------------ Nikhef N251 Tel: +31 20 592 2131 Fax: +31 20 592 5155 ------------------------------------------ CERN, 13-1-018 Tel: +41 22 767 4024 Fax: +41 22 766 8109 ------------------------------------------ ________________________________ From: Marcel Merk [marcel.merk@nikhef.nl] Sent: 02 July 2013 10:51 To: Rob Lambert Cc: bfys-physics@nikhef.nl Subject: Re: [Bfys-physics] B->K*mumu paper

Hi Rob, Let me just clarify: you question the claim of 0.5%, is that correct? just to make sure I understand which point you comment on. cheers, - Marcel

On 2 July 2013 10:06, Rob Lambert <Rob.Lambert@cern.chmailto:Rob.Lambert@cern.ch> wrote: Erm, just some statistical queries.. what about chebychev's inequality:

http://en.wikipedia.org/wiki/Chebyshev%27s_inequality

"No more than 1/k^{2} of the distribution's values can be more than k standard deviations away from the mean"

In this case it is 1/(3.7*3.7) which is 7%, and 24*7% is 1.75, so no more than 1.75 of the observables should be this far away. One deviation this large is perfectly expected from Chebychev's inequality.

Thanks,

Rob

------------------------------------------ Robert Lambert FOM-VU-NIKHEF-Bfys LHCb Email: rob.lambert@cern.chmailto:rob.lambert@cern.ch ------------------------------------------ Nikhef N251 Tel: +31 20 592 2131tel:%2B31%2020%20592%202131 Fax: +31 20 592 5155tel:%2B31%2020%20592%205155 ------------------------------------------ CERN, 13-1-018 Tel: +41 22 767 4024tel:%2B41%2022%20767%204024 Fax: +41 22 766 8109tel:%2B41%2022%20766%208109 ------------------------------------------ ________________________________ From: bfys-physics-bounces@nikhef.nlmailto:bfys-physics-bounces@nikhef.nl [bfys-physics-bounces@nikhef.nlmailto:bfys-physics-bounces@nikhef.nl] on behalf of Marcel Merk [marcel.merk@nikhef.nlmailto:marcel.merk@nikhef.nl] Sent: 01 July 2013 18:01 To: bfys-physics@nikhef.nlmailto:bfys-physics@nikhef.nl Subject: [Bfys-physics] B->K*mumu paper

Dear bfys-physics friend, This is to let you know that I collect comments to the paper below. To stimulate you to read it: we claim that the probability that the measurement is consistent with the Standard Model is only 0.5%. A local discrepancy of 3.7 sigma is observed.

LHCb PAPER-2013-037: "Measurement of form factor independent observables in the decay $B^0\to K^{*0}\mu^+\mu^-$" Link : https://cds.cern.ch/record/1557918 Deadline : 09-Jul-2013

best regards, - Marcel

_______________________________________________ Bfys-physics mailing list Bfys-physics@nikhef.nlmailto:Bfys-physics@nikhef.nl https://mailman.nikhef.nl/mailman/listinfo/bfys-physics

(24*0.0001*2) ^ missed a zero ....

Rob

------------------------------------------ Robert Lambert FOM-VU-NIKHEF-Bfys LHCb Email: rob.lambert@cern.ch ------------------------------------------ Nikhef N251 Tel: +31 20 592 2131 Fax: +31 20 592 5155 ------------------------------------------ CERN, 13-1-018 Tel: +41 22 767 4024 Fax: +41 22 766 8109 ------------------------------------------ ________________________________ From: Rob Lambert Sent: 02 July 2013 11:29 To: Diego Martinez Santos; Marcel Merk Cc: bfys-physics@nikhef.nl Subject: RE: [Bfys-physics] B->K*mumu paper

Hi Diego,

Well, I'm 100% sure that's right because it's just 24*0.001*2, which is the integrated probability for a two-sided Gaussian test, but the question isn't "what's the probability of that happening" which is actually "1", because it did actually happen, but "how many such deviations would you need to see before you can invalidate the null hypothesis of no deviation", and this number is given by Chebychev's equality... well, I thought it was, anyway.

Thanks,

Rob

------------------------------------------ Robert Lambert FOM-VU-NIKHEF-Bfys LHCb Email: rob.lambert@cern.ch ------------------------------------------ Nikhef N251 Tel: +31 20 592 2131 Fax: +31 20 592 5155 ------------------------------------------ CERN, 13-1-018 Tel: +41 22 767 4024 Fax: +41 22 766 8109 ------------------------------------------ ________________________________ From: Diego Martinez Santos Sent: 02 July 2013 11:17 To: Rob Lambert; Marcel Merk Cc: bfys-physics@nikhef.nl Subject: RE: [Bfys-physics] B->K*mumu paper

(PS: 0.566 \pm 0.033. forgot the % of course)

________________________________ Dende: Diego Martinez Santos Enviado: martes, 02 de xullo de 2013 11:16 Ata: Rob Lambert; Marcel Merk Cc: bfys-physics@nikhef.nl Asunto: RE: [Bfys-physics] B->K*mumu paper

Hi

I;ve just made quick toyMC, I get that the probability(maxSigma) > 3.7 for experiments consisting

of 24 measurements is 0.566 \pm 0.033.

Voila the code (needs SetupUrania):

from scipy import random as rnd from Urania import * AccessPackage("Bs2MuMu")

from RTuple import *

def do_set_of_experiments(N): l = [] for i in range(N): l.append(abs(rnd.normal())) l.sort() l.reverse() return l[0]

def do_test(N, Nexp = 50000): tup = RTuple("File_test",["maxSigma/F"]) for i in range(Nexp): tup.fillItem("maxSigma",do_set_of_experiments(24)) tup.fill() tup.close()

Then you just call do_test(24) and analyse the produced Ntuple.

Cheers;

Diego

________________________________ Dende: bfys-physics-bounces@nikhef.nl [bfys-physics-bounces@nikhef.nl] en nome de Rob Lambert [Rob.Lambert@cern.ch] Enviado: martes, 02 de xullo de 2013 10:53 Ata: Marcel Merk Cc: bfys-physics@nikhef.nl Asunto: Re: [Bfys-physics] B->K*mumu paper

Yes. Chebychev's inequality gives an estimate for how often a deviation this large this should happen, and it seems to be perfectly within the realms of probability, in fact it seems to be expected, and even two such deviations would not be strange.

Gerhard might want to correct me here, though...

Cheers,

Rob

------------------------------------------ Robert Lambert FOM-VU-NIKHEF-Bfys LHCb Email: rob.lambert@cern.ch ------------------------------------------ Nikhef N251 Tel: +31 20 592 2131 Fax: +31 20 592 5155 ------------------------------------------ CERN, 13-1-018 Tel: +41 22 767 4024 Fax: +41 22 766 8109 ------------------------------------------ ________________________________ From: Marcel Merk [marcel.merk@nikhef.nl] Sent: 02 July 2013 10:51 To: Rob Lambert Cc: bfys-physics@nikhef.nl Subject: Re: [Bfys-physics] B->K*mumu paper

Hi Rob, Let me just clarify: you question the claim of 0.5%, is that correct? just to make sure I understand which point you comment on. cheers, - Marcel

On 2 July 2013 10:06, Rob Lambert <Rob.Lambert@cern.chmailto:Rob.Lambert@cern.ch> wrote: Erm, just some statistical queries.. what about chebychev's inequality:

http://en.wikipedia.org/wiki/Chebyshev%27s_inequality

"No more than 1/k^{2} of the distribution's values can be more than k standard deviations away from the mean"

In this case it is 1/(3.7*3.7) which is 7%, and 24*7% is 1.75, so no more than 1.75 of the observables should be this far away. One deviation this large is perfectly expected from Chebychev's inequality.

Thanks,

Rob

------------------------------------------ Robert Lambert FOM-VU-NIKHEF-Bfys LHCb Email: rob.lambert@cern.chmailto:rob.lambert@cern.ch ------------------------------------------ Nikhef N251 Tel: +31 20 592 2131tel:%2B31%2020%20592%202131 Fax: +31 20 592 5155tel:%2B31%2020%20592%205155 ------------------------------------------ CERN, 13-1-018 Tel: +41 22 767 4024tel:%2B41%2022%20767%204024 Fax: +41 22 766 8109tel:%2B41%2022%20766%208109 ------------------------------------------ ________________________________ From: bfys-physics-bounces@nikhef.nlmailto:bfys-physics-bounces@nikhef.nl [bfys-physics-bounces@nikhef.nlmailto:bfys-physics-bounces@nikhef.nl] on behalf of Marcel Merk [marcel.merk@nikhef.nlmailto:marcel.merk@nikhef.nl] Sent: 01 July 2013 18:01 To: bfys-physics@nikhef.nlmailto:bfys-physics@nikhef.nl Subject: [Bfys-physics] B->K*mumu paper

Dear bfys-physics friend, This is to let you know that I collect comments to the paper below. To stimulate you to read it: we claim that the probability that the measurement is consistent with the Standard Model is only 0.5%. A local discrepancy of 3.7 sigma is observed.

LHCb PAPER-2013-037: "Measurement of form factor independent observables in the decay $B^0\to K^{*0}\mu^+\mu^-$" Link : https://cds.cern.ch/record/1557918 Deadline : 09-Jul-2013

best regards, - Marcel

_______________________________________________ Bfys-physics mailing list Bfys-physics@nikhef.nlmailto:Bfys-physics@nikhef.nl https://mailman.nikhef.nl/mailman/listinfo/bfys-physics

Ciao Rob,

I don't understand completely what is your point: - that they should not claim any discrepancy because globally the agreement is fine - or that the final claim of 2.8 sigma global tension is wrong.

cheers, Francesco

On Tuesday 02 July 2013 09:33:44 Rob Lambert wrote: |Hi Again, | |Another way to think about it is to fit a Gaussian to the 24 data points, and |analyse whether one of them is an outlier or not. This is analytic given a |Gaussian distribution with one outlier and reduces to the Chebychev |inequality (or at least, I thought it did). | |Thanks, | |Rob | |------------------------------------------ |Robert Lambert |FOM-VU-NIKHEF-Bfys LHCb |Email: rob.lambert@cern.ch |------------------------------------------ |Nikhef N251 |Tel: +31 20 592 2131 |Fax: +31 20 592 5155 |------------------------------------------ |CERN, 13-1-018 |Tel: +41 22 767 4024 |Fax: +41 22 766 8109 |------------------------------------------ |________________________________ |From: bfys-physics-bounces@nikhef.nl [bfys-physics-bounces@nikhef.nl] on |behalf of Rob Lambert [Rob.Lambert@cern.ch] Sent: 02 July 2013 11:30 |To: Diego Martinez Santos; Marcel Merk |Cc: bfys-physics@nikhef.nl |Subject: Re: [Bfys-physics] B->K*mumu paper | |(24*0.0001*2) | ^ missed a zero .... | |Rob | |------------------------------------------ |Robert Lambert |FOM-VU-NIKHEF-Bfys LHCb |Email: rob.lambert@cern.ch |------------------------------------------ |Nikhef N251 |Tel: +31 20 592 2131 |Fax: +31 20 592 5155 |------------------------------------------ |CERN, 13-1-018 |Tel: +41 22 767 4024 |Fax: +41 22 766 8109 |------------------------------------------ |________________________________ |From: Rob Lambert |Sent: 02 July 2013 11:29 |To: Diego Martinez Santos; Marcel Merk |Cc: bfys-physics@nikhef.nl |Subject: RE: [Bfys-physics] B->K*mumu paper | |Hi Diego, | |Well, I'm 100% sure that's right because it's just 24*0.001*2, which is the |integrated probability for a two-sided Gaussian test, but the question isn't |"what's the probability of that happening" which is actually "1", because it |did actually happen, but "how many such deviations would you need to see |before you can invalidate the null hypothesis of no deviation", and this |number is given by Chebychev's equality... well, I thought it was, anyway. | |Thanks, | |Rob | |------------------------------------------ |Robert Lambert |FOM-VU-NIKHEF-Bfys LHCb |Email: rob.lambert@cern.ch |------------------------------------------ |Nikhef N251 |Tel: +31 20 592 2131 |Fax: +31 20 592 5155 |------------------------------------------ |CERN, 13-1-018 |Tel: +41 22 767 4024 |Fax: +41 22 766 8109 |------------------------------------------ |________________________________ |From: Diego Martinez Santos |Sent: 02 July 2013 11:17 |To: Rob Lambert; Marcel Merk |Cc: bfys-physics@nikhef.nl |Subject: RE: [Bfys-physics] B->K*mumu paper | | |(PS: 0.566 \pm 0.033. forgot the % of course) | |________________________________ |Dende: Diego Martinez Santos |Enviado: martes, 02 de xullo de 2013 11:16 |Ata: Rob Lambert; Marcel Merk |Cc: bfys-physics@nikhef.nl |Asunto: RE: [Bfys-physics] B->K*mumu paper | | |Hi | | | |I;ve just made quick toyMC, I get that the probability(maxSigma) > 3.7 for |experiments consisting | |of 24 measurements is 0.566 \pm 0.033. | | | | | |Voila the code (needs SetupUrania): | | | |from scipy import random as rnd |from Urania import * |AccessPackage("Bs2MuMu") | |from RTuple import * | | | |def do_set_of_experiments(N): | l = [] | for i in range(N): | l.append(abs(rnd.normal())) | l.sort() | l.reverse() | return l[0] | |def do_test(N, Nexp = 50000): | tup = RTuple("File_test",["maxSigma/F"]) | for i in range(Nexp): | tup.fillItem("maxSigma",do_set_of_experiments(24)) | tup.fill() | tup.close() | | | | | |Then you just call do_test(24) and analyse the produced Ntuple. | | | | | | | |Cheers; | | | |Diego | | | | | | | | | |________________________________ |Dende: bfys-physics-bounces@nikhef.nl [bfys-physics-bounces@nikhef.nl] en |nome de Rob Lambert [Rob.Lambert@cern.ch] Enviado: martes, 02 de xullo de |2013 10:53 |Ata: Marcel Merk |Cc: bfys-physics@nikhef.nl |Asunto: Re: [Bfys-physics] B->K*mumu paper | |Yes. Chebychev's inequality gives an estimate for how often a deviation this |large this should happen, and it seems to be perfectly within the realms of |probability, in fact it seems to be expected, and even two such deviations |would not be strange. | |Gerhard might want to correct me here, though... | |Cheers, | |Rob | |------------------------------------------ |Robert Lambert |FOM-VU-NIKHEF-Bfys LHCb |Email: rob.lambert@cern.ch |------------------------------------------ |Nikhef N251 |Tel: +31 20 592 2131 |Fax: +31 20 592 5155 |------------------------------------------ |CERN, 13-1-018 |Tel: +41 22 767 4024 |Fax: +41 22 766 8109 |------------------------------------------ |________________________________ |From: Marcel Merk [marcel.merk@nikhef.nl] |Sent: 02 July 2013 10:51 |To: Rob Lambert |Cc: bfys-physics@nikhef.nl |Subject: Re: [Bfys-physics] B->K*mumu paper | |Hi Rob, |Let me just clarify: you question the claim of 0.5%, is that correct? |just to make sure I understand which point you comment on. |cheers, |- Marcel | | |On 2 July 2013 10:06, Rob Lambert |<Rob.Lambert@cern.chmailto:Rob.Lambert@cern.ch> wrote: Erm, just some |statistical queries.. what about chebychev's inequality: | |http://en.wikipedia.org/wiki/Chebyshev%27s_inequality | |"No more than 1/k^{2} of the distribution's values can be more than k |standard deviations away from the mean" | |In this case it is 1/(3.7*3.7) which is 7%, and 24*7% is 1.75, so no more |than 1.75 of the observables should be this far away. One deviation this |large is perfectly expected from Chebychev's inequality. | |Thanks, | |Rob | |------------------------------------------ |Robert Lambert |FOM-VU-NIKHEF-Bfys LHCb |Email: rob.lambert@cern.chmailto:rob.lambert@cern.ch |------------------------------------------ |Nikhef N251 |Tel: +31 20 592 2131tel:%2B31%2020%20592%202131 |Fax: +31 20 592 5155tel:%2B31%2020%20592%205155 |------------------------------------------ |CERN, 13-1-018 |Tel: +41 22 767 4024tel:%2B41%2022%20767%204024 |Fax: +41 22 766 8109tel:%2B41%2022%20766%208109 |------------------------------------------ |________________________________ |From: bfys-physics-bounces@nikhef.nlmailto:bfys-physics-bounces@nikhef.nl |[bfys-physics-bounces@nikhef.nlmailto:bfys-physics-bounces@nikhef.nl] on |behalf of Marcel Merk [marcel.merk@nikhef.nlmailto:marcel.merk@nikhef.nl] |Sent: 01 July 2013 18:01 |To: bfys-physics@nikhef.nlmailto:bfys-physics@nikhef.nl |Subject: [Bfys-physics] B->K*mumu paper | |Dear bfys-physics friend, |This is to let you know that I collect comments to the paper below. |To stimulate you to read it: we claim that the probability that the |measurement is consistent with the Standard Model is only 0.5%. A local |discrepancy of 3.7 sigma is observed. | | LHCb PAPER-2013-037: | "Measurement of form factor independent observables | in the decay $B^0\to K^{*0}\mu^+\mu^-$" | Link : https://cds.cern.ch/record/1557918 | Deadline : 09-Jul-2013 | |best regards, |- Marcel | |_______________________________________________ |Bfys-physics mailing list |Bfys-physics@nikhef.nlmailto:Bfys-physics@nikhef.nl |https://mailman.nikhef.nl/mailman/listinfo/bfys-physics

--- Francesco Dettori Nikhef - VU

+31 20 592 2230 (Nikhef) +41 22 76 76284 (CERN)

Ciao,

yes I perfectly agree with you, I was just trying to summarise possible Rob points :)

Francesco

On Tuesday 02 July 2013 09:56:21 Diego Martinez Santos wrote: |Ciao | |I'm not sure they speak about "global tension", I mean, what they say |is different from making a chi2 with 24 DoF. From the abstract: |"Considering the 24 measurements as independent, the probability to observe |such a discrepancy in one of them is 0:5%." | |i.e, I guess they just mean that in 0.5% of experiments made of 24 |indepenendent measurements you have at least a discrepancy of 3.7 s or more |(and seem consistent with the toy I made). | |Cheers; | |Diego | |________________________________________ |Dende: Francesco Dettori [fdettori@nikhef.nl] |Enviado: martes, 02 de xullo de 2013 11:46 |Ata: bfys-physics@nikhef.nl |Cc: Rob Lambert; Diego Martinez Santos; Marcel Merk |Asunto: Re: [Bfys-physics] B->K*mumu paper | |Ciao Rob, | |I don't understand completely what is your point: |- that they should not claim any discrepancy because globally the agreement |is fine |- or that the final claim of 2.8 sigma global tension is wrong. | |cheers, |Francesco | |On Tuesday 02 July 2013 09:33:44 Rob Lambert wrote: ||Hi Again, || ||Another way to think about it is to fit a Gaussian to the 24 data points, ||and analyse whether one of them is an outlier or not. This is analytic ||given a Gaussian distribution with one outlier and reduces to the Chebychev ||inequality (or at least, I thought it did). || ||Thanks, || ||Rob || ||------------------------------------------ ||Robert Lambert ||FOM-VU-NIKHEF-Bfys LHCb ||Email: rob.lambert@cern.ch ||------------------------------------------ ||Nikhef N251 ||Tel: +31 20 592 2131 ||Fax: +31 20 592 5155 ||------------------------------------------ ||CERN, 13-1-018 ||Tel: +41 22 767 4024 ||Fax: +41 22 766 8109 ||------------------------------------------ ||________________________________ ||From: bfys-physics-bounces@nikhef.nl [bfys-physics-bounces@nikhef.nl] on ||behalf of Rob Lambert [Rob.Lambert@cern.ch] Sent: 02 July 2013 11:30 ||To: Diego Martinez Santos; Marcel Merk ||Cc: bfys-physics@nikhef.nl ||Subject: Re: [Bfys-physics] B->K*mumu paper || ||(24*0.0001*2) || || ^ missed a zero .... || ||Rob || ||------------------------------------------ ||Robert Lambert ||FOM-VU-NIKHEF-Bfys LHCb ||Email: rob.lambert@cern.ch ||------------------------------------------ ||Nikhef N251 ||Tel: +31 20 592 2131 ||Fax: +31 20 592 5155 ||------------------------------------------ ||CERN, 13-1-018 ||Tel: +41 22 767 4024 ||Fax: +41 22 766 8109 ||------------------------------------------ ||________________________________ ||From: Rob Lambert ||Sent: 02 July 2013 11:29 ||To: Diego Martinez Santos; Marcel Merk ||Cc: bfys-physics@nikhef.nl ||Subject: RE: [Bfys-physics] B->K*mumu paper || ||Hi Diego, || ||Well, I'm 100% sure that's right because it's just 24*0.001*2, which is the ||integrated probability for a two-sided Gaussian test, but the question isn't ||"what's the probability of that happening" which is actually "1", because it ||did actually happen, but "how many such deviations would you need to see ||before you can invalidate the null hypothesis of no deviation", and this ||number is given by Chebychev's equality... well, I thought it was, anyway. || ||Thanks, || ||Rob || ||------------------------------------------ ||Robert Lambert ||FOM-VU-NIKHEF-Bfys LHCb ||Email: rob.lambert@cern.ch ||------------------------------------------ ||Nikhef N251 ||Tel: +31 20 592 2131 ||Fax: +31 20 592 5155 ||------------------------------------------ ||CERN, 13-1-018 ||Tel: +41 22 767 4024 ||Fax: +41 22 766 8109 ||------------------------------------------ ||________________________________ ||From: Diego Martinez Santos ||Sent: 02 July 2013 11:17 ||To: Rob Lambert; Marcel Merk ||Cc: bfys-physics@nikhef.nl ||Subject: RE: [Bfys-physics] B->K*mumu paper || || ||(PS: 0.566 \pm 0.033. forgot the % of course) || ||________________________________ ||Dende: Diego Martinez Santos ||Enviado: martes, 02 de xullo de 2013 11:16 ||Ata: Rob Lambert; Marcel Merk ||Cc: bfys-physics@nikhef.nl ||Asunto: RE: [Bfys-physics] B->K*mumu paper || || ||Hi || || || ||I;ve just made quick toyMC, I get that the probability(maxSigma) > 3.7 for ||experiments consisting || ||of 24 measurements is 0.566 \pm 0.033. || || || || || ||Voila the code (needs SetupUrania): || || || ||from scipy import random as rnd ||from Urania import * ||AccessPackage("Bs2MuMu") || ||from RTuple import * || ||def do_set_of_experiments(N): || l = [] || || for i in range(N): || l.append(abs(rnd.normal())) || || l.sort() || l.reverse() || return l[0] || ||def do_test(N, Nexp = 50000): || tup = RTuple("File_test",["maxSigma/F"]) || || for i in range(Nexp): || tup.fillItem("maxSigma",do_set_of_experiments(24)) || tup.fill() || || tup.close() || ||Then you just call do_test(24) and analyse the produced Ntuple. || || || || || || || ||Cheers; || || || ||Diego || || || || || || || || || ||________________________________ ||Dende: bfys-physics-bounces@nikhef.nl [bfys-physics-bounces@nikhef.nl] en ||nome de Rob Lambert [Rob.Lambert@cern.ch] Enviado: martes, 02 de xullo de ||2013 10:53 ||Ata: Marcel Merk ||Cc: bfys-physics@nikhef.nl ||Asunto: Re: [Bfys-physics] B->K*mumu paper || ||Yes. Chebychev's inequality gives an estimate for how often a deviation this ||large this should happen, and it seems to be perfectly within the realms of ||probability, in fact it seems to be expected, and even two such deviations ||would not be strange. || ||Gerhard might want to correct me here, though... || ||Cheers, || ||Rob || ||------------------------------------------ ||Robert Lambert ||FOM-VU-NIKHEF-Bfys LHCb ||Email: rob.lambert@cern.ch ||------------------------------------------ ||Nikhef N251 ||Tel: +31 20 592 2131 ||Fax: +31 20 592 5155 ||------------------------------------------ ||CERN, 13-1-018 ||Tel: +41 22 767 4024 ||Fax: +41 22 766 8109 ||------------------------------------------ ||________________________________ ||From: Marcel Merk [marcel.merk@nikhef.nl] ||Sent: 02 July 2013 10:51 ||To: Rob Lambert ||Cc: bfys-physics@nikhef.nl ||Subject: Re: [Bfys-physics] B->K*mumu paper || ||Hi Rob, ||Let me just clarify: you question the claim of 0.5%, is that correct? ||just to make sure I understand which point you comment on. ||cheers, ||- Marcel || || ||On 2 July 2013 10:06, Rob Lambert ||<Rob.Lambert@cern.chmailto:Rob.Lambert@cern.ch> wrote: Erm, just some ||statistical queries.. what about chebychev's inequality: || ||http://en.wikipedia.org/wiki/Chebyshev%27s_inequality || ||"No more than 1/k^{2} of the distribution's values can be more than k ||standard deviations away from the mean" || ||In this case it is 1/(3.7*3.7) which is 7%, and 24*7% is 1.75, so no more ||than 1.75 of the observables should be this far away. One deviation this ||large is perfectly expected from Chebychev's inequality. || ||Thanks, || ||Rob || ||------------------------------------------ ||Robert Lambert ||FOM-VU-NIKHEF-Bfys LHCb ||Email: rob.lambert@cern.chmailto:rob.lambert@cern.ch ||------------------------------------------ ||Nikhef N251 ||Tel: +31 20 592 2131tel:%2B31%2020%20592%202131 ||Fax: +31 20 592 5155tel:%2B31%2020%20592%205155 ||------------------------------------------ ||CERN, 13-1-018 ||Tel: +41 22 767 4024tel:%2B41%2022%20767%204024 ||Fax: +41 22 766 8109tel:%2B41%2022%20766%208109 ||------------------------------------------ ||________________________________ ||From: bfys-physics-bounces@nikhef.nlmailto:bfys-physics-bounces@nikhef.nl ||[bfys-physics-bounces@nikhef.nlmailto:bfys-physics-bounces@nikhef.nl] on ||behalf of Marcel Merk [marcel.merk@nikhef.nlmailto:marcel.merk@nikhef.nl] ||Sent: 01 July 2013 18:01 ||To: bfys-physics@nikhef.nlmailto:bfys-physics@nikhef.nl ||Subject: [Bfys-physics] B->K*mumu paper || ||Dear bfys-physics friend, ||This is to let you know that I collect comments to the paper below. ||To stimulate you to read it: we claim that the probability that the ||measurement is consistent with the Standard Model is only 0.5%. A local ||discrepancy of 3.7 sigma is observed. || || LHCb PAPER-2013-037: || "Measurement of form factor independent observables || || in the decay $B^0\to K^{*0}\mu^+\mu^-$" || || Link : https://cds.cern.ch/record/1557918 || Deadline : 09-Jul-2013 || ||best regards, ||- Marcel || ||_______________________________________________ ||Bfys-physics mailing list ||Bfys-physics@nikhef.nlmailto:Bfys-physics@nikhef.nl ||https://mailman.nikhef.nl/mailman/listinfo/bfys-physics | |--- | Francesco Dettori | Nikhef - VU | | +31 20 592 2230 (Nikhef) | +41 22 76 76284 (CERN)

--- Francesco Dettori Nikhef - VU

+31 20 592 2230 (Nikhef) +41 22 76 76284 (CERN)

Hi,

I actually do not agree. The 0.5% is the correct percentage to quote computed from the correct distribution. Chebychev inequality is more referring to the opposite: you would not expect that more than 1/k^2 would fall outside k sigmas. However that does not tell you how probably that is.

On the other side the physics importance is relevant: the SM cannot be excluded if you consider those measurements as independent. However those are not independent because they are sensitive to different coefficients.

my two cents, cheers Francesco

On Tuesday 02 July 2013 10:22:28 Rob Lambert wrote: |Hi again, | |Well, for me, concentrating on the 0.5% is over-stating the evidence. The SM |is still perfectly valid if one of 24 measurements shows a 3.7 sigma |discrepancy. Perhaps an added line (footnote?) with respect to the Chebychev |inequality would be prudent in the paper ("Using the rule-of-thumb of |Chebychev's inequality, 7% of the 24 measured values could be expected to |fall outside of the 3.7 sigma range, and thus our current results show no |significant inconsistency with the Standard Model."). | |Thanks, | |Rob | |------------------------------------------ |Robert Lambert |FOM-VU-NIKHEF-Bfys LHCb |Email: rob.lambert@cern.ch |------------------------------------------ |Nikhef N251 |Tel: +31 20 592 2131 |Fax: +31 20 592 5155 |------------------------------------------ |CERN, 13-1-018 |Tel: +41 22 767 4024 |Fax: +41 22 766 8109 |------------------------------------------ | |________________________________________ |From: Francesco Dettori [fdettori@nikhef.nl] |Sent: 02 July 2013 11:58 |To: Diego Martinez Santos |Cc: bfys-physics@nikhef.nl; Rob Lambert; Marcel Merk |Subject: Re: [Bfys-physics] B->K*mumu paper | |Ciao, | |yes I perfectly agree with you, I was just trying to summarise possible Rob |points :) | | |Francesco | |On Tuesday 02 July 2013 09:56:21 Diego Martinez Santos wrote: ||Ciao || ||I'm not sure they speak about "global tension", I mean, what they say ||is different from making a chi2 with 24 DoF. From the abstract: ||"Considering the 24 measurements as independent, the probability to observe ||such a discrepancy in one of them is 0:5%." || ||i.e, I guess they just mean that in 0.5% of experiments made of 24 ||indepenendent measurements you have at least a discrepancy of 3.7 s or more ||(and seem consistent with the toy I made). || ||Cheers; || ||Diego || ||________________________________________ ||Dende: Francesco Dettori [fdettori@nikhef.nl] ||Enviado: martes, 02 de xullo de 2013 11:46 ||Ata: bfys-physics@nikhef.nl ||Cc: Rob Lambert; Diego Martinez Santos; Marcel Merk ||Asunto: Re: [Bfys-physics] B->K*mumu paper || ||Ciao Rob, || ||I don't understand completely what is your point: ||- that they should not claim any discrepancy because globally the agreement ||is fine ||- or that the final claim of 2.8 sigma global tension is wrong. || ||cheers, ||Francesco || ||On Tuesday 02 July 2013 09:33:44 Rob Lambert wrote: |||Hi Again, ||| |||Another way to think about it is to fit a Gaussian to the 24 data points, |||and analyse whether one of them is an outlier or not. This is analytic |||given a Gaussian distribution with one outlier and reduces to the Chebychev |||inequality (or at least, I thought it did). ||| |||Thanks, ||| |||Rob ||| |||------------------------------------------ |||Robert Lambert |||FOM-VU-NIKHEF-Bfys LHCb |||Email: rob.lambert@cern.ch |||------------------------------------------ |||Nikhef N251 |||Tel: +31 20 592 2131 |||Fax: +31 20 592 5155 |||------------------------------------------ |||CERN, 13-1-018 |||Tel: +41 22 767 4024 |||Fax: +41 22 766 8109 |||------------------------------------------ |||________________________________ |||From: bfys-physics-bounces@nikhef.nl [bfys-physics-bounces@nikhef.nl] on |||behalf of Rob Lambert [Rob.Lambert@cern.ch] Sent: 02 July 2013 11:30 |||To: Diego Martinez Santos; Marcel Merk |||Cc: bfys-physics@nikhef.nl |||Subject: Re: [Bfys-physics] B->K*mumu paper ||| |||(24*0.0001*2) ||| ||| ^ missed a zero .... ||| |||Rob ||| |||------------------------------------------ |||Robert Lambert |||FOM-VU-NIKHEF-Bfys LHCb |||Email: rob.lambert@cern.ch |||------------------------------------------ |||Nikhef N251 |||Tel: +31 20 592 2131 |||Fax: +31 20 592 5155 |||------------------------------------------ |||CERN, 13-1-018 |||Tel: +41 22 767 4024 |||Fax: +41 22 766 8109 |||------------------------------------------ |||________________________________ |||From: Rob Lambert |||Sent: 02 July 2013 11:29 |||To: Diego Martinez Santos; Marcel Merk |||Cc: bfys-physics@nikhef.nl |||Subject: RE: [Bfys-physics] B->K*mumu paper ||| |||Hi Diego, ||| |||Well, I'm 100% sure that's right because it's just 24*0.001*2, which is the |||integrated probability for a two-sided Gaussian test, but the question |||isn't |||"what's the probability of that happening" which is actually "1", because |||it |||did actually happen, but "how many such deviations would you need to see |||before you can invalidate the null hypothesis of no deviation", and this |||number is given by Chebychev's equality... well, I thought it was, anyway. ||| |||Thanks, ||| |||Rob ||| |||------------------------------------------ |||Robert Lambert |||FOM-VU-NIKHEF-Bfys LHCb |||Email: rob.lambert@cern.ch |||------------------------------------------ |||Nikhef N251 |||Tel: +31 20 592 2131 |||Fax: +31 20 592 5155 |||------------------------------------------ |||CERN, 13-1-018 |||Tel: +41 22 767 4024 |||Fax: +41 22 766 8109 |||------------------------------------------ |||________________________________ |||From: Diego Martinez Santos |||Sent: 02 July 2013 11:17 |||To: Rob Lambert; Marcel Merk |||Cc: bfys-physics@nikhef.nl |||Subject: RE: [Bfys-physics] B->K*mumu paper ||| ||| |||(PS: 0.566 \pm 0.033. forgot the % of course) ||| |||________________________________ |||Dende: Diego Martinez Santos |||Enviado: martes, 02 de xullo de 2013 11:16 |||Ata: Rob Lambert; Marcel Merk |||Cc: bfys-physics@nikhef.nl |||Asunto: RE: [Bfys-physics] B->K*mumu paper ||| ||| |||Hi ||| ||| ||| |||I;ve just made quick toyMC, I get that the probability(maxSigma) > 3.7 for |||experiments consisting ||| |||of 24 measurements is 0.566 \pm 0.033. ||| ||| ||| ||| ||| |||Voila the code (needs SetupUrania): ||| ||| ||| |||from scipy import random as rnd |||from Urania import * |||AccessPackage("Bs2MuMu") ||| |||from RTuple import * ||| |||def do_set_of_experiments(N): ||| l = [] ||| ||| for i in range(N): ||| l.append(abs(rnd.normal())) ||| ||| l.sort() ||| l.reverse() ||| return l[0] ||| |||def do_test(N, Nexp = 50000): ||| tup = RTuple("File_test",["maxSigma/F"]) ||| ||| for i in range(Nexp): ||| tup.fillItem("maxSigma",do_set_of_experiments(24)) ||| tup.fill() ||| ||| tup.close() ||| |||Then you just call do_test(24) and analyse the produced Ntuple. ||| ||| ||| ||| ||| ||| ||| |||Cheers; ||| ||| ||| |||Diego ||| ||| ||| ||| ||| ||| ||| ||| ||| |||________________________________ |||Dende: bfys-physics-bounces@nikhef.nl [bfys-physics-bounces@nikhef.nl] en |||nome de Rob Lambert [Rob.Lambert@cern.ch] Enviado: martes, 02 de xullo de |||2013 10:53 |||Ata: Marcel Merk |||Cc: bfys-physics@nikhef.nl |||Asunto: Re: [Bfys-physics] B->K*mumu paper ||| |||Yes. Chebychev's inequality gives an estimate for how often a deviation |||this |||large this should happen, and it seems to be perfectly within the realms of |||probability, in fact it seems to be expected, and even two such deviations |||would not be strange. ||| |||Gerhard might want to correct me here, though... ||| |||Cheers, ||| |||Rob ||| |||------------------------------------------ |||Robert Lambert |||FOM-VU-NIKHEF-Bfys LHCb |||Email: rob.lambert@cern.ch |||------------------------------------------ |||Nikhef N251 |||Tel: +31 20 592 2131 |||Fax: +31 20 592 5155 |||------------------------------------------ |||CERN, 13-1-018 |||Tel: +41 22 767 4024 |||Fax: +41 22 766 8109 |||------------------------------------------ |||________________________________ |||From: Marcel Merk [marcel.merk@nikhef.nl] |||Sent: 02 July 2013 10:51 |||To: Rob Lambert |||Cc: bfys-physics@nikhef.nl |||Subject: Re: [Bfys-physics] B->K*mumu paper ||| |||Hi Rob, |||Let me just clarify: you question the claim of 0.5%, is that correct? |||just to make sure I understand which point you comment on. |||cheers, |||- Marcel ||| ||| |||On 2 July 2013 10:06, Rob Lambert |||<Rob.Lambert@cern.chmailto:Rob.Lambert@cern.ch> wrote: Erm, just some |||statistical queries.. what about chebychev's inequality: ||| |||http://en.wikipedia.org/wiki/Chebyshev%27s_inequality ||| |||"No more than 1/k^{2} of the distribution's values can be more than k |||standard deviations away from the mean" ||| |||In this case it is 1/(3.7*3.7) which is 7%, and 24*7% is 1.75, so no more |||than 1.75 of the observables should be this far away. One deviation this |||large is perfectly expected from Chebychev's inequality. ||| |||Thanks, ||| |||Rob ||| |||------------------------------------------ |||Robert Lambert |||FOM-VU-NIKHEF-Bfys LHCb |||Email: rob.lambert@cern.chmailto:rob.lambert@cern.ch |||------------------------------------------ |||Nikhef N251 |||Tel: +31 20 592 2131tel:%2B31%2020%20592%202131 |||Fax: +31 20 592 5155tel:%2B31%2020%20592%205155 |||------------------------------------------ |||CERN, 13-1-018 |||Tel: +41 22 767 4024tel:%2B41%2022%20767%204024 |||Fax: +41 22 766 8109tel:%2B41%2022%20766%208109 |||------------------------------------------ |||________________________________ |||From: bfys-physics-bounces@nikhef.nlmailto:bfys-physics-bounces@nikhef.nl |||[bfys-physics-bounces@nikhef.nlmailto:bfys-physics-bounces@nikhef.nl] on |||behalf of Marcel Merk [marcel.merk@nikhef.nlmailto:marcel.merk@nikhef.nl] |||Sent: 01 July 2013 18:01 |||To: bfys-physics@nikhef.nlmailto:bfys-physics@nikhef.nl |||Subject: [Bfys-physics] B->K*mumu paper ||| |||Dear bfys-physics friend, |||This is to let you know that I collect comments to the paper below. |||To stimulate you to read it: we claim that the probability that the |||measurement is consistent with the Standard Model is only 0.5%. A local |||discrepancy of 3.7 sigma is observed. ||| ||| LHCb PAPER-2013-037: ||| "Measurement of form factor independent observables ||| ||| in the decay $B^0\to K^{*0}\mu^+\mu^-$" ||| ||| Link : https://cds.cern.ch/record/1557918 ||| Deadline : 09-Jul-2013 ||| |||best regards, |||- Marcel ||| |||_______________________________________________ |||Bfys-physics mailing list |||Bfys-physics@nikhef.nlmailto:Bfys-physics@nikhef.nl |||https://mailman.nikhef.nl/mailman/listinfo/bfys-physics || ||--- || || Francesco Dettori || Nikhef - VU || || +31 20 592 2230 (Nikhef) || +41 22 76 76284 (CERN) | |--- | Francesco Dettori | Nikhef - VU | | +31 20 592 2230 (Nikhef) | +41 22 76 76284 (CERN) | |_______________________________________________ |Bfys-physics mailing list |Bfys-physics@nikhef.nl |https://mailman.nikhef.nl/mailman/listinfo/bfys-physics

--- Francesco Dettori Nikhef - VU

+31 20 592 2230 (Nikhef) +41 22 76 76284 (CERN)

Hi Rob,

I think that the only claim is that "there is 0.5% probability of this to happen given the standard model", which means only that if you repeat this full experiment 20 times 1 will give you this result if Nature works as the Standard Model. It does not (as all the frequentist statements) say nothing about the probability of the theory being true given the data.

cheers, Francesco

|I was assuming that all the measurements together are measuring the same |thing, that is "does the SM work", but if we formulate instead 24 different |questions, "does the SM work here, there, over there, in this corner" etc., |then the 0.5% is obviously very correct. | |I think what's there in the paper is fine, but wonder always if it isn't |stated too strongly, | |This statement would be incorrect: "we find 0.5%, which means we are 99.5% |confident the SM is wrong" | |What we actually have is: 23 different measurements that the SM is perfectly |fine, and one measurement which has a reasonable statistical deviation, |given that we measure 24 things. So we are very certain the SM is correct. | |Thanks, | |Rob | |------------------------------------------ |Robert Lambert |FOM-VU-NIKHEF-Bfys LHCb |Email: rob.lambert@cern.ch |------------------------------------------ |Nikhef N251 |Tel: +31 20 592 2131 |Fax: +31 20 592 5155 |------------------------------------------ |CERN, 13-1-018 |Tel: +41 22 767 4024 |Fax: +41 22 766 8109 |------------------------------------------ | |________________________________________ |From: Francesco Dettori [fdettori@nikhef.nl] |Sent: 02 July 2013 12:43 |To: bfys-physics@nikhef.nl |Cc: Rob Lambert; Diego Martinez Santos; Marcel Merk |Subject: Re: [Bfys-physics] B->K*mumu paper | |Hi, | |I actually do not agree. The 0.5% is the correct percentage to quote computed |from the correct distribution. |Chebychev inequality is more referring to the opposite: you would not expect |that more than 1/k^2 would fall outside k sigmas. However that does not tell |you how probably that is. | |On the other side the physics importance is relevant: the SM cannot be |excluded if you consider those measurements as independent. |However those are not independent because they are sensitive to different |coefficients. | |my two cents, |cheers |Francesco | |On Tuesday 02 July 2013 10:22:28 Rob Lambert wrote: ||Hi again, || ||Well, for me, concentrating on the 0.5% is over-stating the evidence. The SM ||is still perfectly valid if one of 24 measurements shows a 3.7 sigma ||discrepancy. Perhaps an added line (footnote?) with respect to the Chebychev ||inequality would be prudent in the paper ("Using the rule-of-thumb of ||Chebychev's inequality, 7% of the 24 measured values could be expected to ||fall outside of the 3.7 sigma range, and thus our current results show no ||significant inconsistency with the Standard Model."). || ||Thanks, || ||Rob || ||------------------------------------------ ||Robert Lambert ||FOM-VU-NIKHEF-Bfys LHCb ||Email: rob.lambert@cern.ch ||------------------------------------------ ||Nikhef N251 ||Tel: +31 20 592 2131 ||Fax: +31 20 592 5155 ||------------------------------------------ ||CERN, 13-1-018 ||Tel: +41 22 767 4024 ||Fax: +41 22 766 8109 ||------------------------------------------ || ||________________________________________ ||From: Francesco Dettori [fdettori@nikhef.nl] ||Sent: 02 July 2013 11:58 ||To: Diego Martinez Santos ||Cc: bfys-physics@nikhef.nl; Rob Lambert; Marcel Merk ||Subject: Re: [Bfys-physics] B->K*mumu paper || ||Ciao, || ||yes I perfectly agree with you, I was just trying to summarise possible Rob ||points :) || || ||Francesco || ||On Tuesday 02 July 2013 09:56:21 Diego Martinez Santos wrote: |||Ciao ||| |||I'm not sure they speak about "global tension", I mean, what they say |||is different from making a chi2 with 24 DoF. From the abstract: |||"Considering the 24 measurements as independent, the probability to observe |||such a discrepancy in one of them is 0:5%." ||| |||i.e, I guess they just mean that in 0.5% of experiments made of 24 |||indepenendent measurements you have at least a discrepancy of 3.7 s or more |||(and seem consistent with the toy I made). ||| |||Cheers; ||| |||Diego ||| |||________________________________________ |||Dende: Francesco Dettori [fdettori@nikhef.nl] |||Enviado: martes, 02 de xullo de 2013 11:46 |||Ata: bfys-physics@nikhef.nl |||Cc: Rob Lambert; Diego Martinez Santos; Marcel Merk |||Asunto: Re: [Bfys-physics] B->K*mumu paper ||| |||Ciao Rob, ||| |||I don't understand completely what is your point: |||- that they should not claim any discrepancy because globally the agreement |||is fine |||- or that the final claim of 2.8 sigma global tension is wrong. ||| |||cheers, |||Francesco ||| |||On Tuesday 02 July 2013 09:33:44 Rob Lambert wrote: ||||Hi Again, |||| ||||Another way to think about it is to fit a Gaussian to the 24 data points, ||||and analyse whether one of them is an outlier or not. This is analytic ||||given a Gaussian distribution with one outlier and reduces to the ||||Chebychev ||||inequality (or at least, I thought it did). |||| ||||Thanks, |||| ||||Rob |||| ||||------------------------------------------ ||||Robert Lambert ||||FOM-VU-NIKHEF-Bfys LHCb ||||Email: rob.lambert@cern.ch ||||------------------------------------------ ||||Nikhef N251 ||||Tel: +31 20 592 2131 ||||Fax: +31 20 592 5155 ||||------------------------------------------ ||||CERN, 13-1-018 ||||Tel: +41 22 767 4024 ||||Fax: +41 22 766 8109 ||||------------------------------------------ ||||________________________________ ||||From: bfys-physics-bounces@nikhef.nl [bfys-physics-bounces@nikhef.nl] on ||||behalf of Rob Lambert [Rob.Lambert@cern.ch] Sent: 02 July 2013 11:30 ||||To: Diego Martinez Santos; Marcel Merk ||||Cc: bfys-physics@nikhef.nl ||||Subject: Re: [Bfys-physics] B->K*mumu paper |||| ||||(24*0.0001*2) |||| |||| ^ missed a zero .... |||| ||||Rob |||| ||||------------------------------------------ ||||Robert Lambert ||||FOM-VU-NIKHEF-Bfys LHCb ||||Email: rob.lambert@cern.ch ||||------------------------------------------ ||||Nikhef N251 ||||Tel: +31 20 592 2131 ||||Fax: +31 20 592 5155 ||||------------------------------------------ ||||CERN, 13-1-018 ||||Tel: +41 22 767 4024 ||||Fax: +41 22 766 8109 ||||------------------------------------------ ||||________________________________ ||||From: Rob Lambert ||||Sent: 02 July 2013 11:29 ||||To: Diego Martinez Santos; Marcel Merk ||||Cc: bfys-physics@nikhef.nl ||||Subject: RE: [Bfys-physics] B->K*mumu paper |||| ||||Hi Diego, |||| ||||Well, I'm 100% sure that's right because it's just 24*0.001*2, which is ||||the ||||integrated probability for a two-sided Gaussian test, but the question ||||isn't ||||"what's the probability of that happening" which is actually "1", because ||||it ||||did actually happen, but "how many such deviations would you need to see ||||before you can invalidate the null hypothesis of no deviation", and this ||||number is given by Chebychev's equality... well, I thought it was, anyway. |||| ||||Thanks, |||| ||||Rob |||| ||||------------------------------------------ ||||Robert Lambert ||||FOM-VU-NIKHEF-Bfys LHCb ||||Email: rob.lambert@cern.ch ||||------------------------------------------ ||||Nikhef N251 ||||Tel: +31 20 592 2131 ||||Fax: +31 20 592 5155 ||||------------------------------------------ ||||CERN, 13-1-018 ||||Tel: +41 22 767 4024 ||||Fax: +41 22 766 8109 ||||------------------------------------------ ||||________________________________ ||||From: Diego Martinez Santos ||||Sent: 02 July 2013 11:17 ||||To: Rob Lambert; Marcel Merk ||||Cc: bfys-physics@nikhef.nl ||||Subject: RE: [Bfys-physics] B->K*mumu paper |||| |||| ||||(PS: 0.566 \pm 0.033. forgot the % of course) |||| ||||________________________________ ||||Dende: Diego Martinez Santos ||||Enviado: martes, 02 de xullo de 2013 11:16 ||||Ata: Rob Lambert; Marcel Merk ||||Cc: bfys-physics@nikhef.nl ||||Asunto: RE: [Bfys-physics] B->K*mumu paper |||| |||| ||||Hi |||| |||| |||| ||||I;ve just made quick toyMC, I get that the probability(maxSigma) > 3.7 for ||||experiments consisting |||| ||||of 24 measurements is 0.566 \pm 0.033. |||| |||| |||| |||| |||| ||||Voila the code (needs SetupUrania): |||| |||| |||| ||||from scipy import random as rnd ||||from Urania import * ||||AccessPackage("Bs2MuMu") |||| ||||from RTuple import * |||| ||||def do_set_of_experiments(N): |||| l = [] |||| |||| for i in range(N): |||| l.append(abs(rnd.normal())) |||| |||| l.sort() |||| l.reverse() |||| return l[0] |||| ||||def do_test(N, Nexp = 50000): |||| tup = RTuple("File_test",["maxSigma/F"]) |||| |||| for i in range(Nexp): |||| tup.fillItem("maxSigma",do_set_of_experiments(24)) |||| tup.fill() |||| |||| tup.close() |||| ||||Then you just call do_test(24) and analyse the produced Ntuple. |||| |||| |||| |||| |||| |||| |||| ||||Cheers; |||| |||| |||| ||||Diego |||| |||| |||| |||| |||| |||| |||| |||| |||| ||||________________________________ ||||Dende: bfys-physics-bounces@nikhef.nl [bfys-physics-bounces@nikhef.nl] en ||||nome de Rob Lambert [Rob.Lambert@cern.ch] Enviado: martes, 02 de xullo de ||||2013 10:53 ||||Ata: Marcel Merk ||||Cc: bfys-physics@nikhef.nl ||||Asunto: Re: [Bfys-physics] B->K*mumu paper |||| ||||Yes. Chebychev's inequality gives an estimate for how often a deviation ||||this ||||large this should happen, and it seems to be perfectly within the realms ||||of ||||probability, in fact it seems to be expected, and even two such deviations ||||would not be strange. |||| ||||Gerhard might want to correct me here, though... |||| ||||Cheers, |||| ||||Rob |||| ||||------------------------------------------ ||||Robert Lambert ||||FOM-VU-NIKHEF-Bfys LHCb ||||Email: rob.lambert@cern.ch ||||------------------------------------------ ||||Nikhef N251 ||||Tel: +31 20 592 2131 ||||Fax: +31 20 592 5155 ||||------------------------------------------ ||||CERN, 13-1-018 ||||Tel: +41 22 767 4024 ||||Fax: +41 22 766 8109 ||||------------------------------------------ ||||________________________________ ||||From: Marcel Merk [marcel.merk@nikhef.nl] ||||Sent: 02 July 2013 10:51 ||||To: Rob Lambert ||||Cc: bfys-physics@nikhef.nl ||||Subject: Re: [Bfys-physics] B->K*mumu paper |||| ||||Hi Rob, ||||Let me just clarify: you question the claim of 0.5%, is that correct? ||||just to make sure I understand which point you comment on. ||||cheers, ||||- Marcel |||| |||| ||||On 2 July 2013 10:06, Rob Lambert ||||<Rob.Lambert@cern.chmailto:Rob.Lambert@cern.ch> wrote: Erm, just some ||||statistical queries.. what about chebychev's inequality: |||| ||||http://en.wikipedia.org/wiki/Chebyshev%27s_inequality |||| ||||"No more than 1/k^{2} of the distribution's values can be more than k ||||standard deviations away from the mean" |||| ||||In this case it is 1/(3.7*3.7) which is 7%, and 24*7% is 1.75, so no more ||||than 1.75 of the observables should be this far away. One deviation this ||||large is perfectly expected from Chebychev's inequality. |||| ||||Thanks, |||| ||||Rob |||| ||||------------------------------------------ ||||Robert Lambert ||||FOM-VU-NIKHEF-Bfys LHCb ||||Email: rob.lambert@cern.chmailto:rob.lambert@cern.ch ||||------------------------------------------ ||||Nikhef N251 ||||Tel: +31 20 592 2131tel:%2B31%2020%20592%202131 ||||Fax: +31 20 592 5155tel:%2B31%2020%20592%205155 ||||------------------------------------------ ||||CERN, 13-1-018 ||||Tel: +41 22 767 4024tel:%2B41%2022%20767%204024 ||||Fax: +41 22 766 8109tel:%2B41%2022%20766%208109 ||||------------------------------------------ ||||________________________________ ||||From: ||||bfys-physics-bounces@nikhef.nlmailto:bfys-physics-bounces@nikhef.nl ||||[bfys-physics-bounces@nikhef.nlmailto:bfys-physics-bounces@nikhef.nl] on ||||behalf of Marcel Merk ||||[marcel.merk@nikhef.nlmailto:marcel.merk@nikhef.nl] ||||Sent: 01 July 2013 18:01 ||||To: bfys-physics@nikhef.nlmailto:bfys-physics@nikhef.nl ||||Subject: [Bfys-physics] B->K*mumu paper |||| ||||Dear bfys-physics friend, ||||This is to let you know that I collect comments to the paper below. ||||To stimulate you to read it: we claim that the probability that the ||||measurement is consistent with the Standard Model is only 0.5%. A local ||||discrepancy of 3.7 sigma is observed. |||| |||| LHCb PAPER-2013-037: |||| "Measurement of form factor independent observables |||| |||| in the decay $B^0\to K^{*0}\mu^+\mu^-$" |||| |||| Link : https://cds.cern.ch/record/1557918 |||| Deadline : 09-Jul-2013 |||| ||||best regards, ||||- Marcel |||| ||||_______________________________________________ ||||Bfys-physics mailing list ||||Bfys-physics@nikhef.nlmailto:Bfys-physics@nikhef.nl ||||https://mailman.nikhef.nl/mailman/listinfo/bfys-physics ||| |||--- ||| ||| Francesco Dettori ||| Nikhef - VU ||| ||| +31 20 592 2230 (Nikhef) ||| +41 22 76 76284 (CERN) || ||--- || || Francesco Dettori || Nikhef - VU || || +31 20 592 2230 (Nikhef) || +41 22 76 76284 (CERN) || ||_______________________________________________ ||Bfys-physics mailing list ||Bfys-physics@nikhef.nl ||https://mailman.nikhef.nl/mailman/listinfo/bfys-physics | |--- | Francesco Dettori | Nikhef - VU | | +31 20 592 2230 (Nikhef) | +41 22 76 76284 (CERN)

--- Francesco Dettori Nikhef - VU

+31 20 592 2230 (Nikhef) +41 22 76 76284 (CERN)

Hi Rob

I don't agree. In particular, it seems that chebychev thing does not make any assumption of Gaussian errors. And w/o knowing that, it can only give an upper(lower) limit of actual p-values, i.e, understated significances. I don't think has nothing to do with Bayesian/frequentist discussion. And looking what the link says: "under Chebyshev's Inequality just 75% of values lie within two standard deviations of the mean and 89% of values within three standard deviations.[1][2]"

then those "p-values" are rather different from what a reader would expect. So, to me, 0.5% seems a correct frequentist p-value. Only possible criticism I could think of, is that it is chosen a posteriori, and perhaps subject to some bias because of that.

Cheers;

Diego

________________________________________ Dende: Rob Lambert Enviado: martes, 02 de xullo de 2013 13:15 Ata: Francesco Dettori Cc: bfys-physics@nikhef.nl; Diego Martinez Santos; Marcel Merk Asunto: RE: [Bfys-physics] B->K*mumu paper

Hi Francesco,

Yes, you're right it's probably a Bayesian vs. frequentist difference in the end.

Searching for deviations from your model has a known power, which we are reminded of by the Chebychev inequality, which first assumes all measurements are from the same distribution (in this case, mean of zero, deviation of 1), and asks how many standard deviations can you see before you get worried. for 25 measurements it is 5-sigma.

So, we shouldn't be worried about this deviation yet, and I'm not sure if this comes across in the paper.

Thanks,

Rob

------------------------------------------ Robert Lambert FOM-VU-NIKHEF-Bfys LHCb Email: rob.lambert@cern.ch ------------------------------------------ Nikhef N251 Tel: +31 20 592 2131 Fax: +31 20 592 5155 ------------------------------------------ CERN, 13-1-018 Tel: +41 22 767 4024 Fax: +41 22 766 8109 ------------------------------------------

________________________________________ From: Francesco Dettori [fdettori@nikhef.nl] Sent: 02 July 2013 13:03 To: Rob Lambert Cc: bfys-physics@nikhef.nl; Diego Martinez Santos; Marcel Merk Subject: Re: [Bfys-physics] B->K*mumu paper

Hi Rob,

I think that the only claim is that "there is 0.5% probability of this to happen given the standard model", which means only that if you repeat this full experiment 20 times 1 will give you this result if Nature works as the Standard Model. It does not (as all the frequentist statements) say nothing about the probability of the theory being true given the data.

cheers, Francesco

|I was assuming that all the measurements together are measuring the same |thing, that is "does the SM work", but if we formulate instead 24 different |questions, "does the SM work here, there, over there, in this corner" etc., |then the 0.5% is obviously very correct. | |I think what's there in the paper is fine, but wonder always if it isn't |stated too strongly, | |This statement would be incorrect: "we find 0.5%, which means we are 99.5% |confident the SM is wrong" | |What we actually have is: 23 different measurements that the SM is perfectly |fine, and one measurement which has a reasonable statistical deviation, |given that we measure 24 things. So we are very certain the SM is correct. | |Thanks, | |Rob | |------------------------------------------ |Robert Lambert |FOM-VU-NIKHEF-Bfys LHCb |Email: rob.lambert@cern.ch |------------------------------------------ |Nikhef N251 |Tel: +31 20 592 2131 |Fax: +31 20 592 5155 |------------------------------------------ |CERN, 13-1-018 |Tel: +41 22 767 4024 |Fax: +41 22 766 8109 |------------------------------------------ | |________________________________________ |From: Francesco Dettori [fdettori@nikhef.nl] |Sent: 02 July 2013 12:43 |To: bfys-physics@nikhef.nl |Cc: Rob Lambert; Diego Martinez Santos; Marcel Merk |Subject: Re: [Bfys-physics] B->K*mumu paper | |Hi, | |I actually do not agree. The 0.5% is the correct percentage to quote computed |from the correct distribution. |Chebychev inequality is more referring to the opposite: you would not expect |that more than 1/k^2 would fall outside k sigmas. However that does not tell |you how probably that is. | |On the other side the physics importance is relevant: the SM cannot be |excluded if you consider those measurements as independent. |However those are not independent because they are sensitive to different |coefficients. | |my two cents, |cheers |Francesco | |On Tuesday 02 July 2013 10:22:28 Rob Lambert wrote: ||Hi again, || ||Well, for me, concentrating on the 0.5% is over-stating the evidence. The SM ||is still perfectly valid if one of 24 measurements shows a 3.7 sigma ||discrepancy. Perhaps an added line (footnote?) with respect to the Chebychev ||inequality would be prudent in the paper ("Using the rule-of-thumb of ||Chebychev's inequality, 7% of the 24 measured values could be expected to ||fall outside of the 3.7 sigma range, and thus our current results show no ||significant inconsistency with the Standard Model."). || ||Thanks, || ||Rob || ||------------------------------------------ ||Robert Lambert ||FOM-VU-NIKHEF-Bfys LHCb ||Email: rob.lambert@cern.ch ||------------------------------------------ ||Nikhef N251 ||Tel: +31 20 592 2131 ||Fax: +31 20 592 5155 ||------------------------------------------ ||CERN, 13-1-018 ||Tel: +41 22 767 4024 ||Fax: +41 22 766 8109 ||------------------------------------------ || ||________________________________________ ||From: Francesco Dettori [fdettori@nikhef.nl] ||Sent: 02 July 2013 11:58 ||To: Diego Martinez Santos ||Cc: bfys-physics@nikhef.nl; Rob Lambert; Marcel Merk ||Subject: Re: [Bfys-physics] B->K*mumu paper || ||Ciao, || ||yes I perfectly agree with you, I was just trying to summarise possible Rob ||points :) || || ||Francesco || ||On Tuesday 02 July 2013 09:56:21 Diego Martinez Santos wrote: |||Ciao ||| |||I'm not sure they speak about "global tension", I mean, what they say |||is different from making a chi2 with 24 DoF. From the abstract: |||"Considering the 24 measurements as independent, the probability to observe |||such a discrepancy in one of them is 0:5%." ||| |||i.e, I guess they just mean that in 0.5% of experiments made of 24 |||indepenendent measurements you have at least a discrepancy of 3.7 s or more |||(and seem consistent with the toy I made). ||| |||Cheers; ||| |||Diego ||| |||________________________________________ |||Dende: Francesco Dettori [fdettori@nikhef.nl] |||Enviado: martes, 02 de xullo de 2013 11:46 |||Ata: bfys-physics@nikhef.nl |||Cc: Rob Lambert; Diego Martinez Santos; Marcel Merk |||Asunto: Re: [Bfys-physics] B->K*mumu paper ||| |||Ciao Rob, ||| |||I don't understand completely what is your point: |||- that they should not claim any discrepancy because globally the agreement |||is fine |||- or that the final claim of 2.8 sigma global tension is wrong. ||| |||cheers, |||Francesco ||| |||On Tuesday 02 July 2013 09:33:44 Rob Lambert wrote: ||||Hi Again, |||| ||||Another way to think about it is to fit a Gaussian to the 24 data points, ||||and analyse whether one of them is an outlier or not. This is analytic ||||given a Gaussian distribution with one outlier and reduces to the ||||Chebychev ||||inequality (or at least, I thought it did). |||| ||||Thanks, |||| ||||Rob |||| ||||------------------------------------------ ||||Robert Lambert ||||FOM-VU-NIKHEF-Bfys LHCb ||||Email: rob.lambert@cern.ch ||||------------------------------------------ ||||Nikhef N251 ||||Tel: +31 20 592 2131 ||||Fax: +31 20 592 5155 ||||------------------------------------------ ||||CERN, 13-1-018 ||||Tel: +41 22 767 4024 ||||Fax: +41 22 766 8109 ||||------------------------------------------ ||||________________________________ ||||From: bfys-physics-bounces@nikhef.nl [bfys-physics-bounces@nikhef.nl] on ||||behalf of Rob Lambert [Rob.Lambert@cern.ch] Sent: 02 July 2013 11:30 ||||To: Diego Martinez Santos; Marcel Merk ||||Cc: bfys-physics@nikhef.nl ||||Subject: Re: [Bfys-physics] B->K*mumu paper |||| ||||(24*0.0001*2) |||| |||| ^ missed a zero .... |||| ||||Rob |||| ||||------------------------------------------ ||||Robert Lambert ||||FOM-VU-NIKHEF-Bfys LHCb ||||Email: rob.lambert@cern.ch ||||------------------------------------------ ||||Nikhef N251 ||||Tel: +31 20 592 2131 ||||Fax: +31 20 592 5155 ||||------------------------------------------ ||||CERN, 13-1-018 ||||Tel: +41 22 767 4024 ||||Fax: +41 22 766 8109 ||||------------------------------------------ ||||________________________________ ||||From: Rob Lambert ||||Sent: 02 July 2013 11:29 ||||To: Diego Martinez Santos; Marcel Merk ||||Cc: bfys-physics@nikhef.nl ||||Subject: RE: [Bfys-physics] B->K*mumu paper |||| ||||Hi Diego, |||| ||||Well, I'm 100% sure that's right because it's just 24*0.001*2, which is ||||the ||||integrated probability for a two-sided Gaussian test, but the question ||||isn't ||||"what's the probability of that happening" which is actually "1", because ||||it ||||did actually happen, but "how many such deviations would you need to see ||||before you can invalidate the null hypothesis of no deviation", and this ||||number is given by Chebychev's equality... well, I thought it was, anyway. |||| ||||Thanks, |||| ||||Rob |||| ||||------------------------------------------ ||||Robert Lambert ||||FOM-VU-NIKHEF-Bfys LHCb ||||Email: rob.lambert@cern.ch ||||------------------------------------------ ||||Nikhef N251 ||||Tel: +31 20 592 2131 ||||Fax: +31 20 592 5155 ||||------------------------------------------ ||||CERN, 13-1-018 ||||Tel: +41 22 767 4024 ||||Fax: +41 22 766 8109 ||||------------------------------------------ ||||________________________________ ||||From: Diego Martinez Santos ||||Sent: 02 July 2013 11:17 ||||To: Rob Lambert; Marcel Merk ||||Cc: bfys-physics@nikhef.nl ||||Subject: RE: [Bfys-physics] B->K*mumu paper |||| |||| ||||(PS: 0.566 \pm 0.033. forgot the % of course) |||| ||||________________________________ ||||Dende: Diego Martinez Santos ||||Enviado: martes, 02 de xullo de 2013 11:16 ||||Ata: Rob Lambert; Marcel Merk ||||Cc: bfys-physics@nikhef.nl ||||Asunto: RE: [Bfys-physics] B->K*mumu paper |||| |||| ||||Hi |||| |||| |||| ||||I;ve just made quick toyMC, I get that the probability(maxSigma) > 3.7 for ||||experiments consisting |||| ||||of 24 measurements is 0.566 \pm 0.033. |||| |||| |||| |||| |||| ||||Voila the code (needs SetupUrania): |||| |||| |||| ||||from scipy import random as rnd ||||from Urania import * ||||AccessPackage("Bs2MuMu") |||| ||||from RTuple import * |||| ||||def do_set_of_experiments(N): |||| l = [] |||| |||| for i in range(N): |||| l.append(abs(rnd.normal())) |||| |||| l.sort() |||| l.reverse() |||| return l[0] |||| ||||def do_test(N, Nexp = 50000): |||| tup = RTuple("File_test",["maxSigma/F"]) |||| |||| for i in range(Nexp): |||| tup.fillItem("maxSigma",do_set_of_experiments(24)) |||| tup.fill() |||| |||| tup.close() |||| ||||Then you just call do_test(24) and analyse the produced Ntuple. |||| |||| |||| |||| |||| |||| |||| ||||Cheers; |||| |||| |||| ||||Diego |||| |||| |||| |||| |||| |||| |||| |||| |||| ||||________________________________ ||||Dende: bfys-physics-bounces@nikhef.nl [bfys-physics-bounces@nikhef.nl] en ||||nome de Rob Lambert [Rob.Lambert@cern.ch] Enviado: martes, 02 de xullo de ||||2013 10:53 ||||Ata: Marcel Merk ||||Cc: bfys-physics@nikhef.nl ||||Asunto: Re: [Bfys-physics] B->K*mumu paper |||| ||||Yes. Chebychev's inequality gives an estimate for how often a deviation ||||this ||||large this should happen, and it seems to be perfectly within the realms ||||of ||||probability, in fact it seems to be expected, and even two such deviations ||||would not be strange. |||| ||||Gerhard might want to correct me here, though... |||| ||||Cheers, |||| ||||Rob |||| ||||------------------------------------------ ||||Robert Lambert ||||FOM-VU-NIKHEF-Bfys LHCb ||||Email: rob.lambert@cern.ch ||||------------------------------------------ ||||Nikhef N251 ||||Tel: +31 20 592 2131 ||||Fax: +31 20 592 5155 ||||------------------------------------------ ||||CERN, 13-1-018 ||||Tel: +41 22 767 4024 ||||Fax: +41 22 766 8109 ||||------------------------------------------ ||||________________________________ ||||From: Marcel Merk [marcel.merk@nikhef.nl] ||||Sent: 02 July 2013 10:51 ||||To: Rob Lambert ||||Cc: bfys-physics@nikhef.nl ||||Subject: Re: [Bfys-physics] B->K*mumu paper |||| ||||Hi Rob, ||||Let me just clarify: you question the claim of 0.5%, is that correct? ||||just to make sure I understand which point you comment on. ||||cheers, ||||- Marcel |||| |||| ||||On 2 July 2013 10:06, Rob Lambert ||||<Rob.Lambert@cern.chmailto:Rob.Lambert@cern.ch> wrote: Erm, just some ||||statistical queries.. what about chebychev's inequality: |||| ||||http://en.wikipedia.org/wiki/Chebyshev%27s_inequality |||| ||||"No more than 1/k^{2} of the distribution's values can be more than k ||||standard deviations away from the mean" |||| ||||In this case it is 1/(3.7*3.7) which is 7%, and 24*7% is 1.75, so no more ||||than 1.75 of the observables should be this far away. One deviation this ||||large is perfectly expected from Chebychev's inequality. |||| ||||Thanks, |||| ||||Rob |||| ||||------------------------------------------ ||||Robert Lambert ||||FOM-VU-NIKHEF-Bfys LHCb ||||Email: rob.lambert@cern.chmailto:rob.lambert@cern.ch ||||------------------------------------------ ||||Nikhef N251 ||||Tel: +31 20 592 2131tel:%2B31%2020%20592%202131 ||||Fax: +31 20 592 5155tel:%2B31%2020%20592%205155 ||||------------------------------------------ ||||CERN, 13-1-018 ||||Tel: +41 22 767 4024tel:%2B41%2022%20767%204024 ||||Fax: +41 22 766 8109tel:%2B41%2022%20766%208109 ||||------------------------------------------ ||||________________________________ ||||From: ||||bfys-physics-bounces@nikhef.nlmailto:bfys-physics-bounces@nikhef.nl ||||[bfys-physics-bounces@nikhef.nlmailto:bfys-physics-bounces@nikhef.nl] on ||||behalf of Marcel Merk ||||[marcel.merk@nikhef.nlmailto:marcel.merk@nikhef.nl] ||||Sent: 01 July 2013 18:01 ||||To: bfys-physics@nikhef.nlmailto:bfys-physics@nikhef.nl ||||Subject: [Bfys-physics] B->K*mumu paper |||| ||||Dear bfys-physics friend, ||||This is to let you know that I collect comments to the paper below. ||||To stimulate you to read it: we claim that the probability that the ||||measurement is consistent with the Standard Model is only 0.5%. A local ||||discrepancy of 3.7 sigma is observed. |||| |||| LHCb PAPER-2013-037: |||| "Measurement of form factor independent observables |||| |||| in the decay $B^0\to K^{*0}\mu^+\mu^-$" |||| |||| Link : https://cds.cern.ch/record/1557918 |||| Deadline : 09-Jul-2013 |||| ||||best regards, ||||- Marcel |||| ||||_______________________________________________ ||||Bfys-physics mailing list ||||Bfys-physics@nikhef.nlmailto:Bfys-physics@nikhef.nl ||||https://mailman.nikhef.nl/mailman/listinfo/bfys-physics ||| |||--- ||| ||| Francesco Dettori ||| Nikhef - VU ||| ||| +31 20 592 2230 (Nikhef) ||| +41 22 76 76284 (CERN) || ||--- || || Francesco Dettori || Nikhef - VU || || +31 20 592 2230 (Nikhef) || +41 22 76 76284 (CERN) || ||_______________________________________________ ||Bfys-physics mailing list ||Bfys-physics@nikhef.nl ||https://mailman.nikhef.nl/mailman/listinfo/bfys-physics | |--- | Francesco Dettori | Nikhef - VU | | +31 20 592 2230 (Nikhef) | +41 22 76 76284 (CERN)

--- Francesco Dettori Nikhef - VU

+31 20 592 2230 (Nikhef) +41 22 76 76284 (CERN)

Hi

Yeah. I guess the authors should have defined a priori which would be the actual test. But in any case LHCb has 126 papers and 110 Conf notes. Thus (ignoring the possibility of unacounted systematic or actual NP), it's kind of reasonable to have one result with a 0.5% p-value even if everything is SM like.

Cheers;

Diego ________________________________________ Dende: Rob Lambert Enviado: martes, 02 de xullo de 2013 14:06 Ata: Diego Martinez Santos; Francesco Dettori Cc: bfys-physics@nikhef.nl; Marcel Merk Asunto: RE: [Bfys-physics] B->K*mumu paper

Hi Diego,

Thanks for the explanation. I think I basically agree with what you say, but I am still more conservative on how to interpret a single (small) deviation in a set of measurements which agree.

Cheers,

Rob

------------------------------------------ Robert Lambert FOM-VU-NIKHEF-Bfys LHCb Email: rob.lambert@cern.ch ------------------------------------------ Nikhef N251 Tel: +31 20 592 2131 Fax: +31 20 592 5155 ------------------------------------------ CERN, 13-1-018 Tel: +41 22 767 4024 Fax: +41 22 766 8109 ------------------------------------------

________________________________________ From: Diego Martinez Santos Sent: 02 July 2013 13:54 To: Rob Lambert; Francesco Dettori Cc: bfys-physics@nikhef.nl; Marcel Merk Subject: RE: [Bfys-physics] B->K*mumu paper

Hi Rob

I don't agree. In particular, it seems that chebychev thing does not make any assumption of Gaussian errors. And w/o knowing that, it can only give an upper(lower) limit of actual p-values, i.e, understated significances. I don't think has nothing to do with Bayesian/frequentist discussion. And looking what the link says: "under Chebyshev's Inequality just 75% of values lie within two standard deviations of the mean and 89% of values within three standard deviations.[1][2]"

then those "p-values" are rather different from what a reader would expect. So, to me, 0.5% seems a correct frequentist p-value. Only possible criticism I could think of, is that it is chosen a posteriori, and perhaps subject to some bias because of that.

Cheers;

Diego

________________________________________ Dende: Rob Lambert Enviado: martes, 02 de xullo de 2013 13:15 Ata: Francesco Dettori Cc: bfys-physics@nikhef.nl; Diego Martinez Santos; Marcel Merk Asunto: RE: [Bfys-physics] B->K*mumu paper

Hi Francesco,

Yes, you're right it's probably a Bayesian vs. frequentist difference in the end.

Searching for deviations from your model has a known power, which we are reminded of by the Chebychev inequality, which first assumes all measurements are from the same distribution (in this case, mean of zero, deviation of 1), and asks how many standard deviations can you see before you get worried. for 25 measurements it is 5-sigma.

So, we shouldn't be worried about this deviation yet, and I'm not sure if this comes across in the paper.

Thanks,

Rob

------------------------------------------ Robert Lambert FOM-VU-NIKHEF-Bfys LHCb Email: rob.lambert@cern.ch ------------------------------------------ Nikhef N251 Tel: +31 20 592 2131 Fax: +31 20 592 5155 ------------------------------------------ CERN, 13-1-018 Tel: +41 22 767 4024 Fax: +41 22 766 8109 ------------------------------------------

________________________________________ From: Francesco Dettori [fdettori@nikhef.nl] Sent: 02 July 2013 13:03 To: Rob Lambert Cc: bfys-physics@nikhef.nl; Diego Martinez Santos; Marcel Merk Subject: Re: [Bfys-physics] B->K*mumu paper

Hi Rob,

I think that the only claim is that "there is 0.5% probability of this to happen given the standard model", which means only that if you repeat this full experiment 20 times 1 will give you this result if Nature works as the Standard Model. It does not (as all the frequentist statements) say nothing about the probability of the theory being true given the data.

cheers, Francesco

|I was assuming that all the measurements together are measuring the same |thing, that is "does the SM work", but if we formulate instead 24 different |questions, "does the SM work here, there, over there, in this corner" etc., |then the 0.5% is obviously very correct. | |I think what's there in the paper is fine, but wonder always if it isn't |stated too strongly, | |This statement would be incorrect: "we find 0.5%, which means we are 99.5% |confident the SM is wrong" | |What we actually have is: 23 different measurements that the SM is perfectly |fine, and one measurement which has a reasonable statistical deviation, |given that we measure 24 things. So we are very certain the SM is correct. | |Thanks, | |Rob | |------------------------------------------ |Robert Lambert |FOM-VU-NIKHEF-Bfys LHCb |Email: rob.lambert@cern.ch |------------------------------------------ |Nikhef N251 |Tel: +31 20 592 2131 |Fax: +31 20 592 5155 |------------------------------------------ |CERN, 13-1-018 |Tel: +41 22 767 4024 |Fax: +41 22 766 8109 |------------------------------------------ | |________________________________________ |From: Francesco Dettori [fdettori@nikhef.nl] |Sent: 02 July 2013 12:43 |To: bfys-physics@nikhef.nl |Cc: Rob Lambert; Diego Martinez Santos; Marcel Merk |Subject: Re: [Bfys-physics] B->K*mumu paper | |Hi, | |I actually do not agree. The 0.5% is the correct percentage to quote computed |from the correct distribution. |Chebychev inequality is more referring to the opposite: you would not expect |that more than 1/k^2 would fall outside k sigmas. However that does not tell |you how probably that is. | |On the other side the physics importance is relevant: the SM cannot be |excluded if you consider those measurements as independent. |However those are not independent because they are sensitive to different |coefficients. | |my two cents, |cheers |Francesco | |On Tuesday 02 July 2013 10:22:28 Rob Lambert wrote: ||Hi again, || ||Well, for me, concentrating on the 0.5% is over-stating the evidence. The SM ||is still perfectly valid if one of 24 measurements shows a 3.7 sigma ||discrepancy. Perhaps an added line (footnote?) with respect to the Chebychev ||inequality would be prudent in the paper ("Using the rule-of-thumb of ||Chebychev's inequality, 7% of the 24 measured values could be expected to ||fall outside of the 3.7 sigma range, and thus our current results show no ||significant inconsistency with the Standard Model."). || ||Thanks, || ||Rob || ||------------------------------------------ ||Robert Lambert ||FOM-VU-NIKHEF-Bfys LHCb ||Email: rob.lambert@cern.ch ||------------------------------------------ ||Nikhef N251 ||Tel: +31 20 592 2131 ||Fax: +31 20 592 5155 ||------------------------------------------ ||CERN, 13-1-018 ||Tel: +41 22 767 4024 ||Fax: +41 22 766 8109 ||------------------------------------------ || ||________________________________________ ||From: Francesco Dettori [fdettori@nikhef.nl] ||Sent: 02 July 2013 11:58 ||To: Diego Martinez Santos ||Cc: bfys-physics@nikhef.nl; Rob Lambert; Marcel Merk ||Subject: Re: [Bfys-physics] B->K*mumu paper || ||Ciao, || ||yes I perfectly agree with you, I was just trying to summarise possible Rob ||points :) || || ||Francesco || ||On Tuesday 02 July 2013 09:56:21 Diego Martinez Santos wrote: |||Ciao ||| |||I'm not sure they speak about "global tension", I mean, what they say |||is different from making a chi2 with 24 DoF. From the abstract: |||"Considering the 24 measurements as independent, the probability to observe |||such a discrepancy in one of them is 0:5%." ||| |||i.e, I guess they just mean that in 0.5% of experiments made of 24 |||indepenendent measurements you have at least a discrepancy of 3.7 s or more |||(and seem consistent with the toy I made). ||| |||Cheers; ||| |||Diego ||| |||________________________________________ |||Dende: Francesco Dettori [fdettori@nikhef.nl] |||Enviado: martes, 02 de xullo de 2013 11:46 |||Ata: bfys-physics@nikhef.nl |||Cc: Rob Lambert; Diego Martinez Santos; Marcel Merk |||Asunto: Re: [Bfys-physics] B->K*mumu paper ||| |||Ciao Rob, ||| |||I don't understand completely what is your point: |||- that they should not claim any discrepancy because globally the agreement |||is fine |||- or that the final claim of 2.8 sigma global tension is wrong. ||| |||cheers, |||Francesco ||| |||On Tuesday 02 July 2013 09:33:44 Rob Lambert wrote: ||||Hi Again, |||| ||||Another way to think about it is to fit a Gaussian to the 24 data points, ||||and analyse whether one of them is an outlier or not. This is analytic ||||given a Gaussian distribution with one outlier and reduces to the ||||Chebychev ||||inequality (or at least, I thought it did). |||| ||||Thanks, |||| ||||Rob |||| ||||------------------------------------------ ||||Robert Lambert ||||FOM-VU-NIKHEF-Bfys LHCb ||||Email: rob.lambert@cern.ch ||||------------------------------------------ ||||Nikhef N251 ||||Tel: +31 20 592 2131 ||||Fax: +31 20 592 5155 ||||------------------------------------------ ||||CERN, 13-1-018 ||||Tel: +41 22 767 4024 ||||Fax: +41 22 766 8109 ||||------------------------------------------ ||||________________________________ ||||From: bfys-physics-bounces@nikhef.nl [bfys-physics-bounces@nikhef.nl] on ||||behalf of Rob Lambert [Rob.Lambert@cern.ch] Sent: 02 July 2013 11:30 ||||To: Diego Martinez Santos; Marcel Merk ||||Cc: bfys-physics@nikhef.nl ||||Subject: Re: [Bfys-physics] B->K*mumu paper |||| ||||(24*0.0001*2) |||| |||| ^ missed a zero .... |||| ||||Rob |||| ||||------------------------------------------ ||||Robert Lambert ||||FOM-VU-NIKHEF-Bfys LHCb ||||Email: rob.lambert@cern.ch ||||------------------------------------------ ||||Nikhef N251 ||||Tel: +31 20 592 2131 ||||Fax: +31 20 592 5155 ||||------------------------------------------ ||||CERN, 13-1-018 ||||Tel: +41 22 767 4024 ||||Fax: +41 22 766 8109 ||||------------------------------------------ ||||________________________________ ||||From: Rob Lambert ||||Sent: 02 July 2013 11:29 ||||To: Diego Martinez Santos; Marcel Merk ||||Cc: bfys-physics@nikhef.nl ||||Subject: RE: [Bfys-physics] B->K*mumu paper |||| ||||Hi Diego, |||| ||||Well, I'm 100% sure that's right because it's just 24*0.001*2, which is ||||the ||||integrated probability for a two-sided Gaussian test, but the question ||||isn't ||||"what's the probability of that happening" which is actually "1", because ||||it ||||did actually happen, but "how many such deviations would you need to see ||||before you can invalidate the null hypothesis of no deviation", and this ||||number is given by Chebychev's equality... well, I thought it was, anyway. |||| ||||Thanks, |||| ||||Rob |||| ||||------------------------------------------ ||||Robert Lambert ||||FOM-VU-NIKHEF-Bfys LHCb ||||Email: rob.lambert@cern.ch ||||------------------------------------------ ||||Nikhef N251 ||||Tel: +31 20 592 2131 ||||Fax: +31 20 592 5155 ||||------------------------------------------ ||||CERN, 13-1-018 ||||Tel: +41 22 767 4024 ||||Fax: +41 22 766 8109 ||||------------------------------------------ ||||________________________________ ||||From: Diego Martinez Santos ||||Sent: 02 July 2013 11:17 ||||To: Rob Lambert; Marcel Merk ||||Cc: bfys-physics@nikhef.nl ||||Subject: RE: [Bfys-physics] B->K*mumu paper |||| |||| ||||(PS: 0.566 \pm 0.033. forgot the % of course) |||| ||||________________________________ ||||Dende: Diego Martinez Santos ||||Enviado: martes, 02 de xullo de 2013 11:16 ||||Ata: Rob Lambert; Marcel Merk ||||Cc: bfys-physics@nikhef.nl ||||Asunto: RE: [Bfys-physics] B->K*mumu paper |||| |||| ||||Hi |||| |||| |||| ||||I;ve just made quick toyMC, I get that the probability(maxSigma) > 3.7 for ||||experiments consisting |||| ||||of 24 measurements is 0.566 \pm 0.033. |||| |||| |||| |||| |||| ||||Voila the code (needs SetupUrania): |||| |||| |||| ||||from scipy import random as rnd ||||from Urania import * ||||AccessPackage("Bs2MuMu") |||| ||||from RTuple import * |||| ||||def do_set_of_experiments(N): |||| l = [] |||| |||| for i in range(N): |||| l.append(abs(rnd.normal())) |||| |||| l.sort() |||| l.reverse() |||| return l[0] |||| ||||def do_test(N, Nexp = 50000): |||| tup = RTuple("File_test",["maxSigma/F"]) |||| |||| for i in range(Nexp): |||| tup.fillItem("maxSigma",do_set_of_experiments(24)) |||| tup.fill() |||| |||| tup.close() |||| ||||Then you just call do_test(24) and analyse the produced Ntuple. |||| |||| |||| |||| |||| |||| |||| ||||Cheers; |||| |||| |||| ||||Diego |||| |||| |||| |||| |||| |||| |||| |||| |||| ||||________________________________ ||||Dende: bfys-physics-bounces@nikhef.nl [bfys-physics-bounces@nikhef.nl] en ||||nome de Rob Lambert [Rob.Lambert@cern.ch] Enviado: martes, 02 de xullo de ||||2013 10:53 ||||Ata: Marcel Merk ||||Cc: bfys-physics@nikhef.nl ||||Asunto: Re: [Bfys-physics] B->K*mumu paper |||| ||||Yes. Chebychev's inequality gives an estimate for how often a deviation ||||this ||||large this should happen, and it seems to be perfectly within the realms ||||of ||||probability, in fact it seems to be expected, and even two such deviations ||||would not be strange. |||| ||||Gerhard might want to correct me here, though... |||| ||||Cheers, |||| ||||Rob |||| ||||------------------------------------------ ||||Robert Lambert ||||FOM-VU-NIKHEF-Bfys LHCb ||||Email: rob.lambert@cern.ch ||||------------------------------------------ ||||Nikhef N251 ||||Tel: +31 20 592 2131 ||||Fax: +31 20 592 5155 ||||------------------------------------------ ||||CERN, 13-1-018 ||||Tel: +41 22 767 4024 ||||Fax: +41 22 766 8109 ||||------------------------------------------ ||||________________________________ ||||From: Marcel Merk [marcel.merk@nikhef.nl] ||||Sent: 02 July 2013 10:51 ||||To: Rob Lambert ||||Cc: bfys-physics@nikhef.nl ||||Subject: Re: [Bfys-physics] B->K*mumu paper |||| ||||Hi Rob, ||||Let me just clarify: you question the claim of 0.5%, is that correct? ||||just to make sure I understand which point you comment on. ||||cheers, ||||- Marcel |||| |||| ||||On 2 July 2013 10:06, Rob Lambert ||||<Rob.Lambert@cern.chmailto:Rob.Lambert@cern.ch> wrote: Erm, just some ||||statistical queries.. what about chebychev's inequality: |||| ||||http://en.wikipedia.org/wiki/Chebyshev%27s_inequality |||| ||||"No more than 1/k^{2} of the distribution's values can be more than k ||||standard deviations away from the mean" |||| ||||In this case it is 1/(3.7*3.7) which is 7%, and 24*7% is 1.75, so no more ||||than 1.75 of the observables should be this far away. One deviation this ||||large is perfectly expected from Chebychev's inequality. |||| ||||Thanks, |||| ||||Rob |||| ||||------------------------------------------ ||||Robert Lambert ||||FOM-VU-NIKHEF-Bfys LHCb ||||Email: rob.lambert@cern.chmailto:rob.lambert@cern.ch ||||------------------------------------------ ||||Nikhef N251 ||||Tel: +31 20 592 2131tel:%2B31%2020%20592%202131 ||||Fax: +31 20 592 5155tel:%2B31%2020%20592%205155 ||||------------------------------------------ ||||CERN, 13-1-018 ||||Tel: +41 22 767 4024tel:%2B41%2022%20767%204024 ||||Fax: +41 22 766 8109tel:%2B41%2022%20766%208109 ||||------------------------------------------ ||||________________________________ ||||From: ||||bfys-physics-bounces@nikhef.nlmailto:bfys-physics-bounces@nikhef.nl ||||[bfys-physics-bounces@nikhef.nlmailto:bfys-physics-bounces@nikhef.nl] on ||||behalf of Marcel Merk ||||[marcel.merk@nikhef.nlmailto:marcel.merk@nikhef.nl] ||||Sent: 01 July 2013 18:01 ||||To: bfys-physics@nikhef.nlmailto:bfys-physics@nikhef.nl ||||Subject: [Bfys-physics] B->K*mumu paper |||| ||||Dear bfys-physics friend, ||||This is to let you know that I collect comments to the paper below. ||||To stimulate you to read it: we claim that the probability that the ||||measurement is consistent with the Standard Model is only 0.5%. A local ||||discrepancy of 3.7 sigma is observed. |||| |||| LHCb PAPER-2013-037: |||| "Measurement of form factor independent observables |||| |||| in the decay $B^0\to K^{*0}\mu^+\mu^-$" |||| |||| Link : https://cds.cern.ch/record/1557918 |||| Deadline : 09-Jul-2013 |||| ||||best regards, ||||- Marcel |||| ||||_______________________________________________ ||||Bfys-physics mailing list ||||Bfys-physics@nikhef.nlmailto:Bfys-physics@nikhef.nl ||||https://mailman.nikhef.nl/mailman/listinfo/bfys-physics ||| |||--- ||| ||| Francesco Dettori ||| Nikhef - VU ||| ||| +31 20 592 2230 (Nikhef) ||| +41 22 76 76284 (CERN) || ||--- || || Francesco Dettori || Nikhef - VU || || +31 20 592 2230 (Nikhef) || +41 22 76 76284 (CERN) || ||_______________________________________________ ||Bfys-physics mailing list ||Bfys-physics@nikhef.nl ||https://mailman.nikhef.nl/mailman/listinfo/bfys-physics | |--- | Francesco Dettori | Nikhef - VU | | +31 20 592 2230 (Nikhef) | +41 22 76 76284 (CERN)

--- Francesco Dettori Nikhef - VU

+31 20 592 2230 (Nikhef) +41 22 76 76284 (CERN)

Hi Marcel,

I missed the meeting. Here's what I have noted

title: The CERN-PH number cannot be right L.17-20: why the parentheses (Bbar), (K*bar) ? I guess this is copied from the previous paper but not needed here. Table 1: why so early? L. 184: That should be a new paragraph. L. 200: This gets me suspicious. Why are the fit values not identical? "With Ref [8]." -> "With those of Ref [8]." References: check the all and use the template. [5]: backword? [13] and [15] : Augusto is not spelled the same way. [17]: Get that from the template to have Sjöstrand correct. Fig. 4: You seem to have an excess in the first 2 bins in all plots. Do you control your partially reconstructed backgrounds?

Cheers,

Patrick

On 07/05/2013 03:01 PM, Niels Tuning wrote:

Hi Marcel,

I would have the following suggestions:

1. Remove 'Gaussian' from abstract. (What other standard deviation

could it be? )

2. Add a sentence to give a feeling what S5 is, and at the same time

use the occassion to state that a simple counting experiment gives consistent results (as shown in Suppl. Mat. in Fig. 5). How about:

"The value of S5 quantifies the asymetry between events with positive and negative value of cos theta_k for -pi/2<phi<+pi/2, averaged with the opposite assymetry of events with |phi|>3pi/2. This asymmetry as determined from a counting experiment is consistent with the asymmetry S5 determined from the fit."

(And perhaps we should suggest to add Fig.48 of the ANA to the Suppl.Mat!)

Cheers, Niels

On Mon, 1 Jul 2013, Marcel Merk wrote:

...

Dear bfys-physics friend,This is to let you know that I collect comments to the paper below. To stimulate you to read it: we claim that the probability that the measurement is consistent with the Standard Model is only 0.5%. A local discrepancy of 3.7 sigma is observed.

LHCb PAPER-2013-037: "Measurement of form factor independent observables in the decay $B^0\to K^{*0}\mu^+\mu^-$" Link : https://cds.cern.ch/record/1557918 Deadline : 09-Jul-2013

best regards,

• Marcel

---

Bfys-physics mailing list Bfys-physics@nikhef.nl https://mailman.nikhef.nl/mailman/listinfo/bfys-physics