Bayesian statistics Tokyo.R#94

Transcript

1. #94 @kilometer00 2021.09.11 BeginneR Session -- Bayesian statistics --

2. Who！？ Who?

3. Who！？ ・ @kilometer ・Postdoc Researcher (Ph.D. Eng.) ・Neuroscience ・Computational Behavior

   ・Functional brain imaging ・R： ~ 10 years

4. 宣伝!!（書籍の翻訳に参加しました。） 絶賛販売中！

5. 宣伝!!（筆頭論⽂が出版されました！！）

6. BeginneR Session

7. -FU`TTUBSU3 ɾ'SFF ɾ -PXJOTUBMMBUJPODPTUGPSCBTJDFOWJSPONFOU ɾ'VMMSBOHFPGGVODUJPOTGPSEBUBTDJFODF ɾ.BOZFYUFOTJPOT QBDLBHFT ɾ4USPOHDPNNVOJUZˡ QPTJUJPOUBML

8. -FU`TTUBSU3 ɾ'SFF ɾ -PXJOTUBMMBUJPODPTUGPSCBTJDFOWJSPONFOU ɾ'VMMSBOHFPGGVODUJPOTGPSEBUBTDJFODF ɾ.BOZFYUFOTJPOT QBDLBHFT ɾ4USPOHDPNNVOJUZˡ QPTJUJPOUBML https://tokyor.connpass.com/

9. -FU`TTUBSU3 ɾ'SFF ɾ -PXJOTUBMMBUJPODPTUGPSCBTJDFOWJSPONFOU ɾ'VMMSBOHFPGGVODUJPOTGPSEBUBTDJFODF ɾ.BOZFYUFOTJPOT QBDLBHFT ɾ4USPOHDPNNVOJUZˡ QPTJUJPOUBML h0ps://tokyor.connpass.com/

   SXBLBMBOH TMBDLXPSLTQBDF .FNCFSਓ

10. 3Λ࢝ΊΑ͏ 【Step】 1. Install R 2. Install RStudio

11. *OTUBMM3 ☝

12. *OTUBMM34UVEJP ౷߹։ൃ؀ڥ JOUFHSBUFEEFWFMPQNFOUFOWJSPONFOU *%& ☝

13. ☝ *OTUBMM34UVEJP ౷߹։ൃ؀ڥ JOUFHSBUFEEFWFMPQNFOUFOWJSPONFOU *%&

14. )PXUPVTF34UVEJP 4DSJQUFEJUPS $POTPMF &OWJSPONFOU QMPU FUD 1 write 2 select

    3 run(⌘ + ↩) output

15. )PXUPVTF34UVEJP

16. )PXUPVTF34UVEJP

17. > x + y

    [1] 3 4DSJQUFEJUPS $POTPMFPVUQVU )PXUPVTF34UVEJP

18. > x +

    y [1] 4 ಉ͡ม਺໊ʹ୅ೖ͢Δͱ্ॻ͖͞ΕΔ DPNNFOUPVU 4DSJQUFEJUPS $POTPMFPVUQVU )PXUPVTF34UVEJP

19. QBDLBHFT $3"/ 5IF$PNQSFIFOTJWF3"SDIJWF/FUXPSL 0GGJDJBM3QBDLBHFSFQPTJUPSZ h0ps://cran.r-project.org/ 2021.09.04

20. $dyverse: データサイエンス関連パッケージ群をまとめたパッケージ ・dplyr: テーブルデータの加⼯・集計 ・ggplot2:

    グラフの描画 ・stringr: ⽂字列加⼯ ・$dyr: データの整形や変形 ・purrrr: 関数型プログラミング⽤ ・magri7r: パイプ演算⼦%>%を提供 *OTUBMMQBDLBHFGSPN$3"/ QBDLBHFT $3"/ 5IF$PNQSFIFOTJWF3"SDIJWF/FUXPSL 0⒏DJBM3QBDLBHFSFQPTJUPSZ https://cran.r-project.org/

21. 0367*22(4*,1*/.6&41/6 ) $70-98.56.$' 20+5*59&4*,1*/. ) $70-98.56.$' 20+5*59&70-98.56.'###%# !" "UUBDIUIFQBDLBHF QBDLBHFT

    $3"/ 5IF$PNQSFIFOTJWF3"SDIJWF/FUXPSL 0GGJDJBM3QBDLBHFSFQPTJUPSZ h0ps://cran.r-project.org/ *OTUBMMQBDLBHFGSPN$3"/

22. Stan A state-of-the-art platform for statistical modeling R A free

    so4ware environment for sta7s7cal compu7ng and graphics. {rstan} package A pla:orm using stan from R
23. None

24. BeginneR

25. Before After BeginneR Session BeginneR BeginneR

26. BeginneR Advanced Hoxo_m If I have seen further it is

    by standing on the shoulders of Giants. -- Sir Isaac Newton, 1676

27. #94 @kilometer00 BeginneR Session -- Bayesian statistics --

28. Experiment hypothesis observation principle phenotype model data Truth Knowledge f

    X (unknown)

29. “Hypothesis driven” “Data driven” Experimental design A B Front Back

    Right Left VerAcal Up A B

30. Strong hypothesis obs. principle phenotype f Weak hypothesis obs. principle

    phenotype model Complex data f model Simple data “Hypothesis driven” “Data driven” Experimental design X X

31. Strong hypothesis obs. principle phenotype f X Weak hypothesis obs.

    principle phenotype model Complex data f X model Simple data “Hypothesis driven” “Data driven” Experimental design ここが気になる（気になりだす）

32. Hypothesis ObservaEon Truth Knowledge principle phenotype model data Dice with

    α faces (regular polyhedron) ! = 5 ?

33. Dice with α faces ! = 5 $ % =

    ! α = 4 = 0 $ % = ! α = 6 = 1 6 $ % = ! α = 8 = 1 8 $ % = ! α = 12 = 1 12 $ % = ! α = 20 = 1 20 likelihood maximum likelihood

34. Dice with α faces ! = {5, 4, 3, 4,

    2, 1, 2, 3, 1, 4} $ % = ! α = 4 = 0 $ % = ! α = 6 = 1 6!" $ % = ! α = 8 = 1 8!" $ % = ! α = 12 = 1 12!" $ % = ! α = 20 = 1 20!" likelihood maximum likelihood

35. Could you find α ?

    Yes, yes, yes. αis 6!! Why do you think so? Because, arg max! - . α = 6 !! Hmmm......, so......, how about ? $(α = 6) Oh, it is " #!"!! ......nnNNNNO!!! WHAT!!????

36. Hmmm......, so, how about

    ? $(α = 6) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} $ % = ! α = 6 = 1 6!" maximum likelihood ! α = 6 % = & !!??

37. Probability distribution $(% = !) ! % $(% = !|α

    = 6) #(% = '|α) parameter data

38. Probability distribution $(%) ! % arg max! -(2|α) 1 6!"

    α = 6 α = 8 α = 12 $(4) α 4 -(5 = α|2 = .) ! = # α = 20

39. Probability distribuEon $#(%) ! % arg max! -$ (2|α) 1

    6!" $$(4) α 4 -! (5 = α|2 = .) ! = # α = 6 α = 8 α = 12 α = 20

40. Probability distribuEon $#(%) ! % arg max! -$ (2|α) 1

    6!" $$(4) α 4 -! (5 = α|2 = .) ! = # ' 5 : α → & ' 6 : & → α α = 6 α = 8 α = 12 α = 20

41. CondiEonal probability "($) "(&) " $ ∩ & = "(&

    ∩ $)

42. CondiEonal probability "($) "(&) "! $ ∩ & = ""

    (& ∩ $)

43. CondiEonal probability "($) "(&) ! 7 * ∗ ! 8

    , * = ! 7 *|, ∗ ! 8 ,

44. Bayes’ theorem ! 7 *|, = ! 8 , *

    ∗ ! 7 (*) ! 8 , "! $ ∩ & = "" (& ∩ $) ! 7 * ∗ ! 8 , * = ! 7 *|, ∗ ! 8 ,

45. ! 7 *|, = ! 8 , * ∗ !

    7 (*) ! 8 , $! ) = α|+ = ! = $" + = ! ) = α ∗ $! (α) $" ! ' 5 : α → & ' 6 : & → α Bayes’ theorem

46. ! 7 *|, = ! 8 , * ∗ !

    7 (*) ! 8 , $! ) = α|+ = ! = $" + = ! ) = α ∗ $! (α) $" ! ' 5 : α → & ' 6 : & → α likelihood Bayes’ theorem

47. ! 7 *|, = ! 8 , * ∗ !

    7 (*) ! 8 , $! α|! = $" ! α ∗ $! (α) $" ! ' 5 : α → & ' 6 : & → α likelihood Bayes’ theorem

48. ! 7 *|, = ! 8 , * ∗ !

    7 (*) ! 8 , $! α|! = $" ! α ∗ $! () = α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood Bayes’ theorem

49. $! α|! = $" ! α ∗ $! () =

    α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood $$ 4 = α = $$ 4 = α|1 = $$ 4 = α|% = 9 %: 9 → ! sample space

50. $! α|! = $" ! α ∗ $! () =

    α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood $$ 4 = α = $$ 4 = α|1 = $$ 4 = α|% = 9 %: 9 → ! sample space $# % = ! = $# % = !|1 = $# % = !|4 = < 4: < → α sample space

51. $! α|! = $" ! α ∗ $! () =

    α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood $$ 4 = α = $$ 4 = α|% = 9 $# % = ! = $# % = !|4 = < = = ∀$ $# % = !|4 = α ∗ $$ 4 = α|% = 9 marginaliza7on α ∈ {4, 6, 8, 12, 20}

52. $! α|! = $" ! α ∗ $! () =

    α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood = = ∀$ $# !|α ∗ $$ α|9 marginalization α ∈ {4, 6, 8, 12, 20} $$ 4 = α = $$ α|9 $# % = ! = $# !|<

53. $! α|! = $" ! α ∗ $! () =

    α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood = = ∀$ $# !|α ∗ $$ α|9 marginaliza7on α ∈ {4, 6, 8, 12, 20} likelihood $$ 4 = α = $$ α|9 $# % = ! = $# !|<

54. $! α|! = $" ! α ∗ $! () =

    α) $" + = ! ' 5 : α → & ' 6 : & → α likelihood $$ 4 = α = $$ α|9 $# % = ! = $# !|< = = ∀$ $# !|α ∗ $$ α|9 marginalization α ∈ {4, 6, 8, 12, 20} likelihood

55. $! α|! = $" ! α ∗ $! (α) $"

    ! ' 5 : α → & ' 6 : & → α likelihood = $" ! α ∗ $! (α|-) Σ∀! $" !|α ∗ $! α|-

56. Dice with α faces ! = {5, 4, 3, 4,

    2, 1, 2, 3, 1, 4} $ % = ! α = 4 = 0 $ % = ! α = 6 = 1 6!" $ % = ! α = 8 = 1 8!" $ % = ! α = 12 = 1 12!" $ % = ! α = 20 = 1 20!" likelihood

57. $! α|! = $" ! α ∗ $! (α|-) Σ∀!

    $" !|α ∗ $! α|- ' 5 : α → & ' 6 : & → α likelihood $! () = α|+ = -)

58. $! α|! = $" ! α ∗ $! (α|-) Σ∀!

    $" !|α ∗ $! α|- ' 5 : α → & ' 6 : & → α likelihood $! () = α|+ = -) %: 9 → ! 9 : sample space of data ! (20!"= 1,024,000,000,000 pa+ern)

59. $! α|! = $" ! α ∗ $! (α|-) Σ∀!

    $" !|α ∗ $! α|- ' 5 : α → & ' 6 : & → α likelihood $! () = α|+ = -) %: 9 → ! 9 : sample space of data ! (20$%= 1,024,000,000,000 paHern)
60. None

61. $! α|! = $" ! α ∗ $! (α|-) Σ∀!

    $" !|α ∗ $! α|- ' 5 : α → & ' 6 : & → α likelihood $! () = α|+ = -) + ≅ +′ approximation $! ) = ∀α + = -& = 1 5 α ∈ {4, 6, 8, 12, 20}

62. $! α|! ≅ $" ! α ∗ $! (α|-′) Σ∀!

    $" !|α ∗ $! α|-′ ' 5 : α → & ' 6 : & → α likelihood = -$ . α Σ∀! -$ .|α = -$ . α -$ . 4 + -$ . 6 + -$ . 8 + -$ . 12 + -$ . 20 ≈ -$ . α 1.7485A − 08 &! ∀α (" = 1 5

63. Hmmm......, so, how many ?

    $(α = 6) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} $ % = ! α = 6 = 1 6!" maximum likelihood $$ 4 = 6|! ≅ $# % = ! 4 = 6 1.7485C − 08 ≈ 94.58%

64. $$ 6|! ≈ 94.58% $$ 6|9′ = 20% $$ 8|!

    ≈ 5.32% $$ 8|9′ = 20% $$ 12|! ≈ 0.09% $$ 12|9′ = 20% $$ 20|! ≈ 0.0005% $$ 20|9′ = 20% $$ 4|! = 0% $$ 4|9′ = 20% prior probability posterior probability Maximum a posteriori (MAP) estimation arg max! $! α ! = 6

65. Hmmm......, so, how many ?

    $(α = 6) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} $ % = ! α = 6 = 1 6!" maximum likelihood $$ 4 = 6|! ≈ 94.58% maximum posteriori prob.

66. Hmmm......, so, how about ?

    $(α = 6) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} $ % = ! α = 6 = 1 6!" maximum likelihood $$ 4 = 6|! ≈ 94.58% maximum posteriori prob. Could you predict & II?

67. Dice with α faces ! = {5, 4, 3, 4,

    2, 1, 2, 3, 1, 4} $# !!! ≤ 6|4 ∗ $$ 4|! = 0% $# !!! ≤ 6|6 ∗ $$ 6|! ≈ 94.58% $# !!! ≤ 6|8 ∗ $$ 8|! ≈ 3.99% $# !!! ≤ 6|12 ∗ $$ 12|! ≈ 0.046% $# !!! ≤ 6|20 ∗ $$ 20|! ≈ 0.0001% $# !!! ≤ 6 = = ∀$ {$# !!! ≤ 6|α ∗ $$ α|! } ≈ 98.62% predic$ve probability

68. Could you predict & II?

    $ ) = 6 ! ≈ 94.58% $ !$$ ≤ 6 ! ≈ 98.62% and Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4}

69. Could you predict & II?

    $ ) = 6 ! ≈ 94.58% $ !$$ ≤ 6 ! ≈ 98.62% and Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} OK, let’s try !!!!!

70. !!! = 8 Dice with

    α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4}

71. $ ) = 6 !

    ≈ 94.58% $ !$$ ≤ 6 ! ≈ 98.62% Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} OK, let’s try "!!!! !)) = 8 " $ = 6 {,, ,## } = 0%

72. "$ α|, ≅ "% , α ∗ "$ (α|4′) "%

    (,) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} prior likelihood posterior /( ∀α 1) = 1 5

73. "$ α|, ≅ "% , α ∗ "$ (α|4′) "%

    (,) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} prior likelihood posterior /( ∀α 1) = 1 5 "$ α| ́ , ≅ "% ́ , α ∗ "$ (α|4′′) "% ( ́ ,) Dice with α faces ́ ! = {!, 8}

74. "$ α|, ≅ "% , α ∗ "$ (α|4′) "%

    (,) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} prior likelihood posterior /( ∀α 1) = 1 5 "$ α| ́ , ≅ "% ́ , α ∗ "$ (α|4′′) "% ( ́ ,) Dice with α faces ́ ! = {!, 8}

75. "$ α|, ≅ "% , α ∗ "$ (α|4′) "%

    (,) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} prior likelihood posterior /( ∀α 1) = 1 5 "$ α| ́ , ≅ "% ́ , α ∗ "$ (α|,) "% ( ́ ,) Dice with α faces ́ ! = {!, 8}

76. Dice with α faces ! = {5, 4, 3, 4,

    2, 1, 2, 3, 1, 4} ́ ! = {!, 8} Non-informa$ve prior distribu$on 20% 20% 20% 20% 20% 0% 94.58% 5.32% 0.09% 0.005% 0% 0% 99.98% 0.02% 0.000004% -! (α|C′) -! (α|.) -! (α| ́ .)

77. $ ) = 8 ́

    ! ≈ 99.98% $ !$' ≤ 8 ́ ! ≈ 99.98% Dice with α faces ́ ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4, 8} OK!! Let’s try !!"!! COME OOON

78. No one knows what happened to them......

79. Hypothesis ObservaEon Truth Knowledge principle phenotype model data Dice with

    α faces (regular polyhedron) ! = 5 ?

80. Hmmm......, so, how about

    ? $(α = 6) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} $ % = ! α = 6 = 1 6!" maximum likelihood ! α = 6 % = & !!??

81. ! 7 *|, = ! 8 , * ∗ !

    7 (*) ! 8 , $! ) = α|+ = ! = $" + = ! ) = α ∗ $! (α) $" ! ' 5 : α → & ' 6 : & → α likelihood Bayes’ theorem

82. $! α|! ≅ $" ! α ∗ $! (α|-′) Σ∀!

    $" !|α ∗ $! α|-′ ' 5 : α → & ' 6 : & → α likelihood = -$ . α Σ∀! -$ .|α = -$ . α -$ . 4 + -$ . 6 + -$ . 8 + -$ . 12 + -$ . 20 ≈ -$ . α 1.7485A − 08 &! ∀α (" = 1 5

83. $$ 6|! ≈ 94.58% $$ 6|9′ = 20% $$ 8|!

    ≈ 5.32% $$ 8|9′ = 20% $$ 12|! ≈ 0.09% $$ 12|9′ = 20% $$ 20|! ≈ 0.0005% $$ 20|9′ = 20% $$ 4|! = 0% $$ 4|9′ = 20% prior probability posterior probability Maximum a posteriori probability (MAP) estimation arg max! $! α ! = 6

84. Dice with α faces ! = {5, 4, 3, 4,

    2, 1, 2, 3, 1, 4} $# !!! ≤ 6|4 ∗ $$ 4|! = 0% $# !!! ≤ 6|6 ∗ $$ 6|! ≈ 94.58% $# !!! ≤ 6|8 ∗ $$ 8|! ≈ 3.99% $# !!! ≤ 6|12 ∗ $$ 12|! ≈ 0.046% $# !!! ≤ 6|20 ∗ $$ 20|! ≈ 0.0001% $# !!! ≤ 6 = = ∀$ {$# !!! ≤ 6|α ∗ $$ α|! } ≈ 98.62% predic$ve probability

85. "$ α|, ≅ "% , α ∗ "$ (α|4′) "%

    (,) Dice with α faces ! = {5, 4, 3, 4, 2, 1, 2, 3, 1, 4} prior likelihood posterior /( ∀α 1) = 1 5 "$ α| ́ , ≅ "% ́ , α ∗ "$ (α|4′′) "% ( ́ ,) Dice with α faces ́ ! = {!, 8}

86. Experiment hypothesis observa$on principle phenotype model data Truth Knowledge f

    X (unknown)

87. Strong hypothesis obs. principle phenotype f Weak hypothesis obs. principle

    phenotype model Complex data f model Simple data “Hypothesis driven” “Data driven” Experimental design X X

88. α ' -(.|α) α |' -(α|.) %|' -(2|α)- α .

    prior distribution posterior distribuBon data predictive distribution $! α ∗ $" ! α $" ! = $! α|! likelihood prior posterior Bayes’ theorem

89. α ' -(.|α) α |' -(α|.) %|' -(2|α)- α .

    prior distribution posterior distribuBon data predictive distribution $! α ∗ $" ! α $" ! = $! α|! likelihood prior posterior Bayes’ theorem Truth

90. α ' -(.|α) α |' -(α|.) %|' -(2|α)- α .

    prior distribuBon posterior distribuBon data predicBve distribuBon $! α ∗ $" ! α $" ! = $! α|! likelihood prior posterior Bayes’ theorem #(%|') .(%) Truth L&'(M| $ Kullback-Leibler divergence

91. α ' -(.|α) α |' -(α|.) %|' -(2|α)- α .

    prior distribuBon posterior distribuBon data predicBve distribuBon $! α ∗ $" ! α $" ! = $! α|! likelihood prior posterior Bayes’ theorem #(%|') .(%) Truth L&'(M| $ = −N( + P KL divergence Entropy Generalization error

92. /!" (.| # = Q[S $ − S(M)] = Q[(−log

    $ ) − (−log M )] = Q log ( ) = ∫ M % ∗ log ((#) )(,|#) Y% = ∫ M % ∗ log M(!) Y% − ∫ M % ∗ log $ % ! Y% = −Q S M − ∫ M % ∗ log $ % ! Y% B( C Entropy Generaliza$on error

93. α ' -(.|α) α |' -(α|.) %|' -(2|α)- α .

    prior distribuBon posterior distribution data predictive distribution $! α ∗ $" ! α $" ! = $! α|! likelihood prior posterior Bayes’ theorem #(%|') .(%) Truth L&'(M| $ = −N( + P KL divergence Entropy GeneralizaBon error arg min) L&'(M| $ ⟺ arg min) P P ≅ WAIC Watanabe Akaike InformaAon Criterion

94. Experiment hypothesis observa$on principle phenotype model data Truth Knowledge f

    X (unknown)

95. Anaïs Nin – “Life shrinks or expands in proporRon to

    one’s courage.” h0ps://images.gr-assets.com

96. Before ABer BeginneR Session BeginneR BeginneR

97. Enjoy!!