Improving the Sensitivity of Online Controlled Experiments: Case Studies at Netflix Huizhi Xie Netflix 100 Winchester Circle Los Gatos, California, USA kxie@netflix.com Juliette Aurisset Netflix 100 Winchester Circle Los Gatos, California, USA jaurisset@netflix.com ABSTRACT Controlled experiments are widely regarded as the most sci- entific way to establish a true causal relationship between produ
1 | \ " x x 100 100 \ " \ " \ " (formalization) ( lassi al) (intuitionisti ) (pred- i ate logi ) (quanti ation) ( rst order) 1.1 \ " \ " ^ _ ! > ? 8 9 1.1 ( ) (language) L � (fun tion symbol) ff;g;h;::: g � (predi ate symbol) fp;q;r;:::g f p (arity) n 2 N f p n f(n), p(n) n = 0 0 ( onstant symbol) 0 (propositional symbol) 1 1.2 ( L0) � f0(0);S(1);+(2);�(2)g � f=(2)g + � = 0 0 S x x+1 1.3 ( L1) � f
I've been thinking about the notion of "reasoning locally" recently. Enabling local reasoning is the only way to scale software development past some number of lines or complexity. When reasoning locally, one only needs to understand a small subset, hundreds of lines, to safely make changes in programs comprising millions.I find types helps massively with this. A function with well-constrained inp
ããã¡æ ç»ç¥ãããä¸åº¦è¦³ããæ ç»ã¯ã³ã¬ã¯ã·ã§ã³ãããã¨ãè²·ã£ããããã£ã¹ã¯ã«ç¼ããããããããã®æã«ãã³ããã ã俺ã®å¥½ããªæ ç»ã¯ããã¥ã¢ãã¼ã»ã¯ã¼ã«ãããªãã ãã ã¨è¨ã£ããç§ã¯ã³ã£ãããããç¹°ãè¿ã観ããæ ç»ã¨ãã¦èªåã¯ããã»ã©å¿ã«æ®ã£ã¦ããªãã£ãã¨ããã®ããããããã³ãããã®æ ç»ãä¸æ¬ã«ããããã¨ãæå¤ã ã£ãã ããªã¦ããè¶ å¤§ä½ã好ããªäººãªã®ã§ãã¦ã£ãããã¢ã¤ã¢ã³ãã³ããä¸çªå¥½ãã ã¨ãè¨ãã¨æã£ã¦ããã®ã ã ããã¦ããã³ãã好ãã ã¨ããããã¥ã¢ãã¼ã»ã¯ã¼ã«ããããã®å¾ãä½åº¦ã観ããã³ã¬ã¯ã·ã§ã³ã«å ããå¾ã«èå³ãæã£ã¦è¦³ã¦ã¿ãã¨ãã¨ã¦ãããæ ç»ã ã£ãããã®å¾ãç£ç£ã¯ãã¼ãã»ã°ã©ããã£ãã§æåã«ãªã£ãã ã¢ã«ãã©ã³ã½ã»ãã¥ã¢ãã³ç£ç£ã§ããã ä»åãããã³ãºã©ããªã³ã¹ãã«ã¤ãã¦è¨äºãæ¸ãããã«ãããã§æ ç»ã«ã¤ãã¦èª¿ã¹ã¦ããããã®ã¬ã«ã¢ã»ãã«ã»ããç£ç£ã¯ã¡ãã·ã³ã®äººã§ãã¢ã«ãã©ã³ã½ã»ãã¥ã¢ã
ã®ã£ãã©ã¤ãºã®éç°ã§ãã Optimizelyãã¯ããã¨ããABãã¹ããã¼ã«ã¯ã1è¡ã¿ã°ãè¨ç½®ãã¦ãã¾ãã°ãã®æ¥ããããã«ABãã¹ããåºæ¥ã¦ãã¾ãã»ã©ãå°å ¥èªä½ã¯ç°¡åã§ãã ã§ããæã ãABãã¹ããã¼ã«ã®å°å ¥ãããã¦ããã ãéã¯ããããªãABãã¹ããå®æ½ããã®ã§ã¯ãªããã¾ããAAãã¹ããã¨ãããã®ãå®æ½ãããã¨ããªã¹ã¹ã¡ãã¦ãã¾ããAAãã¹ãã¨ã¯ããªãªã¸ãã«ã¨ããªã¨ã¼ã·ã§ã³ã§ä¸åã®è¦ç´ ãå¤æ´ãããã¨ãªããåä¸ã®æ¯çã§åºãåãã¦ãçµæãã¬ãã¥ã¼ããææ³ã§ãã ãªãã§ãããããããªãã¨ããã®ãï¼çåã«æãæ¹ãããã£ãããã¨æãã¾ãããAAãã¹ãã¯ãABãã¹ãã®çµæãæ£ããè©ä¾¡ããããã«æ¬ ãããã¨ãã§ããªã大äºãªä½æ¥ãªã®ã§ãã ã©ããããã¨ãï¼æ¬ç¨¿ã§ã説æãã¾ãããã ABãã¹ãã®çµæã®è©ä¾¡ã«ãå°ãã§ã¯ãªãã§ããï¼ ABãã¹ãããã¦ã¿ãã¯ããããåã£ãã®ãè² ããã®ãããåãããªã ABãã¹ãæã¯
ã©ã³ãã³ã°
ã©ã³ãã³ã°
ã©ã³ãã³ã°
ãªãªã¼ã¹ãé害æ å ±ãªã©ã®ãµã¼ãã¹ã®ãç¥ãã
ææ°ã®äººæ°ã¨ã³ããªã¼ã®é ä¿¡
å¦çãå®è¡ä¸ã§ã
j次ã®ããã¯ãã¼ã¯
kåã®ããã¯ãã¼ã¯
lãã¨ã§èªã
eã³ã¡ã³ãä¸è¦§ãéã
oãã¼ã¸ãéã
{{#tags}}- {{label}}
{{/tags}}