pixiv private isucon 2016 ��ά (5/5)
����������:
�ܥȥ�ͥå�õ��
����ϰ��ֽŤ��ä��ȥåץڡ����˥ڡ�������å����Ƴ�����Ʒ�Ū�˹�®��������ΤΡ�CPU�ʳ����ܥȥ�ͥå��ˤʤä��礷����®���ϤǤ��ޤ���Ǥ����� ������������ dstat �����Ω�Ĥ��Ȥ�¿���Ǥ���
----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system-- usr sys idl wai hiq siq| read writ| recv send| in out | int csw 2 0 98 0 0 0| 596k 271k| 0 0 | 0 0 | 606 755 1 0 100 0 0 0| 0 0 | 192B 1190B| 0 0 | 119 177 0 0 100 0 0 0| 0 0 | 66B 350B| 0 0 | 103 158 1 0 98 1 0 0| 512k 0 | 537B 783B| 0 0 | 962 1035 48 7 38 3 0 5| 16M 728k|4590k 60M| 0 0 | 24k 20k 44 7 36 5 0 8| 11M 376k|6568k 71M| 0 0 | 34k 26k 44 11 35 4 0 7| 18M 312k|6190k 71M| 0 0 | 33k 27k 48 8 33 3 0 8| 14M 272k|7784k 71M| 0 0 | 31k 24k 41 7 42 4 0 6| 17M 352k|8097k 72M| 0 0 | 31k 23k 44 9 38 4 1 5| 14M 256k|8019k 71M| 0 0 | 33k 25k 44 10 35 3 0 8| 10M 448k|7705k 71M| 0 0 | 32k 25k 36 9 46 3 0 6| 14M 224k|7196k 71M| 0 0 | 32k 22k 38 10 39 6 0 7| 16M 18M|7812k 71M| 0 0 | 31k 22k 47 9 28 11 0 4| 20M 47M|6683k 72M| 0 0 | 29k 19k 45 7 39 3 0 6| 10M 696k|6113k 71M| 0 0 | 33k 24k 42 7 39 4 0 8| 12M 672k|3825k 71M| 0 0 | 33k 24k 51 8 31 3 0 6| 13M 512k|6750k 71M| 0 0 | 31k 22k 44 9 39 2 0 5|5496k 648k|8045k 71M| 0 0 | 32k 24k 53 9 28 3 0 6| 20M 544k|5832k 71M| 0 0 | 29k 20k 41 7 42 3 0 7| 14M 616k|7459k 71M| 0 0 | 35k 24k 48 9 31 4 0 8| 17M 608k|5869k 71M| 0 0 | 30k 23k 38 11 40 3 0 8| 14M 752k|8956k 71M| 0 0 | 32k 24k 44 11 36 4 0 5| 13M 576k|7181k 71M| 0 0 | 33k 25k 47 7 35 7 0 5| 14M 62M|5753k 65M| 0 0 | 30k 22k ...
net �� send �� 71M ��ĥ���դ��Ƥ�Τ��ܤˤĤ��ޤ��͡����ץꥵ���С��� c4.large �ʤΤǤ���ʤ��Ǥ��礦��
�Ǥ����� pixiv ����� Blog ���� �Υ����꡼�󥷥�åȤ򸫤�ȡ�¿���Υ����ब 40k ����Ķ���Ƥ��ơ��ȥåץ������180k������ã���Ƥ��ޤ������ä���40k���μ������Ӱ���ɤˤ֤Ĥ��äƤ�Τˡ��Ѥ����� (Twitter �� pixiv �� @catatsuy ����ˡ��ݡ�����Υ������ȥ��ޥ�ɥ饤��ˤ��٥���ޡ������Υ�������Ʊ��Ǥ��뤳�Ȥ��ǧ���ޤ�����)
���ơ�2���ܤε����ǡ����Τ褦�˸��äƤ��ޤ����͡�
�Ӱ�� /image/ ������Ū���Ӱ�ͥå��ˤʤ�褦���ä��顢�����Ϻư��̤��ˤ����Τǡ��إå�����Ŭ�ڤ����ꤹ��Х��饤����Ȥ� "304 Not Modified" ���֤���褦�ˤʤä��ç¤ï¿½Ê¥Ö¥ì¡¼ï¿½ï¿½ï¿½ï¿½ï¿½ë¡¼ï¿½ï¿½ï¿½ï¿½ï¿½ë¤«ï¿½â¤·ï¿½ï¿½Ê¤ï¿½ï¿½ï¿½
��˥��饤����ȤΥ���å�������˱ƶ���Ϳ�������ʥإå������ɲä��Ƥߤޤ��礦�� "nginx ��Ū�ե����� ����å���" �ǥ����äƤ���äݤ��Τ�õ���ޤ�
upstream app { server 127.0.0.1:8080; keepalive 64; } server { listen 80; client_max_body_size 10m; root /home/isucon/private_isu/webapp/public/; location / { try_files $uri @app; } location ~* \.(?:ico|css|js|gif|jpe?g|png)$ { try_files $uri @app; expires max; add_header Pragma public; add_header Cache-Control "public, must-revalidate, proxy-revalidate"; etag off; } location @app { proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_pass http://app; } }
��¬���Ƥߤޤ���
������:
{"pass":true,"score":182435,"success":174747,"fail":0,"messages":[]}
pixiv ���� isucon �Υȥåפ�Ʊ�����餤�Υ��������Ǥޤ�����(����ϥץ��ե�������ä��ꤤ���������ʤ���Υ������ʤΤǡ��ž夲������⤦�������Ӥ�Ϥ��Ǥ�)
top, dstat:
Tasks: 81 total, 3 running, 78 sleeping, 0 stopped, 0 zombie %Cpu(s): 80.7 us, 11.5 sy, 0.0 ni, 3.0 id, 0.5 wa, 0.0 hi, 4.2 si, 0.0 st KiB Mem: 1022972 total, 954144 used, 68828 free, 25432 buffers KiB Swap: 0 total, 0 used, 0 free. 557948 cached Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1397 isucon 20 0 377096 97572 8880 S 100.5 9.5 0:27.35 app.dc4a3eaa 731 mysql 20 0 1059292 183956 10552 S 65.9 18.0 2:26.08 mysqld 1396 www-data 20 0 92088 5304 3144 R 18.6 0.5 0:04.99 nginx 1069 isucon 20 0 33140 10476 2636 S 5.7 1.0 0:12.10 tmux 308 memcache 20 0 334624 9744 2228 S 0.7 1.0 0:01.38 memcached 110 root 20 0 0 0 0 S 0.3 0.0 0:00.08 jbd2/xvda2-8 1034 root 20 0 0 0 0 S 0.3 0.0 0:02.89 kworker/1:0
----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system-- usr sys idl wai hiq siq| read writ| recv send| in out | int csw 0 0 100 0 0 0| 0 0 | 0 0 | 0 0 | 45 76 21 5 71 2 0 1| 11M 0 |1987k 12M| 0 0 |8731 9184 73 15 5 2 0 5| 17M 904k|4574k 25M| 0 0 | 19k 20k 78 13 3 1 0 6| 20M 272k|3815k 18M| 0 0 | 15k 17k 80 9 6 2 0 4| 25M 272k|7221k 20M| 0 0 | 16k 17k 81 11 4 1 0 4| 18M 272k|5953k 20M| 0 0 | 15k 16k 81 10 4 0 0 5| 24M 256k|5363k 24M| 0 0 | 15k 17k 76 13 6 2 0 4| 20M 272k|8293k 18M| 0 0 | 16k 17k 82 11 2 1 0 5| 15M 552k|2665k 18M| 0 0 | 16k 16k 80 14 3 1 0 4| 14M 408k|2870k 15M| 0 0 | 15k 16k 79 12 5 1 0 4| 16M 360k|5475k 19M| 0 0 | 17k 18k 79 11 5 2 0 3| 16M 65M|5456k 21M| 0 0 | 15k 16k 77 12 6 1 0 4| 19M 760k|6763k 23M| 0 0 | 19k 20k 79 12 4 2 0 4| 23M 592k|6824k 19M| 0 0 | 16k 17k 79 12 5 2 0 3| 11M 632k|5994k 20M| 0 0 | 16k 18k 82 11 3 1 0 4| 11M 680k|3447k 16M| 0 0 | 15k 17k 83 11 2 0 0 4| 13M 792k|5076k 18M| 0 0 | 16k 18k ...
CPU��Ȥ��ڤ�褦�ˤʤä��Τ�Ƚ��ޤ���
access.log ����:
Request by count 117444 GET /image/* 7987 GET / 6394 GET /js/jquery-2.2.0.js 6394 GET /css/style.css 6394 GET /js/jquery.timeago.ja.js 6394 GET /favicon.ico 6394 GET /js/jquery.timeago.js 6394 GET /js/main.js 4028 POST /login 1459 GET /login 1176 GET /posts/* 959 POST /register 958 GET /admin/banned 873 GET /@user 731 GET /logout 609 POST / 330 GET /posts?max_created_at= 268 POST /comment 1 GET /initialize Request by total time 98.996 0.299987878788 GET /posts?max_created_at= 84.66 0.0105997245524 GET / 84.077 0.0963081328751 GET /@user 52.172 0.0129523336643 POST /login 39.666 0.0337295918367 GET /posts/* 33.621 0.0552068965517 POST / 19.801 0.0738843283582 POST /comment 17.144 0.0117505140507 GET /login ... Request by out bytes 894159053 7613 GET /image/* 185613643 23239 GET / 16456443 18850 GET /@user 11202870 33948 GET /posts?max_created_at= 4098331 3484 GET /posts/* ...
image ��ž���̤������˸��ä����Ȥ���ǧ�Ǥ��ޤ����ޤ������פ����� access.log �򸫤�ȡ������̤� 304 ���֤��Ƥ��뤳�Ȥ�Ƚ��ޤ���
(;��) �ɤ�Ķ����
2���ܤε����� 304 Not Modified ���֤��뤳�Ȥ�ͽ�ۤ��Ƥ����Τϡ������ѡ��Ǥ�ʤ�Ǥ⤢��ޤ��󡣰��ǯ (2014ǯ) �η辡�Ǥ��ä���Ǥ������λ��Ϥ����ɤ�Ķ����줺�������פ��򤷤ޤ�����
���λ����ɤ�Ķ���������ब2�����ढ��ޤ�������ͥ�����������Ū�ե�������֤������ nginx ������Υƥ�ץ�ʤ�����������Υ���סˤ�Ȥä��餤���ʤꥹ�������夬�ä����ɲ��������ä��Τ����ꤷ���ܿͤˤ�ʬ����ʤ��ä���������ͥ��������Ͻ�ͥ��������Υ������򸫤ơ�ɬ���ɤ�Ķ������ˡ������Ϥ����פȻ�Ժ�����������ɤ�Ķ�����ȵ������Ƥ��ޤ����ʶ�̣���������������� blog ������õ���ƤߤƤ���������
�������Υ���פ������ϥޤ롢�Ȥ����ɤ�Ķ�����Ͻ����λ�ʪ�Ǥ��� �Ŀ�Ū�ˤϥ��塼�˥󥰤���Ȥ��ϥѥ�᡼������1��1�Ĥ����äƸ��̳�ǧ�������Ȥ����Ǥ�����������Ȼ��֤���ä����ʤ������٥���ޡ������ε�ư�ˤĤ��Ƥ�ͽ�ۤ����Ƥ��ʤ����ɤ�Ķ�����ʤ��Τǡ���Ϥ�������Υ���פ϶��Ϥ����ˤʤ�ޤ���
�������٥���ޡ�������ư���ͽ�ۤ���Ժ������������ϡ��������¦�ο������ɤ� ISUCON �ϡ���ǽ���Τ����Ժ����򤹤뤿��ΰ����Ф�(����Ǥ����� HTTP �إå����ȥ���å���˴ؤ����μ�)�������Ϥʤɤ������ޤ��� ���˷辡��ͥ������������Ϥɤ�⤳��ǽ�Ϥ��ä�ͥ��Ƥ�����������Ȼפ��ޤ����񤷤��Ǥ���ͥ�����ܻؤ����ٶ�����ϣ�򤷤ޤ��礦��
���ҹʤ�
CPU�ͥå��ˤʤä����ȤǤ�����CPU�ץ��ե�����򸫤Ƥߤޤ��礦��
(pprof) top30 -cum 7.63s of 29.87s total (25.54%) Dropped 601 nodes (cum <= 0.15s) Showing top 30 nodes out of 296 (cum >= 4.36s) flat flat% sum% cum cum% 0 0% 0% 28.68s 96.02% runtime.goexit 0 0% 0% 24.58s 82.29% net/http.(*conn).serve 0 0% 0% 23.35s 78.17% net/http.(*ServeMux).ServeHTTP 0 0% 0% 23.35s 78.17% net/http.serverHandler.ServeHTTP 0 0% 0% 23.32s 78.07% github.com/zenazn/goji/web.(*Mux).ServeHTTP 0 0% 0% 23.30s 78.00% github.com/zenazn/goji/web.(*cStack).ServeHTTP 0 0% 0% 23.30s 78.00% github.com/zenazn/goji/web/middleware.RequestID.func1 0 0% 0% 23.30s 78.00% net/http.HandlerFunc.ServeHTTP 0 0% 0% 23.25s 77.84% github.com/zenazn/goji/web/middleware.Logger.func1 0 0% 0% 22.71s 76.03% github.com/zenazn/goji/web.(*mStack).newStack.func1 0.01s 0.033% 0.033% 22.71s 76.03% github.com/zenazn/goji/web.(*router).route 0 0% 0.033% 22.71s 76.03% github.com/zenazn/goji/web/middleware.AutomaticOptions.func1 0 0% 0.033% 22.71s 76.03% github.com/zenazn/goji/web/middleware.Recoverer.func1 0.01s 0.033% 0.067% 18.11s 60.63% github.com/zenazn/goji/web.netHTTPHandlerFuncWrap.ServeHTTPC 0 0% 0.067% 12.85s 43.02% main.getPosts 0 0% 0.067% 12.46s 41.71% github.com/jmoiron/sqlx.(*DB).Select 0 0% 0.067% 12.46s 41.71% github.com/jmoiron/sqlx.Select 0.07s 0.23% 0.3% 11.69s 39.14% github.com/jmoiron/sqlx.scanAll 1.72s 5.76% 6.06% 9.78s 32.74% runtime.mallocgc 0.16s 0.54% 6.60% 8.62s 28.86% runtime.systemstack 5.02s 16.81% 23.40% 7.29s 24.41% runtime.scanobject 0.15s 0.5% 23.90% 5.91s 19.79% runtime.newobject 0 0% 23.90% 5.33s 17.84% runtime.gcAssistAlloc 0 0% 23.90% 5.33s 17.84% runtime.gcAssistAlloc.func1 0.04s 0.13% 24.04% 5.33s 17.84% runtime.gcDrainN 0.04s 0.13% 24.17% 5.22s 17.48% database/sql.(*Rows).Next 0.03s 0.1% 24.27% 5.04s 16.87% github.com/go-sql-driver/mysql.(*textRows).Next 0.34s 1.14% 25.41% 5.01s 16.77% github.com/go-sql-driver/mysql.(*textRows).readRow 0 0% 25.41% 4.55s 15.23% github.com/zenazn/goji/web.handlerFuncWrap.ServeHTTPC 0.04s 0.13% 25.54% 4.36s 14.60% main.makePosts (pprof) list getPosts Total: 29.87s ROUTINE ======================== main.getPosts in /home/isucon/private_isu/webapp/golang/app.go 0 12.85s (flat, cum) 43.02% of Total . . 593: fmt.Println(terr) . . 594: return . . 595: } . . 596: . . 597: results := []Post{} . 11.19s 598: rerr := db.Select(&results, "SELECT `id`, `user_id`, `body`, `mime`, `created_at` FROM `posts` WHERE `created_at` <= ? ORDER BY `created_at` DESC", t.Format(ISO8601_FORMAT )) . . 599: if rerr != nil { . . 600: fmt.Println(rerr) . . 601: return . . 602: } . . 603: . 1.03s 604: posts, merr := makePosts(results, getCSRFToken(r), false) . . 605: if merr != nil { . . 606: fmt.Println(merr) . . 607: return . . 608: } . . 609: . . 610: if len(posts) == 0 { . . 611: w.WriteHeader(http.StatusNotFound) . . 612: return . . 613: } . . 614: . . 615: fmap := template.FuncMap{ . . 616: "imageURL": imageURL, . . 617: } . . 618: . . 619: template.Must(template.New("posts.html").Funcs(fmap).ParseFiles( . 10ms 620: getTemplPath("posts.html"), . . 621: getTemplPath("post.html"), . 620ms 622: )).Execute(w, posts) . . 623:} . . 624: . . 625:func getPostsID(c web.C, w http.ResponseWriter, r *http.Request) { . . 626: pid, err := strconv.Atoi(c.URLParams["id"]) . . 627: if err != nil { (pprof) list makePosts Total: 29.87s ROUTINE ======================== main.makePosts in /home/isucon/private_isu/webapp/golang/app.go 40ms 4.36s (flat, cum) 14.60% of Total . . 188:} . . 189: . . 190:func makePosts(results []Post, CSRFToken string, allComments bool) ([]Post, error) { . . 191: var posts []Post . . 192: . 20ms 193: for _, p := range results { . 720ms 194: err := db.Get(&p.CommentCount, "SELECT COUNT(*) AS `count` FROM `comments` WHERE `post_id` = ?", p.ID) . . 195: if err != nil { . . 196: return nil, err . . 197: } . . 198: . . 199: query := "SELECT * FROM `comments` WHERE `post_id` = ? ORDER BY `created_at` DESC" . . 200: if !allComments { . . 201: query += " LIMIT 3" . . 202: } . . 203: var comments []Comment . 920ms 204: cerr := db.Select(&comments, query, p.ID) . . 205: if cerr != nil { . . 206: return nil, cerr . . 207: } . . 208: . . 209: for i := 0; i < len(comments); i++ { 20ms 1.89s 210: uerr := db.Get(&comments[i].User, "SELECT * FROM `users` WHERE `id` = ?", comments[i].UserID) . . 211: if uerr != nil { . . 212: return nil, uerr . . 213: } . . 214: } . . 215: . . 216: // reverse . . 217: for i, j := 0, len(comments)-1; i < j; i, j = i+1, j-1 { . . 218: comments[i], comments[j] = comments[j], comments[i] . . 219: } . . 220: . . 221: p.Comments = comments . . 222: 20ms 780ms 223: perr := db.Get(&p.User, "SELECT * FROM `users` WHERE `id` = ?", p.UserID) . . 224: if perr != nil { . . 225: return nil, perr . . 226: } . . 227: . . 228: p.CSRFToken = CSRFToken . . 229: . . 230: if p.User.DelFlg == 0 { . 30ms 231: posts = append(posts, p) . . 232: } . . 233: if len(posts) >= postsPerPage { . . 234: break . . 235: } . . 236: }
getPosts
���Ť��Τϡ����� getIndex
���Ť��Τ�ľ��������Ʊ������ LIMIT ������ɤ������Ǥ��͡�
makePosts
�⥯������ꤲ�����Ƥ���Τǡ� Post �μ����Ȥ��� User �μ����� INNER JOIN �ǰ쵤�ˤ��롢�����Ȱ����ϥ����˥���å��夷�Ƥ��ޤ����Ȥˤ��ޤ���
���ȥƥ�ץ졼�ȤΥ���ѥ��������ˤ���褦�ˤ��Ƥ����ޤ��� Go ���ץ�¦�ǽ��Ϥ��Ƥ�������� tmux �� ssh �� CPU �򿩤äƤ�ΤǤ��������ߤ�Ƥ��ޤ��ޤ��礦�� (nginx �Υ�����ߤ��Τϰ��ֺǸ�Ǥ�)
diff --git a/app.go b/app.go index f246ad6..eca8413 100644 --- a/app.go +++ b/app.go @@ -187,54 +184,78 @@ func getFlash(w http.ResponseWriter, r *http.Request, key string) string { } } -func makePosts(results []Post, CSRFToken string, allComments bool) ([]Post, error) { - var posts []Post +var ( + commentM sync.Mutex + commentStore map[int][]Comment = make(map[int][]Comment) +) - for _, p := range results { - err := db.Get(&p.CommentCount, "SELECT COUNT(*) AS `count` FROM `comments` WHERE `post_id` = ?", p.ID) +func getCommentsLocked(postID int) []Comment { + if cs, ok := commentStore[postID]; ok { + return cs + } + + var cs []Comment + query := ("SELECT comments.id, comments.comment, comments.created_at, users.id, users.account_name " + + " FROM `comments` INNER JOIN users ON comments.user_id = users.id " + + " WHERE `post_id` = ? ORDER BY comments.`created_at`") + + rows, err := db.Query(query, postID) + if err != nil { + log.Println(err) + return cs + } + for rows.Next() { + var c Comment + err := rows.Scan(&c.ID, &c.Comment, &c.CreatedAt, &c.User.ID, &c.User.AccountName) if err != nil { - return nil, err + log.Println(err) + continue } + cs = append(cs, c) + } + rows.Close() - query := "SELECT * FROM `comments` WHERE `post_id` = ? ORDER BY `created_at` DESC" - if !allComments { - query += " LIMIT 3" - } - var comments []Comment - cerr := db.Select(&comments, query, p.ID) - if cerr != nil { - return nil, cerr - } + commentStore[postID] = cs + return cs +} - for i := 0; i < len(comments); i++ { - uerr := db.Get(&comments[i].User, "SELECT * FROM `users` WHERE `id` = ?", comments[i].UserID) - if uerr != nil { - return nil, uerr - } - } +func getComments(postID int) []Comment { + commentM.Lock() + defer commentM.Unlock() + return getCommentsLocked(postID) +} - // reverse - for i, j := 0, len(comments)-1; i < j; i, j = i+1, j-1 { - comments[i], comments[j] = comments[j], comments[i] - } +func appendComent(c Comment) { + commentM.Lock() + cs := getCommentsLocked(c.PostID) + commentStore[c.PostID] = append(cs, c) + commentM.Unlock() +} + +func makePosts(results []Post, CSRFToken string, allComments bool) ([]Post, error) { + var posts []Post + for _, p := range results { + comments := getComments(p.ID) + if !allComments && len(comments) > 3 { + comments = comments[len(comments)-3:] + } p.Comments = comments + p.CSRFToken = CSRFToken perr := db.Get(&p.User, "SELECT * FROM `users` WHERE `id` = ?", p.UserID) if perr != nil { return nil, perr } - p.CSRFToken = CSRFToken - - if p.User.DelFlg == 0 { - posts = append(posts, p) + if p.User.DelFlg != 0 { + continue } + posts = append(posts, p) if len(posts) >= postsPerPage { break } } - return posts, nil } @@ -411,8 +432,9 @@ func getLogout(w http.ResponseWriter, r *http.Request) { } var ( - indexTemplate *template.Template - postsTemplate *template.Template + indexTemplate *template.Template + postsTemplate *template.Template + accountNameTempalte *template.Template indexPostsM sync.Mutex indexPostsT time.Time @@ -434,6 +456,13 @@ func init() { getTemplPath("posts.html"), getTemplPath("post.html"), )) + + accountNameTempalte = template.Must(template.New("layout.html").Funcs(fmap).ParseFiles( + getTemplPath("layout.html"), + getTemplPath("user.html"), + getTemplPath("posts.html"), + getTemplPath("post.html"), + )) } func renderIndexPosts() { @@ -506,7 +535,6 @@ func getAccountName(c web.C, w http.ResponseWriter, r *http.Request) { } results := []Post{} - rerr := db.Select(&results, "SELECT `id`, `user_id`, `body`, `mime`, `created_at` FROM `posts` WHERE `user_id` = ? ORDER BY `created_at` DESC", user.ID) if rerr != nil { fmt.Println(rerr) @@ -527,12 +555,10 @@ func getAccountName(c web.C, w http.ResponseWriter, r *http.Request) { } postIDs := []int{} - perr := db.Select(&postIDs, "SELECT `id` FROM `posts` WHERE `user_id` = ?", user.ID) - if perr != nil { - fmt.Println(perr) - return + for _, r := range results { + postIDs = append(postIDs, r.ID) } - postCount := len(postIDs) + postCount := len(results) commentedCount := 0 if postCount > 0 { @@ -557,16 +583,7 @@ func getAccountName(c web.C, w http.ResponseWriter, r *http.Request) { me := getSessionUser(r) - fmap := template.FuncMap{ - "imageURL": imageURL, - } - - template.Must(template.New("layout.html").Funcs(fmap).ParseFiles( - getTemplPath("layout.html"), - getTemplPath("user.html"), - getTemplPath("posts.html"), - getTemplPath("post.html"), - )).Execute(w, struct { + accountNameTempalte.Execute(w, struct { Posts []Post User User PostCount int @@ -595,7 +612,7 @@ func getPosts(w http.ResponseWriter, r *http.Request) { } results := []Post{} - rerr := db.Select(&results, "SELECT `id`, `user_id`, `body`, `mime`, `created_at` FROM `posts` WHERE `created_at` <= ? ORDER BY `created_at` DESC", t.Format(ISO8601_FORMAT)) + rerr := db.Select(&results, "SELECT posts.`id`, `user_id`, `body`, `mime`, posts.`created_at` FROM `posts` INNER JOIN `users` ON posts.user_id=users.id WHERE users.del_flg = 0 AND posts.`created_at` <= ? ORDER BY `created_at` DESC LIMIT 20", t.Format(ISO8601_FORMAT)) if rerr != nil { fmt.Println(rerr) return @@ -806,9 +823,24 @@ func postComment(w http.ResponseWriter, r *http.Request) { return } - query := "INSERT INTO `comments` (`post_id`, `user_id`, `comment`) VALUES (?,?,?)" - db.Exec(query, postID, me.ID, r.FormValue("comment")) - + now := time.Now() + commentStr := r.FormValue("comment") + query := "INSERT INTO `comments` (`post_id`, `user_id`, `comment`, `created_at`) VALUES (?,?,?,?)" + res, err := db.Exec(query, postID, me.ID, commentStr, now) + if err != nil { + log.Println(err) + return + } + lid, _ := res.LastInsertId() + c := Comment{ + ID: int(lid), + PostID: postID, + UserID: me.ID, + Comment: commentStr, + CreatedAt: now, + User: me, + } + appendComent(c) renderIndexPosts() http.Redirect(w, r, fmt.Sprintf("/posts/%d", postID), http.StatusFound) } @@ -921,6 +953,7 @@ func main() { go http.ListenAndServe(":3000", nil) + goji.DefaultMux = web.New() goji.Get("/initialize", getInitialize) goji.Get("/login", getLogin) goji.Post("/login", postLogin)
��¬
������:
{"pass":true,"score":271542,"success":256436,"fail":0,"messages":[]}
pixiv ���� isucon ��ͥ��������Υ�������Ķ���뤳�Ȥ��Ǥ����Ȼפ��ޤ���
top:
ks: 82 total, 4 running, 78 sleeping, 0 stopped, 0 zombie %Cpu(s): 57.1 us, 13.8 sy, 0.0 ni, 16.5 id, 5.1 wa, 0.0 hi, 7.3 si, 0.2 st KiB Mem: 1022972 total, 926504 used, 96468 free, 27152 buffers KiB Swap: 0 total, 0 used, 0 free. 432620 cached Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1494 isucon 20 0 544796 191292 8940 S 81.5 18.7 0:29.06 app.7f063c1 731 mysql 20 0 1125024 182444 10552 S 44.9 17.8 3:04.05 mysqld 1524 www-data 20 0 92056 5276 3140 R 27.6 0.5 0:09.65 nginx 308 memcache 20 0 336672 11756 2228 S 1.7 1.1 0:02.25 memcached 31 root 20 0 0 0 0 S 0.7 0.0 0:00.40 kswapd0 89 root 0 -20 0 0 0 S 0.3 0.0 0:00.73 kworker/0:1H 1 root 20 0 28636 4736 3100 S 0.0 0.5 0:01.31 systemd
����ä� idle ���ФƤ������ʡ�
dstat:
----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system-- usr sys idl wai hiq siq| read writ| recv send| in out | int csw 0 0 100 0 0 0| 0 0 | 0 0 | 0 0 | 82 283 11 3 82 2 0 2| 12M 0 |1684k 15M| 0 0 |7607 5952 55 13 17 9 0 7| 49M 864k| 11M 44M| 0 0 | 27k 20k 58 15 14 8 0 6| 48M 416k| 13M 43M| 0 0 | 28k 20k 48 14 21 12 0 5| 43M 63M| 12M 40M| 0 0 | 25k 19k 55 17 14 7 0 7| 48M 528k| 14M 42M| 0 0 | 25k 18k 55 12 19 8 0 6| 45M 488k| 12M 42M| 0 0 | 27k 20k 52 17 16 6 0 8| 35M 592k| 14M 38M| 0 0 | 27k 19k 50 12 19 14 0 6| 34M 59M| 11M 38M| 0 0 | 25k 17k 56 12 18 6 0 7| 42M 720k| 11M 45M| 0 0 | 28k 20k 51 15 18 9 0 8| 45M 704k| 14M 39M| 0 0 | 26k 19k 52 17 17 7 0 7| 39M 3936k| 14M 45M| 0 0 | 27k 19k 47 17 18 13 0 5| 41M 56M| 12M 39M| 0 0 | 24k 17k 53 13 19 9 0 6| 45M 848k| 13M 41M| 0 0 | 27k 20k 57 15 16 6 0 6| 38M 776k| 13M 42M| 0 0 | 27k 19k 53 14 20 9 0 4| 28M 58M| 11M 29M| 0 0 | 24k 16k 53 13 19 6 0 8| 44M 672k| 13M 38M| 0 0 | 26k 18k 57 13 16 9 0 6| 46M 872k| 10M 38M| 0 0 | 25k 18k 53 12 22 7 0 6| 44M 776k| 12M 41M| 0 0 | 28k 20k 54 16 16 8 0 6| 33M 57M|9049k 33M| 0 0 | 24k 18k 51 16 21 5 0 7| 32M 888k| 12M 38M| 0 0 | 27k 19k 54 13 19 7 0 7| 49M 2264k| 11M 46M| 0 0 | 26k 18k 54 14 19 6 0 7| 36M 680k| 12M 42M| 0 0 | 28k 19k 48 16 19 10 0 7| 33M 52M| 14M 31M| 0 0 | 23k 16k 55 15 15 7 0 7| 42M 720k| 12M 34M| 0 0 | 26k 20k 59 14 17 5 0 6| 33M 744k| 12M 35M| 0 0 | 25k 18k 47 15 25 7 0 6| 34M 736k|9604k 43M| 0 0 | 28k 19k 47 11 27 8 0 6| 31M 51M| 12M 39M| 0 0 | 25k 17k 54 18 17 5 0 7| 34M 816k| 12M 33M| 0 0 | 27k 20k 55 16 16 6 0 6| 35M 888k| 14M 34M| 0 0 | 27k 19k 57 13 18 6 0 6| 34M 720k| 11M 37M| 0 0 | 27k 19k 52 12 18 12 0 6| 36M 53M| 11M 34M| 0 0 | 23k 17k 62 12 15 4 0 7| 26M 720k|9309k 29M| 0 0 | 25k 19k 46 20 19 5 0 9| 30M 560k| 16M 35M| 0 0 | 28k 20k 59 12 20 5 0 5| 23M 544k| 10M 36M| 0 0 | 25k 17k 53 13 21 8 0 5| 25M 47M| 11M 31M| 0 0 | 23k 16k 54 14 18 6 0 9| 38M 584k| 14M 35M| 0 0 | 27k 21k ...
�Ӱ��Ⱦʬ���ȤäƤ��ޤ��͡��ç¤ï¿½ï¿½ js ������Τǡ� `nginx �� gzip_static �⥸�塼���Ȥ��ʤɤǤ⤦�����Ӱ�ò³«¤ï¿½ï¿½ï¿½ï¿½ï¿½È»×¤ï¿½ï¿½Þ¤ï¿½ï¿½ï¿½
pprof:
(pprof) top30 -cum 6.95s of 22.25s total (31.24%) Dropped 621 nodes (cum <= 0.11s) Showing top 30 nodes out of 321 (cum >= 2.44s) flat flat% sum% cum cum% 0 0% 0% 20.30s 91.24% runtime.goexit 0.02s 0.09% 0.09% 17.03s 76.54% net/http.(*conn).serve 0 0% 0.09% 14.60s 65.62% net/http.(*ServeMux).ServeHTTP 0 0% 0.09% 14.60s 65.62% net/http.serverHandler.ServeHTTP 0 0% 0.09% 14.58s 65.53% github.com/zenazn/goji/web.(*Mux).ServeHTTP 0 0% 0.09% 14.57s 65.48% github.com/zenazn/goji/web.(*cStack).ServeHTTP 0 0% 0.09% 14.57s 65.48% github.com/zenazn/goji/web.(*mStack).newStack.func1 0 0% 0.09% 14.57s 65.48% github.com/zenazn/goji/web.(*router).route 0 0% 0.09% 14.57s 65.48% net/http.HandlerFunc.ServeHTTP 0 0% 0.09% 10.03s 45.08% github.com/zenazn/goji/web.netHTTPHandlerFuncWrap.ServeHTTPC 0.01s 0.045% 0.13% 6.12s 27.51% html/template.(*Template).Execute 0 0% 0.13% 5.58s 25.08% text/template.(*Template).Execute 0.12s 0.54% 0.67% 5.57s 25.03% text/template.(*state).walk 0.01s 0.045% 0.72% 5.20s 23.37% text/template.(*state).walkTemplate 0.04s 0.18% 0.9% 4.68s 21.03% text/template.(*state).walkRange 0.01s 0.045% 0.94% 4.65s 20.90% text/template.(*state).walkRange.func1 0.97s 4.36% 5.30% 4.60s 20.67% runtime.mallocgc 0 0% 5.30% 4.46s 20.04% github.com/zenazn/goji/web.handlerFuncWrap.ServeHTTPC 0.23s 1.03% 6.34% 4.38s 19.69% runtime.systemstack 0.04s 0.18% 6.52% 4.07s 18.29% text/template.(*state).evalPipeline 0.08s 0.36% 6.88% 4.03s 18.11% text/template.(*state).evalCommand 2.54s 11.42% 18.29% 3.71s 16.67% runtime.scanobject 0.10s 0.45% 18.74% 3.09s 13.89% text/template.(*state).evalCall 0.03s 0.13% 18.88% 3.07s 13.80% text/template.(*state).evalFunction 0 0% 18.88% 2.99s 13.44% main.getAccountName 2.48s 11.15% 30.02% 2.70s 12.13% syscall.Syscall 0 0% 30.02% 2.62s 11.78% main.getPosts 0.01s 0.045% 30.07% 2.56s 11.51% reflect.Value.Call 0.26s 1.17% 31.24% 2.52s 11.33% reflect.Value.call 0 0% 31.24% 2.44s 10.97% github.com/jmoiron/sqlx.(*DB).Get
�ƥ�ץ졼�Ȥμ¹ԤȤ��Ť��Τȡ�����GC���Ť��Ǥ��͡�
�ƥ�ץ졼�Ȥ��Ť��Τϡ��ڡ�������å�����ȥȥåץڡ����ʳ��Ϥɤ줯�餤����å���ҥå�Ψ��������뤫�狼��ޤ��� valyala/quicktemplate �ʤɤΡ���ե쥯������¿�Ѥ��ʤ���®�ʥƥ�ץ졼�ȥ��󥸥�˾�괹����ۤ����ᤤ���⤷��ޤ���
GC�����ϡ�������¹Բ���ò¤µ¤ï¿½Ëºï¿½Ã¤Æ¤ï¿½ï¿½ï¿½ï¿½Ð¡ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½Ö¥ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½È¤ï¿½ï¿½ï¿½ï¸ºï¿½ï¿½ï¿½ï¿½Î¤ï¿½Í���������ɤ��ǥ��������Ȥ�¿ȯ���Ƥ���Τ����ǧ���������� pprof -alloc_objects
��ȤäƤߤƤ���������
�ޤȤ�
������: 4745 (�������) -> 39743 (����) -> 182435 (Not Modified) -> 271542 (�ǽ���)
�ɤ�Ķ�������̤ǡ����ޤޤǿ���Ǻ��Ǥ������������쵤�˥����פ��ޤ�����
��ά���Ϥ����ޤǤˤ��ޤ������Ǥ���Ф���˥��塼�˥󥰤�³���ƤߤƤ��������������Ǽºݤ˼��ư�����Ƥ��ʤ����Ȥϡ����֤ǤϤۤܳμ¤ˤǤ��ޤ��� ��ξ��ϡ����Τ褦�ʥ��塼�˥󥰤�500k����Ķ���ޤ�����������Ǥ�ޤ��Ӱ��Ȥ��ڤäƤ��ʤ��� nginx ���� app �� MySQL �����Ѥ���CPU�������ç¤ï¿½ï¿½ï¿½Î¤Ç¤ï¿½ï¿½ï¿½Ë¾ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½Ï¤ï¿½ï¿½Ç¤ï¿½ï¿½ï¿½
- gzip_static ��Ȥ� (�������Ͼ夬��ޤ���Ǥ��������Ӱ��;͵����ޤ���)
- �ƥ�ץ졼�Ȥ��Ť� posts ����ʬ�� valyala/quicktemplate ��ȤäƤߤ�
- �桼�����ò¥ª¥ï¿½ï¿½ï¿½ê¥ï¿½ï¿½Ã¥ï¿½ï¿½å¤·ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½JOIN�����
- ���å���󥹥ȥ��� memcached ���饪�������ڤ��ؤ���
- nginx �� app �δ֤� unix domain socket�����ڤ��ؤ���
- nginx ���Ť��ʤäƤ��Ƥ����Τ� worker_processes �� 2 �ˤ���
- �����ϤΥꥯ�����ȤǤ�����
time.Sleep()
���Ԥ����ơ��������ؤαƶ����ç¤ï¿½ï¿½ï¿½ï¿½ï¿½Ê¥ê¥ªï¿½Î½ï¿½ï¿½ï¿½ï¿½Ë¥ê¥½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ - ���������������ڤ� (�ǽ����Ū�ե�����������Ǹ�������Υ�������������ä�)
�����ޤǤΥ��ߥåȥ����� methane/pixiv-private-isucon-2016 ���֤��Ƥ���ޤ��� �Ǥ������Ǥ���������Ϥ� 500k ��Ķ�����ܻؤ��ƤߤƤ���������
�Ǹ�ˡ�ISUCON �������ˤȤƤ��ɤ������󶡤���ĺ���� pixiv ����˴��դ��ޤ���
@methane