Qudoを使ったJob queue system

Transcript

1. 2VEPΛ࢖ͬͨ KPCRVFVFTZTUFN id:shiba_yu36 2012/03/17 Kyoto.pm Tech Talks #01 2012೥3݄18೔೔༵೔

2. ࣗݾ঺հ 2012೥3݄18೔೔༵೔

3. TIJCB@ZV TIJCBZV PSFUD 2012೥3݄18೔೔༵೔

4. !גࣜձࣾ͸ͯͳ • ՝ۚ΍λΠϜϥΠϯͳͲϓϥοτ ϑΥʔϜ෦෼Λ୲౰ • ֎෦αʔϏε࿈ܞ • ςετ؀ڥ੔උ 2012೥3݄18೔೔༵೔

5. QFSM w,ZPUPQN w$1"/ w8FC4FSWJDF#JUMZ w%#*Y%BUB'BDUPSZ w1SF1"/΋ͨ·ʹख఻ͬͯ·͢ 2012೥3݄18೔೔༵೔

6. 1SF1"/։ൃͰKPCRVFVFTZTUFNΛ ࢖͏ػձ͕͋ͬͨͷͰɺͦͷ࢓૊Έͱ 2VEPʹΑΔ࣮૷Λ঺հ͠·͢ɻ 2012೥3݄18೔೔༵೔

7. ΞδΣϯμ wKPCRVFVFTZTUFN w2VEPΛ࢖࣮ͬͨ૷ w࢖ͬͯΈͯ ͋·ΓKPCRVFVFTZTUFNΛ ࢖ͬͨ͜ͱͷͳ͍ਓ޲͚ 2012೥3݄18೔೔༵೔

8. 2012೥3݄18೔೔༵೔

9. KPCRVFVFTZTUFNʁ ΞϓϦέʔγϣϯͷόοΫάϥ΢ϯ υଆͰඇಉظͰॲཧΛ࣮ߦ͢ΔͨΊ ͷ࢓૊Έ 2012೥3݄18೔೔༵೔

10. ϝϦοτ wॲཧΛޙճ͠ wϦΞϧλΠϜੑ͕ඞཁͳ͍΋ͷ wΤϥʔϋϯυϦϯά wϦτϥΠ wεέʔϧ 2012೥3݄18೔೔༵೔

11. KPCRVFVFTZTUFNͷߏ੒ w+PCΛFORVFVF͢Δ෦෼ DMJFOU w+PCͷॲཧຊମ XPSLFS w+PCͷ࣮ߦͷ؅ཧ XPSLFSNBOBHFS 2012೥3݄18೔೔༵೔

12. Ұྫ 2012೥3݄18೔೔༵೔

13. QFSMͰͷ KPCRVFVFTZTUFN w(FBSNBO wதؒEBFNPOར༻ w5IF4DIXBSU[ w3%#.4Λ࢖͏ wଟ͘ͷ࣮੷͕͋Δ 2012೥3݄18೔೔༵೔

14. QFSMͰͷ KPCRVFVFTZTUFN w 2VEP OFLPLBL͞Μ NBTBSU[͞Μ w ܰྔKPCRVFVFTZTUFN w +POL

    w ࠷௿ݶͷKPCRVFVF"1*Λ࣮૷ w $MVUDI w EJTUSJCVUFEKPCRVFVF w DMJFOU͕ॲཧ͢ΔXPSLFSΛܾఆ 2012೥3݄18೔೔༵೔

15. 2VEPΛ࢖࣮ͬͨ૷ 2012೥3݄18೔೔༵೔

16. ͳͥ2VEP w1SF1"/ͰNPEVMF͕౤ߘ͞ΕͨΒɺ͍ ͍ײ͡ʹ௨஌͍ͨ͠ wHJUIVCͷϑΥϩʔ wΩʔϫʔυʹϚονͰ௨஌ wॲཧ͕গ͠ॏ͍ͷͰɺ·ͣKPCRVFVF TZTUFNΛಋೖ͔ͨͬͨ͠ 2012೥3݄18೔೔༵೔

17. ͳͥ2VEP w5IF4DIXBSU[͸ΦʔόʔεϖοΫײ wͨ͘͞ΜΧελϚΠζ͍ͨ͠Θ͚Ͱ ΋ͳ͍ 2VEP͕ͪΐ͏ͲΑͦ͞͏ 2012೥3݄18೔೔༵೔

18. %#४උ RVEPECRVEPVTFSSPPU SECNTNZTRMVTF@JOOPEC 2012೥3݄18೔೔༵೔

19. $MJFOU 2012೥3݄18೔೔༵೔

20. $MJFOU package Sample::Qudo::Client; use parent qw(Qudo); sub new { my

    ($class, $args) = @_; $class->SUPER::new( databases => [+{ dsn => 'dbi:mysql:qudo_sample', username => 'root', password => '', }], default_hooks => ['Qudo::Hook::Serialize::JSON'], ); } 2012೥3݄18೔೔༵೔

21. $MJFOU $qudo->enqueue( "Sample::Qudo::Worker::Echo", { arg => {message => 'hoge'} },

    ); 2012೥3݄18೔೔༵೔

22. 8PSLFS 2012೥3݄18೔೔༵೔

23. 8PSLFS package Sample::Qudo::Worker::Echo; use parent qw(Qudo::Worker); sub work { my

    ($self , $job ) = @_; my $arg = $job->arg; my $message = $arg->{message}; sleep 2; # heavy task... warn $message; $job->completed(); # or $job->abort } 2012೥3݄18೔೔༵೔

24. 8PSLFS.BOBHFS 2012೥3݄18೔೔༵೔

25. 8PSLFS.BOBHFS # worker.pl my $qudo = Qudo->new( databases => [+{

    dsn => 'dbi:mysql:qudo_sample', username => 'root', password => '', }], default_hooks => ['Qudo::Hook::Serialize::JSON'], manager_abilities => [qw/Sample::Qudo::Worker::Echo/], ); $qudo->work; 2012೥3݄18೔೔༵೔

26. 8PSLFS.BOBHFS QBSBMMFM my $manager = Qudo::Parallel::Manager->new( databases => ..., default_hooks

    => ..., manager_abilities => ..., work_delay => 5, max_workers => 3, min_spare_workers => 3, auto_load_worker => 1, ); $manager->run; 2012೥3݄18೔೔༵೔

27. %FNP XPSLFSNBOBHFS QBSBMMFM XPSLFS͸JOTFSU͞Εͨ৘ใΛXBSO͢Δ͚ͩ 2012೥3݄18೔೔༵೔

28. EFQMPZ QFSMXPSLFSQM 2012೥3݄18೔೔༵೔

29. EFQMPZ EBFNPOUPPMT # run.sh #!/bin/sh exec 2>&1 cd /path/Sample-Worker ||

    exit 1 exec setuidgid app /usr/bin/perl worker.pl svc -u /etc/service/Sample-Worker #start svc -d /etc/service/Sample-Worker #stop 2012೥3݄18೔೔༵೔

30. ࢖ͬͯΈͯ 2012೥3݄18೔೔༵೔

31. ಛ௃ ͱʹ͔͘؆୯ ͙͢ʹKPCRVFVFͷ࢓૊ΈΛ ࢖ͬͯΈ͍ͨͳΒ࠷ద 2012೥3݄18೔೔༵೔

32. ͨͩ͠ w࢖ΘΕ͍ͯΔͱ͜Ζ͸ͦ͜·Ͱଟ͘ͳ ͍ w5IF4DIXBSU[ͱ͔ w+PC؅ཧͷ࢓૊ΈࣗମΛΧελϚΠζ͠ Α͏ͱ͢Δͱ೉͍͠ w+POLͱ͔ 2012೥3݄18೔೔༵೔

33. wKPCRVFVFTZTUFNͱ͔࢖ͬͨ͜ͱແ ͍ wखܰʹKPCRVFVFTZTUFNΛ࢖ͬͯΈ ͍ͨ Ұ౓2VEPΛ࢖ͬͯΈΔͱྑ͍Ͱ͢ʂ 2012೥3݄18೔೔༵೔

34. 2012೥3݄18೔೔༵೔

35. ·ͱΊ wKPCRVFVFTZTUFNͷ؆୯ͳઆ໌ w2VEPͰͷ࣮૷ͱͦͷಛ௃ͷ঺հ wͱʹ͔͘؆୯ w৴པੑ5IF4DIXBSU[ wΧελϚΠζ+POL 2012೥3݄18೔೔༵೔