Mitchell Hashimoto, HashiCorp
•
62 likes•12,288 views
This document discusses the history and future of operations (ops) and infrastructure management. It outlines how infrastructure has evolved from single manually configured servers to cloud-based infrastructure with immutable servers. Immutable infrastructure involves replacing servers instead of modifying them, using pre-built machine images. Tools like Packer, Vagrant, and Serf help enable immutable infrastructure by automating the creation of machine images and handling service orchestration outside of images. This approach provides benefits like speed, repeatability, stability and testability compared to traditional mutable infrastructure management.
Mitchell Hashimoto, HashiCorp
- 1. Towards FutureOps: Stable, repeatable, environments from dev to prod.
- 2. I’m Mitchell Hashimoto Also known as @mitchellh
- 5. History of Ops Or, the history of pretending that anything works.
- 7. Single Dedicated Server • Manually setup
- 8. Single Dedicated Server • • Manually setup Expensive to replace (time and money)
- 9. Single Dedicated Server • • • Manually setup Expensive to replace (time and money) Professionals, usually
- 10. What is hard/slow/expensive? • • • • Acquiring hardware Manual setup Expertise Downtime, Failures
- 12. Multiple Dedicated Servers • Golden images
- 13. Multiple Dedicated Servers • • Golden images Manually made
- 14. Multiple Dedicated Servers • • • Golden images Manually made Slow to change, expensive to grow
- 15. Multiple Dedicated Servers • Eventually: config management (CFEngine, Puppet, Chef)
- 16. What is hard/slow/expensive? • • • • Acquiring hardware Making images Expertise Downtime, Failures
- 18. The VPS Server • Manually made (cargo cult, usually)
- 19. The VPS Server • • Manually made (cargo cult, usually) Cheaper to replace
- 20. The VPS Server • • • Manually made (cargo cult, usually) Cheaper to replace Hobbyist, startup
- 21. What is hard/slow/expensive? • • • Expertise Downtime, Failures Rebuilding it
- 22. The Cloud
- 23. The Cloud • Many cheap servers
- 24. The Cloud • • Many cheap servers Seconds to create
- 25. The Cloud • • • Many cheap servers Seconds to create Configuration management
- 26. What is hard/slow/expensive? • • • • Configuration mgmt Portability Repeatability Can be expensive
- 27. The Ops of Tomorrow Better, faster, stronger
- 29. Immutable Infrastructure Instead of modifying or maintaining a server, replace it with a new one.
- 30. Immutable Infrastructure Servers don’t change anymore, they’re pre-built, static images that are tested/verified.
- 31. Immutable Wins • • • Super fast server deployment Failure expected, not a big deal Testable, more stable
- 32. Automation Critical • • • • Machine Images Software install/configure Failure detection, recovery etc…
- 33. Machine Images Easily build machine images for multiple platforms with Packer.
- 34. Machine image: A single deployable unit that contains a pre-configured OS and software.
- 35. Ew... Images. The anti-image stigma
- 36. DevOps has historically been against machine images.
- 37. Important to understand why DevOps has historically been against machine images.
- 38. Golden images used to be the way.
- 40. Getting any changes (ops or dev) in was slow and frustrating.
- 41. Necessary evil while tooling wasn’t as good as it is today.
- 42. With modern config management, easy to simply ignore images.
- 43. Machine images also have a ton of benefits.
- 47. Analogy: Ops without machine images is like applications without binaries
- 48. Put it another way: There is no promise of reproducible builds in ops, and no binary-like snapshots
- 50. Source Code libA 1.0 Binary libB 1.0 libC 1.0
- 51. Source Code libA 2.0 Binary libB 1.0 libC 1.0
- 52. Source Code libA 2.0 COMPILE FAILED libB 1.0 libC 1.0
- 53. Chef, Puppet, Scripts, etc. New Server Ready Server
- 54. Chef, Puppet, Scripts, etc. New Server Packages Ready Server Network Chef/Puppet Changes
- 55. Chef, Puppet, Scripts, etc. SERVER SETUP FAILED New Server Package Changes Network Chef/Puppet Changes
- 56. Chef, Puppet, Scripts, etc. SERVER SETUP FAILED New Server Packages Network Unreliable Chef/Puppet Changes
- 57. Chef, Puppet, Scripts, etc. SERVER SETUP FAILED New Server Packages Network Chef/Puppet Changed
- 58. Machine Image New Server Ready Server
- 59. Machine Image New Server Ready Server
- 60. Packer Embraces modern best practices and automates building images
- 61. Takes template input and repeatably produces image output.
- 62. $ packer build ! -var ‘aws_access_key=YOUR KEY’ ! -var ‘aws_secret_key=YOUR SECRET’ ! template.json! ...
- 63. Automated, repeatable, fast, uses Chef/Puppet/ Shell/etc. Gets rid of old downsides.
- 64. Can also produce Vagrant boxes for development, at the same time.
- 65. Workflows What does this look like in practice?
- 66. Ops Ops Push Packer Build Machine Image(s)
- 67. Ops Ops Push Packer Build Machine Image(s) Setup users, install software, configure software, configure services. No orchestration.
- 68. Server Deploy Request Server Running Server
- 69. Server Deploy • Server started with Packerbuilt image • Running server has all preinstalled software • Service orchestration runs now Request Server Running Server
- 70. Development “vagrant up” Ready to work
- 71. Development • Vagrant downloads the latest machine image created by Packer • “vagrant up” takes seconds to minutes and runs, nearly identical to production “vagrant up” Ready to work
- 72. Packer Wins • • • • Automated, repeatable images Fast deployment from dev to prod Testable, very stable Embraces configuration mgmt
- 73. SERF Service orchestration, discovery, failure detection.
- 74. Remember, Chef/Puppet/etc. aren’t doing service orchestration for the image.
- 75. Service Discovery: Web servers in a load balancer, Memcache in a cluster, MySQL master/slave, etc.
- 76. Service Orchestration: Add this node to your cluster, deploy this application, restart, etc.
- 77. Serf is a decentralized, highly available, fault tolerant, and lightweight solution to service orchestration and discovery.
- 78. Example Web servers in a load balancer
- 79. Run the Agent $ serf agent ! ! -event-handler=“handle.sh”! …
- 80. Join a Cluster $ serf join 10.0.1.5! …
- 81. Serf Events Serf Agent Handle.sh Add or Remove Node from LB
- 82. Serf Events Serf Agent handle.sh Add or Remove Node from LB Serf invokes event handlers for any event: member-join, member-leave, member-failed, user.
- 83. How? How does Serf do what it does?
- 86. CUSTOM EVENTS
- 87. Config Management Chef/Puppet Run Configure Serf Agent Services Don’t start Serf. Serf starts on boot.
- 88. Config Management Chef/Puppet Run Configure Serf agent init script Don’t start Serf. Serf starts on boot. Chef and Puppet just configure the Serf init script to start the agent.
- 89. Serf Wins • • • No orchestration in image build Fast membership detection, don’t wait for Chef/Puppet Easily and infinitely extensible
- 90. The Future is Immutable Big wins, little downsides.
- 91. As servers become cheaper, more ondemand, and infrastructures become more distributed, immutable infrastructure will win.
- 92. Important to remember: You don’t have to go all in on immutable. You can start with certain servers. Example: databases are hard, just ignore that for awhile.
- 93. Mostly Immutable There are still big benefits for going mostly immutable. Things like quick config changes and deploys can still be mutable. Serf is good for this.
- 94. The Wins • • • • • Super fast deployment Repeatability High failure tolerance Improved stability and testability Versioning
- 95. The Downsides • • • Requires proper mindset Tooling is new Deploys, simple config changes
- 96. Thank you!