In 2012 we made the very difficult decision to entirely re-engineer our existing monolithic LAMP application from the ground up in order to address some growing concerns about it's long term viability as a platform.\n\n
Full application re-write is almost always never the answer, because of the risks involved. However the situation warranted drastic action as it was clear that the existing product was going to face severe scaling issues. We felt it better address these sooner rather than later and also take the opportunity to improve the international architecture and also to refactor the database in. order that it better matched the changes in core functionality.\n\n
PostgreSQL was chosen for its reputation as being solid ACID compliant database backend, it was available as an offering AWS RDS service which reduced the management overhead of us having to configure it ourselves. In order to reduce read load on the primary database we implemented an Elasticsearch layer for fast and scalable search operations. Synchronisation of these indexes was to be achieved through the use of Sidekiq's Redis based background workers on Amazon ElastiCache. Again the AWS solution here looked to be an easy way to keep our involvement in managing this part of the platform at a minimum. Allowing us to focus on our core business.\n\n
Rails ls was chosen for its ability to quickly get core functionality up and running, its MVC architecture and also its focus on Test Driven Development using RSpec and Selenium with Travis CI providing continual integration. We also liked Ruby for its terse, clean and elegant syntax. Though YMMV on that one!\n\n
Unicorn was chosen for its continual deployment and reputation as a reliable application server, nginx for its reputation as a fast and stable reverse-proxy. We also took advantage of the Amazon CloudFront CDN here to further improve performance by caching static assets globally.\n\n
We tried to strike a balance between having control over management and configuration of our core application with the convenience of being able to leverage AWS hosted services for ancillary functions (Amazon SES , Amazon SQS Amazon Route 53 all hosted securely inside Amazon VPC of course!). \n\n
Whilst there is some compromise here with potential vendor lock in, the tasks being performed by these ancillary services are no particularly specialised which should mitigate this risk. Furthermore we have already containerised the stack in our development using Docker environment, and looking to how best to bring this into production - potentially using Amazon EC2 Container Service ","rawContent":"In 2012 we made the very difficult decision to entirely re-engineer our existing monolithic LAMP application from the ground up in order to address some growing concerns about it's long term viability as a platform.\n\nFull application re-write is almost always never the answer, because of the risks involved. However the situation warranted drastic action as it was clear that the existing product was going to face severe scaling issues. We felt it better address these sooner rather than later and also take the opportunity to improve the international architecture and also to refactor the database in. order that it better matched the changes in core functionality.\n\n@{PostgreSQL}|tool:1028| was chosen for its reputation as being solid ACID compliant database backend, it was available as an offering AWS RDS service which reduced the management overhead of us having to configure it ourselves. In order to reduce read load on the primary database we implemented an @{Elasticsearch}|tool:841| layer for fast and scalable search operations. Synchronisation of these indexes was to be achieved through the use of @{Sidekiq}|tool:1078|'s @{Redis}|tool:1031| based background workers on @{Amazon ElastiCache}|tool:392|. Again the AWS solution here looked to be an easy way to keep our involvement in managing this part of the platform at a minimum. Allowing us to focus on our core business.\n\n@{Rails}|tool:990| ls was chosen for its ability to quickly get core functionality up and running, its MVC architecture and also its focus on Test Driven Development using @{RSpec}|tool:2539| and @{Selenium}|tool:1517| with @{Travis CI}|tool:460| providing continual integration. We also liked @{Ruby}|tool:989| for its terse, clean and elegant syntax. Though YMMV on that one!\n\n@{Unicorn}|tool:1053| was chosen for its continual deployment and reputation as a reliable application server, @{nginx}|tool:1052| for its reputation as a fast and stable reverse-proxy. We also took advantage of the @{Amazon CloudFront}|tool:13| CDN here to further improve performance by caching static assets globally.\n\nWe tried to strike a balance between having control over management and configuration of our core application with the convenience of being able to leverage AWS hosted services for ancillary functions (@{Amazon SES}|tool:17| , @{Amazon SQS}|tool:395| @{Amazon Route 53}|tool:38| all hosted securely inside @{Amazon VPC}|tool:388| of course!). \n\nWhilst there is some compromise here with potential vendor lock in, the tasks being performed by these ancillary services are no particularly specialised which should mitigate this risk. Furthermore we have already containerised the stack in our development using @{Docker}|tool:586| environment, and looking to how best to bring this into production - potentially using @{Amazon EC2 Container Service}|tool:1908| ","publishedAt":"2019-03-25T16:09:44Z","private":false,"sample":false,"commentsCount":0,"upvotesCount":8,"upvoted":false,"flagged":false,"bookmarked":false,"viewCount":828026,"draft":false,"createdAt":"2019-03-25T16:09:53Z","decisionType":"freeform","answers({\"first\":2})":{"type":"id","generated":true,"id":"$StackDecision:101812154729512342.answers({\"first\":2})","typename":"StackDecisionConnection"},"subjectTools":[],"fromTools":[],"toTools":[],"deleted":false,"link":null,"company":{"type":"id","generated":false,"id":"Company:101231722605166946","typename":"Company"},"stack":null,"topics":[],"services":[{"type":"id","generated":false,"id":"Tool:101231773853533403","typename":"Tool"},{"type":"id","generated":false,"id":"Tool:101231773774313645","typename":"Tool"},{"type":"id","generated":false,"id":"Tool:101231773873140185","typename":"Tool"},{"type":"id","generated":false,"id":"Tool:101231773854768982","typename":"Tool"},{"type":"id","generated":false,"id":"Tool:101231773616958547","typename":"Tool"},{"type":"id","generated":false,"id":"Tool:101231773835996572","typename":"Tool"},{"type":"id","generated":false,"id":"Tool:101231774472965337","typename":"Tool"},{"type":"id","generated":false,"id":"Tool:101231774063724150","typename":"Tool"},{"type":"id","generated":false,"id":"Tool:101231773636491087","typename":"Tool"},{"type":"id","generated":false,"id":"Tool:101231773835511567","typename":"Tool"},{"type":"id","generated":false,"id":"Tool:101231773862206820","typename":"Tool"},{"type":"id","generated":false,"id":"Tool:101231773861864225","typename":"Tool"},{"type":"id","generated":false,"id":"Tool:101231773403188549","typename":"Tool"},{"type":"id","generated":false,"id":"Tool:101231773405155016","typename":"Tool"},{"type":"id","generated":false,"id":"Tool:101231773618167288","typename":"Tool"},{"type":"id","generated":false,"id":"Tool:101231773418092710","typename":"Tool"},{"type":"id","generated":false,"id":"Tool:101231773615096541","typename":"Tool"},{"type":"id","generated":false,"id":"Tool:101231773669807419","typename":"Tool"},{"type":"id","generated":false,"id":"Tool:101231774215500019","typename":"Tool"}],"user":{"type":"id","generated":false,"id":"User:101232110333797933","typename":"User"},"rootComments":[],"__typename":"StackDecision","parent":null},"$StackDecision:101812154729512342.permissions":{"edit":false,"delete":false,"__typename":"Permissions"},"$StackDecision:101812154729512342.answers({\"first\":2})":{"count":0,"pageInfo":{"type":"id","generated":true,"id":"$StackDecision:101812154729512342.answers({\"first\":2}).pageInfo","typename":"PageInfo"},"edges":[],"__typename":"StackDecisionConnection"},"$StackDecision:101812154729512342.answers({\"first\":2}).pageInfo":{"hasNextPage":false,"endCursor":null,"__typename":"PageInfo"},"Company:101231722605166946":{"id":"101231722605166946","path":"/companies/bettison-org-limited","imageUrl":"https://img.stackshare.io/company/28600/default_59f36dcba152abf2649245238d99b888f3adfa58.png","name":"Bettison.org Limited","slug":"bettison-org-limited","features":[],"__typename":"Company"},"Tool:101231773774313645":{"id":"101231773774313645","name":"Elasticsearch","slug":"elasticsearch","title":"Open Source, Distributed, RESTful Search Engine","verified":true,"imageUrl":"https://img.stackshare.io/service/841/Image_2019-05-20_at_4.58.04_PM.png","canonicalUrl":"/elasticsearch","path":"/elasticsearch","votes":1648,"fans":32682,"stacks":34520,"followingTool":false,"followContext":null,"__typename":"Tool"},"Tool:101231773873140185":{"id":"101231773873140185","name":"Sidekiq","slug":"sidekiq","title":"Simple, efficient background processing for Ruby","verified":false,"imageUrl":"https://img.stackshare.io/service/1078/4b7277462dadad85454ab427ce3f0ca7.png","canonicalUrl":"/sidekiq","path":"/sidekiq","votes":408,"fans":911,"stacks":1118,"followingTool":false,"followContext":null,"__typename":"Tool"},"Tool:101231773854768982":{"id":"101231773854768982","name":"Redis","slug":"redis","title":"Open source (BSD licensed), in-memory data structure store","verified":false,"imageUrl":"https://img.stackshare.io/service/1031/default_cbce472cd134adc6688572f999e9122b9657d4ba.png","canonicalUrl":"/redis","path":"/redis","votes":3943,"fans":54921,"stacks":59443,"followingTool":false,"followContext":null,"__typename":"Tool"},"Tool:101231773616958547":{"id":"101231773616958547","name":"Amazon ElastiCache","slug":"amazon-elasticache","title":"Deploy, operate, and scale an in-memory cache in the cloud","verified":false,"imageUrl":"https://img.stackshare.io/service/392/amazon-elasticache.png","canonicalUrl":"/amazon-elasticache","path":"/amazon-elasticache","votes":151,"fans":1319,"stacks":1324,"followingTool":false,"followContext":null,"__typename":"Tool"},"Tool:101231773835996572":{"id":"101231773835996572","name":"Rails","slug":"rails","title":"Web development that doesn't hurt","verified":false,"imageUrl":"https://img.stackshare.io/service/990/x57_Lorv.png","canonicalUrl":"/rails","path":"/rails","votes":5441,"fans":16797,"stacks":19470,"followingTool":false,"followContext":null,"__typename":"Tool"},"Tool:101231774472965337":{"id":"101231774472965337","name":"RSpec","slug":"rspec","title":"Behaviour Driven Development for Ruby","verified":false,"imageUrl":"https://img.stackshare.io/service/2539/logo.png","canonicalUrl":"/rspec","path":"/rspec","votes":0,"fans":489,"stacks":2628,"followingTool":false,"followContext":null,"__typename":"Tool"},"Tool:101231774063724150":{"id":"101231774063724150","name":"Selenium","slug":"selenium","title":"Web Browser Automation","verified":false,"imageUrl":"https://img.stackshare.io/service/1517/sbUizSli_400x400.jpg","canonicalUrl":"/selenium","path":"/selenium","votes":527,"fans":15415,"stacks":15623,"followingTool":false,"followContext":null,"__typename":"Tool"},"Tool:101231773636491087":{"id":"101231773636491087","name":"Travis CI","slug":"travis-ci","title":"A hosted continuous integration service for open source and private projects","verified":true,"imageUrl":"https://img.stackshare.io/service/460/Lu6cGu0z_400x400.png","canonicalUrl":"/travis-ci","path":"/travis-ci","votes":1749,"fans":9389,"stacks":26381,"followingTool":false,"followContext":null,"__typename":"Tool"},"Tool:101231773835511567":{"id":"101231773835511567","name":"Ruby","slug":"ruby","title":"A dynamic, interpreted, open source programming language with a focus on simplicity and productivity","verified":false,"imageUrl":"https://img.stackshare.io/service/989/ruby.png","canonicalUrl":"/ruby","path":"/ruby","votes":3973,"fans":26486,"stacks":41823,"followingTool":false,"followContext":null,"__typename":"Tool"},"Tool:101231773862206820":{"id":"101231773862206820","name":"Unicorn","slug":"unicorn","title":"Rack HTTP server for fast clients and Unix","verified":false,"imageUrl":"https://img.stackshare.io/service/1053/unicorn.png","canonicalUrl":"/unicorn","path":"/unicorn","votes":295,"fans":498,"stacks":494,"followingTool":false,"followContext":null,"__typename":"Tool"},"Tool:101231773403188549":{"id":"101231773403188549","name":"Amazon CloudFront","slug":"amazon-cloudfront","title":"Content delivery with low latency and high data transfer speeds","verified":false,"imageUrl":"https://img.stackshare.io/service/13/amazon-cloudfront.png","canonicalUrl":"/amazon-cloudfront","path":"/amazon-cloudfront","votes":935,"fans":13594,"stacks":21286,"followingTool":false,"followContext":null,"__typename":"Tool"},"Tool:101231773405155016":{"id":"101231773405155016","name":"Amazon SES","slug":"amazon-ses","title":"Bulk and transactional email-sending service.","verified":false,"imageUrl":"https://img.stackshare.io/service/17/amazon-ses.png","canonicalUrl":"/amazon-ses","path":"/amazon-ses","votes":328,"fans":7666,"stacks":9844,"followingTool":false,"followContext":null,"__typename":"Tool"},"Tool:101231773618167288":{"id":"101231773618167288","name":"Amazon SQS","slug":"amazon-sqs","title":"Fully managed message queuing service","verified":false,"imageUrl":"https://img.stackshare.io/service/395/amazon-sqs.png","canonicalUrl":"/amazon-sqs","path":"/amazon-sqs","votes":171,"fans":2513,"stacks":2237,"followingTool":false,"followContext":null,"__typename":"Tool"},"Tool:101231773418092710":{"id":"101231773418092710","name":"Amazon Route 53","slug":"amazon-route-53","title":"A highly available and scalable Domain Name System (DNS) web service.","verified":false,"imageUrl":"https://img.stackshare.io/service/38/amazon-route-53.png","canonicalUrl":"/amazon-route-53","path":"/amazon-route-53","votes":678,"fans":12150,"stacks":14429,"followingTool":false,"followContext":null,"__typename":"Tool"},"Tool:101231773615096541":{"id":"101231773615096541","name":"Amazon VPC","slug":"amazon-vpc","title":"Provision a logically isolated section of the AWS Cloud and launch AWS resources in a virtual network that you define","verified":false,"imageUrl":"https://img.stackshare.io/service/388/amazon-vpc.png","canonicalUrl":"/amazon-vpc","path":"/amazon-vpc","votes":46,"fans":1018,"stacks":1381,"followingTool":false,"followContext":null,"__typename":"Tool"},"Tool:101231774215500019":{"id":"101231774215500019","name":"Amazon EC2 Container Service","slug":"amazon-ec2-container-service","title":"Container management service that supports Docker containers","verified":true,"imageUrl":"https://img.stackshare.io/service/1908/amazon-ecs.png","canonicalUrl":"/amazon-ec2-container-service","path":"/amazon-ec2-container-service","votes":325,"fans":12919,"stacks":14160,"followingTool":false,"followContext":null,"__typename":"Tool"}}
In 2012 we made the very difficult decision to entirely re-engineer our existing monolithic LAMP application from the ground up in order to address some growing concerns about it's long term viability as a platform.
Full application re-write is almost always never the answer, because of the risks involved. However the situation warranted drastic action as it was clear that the existing product was going to face severe scaling issues. We felt it better address these sooner rather than later and also take the opportunity to improve the international architecture and also to refactor the database in. order that it better matched the changes in core functionality.
PostgreSQL was chosen for its reputation as being solid ACID compliant database backend, it was available as an offering AWS RDS service which reduced the management overhead of us having to configure it ourselves. In order to reduce read load on the primary database we implemented an Elasticsearch layer for fast and scalable search operations. Synchronisation of these indexes was to be achieved through the use of Sidekiq's Redis based background workers on Amazon ElastiCache. Again the AWS solution here looked to be an easy way to keep our involvement in managing this part of the platform at a minimum. Allowing us to focus on our core business.
Rails ls was chosen for its ability to quickly get core functionality up and running, its MVC architecture and also its focus on Test Driven Development using RSpec and Selenium with Travis CI providing continual integration. We also liked Ruby for its terse, clean and elegant syntax. Though YMMV on that one!
Unicorn was chosen for its continual deployment and reputation as a reliable application server, nginx for its reputation as a fast and stable reverse-proxy. We also took advantage of the Amazon CloudFront CDN here to further improve performance by caching static assets globally.
We tried to strike a balance between having control over management and configuration of our core application with the convenience of being able to leverage AWS hosted services for ancillary functions (Amazon SES , Amazon SQSAmazon Route 53 all hosted securely inside Amazon VPC of course!).
Whilst there is some compromise here with potential vendor lock in, the tasks being performed by these ancillary services are no particularly specialised which should mitigate this risk. Furthermore we have already containerised the stack in our development using Docker environment, and looking to how best to bring this into production - potentially using Amazon EC2 Container Service
