Last active
December 26, 2024 13:34
-
-
Save vasanthk/485d1c25737e8e72759f to your computer and use it in GitHub Desktop.
Revisions
-
vasanthk revised this gist
Sep 18, 2022 . 1 changed file with 0 additions and 2 deletions.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -136,8 +136,6 @@ **Links** [System Design Interviewing](http://www.hiredintech.com/system-design/) [Scalability for Dummies](http://www.lecloud.net/tagged/scalability) -
vasanthk revised this gist
Jan 13, 2020 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -104,7 +104,7 @@ * Web Workers * Server Side rendering * Asynchronous loading of assets (Lazy load items) * Minimizing network requests (Http2 + bundling/sprites etc) * Developer productivity/Tooling * Accessibility * Internationalization -
vasanthk revised this gist
Aug 11, 2017 . 1 changed file with 2 additions and 2 deletions.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -70,7 +70,7 @@ <img src="http://lethain.com/static/blog/intro_arch/platform_layer.png" alt="Platform Layer"/> </p> ## Key topics for designing a system 1) **Concurrency** * Do you understand threads, deadlock, and starvation? Do you know how to parallelize algorithms? Do you understand consistency and coherence? @@ -91,7 +91,7 @@ * Are you thinking about how things can fail, especially in a distributed environment? Do know how to design a system to cope with network failures? Do you understand durability? ## Web App System design considerations: * Security (CORS) * Using CDN * A content delivery network (CDN) is a system of distributed servers (network) that deliver webpages and other Web content to a user based on the geographic locations of the user, the origin of the webpage and a content delivery server. -
vasanthk revised this gist
Mar 28, 2017 . 1 changed file with 2 additions and 2 deletions.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,8 +1,8 @@ # System Design Cheatsheet > Picking the right architecture = Picking the right battles + Managing trade-offs ## Basic Steps 1) **Clarify and agree on the scope of the system** * **User cases** (description of sequences of events that, taken together, lead to a system doing something useful) -
vasanthk revised this gist
Apr 18, 2016 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,4 +1,4 @@ #System Design Cheatsheet > Picking the right architecture = Picking the right battles + Managing trade-offs -
vasanthk revised this gist
Apr 18, 2016 . 1 changed file with 3 additions and 1 deletion.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -148,4 +148,6 @@ [Scalable Web Architecture and Distributed Systems](http://www.aosabook.org/en/distsys.html) [What is the best way to design a web site to be highly scalable?](http://programmers.stackexchange.com/a/108679/62739) [How web works?](https://github.com/vasanthk/how-web-works) -
vasanthk revised this gist
Apr 17, 2016 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,4 +1,4 @@ #System Design Interview Cheatsheet > Picking the right architecture = Picking the right battles + Managing trade-offs -
vasanthk revised this gist
Apr 17, 2016 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,4 +1,4 @@ #Cracking the System Design Interview > Picking the right architecture = Picking the right battles + Managing trade-offs -
vasanthk revised this gist
Feb 25, 2016 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -5,7 +5,7 @@ ##Basic Steps 1) **Clarify and agree on the scope of the system** * **User cases** (description of sequences of events that, taken together, lead to a system doing something useful) * Who is going to use it? * How are they going to use it? * **Constraints** -
vasanthk revised this gist
Feb 6, 2016 . 1 changed file with 2 additions and 2 deletions.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,5 +1,7 @@ #System Design > Picking the right architecture = Picking the right battles + Managing trade-offs ##Basic Steps 1) **Clarify and agree on the scope of the system** @@ -132,8 +134,6 @@ * Deploy Process * Continuous Integration (Travis CI, Jenkins) **Links** [How to rock a systems design interview](http://www.palantir.com/2011/10/how-to-rock-a-systems-design-interview/) -
vasanthk revised this gist
Feb 6, 2016 . 1 changed file with 24 additions and 1 deletion.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -89,7 +89,7 @@ * Are you thinking about how things can fail, especially in a distributed environment? Do know how to design a system to cope with network failures? Do you understand durability? ##Web App System design considerations: * Security (CORS) * Using CDN * A content delivery network (CDN) is a system of distributed servers (network) that deliver webpages and other Web content to a user based on the geographic locations of the user, the origin of the webpage and a content delivery server. @@ -109,6 +109,29 @@ * Responsive design * Browser compatibility ## Working Components of Front-end Architecture * Code * HTML5/WAI-ARIA * CSS/Sass Code standards and organization * Object-Oriented approach (how do objects break down and get put together) * JS frameworks/organization/performance optimization techniques * Asset Delivery - Front-end Ops * Documentation * Onboarding Docs * Styleguide/Pattern Library * Architecture Diagrams (code flow, tool chain) * Testing * Performance Testing * Visual Regression * Unit Testing * End-to-End Testing * Process * Git Workflow * Dependency Management (npm, Bundler, Bower) * Build Systems (Grunt/Gulp) * Deploy Process * Continuous Integration (Travis CI, Jenkins) > Picking the right architecture = Picking the right battles + Managing trade-offs **Links** -
vasanthk revised this gist
Feb 6, 2016 . 1 changed file with 6 additions and 0 deletions.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -100,11 +100,17 @@ * Offline support/Progressive enhancement * Service Workers * Web Workers * Server Side rendering * Asynchronous loading of assets (Lazy load items) * Minimizing netwrok requests (Http2 + bundling/sprites etc) * Developer productivity/Tooling * Accessibility * Internationalization * Responsive design * Browser compatibility > Picking the right architecture = Picking the right battles + Managing trade-offs **Links** [How to rock a systems design interview](http://www.palantir.com/2011/10/how-to-rock-a-systems-design-interview/) -
vasanthk revised this gist
Feb 6, 2016 . 1 changed file with 8 additions and 0 deletions.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -60,6 +60,14 @@ * **Map-Reduce** * For sufficiently small systems you can often get away with adhoc queries on a SQL database, but that approach may not scale up trivially once the quantity of data stored or write-load requires sharding your database, and will usually require dedicated slaves for the purpose of performing these queries (at which point, maybe you'd rather use a system designed for analyzing large quantities of data, rather than fighting your database). * Adding a map-reduce layer makes it possible to perform data and/or processing intensive operations in a reasonable amount of time. You might use it for calculating suggested users in a social graph, or for generating analytics reports. eg. Hadoop, and maybe Hive or HBase. * **Platform Layer (Services)** * Separating the platform and web application allow you to scale the pieces independently. If you add a new API, you can add platform servers without adding unnecessary capacity for your web application tier. * Adding a platform layer can be a way to reuse your infrastructure for multiple products or interfaces (a web application, an API, an iPhone app, etc) without writing too much redundant boilerplate code for dealing with caches, databases, etc. <p align="center"> <img src="http://lethain.com/static/blog/intro_arch/platform_layer.png" alt="Platform Layer"/> </p> ##Key topics for designing a system 1) **Concurrency** -
vasanthk revised this gist
Feb 6, 2016 . 1 changed file with 3 additions and 1 deletion.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -57,7 +57,9 @@ * Database replication is the frequent electronic copying data from a database in one computer or server to a database in another so that all users share the same level of information. The result is a distributed database in which users can access data relevant to their tasks without interfering with the work of others. The implementation of database replication for the purpose of eliminating data ambiguity or inconsistency among users is known as normalization. * **Database partitioning** * Partitioning of relational data usually refers to decomposing your tables either row-wise (horizontally) or column-wise (vertically). * **Map-Reduce** * For sufficiently small systems you can often get away with adhoc queries on a SQL database, but that approach may not scale up trivially once the quantity of data stored or write-load requires sharding your database, and will usually require dedicated slaves for the purpose of performing these queries (at which point, maybe you'd rather use a system designed for analyzing large quantities of data, rather than fighting your database). * Adding a map-reduce layer makes it possible to perform data and/or processing intensive operations in a reasonable amount of time. You might use it for calculating suggested users in a social graph, or for generating analytics reports. eg. Hadoop, and maybe Hive or HBase. ##Key topics for designing a system 1) **Concurrency** -
vasanthk revised this gist
Feb 6, 2016 . 1 changed file with 5 additions and 1 deletion.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -85,7 +85,11 @@ * A content delivery network (CDN) is a system of distributed servers (network) that deliver webpages and other Web content to a user based on the geographic locations of the user, the origin of the webpage and a content delivery server. * This service is effective in speeding the delivery of content of websites with high traffic and websites that have global reach. The closer the CDN server is to the user geographically, the faster the content will be delivered to the user. * CDNs also provide protection from large surges in traffic. * Full Text Search * Using Sphinx/Lucene/Solr - which achieve fast search responses because, instead of searching the text directly, it searches an index instead. * Offline support/Progressive enhancement * Service Workers * Web Workers * Accessibility * Internationalization * Responsive design -
vasanthk revised this gist
Feb 6, 2016 . 1 changed file with 5 additions and 1 deletion.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -19,7 +19,7 @@ * eg. Usually a scalable system includes webserver (load balancer), service (service partition), database (master/slave database cluster) and caching systems. 3) **Component Design** * Component + specific **APIs** required for each of them. * **Object oriented design** for functionalities. * Map features to modules: One scenario for one module. * Consider the relationships among modules: @@ -81,6 +81,10 @@ ##Web UI System design topics: * Security (CORS) * Using CDN * A content delivery network (CDN) is a system of distributed servers (network) that deliver webpages and other Web content to a user based on the geographic locations of the user, the origin of the webpage and a content delivery server. * This service is effective in speeding the delivery of content of websites with high traffic and websites that have global reach. The closer the CDN server is to the user geographically, the faster the content will be delivered to the user. * CDNs also provide protection from large surges in traffic. * Offline support/Progressive enhancement? * Accessibility * Internationalization -
vasanthk revised this gist
Feb 6, 2016 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -41,7 +41,7 @@ * Load balancing helps you scale horizontally across an ever-increasing number of servers, but caching will enable you to make vastly better use of the resources you already have, as well as making otherwise unattainable product requirements feasible. * **Application caching** requires explicit integration in the application code itself. Usually it will check if a value is in the cache; if not, retrieve the value from the database. * **Database caching** tends to be "free". When you flip your database on, you're going to get some level of default configuration which will provide some degree of caching and performance. Those initial settings will be optimized for a generic usecase, and by tweaking them to your system's access patterns you can generally squeeze a great deal of performance improvement. * **In-memory caches** are most potent in terms of raw performance. This is because they store their entire set of data in memory and accesses to RAM are orders of magnitude faster than those to disk. eg. Memcached or Redis. * eg. Precalculating results (e.g. the number of visits from each referring domain for the previous day), * eg. Pre-generating expensive indexes (e.g. suggested stories based on a user's click history) * eg. Storing copies of frequently accessed data in a faster backend (e.g. Memcache instead of PostgreSQL. -
vasanthk revised this gist
Feb 6, 2016 . 1 changed file with 4 additions and 2 deletions.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -38,8 +38,10 @@ * **Horizontal scaling** * You scale by adding more machines into your pool of resources. * **Caching** * Load balancing helps you scale horizontally across an ever-increasing number of servers, but caching will enable you to make vastly better use of the resources you already have, as well as making otherwise unattainable product requirements feasible. * **Application caching** requires explicit integration in the application code itself. Usually it will check if a value is in the cache; if not, retrieve the value from the database. * **Database caching** tends to be "free". When you flip your database on, you're going to get some level of default configuration which will provide some degree of caching and performance. Those initial settings will be optimized for a generic usecase, and by tweaking them to your system's access patterns you can generally squeeze a great deal of performance improvement. * In-memory caches are most potent in terms of raw performance. This is because they store their entire set of data in memory and accesses to RAM are orders of magnitude faster than those to disk. eg. Memcached or Redis. * eg. Precalculating results (e.g. the number of visits from each referring domain for the previous day), * eg. Pre-generating expensive indexes (e.g. suggested stories based on a user's click history) * eg. Storing copies of frequently accessed data in a faster backend (e.g. Memcache instead of PostgreSQL. -
vasanthk revised this gist
Feb 6, 2016 . 1 changed file with 25 additions and 24 deletions.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -2,77 +2,78 @@ ##Basic Steps 1) **Clarify and agree on the scope of the system** * **Use cases** (description of sequences of events that, taken together, lead to a system doing something useful) * Who is going to use it? * How are they going to use it? * **Constraints** * Mainly identify **traffic and data handling** constraints at scale. * Scale of the system such as requests per second, requests types, data written per second, data read per second) * Special system requirements such as multi-threading, read or write oriented. 2) **High level architecture design (Abstract design)** * Sketch the important components and connections between them, but don't go into some details. * Application service layer (serves the requests) * List different services required. * Data Storage layer * eg. Usually a scalable system includes webserver (load balancer), service (service partition), database (master/slave database cluster) and caching systems. 3) **Component Design** * Component + specific APIs required for each of them. * **Object oriented design** for functionalities. * Map features to modules: One scenario for one module. * Consider the relationships among modules: * Certain functions must have unique instance (Singletons) * Core object can be made up of many other objects (composition). * One object is another object (inheritance) * **Database schema design.** 4) **Understanding Bottlenecks** * Perhaps your system needs a load balancer and many machines behind it to handle the user requests. * Or maybe the data is so huge that you need to distribute your database on multiple machines. What are some of the downsides that occur from doing that? * Is the database too slow and does it need some in-memory caching? 5) **Scaling** your abstract design * **Vertical scaling** * You scale by adding more power (CPU, RAM) to your existing machine. * **Horizontal scaling** * You scale by adding more machines into your pool of resources. * **Caching** * Load balancing helps you scale horizontally across an ever-increasing number of servers, but caching will enable you to make vastly better use of the resources you already have, as well as making otherwise unattainable product requirements feasible. * In-memory caches like Memcached or Redis are good choices. Please never do file-based caching, it makes cloning and auto-scaling of your servers just a pain. * eg. Precalculating results (e.g. the number of visits from each referring domain for the previous day), * eg. Pre-generating expensive indexes (e.g. suggested stories based on a user's click history) * eg. Storing copies of frequently accessed data in a faster backend (e.g. Memcache instead of PostgreSQL. * **Load balancing** * Public servers of a scalable web service are hidden behind a load balancer. This load balancer evenly distributes load (requests from your users) onto your group/cluster of application servers. * Types: Smart client (hard to get it perfect), Hardware load balancers ($$$ but reliable), Software load balancers (hybrid - works for most systems) <p align="center"> <img src="http://lethain.com/static/blog/intro_arch/load_balance.png" alt="Load Balancing"/> </p> * **Database replication** * Database replication is the frequent electronic copying data from a database in one computer or server to a database in another so that all users share the same level of information. The result is a distributed database in which users can access data relevant to their tasks without interfering with the work of others. The implementation of database replication for the purpose of eliminating data ambiguity or inconsistency among users is known as normalization. * **Database partitioning** * Partitioning of relational data usually refers to decomposing your tables either row-wise (horizontally) or column-wise (vertically). ##Key topics for designing a system 1) **Concurrency** * Do you understand threads, deadlock, and starvation? Do you know how to parallelize algorithms? Do you understand consistency and coherence? 2) **Networking** * Do you roughly understand IPC and TCP/IP? Do you know the difference between throughput and latency, and when each is the relevant factor? 3) **Abstraction** * You should understand the systems you’re building upon. Do you know roughly how an OS, file system, and database work? Do you know about the various levels of caching in a modern OS? 4) **Real-World Performance** * You should be familiar with the speed of everything your computer can do, including the relative performance of RAM, disk, SSD and your network. 5) **Estimation** * Estimation, especially in the form of a back-of-the-envelope calculation, is important because it helps you narrow down the list of possible solutions to only the ones that are feasible. Then you have only a few prototypes or micro-benchmarks to write. 6) **Availability & Reliability** * Are you thinking about how things can fail, especially in a distributed environment? Do know how to design a system to cope with network failures? Do you understand durability? -
vasanthk revised this gist
Feb 6, 2016 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -2,7 +2,7 @@ ##Basic Steps 1) *Clarify and agree on the scope of the system* * Use cases (description of sequences of events that, taken together, lead to a system doing something useful) * Who is going to use it? * How are they going to use it? -
vasanthk revised this gist
Feb 6, 2016 . 1 changed file with 3 additions and 1 deletion.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -46,7 +46,9 @@ * Public servers of a scalable web service are hidden behind a load balancer. This load balancer evenly distributes load (requests from your users) onto your group/cluster of application servers. * Types: Smart client (hard to get it perfect), Hardware load balancers ($$$ but reliable), Software load balancers (hybrid - works for most systems) <p align="center"> <img src="http://lethain.com/static/blog/intro_arch/load_balance.png" alt="Load Balancing"/> </p> * Database replication * Database replication is the frequent electronic copying data from a database in one computer or server to a database in another so that all users share the same level of information. The result is a distributed database in which users can access data relevant to their tasks without interfering with the work of others. The implementation of database replication for the purpose of eliminating data ambiguity or inconsistency among users is known as normalization. -
vasanthk revised this gist
Feb 6, 2016 . 1 changed file with 2 additions and 2 deletions.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -43,11 +43,11 @@ * eg. Pre-generating expensive indexes (e.g. suggested stories based on a user's click history) * eg. Storing copies of frequently accessed data in a faster backend (e.g. Memcache instead of PostgreSQL. * Load balancing * Public servers of a scalable web service are hidden behind a load balancer. This load balancer evenly distributes load (requests from your users) onto your group/cluster of application servers. * Types: Smart client (hard to get it perfect), Hardware load balancers ($$$ but reliable), Software load balancers (hybrid - works for most systems) ![Load Balancing](http://lethain.com/static/blog/intro_arch/load_balance.png) * Database replication * Database replication is the frequent electronic copying data from a database in one computer or server to a database in another so that all users share the same level of information. The result is a distributed database in which users can access data relevant to their tasks without interfering with the work of others. The implementation of database replication for the purpose of eliminating data ambiguity or inconsistency among users is known as normalization. * Database partitioning -
vasanthk revised this gist
Feb 6, 2016 . 1 changed file with 3 additions and 0 deletions.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -43,6 +43,9 @@ * eg. Pre-generating expensive indexes (e.g. suggested stories based on a user's click history) * eg. Storing copies of frequently accessed data in a faster backend (e.g. Memcache instead of PostgreSQL. * Load balancing ![Load Balancing](http://lethain.com/static/blog/intro_arch/load_balance.png) * Public servers of a scalable web service are hidden behind a load balancer. This load balancer evenly distributes load (requests from your users) onto your group/cluster of application servers. * Types: Smart client (hard to get it perfect), Hardware load balancers ($$$ but reliable), Software load balancers (hybrid - works for most systems) * Database replication -
vasanthk revised this gist
Feb 6, 2016 . 1 changed file with 3 additions and 3 deletions.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -36,15 +36,15 @@ * You scale by adding more power (CPU, RAM) to your existing machine. * Horizontal scaling * You scale by adding more machines into your pool of resources. * Caching * Load balancing helps you scale horizontally across an ever-increasing number of servers, but caching will enable you to make vastly better use of the resources you already have, as well as making otherwise unattainable product requirements feasible. * In-memory caches like Memcached or Redis are good choices. Please never do file-based caching, it makes cloning and auto-scaling of your servers just a pain. * eg. Precalculating results (e.g. the number of visits from each referring domain for the previous day), * eg. Pre-generating expensive indexes (e.g. suggested stories based on a user's click history) * eg. Storing copies of frequently accessed data in a faster backend (e.g. Memcache instead of PostgreSQL. * Load balancing * Public servers of a scalable web service are hidden behind a load balancer. This load balancer evenly distributes load (requests from your users) onto your group/cluster of application servers. * Types: Smart client (hard to get it perfect), Hardware load balancers ($$$ but reliable), Software load balancers (hybrid - works for most systems) * Database replication * Database replication is the frequent electronic copying data from a database in one computer or server to a database in another so that all users share the same level of information. The result is a distributed database in which users can access data relevant to their tasks without interfering with the work of others. The implementation of database replication for the purpose of eliminating data ambiguity or inconsistency among users is known as normalization. * Database partitioning -
vasanthk revised this gist
Feb 6, 2016 . 1 changed file with 6 additions and 2 deletions.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -36,11 +36,15 @@ * You scale by adding more power (CPU, RAM) to your existing machine. * Horizontal scaling * You scale by adding more machines into your pool of resources. * Load balancing * Public servers of a scalable web service are hidden behind a load balancer. This load balancer evenly distributes load (requests from your users) onto your group/cluster of application servers. * Types: Smart client (hard to get it perfect), Hardware load balancers ($$$ but reliable), Software load balancers (hybrid - works for most systems) * Caching * Load balancing helps you scale horizontally across an ever-increasing number of servers, but caching will enable you to make vastly better use of the resources you already have, as well as making otherwise unattainable product requirements feasible. * In-memory caches like Memcached or Redis are good choices. Please never do file-based caching, it makes cloning and auto-scaling of your servers just a pain. * eg. Precalculating results (e.g. the number of visits from each referring domain for the previous day), * eg. Pre-generating expensive indexes (e.g. suggested stories based on a user's click history) * eg. Storing copies of frequently accessed data in a faster backend (e.g. Memcache instead of PostgreSQL. * Database replication * Database replication is the frequent electronic copying data from a database in one computer or server to a database in another so that all users share the same level of information. The result is a distributed database in which users can access data relevant to their tasks without interfering with the work of others. The implementation of database replication for the purpose of eliminating data ambiguity or inconsistency among users is known as normalization. * Database partitioning -
vasanthk revised this gist
Feb 6, 2016 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -40,7 +40,7 @@ * In-memory caches like Memcached or Redis are good choices. Please never do file-based caching, it makes cloning and auto-scaling of your servers just a pain. * Load balancing * Public servers of a scalable web service are hidden behind a load balancer. This load balancer evenly distributes load (requests from your users) onto your group/cluster of application servers. * Types: Smart client (hard to get it perfect), Hardware load balancers ($$$ but reliable), Software load balancers (hybrid - works for most systems) * Database replication * Database replication is the frequent electronic copying data from a database in one computer or server to a database in another so that all users share the same level of information. The result is a distributed database in which users can access data relevant to their tasks without interfering with the work of others. The implementation of database replication for the purpose of eliminating data ambiguity or inconsistency among users is known as normalization. * Database partitioning -
vasanthk revised this gist
Feb 6, 2016 . 1 changed file with 1 addition and 0 deletions.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -40,6 +40,7 @@ * In-memory caches like Memcached or Redis are good choices. Please never do file-based caching, it makes cloning and auto-scaling of your servers just a pain. * Load balancing * Public servers of a scalable web service are hidden behind a load balancer. This load balancer evenly distributes load (requests from your users) onto your group/cluster of application servers. * Types: Smart client (hard to get it perfect), Hardware load balancers ($$$ but reliable), Software load balancers (hybrid) * Database replication * Database replication is the frequent electronic copying data from a database in one computer or server to a database in another so that all users share the same level of information. The result is a distributed database in which users can access data relevant to their tasks without interfering with the work of others. The implementation of database replication for the purpose of eliminating data ambiguity or inconsistency among users is known as normalization. * Database partitioning -
vasanthk revised this gist
Feb 6, 2016 . 1 changed file with 3 additions and 1 deletion.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -86,4 +86,6 @@ [Scalable System Design Patterns](http://horicky.blogspot.com/2010/10/scalable-system-design-patterns.html) [Scalable Web Architecture and Distributed Systems](http://www.aosabook.org/en/distsys.html) [What is the best way to design a web site to be highly scalable?](http://programmers.stackexchange.com/a/108679/62739) -
vasanthk revised this gist
Feb 6, 2016 . 1 changed file with 2 additions and 2 deletions.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -45,7 +45,7 @@ * Database partitioning * Partitioning of relational data usually refers to decomposing your tables either row-wise (horizontally) or column-wise (vertically). ##Key topics for designing a system 1) Concurrency * Do you understand threads, deadlock, and starvation? Do you know how to parallelize algorithms? Do you understand consistency and coherence? @@ -66,7 +66,7 @@ * Are you thinking about how things can fail, especially in a distributed environment? Do know how to design a system to cope with network failures? Do you understand durability? ##Web UI System design topics: * Security (CORS) * Offline support/Progressive enhancement? * Accessibility -
vasanthk revised this gist
Feb 6, 2016 . 1 changed file with 2 additions and 2 deletions.There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,6 +1,6 @@ #System Design ##Basic Steps 1) Clarify and agree on the scope of the system * Use cases (description of sequences of events that, taken together, lead to a system doing something useful)
NewerOlder