-
What is orchestration?
-
What we have built?
-
Plans
Auto-Deployment
Scheduling / HW Sharing
Monitoring
Self-Healing
Minimum components
Availability first
Introspection
Minimum configuration
Upgrades
Only One Orchestration
Security
Network Async and Lossy
Leader Election
Minimum components
Availability first
Introspection
Minimum configuration
vagga
lithos
cantal
verwalter
$ git clone git@host:proj
$ cd proj
$ vagga
Available commands:
run Start app with postgres and redis
doc Build docs with sphinx
test Run unit tests
js Compile javascripts
$ vagga run
Okay, server started at http://localhost:8080
$ vagga doc
Installing sphinx
...
Docs are built in doc/_build/html/index.html
$ git pull
$ vagga test
Requirements txt changed
Rebuilding container "test"
................
Ok 20 successful
Self-documentation
IDE folders with deps
Versioning
Continuous Integration
Production-grade containers
Start process
Set up environment
Mount file systems
Restart on failure
Downloading
Build
Network API
Configuration files
introspection
security
Sandbox config
Read-only/nosuid/noexec
Fixed command-line (*)
(*) You control only version and number of instances
System metrics
CGroups
Application metrics
Peer discovery
Aggregated stats
3k-10k metrics in 10-40ms
1h in 15-35MiB
Debug build
10 Django 1 Posgres 1 Nginx
| → | jupyter 5D mars 3D 1N pluto 2D 1P
|
| < 10K rpm | > 10K rpm |
|---|
Django | 10 | 20 |
Celery | 5 | 2 |
Inputs
Leader-Election
Monolithic Scheduler
Render
Web UI
Raft-like protocol
No fsync
Low Traffic
Consistency
Side-Effect Free
Reproducible
Transformer
10 Django 1 Posgres 1 Nginx
| → | jupyter 5D mars 3D 1N pluto 2D 1P
|
7.5K rpm |
JSON
Handlebars -> nginx.conf
nginx -t
nginx -s reload
10 Djangos → 5 Running, 77 Err/sec → 1 min → Blacklist
Met → Sch → Met → Sch
Ship It
Dokku-like thing
HTTP Routing
",d.firstChild.style.width="5px",b.shrinkWrapBlocks=d.offsetWidth!==3,c.style.zoom=1),i.removeChild(c),c=d=f=g=null}),i.removeChild(n),c=d=f=g=h=i=n=null,b}();var H=/(?:\{[\s\S]*\}|\[[\s\S]*\])$/,I=/([A-Z])/g;p.extend({cache:{},deletedIds:[],uuid:0,expando:"jQuery"+(p.fn.jquery+Math.random()).replace(/\D/g,""),noData:{embed:!0,object:"clsid:D27CDB6E-AE6D-11cf-96B8-444553540000",applet:!0},hasData:function(a){return a=a.nodeType?p.cache[a[p.expando]]:a[p.expando],!!a&&!K(a)},data:function(a,c,d,e){if(!p.acceptData(a))return;var f,g,h=p.expando,i=typeof c=="string",j=a.nodeType,k=j?p.cache:a,l=j?a[h]:a[h]&&h;if((!l||!k[l]||!e&&!k[l].data)&&i&&d===b)return;l||(j?a[h]=l=p.deletedIds.pop()||p.guid++:l=h),k[l]||(k[l]={},j||(k[l].toJSON=p.noop));if(typeof c=="object"||typeof c=="function")e?k[l]=p.extend(k[l],c):k[l].data=p.extend(k[l].data,c);return f=k[l],e||(f.data||(f.data={}),f=f.data),d!==b&&(f[p.camelCase(c)]=d),i?(g=f[c],g==null&&(g=f[p.camelCase(c)])):g=f,g},removeData:function(a,b,c){if(!p.acceptData(a))return;var d,e,f,g=a.nodeType,h=g?p.cache:a,i=g?a[p.expando]:p.expando;if(!h[i])return;if(b){d=c?h[i]:h[i].data;if(d){p.isArray(b)||(b in d?b=[b]:(b=p.camelCase(b),b in d?b=[b]:b=b.split(" ")));for(e=0,f=b.length;e