There was a lot in this that resonated with my experience at Microsoft, but there’s one bit near the end that needs amplifying:
Location based salaries are discriminatory
If you have set up a company somewhere expensive (Silicon Valley, Seattle, London, wherever) and you require people to relocate there then you need to pay them for the cost of living premium. If you are allowing people to work anywhere in the world then there is no excuse for penalising the people who choose to live somewhere cheap (though requiring some minimal level of available Internet connectivity for remote work locations is fine).
Anything where you’re paying for input not output is a bad sign. The one that really annoyed me was that MS allowed you to buy a week of extra vacation time. People who take a healthy amount of vacation and don’t work long hours are vastly more productive than people who work for a long time without a break. The company is offering you the ability to pay them so that you can work more productively. This is totally the wrong incentive. This is why my contract at SCI, which is the template for people joining my team, has a requirement on the minimum amounts of leave people take, not the maximum. Smart people are far more likely to have their productivity suffer because they don’t stop working than because they take too much time off (for a lot of people I’ve worked with, the two weeks after they get back from a week off are far more productive than the four weeks before). Taking time off isn’t something that the company allows you to do as a reward, it’s something that the company requires that you do to ensure that you’re working at peak productivity. A tired person can introduce a bug in five minutes that it takes a week of debugging to fix. Managers need to be able to tell people that they’re working too hard and that they should stop for a few days.
If you are allowing people to work anywhere in the world then there is no excuse for penalising the people who choose to live somewhere cheap
I strongly agree with this. People’s cost of living can vary widely, and location is just one component. But we would never think it was OK to vary salary based on other lifestyle differences that affect those costs: whether someone is married or single, whether they have kids, whether they drive or use public transportation, whether they live in a house or an apartment.
I agree with your point, but I’d be willing to bet that there are a ton of people with kids who would totally support this. I mean, unpopular opinion, but parental leave is basically a bonus you get for making babies (not that I think it shouldn’t exist, but if there’s 8 weeks of parental leave, then everyone should be able to take 8 weeks off every few years or whatever).
To add to this, for mothers who give birth, it is typically at least 6 weeks of medical leave. Giving birth to a child is a physically traumatic event. Being a male I have obviously never experienced it first hand but I have witnessed my partner go through it twice. The suggestion that maternity leave is anything even close to resembling “time off” is, frankly, borderline offensive (and the fact that in the US paid maternity leave is not even legally required is practically criminal).
Heh yeah unlike vacation time where people come back more productive, I can only imagine that parental leave is seen as “they’re going to be scattered and sleep-deprived anyway… and that’s not going to be good for anyone”
A long time ago, I worked in a web dev startup where both founders were absolutely adamant that employees take as much parental leave as they were entitled to. Both had worked in places where the opposite was true, both had ended up missing out on the earliest bits of their kids’ lives (and one of them would sometimes say that it had also put an unreasonable burden on his wife), and both had regretted it deeply. Neither wanted to foist that upon anyone else, ever. It wasn’t directly relevant to me since I neither have nor want kids, but it’s the kind of little thing that reminds you that there are workplaces not run by complete sociopaths.
The same place also deliberately tried to avoid the kind of “workday as LAN party” grind that burns out everyone but the youngest devs, and pushes out older people who have family lives.
It then got acquired by a larger company, and the previously very healthy company culture promptly went to shit.
That’s true, but it’s usually still by choice. If I decide to take 8 weeks of to help the local firefighters, should it get the same preferential treatment? Not an easy question.
I think at some point it becomes a matter of choice vs. right. Do you have the right to spend 8 weeks helping the local firefighters? Usually no, in most cultures, but honestly I think it’d be kinda nice. Do you have the right to have children if you want to? Generally it’s considered pretty important that the answer is “yes” in most places, so making it so that your workplace has to give you leave for it is pretty reasonable. As gpanders points out, a significant amount of it is medical leave.
I think most people understand that some people have additional needs and that it’s okay, even virtuous, for society to compensate them for that.
Famously this idea is described as: “From each according to their ability, to each according to their need”
Now, I think everyone should have more holiday, but while we live in a system with limited paid time off then it makes sense to me that people with needy dependents should get proportionally more.
But people who live in a super expensive city literally need more. Having children is a lifestyle choice, just like living in an expensive city. I would take a different view if we, as a species, were critically under-populated. But since we’re not, I see no significant difference.
Edit: for something, like a disability, that is a condition, not a choice, I absolutely support accommodations, including additional pay and/or time off (whatever is helpful).
Raising children is labour and should be compensated. Extra paid time off for parents is part of that.
Also, for many people children are not really a choice. They’re accidental or it is unthinkable not to have them due to social conditioning, biological imperative, etc.
I’m not accusing you of this, but given our capitalist context, arguments that having children is a choice like any other tend to end up implying that poor people shouldn’t have kids, which is fucked up.
We could solve this as a society without prioritising parents especially by backing big transfers of income and political power to regular people.
Lots of people do realise they’re getting ripped off and that life could be better for everyone, but it won’t happen without more people getting organised. If you’re reading this and you’re not already in a feisty union or in some way building and exercising collective power then this is a good time to start, even if all you want is to protect your own quality of life.
Good international orgs: Industrial Workers of the World (IWW)
Good orgs in the UK: Unite, Acorn, Peoples Momentum
Good orgs in the USA: IWW, Democratic Socialists of America, your local renter or community union
I originally wrote a long sibling comment, but I’ll pare it down to a reply agreeing with and adding to your excellent comment!
Suppose that half of people in the US who have children had a choice, and that we want to accept their additional need as a reason for them to receive proportionally more. There are reasons over whose causes people have even less choice for which they might want to live in a city.
One reason might be that someone’s family has lived in a city for generations; do they have more an obligation to uproot their parents and grandparents to a lower cost-of-living area than someone has an obligation not to have a child?
Another reason might be that someone belongs to a minority group, in consequence of which, in defiance of reason but in light of their personal experience, they feel more welcome in cities than in lower cost-of-living areas. I can personally testify to this.
I think that having salaries that are not based on location is very reasonable. I think location-based salaries are also a decent compromise. I personally do not think that adjusting salaries based on the number of kids someone has or their identity etc. makes sense. I could see myself being swayed by new arguments.
Smart people are far more likely to have their productivity suffer because they don’t stop working than because they take too much time off (for a lot of people I’ve worked with, the two weeks after they get back from a week off are far more productive than the four weeks before). Taking time off isn’t something that the company allows you to do as a reward, it’s something that the company requires that you do to ensure that you’re working at peak productivity.
…and I think this is a very fundamental difference in different types of work. There’s design/project work, where you’re hired for your brain: deadlines are relatively distant, you build and design things, you make decisions and solve problems, and the quality and success of the result is very decoupled from the amount of hours put into it. And then there’s operations work, where you are hired for your output: not just flipping burgers or occupying a slot in a factory floor, but things like tech support, on-call or 24hr NOC/sysadmin work, working in construction or operating machinery, etc. In those your productivity is VERY coupled to the hours put into it: if you are present then the business is operating whether you don’t have anything or whether you’re working your ass off, and if you are not present then the business is not operating whether there is 1 Task Needing Doing or 1000. In those jobs a LOT of effort goes into scheduling people, and reliability and flexibility are valued much, much more highly by the business.
This is a distinction that most people don’t seem to know exists, for whatever reason. Nobody seems to talk about it but it very fundamentally shapes how the expectations of industry are shaped, which then shapes the laws and regulations and contracts and stuff that go into it. Maybe most people tend to work one or the other type of job their whole life?
I think there are two kinds of operations work, from your characterisation. If you are the receptionist or security guard, for example, there’s a huge difference between not doing your job at all and doing it badly. The gap between doing it badly and doing it well can also be large but, in normal circumstances, it is less than the complete shutdown that happens if you do nothing. In contrast, SRE-type work often involves doing the kind of things that, if you don’t do them until tomorrow then it’s not a disaster but if you do them wrong can result in data loss, reputation loss, and even penalty clauses for a customer.
Even this isn’t so clear cut. A receptionist who is overstressed and snaps at the wrong person may lose a million dollar contract.
The thing that surprised me recently was reading productivity studies on construction workers, which showed exactly the same shape curves as for knowledge workers. It turns out that making mistakes from from tiredness that offset benefits from working more is pretty universal.
Oh, it’s certainly not a clear cut division. But in my experience the differences between the two are much bigger than the differences within the two. They result in very different mindsets for me.
… which showed exactly the same shape curves as for knowledge workers.
Oooooh, interesting! Not surprising, but not entirely obvious. Do you remember where I can find those?
I’ve heard something similar from the local railway company, where (in the IT development department) you get more free time for a cut in pay..
A friend of mine sounded very happy about it at the time, you make a compelling argument against it. At the very least, the burnout that might result should be something that costs the company a lot more than what it saves by cutting pay.
Everyone one wants US salaries but nobody’s willing to give up all the local benefits (5 weeks paid holidays, 35h per week, strong worker rights, nationalized retirement, free healthcare…)
On the other hand, if someone is happy to work for $50k because they’re 12 timezone hours away and live on a beach in a $10k bungalow and they have full time domestic help for a few thousand bucks a year, I don’t know why a company should hire them at $500k just because a few other employees live in Palo Alto.
Companies will only pay what they have to pay. That’s how free markets work.
It gets more complicated when it’s an employee’s choice to move from Manhattan to Vietnam, of course. Should the company reduce their pay just because the cost of living for the remote employee has dropped? That seems creepy.
There aren’t that many highly paid jobs in which you can just choose to live anywhere. Programmers are pretty lucky.
Companies will only pay what they have to pay. That’s how free markets work.
Markets will eventually price goods based on both supply and demand. Prices go up if demand outstrips supply and up if supply outstrips demand. The company offering $50K is usually not the only source of demand.
If you make this person a $50K offer, then another company can offer them $100K and they’ll be really motivated to take it and that company is still saving money relative to hiring someone for $500K. Retention is usually cheaper than hiring, so offering a salary that encourages people to leave is a bad long-term strategy. We’ve seen this with Indian outsourcing companies: their competent workers rapidly get hired away at higher salaries and so they’re left with inexperienced or incompetent people (or they have to match international salary rates).
If you’re hiring remote workers anywhere in the world, then you’re competing for talent not just with companies local to that person but with any other company that offers remote work. If you have someone who you’re paying $50K, they don’t have to offer $500K to steal them, $100K is fine. But then someone else comes along and hires them for $200K, and then someone else offers $300K. And they’re still cheaper than those $500K folks, so the ones in the bay area are getting fewer offers. If you’re getting $500K of value from someone doing the job, then it makes sense to offer $400K to hire someone talented away from the competition. If you’re getting $1M of value from someone, then offering $500K anywhere in the world gives you a much better pool of applicants than offering $500K to people in the Manhattan and $50K to people in Vietnam.
Salaries probably won’t eventually converge on the current bay area rates, but there are a lot more smart people that don’t live in the bay area than do and an increasing number of companies competing for their labour. When I was looking for a job last year, I got several pings from companies that do full remote and don’t do location-based salaries. If you do, you’re probably not competitive as an employer.
I’ve hired a few thousand people across 6 continents, and (unfortunately) had to lay off some as well. Everything that you wrote in this response is correct in my experience. I’m not arguing for or against, or defending, or promoting. Like what you wrote here, I was simply pointing out reality. Reality doesn’t ever seem to care what I think about it.
I have always paid well when I hired people. But when hiring someone in Prague or New Delhi or St Petersberg, I wasn’t typically offering Bay Area rates. I may have paid better than what most people in Czech Republic or India or Russia make, but as you suggest, I was offering based on the market prices and how badly I wanted / needed a particular employee, and how badly other companies wanted them. I’m super proud to have had a lot of employees targeted by other companies for recruitment, and super happy for their successes after leaving my teams. I always tried to make it hard for other companies to poach, but good people are always worth paying dearly to have, and it’s also good for people to get to try out some new challenges from time to time.
This brings back so many memories of my time at the birdsite: spending the first month adding metrics just to find out what was slow (nobody knew), and later wondering why they hired a thousand engineers who don’t seem to work on any of the major problems.
I’m really glad to read that Gitlab didn’t fire the person who deleted their database. Having a deleted database without working backups is a systemic problem that the whole company is responsible for, and too often a startup will just pick a scapegoat, like the last person who touched it.
Thanks for writing this, this was fascinating to read. Also it’s a little surreal to have you mention various things happening and going “oh shit I read about that in the news, right!”
…how the [laptop management] changes in general gave off a vibe of solutions in search of problems to justify their existence.
Yeah this is the baaaane of corporate life. The earlier mention of database sharding is another example. Various accounting and travel policies tend to be similar.
GitLab offers two types of product: self-hosted installations and a software as a service (SaaS) offering. I believe most companies won’t be able to effectively offer such a setup, … the two types of setups also come with different requirements and ways of applying updates.
This whole section is actually really insightful! It sounds very much a centrailzed-vs-decentralized problem.
It’s interesting to see the section suggesting that a company can’t do both a service and an on premises product.
This effectively results in two code paths in many parts of your platform: one for the SaaS version, and one for the self-hosted version.
I think if you end up treating them as two different things that’s probably right, but I think that’s generally a mistake.
When I was at Joyent we went to great lengths to just have the one body of software, which we deployed both in our own public cloud service and sold to people on premises to have their own private cloud environment. While we had obviously a very close relationship with the operations group running the public cloud, we generally tried to make the code the same in all cases. If there were differences, they’d be differences of configuration wherever possible. I think this is critical, but doable, if you want to offer both a hosted service and something people can self host.
What kind of customers did you have? I suspect for something like GitLab, a lot of the people using the self-hosted option have far more conservative operations folks than an internal team for a SaaS company. If I were using it, I’d probably want something like a six-month update cycle (with security fixes pushed automatically). This means that you need the ability to maintain security back ports to at least one release for the self-hosted version and you probably don’t want to ship new features in a release unless they’re stable. In contrast, for the SaaS version, new features can go in experimentally every day, possibly with a feature flag for A/B testing and be part of an active UAT cycle.
It’s been five years at this point so my memory is a bit hazy. There were some retail and e-commerce outfits, and some content producing companies with online publications, and various other bits and pieces. It was infrastructure stuff: VMs/containers and object storage.
Certainly some customers were relatively conservative in their update cadence, but mostly it was just that they needed to schedule around critical periods for their respective businesses.
I don’t really believe in A/B testing; it feels exploitative at best, and a bit like gaslighting at worst. We did lots of testing but it was generally before we would integrate new features. We were big believers in FCS (first customer ship) quality all the time; if it didn’t work yet it didn’t go into the main branch. When you stick as closely as possible to that philosophy, there’s vastly less risk involved in cutting a new release. By contrast, back porting piecemeal fixes to old “stable” branches is expensive and error prone – how do you really know you got everything you need for a complete fix?
The time it takes to deploy code is vital to the success of an organization
I believe in this strongly. And the good news is that commit to test to deploy, and the frequency of deploys, are one of the easier things to measure. I’d argue it’s a lot easier and effective for most teams to optimize for deployment speed than to (prematurely) optimize for an SLA type objective.
At the beginning they sound very happy with working remotely, while at the end it sounds like working in the office would have helped to reduce the loss in social interactions. It’s understandable for gitlab to stop company-wide meetups with that many employees, though it should have been possible to meetup in smaller groups.
It’s hard to coordinate even on a smaller scale (~100 people) like us at thoughtbot. Visa requirements and other country-specific bullshit get in the way very easily.
Doing smaller ones (for folks who are geographically closer) is a good alternative.
There was a lot in this that resonated with my experience at Microsoft, but there’s one bit near the end that needs amplifying:
If you have set up a company somewhere expensive (Silicon Valley, Seattle, London, wherever) and you require people to relocate there then you need to pay them for the cost of living premium. If you are allowing people to work anywhere in the world then there is no excuse for penalising the people who choose to live somewhere cheap (though requiring some minimal level of available Internet connectivity for remote work locations is fine).
Anything where you’re paying for input not output is a bad sign. The one that really annoyed me was that MS allowed you to buy a week of extra vacation time. People who take a healthy amount of vacation and don’t work long hours are vastly more productive than people who work for a long time without a break. The company is offering you the ability to pay them so that you can work more productively. This is totally the wrong incentive. This is why my contract at SCI, which is the template for people joining my team, has a requirement on the minimum amounts of leave people take, not the maximum. Smart people are far more likely to have their productivity suffer because they don’t stop working than because they take too much time off (for a lot of people I’ve worked with, the two weeks after they get back from a week off are far more productive than the four weeks before). Taking time off isn’t something that the company allows you to do as a reward, it’s something that the company requires that you do to ensure that you’re working at peak productivity. A tired person can introduce a bug in five minutes that it takes a week of debugging to fix. Managers need to be able to tell people that they’re working too hard and that they should stop for a few days.
I strongly agree with this. People’s cost of living can vary widely, and location is just one component. But we would never think it was OK to vary salary based on other lifestyle differences that affect those costs: whether someone is married or single, whether they have kids, whether they drive or use public transportation, whether they live in a house or an apartment.
I agree with your point, but I’d be willing to bet that there are a ton of people with kids who would totally support this. I mean, unpopular opinion, but parental leave is basically a bonus you get for making babies (not that I think it shouldn’t exist, but if there’s 8 weeks of parental leave, then everyone should be able to take 8 weeks off every few years or whatever).
I think the key difference is that parental leave is very much not “time off”. You are gonna work your ass off in that time, no matter what happens.
To add to this, for mothers who give birth, it is typically at least 6 weeks of medical leave. Giving birth to a child is a physically traumatic event. Being a male I have obviously never experienced it first hand but I have witnessed my partner go through it twice. The suggestion that maternity leave is anything even close to resembling “time off” is, frankly, borderline offensive (and the fact that in the US paid maternity leave is not even legally required is practically criminal).
Heh yeah unlike vacation time where people come back more productive, I can only imagine that parental leave is seen as “they’re going to be scattered and sleep-deprived anyway… and that’s not going to be good for anyone”
A long time ago, I worked in a web dev startup where both founders were absolutely adamant that employees take as much parental leave as they were entitled to. Both had worked in places where the opposite was true, both had ended up missing out on the earliest bits of their kids’ lives (and one of them would sometimes say that it had also put an unreasonable burden on his wife), and both had regretted it deeply. Neither wanted to foist that upon anyone else, ever. It wasn’t directly relevant to me since I neither have nor want kids, but it’s the kind of little thing that reminds you that there are workplaces not run by complete sociopaths.
The same place also deliberately tried to avoid the kind of “workday as LAN party” grind that burns out everyone but the youngest devs, and pushes out older people who have family lives.
It then got acquired by a larger company, and the previously very healthy company culture promptly went to shit.
That’s true, but it’s usually still by choice. If I decide to take 8 weeks of to help the local firefighters, should it get the same preferential treatment? Not an easy question.
I think at some point it becomes a matter of choice vs. right. Do you have the right to spend 8 weeks helping the local firefighters? Usually no, in most cultures, but honestly I think it’d be kinda nice. Do you have the right to have children if you want to? Generally it’s considered pretty important that the answer is “yes” in most places, so making it so that your workplace has to give you leave for it is pretty reasonable. As gpanders points out, a significant amount of it is medical leave.
What do you mean by “right”?
Legally speaking there is no law that forbids you to help the firefighters. There is also no law that forces the firefighters to have you join them.
In the same way, there is no law that forbids you to make children. There is also no law that forces your potential partner to help you with that.
To me that really looks like the same situation, legally speaking.
I think most people understand that some people have additional needs and that it’s okay, even virtuous, for society to compensate them for that.
Famously this idea is described as: “From each according to their ability, to each according to their need”
Now, I think everyone should have more holiday, but while we live in a system with limited paid time off then it makes sense to me that people with needy dependents should get proportionally more.
But people who live in a super expensive city literally need more. Having children is a lifestyle choice, just like living in an expensive city. I would take a different view if we, as a species, were critically under-populated. But since we’re not, I see no significant difference.
Edit: for something, like a disability, that is a condition, not a choice, I absolutely support accommodations, including additional pay and/or time off (whatever is helpful).
Raising children is labour and should be compensated. Extra paid time off for parents is part of that.
Also, for many people children are not really a choice. They’re accidental or it is unthinkable not to have them due to social conditioning, biological imperative, etc.
I’m not accusing you of this, but given our capitalist context, arguments that having children is a choice like any other tend to end up implying that poor people shouldn’t have kids, which is fucked up.
We could solve this as a society without prioritising parents especially by backing big transfers of income and political power to regular people.
I support this wholeheartedly. Making life better for everyone makes life better for parents (and maybe even disproportionately so).
Lots of people do realise they’re getting ripped off and that life could be better for everyone, but it won’t happen without more people getting organised. If you’re reading this and you’re not already in a feisty union or in some way building and exercising collective power then this is a good time to start, even if all you want is to protect your own quality of life.
I originally wrote a long sibling comment, but I’ll pare it down to a reply agreeing with and adding to your excellent comment!
Suppose that half of people in the US who have children had a choice, and that we want to accept their additional need as a reason for them to receive proportionally more. There are reasons over whose causes people have even less choice for which they might want to live in a city.
One reason might be that someone’s family has lived in a city for generations; do they have more an obligation to uproot their parents and grandparents to a lower cost-of-living area than someone has an obligation not to have a child?
Another reason might be that someone belongs to a minority group, in consequence of which, in defiance of reason but in light of their personal experience, they feel more welcome in cities than in lower cost-of-living areas. I can personally testify to this.
I think that having salaries that are not based on location is very reasonable. I think location-based salaries are also a decent compromise. I personally do not think that adjusting salaries based on the number of kids someone has or their identity etc. makes sense. I could see myself being swayed by new arguments.
Tell me you do not have children, without telling me you do not have children.
…and I think this is a very fundamental difference in different types of work. There’s design/project work, where you’re hired for your brain: deadlines are relatively distant, you build and design things, you make decisions and solve problems, and the quality and success of the result is very decoupled from the amount of hours put into it. And then there’s operations work, where you are hired for your output: not just flipping burgers or occupying a slot in a factory floor, but things like tech support, on-call or 24hr NOC/sysadmin work, working in construction or operating machinery, etc. In those your productivity is VERY coupled to the hours put into it: if you are present then the business is operating whether you don’t have anything or whether you’re working your ass off, and if you are not present then the business is not operating whether there is 1 Task Needing Doing or 1000. In those jobs a LOT of effort goes into scheduling people, and reliability and flexibility are valued much, much more highly by the business.
This is a distinction that most people don’t seem to know exists, for whatever reason. Nobody seems to talk about it but it very fundamentally shapes how the expectations of industry are shaped, which then shapes the laws and regulations and contracts and stuff that go into it. Maybe most people tend to work one or the other type of job their whole life?
I think there are two kinds of operations work, from your characterisation. If you are the receptionist or security guard, for example, there’s a huge difference between not doing your job at all and doing it badly. The gap between doing it badly and doing it well can also be large but, in normal circumstances, it is less than the complete shutdown that happens if you do nothing. In contrast, SRE-type work often involves doing the kind of things that, if you don’t do them until tomorrow then it’s not a disaster but if you do them wrong can result in data loss, reputation loss, and even penalty clauses for a customer.
Even this isn’t so clear cut. A receptionist who is overstressed and snaps at the wrong person may lose a million dollar contract.
The thing that surprised me recently was reading productivity studies on construction workers, which showed exactly the same shape curves as for knowledge workers. It turns out that making mistakes from from tiredness that offset benefits from working more is pretty universal.
Oh, it’s certainly not a clear cut division. But in my experience the differences between the two are much bigger than the differences within the two. They result in very different mindsets for me.
Oooooh, interesting! Not surprising, but not entirely obvious. Do you remember where I can find those?
I’ve heard something similar from the local railway company, where (in the IT development department) you get more free time for a cut in pay..
A friend of mine sounded very happy about it at the time, you make a compelling argument against it. At the very least, the burnout that might result should be something that costs the company a lot more than what it saves by cutting pay.
Everyone one wants US salaries but nobody’s willing to give up all the local benefits (5 weeks paid holidays, 35h per week, strong worker rights, nationalized retirement, free healthcare…)
On the other hand, if someone is happy to work for $50k because they’re 12 timezone hours away and live on a beach in a $10k bungalow and they have full time domestic help for a few thousand bucks a year, I don’t know why a company should hire them at $500k just because a few other employees live in Palo Alto.
Companies will only pay what they have to pay. That’s how free markets work.
It gets more complicated when it’s an employee’s choice to move from Manhattan to Vietnam, of course. Should the company reduce their pay just because the cost of living for the remote employee has dropped? That seems creepy.
There aren’t that many highly paid jobs in which you can just choose to live anywhere. Programmers are pretty lucky.
Markets will eventually price goods based on both supply and demand. Prices go up if demand outstrips supply and up if supply outstrips demand. The company offering $50K is usually not the only source of demand.
If you make this person a $50K offer, then another company can offer them $100K and they’ll be really motivated to take it and that company is still saving money relative to hiring someone for $500K. Retention is usually cheaper than hiring, so offering a salary that encourages people to leave is a bad long-term strategy. We’ve seen this with Indian outsourcing companies: their competent workers rapidly get hired away at higher salaries and so they’re left with inexperienced or incompetent people (or they have to match international salary rates).
If you’re hiring remote workers anywhere in the world, then you’re competing for talent not just with companies local to that person but with any other company that offers remote work. If you have someone who you’re paying $50K, they don’t have to offer $500K to steal them, $100K is fine. But then someone else comes along and hires them for $200K, and then someone else offers $300K. And they’re still cheaper than those $500K folks, so the ones in the bay area are getting fewer offers. If you’re getting $500K of value from someone doing the job, then it makes sense to offer $400K to hire someone talented away from the competition. If you’re getting $1M of value from someone, then offering $500K anywhere in the world gives you a much better pool of applicants than offering $500K to people in the Manhattan and $50K to people in Vietnam.
Salaries probably won’t eventually converge on the current bay area rates, but there are a lot more smart people that don’t live in the bay area than do and an increasing number of companies competing for their labour. When I was looking for a job last year, I got several pings from companies that do full remote and don’t do location-based salaries. If you do, you’re probably not competitive as an employer.
I’ve hired a few thousand people across 6 continents, and (unfortunately) had to lay off some as well. Everything that you wrote in this response is correct in my experience. I’m not arguing for or against, or defending, or promoting. Like what you wrote here, I was simply pointing out reality. Reality doesn’t ever seem to care what I think about it.
I have always paid well when I hired people. But when hiring someone in Prague or New Delhi or St Petersberg, I wasn’t typically offering Bay Area rates. I may have paid better than what most people in Czech Republic or India or Russia make, but as you suggest, I was offering based on the market prices and how badly I wanted / needed a particular employee, and how badly other companies wanted them. I’m super proud to have had a lot of employees targeted by other companies for recruitment, and super happy for their successes after leaving my teams. I always tried to make it hard for other companies to poach, but good people are always worth paying dearly to have, and it’s also good for people to get to try out some new challenges from time to time.
This brings back so many memories of my time at the birdsite: spending the first month adding metrics just to find out what was slow (nobody knew), and later wondering why they hired a thousand engineers who don’t seem to work on any of the major problems.
I’m really glad to read that Gitlab didn’t fire the person who deleted their database. Having a deleted database without working backups is a systemic problem that the whole company is responsible for, and too often a startup will just pick a scapegoat, like the last person who touched it.
Thanks for writing this, this was fascinating to read. Also it’s a little surreal to have you mention various things happening and going “oh shit I read about that in the news, right!”
Yeah this is the baaaane of corporate life. The earlier mention of database sharding is another example. Various accounting and travel policies tend to be similar.
This whole section is actually really insightful! It sounds very much a centrailzed-vs-decentralized problem.
It’s interesting to see the section suggesting that a company can’t do both a service and an on premises product.
I think if you end up treating them as two different things that’s probably right, but I think that’s generally a mistake.
When I was at Joyent we went to great lengths to just have the one body of software, which we deployed both in our own public cloud service and sold to people on premises to have their own private cloud environment. While we had obviously a very close relationship with the operations group running the public cloud, we generally tried to make the code the same in all cases. If there were differences, they’d be differences of configuration wherever possible. I think this is critical, but doable, if you want to offer both a hosted service and something people can self host.
What kind of customers did you have? I suspect for something like GitLab, a lot of the people using the self-hosted option have far more conservative operations folks than an internal team for a SaaS company. If I were using it, I’d probably want something like a six-month update cycle (with security fixes pushed automatically). This means that you need the ability to maintain security back ports to at least one release for the self-hosted version and you probably don’t want to ship new features in a release unless they’re stable. In contrast, for the SaaS version, new features can go in experimentally every day, possibly with a feature flag for A/B testing and be part of an active UAT cycle.
I’ve been running gitlab EE for some time. We always wait for the X.1 release if we can, as we’ve had some bad experience with major version releases.
But gitlab will still send you a mail at 1 AM about the incoming security fix for the same day (which arrives at 11PM).
It’s been five years at this point so my memory is a bit hazy. There were some retail and e-commerce outfits, and some content producing companies with online publications, and various other bits and pieces. It was infrastructure stuff: VMs/containers and object storage.
Certainly some customers were relatively conservative in their update cadence, but mostly it was just that they needed to schedule around critical periods for their respective businesses.
I don’t really believe in A/B testing; it feels exploitative at best, and a bit like gaslighting at worst. We did lots of testing but it was generally before we would integrate new features. We were big believers in FCS (first customer ship) quality all the time; if it didn’t work yet it didn’t go into the main branch. When you stick as closely as possible to that philosophy, there’s vastly less risk involved in cutting a new release. By contrast, back porting piecemeal fixes to old “stable” branches is expensive and error prone – how do you really know you got everything you need for a complete fix?
I believe in this strongly. And the good news is that commit to test to deploy, and the frequency of deploys, are one of the easier things to measure. I’d argue it’s a lot easier and effective for most teams to optimize for deployment speed than to (prematurely) optimize for an SLA type objective.
At the beginning they sound very happy with working remotely, while at the end it sounds like working in the office would have helped to reduce the loss in social interactions. It’s understandable for gitlab to stop company-wide meetups with that many employees, though it should have been possible to meetup in smaller groups.
It’s hard to coordinate even on a smaller scale (~100 people) like us at thoughtbot. Visa requirements and other country-specific bullshit get in the way very easily.
Doing smaller ones (for folks who are geographically closer) is a good alternative.