Change the future

Hermes - Python messaging at Spotify

Neville Li

Audience level:
Intermediate
Category:
Distributed Computing

Description

How Spotify builds large scale distributed system with Python and the Hermes framework.

Abstract

Hermes is a network messaging stack created at Spotify, on top of existing open source stack such as ZeroMQ, protobuf and gevent. We are migrating backend service from legacy stack to Hermes and this session explains the rationale behind some of the choices. It also gives a brief introduction of some typical network topology for backend messaging at Spotify. Finally we discuss some lessons learned from building such large scale Python systems and the problems we are facing.

Sponsors