Scaling I/O Bound Microservices
In this meetup, we will continue our #2ndhalf journey to the next^2. You can see the talks from the 1st meetup of this series here - http://bit.ly/second-half-p1 Nowadays, scaling and auto-scaling have become relatively easy tasks. Everyone knows how to set up auto-scaling environments - Auto-Scaling groups, Swarm, Kubernetes, etc.
But when we try to scale I/O Bound workloads:
- Message queues (Kafka, Rabbit, NATS)
- Distributed databases (Hadoop, Cassandra)
- Storage subsystems (CEPH, GlusterFS, HDFS), the traditional auto-scaling mechanisms are just not enough.
Heavy calculations must be performed to determine the I/O bottlenecks. Rebalancing the data after a scaling event can take up to hours depending on your data & could, resulting in data loss if not properly designed.
We will deep dive into this type of workload and walk you through code samples you can apply in your own environment.
We will contact you as soon as possible.