Time-Based Data Aggregation with Cassandra
For collecting time-based data, for example, such that is streamed from single or multiple resource monitors including CPU, disk usage etc., data volumes can pretty quickly become daunting and unusable even with the most modern relational database.
In such cases, using a NoSQL DB may be a good idea. Modeling the data isn't as trivial as inserting each new time-based sample into a new row - rather, the recommended model is actually aggregating the data horizontally by adding new columns whose name is the collection timestamp and the value is the measured metric.
Below is a summary of all resources I've used to develop a solution for the problem of this type using Cassandra DB - for some reason they were not easy to gather, hoping this summary may help whoever is implementing a solution for a similar problem.
- Basic Time Series with Cassandra:
- Modeling Time Series Data on top of Cassandra:
- Cassandra Developer Center :: Advanced Time Series with Cassandra
- Also, it is probably a good idea to read about Cassandra's Time UUIDs, which you will need to use as column names, explained in:
- Lastly, if using Hector as Cassandra driver (recommended), it probably makes more sense to use its me.prettyprint.cassandra.utils.TimeUUIDUtils
Hopefully, a more concrete code example will soon follow soon...