Ottla: our kafka framework open sourced

[rmangi]

Hello all.

Just finished up a hack week at work and finally got to open source our kafka framework that we use for high throughput consumers (thousands of rps) internally. It’s quite stable, we’ve been using it for years. We wanted more opportunities to build off of it so we bumped the core libraries and have been focused on writing tests and docs. I’ve left it at 0.9-SNAPSHOT until we’re happy with those changes.

It’s “opinionated” so it’s not going to solve all use-cases but we really like it for building individual “cogs” in larger pipelines because it’s simple and surprisingly flexible.

chartbeat-labs/ottla

[daslu]

I’m so happy there is a library named after Ottla Kafka.

[rmangi]

All credit to a woman on our marketing team. We had a much less creative internal name and I asked the company for ideas. We loved everything about the suggestion, it was an easy choice.

[d-t-w]

I assumed this would be another syntactic-sugar wrapper on Kafka Consumers, but it isn’t.

It’s a consumer impl (using the standard java base classes) that has an opinionated approach to set-partition consumption.

It’s interesting! I guess like a very light Kafka Streams for specific use cases. Nice one.

[rmangi]

Thanks a lot! Yes, it’s a very different approach which is why I felt strongly about open sourcing it. I like it when a very complicated system can be made simple by approaching it from a different angle.