DevOps Tension and the Honeycomb Solution
The “DevOps” phrase may be easy to say, but it masks an underlying tension between development and operations that can lead to poor communication and ultimately poor management of increasingly complex tech stacks. Addressing this DevOps tension and communication gap is a challenge that the founders of an observability startup called Honeycomb aim to solve.
Honeycomb’s Story
Honeycomb’s story starts circa 2012, when two engineers, Christine Yen and Charity Majors, found themselves working together at Parse, which developed a back-end for mobile apps. Yen was a developer while Majors was more inclined towards the infrastructure. A friendship blossomed between them, despite differences in jobs and natural talents.
“Everything was so fast you could churn through tons and tons and tons of data, spit out a chart, and then say ‘Drill into this one app and this one SDK version,'” Yen said. “Because we were a platform, we wouldn’t know if there was a Russian dating app that would launch, that would use a new endpoint in an unexpected way and take down our Mongo cluster. But it happened all the time, and so we had to figure out how to get ahead of that.”
Emboldened by the Scuba experience, Yen and Majors left Facebook and founded Honeycomb in 2016. The pair had an idea for a new kind of observability tool that could express operations concepts in a developers’ language, and (hopefully) bridge the gap between Dev and Ops, thereby bringing back some semblance of simplicity—or at least understandability.
Honeycomb’s Technology
At a technical level, Yen and Majors made a few decisions in 2016 that set Honeycomb on its path. It decided there was no good reason to split up the Holy Trinity of observability data—logs, metrics, and traces—as some observability companies do. By storing logs, metrics, and traces together, it would be easier to correlate details found in them, which would accelerate the remediation of problems.
The second big architectural decision the company made was it selected a column-oriented database in which to store all this observability data. This made Honeycomb more like Snowflake than Datadog, Yen said.
“We didn’t invent column stores, and we’ll be the first to tell you that there’s nothing, honestly, all that special about column stores,” she told BigDATAwire in an interview at re:Invent 2024 last week. “But we were the first people to really start to question, should we maybe not have logs over here and metrics over here forever? Should we maybe try to find ways [to keep them together], especially if they’re talking about the same thing that happened in an application?”
Providing Context
Instead of spending time and effort trying to join these data types together so they can be analyzed, Honeycomb’s design naturally stores them together, using the OpenTelemetry (OTEL) format. That not only eliminates the need for data integration, but it helps with the next capability that Honeycomb is known for: Providing context to the complex metadata it collects, which in turn helps turn the technical infrastructure jargon that operations folks speak into the business language that developers speak.
“It meant that previously Charity would be like ‘Christine, what did you do to the write throughput on my Cassandra instance?’ And I’d be like, let me try to figure it out. I’m really scared,” Yen said. “It turned those conversations into, ‘Hey, Christine, I saw elevated write throughput on my Cassandra instances. So I looked, and it seems that that elevated traffic is driven by this app on this endpoint. What’s going on?'”
And I’d be like, Oh, that’s a regression,” she continued. “Now let me go look, because now it’s in my world. Now I can go try to reproduce it in a test. Now I can go look at the business logic. Now I understand why it matters, because I know which apps you’re talking about, and that’s the way that it changed how we work together as an engineering team.”
Conclusion
Honeycomb’s approach seems to be resonating, as the San Francisco-based company has already signed 800 paying customers, including companies like Vanguard, Fender Guitars, and Jack Henry & Associates. The company, which has raised about $147 million through a Series D round, also last week picked up a 2024 BigDATAwire Readers’ Choice Award in the category of Best Big Data Product: DataOps and Observability.
Frequently Asked Questions
Q: What problem does Honeycomb aim to solve?
A: Honeycomb aims to solve the DevOps tension and communication gap between development and operations teams.
Q: Who founded Honeycomb?
A: Honeycomb was founded by Christine Yen and Charity Majors in 2016.
Q: What is Honeycomb’s technical approach?
A: Honeycomb’s technical approach includes storing logs, metrics, and traces together in a column-oriented database, and using OpenTelemetry (OTEL) format.
Q: What is the purpose of Honeycomb’s design?
A: The purpose of Honeycomb’s design is to provide context to the complex metadata it collects, turning technical infrastructure jargon into business language that developers understand.
Q: What recognition has Honeycomb received?
A: Honeycomb received a 2024 BigDATAwire Readers’ Choice Award in the category of Best Big Data Product: DataOps and Observability.
