Learning Observability

Over the last few years, something I've been hearing more and more about is the term "Observability" for the systems that we build. I came across this term after following Charity Majors on Twitter, with her forays into Honeycomb.io and the hard lessons she's learned over the years and how much things suck for complex distributed systems. And, as is usually the case, I'm a bit late to the party.

To my admittedly barebones knowledge, observability is defined as:

The ability to ask your system a question, any question. Even questions you didn't know you need to ask. And you don't need to ship code to do it.

Ok cool, so this is where my brain starts to melt. I really struggle to understand the level of abstraction required in your instrumentation to be able to achieve this. This is part of what I'm planning on diving into. I see tools like Application Insights, Splunk and even Seq, and I'm thinking "ok, this sort of sounds like what they're talking about... maybe...". The level of granularity and flexibility these tools allow is incredible and seriously shifted my perspective on how useful and flexible logging can actually be. Tools like KQL and Seq's own internal query language are really incredible, and worth exploring on their own merits.

Is "observability" just another term for "structured logging" (ala Seq)? I don't know. Am I just using a different term for the same thing (and God only knows how many times that type of things bites us as developers!)? I don't know. I'm starting to feel like the meme where someone is trying to budget but has no idea and I'm here like 'omg help I don't know how to run a logging tool'.

Another thing I'm planning on looking at is how do you take a system that is heavily embedded in a non-observability style of monitoring or logging, and make it observable? Are these things contradictory?

After mulling this around again recently when I thought I had some PD coming up, I did what all good developers do... I gathered a bunch of links, chucked them into a OneNote page with the intent of never looking at them again!

And then, for some reason, today at about 4.45pm on a bloody Friday, my brain was like 'hey Adam would you like some MOTIVATION', and then I really didn't have a choice and here we are. So I'm going to list a bunch of links I've come across, with the intent (lol) of doing some reading over the weekend. Then I'll do another blog post.

Hopefully.

A neat little podcast I just stumbled across on Twitter, called O11ycast. I listened to the first episode this afternoon, and it was pretty neat.

What is Observability, from the folks at Dynatrace.

Intro to Observability, from the folks at Honeycomb.

Learn the Elk Stack, whatever that is, from the folks at Logz.io.

Grafana vs Kibana, whatever those are lol, also from the folks at Logz.io.

Pray for me dear reader!

But that's it for now! It's time for some long overdue CSGO.