The problem is that they’re all on different servers. Once you use log aggregation stuff like DataDog, Splunk, or Kibana you get it, but before it’s hard to see the benefits. Stuff like being able to see a timestamp of when an error first appeared and then from the same place see what other stuff happened around the same time.
If I had dozens or hundreds of servers that would make a huge difference, but for under a dozen I think the cost of setting that all up isn’t worth the added benefit. Plus if the log aggregation goes down (which I’ve seen happen with some really hairy issues) you’re back to grepping files so it’s good to know how.
Totally. I’m talking more from the enterprise perspective. Even apart from that I’m not sure if the cost is worth it at that scale. Even using foss solutions the dev hours setting it up might not be worth it.
I feel you. The problem with a lot of Elastic style document search engines is that they don’t ever let you search by very explicit terms because of how the index is built. I believe the pros outweigh the cons but I often wish I could “drop into” grep, less, and others from within the log aggregation tool.
Searching a log file? I want
less. Searching all log files? I want log aggregation lol.If I knew what I was looking for I could grep all the log files and pipe the output to another file to aggregate them.
The problem is that they’re all on different servers. Once you use log aggregation stuff like DataDog, Splunk, or Kibana you get it, but before it’s hard to see the benefits. Stuff like being able to see a timestamp of when an error first appeared and then from the same place see what other stuff happened around the same time.
If I had dozens or hundreds of servers that would make a huge difference, but for under a dozen I think the cost of setting that all up isn’t worth the added benefit. Plus if the log aggregation goes down (which I’ve seen happen with some really hairy issues) you’re back to grepping files so it’s good to know how.
Totally. I’m talking more from the enterprise perspective. Even apart from that I’m not sure if the cost is worth it at that scale. Even using foss solutions the dev hours setting it up might not be worth it.
Exactly.
One log file, or all, I want grep or awk, maybe with find in front, possibly throw some jq on top if something is logging big json blobs.
I feel you. The problem with a lot of Elastic style document search engines is that they don’t ever let you search by very explicit terms because of how the index is built. I believe the pros outweigh the cons but I often wish I could “drop into” grep, less, and others from within the log aggregation tool.
That’s a lot slower at scale than something like Loki.