Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Because most logs are unstructured.

> Even at very immature organizations, log data within a service is usually structured.

unless the framework provides it by default, I've never seen this actually happen in real life. Sure I've seen a lot of custom telegraf configs, status end points and the like, but never actual working structured logging.

When I have seen structure logs, each team did it differently, The "ontology" was different (protip: if you're ever discussing ontology in logging then you might as well scream and run away.)



I suspect you and the parent are using different meanings of the word "structured". They're not totally random or they wouldn't be usable. It's a question of what the structuring principle is.


How many unique message formats do you think exist in your org?

actually, how many of your messages include the time and date, and how many different ways of displaying timestamp exist in those messages?

That is why I say logs are unstructured, because all but a very few places actually have the discipline to enforce a single log structure.


Am I crazy here? We run all of our app logs and error logs through LogStash and just have a few filters in there to normalize stuff like the timestamp. Honestly the only peace of data that absolutely HAS to be standardized because that’s the piece of data that splits our log indexes, is the primary sorting mechanism, and at what point we roll up an index into some aggregates and then compress and send it to cold storage.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: