JMK no matter what

how high-frequency logging should be done

If the log is to be analyzed mechanically in any way...

  • use a key-value scheme rather than plain text. you can go even further can write binary dumps, so you can shave off extra microseconds to string-ize these messages. (later, write a separate decoder which will eat the dumps and spit out human-readable dumps.)
  • use a single storage of logs rather than multiple files, and tag each message with what kind of message that is.
  • decide what kind of information should be included in the logs before going prod.
  • when logging starts, put some revision info or whatever at the beginning.
  • use timestamps as accurate as possible.
  • don't do any rounding or trim-off.

I hate parsing ambiguous logs and trying to make any sense from them. I hate, even more, dealing with log formats changing constantly in subtle ways....

2010-08-18 06:34:12 | JM | /writings/cs/ | 0 Comments

Leave a comment

春來不似春

About

Eventstream

Pages

Guestbook

Search

Site Admin

Recent Comments