Re: analysis of java application logs

From:

Tom Anderson <twic@urchin.earth.li>

Newsgroups:

comp.lang.java.programmer

Date:

Tue, 24 May 2011 15:04:41 +0100

Message-ID:

<alpine.DEB.2.00.1105241504000.27717@urchin.earth.li>

On Mon, 23 May 2011, Ulrich Scholz wrote:

I'm looking for an approach to the problem of analyzing application
log files.

I need to analyse Java log files from applications (i.e., not logs of
web servers). These logs contain Java exceptions, thread dumps, and
free-form log4j messages issued by log statements inserted by
programmers during development. Right now, these man-made log entries
do not have any specific format.

What I'm looking for is a tool and/or strategy that supports in lexing/
parsing, tagging, and analysing the log entries. Because there is only
little defined syntax and grammar - and because you might not know
what you are looking for - the task requires the quick issuing of
queries against the log data base. Some sort of visualization would be
nice, too.

Pointers to existing tools and approaches as well as appropriate tools/
algorithms to develop the required system would be welcome.

grep.

For particularly thorny bits of analysis, sed.

There are those who advocate awk for this sort of thing, but frankly that
seems like overkill.

tom

--
shouting drunkenly about 6502 assembler at parties