Logstalgia - Custom Log Format

[ovflowd]

Hi i'm wondering how i create a custom log format for Java's Jersey "Web Server"
An example of log entry is:

[2016-11-02 03:36:47,434]   INFO - erver.RequestLog.bundleProcess - 172.16.28.6 - - [02/Nov/2016:05:36:47 +0000] "GET /hub/api/rest/oauth2/target HTTP/1.1" 200 9 "-" "Jersey/2.22.2 (Apache HttpClient 4.5)" 0 

How i adapt Logstalgia for that?

[Bifrozt]

You can do this by "converting" the custom log entries into the format Logstalgia expects them to be. This example shows you how firewall logs can be transformed into a format that can be read by Logstalgia.

[ovflowd]

Yes i did that using awk, it converts now to Apache Standards Access Log format.
But when i use --sync the Logstalgia shows nothing. Removing it, can only see past things. Going in live makes Logstalgia closes suddently.

I only can access the past data by mounting SSHFS Partition. Doing over SSH with cat/tail doesn't work.
Tested with other Apache HTTP Server and worked.

What i can do to fix it?

[skakri]

I'm not sure if this is the case but maybe it's related to pipe buffering (you didn't specify your log parsing command)?

Related:
http://superuser.com/questions/742238/piping-tail-f-into-awk
http://unix.stackexchange.com/questions/25372/turn-off-buffering-in-pipe

[ovflowd]

Maybe be this. I think it's the file lock.
Will give a look...

[ovflowd]

Tried this:
ssh -x -p22 -l XXX IP stdbuf -i0 -o0 -e0 tail -f /usr/share/hub/logs/internal/services/bundleProcess/access.log | awk -F" " '{print $7, $8, $9, $10, $11, $12, $13, $14, $15, $16}' | logstalgia --sync --disable-glow

And didn't worked. Logstalgia also didn't opened.

[mikedopp]

Ideas on how I would use IIS logs with this? what format I need?

[KurtDeGreeff]

I 'm also highly interested in parsing and visualizing IIS logs..are you aware it can be done with the custom format? Many thanks