Skip to content

About

Semantic log compression, search and alerting prototype

Resources

Stars

0 stars

Watchers

1 watching

Forks

Repository files navigation

Semantic log compression, search and alerting prototype
-------------------------------------------------------

The aim of this software is to identify (with some smart logic,
eventually with some AI) semantic similarities between lines of log
files and to use these similarities for a high compression ratio.

The 'slc' tool performs the extraction of attributes and the compression
and index creation, and the 'slcgrep' tool is for indexed search in
analogy to the regular grep. Additionally, the 'slcnotify' tool is
available to trigger hooks upon occurrence of an attribute value.
Finally, 'netlogserve' is used to emulate a networked logging service.

Tool installation:

 Run `make install'. This is unpythonic but pragmatic for now.
 Should work on Debian/Ubuntu type systems.

Tool invocation:

 Run `slc -h' to find out about command line parameters.
 Run the chain
   netlogserve examples/admin.log | slc -j - | slcnotify portnumber:8008
 to replay the example logfile, analyse it, and react on occurrences of
 a specified port number.

Documentation:

 Read this blog post for more information:
https://blog.zhaw.ch/splab/2020/03/01/log-files-semantic-compression-and-learned-attribute-based-notifications/

About

Semantic log compression, search and alerting prototype

Resources

Stars

0 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages