Skip to content

Return None instead of skipping lines without timestamp in year_rollo…#1753

Open
EmperorValkorion wants to merge 1 commit into
fox-it:mainfrom
EmperorValkorion:yrh-multiline-fix
Open

Return None instead of skipping lines without timestamp in year_rollo…#1753
EmperorValkorion wants to merge 1 commit into
fox-it:mainfrom
EmperorValkorion:yrh-multiline-fix

Conversation

@EmperorValkorion

Copy link
Copy Markdown

Yield ts as None instead of skipping lines in year_rollover_helper that have no timestamp

This way the helper is also compatible with multiline logs. Also updated test_messages.py, since the new logic caused the number of results for the year_rollover tests to change from 2 to 3. Maybe worth looking into if messages plugin should be updated with the new logic. @JSCU-CNI was there a reason that the original implemention skipped lines without a valid timestamp fully instead of just yielding ts as None?

@JSCU-CNI

Copy link
Copy Markdown
Contributor

@JSCU-CNI was there a reason that the original implemention skipped lines without a valid timestamp fully instead of just yielding ts as None?

Yes, this method was not created with multi-line log messages in mind, but for structured log messages. If a structured log message is malformed we chose to discard it as the forensic value of the record can not be determined without manual review (e.g. we do not know what part of the record was malformed).

Yielding the message without a timestamp seems fine to me, I do not have a strong opinion on that. The previous behavior was mainly intended to catch the attention of the user - hopefully causing them to investigate and either 1) fix the parser or 2) conclude the log is partially corrupted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants