10 Commits

Author SHA1 Message Date
Srikanth Chekuri
2f949d2738
chore: address large number of staticcheck issues (#5176)
* chore: address large number of staticcheck issues

* chore: fix tests

* chore: fix more issue

* chore: fix options
2024-06-11 20:10:38 +05:30
Raj Kamal Singh
96e7505922
Chore: logs pipelines cover all available processors with tests (#4454)
* chore: add explicit happy case test for regex parsing processor

* chore: add explicit happy case test for grok parsing processor

* chore: add explicit happy case test for JSON parsing processor

* chore: log pipelines: move trace parser test to processors_test.go

* chore: add explicit happy case test for ADD processor

* chore: add explicit happy case test for remove processor

* chore: add explicit happy case test for copy processor

* chore: add explicit happy case test for move processor

* fix: fix broken grok parser test: change test int value to int64

---------

Co-authored-by: Nityananda Gohain <nityanandagohain@gmail.com>
2024-04-04 18:48:01 +05:30
Raj Kamal Singh
aad44a1037
Feat: QS: logs pipelines severity parsing processor (#4132)
* chore: update test helper for making logs

* chore: add happy case test for severity parser

* feat: get severity parsing processor working and add more happy path tests

* chore: add test for validating severity parser doesn't spam collector logs

* chore: add if condition to generated severity_parser operators

* chore: add postablePipeline validation for severity parser

* chore: minor cleanup in tests

* chore: allow trace and fatal in severity mappings
2023-12-01 17:22:22 +05:30
Raj Kamal Singh
1b6b3c2fdf
Feat: query service: logs pipelines timestamp parsing processor (#4105)
* chore: relocate tests for trace and grok parsing processor

* chore: add test for timestamp parsing processor

* feat: update PipelineOperator model for time parser fields and get tests passing

* chore: add test cases for validating time parser fails silently on mismatched logs

* chore: add helper for generating regex for strptime layouts

* feat: time_parser ignore logs whose parseFrom value doesn't match strptime layout

* chore: escape regex special chars if any in the layout string before preparing regex

* chore: add operator.If on time_parser when using layout type epoch

* chore: finish up with operator.If on time_parser for  layout type

* chore: postable pipeline validation for time parser

* chore: some cleanup

* chore: some more cleanup

* chore: add validation for strptime layouts in postable pipelines

---------

Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
2023-11-29 18:55:01 +05:30
Raj Kamal Singh
050b866173
chore: return warning logs too from collector simulator (#3888)
* chore: return warning logs too from collector simulator

* chore: also return collector logs in preview API response to help with debugging

---------

Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
2023-11-03 17:41:09 +05:30
Raj Kamal Singh
626da7533e
Fix: log pipelines: generated operators should have appropriate if condition to avoid spamming collector logs (#3870)
* chore: experiment with using a tmp file for getting collector logs in simulator

* chore: collector simulator: cleaned up tmp file based collector logs capture

* chore: add test validating regex proc doesn't error for logs that dont match

* chore: return collector error logs from pipeline preview API

* chore: add test validating regex processor doesn't log errors for mismatched logs

* chore: add if condition for generated regex processors

* chore: add test case validating json parser ignore non json logs

* chore: add if condition for operator generated for json parser

* chore: add test case validating move processor ignores logs with missing field

* chore: add if condition for operator generated for move

* chore: add test case validating copy processor ignores logs with missing field

* chore: add if condition for operator generated for copy

* chore: add test case validating remove processor ignores logs with missing field

* chore: add if condition for operator generated for remove

* chore: log pipelines: ensuring json parser ignores log if json field is missing

* chore: log pipelines: ensure regex parser ignores log if field is missing
2023-11-01 22:12:35 +05:30
Raj Kamal Singh
9c21449239
chore: empty out trace parser fields if parseFrom is empty (#3850) 2023-11-01 10:10:36 +05:30
Raj Kamal Singh
79aef73767
Fix: Query Service: get trace parser working in log parsing pipelines (#3820)
* chore: add test for ensuring pipeline previews work for trace parser processors

* chore: updates to trace parser validation in postable pipelines

* chore: extract auth.randomHex into utils.RandomHex for reuse

* chore: get trace parser preview test passing

* chore: start with JSON serialized trace parser in test to cover deserialization

* chore: address PR feedback
2023-10-29 16:58:31 +05:30
Raj Kamal Singh
f427bac993
Fix: grok parser in pipeline previews (#3810)
* chore: add test for previewing pipeline with grok parser

* chore: import grok parser in logparsing preview to ensure its a registered logtransform op

* chore: add dependencies for grok parsing operator
2023-10-26 19:37:13 +05:30
Raj Kamal Singh
718eb7b381
QS: logparsingpipeline previews (#3694)
* chore: rename model.GetLogsResponse -> model.SignozLog for use in both requests and responses

* feat: add test for simulating log pipelines processing

* feat: get pipeline preview tests passing

* chore: cleanup
2023-10-09 15:25:13 +05:30