5 Commits

Author SHA1 Message Date
Raj Kamal Singh
aad44a1037
Feat: QS: logs pipelines severity parsing processor (#4132)
* chore: update test helper for making logs

* chore: add happy case test for severity parser

* feat: get severity parsing processor working and add more happy path tests

* chore: add test for validating severity parser doesn't spam collector logs

* chore: add if condition to generated severity_parser operators

* chore: add postablePipeline validation for severity parser

* chore: minor cleanup in tests

* chore: allow trace and fatal in severity mappings
2023-12-01 17:22:22 +05:30
Raj Kamal Singh
1b6b3c2fdf
Feat: query service: logs pipelines timestamp parsing processor (#4105)
* chore: relocate tests for trace and grok parsing processor

* chore: add test for timestamp parsing processor

* feat: update PipelineOperator model for time parser fields and get tests passing

* chore: add test cases for validating time parser fails silently on mismatched logs

* chore: add helper for generating regex for strptime layouts

* feat: time_parser ignore logs whose parseFrom value doesn't match strptime layout

* chore: escape regex special chars if any in the layout string before preparing regex

* chore: add operator.If on time_parser when using layout type epoch

* chore: finish up with operator.If on time_parser for  layout type

* chore: postable pipeline validation for time parser

* chore: some cleanup

* chore: some more cleanup

* chore: add validation for strptime layouts in postable pipelines

---------

Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
2023-11-29 18:55:01 +05:30
Raj Kamal Singh
79aef73767
Fix: Query Service: get trace parser working in log parsing pipelines (#3820)
* chore: add test for ensuring pipeline previews work for trace parser processors

* chore: updates to trace parser validation in postable pipelines

* chore: extract auth.randomHex into utils.RandomHex for reuse

* chore: get trace parser preview test passing

* chore: start with JSON serialized trace parser in test to cover deserialization

* chore: address PR feedback
2023-10-29 16:58:31 +05:30
Raj Kamal Singh
3db8a25eb9
Add support in query service for querybuilder filterset based log pipelines (#3560)
* chore: use v3.Filterset as pipeline filters in logparsing pipelines integration tests

* chore: get logparsing integration tests passing with filterset based pipeline

* chore: get all other breaking tests passing

* chore: move models.logparsingpipeline to logparsingpipeline.model

* chore: implement Valuer and Scanner interfaces for v3.FilterSet
2023-09-20 12:39:34 +05:30
Nityananda Gohain
1a3e46cecd
feat: integrate pipelines API (#2457)
* chore: integrate pipelines API

* fix: limit support integrated in pipelines

* fix: interface to string

* fix: json parser and allow deleting all pipelines

* fix: output modified if operators are disabled

* fix: validation updated for operators

* fix: expression check added

* fix: regex expression check added

* fix: remove operator validation updated

* fix: tests updated for pipeline builder

* fix: fix error messages in http handler

* fix: dont return payload if there is an error

* fix: extracting userId from context moved to auth package

* fix: api errors moved to http handler

* fix: get version logic updated

* fix: deployment result message updated

* fix: pipeline builder edgecase fixed and tests updated

* fix: get failing postablePipeline tests to pass

---------

Co-authored-by: Vishal Sharma <makeavish786@gmail.com>
Co-authored-by: Raj <rkssisodiya@gmail.com>
2023-07-31 21:34:42 +05:30