21 Commits

Author SHA1 Message Date
Vibhu Pandey
9e449e2858
feat(auth): drop group table (#7672)
### Summary

drop group table
2025-04-26 15:50:02 +05:30
Nityananda Gohain
48936bed9b
chore: multitenancy in integrations (#7507)
* chore: multitenancy in integrations

* chore: multitenancy in cloud integration accounts

* chore: changes to cloudintegrationservice

* chore: rename migration

* chore: update scan function

* chore: update scan function

* chore: fix migration

* chore: fix struct

* chore: remove unwanted code

* chore: update scan function

* chore: migrate user and pat for integrations

* fix: changes to the user for integrations

* fix: address comments

* fix: copy created_at

* fix: update non revoked token

* chore: don't allow deleting pat and user for integrations

* fix: address comments

* chore: address comments

* chore: add checks for fk in dialect

* fix: service migration

* fix: don't update user if user is already migrated

* fix: update correct service config

* fix: remove unwanted code

* fix: remove migration for multiple same services which is not required

* fix: fix migration and disable disaboard if metrics disabled

* fix: don't use ee types

---------

Co-authored-by: Vikrant Gupta <vikrant@signoz.io>
2025-04-15 15:35:36 +00:00
Nityananda Gohain
71d1dfe9bd
chore: use new uuid in pipelines (#7487) 2025-03-31 16:45:00 +05:30
Nityananda Gohain
8fc43a00f8
fix: collector connection to opamp without orgID (#7474) 2025-03-28 20:19:37 +05:30
Nityananda Gohain
1dfebed93a
fix: pipelines postgres support and multitenancy (#7371)
* fix: pipelines postgres support and multitenancy

* fix: minor fixes

* fix: address minor comments

* fix: rename package pipelinetypes
2025-03-24 10:17:12 +05:30
Shivanshu Raj Shrivastava
efd4e30edf
fix: publish signoz as package (#7378)
Signed-off-by: Shivanshu Raj Shrivastava <shivanshu1333@gmail.com>
2025-03-20 15:31:41 +00:00
Nityananda Gohain
c3951afdfd
fix: refactor auth package (#7110)
* fix: refactor auth package

* fix: minor changes

* fix: refactor jwt

* fix: add tests and address comments

* fix: address comments

* fix: add uncomitted file

* fix: address comments

* fix: update tests
2025-02-17 18:16:41 +05:30
Vibhu Pandey
dc15ee8176
feat(sqlmigration): consolidate all sqlmigrations into one package (#7018)
* feat(sqlmigration): add sqlmigrations

* feat(sqlmigration): test sqlmigrations

* feat(sqlmigration): add remaining factories

* feat(sqlmigration): consolidate into single package

* fix(telemetrystore): remove existing env variables

* fix(telemetrystore): fix DSN
2025-02-04 09:23:36 +00:00
Vibhu Pandey
837f434fe9
feat(sqlstore): open a connection to sql once (#6867) 2025-01-22 07:19:38 +00:00
Raj Kamal Singh
cd9f27ab08
Fix: QS: logs pipelines: better validation of pipelines being saved (#6652)
* chore: add test validating invalid field paths in pipeline operators are rejected

* chore: refactor posted pipelines validation to use a controller method

* fix: run a collector simulation to validate pipeline config being saved

* chore: minor cleanup
2024-12-18 10:42:14 +05:30
Srikanth Chekuri
ae594061e9
chore: fix query-service logging (#4696) 2024-03-27 00:07:29 +05:30
Raj Kamal Singh
9ace374855
Feat: QS: Log Pipelines for installed integrations (#4674)
* chore: refactor: inject sqlx.DB into opamp.initDB instead of DB file name

* chore: reorganize test utils a little

* chore: add test validating pipelines for installed integrations show up in pipelines list

* chore: get basic integration pipelines testcase passing

* chore: reconcile experimental changes with latest state of develop

* chore: add integration test for reordering of pipelines

* chore: marker for integration pipelines using Id

* chore: hookup propagation of installed integration pipelines by opamp

* chore: add util for mapping slices

* chore: add support for reordering integration pipelines

* chore: exclude user saved integration pipelines if no longer installed

* chore: flesh out rest of intgeration pipelines scenarios

* chore: handle scenario when an integration is installed before any pipelines exist

* chore: notify agentConf of update after uninstalling an integration

* chore: some minor cleanup

* chore: some more cleanup

* chore: update ee server for changed controllers

* chore: some more cleanup

* chore: change builtin integration id prefix to avoid using colons that break yaml

* chore: update builtin integration id in test
2024-03-11 14:15:11 +05:30
Raj Kamal Singh
581bd07b35
Fix: query service: escape dollars in collector config generated for pipelines. (#4242)
* chore: add test validating pipeline config containing dollar works

* chore: collector simulator: use fileprovider with expandconverter like actual collector

* chore: get tests passing

* chore: take a configGenerator func in simulation and unify config generation logic

* fix: escape $ in config generated for log pipelines
2023-12-19 21:24:53 +05:30
Raj Kamal Singh
3a1e8d523a
Fix: qs: allow saving pipelines without connected agents (#4189)
* chore: add test validating pipelines can be saved without connected agents

* chore: allow pipelines to be saved without connected agents
2023-12-09 10:17:06 +05:30
Raj Kamal Singh
050b866173
chore: return warning logs too from collector simulator (#3888)
* chore: return warning logs too from collector simulator

* chore: also return collector logs in preview API response to help with debugging

---------

Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
2023-11-03 17:41:09 +05:30
Raj Kamal Singh
626da7533e
Fix: log pipelines: generated operators should have appropriate if condition to avoid spamming collector logs (#3870)
* chore: experiment with using a tmp file for getting collector logs in simulator

* chore: collector simulator: cleaned up tmp file based collector logs capture

* chore: add test validating regex proc doesn't error for logs that dont match

* chore: return collector error logs from pipeline preview API

* chore: add test validating regex processor doesn't log errors for mismatched logs

* chore: add if condition for generated regex processors

* chore: add test case validating json parser ignore non json logs

* chore: add if condition for operator generated for json parser

* chore: add test case validating move processor ignores logs with missing field

* chore: add if condition for operator generated for move

* chore: add test case validating copy processor ignores logs with missing field

* chore: add if condition for operator generated for copy

* chore: add test case validating remove processor ignores logs with missing field

* chore: add if condition for operator generated for remove

* chore: log pipelines: ensuring json parser ignores log if json field is missing

* chore: log pipelines: ensure regex parser ignores log if field is missing
2023-11-01 22:12:35 +05:30
Raj Kamal Singh
546d98ca9c
feat: agentConf.manager as opamp conf provider (#3747)
* chore: add integration test for log pipelines being recommended to agents on connection

* chore: agentConf.Manager as AgentConfigProvider

* feat: logparsingpipelines as an AgentFeature

* chore: some cleanup

* feat: update agentConf init in ee query-service

* chore: more cleanup and move opamp/logspipeline -> logparsingpipeline/collector_config.go

* chore: some more cleanup

* chore: set agent.RemoteConfig after broadcasting conf to all agents

* chore: add test scenario for post remote conf application
2023-10-15 21:04:19 +05:30
Raj Kamal Singh
a84a70df14
QS: logs pipelines preview http handler (#3701)
* feat: add logsparsingpipeline controller api for pipeline previews

* feat: add http handler and route for generating pipeline previews

* feat: use a response model for pipeline previews response
2023-10-10 14:09:55 +05:30
Raj Kamal Singh
3db8a25eb9
Add support in query service for querybuilder filterset based log pipelines (#3560)
* chore: use v3.Filterset as pipeline filters in logparsing pipelines integration tests

* chore: get logparsing integration tests passing with filterset based pipeline

* chore: get all other breaking tests passing

* chore: move models.logparsingpipeline to logparsingpipeline.model

* chore: implement Valuer and Scanner interfaces for v3.FilterSet
2023-09-20 12:39:34 +05:30
Raj Kamal Singh
dfd94f67bd
chore: return 400 http status on pipeline validation error (#3472)
* chore: add integration test for log parsing pipelines validation

* chore: add helpers for creating unauthorized, unavailable & not found api errors

* chore: return *model.APIError from logpipeline and agentConf functions

* chore: some cleanup

* chore: some more cleanup

* chore: one more round of cleanups
2023-09-10 16:48:29 +05:30
Nityananda Gohain
1a3e46cecd
feat: integrate pipelines API (#2457)
* chore: integrate pipelines API

* fix: limit support integrated in pipelines

* fix: interface to string

* fix: json parser and allow deleting all pipelines

* fix: output modified if operators are disabled

* fix: validation updated for operators

* fix: expression check added

* fix: regex expression check added

* fix: remove operator validation updated

* fix: tests updated for pipeline builder

* fix: fix error messages in http handler

* fix: dont return payload if there is an error

* fix: extracting userId from context moved to auth package

* fix: api errors moved to http handler

* fix: get version logic updated

* fix: deployment result message updated

* fix: pipeline builder edgecase fixed and tests updated

* fix: get failing postablePipeline tests to pass

---------

Co-authored-by: Vishal Sharma <makeavish786@gmail.com>
Co-authored-by: Raj <rkssisodiya@gmail.com>
2023-07-31 21:34:42 +05:30