160 Commits

Author SHA1 Message Date
Nityananda Gohain
1dfebed93a
fix: pipelines postgres support and multitenancy (#7371)
* fix: pipelines postgres support and multitenancy

* fix: minor fixes

* fix: address minor comments

* fix: rename package pipelinetypes
2025-03-24 10:17:12 +05:30
Shivanshu Raj Shrivastava
efd4e30edf
fix: publish signoz as package (#7378)
Signed-off-by: Shivanshu Raj Shrivastava <shivanshu1333@gmail.com>
2025-03-20 15:31:41 +00:00
Nityananda Gohain
a9618886b9
fix: support multitenancy in dashboards & savedviews (#7237)
* fix: support multitenancy in dashboards

* fix: support multitenancy in saved views

* fix: move migrations to provider file

* fix: remove getUserFromClaims and use new errors

* fix: remove getUserFromClaims and use new errors

* fix: use new errors in dashboards.go

* Update ee/query-service/app/api/dashboard.go

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* fix: minor changes

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-03-12 17:18:11 +05:30
Nityananda Gohain
2d73f91380
Fix: Multitenancy support for ORG (#7155)
* fix: support multitenancy in org

* fix: register and login working now

* fix: changes to migration

* fix: migrations run both on sqlite and postgres

* fix: remove user flags from fe and be

* fix: remove ingestion keys from update

* fix: multitenancy support for apdex settings

* fix: render ts for users correctly

* fix: fix migration to run for new tenants

* fix: clean up migrations

* fix: address comments

* Update pkg/sqlmigration/013_update_organization.go

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* fix: fix build

* fix: force invites with org id

* Update pkg/query-service/auth/auth.go

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* fix: address comments

* fix: address comments

* fix: provier with their own dialect

* fix: update dialects

* fix: remove unwanted change

* Update pkg/query-service/app/http_handler.go

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* fix: different files for types

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-03-06 15:39:45 +05:30
Raj Kamal Singh
bc907b9e61
chore: AWS Integration: include CWAgent metrics namespace for ec2 (#7165)
* chore: include CWAgent namespace for ec2 metrics

* chore: update aws integration service metrics connection status queried labels

* chore: fix breaking test
2025-02-21 10:45:56 +05:30
Nityananda Gohain
eba2049a4d
fix: signoz shouldn't panic with postgres provider (#7138)
All necessary changes so that whatever initalize SQL commans run, they are moved to bun so that it works with both sqlite and postgres.
2025-02-18 11:08:09 +05:30
Nityananda Gohain
c3951afdfd
fix: refactor auth package (#7110)
* fix: refactor auth package

* fix: minor changes

* fix: refactor jwt

* fix: add tests and address comments

* fix: address comments

* fix: add uncomitted file

* fix: address comments

* fix: update tests
2025-02-17 18:16:41 +05:30
Raj Kamal Singh
169bd3f65b
Feat: aws integrations: dashboards (#7058)
* chore: get started on dashboards plumbing for AWS integration services

* feat: cloud integrations: include cloud integrations dashboards in dashboards list

* feat: cloud integrations: get cloud integration dashboard by id

* feat: dashboard url in cloud integrations svc detail

* feat: ec2 overview dashboard

* chore: add ec2 overview dashboard image

* chore: finish up with v0 definitions for EC2 and RDS

* chore: some cleanup

* chore: fix broken test

* chore: add connection url composition

---------

Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
2025-02-13 14:15:51 +00:00
Raj Kamal Singh
acd9b97ee3
Feat: aws integrations: service connection status (#7032)
* feat: add ability to get latest metric received ts by labelValues filter

* feat: svc metrics connection status check

* feat: aws integration svc logs connection check

* chore: fix broken test

* chore: address PR review comments

* chore: address PR feedback

* chore: fix broken test expectation

* fix: use resource filter for logs connection status
2025-02-06 13:08:47 +05:30
Raj Kamal Singh
3b550c485d
Feat: cloud integrations: agent check-in api (#7004)
* chore: cloudintegrations: rename rds def

* feat: cloudintegrations: shape of agent check in response for v0 release

* chore: cloudintegrations: add validation for response expected after agent check in

* chore: cloudintegrations: accumulate teletry collection strategies for enabled services

* chore: cloudintegrations: use map struct to parse from map to struct with json tags

* chore: cloudintegrations: telemetry collection strategy for services

* chore: cloudintegrations: wrap up test for agent check in resp

* chore: some cleanup

* chore: some cleanup

* chore: some minor renaming

* chore: address review comment

* chore: some cleanup

---------

Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
2025-02-03 20:52:15 +05:30
Nityananda Gohain
d1e7cc128f
fix: telemetry store (#6923)
* fix: inital changes for telemetry store

* fix: add tests and use proper config for conn

* fix: add telemetry store test

* fix: add backward compatibility for old variables and update example conf

* fix: move wrapper to telemetry store

* fix: no need to pass query for settings

* fix: remove redundant config for ch conn

* fix: use clickhouse dsn instead

* fix: update example config

* fix: update backward compatibility code

* fix: use hooks in telemetrystore

* fix: address minor comments

---------

Co-authored-by: Vibhu Pandey <vibhupandey28@gmail.com>
2025-01-30 10:21:55 +00:00
Vikrant Gupta
1e61e6c2f6
feat(trace-details): query service changes for trace details (#6906)
* feat(trace-details): query service changes for trace details

* feat(trace-detail): refactor query service trace apis

* feat(trace-detail): minor bug fixes

* feat(trace-detail): address review comments

* feat(trace-detail): address review comments
2025-01-24 00:16:38 +05:30
Vibhu Pandey
837f434fe9
feat(sqlstore): open a connection to sql once (#6867) 2025-01-22 07:19:38 +00:00
Raj Kamal Singh
8e11a988be
chore: cloud integrations: include cloud account id in account status response (#6833) 2025-01-16 22:51:35 +05:30
Raj Kamal Singh
bab8c8274c
feat: aws integration UI facing api: services (#6803)
* feat: cloud service integrations: get model and repo interface started

* feat: cloud service integrations: flesh out more of cloud services model

* feat: cloud integrations: reorganize things a little

* feat: cloud integrations: get svc controller started

* feat: cloud integrations: add stubs for EC2 and RDS postgres services

* feat: cloud integrations: add validation for listing and getting available svcs and some cleanup

* feat: cloud integrations: refactor helpers in existing integrations code for reuse

* feat: cloud integrations: parsing of cloud service definitions

* feat: cloud integrations: impl for getCloudProviderService

* feat: cloud integrations: some reorganization

* feat: cloud integrations: some more cleanup

* feat: cloud integrations: add validation for listing available cloud provider services

* feat: cloud integrations: API endpoint for listing available cloud provider services

* feat: cloud integrations: add validation for getting details of a particular service

* feat: cloud integrations: API endpoint for getting details of a service

* feat: cloud integrations: add controller validation for configuring cloud services

* feat: cloud integrations: get serviceConfigRepo started

* feat: cloud integrations: service config in service list summaries when queried for cloud account id

* feat: cloud integrations: only a supported service for a connected cloud account can be configured

* feat: cloud integrations: add validation for configuring services via the API

* feat: cloud integrations: API for configuring services

* feat: cloud integrations: some cleanup

* feat: cloud integrations: fix broken test

---------

Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
2025-01-16 17:36:09 +05:30
Raj Kamal Singh
d5b847c091
feat: aws integration: UI facing QS api for cloud account management (#6771)
* feat: init app/cloud_integrations

* feat: get API test started for cloudintegrations account lifecycle

* feat: cloudintegrations: get controller started

* feat: cloud integrations: add cloudintegrations.Controller to APIHandler and servers

* feat: cloud integrations: get routes started

* feat: cloud integrations: get accounts table schema started

* feat: cloud integrations: get cloudProviderAccountsSQLRepository started

* feat: cloud integrations: cloudProviderAccountsSQLRepository.listAccounts

* feat: cloud integrations: http handler and controller plumbing for /generate-connection-url

* feat: cloud integrations: cloudProviderAccountsSQLRepository.upsert

* feat: cloud integrations: finish up with /generate-connection-url

* feat: cloud integrations: add cloudProviderAccountsRepository.get

* feat: cloud integrations: add API test expectation for being able to get account status

* feat: cloud integrations: add http handler and controller method for getting account status

* feat: cloud integrations: ensure unconnected accounts aren't included in list of connected accounts

* feat: cloud integrations: add test expectation for agent check in request

* feat: cloud integrations: agent check in API

* feat: cloud integrations: ensure polling for status after agent check in works

* feat: cloud integrations: ensure account included in connected account list after agent check in

* feat: cloud integrations: add API expectation for updating account config

* feat: cloud integrations: API for updating cloud account config

* feat: cloud integrations: expectation for agent receiving latest config after account config update

* feat: cloud integrations: expectation for disconnecting cloud accounts from UI

* feat: cloud integrations: API for disconnecting cloud accounts

* feat: cloud integrations: some cleanup

* feat: cloud integrations: some more cleanup

* feat: cloud integrations: repo: scope rows by cloud provider

* feat: testutils: refactor out helper for creating a test sqlite DB

* feat: cloud integrations: controller: add test validating regeneration of connection url

* feat: cloud integrations: controller: validations for agent check ins

* feat: cloud integrations: connected account response structure

* feat: cloud integrations: API response account structure

* feat: cloud integrations: some more cleanup

* feat: cloud integrations: remove cloudProviderAccountsRepository.GetById

* feat: cloud integrations: shouldn't be able to disconnect non-existent account

* feat: cloud integrations: validate agents can't check in to cloud account with 2 signoz ids

* feat: cloud integrations: ensure agents can't check in to cloud account with 2 signoz ids

* feat: cloud integrations: remove stray import of ee/model in cloudintegrations controller
2025-01-10 18:43:35 +05:30
Raj Kamal Singh
cd9f27ab08
Fix: QS: logs pipelines: better validation of pipelines being saved (#6652)
* chore: add test validating invalid field paths in pipeline operators are rejected

* chore: refactor posted pipelines validation to use a controller method

* fix: run a collector simulation to validate pipeline config being saved

* chore: minor cleanup
2024-12-18 10:42:14 +05:30
Srikanth Chekuri
e6766023dd
chore: use tag attributes v2 table (#6616) 2024-12-16 09:59:16 +00:00
Nityananda Gohain
67058b2a17
feat: trace v4 integration (#6226)
* feat: trace v4 inital commit

* fix: add remaining files

* fix: integrate with querier

* fix: get trace by id api updated

* fix: add servicename resource filter

* fix: tests

* fix: use correct prepQUery

* fix: services page

* fix: minor fixes to use the new table in api's and querier

* fix: add support for window based pagination

* feat: support for faster trace detail

* fix: searchTraces

* fix: attribute enrichment updated and issue in group by

* fix: issues in group by

* fix: enrichment using alias

* fix: test file added

* fix: tests

* fix: group by with filters

* fix: add subquery

* fix: trigger builde

* fix: update pagination logic and few ch column names

* fix: update qb

* fix: add tests

* feat: minor fixes

* fix: update pagination logic

* fix: update pagination logic

* fix: remove utils

* fix: remove unwanted API's

* fix: attribute and attribute values v2

* fix: autocomplete api's updated

* fix: tests fixed

* feat: minor fixes

* fix: update telemetry functions

* fix: dont use alias, use proper col names

* fix: move models to it's own file

* fix: minor fixes

* fix: address comments

* fix: add to serviceoverview function

* fix: add changes to overview function

* fix: address comments

* fix: remove printlines

---------

Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
2024-11-22 12:00:29 +05:30
Akira Hayashi
2d732ae4a9
chore: run all query-service tests (#6098) 2024-10-20 15:24:12 +00:00
Prashant Shahi
a6e492880d
fix(docker): use env prefix for boolean in collector config (#6199)
### Summary

Fixes for `expected type 'bool', got unconvertible type 'string'` errors.

Signed-off-by: Prashant Shahi <prashant@signoz.io>
2024-10-16 15:07:02 +05:30
Prashant Shahi
5588c7dd3f
revert(signoz): pin versions: Schema Migrator 0.102.10
Signed-off-by: Prashant Shahi <prashant@signoz.io>
2024-10-10 19:41:47 +05:30
Prashant Shahi
b70d50f2b3
chore: pin versions: SigNoz 0.56.0, OtelCollector 0.102.12, Alertmanager 0.23.7
Signed-off-by: Prashant Shahi <prashant@signoz.io>
2024-10-10 15:39:49 +05:30
Prashant Shahi
bc8891d2f8
Sync/post release v0.55 (#6092) 2024-09-30 18:02:56 +05:30
Raj Kamal Singh
fc8391c5aa
Feat: logs filter suggestions higher rank for special resource attribs like service.name and env etc (#6060)
* chore: upgrade signoz-otel-collector dependency to v0.102.10

* feat: first stab at ranking resource attribs higher

* chore: add test todo for validating resource attribs get ranked higher in logs filter suggestions

* chore: add test validating higher ranking for special resource attribs

* chore: some cleanup

* chore: some more cleanup
2024-09-26 15:26:14 +05:30
Raj Kamal Singh
8c891f0e87
Fix: cheaper query for fetching log attribute values for filter suggestions (#5989)
* chore: change query for fetching multiple log attribs to make sure it is always cheap

* chore: get filter suggestions tests passing
2024-09-17 15:49:14 +05:30
Nityananda Gohain
10ebd0cad6
feat: use new schema flag (#5930) 2024-09-12 10:58:07 +05:30
Raj Kamal Singh
7844522691
Chore: qs filter suggestions: example queries for multiple top attributes (#5703)
* chore: put together helper for fetching values for multiple attributes

* chore: poc: use helper for filter suggestions

* chore: add a working impl for getting attrib values for multiple attributes

* chore: start updating integration test to account for new approach for getting log attrib values

* chore: use a global zap logger in filter suggestion tests

* chore: fix attrib values clickhouse query expectation

* chore: only query values for actual attributes when generating example queries

* chore: update clickhouse-go-mock

* chore: cleanup: separate params for attributesLimit and examplesLimit for filter suggestions

* chore: some test cleanup

* chore: some more cleanup

* chore: some more cleanup

---------

Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
2024-09-09 10:12:36 +05:30
Nityananda Gohain
9f481aacff
feat: enable macro (#5760) 2024-08-26 11:57:18 +05:30
Raj Kamal Singh
eb146491f2
Feat: QS: query builder suggestions api v0 (#5634)
* chore: stash initial work with API signature

* chore: put together setup for integration testing filter suggestions

* feat: filter suggestions: suggest attribs using existing autocomplete logic

* chore: filter suggestions test: add expectation for example queries

* feat: filter suggestions: default suggestions when data yet to be received

* feat: finish plumbing basic example queries

* chore: add test for filter suggestions with an existing query

* feat: filter suggestions: don't suggest attribs already included in existing filter

* chore: generate example queries by including existing filter first

* chore: upgrade ClickHouse-go-mock

* chore: some cleanup of reader.GetQBFilterSuggestionsForLogs

* chore: some cleanup of filter suggestion tests

* chore: some cleanup to http handler and request parsing logic for filter suggestions

* chore: remove expectation that attrib suggestions won't contain attribs already used in filter
2024-08-08 09:27:41 +05:30
Prashant Shahi
fff61379fe
fix: mount root path in /hostfs for hostmetrics (#5534)
Signed-off-by: Prashant Shahi <prashant@signoz.io>
2024-08-02 16:06:21 +05:30
Prashant Shahi
e542d2ee09
chore(signoz): 📌 pin versions: SigNoz 0.49.1, SigNoz OtelCollector 0.102.2
Signed-off-by: Prashant Shahi <prashant@signoz.io>
2024-07-04 22:24:57 +05:30
Prashant Shahi
216ad36234
chore(signoz): 📌 pin versions: SigNoz 0.49.0, SigNoz OtelCollector 0.102.1
Signed-off-by: Prashant Shahi <prashant@signoz.io>
2024-07-03 16:53:11 +05:30
Prashant Shahi
537641000d
chore(signoz): 📌 pin versions: SigNoz 0.48.0, SigNoz OtelCollector 0.102.0
Signed-off-by: Prashant Shahi <prashant@signoz.io>
2024-06-19 21:14:45 +05:30
Prashant Shahi
4916cf5083 Merge branch 'main' into release/v0.48.x 2024-06-19 20:57:30 +05:30
Srikanth Chekuri
2f949d2738
chore: address large number of staticcheck issues (#5176)
* chore: address large number of staticcheck issues

* chore: fix tests

* chore: fix more issue

* chore: fix options
2024-06-11 20:10:38 +05:30
Prashant Shahi
4d375e7cc3
chore(signoz): 📌 pin versions: SigNoz 0.47.0, SigNoz OtelCollector 0.88.26
Signed-off-by: Prashant Shahi <prashant@signoz.io>
2024-06-05 19:12:50 +05:30
Prashant Shahi
52c8584e63
chore(signoz): 📌 pin versions: SigNoz 0.46.0, SigNoz OtelCollector 0.88.24
Signed-off-by: Prashant Shahi <prashant@signoz.io>
2024-05-22 21:45:38 +05:30
Prashant Shahi
9a908c3f76 Merge branch 'main' into release/v0.46.x 2024-05-22 21:42:34 +05:30
Raj Kamal Singh
5a778dcb18
Chore: integrations: populate updatedAt for integration dashboards (#5019)
* chore: add test for updatedAt value being populated in integration dashboards and get it passing

* chore: also populate createdAt, createBy and updateBy for instaled integration dashboards

* chore: update clickhouse integration config instructions
2024-05-16 21:44:46 +05:30
Prashant Shahi
47dcd994f0
chore(signoz): 📌 pin versions: SigNoz 0.45.0, SigNoz OtelCollector 0.88.22
Signed-off-by: Prashant Shahi <prashant@signoz.io>
2024-05-08 20:03:12 +05:30
Prashant Shahi
5ec2f17d09
chore: pin SigNoz OtelCollector 0.88.21 and update ClickHouse dsn (#4909)
* chore: 📌 pin versions: SigNoz OtelCollector 0.88.21
* chore(clickhouse): update dsn as per the new parsing logic

---------

Signed-off-by: Prashant Shahi <prashant@signoz.io>
2024-04-24 21:15:07 +05:30
Prashant Shahi
a45fb8ec0c
fix(clickhouse): update endpoint of the healthcheck in deployment files (#4908)
Signed-off-by: Prashant Shahi <prashant@signoz.io>
2024-04-24 19:06:05 +05:30
Prashant Shahi
cd63dd972d
chore(pre-release): 📌 pin versions: SigNoz 0.43.0, SigNoz OtelCollector 0.88.20
Signed-off-by: Prashant Shahi <prashant@signoz.io>
2024-04-10 23:55:39 +05:45
Ankit Nayan
cff20f88cd merging main 2024-03-30 18:30:46 +01:00
Raj Kamal Singh
990fc83269
Feat/integrations v0 mongo and nginx (#4763)
* feat: flesh out pre-requisites for collecting mongodb logs and metrics

* chore: remove stale pipelines in bundled integrations

* chore: clean up 'collect metrics' step for mongodb

* feat: add instructions for collecting and parsing mongodb logs

* feat: add metrics and logs attributes to mongodb data collected list

* feat: nginx logs collection instructions and some other cleanup

* feat: add list of parsed log attributes to data collected list for nginx

* chore: do not run pipeline population integration test if no built-in integration has a pipeline
2024-03-28 19:57:07 +05:30
Prashant Shahi
f8e8132b58
chore(signoz): 📌 pin versions: SigNoz 0.42.0, SigNoz OtelCollector 0.88.17
Signed-off-by: Prashant Shahi <prashant@signoz.io>
2024-03-27 23:23:15 +05:45
Prashant Shahi
c2556facc2 Merge branch 'main' into release/v0.42.0 2024-03-27 22:49:48 +05:45
Srikanth Chekuri
ae594061e9
chore: fix query-service logging (#4696) 2024-03-27 00:07:29 +05:30
Raj Kamal Singh
43f9830e8d
Feat: integrations v0 metrics connection status (#4715)
* chore: add test expectations for integration metrics connection status

* chore: reorg logs connection status calculation for parallelization

* chore: add interface for reader.GetMetricLastReceivedTsMillis

* chore: add plumbing for calculating integration metrics connection status

* chore: impl and test mocks for reader.GetMetricReceivedLatest

* chore: wrap things up and get test passing

* chore: some cleanup

* chore: some more cleanup

* chore: use prom metric names for integration connection test
2024-03-18 10:01:53 +05:30