* feat(sqlmigration): update the alertmanager tables
* feat(sqlmigration): update the alertmanager tables
* feat(sqlmigration): make the preference package multi tenant
* feat(preference): address nit pick comments
* feat(preference): added the cascade delete for preferences
* feat(sqlmigration): update apdex and TTL status tables (#7481)
* feat(sqlmigration): update the apdex and ttl tables
* feat(sqlmigration): register the new migration and rename table
* feat(sqlmigration): fix the ttl queries
* feat(sqlmigration): update the TTL and apdex tables
* feat(sqlmigration): update the TTL and apdex tables
* fix: support multitenancy in dashboards
* fix: support multitenancy in saved views
* fix: move migrations to provider file
* fix: remove getUserFromClaims and use new errors
* fix: remove getUserFromClaims and use new errors
* fix: use new errors in dashboards.go
* Update ee/query-service/app/api/dashboard.go
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* fix: minor changes
---------
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* fix: support multitenancy in org
* fix: register and login working now
* fix: changes to migration
* fix: migrations run both on sqlite and postgres
* fix: remove user flags from fe and be
* fix: remove ingestion keys from update
* fix: multitenancy support for apdex settings
* fix: render ts for users correctly
* fix: fix migration to run for new tenants
* fix: clean up migrations
* fix: address comments
* Update pkg/sqlmigration/013_update_organization.go
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* fix: fix build
* fix: force invites with org id
* Update pkg/query-service/auth/auth.go
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* fix: address comments
* fix: address comments
* fix: provier with their own dialect
* fix: update dialects
* fix: remove unwanted change
* Update pkg/query-service/app/http_handler.go
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* fix: different files for types
---------
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* chore: include CWAgent namespace for ec2 metrics
* chore: update aws integration service metrics connection status queried labels
* chore: fix breaking test
* chore: get started on dashboards plumbing for AWS integration services
* feat: cloud integrations: include cloud integrations dashboards in dashboards list
* feat: cloud integrations: get cloud integration dashboard by id
* feat: dashboard url in cloud integrations svc detail
* feat: ec2 overview dashboard
* chore: add ec2 overview dashboard image
* chore: finish up with v0 definitions for EC2 and RDS
* chore: some cleanup
* chore: fix broken test
* chore: add connection url composition
---------
Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
* chore: cloudintegrations: rename rds def
* feat: cloudintegrations: shape of agent check in response for v0 release
* chore: cloudintegrations: add validation for response expected after agent check in
* chore: cloudintegrations: accumulate teletry collection strategies for enabled services
* chore: cloudintegrations: use map struct to parse from map to struct with json tags
* chore: cloudintegrations: telemetry collection strategy for services
* chore: cloudintegrations: wrap up test for agent check in resp
* chore: some cleanup
* chore: some cleanup
* chore: some minor renaming
* chore: address review comment
* chore: some cleanup
---------
Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
* fix: inital changes for telemetry store
* fix: add tests and use proper config for conn
* fix: add telemetry store test
* fix: add backward compatibility for old variables and update example conf
* fix: move wrapper to telemetry store
* fix: no need to pass query for settings
* fix: remove redundant config for ch conn
* fix: use clickhouse dsn instead
* fix: update example config
* fix: update backward compatibility code
* fix: use hooks in telemetrystore
* fix: address minor comments
---------
Co-authored-by: Vibhu Pandey <vibhupandey28@gmail.com>
* feat: cloud service integrations: get model and repo interface started
* feat: cloud service integrations: flesh out more of cloud services model
* feat: cloud integrations: reorganize things a little
* feat: cloud integrations: get svc controller started
* feat: cloud integrations: add stubs for EC2 and RDS postgres services
* feat: cloud integrations: add validation for listing and getting available svcs and some cleanup
* feat: cloud integrations: refactor helpers in existing integrations code for reuse
* feat: cloud integrations: parsing of cloud service definitions
* feat: cloud integrations: impl for getCloudProviderService
* feat: cloud integrations: some reorganization
* feat: cloud integrations: some more cleanup
* feat: cloud integrations: add validation for listing available cloud provider services
* feat: cloud integrations: API endpoint for listing available cloud provider services
* feat: cloud integrations: add validation for getting details of a particular service
* feat: cloud integrations: API endpoint for getting details of a service
* feat: cloud integrations: add controller validation for configuring cloud services
* feat: cloud integrations: get serviceConfigRepo started
* feat: cloud integrations: service config in service list summaries when queried for cloud account id
* feat: cloud integrations: only a supported service for a connected cloud account can be configured
* feat: cloud integrations: add validation for configuring services via the API
* feat: cloud integrations: API for configuring services
* feat: cloud integrations: some cleanup
* feat: cloud integrations: fix broken test
---------
Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
* feat: init app/cloud_integrations
* feat: get API test started for cloudintegrations account lifecycle
* feat: cloudintegrations: get controller started
* feat: cloud integrations: add cloudintegrations.Controller to APIHandler and servers
* feat: cloud integrations: get routes started
* feat: cloud integrations: get accounts table schema started
* feat: cloud integrations: get cloudProviderAccountsSQLRepository started
* feat: cloud integrations: cloudProviderAccountsSQLRepository.listAccounts
* feat: cloud integrations: http handler and controller plumbing for /generate-connection-url
* feat: cloud integrations: cloudProviderAccountsSQLRepository.upsert
* feat: cloud integrations: finish up with /generate-connection-url
* feat: cloud integrations: add cloudProviderAccountsRepository.get
* feat: cloud integrations: add API test expectation for being able to get account status
* feat: cloud integrations: add http handler and controller method for getting account status
* feat: cloud integrations: ensure unconnected accounts aren't included in list of connected accounts
* feat: cloud integrations: add test expectation for agent check in request
* feat: cloud integrations: agent check in API
* feat: cloud integrations: ensure polling for status after agent check in works
* feat: cloud integrations: ensure account included in connected account list after agent check in
* feat: cloud integrations: add API expectation for updating account config
* feat: cloud integrations: API for updating cloud account config
* feat: cloud integrations: expectation for agent receiving latest config after account config update
* feat: cloud integrations: expectation for disconnecting cloud accounts from UI
* feat: cloud integrations: API for disconnecting cloud accounts
* feat: cloud integrations: some cleanup
* feat: cloud integrations: some more cleanup
* feat: cloud integrations: repo: scope rows by cloud provider
* feat: testutils: refactor out helper for creating a test sqlite DB
* feat: cloud integrations: controller: add test validating regeneration of connection url
* feat: cloud integrations: controller: validations for agent check ins
* feat: cloud integrations: connected account response structure
* feat: cloud integrations: API response account structure
* feat: cloud integrations: some more cleanup
* feat: cloud integrations: remove cloudProviderAccountsRepository.GetById
* feat: cloud integrations: shouldn't be able to disconnect non-existent account
* feat: cloud integrations: validate agents can't check in to cloud account with 2 signoz ids
* feat: cloud integrations: ensure agents can't check in to cloud account with 2 signoz ids
* feat: cloud integrations: remove stray import of ee/model in cloudintegrations controller
* chore: add test validating invalid field paths in pipeline operators are rejected
* chore: refactor posted pipelines validation to use a controller method
* fix: run a collector simulation to validate pipeline config being saved
* chore: minor cleanup
* feat: trace v4 inital commit
* fix: add remaining files
* fix: integrate with querier
* fix: get trace by id api updated
* fix: add servicename resource filter
* fix: tests
* fix: use correct prepQUery
* fix: services page
* fix: minor fixes to use the new table in api's and querier
* fix: add support for window based pagination
* feat: support for faster trace detail
* fix: searchTraces
* fix: attribute enrichment updated and issue in group by
* fix: issues in group by
* fix: enrichment using alias
* fix: test file added
* fix: tests
* fix: group by with filters
* fix: add subquery
* fix: trigger builde
* fix: update pagination logic and few ch column names
* fix: update qb
* fix: add tests
* feat: minor fixes
* fix: update pagination logic
* fix: update pagination logic
* fix: remove utils
* fix: remove unwanted API's
* fix: attribute and attribute values v2
* fix: autocomplete api's updated
* fix: tests fixed
* feat: minor fixes
* fix: update telemetry functions
* fix: dont use alias, use proper col names
* fix: move models to it's own file
* fix: minor fixes
* fix: address comments
* fix: add to serviceoverview function
* fix: add changes to overview function
* fix: address comments
* fix: remove printlines
---------
Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
* chore: upgrade signoz-otel-collector dependency to v0.102.10
* feat: first stab at ranking resource attribs higher
* chore: add test todo for validating resource attribs get ranked higher in logs filter suggestions
* chore: add test validating higher ranking for special resource attribs
* chore: some cleanup
* chore: some more cleanup
* chore: put together helper for fetching values for multiple attributes
* chore: poc: use helper for filter suggestions
* chore: add a working impl for getting attrib values for multiple attributes
* chore: start updating integration test to account for new approach for getting log attrib values
* chore: use a global zap logger in filter suggestion tests
* chore: fix attrib values clickhouse query expectation
* chore: only query values for actual attributes when generating example queries
* chore: update clickhouse-go-mock
* chore: cleanup: separate params for attributesLimit and examplesLimit for filter suggestions
* chore: some test cleanup
* chore: some more cleanup
* chore: some more cleanup
---------
Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
* chore: stash initial work with API signature
* chore: put together setup for integration testing filter suggestions
* feat: filter suggestions: suggest attribs using existing autocomplete logic
* chore: filter suggestions test: add expectation for example queries
* feat: filter suggestions: default suggestions when data yet to be received
* feat: finish plumbing basic example queries
* chore: add test for filter suggestions with an existing query
* feat: filter suggestions: don't suggest attribs already included in existing filter
* chore: generate example queries by including existing filter first
* chore: upgrade ClickHouse-go-mock
* chore: some cleanup of reader.GetQBFilterSuggestionsForLogs
* chore: some cleanup of filter suggestion tests
* chore: some cleanup to http handler and request parsing logic for filter suggestions
* chore: remove expectation that attrib suggestions won't contain attribs already used in filter
* chore: add test for updatedAt value being populated in integration dashboards and get it passing
* chore: also populate createdAt, createBy and updateBy for instaled integration dashboards
* chore: update clickhouse integration config instructions
* feat: flesh out pre-requisites for collecting mongodb logs and metrics
* chore: remove stale pipelines in bundled integrations
* chore: clean up 'collect metrics' step for mongodb
* feat: add instructions for collecting and parsing mongodb logs
* feat: add metrics and logs attributes to mongodb data collected list
* feat: nginx logs collection instructions and some other cleanup
* feat: add list of parsed log attributes to data collected list for nginx
* chore: do not run pipeline population integration test if no built-in integration has a pipeline
* chore: add test expectations for integration metrics connection status
* chore: reorg logs connection status calculation for parallelization
* chore: add interface for reader.GetMetricLastReceivedTsMillis
* chore: add plumbing for calculating integration metrics connection status
* chore: impl and test mocks for reader.GetMetricReceivedLatest
* chore: wrap things up and get test passing
* chore: some cleanup
* chore: some more cleanup
* chore: use prom metric names for integration connection test
* chore: add test for dashboards for installed integrations
* feat: include dashboards for installed integrations in API response
* chore: add test expectation for getting installed integration dashboard by id
* feat: add support for retrieving installed integration dashboards by id
* chore: add dashboard id validation for integrations
* chore: refactor: inject sqlx.DB into opamp.initDB instead of DB file name
* chore: reorganize test utils a little
* chore: add test validating pipelines for installed integrations show up in pipelines list
* chore: get basic integration pipelines testcase passing
* chore: reconcile experimental changes with latest state of develop
* chore: add integration test for reordering of pipelines
* chore: marker for integration pipelines using Id
* chore: hookup propagation of installed integration pipelines by opamp
* chore: add util for mapping slices
* chore: add support for reordering integration pipelines
* chore: exclude user saved integration pipelines if no longer installed
* chore: flesh out rest of intgeration pipelines scenarios
* chore: handle scenario when an integration is installed before any pipelines exist
* chore: notify agentConf of update after uninstalling an integration
* chore: some minor cleanup
* chore: some more cleanup
* chore: update ee server for changed controllers
* chore: some more cleanup
* chore: change builtin integration id prefix to avoid using colons that break yaml
* chore: update builtin integration id in test
* chore: add integration attribs for connection tests and status
* chore: add connection status to integration details response
* chore: update integration lifecycle test to check for connection status too
* feat: add GetIntegrationConnectionTests to integrations manager and controller
* chore: add tests for querying integration connection status
* feat: add http API support for integration connection status
* chore: some cleanups
* chore: use PostableRule for integration alerts
* chore: some more cleanup
* chore: add http api test for signoz integrations
* chore: add controller for integrations
* chore: add http API handlers for integrations API
* chore: hook up integrations API in new servers
* chore: add remaining fields in Integration DTO
---------
Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
* fix: improve user telemetry
- move GetEmailFromJwt to common function
- update AttachJwtToContext() to use standard way of attaching value to context
- update userEmail in every possible sendEvent call
- send groupId in sendEvent call for SaaS operator analytics
* chore: added DEFAULT_CLOUD_EMAIL const
* chore: add AttachJwtToContext to analytics middleware
* test: added AttachJwtToContext to logs pipelines
* chore: add integration test for log pipelines being recommended to agents on connection
* chore: agentConf.Manager as AgentConfigProvider
* feat: logparsingpipelines as an AgentFeature
* chore: some cleanup
* feat: update agentConf init in ee query-service
* chore: more cleanup and move opamp/logspipeline -> logparsingpipeline/collector_config.go
* chore: some more cleanup
* chore: set agent.RemoteConfig after broadcasting conf to all agents
* chore: add test scenario for post remote conf application
* feat: add interface for opamp.AgentConfigProvider
* feat: add iface and plumbing for generating recommended conf in opamp/agent
* feat: get opamp server config provider tests started
* chore: add test scenario for agent connection without a config recommendation
* chore: add test scenario for agent connection with a config recommendation
* chore: add test for validating config deployment status gets reported
* chore: add test for rolling out latest config recommendations when config changes
* chore: wrap up opamp server lifecycle tests
* chore: some tests cleanup
* chore: get all tests passing
* chore: update opamp server init logic in ee query service
* chore: some cleanup
* chore: some final cleanup
* chore: use v3.Filterset as pipeline filters in logparsing pipelines integration tests
* chore: get logparsing integration tests passing with filterset based pipeline
* chore: get all other breaking tests passing
* chore: move models.logparsingpipeline to logparsingpipeline.model
* chore: implement Valuer and Scanner interfaces for v3.FilterSet
* chore: add integration test for log parsing pipelines validation
* chore: add helpers for creating unauthorized, unavailable & not found api errors
* chore: return *model.APIError from logpipeline and agentConf functions
* chore: some cleanup
* chore: some more cleanup
* chore: one more round of cleanups
* fix: avoid pipeline builder panic if first op is disabled
* chore: use pipeline builder test to trigger getOperator panic
---------
Co-authored-by: Nityananda Gohain <nityanandagohain@gmail.com>
