35 Commits

Author SHA1 Message Date
shivanshu
aabf364cc6
feat: add partition level granularity 2024-08-27 18:27:44 +05:30
shivanshu
8d655bf419
chore: use MinAllowedStepInterval 2024-08-27 14:04:00 +05:30
shivanshu
90cb8ba9a1
chore: modify producer output 2024-08-26 20:35:08 +05:30
shivanshu
f508ee7521
chore: query, response update 2024-08-26 20:28:22 +05:30
shivanshu
413caad0d8
chore: cleanup 2024-08-26 20:28:22 +05:30
shivanshu
666f601ecd
feat: change to builder queries 2024-08-26 20:28:22 +05:30
shivanshu
5cdcbef00c
feat: add network latency for kafka 2024-08-26 20:28:22 +05:30
shivanshu
3b94dab3ce
chore: small patch to fix consumer_group check 2024-08-26 15:36:18 +05:30
shivanshu
c957c0f757
chore: addressing review comments 2024-08-05 18:14:40 +05:30
shivanshu
3ff0aa4b4b
chore: consumer group filtering 2024-08-05 18:09:58 +05:30
shivanshu
063c9adba6
chore: pr-reviews 2024-08-05 18:09:58 +05:30
shivanshu
5c3ce146fa
chore: add queue type 2024-08-05 18:09:58 +05:30
shivanshu
481bb6e8b8
feat: add consumer and producer APIs 2024-08-05 18:09:58 +05:30
Raj Kamal Singh
59121bd932
chore: nginx integration: add note about adjusting regex if using custom log format (#5615) 2024-07-31 17:52:51 +05:30
Srikanth Chekuri
6ee9705599
chore: bump SigNoz/signoz-otel-collector and SigNoz/prometheus (#5294) 2024-06-20 18:33:45 +05:30
Srikanth Chekuri
2f949d2738
chore: address large number of staticcheck issues (#5176)
* chore: address large number of staticcheck issues

* chore: fix tests

* chore: fix more issue

* chore: fix options
2024-06-11 20:10:38 +05:30
Srikanth Chekuri
592073a564
chore: add AWS RDS and ElastiCache integration (#5071) 2024-06-02 14:42:05 +05:30
Raj Kamal Singh
5a778dcb18
Chore: integrations: populate updatedAt for integration dashboards (#5019)
* chore: add test for updatedAt value being populated in integration dashboards and get it passing

* chore: also populate createdAt, createBy and updateBy for instaled integration dashboards

* chore: update clickhouse integration config instructions
2024-05-16 21:44:46 +05:30
Raj Kamal Singh
3efd9801a1
Chore: restrict logs connection test for integrations to use log attributes for identifying logs (#4977)
* chore: change logs connection test spec to be based on an attrib value

* chore: disallow unknown fields while unmarshalling JSON for an integration

* chore: add description field to collected metric spec

* chore: update logs connection test for builtin integrations

* chore: update logic for calculating logs connection status
2024-05-15 14:36:52 +05:30
Raj Kamal Singh
e6e0a59f5f
Feat: integrations: clickhouse (#4879)
* chore: get built-in clickhouse integration started

* chore: update config pre-requisites for clickhouse integration

* chore: add details of metrics data collected for clickhouse integration

* chore: clickhouse integration: move list of data-collected to its own file

* chore: clickhouse integration: get overview dashboard started

* chore: start with logs collection instructions for clickhouse

* chore: regex parsing for clickhouse text logs

* chore: timestamp parsing for clickhouse logs

* chore: severity parsing for clickhouse logs

* chore: clickhouse logs parsing: move parsed message to body if available

* chore: update pre-reqs for collecting from system.query_log table

* feat: add instructions for collecting from system.query_log table

* feat: add logs attribs collected

* chore: some cleanup of clickhouse overview dashboard

* feat: finish up with clickhouse overview dashboard for clickhouse integration
2024-04-26 09:45:57 +05:30
Raj Kamal Singh
39e0ef68ca
chore: integration instructions: add typical log file locations on macOS (#4779) 2024-04-01 12:06:08 +05:30
Raj Kamal Singh
990fc83269
Feat/integrations v0 mongo and nginx (#4763)
* feat: flesh out pre-requisites for collecting mongodb logs and metrics

* chore: remove stale pipelines in bundled integrations

* chore: clean up 'collect metrics' step for mongodb

* feat: add instructions for collecting and parsing mongodb logs

* feat: add metrics and logs attributes to mongodb data collected list

* feat: nginx logs collection instructions and some other cleanup

* feat: add list of parsed log attributes to data collected list for nginx

* chore: do not run pipeline population integration test if no built-in integration has a pipeline
2024-03-28 19:57:07 +05:30
Raj Kamal Singh
0ac9f6f663
Feat: QS: redis integration v0: instructions for collecting and parsing logs (#4753)
* chore: minor cleanups to postgres integration instructions

* chore: update instructions for connecting redis integration

* feat: add instructions for collecting redis logs

* chore: flesh out prerequisites for connecting redis integration

* chore: add list of metrics collected for redis
2024-03-27 20:03:27 +05:30
Raj Kamal Singh
f24135f5b0
Feat: QS: postgres integration: instructions for collecting and parsing logs (#4738)
* chore: offer metrics config instructions for signoz cloud only

* chore: some more cleanups

* chore: get log collection instructions started

* feat: flesh out log collection otel config for postgres

* chore: some cleanup

* chore: some more cleanup

* chore: some more cleanup
2024-03-23 11:39:28 +05:30
Raj Kamal Singh
f4e94c0ad1
chore: update postgres config instructions to work for both signoz cloud and self-hosted (#4718) 2024-03-19 12:28:22 +05:30
Raj Kamal Singh
6f3183823f
Feat: postgres integration v0 (#4704)
* chore: update annotations for pre blocks in configuration instructions

* chore: update list of collected metrics for postgres integration

* chore: change non-string units to string in metrics collected list

* chore: some cleanups for postgres config instructions

* chore: some cleanup to metrics connection status resource labels

* chore: remove stub pipeline in postgres integration - no interesting log parsing to be done
2024-03-18 18:20:12 +05:30
Raj Kamal Singh
43f9830e8d
Feat: integrations v0 metrics connection status (#4715)
* chore: add test expectations for integration metrics connection status

* chore: reorg logs connection status calculation for parallelization

* chore: add interface for reader.GetMetricLastReceivedTsMillis

* chore: add plumbing for calculating integration metrics connection status

* chore: impl and test mocks for reader.GetMetricReceivedLatest

* chore: wrap things up and get test passing

* chore: some cleanup

* chore: some more cleanup

* chore: use prom metric names for integration connection test
2024-03-18 10:01:53 +05:30
Raj Kamal Singh
5a2d729ba9
Feat: qs autopopulate installed integration dashboards (#4680)
* chore: add test for dashboards for installed integrations

* feat: include dashboards for installed integrations in API response

* chore: add test expectation for getting installed integration dashboard by id

* feat: add support for retrieving installed integration dashboards by id

* chore: add dashboard id validation for integrations
2024-03-11 20:06:59 +05:30
Raj Kamal Singh
9ace374855
Feat: QS: Log Pipelines for installed integrations (#4674)
* chore: refactor: inject sqlx.DB into opamp.initDB instead of DB file name

* chore: reorganize test utils a little

* chore: add test validating pipelines for installed integrations show up in pipelines list

* chore: get basic integration pipelines testcase passing

* chore: reconcile experimental changes with latest state of develop

* chore: add integration test for reordering of pipelines

* chore: marker for integration pipelines using Id

* chore: hookup propagation of installed integration pipelines by opamp

* chore: add util for mapping slices

* chore: add support for reordering integration pipelines

* chore: exclude user saved integration pipelines if no longer installed

* chore: flesh out rest of intgeration pipelines scenarios

* chore: handle scenario when an integration is installed before any pipelines exist

* chore: notify agentConf of update after uninstalling an integration

* chore: some minor cleanup

* chore: some more cleanup

* chore: update ee server for changed controllers

* chore: some more cleanup

* chore: change builtin integration id prefix to avoid using colons that break yaml

* chore: update builtin integration id in test
2024-03-11 14:15:11 +05:30
Vikrant Gupta
a4d5774ae3
fix: update the md file for integrations config (#4677) 2024-03-11 11:48:36 +05:30
Raj Kamal Singh
a295bf2fb6
Feat: QS: structure for built in integrations (#4655)
* feat: get builtin integrations started with nginx

* feat: get started with embedding and parsing of builtin integrations

* chore: add icons for nginx and redis integrations

* chore: stash current state of work

* chore: remove all yaml annotations since moved to JSON assets for bundled integrations

* chore: add file uri hydration in integration spec

* chore: refactor file uri hydration logic

* chore: add support for referencing JSON files with file uri

* chore: bring in initial integration assets

* chore: hookup builtin integrations and get all tests passing

* chore: update icons for postgres and mongo and some cleanup

* chore: some more cleanup

---------

Co-authored-by: Raj Singh <raj@Rajs-MacBook-Pro.local>
2024-03-07 19:26:20 +05:30
Raj Kamal Singh
7136ecc2fe
chore: accept connection test lookback seconds as request param (#4650)
Also removes connectionStatus from integration details.
2024-03-06 11:06:04 +05:30
Raj Kamal Singh
ab5285dee6
Feat: qs api integration connection status (#4628)
* chore: add integration attribs for connection tests and status

* chore: add connection status to integration details response

* chore: update integration lifecycle test to check for connection status too

* feat: add GetIntegrationConnectionTests to integrations manager and controller

* chore: add tests for querying integration connection status

* feat: add http API support for integration connection status

* chore: some cleanups

* chore: use PostableRule for integration alerts

* chore: some more cleanup
2024-03-05 15:23:56 +05:30
Raj Kamal Singh
0870030d1c
Feat: qs integrations http api (#4622)
* chore: add http api test for signoz integrations

* chore: add controller for integrations

* chore: add http API handlers for integrations API

* chore: hook up integrations API in new servers

* chore: add remaining fields in Integration DTO

---------

Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
2024-03-02 10:11:51 +05:30
Raj Kamal Singh
ddaa464d97
feat: QS package for integrations (#4578)
* chore: bring in latest state of QS api work for integrations

* chore: integrations v0 qs API: refactor installed integration struct

* chore: finish up with integration lifecycle tests

* chore: some cleanup

* chore: some more cleanup

* chore: some more cleanup

* chore: some more cleanup

* chore: some more cleanup

---------

Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
2024-02-28 09:54:50 +05:30