diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 2781d2a0c6..6ea316135a 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -4,4 +4,4 @@ * @ankitnayan /frontend/ @palashgdev @pranshuchittora /deploy/ @prashant-shahi -/pkg/query-service/ @srikanthccv @makeavish @nityanandagohain +/pkg/query-service/ @srikanthccv diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 54ff60451b..686bcdba58 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,122 +1,331 @@ -# How to Contribute +# Contributing Guidelines -There are primarily 2 areas in which you can contribute in SigNoz +## Welcome to SigNoz Contributing section πŸŽ‰ -- Frontend ( written in Typescript, React) -- Backend - ( Query Service - written in Go) +Hi there! We're thrilled that you'd like to contribute to this project, thank you for your interest. Whether it's a bug report, new feature, correction, or additional documentation, we greatly value feedback and contributions from our community. -Depending upon your area of expertise & interest, you can chose one or more to contribute. Below are detailed instructions to contribute in each area +Please read through this document before submitting any issues or pull requests to ensure we have all the necessary information to effectively respond to your bug report or contribution. -> Please note: If you want to work on an issue, please ask the maintainers to assign the issue to you before starting work on it. This would help us understand who is working on an issue and prevent duplicate work. πŸ™πŸ» +- We accept contributions made to the [SigNoz `develop` branch]() +- Find all SigNoz Docker Hub images here + - [signoz/frontend](https://hub.docker.com/r/signoz/frontend) + - [signoz/query-service](https://hub.docker.com/r/signoz/query-service) + - [signoz/otelcontribcol](https://hub.docker.com/r/signoz/otelcontribcol) -> If you just raise a PR, without the corresponding issue being assigned to you - it may not be accepted. +## Finding contributions to work on πŸ’¬ -# Develop Frontend +Looking at the existing issues is a great way to find something to contribute on. +Also, have a look at these [good first issues label](https://github.com/SigNoz/signoz/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) to start with. -Need to update [https://github.com/SigNoz/signoz/tree/main/frontend](https://github.com/SigNoz/signoz/tree/main/frontend) -### Contribute to Frontend with Docker installation of SigNoz +## Sections: +- [General Instructions](#1-general-instructions-) + - [For Creating Issue(s)](#11-for-creating-issues) + - [For Pull Requests(s)](#12-for-pull-requests) +- [How to Contribute](#2-how-to-contribute-%EF%B8%8F) +- [Develop Frontend](#3-develop-frontend-) + - [Contribute to Frontend with Docker installation of SigNoz](#31-contribute-to-frontend-with-docker-installation-of-signoz) + - [Contribute to Frontend without installing SigNoz backend](#32-contribute-to-frontend-without-installing-signoz-backend) +- [Contribute to Backend (Query-Service)](#4-contribute-to-backend-query-service-) + - [To run ClickHouse setup](#41-to-run-clickhouse-setup-recommended-for-local-development) +- [Contribute to SigNoz Helm Chart](#5-contribute-to-signoz-helm-chart-) + - [To run helm chart for local development](#51-to-run-helm-chart-for-local-development) +- [Other Ways to Contribute](#other-ways-to-contribute) -- `git clone https://github.com/SigNoz/signoz.git && cd signoz` -- comment out frontend service section at `deploy/docker/clickhouse-setup/docker-compose.yaml#L62` -- run `cd deploy` to move to deploy directory -- Install signoz locally without the frontend - - Add below configuration to query-service section at `docker/clickhouse-setup/docker-compose.yaml#L38` +# 1. General Instructions πŸ“ - ```docker +## 1.1 For Creating Issue(s) +Before making any significant changes and before filing a new issue, please check [existing open](https://github.com/SigNoz/signoz/issues?q=is%3Aopen+is%3Aissue), or [recently closed](https://github.com/SigNoz/signoz/issues?q=is%3Aissue+is%3Aclosed) issues to make sure somebody else hasn't already reported the issue. Please try to include as much information as you can. + +**Issue Types** - [Bug Report](https://github.com/SigNoz/signoz/issues/new?assignees=&labels=&template=bug_report.md&title=) | [Feature Request](https://github.com/SigNoz/signoz/issues/new?assignees=&labels=&template=feature_request.md&title=) | [Performance Issue Report](https://github.com/SigNoz/signoz/issues/new?assignees=&labels=&template=performance-issue-report.md&title=) | [Report a Security Vulnerability](https://github.com/SigNoz/signoz/security/policy) + +#### Details like these are incredibly useful: + +- **Requirement** - what kind of use case are you trying to solve? +- **Proposal** - what do you suggest to solve the problem or improve the existing + situation? +- Any open questions to address❓ + +#### If you are reporting a bug, details like these are incredibly useful: + +- A reproducible test case or series of steps. +- The version of our code being used. +- Any modifications you've made relevant to the bug🐞. +- Anything unusual about your environment or deployment. + +Discussing your proposed changes ahead of time will make the contribution +process smooth for everyone πŸ™Œ. + + **[`^top^`](#)** + +
+ +## 1.2 For Pull Request(s) + +Contributions via pull requests are much appreciated. Once the approach is agreed upon βœ…, make your changes and open a Pull Request(s). +Before sending us a pull request, please ensure that, + +- Fork the SigNoz repo on GitHub, clone it on your machine. +- Create a branch with your changes. +- You are working against the latest source on the `develop` branch. +- Modify the source; please focus only on the specific change you are contributing. +- Ensure local tests pass. +- Commit to your fork using clear commit messages. +- Send us a pull request, answering any default questions in the pull request interface. +- Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation +- Once you've pushed your commits to GitHub, make sure that your branch can be auto-merged (there are no merge conflicts). If not, on your computer, merge main into your branch, resolve any merge conflicts, make sure everything still runs correctly and passes all the tests, and then push up those changes. +- Once the change has been approved and merged, we will inform you in a comment. + + +GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and +[creating a pull request](https://help.github.com/articles/creating-a-pull-request/). + +**Note:** Unless your change is small, **please** consider submitting different Pull Rrequest(s): + +* 1️⃣ First PR should include the overall structure of the new component: + * Readme, configuration, interfaces or base classes, etc... + * This PR is usually trivial to review, so the size limit does not apply to + it. +* 2️⃣ Second PR should include the concrete implementation of the component. If the + size of this PR is larger than the recommended size, consider **splitting** βš”οΈ it into + multiple PRs. +* If there are multiple sub-component then ideally each one should be implemented as + a **separate** pull request. +* Last PR should include changes to **any user-facing documentation.** And should include + end-to-end tests if applicable. The component must be enabled + only after sufficient testing, and there is enough confidence in the + stability and quality of the component. + + +You can always reach out to `ankit@signoz.io` to understand more about the repo and product. We are very responsive over email and [SLACK](https://signoz.io/slack). + +### Pointers: +- If you find any **bugs** β†’ please create an [**issue.**](https://github.com/SigNoz/signoz/issues/new?assignees=&labels=&template=bug_report.md&title=) +- If you find anything **missing** in documentation β†’ you can create an issue with the label **`documentation`**. +- If you want to build any **new feature** β†’ please create an [issue with the label **`enhancement`**.](https://github.com/SigNoz/signoz/issues/new?assignees=&labels=&template=feature_request.md&title=) +- If you want to **discuss** something about the product, start a new [**discussion**.](https://github.com/SigNoz/signoz/discussions) + +
+ +### Conventions to follow when submitting Commits and Pull Request(s). + +We try to follow [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/), more specifically the commits and PRs **should have type specifiers** prefixed in the name. [This](https://www.conventionalcommits.org/en/v1.0.0/#specification) should give you a better idea. + +e.g. If you are submitting a fix for an issue in frontend, the PR name should be prefixed with **`fix(FE):`** + +- Follow [GitHub Flow](https://guides.github.com/introduction/flow/) guidelines for your contribution flows. + +- Feel free to ping us on [`#contributing`](https://signoz-community.slack.com/archives/C01LWQ8KS7M) or [`#contributing-frontend`](https://signoz-community.slack.com/archives/C027134DM8B) on our slack community if you need any help on this :) + + **[`^top^`](#)** + +
+ +# 2. How to Contribute πŸ™‹πŸ»β€β™‚οΈ + +#### There are primarily 2 areas in which you can contribute to SigNoz + +- [**Frontend**](#3-develop-frontend-) (Written in Typescript, React) +- [**Backend**](#4-contribute-to-backend-query-service-) (Query Service, written in Go) + +Depending upon your area of expertise & interest, you can choose one or more to contribute. Below are detailed instructions to contribute in each area. + +**Please note:** If you want to work on an issue, please ask the maintainers to assign the issue to you before starting work on it. This would help us understand who is working on an issue and prevent duplicate work. πŸ™πŸ» + +⚠️ If you just raise a PR, without the corresponding issue being assigned to you - it may not be accepted. + + **[`^top^`](#)** + +
+ +# 3. Develop Frontend 🌚 + +**Need to Update: [https://github.com/SigNoz/signoz/tree/develop/frontend](https://github.com/SigNoz/signoz/tree/develop/frontend)** + +Also, have a look at [Frontend README.md](https://github.com/SigNoz/signoz/blob/develop/frontend/README.md) sections for more info on how to setup SigNoz frontend locally (with and without Docker). + +## 3.1 Contribute to Frontend with Docker installation of SigNoz + +- Clone the SigNoz repository and cd into signoz directory, + ``` + git clone https://github.com/SigNoz/signoz.git && cd signoz + ``` +- Comment out `frontend` service section at [`deploy/docker/clickhouse-setup/docker-compose.yaml#L68`](https://github.com/SigNoz/signoz/blob/develop/deploy/docker/clickhouse-setup/docker-compose.yaml#L68) + +![develop-frontend](https://user-images.githubusercontent.com/52788043/179009217-6692616b-17dc-4d27-b587-9d007098d739.jpeg) + + +- run `cd deploy` to move to deploy directory, +- Install signoz locally **without** the frontend, + - Add / Uncomment the below configuration to query-service section at [`deploy/docker/clickhouse-setup/docker-compose.yaml#L47`](https://github.com/SigNoz/signoz/blob/develop/deploy/docker/clickhouse-setup/docker-compose.yaml#L47) + ``` ports: - "8080:8080" ``` - - If you are using x86_64 processors (All Intel/AMD processors) run `sudo docker-compose -f docker/clickhouse-setup/docker-compose.yaml up -d` - - If you are on arm64 processors (Apple M1 Macbooks) run `sudo docker-compose -f docker/clickhouse-setup/docker-compose.arm.yaml up -d` -- `cd ../frontend` and change baseURL to `http://localhost:8080` in file `src/constants/env.ts` -- `yarn install` -- `yarn dev` +query service + + - Next run, + ``` + sudo docker-compose -f docker/clickhouse-setup/docker-compose.yaml up -d + ``` +- `cd ../frontend` and change baseURL in file [`frontend/src/constants/env.ts#L2`](https://github.com/SigNoz/signoz/blob/develop/frontend/src/constants/env.ts#L2) and for that, you need to create a `.env` file in the `frontend` directory with the following environment variable (`FRONTEND_API_ENDPOINT`) matching your configuration. -> Notes for Maintainers/Contributors who will change Line Numbers of Frontend & Query-Section. Please Update Line Numbers in `./scripts/commentLinesForSetup.sh` + If you have backend api exposed via frontend nginx: + ``` + FRONTEND_API_ENDPOINT=http://localhost:3301 + ``` + If not: + ``` + FRONTEND_API_ENDPOINT=http://localhost:8080 + ``` -### Contribute to Frontend without installing SigNoz backend +- Next, + ``` + yarn install + yarn dev + ``` -If you don't want to install SigNoz backend just for doing frontend development, we can provide you with test environments which you can use as the backend. Please ping us in #contributing channel in our [slack community](https://signoz.io/slack) and we will DM you with `` +### Important Notes: +The Maintainers / Contributors who will change Line Numbers of `Frontend` & `Query-Section`, please update line numbers in [`/.scripts/commentLinesForSetup.sh`](https://github.com/SigNoz/signoz/blob/develop/.scripts/commentLinesForSetup.sh) -- `git clone https://github.com/SigNoz/signoz.git && cd signoz/frontend` -- Create a file `.env` with `FRONTEND_API_ENDPOINT=` -- `yarn install` -- `yarn dev` + **[`^top^`](#)** -**_Frontend should now be accessible at `http://localhost:3301/application`_** +## 3.2 Contribute to Frontend without installing SigNoz backend -# Contribute to Query-Service +If you don't want to install the SigNoz backend just for doing frontend development, we can provide you with test environments that you can use as the backend. -Need to update [https://github.com/SigNoz/signoz/tree/main/pkg/query-service](https://github.com/SigNoz/signoz/tree/main/pkg/query-service) +- Clone the SigNoz repository and cd into signoz/frontend directory, + ``` + git clone https://github.com/SigNoz/signoz.git && cd signoz/frontend + ```` +- Create a file `.env` in the `frontend` directory with `FRONTEND_API_ENDPOINT=` +- Next, + ``` + yarn install + yarn dev + ``` -### To run ClickHouse setup (recommended for local development) +Please ping us in the [`#contributing`](https://signoz-community.slack.com/archives/C01LWQ8KS7M) channel or ask `@Prashant Shahi` in our [Slack Community](https://signoz.io/slack) and we will DM you with ``. -- git clone https://github.com/SigNoz/signoz.git -- run `cd signoz` to move to signoz directory -- run `sudo make dev-setup` to configure local setup to run query-service -- comment out frontend service section at `docker/clickhouse-setup/docker-compose.yaml` -- comment out query-service section at `docker/clickhouse-setup/docker-compose.yaml` -- add below configuration to clickhouse section at `docker/clickhouse-setup/docker-compose.yaml` -```docker - expose: - - 9000 - ports: - - 9001:9000 +**Frontend should now be accessible at** [`http://localhost:3301/application`](http://localhost:3301/application) + + **[`^top^`](#)** + +
+ +# 4. Contribute to Backend (Query-Service) πŸŒ‘ + +[**https://github.com/SigNoz/signoz/tree/develop/pkg/query-service**](https://github.com/SigNoz/signoz/tree/develop/pkg/query-service) + +## 4.1 To run ClickHouse setup (recommended for local development) + +- Clone the SigNoz repository and cd into signoz directory, + ``` + git clone https://github.com/SigNoz/signoz.git && cd signoz + ``` +- run `sudo make dev-setup` to configure local setup to run query-service, +- Comment out `frontend` service section at [`deploy/docker/clickhouse-setup/docker-compose.yaml#L68`](https://github.com/SigNoz/signoz/blob/develop/deploy/docker/clickhouse-setup/docker-compose.yaml#L68) +develop-frontend + +- Comment out `query-service` section at [`deploy/docker/clickhouse-setup/docker-compose.yaml#L41`,](https://github.com/SigNoz/signoz/blob/develop/deploy/docker/clickhouse-setup/docker-compose.yaml#L41) +Screenshot 2022-07-14 at 22 48 07 + +- add below configuration to `clickhouse` section at [`deploy/docker/clickhouse-setup/docker-compose.yaml`,](https://github.com/SigNoz/signoz/blob/develop/deploy/docker/clickhouse-setup/docker-compose.yaml) + ``` + ports: + - 9001:9000 + ``` +Screenshot 2022-07-14 at 22 50 37 + +- run `cd pkg/query-service/` to move to `query-service` directory, +- Then, you need to create a `.env` file with the following environment variable + ``` + SIGNOZ_LOCAL_DB_PATH="./signoz.db" + ``` +to set your local environment with the right `RELATIONAL_DATASOURCE_PATH` as mentioned in [`./constants/constants.go#L38`,](https://github.com/SigNoz/signoz/blob/develop/pkg/query-service/constants/constants.go#L38) + +- Now, install SigNoz locally **without** the `frontend` and `query-service`, + - If you are using `x86_64` processors (All Intel/AMD processors) run `sudo make run-x86` + - If you are on `arm64` processors (Apple M1 Macs) run `sudo make run-arm` + +#### Run locally, ``` - -- run `cd pkg/query-service/` to move to query-service directory -- Open ./constants/constants.go - - Replace ```const RELATIONAL_DATASOURCE_PATH = "/var/lib/signoz/signoz.db"``` \ - with ```const RELATIONAL_DATASOURCE_PATH = "./signoz.db".``` - -- Install signoz locally without the frontend and query-service - - If you are using x86_64 processors (All Intel/AMD processors) run `sudo make run-x86` - - If you are on arm64 processors (Apple M1 Macbooks) run `sudo make run-arm` - -#### Run locally -```console ClickHouseUrl=tcp://localhost:9001 STORAGE=clickhouse go run main.go ``` -> Notes for Maintainers/Contributors who will change Line Numbers of Frontend & Query-Section. Please Update Line Numbers in `./scripts/commentLinesForSetup.sh` +#### Build and Run locally +``` +cd pkg/query-service +go build -o build/query-service main.go +ClickHouseUrl=tcp://localhost:9001 STORAGE=clickhouse build/query-service +``` -**_Query Service should now be available at `http://localhost:8080`_** +#### Docker Images +The docker images of query-service is available at https://hub.docker.com/r/signoz/query-service -> If you want to see how, frontend plays with query service, you can run frontend also in you local env with the baseURL changed to `http://localhost:8080` in file `src/constants/env.ts` as the query-service is now running at port `8080` +``` +docker pull signoz/query-service +``` + +``` +docker pull signoz/query-service:latest +``` + +``` +docker pull signoz/query-service:develop +``` + +### Important Note: +The Maintainers / Contributors who will change Line Numbers of `Frontend` & `Query-Section`, please update line numbers in [`/.scripts/commentLinesForSetup.sh`](https://github.com/SigNoz/signoz/blob/develop/.scripts/commentLinesForSetup.sh) + + + +**Query Service should now be available at** [`http://localhost:8080`](http://localhost:8080) + +If you want to see how the frontend plays with query service, you can run the frontend also in your local env with the baseURL changed to `http://localhost:8080` in file [`frontend/src/constants/env.ts`](https://github.com/SigNoz/signoz/blob/develop/frontend/src/constants/env.ts) as the `query-service` is now running at port `8080`. ---- +> To use it on your forked repo, edit the 'Open in Gitpod' button URL to `https://gitpod.io/#https://github.com//signoz` --> -# Contribute to SigNoz Helm Chart + **[`^top^`](#)** + +
-Need to update [https://github.com/SigNoz/charts](https://github.com/SigNoz/charts). +# 5. Contribute to SigNoz Helm Chart πŸ“Š -### To run helm chart for local development +**Need to Update: [https://github.com/SigNoz/charts](https://github.com/SigNoz/charts).** -- run `git clone https://github.com/SigNoz/charts.git` followed by `cd charts` -- it is recommended to use lightweight kubernetes (k8s) cluster for local development: +## 5.1 To run helm chart for local development + +- Clone the SigNoz repository and cd into charts directory, + ``` + git clone https://github.com/SigNoz/charts.git && cd charts + ``` +- It is recommended to use lightweight kubernetes (k8s) cluster for local development: - [kind](https://kind.sigs.k8s.io/docs/user/quick-start/#installation) - [k3d](https://k3d.io/#installation) - [minikube](https://minikube.sigs.k8s.io/docs/start/) -- create a k8s cluster and make sure `kubectl` points to the locally created k8s cluster -- run `make dev-install` to install SigNoz chart with `my-release` release name in `platform` namespace. -- run `kubectl -n platform port-forward svc/my-release-signoz-frontend 3301:3301` to make SigNoz UI available at [localhost:3301](http://localhost:3301) +- create a k8s cluster and make sure `kubectl` points to the locally created k8s cluster, +- run `make dev-install` to install SigNoz chart with `my-release` release name in `platform` namespace, +- next run, + ``` + kubectl -n platform port-forward svc/my-release-signoz-frontend 3301:3301 + ``` +to make SigNoz UI available at [localhost:3301](http://localhost:3301) -**To install HotROD sample app:** +**5.1.1 To install the HotROD sample app:** ```bash curl -sL https://github.com/SigNoz/signoz/raw/main/sample-apps/hotrod/hotrod-install.sh \ | HELM_RELEASE=my-release SIGNOZ_NAMESPACE=platform bash ``` -**To load data with HotROD sample app:** +**5.1.2 To load data with the HotROD sample app:** ```bash kubectl -n sample-application run strzal --image=djbingham/curl \ @@ -124,7 +333,7 @@ kubectl -n sample-application run strzal --image=djbingham/curl \ 'locust_count=6' -F 'hatch_rate=2' http://locust-master:8089/swarm ``` -**To stop the load generation:** +**5.1.3 To stop the load generation:** ```bash kubectl -n sample-application run strzal --image=djbingham/curl \ @@ -132,59 +341,32 @@ kubectl -n sample-application run strzal --image=djbingham/curl \ http://locust-master:8089/stop ``` -**To delete HotROD sample app:** +**5.1.4 To delete the HotROD sample app:** ```bash curl -sL https://github.com/SigNoz/signoz/raw/main/sample-apps/hotrod/hotrod-delete.sh \ | HOTROD_NAMESPACE=sample-application bash ``` + **[`^top^`](#)** + --- -## General Instructions +## Other Ways to Contribute -**Before making any significant changes, please open an issue**. Each issue -should describe the following: +There are many other ways to get involved with the community and to participate in this project: -* Requirement - what kind of use case are you trying to solve? -* Proposal - what do you suggest to solve the problem or improve the existing - situation? -* Any open questions to address - -Discussing your proposed changes ahead of time will make the contribution -process smooth for everyone. Once the approach is agreed upon, make your changes -and open a pull request(s). Unless your change is small, Please consider submitting different PRs: - -* First PR should include the overall structure of the new component: - * Readme, configuration, interfaces or base classes etc... - * This PR is usually trivial to review, so the size limit does not apply to - it. -* Second PR should include the concrete implementation of the component. If the - size of this PR is larger than the recommended size consider splitting it in - multiple PRs. -* If there are multiple sub-component then ideally each one should be implemented as - a separate pull request. -* Last PR should include changes to any user facing documentation. And should include - end to end tests if applicable. The component must be enabled - only after sufficient testing, and there is enough confidence in the - stability and quality of the component. +- Use the product, submitting GitHub issues when a problem is found. +- Help code review pull requests and participate in issue threads. +- Submit a new feature request as an issue. +- Help answer questions on forums such as Stack Overflow and [SigNoz Community Slack Channel](https://signoz.io/slack). +- Tell others about the project on Twitter, your blog, etc. -You can always reach out to `ankit@signoz.io` to understand more about the repo and product. We are very responsive over email and [slack](https://signoz.io/slack). +## License -- If you find any bugs, please create an issue -- If you find anything missing in documentation, you can create an issue with label **documentation** -- If you want to build any new feature, please create an issue with label `enhancement` -- If you want to discuss something about the product, start a new [discussion](https://github.com/SigNoz/signoz/discussions) +By contributing to SigNoz, you agree that your contributions will be licensed under its MIT license. -### Conventions to follow when submitting commits, PRs +Again, Feel free to ping us on [`#contributing`](https://signoz-community.slack.com/archives/C01LWQ8KS7M) or [`#contributing-frontend`](https://signoz-community.slack.com/archives/C027134DM8B) on our slack community if you need any help on this :) -1. We try to follow https://www.conventionalcommits.org/en/v1.0.0/ - -More specifically the commits and PRs should have type specifiers prefixed in the name. [This](https://www.conventionalcommits.org/en/v1.0.0/#specification) should give you a better idea. - -e.g. If you are submitting a fix for an issue in frontend - PR name should be prefixed with `fix(FE):` - -2. Follow [GitHub Flow](https://guides.github.com/introduction/flow/) guidelines for your contribution flows - -3. Feel free to ping us on `#contributing` or `#contributing-frontend` on our slack community if you need any help on this :) +Thank You! diff --git a/deploy/docker-swarm/clickhouse-setup/clickhouse-storage.xml b/deploy/docker-swarm/clickhouse-setup/clickhouse-storage.xml index aab0c15da7..2b2f4010ac 100644 --- a/deploy/docker-swarm/clickhouse-setup/clickhouse-storage.xml +++ b/deploy/docker-swarm/clickhouse-setup/clickhouse-storage.xml @@ -20,6 +20,7 @@ s3 + 0 diff --git a/deploy/docker-swarm/clickhouse-setup/docker-compose.yaml b/deploy/docker-swarm/clickhouse-setup/docker-compose.yaml index 1003dda437..9fbf9e0632 100644 --- a/deploy/docker-swarm/clickhouse-setup/docker-compose.yaml +++ b/deploy/docker-swarm/clickhouse-setup/docker-compose.yaml @@ -27,7 +27,7 @@ services: retries: 3 alertmanager: - image: signoz/alertmanager:0.23.0-0.1 + image: signoz/alertmanager:0.23.0-0.2 volumes: - ./data/alertmanager:/data command: @@ -40,7 +40,7 @@ services: condition: on-failure query-service: - image: signoz/query-service:0.10.0 + image: signoz/query-service:0.10.1 command: ["-config=/root/config/prometheus.yml"] # ports: # - "6060:6060" # pprof port @@ -68,7 +68,7 @@ services: - clickhouse frontend: - image: signoz/frontend:0.10.0 + image: signoz/frontend:0.10.1 deploy: restart_policy: condition: on-failure @@ -81,7 +81,7 @@ services: - ../common/nginx-config.conf:/etc/nginx/conf.d/default.conf otel-collector: - image: signoz/otelcontribcol:0.45.1-1.1 + image: signoz/otelcontribcol:0.45.1-1.3 command: ["--config=/etc/otel-collector-config.yaml"] volumes: - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml @@ -111,7 +111,7 @@ services: - clickhouse otel-collector-metrics: - image: signoz/otelcontribcol:0.45.1-1.1 + image: signoz/otelcontribcol:0.45.1-1.3 command: ["--config=/etc/otel-collector-metrics-config.yaml"] volumes: - ./otel-collector-metrics-config.yaml:/etc/otel-collector-metrics-config.yaml diff --git a/deploy/docker-swarm/clickhouse-setup/otel-collector-metrics-config.yaml b/deploy/docker-swarm/clickhouse-setup/otel-collector-metrics-config.yaml index a01f356437..ecaee5977a 100644 --- a/deploy/docker-swarm/clickhouse-setup/otel-collector-metrics-config.yaml +++ b/deploy/docker-swarm/clickhouse-setup/otel-collector-metrics-config.yaml @@ -5,9 +5,11 @@ receivers: # otel-collector internal metrics - job_name: "otel-collector" scrape_interval: 60s - static_configs: - - targets: - - otel-collector:8888 + dns_sd_configs: + - names: + - 'tasks.otel-collector' + type: 'A' + port: 8888 # otel-collector-metrics internal metrics - job_name: "otel-collector-metrics" scrape_interval: 60s @@ -17,9 +19,11 @@ receivers: # SigNoz span metrics - job_name: "signozspanmetrics-collector" scrape_interval: 60s - static_configs: - - targets: - - otel-collector:8889 + dns_sd_configs: + - names: + - 'tasks.otel-collector' + type: 'A' + port: 8889 processors: batch: diff --git a/deploy/docker/clickhouse-setup/clickhouse-storage.xml b/deploy/docker/clickhouse-setup/clickhouse-storage.xml index aab0c15da7..2b2f4010ac 100644 --- a/deploy/docker/clickhouse-setup/clickhouse-storage.xml +++ b/deploy/docker/clickhouse-setup/clickhouse-storage.xml @@ -20,6 +20,7 @@ s3 + 0 diff --git a/deploy/docker/clickhouse-setup/docker-compose.yaml b/deploy/docker/clickhouse-setup/docker-compose.yaml index 3b3403a480..2892cb89a2 100644 --- a/deploy/docker/clickhouse-setup/docker-compose.yaml +++ b/deploy/docker/clickhouse-setup/docker-compose.yaml @@ -25,7 +25,7 @@ services: retries: 3 alertmanager: - image: signoz/alertmanager:0.23.0-0.1 + image: signoz/alertmanager:0.23.0-0.2 volumes: - ./data/alertmanager:/data depends_on: @@ -39,7 +39,7 @@ services: # Notes for Maintainers/Contributors who will change Line Numbers of Frontend & Query-Section. Please Update Line Numbers in `./scripts/commentLinesForSetup.sh` & `./CONTRIBUTING.md` query-service: - image: signoz/query-service:0.10.0 + image: signoz/query-service:0.10.1 container_name: query-service command: ["-config=/root/config/prometheus.yml"] # ports: @@ -66,7 +66,7 @@ services: condition: service_healthy frontend: - image: signoz/frontend:0.10.0 + image: signoz/frontend:0.10.1 container_name: frontend restart: on-failure depends_on: @@ -78,7 +78,7 @@ services: - ../common/nginx-config.conf:/etc/nginx/conf.d/default.conf otel-collector: - image: signoz/otelcontribcol:0.45.1-1.1 + image: signoz/otelcontribcol:0.45.1-1.3 command: ["--config=/etc/otel-collector-config.yaml"] volumes: - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml @@ -103,7 +103,7 @@ services: condition: service_healthy otel-collector-metrics: - image: signoz/otelcontribcol:0.45.1-1.1 + image: signoz/otelcontribcol:0.45.1-1.3 command: ["--config=/etc/otel-collector-metrics-config.yaml"] volumes: - ./otel-collector-metrics-config.yaml:/etc/otel-collector-metrics-config.yaml diff --git a/deploy/install.sh b/deploy/install.sh index 492336a566..34ca5356bb 100755 --- a/deploy/install.sh +++ b/deploy/install.sh @@ -204,9 +204,14 @@ start_docker() { echo "Starting docker service" $sudo_cmd systemctl start docker.service fi + # if [[ -z $sudo_cmd ]]; then + # docker ps > /dev/null && true + # if [[ $? -ne 0 ]]; then + # request_sudo + # fi + # fi if [[ -z $sudo_cmd ]]; then - docker ps > /dev/null && true - if [[ $? -ne 0 ]]; then + if ! docker ps > /dev/null && true; then request_sudo fi fi @@ -268,8 +273,12 @@ request_sudo() { if (( $EUID != 0 )); then sudo_cmd="sudo" echo -e "Please enter your sudo password, if prompt." - $sudo_cmd -l | grep -e "NOPASSWD: ALL" > /dev/null - if [[ $? -ne 0 ]] && ! $sudo_cmd -v; then + # $sudo_cmd -l | grep -e "NOPASSWD: ALL" > /dev/null + # if [[ $? -ne 0 ]] && ! $sudo_cmd -v; then + # echo "Need sudo privileges to proceed with the installation." + # exit 1; + # fi + if ! $sudo_cmd -l | grep -e "NOPASSWD: ALL" > /dev/null && ! $sudo_cmd -v; then echo "Need sudo privileges to proceed with the installation." exit 1; fi @@ -303,8 +312,13 @@ echo -e "🌏 Detecting your OS ...\n" check_os # Obtain unique installation id -sysinfo="$(uname -a)" -if [[ $? -ne 0 ]]; then +# sysinfo="$(uname -a)" +# if [[ $? -ne 0 ]]; then +# uuid="$(uuidgen)" +# uuid="${uuid:-$(cat /proc/sys/kernel/random/uuid)}" +# sysinfo="${uuid:-$(cat /proc/sys/kernel/random/uuid)}" +# fi +if ! sysinfo="$(uname -a)"; then uuid="$(uuidgen)" uuid="${uuid:-$(cat /proc/sys/kernel/random/uuid)}" sysinfo="${uuid:-$(cat /proc/sys/kernel/random/uuid)}" diff --git a/frontend/package.json b/frontend/package.json index 868e95dce7..1fd5563c0b 100644 --- a/frontend/package.json +++ b/frontend/package.json @@ -16,6 +16,7 @@ "playwright": "NODE_ENV=testing playwright test --config=./playwright.config.ts", "playwright:local:debug": "PWDEBUG=console yarn playwright --headed --browser=chromium", "playwright:codegen:local":"playwright codegen http://localhost:3301", + "playwright:codegen:local:auth":"yarn playwright:codegen:local --load-storage=tests/auth.json", "husky:configure": "cd .. && husky install frontend/.husky && cd frontend && chmod ug+x .husky/*", "commitlint": "commitlint --edit $1" }, diff --git a/frontend/playwright.config.ts b/frontend/playwright.config.ts index 6733c67536..0b24052fc8 100644 --- a/frontend/playwright.config.ts +++ b/frontend/playwright.config.ts @@ -14,8 +14,8 @@ const config: PlaywrightTestConfig = { baseURL: process.env.PLAYWRIGHT_TEST_BASE_URL || 'http://localhost:3301', }, updateSnapshots: 'all', - fullyParallel: false, - quiet: true, + fullyParallel: !!process.env.CI, + quiet: false, testMatch: ['**/*.spec.ts'], reporter: process.env.CI ? 'github' : 'list', }; diff --git a/frontend/public/locales/en-GB/alerts.json b/frontend/public/locales/en-GB/alerts.json index e67bd35273..cae309fd45 100644 --- a/frontend/public/locales/en-GB/alerts.json +++ b/frontend/public/locales/en-GB/alerts.json @@ -1,4 +1,11 @@ { + "target_missing": "Please enter a threshold to proceed", + "rule_test_fired": "Test notification sent successfully", + "no_alerts_found": "No alerts found during the evaluation. This happens when rule condition is unsatisfied. You may adjust the rule threshold and retry.", + "button_testrule": "Test Notification", + "label_channel_select": "Notification Channels", + "placeholder_channel_select": "select one or more channels", + "channel_select_tooltip": "Leave empty to send this alert on all the configured channels", "preview_chart_unexpected_error": "An unexpeced error occurred updating the chart, please check your query.", "preview_chart_threshold_label": "Threshold", "placeholder_label_key_pair": "Click here to enter a label (key value pairs)", diff --git a/frontend/public/locales/en-GB/channels.json b/frontend/public/locales/en-GB/channels.json index 5e670cc536..027501f69d 100644 --- a/frontend/public/locales/en-GB/channels.json +++ b/frontend/public/locales/en-GB/channels.json @@ -1,4 +1,14 @@ { + "channel_delete_unexp_error": "Something went wrong", + "channel_delete_success": "Channel Deleted Successfully", + "column_channel_name": "Name", + "column_channel_type": "Type", + "column_channel_action": "Action", + "column_channel_edit": "Edit", + "button_new_channel": "New Alert Channel", + "tooltip_notification_channels": "More details on how to setting notification channels", + "sending_channels_note": "The alerts will be sent to all the configured channels.", + "loading_channels_message": "Loading Channels..", "page_title_create": "New Notification Channels", "page_title_edit": "Edit Notification Channels", "button_save_channel": "Save", diff --git a/frontend/public/locales/en/alerts.json b/frontend/public/locales/en/alerts.json index e67bd35273..cae309fd45 100644 --- a/frontend/public/locales/en/alerts.json +++ b/frontend/public/locales/en/alerts.json @@ -1,4 +1,11 @@ { + "target_missing": "Please enter a threshold to proceed", + "rule_test_fired": "Test notification sent successfully", + "no_alerts_found": "No alerts found during the evaluation. This happens when rule condition is unsatisfied. You may adjust the rule threshold and retry.", + "button_testrule": "Test Notification", + "label_channel_select": "Notification Channels", + "placeholder_channel_select": "select one or more channels", + "channel_select_tooltip": "Leave empty to send this alert on all the configured channels", "preview_chart_unexpected_error": "An unexpeced error occurred updating the chart, please check your query.", "preview_chart_threshold_label": "Threshold", "placeholder_label_key_pair": "Click here to enter a label (key value pairs)", diff --git a/frontend/public/locales/en/channels.json b/frontend/public/locales/en/channels.json index 5e670cc536..027501f69d 100644 --- a/frontend/public/locales/en/channels.json +++ b/frontend/public/locales/en/channels.json @@ -1,4 +1,14 @@ { + "channel_delete_unexp_error": "Something went wrong", + "channel_delete_success": "Channel Deleted Successfully", + "column_channel_name": "Name", + "column_channel_type": "Type", + "column_channel_action": "Action", + "column_channel_edit": "Edit", + "button_new_channel": "New Alert Channel", + "tooltip_notification_channels": "More details on how to setting notification channels", + "sending_channels_note": "The alerts will be sent to all the configured channels.", + "loading_channels_message": "Loading Channels..", "page_title_create": "New Notification Channels", "page_title_edit": "Edit Notification Channels", "button_save_channel": "Save", diff --git a/frontend/src/api/alerts/patch.ts b/frontend/src/api/alerts/patch.ts new file mode 100644 index 0000000000..920b53ae9f --- /dev/null +++ b/frontend/src/api/alerts/patch.ts @@ -0,0 +1,26 @@ +import axios from 'api'; +import { ErrorResponseHandler } from 'api/ErrorResponseHandler'; +import { AxiosError } from 'axios'; +import { ErrorResponse, SuccessResponse } from 'types/api'; +import { PayloadProps, Props } from 'types/api/alerts/patch'; + +const patch = async ( + props: Props, +): Promise | ErrorResponse> => { + try { + const response = await axios.patch(`/rules/${props.id}`, { + ...props.data, + }); + + return { + statusCode: 200, + error: null, + message: response.data.status, + payload: response.data.data, + }; + } catch (error) { + return ErrorResponseHandler(error as AxiosError); + } +}; + +export default patch; diff --git a/frontend/src/api/alerts/testAlert.ts b/frontend/src/api/alerts/testAlert.ts new file mode 100644 index 0000000000..a30e977a10 --- /dev/null +++ b/frontend/src/api/alerts/testAlert.ts @@ -0,0 +1,26 @@ +import axios from 'api'; +import { ErrorResponseHandler } from 'api/ErrorResponseHandler'; +import { AxiosError } from 'axios'; +import { ErrorResponse, SuccessResponse } from 'types/api'; +import { PayloadProps, Props } from 'types/api/alerts/testAlert'; + +const testAlert = async ( + props: Props, +): Promise | ErrorResponse> => { + try { + const response = await axios.post('/testRule', { + ...props.data, + }); + + return { + statusCode: 200, + error: null, + message: response.data.status, + payload: response.data.data, + }; + } catch (error) { + return ErrorResponseHandler(error as AxiosError); + } +}; + +export default testAlert; diff --git a/frontend/src/api/metrics/getTopLevelOperations.ts b/frontend/src/api/metrics/getTopLevelOperations.ts new file mode 100644 index 0000000000..5ecfd2a67a --- /dev/null +++ b/frontend/src/api/metrics/getTopLevelOperations.ts @@ -0,0 +1,24 @@ +import axios from 'api'; +import { ErrorResponseHandler } from 'api/ErrorResponseHandler'; +import { AxiosError } from 'axios'; +import { ErrorResponse, SuccessResponse } from 'types/api'; +import { PayloadProps, Props } from 'types/api/metrics/getTopLevelOperations'; + +const getTopLevelOperations = async ( + props: Props, +): Promise | ErrorResponse> => { + try { + const response = await axios.post(`/service/top_level_operations`); + + return { + statusCode: 200, + error: null, + message: response.data.status, + payload: response.data[props.service], + }; + } catch (error) { + return ErrorResponseHandler(error as AxiosError); + } +}; + +export default getTopLevelOperations; diff --git a/frontend/src/api/metrics/getTopEndPoints.ts b/frontend/src/api/metrics/getTopOperations.ts similarity index 73% rename from frontend/src/api/metrics/getTopEndPoints.ts rename to frontend/src/api/metrics/getTopOperations.ts index db78aae9e3..cf07f0ee5d 100644 --- a/frontend/src/api/metrics/getTopEndPoints.ts +++ b/frontend/src/api/metrics/getTopOperations.ts @@ -2,13 +2,13 @@ import axios from 'api'; import { ErrorResponseHandler } from 'api/ErrorResponseHandler'; import { AxiosError } from 'axios'; import { ErrorResponse, SuccessResponse } from 'types/api'; -import { PayloadProps, Props } from 'types/api/metrics/getTopEndPoints'; +import { PayloadProps, Props } from 'types/api/metrics/getTopOperations'; -const getTopEndPoints = async ( +const getTopOperations = async ( props: Props, ): Promise | ErrorResponse> => { try { - const response = await axios.post(`/service/top_endpoints`, { + const response = await axios.post(`/service/top_operations`, { start: `${props.start}`, end: `${props.end}`, service: props.service, @@ -26,4 +26,4 @@ const getTopEndPoints = async ( } }; -export default getTopEndPoints; +export default getTopOperations; diff --git a/frontend/src/container/AllAlertChannels/AlertChannels.tsx b/frontend/src/container/AllAlertChannels/AlertChannels.tsx index 974530c6e5..762304a871 100644 --- a/frontend/src/container/AllAlertChannels/AlertChannels.tsx +++ b/frontend/src/container/AllAlertChannels/AlertChannels.tsx @@ -5,6 +5,7 @@ import ROUTES from 'constants/routes'; import useComponentPermission from 'hooks/useComponentPermission'; import history from 'lib/history'; import React, { useCallback, useState } from 'react'; +import { useTranslation } from 'react-i18next'; import { useSelector } from 'react-redux'; import { generatePath } from 'react-router-dom'; import { AppState } from 'store/reducers'; @@ -14,6 +15,7 @@ import AppReducer from 'types/reducer/app'; import Delete from './Delete'; function AlertChannels({ allChannels }: AlertChannelsProps): JSX.Element { + const { t } = useTranslation(['channels']); const [notifications, Element] = notification.useNotification(); const [channels, setChannels] = useState(allChannels); const { role } = useSelector((state) => state.app); @@ -29,12 +31,12 @@ function AlertChannels({ allChannels }: AlertChannelsProps): JSX.Element { const columns: ColumnsType = [ { - title: 'Name', + title: t('column_channel_name'), dataIndex: 'name', key: 'name', }, { - title: 'Type', + title: t('column_channel_type'), dataIndex: 'type', key: 'type', }, @@ -42,14 +44,14 @@ function AlertChannels({ allChannels }: AlertChannelsProps): JSX.Element { if (action) { columns.push({ - title: 'Action', + title: t('column_channel_action'), dataIndex: 'id', key: 'action', align: 'center', render: (id: string): JSX.Element => ( <> diff --git a/frontend/src/container/AllAlertChannels/Delete.tsx b/frontend/src/container/AllAlertChannels/Delete.tsx index 85116fd922..75555e199c 100644 --- a/frontend/src/container/AllAlertChannels/Delete.tsx +++ b/frontend/src/container/AllAlertChannels/Delete.tsx @@ -1,29 +1,31 @@ import { Button } from 'antd'; import { NotificationInstance } from 'antd/lib/notification'; -import deleteAlert from 'api/channels/delete'; +import deleteChannel from 'api/channels/delete'; import React, { useState } from 'react'; +import { useTranslation } from 'react-i18next'; import { Channels } from 'types/api/channels/getAll'; function Delete({ notifications, setChannels, id }: DeleteProps): JSX.Element { + const { t } = useTranslation(['channels']); const [loading, setLoading] = useState(false); const onClickHandler = async (): Promise => { try { setLoading(true); - const response = await deleteAlert({ + const response = await deleteChannel({ id, }); if (response.statusCode === 200) { notifications.success({ message: 'Success', - description: 'Channel Deleted Successfully', + description: t('channel_delete_success'), }); setChannels((preChannels) => preChannels.filter((e) => e.id !== id)); } else { notifications.error({ message: 'Error', - description: response.error || 'Something went wrong', + description: response.error || t('channel_delete_unexp_error'), }); } setLoading(false); @@ -31,7 +33,9 @@ function Delete({ notifications, setChannels, id }: DeleteProps): JSX.Element { notifications.error({ message: 'Error', description: - error instanceof Error ? error.toString() : 'Something went wrong', + error instanceof Error + ? error.toString() + : t('channel_delete_unexp_error'), }); setLoading(false); } diff --git a/frontend/src/container/AllAlertChannels/index.tsx b/frontend/src/container/AllAlertChannels/index.tsx index 44ab948f0b..99636806ea 100644 --- a/frontend/src/container/AllAlertChannels/index.tsx +++ b/frontend/src/container/AllAlertChannels/index.tsx @@ -8,16 +8,18 @@ import useComponentPermission from 'hooks/useComponentPermission'; import useFetch from 'hooks/useFetch'; import history from 'lib/history'; import React, { useCallback } from 'react'; +import { useTranslation } from 'react-i18next'; import { useSelector } from 'react-redux'; import { AppState } from 'store/reducers'; import AppReducer from 'types/reducer/app'; import AlertChannelsComponent from './AlertChannels'; -import { Button, ButtonContainer } from './styles'; +import { Button, ButtonContainer, RightActionContainer } from './styles'; const { Paragraph } = Typography; function AlertChannels(): JSX.Element { + const { t } = useTranslation(['channels']); const { role } = useSelector((state) => state.app); const [addNewChannelPermission] = useComponentPermission( ['add_new_channel'], @@ -34,28 +36,28 @@ function AlertChannels(): JSX.Element { } if (loading || payload === undefined) { - return ; + return ; } return ( <> - The latest added channel is used as the default channel for sending alerts + {t('sending_channels_note')} -
+ {addNewChannelPermission && ( )} -
+ diff --git a/frontend/src/container/AllAlertChannels/styles.ts b/frontend/src/container/AllAlertChannels/styles.ts index b2d03a4cea..209860b867 100644 --- a/frontend/src/container/AllAlertChannels/styles.ts +++ b/frontend/src/container/AllAlertChannels/styles.ts @@ -1,6 +1,13 @@ import { Button as ButtonComponent } from 'antd'; import styled from 'styled-components'; +export const RightActionContainer = styled.div` + &&& { + display: flex; + align-items: center; + } +`; + export const ButtonContainer = styled.div` &&& { display: flex; diff --git a/frontend/src/container/FormAlertRules/BasicInfo.tsx b/frontend/src/container/FormAlertRules/BasicInfo.tsx index c977c82a4e..6bfbfffd03 100644 --- a/frontend/src/container/FormAlertRules/BasicInfo.tsx +++ b/frontend/src/container/FormAlertRules/BasicInfo.tsx @@ -4,9 +4,12 @@ import React from 'react'; import { useTranslation } from 'react-i18next'; import { AlertDef, Labels } from 'types/api/alerts/def'; +import ChannelSelect from './ChannelSelect'; import LabelSelect from './labels'; import { + ChannelSelectTip, FormContainer, + FormItemMedium, InputSmall, SeveritySelect, StepHeading, @@ -80,7 +83,7 @@ function BasicInfo({ alertDef, setAlertDef }: BasicInfoProps): JSX.Element { }} /> - + { setAlertDef({ @@ -92,7 +95,19 @@ function BasicInfo({ alertDef, setAlertDef }: BasicInfoProps): JSX.Element { }} initialValues={alertDef.labels} /> - + + + { + setAlertDef({ + ...alertDef, + preferredChannels: s, + }); + }} + /> + {t('channel_select_tooltip')} + ); diff --git a/frontend/src/container/FormAlertRules/ChannelSelect/index.tsx b/frontend/src/container/FormAlertRules/ChannelSelect/index.tsx new file mode 100644 index 0000000000..99c3038a42 --- /dev/null +++ b/frontend/src/container/FormAlertRules/ChannelSelect/index.tsx @@ -0,0 +1,70 @@ +import { notification, Select } from 'antd'; +import getChannels from 'api/channels/getAll'; +import useFetch from 'hooks/useFetch'; +import React from 'react'; +import { useTranslation } from 'react-i18next'; + +import { StyledSelect } from './styles'; + +export interface ChannelSelectProps { + currentValue?: string[]; + onSelectChannels: (s: string[]) => void; +} + +function ChannelSelect({ + currentValue, + onSelectChannels, +}: ChannelSelectProps): JSX.Element | null { + // init namespace for translations + const { t } = useTranslation('alerts'); + + const { loading, payload, error, errorMessage } = useFetch(getChannels); + + const handleChange = (value: string[]): void => { + onSelectChannels(value); + }; + + if (error && errorMessage !== '') { + notification.error({ + message: 'Error', + description: errorMessage, + }); + } + const renderOptions = (): React.ReactNode[] => { + const children: React.ReactNode[] = []; + + if (loading || payload === undefined || payload.length === 0) { + return children; + } + + payload.forEach((o) => { + children.push( + + {o.name} + , + ); + }); + + return children; + }; + return ( + { + handleChange(value as string[]); + }} + optionLabelProp="label" + > + {renderOptions()} + + ); +} + +ChannelSelect.defaultProps = { + currentValue: [], +}; +export default ChannelSelect; diff --git a/frontend/src/container/FormAlertRules/ChannelSelect/styles.ts b/frontend/src/container/FormAlertRules/ChannelSelect/styles.ts new file mode 100644 index 0000000000..7a59e38767 --- /dev/null +++ b/frontend/src/container/FormAlertRules/ChannelSelect/styles.ts @@ -0,0 +1,6 @@ +import { Select } from 'antd'; +import styled from 'styled-components'; + +export const StyledSelect = styled(Select)` + border-radius: 4px; +`; diff --git a/frontend/src/container/FormAlertRules/ChartPreview/index.tsx b/frontend/src/container/FormAlertRules/ChartPreview/index.tsx index b6f51227da..6243c1d4d4 100644 --- a/frontend/src/container/FormAlertRules/ChartPreview/index.tsx +++ b/frontend/src/container/FormAlertRules/ChartPreview/index.tsx @@ -21,7 +21,7 @@ export interface ChartPreviewProps { selectedTime?: timePreferenceType; selectedInterval?: Time; headline?: JSX.Element; - threshold?: number; + threshold?: number | undefined; } function ChartPreview({ @@ -35,7 +35,7 @@ function ChartPreview({ }: ChartPreviewProps): JSX.Element | null { const { t } = useTranslation('alerts'); const staticLine: StaticLineProps | undefined = - threshold && threshold > 0 + threshold !== undefined ? { yMin: threshold, yMax: threshold, @@ -66,8 +66,12 @@ function ChartPreview({ }), enabled: query != null && - (query.queryType !== EQueryType.PROM || - (query.promQL?.length > 0 && query.promQL[0].query !== '')), + ((query.queryType === EQueryType.PROM && + query.promQL?.length > 0 && + query.promQL[0].query !== '') || + (query.queryType === EQueryType.QUERY_BUILDER && + query.metricsBuilder?.queryBuilder?.length > 0 && + query.metricsBuilder?.queryBuilder[0].metricName !== '')), }); const chartDataSet = queryResponse.isError @@ -113,7 +117,7 @@ ChartPreview.defaultProps = { selectedTime: 'GLOBAL_TIME', selectedInterval: '5min', headline: undefined, - threshold: 0, + threshold: undefined, }; export default ChartPreview; diff --git a/frontend/src/container/FormAlertRules/RuleOptions.tsx b/frontend/src/container/FormAlertRules/RuleOptions.tsx index d9aff8bfb1..968ce695e3 100644 --- a/frontend/src/container/FormAlertRules/RuleOptions.tsx +++ b/frontend/src/container/FormAlertRules/RuleOptions.tsx @@ -156,7 +156,9 @@ function RuleOptions({ ...alertDef, condition: { ...alertDef.condition, - target: (value as number) || undefined, + op: alertDef.condition?.op || defaultCompareOp, + matchType: alertDef.condition?.matchType || defaultMatchType, + target: value as number, }, }); }} diff --git a/frontend/src/container/FormAlertRules/index.tsx b/frontend/src/container/FormAlertRules/index.tsx index 38fcaad04d..022e913f8e 100644 --- a/frontend/src/container/FormAlertRules/index.tsx +++ b/frontend/src/container/FormAlertRules/index.tsx @@ -1,6 +1,7 @@ import { ExclamationCircleOutlined, SaveOutlined } from '@ant-design/icons'; import { FormInstance, Modal, notification, Typography } from 'antd'; import saveAlertApi from 'api/alerts/save'; +import testAlertApi from 'api/alerts/testAlert'; import ROUTES from 'constants/routes'; import QueryTypeTag from 'container/NewWidget/LeftContainer/QueryTypeTag'; import PlotTag from 'container/NewWidget/LeftContainer/WidgetGraph/PlotTag'; @@ -83,7 +84,7 @@ function FormAlertRules({ // staged query is used to display chart preview const [stagedQuery, setStagedQuery] = useState(); - const debouncedStagedQuery = useDebounce(stagedQuery, 500); + const debouncedStagedQuery = useDebounce(stagedQuery, 1000); // this use effect initiates staged query and // other queries based on server data. @@ -143,10 +144,74 @@ function FormAlertRules({ }); } }; + const validatePromParams = useCallback((): boolean => { + let retval = true; + if (queryCategory !== EQueryType.PROM) return retval; + + if (!promQueries || Object.keys(promQueries).length === 0) { + notification.error({ + message: 'Error', + description: t('promql_required'), + }); + return false; + } + + Object.keys(promQueries).forEach((key) => { + if (promQueries[key].query === '') { + notification.error({ + message: 'Error', + description: t('promql_required'), + }); + retval = false; + } + }); + + return retval; + }, [t, promQueries, queryCategory]); + + const validateQBParams = useCallback((): boolean => { + let retval = true; + if (queryCategory !== EQueryType.QUERY_BUILDER) return true; + + if (!metricQueries || Object.keys(metricQueries).length === 0) { + notification.error({ + message: 'Error', + description: t('condition_required'), + }); + return false; + } + + if (!alertDef.condition?.target) { + notification.error({ + message: 'Error', + description: t('target_missing'), + }); + return false; + } + + Object.keys(metricQueries).forEach((key) => { + if (metricQueries[key].metricName === '') { + notification.error({ + message: 'Error', + description: t('metricname_missing', { where: metricQueries[key].name }), + }); + retval = false; + } + }); + + Object.keys(formulaQueries).forEach((key) => { + if (formulaQueries[key].expression === '') { + notification.error({ + message: 'Error', + description: t('expression_missing', formulaQueries[key].name), + }); + retval = false; + } + }); + return retval; + }, [t, alertDef, queryCategory, metricQueries, formulaQueries]); const isFormValid = useCallback((): boolean => { - let retval = true; - if (!alertDef.alert || alertDef.alert === '') { notification.error({ message: 'Error', @@ -155,56 +220,14 @@ function FormAlertRules({ return false; } - if ( - queryCategory === EQueryType.PROM && - (!promQueries || Object.keys(promQueries).length === 0) - ) { - notification.error({ - message: 'Error', - description: t('promql_required'), - }); + if (!validatePromParams()) { return false; } - if ( - (queryCategory === EQueryType.QUERY_BUILDER && !metricQueries) || - Object.keys(metricQueries).length === 0 - ) { - notification.error({ - message: 'Error', - description: t('condition_required'), - }); - return false; - } - - Object.keys(metricQueries).forEach((key) => { - if (metricQueries[key].metricName === '') { - retval = false; - notification.error({ - message: 'Error', - description: t('metricname_missing', { where: metricQueries[key].name }), - }); - } - }); - - Object.keys(formulaQueries).forEach((key) => { - if (formulaQueries[key].expression === '') { - retval = false; - notification.error({ - message: 'Error', - description: t('expression_missing', formulaQueries[key].name), - }); - } - }); - - return retval; - }, [t, alertDef, queryCategory, metricQueries, formulaQueries, promQueries]); - - const saveRule = useCallback(async () => { - if (!isFormValid()) { - return; - } + return validateQBParams(); + }, [t, validateQBParams, alertDef, validatePromParams]); + const preparePostData = (): AlertDef => { const postableAlert: AlertDef = { ...alertDef, source: window?.location.toString(), @@ -219,6 +242,22 @@ function FormAlertRules({ }, }, }; + return postableAlert; + }; + + const memoizedPreparePostData = useCallback(preparePostData, [ + queryCategory, + alertDef, + metricQueries, + formulaQueries, + promQueries, + ]); + + const saveRule = useCallback(async () => { + if (!isFormValid()) { + return; + } + const postableAlert = memoizedPreparePostData(); setLoading(true); try { @@ -235,7 +274,7 @@ function FormAlertRules({ description: !ruleId || ruleId === 0 ? t('rule_created') : t('rule_edited'), }); - console.log('invalidting cache'); + // invalidate rule in cache ruleCache.invalidateQueries(['ruleId', ruleId]); @@ -249,24 +288,13 @@ function FormAlertRules({ }); } } catch (e) { - console.log('save alert api failed:', e); notification.error({ message: 'Error', description: t('unexpected_error'), }); } setLoading(false); - }, [ - t, - isFormValid, - queryCategory, - ruleId, - alertDef, - metricQueries, - formulaQueries, - promQueries, - ruleCache, - ]); + }, [t, isFormValid, ruleId, ruleCache, memoizedPreparePostData]); const onSaveHandler = useCallback(async () => { const content = ( @@ -287,6 +315,44 @@ function FormAlertRules({ }); }, [t, saveRule, queryCategory]); + const onTestRuleHandler = useCallback(async () => { + if (!isFormValid()) { + return; + } + const postableAlert = memoizedPreparePostData(); + + setLoading(true); + try { + const response = await testAlertApi({ data: postableAlert }); + + if (response.statusCode === 200) { + const { payload } = response; + if (payload?.alertCount === 0) { + notification.error({ + message: 'Error', + description: t('no_alerts_found'), + }); + } else { + notification.success({ + message: 'Success', + description: t('rule_test_fired'), + }); + } + } else { + notification.error({ + message: 'Error', + description: response.error || t('unexpected_error'), + }); + } + } catch (e) { + notification.error({ + message: 'Error', + description: t('unexpected_error'), + }); + } + setLoading(false); + }, [t, isFormValid, memoizedPreparePostData]); + const renderBasicInfo = (): JSX.Element => ( ); @@ -353,6 +419,14 @@ function FormAlertRules({ > {ruleId > 0 ? t('button_savechanges') : t('button_createrule')} + + {' '} + {t('button_testrule')} + ` - width: 70%; - border-radisu: 4px; + border-radius: 4px; background: ${({ isDarkMode }): string => (isDarkMode ? '#000' : '#fff')}; flex: 1; display: flex; diff --git a/frontend/src/container/FormAlertRules/styles.ts b/frontend/src/container/FormAlertRules/styles.ts index 3c64414cbe..c1a7ad2aa8 100644 --- a/frontend/src/container/FormAlertRules/styles.ts +++ b/frontend/src/container/FormAlertRules/styles.ts @@ -1,4 +1,15 @@ -import { Button, Card, Col, Form, Input, InputNumber, Row, Select } from 'antd'; +import { + Button, + Card, + Col, + Form, + Input, + InputNumber, + Row, + Select, + Typography, +} from 'antd'; +import FormItem from 'antd/lib/form/FormItem'; import TextArea from 'antd/lib/input/TextArea'; import styled from 'styled-components'; @@ -67,21 +78,19 @@ export const InlineSelect = styled(Select)` `; export const SeveritySelect = styled(Select)` - width: 15% !important; + width: 25% !important; `; export const InputSmall = styled(Input)` width: 40% !important; `; -export const FormContainer = styled.div` +export const FormContainer = styled(Card)` padding: 2em; margin-top: 1rem; display: flex; flex-direction: column; - background: #141414; border-radius: 4px; - border: 1px solid #303030; `; export const ThresholdInput = styled(InputNumber)` @@ -101,3 +110,11 @@ export const ThresholdInput = styled(InputNumber)` export const TextareaMedium = styled(TextArea)` width: 70%; `; + +export const FormItemMedium = styled(FormItem)` + width: 70%; +`; + +export const ChannelSelectTip = styled(Typography.Text)` + color: hsla(0, 0%, 100%, 0.3); +`; diff --git a/frontend/src/container/ListAlertRules/DeleteAlert.tsx b/frontend/src/container/ListAlertRules/DeleteAlert.tsx index f479de38ab..ac91bfe2f2 100644 --- a/frontend/src/container/ListAlertRules/DeleteAlert.tsx +++ b/frontend/src/container/ListAlertRules/DeleteAlert.tsx @@ -1,10 +1,11 @@ -import { Button } from 'antd'; import { NotificationInstance } from 'antd/lib/notification/index'; import deleteAlerts from 'api/alerts/delete'; import { State } from 'hooks/useFetch'; import React, { useState } from 'react'; import { PayloadProps as DeleteAlertPayloadProps } from 'types/api/alerts/delete'; -import { Alerts } from 'types/api/alerts/getAll'; +import { GettableAlert } from 'types/api/alerts/get'; + +import { ColumnButton } from './styles'; function DeleteAlert({ id, @@ -72,20 +73,20 @@ function DeleteAlert({ }; return ( - + ); } interface DeleteAlertProps { - id: Alerts['id']; - setData: React.Dispatch>; + id: GettableAlert['id']; + setData: React.Dispatch>; notifications: NotificationInstance; } diff --git a/frontend/src/container/ListAlertRules/ListAlert.tsx b/frontend/src/container/ListAlertRules/ListAlert.tsx index 4df6290725..1981e8bfd8 100644 --- a/frontend/src/container/ListAlertRules/ListAlert.tsx +++ b/frontend/src/container/ListAlertRules/ListAlert.tsx @@ -1,6 +1,6 @@ /* eslint-disable react/display-name */ import { PlusOutlined } from '@ant-design/icons'; -import { notification, Tag, Typography } from 'antd'; +import { notification, Typography } from 'antd'; import Table, { ColumnsType } from 'antd/lib/table'; import TextToolTip from 'components/TextToolTip'; import ROUTES from 'constants/routes'; @@ -13,15 +13,16 @@ import { UseQueryResult } from 'react-query'; import { useSelector } from 'react-redux'; import { AppState } from 'store/reducers'; import { ErrorResponse, SuccessResponse } from 'types/api'; -import { Alerts } from 'types/api/alerts/getAll'; +import { GettableAlert } from 'types/api/alerts/get'; import AppReducer from 'types/reducer/app'; import DeleteAlert from './DeleteAlert'; -import { Button, ButtonContainer } from './styles'; +import { Button, ButtonContainer, ColumnButton, StyledTag } from './styles'; import Status from './TableComponents/Status'; +import ToggleAlertState from './ToggleAlertState'; function ListAlert({ allAlertRules, refetch }: ListAlertProps): JSX.Element { - const [data, setData] = useState(allAlertRules || []); + const [data, setData] = useState(allAlertRules || []); const { t } = useTranslation('common'); const { role } = useSelector((state) => state.app); const [addNewAlert, action] = useComponentPermission( @@ -53,22 +54,27 @@ function ListAlert({ allAlertRules, refetch }: ListAlertProps): JSX.Element { history.push(`${ROUTES.EDIT_ALERTS}?ruleId=${id}`); }; - const columns: ColumnsType = [ + const columns: ColumnsType = [ { title: 'Status', dataIndex: 'state', key: 'state', sorter: (a, b): number => - b.labels.severity.length - a.labels.severity.length, + (b.state ? b.state.charCodeAt(0) : 1000) - + (a.state ? a.state.charCodeAt(0) : 1000), render: (value): JSX.Element => , }, { title: 'Alert Name', dataIndex: 'alert', key: 'name', - sorter: (a, b): number => a.name.charCodeAt(0) - b.name.charCodeAt(0), + sorter: (a, b): number => + (a.alert ? a.alert.charCodeAt(0) : 1000) - + (b.alert ? b.alert.charCodeAt(0) : 1000), render: (value, record): JSX.Element => ( - onEditHandler(record.id.toString())}> + onEditHandler(record.id ? record.id.toString() : '')} + > {value} ), @@ -78,7 +84,8 @@ function ListAlert({ allAlertRules, refetch }: ListAlertProps): JSX.Element { dataIndex: 'labels', key: 'severity', sorter: (a, b): number => - a.labels.severity.length - b.labels.severity.length, + (a.labels ? a.labels.severity.length : 0) - + (b.labels ? b.labels.severity.length : 0), render: (value): JSX.Element => { const objectKeys = Object.keys(value); const withSeverityKey = objectKeys.find((e) => e === 'severity') || ''; @@ -92,6 +99,7 @@ function ListAlert({ allAlertRules, refetch }: ListAlertProps): JSX.Element { dataIndex: 'labels', key: 'tags', align: 'center', + width: 350, render: (value): JSX.Element => { const objectKeys = Object.keys(value); const withOutSeverityKeys = objectKeys.filter((e) => e !== 'severity'); @@ -104,9 +112,9 @@ function ListAlert({ allAlertRules, refetch }: ListAlertProps): JSX.Element { <> {withOutSeverityKeys.map((e) => { return ( - + {e}: {value[e]} - + ); })} @@ -120,14 +128,19 @@ function ListAlert({ allAlertRules, refetch }: ListAlertProps): JSX.Element { title: 'Action', dataIndex: 'id', key: 'action', - render: (id: Alerts['id']): JSX.Element => { + render: (id: GettableAlert['id'], record): JSX.Element => { return ( <> - + - + + + ); }, @@ -159,8 +172,10 @@ function ListAlert({ allAlertRules, refetch }: ListAlertProps): JSX.Element { } interface ListAlertProps { - allAlertRules: Alerts[]; - refetch: UseQueryResult>['refetch']; + allAlertRules: GettableAlert[]; + refetch: UseQueryResult< + ErrorResponse | SuccessResponse + >['refetch']; } export default ListAlert; diff --git a/frontend/src/container/ListAlertRules/TableComponents/Status.tsx b/frontend/src/container/ListAlertRules/TableComponents/Status.tsx index 33de5fb1db..d935b8d5ba 100644 --- a/frontend/src/container/ListAlertRules/TableComponents/Status.tsx +++ b/frontend/src/container/ListAlertRules/TableComponents/Status.tsx @@ -1,6 +1,6 @@ import { Tag } from 'antd'; import React from 'react'; -import { Alerts } from 'types/api/alerts/getAll'; +import { GettableAlert } from 'types/api/alerts/get'; function Status({ status }: StatusProps): JSX.Element { switch (status) { @@ -16,14 +16,18 @@ function Status({ status }: StatusProps): JSX.Element { return Firing; } + case 'disabled': { + return Disabled; + } + default: { - return Unknown Status; + return Unknown; } } } interface StatusProps { - status: Alerts['state']; + status: GettableAlert['state']; } export default Status; diff --git a/frontend/src/container/ListAlertRules/ToggleAlertState.tsx b/frontend/src/container/ListAlertRules/ToggleAlertState.tsx new file mode 100644 index 0000000000..9b367ea891 --- /dev/null +++ b/frontend/src/container/ListAlertRules/ToggleAlertState.tsx @@ -0,0 +1,108 @@ +import { notification } from 'antd'; +import patchAlert from 'api/alerts/patch'; +import { State } from 'hooks/useFetch'; +import React, { useState } from 'react'; +import { GettableAlert } from 'types/api/alerts/get'; +import { PayloadProps as PatchPayloadProps } from 'types/api/alerts/patch'; + +import { ColumnButton } from './styles'; + +function ToggleAlertState({ + id, + disabled, + setData, +}: ToggleAlertStateProps): JSX.Element { + const [apiStatus, setAPIStatus] = useState>({ + error: false, + errorMessage: '', + loading: false, + success: false, + payload: undefined, + }); + + const defaultErrorMessage = 'Something went wrong'; + + const onToggleHandler = async ( + id: number, + disabled: boolean, + ): Promise => { + try { + setAPIStatus((state) => ({ + ...state, + loading: true, + })); + + const response = await patchAlert({ + id, + data: { + disabled, + }, + }); + + if (response.statusCode === 200) { + setData((state) => { + return state.map((alert) => { + if (alert.id === id) { + return { + ...alert, + disabled: response.payload.disabled, + state: response.payload.state, + }; + } + return alert; + }); + }); + + setAPIStatus((state) => ({ + ...state, + loading: false, + payload: response.payload, + })); + notification.success({ + message: 'Success', + }); + } else { + setAPIStatus((state) => ({ + ...state, + loading: false, + error: true, + errorMessage: response.error || defaultErrorMessage, + })); + + notification.error({ + message: response.error || defaultErrorMessage, + }); + } + } catch (error) { + setAPIStatus((state) => ({ + ...state, + loading: false, + error: true, + errorMessage: defaultErrorMessage, + })); + + notification.error({ + message: defaultErrorMessage, + }); + } + }; + + return ( + => onToggleHandler(id, !disabled)} + type="link" + > + {disabled ? 'Enable' : 'Disable'} + + ); +} + +interface ToggleAlertStateProps { + id: GettableAlert['id']; + disabled: boolean; + setData: React.Dispatch>; +} + +export default ToggleAlertState; diff --git a/frontend/src/container/ListAlertRules/styles.ts b/frontend/src/container/ListAlertRules/styles.ts index fa993568fb..67748b21c0 100644 --- a/frontend/src/container/ListAlertRules/styles.ts +++ b/frontend/src/container/ListAlertRules/styles.ts @@ -1,4 +1,4 @@ -import { Button as ButtonComponent } from 'antd'; +import { Button as ButtonComponent, Tag } from 'antd'; import styled from 'styled-components'; export const ButtonContainer = styled.div` @@ -12,6 +12,20 @@ export const ButtonContainer = styled.div` export const Button = styled(ButtonComponent)` &&& { - margin-left: 1rem; + margin-left: 1em; + } +`; + +export const ColumnButton = styled(ButtonComponent)` + &&& { + padding-left: 0; + padding-right: 0; + margin-right: 1.5em; + } +`; + +export const StyledTag = styled(Tag)` + &&& { + white-space: normal; } `; diff --git a/frontend/src/container/MetricsApplication/Tabs/Overview.tsx b/frontend/src/container/MetricsApplication/Tabs/Overview.tsx index 803ed91bcc..2dbf2d33fd 100644 --- a/frontend/src/container/MetricsApplication/Tabs/Overview.tsx +++ b/frontend/src/container/MetricsApplication/Tabs/Overview.tsx @@ -15,7 +15,7 @@ import { PromQLWidgets } from 'types/api/dashboard/getAll'; import MetricReducer from 'types/reducer/metrics'; import { Card, Col, GraphContainer, GraphTitle, Row } from '../styles'; -import TopEndpointsTable from '../TopEndpointsTable'; +import TopOperationsTable from '../TopOperationsTable'; import { Button } from './styles'; function Application({ getWidget }: DashboardProps): JSX.Element { @@ -23,11 +23,13 @@ function Application({ getWidget }: DashboardProps): JSX.Element { const selectedTimeStamp = useRef(0); const { - topEndPoints, + topOperations, serviceOverview, resourceAttributePromQLQuery, resourceAttributeQueries, + topLevelOperations, } = useSelector((state) => state.metrics); + const operationsRegex = topLevelOperations.join('|'); const selectedTraceTags: string = JSON.stringify( convertRawQueriesToTraceSelectedTags(resourceAttributeQueries, 'array') || [], @@ -107,7 +109,7 @@ function Application({ getWidget }: DashboardProps): JSX.Element { - Application latency + Latency { - onClickHandler(ChartEvent, activeElements, chart, data, 'Application'); + onClickHandler(ChartEvent, activeElements, chart, data, 'Service'); }} - name="application_latency" + name="service_latency" type="line" data={{ datasets: [ @@ -175,7 +177,7 @@ function Application({ getWidget }: DashboardProps): JSX.Element { - Requests + Rate (ops/s) { - onClickHandler(event, element, chart, data, 'Request'); + onClickHandler(event, element, chart, data, 'Rate'); }} widget={getWidget([ { - query: `sum(rate(signoz_latency_count{service_name="${servicename}", span_kind="SPAN_KIND_SERVER"${resourceAttributePromQLQuery}}[5m]))`, - legend: 'Requests', + query: `sum(rate(signoz_latency_count{service_name="${servicename}", operation=~"${operationsRegex}"${resourceAttributePromQLQuery}}[5m]))`, + legend: 'Operations', }, ])} - yAxisUnit="reqps" + yAxisUnit="ops" /> @@ -227,7 +229,7 @@ function Application({ getWidget }: DashboardProps): JSX.Element { }} widget={getWidget([ { - query: `max(sum(rate(signoz_calls_total{service_name="${servicename}", span_kind="SPAN_KIND_SERVER", status_code="STATUS_CODE_ERROR"${resourceAttributePromQLQuery}}[5m]) OR rate(signoz_calls_total{service_name="${servicename}", span_kind="SPAN_KIND_SERVER", http_status_code=~"5.."${resourceAttributePromQLQuery}}[5m]))*100/sum(rate(signoz_calls_total{service_name="${servicename}", span_kind="SPAN_KIND_SERVER"${resourceAttributePromQLQuery}}[5m]))) < 1000 OR vector(0)`, + query: `max(sum(rate(signoz_calls_total{service_name="${servicename}", operation=~"${operationsRegex}", status_code="STATUS_CODE_ERROR"${resourceAttributePromQLQuery}}[5m]) OR rate(signoz_calls_total{service_name="${servicename}", operation=~"${operationsRegex}", http_status_code=~"5.."${resourceAttributePromQLQuery}}[5m]))*100/sum(rate(signoz_calls_total{service_name="${servicename}", operation=~"${operationsRegex}"${resourceAttributePromQLQuery}}[5m]))) < 1000 OR vector(0)`, legend: 'Error Percentage', }, ])} @@ -239,7 +241,7 @@ function Application({ getWidget }: DashboardProps): JSX.Element { - + diff --git a/frontend/src/container/MetricsApplication/TopEndpointsTable.tsx b/frontend/src/container/MetricsApplication/TopOperationsTable.tsx similarity index 89% rename from frontend/src/container/MetricsApplication/TopEndpointsTable.tsx rename to frontend/src/container/MetricsApplication/TopOperationsTable.tsx index 5ede2d9c6a..4f91a97781 100644 --- a/frontend/src/container/MetricsApplication/TopEndpointsTable.tsx +++ b/frontend/src/container/MetricsApplication/TopOperationsTable.tsx @@ -11,7 +11,7 @@ import { AppState } from 'store/reducers'; import { GlobalReducer } from 'types/reducer/globalTime'; import MetricReducer from 'types/reducer/metrics'; -function TopEndpointsTable(props: TopEndpointsTableProps): JSX.Element { +function TopOperationsTable(props: TopOperationsTableProps): JSX.Element { const { minTime, maxTime } = useSelector( (state) => state.globalTime, ); @@ -85,7 +85,7 @@ function TopEndpointsTable(props: TopEndpointsTableProps): JSX.Element { title: 'Number of Calls', dataIndex: 'numCalls', key: 'numCalls', - sorter: (a: TopEndpointListItem, b: TopEndpointListItem): number => + sorter: (a: TopOperationListItem, b: TopOperationListItem): number => a.numCalls - b.numCalls, }, ]; @@ -94,7 +94,7 @@ function TopEndpointsTable(props: TopEndpointsTableProps): JSX.Element { { - return 'Top Endpoints'; + return 'Key Operations'; }} tableLayout="fixed" dataSource={data} @@ -104,7 +104,7 @@ function TopEndpointsTable(props: TopEndpointsTableProps): JSX.Element { ); } -interface TopEndpointListItem { +interface TopOperationListItem { p50: number; p95: number; p99: number; @@ -112,10 +112,10 @@ interface TopEndpointListItem { name: string; } -type DataProps = TopEndpointListItem; +type DataProps = TopOperationListItem; -interface TopEndpointsTableProps { - data: TopEndpointListItem[]; +interface TopOperationsTableProps { + data: TopOperationListItem[]; } -export default TopEndpointsTable; +export default TopOperationsTable; diff --git a/frontend/src/container/MetricsTable/index.tsx b/frontend/src/container/MetricsTable/index.tsx index cc0778c80e..e81a7badfc 100644 --- a/frontend/src/container/MetricsTable/index.tsx +++ b/frontend/src/container/MetricsTable/index.tsx @@ -56,14 +56,14 @@ function Metrics(): JSX.Element { render: (value: number): string => (value / 1000000).toFixed(2), }, { - title: 'Error Rate (% of requests)', + title: 'Error Rate (% of total)', dataIndex: 'errorRate', key: 'errorRate', sorter: (a: DataProps, b: DataProps): number => a.errorRate - b.errorRate, render: (value: number): string => value.toFixed(2), }, { - title: 'Requests Per Second', + title: 'Operations Per Second', dataIndex: 'callRate', key: 'callRate', sorter: (a: DataProps, b: DataProps): number => a.callRate - b.callRate, diff --git a/frontend/src/container/TopNav/DateTimeSelection/config.ts b/frontend/src/container/TopNav/DateTimeSelection/config.ts index 69bdde40c7..59715d1f86 100644 --- a/frontend/src/container/TopNav/DateTimeSelection/config.ts +++ b/frontend/src/container/TopNav/DateTimeSelection/config.ts @@ -42,8 +42,9 @@ export interface Option { } export const ServiceMapOptions: Option[] = [ - { value: '1min', label: 'Last 1 min' }, { value: '5min', label: 'Last 5 min' }, + { value: '15min', label: 'Last 15 min' }, + { value: '30min', label: 'Last 30 min' }, ]; export const getDefaultOption = (route: string): Time => { diff --git a/frontend/src/container/TriggeredAlerts/Filter.tsx b/frontend/src/container/TriggeredAlerts/Filter.tsx index ae61fbc35a..601651cdff 100644 --- a/frontend/src/container/TriggeredAlerts/Filter.tsx +++ b/frontend/src/container/TriggeredAlerts/Filter.tsx @@ -2,7 +2,7 @@ import type { SelectProps } from 'antd'; import { Tag } from 'antd'; import React, { useCallback, useMemo } from 'react'; -import { Alerts } from 'types/api/alerts/getAll'; +import { Alerts } from 'types/api/alerts/getTriggered'; import { Container, Select } from './styles'; diff --git a/frontend/src/container/TriggeredAlerts/FilteredTable/ExapandableRow.tsx b/frontend/src/container/TriggeredAlerts/FilteredTable/ExapandableRow.tsx index fab66e242d..388e2d7499 100644 --- a/frontend/src/container/TriggeredAlerts/FilteredTable/ExapandableRow.tsx +++ b/frontend/src/container/TriggeredAlerts/FilteredTable/ExapandableRow.tsx @@ -2,7 +2,7 @@ import { Tag, Typography } from 'antd'; import convertDateToAmAndPm from 'lib/convertDateToAmAndPm'; import getFormattedDate from 'lib/getFormatedDate'; import React from 'react'; -import { Alerts } from 'types/api/alerts/getAll'; +import { Alerts } from 'types/api/alerts/getTriggered'; import Status from '../TableComponents/AlertStatus'; import { TableCell, TableRow } from './styles'; diff --git a/frontend/src/container/TriggeredAlerts/FilteredTable/TableRow.tsx b/frontend/src/container/TriggeredAlerts/FilteredTable/TableRow.tsx index 2f446adcf5..97619b5f12 100644 --- a/frontend/src/container/TriggeredAlerts/FilteredTable/TableRow.tsx +++ b/frontend/src/container/TriggeredAlerts/FilteredTable/TableRow.tsx @@ -1,7 +1,7 @@ import { MinusSquareOutlined, PlusSquareOutlined } from '@ant-design/icons'; import { Tag } from 'antd'; import React, { useState } from 'react'; -import { Alerts } from 'types/api/alerts/getAll'; +import { Alerts } from 'types/api/alerts/getTriggered'; import ExapandableRow from './ExapandableRow'; import { IconContainer, StatusContainer, TableCell, TableRow } from './styles'; diff --git a/frontend/src/container/TriggeredAlerts/FilteredTable/index.tsx b/frontend/src/container/TriggeredAlerts/FilteredTable/index.tsx index 8c8f47fdfd..a9e56d903d 100644 --- a/frontend/src/container/TriggeredAlerts/FilteredTable/index.tsx +++ b/frontend/src/container/TriggeredAlerts/FilteredTable/index.tsx @@ -1,6 +1,6 @@ import groupBy from 'lodash-es/groupBy'; import React, { useMemo } from 'react'; -import { Alerts } from 'types/api/alerts/getAll'; +import { Alerts } from 'types/api/alerts/getTriggered'; import { Value } from '../Filter'; import { FilterAlerts } from '../utils'; diff --git a/frontend/src/container/TriggeredAlerts/NoFilterTable.tsx b/frontend/src/container/TriggeredAlerts/NoFilterTable.tsx index a9c8064616..ac4e45131a 100644 --- a/frontend/src/container/TriggeredAlerts/NoFilterTable.tsx +++ b/frontend/src/container/TriggeredAlerts/NoFilterTable.tsx @@ -5,7 +5,7 @@ import AlertStatus from 'container/TriggeredAlerts/TableComponents/AlertStatus'; import convertDateToAmAndPm from 'lib/convertDateToAmAndPm'; import getFormattedDate from 'lib/getFormatedDate'; import React from 'react'; -import { Alerts } from 'types/api/alerts/getAll'; +import { Alerts } from 'types/api/alerts/getTriggered'; import { Value } from './Filter'; import { FilterAlerts } from './utils'; diff --git a/frontend/src/container/TriggeredAlerts/TriggeredAlert.tsx b/frontend/src/container/TriggeredAlerts/TriggeredAlert.tsx index 425334e7ba..b12a09d5e4 100644 --- a/frontend/src/container/TriggeredAlerts/TriggeredAlert.tsx +++ b/frontend/src/container/TriggeredAlerts/TriggeredAlert.tsx @@ -1,7 +1,7 @@ import getTriggeredApi from 'api/alerts/getTriggered'; import useInterval from 'hooks/useInterval'; import React, { useState } from 'react'; -import { Alerts } from 'types/api/alerts/getAll'; +import { Alerts } from 'types/api/alerts/getTriggered'; import Filter, { Value } from './Filter'; import FilteredTable from './FilteredTable'; diff --git a/frontend/src/container/TriggeredAlerts/utils.ts b/frontend/src/container/TriggeredAlerts/utils.ts index aab179e1cf..67d3024d6a 100644 --- a/frontend/src/container/TriggeredAlerts/utils.ts +++ b/frontend/src/container/TriggeredAlerts/utils.ts @@ -1,4 +1,4 @@ -import { Alerts } from 'types/api/alerts/getAll'; +import { Alerts } from 'types/api/alerts/getTriggered'; import { Value } from './Filter'; diff --git a/frontend/src/modules/Servicemap/ServiceMap.tsx b/frontend/src/modules/Servicemap/ServiceMap.tsx index 03256dde59..7bc44d0d5f 100644 --- a/frontend/src/modules/Servicemap/ServiceMap.tsx +++ b/frontend/src/modules/Servicemap/ServiceMap.tsx @@ -45,6 +45,9 @@ interface graphLink { source: string; target: string; value: number; + callRate: number; + errorRate: number; + p99: number; } export interface graphDataType { nodes: graphNode[]; @@ -96,16 +99,16 @@ function ServiceMap(props: ServiceMapProps): JSX.Element { const graphData = { nodes, links }; return ( - + /> */} d.target} linkDirectionalParticles="value" linkDirectionalParticleSpeed={(d) => d.value} @@ -124,7 +127,7 @@ function ServiceMap(props: ServiceMapProps): JSX.Element { ctx.fillStyle = isDarkMode ? '#ffffff' : '#000000'; ctx.fillText(label, node.x, node.y); }} - onNodeClick={(node) => { + onLinkHover={(node) => { const tooltip = document.querySelector('.graph-tooltip'); if (tooltip && node) { tooltip.innerHTML = getTooltip(node); diff --git a/frontend/src/modules/Servicemap/utils.ts b/frontend/src/modules/Servicemap/utils.ts index 6bec25f8a6..f1da9e3c3a 100644 --- a/frontend/src/modules/Servicemap/utils.ts +++ b/frontend/src/modules/Servicemap/utils.ts @@ -1,12 +1,13 @@ /*eslint-disable*/ //@ts-nocheck -import { cloneDeep, find, maxBy, uniq, uniqBy } from 'lodash-es'; +import { cloneDeep, find, maxBy, uniq, uniqBy, groupBy, sumBy } from 'lodash-es'; import { graphDataType } from './ServiceMap'; const MIN_WIDTH = 10; const MAX_WIDTH = 20; const DEFAULT_FONT_SIZE = 6; + export const getDimensions = (num, highest) => { const percentage = (num / highest) * 100; const width = (percentage * (MAX_WIDTH - MIN_WIDTH)) / 100 + MIN_WIDTH; @@ -18,19 +19,30 @@ export const getDimensions = (num, highest) => { }; export const getGraphData = (serviceMap, isDarkMode): graphDataType => { - const { items, services } = serviceMap; + const { items } = serviceMap; + const services = Object.values(groupBy(items, 'child')).map((e) => { + return { + serviceName: e[0].child, + errorRate: sumBy(e, 'errorRate'), + callRate: sumBy(e, 'callRate'), + } + }); const highestCallCount = maxBy(items, (e) => e?.callCount)?.callCount; const highestCallRate = maxBy(services, (e) => e?.callRate)?.callRate; + const divNum = Number( String(1).padEnd(highestCallCount.toString().length, '0'), ); const links = cloneDeep(items).map((node) => { - const { parent, child, callCount } = node; + const { parent, child, callCount, callRate, errorRate, p99 } = node; return { source: parent, target: child, value: (100 - callCount / divNum) * 0.03, + callRate, + errorRate, + p99, }; }); const uniqParent = uniqBy(cloneDeep(items), 'parent').map((e) => e.parent); @@ -47,15 +59,10 @@ export const getGraphData = (serviceMap, isDarkMode): graphDataType => { width: MIN_WIDTH, color, nodeVal: MIN_WIDTH, - callRate: 0, - errorRate: 0, - p99: 0, }; } if (service.errorRate > 0) { color = isDarkMode ? '#DB836E' : '#F98989'; - } else if (service.fourXXRate > 0) { - color = isDarkMode ? '#C79931' : '#F9DA7B'; } const { fontSize, width } = getDimensions(service.callRate, highestCallRate); return { @@ -65,9 +72,6 @@ export const getGraphData = (serviceMap, isDarkMode): graphDataType => { width, color, nodeVal: width, - callRate: service.callRate.toFixed(2), - errorRate: service.errorRate, - p99: service.p99, }; }); return { @@ -90,25 +94,31 @@ export const getZoomPx = (): number => { return 190; }; -export const getTooltip = (node: { +const getRound2DigitsAfterDecimal = (num: number) => { + if (num === 0) { + return 0; + } + return num.toFixed(20).match(/^-?\d*\.?0*\d{0,2}/)[0]; +} + +export const getTooltip = (link: { p99: number; errorRate: number; callRate: number; id: string; }) => { return `
-
${node.id}
P99 latency:
-
${node.p99 / 1000000}ms
+
${getRound2DigitsAfterDecimal(link.p99/ 1000000)}ms
Request:
-
${node.callRate}/sec
+
${getRound2DigitsAfterDecimal(link.callRate)}/sec
Error Rate:
-
${node.errorRate}%
+
${getRound2DigitsAfterDecimal(link.errorRate)}%
`; }; diff --git a/frontend/src/store/actions/metrics/getInitialData.ts b/frontend/src/store/actions/metrics/getInitialData.ts index f994a35c94..0f607f6ea5 100644 --- a/frontend/src/store/actions/metrics/getInitialData.ts +++ b/frontend/src/store/actions/metrics/getInitialData.ts @@ -3,7 +3,8 @@ // import getExternalError from 'api/metrics/getExternalError'; // import getExternalService from 'api/metrics/getExternalService'; import getServiceOverview from 'api/metrics/getServiceOverview'; -import getTopEndPoints from 'api/metrics/getTopEndPoints'; +import getTopLevelOperations from 'api/metrics/getTopLevelOperations'; +import getTopOperations from 'api/metrics/getTopOperations'; import { AxiosError } from 'axios'; import GetMinMax from 'lib/getMinMax'; import getStep from 'lib/getStep'; @@ -46,7 +47,8 @@ export const GetInitialData = ( // getExternalErrorResponse, // getExternalServiceResponse, getServiceOverviewResponse, - getTopEndPointsResponse, + getTopOperationsResponse, + getTopLevelOperationsResponse, ] = await Promise.all([ // getDBOverView({ // ...props, @@ -67,12 +69,15 @@ export const GetInitialData = ( step: getStep({ start: minTime, end: maxTime, inputFormat: 'ns' }), selectedTags: props.selectedTags, }), - getTopEndPoints({ + getTopOperations({ end: maxTime, service: props.serviceName, start: minTime, selectedTags: props.selectedTags, }), + getTopLevelOperations({ + service: props.serviceName, + }), ]); if ( @@ -81,7 +86,8 @@ export const GetInitialData = ( // getExternalErrorResponse.statusCode === 200 && // getExternalServiceResponse.statusCode === 200 && getServiceOverviewResponse.statusCode === 200 && - getTopEndPointsResponse.statusCode === 200 + getTopOperationsResponse.statusCode === 200 && + getTopLevelOperationsResponse.statusCode === 200 ) { dispatch({ type: 'GET_INTIAL_APPLICATION_DATA', @@ -91,7 +97,8 @@ export const GetInitialData = ( // externalError: getExternalErrorResponse.payload, // externalService: getExternalServiceResponse.payload, serviceOverview: getServiceOverviewResponse.payload, - topEndPoints: getTopEndPointsResponse.payload, + topOperations: getTopOperationsResponse.payload, + topLevelOperations: getTopLevelOperationsResponse.payload, }, }); } else { @@ -99,8 +106,9 @@ export const GetInitialData = ( type: 'GET_INITIAL_APPLICATION_ERROR', payload: { errorMessage: - getTopEndPointsResponse.error || + getTopOperationsResponse.error || getServiceOverviewResponse.error || + getTopLevelOperationsResponse.error || // getExternalServiceResponse.error || // getExternalErrorResponse.error || // getExternalAverageDurationResponse.error || diff --git a/frontend/src/store/actions/serviceMap.ts b/frontend/src/store/actions/serviceMap.ts index 36d8e5ba97..e3f527fc57 100644 --- a/frontend/src/store/actions/serviceMap.ts +++ b/frontend/src/store/actions/serviceMap.ts @@ -6,26 +6,16 @@ import { ActionTypes } from './types'; export interface ServiceMapStore { items: ServicesMapItem[]; - services: ServicesItem[]; loading: boolean; } -export interface ServicesItem { - serviceName: string; - p99: number; - avgDuration: number; - numCalls: number; - callRate: number; - numErrors: number; - errorRate: number; - num4XX: number; - fourXXRate: number; -} - export interface ServicesMapItem { parent: string; child: string; callCount: number; + callRate: number; + errorRate: number; + p99: number; } export interface ServiceMapItemAction { @@ -33,11 +23,6 @@ export interface ServiceMapItemAction { payload: ServicesMapItem[]; } -export interface ServicesAction { - type: ActionTypes.getServices; - payload: ServicesItem[]; -} - export interface ServiceMapLoading { type: ActionTypes.serviceMapLoading; payload: { @@ -55,19 +40,13 @@ export const getDetailedServiceMapItems = (globalTime: GlobalTime) => { end, tags: [], }; - const [serviceMapDependenciesResponse, response] = await Promise.all([ - api.post(`/serviceMapDependencies`, serviceMapPayload), - api.post(`/services`, serviceMapPayload), + const [dependencyGraphResponse] = await Promise.all([ + api.post(`/dependency_graph`, serviceMapPayload), ]); - dispatch({ - type: ActionTypes.getServices, - payload: response.data, - }); - dispatch({ type: ActionTypes.getServiceMapItems, - payload: serviceMapDependenciesResponse.data, + payload: dependencyGraphResponse.data, }); dispatch({ diff --git a/frontend/src/store/actions/types.ts b/frontend/src/store/actions/types.ts index 702997d49b..96d3f63538 100644 --- a/frontend/src/store/actions/types.ts +++ b/frontend/src/store/actions/types.ts @@ -1,8 +1,4 @@ -import { - ServiceMapItemAction, - ServiceMapLoading, - ServicesAction, -} from './serviceMap'; +import { ServiceMapItemAction, ServiceMapLoading } from './serviceMap'; import { GetUsageDataAction } from './usage'; export enum ActionTypes { @@ -17,6 +13,5 @@ export enum ActionTypes { export type Action = | GetUsageDataAction - | ServicesAction | ServiceMapItemAction | ServiceMapLoading; diff --git a/frontend/src/store/index.ts b/frontend/src/store/index.ts index 3ff983b82e..6796207357 100644 --- a/frontend/src/store/index.ts +++ b/frontend/src/store/index.ts @@ -18,4 +18,8 @@ const store = createStore( ), ); +if (window !== undefined) { + window.store = store; +} + export default store; diff --git a/frontend/src/store/reducers/global.ts b/frontend/src/store/reducers/global.ts index 084e7cd377..896c6586e3 100644 --- a/frontend/src/store/reducers/global.ts +++ b/frontend/src/store/reducers/global.ts @@ -10,7 +10,9 @@ const intitalState: GlobalReducer = { maxTime: Date.now() * 1000000, minTime: (Date.now() - 15 * 60 * 1000) * 1000000, loading: true, - selectedTime: getDefaultOption(window.location.pathname), + selectedTime: getDefaultOption( + typeof window !== 'undefined' ? window?.location?.pathname : '', + ), }; const globalTimeReducer = ( diff --git a/frontend/src/store/reducers/metric.ts b/frontend/src/store/reducers/metric.ts index 72b24a6b5b..2cb316d2c1 100644 --- a/frontend/src/store/reducers/metric.ts +++ b/frontend/src/store/reducers/metric.ts @@ -21,7 +21,7 @@ const InitialValue: InitialValueTypes = { services: [], dbOverView: [], externalService: [], - topEndPoints: [], + topOperations: [], externalAverageDuration: [], externalError: [], serviceOverview: [], @@ -29,6 +29,7 @@ const InitialValue: InitialValueTypes = { resourceAttributePromQLQuery: resourceAttributesQueryToPromQL( GetResourceAttributeQueriesFromURL() || [], ), + topLevelOperations: [], }; const metrics = ( @@ -88,22 +89,24 @@ const metrics = ( case GET_INTIAL_APPLICATION_DATA: { const { // dbOverView, - topEndPoints, + topOperations, serviceOverview, // externalService, // externalAverageDuration, // externalError, + topLevelOperations, } = action.payload; return { ...state, // dbOverView, - topEndPoints, + topOperations, serviceOverview, // externalService, // externalAverageDuration, // externalError, metricsApplicationLoading: false, + topLevelOperations, }; } diff --git a/frontend/src/store/reducers/serviceMap.ts b/frontend/src/store/reducers/serviceMap.ts index 18ec21a9ec..04b724615b 100644 --- a/frontend/src/store/reducers/serviceMap.ts +++ b/frontend/src/store/reducers/serviceMap.ts @@ -2,7 +2,6 @@ import { Action, ActionTypes, ServiceMapStore } from 'store/actions'; const initialState: ServiceMapStore = { items: [], - services: [], loading: true, }; @@ -16,11 +15,6 @@ export const ServiceMapReducer = ( ...state, items: action.payload, }; - case ActionTypes.getServices: - return { - ...state, - services: action.payload, - }; case ActionTypes.serviceMapLoading: { return { ...state, diff --git a/frontend/src/types/actions/metrics.ts b/frontend/src/types/actions/metrics.ts index 382e56b560..bc48f0929f 100644 --- a/frontend/src/types/actions/metrics.ts +++ b/frontend/src/types/actions/metrics.ts @@ -5,7 +5,7 @@ import { IResourceAttributeQuery } from 'container/MetricsApplication/ResourceAttributesFilter/types'; import { ServicesList } from 'types/api/metrics/getService'; import { ServiceOverview } from 'types/api/metrics/getServiceOverview'; -import { TopEndPoints } from 'types/api/metrics/getTopEndPoints'; +import { TopOperations } from 'types/api/metrics/getTopOperations'; export const GET_SERVICE_LIST_SUCCESS = 'GET_SERVICE_LIST_SUCCESS'; export const GET_SERVICE_LIST_LOADING_START = 'GET_SERVICE_LIST_LOADING_START'; @@ -38,12 +38,13 @@ export interface GetServiceListError { export interface GetInitialApplicationData { type: typeof GET_INTIAL_APPLICATION_DATA; payload: { - topEndPoints: TopEndPoints[]; + topOperations: TopOperations[]; // dbOverView: DBOverView[]; // externalService: ExternalService[]; // externalAverageDuration: ExternalAverageDuration[]; // externalError: ExternalError[]; serviceOverview: ServiceOverview[]; + topLevelOperations: string[]; }; } diff --git a/frontend/src/types/api/alerts/def.ts b/frontend/src/types/api/alerts/def.ts index 060bdc4d73..f417678ee1 100644 --- a/frontend/src/types/api/alerts/def.ts +++ b/frontend/src/types/api/alerts/def.ts @@ -18,6 +18,8 @@ export interface AlertDef { annotations?: Labels; evalWindow?: string; source?: string; + disabled?: boolean; + preferredChannels?: string[]; } export interface RuleCondition { diff --git a/frontend/src/types/api/alerts/delete.ts b/frontend/src/types/api/alerts/delete.ts index 24dbdc1d8a..5c842ea34c 100644 --- a/frontend/src/types/api/alerts/delete.ts +++ b/frontend/src/types/api/alerts/delete.ts @@ -1,7 +1,7 @@ -import { Alerts } from './getAll'; +import { AlertDef } from './def'; export interface Props { - id: Alerts['id']; + id: AlertDef['id']; } export interface PayloadProps { diff --git a/frontend/src/types/api/alerts/get.ts b/frontend/src/types/api/alerts/get.ts index 69eef474e1..78b637c140 100644 --- a/frontend/src/types/api/alerts/get.ts +++ b/frontend/src/types/api/alerts/get.ts @@ -4,6 +4,13 @@ export interface Props { id: AlertDef['id']; } +export interface GettableAlert extends AlertDef { + id: number; + alert: string; + state: string; + disabled: boolean; +} + export type PayloadProps = { - data: AlertDef; + data: GettableAlert; }; diff --git a/frontend/src/types/api/alerts/getAll.ts b/frontend/src/types/api/alerts/getAll.ts index 501c34a4cb..58351ed703 100644 --- a/frontend/src/types/api/alerts/getAll.ts +++ b/frontend/src/types/api/alerts/getAll.ts @@ -1,32 +1,3 @@ -export interface Alerts { - labels: AlertsLabel; - annotations: { - description: string; - summary: string; - [key: string]: string; - }; - state: string; - name: string; - id: number; - endsAt: string; - fingerprint: string; - generatorURL: string; - receivers: Receivers[]; - startsAt: string; - status: { - inhibitedBy: []; - silencedBy: []; - state: string; - }; - updatedAt: string; -} +import { GettableAlert } from './get'; -interface Receivers { - name: string; -} - -interface AlertsLabel { - [key: string]: string; -} - -export type PayloadProps = Alerts[]; +export type PayloadProps = GettableAlert[]; diff --git a/frontend/src/types/api/alerts/getGroups.ts b/frontend/src/types/api/alerts/getGroups.ts index f7dac48a14..71979d116d 100644 --- a/frontend/src/types/api/alerts/getGroups.ts +++ b/frontend/src/types/api/alerts/getGroups.ts @@ -1,4 +1,4 @@ -import { Alerts } from './getAll'; +import { AlertDef } from './def'; export interface Props { silenced: boolean; @@ -7,8 +7,8 @@ export interface Props { [key: string]: string | boolean; } export interface Group { - alerts: Alerts[]; - label: Alerts['labels']; + alerts: AlertDef[]; + label: AlertDef['labels']; receiver: { [key: string]: string; }; diff --git a/frontend/src/types/api/alerts/getTriggered.ts b/frontend/src/types/api/alerts/getTriggered.ts index 8b0e50a279..97d116b431 100644 --- a/frontend/src/types/api/alerts/getTriggered.ts +++ b/frontend/src/types/api/alerts/getTriggered.ts @@ -1,4 +1,33 @@ -import { Alerts } from './getAll'; +export interface Alerts { + labels: AlertsLabel; + annotations: { + description: string; + summary: string; + [key: string]: string; + }; + state: string; + name: string; + id: number; + endsAt: string; + fingerprint: string; + generatorURL: string; + receivers: Receivers[]; + startsAt: string; + status: { + inhibitedBy: []; + silencedBy: []; + state: string; + }; + updatedAt: string; +} + +interface Receivers { + name: string; +} + +interface AlertsLabel { + [key: string]: string; +} export interface Props { silenced: boolean; diff --git a/frontend/src/types/api/alerts/patch.ts b/frontend/src/types/api/alerts/patch.ts new file mode 100644 index 0000000000..fab1e67cfe --- /dev/null +++ b/frontend/src/types/api/alerts/patch.ts @@ -0,0 +1,12 @@ +import { GettableAlert } from './get'; + +export type PayloadProps = GettableAlert; + +export interface PatchProps { + disabled?: boolean; +} + +export interface Props { + id?: number; + data: PatchProps; +} diff --git a/frontend/src/types/api/alerts/testAlert.ts b/frontend/src/types/api/alerts/testAlert.ts new file mode 100644 index 0000000000..f0928275be --- /dev/null +++ b/frontend/src/types/api/alerts/testAlert.ts @@ -0,0 +1,10 @@ +import { AlertDef } from 'types/api/alerts/def'; + +export interface Props { + data: AlertDef; +} + +export interface PayloadProps { + alertCount: number; + message: string; +} diff --git a/frontend/src/types/api/metrics/getTopLevelOperations.ts b/frontend/src/types/api/metrics/getTopLevelOperations.ts new file mode 100644 index 0000000000..c4e88aed08 --- /dev/null +++ b/frontend/src/types/api/metrics/getTopLevelOperations.ts @@ -0,0 +1,7 @@ +export type TopLevelOperations = string[]; + +export interface Props { + service: string; +} + +export type PayloadProps = TopLevelOperations; diff --git a/frontend/src/types/api/metrics/getTopEndPoints.ts b/frontend/src/types/api/metrics/getTopOperations.ts similarity index 74% rename from frontend/src/types/api/metrics/getTopEndPoints.ts rename to frontend/src/types/api/metrics/getTopOperations.ts index c86d5fd115..f30c01251f 100644 --- a/frontend/src/types/api/metrics/getTopEndPoints.ts +++ b/frontend/src/types/api/metrics/getTopOperations.ts @@ -1,6 +1,6 @@ import { Tags } from 'types/reducer/trace'; -export interface TopEndPoints { +export interface TopOperations { name: string; numCalls: number; p50: number; @@ -15,4 +15,4 @@ export interface Props { selectedTags: Tags[]; } -export type PayloadProps = TopEndPoints[]; +export type PayloadProps = TopOperations[]; diff --git a/frontend/src/types/reducer/metrics.ts b/frontend/src/types/reducer/metrics.ts index d5b500f109..7903b2c21a 100644 --- a/frontend/src/types/reducer/metrics.ts +++ b/frontend/src/types/reducer/metrics.ts @@ -5,7 +5,7 @@ import { ExternalError } from 'types/api/metrics/getExternalError'; import { ExternalService } from 'types/api/metrics/getExternalService'; import { ServicesList } from 'types/api/metrics/getService'; import { ServiceOverview } from 'types/api/metrics/getServiceOverview'; -import { TopEndPoints } from 'types/api/metrics/getTopEndPoints'; +import { TopOperations } from 'types/api/metrics/getTopOperations'; interface MetricReducer { services: ServicesList[]; @@ -15,12 +15,13 @@ interface MetricReducer { errorMessage: string; dbOverView: DBOverView[]; externalService: ExternalService[]; - topEndPoints: TopEndPoints[]; + topOperations: TopOperations[]; externalAverageDuration: ExternalAverageDuration[]; externalError: ExternalError[]; serviceOverview: ServiceOverview[]; resourceAttributeQueries: IResourceAttributeQuery[]; resourceAttributePromQLQuery: string; + topLevelOperations: string[]; } export default MetricReducer; diff --git a/frontend/tests/expectionDetails/index.spec.ts b/frontend/tests/expectionDetails/index.spec.ts new file mode 100644 index 0000000000..f5dbb8c923 --- /dev/null +++ b/frontend/tests/expectionDetails/index.spec.ts @@ -0,0 +1,101 @@ +import { expect, Page, test } from '@playwright/test'; +import ROUTES from 'constants/routes'; + +import allErrorList from '../fixtures/api/allErrors/200.json'; +import errorDetailSuccess from '../fixtures/api/errorDetails/200.json'; +import errorDetailNotFound from '../fixtures/api/errorDetails/404.json'; +import nextPreviousSuccess from '../fixtures/api/getNextPrev/200.json'; +import { loginApi } from '../fixtures/common'; +import { JsonApplicationType } from '../fixtures/constant'; + +let page: Page; +const timestamp = '1657794588955274000'; + +test.describe('Expections Details', async () => { + test.beforeEach(async ({ baseURL, browser }) => { + const context = await browser.newContext({ storageState: 'tests/auth.json' }); + const newPage = await context.newPage(); + + await loginApi(newPage); + + await newPage.goto(`${baseURL}${ROUTES.APPLICATION}`); + + page = newPage; + }); + + test('Should have not found when api return 404', async () => { + await Promise.all([ + page.route('**/errorFromGroupID**', (route) => + route.fulfill({ + status: 404, + contentType: JsonApplicationType, + body: JSON.stringify(errorDetailNotFound), + }), + ), + page.route('**/nextPrevErrorIDs**', (route) => + route.fulfill({ + status: 404, + contentType: JsonApplicationType, + body: JSON.stringify([]), + }), + ), + ]); + + await page.goto( + `${ROUTES.ERROR_DETAIL}?groupId=${allErrorList[0].groupID}×tamp=${timestamp}`, + { + waitUntil: 'networkidle', + }, + ); + + const NoDataLocator = page.locator('text=Not Found'); + const isVisible = await NoDataLocator.isVisible(); + const text = await NoDataLocator.textContent(); + + expect(isVisible).toBe(true); + expect(text).toBe('Not Found'); + expect(await page.screenshot()).toMatchSnapshot(); + }); + + test('Render Success Data when 200 from details page', async () => { + await Promise.all([ + page.route('**/errorFromGroupID**', (route) => + route.fulfill({ + status: 200, + contentType: JsonApplicationType, + body: JSON.stringify(errorDetailSuccess), + }), + ), + page.route('**/nextPrevErrorIDs**', (route) => + route.fulfill({ + status: 200, + contentType: JsonApplicationType, + body: JSON.stringify(nextPreviousSuccess), + }), + ), + ]); + + await page.goto( + `${ROUTES.ERROR_DETAIL}?groupId=${allErrorList[0].groupID}×tamp=${timestamp}`, + { + waitUntil: 'networkidle', + }, + ); + + const traceDetailButton = page.locator('text=See the error in trace graph'); + const olderButton = page.locator('text=Older'); + const newerButton = page.locator(`text=Newer`); + + expect(await traceDetailButton.isVisible()).toBe(true); + expect(await olderButton.isVisible()).toBe(true); + expect(await newerButton.isVisible()).toBe(true); + + expect(await traceDetailButton.textContent()).toBe( + 'See the error in trace graph', + ); + expect(await olderButton.textContent()).toBe('Older'); + expect(await newerButton.textContent()).toBe('Newer'); + + expect(await page.screenshot()).toMatchSnapshot(); + }); +}); diff --git a/frontend/tests/expectionDetails/index.spec.ts-snapshots/Expections-Details-Render-Success-Data-when-200-from-details-page-1-Signoz-darwin.png b/frontend/tests/expectionDetails/index.spec.ts-snapshots/Expections-Details-Render-Success-Data-when-200-from-details-page-1-Signoz-darwin.png new file mode 100644 index 0000000000..ed84333d67 Binary files /dev/null and b/frontend/tests/expectionDetails/index.spec.ts-snapshots/Expections-Details-Render-Success-Data-when-200-from-details-page-1-Signoz-darwin.png differ diff --git a/frontend/tests/expectionDetails/index.spec.ts-snapshots/Expections-Details-Should-have-not-found-when-api-return-404-1-Signoz-darwin.png b/frontend/tests/expectionDetails/index.spec.ts-snapshots/Expections-Details-Should-have-not-found-when-api-return-404-1-Signoz-darwin.png new file mode 100644 index 0000000000..703767c1b0 Binary files /dev/null and b/frontend/tests/expectionDetails/index.spec.ts-snapshots/Expections-Details-Should-have-not-found-when-api-return-404-1-Signoz-darwin.png differ diff --git a/frontend/tests/expections/index.spec.ts b/frontend/tests/expections/index.spec.ts new file mode 100644 index 0000000000..ecf790b7ed --- /dev/null +++ b/frontend/tests/expections/index.spec.ts @@ -0,0 +1,148 @@ +import { expect, Page, test } from '@playwright/test'; +import ROUTES from 'constants/routes'; + +import successAllErrors from '../fixtures/api/allErrors/200.json'; +import { loginApi } from '../fixtures/common'; +import { JsonApplicationType } from '../fixtures/constant'; + +const noDataTableData = async (page: Page): Promise => { + const text = page.locator('text=No Data'); + + expect(text).toBeVisible(); + expect(text).toHaveText('No Data'); + + const textType = [ + 'Exception Type', + 'Error Message', + 'Last Seen', + 'First Seen', + 'Application', + ]; + + textType.forEach(async (text) => { + const textLocator = page.locator(`text=${text}`); + + const textContent = await textLocator.textContent(); + + expect(textContent).toBe(text); + expect(textLocator).not.toBeNull(); + + expect(textLocator).toBeVisible(); + await expect(textLocator).toHaveText(`${text}`); + }); +}; + +let page: Page; + +test.describe('Expections page', async () => { + test.beforeEach(async ({ baseURL, browser }) => { + const context = await browser.newContext({ storageState: 'tests/auth.json' }); + const newPage = await context.newPage(); + + await loginApi(newPage); + + await newPage.goto(`${baseURL}${ROUTES.APPLICATION}`); + + page = newPage; + }); + + test('Should have a valid route', async () => { + await page.goto(ROUTES.ALL_ERROR); + + await expect(page).toHaveURL(ROUTES.ALL_ERROR); + expect(await page.screenshot()).toMatchSnapshot(); + }); + + test('Should have a valid Breadcrumbs', async () => { + await page.goto(ROUTES.ALL_ERROR, { + waitUntil: 'networkidle', + }); + + const expectionsLocator = page.locator('a:has-text("Exceptions")'); + + await expect(expectionsLocator).toBeVisible(); + await expect(expectionsLocator).toHaveText('Exceptions'); + await expect(expectionsLocator).toHaveAttribute('href', ROUTES.ALL_ERROR); + expect(await page.screenshot()).toMatchSnapshot(); + }); + + test('Should render the page with 404 status', async () => { + await page.route('**listErrors', (route) => + route.fulfill({ + status: 404, + contentType: JsonApplicationType, + body: JSON.stringify([]), + }), + ); + + await page.goto(ROUTES.ALL_ERROR, { + waitUntil: 'networkidle', + }); + + await noDataTableData(page); + expect(await page.screenshot()).toMatchSnapshot(); + }); + + test('Should render the page with 500 status in antd notification with no data antd table', async () => { + await page.route(`**/listErrors**`, (route) => + route.fulfill({ + status: 500, + contentType: JsonApplicationType, + body: JSON.stringify([]), + }), + ); + + await page.goto(ROUTES.ALL_ERROR, { + waitUntil: 'networkidle', + }); + + const text = 'Something went wrong'; + + const el = page.locator(`text=${text}`); + + expect(el).toBeVisible(); + expect(el).toHaveText(`${text}`); + expect(await el.getAttribute('disabled')).toBe(null); + + await noDataTableData(page); + expect(await page.screenshot()).toMatchSnapshot(); + }); + + test('Should render data in antd table', async () => { + await Promise.all([ + page.route(`**/listErrors**`, (route) => + route.fulfill({ + status: 200, + contentType: JsonApplicationType, + body: JSON.stringify(successAllErrors), + }), + ), + + page.route('**/countErrors**', (route) => + route.fulfill({ + status: 200, + contentType: JsonApplicationType, + body: JSON.stringify(200), + }), + ), + ]); + + await page.goto(ROUTES.ALL_ERROR, { + waitUntil: 'networkidle', + }); + + await page.evaluate(() => window.scrollTo(0, document.body.scrollHeight)); + + const expectionType = page.locator( + `td:has-text("${successAllErrors[1].exceptionType}")`, + ); + + expect(expectionType).toBeVisible(); + + const second = page.locator('li > a:has-text("2") >> nth=0'); + const isVisisble = await second.isVisible(); + + expect(isVisisble).toBe(true); + expect(await page.screenshot()).toMatchSnapshot(); + }); +}); diff --git a/frontend/tests/expections/index.spec.ts-snapshots/Expections-page-Should-have-a-valid-Breadcrumbs-1-Signoz-darwin.png b/frontend/tests/expections/index.spec.ts-snapshots/Expections-page-Should-have-a-valid-Breadcrumbs-1-Signoz-darwin.png new file mode 100644 index 0000000000..a482ae42a6 Binary files /dev/null and b/frontend/tests/expections/index.spec.ts-snapshots/Expections-page-Should-have-a-valid-Breadcrumbs-1-Signoz-darwin.png differ diff --git a/frontend/tests/expections/index.spec.ts-snapshots/Expections-page-Should-have-a-valid-route-1-Signoz-darwin.png b/frontend/tests/expections/index.spec.ts-snapshots/Expections-page-Should-have-a-valid-route-1-Signoz-darwin.png new file mode 100644 index 0000000000..fc6187b788 Binary files /dev/null and b/frontend/tests/expections/index.spec.ts-snapshots/Expections-page-Should-have-a-valid-route-1-Signoz-darwin.png differ diff --git a/frontend/tests/expections/index.spec.ts-snapshots/Expections-page-Should-render-data-in-antd-table-1-Signoz-darwin.png b/frontend/tests/expections/index.spec.ts-snapshots/Expections-page-Should-render-data-in-antd-table-1-Signoz-darwin.png new file mode 100644 index 0000000000..01e97f5f70 Binary files /dev/null and b/frontend/tests/expections/index.spec.ts-snapshots/Expections-page-Should-render-data-in-antd-table-1-Signoz-darwin.png differ diff --git a/frontend/tests/expections/index.spec.ts-snapshots/Expections-page-Should-render-the-page-with-404-status-1-Signoz-darwin.png b/frontend/tests/expections/index.spec.ts-snapshots/Expections-page-Should-render-the-page-with-404-status-1-Signoz-darwin.png new file mode 100644 index 0000000000..a482ae42a6 Binary files /dev/null and b/frontend/tests/expections/index.spec.ts-snapshots/Expections-page-Should-render-the-page-with-404-status-1-Signoz-darwin.png differ diff --git a/frontend/tests/expections/index.spec.ts-snapshots/Expections-page-Should-render-the-page-with-50-26a88--in-antd-notification-with-no-data-antd-table-1-Signoz-darwin.png b/frontend/tests/expections/index.spec.ts-snapshots/Expections-page-Should-render-the-page-with-50-26a88--in-antd-notification-with-no-data-antd-table-1-Signoz-darwin.png new file mode 100644 index 0000000000..b02465a93c Binary files /dev/null and b/frontend/tests/expections/index.spec.ts-snapshots/Expections-page-Should-render-the-page-with-50-26a88--in-antd-notification-with-no-data-antd-table-1-Signoz-darwin.png differ diff --git a/frontend/tests/fixtures/api/allErrors/200.json b/frontend/tests/fixtures/api/allErrors/200.json new file mode 100644 index 0000000000..d91c7ad061 --- /dev/null +++ b/frontend/tests/fixtures/api/allErrors/200.json @@ -0,0 +1,92 @@ +[ + { + "exceptionType": "ConnectionError", + "exceptionMessage": "HTTPSConnectionPool(host='run.mocekdy.io', port=443): Max retries exceeded with url: /v3/1cwb67153-a6ac-4aae-aca6-273ed68b5d9e (Caused by NewConnectionError('\u003curllib3.connection.HTTPSConnection object at 0x108ce9c10\u003e: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))", + "exceptionCount": 2, + "lastSeen": "2022-07-14T10:29:48.955274Z", + "firstSeen": "2022-07-14T10:29:48.950721Z", + "serviceName": "1rfflaskAp", + "groupID": "e24d35bda98c5499a5c8df3ba61b0238" + }, + { + "exceptionType": "NameError", + "exceptionMessage": "name 'listf' is not defined", + "exceptionCount": 8, + "lastSeen": "2022-07-14T10:30:42.411035Z", + "firstSeen": "2022-07-14T10:29:45.426784Z", + "serviceName": "1rfflaskAp", + "groupID": "efc46adcd5e87b65f8f244cba683b265" + }, + { + "exceptionType": "ZeroDivisionError", + "exceptionMessage": "division by zero", + "exceptionCount": 1, + "lastSeen": "2022-07-14T10:29:54.195996Z", + "firstSeen": "2022-07-14T10:29:54.195996Z", + "serviceName": "1rfflaskAp", + "groupID": "a49058b540eef9aefe159d84f1a2b6df" + }, + { + "exceptionType": "MaxRetryError", + "exceptionMessage": "HTTPSConnectionPool(host='rufn.fmoceky.io', port=443): Max retries exceeded with url: /v3/b851a5c6-ab54-495a-be04-69834ae0d2a7 (Caused by NewConnectionError('\u003curllib3.connection.HTTPSConnection object at 0x108ec2640\u003e: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))", + "exceptionCount": 1, + "lastSeen": "2022-07-14T10:29:49.471402Z", + "firstSeen": "2022-07-14T10:29:49.471402Z", + "serviceName": "1rfflaskAp", + "groupID": "e59d39239f4d48842d83e3cc4cf53249" + }, + { + "exceptionType": "MaxRetryError", + "exceptionMessage": "HTTPSConnectionPool(host='run.mocekdy.io', port=443): Max retries exceeded with url: /v3/1cwb67153-a6ac-4aae-aca6-273ed68b5d9e (Caused by NewConnectionError('\u003curllib3.connection.HTTPSConnection object at 0x108ce9c10\u003e: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))", + "exceptionCount": 1, + "lastSeen": "2022-07-14T10:29:48.947579Z", + "firstSeen": "2022-07-14T10:29:48.947579Z", + "serviceName": "1rfflaskAp", + "groupID": "14d18a6fb1cd3f541de1566530e75486" + }, + { + "exceptionType": "ConnectionError", + "exceptionMessage": "HTTPSConnectionPool(host='rufn.fmoceky.io', port=443): Max retries exceeded with url: /v3/b851a5c6-ab54-495a-be04-69834ae0d2a7 (Caused by NewConnectionError('\u003curllib3.connection.HTTPSConnection object at 0x108ec2640\u003e: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))", + "exceptionCount": 2, + "lastSeen": "2022-07-14T10:29:49.476718Z", + "firstSeen": "2022-07-14T10:29:49.472271Z", + "serviceName": "1rfflaskAp", + "groupID": "bf6d88d10397ca3194b96a10f4719031" + }, + { + "exceptionType": "github.com/gin-gonic/gin.Error", + "exceptionMessage": "Sample Error", + "exceptionCount": 6, + "lastSeen": "2022-07-15T18:55:32.3538096Z", + "firstSeen": "2022-07-14T14:47:19.874387Z", + "serviceName": "goApp", + "groupID": "b4fd099280072d45318e1523d82aa9c1" + }, + { + "exceptionType": "MaxRetryError", + "exceptionMessage": "HTTPSConnectionPool(host='rufn.fmoceky.io', port=443): Max retries exceeded with url: /v3/b851a5c6-ab54-495a-be04-69834ae0d2a7 (Caused by NewConnectionError('\u003curllib3.connection.HTTPSConnection object at 0x10801b490\u003e: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))", + "exceptionCount": 1, + "lastSeen": "2022-07-14T11:07:06.560593Z", + "firstSeen": "2022-07-14T11:07:06.560593Z", + "serviceName": "samplFlaskApp", + "groupID": "1945671c945b10641e73b0fe28c4d486" + }, + { + "exceptionType": "ConnectionError", + "exceptionMessage": "HTTPSConnectionPool(host='rufn.fmoceky.io', port=443): Max retries exceeded with url: /v3/b851a5c6-ab54-495a-be04-69834ae0d2a7 (Caused by NewConnectionError('\u003curllib3.connection.HTTPSConnection object at 0x10801b490\u003e: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))", + "exceptionCount": 2, + "lastSeen": "2022-07-14T11:07:06.56493Z", + "firstSeen": "2022-07-14T11:07:06.561074Z", + "serviceName": "samplFlaskApp", + "groupID": "5bea5295cac187404005f9c96e71aa53" + }, + { + "exceptionType": "ConnectionError", + "exceptionMessage": "HTTPSConnectionPool(host='rufn.fmoceky.io', port=443): Max retries exceeded with url: /v3/b851a5c6-ab54-495a-be04-69834ae0d2a7 (Caused by NewConnectionError('\u003curllib3.connection.HTTPSConnection object at 0x108031820\u003e: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))", + "exceptionCount": 2, + "lastSeen": "2022-07-14T11:07:06.363977Z", + "firstSeen": "2022-07-14T11:07:06.361163Z", + "serviceName": "samplFlaskApp", + "groupID": "52a1fbe033453d806c0f24ba39168a78" + } +] diff --git a/frontend/tests/fixtures/api/errorDetails/200.json b/frontend/tests/fixtures/api/errorDetails/200.json new file mode 100644 index 0000000000..09f9bfe6cc --- /dev/null +++ b/frontend/tests/fixtures/api/errorDetails/200.json @@ -0,0 +1,12 @@ +{ + "errorId": "56c8572c51e94bc9a2f501a81390a054", + "exceptionEscaped": false, + "exceptionMessage": "HTTPSConnectionPool(host='run.mocekdy.io', port=443): Max retries exceeded with url: /v3/1cwb67153-a6ac-4aae-aca6-273ed68b5d9e (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))", + "exceptionStacktrace": "Traceback (most recent call last):\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n conn = connection.create_connection(\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socket.py\", line 954, in getaddrinfo\n for res in _socket.getaddrinfo(host, port, family, type, proto, flags):\nsocket.gaierror: [Errno 8] nodename nor servname provided, or not known\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 699, in urlopen\n httplib_response = self._make_request(\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 382, in _make_request\n self._validate_conn(conn)\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 1010, in _validate_conn\n conn.connect()\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/urllib3/connection.py\", line 358, in connect\n conn = self._new_conn()\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n raise NewConnectionError(\nurllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/requests/adapters.py\", line 439, in send\n resp = conn.urlopen(\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/opentelemetry/instrumentation/urllib3/__init__.py\", line 181, in instrumented_urlopen\n response = wrapped(*args, **kwargs)\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 755, in urlopen\n retries = retries.increment(\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/urllib3/util/retry.py\", line 574, in increment\n raise MaxRetryError(_pool, url, error or ResponseError(cause))\nurllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='run.mocekdy.io', port=443): Max retries exceeded with url: /v3/1cwb67153-a6ac-4aae-aca6-273ed68b5d9e (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/opentelemetry/trace/__init__.py\", line 541, in use_span\n yield span\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/flask/app.py\", line 2073, in wsgi_app\n response = self.full_dispatch_request()\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/flask/app.py\", line 1518, in full_dispatch_request\n rv = self.handle_user_exception(e)\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/flask/app.py\", line 1516, in full_dispatch_request\n rv = self.dispatch_request()\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/flask/app.py\", line 1502, in dispatch_request\n return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)\n File \"/Users/makeavish/signoz/sample-flask-app/app.py\", line 45, in lists\n response = requests.get('https://run.mocekdy.io/v3/1cwb67153-a6ac-4aae-aca6-273ed68b5d9e')\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/requests/api.py\", line 75, in get\n return request('get', url, params=params, **kwargs)\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/requests/api.py\", line 61, in request\n return session.request(method=method, url=url, **kwargs)\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/opentelemetry/instrumentation/requests/__init__.py\", line 122, in instrumented_request\n return _instrumented_requests_call(\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/opentelemetry/instrumentation/requests/__init__.py\", line 199, in _instrumented_requests_call\n raise exception.with_traceback(exception.__traceback__)\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/opentelemetry/instrumentation/requests/__init__.py\", line 180, in _instrumented_requests_call\n result = call_wrapped() # *** PROCEED\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/opentelemetry/instrumentation/requests/__init__.py\", line 120, in call_wrapped\n return wrapped_request(self, method, url, *args, **kwargs)\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/requests/sessions.py\", line 542, in request\n resp = self.send(prep, **send_kwargs)\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/opentelemetry/instrumentation/requests/__init__.py\", line 142, in instrumented_send\n return _instrumented_requests_call(\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/opentelemetry/instrumentation/requests/__init__.py\", line 152, in _instrumented_requests_call\n return call_wrapped()\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/opentelemetry/instrumentation/requests/__init__.py\", line 140, in call_wrapped\n return wrapped_send(self, request, **kwargs)\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/requests/sessions.py\", line 655, in send\n r = adapter.send(request, **kwargs)\n File \"/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/requests/adapters.py\", line 516, in send\n raise ConnectionError(e, request=request)\nrequests.exceptions.ConnectionError: HTTPSConnectionPool(host='run.mocekdy.io', port=443): Max retries exceeded with url: /v3/1cwb67153-a6ac-4aae-aca6-273ed68b5d9e (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))\n", + "exceptionType": "ConnectionError", + "groupID": "e24d35bda98c5499a5c8df3ba61b0238", + "serviceName": "1rfflaskAp", + "spanID": "d5c8f8e860e77255", + "timestamp": "2022-07-14T10:29:48.955274Z", + "traceID": "2a149caa7415389aa25e7c80f1527c9c" +} diff --git a/frontend/tests/fixtures/api/errorDetails/404.json b/frontend/tests/fixtures/api/errorDetails/404.json new file mode 100644 index 0000000000..dd843ec7e1 --- /dev/null +++ b/frontend/tests/fixtures/api/errorDetails/404.json @@ -0,0 +1,5 @@ +{ + "error": "Error/Exception not found", + "errorType": "not_found", + "status": "error" +} diff --git a/frontend/tests/fixtures/api/getNextPrev/200.json b/frontend/tests/fixtures/api/getNextPrev/200.json new file mode 100644 index 0000000000..b98c94a369 --- /dev/null +++ b/frontend/tests/fixtures/api/getNextPrev/200.json @@ -0,0 +1,7 @@ +{ + "nextErrorID": "", + "nextTimestamp": "0001-01-01T00:00:00Z", + "prevErrorID": "217133e5f7df429abd31b507859ea513", + "prevTimestamp": "2022-07-14T10:29:48.950721Z", + "groupID": "e24d35bda98c5499a5c8df3ba61b0238" +} diff --git a/frontend/tests/fixtures/constant.ts b/frontend/tests/fixtures/constant.ts index ac20029c4a..525ed5a0ec 100644 --- a/frontend/tests/fixtures/constant.ts +++ b/frontend/tests/fixtures/constant.ts @@ -6,3 +6,5 @@ export const validPassword = 'SamplePassword98@@'; export const getStartedButtonSelector = 'button[data-attr="signup"]'; export const confirmPasswordSelector = '#password-confirm-error'; + +export const JsonApplicationType = 'application/json'; diff --git a/frontend/tests/login/fail.spec.ts b/frontend/tests/login/fail.spec.ts index 5366d7240c..760556c95e 100644 --- a/frontend/tests/login/fail.spec.ts +++ b/frontend/tests/login/fail.spec.ts @@ -24,5 +24,6 @@ test.describe('Version API fail while loading login page', async () => { expect(el).toBeVisible(); expect(el).toHaveText(`${text}`); expect(await el.getAttribute('disabled')).toBe(null); + expect(await page.screenshot()).toMatchSnapshot(); }); }); diff --git a/frontend/tests/login/fail.spec.ts-snapshots/Version-API-fail-while-loading-login-page-Something-went-wrong-1-Signoz-darwin.png b/frontend/tests/login/fail.spec.ts-snapshots/Version-API-fail-while-loading-login-page-Something-went-wrong-1-Signoz-darwin.png new file mode 100644 index 0000000000..c969d1f3d4 Binary files /dev/null and b/frontend/tests/login/fail.spec.ts-snapshots/Version-API-fail-while-loading-login-page-Something-went-wrong-1-Signoz-darwin.png differ diff --git a/frontend/tests/login/index.spec.ts b/frontend/tests/login/index.spec.ts index ec735460ab..24a205e197 100644 --- a/frontend/tests/login/index.spec.ts +++ b/frontend/tests/login/index.spec.ts @@ -45,5 +45,6 @@ test.describe('Login Page', () => { element.isVisible(); const text = await element.innerText(); expect(text).toBe(`SigNoz ${version}`); + expect(await page.screenshot()).toMatchSnapshot(); }); }); diff --git a/frontend/tests/login/index.spec.ts-snapshots/Login-Page-Version-of-the-application-when-api-returns-200-1-Signoz-darwin.png b/frontend/tests/login/index.spec.ts-snapshots/Login-Page-Version-of-the-application-when-api-returns-200-1-Signoz-darwin.png new file mode 100644 index 0000000000..b3ce5b590a Binary files /dev/null and b/frontend/tests/login/index.spec.ts-snapshots/Login-Page-Version-of-the-application-when-api-returns-200-1-Signoz-darwin.png differ diff --git a/frontend/tests/service/index.spec.ts b/frontend/tests/service/index.spec.ts index ae708322ed..5e90209fbe 100644 --- a/frontend/tests/service/index.spec.ts +++ b/frontend/tests/service/index.spec.ts @@ -16,7 +16,17 @@ test.describe('Service Page', () => { page = newPage; }); + test('Serice Page is rendered', async ({ baseURL }) => { await expect(page).toHaveURL(`${baseURL}${ROUTES.APPLICATION}`); + expect(await page.screenshot()).toMatchSnapshot(); + }); + + test('Logged In must be true', async () => { + const { app } = await page.evaluate(() => window.store.getState()); + + const { isLoggedIn } = app; + + expect(isLoggedIn).toBe(true); }); }); diff --git a/frontend/tests/service/index.spec.ts-snapshots/Service-Page-Serice-Page-is-rendered-1-Signoz-darwin.png b/frontend/tests/service/index.spec.ts-snapshots/Service-Page-Serice-Page-is-rendered-1-Signoz-darwin.png new file mode 100644 index 0000000000..2be4376847 Binary files /dev/null and b/frontend/tests/service/index.spec.ts-snapshots/Service-Page-Serice-Page-is-rendered-1-Signoz-darwin.png differ diff --git a/frontend/tests/signup/index.spec.ts b/frontend/tests/signup/index.spec.ts index afdc98f140..ed0f16047d 100644 --- a/frontend/tests/signup/index.spec.ts +++ b/frontend/tests/signup/index.spec.ts @@ -77,6 +77,7 @@ test.describe('Sign Up Page', () => { await buttonSignupButton.click(); expect(page).toHaveURL(`${baseURL}${ROUTES.SIGN_UP}`); + expect(await page.screenshot()).toMatchSnapshot(); }); test('Invite link validation', async ({ baseURL, page }) => { @@ -87,6 +88,7 @@ test.describe('Sign Up Page', () => { const messageText = await page.locator(`text=${message}`).innerText(); expect(messageText).toBe(message); + expect(await page.screenshot()).toMatchSnapshot(); }); test('User Sign up with valid details', async ({ baseURL, page, context }) => { @@ -125,6 +127,7 @@ test.describe('Sign Up Page', () => { await context.storageState({ path: 'tests/auth.json', }); + expect(await page.screenshot()).toMatchSnapshot(); }); test('Empty name with valid details', async ({ baseURL, page }) => { @@ -142,6 +145,7 @@ test.describe('Sign Up Page', () => { const gettingStartedButton = page.locator(getStartedButtonSelector); expect(await gettingStartedButton.isDisabled()).toBe(true); + expect(await page.screenshot()).toMatchSnapshot(); }); test('Empty Company name with valid details', async ({ baseURL, page }) => { @@ -159,6 +163,7 @@ test.describe('Sign Up Page', () => { const gettingStartedButton = page.locator(getStartedButtonSelector); expect(await gettingStartedButton.isDisabled()).toBe(true); + expect(await page.screenshot()).toMatchSnapshot(); }); test('Empty Email with valid details', async ({ baseURL, page }) => { @@ -176,6 +181,7 @@ test.describe('Sign Up Page', () => { const gettingStartedButton = page.locator(getStartedButtonSelector); expect(await gettingStartedButton.isDisabled()).toBe(true); + expect(await page.screenshot()).toMatchSnapshot(); }); test('Empty Password and confirm password with valid details', async ({ @@ -200,6 +206,7 @@ test.describe('Sign Up Page', () => { // password validation message is not present const locator = await page.locator(confirmPasswordSelector).isVisible(); expect(locator).toBe(false); + expect(await page.screenshot()).toMatchSnapshot(); }); test('Miss Match Password and confirm password with valid details', async ({ @@ -220,5 +227,6 @@ test.describe('Sign Up Page', () => { // password validation message is not present const locator = await page.locator(confirmPasswordSelector).isVisible(); expect(locator).toBe(true); + expect(await page.screenshot()).toMatchSnapshot(); }); }); diff --git a/frontend/tests/signup/index.spec.ts-snapshots/Sign-Up-Page-Empty-Company-name-with-valid-details-1-Signoz-darwin.png b/frontend/tests/signup/index.spec.ts-snapshots/Sign-Up-Page-Empty-Company-name-with-valid-details-1-Signoz-darwin.png new file mode 100644 index 0000000000..82de95c0d1 Binary files /dev/null and b/frontend/tests/signup/index.spec.ts-snapshots/Sign-Up-Page-Empty-Company-name-with-valid-details-1-Signoz-darwin.png differ diff --git a/frontend/tests/signup/index.spec.ts-snapshots/Sign-Up-Page-Empty-Email-with-valid-details-1-Signoz-darwin.png b/frontend/tests/signup/index.spec.ts-snapshots/Sign-Up-Page-Empty-Email-with-valid-details-1-Signoz-darwin.png new file mode 100644 index 0000000000..f2fe6360c9 Binary files /dev/null and b/frontend/tests/signup/index.spec.ts-snapshots/Sign-Up-Page-Empty-Email-with-valid-details-1-Signoz-darwin.png differ diff --git a/frontend/tests/signup/index.spec.ts-snapshots/Sign-Up-Page-Empty-Password-and-confirm-password-with-valid-details-1-Signoz-darwin.png b/frontend/tests/signup/index.spec.ts-snapshots/Sign-Up-Page-Empty-Password-and-confirm-password-with-valid-details-1-Signoz-darwin.png new file mode 100644 index 0000000000..e5c268cb83 Binary files /dev/null and b/frontend/tests/signup/index.spec.ts-snapshots/Sign-Up-Page-Empty-Password-and-confirm-password-with-valid-details-1-Signoz-darwin.png differ diff --git a/frontend/tests/signup/index.spec.ts-snapshots/Sign-Up-Page-Empty-name-with-valid-details-1-Signoz-darwin.png b/frontend/tests/signup/index.spec.ts-snapshots/Sign-Up-Page-Empty-name-with-valid-details-1-Signoz-darwin.png new file mode 100644 index 0000000000..5875c39dca Binary files /dev/null and b/frontend/tests/signup/index.spec.ts-snapshots/Sign-Up-Page-Empty-name-with-valid-details-1-Signoz-darwin.png differ diff --git a/frontend/tests/signup/index.spec.ts-snapshots/Sign-Up-Page-Invite-link-validation-1-Signoz-darwin.png b/frontend/tests/signup/index.spec.ts-snapshots/Sign-Up-Page-Invite-link-validation-1-Signoz-darwin.png new file mode 100644 index 0000000000..a514f42f06 Binary files /dev/null and b/frontend/tests/signup/index.spec.ts-snapshots/Sign-Up-Page-Invite-link-validation-1-Signoz-darwin.png differ diff --git a/frontend/tests/signup/index.spec.ts-snapshots/Sign-Up-Page-Miss-Match-Password-and-confirm-password-with-valid-details-1-Signoz-darwin.png b/frontend/tests/signup/index.spec.ts-snapshots/Sign-Up-Page-Miss-Match-Password-and-confirm-password-with-valid-details-1-Signoz-darwin.png new file mode 100644 index 0000000000..b661ff5744 Binary files /dev/null and b/frontend/tests/signup/index.spec.ts-snapshots/Sign-Up-Page-Miss-Match-Password-and-confirm-password-with-valid-details-1-Signoz-darwin.png differ diff --git a/frontend/tests/signup/index.spec.ts-snapshots/Sign-Up-Page-User-Sign-up-with-valid-details-1-Signoz-darwin.png b/frontend/tests/signup/index.spec.ts-snapshots/Sign-Up-Page-User-Sign-up-with-valid-details-1-Signoz-darwin.png new file mode 100644 index 0000000000..ef4ea46656 Binary files /dev/null and b/frontend/tests/signup/index.spec.ts-snapshots/Sign-Up-Page-User-Sign-up-with-valid-details-1-Signoz-darwin.png differ diff --git a/frontend/tests/signup/index.spec.ts-snapshots/Sign-Up-Page-When-User-successfull-signup-and-logged-in-he-should-be-redirected-to-dashboard-1-Signoz-darwin.png b/frontend/tests/signup/index.spec.ts-snapshots/Sign-Up-Page-When-User-successfull-signup-and-logged-in-he-should-be-redirected-to-dashboard-1-Signoz-darwin.png new file mode 100644 index 0000000000..e6b904dc77 Binary files /dev/null and b/frontend/tests/signup/index.spec.ts-snapshots/Sign-Up-Page-When-User-successfull-signup-and-logged-in-he-should-be-redirected-to-dashboard-1-Signoz-darwin.png differ diff --git a/pkg/query-service/Dockerfile b/pkg/query-service/Dockerfile index 3689b2bf23..eec478aaef 100644 --- a/pkg/query-service/Dockerfile +++ b/pkg/query-service/Dockerfile @@ -20,7 +20,7 @@ RUN go mod download -x # Add the sources and proceed with build ADD . . -RUN go build -a -ldflags "-linkmode external -extldflags '-static' -s -w $LD_FLAGS" -o ./bin/query-service ./main.go +RUN go build -tags timetzdata -a -ldflags "-linkmode external -extldflags '-static' -s -w $LD_FLAGS" -o ./bin/query-service ./main.go RUN chmod +x ./bin/query-service diff --git a/pkg/query-service/README.md b/pkg/query-service/README.md index bac8851855..10a7f78fbc 100644 --- a/pkg/query-service/README.md +++ b/pkg/query-service/README.md @@ -6,8 +6,37 @@ Query service is the interface between frontend and databases. It is written in - parse response from databases and handle error if any - clickhouse response in the format accepted by Frontend +# Complete the clickhouse setup locally. +https://github.com/SigNoz/signoz/blob/main/CONTRIBUTING.md#to-run-clickhouse-setup-recommended-for-local-development + +- Comment out the query-service and the frontend section in `signoz/deploy/docker/clickhouse-setup/docker-compose.yaml` +- Change the alertmanager section in `signoz/deploy/docker/clickhouse-setup/docker-compose.yaml` as follows: +```console +alertmanager: + image: signoz/alertmanager:0.23.0-0.1 + volumes: + - ./data/alertmanager:/data + expose: + - "9093" + ports: + - "8080:9093" + # depends_on: + # query-service: + # condition: service_healthy + restart: on-failure + command: + - --queryService.url=http://172.17.0.1:8085 + - --storage.path=/data +``` +- Run the following: +```console +cd signoz/ +If you are using x86_64 processors (All Intel/AMD processors) run sudo make run-x86 +If you are on arm64 processors (Apple M1 Macs) run sudo make run-arm +``` + +#### Backend Configuration -#### Configuration - Open ./constants/constants.go - Replace ```const RELATIONAL_DATASOURCE_PATH = "/var/lib/signoz/signoz.db"``` \ with ```const RELATIONAL_DATASOURCE_PATH = "./signoz.db".``` @@ -15,8 +44,9 @@ Query service is the interface between frontend and databases. It is written in - Query Service needs below `env` variables to run: ``` - ClickHouseUrl=tcp://localhost:9001 - STORAGE=clickhouse + export ClickHouseUrl=tcp://localhost:9001 + export STORAGE=clickhouse + export ALERTMANAGER_API_PREFIX=http://localhost:9093/api/ ``` @@ -28,5 +58,24 @@ go build -o build/query-service main.go ClickHouseUrl=tcp://localhost:9001 STORAGE=clickhouse build/query-service ``` +# Frontend Configuration for local query-service. + +- Set the following environment variables +```console +export FRONTEND_API_ENDPOINT=http://localhost:8080 +``` + +- Run the following +```console +cd signoz\frontend\ +yarn install +yarn dev +``` + +## Note: +If you use go version 1.18 for development and contributions, then please checkout the following issue. +https://github.com/SigNoz/signoz/issues/1371 + + #### Docker Images The docker images of query-service is available at https://hub.docker.com/r/signoz/query-service diff --git a/pkg/query-service/app/clickhouseReader/options.go b/pkg/query-service/app/clickhouseReader/options.go index 99fe5080ae..29816c2a08 100644 --- a/pkg/query-service/app/clickhouseReader/options.go +++ b/pkg/query-service/app/clickhouseReader/options.go @@ -18,16 +18,19 @@ const ( ) const ( - defaultDatasource string = "tcp://localhost:9000" - defaultTraceDB string = "signoz_traces" - defaultOperationsTable string = "signoz_operations" - defaultIndexTable string = "signoz_index_v2" - defaultErrorTable string = "signoz_error_index_v2" - defaulDurationTable string = "durationSortMV" - defaultSpansTable string = "signoz_spans" - defaultWriteBatchDelay time.Duration = 5 * time.Second - defaultWriteBatchSize int = 10000 - defaultEncoding Encoding = EncodingJSON + defaultDatasource string = "tcp://localhost:9000" + defaultTraceDB string = "signoz_traces" + defaultOperationsTable string = "signoz_operations" + defaultIndexTable string = "signoz_index_v2" + defaultErrorTable string = "signoz_error_index_v2" + defaultDurationTable string = "durationSortMV" + defaultUsageExplorerTable string = "usage_explorer" + defaultSpansTable string = "signoz_spans" + defaultDependencyGraphTable string = "dependency_graph_minutes" + defaultTopLevelOperationsTable string = "top_level_operations" + defaultWriteBatchDelay time.Duration = 5 * time.Second + defaultWriteBatchSize int = 10000 + defaultEncoding Encoding = EncodingJSON ) const ( @@ -43,19 +46,22 @@ const ( // NamespaceConfig is Clickhouse's internal configuration data type namespaceConfig struct { - namespace string - Enabled bool - Datasource string - TraceDB string - OperationsTable string - IndexTable string - DurationTable string - SpansTable string - ErrorTable string - WriteBatchDelay time.Duration - WriteBatchSize int - Encoding Encoding - Connector Connector + namespace string + Enabled bool + Datasource string + TraceDB string + OperationsTable string + IndexTable string + DurationTable string + UsageExplorerTable string + SpansTable string + ErrorTable string + DependencyGraphTable string + TopLevelOperationsTable string + WriteBatchDelay time.Duration + WriteBatchSize int + Encoding Encoding + Connector Connector } // Connecto defines how to connect to the database @@ -102,19 +108,22 @@ func NewOptions(datasource string, primaryNamespace string, otherNamespaces ...s options := &Options{ primary: &namespaceConfig{ - namespace: primaryNamespace, - Enabled: true, - Datasource: datasource, - TraceDB: defaultTraceDB, - OperationsTable: defaultOperationsTable, - IndexTable: defaultIndexTable, - ErrorTable: defaultErrorTable, - DurationTable: defaulDurationTable, - SpansTable: defaultSpansTable, - WriteBatchDelay: defaultWriteBatchDelay, - WriteBatchSize: defaultWriteBatchSize, - Encoding: defaultEncoding, - Connector: defaultConnector, + namespace: primaryNamespace, + Enabled: true, + Datasource: datasource, + TraceDB: defaultTraceDB, + OperationsTable: defaultOperationsTable, + IndexTable: defaultIndexTable, + ErrorTable: defaultErrorTable, + DurationTable: defaultDurationTable, + UsageExplorerTable: defaultUsageExplorerTable, + SpansTable: defaultSpansTable, + DependencyGraphTable: defaultDependencyGraphTable, + TopLevelOperationsTable: defaultTopLevelOperationsTable, + WriteBatchDelay: defaultWriteBatchDelay, + WriteBatchSize: defaultWriteBatchSize, + Encoding: defaultEncoding, + Connector: defaultConnector, }, others: make(map[string]*namespaceConfig, len(otherNamespaces)), } diff --git a/pkg/query-service/app/clickhouseReader/reader.go b/pkg/query-service/app/clickhouseReader/reader.go index 6716373c8b..65758f00e2 100644 --- a/pkg/query-service/app/clickhouseReader/reader.go +++ b/pkg/query-service/app/clickhouseReader/reader.go @@ -47,16 +47,17 @@ import ( ) const ( - primaryNamespace = "clickhouse" - archiveNamespace = "clickhouse-archive" - signozTraceDBName = "signoz_traces" - signozDurationMVTable = "durationSort" - signozSpansTable = "signoz_spans" - signozErrorIndexTable = "signoz_error_index_v2" - signozTraceTableName = "signoz_index_v2" - signozMetricDBName = "signoz_metrics" - signozSampleTableName = "samples_v2" - signozTSTableName = "time_series_v2" + primaryNamespace = "clickhouse" + archiveNamespace = "clickhouse-archive" + signozTraceDBName = "signoz_traces" + signozDurationMVTable = "durationSort" + signozUsageExplorerTable = "usage_explorer" + signozSpansTable = "signoz_spans" + signozErrorIndexTable = "signoz_error_index_v2" + signozTraceTableName = "signoz_index_v2" + signozMetricDBName = "signoz_metrics" + signozSampleTableName = "samples_v2" + signozTSTableName = "time_series_v2" minTimespanForProgressiveSearch = time.Hour minTimespanForProgressiveSearchMargin = time.Minute @@ -75,16 +76,19 @@ var ( // SpanWriter for reading spans from ClickHouse type ClickHouseReader struct { - db clickhouse.Conn - localDB *sqlx.DB - traceDB string - operationsTable string - durationTable string - indexTable string - errorTable string - spansTable string - queryEngine *promql.Engine - remoteStorage *remote.Storage + db clickhouse.Conn + localDB *sqlx.DB + traceDB string + operationsTable string + durationTable string + usageExplorerTable string + indexTable string + errorTable string + spansTable string + dependencyGraphTable string + topLevelOperationsTable string + queryEngine *promql.Engine + remoteStorage *remote.Storage promConfigFile string promConfig *config.Config @@ -111,16 +115,19 @@ func NewReader(localDB *sqlx.DB, configFile string) *ClickHouseReader { } return &ClickHouseReader{ - db: db, - localDB: localDB, - traceDB: options.primary.TraceDB, - alertManager: alertManager, - operationsTable: options.primary.OperationsTable, - indexTable: options.primary.IndexTable, - errorTable: options.primary.ErrorTable, - durationTable: options.primary.DurationTable, - spansTable: options.primary.SpansTable, - promConfigFile: configFile, + db: db, + localDB: localDB, + traceDB: options.primary.TraceDB, + alertManager: alertManager, + operationsTable: options.primary.OperationsTable, + indexTable: options.primary.IndexTable, + errorTable: options.primary.ErrorTable, + usageExplorerTable: options.primary.UsageExplorerTable, + durationTable: options.primary.DurationTable, + spansTable: options.primary.SpansTable, + dependencyGraphTable: options.primary.DependencyGraphTable, + topLevelOperationsTable: options.primary.TopLevelOperationsTable, + promConfigFile: configFile, } } @@ -374,14 +381,21 @@ func (r *ClickHouseReader) GetChannel(id string) (*model.ChannelItem, *model.Api idInt, _ := strconv.Atoi(id) channel := model.ChannelItem{} - query := fmt.Sprintf("SELECT id, created_at, updated_at, name, type, data data FROM notification_channels WHERE id=%d", idInt) + query := "SELECT id, created_at, updated_at, name, type, data data FROM notification_channels WHERE id=? " - err := r.localDB.Get(&channel, query) + stmt, err := r.localDB.Preparex(query) - zap.S().Info(query) + zap.S().Info(query, idInt) if err != nil { - zap.S().Debug("Error in processing sql query: ", err) + zap.S().Debug("Error in preparing sql query for GetChannel : ", err) + return nil, &model.ApiError{Typ: model.ErrorInternal, Err: err} + } + + err = stmt.Get(&channel, idInt) + + if err != nil { + zap.S().Debug(fmt.Sprintf("Error in getting channel with id=%d : ", idInt), err) return nil, &model.ApiError{Typ: model.ErrorInternal, Err: err} } @@ -650,103 +664,153 @@ func (r *ClickHouseReader) GetServicesList(ctx context.Context) (*[]string, erro return &services, nil } +func (r *ClickHouseReader) GetTopLevelOperations(ctx context.Context) (*map[string][]string, *model.ApiError) { + + operations := map[string][]string{} + query := fmt.Sprintf(`SELECT DISTINCT name, serviceName FROM %s.%s`, r.traceDB, r.topLevelOperationsTable) + + rows, err := r.db.Query(ctx, query) + + if err != nil { + zap.S().Error("Error in processing sql query: ", err) + return nil, &model.ApiError{Typ: model.ErrorExec, Err: fmt.Errorf("Error in processing sql query")} + } + + defer rows.Close() + for rows.Next() { + var name, serviceName string + if err := rows.Scan(&name, &serviceName); err != nil { + return nil, &model.ApiError{Typ: model.ErrorInternal, Err: fmt.Errorf("Error in reading data")} + } + if _, ok := operations[serviceName]; !ok { + operations[serviceName] = []string{} + } + operations[serviceName] = append(operations[serviceName], name) + } + return &operations, nil +} + func (r *ClickHouseReader) GetServices(ctx context.Context, queryParams *model.GetServicesParams) (*[]model.ServiceItem, *model.ApiError) { if r.indexTable == "" { return nil, &model.ApiError{Typ: model.ErrorExec, Err: ErrNoIndexTable} } + topLevelOps, apiErr := r.GetTopLevelOperations(ctx) + if apiErr != nil { + return nil, apiErr + } + serviceItems := []model.ServiceItem{} + var wg sync.WaitGroup + // limit the number of concurrent queries to not overload the clickhouse server + sem := make(chan struct{}, 10) + var mtx sync.RWMutex - query := fmt.Sprintf("SELECT serviceName, quantile(0.99)(durationNano) as p99, avg(durationNano) as avgDuration, count(*) as numCalls FROM %s.%s WHERE timestamp>='%s' AND timestamp<='%s' AND kind='2'", r.traceDB, r.indexTable, strconv.FormatInt(queryParams.Start.UnixNano(), 10), strconv.FormatInt(queryParams.End.UnixNano(), 10)) - args := []interface{}{} - args, errStatus := buildQueryWithTagParams(ctx, queryParams.Tags, &query, args) - if errStatus != nil { - return nil, errStatus + for svc, ops := range *topLevelOps { + sem <- struct{}{} + wg.Add(1) + go func(svc string, ops []string) { + defer wg.Done() + defer func() { <-sem }() + var serviceItem model.ServiceItem + var numErrors uint64 + query := fmt.Sprintf( + `SELECT + quantile(0.99)(durationNano) as p99, + avg(durationNano) as avgDuration, + count(*) as numCalls + FROM %s.%s + WHERE serviceName = @serviceName AND name In [@names] AND timestamp>= @start AND timestamp<= @end`, + r.traceDB, r.indexTable, + ) + errorQuery := fmt.Sprintf( + `SELECT + count(*) as numErrors + FROM %s.%s + WHERE serviceName = @serviceName AND name In [@names] AND timestamp>= @start AND timestamp<= @end AND statusCode=2`, + r.traceDB, r.indexTable, + ) + + args := []interface{}{} + args = append(args, + clickhouse.Named("start", strconv.FormatInt(queryParams.Start.UnixNano(), 10)), + clickhouse.Named("end", strconv.FormatInt(queryParams.End.UnixNano(), 10)), + clickhouse.Named("serviceName", svc), + clickhouse.Named("names", ops), + ) + args, errStatus := buildQueryWithTagParams(ctx, queryParams.Tags, &query, args) + if errStatus != nil { + zap.S().Error("Error in processing sql query: ", errStatus) + return + } + err := r.db.QueryRow( + ctx, + query, + args..., + ).ScanStruct(&serviceItem) + + if err != nil { + zap.S().Error("Error in processing sql query: ", err) + return + } + + err = r.db.QueryRow(ctx, errorQuery, args...).Scan(&numErrors) + if err != nil { + zap.S().Error("Error in processing sql query: ", err) + return + } + + serviceItem.ServiceName = svc + serviceItem.NumErrors = numErrors + mtx.Lock() + serviceItems = append(serviceItems, serviceItem) + mtx.Unlock() + }(svc, ops) } - query += " GROUP BY serviceName ORDER BY p99 DESC" - err := r.db.Select(ctx, &serviceItems, query, args...) + wg.Wait() - zap.S().Info(query) - - if err != nil { - zap.S().Debug("Error in processing sql query: ", err) - return nil, &model.ApiError{Typ: model.ErrorExec, Err: fmt.Errorf("Error in processing sql query")} + for idx := range serviceItems { + serviceItems[idx].CallRate = float64(serviceItems[idx].NumCalls) / float64(queryParams.Period) + serviceItems[idx].ErrorRate = float64(serviceItems[idx].NumErrors) * 100 / float64(serviceItems[idx].NumCalls) } - - ////////////////// Below block gets 5xx of services - serviceErrorItems := []model.ServiceItem{} - - query = fmt.Sprintf("SELECT serviceName, count(*) as numErrors FROM %s.%s WHERE timestamp>='%s' AND timestamp<='%s' AND kind='2' AND (statusCode>=500 OR statusCode=2)", r.traceDB, r.indexTable, strconv.FormatInt(queryParams.Start.UnixNano(), 10), strconv.FormatInt(queryParams.End.UnixNano(), 10)) - args = []interface{}{} - args, errStatus = buildQueryWithTagParams(ctx, queryParams.Tags, &query, args) - if errStatus != nil { - return nil, errStatus - } - query += " GROUP BY serviceName" - err = r.db.Select(ctx, &serviceErrorItems, query, args...) - - zap.S().Info(query) - - if err != nil { - zap.S().Debug("Error in processing sql query: ", err) - return nil, &model.ApiError{Typ: model.ErrorExec, Err: fmt.Errorf("Error in processing sql query")} - } - - m5xx := make(map[string]uint64) - - for j := range serviceErrorItems { - m5xx[serviceErrorItems[j].ServiceName] = serviceErrorItems[j].NumErrors - } - /////////////////////////////////////////// - - ////////////////// Below block gets 4xx of services - - service4xxItems := []model.ServiceItem{} - - query = fmt.Sprintf("SELECT serviceName, count(*) as num4xx FROM %s.%s WHERE timestamp>='%s' AND timestamp<='%s' AND kind='2' AND statusCode>=400 AND statusCode<500", r.traceDB, r.indexTable, strconv.FormatInt(queryParams.Start.UnixNano(), 10), strconv.FormatInt(queryParams.End.UnixNano(), 10)) - args = []interface{}{} - args, errStatus = buildQueryWithTagParams(ctx, queryParams.Tags, &query, args) - if errStatus != nil { - return nil, errStatus - } - query += " GROUP BY serviceName" - err = r.db.Select(ctx, &service4xxItems, query, args...) - - zap.S().Info(query) - - if err != nil { - zap.S().Debug("Error in processing sql query: ", err) - return nil, &model.ApiError{Typ: model.ErrorExec, Err: fmt.Errorf("Error in processing sql query")} - } - - m4xx := make(map[string]uint64) - - for j := range service4xxItems { - m4xx[service4xxItems[j].ServiceName] = service4xxItems[j].Num4XX - } - - for i := range serviceItems { - if val, ok := m5xx[serviceItems[i].ServiceName]; ok { - serviceItems[i].NumErrors = val - } - if val, ok := m4xx[serviceItems[i].ServiceName]; ok { - serviceItems[i].Num4XX = val - } - serviceItems[i].CallRate = float64(serviceItems[i].NumCalls) / float64(queryParams.Period) - serviceItems[i].FourXXRate = float64(serviceItems[i].Num4XX) * 100 / float64(serviceItems[i].NumCalls) - serviceItems[i].ErrorRate = float64(serviceItems[i].NumErrors) * 100 / float64(serviceItems[i].NumCalls) - } - return &serviceItems, nil } func (r *ClickHouseReader) GetServiceOverview(ctx context.Context, queryParams *model.GetServiceOverviewParams) (*[]model.ServiceOverviewItem, *model.ApiError) { + topLevelOps, apiErr := r.GetTopLevelOperations(ctx) + if apiErr != nil { + return nil, apiErr + } + ops, ok := (*topLevelOps)[queryParams.ServiceName] + if !ok { + return nil, &model.ApiError{Typ: model.ErrorNotFound, Err: fmt.Errorf("Service not found")} + } + + namedArgs := []interface{}{ + clickhouse.Named("interval", strconv.Itoa(int(queryParams.StepSeconds/60))), + clickhouse.Named("start", strconv.FormatInt(queryParams.Start.UnixNano(), 10)), + clickhouse.Named("end", strconv.FormatInt(queryParams.End.UnixNano(), 10)), + clickhouse.Named("serviceName", queryParams.ServiceName), + clickhouse.Named("names", ops), + } + serviceOverviewItems := []model.ServiceOverviewItem{} - query := fmt.Sprintf("SELECT toStartOfInterval(timestamp, INTERVAL %s minute) as time, quantile(0.99)(durationNano) as p99, quantile(0.95)(durationNano) as p95,quantile(0.50)(durationNano) as p50, count(*) as numCalls FROM %s.%s WHERE timestamp>='%s' AND timestamp<='%s' AND kind='2' AND serviceName='%s'", strconv.Itoa(int(queryParams.StepSeconds/60)), r.traceDB, r.indexTable, strconv.FormatInt(queryParams.Start.UnixNano(), 10), strconv.FormatInt(queryParams.End.UnixNano(), 10), queryParams.ServiceName) + query := fmt.Sprintf(` + SELECT + toStartOfInterval(timestamp, INTERVAL @interval minute) as time, + quantile(0.99)(durationNano) as p99, + quantile(0.95)(durationNano) as p95, + quantile(0.50)(durationNano) as p50, + count(*) as numCalls + FROM %s.%s + WHERE serviceName = @serviceName AND name In [@names] AND timestamp>= @start AND timestamp<= @end`, + r.traceDB, r.indexTable, + ) args := []interface{}{} + args = append(args, namedArgs...) args, errStatus := buildQueryWithTagParams(ctx, queryParams.Tags, &query, args) if errStatus != nil { return nil, errStatus @@ -754,17 +818,25 @@ func (r *ClickHouseReader) GetServiceOverview(ctx context.Context, queryParams * query += " GROUP BY time ORDER BY time DESC" err := r.db.Select(ctx, &serviceOverviewItems, query, args...) - zap.S().Info(query) + zap.S().Debug(query) if err != nil { - zap.S().Debug("Error in processing sql query: ", err) + zap.S().Error("Error in processing sql query: ", err) return nil, &model.ApiError{Typ: model.ErrorExec, Err: fmt.Errorf("Error in processing sql query")} } serviceErrorItems := []model.ServiceErrorItem{} - query = fmt.Sprintf("SELECT toStartOfInterval(timestamp, INTERVAL %s minute) as time, count(*) as numErrors FROM %s.%s WHERE timestamp>='%s' AND timestamp<='%s' AND kind='2' AND serviceName='%s' AND hasError=true", strconv.Itoa(int(queryParams.StepSeconds/60)), r.traceDB, r.indexTable, strconv.FormatInt(queryParams.Start.UnixNano(), 10), strconv.FormatInt(queryParams.End.UnixNano(), 10), queryParams.ServiceName) + query = fmt.Sprintf(` + SELECT + toStartOfInterval(timestamp, INTERVAL @interval minute) as time, + count(*) as numErrors + FROM %s.%s + WHERE serviceName = @serviceName AND name In [@names] AND timestamp>= @start AND timestamp<= @end AND statusCode=2`, + r.traceDB, r.indexTable, + ) args = []interface{}{} + args = append(args, namedArgs...) args, errStatus = buildQueryWithTagParams(ctx, queryParams.Tags, &query, args) if errStatus != nil { return nil, errStatus @@ -772,10 +844,10 @@ func (r *ClickHouseReader) GetServiceOverview(ctx context.Context, queryParams * query += " GROUP BY time ORDER BY time DESC" err = r.db.Select(ctx, &serviceErrorItems, query, args...) - zap.S().Info(query) + zap.S().Debug(query) if err != nil { - zap.S().Debug("Error in processing sql query: ", err) + zap.S().Error("Error in processing sql query: ", err) return nil, &model.ApiError{Typ: model.ErrorExec, Err: fmt.Errorf("Error in processing sql query")} } @@ -1516,45 +1588,67 @@ func (r *ClickHouseReader) GetTagValues(ctx context.Context, queryParams *model. return &cleanedTagValues, nil } -func (r *ClickHouseReader) GetTopEndpoints(ctx context.Context, queryParams *model.GetTopEndpointsParams) (*[]model.TopEndpointsItem, *model.ApiError) { +func (r *ClickHouseReader) GetTopOperations(ctx context.Context, queryParams *model.GetTopOperationsParams) (*[]model.TopOperationsItem, *model.ApiError) { - var topEndpointsItems []model.TopEndpointsItem + namedArgs := []interface{}{ + clickhouse.Named("start", strconv.FormatInt(queryParams.Start.UnixNano(), 10)), + clickhouse.Named("end", strconv.FormatInt(queryParams.End.UnixNano(), 10)), + clickhouse.Named("serviceName", queryParams.ServiceName), + } - query := fmt.Sprintf("SELECT quantile(0.5)(durationNano) as p50, quantile(0.95)(durationNano) as p95, quantile(0.99)(durationNano) as p99, COUNT(1) as numCalls, name FROM %s.%s WHERE timestamp >= '%s' AND timestamp <= '%s' AND kind='2' and serviceName='%s'", r.traceDB, r.indexTable, strconv.FormatInt(queryParams.Start.UnixNano(), 10), strconv.FormatInt(queryParams.End.UnixNano(), 10), queryParams.ServiceName) + var topOperationsItems []model.TopOperationsItem + + query := fmt.Sprintf(` + SELECT + quantile(0.5)(durationNano) as p50, + quantile(0.95)(durationNano) as p95, + quantile(0.99)(durationNano) as p99, + COUNT(*) as numCalls, + name + FROM %s.%s + WHERE serviceName = @serviceName AND timestamp>= @start AND timestamp<= @end`, + r.traceDB, r.indexTable, + ) args := []interface{}{} + args = append(args, namedArgs...) args, errStatus := buildQueryWithTagParams(ctx, queryParams.Tags, &query, args) if errStatus != nil { return nil, errStatus } - query += " GROUP BY name" - err := r.db.Select(ctx, &topEndpointsItems, query, args...) + query += " GROUP BY name ORDER BY p99 DESC LIMIT 10" + err := r.db.Select(ctx, &topOperationsItems, query, args...) - zap.S().Info(query) + zap.S().Debug(query) if err != nil { - zap.S().Debug("Error in processing sql query: ", err) + zap.S().Error("Error in processing sql query: ", err) return nil, &model.ApiError{Typ: model.ErrorExec, Err: fmt.Errorf("Error in processing sql query")} } - if topEndpointsItems == nil { - topEndpointsItems = []model.TopEndpointsItem{} + if topOperationsItems == nil { + topOperationsItems = []model.TopOperationsItem{} } - return &topEndpointsItems, nil + return &topOperationsItems, nil } func (r *ClickHouseReader) GetUsage(ctx context.Context, queryParams *model.GetUsageParams) (*[]model.UsageItem, error) { var usageItems []model.UsageItem - + namedArgs := []interface{}{ + clickhouse.Named("interval", queryParams.StepHour), + clickhouse.Named("start", strconv.FormatInt(queryParams.Start.UnixNano(), 10)), + clickhouse.Named("end", strconv.FormatInt(queryParams.End.UnixNano(), 10)), + } var query string if len(queryParams.ServiceName) != 0 { - query = fmt.Sprintf("SELECT toStartOfInterval(timestamp, INTERVAL %d HOUR) as time, count(1) as count FROM %s.%s WHERE serviceName='%s' AND timestamp>='%s' AND timestamp<='%s' GROUP BY time ORDER BY time ASC", queryParams.StepHour, r.traceDB, r.indexTable, queryParams.ServiceName, strconv.FormatInt(queryParams.Start.UnixNano(), 10), strconv.FormatInt(queryParams.End.UnixNano(), 10)) + namedArgs = append(namedArgs, clickhouse.Named("serviceName", queryParams.ServiceName)) + query = fmt.Sprintf("SELECT toStartOfInterval(timestamp, INTERVAL @interval HOUR) as time, sum(count) as count FROM %s.%s WHERE service_name=@serviceName AND timestamp>=@start AND timestamp<=@end GROUP BY time ORDER BY time ASC", r.traceDB, r.usageExplorerTable) } else { - query = fmt.Sprintf("SELECT toStartOfInterval(timestamp, INTERVAL %d HOUR) as time, count(1) as count FROM %s.%s WHERE timestamp>='%s' AND timestamp<='%s' GROUP BY time ORDER BY time ASC", queryParams.StepHour, r.traceDB, r.indexTable, strconv.FormatInt(queryParams.Start.UnixNano(), 10), strconv.FormatInt(queryParams.End.UnixNano(), 10)) + query = fmt.Sprintf("SELECT toStartOfInterval(timestamp, INTERVAL @interval HOUR) as time, sum(count) as count FROM %s.%s WHERE timestamp>=@start AND timestamp<=@end GROUP BY time ORDER BY time ASC", r.traceDB, r.usageExplorerTable) } - err := r.db.Select(ctx, &usageItems, query) + err := r.db.Select(ctx, &usageItems, query, namedArgs...) zap.S().Info(query) @@ -1614,48 +1708,50 @@ func interfaceArrayToStringArray(array []interface{}) []string { return strArray } -func (r *ClickHouseReader) GetServiceMapDependencies(ctx context.Context, queryParams *model.GetServicesParams) (*[]model.ServiceMapDependencyResponseItem, error) { - serviceMapDependencyItems := []model.ServiceMapDependencyItem{} +func (r *ClickHouseReader) GetDependencyGraph(ctx context.Context, queryParams *model.GetServicesParams) (*[]model.ServiceMapDependencyResponseItem, error) { - query := fmt.Sprintf(`SELECT spanID, parentSpanID, serviceName FROM %s.%s WHERE timestamp>='%s' AND timestamp<='%s'`, r.traceDB, r.indexTable, strconv.FormatInt(queryParams.Start.UnixNano(), 10), strconv.FormatInt(queryParams.End.UnixNano(), 10)) + response := []model.ServiceMapDependencyResponseItem{} - err := r.db.Select(ctx, &serviceMapDependencyItems, query) + args := []interface{}{} + args = append(args, + clickhouse.Named("start", uint64(queryParams.Start.Unix())), + clickhouse.Named("end", uint64(queryParams.End.Unix())), + clickhouse.Named("duration", uint64(queryParams.End.Unix()-queryParams.Start.Unix())), + ) - zap.S().Info(query) + query := fmt.Sprintf(` + WITH + quantilesMergeState(0.5, 0.75, 0.9, 0.95, 0.99)(duration_quantiles_state) AS duration_quantiles_state, + finalizeAggregation(duration_quantiles_state) AS result + SELECT + src as parent, + dest as child, + result[1] AS p50, + result[2] AS p75, + result[3] AS p90, + result[4] AS p95, + result[5] AS p99, + sum(total_count) as callCount, + sum(total_count)/ @duration AS callRate, + sum(error_count)/sum(total_count) as errorRate + FROM %s.%s + WHERE toUInt64(toDateTime(timestamp)) >= @start AND toUInt64(toDateTime(timestamp)) <= @end + GROUP BY + src, + dest`, + r.traceDB, r.dependencyGraphTable, + ) + + zap.S().Debug(query, args) + + err := r.db.Select(ctx, &response, query, args...) if err != nil { - zap.S().Debug("Error in processing sql query: ", err) + zap.S().Error("Error in processing sql query: ", err) return nil, fmt.Errorf("Error in processing sql query") } - serviceMap := make(map[string]*model.ServiceMapDependencyResponseItem) - - spanId2ServiceNameMap := make(map[string]string) - for i := range serviceMapDependencyItems { - spanId2ServiceNameMap[serviceMapDependencyItems[i].SpanId] = serviceMapDependencyItems[i].ServiceName - } - for i := range serviceMapDependencyItems { - parent2childServiceName := spanId2ServiceNameMap[serviceMapDependencyItems[i].ParentSpanId] + "-" + spanId2ServiceNameMap[serviceMapDependencyItems[i].SpanId] - if _, ok := serviceMap[parent2childServiceName]; !ok { - serviceMap[parent2childServiceName] = &model.ServiceMapDependencyResponseItem{ - Parent: spanId2ServiceNameMap[serviceMapDependencyItems[i].ParentSpanId], - Child: spanId2ServiceNameMap[serviceMapDependencyItems[i].SpanId], - CallCount: 1, - } - } else { - serviceMap[parent2childServiceName].CallCount++ - } - } - - retMe := make([]model.ServiceMapDependencyResponseItem, 0, len(serviceMap)) - for _, dependency := range serviceMap { - if dependency.Parent == "" { - continue - } - retMe = append(retMe, *dependency) - } - - return &retMe, nil + return &response, nil } func (r *ClickHouseReader) GetFilteredSpansAggregates(ctx context.Context, queryParams *model.GetFilteredSpanAggregatesParams) (*model.GetFilteredSpansAggregatesResponse, *model.ApiError) { @@ -1895,7 +1991,7 @@ func (r *ClickHouseReader) SetTTL(ctx context.Context, switch params.Type { case constants.TraceTTL: - tableNameArray := []string{signozTraceDBName + "." + signozTraceTableName, signozTraceDBName + "." + signozDurationMVTable, signozTraceDBName + "." + signozSpansTable, signozTraceDBName + "." + signozErrorIndexTable} + tableNameArray := []string{signozTraceDBName + "." + signozTraceTableName, signozTraceDBName + "." + signozDurationMVTable, signozTraceDBName + "." + signozSpansTable, signozTraceDBName + "." + signozErrorIndexTable, signozTraceDBName + "." + signozUsageExplorerTable, signozTraceDBName + "." + defaultDependencyGraphTable} for _, tableName = range tableNameArray { statusItem, err := r.checkTTLStatusItem(ctx, tableName) if err != nil { @@ -2170,7 +2266,7 @@ func (r *ClickHouseReader) GetTTL(ctx context.Context, ttlParams *model.GetTTLPa switch ttlParams.Type { case constants.TraceTTL: - tableNameArray := []string{signozTraceDBName + "." + signozTraceTableName, signozTraceDBName + "." + signozDurationMVTable, signozTraceDBName + "." + signozSpansTable, signozTraceDBName + "." + signozErrorIndexTable} + tableNameArray := []string{signozTraceDBName + "." + signozTraceTableName, signozTraceDBName + "." + signozDurationMVTable, signozTraceDBName + "." + signozSpansTable, signozTraceDBName + "." + signozErrorIndexTable, signozTraceDBName + "." + signozUsageExplorerTable, signozTraceDBName + "." + defaultDependencyGraphTable} status, err := r.setTTLQueryStatus(ctx, tableNameArray) if err != nil { return nil, err diff --git a/pkg/query-service/app/http_handler.go b/pkg/query-service/app/http_handler.go index 5d7b8cce5c..a7a0a40c70 100644 --- a/pkg/query-service/app/http_handler.go +++ b/pkg/query-service/app/http_handler.go @@ -304,11 +304,14 @@ func (aH *APIHandler) RegisterRoutes(router *mux.Router) { router.HandleFunc("/api/v1/channels/{id}", AdminAccess(aH.deleteChannel)).Methods(http.MethodDelete) router.HandleFunc("/api/v1/channels", EditAccess(aH.createChannel)).Methods(http.MethodPost) router.HandleFunc("/api/v1/testChannel", EditAccess(aH.testChannel)).Methods(http.MethodPost) + router.HandleFunc("/api/v1/rules", ViewAccess(aH.listRules)).Methods(http.MethodGet) router.HandleFunc("/api/v1/rules/{id}", ViewAccess(aH.getRule)).Methods(http.MethodGet) router.HandleFunc("/api/v1/rules", EditAccess(aH.createRule)).Methods(http.MethodPost) router.HandleFunc("/api/v1/rules/{id}", EditAccess(aH.editRule)).Methods(http.MethodPut) router.HandleFunc("/api/v1/rules/{id}", EditAccess(aH.deleteRule)).Methods(http.MethodDelete) + router.HandleFunc("/api/v1/rules/{id}", EditAccess(aH.patchRule)).Methods(http.MethodPatch) + router.HandleFunc("/api/v1/testRule", EditAccess(aH.testRule)).Methods(http.MethodPost) router.HandleFunc("/api/v1/dashboards", ViewAccess(aH.getDashboards)).Methods(http.MethodGet) router.HandleFunc("/api/v1/dashboards", EditAccess(aH.createDashboards)).Methods(http.MethodPost) @@ -321,10 +324,11 @@ func (aH *APIHandler) RegisterRoutes(router *mux.Router) { router.HandleFunc("/api/v1/services", ViewAccess(aH.getServices)).Methods(http.MethodPost) router.HandleFunc("/api/v1/services/list", aH.getServicesList).Methods(http.MethodGet) router.HandleFunc("/api/v1/service/overview", ViewAccess(aH.getServiceOverview)).Methods(http.MethodPost) - router.HandleFunc("/api/v1/service/top_endpoints", ViewAccess(aH.getTopEndpoints)).Methods(http.MethodPost) + router.HandleFunc("/api/v1/service/top_operations", ViewAccess(aH.getTopOperations)).Methods(http.MethodPost) + router.HandleFunc("/api/v1/service/top_level_operations", ViewAccess(aH.getServicesTopLevelOps)).Methods(http.MethodPost) router.HandleFunc("/api/v1/traces/{traceId}", ViewAccess(aH.searchTraces)).Methods(http.MethodGet) router.HandleFunc("/api/v1/usage", ViewAccess(aH.getUsage)).Methods(http.MethodGet) - router.HandleFunc("/api/v1/serviceMapDependencies", ViewAccess(aH.serviceMapDependencies)).Methods(http.MethodPost) + router.HandleFunc("/api/v1/dependency_graph", ViewAccess(aH.dependencyGraph)).Methods(http.MethodPost) router.HandleFunc("/api/v1/settings/ttl", AdminAccess(aH.setTTL)).Methods(http.MethodPost) router.HandleFunc("/api/v1/settings/ttl", ViewAccess(aH.getTTL)).Methods(http.MethodGet) @@ -769,6 +773,32 @@ func (aH *APIHandler) createDashboards(w http.ResponseWriter, r *http.Request) { } +func (aH *APIHandler) testRule(w http.ResponseWriter, r *http.Request) { + + defer r.Body.Close() + body, err := ioutil.ReadAll(r.Body) + if err != nil { + zap.S().Errorf("Error in getting req body in test rule API\n", err) + respondError(w, &model.ApiError{Typ: model.ErrorBadData, Err: err}, nil) + return + } + + ctx, cancel := context.WithTimeout(context.Background(), 1*time.Minute) + defer cancel() + + alertCount, apiRrr := aH.ruleManager.TestNotification(ctx, string(body)) + if apiRrr != nil { + respondError(w, apiRrr, nil) + return + } + + response := map[string]interface{}{ + "alertCount": alertCount, + "message": "notification sent", + } + aH.respond(w, response) +} + func (aH *APIHandler) deleteRule(w http.ResponseWriter, r *http.Request) { id := mux.Vars(r)["id"] @@ -784,6 +814,28 @@ func (aH *APIHandler) deleteRule(w http.ResponseWriter, r *http.Request) { } +// patchRule updates only requested changes in the rule +func (aH *APIHandler) patchRule(w http.ResponseWriter, r *http.Request) { + id := mux.Vars(r)["id"] + + defer r.Body.Close() + body, err := ioutil.ReadAll(r.Body) + if err != nil { + zap.S().Errorf("msg: error in getting req body of patch rule API\n", "\t error:", err) + respondError(w, &model.ApiError{Typ: model.ErrorBadData, Err: err}, nil) + return + } + + gettableRule, err := aH.ruleManager.PatchRule(string(body), id) + + if err != nil { + respondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil) + return + } + + aH.respond(w, gettableRule) +} + func (aH *APIHandler) editRule(w http.ResponseWriter, r *http.Request) { id := mux.Vars(r)["id"] @@ -939,6 +991,7 @@ func (aH *APIHandler) createRule(w http.ResponseWriter, r *http.Request) { aH.respond(w, "rule successfully added") } + func (aH *APIHandler) queryRangeMetricsFromClickhouse(w http.ResponseWriter, r *http.Request) { } @@ -1080,14 +1133,14 @@ func (aH *APIHandler) submitFeedback(w http.ResponseWriter, r *http.Request) { } -func (aH *APIHandler) getTopEndpoints(w http.ResponseWriter, r *http.Request) { +func (aH *APIHandler) getTopOperations(w http.ResponseWriter, r *http.Request) { - query, err := parseGetTopEndpointsRequest(r) + query, err := parseGetTopOperationsRequest(r) if aH.handleError(w, err, http.StatusBadRequest) { return } - result, apiErr := (*aH.reader).GetTopEndpoints(r.Context(), query) + result, apiErr := (*aH.reader).GetTopOperations(r.Context(), query) if apiErr != nil && aH.handleError(w, apiErr.Err, http.StatusInternalServerError) { return @@ -1129,6 +1182,17 @@ func (aH *APIHandler) getServiceOverview(w http.ResponseWriter, r *http.Request) } +func (aH *APIHandler) getServicesTopLevelOps(w http.ResponseWriter, r *http.Request) { + + result, apiErr := (*aH.reader).GetTopLevelOperations(r.Context()) + if apiErr != nil { + respondError(w, apiErr, nil) + return + } + + aH.writeJSON(w, r, result) +} + func (aH *APIHandler) getServices(w http.ResponseWriter, r *http.Request) { query, err := parseGetServicesRequest(r) @@ -1150,14 +1214,14 @@ func (aH *APIHandler) getServices(w http.ResponseWriter, r *http.Request) { aH.writeJSON(w, r, result) } -func (aH *APIHandler) serviceMapDependencies(w http.ResponseWriter, r *http.Request) { +func (aH *APIHandler) dependencyGraph(w http.ResponseWriter, r *http.Request) { query, err := parseGetServicesRequest(r) if aH.handleError(w, err, http.StatusBadRequest) { return } - result, err := (*aH.reader).GetServiceMapDependencies(r.Context(), query) + result, err := (*aH.reader).GetDependencyGraph(r.Context(), query) if aH.handleError(w, err, http.StatusBadRequest) { return } diff --git a/pkg/query-service/app/parser.go b/pkg/query-service/app/parser.go index e81b986a3d..6991a03156 100644 --- a/pkg/query-service/app/parser.go +++ b/pkg/query-service/app/parser.go @@ -32,8 +32,8 @@ func parseUser(r *http.Request) (*model.User, error) { return &user, nil } -func parseGetTopEndpointsRequest(r *http.Request) (*model.GetTopEndpointsParams, error) { - var postData *model.GetTopEndpointsParams +func parseGetTopOperationsRequest(r *http.Request) (*model.GetTopOperationsParams, error) { + var postData *model.GetTopOperationsParams err := json.NewDecoder(r.Body).Decode(&postData) if err != nil { @@ -467,8 +467,8 @@ func parseCountErrorsRequest(r *http.Request) (*model.CountErrorsParams, error) } params := &model.CountErrorsParams{ - Start: startTime, - End: endTime, + Start: startTime, + End: endTime, } return params, nil diff --git a/pkg/query-service/app/server.go b/pkg/query-service/app/server.go index 1815e5c7f0..845b75e9c4 100644 --- a/pkg/query-service/app/server.go +++ b/pkg/query-service/app/server.go @@ -140,7 +140,7 @@ func (s *Server) createPrivateServer(api *APIHandler) (*http.Server, error) { //todo(amol): find out a way to add exact domain or // ip here for alert manager AllowedOrigins: []string{"*"}, - AllowedMethods: []string{"GET", "DELETE", "POST", "PUT"}, + AllowedMethods: []string{"GET", "DELETE", "POST", "PUT", "PATCH"}, AllowedHeaders: []string{"Accept", "Authorization", "Content-Type"}, }) @@ -165,7 +165,7 @@ func (s *Server) createPublicServer(api *APIHandler) (*http.Server, error) { c := cors.New(cors.Options{ AllowedOrigins: []string{"*"}, - AllowedMethods: []string{"GET", "DELETE", "POST", "PUT"}, + AllowedMethods: []string{"GET", "DELETE", "POST", "PUT", "PATCH"}, AllowedHeaders: []string{"Accept", "Authorization", "Content-Type"}, }) diff --git a/pkg/query-service/config/prometheus.yml b/pkg/query-service/config/prometheus.yml index c515a46662..d7c0ce6911 100644 --- a/pkg/query-service/config/prometheus.yml +++ b/pkg/query-service/config/prometheus.yml @@ -23,4 +23,4 @@ scrape_configs: remote_read: - - url: tcp://localhost:9001/?database=signoz_metrics + - url: tcp://localhost:9000/?database=signoz_metrics diff --git a/pkg/query-service/go.mod b/pkg/query-service/go.mod index 1d5dfa8e8d..34ddceace9 100644 --- a/pkg/query-service/go.mod +++ b/pkg/query-service/go.mod @@ -11,7 +11,7 @@ require ( github.com/gorilla/mux v1.8.0 github.com/gosimple/slug v1.10.0 github.com/jmoiron/sqlx v1.3.4 - github.com/json-iterator/go v1.1.10 + github.com/json-iterator/go v1.1.12 github.com/mattn/go-sqlite3 v1.14.8 github.com/minio/minio-go/v6 v6.0.57 github.com/oklog/oklog v0.3.2 @@ -92,7 +92,7 @@ require ( github.com/mitchellh/go-homedir v1.1.0 // indirect github.com/mitchellh/go-testing-interface v1.14.1 // indirect github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect - github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742 // indirect + github.com/modern-go/reflect2 v1.0.2 // indirect github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223 // indirect github.com/oklog/run v1.1.0 // indirect github.com/oklog/ulid v0.3.1-0.20170117200651-66bb6560562f // indirect diff --git a/pkg/query-service/go.sum b/pkg/query-service/go.sum index d69fb82481..4fbb2b1c25 100644 --- a/pkg/query-service/go.sum +++ b/pkg/query-service/go.sum @@ -295,6 +295,8 @@ github.com/json-iterator/go v0.0.0-20180612202835-f2b4162afba3/go.mod h1:+SdeFBv github.com/json-iterator/go v1.1.9/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= github.com/json-iterator/go v1.1.10 h1:Kz6Cvnvv2wGdaG/V8yMvfkmNiXq9Ya2KUv4rouJJr68= github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= +github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM= +github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo= github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU= github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk= github.com/jtolds/gls v4.20.0+incompatible h1:xdiiI2gbIgH/gLH7ADydsJ1uDOEzR8yvV7C0MuV77Wo= @@ -343,6 +345,8 @@ github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742 h1:Esafd1046DLDQ0W1YjYsBW+p8U2u7vzgW2SQVmlNazg= github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= +github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M= +github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk= github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223 h1:F9x/1yl3T2AeKLr2AMdilSD8+f9bvMnNN8VS5iDtovc= github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= github.com/oklog/oklog v0.2.3-0.20170918173356-f857583a70c3/go.mod h1:FCV+B7mhrz4o+ueLpx+KqkyXRGMWOYEvfiXtdGtbWGs= diff --git a/pkg/query-service/integrations/alertManager/model.go b/pkg/query-service/integrations/alertManager/model.go index bb709e430f..19371a9bfd 100644 --- a/pkg/query-service/integrations/alertManager/model.go +++ b/pkg/query-service/integrations/alertManager/model.go @@ -40,6 +40,8 @@ type Alert struct { StartsAt time.Time `json:"startsAt,omitempty"` EndsAt time.Time `json:"endsAt,omitempty"` GeneratorURL string `json:"generatorURL,omitempty"` + + Receivers []string `json:"receivers,omitempty"` } // Name returns the name of the alert. It is equivalent to the "alertname" label. @@ -53,7 +55,7 @@ func (a *Alert) Hash() uint64 { } func (a *Alert) String() string { - s := fmt.Sprintf("%s[%s]", a.Name(), fmt.Sprintf("%016x", a.Hash())[:7]) + s := fmt.Sprintf("%s[%s][%s]", a.Name(), fmt.Sprintf("%016x", a.Hash())[:7], a.Receivers) if a.Resolved() { return s + "[resolved]" } diff --git a/pkg/query-service/interfaces/interface.go b/pkg/query-service/interfaces/interface.go index 5e9d01be8b..14bc4b5d63 100644 --- a/pkg/query-service/interfaces/interface.go +++ b/pkg/query-service/interfaces/interface.go @@ -20,11 +20,13 @@ type Reader interface { GetInstantQueryMetricsResult(ctx context.Context, query *model.InstantQueryMetricsParams) (*promql.Result, *stats.QueryStats, *model.ApiError) GetQueryRangeResult(ctx context.Context, query *model.QueryRangeParams) (*promql.Result, *stats.QueryStats, *model.ApiError) GetServiceOverview(ctx context.Context, query *model.GetServiceOverviewParams) (*[]model.ServiceOverviewItem, *model.ApiError) + GetTopLevelOperations(ctx context.Context) (*map[string][]string, *model.ApiError) GetServices(ctx context.Context, query *model.GetServicesParams) (*[]model.ServiceItem, *model.ApiError) - GetTopEndpoints(ctx context.Context, query *model.GetTopEndpointsParams) (*[]model.TopEndpointsItem, *model.ApiError) + GetTopOperations(ctx context.Context, query *model.GetTopOperationsParams) (*[]model.TopOperationsItem, *model.ApiError) GetUsage(ctx context.Context, query *model.GetUsageParams) (*[]model.UsageItem, error) GetServicesList(ctx context.Context) (*[]string, error) - GetServiceMapDependencies(ctx context.Context, query *model.GetServicesParams) (*[]model.ServiceMapDependencyResponseItem, error) + GetDependencyGraph(ctx context.Context, query *model.GetServicesParams) (*[]model.ServiceMapDependencyResponseItem, error) + GetTTL(ctx context.Context, ttlParams *model.GetTTLParams) (*model.GetTTLResponseItem, *model.ApiError) // GetDisks returns a list of disks configured in the underlying DB. It is supported by diff --git a/pkg/query-service/model/queryParams.go b/pkg/query-service/model/queryParams.go index 2b964597ab..a215fb8d9b 100644 --- a/pkg/query-service/model/queryParams.go +++ b/pkg/query-service/model/queryParams.go @@ -135,7 +135,7 @@ type MetricAutocompleteTagParams struct { TagKey string } -type GetTopEndpointsParams struct { +type GetTopOperationsParams struct { StartTime string `json:"start"` EndTime string `json:"end"` ServiceName string `json:"service"` diff --git a/pkg/query-service/model/response.go b/pkg/query-service/model/response.go index 0bdaf02dc7..06ee6e6e2f 100644 --- a/pkg/query-service/model/response.go +++ b/pkg/query-service/model/response.go @@ -3,6 +3,7 @@ package model import ( "encoding/json" "fmt" + "math" "strconv" "time" @@ -205,19 +206,13 @@ func (item *SearchSpanReponseItem) GetValues() []interface{} { return returnArray } -type ServiceMapDependencyItem struct { - SpanId string `json:"spanId,omitempty" ch:"spanID"` - ParentSpanId string `json:"parentSpanId,omitempty" ch:"parentSpanID"` - ServiceName string `json:"serviceName,omitempty" ch:"serviceName"` -} - type UsageItem struct { Time time.Time `json:"time,omitempty" ch:"time"` Timestamp uint64 `json:"timestamp" ch:"timestamp"` Count uint64 `json:"count" ch:"count"` } -type TopEndpointsItem struct { +type TopOperationsItem struct { Percentile50 float64 `json:"p50" ch:"p50"` Percentile95 float64 `json:"p95" ch:"p95"` Percentile99 float64 `json:"p99" ch:"p99"` @@ -232,10 +227,18 @@ type TagFilters struct { type TagValues struct { TagValues string `json:"tagValues" ch:"tagValues"` } + type ServiceMapDependencyResponseItem struct { - Parent string `json:"parent,omitempty" ch:"parent"` - Child string `json:"child,omitempty" ch:"child"` - CallCount int `json:"callCount,omitempty" ch:"callCount"` + Parent string `json:"parent" ch:"parent"` + Child string `json:"child" ch:"child"` + CallCount uint64 `json:"callCount" ch:"callCount"` + CallRate float64 `json:"callRate" ch:"callRate"` + ErrorRate float64 `json:"errorRate" ch:"errorRate"` + P99 float64 `json:"p99" ch:"p99"` + P95 float64 `json:"p95" ch:"p95"` + P90 float64 `json:"p90" ch:"p90"` + P75 float64 `json:"p75" ch:"p75"` + P50 float64 `json:"p50" ch:"p50"` } type GetFilteredSpansAggregatesResponse struct { @@ -403,3 +406,30 @@ func (p *MetricPoint) MarshalJSON() ([]byte, error) { v := strconv.FormatFloat(p.Value, 'f', -1, 64) return json.Marshal([...]interface{}{float64(p.Timestamp) / 1000, v}) } + +// MarshalJSON implements json.Marshaler. +func (s *ServiceItem) MarshalJSON() ([]byte, error) { + // If a service didn't not send any data in the last interval duration + // it's values such as 99th percentile will return as NaN and + // json encoding doesn't support NaN + // We still want to show it in the UI, so we'll replace NaN with 0 + type Alias ServiceItem + if math.IsInf(s.AvgDuration, 0) || math.IsNaN(s.AvgDuration) { + s.AvgDuration = 0 + } + if math.IsInf(s.CallRate, 0) || math.IsNaN(s.CallRate) { + s.CallRate = 0 + } + if math.IsInf(s.ErrorRate, 0) || math.IsNaN(s.ErrorRate) { + s.ErrorRate = 0 + } + if math.IsInf(s.Percentile99, 0) || math.IsNaN(s.Percentile99) { + s.Percentile99 = 0 + } + + return json.Marshal(&struct { + *Alias + }{ + Alias: (*Alias)(s), + }) +} diff --git a/pkg/query-service/rules/alerting.go b/pkg/query-service/rules/alerting.go index a4768b4036..b7655733d0 100644 --- a/pkg/query-service/rules/alerting.go +++ b/pkg/query-service/rules/alerting.go @@ -2,12 +2,18 @@ package rules import ( "encoding/json" + "fmt" "github.com/pkg/errors" "go.signoz.io/query-service/model" "go.signoz.io/query-service/utils/labels" + "net/url" + "strings" "time" ) +// this file contains common structs and methods used by +// rule engine + // how long before re-sending the alert const resolvedRetention = 15 * time.Minute @@ -17,6 +23,8 @@ const ( // AlertForStateMetricName is the metric name for 'for' state of alert. alertForStateMetricName = "ALERTS_FOR_STATE" + + TestAlertPostFix = "_TEST_ALERT" ) type RuleType string @@ -41,6 +49,7 @@ const ( StateInactive AlertState = iota StatePending StateFiring + StateDisabled ) func (s AlertState) String() string { @@ -51,6 +60,8 @@ func (s AlertState) String() string { return "pending" case StateFiring: return "firing" + case StateDisabled: + return "disabled" } panic(errors.Errorf("unknown alert state: %d", s)) } @@ -63,6 +74,9 @@ type Alert struct { GeneratorURL string + // list of preferred receivers, e.g. slack + Receivers []string + Value float64 ActiveAt time.Time FiredAt time.Time @@ -71,7 +85,6 @@ type Alert struct { ValidUntil time.Time } -// todo(amol): need to review this with ankit func (a *Alert) needsSending(ts time.Time, resendDelay time.Duration) bool { if a.State == StatePending { return false @@ -198,3 +211,30 @@ func (d *Duration) UnmarshalJSON(b []byte) error { return errors.New("invalid duration") } } + +// prepareRuleGeneratorURL creates an appropriate url +// for the rule. the URL is sent in slack messages as well as +// to other systems and allows backtracking to the rule definition +// from the third party systems. +func prepareRuleGeneratorURL(ruleId string, source string) string { + if source == "" { + return source + } + + // check if source is a valid url + _, err := url.Parse(source) + if err != nil { + return "" + } + // since we capture window.location when a new rule is created + // we end up with rulesource host:port/alerts/new. in this case + // we want to replace new with rule id parameter + + hasNew := strings.LastIndex(source, "new") + if hasNew > -1 { + ruleURL := fmt.Sprintf("%sedit?ruleId=%s", source[0:hasNew], ruleId) + return ruleURL + } + + return source +} diff --git a/pkg/query-service/rules/apiParams.go b/pkg/query-service/rules/apiParams.go index 6f3b466d11..1d488c026d 100644 --- a/pkg/query-service/rules/apiParams.go +++ b/pkg/query-service/rules/apiParams.go @@ -18,6 +18,16 @@ import ( // this file contains api request and responses to be // served over http +// newApiErrorInternal returns a new api error object of type internal +func newApiErrorInternal(err error) *model.ApiError { + return &model.ApiError{Typ: model.ErrorInternal, Err: err} +} + +// newApiErrorBadData returns a new api error object of bad request type +func newApiErrorBadData(err error) *model.ApiError { + return &model.ApiError{Typ: model.ErrorBadData, Err: err} +} + // PostableRule is used to create alerting rule from HTTP api type PostableRule struct { Alert string `yaml:"alert,omitempty" json:"alert,omitempty"` @@ -30,9 +40,13 @@ type PostableRule struct { Labels map[string]string `yaml:"labels,omitempty" json:"labels,omitempty"` Annotations map[string]string `yaml:"annotations,omitempty" json:"annotations,omitempty"` + Disabled bool `json:"disabled"` + // Source captures the source url where rule has been created Source string `json:"source,omitempty"` + PreferredChannels []string `json:"preferredChannels,omitempty"` + // legacy Expr string `yaml:"expr,omitempty" json:"expr,omitempty"` OldYaml string `json:"yaml,omitempty"` @@ -43,16 +57,23 @@ func ParsePostableRule(content []byte) (*PostableRule, []error) { } func parsePostableRule(content []byte, kind string) (*PostableRule, []error) { - rule := PostableRule{} + return parseIntoRule(PostableRule{}, content, kind) +} + +// parseIntoRule loads the content (data) into PostableRule and also +// validates the end result +func parseIntoRule(initRule PostableRule, content []byte, kind string) (*PostableRule, []error) { + + rule := &initRule var err error if kind == "json" { - if err = json.Unmarshal(content, &rule); err != nil { + if err = json.Unmarshal(content, rule); err != nil { zap.S().Debugf("postable rule content", string(content), "\t kind:", kind) return nil, []error{fmt.Errorf("failed to load json")} } } else if kind == "yaml" { - if err = yaml.Unmarshal(content, &rule); err != nil { + if err = yaml.Unmarshal(content, rule); err != nil { zap.S().Debugf("postable rule content", string(content), "\t kind:", kind) return nil, []error{fmt.Errorf("failed to load yaml")} } @@ -105,7 +126,8 @@ func parsePostableRule(content []byte, kind string) (*PostableRule, []error) { if errs := rule.Validate(); len(errs) > 0 { return nil, errs } - return &rule, []error{} + + return rule, []error{} } func isValidLabelName(ln string) bool { @@ -213,18 +235,7 @@ type GettableRules struct { // GettableRule has info for an alerting rules. type GettableRule struct { - Labels map[string]string `json:"labels"` - Annotations map[string]string `json:"annotations"` - State string `json:"state"` - Alert string `json:"alert"` - // Description string `yaml:"description,omitempty" json:"description,omitempty"` - - Id string `json:"id"` - RuleType RuleType `yaml:"ruleType,omitempty" json:"ruleType,omitempty"` - EvalWindow Duration `yaml:"evalWindow,omitempty" json:"evalWindow,omitempty"` - Frequency Duration `yaml:"frequency,omitempty" json:"frequency,omitempty"` - RuleCondition RuleCondition `yaml:"condition,omitempty" json:"condition,omitempty"` - - // ActiveAt *time.Time `json:"activeAt,omitempty"` - // Value float64 `json:"value"` + Id string `json:"id"` + State string `json:"state"` + PostableRule } diff --git a/pkg/query-service/rules/manager.go b/pkg/query-service/rules/manager.go index 9a040fdf74..bf1e70b956 100644 --- a/pkg/query-service/rules/manager.go +++ b/pkg/query-service/rules/manager.go @@ -4,6 +4,7 @@ import ( "context" "encoding/json" "fmt" + "github.com/google/uuid" "sort" "strconv" "strings" @@ -19,6 +20,8 @@ import ( // opentracing "github.com/opentracing/opentracing-go" am "go.signoz.io/query-service/integrations/alertManager" + "go.signoz.io/query-service/model" + "go.signoz.io/query-service/utils/labels" ) // namespace for prom metrics @@ -29,8 +32,15 @@ func ruleIdFromTaskName(n string) string { return strings.Split(n, "-groupname")[0] } -func prepareTaskName(ruleId int64) string { - return fmt.Sprintf("%d-groupname", ruleId) +func prepareTaskName(ruleId interface{}) string { + switch ruleId.(type) { + case int, int64: + return fmt.Sprintf("%d-groupname", ruleId) + case string: + return fmt.Sprintf("%s-groupname", ruleId) + default: + return fmt.Sprintf("%v-groupname", ruleId) + } } // ManagerOptions bundles options for the Manager. @@ -170,10 +180,11 @@ func (m *Manager) initiate() error { continue } } - - err := m.addTask(parsedRule, taskName) - if err != nil { - zap.S().Errorf("failed to load the rule definition (%s): %v", taskName, err) + if !parsedRule.Disabled { + err := m.addTask(parsedRule, taskName) + if err != nil { + zap.S().Errorf("failed to load the rule definition (%s): %v", taskName, err) + } } } @@ -206,7 +217,7 @@ func (m *Manager) Stop() { // EditRuleDefinition writes the rule definition to the // datastore and also updates the rule executor func (m *Manager) EditRule(ruleStr string, id string) error { - // todo(amol): fetch recent rule from db first + parsedRule, errs := ParsePostableRule([]byte(ruleStr)) if len(errs) > 0 { @@ -221,16 +232,9 @@ func (m *Manager) EditRule(ruleStr string, id string) error { } if !m.opts.DisableRules { - err = m.editTask(parsedRule, taskName) - if err != nil { - // todo(amol): using tx with sqllite3 is gets - // database locked. need to research and resolve this - //tx.Rollback() - return err - } + return m.syncRuleStateWithTask(taskName, parsedRule) } - // return tx.Commit() return nil } @@ -249,8 +253,7 @@ func (m *Manager) editTask(rule *PostableRule, taskName string) error { // it to finish the current iteration. Then copy it into the new group. oldTask, ok := m.tasks[taskName] if !ok { - zap.S().Errorf("msg:", "rule task not found, edit task failed", "\t task name:", taskName) - return errors.New("rule task not found, edit task failed") + zap.S().Warnf("msg:", "rule task not found, a new task will be created ", "\t task name:", taskName) } delete(m.tasks, taskName) @@ -281,10 +284,7 @@ func (m *Manager) DeleteRule(id string) error { taskName := prepareTaskName(int64(idInt)) if !m.opts.DisableRules { - if err := m.deleteTask(taskName); err != nil { - zap.S().Errorf("msg: ", "failed to unload the rule task from memory, please retry", "\t ruleid: ", id) - return err - } + m.deleteTask(taskName) } if _, _, err := m.ruleDB.DeleteRuleTx(id); err != nil { @@ -295,7 +295,7 @@ func (m *Manager) DeleteRule(id string) error { return nil } -func (m *Manager) deleteTask(taskName string) error { +func (m *Manager) deleteTask(taskName string) { m.mtx.Lock() defer m.mtx.Unlock() @@ -305,11 +305,8 @@ func (m *Manager) deleteTask(taskName string) error { delete(m.tasks, taskName) delete(m.rules, ruleIdFromTaskName(taskName)) } else { - zap.S().Errorf("msg:", "rule not found for deletion", "\t name:", taskName) - return fmt.Errorf("rule not found") + zap.S().Info("msg: ", "rule not found for deletion", "\t name:", taskName) } - - return nil } // CreateRule stores rule def into db and also @@ -386,12 +383,8 @@ func (m *Manager) prepareTask(acquireLock bool, r *PostableRule, taskName string // create a threshold rule tr, err := NewThresholdRule( ruleId, - r.Alert, - r.RuleCondition, - time.Duration(r.EvalWindow), - r.Labels, - r.Annotations, - r.Source, + r, + ThresholdRuleOpts{}, ) if err != nil { @@ -411,14 +404,9 @@ func (m *Manager) prepareTask(acquireLock bool, r *PostableRule, taskName string // create promql rule pr, err := NewPromRule( ruleId, - r.Alert, - r.RuleCondition, - time.Duration(r.EvalWindow), - r.Labels, - r.Annotations, - // required as promql engine works with logger and not zap + r, log.With(m.logger, "alert", r.Alert), - r.Source, + PromRuleOpts{}, ) if err != nil { @@ -526,6 +514,7 @@ func (m *Manager) prepareNotifyFunc() NotifyFunc { Labels: alert.Labels, Annotations: alert.Annotations, GeneratorURL: generatorURL, + Receivers: alert.Receivers, } if !alert.ResolvedAt.IsZero() { a.EndsAt = alert.ResolvedAt @@ -555,6 +544,9 @@ func (m *Manager) ListRuleStates() (*GettableRules, error) { // fetch rules from DB storedRules, err := m.ruleDB.GetStoredRules() + if err != nil { + return nil, err + } // initiate response object resp := make([]*GettableRule, 0) @@ -571,7 +563,8 @@ func (m *Manager) ListRuleStates() (*GettableRules, error) { // fetch state of rule from memory if rm, ok := m.rules[ruleResponse.Id]; !ok { - zap.S().Warnf("msg:", "invalid rule id found while fetching list of rules", "\t err:", err, "\t rule_id:", ruleResponse.Id) + ruleResponse.State = StateDisabled.String() + ruleResponse.Disabled = true } else { ruleResponse.State = rm.State().String() } @@ -593,3 +586,185 @@ func (m *Manager) GetRule(id string) (*GettableRule, error) { r.Id = fmt.Sprintf("%d", s.Id) return r, nil } + +// syncRuleStateWithTask ensures that the state of a stored rule matches +// the task state. For example - if a stored rule is disabled, then +// there is no task running against it. +func (m *Manager) syncRuleStateWithTask(taskName string, rule *PostableRule) error { + + if rule.Disabled { + // check if rule has any task running + if _, ok := m.tasks[taskName]; ok { + // delete task from memory + m.deleteTask(taskName) + } + } else { + // check if rule has a task running + if _, ok := m.tasks[taskName]; !ok { + // rule has not task, start one + if err := m.addTask(rule, taskName); err != nil { + return err + } + } + } + return nil +} + +// PatchRule supports attribute level changes to the rule definition unlike +// EditRule, which updates entire rule definition in the DB. +// the process: +// - get the latest rule from db +// - over write the patch attributes received in input (ruleStr) +// - re-deploy or undeploy task as necessary +// - update the patched rule in the DB +func (m *Manager) PatchRule(ruleStr string, ruleId string) (*GettableRule, error) { + + if ruleId == "" { + return nil, fmt.Errorf("id is mandatory for patching rule") + } + + taskName := prepareTaskName(ruleId) + + // retrieve rule from DB + storedJSON, err := m.ruleDB.GetStoredRule(ruleId) + if err != nil { + zap.S().Errorf("msg:", "failed to get stored rule with given id", "\t error:", err) + return nil, err + } + + // storedRule holds the current stored rule from DB + storedRule := PostableRule{} + if err := json.Unmarshal([]byte(storedJSON.Data), &storedRule); err != nil { + zap.S().Errorf("msg:", "failed to get unmarshal stored rule with given id", "\t error:", err) + return nil, err + } + + // patchedRule is combo of stored rule and patch received in the request + patchedRule, errs := parseIntoRule(storedRule, []byte(ruleStr), "json") + if len(errs) > 0 { + zap.S().Errorf("failed to parse rules:", errs) + // just one rule is being parsed so expect just one error + return nil, errs[0] + } + + // deploy or un-deploy task according to patched (new) rule state + if err := m.syncRuleStateWithTask(taskName, patchedRule); err != nil { + zap.S().Errorf("failed to sync stored rule state with the task") + return nil, err + } + + // prepare rule json to write to update db + patchedRuleBytes, err := json.Marshal(patchedRule) + if err != nil { + return nil, err + } + + // write updated rule to db + if _, _, err = m.ruleDB.EditRuleTx(string(patchedRuleBytes), ruleId); err != nil { + // write failed, rollback task state + + // restore task state from the stored rule + if err := m.syncRuleStateWithTask(taskName, &storedRule); err != nil { + zap.S().Errorf("msg: ", "failed to restore rule after patch failure", "\t error:", err) + } + + return nil, err + } + + // prepare http response + response := GettableRule{ + Id: ruleId, + PostableRule: *patchedRule, + } + + // fetch state of rule from memory + if rm, ok := m.rules[ruleId]; !ok { + response.State = StateDisabled.String() + response.Disabled = true + } else { + response.State = rm.State().String() + } + + return &response, nil +} + +// TestNotification prepares a dummy rule for given rule parameters and +// sends a test notification. returns alert count and error (if any) +func (m *Manager) TestNotification(ctx context.Context, ruleStr string) (int, *model.ApiError) { + + parsedRule, errs := ParsePostableRule([]byte(ruleStr)) + + if len(errs) > 0 { + zap.S().Errorf("msg: failed to parse rule from request:", "\t error: ", errs) + return 0, newApiErrorBadData(errs[0]) + } + + var alertname = parsedRule.Alert + if alertname == "" { + // alertname is not mandatory for testing, so picking + // a random string here + alertname = uuid.New().String() + } + + // append name to indicate this is test alert + parsedRule.Alert = fmt.Sprintf("%s%s", alertname, TestAlertPostFix) + + var rule Rule + var err error + + if parsedRule.RuleType == RuleTypeThreshold { + + // add special labels for test alerts + parsedRule.Labels[labels.AlertAdditionalInfoLabel] = fmt.Sprintf("The rule threshold is set to %.4f, and the observed metric value is {{$value}}.", *parsedRule.RuleCondition.Target) + parsedRule.Annotations[labels.AlertSummaryLabel] = fmt.Sprintf("The rule threshold is set to %.4f, and the observed metric value is {{$value}}.", *parsedRule.RuleCondition.Target) + parsedRule.Labels[labels.RuleSourceLabel] = "" + parsedRule.Labels[labels.AlertRuleIdLabel] = "" + + // create a threshold rule + rule, err = NewThresholdRule( + alertname, + parsedRule, + ThresholdRuleOpts{ + SendUnmatched: true, + SendAlways: true, + }, + ) + + if err != nil { + zap.S().Errorf("msg: failed to prepare a new threshold rule for test:", "\t error: ", err) + return 0, newApiErrorBadData(err) + } + + } else if parsedRule.RuleType == RuleTypeProm { + + // create promql rule + rule, err = NewPromRule( + alertname, + parsedRule, + log.With(m.logger, "alert", alertname), + PromRuleOpts{ + SendAlways: true, + }, + ) + + if err != nil { + zap.S().Errorf("msg: failed to prepare a new promql rule for test:", "\t error: ", err) + return 0, newApiErrorBadData(err) + } + } else { + return 0, newApiErrorBadData(fmt.Errorf("failed to derive ruletype with given information")) + } + + // set timestamp to current utc time + ts := time.Now().UTC() + + count, err := rule.Eval(ctx, ts, m.opts.Queriers) + if err != nil { + zap.S().Warn("msg:", "Evaluating rule failed", "\t rule:", rule, "\t err: ", err) + return 0, newApiErrorInternal(fmt.Errorf("rule evaluation failed")) + } + alertsFound := count.(int) + rule.SendAlerts(ctx, ts, 0, time.Duration(1*time.Minute), m.prepareNotifyFunc()) + + return alertsFound, nil +} diff --git a/pkg/query-service/rules/promRule.go b/pkg/query-service/rules/promRule.go index 669d6e3845..761ca8ddee 100644 --- a/pkg/query-service/rules/promRule.go +++ b/pkg/query-service/rules/promRule.go @@ -18,6 +18,12 @@ import ( yaml "gopkg.in/yaml.v2" ) +type PromRuleOpts struct { + // SendAlways will send alert irresepective of resendDelay + // or other params + SendAlways bool +} + type PromRule struct { id string name string @@ -29,6 +35,8 @@ type PromRule struct { labels plabels.Labels annotations plabels.Labels + preferredChannels []string + mtx sync.Mutex evaluationDuration time.Duration evaluationTimestamp time.Time @@ -41,42 +49,50 @@ type PromRule struct { active map[uint64]*Alert logger log.Logger + opts PromRuleOpts } func NewPromRule( id string, - name string, - ruleCondition *RuleCondition, - evalWindow time.Duration, - labels, annotations map[string]string, + postableRule *PostableRule, logger log.Logger, - source string, + opts PromRuleOpts, ) (*PromRule, error) { - if int64(evalWindow) == 0 { - evalWindow = 5 * time.Minute - } - - if ruleCondition == nil { + if postableRule.RuleCondition == nil { return nil, fmt.Errorf("no rule condition") - } else if !ruleCondition.IsValid() { + } else if !postableRule.RuleCondition.IsValid() { return nil, fmt.Errorf("invalid rule condition") } - zap.S().Info("msg:", "creating new alerting rule", "\t name:", name, "\t condition:", ruleCondition.String()) + p := PromRule{ + id: id, + name: postableRule.Alert, + source: postableRule.Source, + ruleCondition: postableRule.RuleCondition, + evalWindow: time.Duration(postableRule.EvalWindow), + labels: plabels.FromMap(postableRule.Labels), + annotations: plabels.FromMap(postableRule.Annotations), + preferredChannels: postableRule.PreferredChannels, + health: HealthUnknown, + active: map[uint64]*Alert{}, + logger: logger, + opts: opts, + } - return &PromRule{ - id: id, - name: name, - source: source, - ruleCondition: ruleCondition, - evalWindow: evalWindow, - labels: plabels.FromMap(labels), - annotations: plabels.FromMap(annotations), - health: HealthUnknown, - active: map[uint64]*Alert{}, - logger: logger, - }, nil + if int64(p.evalWindow) == 0 { + p.evalWindow = 5 * time.Minute + } + query, err := p.getPqlQuery() + + if err != nil { + // can not generate a valid prom QL query + return nil, err + } + + zap.S().Info("msg:", "creating new alerting rule", "\t name:", p.name, "\t condition:", p.ruleCondition.String(), "\t query:", query) + + return &p, nil } func (r *PromRule) Name() string { @@ -96,7 +112,11 @@ func (r *PromRule) Type() RuleType { } func (r *PromRule) GeneratorURL() string { - return r.source + return prepareRuleGeneratorURL(r.ID(), r.source) +} + +func (r *PromRule) PreferredChannels() []string { + return r.preferredChannels } func (r *PromRule) SetLastError(err error) { @@ -167,24 +187,6 @@ func (r *PromRule) sample(alert *Alert, ts time.Time) pql.Sample { return s } -// forStateSample returns the sample for ALERTS_FOR_STATE. -func (r *PromRule) forStateSample(alert *Alert, ts time.Time, v float64) pql.Sample { - lb := plabels.NewBuilder(r.labels) - alertLabels := alert.Labels.(plabels.Labels) - for _, l := range alertLabels { - lb.Set(l.Name, l.Value) - } - - lb.Set(plabels.MetricName, alertForStateMetricName) - lb.Set(plabels.AlertName, r.name) - - s := pql.Sample{ - Metric: lb.Labels(), - Point: pql.Point{T: timestamp.FromTime(ts), V: v}, - } - return s -} - // GetEvaluationDuration returns the time in seconds it took to evaluate the alerting rule. func (r *PromRule) GetEvaluationDuration() time.Duration { r.mtx.Lock() @@ -260,7 +262,7 @@ func (r *PromRule) ForEachActiveAlert(f func(*Alert)) { func (r *PromRule) SendAlerts(ctx context.Context, ts time.Time, resendDelay time.Duration, interval time.Duration, notifyFunc NotifyFunc) { alerts := []*Alert{} r.ForEachActiveAlert(func(alert *Alert) { - if alert.needsSending(ts, resendDelay) { + if r.opts.SendAlways || alert.needsSending(ts, resendDelay) { alert.LastSentAt = ts // Allow for two Eval or Alertmanager send failures. delta := resendDelay @@ -284,7 +286,6 @@ func (r *PromRule) getPqlQuery() (string, error) { if query == "" { return query, fmt.Errorf("a promquery needs to be set for this rule to function") } - if r.ruleCondition.Target != nil && r.ruleCondition.CompareOp != CompareOpNone { query = fmt.Sprintf("%s %s %f", query, ResolveCompareOp(r.ruleCondition.CompareOp), *r.ruleCondition.Target) return query, nil @@ -316,7 +317,7 @@ func (r *PromRule) Eval(ctx context.Context, ts time.Time, queriers *Queriers) ( defer r.mtx.Unlock() resultFPs := map[uint64]struct{}{} - var vec pql.Vector + var alerts = make(map[uint64]*Alert, len(res)) for _, smpl := range res { @@ -353,6 +354,7 @@ func (r *PromRule) Eval(ctx context.Context, ts time.Time, queriers *Queriers) ( for _, l := range r.labels { lb.Set(l.Name, expand(l.Value)) } + lb.Set(qslabels.AlertNameLabel, r.Name()) lb.Set(qslabels.AlertRuleIdLabel, r.ID()) lb.Set(qslabels.RuleSourceLabel, r.GeneratorURL()) @@ -382,6 +384,7 @@ func (r *PromRule) Eval(ctx context.Context, ts time.Time, queriers *Queriers) ( State: StatePending, Value: smpl.V, GeneratorURL: r.GeneratorURL(), + Receivers: r.preferredChannels, } } @@ -392,6 +395,7 @@ func (r *PromRule) Eval(ctx context.Context, ts time.Time, queriers *Queriers) ( if alert, ok := r.active[h]; ok && alert.State != StateInactive { alert.Value = a.Value alert.Annotations = a.Annotations + alert.Receivers = r.preferredChannels continue } @@ -422,18 +426,19 @@ func (r *PromRule) Eval(ctx context.Context, ts time.Time, queriers *Queriers) ( } r.health = HealthGood r.lastError = err - return vec, nil + return len(r.active), nil } func (r *PromRule) String() string { ar := PostableRule{ - Alert: r.name, - RuleCondition: r.ruleCondition, - EvalWindow: Duration(r.evalWindow), - Labels: r.labels.Map(), - Annotations: r.annotations.Map(), + Alert: r.name, + RuleCondition: r.ruleCondition, + EvalWindow: Duration(r.evalWindow), + Labels: r.labels.Map(), + Annotations: r.annotations.Map(), + PreferredChannels: r.preferredChannels, } byt, err := yaml.Marshal(ar) diff --git a/pkg/query-service/rules/promRuleTask.go b/pkg/query-service/rules/promRuleTask.go index c06d3e1135..fa84d80d57 100644 --- a/pkg/query-service/rules/promRuleTask.go +++ b/pkg/query-service/rules/promRuleTask.go @@ -6,7 +6,6 @@ import ( "github.com/go-kit/log" opentracing "github.com/opentracing/opentracing-go" plabels "github.com/prometheus/prometheus/pkg/labels" - pql "github.com/prometheus/prometheus/promql" "go.uber.org/zap" "sort" "sync" @@ -313,7 +312,6 @@ func (g *PromRuleTask) CopyState(fromTask Task) error { // Eval runs a single evaluation cycle in which all rules are evaluated sequentially. func (g *PromRuleTask) Eval(ctx context.Context, ts time.Time) { zap.S().Info("promql rule task:", g.name, "\t eval started at:", ts) - var samplesTotal float64 for i, rule := range g.rules { if rule == nil { continue @@ -336,7 +334,7 @@ func (g *PromRuleTask) Eval(ctx context.Context, ts time.Time) { rule.SetEvaluationTimestamp(t) }(time.Now()) - data, err := rule.Eval(ctx, ts, g.opts.Queriers) + _, err := rule.Eval(ctx, ts, g.opts.Queriers) if err != nil { rule.SetHealth(HealthBad) rule.SetLastError(err) @@ -350,21 +348,8 @@ func (g *PromRuleTask) Eval(ctx context.Context, ts time.Time) { //} return } - vector := data.(pql.Vector) - samplesTotal += float64(len(vector)) - rule.SendAlerts(ctx, ts, g.opts.ResendDelay, g.frequency, g.notify) - seriesReturned := make(map[string]plabels.Labels, len(g.seriesInPreviousEval[i])) - - defer func() { - g.seriesInPreviousEval[i] = seriesReturned - }() - - for _, s := range vector { - seriesReturned[s.Metric.String()] = s.Metric - } - }(i, rule) } } diff --git a/pkg/query-service/rules/rule.go b/pkg/query-service/rules/rule.go index ba5c934172..9a2ac1bad0 100644 --- a/pkg/query-service/rules/rule.go +++ b/pkg/query-service/rules/rule.go @@ -19,6 +19,8 @@ type Rule interface { State() AlertState ActiveAlerts() []*Alert + PreferredChannels() []string + Eval(context.Context, time.Time, *Queriers) (interface{}, error) String() string // Query() string diff --git a/pkg/query-service/rules/ruleTask.go b/pkg/query-service/rules/ruleTask.go index 59b25f05e0..4075d9888e 100644 --- a/pkg/query-service/rules/ruleTask.go +++ b/pkg/query-service/rules/ruleTask.go @@ -14,17 +14,15 @@ import ( // RuleTask holds a rule (with composite queries) // and evaluates the rule at a given frequency type RuleTask struct { - name string - file string - frequency time.Duration - rules []Rule - seriesInPreviousEval []map[string]labels.Labels // One per Rule. - staleSeries []labels.Labels - opts *ManagerOptions - mtx sync.Mutex - evaluationDuration time.Duration - evaluationTime time.Duration - lastEvaluation time.Time + name string + file string + frequency time.Duration + rules []Rule + opts *ManagerOptions + mtx sync.Mutex + evaluationDuration time.Duration + evaluationTime time.Duration + lastEvaluation time.Time markStale bool done chan struct{} @@ -46,16 +44,15 @@ func newRuleTask(name, file string, frequency time.Duration, rules []Rule, opts zap.S().Info("msg:", "initiating a new rule task", "\t name:", name, "\t frequency:", frequency) return &RuleTask{ - name: name, - file: file, - pause: false, - frequency: frequency, - rules: rules, - opts: opts, - seriesInPreviousEval: make([]map[string]labels.Labels, len(rules)), - done: make(chan struct{}), - terminated: make(chan struct{}), - notify: notify, + name: name, + file: file, + pause: false, + frequency: frequency, + rules: rules, + opts: opts, + done: make(chan struct{}), + terminated: make(chan struct{}), + notify: notify, } } @@ -126,24 +123,6 @@ func (g *RuleTask) Run(ctx context.Context) { tick := time.NewTicker(g.frequency) defer tick.Stop() - // defer cleanup - defer func() { - if !g.markStale { - return - } - go func(now time.Time) { - for _, rule := range g.seriesInPreviousEval { - for _, r := range rule { - g.staleSeries = append(g.staleSeries, r) - } - } - // That can be garbage collected at this point. - g.seriesInPreviousEval = nil - - }(time.Now()) - - }() - iter() // let the group iterate and run @@ -285,17 +264,15 @@ func (g *RuleTask) CopyState(fromTask Task) error { ruleMap[nameAndLabels] = append(l, fi) } - for i, rule := range g.rules { + for _, rule := range g.rules { nameAndLabels := nameAndLabels(rule) indexes := ruleMap[nameAndLabels] if len(indexes) == 0 { continue } fi := indexes[0] - g.seriesInPreviousEval[i] = from.seriesInPreviousEval[fi] ruleMap[nameAndLabels] = indexes[1:] - // todo(amol): support other rules too here ar, ok := rule.(*ThresholdRule) if !ok { continue @@ -310,18 +287,6 @@ func (g *RuleTask) CopyState(fromTask Task) error { } } - // Handle deleted and unmatched duplicate rules. - // todo(amol): possibly not needed any more - g.staleSeries = from.staleSeries - for fi, fromRule := range from.rules { - nameAndLabels := nameAndLabels(fromRule) - l := ruleMap[nameAndLabels] - if len(l) != 0 { - for _, series := range from.seriesInPreviousEval[fi] { - g.staleSeries = append(g.staleSeries, series) - } - } - } return nil } @@ -330,7 +295,6 @@ func (g *RuleTask) Eval(ctx context.Context, ts time.Time) { zap.S().Debugf("msg:", "rule task eval started", "\t name:", g.name, "\t start time:", ts) - var samplesTotal float64 for i, rule := range g.rules { if rule == nil { continue @@ -353,7 +317,7 @@ func (g *RuleTask) Eval(ctx context.Context, ts time.Time) { rule.SetEvaluationTimestamp(t) }(time.Now()) - data, err := rule.Eval(ctx, ts, g.opts.Queriers) + _, err := rule.Eval(ctx, ts, g.opts.Queriers) if err != nil { rule.SetHealth(HealthBad) rule.SetLastError(err) @@ -368,18 +332,8 @@ func (g *RuleTask) Eval(ctx context.Context, ts time.Time) { return } - vector := data.(Vector) - samplesTotal += float64(len(vector)) - rule.SendAlerts(ctx, ts, g.opts.ResendDelay, g.frequency, g.notify) - seriesReturned := make(map[string]labels.Labels, len(g.seriesInPreviousEval[i])) - - for _, s := range vector { - seriesReturned[s.Metric.String()] = s.Metric - } - - g.seriesInPreviousEval[i] = seriesReturned }(i, rule) } } diff --git a/pkg/query-service/rules/templates.go b/pkg/query-service/rules/templates.go index 4789780ffc..3d9aa518d8 100644 --- a/pkg/query-service/rules/templates.go +++ b/pkg/query-service/rules/templates.go @@ -17,6 +17,9 @@ import ( "go.signoz.io/query-service/utils/times" ) +// this file contains all the methods and structs +// related to go templating in rule labels and annotations + type tmplQueryRecord struct { Labels map[string]string Value float64 diff --git a/pkg/query-service/rules/thresholdRule.go b/pkg/query-service/rules/thresholdRule.go index 8f734c113d..5234e88a72 100644 --- a/pkg/query-service/rules/thresholdRule.go +++ b/pkg/query-service/rules/thresholdRule.go @@ -32,6 +32,7 @@ type ThresholdRule struct { labels labels.Labels annotations labels.Labels + preferredChannels []string mtx sync.Mutex evaluationDuration time.Duration evaluationTimestamp time.Time @@ -42,41 +43,54 @@ type ThresholdRule struct { // map of active alerts active map[uint64]*Alert + + opts ThresholdRuleOpts +} + +type ThresholdRuleOpts struct { + // sendUnmatched sends observed metric values + // even if they dont match the rule condition. this is + // useful in testing the rule + SendUnmatched bool + + // sendAlways will send alert irresepective of resendDelay + // or other params + SendAlways bool } func NewThresholdRule( id string, - name string, - ruleCondition *RuleCondition, - evalWindow time.Duration, - l, a map[string]string, - source string, + p *PostableRule, + opts ThresholdRuleOpts, ) (*ThresholdRule, error) { - if int64(evalWindow) == 0 { - evalWindow = 5 * time.Minute - } - - if ruleCondition == nil { + if p.RuleCondition == nil { return nil, fmt.Errorf("no rule condition") - } else if !ruleCondition.IsValid() { + } else if !p.RuleCondition.IsValid() { return nil, fmt.Errorf("invalid rule condition") } - zap.S().Info("msg:", "creating new alerting rule", "\t name:", name, "\t condition:", ruleCondition.String()) + t := ThresholdRule{ + id: id, + name: p.Alert, + source: p.Source, + ruleCondition: p.RuleCondition, + evalWindow: time.Duration(p.EvalWindow), + labels: labels.FromMap(p.Labels), + annotations: labels.FromMap(p.Annotations), + preferredChannels: p.PreferredChannels, + health: HealthUnknown, + active: map[uint64]*Alert{}, + opts: opts, + } - return &ThresholdRule{ - id: id, - name: name, - source: source, - ruleCondition: ruleCondition, - evalWindow: evalWindow, - labels: labels.FromMap(l), - annotations: labels.FromMap(a), + if int64(t.evalWindow) == 0 { + t.evalWindow = 5 * time.Minute + } - health: HealthUnknown, - active: map[uint64]*Alert{}, - }, nil + zap.S().Info("msg:", "creating new alerting rule", "\t name:", t.name, "\t condition:", t.ruleCondition.String(), "\t generatorURL:", t.GeneratorURL()) + + return &t, nil } func (r *ThresholdRule) Name() string { @@ -92,7 +106,11 @@ func (r *ThresholdRule) Condition() *RuleCondition { } func (r *ThresholdRule) GeneratorURL() string { - return r.source + return prepareRuleGeneratorURL(r.ID(), r.source) +} + +func (r *ThresholdRule) PreferredChannels() []string { + return r.preferredChannels } func (r *ThresholdRule) target() *float64 { @@ -102,6 +120,14 @@ func (r *ThresholdRule) target() *float64 { return r.ruleCondition.Target } +func (r *ThresholdRule) targetVal() float64 { + if r.ruleCondition == nil || r.ruleCondition.Target == nil { + return 0 + } + + return *r.ruleCondition.Target +} + func (r *ThresholdRule) matchType() MatchType { if r.ruleCondition == nil { return AtleastOnce @@ -185,25 +211,7 @@ func (r *ThresholdRule) sample(alert *Alert, ts time.Time) Sample { Metric: lb.Labels(), Point: Point{T: timestamp.FromTime(ts), V: 1}, } - return s -} -// forStateSample returns the sample for ALERTS_FOR_STATE. -func (r *ThresholdRule) forStateSample(alert *Alert, ts time.Time, v float64) Sample { - lb := labels.NewBuilder(r.labels) - - alertLabels := alert.Labels.(labels.Labels) - for _, l := range alertLabels { - lb.Set(l.Name, l.Value) - } - - lb.Set(labels.MetricNameLabel, alertForStateMetricName) - lb.Set(labels.AlertNameLabel, r.name) - - s := Sample{ - Metric: lb.Labels(), - Point: Point{T: timestamp.FromTime(ts), V: v}, - } return s } @@ -231,9 +239,9 @@ func (r *ThresholdRule) GetEvaluationTimestamp() time.Time { // State returns the maximum state of alert instances for this rule. // StateFiring > StatePending > StateInactive func (r *ThresholdRule) State() AlertState { + r.mtx.Lock() defer r.mtx.Unlock() - maxState := StateInactive for _, a := range r.active { if a.State > maxState { @@ -280,10 +288,10 @@ func (r *ThresholdRule) ForEachActiveAlert(f func(*Alert)) { } func (r *ThresholdRule) SendAlerts(ctx context.Context, ts time.Time, resendDelay time.Duration, interval time.Duration, notifyFunc NotifyFunc) { - zap.S().Info("msg:", "initiating send alerts (if any)", "\t rule:", r.Name()) + zap.S().Info("msg:", "sending alerts", "\t rule:", r.Name()) alerts := []*Alert{} r.ForEachActiveAlert(func(alert *Alert) { - if alert.needsSending(ts, resendDelay) { + if r.opts.SendAlways || alert.needsSending(ts, resendDelay) { alert.LastSentAt = ts // Allow for two Eval or Alertmanager send failures. delta := resendDelay @@ -477,14 +485,16 @@ func (r *ThresholdRule) runChQuery(ctx context.Context, db clickhouse.Conn, quer } } } + zap.S().Debugf("ruleid:", r.ID(), "\t resultmap(potential alerts):", len(resultMap)) for _, sample := range resultMap { - // check alert rule condition before dumping results - if r.CheckCondition(sample.Point.V) { + // check alert rule condition before dumping results, if sendUnmatchedResults + // is set then add results irrespective of condition + if r.opts.SendUnmatched || r.CheckCondition(sample.Point.V) { result = append(result, sample) } } - + zap.S().Debugf("ruleid:", r.ID(), "\t result (found alerts):", len(result)) return result, nil } @@ -545,7 +555,6 @@ func (r *ThresholdRule) Eval(ctx context.Context, ts time.Time, queriers *Querie defer r.mtx.Unlock() resultFPs := map[uint64]struct{}{} - var vec Vector var alerts = make(map[uint64]*Alert, len(res)) for _, smpl := range res { @@ -559,6 +568,7 @@ func (r *ThresholdRule) Eval(ctx context.Context, ts time.Time, queriers *Querie // who are not used to Go's templating system. defs := "{{$labels := .Labels}}{{$value := .Value}}" + // utility function to apply go template on labels and annots expand := func(text string) string { tmpl := NewTemplateExpander( @@ -613,6 +623,7 @@ func (r *ThresholdRule) Eval(ctx context.Context, ts time.Time, queriers *Querie State: StatePending, Value: smpl.V, GeneratorURL: r.GeneratorURL(), + Receivers: r.preferredChannels, } } @@ -626,6 +637,7 @@ func (r *ThresholdRule) Eval(ctx context.Context, ts time.Time, queriers *Querie alert.Value = a.Value alert.Annotations = a.Annotations + alert.Receivers = r.preferredChannels continue } @@ -656,18 +668,19 @@ func (r *ThresholdRule) Eval(ctx context.Context, ts time.Time, queriers *Querie } r.health = HealthGood r.lastError = err - return vec, nil + return len(r.active), nil } func (r *ThresholdRule) String() string { ar := PostableRule{ - Alert: r.name, - RuleCondition: r.ruleCondition, - EvalWindow: Duration(r.evalWindow), - Labels: r.labels.Map(), - Annotations: r.annotations.Map(), + Alert: r.name, + RuleCondition: r.ruleCondition, + EvalWindow: Duration(r.evalWindow), + Labels: r.labels.Map(), + Annotations: r.annotations.Map(), + PreferredChannels: r.preferredChannels, } byt, err := yaml.Marshal(ar) diff --git a/pkg/query-service/tests/test-deploy/clickhouse-storage.xml b/pkg/query-service/tests/test-deploy/clickhouse-storage.xml index eaf1e7e99d..f444bf43b4 100644 --- a/pkg/query-service/tests/test-deploy/clickhouse-storage.xml +++ b/pkg/query-service/tests/test-deploy/clickhouse-storage.xml @@ -20,6 +20,7 @@ s3 + 0 diff --git a/pkg/query-service/tests/test-deploy/docker-compose.yaml b/pkg/query-service/tests/test-deploy/docker-compose.yaml index 794d398826..d6d9b9c1eb 100644 --- a/pkg/query-service/tests/test-deploy/docker-compose.yaml +++ b/pkg/query-service/tests/test-deploy/docker-compose.yaml @@ -24,7 +24,7 @@ services: - "8123:8123" alertmanager: - image: signoz/alertmanager:0.23.0-0.1 + image: signoz/alertmanager:0.23.0-0.2 depends_on: - query-service restart: on-failure @@ -59,7 +59,7 @@ services: condition: service_healthy otel-collector: - image: signoz/otelcontribcol:0.45.1-1.1 + image: signoz/otelcontribcol:0.45.1-1.3 command: ["--config=/etc/otel-collector-config.yaml"] volumes: - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml @@ -74,7 +74,7 @@ services: condition: service_healthy otel-collector-metrics: - image: signoz/otelcontribcol:0.45.1-1.1 + image: signoz/otelcontribcol:0.45.1-1.3 command: ["--config=/etc/otel-collector-metrics-config.yaml"] volumes: - ./otel-collector-metrics-config.yaml:/etc/otel-collector-metrics-config.yaml diff --git a/pkg/query-service/utils/labels/labels.go b/pkg/query-service/utils/labels/labels.go index 52ef1867f5..b7f6fdc09d 100644 --- a/pkg/query-service/utils/labels/labels.go +++ b/pkg/query-service/utils/labels/labels.go @@ -24,6 +24,10 @@ const ( AlertRuleIdLabel = "ruleId" RuleSourceLabel = "ruleSource" + + RuleThresholdLabel = "threshold" + AlertAdditionalInfoLabel = "additionalInfo" + AlertSummaryLabel = "summary" ) // Label is a key/value pair of strings.