Compare commits

..
Author SHA1 Message Date
telackey 3ba9436863 Add git to webapp base container. 2024-06-24 23:57:15 -05:00
dboreham b4e82ebc19 Merge pull request 'Fix mainnet laconic deploy setup' (#850) from dboreham/fix-mainnet-laconic-init into main
Lint Checks / Run linter (push) Successful in 48s
Publish / Build and publish (push) Successful in 1m15s
Webapp Test / Run webapp test suite (push) Successful in 4m19s
Smoke Test / Run basic test suite (push) Successful in 3m52s
Deploy Test / Run deploy test suite (push) Successful in 4m47s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 13m19s
Fixturenet-Eth-Plugeth-Test / Run an Ethereum plugeth fixturenet test (push) Failing after 56m43s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 7m23s
Fixturenet-Eth-Plugeth-Arm-Test / Run an Ethereum plugeth fixturenet test (push) Failing after 1h12m31s
Database Test / Run database hosting test on kind/k8s (push) Successful in 10m46s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 3m36s
External Stack Test / Run external stack test suite (push) Successful in 4m30s
Reviewed-on: #850
2024-06-22 01:25:07 +00:00
dboreham 2364924a59 Fix mainnet laconic deploy setup
Lint Checks / Run linter (pull_request) Successful in 31s
Deploy Test / Run deploy test suite (pull_request) Successful in 5m6s
K8s Deploy Test / Run deploy test suite on kind/k8s (pull_request) Successful in 8m9s
Webapp Test / Run webapp test suite (pull_request) Successful in 4m33s
Smoke Test / Run basic test suite (pull_request) Successful in 4m9s
2024-06-21 19:24:33 -06:00
nabarun a223797b4a Update graph-node dashboard to show individual subgraph increase in query count (#846)
Lint Checks / Run linter (push) Successful in 35s
Publish / Build and publish (push) Successful in 1m10s
Deploy Test / Run deploy test suite (push) Successful in 4m42s
Webapp Test / Run webapp test suite (push) Successful in 4m18s
Smoke Test / Run basic test suite (push) Successful in 3m53s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 13m13s
Fixturenet-Eth-Plugeth-Test / Run an Ethereum plugeth fixturenet test (push) Failing after 51m15s
Fixturenet-Eth-Plugeth-Arm-Test / Run an Ethereum plugeth fixturenet test (push) Failing after 55m9s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 7m20s
Database Test / Run database hosting test on kind/k8s (push) Successful in 9m4s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 3m35s
External Stack Test / Run external stack test suite (push) Successful in 4m24s
Part of [Metrics and logging for GQL queries in watcher](https://www.notion.so/Metrics-and-logging-for-GQL-queries-in-watcher-928c692292b140a2a4f52cda9795df5e)

Reviewed-on: #846
Co-authored-by: Nabarun <nabarun@deepstacksoft.com>
Co-committed-by: Nabarun <nabarun@deepstacksoft.com>
2024-06-20 09:27:23 +00:00
nabarun b8004e9870 Add Grafana panels for graph-node subgraph GQL queries (#845)
Lint Checks / Run linter (push) Successful in 42s
Publish / Build and publish (push) Successful in 1m22s
Deploy Test / Run deploy test suite (push) Successful in 5m12s
Webapp Test / Run webapp test suite (push) Successful in 4m34s
Smoke Test / Run basic test suite (push) Successful in 3m59s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 13m17s
Fixturenet-Eth-Plugeth-Test / Run an Ethereum plugeth fixturenet test (push) Failing after 51m47s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 7m40s
Fixturenet-Eth-Plugeth-Arm-Test / Run an Ethereum plugeth fixturenet test (push) Failing after 1h38m30s
Database Test / Run database hosting test on kind/k8s (push) Successful in 9m22s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 3m49s
External Stack Test / Run external stack test suite (push) Successful in 4m50s
Part of [Deploy v2 and updated v3 sushiswap subgraphs](https://www.notion.so/Deploy-v2-and-updated-v3-sushiswap-subgraphs-e331945fdeea487c890706fc22c6cc94)

Reviewed-on: #845
Co-authored-by: Nabarun <nabarun@deepstacksoft.com>
Co-committed-by: Nabarun <nabarun@deepstacksoft.com>
2024-06-19 10:40:54 +00:00
prathamesh 3fd99a1522 Handle race condition in laconic registry CLI tests (#843)
Lint Checks / Run linter (push) Successful in 33s
Publish / Build and publish (push) Successful in 1m16s
Deploy Test / Run deploy test suite (push) Successful in 5m11s
Webapp Test / Run webapp test suite (push) Successful in 4m9s
Smoke Test / Run basic test suite (push) Successful in 3m46s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 13m37s
Part of cerc-io/laconic-registry-cli#63

Reviewed-on: #843
Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
2024-06-19 06:41:58 +00:00
nabarun 842d999792 Add alert rules for secured secured-finance subgraph watcher (#842)
Lint Checks / Run linter (push) Successful in 38s
Publish / Build and publish (push) Successful in 1m22s
Webapp Test / Run webapp test suite (push) Successful in 4m44s
Smoke Test / Run basic test suite (push) Successful in 4m34s
Deploy Test / Run deploy test suite (push) Successful in 5m25s
Fixturenet-Laconicd-Test / Run Laconicd fixturenet and Laconic CLI tests (push) Successful in 13m30s
Fixturenet-Eth-Plugeth-Test / Run an Ethereum plugeth fixturenet test (push) Failing after 53m10s
Fixturenet-Eth-Plugeth-Arm-Test / Run an Ethereum plugeth fixturenet test (push) Failing after 1h1m56s
K8s Deploy Test / Run deploy test suite on kind/k8s (push) Successful in 7m37s
Database Test / Run database hosting test on kind/k8s (push) Successful in 10m23s
Container Registry Test / Run contaier registry hosting test on kind/k8s (push) Successful in 3m56s
External Stack Test / Run external stack test suite (push) Successful in 4m46s
Part of [Generate secured-finance subgraph watcher with codegen](https://www.notion.so/Generate-secured-finance-subgraph-watcher-with-codegen-2923413e0af54ea787c5435d6966f3bb)
- Update watcher dashboard labels

Reviewed-on: #842
Co-authored-by: Nabarun <nabarun@deepstacksoft.com>
Co-committed-by: Nabarun <nabarun@deepstacksoft.com>
2024-06-18 12:28:02 +00:00
7 changed files with 723 additions and 9 deletions
@@ -4,3 +4,4 @@ Trigger
Trigger
Trigger
Trigger
Trigger
@@ -18,7 +18,7 @@
"editable": true,
"fiscalYearStartMonth": 0,
"graphTooltip": 0,
"id": 6,
"id": 39,
"links": [],
"liveNow": false,
"panels": [
@@ -1202,6 +1202,640 @@
],
"title": "ETH RPC failed requests rate",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "PBFA97CFB590B2093"
},
"gridPos": {
"h": 2,
"w": 24,
"x": 0,
"y": 22
},
"id": 32,
"options": {
"code": {
"language": "plaintext",
"showLineNumbers": false,
"showMiniMap": false
},
"content": "<center style=\"margin-top: 1rem\"> \n <h1>GraphQL Server</h1> \n</center>",
"mode": "html"
},
"pluginVersion": "10.2.3",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "PBFA97CFB590B2093"
},
"refId": "A"
}
],
"transparent": true,
"type": "text"
},
{
"cards": {},
"color": {
"cardColor": "#b4ff00",
"colorScale": "sqrt",
"colorScheme": "interpolatePlasma",
"exponent": 0.5,
"mode": "spectrum"
},
"dataFormat": "tsbuckets",
"datasource": {
"type": "prometheus",
"uid": "PBFA97CFB590B2093"
},
"fieldConfig": {
"defaults": {
"custom": {
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"scaleDistribution": {
"type": "linear"
}
}
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 6,
"x": 0,
"y": 24
},
"heatmap": {},
"hideZeroBuckets": true,
"highlightCards": true,
"id": 29,
"legend": {
"show": false
},
"links": [],
"options": {
"calculate": false,
"calculation": {},
"cellGap": 2,
"cellValues": {},
"color": {
"exponent": 0.5,
"fill": "#b4ff00",
"mode": "scheme",
"reverse": false,
"scale": "exponential",
"scheme": "Plasma",
"steps": 128
},
"exemplars": {
"color": "rgba(255,0,255,0.7)"
},
"filterValues": {
"le": 1e-9
},
"legend": {
"show": false
},
"rowsFrame": {
"layout": "auto"
},
"showValue": "never",
"tooltip": {
"show": true,
"showColorScale": false,
"yHistogram": false
},
"yAxis": {
"axisPlacement": "left",
"reverse": false,
"unit": "short"
}
},
"pluginVersion": "10.2.3",
"repeatDirection": "v",
"reverseYBuckets": false,
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "PBFA97CFB590B2093"
},
"editorMode": "code",
"expr": "sum(increase(query_execution_time_bucket{deployment=\"[[subgraph_hash]]\"}[1m])) by (le)",
"format": "heatmap",
"instant": false,
"intervalFactor": 1,
"legendFormat": "{{le}}",
"refId": "A"
}
],
"title": "Query Execution Time Histogram ([[subgraph_name]])",
"tooltip": {
"show": true,
"showHistogram": false
},
"type": "heatmap",
"xAxis": {
"show": true
},
"yAxis": {
"format": "short",
"logBase": 1,
"show": true
},
"yBucketBound": "auto"
},
{
"datasource": {
"type": "prometheus",
"uid": "PBFA97CFB590B2093"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"custom": {
"fillOpacity": 80,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineWidth": 1
},
"links": [],
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 80
}
]
}
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 6,
"x": 6,
"y": 24
},
"id": 26,
"links": [],
"options": {
"bucketOffset": 0,
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
}
},
"pluginVersion": "10.2.3",
"repeat": "subgraph_hash",
"repeatDirection": "v",
"targets": [
{
"datasource": {
"uid": "prometheus"
},
"editorMode": "code",
"expr": "sum(increase(query_execution_time_bucket{deployment=\"[[subgraph_hash]]\"}[1m])) by (le)",
"format": "heatmap",
"instant": false,
"intervalFactor": 1,
"legendFormat": "{{le}}",
"refId": "A"
}
],
"title": "Query Execution Time Histogram ([[subgraph_name]])",
"type": "histogram"
},
{
"datasource": {
"type": "prometheus",
"uid": "PBFA97CFB590B2093"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 80,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "never",
"spanNulls": true,
"stacking": {
"group": "A",
"mode": "normal"
},
"thresholdsStyle": {
"mode": "off"
}
},
"links": [],
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 80
}
]
},
"unit": "locale"
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 24
},
"id": 36,
"links": [],
"options": {
"legend": {
"calcs": [],
"displayMode": "table",
"placement": "bottom",
"showLegend": false
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
},
"pluginVersion": "10.2.3",
"repeatDirection": "v",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "PBFA97CFB590B2093"
},
"disableTextWrap": false,
"editorMode": "builder",
"exemplar": false,
"expr": "sum(increase(query_execution_time_count{deployment=\"$subgraph_hash\"}[1m]))",
"format": "time_series",
"fullMetaSearch": false,
"includeNullMetadata": true,
"instant": false,
"intervalFactor": 1,
"legendFormat": "{{subgraph_deployment}}",
"range": true,
"refId": "A",
"useBackend": false
}
],
"title": "Queries ([[subgraph_name]])",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "PBFA97CFB590B2093"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "Queries per minute per minute",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 2,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"links": [],
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 80
}
]
},
"unit": "locale"
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 32
},
"id": 27,
"links": [],
"options": {
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
},
"pluginVersion": "10.2.3",
"repeatDirection": "v",
"targets": [
{
"datasource": {
"uid": "prometheus"
},
"editorMode": "code",
"expr": "sum(rate(query_execution_time_count[1m])) by (deployment)",
"format": "time_series",
"instant": false,
"intervalFactor": 1,
"legendFormat": "{{deployment}}",
"refId": "A"
}
],
"title": "Query Rate (All Subgraphs)",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "PBFA97CFB590B2093"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 80,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "never",
"spanNulls": true,
"stacking": {
"group": "A",
"mode": "normal"
},
"thresholdsStyle": {
"mode": "off"
}
},
"links": [],
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 80
}
]
},
"unit": "locale"
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 32
},
"id": 67,
"links": [],
"options": {
"legend": {
"calcs": [
"sum"
],
"displayMode": "table",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
},
"pluginVersion": "10.2.3",
"repeatDirection": "v",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "PBFA97CFB590B2093"
},
"editorMode": "code",
"expr": "sum(increase(query_execution_time_count[1m])) by (deployment)",
"format": "time_series",
"instant": false,
"intervalFactor": 1,
"legendFormat": "{{deployment}}",
"refId": "A"
}
],
"title": "Queries (All Subgraphs)",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "PBFA97CFB590B2093"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 70,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "never",
"spanNulls": true,
"stacking": {
"group": "A",
"mode": "normal"
},
"thresholdsStyle": {
"mode": "off"
}
},
"links": [],
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 80
}
]
},
"unit": "locale"
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 40
},
"id": 41,
"links": [],
"options": {
"legend": {
"calcs": [],
"displayMode": "table",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
},
"pluginVersion": "10.2.3",
"repeatDirection": "v",
"targets": [
{
"datasource": {
"uid": "prometheus"
},
"editorMode": "code",
"expr": "increase(query_execution_time_count{status=\"failed\"}[1m])",
"format": "time_series",
"instant": false,
"intervalFactor": 1,
"legendFormat": "{{deployment}}",
"refId": "A"
}
],
"title": "Failed Queries (All Subgraphs)",
"type": "timeseries"
}
],
"refresh": "10s",
@@ -322,7 +322,7 @@
"useBackend": false
}
],
"title": "Latest processed block",
"title": "Event processed block",
"type": "stat"
},
{
@@ -391,7 +391,7 @@
"useBackend": false
}
],
"title": "Latest indexed block",
"title": "Inserted processed block",
"type": "stat"
},
{
@@ -849,3 +849,81 @@ groups:
annotations:
summary: Watcher {{ index $labels "instance" }} of group {{ index $labels "job" }} is falling behind external head by {{ index $values "diff" }}
isPaused: false
# Secured Finance
- uid: secured_finance_diff_external
title: secured_finance_watcher_head_tracking
condition: condition
data:
- refId: diff
relativeTimeRange:
from: 600
to: 0
datasourceUid: PBFA97CFB590B2093
model:
datasource:
type: prometheus
uid: PBFA97CFB590B2093
disableTextWrap: false
editorMode: code
expr: latest_block_number{instance="external"} - on(chain) group_right sync_status_block_number{job="secured-finance", instance="secured-finance", kind="latest_indexed"}
fullMetaSearch: false
includeNullMetadata: true
instant: true
intervalMs: 1000
legendFormat: __auto
maxDataPoints: 43200
range: false
refId: diff
useBackend: false
- refId: latest_external
relativeTimeRange:
from: 600
to: 0
datasourceUid: PBFA97CFB590B2093
model:
datasource:
type: prometheus
uid: PBFA97CFB590B2093
editorMode: code
expr: latest_block_number{chain="filecoin"}
hide: false
instant: true
legendFormat: __auto
range: false
refId: latest_external
- refId: condition
relativeTimeRange:
from: 600
to: 0
datasourceUid: __expr__
model:
conditions:
- evaluator:
params:
- 0
- 0
type: gt
operator:
type: and
query:
params: []
reducer:
params: []
type: avg
type: query
datasource:
name: Expression
type: __expr__
uid: __expr__
expression: ${diff} >= 16
intervalMs: 1000
maxDataPoints: 43200
refId: condition
type: math
noDataState: Alerting
execErrState: Alerting
for: 15m
annotations:
summary: Watcher {{ index $labels "instance" }} of group {{ index $labels "job" }} is falling behind external head by {{ index $values "diff" }}
isPaused: false
@@ -32,7 +32,7 @@ RUN \
# [Optional] Uncomment this section to install additional OS packages.
RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
&& apt-get -y install --no-install-recommends jq gettext-base
&& apt-get -y install --no-install-recommends jq gettext-base git
# [Optional] Uncomment if you want to install an additional version of node using nvm
# ARG EXTRA_NODE_VERSION=10
@@ -14,7 +14,8 @@
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
from stack_orchestrator.util import get_yaml
from stack_orchestrator.deploy.deploy_types import DeployCommandContext, LaconicStackSetupCommand, DeploymentContext
from stack_orchestrator.deploy.deploy_types import DeployCommandContext, LaconicStackSetupCommand
from stack_orchestrator.deploy.deployment_context import DeploymentContext
from stack_orchestrator.deploy.stack_state import State
from stack_orchestrator.deploy.deploy_util import VolumeMapping, run_container_command
from stack_orchestrator.command_types import CommandOptions
+4 -4
View File
@@ -22,16 +22,16 @@ echo "$(date +"%Y-%m-%d %T"): Stack started"
# Verify that the fixturenet is up and running
$TEST_TARGET_SO --stack fixturenet-laconicd deploy --cluster laconicd ps
# Wait for the laconid endpoint to come up
echo "Waiting for the RPC endpoint to come up"
docker exec laconicd-laconicd-1 sh -c "curl --retry 20 --retry-delay 3 --retry-connrefused http://127.0.0.1:9473/api"
# Get the fixturenet account address
laconicd_account_address=$(docker exec laconicd-laconicd-1 laconicd keys list | awk '/- address:/ {print $3}')
# Copy over config
docker exec laconicd-cli-1 cp config.yml laconic-registry-cli/
# Wait for the laconid endpoint to come up
echo "Waiting for the RPC endpoint to come up"
docker exec laconicd-laconicd-1 sh -c "curl --retry 20 --retry-delay 3 --retry-connrefused http://127.0.0.1:9473/api"
# Run the tests
echo "Running the tests"
docker exec -e TEST_ACCOUNT=$laconicd_account_address laconicd-cli-1 sh -c 'cd laconic-registry-cli && yarn && yarn test'