Archived
Compare commits
7
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 zramsay
|
22febffb09 | ||
 telackey
|
5308ab1e4e | ||
 prathamesh
|
cd50832038 | ||
|
prathamesh
|
aeddc82ebc | ||
|
prathamesh
|
17e860d6e4 | ||
|
telackey
|
523b5779be | ||
|
telackey
|
62f7ce649d |
@@ -0,0 +1,271 @@
|
||||
# Deploying WebApps from the Laconic Registry to Kubernetes
|
||||
|
||||
First, ensure you have `laconicd` and the console running via [this tutorial](./laconicd-with-console.md).
|
||||
|
||||
## Setup Kubernetes
|
||||
|
||||
If merely requesting deployment, access to the Laconic registry is sufficient, but to *process* deployment requests, Kubernetes configuration is also necessary. The configuration format is the same as used by `kubectl`, and should be places in `/etc/config/kube.yml`
|
||||
|
||||
## Setup the Deployer
|
||||
|
||||
### Set envs
|
||||
|
||||
|
||||
```
|
||||
$DEPLOYMENT_DNS_SUFFIX
|
||||
$DEPLOYMENT_RECORD_NAMESPACE
|
||||
$IMAGE_REGISTRY
|
||||
$IMAGE_REGISTRY_CREDS
|
||||
$IMAGE_REGISTRY_USER
|
||||
|
||||
```
|
||||
|
||||
|
||||
```
|
||||
laconic-so --stack webapp-deployer-backend setup-repositories
|
||||
laconic-so --stack webapp-deployer-backend build-containers
|
||||
laconic-so --stack webapp-deployer-backend deploy up
|
||||
```
|
||||
|
||||
## Setup Your Account
|
||||
|
||||
Writing to the Laconic Registry, in order to create records for applications and deployment requests, requires a user key, and a bond with adequate funds available.
|
||||
|
||||
### Install the registry CLI
|
||||
|
||||
```
|
||||
npm config set @cerc-io:registry https://git.vdb.to/api/packages/cerc-io/npm/
|
||||
npm install -g @cerc-io/laconic-registry-cli
|
||||
```
|
||||
|
||||
Alternatively, use the pre-configured container that was setup alongside `laconicd`
|
||||
|
||||
### Get your private key
|
||||
|
||||
```
|
||||
exec into laconicd
|
||||
```
|
||||
|
||||
### config.yml
|
||||
|
||||
```
|
||||
services:
|
||||
cns:
|
||||
restEndpoint: http://console.laconic.com:1317
|
||||
gqlEndpoint: http://console.laconic.com:9473/api
|
||||
userKey: 3d4789e88508c6230d973ccea283032d3e3948775dbe02f4f0a80dc6c1f7c8d5
|
||||
# bondId: 3c61577f3c6197ced599e2fc21ccf76f43373004fd29c29f2e560c77f7c4bb6d
|
||||
chainId: laconic_9000-1
|
||||
gas: 550000
|
||||
fees: 200000aphoton
|
||||
```
|
||||
|
||||
### Create a bond
|
||||
|
||||
```
|
||||
laconic cns -c config.yml bond create --type aphoton --quantity 1000000000 --gas 200000 --fees 200000aphoton
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
# Creating Deployments
|
||||
|
||||
Deploying an application from the registry requires (and generates)
|
||||
several records:
|
||||
|
||||
1. An `ApplicationRecord` which describes the application itself, including its name, version, and repository.
|
||||
2. An `ApplicationDeploymentRequest` which references an`ApplicationRecord`, and provides additional information such as the DNS name and configuration.
|
||||
3. A `DnsRecord`, which contains the FQDN of the deployment. The ownership of this record will be checked against any future deployment or removal requests.
|
||||
4. An `ApplicationDeploymentRecord` which records the successful processing of the deployment request.
|
||||
|
||||
Additionally, since names need to be registered, namespace authorities need to be reserved, for example:
|
||||
|
||||
```
|
||||
$ laconic -c $LACONIC_CONFIG cns authority reserve my-org-name
|
||||
|
||||
$ laconic -c $LACONIC_CONFIG cns authority bond set my-org-name 0e9176d854bc3c20528b6361aab632f0b252a0f69717bf035fa68d1ef7647ba7
|
||||
```
|
||||
|
||||
## Application Records
|
||||
|
||||
The `ApplicationRecord` should, at a minimum, specify the name of the application (`name`), its version (`app_version`), type (`app_type`), one or more repository URLs (`repository`), and the repository reference (eg, branch, tag, hash, etc.) to use (`repository_ref`).
|
||||
|
||||
```
|
||||
$ cat app.yml
|
||||
record:
|
||||
type: ApplicationRecord
|
||||
version: 0.0.4
|
||||
name: "@my-org-name/my-test-webapp"
|
||||
repository:
|
||||
- "https://github.com/my-org-name/my-test-webapp"
|
||||
repository_ref: "v0.1.5"
|
||||
app_version: "0.1.5"
|
||||
app_type: "webapp"
|
||||
|
||||
$ laconic -c $LACONIC_CONFIG cns record publish -f app.yml
|
||||
bafyreihwvu6ynmk4nfrxg2vdcx2ep3tqry775ksqyehjitj2i4kphhyuky
|
||||
```
|
||||
|
||||
One or more names should be registered for the application, which deployment requests can reference.
|
||||
|
||||
```
|
||||
$ laconic -c $LACONIC_CONFIG cns name set "crn://my-org-name/applications/my-test-webapp" bafyreihwvu6ynmk4nfrxg2vdcx2ep3tqry775ksqyehjitj2i4kphhyuky
|
||||
|
||||
$ laconic -c $LACONIC_CONFIG cns name set "crn://my-org-name/applications/my-test-webapp@0.1.5" bafyreihwvu6ynmk4nfrxg2vdcx2ep3tqry775ksqyehjitj2i4kphhyuky
|
||||
|
||||
$ laconic -c $LACONIC_CONFIG cns record get --id bafyreihwvu6ynmk4nfrxg2vdcx2ep3tqry775ksqyehjitj2i4kphhyuky
|
||||
[
|
||||
{
|
||||
"id": "bafyreihwvu6ynmk4nfrxg2vdcx2ep3tqry775ksqyehjitj2i4kphhyuky",
|
||||
"names": [
|
||||
"crn://my-org-name/applications/my-test-webapp",
|
||||
"crn://my-org-name/applications/my-test-webapp@0.1.5",
|
||||
],
|
||||
"owners": [
|
||||
"2671D38525BDC91A5DF4794EF2059D5771133702"
|
||||
],
|
||||
"bondId": "0e9176d854bc3c20528b6361aab632f0b252a0f69717bf035fa68d1ef7647ba7",
|
||||
"createTime": "2024-01-12T20:30:33Z",
|
||||
"expiryTime": "2025-01-11T20:30:33Z",
|
||||
"attributes": {
|
||||
"app_type": "webapp",
|
||||
"app_version": "0.1.5",
|
||||
"name": "@my-org-name/my-test-webapp",
|
||||
"repository": [
|
||||
"https://github.com/my-org-name/my-test-webapp"
|
||||
],
|
||||
"repository_ref": "v0.1.5",
|
||||
"type": "ApplicationRecord",
|
||||
"version": "0.0.4"
|
||||
}
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
## Application Deployment Requests
|
||||
|
||||
To create the deployment of specific application, and `ApplicationDeploymentRequest` must be published. This request must reference the application to be deployed (`application`) and may optionally provide configuration (`config`) and a DNS name (`dns`). If no DNS name is supplied, one will be generated. A supplied DNS is usually just the short hostname, not the FQDN, since the suffix is supplied by the deployer.
|
||||
|
||||
```
|
||||
$ cat req.yml
|
||||
record:
|
||||
type: ApplicationDeploymentRequest
|
||||
version: 1.0.0
|
||||
name: "my-org-name/my-test-webapp@0.1.5"
|
||||
application: "crn://my-org-name/applications/my-test-webapp@0.1.5"
|
||||
dns: "my-test-app"
|
||||
config:
|
||||
env:
|
||||
CERC_WEBAPP_DEBUG: my_debug_value_here
|
||||
|
||||
$ laconic -c $LACONIC_CONFIG cns record publish -f req.yml
|
||||
bafyreihtpvwjka5ecjmca46y4dip5gb2h25vvmc7t27d7g4zecngjssvky
|
||||
```
|
||||
|
||||
It is not necessary to assign a name to the request.
|
||||
|
||||
## Building and Deploying
|
||||
|
||||
Building and deploying will happen automatically for records published the production registry, but in other environments it can be triggered manually.
|
||||
|
||||
Laconic `stack-orchestrator` is used build and launch the application. It will clone the repository, build the application, upload the container to a registry, and launch the instance in Kubernetes, with
|
||||
automatic DNS and TLS provisioning.
|
||||
|
||||
```
|
||||
$ laconic-so deploy-webapp-from-registry \
|
||||
--kube-config $KUBE_CONFIG \
|
||||
--image-registry registry.mydeployer.org/app-registry \
|
||||
--deployment-parent-dir /opt/deployments \
|
||||
--laconic-config $LACONIC_CONFIG \
|
||||
--dns-suffix mydeployer.servesthe.world \
|
||||
--record-namespace-dns crn://my-deployer-org/dns \
|
||||
--record-namespace-deployments crn://my-deployer-org/deployments \
|
||||
--request-id bafyreihtpvwjka5ecjmca46y4dip5gb2h25vvmc7t27d7g4zecngjssvky
|
||||
```
|
||||
|
||||
When deployment is complete, an `ApplicationDeploymentRecord` will be created:
|
||||
|
||||
```
|
||||
$ laconic -c $LACONIC_CONFIG cns name resolve crn://my-deployer-org/deployments/my-test-app.mydeployer.servesthe.world
|
||||
[
|
||||
{
|
||||
"id": "bafyreiemfmxsue4svzys6tcwsqmhfjeyoo3gp63n7kydi2izbrwuzd4rga",
|
||||
"names": [
|
||||
"crn://my-deployer-org/deployments/my-test-app.laconic.servesthe.world"
|
||||
],
|
||||
"owners": [
|
||||
"2671D38525BDC91A5DF4794EF2059D5771133702"
|
||||
],
|
||||
"bondId": "0e9176d854bc3c20528b6361aab632f0b252a0f69717bf035fa68d1ef7647ba7",
|
||||
"createTime": "2024-01-11T20:45:29Z",
|
||||
"expiryTime": "2025-01-10T20:45:29Z",
|
||||
"attributes": {
|
||||
"application": "bafyreibs6y7jhgjlsoxyqlugvkweanp3bi7ippfmhehk2rslcpoaqic2xi",
|
||||
"dns": "bafyreicpac7tar5ua5e42zo7d5zwyp5yv7iord23p3gdox7zdfslrarjle",
|
||||
"meta": {
|
||||
"config": "da39a3ee5e6b4b0d3255bfef95601890afd80709",
|
||||
"so": "82a5f9a3e4924cbb92e28be68759a487"
|
||||
},
|
||||
"name": "@cerc-io/my-test-webapp",
|
||||
"request": "bafyreiex7vmtruiiasra4wvqe7wirl6iuwiso6p3cvb3sjhq55e74zh4ke",
|
||||
"type": "ApplicationDeploymentRecord",
|
||||
"url": "https://my-test-app.mydeployer.servesthe.world",
|
||||
"version": "0.0.1"
|
||||
}
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
## Checking Status
|
||||
|
||||
The status of the deployment may be checked with `stack-orchestrator`.
|
||||
|
||||
```
|
||||
$ laconic-so deployment --dir /opt/deployments/my-test-app.snowball.servesthe.world status
|
||||
Ingress:
|
||||
Hostname: my-test-app.mydeployer.servesthe.world
|
||||
IP: 204.130.133.199
|
||||
TLS: notBefore: 2024-01-12T23:11:09Z, notAfter: 2024-04-11T23:11:08Z
|
||||
|
||||
Pods:
|
||||
default/laconic-9560ffc64512e453-deployment-fb58d756f-l77pm: Running (2024-01-13 00:10:42+00:00)
|
||||
```
|
||||
|
||||
# Removing Deployments
|
||||
|
||||
As with deployment, removal involves publishing a request, which is then fulfilled by deployment processor.
|
||||
|
||||
## Application Deployment Removal Requests
|
||||
|
||||
```
|
||||
$ cat apprm.yml
|
||||
record:
|
||||
type: ApplicationDeploymentRemovalRequest
|
||||
version: 1.0.0
|
||||
deployment: bafyreiemfmxsue4svzys6tcwsqmhfjeyoo3gp63n7kydi2izbrwuzd4rga
|
||||
|
||||
$ laconic -c $LACONIC_CONFIG cns record publish -f apprm.yml
|
||||
bafyreiejnqhsn5ibc3c6pzlsc26co3mt63djdeuntt3gmfuckcntdmisge
|
||||
```
|
||||
|
||||
## Removal
|
||||
|
||||
As with deployment, for records publish to the production service removal should be processed automatically. In other environments, removal can be processed with `stack-orchestrator`.
|
||||
|
||||
```
|
||||
$ laconic-so undeploy-webapp-from-registry \
|
||||
--deployment-parent-dir /opt/deployments \
|
||||
--laconic-config ~/.laconic/local.yml \
|
||||
--request-id bafyreiejnqhsn5ibc3c6pzlsc26co3mt63djdeuntt3gmfuckcntdmisge
|
||||
|
||||
Request bafyreiejnqhsn5ibc3c6pzlsc26co3mt63djdeuntt3gmfuckcntdmisge needs to processed.
|
||||
Found 1 unsatisfied request(s) to process.
|
||||
Matched deployment ownership: 2671D38525BDC91A5DF4794EF2059D5771133702
|
||||
record:
|
||||
type: ApplicationDeploymentRemovalRecord
|
||||
deployment: bafyreiemfmxsue4svzys6tcwsqmhfjeyoo3gp63n7kydi2izbrwuzd4rga
|
||||
request: bafyreiejnqhsn5ibc3c6pzlsc26co3mt63djdeuntt3gmfuckcntdmisge
|
||||
version: 1.0.0
|
||||
```
|
||||
@@ -0,0 +1,76 @@
|
||||
version: '3.2'
|
||||
|
||||
services:
|
||||
ajna-watcher-db:
|
||||
restart: unless-stopped
|
||||
image: postgres:14-alpine
|
||||
environment:
|
||||
- POSTGRES_USER=vdbm
|
||||
- POSTGRES_MULTIPLE_DATABASES=ajna-watcher,ajna-watcher-job-queue
|
||||
- POSTGRES_EXTENSION=ajna-watcher-job-queue:pgcrypto
|
||||
- POSTGRES_PASSWORD=password
|
||||
volumes:
|
||||
- ../config/postgresql/multiple-postgressql-databases.sh:/docker-entrypoint-initdb.d/multiple-postgressql-databases.sh
|
||||
- ajna_watcher_db_data:/var/lib/postgresql/data
|
||||
ports:
|
||||
- "5432"
|
||||
healthcheck:
|
||||
test: ["CMD", "nc", "-v", "localhost", "5432"]
|
||||
interval: 20s
|
||||
timeout: 5s
|
||||
retries: 15
|
||||
start_period: 10s
|
||||
|
||||
ajna-watcher-job-runner:
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
ajna-watcher-db:
|
||||
condition: service_healthy
|
||||
image: cerc/watcher-ajna:local
|
||||
environment:
|
||||
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
|
||||
CERC_ETH_RPC_ENDPOINT: ${CERC_ETH_RPC_ENDPOINT}
|
||||
command: ["bash", "./start-job-runner.sh"]
|
||||
volumes:
|
||||
- ../config/watcher-ajna/watcher-config-template.toml:/app/environments/watcher-config-template.toml
|
||||
- ../config/watcher-ajna/start-job-runner.sh:/app/start-job-runner.sh
|
||||
ports:
|
||||
- "9000"
|
||||
healthcheck:
|
||||
test: ["CMD", "nc", "-v", "localhost", "9000"]
|
||||
interval: 20s
|
||||
timeout: 5s
|
||||
retries: 15
|
||||
start_period: 5s
|
||||
extra_hosts:
|
||||
- "host.docker.internal:host-gateway"
|
||||
|
||||
ajna-watcher-server:
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
ajna-watcher-db:
|
||||
condition: service_healthy
|
||||
ajna-watcher-job-runner:
|
||||
condition: service_healthy
|
||||
image: cerc/watcher-ajna:local
|
||||
environment:
|
||||
CERC_SCRIPT_DEBUG: ${CERC_SCRIPT_DEBUG}
|
||||
CERC_ETH_RPC_ENDPOINT: ${CERC_ETH_RPC_ENDPOINT}
|
||||
command: ["bash", "./start-server.sh"]
|
||||
volumes:
|
||||
- ../config/watcher-ajna/watcher-config-template.toml:/app/environments/watcher-config-template.toml
|
||||
- ../config/watcher-ajna/start-server.sh:/app/start-server.sh
|
||||
ports:
|
||||
- "3008"
|
||||
- "9001"
|
||||
healthcheck:
|
||||
test: ["CMD", "nc", "-v", "localhost", "3008"]
|
||||
interval: 20s
|
||||
timeout: 5s
|
||||
retries: 15
|
||||
start_period: 5s
|
||||
extra_hosts:
|
||||
- "host.docker.internal:host-gateway"
|
||||
|
||||
volumes:
|
||||
ajna_watcher_db_data:
|
||||
@@ -50,22 +50,6 @@ groups:
|
||||
legendFormat: __auto
|
||||
range: false
|
||||
refId: latest_external
|
||||
- refId: latest_indexed
|
||||
relativeTimeRange:
|
||||
from: 600
|
||||
to: 0
|
||||
datasourceUid: PBFA97CFB590B2093
|
||||
model:
|
||||
datasource:
|
||||
type: prometheus
|
||||
uid: PBFA97CFB590B2093
|
||||
editorMode: code
|
||||
expr: sync_status_block_number{job="azimuth", instance="azimuth", kind="latest_indexed"}
|
||||
hide: false
|
||||
instant: true
|
||||
legendFormat: __auto
|
||||
range: false
|
||||
refId: latest_indexed
|
||||
- refId: condition
|
||||
relativeTimeRange:
|
||||
from: 600
|
||||
@@ -142,22 +126,6 @@ groups:
|
||||
legendFormat: __auto
|
||||
range: false
|
||||
refId: latest_external
|
||||
- refId: latest_indexed
|
||||
relativeTimeRange:
|
||||
from: 600
|
||||
to: 0
|
||||
datasourceUid: PBFA97CFB590B2093
|
||||
model:
|
||||
datasource:
|
||||
type: prometheus
|
||||
uid: PBFA97CFB590B2093
|
||||
editorMode: code
|
||||
expr: sync_status_block_number{job="azimuth", instance="censures", kind="latest_indexed"}
|
||||
hide: false
|
||||
instant: true
|
||||
legendFormat: __auto
|
||||
range: false
|
||||
refId: latest_indexed
|
||||
- refId: condition
|
||||
relativeTimeRange:
|
||||
from: 600
|
||||
@@ -234,22 +202,6 @@ groups:
|
||||
legendFormat: __auto
|
||||
range: false
|
||||
refId: latest_external
|
||||
- refId: latest_indexed
|
||||
relativeTimeRange:
|
||||
from: 600
|
||||
to: 0
|
||||
datasourceUid: PBFA97CFB590B2093
|
||||
model:
|
||||
datasource:
|
||||
type: prometheus
|
||||
uid: PBFA97CFB590B2093
|
||||
editorMode: code
|
||||
expr: sync_status_block_number{job="azimuth", instance="claims", kind="latest_indexed"}
|
||||
hide: false
|
||||
instant: true
|
||||
legendFormat: __auto
|
||||
range: false
|
||||
refId: latest_indexed
|
||||
- refId: condition
|
||||
relativeTimeRange:
|
||||
from: 600
|
||||
@@ -326,22 +278,6 @@ groups:
|
||||
legendFormat: __auto
|
||||
range: false
|
||||
refId: latest_external
|
||||
- refId: latest_indexed
|
||||
relativeTimeRange:
|
||||
from: 600
|
||||
to: 0
|
||||
datasourceUid: PBFA97CFB590B2093
|
||||
model:
|
||||
datasource:
|
||||
type: prometheus
|
||||
uid: PBFA97CFB590B2093
|
||||
editorMode: code
|
||||
expr: sync_status_block_number{job="azimuth", instance="conditional_star_release", kind="latest_indexed"}
|
||||
hide: false
|
||||
instant: true
|
||||
legendFormat: __auto
|
||||
range: false
|
||||
refId: latest_indexed
|
||||
- refId: condition
|
||||
relativeTimeRange:
|
||||
from: 600
|
||||
@@ -418,22 +354,6 @@ groups:
|
||||
legendFormat: __auto
|
||||
range: false
|
||||
refId: latest_external
|
||||
- refId: latest_indexed
|
||||
relativeTimeRange:
|
||||
from: 600
|
||||
to: 0
|
||||
datasourceUid: PBFA97CFB590B2093
|
||||
model:
|
||||
datasource:
|
||||
type: prometheus
|
||||
uid: PBFA97CFB590B2093
|
||||
editorMode: code
|
||||
expr: sync_status_block_number{job="azimuth", instance="delegated_sending", kind="latest_indexed"}
|
||||
hide: false
|
||||
instant: true
|
||||
legendFormat: __auto
|
||||
range: false
|
||||
refId: latest_indexed
|
||||
- refId: condition
|
||||
relativeTimeRange:
|
||||
from: 600
|
||||
@@ -510,22 +430,6 @@ groups:
|
||||
legendFormat: __auto
|
||||
range: false
|
||||
refId: latest_external
|
||||
- refId: latest_indexed
|
||||
relativeTimeRange:
|
||||
from: 600
|
||||
to: 0
|
||||
datasourceUid: PBFA97CFB590B2093
|
||||
model:
|
||||
datasource:
|
||||
type: prometheus
|
||||
uid: PBFA97CFB590B2093
|
||||
editorMode: code
|
||||
expr: sync_status_block_number{job="azimuth", instance="ecliptic", kind="latest_indexed"}
|
||||
hide: false
|
||||
instant: true
|
||||
legendFormat: __auto
|
||||
range: false
|
||||
refId: latest_indexed
|
||||
- refId: condition
|
||||
relativeTimeRange:
|
||||
from: 600
|
||||
@@ -602,22 +506,6 @@ groups:
|
||||
legendFormat: __auto
|
||||
range: false
|
||||
refId: latest_external
|
||||
- refId: latest_indexed
|
||||
relativeTimeRange:
|
||||
from: 600
|
||||
to: 0
|
||||
datasourceUid: PBFA97CFB590B2093
|
||||
model:
|
||||
datasource:
|
||||
type: prometheus
|
||||
uid: PBFA97CFB590B2093
|
||||
editorMode: code
|
||||
expr: sync_status_block_number{job="azimuth", instance="azimuth", kind="latest_indexed"}
|
||||
hide: false
|
||||
instant: true
|
||||
legendFormat: __auto
|
||||
range: false
|
||||
refId: latest_indexed
|
||||
- refId: condition
|
||||
relativeTimeRange:
|
||||
from: 600
|
||||
@@ -694,22 +582,6 @@ groups:
|
||||
legendFormat: __auto
|
||||
range: false
|
||||
refId: latest_external
|
||||
- refId: latest_indexed
|
||||
relativeTimeRange:
|
||||
from: 600
|
||||
to: 0
|
||||
datasourceUid: PBFA97CFB590B2093
|
||||
model:
|
||||
datasource:
|
||||
type: prometheus
|
||||
uid: PBFA97CFB590B2093
|
||||
editorMode: code
|
||||
expr: sync_status_block_number{job="azimuth", instance="polls", kind="latest_indexed"}
|
||||
hide: false
|
||||
instant: true
|
||||
legendFormat: __auto
|
||||
range: false
|
||||
refId: latest_indexed
|
||||
- refId: condition
|
||||
relativeTimeRange:
|
||||
from: 600
|
||||
@@ -788,22 +660,6 @@ groups:
|
||||
legendFormat: __auto
|
||||
range: false
|
||||
refId: latest_external
|
||||
- refId: latest_indexed
|
||||
relativeTimeRange:
|
||||
from: 600
|
||||
to: 0
|
||||
datasourceUid: PBFA97CFB590B2093
|
||||
model:
|
||||
datasource:
|
||||
type: prometheus
|
||||
uid: PBFA97CFB590B2093
|
||||
editorMode: code
|
||||
expr: sync_status_block_number{job="sushi", instance="sushiswap", kind="latest_indexed"}
|
||||
hide: false
|
||||
instant: true
|
||||
legendFormat: __auto
|
||||
range: false
|
||||
refId: latest_indexed
|
||||
- refId: condition
|
||||
relativeTimeRange:
|
||||
from: 600
|
||||
@@ -880,22 +736,6 @@ groups:
|
||||
legendFormat: __auto
|
||||
range: false
|
||||
refId: latest_external
|
||||
- refId: latest_indexed
|
||||
relativeTimeRange:
|
||||
from: 600
|
||||
to: 0
|
||||
datasourceUid: PBFA97CFB590B2093
|
||||
model:
|
||||
datasource:
|
||||
type: prometheus
|
||||
uid: PBFA97CFB590B2093
|
||||
editorMode: code
|
||||
expr: sync_status_block_number{job="sushi", instance="merkl_sushiswap", kind="latest_indexed"}
|
||||
hide: false
|
||||
instant: true
|
||||
legendFormat: __auto
|
||||
range: false
|
||||
refId: latest_indexed
|
||||
- refId: condition
|
||||
relativeTimeRange:
|
||||
from: 600
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
#!/bin/sh
|
||||
|
||||
set -e
|
||||
if [ -n "$CERC_SCRIPT_DEBUG" ]; then
|
||||
set -x
|
||||
fi
|
||||
set -u
|
||||
|
||||
echo "Using ETH RPC endpoint ${CERC_ETH_RPC_ENDPOINT}"
|
||||
|
||||
# Read in the config template TOML file and modify it
|
||||
WATCHER_CONFIG_TEMPLATE=$(cat environments/watcher-config-template.toml)
|
||||
WATCHER_CONFIG=$(echo "$WATCHER_CONFIG_TEMPLATE" | \
|
||||
sed -E "s|REPLACE_WITH_CERC_ETH_RPC_ENDPOINT|${CERC_ETH_RPC_ENDPOINT}| ")
|
||||
|
||||
# Write the modified content to a new file
|
||||
echo "$WATCHER_CONFIG" > environments/local.toml
|
||||
|
||||
echo "Running job-runner..."
|
||||
DEBUG=vulcanize:* exec node --enable-source-maps dist/job-runner.js
|
||||
@@ -0,0 +1,20 @@
|
||||
#!/bin/sh
|
||||
|
||||
set -e
|
||||
if [ -n "$CERC_SCRIPT_DEBUG" ]; then
|
||||
set -x
|
||||
fi
|
||||
set -u
|
||||
|
||||
echo "Using ETH RPC endpoint ${CERC_ETH_RPC_ENDPOINT}"
|
||||
|
||||
# Read in the config template TOML file and modify it
|
||||
WATCHER_CONFIG_TEMPLATE=$(cat environments/watcher-config-template.toml)
|
||||
WATCHER_CONFIG=$(echo "$WATCHER_CONFIG_TEMPLATE" | \
|
||||
sed -E "s|REPLACE_WITH_CERC_ETH_RPC_ENDPOINT|${CERC_ETH_RPC_ENDPOINT}| ")
|
||||
|
||||
# Write the modified content to a new file
|
||||
echo "$WATCHER_CONFIG" > environments/local.toml
|
||||
|
||||
echo "Running server..."
|
||||
DEBUG=vulcanize:* exec node --enable-source-maps dist/server.js
|
||||
@@ -0,0 +1,98 @@
|
||||
[server]
|
||||
host = "0.0.0.0"
|
||||
port = 3008
|
||||
kind = "active"
|
||||
gqlPath = "/graphql"
|
||||
|
||||
# Checkpointing state.
|
||||
checkpointing = true
|
||||
|
||||
# Checkpoint interval in number of blocks.
|
||||
checkpointInterval = 2000
|
||||
|
||||
# Enable state creation
|
||||
# CAUTION: Disable only if state creation is not desired or can be filled subsequently
|
||||
enableState = false
|
||||
|
||||
subgraphPath = "./subgraph-build"
|
||||
|
||||
# Interval to restart wasm instance periodically
|
||||
wasmRestartBlocksInterval = 20
|
||||
|
||||
# Interval in number of blocks at which to clear entities cache.
|
||||
clearEntitiesCacheInterval = 1000
|
||||
|
||||
# Max block range for which to return events in eventsInRange GQL query.
|
||||
# Use -1 for skipping check on block range.
|
||||
maxEventsBlockRange = 1000
|
||||
|
||||
# Flag to specify whether RPC endpoint supports block hash as block tag parameter
|
||||
rpcSupportsBlockHashParam = false
|
||||
|
||||
# GQL cache settings
|
||||
[server.gqlCache]
|
||||
enabled = true
|
||||
|
||||
# Max in-memory cache size (in bytes) (default 8 MB)
|
||||
# maxCacheSize
|
||||
|
||||
# GQL cache-control max-age settings (in seconds)
|
||||
maxAge = 15
|
||||
timeTravelMaxAge = 86400 # 1 day
|
||||
|
||||
[metrics]
|
||||
host = "0.0.0.0"
|
||||
port = 9000
|
||||
[metrics.gql]
|
||||
port = 9001
|
||||
|
||||
[database]
|
||||
type = "postgres"
|
||||
host = "ajna-watcher-db"
|
||||
port = 5432
|
||||
database = "ajna-watcher"
|
||||
username = "vdbm"
|
||||
password = "password"
|
||||
synchronize = true
|
||||
logging = false
|
||||
|
||||
[upstream]
|
||||
[upstream.ethServer]
|
||||
rpcProviderEndpoint = "REPLACE_WITH_CERC_ETH_RPC_ENDPOINT"
|
||||
|
||||
# Boolean flag to specify if rpc-eth-client should be used for RPC endpoint instead of ipld-eth-client (ipld-eth-server GQL client)
|
||||
rpcClient = true
|
||||
|
||||
# Boolean flag to specify if rpcProviderEndpoint is an FEVM RPC endpoint
|
||||
isFEVM = true
|
||||
|
||||
# Boolean flag to filter event logs by contracts
|
||||
filterLogsByAddresses = true
|
||||
# Boolean flag to filter event logs by topics
|
||||
filterLogsByTopics = true
|
||||
|
||||
[upstream.cache]
|
||||
name = "requests"
|
||||
enabled = false
|
||||
deleteOnStart = false
|
||||
|
||||
[jobQueue]
|
||||
dbConnectionString = "postgres://vdbm:password@ajna-watcher-db/ajna-watcher-job-queue"
|
||||
maxCompletionLagInSecs = 300
|
||||
jobDelayInMilliSecs = 100
|
||||
eventsInBatch = 50
|
||||
subgraphEventsOrder = true
|
||||
# Filecoin block time: https://docs.filecoin.io/basics/the-blockchain/blocks-and-tipsets#blocktime
|
||||
blockDelayInMilliSecs = 30000
|
||||
|
||||
# Boolean to switch between modes of processing events when starting the server.
|
||||
# Setting to true will fetch filtered events and required blocks in a range of blocks and then process them.
|
||||
# Setting to false will fetch blocks consecutively with its events and then process them (Behaviour is followed in realtime processing near head).
|
||||
useBlockRanges = true
|
||||
|
||||
# Block range in which logs are fetched during historical blocks processing
|
||||
historicalLogsBlockRange = 2000
|
||||
|
||||
# Max block range of historical processing after which it waits for completion of events processing
|
||||
# If set to -1 historical processing does not wait for events processing and completes till latest canonical block
|
||||
historicalMaxFetchAhead = 10000
|
||||
+1
-1
@@ -5,7 +5,7 @@ if [ -n "$CERC_SCRIPT_DEBUG" ]; then
|
||||
set -x
|
||||
fi
|
||||
|
||||
registry_command="laconic cns"
|
||||
registry_command="laconic registry"
|
||||
demo_records_dir="scripts/demo-records"
|
||||
|
||||
# Check we have funds
|
||||
|
||||
@@ -11,8 +11,14 @@ CERC_CONTAINER_BUILD_DOCKERFILE=${CERC_CONTAINER_BUILD_DOCKERFILE:-$SCRIPT_DIR/D
|
||||
CERC_CONTAINER_BUILD_TAG=${CERC_CONTAINER_BUILD_TAG:-cerc/nextjs-base:local}
|
||||
|
||||
docker build -t $CERC_CONTAINER_BUILD_TAG ${build_command_args} -f $CERC_CONTAINER_BUILD_DOCKERFILE $CERC_CONTAINER_BUILD_WORK_DIR
|
||||
rc=$?
|
||||
|
||||
if [ $? -eq 0 ] && [ "$CERC_CONTAINER_BUILD_TAG" != "cerc/nextjs-base:local" ]; then
|
||||
if [ $rc -ne 0 ]; then
|
||||
echo "BUILD FAILED" 1>&2
|
||||
exit $rc
|
||||
fi
|
||||
|
||||
if [ "$CERC_CONTAINER_BUILD_TAG" != "cerc/nextjs-base:local" ]; then
|
||||
cat <<EOF
|
||||
|
||||
#################################################################
|
||||
|
||||
@@ -0,0 +1,10 @@
|
||||
FROM node:18.17.1-alpine3.18
|
||||
|
||||
RUN apk --update --no-cache add git python3 alpine-sdk bash curl jq
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY . .
|
||||
|
||||
RUN echo "Installing dependencies and building ajna-watcher-ts" && \
|
||||
yarn && yarn build
|
||||
@@ -0,0 +1,9 @@
|
||||
#!/usr/bin/env bash
|
||||
# Build cerc/watcher-ajna
|
||||
|
||||
source ${CERC_CONTAINER_BASE_DIR}/build-base.sh
|
||||
|
||||
# See: https://stackoverflow.com/a/246128/1701505
|
||||
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
|
||||
|
||||
docker build -t cerc/watcher-ajna:local -f ${SCRIPT_DIR}/Dockerfile ${build_command_args} ${CERC_REPO_BASE_DIR}/ajna-watcher-ts
|
||||
@@ -11,8 +11,14 @@ CERC_CONTAINER_BUILD_DOCKERFILE=${CERC_CONTAINER_BUILD_DOCKERFILE:-$SCRIPT_DIR/D
|
||||
CERC_CONTAINER_BUILD_TAG=${CERC_CONTAINER_BUILD_TAG:-cerc/webapp-base:local}
|
||||
|
||||
docker build -t $CERC_CONTAINER_BUILD_TAG ${build_command_args} -f $CERC_CONTAINER_BUILD_DOCKERFILE $CERC_CONTAINER_BUILD_WORK_DIR
|
||||
rc=$?
|
||||
|
||||
if [ $? -eq 0 ] && [ "$CERC_CONTAINER_BUILD_TAG" != "cerc/webapp-base:local" ]; then
|
||||
if [ $rc -ne 0 ]; then
|
||||
echo "BUILD FAILED" 1>&2
|
||||
exit $rc
|
||||
fi
|
||||
|
||||
if [ "$CERC_CONTAINER_BUILD_TAG" != "cerc/webapp-base:local" ]; then
|
||||
cat <<EOF
|
||||
|
||||
#################################################################
|
||||
|
||||
@@ -0,0 +1,118 @@
|
||||
# Ajna Watcher
|
||||
|
||||
## Setup
|
||||
|
||||
Clone required repositories:
|
||||
|
||||
```bash
|
||||
laconic-so --stack ajna setup-repositories --git-ssh --pull
|
||||
```
|
||||
|
||||
Build the container images:
|
||||
|
||||
```bash
|
||||
laconic-so --stack ajna build-containers
|
||||
```
|
||||
|
||||
## Deploy
|
||||
|
||||
Create a spec file for the deployment:
|
||||
|
||||
```bash
|
||||
laconic-so --stack ajna deploy init --output ajna-spec.yml
|
||||
```
|
||||
|
||||
### Ports
|
||||
|
||||
Edit `network` in the spec file to map container ports to host ports as required:
|
||||
|
||||
```yml
|
||||
...
|
||||
network:
|
||||
ports:
|
||||
ajna-watcher-db:
|
||||
- 15432:5432
|
||||
ajna-watcher-job-runner:
|
||||
- 9000:9000
|
||||
ajna-watcher-server:
|
||||
- 3008:3008
|
||||
- 9001:9001
|
||||
```
|
||||
|
||||
### Create a deployment
|
||||
|
||||
Create a deployment from the spec file:
|
||||
|
||||
```bash
|
||||
laconic-so --stack ajna deploy create --spec-file ajna-spec.yml --deployment-dir ajna-deployment
|
||||
```
|
||||
|
||||
### Configuration
|
||||
|
||||
Inside deployment directory, open the `config.env` file and set following env variables:
|
||||
|
||||
```bash
|
||||
# External Filecoin (ETH RPC) endpoint to point the watcher to
|
||||
CERC_ETH_RPC_ENDPOINT=https://example-lotus-endpoint/rpc/v1
|
||||
```
|
||||
|
||||
### Start the deployment
|
||||
|
||||
```bash
|
||||
laconic-so deployment --dir ajna-deployment start
|
||||
```
|
||||
|
||||
* To list down and monitor the running containers:
|
||||
|
||||
```bash
|
||||
# With status
|
||||
docker ps -a
|
||||
|
||||
# Check logs for a container
|
||||
docker logs -f <CONTAINER_ID>
|
||||
```
|
||||
|
||||
* Open the GQL playground at <http://localhost:3008/graphql>
|
||||
|
||||
```graphql
|
||||
# Example query
|
||||
query {
|
||||
_meta {
|
||||
block {
|
||||
hash
|
||||
number
|
||||
timestamp
|
||||
}
|
||||
deployment
|
||||
hasIndexingErrors
|
||||
}
|
||||
|
||||
accounts {
|
||||
id
|
||||
txCount
|
||||
tokensDelegated
|
||||
rewardsClaimed
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Clean up
|
||||
|
||||
Stop all the ajna services running in background:
|
||||
|
||||
```bash
|
||||
# Only stop the docker containers
|
||||
laconic-so deployment --dir ajna-deployment stop
|
||||
|
||||
# Run 'start' to restart the deployment
|
||||
```
|
||||
|
||||
To stop all the ajna services and also delete data:
|
||||
|
||||
```bash
|
||||
# Stop the docker containers
|
||||
laconic-so deployment --dir ajna-deployment stop --delete-volumes
|
||||
|
||||
# Remove deployment directory (deployment will have to be recreated for a re-run)
|
||||
rm -r ajna-deployment
|
||||
```
|
||||
@@ -0,0 +1,9 @@
|
||||
version: "1.0"
|
||||
name: ajna
|
||||
description: "Ajna watcher stack"
|
||||
repos:
|
||||
- git.vdb.to/cerc-io/ajna-watcher-ts
|
||||
containers:
|
||||
- cerc/watcher-ajna
|
||||
pods:
|
||||
- watcher-ajna
|
||||
@@ -49,7 +49,7 @@ $ laconic-so --stack fixturenet-laconic-loaded deploy logs
|
||||
```
|
||||
## 6. Test with the Registry CLI
|
||||
```
|
||||
$ laconic-so --stack fixturenet-laconic-loaded deploy exec cli "laconic cns status"
|
||||
$ laconic-so --stack fixturenet-laconic-loaded deploy exec cli "laconic registry status"
|
||||
```
|
||||
## 7. View the laconic console
|
||||
Get the URL for the console web app with this command (the port number will be different for each deployment):
|
||||
|
||||
@@ -44,5 +44,5 @@ $ laconic-so --stack fixturenet-laconicd deploy logs
|
||||
```
|
||||
## 6. Test with the Registry CLI
|
||||
```
|
||||
$ laconic-so --stack fixturenet-laconicd deploy exec cli "laconic cns status"
|
||||
$ laconic-so --stack fixturenet-laconicd deploy exec cli "laconic registry status"
|
||||
```
|
||||
|
||||
@@ -16,26 +16,55 @@ laconic-so --stack merkl-sushiswap-v3 build-containers
|
||||
|
||||
## Deploy
|
||||
|
||||
### Configuration
|
||||
|
||||
Create and update an env file to be used in the next step:
|
||||
|
||||
```bash
|
||||
# External Filecoin (ETH RPC) endpoint to point the watcher
|
||||
CERC_ETH_RPC_ENDPOINT=
|
||||
```
|
||||
|
||||
### Deploy the stack
|
||||
Create a spec file for the deployment:
|
||||
|
||||
```bash
|
||||
laconic-so --stack merkl-sushiswap-v3 deploy --cluster merkl_sushiswap_v3 --env-file <PATH_TO_ENV_FILE> up
|
||||
laconic-so --stack merkl-sushiswap-v3 deploy init --output merkl-sushiswap-v3-spec.yml
|
||||
```
|
||||
|
||||
### Ports
|
||||
|
||||
Edit `network` in the spec file to map container ports to host ports as required:
|
||||
|
||||
```
|
||||
...
|
||||
network:
|
||||
ports:
|
||||
merkl-sushiswap-v3-watcher-db:
|
||||
- '5432'
|
||||
merkl-sushiswap-v3-watcher-job-runner:
|
||||
- 9002:9000
|
||||
merkl-sushiswap-v3-watcher-server:
|
||||
- 127.0.0.1:3007:3008
|
||||
- 9003:9001
|
||||
```
|
||||
|
||||
### Create a deployment
|
||||
|
||||
Create a deployment from the spec file:
|
||||
|
||||
```bash
|
||||
laconic-so --stack merkl-sushiswap-v3 deploy create --spec-file merkl-sushiswap-v3-spec.yml --deployment-dir merkl-sushiswap-v3-deployment
|
||||
```
|
||||
|
||||
### Configuration
|
||||
|
||||
Inside deployment directory, open the `config.env` file and set following env variables:
|
||||
|
||||
```bash
|
||||
# External Filecoin (ETH RPC) endpoint to point the watcher to
|
||||
CERC_ETH_RPC_ENDPOINT=https://example-lotus-endpoint/rpc/v1
|
||||
```
|
||||
|
||||
### Start the deployment
|
||||
|
||||
```bash
|
||||
laconic-so deployment --dir merkl-sushiswap-v3-deployment start
|
||||
```
|
||||
|
||||
* To list down and monitor the running containers:
|
||||
|
||||
```bash
|
||||
laconic-so --stack merkl-sushiswap-v3 deploy --cluster merkl_sushiswap_v3 ps
|
||||
|
||||
# With status
|
||||
docker ps -a
|
||||
|
||||
@@ -46,6 +75,7 @@ laconic-so --stack merkl-sushiswap-v3 deploy --cluster merkl_sushiswap_v3 --env-
|
||||
* Open the GQL playground at http://localhost:3007/graphql
|
||||
|
||||
```graphql
|
||||
# Example query
|
||||
{
|
||||
_meta {
|
||||
block {
|
||||
@@ -54,7 +84,7 @@ laconic-so --stack merkl-sushiswap-v3 deploy --cluster merkl_sushiswap_v3 --env-
|
||||
}
|
||||
hasIndexingErrors
|
||||
}
|
||||
|
||||
|
||||
factories {
|
||||
id
|
||||
poolCount
|
||||
@@ -64,18 +94,21 @@ laconic-so --stack merkl-sushiswap-v3 deploy --cluster merkl_sushiswap_v3 --env-
|
||||
|
||||
## Clean up
|
||||
|
||||
Stop all the services running in background:
|
||||
Stop all the merkl-sushiswap-v3 services running in background:
|
||||
|
||||
```bash
|
||||
laconic-so --stack merkl-sushiswap-v3 deploy --cluster merkl_sushiswap_v3 down
|
||||
# Only stop the docker containers
|
||||
laconic-so deployment --dir merkl-sushiswap-v3-deployment stop
|
||||
|
||||
# Run 'start' to restart the deployment
|
||||
```
|
||||
|
||||
Clear volumes created by this stack:
|
||||
To stop all the merkl-sushiswap-v3 services and also delete data:
|
||||
|
||||
```bash
|
||||
# List all relevant volumes
|
||||
docker volume ls -q --filter "name=merkl_sushiswap_v3"
|
||||
# Stop the docker containers
|
||||
laconic-so deployment --dir merkl-sushiswap-v3-deployment stop --delete-volumes
|
||||
|
||||
# Remove all the listed volumes
|
||||
docker volume rm $(docker volume ls -q --filter "name=merkl_sushiswap_v3")
|
||||
# Remove deployment directory (deployment will have to be recreated for a re-run)
|
||||
rm -r merkl-sushiswap-v3-deployment
|
||||
```
|
||||
|
||||
@@ -2,7 +2,7 @@ version: "1.0"
|
||||
name: merkl-sushiswap-v3
|
||||
description: "SushiSwap v3 watcher stack"
|
||||
repos:
|
||||
- github.com/cerc-io/merkl-sushiswap-v3-watcher-ts@v0.1.6
|
||||
- github.com/cerc-io/merkl-sushiswap-v3-watcher-ts@v0.1.7
|
||||
containers:
|
||||
- cerc/watcher-merkl-sushiswap-v3
|
||||
pods:
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
version: "0.1"
|
||||
name: monitoring
|
||||
repos:
|
||||
- github.com/cerc-io/watcher-ts@v0.2.79
|
||||
- github.com/cerc-io/watcher-ts@v0.2.81
|
||||
containers:
|
||||
- cerc/watcher-ts
|
||||
pods:
|
||||
|
||||
@@ -55,7 +55,7 @@ ports:
|
||||
Create deployment:
|
||||
|
||||
```bash
|
||||
laconic-so deploy create --spec-file sushiswap-subgraph-spec.yml --deployment-dir sushiswap-subgraph-deployment
|
||||
laconic-so --stack sushiswap-subgraph deploy create --spec-file sushiswap-subgraph-spec.yml --deployment-dir sushiswap-subgraph-deployment
|
||||
```
|
||||
|
||||
## Start the stack
|
||||
|
||||
@@ -16,26 +16,55 @@ laconic-so --stack sushiswap-v3 build-containers
|
||||
|
||||
## Deploy
|
||||
|
||||
### Configuration
|
||||
|
||||
Create and update an env file to be used in the next step:
|
||||
|
||||
```bash
|
||||
# External Filecoin (ETH RPC) endpoint to point the watcher
|
||||
CERC_ETH_RPC_ENDPOINT=
|
||||
```
|
||||
|
||||
### Deploy the stack
|
||||
Create a spec file for the deployment:
|
||||
|
||||
```bash
|
||||
laconic-so --stack sushiswap-v3 deploy --cluster sushiswap_v3 --env-file <PATH_TO_ENV_FILE> up
|
||||
laconic-so --stack sushiswap-v3 deploy init --output sushiswap-v3-spec.yml
|
||||
```
|
||||
|
||||
### Ports
|
||||
|
||||
Edit `network` in the spec file to map container ports to host ports as required:
|
||||
|
||||
```
|
||||
...
|
||||
network:
|
||||
ports:
|
||||
sushiswap-v3-watcher-db:
|
||||
- '5432'
|
||||
sushiswap-v3-watcher-job-runner:
|
||||
- 9000:9000
|
||||
sushiswap-v3-watcher-server:
|
||||
- 127.0.0.1:3008:3008
|
||||
- 9001:9001
|
||||
```
|
||||
|
||||
### Create a deployment
|
||||
|
||||
Create a deployment from the spec file:
|
||||
|
||||
```bash
|
||||
laconic-so --stack sushiswap-v3 deploy create --spec-file sushiswap-v3-spec.yml --deployment-dir sushiswap-v3-deployment
|
||||
```
|
||||
|
||||
### Configuration
|
||||
|
||||
Inside deployment directory, open the `config.env` file and set following env variables:
|
||||
|
||||
```bash
|
||||
# External Filecoin (ETH RPC) endpoint to point the watcher to
|
||||
CERC_ETH_RPC_ENDPOINT=https://example-lotus-endpoint/rpc/v1
|
||||
```
|
||||
|
||||
### Start the deployment
|
||||
|
||||
```bash
|
||||
laconic-so deployment --dir sushiswap-v3-deployment start
|
||||
```
|
||||
|
||||
* To list down and monitor the running containers:
|
||||
|
||||
```bash
|
||||
laconic-so --stack sushiswap-v3 deploy --cluster sushiswap_v3 ps
|
||||
|
||||
# With status
|
||||
docker ps -a
|
||||
|
||||
@@ -43,20 +72,43 @@ laconic-so --stack sushiswap-v3 deploy --cluster sushiswap_v3 --env-file <PATH_T
|
||||
docker logs -f <CONTAINER_ID>
|
||||
```
|
||||
|
||||
* Open the GQL playground at http://localhost:3008/graphql
|
||||
|
||||
```graphql
|
||||
# Example query
|
||||
{
|
||||
_meta {
|
||||
block {
|
||||
number
|
||||
timestamp
|
||||
}
|
||||
hasIndexingErrors
|
||||
}
|
||||
|
||||
factories {
|
||||
id
|
||||
poolCount
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Clean up
|
||||
|
||||
Stop all the services running in background:
|
||||
Stop all the sushiswap-v3 services running in background:
|
||||
|
||||
```bash
|
||||
laconic-so --stack sushiswap-v3 deploy --cluster sushiswap_v3 down
|
||||
# Only stop the docker containers
|
||||
laconic-so deployment --dir sushiswap-v3-deployment stop
|
||||
|
||||
# Run 'start' to restart the deployment
|
||||
```
|
||||
|
||||
Clear volumes created by this stack:
|
||||
To stop all the sushiswap-v3 services and also delete data:
|
||||
|
||||
```bash
|
||||
# List all relevant volumes
|
||||
docker volume ls -q --filter "name=sushiswap_v3"
|
||||
# Stop the docker containers
|
||||
laconic-so deployment --dir sushiswap-v3-deployment stop --delete-volumes
|
||||
|
||||
# Remove all the listed volumes
|
||||
docker volume rm $(docker volume ls -q --filter "name=sushiswap_v3")
|
||||
# Remove deployment directory (deployment will have to be recreated for a re-run)
|
||||
rm -r sushiswap-v3-deployment
|
||||
```
|
||||
|
||||
@@ -2,7 +2,7 @@ version: "1.0"
|
||||
name: sushiswap-v3
|
||||
description: "SushiSwap v3 watcher stack"
|
||||
repos:
|
||||
- github.com/cerc-io/sushiswap-v3-watcher-ts@v0.1.6
|
||||
- github.com/cerc-io/sushiswap-v3-watcher-ts@v0.1.7
|
||||
containers:
|
||||
- cerc/watcher-sushiswap-v3
|
||||
pods:
|
||||
|
||||
@@ -101,7 +101,7 @@ class ClusterInfo:
|
||||
)
|
||||
return service
|
||||
|
||||
def get_ingress(self, use_tls=False):
|
||||
def get_ingress(self, use_tls=False, certificate=None, cluster_issuer="letsencrypt-prod"):
|
||||
# No ingress for a deployment that has no http-proxy defined, for now
|
||||
http_proxy_info_list = self.spec.get_http_proxy()
|
||||
ingress = None
|
||||
@@ -110,32 +110,13 @@ class ClusterInfo:
|
||||
http_proxy_info = http_proxy_info_list[0]
|
||||
if opts.o.debug:
|
||||
print(f"http-proxy: {http_proxy_info}")
|
||||
|
||||
host_name = http_proxy_info["host-name"]
|
||||
|
||||
tls = None
|
||||
tls_issuer = None
|
||||
|
||||
if use_tls:
|
||||
tls_info = http_proxy_info.get("tls", {})
|
||||
tls_hosts = tls_info.get("hosts", [host_name])
|
||||
tls_issuer = tls_info.get("issuer", "letsencrypt-prod")
|
||||
tls_secret_name = f"{self.app_name}-tls"
|
||||
if "secret" in tls_info:
|
||||
# If an existing secret is specified, unset the issuer so that we don't try to re-request it.
|
||||
tls_secret_name = tls_info["secret"]
|
||||
tls_issuer = None
|
||||
|
||||
if opts.o.debug:
|
||||
print(f"TLS hosts/secret: {tls_hosts}/{tls_secret_name}")
|
||||
|
||||
tls = [client.V1IngressTLS(
|
||||
hosts=tls_hosts,
|
||||
secret_name=tls_secret_name
|
||||
)]
|
||||
|
||||
# TODO: good enough parsing for webapp deployment for now
|
||||
host_name = http_proxy_info["host-name"]
|
||||
rules = []
|
||||
tls = [client.V1IngressTLS(
|
||||
hosts=certificate["spec"]["dnsNames"] if certificate else [host_name],
|
||||
secret_name=certificate["spec"]["secretName"] if certificate else f"{self.app_name}-tls"
|
||||
)] if use_tls else None
|
||||
paths = []
|
||||
for route in http_proxy_info["routes"]:
|
||||
path = route["path"]
|
||||
@@ -166,15 +147,17 @@ class ClusterInfo:
|
||||
tls=tls,
|
||||
rules=rules
|
||||
)
|
||||
annotations = {
|
||||
|
||||
ingress_annotations = {
|
||||
"kubernetes.io/ingress.class": "nginx",
|
||||
}
|
||||
if tls_issuer:
|
||||
annotations["cert-manager.io/cluster-issuer"] = tls_issuer
|
||||
if not certificate:
|
||||
ingress_annotations["cert-manager.io/cluster-issuer"] = cluster_issuer
|
||||
|
||||
ingress = client.V1Ingress(
|
||||
metadata=client.V1ObjectMeta(
|
||||
name=f"{self.app_name}-ingress",
|
||||
annotations=annotations
|
||||
annotations=ingress_annotations
|
||||
),
|
||||
spec=spec
|
||||
)
|
||||
|
||||
@@ -169,6 +169,39 @@ class K8sDeployer(Deployer):
|
||||
print("Service created:")
|
||||
print(f"{service_resp}")
|
||||
|
||||
def _find_certificate_for_host_name(self, host_name):
|
||||
all_certificates = self.custom_obj_api.list_namespaced_custom_object(
|
||||
group="cert-manager.io",
|
||||
version="v1",
|
||||
namespace=self.k8s_namespace,
|
||||
plural="certificates"
|
||||
)
|
||||
|
||||
host_parts = host_name.split(".", 1)
|
||||
host_as_wild = None
|
||||
if len(host_parts) == 2:
|
||||
host_as_wild = f"*.{host_parts[1]}"
|
||||
|
||||
now = datetime.utcnow().replace(tzinfo=timezone.utc)
|
||||
fmt = "%Y-%m-%dT%H:%M:%S%z"
|
||||
|
||||
# Walk over all the configured certificates.
|
||||
for cert in all_certificates["items"]:
|
||||
dns = cert["spec"]["dnsNames"]
|
||||
# Check for an exact hostname match or a wildcard match.
|
||||
if host_name in dns or host_as_wild in dns:
|
||||
status = cert.get("status", {})
|
||||
# Check the certificate date.
|
||||
if "notAfter" in status and "notBefore" in status:
|
||||
before = datetime.strptime(status["notBefore"], fmt)
|
||||
after = datetime.strptime(status["notAfter"], fmt)
|
||||
if before < now < after:
|
||||
# Check the status is Ready
|
||||
for condition in status.get("conditions", []):
|
||||
if "True" == condition.get("status") and "Ready" == condition.get("type"):
|
||||
return cert
|
||||
return None
|
||||
|
||||
def up(self, detach, services):
|
||||
if not opts.o.dry_run:
|
||||
if self.is_kind():
|
||||
@@ -189,8 +222,15 @@ class K8sDeployer(Deployer):
|
||||
self._create_volume_data()
|
||||
self._create_deployment()
|
||||
|
||||
http_proxy_info = self.cluster_info.spec.get_http_proxy()
|
||||
# Note: at present we don't support tls for kind (and enabling tls causes errors)
|
||||
ingress: client.V1Ingress = self.cluster_info.get_ingress(use_tls=not self.is_kind())
|
||||
use_tls = http_proxy_info and not self.is_kind()
|
||||
certificate = self._find_certificate_for_host_name(http_proxy_info[0]["host-name"]) if use_tls else None
|
||||
if opts.o.debug:
|
||||
if certificate:
|
||||
print(f"Using existing certificate: {certificate}")
|
||||
|
||||
ingress: client.V1Ingress = self.cluster_info.get_ingress(use_tls=use_tls, certificate=certificate)
|
||||
if ingress:
|
||||
if opts.o.debug:
|
||||
print(f"Sending this ingress: {ingress}")
|
||||
@@ -353,7 +393,8 @@ class K8sDeployer(Deployer):
|
||||
hostname = ingress.spec.rules[0].host
|
||||
ip = ingress.status.load_balancer.ingress[0].ip
|
||||
tls = "notBefore: %s; notAfter: %s; names: %s" % (
|
||||
cert["status"]["notBefore"], cert["status"]["notAfter"], ", ".join(ingress.spec.tls[0].hosts))
|
||||
cert["status"]["notBefore"], cert["status"]["notAfter"], ingress.spec.tls[0].hosts
|
||||
)
|
||||
except: # noqa: E722
|
||||
pass
|
||||
|
||||
|
||||
@@ -15,7 +15,6 @@
|
||||
|
||||
import click
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from urllib.parse import urlparse
|
||||
from tempfile import NamedTemporaryFile
|
||||
@@ -24,7 +23,6 @@ from stack_orchestrator.util import error_exit, global_options2
|
||||
from stack_orchestrator.deploy.deployment_create import init_operation, create_operation
|
||||
from stack_orchestrator.deploy.deploy import create_deploy_context
|
||||
from stack_orchestrator.deploy.deploy_types import DeployCommandContext
|
||||
from stack_orchestrator.deploy.webapp.util import TlsDetails
|
||||
|
||||
|
||||
def _fixup_container_tag(deployment_dir: str, image: str):
|
||||
@@ -38,16 +36,16 @@ def _fixup_container_tag(deployment_dir: str, image: str):
|
||||
wfile.write(contents)
|
||||
|
||||
|
||||
def _fixup_url_spec(spec_file_name: str, url: str, tls_details: TlsDetails = TlsDetails()):
|
||||
def _fixup_url_spec(spec_file_name: str, url: str):
|
||||
# url is like: https://example.com/path
|
||||
parsed_url = urlparse(url)
|
||||
http_proxy_spec = f''' http-proxy:
|
||||
http_proxy_spec = f'''
|
||||
http-proxy:
|
||||
- host-name: {parsed_url.hostname}
|
||||
routes:
|
||||
- path: '{parsed_url.path if parsed_url.path else "/"}'
|
||||
proxy-to: webapp:80
|
||||
{tls_details.to_yaml(indent=6)}
|
||||
'''
|
||||
'''
|
||||
spec_file_path = Path(spec_file_name)
|
||||
with open(spec_file_path) as rfile:
|
||||
contents = rfile.read()
|
||||
@@ -56,8 +54,7 @@ def _fixup_url_spec(spec_file_name: str, url: str, tls_details: TlsDetails = Tls
|
||||
wfile.write(contents)
|
||||
|
||||
|
||||
def create_deployment(ctx, deployment_dir, image, url, kube_config, image_registry, env_file,
|
||||
tls_details: TlsDetails = None):
|
||||
def create_deployment(ctx, deployment_dir, image, url, kube_config, image_registry, env_file):
|
||||
# Do the equivalent of:
|
||||
# 1. laconic-so --stack webapp-template deploy --deploy-to k8s init --output webapp-spec.yml
|
||||
# --config (eqivalent of the contents of my-config.env)
|
||||
@@ -89,7 +86,7 @@ def create_deployment(ctx, deployment_dir, image, url, kube_config, image_regist
|
||||
None
|
||||
)
|
||||
# Add the TLS and DNS spec
|
||||
_fixup_url_spec(spec_file_name, url, tls_details)
|
||||
_fixup_url_spec(spec_file_name, url)
|
||||
create_operation(
|
||||
deploy_command_context,
|
||||
spec_file_name,
|
||||
@@ -119,16 +116,8 @@ def command(ctx):
|
||||
@click.option("--image", help="image to deploy", required=True)
|
||||
@click.option("--url", help="url to serve", required=True)
|
||||
@click.option("--env-file", help="environment file for webapp")
|
||||
@click.option("--tls-host", help="Override TLS hostname (eg, '*.mydomain.com')")
|
||||
@click.option("--tls-secret", help="Override TLS secret name")
|
||||
@click.option("--tls-issuer", help="TLS issuer to use (default: letsencrypt-prod)")
|
||||
@click.pass_context
|
||||
def create(ctx, deployment_dir, image, url, kube_config, image_registry, env_file, tls_host, tls_secret, tls_issuer):
|
||||
def create(ctx, deployment_dir, image, url, kube_config, image_registry, env_file):
|
||||
'''create a deployment for the specified webapp container'''
|
||||
|
||||
if (tls_secret and not tls_host) or (tls_host and not tls_secret):
|
||||
print("Cannot specify --tls-host without --tls-secret", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
return create_deployment(ctx, deployment_dir, image, url, kube_config, image_registry, env_file,
|
||||
TlsDetails(tls_host, tls_secret, tls_issuer))
|
||||
return create_deployment(ctx, deployment_dir, image, url, kube_config, image_registry, env_file)
|
||||
|
||||
@@ -26,7 +26,7 @@ import click
|
||||
|
||||
from stack_orchestrator.deploy.images import remote_image_exists, add_tags_to_image
|
||||
from stack_orchestrator.deploy.webapp import deploy_webapp
|
||||
from stack_orchestrator.deploy.webapp.util import (LaconicRegistryClient, TimedLogger, TlsDetails,
|
||||
from stack_orchestrator.deploy.webapp.util import (LaconicRegistryClient, TimedLogger,
|
||||
build_container_image, push_container_image,
|
||||
file_hash, deploy_to_k8s, publish_deployment,
|
||||
hostname_for_deployment_request, generate_hostname_for_app,
|
||||
@@ -44,8 +44,7 @@ def process_app_deployment_request(
|
||||
kube_config,
|
||||
image_registry,
|
||||
force_rebuild,
|
||||
tls_details,
|
||||
logger,
|
||||
logger
|
||||
):
|
||||
logger.log("BEGIN - process_app_deployment_request")
|
||||
|
||||
@@ -107,7 +106,7 @@ def process_app_deployment_request(
|
||||
(app_deployment_crn, deployment_dir))
|
||||
print("deploy_webapp", deployment_dir)
|
||||
deploy_webapp.create_deployment(ctx, deployment_dir, deployment_container_tag,
|
||||
f"https://{fqdn}", kube_config, image_registry, env_filename, tls_details)
|
||||
f"https://{fqdn}", kube_config, image_registry, env_filename)
|
||||
elif env_filename:
|
||||
shutil.copyfile(env_filename, deployment_config_file)
|
||||
|
||||
@@ -199,14 +198,11 @@ def dump_known_requests(filename, requests, status="SEEN"):
|
||||
@click.option("--exclude-tags", help="Exclude requests with matching tags (comma-separated).", default="")
|
||||
@click.option("--force-rebuild", help="Rebuild even if the image already exists.", is_flag=True)
|
||||
@click.option("--log-dir", help="Output build/deployment logs to directory.", default=None)
|
||||
@click.option("--tls-host", help="Override TLS hostname (eg, '*.mydomain.com')")
|
||||
@click.option("--tls-secret", help="Override TLS secret name")
|
||||
@click.option("--tls-issuer", help="TLS issuer to use (default: letsencrypt-prod)")
|
||||
@click.pass_context
|
||||
def command(ctx, kube_config, laconic_config, image_registry, deployment_parent_dir, # noqa: C901
|
||||
request_id, discover, state_file, only_update_state,
|
||||
dns_suffix, record_namespace_dns, record_namespace_deployments, dry_run,
|
||||
include_tags, exclude_tags, force_rebuild, log_dir, tls_host, tls_secret, tls_issuer):
|
||||
include_tags, exclude_tags, force_rebuild, log_dir):
|
||||
if request_id and discover:
|
||||
print("Cannot specify both --request-id and --discover", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
@@ -224,10 +220,6 @@ def command(ctx, kube_config, laconic_config, image_registry, deployment_parent_
|
||||
print("--dns-suffix, --record-namespace-dns, and --record-namespace-deployments are all required", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
if (tls_secret and not tls_host) or (tls_host and not tls_secret):
|
||||
print("Cannot specify --tls-host without --tls-secret", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
# Split CSV and clean up values.
|
||||
include_tags = [tag.strip() for tag in include_tags.split(",") if tag]
|
||||
exclude_tags = [tag.strip() for tag in exclude_tags.split(",") if tag]
|
||||
@@ -313,7 +305,6 @@ def command(ctx, kube_config, laconic_config, image_registry, deployment_parent_
|
||||
print("Found %d unsatisfied request(s) to process." % len(requests_to_execute))
|
||||
|
||||
if not dry_run:
|
||||
tls_details = TlsDetails(tls_host, tls_secret, tls_issuer)
|
||||
for r in requests_to_execute:
|
||||
dump_known_requests(state_file, [r], "DEPLOYING")
|
||||
status = "ERROR"
|
||||
@@ -343,7 +334,6 @@ def command(ctx, kube_config, laconic_config, image_registry, deployment_parent_
|
||||
kube_config,
|
||||
image_registry,
|
||||
force_rebuild,
|
||||
tls_details,
|
||||
logger
|
||||
)
|
||||
status = "DEPLOYED"
|
||||
|
||||
@@ -26,40 +26,6 @@ import uuid
|
||||
import yaml
|
||||
|
||||
|
||||
class TlsDetails:
|
||||
def __init__(self, host_or_hosts=None, secret_name: str = None, issuer_name: str = None):
|
||||
if host_or_hosts:
|
||||
if isinstance(host_or_hosts, list):
|
||||
self.hosts = host_or_hosts
|
||||
else:
|
||||
self.hosts = [host_or_hosts]
|
||||
else:
|
||||
self.hosts = None
|
||||
self.secret_name = secret_name
|
||||
self.issuer_name = issuer_name
|
||||
|
||||
def to_yaml(self, indent=6):
|
||||
if not self.hosts and not self.secret_name and not self.issuer_name:
|
||||
return ""
|
||||
|
||||
ret = " " * indent + "tls:\n"
|
||||
indent += 2
|
||||
|
||||
if self.issuer_name:
|
||||
ret += " " * indent + "issuer: '%s'\n" % self.issuer_name
|
||||
|
||||
if self.secret_name:
|
||||
ret += " " * indent + "secret: '%s'\n" % self.secret_name
|
||||
|
||||
if self.hosts:
|
||||
ret += " " * indent + "hosts:"
|
||||
indent += 2
|
||||
for h in self.hosts:
|
||||
ret += "\n" + " " * indent + "- '%s'" % h
|
||||
|
||||
return ret
|
||||
|
||||
|
||||
class AttrDict(dict):
|
||||
def __init__(self, *args, **kwargs):
|
||||
super(AttrDict, self).__init__(*args, **kwargs)
|
||||
@@ -128,7 +94,7 @@ class LaconicRegistryClient:
|
||||
)
|
||||
|
||||
def list_records(self, criteria={}, all=False):
|
||||
args = ["laconic", "-c", self.config_file, "cns", "record", "list"]
|
||||
args = ["laconic", "-c", self.config_file, "registry", "record", "list"]
|
||||
|
||||
if all:
|
||||
args.append("--all")
|
||||
@@ -174,7 +140,7 @@ class LaconicRegistryClient:
|
||||
if name in self.cache.name_or_id:
|
||||
return self.cache.name_or_id[name]
|
||||
|
||||
args = ["laconic", "-c", self.config_file, "cns", "name", "resolve", name]
|
||||
args = ["laconic", "-c", self.config_file, "registry", "name", "resolve", name]
|
||||
|
||||
parsed = [AttrDict(r) for r in json.loads(logged_cmd(self.log_file, *args))]
|
||||
if parsed:
|
||||
@@ -199,7 +165,7 @@ class LaconicRegistryClient:
|
||||
"laconic",
|
||||
"-c",
|
||||
self.config_file,
|
||||
"cns",
|
||||
"registry",
|
||||
"record",
|
||||
"get",
|
||||
"--id",
|
||||
@@ -237,7 +203,8 @@ class LaconicRegistryClient:
|
||||
print(open(record_fname, 'r').read(), file=self.log_file)
|
||||
|
||||
new_record_id = json.loads(
|
||||
logged_cmd(self.log_file, "laconic", "-c", self.config_file, "cns", "record", "publish", "--filename", record_fname)
|
||||
logged_cmd(self.log_file, "laconic", "-c", self.config_file, "registry",
|
||||
"record", "publish", "--filename", record_fname)
|
||||
)["id"]
|
||||
for name in names:
|
||||
self.set_name(name, new_record_id)
|
||||
@@ -246,10 +213,10 @@ class LaconicRegistryClient:
|
||||
logged_cmd(self.log_file, "rm", "-rf", tmpdir)
|
||||
|
||||
def set_name(self, name, record_id):
|
||||
logged_cmd(self.log_file, "laconic", "-c", self.config_file, "cns", "name", "set", name, record_id)
|
||||
logged_cmd(self.log_file, "laconic", "-c", self.config_file, "registry", "name", "set", name, record_id)
|
||||
|
||||
def delete_name(self, name):
|
||||
logged_cmd(self.log_file, "laconic", "-c", self.config_file, "cns", "name", "delete", name)
|
||||
logged_cmd(self.log_file, "laconic", "-c", self.config_file, "registry", "name", "delete", name)
|
||||
|
||||
|
||||
def file_hash(filename):
|
||||
|
||||
Reference in New Issue
Block a user