Add ansible playbook to automate service provider setup #10

Merged
nabarun merged 31 commits from deep-stack/testnet-ops:ag-sp-setup into main 2024-10-01 12:17:11 +00:00
37 changed files with 1370 additions and 0 deletions

View File

@ -43,3 +43,4 @@
- [nitro-node-setup](./nitro-nodes-setup/README.md) - [nitro-node-setup](./nitro-nodes-setup/README.md)
- [nitro-bridge-setup](./nitro-bridge-setup/README.md) - [nitro-bridge-setup](./nitro-bridge-setup/README.md)
- [nitro-contracts-setup](./nitro-contracts-setup/README.md) - [nitro-contracts-setup](./nitro-contracts-setup/README.md)
- [service-provider-setup](./service-provider-setup/README.md)

2
service-provider-setup/.gitignore vendored Normal file
View File

@ -0,0 +1,2 @@
vars/*.yml
!vars/*.example.yml

View File

@ -0,0 +1,163 @@
# service-provider-setup
## Setup Ansible
To get started, follow the [installation](../README.md#installation) guide to setup ansible on your machine
## Prerequisites
- Set up a DigitalOcean Droplet with passwordless SSH access
- Buy a domain and configure [nameservers pointing to DigitalOcean](https://docs.digitalocean.com/products/networking/dns/getting-started/dns-registrars/)
- Generate a DigitalOcean access token, used for API authentication and managing cloud resources
## Setup a new User
- Create a new `hosts.ini` file:
```bash
cp ../hosts.example.ini hosts.ini
```
- Edit the [`hosts.ini`](./hosts.ini) file to run the playbook on a remote machine:
```ini
[root_host]
<host_name> ansible_host=<target_ip> ansible_user=<ssh_user> ansible_ssh_common_args='-o ForwardAgent=yes'
```
- Replace `<host_name>` with the desired `hostname` of the remote machine
- Replace `<target_ip>` with the IP address or hostname of the target machine
- Replace `<ssh_user>` with `root`
- Verify that you are able to connect to the host using the following command:
```bash
ansible all -m ping -i hosts.ini
# Expected output:
# <host_name> | SUCCESS => {
# "ansible_facts": {
# "discovered_interpreter_python": "/usr/bin/python3.10"
# },
# "changed": false,
# "ping": "pong"
# }
```
- Setup `user-vars.yml` using the example file
```bash
cp vars/user-vars.example.yml vars/user-vars.yml
```
- Edit the `user-vars.yml` file:
```bash
# name of the user you want to setup on the target host
username: ""
# password of the user you want to setup on the target host
password: ""
# path to the ssh key on your machine, eg: "/home/dev/.ssh/id_rsa.pub"
path_to_ssh_key: ""
```
- Execute the `setup-user.yml` Ansible playbook to create a user with passwordless sudo permissions:
```bash
cd ../
LANG=en_US.utf8 ansible-playbook setup-user.yml -i hosts.ini --extra-vars='{ "target_host": "deployment_host" }'
```
## Become a Service Provider
### Setup
- Copy the vars files:
```bash
cd vars
cp dns-vars.example.yml dns-vars.yml
cp gpg-vars.example.yml gpg-vars.yml
cp k8s-vars.example.yml k8s-vars.yml
cp container-vars.example.yml container-vars.yml
cp webapp-vars.example.yml webapp-vars.yml
cd -
```
- Update the following values in the respective variable files:
```bash
# vars/dns-vars.yml
full_domain: "" # eg: laconic.com
subdomain_prefix: "" # eg: lcn-cad
service_provider_ip: "" # eg: 23.111.78.179
do_api_token: "" # Digital Ocean access token that you generated, eg: dop_v1...
# vars/gpg-vars.yml
gpg_user_name: "" # Full name of the user for the GPG key
gpg_user_email: "" # Email address associated with the GPG key
gpg_passphrase: "" # Passphrase for securing the GPG key
# vars/k8s-vars.yml
target_host: "deployment_host"
org_id: "" # eg: lcn
location_id: "" # eg: cad
base_domain: "" # eg: laconic
support_email: "" # eg: support@laconic.com
# vars/container-vars.yml
container_registry_username: "" # username to login to the container registry
container_registry_password: "" # password to login to the container registry
# vars/webapp-vars.yml
authority_name: "" # eg: my-org-name
cpu_reservation: "" # Minimum number of cpu cores to be used, eg: 2
memory_reservation: "" # Minimum amount of memory in GB to be used, eg: 4G
deployer_gpg_passphrase: "" # passphrase for creating GPG key used by webapp-deployer, eg: SECRET
```
- Update the [`hosts.ini`](./hosts.ini) file:
```ini
[root_host]
<host_name> ansible_host=<target_ip> ansible_user=root ansible_ssh_common_args='-o ForwardAgent=yes'
[deployment_host]
<host_name> ansible_host=<target_ip> ansible_user=<new_username> ansible_ssh_common_args='-o ForwardAgent=yes'
```
- Replace `<host_name>` with the desired `hostname` of the remote machine
- Replace `<target_ip>` with the IP address or hostname of the target machine
- Under `deployment_host`, Replace `<ansible_user>` with the name of the user you have created
- Verify that you are able to connect to the host using the following command:
```bash
ansible all -m ping -i hosts.ini
# Expected output:
# <host_name> | SUCCESS => {
# "ansible_facts": {
# "discovered_interpreter_python": "/usr/bin/python3.10"
# },
# "changed": false,
# "ping": "pong"
# }
```
- Run the `service-provider-setup.yml` ansible-playbook to:
- Create DNS records
- Deploy k8s
- Setup laconicd and laconic console
- Setup container registry
- Deploy the webapp-deployer API and webapp-deployer UI
```bash
LANG=en_US.utf8 ansible-playbook service-provider-setup.yml -i hosts.ini --extra-vars='{ target_host: "deployment_host" }' --user $USER
```

View File

@ -0,0 +1,127 @@
- name: Deploy webapp-deployer backend
hosts: "{{ target_host }}"
environment:
PATH: "{{ ansible_env.PATH }}:/home/{{ansible_user}}/bin"
KUBECONFIG: "{{ ansible_env.HOME }}/.kube/config-default.yaml"
vars_files:
- vars/webapp-vars.yml
- vars/container-vars.yml
- vars/k8s-vars.yml
- vars/dns-vars.yml
tasks:
- name: Ensure gpg-keys directory exists
file:
path: ~/gpg-keys
state: directory
mode: '0700'
- name: Create a GPG key
shell: gpg --batch --passphrase "{{ deployer_gpg_passphrase }}" --quick-generate-key webapp-deployer-api.{{ full_domain }} default default never
- name: Export the public key
shell: gpg --export webapp-deployer-api.{{ full_domain }} > ~/gpg-keys/webapp-deployer-api.{{ full_domain }}.pgp.pub
args:
creates: ~/gpg-keys/webapp-deployer-api.{{ full_domain }}.pgp.pub
- name: Export the GPG private key with passphrase
shell: gpg --pinentry-mode=loopback --passphrase "{{ deployer_gpg_passphrase }}" --export-secret-keys webapp-deployer-api.{{ full_domain }} > ~/gpg-keys/webapp-deployer-api.{{ full_domain }}.pgp.key
- name: Setup repositories for webapp-deployer-backend
command: laconic-so --stack webapp-deployer-backend setup-repositories
- name: Build containers for webapp-deployer-backend
command: laconic-so --stack webapp-deployer-backend build-containers
- name: Ensure the config directory exists
file:
path: "{{ ansible_env.HOME }}/config"
state: directory
- name: Create laconic config file
template:
src: "./templates/laconic.yml.j2"
dest: "config/laconic.yml"
- name: Copy the gpg private key file to config dir
copy:
src: "gpg-keys/webapp-deployer-api.{{ full_domain }}.pgp.key"
dest: "config"
remote_src: true
- name: Copy the gpg public key file to config dir
copy:
src: "gpg-keys/webapp-deployer-api.{{ full_domain }}.pgp.pub"
dest: "config"
remote_src: true
- name: Publish the webapp-deployer record using laconic-so
shell: |
docker run -i -t \
-v /home/{{ ansible_user }}/config:/home/root/config \
cerc/webapp-deployer-backend:local laconic-so publish-deployer-to-registry \
--laconic-config /home/root/config/laconic.yml \
--api-url https://webapp-deployer-api.pwa.{{ full_domain }} \
--public-key-file /home/root/config/webapp-deployer-api.{{ full_domain }}.pgp.pub \
--lrn lrn://{{ authority_name }}/deployers/webapp-deployer-api.{{ full_domain }} \
--min-required-payment 0
register: publish_output
- name: Display publish output
debug:
var: publish_output.stdout
- name: Generate spec file for webapp-deployer-backend
template:
src: "./templates/specs/webapp-deployer.spec.j2"
dest: "webapp-deployer.spec"
- name: Create the deployment directory from the spec file
command: >
laconic-so --stack webapp-deployer-backend deploy create
--deployment-dir webapp-deployer --spec-file webapp-deployer.spec
- name: Update config for webapp-deployer-backend
template:
src: "./templates/configs/webapp-deployer-config.env.j2"
dest: "webapp-deployer/config.env"
- name: Copy the kube config file to webapp-deployer directory
copy:
src: "{{ansible_env.HOME}}/.kube/config-default.yaml"
dest: "webapp-deployer/data/config/kube.yml"
remote_src: true
- name: Create laconic config file
template:
src: "./templates/laconic.yml.j2"
dest: "webapp-deployer/data/config/laconic.yml"
- name: login to the container registry
command: "docker login container-registry.pwa.{{ full_domain }} --username {{ container_registry_username }} --password {{ container_registry_password}}"
- name: Push images to container registry
command: laconic-so deployment --dir webapp-deployer push-images
- name: Start the webapp deployer
command: laconic-so deployment --dir webapp-deployer start
- name: Get the most recent pod for the deployment
shell: kubectl get pods --sort-by=.metadata.creationTimestamp -o jsonpath='{.items[-1].metadata.name}'
register: webapp_deployer_pod
- name: Set pod ID to a variable
set_fact:
pod_id: "{{ webapp_deployer_pod.stdout }}"
- name: Wait for the recent pod to be ready
command: kubectl wait --for=condition=Ready pod/{{ pod_id }} --timeout=300s
register: wait_result
- name: Copy gpg private key file to webapp deployer pod
shell: kubectl cp gpg-keys/webapp-deployer-api.{{ full_domain }}.pgp.key {{ pod_id }}:/app
- name: Copy gpg public key file to webapp deployer pod
shell: kubectl cp gpg-keys/webapp-deployer-api.{{ full_domain }}.pgp.pub {{ pod_id }}:/app

View File

@ -0,0 +1,43 @@
- name: Deploy webapp-deployer ui
hosts: "{{ target_host }}"
environment:
PATH: "{{ ansible_env.PATH }}:/home/{{ansible_user}}/bin"
vars_files:
- vars/webapp-vars.yml
- vars/dns-vars.yml
- vars/k8s-vars.yml
tasks:
- name: Clone webapp-deployment-status-ui repository
git:
repo: "https://git.vdb.to/cerc-io/webapp-deployment-status-ui.git"
dest: "{{ ansible_env.HOME }}/cerc/webapp-deployment-status-ui"
update: yes
- name: Build webapp-deployer-status-ui
command: laconic-so build-webapp --source-repo {{ ansible_env.HOME }}/cerc/webapp-deployment-status-ui
- name: Create a deployment for webapp-ui
command: |
laconic-so deploy-webapp create --kube-config {{ ansible_env.HOME }}/.kube/config-default.yaml
--image-registry container-registry.pwa.{{ full_domain }} --deployment-dir webapp-ui
--image cerc/webapp-deployment-status-ui:local --url https://webapp-deployer-ui.pwa.{{ full_domain }}
--env-file ~/cerc/webapp-deployment-status-ui/.env
- name: Push image to container registry
command: laconic-so deployment --dir webapp-ui push-images
- name: Update config file for webapp ui
template:
src: "./templates/configs/webapp-ui-config.env.j2"
dest: "webapp-ui/config.env"
- name: Start the deployer ui
command: laconic-so deployment --dir webapp-ui start
- name: Create .out file
file:
path: "{{ ansible_env.HOME }}/.out"
state: touch

View File

@ -0,0 +1,77 @@
- name: Setup and run laconic console
hosts: "{{target_host}}"
environment:
PATH: "{{ ansible_env.PATH }}:/home/{{ansible_user}}/bin"
vars_files:
- vars/webapp-vars.yml
- vars/dns-vars.yml
- vars/k8s-vars.yml
tasks:
- name: Clone the stack repo
command: laconic-so fetch-stack git.vdb.to/cerc-io/testnet-laconicd-stack --pull
ignore_errors: yes
- name: Clone required repositories for laconic-console
command: laconic-so --stack ~/cerc/testnet-laconicd-stack/stack-orchestrator/stacks/laconic-console setup-repositories --pull
- name: Build container images
command: laconic-so --stack ~/cerc/testnet-laconicd-stack/stack-orchestrator/stacks/laconic-console build-containers --force-rebuild
- name: Generate spec file for laconic console deployment
template:
src: "./templates/specs/laconic-console-spec.yml.j2"
dest: "laconic-console-spec.yml"
- name: Check if the deployment directory exists
stat:
path: laconic-console-deployment
register: deployment_dir
- name: Create a deployment from the spec file
command: laconic-so --stack ~/cerc/testnet-laconicd-stack/stack-orchestrator/stacks/laconic-console deploy create --spec-file laconic-console-spec.yml --deployment-dir laconic-console-deployment
when: not deployment_dir.stat.exists
- name: Place deployment in the same namespace as fixturenet-laconicd
copy:
src: "fixturenet-laconicd-deployment/deployment.yml"
dest: "laconic-console-deployment/deployment.yml"
remote_src: yes
- name: Fetch user key from laconicd
command: laconic-so deployment --dir fixturenet-laconicd-deployment exec laconicd "echo y | laconicd keys export alice --unarmored-hex --unsafe"
register: alice_pk
- name: Set Private key for console deployment
set_fact:
ALICE_PK: "{{ alice_pk.stdout }}"
- name: Start the laconic console deployment
command: laconic-so deployment --dir laconic-console-deployment start
- name: Create a bond using cli
shell: laconic-so deployment --dir laconic-console-deployment exec cli "laconic registry bond create --type alnt --quantity 1000000000000 --user-key {{ALICE_PK}}" | jq -r '.bondId'
register: bond_id
- name: Set Bond ID for console deployment
set_fact:
BOND_ID: "{{ bond_id.stdout }}"
- name: Stop the console deployment
command: laconic-so deployment --dir laconic-console-deployment stop
- name: Modify the console config with alice_pk and bond_id
template:
src: "./templates/configs/console-config.env.j2"
dest: "laconic-console-deployment/config.env"
- name: Start the laconic console deployment with updated config
command: laconic-so deployment --dir laconic-console-deployment start
- name: Reserve an authority
command: laconic-so deployment --dir laconic-console-deployment exec cli "laconic registry authority reserve {{authority_name}}"
- name: Set authority using bond id
command: laconic-so deployment --dir laconic-console-deployment exec cli "laconic registry authority bond set {{authority_name}} {{BOND_ID}}"

View File

@ -0,0 +1,33 @@
- name: Setup and run fixturnet-laconicd-stack
hosts: "{{ target_host }}"
environment:
PATH: "{{ ansible_env.PATH }}:/home/{{ansible_user}}/bin"
tasks:
- name: Clone the fixturenet-laconicd-stack repo
command: laconic-so fetch-stack git.vdb.to/cerc-io/fixturenet-laconicd-stack --pull
ignore_errors: yes
- name: Setup repos for fixturenet-laconicd
command: laconic-so --stack ~/cerc/fixturenet-laconicd-stack/stack-orchestrator/stacks/fixturenet-laconicd setup-repositories
- name: Build container images
command: laconic-so --stack ~/cerc/fixturenet-laconicd-stack/stack-orchestrator/stacks/fixturenet-laconicd build-containers --force-rebuild
- name: Generate over spec file for laconicd deployment
template:
src: "./templates/specs/fixturenet-laconicd-spec.yml.j2"
dest: "fixturenet-laconicd-spec.yml"
- name: Check if the deployment directory exists
stat:
path: "fixturenet-laconicd-deployment"
register: deployment_dir
- name: Create the deployment from the spec file
command: laconic-so --stack ~/cerc/fixturenet-laconicd-stack/stack-orchestrator/stacks/fixturenet-laconicd deploy create --spec-file fixturenet-laconicd-spec.yml --deployment-dir fixturenet-laconicd-deployment
when: not deployment_dir.stat.exists
- name: Start the deployment
command: laconic-so deployment --dir fixturenet-laconicd-deployment start

View File

@ -0,0 +1,20 @@
- hosts: "{{ target_host }}"
tasks:
- name: Check if .out file exists
stat:
path: "{{ ansible_env.HOME }}/.out"
register: out_file
- name: Exit playbook if .out file exists
fail:
msg: ".out file exists, exiting playbook."
when: out_file.stat.exists
- import_playbook: setup-dns.yml
- import_playbook: setup-system.yml
- import_playbook: setup-k8s.yml
- import_playbook: setup-container-registry.yml
- import_playbook: run-laconicd.yml
- import_playbook: run-laconic-console.yml
- import_playbook: deploy-backend.yml
- import_playbook: deploy-frontend.yml

View File

@ -0,0 +1,161 @@
- name: Setup container registry
hosts: "{{ target_host }}"
environment:
PATH: "{{ ansible_env.PATH }}:/home/{{ansible_user}}/bin"
vars_files:
- vars/k8s-vars.yml
- vars/container-vars.yml
- vars/dns-vars.yml
tasks:
- name: Generate spec file for the container-registry stack
template:
src: "./templates/specs/container-registry.spec.j2"
dest: "{{ansible_env.HOME}}/container-registry.spec"
- name: Create a deployment for the container-registry stack
command: laconic-so --stack container-registry deploy create --deployment-dir container-registry --spec-file container-registry.spec
- name: Base64 encode the container registry credentials
set_fact:
b64_encoded_cred: "{{ (container_registry_username + ':' + container_registry_password) | b64encode }}"
- name: Encrypt the container registry credentials to create an htpasswd file
command: >
htpasswd -bB -c container-registry/configmaps/config/htpasswd
{{ container_registry_username }} {{ container_registry_password }}
register: htpasswd_file
- name: Read the htpasswd file
slurp:
src: "container-registry/configmaps/config/htpasswd"
register: htpasswd_file_content
- name: Extract the hashed password (after the colon)
set_fact:
hashed_password: "{{ (htpasswd_file_content.content | b64decode).split(':')[1] | trim }}"
- name: Create container-registry/my_password.json file
template:
src: "./templates/my_password.json.j2"
dest: "container-registry/my_password.json"
- name: Configure the file container-registry/config.env
copy:
dest: "container-registry/config.env"
content: |
REGISTRY_AUTH=htpasswd
REGISTRY_AUTH_HTPASSWD_REALM="{{org_id}} Service Provider Image Registry"
REGISTRY_AUTH_HTPASSWD_PATH="/config/htpasswd"
REGISTRY_HTTP_SECRET='{{ hashed_password }}'
- name: Set KUBECONFIG environment variable
set_fact:
kubeconfig_path: "{{ ansible_env.HOME }}/.kube/config-default.yaml"
- name: Add the container registry credentials as a secret available to the cluster
command: >
kubectl create secret generic laconic-registry
--from-file=.dockerconfigjson=container-registry/my_password.json
--type=kubernetes.io/dockerconfigjson
environment:
KUBECONFIG: "{{ kubeconfig_path }}"
# TODO: Investigate why container registry throws error if started immediately
- name: Wait for 90 seconds
pause:
seconds: 90
- block:
- name: Get Kubernetes nodes with wide output
command: kubectl get nodes -o wide
environment:
KUBECONFIG: "{{ kubeconfig_path }}"
register: nodes_output
- name: Print output of 'kubectl get nodes -o wide'
debug:
var: nodes_output.stdout
- name: Get all secrets from all namespaces
command: kubectl get secrets --all-namespaces
environment:
KUBECONFIG: "{{ kubeconfig_path }}"
register: secrets_output
- name: Print output of 'kubectl get secrets --all-namespaces'
debug:
var: secrets_output.stdout
- name: Get cluster issuers
command: kubectl get clusterissuer
environment:
KUBECONFIG: "{{ kubeconfig_path }}"
register: clusterissuer_output
- name: Print output of 'kubectl get clusterissuer'
debug:
var: clusterissuer_output.stdout
- name: Get certificates
command: kubectl get certificates
environment:
KUBECONFIG: "{{ kubeconfig_path }}"
register: certificates_output
- name: Print output of 'kubectl get certificates'
debug:
var: certificates_output.stdout
- name: Get DaemonSets in all namespaces
command: kubectl get ds --all-namespaces
environment:
KUBECONFIG: "{{ kubeconfig_path }}"
register: daemonsets_output
- name: Print output of 'kubectl get ds --all-namespaces'
debug:
var: daemonsets_output.stdout
ignore_errors: yes
- name: Deploy the container registry
command: >
laconic-so deployment --dir container-registry start
- name: Get cluster_id from container-registry-deployment
slurp:
src: container-registry/deployment.yml
register: deployment_file
- name: Decode and extract cluster-id
set_fact:
extracted_cluster_id: "{{ deployment_file.content | b64decode | regex_search('cluster-id: (.+)', '\\1') }}"
- name: Set modified cluster-id
set_fact:
formatted_cluster_id: "{{ extracted_cluster_id | replace('[', '') | replace(']', '') | replace(\"'\", '') }}"
- name: Display the cluster ID
debug:
msg: "The cluster ID is: {{ formatted_cluster_id }}"
- name: Annotate ingress for proxy body size
command: >
kubectl annotate ingress {{ formatted_cluster_id }}-ingress nginx.ingress.kubernetes.io/proxy-body-size=0
environment:
KUBECONFIG: "{{ ansible_env.HOME }}/.kube/config-default.yaml"
- name: Annotate ingress for proxy read timeout
command: >
kubectl annotate ingress {{ formatted_cluster_id }}-ingress nginx.ingress.kubernetes.io/proxy-read-timeout=600
environment:
KUBECONFIG: "{{ ansible_env.HOME }}/.kube/config-default.yaml"
- name: Annotate ingress for proxy send timeout
command: >
kubectl annotate ingress {{ formatted_cluster_id }}-ingress nginx.ingress.kubernetes.io/proxy-send-timeout=600
environment:
KUBECONFIG: "{{ ansible_env.HOME }}/.kube/config-default.yaml"

View File

@ -0,0 +1,72 @@
- name: Configure DNS records
hosts: localhost
vars_files:
- vars/dns-vars.yml
tasks:
- name: Create a domain
community.digitalocean.digital_ocean_domain:
state: present
oauth_token: "{{ do_api_token }}"
name: "{{ full_domain }}"
ip: "{{ service_provider_ip }}"
- name: Create record for cluster control machine
community.digitalocean.digital_ocean_domain_record:
state: present
oauth_token: "{{ do_api_token }}"
domain: "{{ full_domain }}"
type: A
name: "{{ subdomain_prefix }}-cluster-control"
data: "{{ service_provider_ip }}"
- name: Create CNAME record for www
community.digitalocean.digital_ocean_domain_record:
state: present
oauth_token: "{{ do_api_token }}"
data: "{{ full_domain }}"
domain: "{{ full_domain }}"
type: CNAME
name: www
ttl: 43200
- name: Create CNAME record for subdomain
community.digitalocean.digital_ocean_domain_record:
state: present
oauth_token: "{{ do_api_token }}"
data: "{{ subdomain_cluster_control }}.{{ full_domain }}"
domain: "{{ full_domain }}"
type: CNAME
name: "{{ subdomain_prefix }}"
ttl: 43200
- name: Create wildcard CNAME record for subdomain
community.digitalocean.digital_ocean_domain_record:
state: present
oauth_token: "{{ do_api_token }}"
data: "{{ subdomain_cluster_control }}.{{ full_domain }}"
domain: "{{ full_domain }}"
type: CNAME
name: "*.{{ subdomain_prefix }}"
ttl: 43200
- name: Create CNAME record for pwa
community.digitalocean.digital_ocean_domain_record:
state: present
oauth_token: "{{ do_api_token }}"
data: "{{ subdomain_cluster_control }}.{{ full_domain }}"
domain: "{{ full_domain }}"
type: CNAME
name: "pwa"
ttl: 43200
- name: Create wildcard CNAME record for pwa
community.digitalocean.digital_ocean_domain_record:
state: present
oauth_token: "{{ do_api_token }}"
data: "{{ subdomain_cluster_control }}.{{ full_domain }}"
domain: "{{ full_domain }}"
type: CNAME
name: "*.pwa"
ttl: 43200

View File

@ -0,0 +1,183 @@
- name: Install Stack Orchestrator if it isn't present
import_playbook: ../stack-orchestrator-setup/setup-laconic-so.yml
- name: Setup k8s
hosts: "{{ target_host }}"
environment:
PATH: "{{ ansible_env.PATH }}:/home/{{ansible_user}}/.local/bin"
VAULT_KEY: "{{ vault_passphrase }}"
vars_files:
- vars/dns-vars.yml
- vars/gpg-vars.yml
- vars/k8s-vars.yml
tasks:
- name: Install Python and pip
apt:
name: "{{ item }}"
state: present
become: yes
loop:
- python3
- python3-pip
- name: Add user to docker group
user:
name: "{{ ansible_user }}"
groups: docker
append: true
- name: Install Ansible on remote host
pip:
name: ansible
extra_args: --user
when: target_host != "localhost"
- name: Ensure ~/.local/bin is in PATH in .bashrc
lineinfile:
path: ~/.bashrc
line: 'export PATH="$HOME/.local/bin:$PATH"'
state: present
create: yes
- name: Ensure ~/.local/bin is in PATH in .zshrc
lineinfile:
path: ~/.zshrc
line: 'export PATH="$HOME/.local/bin:$PATH"'
state: present
create: yes
- name: Clone the service provider template repo
git:
repo: "https://git.vdb.to/cerc-io/service-provider-template.git"
dest: "{{ ansible_env.HOME }}/service-provider-template"
- name: Update .vault/vault-keys file
lineinfile:
path: "service-provider-template/.vault/vault-keys"
regexp: '^.*$'
line: "{{ gpg_key_id }}"
create: yes
- name: Start GPG agent
command: gpg-agent --daemon
ignore_errors: yes
- name: Sign a dummy string using gpg-key
shell: echo "This is a dummy string." | gpg --batch --yes --local-user "{{ gpg_key_id }}" --passphrase "{{ vault_passphrase }}" --pinentry-mode loopback --sign -
- name: Run vault-rekey.sh
shell: bash .vault/vault-rekey.sh
args:
chdir: "service-provider-template"
register: rekey_result
until: rekey_result.stderr == ""
retries: 5
delay: 5
- name: Ensure the target directory exists
file:
path: "{{ ansible_env.HOME }}/service-provider-template"
state: directory
mode: '0755'
- name: Change directory name in group_vars
command: mv lcn_cad {{ org_id }}_{{ location_id }}
args:
chdir: "{{ ansible_env.HOME }}/service-provider-template/group_vars"
- name: Change control directory name in host_vars
command: mv lcn-cad-cluster-control {{ org_id }}-{{ location_id }}-cluster-control
args:
chdir: "{{ ansible_env.HOME }}/service-provider-template/host_vars"
- name: Change daemon directory name in host_vars
command: mv lcn-daemon {{ org_id }}-daemon
args:
chdir: "{{ ansible_env.HOME }}/service-provider-template/host_vars"
- name: Copy control-firewalld.yml to the remote VM
template:
src: ./templates/control-firewalld.yml.j2
dest: "{{ ansible_env.HOME }}/service-provider-template/host_vars/{{ org_id }}-{{ location_id }}-cluster-control/firewalld.yml"
- name: Copy daemon-firewalld.yml to the remote VM
template:
src: ./templates/daemon-firewalld.yml.j2
dest: "{{ ansible_env.HOME }}/service-provider-template/host_vars/{{ org_id }}-daemon/firewalld.yml"
- name: Copy nginx.yml to the remote VM
template:
src: ./templates/nginx.yml.j2
dest: "{{ ansible_env.HOME }}/service-provider-template/host_vars/{{ org_id }}-daemon/nginx.yml"
- name: Copy hosts file to the remote VM
template:
src: ./templates/hosts.j2
dest: "{{ ansible_env.HOME }}/service-provider-template/hosts"
- name: Copy k8s.yml to the remote VM
template:
src: ./templates/k8s.yml.j2
dest: "{{ ansible_env.HOME }}/service-provider-template/group_vars/{{ org_id }}_{{ location_id }}/k8s.yml"
- name: Copy wildcard-pwa-{{ base_domain }}.yaml to the remote VM
template:
src: ./templates/wildcard-pwa-example.yml.j2
dest: "{{ ansible_env.HOME }}/service-provider-template/files/manifests/wildcard-pwa-{{ base_domain }}.yaml"
- name: Delete old wildcard-pwa file
file:
path: "{{ ansible_env.HOME }}/service-provider-template/files/manifests/wildcard-pwa-laconic.yaml"
state: absent
- name: Install required ansible roles
shell: ansible-galaxy install -f -p roles -r roles/requirements.yml
args:
chdir: "{{ ansible_env.HOME }}/service-provider-template"
- name: Install Kubernetes helper tools
shell: ./roles/k8s/files/scripts/get-kube-tools.sh
args:
chdir: "{{ ansible_env.HOME }}/service-provider-template"
become: yes
- name: Update group_vars/all/vault.yml with support email using template
template:
src: ./templates/vault.yml.j2
dest: "{{ ansible_env.HOME }}/service-provider-template/group_vars/all/vault.yml"
- name: Base64 encode DigitalOcean token
set_fact:
b64_encoded_token: "{{ do_api_token | b64encode }}"
- name: Update secret-digitalocean-dns.yaml with encoded token
template:
src: ./templates/secret-digitalocean-dns.yml.j2
dest: "{{ ansible_env.HOME }}/service-provider-template/files/manifests/secret-digitalocean-dns.yaml"
vars:
b64_encoded_token: "{{ b64_encoded_token }}"
- name: Remove k8s-vault.yml file
file:
path: "{{ ansible_env.HOME }}/service-provider-template/group_vars/{{ org_id }}_{{ location_id }}/k8s-vault.yml"
state: absent
- name: Generate token for the cluster
command: ./roles/k8s/files/scripts/token-vault.sh ./group_vars/{{ org_id }}_{{ location_id }}/k8s-vault.yml
args:
chdir: "{{ ansible_env.HOME }}/service-provider-template"
- name: Configure firewalld and nginx
command: ansible-playbook -i hosts site.yml --tags=firewalld,nginx
args:
chdir: "{{ ansible_env.HOME }}/service-provider-template"
environment:
ANSIBLE_HOST_KEY_CHECKING: "False"
- name: Deploy Kubernetes
command: ansible-playbook -i hosts site.yml --tags=k8s --limit={{ org_id }}_{{ location_id }} --user {{ ansible_user }}
args:
chdir: "{{ ansible_env.HOME }}/service-provider-template"

View File

@ -0,0 +1,134 @@
- name: Setup system for the service provider setup
hosts: "{{ target_host }}"
environment:
GNUPGHOME: /home/{{ ansible_user }}/.gnupg
vars_files:
- vars/k8s-vars.yml
- vars/dns-vars.yml
- vars/gpg-vars.yml
tasks:
- name: Install required packages
apt:
name:
- doas
- zsh
- tmux
- git
- jq
- acl
- curl
- wget
- netcat-traditional
- fping
- rsync
- htop
- iotop
- iftop
- tar
- less
- firewalld
- sshguard
- wireguard
- iproute2
- iperf3
- zfsutils-linux
- net-tools
- ca-certificates
- gnupg
- sshpass
- apache2-utils
state: latest
update_cache: true
become: yes
- name: Set unique hostname
hostname:
name: "{{ inventory_hostname }}"
when: ansible_hostname != inventory_hostname
- name: Verify status of firewalld and enable sshguard
systemd:
name: "{{ item }}"
enabled: yes
state: started
loop:
- firewalld
- sshguard
ignore_errors: yes
- name: Disable and remove snapd
block:
- name: Disable snapd services
systemd:
name: "{{ item }}"
enabled: no
state: stopped
loop:
- snapd.service
- snapd.socket
- snapd.seeded
- snapd.snap-repair.timer
ignore_errors: yes
- name: Purge snapd
apt:
name: snapd
state: absent
- name: Remove snap directories
file:
path: "{{ item }}"
state: absent
loop:
- "{{ ansible_env.HOME }}/snap"
- /snap
- /var/snap
- /var/lib/snapd
become: yes
ignore_errors: yes
- name: Ensure GPG directory exists
file:
path: "{{ ansible_env.HOME }}/.gnupg"
state: directory
mode: '0700'
- name: Create GPG key parameters file
copy:
dest: /tmp/gpg_key_params.txt
content: |
Key-Type: RSA
Key-Length: 4096
Subkey-Type: RSA
Name-Real: {{ gpg_user_name }}
Name-Email: {{ gpg_user_email }}
Expire-Date: 0
Passphrase: {{ gpg_passphrase }}
%no-protection
%commit
mode: '0600'
- name: Generate GPG key using the parameter file
command: gpg --batch --gen-key /tmp/gpg_key_params.txt
become_user: "{{ ansible_user }}"
register: gpg_keygen_output
ignore_errors: yes
- name: Show GPG key generation output
debug:
var: gpg_keygen_output.stdout
- name: Fetch the Key ID of the most recently created GPG key
shell: gpg --list-secret-keys --keyid-format=long | grep 'sec' | tail -n 1 | awk -F'/' '{print $2}' | awk '{print $1}'
register: gpg_key_output
- name: Set the GPG key ID to a variable
set_fact:
sec_key_id: "{{ gpg_key_output.stdout }}"
- name: Show GPG Key ID
debug:
msg: "GPG Key ID: {{ sec_key_id }}"

View File

@ -0,0 +1,46 @@
- name: Configure system
hosts: root_host
become: yes
vars_files:
- vars/user-vars.yml
tasks:
- name: Create a user
user:
name: "{{ username }}"
password: "{{ '{{ password }}' | password_hash('sha512') }}"
shell: /bin/bash
state: present
- name: Add user to sudoers group
user:
name: "{{ username }}"
groups: sudo
append: yes
- name: Ensure .ssh directory exists for user
file:
path: /home/{{ username }}/.ssh
state: directory
owner: "{{ username }}"
group: "{{ username }}"
mode: '0700'
- name: Append SSH public key to authorized_keys
lineinfile:
path: /home/{{ username }}/.ssh/authorized_keys
line: "{{ lookup('file', path_to_ssh_key) }}"
create: yes
owner: "{{ username }}"
group: "{{ username }}"
mode: '0600'
state: present
- name: Add user to sudoers for passwordless sudo
lineinfile:
path: /etc/sudoers
state: present
regexp: '^{{ username }} ALL=\(ALL\) NOPASSWD:ALL'
line: '{{ username }} ALL=(ALL) NOPASSWD:ALL'
validate: 'visudo -cf %s'

View File

@ -0,0 +1,5 @@
CERC_LACONICD_USER_KEY={{ALICE_PK}}
CERC_LACONICD_BOND_ID={{BOND_ID}}
CERC_LACONICD_RPC_ENDPOINT=http://{{ org_id }}-{{ location_id }}-cluster-control.{{ full_domain }}:26657
CERC_LACONICD_GQL_ENDPOINT=http://{{ org_id }}-{{ location_id }}-cluster-control.{{ full_domain }}:9473/api
LACONIC_HOSTED_ENDPOINT=http://{{ org_id }}-{{ location_id }}-cluster-control.{{ full_domain }}:9473

View File

@ -0,0 +1,28 @@
DEPLOYMENT_DNS_SUFFIX="pwa.{{ full_domain }}"
# Name of reserved authority
DEPLOYMENT_RECORD_NAMESPACE="{{ authority_name }}"
# url of the deployed docker image registry
IMAGE_REGISTRY="container-registry.pwa.{{ full_domain }}"
# htpasswd credentials
IMAGE_REGISTRY_USER="{{ container_registry_username }}"
IMAGE_REGISTRY_CREDS="{{ container_registry_password }}"
# configs
CLEAN_DEPLOYMENTS=false
CLEAN_LOGS=false
CLEAN_CONTAINERS=false
SYSTEM_PRUNE=false
WEBAPP_IMAGE_PRUNE=true
CHECK_INTERVAL=5
FQDN_POLICY="allow"
# lrn of the webapp deployer
LRN="lrn://{{ authority_name }}/deployers/webapp-deployer-api.{{ full_domain }}"
export OPENPGP_PRIVATE_KEY_FILE="webapp-deployer-api.{{ full_domain }}.pgp.key"
export OPENPGP_PASSPHRASE="{{ deployer_gpg_passphrase }}"
export DEPLOYER_STATE="srv-test/deployments/autodeploy.state"
export UNDEPLOYER_STATE="srv-test/deployments/autoundeploy.state"
export UPLOAD_DIRECTORY="srv-test/uploads"

View File

@ -0,0 +1,3 @@
CERC_WEBAPP_DEBUG=0.1.0
LACONIC_HOSTED_CONFIG_app_api_url=https://webapp-deployer-api.pwa.{{ full_domain }}
LACONIC_HOSTED_CONFIG_app_console_link=http://{{ org_id }}-{{ location_id }}-cluster-control.{{ full_domain }}:9473/console?query=%0A%20%20fragment%20ValueParts%20on%20Value%20%7B%0A%20%20%20%20...%20on%20BooleanValue%20%7B%0A%20%20%20%20%20%20bool%3A%20value%0A%20%20%20%20%7D%0A%20%20%20%20...%20on%20IntValue%20%7B%0A%20%20%20%20%20%20int%3A%20value%0A%20%20%20%20%7D%0A%20%20%20%20...%20on%20FloatValue%20%7B%0A%20%20%20%20%20%20float%3A%20value%0A%20%20%20%20%7D%0A%20%20%20%20...%20on%20StringValue%20%7B%0A%20%20%20%20%20%20string%3A%20value%0A%20%20%20%20%7D%0A%20%20%20%20...%20on%20BytesValue%20%7B%0A%20%20%20%20%20%20bytes%3A%20value%0A%20%20%20%20%7D%0A%20%20%20%20...%20on%20LinkValue%20%7B%0A%20%20%20%20%20%20link%3A%20value%0A%20%20%20%20%7D%0A%20%20%7D%0A%0A%20%20fragment%20AttrParts%20on%20Attribute%20%7B%0A%20%20%20%20key%0A%20%20%20%20value%20%7B%0A%20%20%20%20%20%20...ValueParts%0A%20%20%20%20%20%20...%20on%20ArrayValue%20%7B%0A%20%20%20%20%20%20%20%20value%20%7B%0A%20%20%20%20%20%20%20%20%20%20...ValueParts%0A%20%20%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%20%20%7D%0A%0A%20%20%7B%0A%20%20%20%20getRecordsByIds(ids%3A%20%5B%22#RQID#%22%5D)%20%7B%0A%20%20%20%20%20%20id%0A%20%20%20%20%20%20names%0A%20%20%20%20%20%20bondId%0A%20%20%20%20%20%20createTime%0A%20%20%20%20%20%20expiryTime%0A%20%20%20%20%20%20owners%0A%20%20%20%20%20%20attributes%20%7B%0A%20%20%20%20%20%20%20%20...AttrParts%0A%20%20%20%20%20%20%20%20value%20%7B%0A%20%20%20%20%20%20%20%20%20%20...%20on%20MapValue%20%7B%0A%20%20%20%20%20%20%20%20%20%20%20%20map%3A%20value%20%7B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20...AttrParts%0A%20%20%20%20%20%20%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%20%20%7D%0A

View File

@ -0,0 +1,16 @@
---
firewalld_add:
- name: public
interfaces:
- enp9s0
services:
- http
- https
ports:
- 6443/tcp
- name: trusted
sources:
- 10.42.0.0/16
- 10.43.0.0/16
- {{ service_provider_ip }}

View File

@ -0,0 +1,16 @@
---
firewalld_add:
- name: public
interfaces:
- ens3
services:
- http
- https
ports:
- 26657/tcp
- 26656/tcp
- 1317/tcp
- name: trusted
sources:
- {{ service_provider_ip }}

View File

@ -0,0 +1,12 @@
[all]
{{ org_id }}-daemon ansible_host={{ service_provider_ip }}
{{ org_id }}-{{ location_id }}-cluster-control ansible_host={{ service_provider_ip }}
[so]
{{ org_id }}-daemon
[{{ org_id }}_{{ location_id }}]
{{ org_id }}-{{ location_id }}-cluster-control k8s_node_type=bootstrap k8s_pod_limit=1024 k8s_external_ip={{ service_provider_ip }}
[k8s:children]
{{ org_id }}_{{ location_id }}

View File

@ -0,0 +1,55 @@
---
# default context is used for stack orchestrator deployments, for testing a custom context name can be usefull
#k8s_cluster_name: {{ org_id }}-{{ location_id }}-cluster
k8s_cluster_name: default
k8s_cluster_url: {{ org_id }}-{{ location_id }}-cluster-control.{{ full_domain }}
k8s_taint_servers: false
k8s_acme_email: "{{ support_email }}"
# k3s bundles traefik as the default ingress controller, we will disable it and use nginx instead
k8s_disable:
- traefik
# secrets can be stored in a file or as a template, the template secrets gets dynamically base64 encoded while file based secrets must be encoded by hand
k8s_secrets:
- name: digitalocean-dns
type: file
source: secret-digitalocean-dns.yaml
k8s_manifests:
# ingress controller, replaces traefik which is explicitly disabled
- name: ingress-nginx
type: url
source: https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.10.1/deploy/static/provider/cloud/deploy.yaml
# cert-manager, required for letsencrypt
- name: cert-manager
type: url
source: https://github.com/cert-manager/cert-manager/releases/download/v1.15.1/cert-manager.yaml
# issuer for basic http certs
- name: letsencrypt-prod
type: template
source: shared/clusterissuer-acme.yaml
server: https://acme-v02.api.letsencrypt.org/directory
solvers:
- type: http
ingress: nginx
# issuer for wildcard dns certs
- name: letsencrypt-prod-wild
type: template
source: shared/clusterissuer-acme.yaml
server: https://acme-v02.api.letsencrypt.org/directory
solvers:
- type: dns
provider: digitalocean
tokenref: tokenSecretRef
secret_name: digitalocean-dns
secret_key: access-token
# initiate wildcard cert
- name: pwa.{{ full_domain }}
type: file
source: wildcard-pwa-{{ base_domain }}.yaml

View File

@ -0,0 +1,9 @@
services:
registry:
rpcEndpoint: 'http://{{ subdomain_cluster_control }}.{{ full_domain }}:26657'
gqlEndpoint: 'http://{{ subdomain_cluster_control}}.{{ full_domain }}:9473/api'
userKey: "{{ ALICE_PK }}"
bondId: "{{ BOND_ID }}"
chainId: lorotestnet-1
gas: 200000
fees: 200000alnt

View File

@ -0,0 +1,9 @@
{
"auths": {
"{{container_registry_domain}}": {
"username": "{{ container_registry_username }}",
"password": "{{ hashed_password }}",
"auth": "{{ b64_encoded_cred }}"
}
}
}

View File

@ -0,0 +1,21 @@
---
nginx_packages_intall: false
nginx_server_name_hash: 64
nginx_proxy_read_timeout: 1200
nginx_proxy_send_timeout: 1200
nginx_proxy_connection_timeout: 75
nginx_sites:
- name: {{ org_id }}-console
url: {{ org_id }}-console.{{ full_domain }}
upstream: http://localhost:8080
template: basic-proxy
ssl: true
- name: {{ org_id }}-daemon
url: {{ org_id }}-daemon.{{ full_domain }}
upstream: http://localhost:9473
configs:
- rewrite ^/deployer(/.*)? https://webapp-deployer.pwa.{{full_domain}} permanent
template: websocket-proxy
ssl: true

View File

@ -0,0 +1,12 @@
apiVersion: v1
kind: Namespace
metadata:
name: cert-manager
---
apiVersion: v1
data:
access-token: {{ b64_encoded_token }}
kind: Secret
metadata:
name: digitalocean-dns
namespace: cert-manager

View File

@ -0,0 +1,16 @@
stack: container-registry
deploy-to: k8s
kube-config: /home/{{ ansible_user }}/.kube/config-default.yaml
network:
ports:
registry:
- '5000'
http-proxy:
- host-name: container-registry.pwa.{{full_domain}}
routes:
- path: '/'
proxy-to: registry:5000
volumes:
registry-data:
configmaps:
config: ./configmaps/config

View File

@ -0,0 +1,15 @@
stack:
/home/{{ansible_user}}/cerc/fixturenet-laconicd-stack/stack-orchestrator/stacks/fixturenet-laconicd
deploy-to: compose
network:
ports:
laconicd:
- '6060:6060'
- '26657:26657'
- '26656:26656'
- '9473:9473'
- '9090:9090'
- '1317:1317'
volumes:
laconicd-data: ./data/laconicd-data
genesis-config: ./data/genesis-config

View File

@ -0,0 +1,9 @@
stack:
/home/{{ansible_user}}/cerc/testnet-laconicd-stack/stack-orchestrator/stacks/laconic-console
deploy-to: compose
network:
ports:
console:
- '8080:80'
volumes:
laconic-registry-data: ./data/laconic-registry-data

View File

@ -0,0 +1,35 @@
stack: webapp-deployer-backend
deploy-to: k8s
kube-config: {{ansible_env.HOME}}/.kube/config-default.yaml
image-registry: container-registry.pwa.{{full_domain}}/laconic-registry
network:
ports:
server:
- '9555'
http-proxy:
- host-name: webapp-deployer-api.pwa.{{ full_domain }}
routes:
- path: '/'
proxy-to: server:9555
volumes:
srv:
configmaps:
config: ./data/config
annotations:
container.apparmor.security.beta.kubernetes.io/{name}: unconfined
labels:
container.kubeaudit.io/{name}.allow-disabled-apparmor: "podman"
security:
privileged: true
resources:
containers:
reservations:
cpus: "{{ cpu_reservation }}"
memory: "{{ memory_reservation }}"
limits:
cpus: 6
memory: 16G
volumes:
reservations:
storage: 200G

View File

@ -0,0 +1,2 @@
---
support_email: {{ support_email }}

View File

@ -0,0 +1,15 @@
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: pwa.{{ full_domain }}
namespace: default
spec:
secretName: pwa.{{ full_domain }}
issuerRef:
name: letsencrypt-prod-wild
kind: ClusterIssuer
group: cert-manager.io
commonName: *.pwa.{{ full_domain }}
dnsNames:
- pwa.{{ full_domain }}
- *.pwa.{{ full_domain }}

View File

@ -0,0 +1,3 @@
container_registry_username: ""
container_registry_password: ""
container_registry_domain: "container-registry.pwa.{{ full_domain }}"

View File

@ -0,0 +1,5 @@
full_domain: ""
subdomain_prefix: ""
subdomain_cluster_control: "{{ subdomain_prefix }}-cluster-control"
service_provider_ip: ""
do_api_token: ""

View File

@ -0,0 +1,3 @@
gpg_user_name: ""
gpg_user_email: ""
gpg_passphrase: ""

View File

@ -0,0 +1,8 @@
target_host: "deployment_host"
gpg_key_id: "{{ sec_key_id }}"
vault_passphrase: "{{ gpg_passphrase }}"
org_id: ""
location_id: ""
base_domain: ""
support_email: ""
ansible_ssh_extra_args: '-o StrictHostKeyChecking=no'

View File

@ -0,0 +1,3 @@
username: ""
password: ""
path_to_ssh_key: ""

View File

@ -0,0 +1,6 @@
ALICE_PK: "{{ ALICE_PK }}"
BOND_ID: "{{ BOND_ID }}"
authority_name: ""
cpu_reservation: ""
memory_reservation: ""
deployer_gpg_passphrase: ""

View File

@ -81,9 +81,11 @@ To run the playbook on a remote host:
```bash ```bash
# For bash users # For bash users
echo 'export PATH="$HOME/bin:$PATH"' >> ~/.bashrc echo 'export PATH="$HOME/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc
# For zsh users # For zsh users
echo 'export PATH="$HOME/bin:$PATH"' >> ~/.zshrc echo 'export PATH="$HOME/bin:$PATH"' >> ~/.zshrc
source ~/.zshrc
``` ```
- Once the PATH is set, verify the installation by running the following commands: - Once the PATH is set, verify the installation by running the following commands: