diff --git a/app/data/config/go-nitro/run-nitro-node.sh b/app/data/config/go-nitro/run-nitro-node.sh index 9cc9fccf..edfb1cdf 100755 --- a/app/data/config/go-nitro/run-nitro-node.sh +++ b/app/data/config/go-nitro/run-nitro-node.sh @@ -1,6 +1,5 @@ #!/bin/bash -set -e if [ -n "$CERC_SCRIPT_DEBUG" ]; then set -x fi @@ -30,6 +29,21 @@ fi echo "Running Nitro node" -# TODO Wait for chain endpoint to come up +# Wait till chain endpoint is available +retry_interval=5 +while true; do + # Assuming CERC_NITRO_CHAIN_URL is of format ://host:port + ws_host=$(echo "$CERC_NITRO_CHAIN_URL" | awk -F '://' '{print $2}' | cut -d ':' -f 1) + ws_port=$(echo "$CERC_NITRO_CHAIN_URL" | awk -F '://' '{print $2}' | cut -d ':' -f 2) + nc -z -w 1 "$ws_host" "$ws_port" + + if [ $? -eq 0 ]; then + echo "Chain endpoint is available" + break + fi + + echo "Chain endpoint not yet available, retrying in $retry_interval seconds..." + sleep $retry_interval +done ./nitro -chainurl ${CERC_NITRO_CHAIN_URL} -msgport 3005 -rpcport 4005 -wsmsgport 5005 -pk ${CERC_NITRO_PK} -chainpk ${CERC_NITRO_CHAIN_PK} -naaddress ${NA_ADDRESS} -vpaaddress ${VPA_ADDRESS} -caaddress ${CA_ADDRESS} -usedurablestore ${CERC_NITRO_USE_DURABLE_STORE} -durablestorefolder ${CERC_NITRO_DURABLE_STORE_FOLDER} diff --git a/app/data/stacks/fixturenet-payments/README.md b/app/data/stacks/fixturenet-payments/README.md index 5f208e98..cb2e9422 100644 --- a/app/data/stacks/fixturenet-payments/README.md +++ b/app/data/stacks/fixturenet-payments/README.md @@ -1,5 +1,7 @@ # fixturenet-payments +Instructions to setup and deploy an end-to-end fixturenet-payments stack + ## Setup Clone required repositories: diff --git a/app/data/stacks/fixturenet-payments/demo.md b/app/data/stacks/fixturenet-payments/demo.md index 0b5e658d..579225da 100644 --- a/app/data/stacks/fixturenet-payments/demo.md +++ b/app/data/stacks/fixturenet-payments/demo.md @@ -1,14 +1,25 @@ # Demo -## MobyMask Watcher +Stack components: +* `ipld-eth-db` database for statediffed data +* Local geth + lighthouse blockchain "fixturenet" running in statediffing mode +* `ipld-eth-server` which runs an ETH RPC API and a GQL server; serves data from `ipld-eth-db` +* A go-nitro deployment acting as the Nitro node for `ipld-eth-server` +* A modified reverse payment proxy server (based on the one from go-nitro) that proxies requests to `ipld-eth-server`'s RPC endpoint; it talks to `ipld-eth-server`'s Nitro node to accept and validate payments required for configured RPC requests +* A MobyMask v3 watcher that pays the `ipld-eth-server` for ETH RPC requests +* A MobyMask v3 app that pays the watcher for reads (GQL queries) and writes +* An example ERC20 Ponder app that pays the `ipld-eth-server` for ETH RPC requests -* Check watcher logs and wait for the payment channel to be created with upstream go-nitro node: +## Setup + +* On starting the stack, MobyMask watcher creates a payment channel with the `ipld-eth-server`'s Nitro node. Check watcher logs and wait for the same: ```bash docker logs -f $(docker ps -aq --filter name="mobymask-watcher-server") # Expected output: - # vulcanize:server Using rpcProviderEndpoint as chain URL for Nitro node +0ms + # vulcanize:server Peer ID: 12D3KooWKLqLWU82VU7jmsmQMruRvZWhoBoVsf1UHchM5Nuq9ymY + # vulcanize:server Using chain URL http://fixturenet-eth-geth-1:8546 for Nitro node # ... # ts-nitro:util:nitro Ledger channel created with id 0x65703ccdfacab09ac35367bdbe6c5a337e7a6651aad526807607b1c59b28bc1e # ... @@ -16,7 +27,9 @@ # vulcanize:server Starting server... +0ms ``` -* Export the payment channel id to a variable: +* Keep the above command running to keep track of incoming payments and GQL requests from the MobyMask app + +* In another terminal, export the payment channel id to a variable: ```bash export WATCHER_UPSTREAM_PAYMENT_CHANNEL= @@ -41,43 +54,46 @@ # } ``` -## MobyMask App - -* Track the reverse payment proxy container logs in a terminal: +* In another terminal, check the reverse payment proxy server's logs to keep track of incoming payments and RPC requests: ```bash docker logs -f $(docker ps -aq --filter name="nitro-reverse-payment-proxy") ``` -* Get the geth node’s port mapped to host: +* MetaMask flask wallet setup for running the MobyMask app: - ```bash - docker port payments-fixturenet-eth-geth-1-1 8545 - ``` + * Get the geth node’s port mapped to host: -* In MetaMask Flask, add a custom network with the following settings: + ```bash + docker port payments-fixturenet-eth-geth-1-1 8545 + ``` - ```bash - # New RPC URL - http://127.0.0.1: + * In MetaMask, add a custom network with the following settings: - # Chain ID - 1212 + ```bash + # Network name + Local fixturenet - # Currency symbol - ETH - ``` + # New RPC URL + http://127.0.0.1: -* Import the faucet account in MetaMask and fund an additional account for usage in the app: + # Chain ID + 1212 - ```bash - # Faucet PK - # 0x570b909da9669b2f35a0b1ac70b8358516d55ae1b5b3710e95e9a94395090597 + # Currency symbol + ETH + ``` - # Clear activity tab for the accounts on chain restart - ``` + * Import a faucet account with the following private key: -* Get the generated root invite link for the app from MobyMask contract deployment container logs: + ```bash + # Faucet PK + # 0x570b909da9669b2f35a0b1ac70b8358516d55ae1b5b3710e95e9a94395090597 + ``` + + * Create an additional account for usage in the app; fund it from the faucet account + +* Get the generated root invite link for MobyMask from contract deployment container logs: ```bash docker logs -f $(docker ps -aq --filter name="mobymask-1") @@ -89,9 +105,13 @@ # http://127.0.0.1:3004/#/members?invitation= ``` -* Open app in a browser using the invite link +## Run -* Run the following in the browser console to enable logs: +### MobyMask App + +* Open app in a browser (where MetaMask was setup) using the invite link + +* Run the following in browser console to enable logs: ```bash localStorage.debug = 'ts-nitro:*' @@ -104,19 +124,66 @@ * Click on `Connect Wallet` to connect to MetaMask (make sure that the newly funded account is active) * Click on `Connect Snap` to install/connect snap -* Perform `DIRECT FUND` with the preset amount +* Perform `DIRECT FUND` with the preset amount and wait for the MetaMask confirmation prompt to appear; confirm the transaction and wait for a ledger channel to be created with the watcher -* Perform `VIRTUAL FUND` with amount set to `10000` +* Perform `VIRTUAL FUND` with amount set to `10000` and wait for a payment channel to be created with the watcher * Perform phisher status check queries now that a payment channel is created: - * Check the watcher logs for received payments - * Check the payment proxy server logs for charged RPC requests (`eth_getBlockByHash`, `eth_getBlockByNumber`, `eth_getStorageAt`) made from watcher to upstream ETH server -* Change the amount besides `PAY` button to `>=100` for phisher reports next + * Check the watcher logs for received payments along with the GQL queries: -* Perform a phisher report and check the watcher logs for received payments; the RPC mutation request is sent to geth node and is not charged + ```bash + # Expected output: + # ... + # laconic:payments Serving a paid query for 0x86804299822212c070178B5135Ba6DdAcFC357D3 + # vulcanize:resolver isPhisher 0x98ae4f9e9d01cc892adfe6871e1db0287039e0c183d3b5bb31d724228c114744 0x2B6AFbd4F479cE4101Df722cF4E05F941523EaD9 TWT:ash1 + # vulcanize:indexer isPhisher: db miss, fetching from upstream server + # laconic:payments Making RPC call: eth_chainId + # laconic:payments Making RPC call: eth_getBlockByHash + # laconic:payments Making RPC call: eth_chainId + # laconic:payments Making RPC call: eth_getStorageAt + ``` -* Check the watcher - eth-server payment channel status after a few requests: + * The watcher makes several ETH RPC requests to `ipld-eth-server` to fetch data required for satisfying the GQL request(s); check the payment proxy server logs for charged RPC requests (`eth_getBlockByHash`, `eth_getBlockByNumber`, `eth_getStorageAt`): + + ```bash + # Expected output: + # ... + # {"time":"2023-10-06T06:46:52.769009314Z","level":"DEBUG","msg":"Serving RPC request","method":"eth_chainId"} + # {"time":"2023-10-06T06:46:52.773006426Z","level":"DEBUG","msg":"Serving RPC request","method":"eth_getBlockByNumber"} + # {"time":"2023-10-06T06:46:52.811142054Z","level":"DEBUG","msg":"Request cost","cost-per-byte":1,"response-length":1480,"cost":1480,"method":"eth_getBlockByNumber"} + # {"time":"2023-10-06T06:46:52.811418494Z","level":"DEBUG","msg":"sent message","address":"0xAAA6628Ec44A8a742987EF3A114dDFE2D4F7aDCE","method":"receive_voucher"} + # {"time":"2023-10-06T06:46:52.812557482Z","level":"DEBUG","msg":"Received voucher","delta":5000} + # ... + # {"time":"2023-10-06T06:46:52.87525215Z","level":"DEBUG","msg":"Serving RPC request","method":"eth_getStorageAt"} + # {"time":"2023-10-06T06:46:52.882859654Z","level":"DEBUG","msg":"Request cost","cost-per-byte":1,"response-length":104,"cost":104,"method":"eth_getStorageAt"} + # {"time":"2023-10-06T06:46:52.882946485Z","level":"DEBUG","msg":"sent message","address":"0xAAA6628Ec44A8a742987EF3A114dDFE2D4F7aDCE","method":"receive_voucher"} + # {"time":"2023-10-06T06:46:52.884012641Z","level":"DEBUG","msg":"Received voucher","delta":5000} + # {"time":"2023-10-06T06:46:52.884032961Z","level":"DEBUG","msg":"Destination request","url":"http://ipld-eth-server:8081/"} + ``` + +* Change the amount besides `PAY` button in debug panel to `>=100` for phisher reports next + +* Perform a phisher report and check the watcher logs for received payments: + + ```bash + # Expected output: + # ... + # vulcanize:libp2p-utils [6:50:2] Received a message on mobymask P2P network from peer: 12D3KooWRkxV9SX8uTUZYkbRjai4Fsn7yavB61J5TMnksixsabsP + # ts-nitro:engine {"msg":"Received message","_msg":{"to":"0xBBB676","from":"0x868042","payloadSummaries":[],"proposalSummaries":[],"payments":[{"amount":200,"channelId":"0x557153d729cf3323c0bdb40a36b245f98c2d4562933ba2182c9d61c5cfeda948"}],"rejectedObjectives":[]}} + # laconic:payments Received a payment voucher of 100 from 0x86804299822212c070178B5135Ba6DdAcFC357D3 + # vulcanize:libp2p-utils Payment received for a mutation request from 0x86804299822212c070178B5135Ba6DdAcFC357D3 + # vulcanize:libp2p-utils Transaction receipt for invoke message { + # to: '0x2B6AFbd4F479cE4101Df722cF4E05F941523EaD9', + # blockNumber: 232, + # blockHash: '0x6a188722c102662ea48af3786fe9db0d4b6c7ab7b27473eb0e628cf95746a244', + # transactionHash: '0x6521205db8a905b3222adc2b6855f9b2abc72580624d299bec2a35bcba173efa', + # effectiveGasPrice: '1500000007', + # gasUsed: '113355' + # } + ``` + +* Check the watcher - ipld-eth-server payment channel status after a few requests: ```bash docker exec payments-nitro-rpc-client-1 npm exec -c "nitro-rpc-client get-payment-channel $WATCHER_UPSTREAM_PAYMENT_CHANNEL -h go-nitro -p 4005" @@ -135,7 +202,7 @@ # } ``` -## Ponder App +### ERC20 Ponder App * Run the ponder app in it's container: @@ -150,13 +217,33 @@ # 09:59:14.329 INFO payment Using payment channel 0x10f049519bc3f862e2b26e974be8666886228f30ea54aab06e2f23718afffab0 ``` +* On starting the Ponder app, it creates a payment channel with the `ipld-eth-server`'s Nitro node and then starts the historical sync service + +* The sync service makes several ETH RPC requests to the `ipld-eth-server` to fetch required data; check the payment proxy server logs for charged RPC requests (`eth_getBlockByNumber`, `eth_getLogs`) + + ```bash + # Expected output: + # ... + # {"time":"2023-10-06T06:51:45.214478402Z","level":"DEBUG","msg":"Serving RPC request","method":"eth_getBlockByNumber"} + # {"time":"2023-10-06T06:51:45.22251171Z","level":"DEBUG","msg":"Request cost","cost-per-byte":1,"response-length":576,"cost":576,"method":"eth_getBlockByNumber"} + # {"time":"2023-10-06T06:51:45.222641963Z","level":"DEBUG","msg":"sent message","address":"0xAAA6628Ec44A8a742987EF3A114dDFE2D4F7aDCE","method":"receive_voucher"} + # {"time":"2023-10-06T06:51:45.224042391Z","level":"DEBUG","msg":"Received voucher","delta":5000} + # {"time":"2023-10-06T06:51:45.224061411Z","level":"DEBUG","msg":"Destination request","url":"http://ipld-eth-server:8081/"} + # {"time":"2023-10-06T06:51:45.242064953Z","level":"DEBUG","msg":"Serving RPC request","method":"eth_getLogs"} + # {"time":"2023-10-06T06:51:45.249118517Z","level":"DEBUG","msg":"Request cost","cost-per-byte":1,"response-length":61,"cost":61,"method":"eth_getLogs"} + # {"time":"2023-10-06T06:51:45.249189892Z","level":"DEBUG","msg":"sent message","address":"0xAAA6628Ec44A8a742987EF3A114dDFE2D4F7aDCE","method":"receive_voucher"} + # {"time":"2023-10-06T06:51:45.249743149Z","level":"DEBUG","msg":"Received voucher","delta":5000} + # {"time":"2023-10-06T06:51:45.249760631Z","level":"DEBUG","msg":"Destination request","url":"http://ipld-eth-server:8081/"} + # ... + ``` + * Export the payment channel id to a variable: ```bash export PONDER_UPSTREAM_PAYMENT_CHANNEL= ``` -* Check the ponder - eth-server payment channel status: +* Check the ponder - ipld-eth-server payment channel status: ```bash docker exec payments-nitro-rpc-client-1 npm exec -c "nitro-rpc-client get-payment-channel $PONDER_UPSTREAM_PAYMENT_CHANNEL -h go-nitro -p 4005" @@ -175,23 +262,14 @@ # } ``` -* Check reverse payment proxy server logs for charged RPC requests made from ponder app to upstream ETH server: - - ```bash - # Expected output: - # ... - # {"time":"2023-09-28T09:59:14.499841999Z","level":"DEBUG","msg":"Request cost","cost-per-byte":1,"response-length":61,"cost":61} - # {"time":"2023-09-28T09:59:14.500060006Z","level":"DEBUG","msg":"sent message","address":"0xAAA6628Ec44A8a742987EF3A114dDFE2D4F7aDCE","method":"receive_voucher"} - # {"time":"2023-09-28T09:59:14.501221898Z","level":"DEBUG","msg":"Received voucher","delta":5000} - # {"time":"2023-09-28T09:59:14.501245984Z","level":"DEBUG","msg":"Destination request","url":"http://ipld-eth-server:8081/?method=eth_getLogs"} - ``` - ## Clean Up -* In the MobyMask app, perform `VIRTUAL DEFUND` and `DIRECT DEFUND` (in this order) for closing the payment channel created with watcher +* In the MobyMask app, perform `VIRTUAL DEFUND` and `DIRECT DEFUND` (in order) for closing the payment channel created with watcher * Run the following in the browser console to delete the Nitro node's data: ```bash await clearNodeStorage() ``` + +* On a fresh restart, clear activity tab data in MetaMask for concerned accounts