cerc-io/ipld-eth-beacon-indexer
19
0
Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases 9 Wiki Activity
5beff1e5c6
ipld-eth-beacon-indexer/Makefile

133 lines
2.9 KiB
Makefile
Raw Normal View History

Utilize Ginkgo toreplace `testing` library, and add CI/CD. (#9) * Utilize Ginkgo and replace `testing` library. * Add TOC to docs * Add Docker specific files * Remove -e * Update on-pr-manual.yml * Add depth * Add repositories * Remove $ from path * Update path for make * Setup Go and Ginkgo * Use go mod download * Use go install * Update on-pr-manual.yml * Use latest * Remove install of GINKGO * Add explicit gopath * Explicitly specify the gopath * Update on-pr-manual.yml * Update on-pr-manual.yml * Update on-pr-manual.yml * Update on-pr-manual.yml * Update on-pr-manual.yml * Update on-pr-manual.yml * Use which ginkgo * Try with make now * Final working Make
2022-04-22 12:28:01 +00:00
BIN = $(GOPATH)/bin
BASE = $(GOPATH)/src/$(PACKAGE)
PKGS = go list ./... | grep -v "^vendor/"
# Tools
## Testing library
GINKGO = $(BIN)/ginkgo
$(BIN)/ginkgo:
go install github.com/onsi/ginkgo/ginkgo
.PHONY: installtools
installtools: | $(GINKGO)
echo "Installing tools"
.PHONY: test
test:
go vet ./...
go fmt ./...
$(GINKGO) -r
Feature/11 testing add tests for boot (#12) * Utilize Versioning * Update Testing and CI/CD * Update Testing * Add Linting and Remove timeout * Update Lint * handle errors to make the linter happy
2022-04-22 16:27:54 +00:00
.PHONY: integration-test-ci
integration-test-ci:
go vet ./...
go fmt ./...
$(GINKGO) -r --label-filter integration \
--procs=4 --compilers=4 \
--randomize-all --randomize-suites \
--fail-on-pending --keep-going \
--cover --coverprofile=cover.profile \
--race --trace --json-report=report.json
Capture the head block in the DB entirely. (#27) * -- Intermediary Commit -- Just want to commit my code over the weekend, in case I spill coffee on my workstation. * Create DB models ready for write. * Handle SSE events * Update ref for stack-orchestrator * Use env in one place only. * Boot Application on PR * Update syntax * Update syntax * Correct command * Use bash instead of sh * Use until instead of while * Make linter happy and check sse subscription err * Handle Reorgs - Untested
2022-05-06 15:03:15 +00:00
.PHONY: integration-test-ci-no-race
integration-test-ci-no-race:
go vet ./...
go fmt ./...
$(GINKGO) -r --label-filter integration \
--procs=4 --compilers=4 \
--randomize-all --randomize-suites \
--fail-on-pending --keep-going \
--cover --coverprofile=cover.profile \
--trace --json-report=report.json
Process SSE messages, and include a graceful shutdown
2022-04-27 14:28:42 +00:00
.PHONY: integration-test-local
integration-test-local:
go vet ./...
go fmt ./...
$(GINKGO) -r --label-filter integration \
--procs=4 --compilers=4 \
--randomize-all --randomize-suites \
--fail-on-pending --keep-going \
Capture the head block in the DB entirely. (#27) * -- Intermediary Commit -- Just want to commit my code over the weekend, in case I spill coffee on my workstation. * Create DB models ready for write. * Handle SSE events * Update ref for stack-orchestrator * Use env in one place only. * Boot Application on PR * Update syntax * Update syntax * Correct command * Use bash instead of sh * Use until instead of while * Make linter happy and check sse subscription err * Handle Reorgs - Untested
2022-05-06 15:03:15 +00:00
--trace --race
.PHONY: integration-test-local-no-race
integration-test-local-no-race:
go vet ./...
go fmt ./...
$(GINKGO) -r --label-filter integration \
--procs=4 --compilers=4 \
--randomize-all --randomize-suites \
--fail-on-pending --keep-going \
--trace
Process SSE messages, and include a graceful shutdown
2022-04-27 14:28:42 +00:00
.PHONY: unit-test-local
unit-test-local:
go vet ./...
go fmt ./...
76: Add indexing of ExecutionPayloads (and other Merge-related updates). (#73) 1. Updates or replaces outdated dependencies (eg, replacing a version of the Prysm client with the latest zrnt). 2. Add support for parsing Bellatrix-era BeaconState and BeaconBlocks 3. Adds flags for toggling the processing of BeaconBlocks and BeaconState. This is particularly important because processing and storing the BeaconState at this time would be too expensive to really do (see: Temporarily disable BeaconState indexing #75 and [Feature] Reduce the Amount of DB Space the Beacon Chain Needs #71) 4. Fixes flaky event handling. The previous code would not reconnect in the case of errors with the SSE connection. This enables automatic reconnection in the case of error (default in the updated v2 SSE library dependency), and also adds a timeout so that if no event is received in 2.5x the block time, the SSE connection is closed and re-established. 5. Other refactoring and cleanup (eg, changing the type of slot from int to Slot (uint64)).
2022-09-29 01:39:56 +00:00
$(GINKGO) -r --label-filter 'unit && !flaky' \
--randomize-all --randomize-suites \
--flake-attempts=3 \
--fail-on-pending --keep-going \
--trace
.PHONY: unit-test-local-bellatrix
unit-test-local-bellatrix:
go vet ./...
go fmt ./...
$(GINKGO) -r --label-filter 'unit && !flaky && bellatrix' \
Process SSE messages, and include a graceful shutdown
2022-04-27 14:28:42 +00:00
--randomize-all --randomize-suites \
Slight Performance improvements with Logging (#67) * Slight Performance improvements with Logging * Use exec in docker container * Update packages for dockerfile * Add flakey test and boot command specific logic * Remove excess logging and add config val
2022-06-15 15:49:30 +00:00
--flake-attempts=3 \
Process SSE messages, and include a graceful shutdown
2022-04-27 14:28:42 +00:00
--fail-on-pending --keep-going \
Feature/22 test handling incoming events (#30) * Checkpoint before the weekend * Update location for SetupPostgresDB * Include first functioning tests for processing head * Fix gitignore * Test CaptureHead | Add Metrics | Handle Test Race Conditions This Commit allows us to: * Test the `CaptureHead` function. * Test parsing a single Head message. * Test a Reorg condition. * Add Metrics. This is primarily used for testing but can have future use cases. * Rearrange the test due to race conditions introduced by reusing a variable. `BeforeEach` can't be used to update `BC`. * Update and finalize testing at this stage * Update code and CI/CD * Fix lint errors * Update CICD and fail when file not found. * Update test to have failed as expected.
2022-05-12 13:52:13 +00:00
--trace
Process SSE messages, and include a graceful shutdown
2022-04-27 14:28:42 +00:00
Feature/11 testing add tests for boot (#12) * Utilize Versioning * Update Testing and CI/CD * Update Testing * Add Linting and Remove timeout * Update Lint * handle errors to make the linter happy
2022-04-22 16:27:54 +00:00
.PHONY: unit-test-ci
unit-test-ci:
go vet ./...
go fmt ./...
$(GINKGO) -r --label-filter unit \
Slight Performance improvements with Logging (#67) * Slight Performance improvements with Logging * Use exec in docker container * Update packages for dockerfile * Add flakey test and boot command specific logic * Remove excess logging and add config val
2022-06-15 15:49:30 +00:00
--randomize-all --randomize-suites
--flake-attempts=3 \
Feature/11 testing add tests for boot (#12) * Utilize Versioning * Update Testing and CI/CD * Update Testing * Add Linting and Remove timeout * Update Lint * handle errors to make the linter happy
2022-04-22 16:27:54 +00:00
--fail-on-pending --keep-going \
--cover --coverprofile=cover.profile \
Feature/22 test handling incoming events (#30) * Checkpoint before the weekend * Update location for SetupPostgresDB * Include first functioning tests for processing head * Fix gitignore * Test CaptureHead | Add Metrics | Handle Test Race Conditions This Commit allows us to: * Test the `CaptureHead` function. * Test parsing a single Head message. * Test a Reorg condition. * Add Metrics. This is primarily used for testing but can have future use cases. * Rearrange the test due to race conditions introduced by reusing a variable. `BeforeEach` can't be used to update `BC`. * Update and finalize testing at this stage * Update code and CI/CD * Fix lint errors * Update CICD and fail when file not found. * Update test to have failed as expected.
2022-05-12 13:52:13 +00:00
--trace --json-report=report.json
Feature/11 testing add tests for boot (#12) * Utilize Versioning * Update Testing and CI/CD * Update Testing * Add Linting and Remove timeout * Update Lint * handle errors to make the linter happy
2022-04-22 16:27:54 +00:00
Testing for Batch Processing (#56) * Set starting slot and improve error gap capturing * Set starting slot and improve error gap capturing * Tests + Significant Refactor The code for historical processing has been significantly refactored to use a context to signify a shutdown. There have also been many tests added for historical and knownGaps processing. * Update MhKeys in test * Update correct values * Update Max Retry Genesis is not working as expected. * Ensure we release locks properly * Add ordered testing * Include system tests * Update workflow calls * Add secrets * Add required secrets * update path * Try using the absolute path * Remove volumes at the end. * Update system-tests.yml * Update system-tests.yml * Update test err * Update and test the shutdown * rename ethcl --> eth-beacon * Try forcing /bin/bash for docker-compose * Update system-tests.yml * Update system-tests.yml * Update system-tests.yml * Update system-tests.yml * Update system-tests.yml * Update system-tests.yml * Use single quote cron * Dont run generic on schedule
2022-06-09 21:32:46 +00:00
.PHONY: system-test-ci
system-test-ci:
go vet ./...
go fmt ./...
$(GINKGO) -r --label-filter system \
--randomize-all --randomize-suites \
--fail-on-pending --keep-going \
--cover --coverprofile=cover.profile \
--trace --json-report=report.json
.PHONY: system-test-local
system-test-local:
go vet ./...
go fmt ./...
$(GINKGO) -r --label-filter system \
--randomize-all --randomize-suites \
--fail-on-pending --keep-going \
--trace
Utilize Ginkgo toreplace `testing` library, and add CI/CD. (#9) * Utilize Ginkgo and replace `testing` library. * Add TOC to docs * Add Docker specific files * Remove -e * Update on-pr-manual.yml * Add depth * Add repositories * Remove $ from path * Update path for make * Setup Go and Ginkgo * Use go mod download * Use go install * Update on-pr-manual.yml * Use latest * Remove install of GINKGO * Add explicit gopath * Explicitly specify the gopath * Update on-pr-manual.yml * Update on-pr-manual.yml * Update on-pr-manual.yml * Update on-pr-manual.yml * Update on-pr-manual.yml * Update on-pr-manual.yml * Use which ginkgo * Try with make now * Final working Make
2022-04-22 12:28:01 +00:00
Add first e2e test for post-Merge indexing
2022-10-07 20:22:04 +00:00
.PHONY: e2e-test-local
e2e-test-local:
go vet ./...
go fmt ./...
$(GINKGO) -r --label-filter e2e \
--randomize-all --randomize-suites \
--fail-on-pending --keep-going \
--trace
Utilize Ginkgo toreplace `testing` library, and add CI/CD. (#9) * Utilize Ginkgo and replace `testing` library. * Add TOC to docs * Add Docker specific files * Remove -e * Update on-pr-manual.yml * Add depth * Add repositories * Remove $ from path * Update path for make * Setup Go and Ginkgo * Use go mod download * Use go install * Update on-pr-manual.yml * Use latest * Remove install of GINKGO * Add explicit gopath * Explicitly specify the gopath * Update on-pr-manual.yml * Update on-pr-manual.yml * Update on-pr-manual.yml * Update on-pr-manual.yml * Update on-pr-manual.yml * Update on-pr-manual.yml * Use which ginkgo * Try with make now * Final working Make
2022-04-22 12:28:01 +00:00
.PHONY: build
build:
go fmt ./...
GO111MODULE=on go build
## Build docker image
.PHONY: docker-build
docker-build:
76: Add indexing of ExecutionPayloads (and other Merge-related updates). (#73) 1. Updates or replaces outdated dependencies (eg, replacing a version of the Prysm client with the latest zrnt). 2. Add support for parsing Bellatrix-era BeaconState and BeaconBlocks 3. Adds flags for toggling the processing of BeaconBlocks and BeaconState. This is particularly important because processing and storing the BeaconState at this time would be too expensive to really do (see: Temporarily disable BeaconState indexing #75 and [Feature] Reduce the Amount of DB Space the Beacon Chain Needs #71) 4. Fixes flaky event handling. The previous code would not reconnect in the case of errors with the SSE connection. This enables automatic reconnection in the case of error (default in the updated v2 SSE library dependency), and also adds a timeout so that if no event is received in 2.5x the block time, the SSE connection is closed and re-established. 5. Other refactoring and cleanup (eg, changing the type of slot from int to Slot (uint64)).
2022-09-29 01:39:56 +00:00
docker build -t vulcanize/ipld-eth-beacon-indexer .
Reference in New Issue Copy Permalink