cerc-io/lighthouse
16
0
Fork 0
Code Issues 1 Pull Requests Packages Projects Releases Wiki Activity
34d22b5920
lighthouse/validator_client/slashing_protection/Makefile

42 lines
916 B
Makefile
Raw Normal View History

Minify slashing protection interchange data (#2380) ## Issue Addressed Closes #2354 ## Proposed Changes Add a `minify` method to `slashing_protection::Interchange` that keeps only the maximum-epoch attestation and maximum-slot block for each validator. Specifically, `minify` constructs "synthetic" attestations (with no `signing_root`) containing the maximum source epoch _and_ the maximum target epoch from the input. This is equivalent to the `minify_synth` algorithm that I've formally verified in this repository: https://github.com/michaelsproul/slashing-proofs ## Additional Info Includes the JSON loading optimisation from #2347
2021-06-21 05:46:36 +00:00
TESTS_TAG := v5.1.0
Implement slashing protection interchange format (#1544) ## Issue Addressed Implements support for importing and exporting the slashing protection DB interchange format described here: https://hackmd.io/@sproul/Bk0Y0qdGD Also closes #1584 ## Proposed Changes * [x] Support for serializing and deserializing the format * [x] Support for importing and exporting Lighthouse's database * [x] CLI commands to invoke import and export * [x] Export to minimal format (required when a minimal format has been previously imported) * [x] Tests for export to minimal (utilising mixed importing and attestation signing?) * [x] Tests for import/export of complete format, and import of minimal format * [x] ~~Prevent attestations with sources less than our max source (Danny's suggestion). Required for the fake attestation that we put in for the minimal format to block attestations from source 0.~~ * [x] Add the concept of a "low watermark" for compatibility with the minimal format Bonus! * [x] A fix to a potentially nasty bug involving validators getting re-registered each time the validator client ran! Thankfully, the ordering of keys meant that the validator IDs used for attestations and blocks remained stable -- otherwise we could have had some slashings on our hands! :scream: * [x] Tests to confirm that this bug is indeed vanquished
2020-10-02 01:42:27 +00:00
GENERATE_DIR := generated-tests
OUTPUT_DIR := interchange-tests
TARBALL := $(OUTPUT_DIR)-$(TESTS_TAG).tar.gz
ARCHIVE_URL := https://github.com/eth2-clients/slashing-protection-interchange-tests/tarball/$(TESTS_TAG)
Enable Compatibility with Windows (#2333) ## Issue Addressed Windows incompatibility. ## Proposed Changes On windows, lighthouse needs to default to STDIN as tty doesn't exist. Also Windows uses ACLs for file permissions. So to mirror chmod 600, we will remove every entry in a file's ACL and add only a single SID that is an alias for the file owner. Beyond that, there were several changes made to different unit tests because windows has slightly different error messages as well as frustrating nuances around killing a process :/ ## Additional Info Tested on my Windows VM and it appears to work, also compiled & tested on Linux with these changes. Permissions look correct on both platforms now. Just waiting for my validator to activate on Prater so I can test running full validator client on windows. Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com> Co-authored-by: Michael Sproul <micsproul@gmail.com>
2021-05-19 23:05:16 +00:00
ifeq ($(OS),Windows_NT)
ifeq (, $(shell where rm))
rmfile = if exist $(1) (del /F /Q $(1))
rmdir = if exist $(1) (rmdir /Q /S $(1))
else
rmfile = rm -f $(1)
rmdir = rm -rf $(1)
endif
else
rmfile = rm -f $(1)
rmdir = rm -rf $(1)
endif
Implement slashing protection interchange format (#1544) ## Issue Addressed Implements support for importing and exporting the slashing protection DB interchange format described here: https://hackmd.io/@sproul/Bk0Y0qdGD Also closes #1584 ## Proposed Changes * [x] Support for serializing and deserializing the format * [x] Support for importing and exporting Lighthouse's database * [x] CLI commands to invoke import and export * [x] Export to minimal format (required when a minimal format has been previously imported) * [x] Tests for export to minimal (utilising mixed importing and attestation signing?) * [x] Tests for import/export of complete format, and import of minimal format * [x] ~~Prevent attestations with sources less than our max source (Danny's suggestion). Required for the fake attestation that we put in for the minimal format to block attestations from source 0.~~ * [x] Add the concept of a "low watermark" for compatibility with the minimal format Bonus! * [x] A fix to a potentially nasty bug involving validators getting re-registered each time the validator client ran! Thankfully, the ordering of keys meant that the validator IDs used for attestations and blocks remained stable -- otherwise we could have had some slashings on our hands! :scream: * [x] Tests to confirm that this bug is indeed vanquished
2020-10-02 01:42:27 +00:00
$(OUTPUT_DIR): $(TARBALL)
Enable Compatibility with Windows (#2333) ## Issue Addressed Windows incompatibility. ## Proposed Changes On windows, lighthouse needs to default to STDIN as tty doesn't exist. Also Windows uses ACLs for file permissions. So to mirror chmod 600, we will remove every entry in a file's ACL and add only a single SID that is an alias for the file owner. Beyond that, there were several changes made to different unit tests because windows has slightly different error messages as well as frustrating nuances around killing a process :/ ## Additional Info Tested on my Windows VM and it appears to work, also compiled & tested on Linux with these changes. Permissions look correct on both platforms now. Just waiting for my validator to activate on Prater so I can test running full validator client on windows. Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com> Co-authored-by: Michael Sproul <micsproul@gmail.com>
2021-05-19 23:05:16 +00:00
$(call rmdir,$@)
Implement slashing protection interchange format (#1544) ## Issue Addressed Implements support for importing and exporting the slashing protection DB interchange format described here: https://hackmd.io/@sproul/Bk0Y0qdGD Also closes #1584 ## Proposed Changes * [x] Support for serializing and deserializing the format * [x] Support for importing and exporting Lighthouse's database * [x] CLI commands to invoke import and export * [x] Export to minimal format (required when a minimal format has been previously imported) * [x] Tests for export to minimal (utilising mixed importing and attestation signing?) * [x] Tests for import/export of complete format, and import of minimal format * [x] ~~Prevent attestations with sources less than our max source (Danny's suggestion). Required for the fake attestation that we put in for the minimal format to block attestations from source 0.~~ * [x] Add the concept of a "low watermark" for compatibility with the minimal format Bonus! * [x] A fix to a potentially nasty bug involving validators getting re-registered each time the validator client ran! Thankfully, the ordering of keys meant that the validator IDs used for attestations and blocks remained stable -- otherwise we could have had some slashings on our hands! :scream: * [x] Tests to confirm that this bug is indeed vanquished
2020-10-02 01:42:27 +00:00
mkdir $@
tar --strip-components=1 -xzf $^ -C $@
$(TARBALL):
Pretty-print EIP-3076 tests (#1977) ## Proposed Changes * Pretty-print the EIP-3076 tests to match https://github.com/eth2-clients/slashing-protection-interchange-tests/pull/4 * Move the `curl` invocation that downloads the tests to the test executor, removing the build script (closes #1982)
2020-12-03 22:07:12 +00:00
curl --fail -L -o $@ $(ARCHIVE_URL)
Implement slashing protection interchange format (#1544) ## Issue Addressed Implements support for importing and exporting the slashing protection DB interchange format described here: https://hackmd.io/@sproul/Bk0Y0qdGD Also closes #1584 ## Proposed Changes * [x] Support for serializing and deserializing the format * [x] Support for importing and exporting Lighthouse's database * [x] CLI commands to invoke import and export * [x] Export to minimal format (required when a minimal format has been previously imported) * [x] Tests for export to minimal (utilising mixed importing and attestation signing?) * [x] Tests for import/export of complete format, and import of minimal format * [x] ~~Prevent attestations with sources less than our max source (Danny's suggestion). Required for the fake attestation that we put in for the minimal format to block attestations from source 0.~~ * [x] Add the concept of a "low watermark" for compatibility with the minimal format Bonus! * [x] A fix to a potentially nasty bug involving validators getting re-registered each time the validator client ran! Thankfully, the ordering of keys meant that the validator IDs used for attestations and blocks remained stable -- otherwise we could have had some slashings on our hands! :scream: * [x] Tests to confirm that this bug is indeed vanquished
2020-10-02 01:42:27 +00:00
clean-test-files:
Enable Compatibility with Windows (#2333) ## Issue Addressed Windows incompatibility. ## Proposed Changes On windows, lighthouse needs to default to STDIN as tty doesn't exist. Also Windows uses ACLs for file permissions. So to mirror chmod 600, we will remove every entry in a file's ACL and add only a single SID that is an alias for the file owner. Beyond that, there were several changes made to different unit tests because windows has slightly different error messages as well as frustrating nuances around killing a process :/ ## Additional Info Tested on my Windows VM and it appears to work, also compiled & tested on Linux with these changes. Permissions look correct on both platforms now. Just waiting for my validator to activate on Prater so I can test running full validator client on windows. Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com> Co-authored-by: Michael Sproul <micsproul@gmail.com>
2021-05-19 23:05:16 +00:00
$(call rmdir,$(OUTPUT_DIR))
Implement slashing protection interchange format (#1544) ## Issue Addressed Implements support for importing and exporting the slashing protection DB interchange format described here: https://hackmd.io/@sproul/Bk0Y0qdGD Also closes #1584 ## Proposed Changes * [x] Support for serializing and deserializing the format * [x] Support for importing and exporting Lighthouse's database * [x] CLI commands to invoke import and export * [x] Export to minimal format (required when a minimal format has been previously imported) * [x] Tests for export to minimal (utilising mixed importing and attestation signing?) * [x] Tests for import/export of complete format, and import of minimal format * [x] ~~Prevent attestations with sources less than our max source (Danny's suggestion). Required for the fake attestation that we put in for the minimal format to block attestations from source 0.~~ * [x] Add the concept of a "low watermark" for compatibility with the minimal format Bonus! * [x] A fix to a potentially nasty bug involving validators getting re-registered each time the validator client ran! Thankfully, the ordering of keys meant that the validator IDs used for attestations and blocks remained stable -- otherwise we could have had some slashings on our hands! :scream: * [x] Tests to confirm that this bug is indeed vanquished
2020-10-02 01:42:27 +00:00
clean-archives:
Enable Compatibility with Windows (#2333) ## Issue Addressed Windows incompatibility. ## Proposed Changes On windows, lighthouse needs to default to STDIN as tty doesn't exist. Also Windows uses ACLs for file permissions. So to mirror chmod 600, we will remove every entry in a file's ACL and add only a single SID that is an alias for the file owner. Beyond that, there were several changes made to different unit tests because windows has slightly different error messages as well as frustrating nuances around killing a process :/ ## Additional Info Tested on my Windows VM and it appears to work, also compiled & tested on Linux with these changes. Permissions look correct on both platforms now. Just waiting for my validator to activate on Prater so I can test running full validator client on windows. Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com> Co-authored-by: Michael Sproul <micsproul@gmail.com>
2021-05-19 23:05:16 +00:00
$(call rmfile,$(TARBALL))
Implement slashing protection interchange format (#1544) ## Issue Addressed Implements support for importing and exporting the slashing protection DB interchange format described here: https://hackmd.io/@sproul/Bk0Y0qdGD Also closes #1584 ## Proposed Changes * [x] Support for serializing and deserializing the format * [x] Support for importing and exporting Lighthouse's database * [x] CLI commands to invoke import and export * [x] Export to minimal format (required when a minimal format has been previously imported) * [x] Tests for export to minimal (utilising mixed importing and attestation signing?) * [x] Tests for import/export of complete format, and import of minimal format * [x] ~~Prevent attestations with sources less than our max source (Danny's suggestion). Required for the fake attestation that we put in for the minimal format to block attestations from source 0.~~ * [x] Add the concept of a "low watermark" for compatibility with the minimal format Bonus! * [x] A fix to a potentially nasty bug involving validators getting re-registered each time the validator client ran! Thankfully, the ordering of keys meant that the validator IDs used for attestations and blocks remained stable -- otherwise we could have had some slashings on our hands! :scream: * [x] Tests to confirm that this bug is indeed vanquished
2020-10-02 01:42:27 +00:00
generate:
Enable Compatibility with Windows (#2333) ## Issue Addressed Windows incompatibility. ## Proposed Changes On windows, lighthouse needs to default to STDIN as tty doesn't exist. Also Windows uses ACLs for file permissions. So to mirror chmod 600, we will remove every entry in a file's ACL and add only a single SID that is an alias for the file owner. Beyond that, there were several changes made to different unit tests because windows has slightly different error messages as well as frustrating nuances around killing a process :/ ## Additional Info Tested on my Windows VM and it appears to work, also compiled & tested on Linux with these changes. Permissions look correct on both platforms now. Just waiting for my validator to activate on Prater so I can test running full validator client on windows. Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com> Co-authored-by: Michael Sproul <micsproul@gmail.com>
2021-05-19 23:05:16 +00:00
$(call rmdir,$(GENERATE_DIR))
Implement slashing protection interchange format (#1544) ## Issue Addressed Implements support for importing and exporting the slashing protection DB interchange format described here: https://hackmd.io/@sproul/Bk0Y0qdGD Also closes #1584 ## Proposed Changes * [x] Support for serializing and deserializing the format * [x] Support for importing and exporting Lighthouse's database * [x] CLI commands to invoke import and export * [x] Export to minimal format (required when a minimal format has been previously imported) * [x] Tests for export to minimal (utilising mixed importing and attestation signing?) * [x] Tests for import/export of complete format, and import of minimal format * [x] ~~Prevent attestations with sources less than our max source (Danny's suggestion). Required for the fake attestation that we put in for the minimal format to block attestations from source 0.~~ * [x] Add the concept of a "low watermark" for compatibility with the minimal format Bonus! * [x] A fix to a potentially nasty bug involving validators getting re-registered each time the validator client ran! Thankfully, the ordering of keys meant that the validator IDs used for attestations and blocks remained stable -- otherwise we could have had some slashings on our hands! :scream: * [x] Tests to confirm that this bug is indeed vanquished
2020-10-02 01:42:27 +00:00
cargo run --release --bin test_generator -- $(GENERATE_DIR)
clean: clean-test-files clean-archives
.PHONY: clean clean-archives clean-test-files generate
Reference in New Issue Copy Permalink