## Issue Addressed
Our v1.1.0 automated release failed to upload to Github. The `hub` command failed with a `403`, which seems like this issue: https://github.com/github/hub/issues/2149
## Proposed Changes
The suggested fix in that issue is to set the `$GITHUB_USER` environment variable. I can't really test this because this hasn't been failing on my fork, but seems low risk
Co-authored-by: realbigsean <seananderson33@gmail.com>
Co-authored-by: Paul Hauner <paul@paulhauner.com>
## Issue Addressed
NA
## Proposed Changes
Rust 1.50 has landed 🎉
The shiny new `clippy` peers down upon us mere mortals with disgust. Brutish peasants wrapping our `usize`s in superfluous `Option`s... tsk tsk.
I've performed the goat sacrifice and corrected our evil ways in this PR. Tonight we shall pray that Github Actions bestows the almighty green tick upon us.
## Additional Info
NA
Co-authored-by: realbigsean <seananderson33@gmail.com>
Co-authored-by: Michael Sproul <michael@sigmaprime.io>
Attempt to prevent accidental merges to `stable` due to GitHub's default behaviour of opening PRs against it.
I've intentionally opened this PR against `stable` to test the functionality ;)
## Issue Addressed
N/A
## Proposed Changes
On any tag formatted `v*`, a full multi-arch docker build will be kicked off and automatically pushed to docker hub with the version tag.
This is a bit repetitive, because the image built will usually be the same as the image built on pushes to `stable`, but it seems like the simplest way to go about it and this will also work if we incorporate a workflow with `vX.X.X-rc` tags.
## Additional Info
This may also need to wait for env variable updates: https://github.com/sigp/lighthouse/pull/2135#issuecomment-754977433
Co-authored-by: realbigsean <seananderson33@gmail.com>
## Issue Addressed
Resolves #2120
## Proposed Changes
This updates github actions to use `cross` when compiling linux x86_64 binaries.
## Additional Info
I think we could alternatively be explicit with the version of macOS or ubuntu we are running actions on and that could solve #2120. I'm not sure which method is preferred here though. Github actions supports Ubuntu 16.04
Co-authored-by: realbigsean <seananderson33@gmail.com>
## Issue Addressed
Resolves#1674
## Proposed Changes
- Whenever a tag is pushed with the prefix `v` this workflow is triggered
- creates portable and non-portable binaries for linux x86_64, linux aarch64, macOS
- an attempt at using github actions caching
- signs each binary using GPG
- auto-generates full changelog based on commit messages since the last release
- creates a **draft** release
- hot new formatting (preview [here](https://github.com/realbigsean/lighthouse/releases/tag/v0.9.23))
- has been taking around 35 minutes
## Additional Info
TODOs:
- Figure out how we should automate dockerhub's version tag.
- It'd be quickest just to tag `latest`, but we'd need to make sure the docker workflow completes before this starts
- we do the same cross-compile in the `docker` workflow, we could try to use the same binary
- integrate a similar flow for unstable binaries (`-rc` tag?)
- improve caching, potentially use sccache
- if we start using a self-hosted runner this'll require some re-working
Need to add the following secrets to Github:
- `GPG_PASSPHRASE`
- ~~`GPG_PUBLIC_KEY`~~ hard-coded this, because it was tough manage as a secret
- `GPG_SIGNING_KEY`
Co-authored-by: realbigsean <seananderson33@gmail.com>
## Issue Addressed
N/A
## Proposed Changes
I didn't realize the `PORTABLE` env variable is only picked up by `install` in the `Makefile` so we are still getting `SIGILL`s:
https://github.com/sigp/lighthouse/runs/1565004525?check_suite_focus=true
## Additional Info
Co-authored-by: realbigsean <seananderson33@gmail.com>
## Issue Addressed
NA
## Proposed Changes
Fixes problems with slot times below 1 second which got revealed by running the syncing simulator with the default speedup time.
## Issue Addressed
N/A
## Proposed Changes
Add some caching to the test suite and to the aarch64 cross-compile in the docker build.
## Additional Info
Cache hits only occur if the Cargo.lock file is unchanged, Github Actions runner OS matches, and the cache is "in scope". Some documentation on github actions cache scoping is here:
https://docs.github.com/en/free-pro-team@latest/actions/guides/caching-dependencies-to-speed-up-workflows#matching-a-cache-key
I'm not sure how frequently we'll get cache hits, I imagine only on smaller PR's or updates to the same PR. And there is a cache size limit that we may end up reaching quickly. But Github actions handles evictions if we go over that limit.
Not sure how much of an impact this will end up having but I don't really see a downside to trying it out.
Co-authored-by: realbigsean <seananderson33@gmail.com>
## Issue Addressed
N/A
## Proposed Changes
- hardcode `ubuntu-18.04` -- I don't think this was causing us issues, but github actions is in the process of migrating `ubuntu-latest` from Ubuntu 18 -> 20.. so just in case
- different source of emulation dependencies -> https://github.com/tonistiigi/binfmt
- this one is explicitly referenced in the `buildx` github docs
- install emulation dependencies and run `docker buildx` in the same `run` command
- enable `buildx` with `DOCKER_CLI_EXPERIMENTAL: enabled` rather than re-building it
## Additional Info
N/A
Co-authored-by: realbigsean <seananderson33@gmail.com>
## Issue Addressed
Resolves#1512
## Proposed Changes
- Adds a new docker Github Actions workflow
- Removes the Dockerhub hook
- Adds a new Dockerfile for use with pre-existing cross-compiled binaries
- on pushes to `unstable`
- builds an ARM64 image and tags it `latest-arm64-unstable`
- builds an AMD64 image and tags it `latest-amd64-unstable`
- builds an multiarch image by creating a manifest list referencing the prior two images and tags it `latest-unstable`
- on pushes to `stable`
- builds an ARM64 image and tags it `latest-arm64`
- builds an AMD64 image and tags it `latest-amd64`
- builds an multiarch image by creating a manifest list referencing the prior two images and tags it `latest`
## Additional Info
- for ARM64, first `cross` is used to cross compile the `lighthouse` and `lcli` binaries, then `docker buildx` is installed to actually build the docker image for the correct target platform. The image build pretty much just copies the binaries from local into the docker image (thanks @michaelsproul :) )
- The AMD64 and ARM64 builds run in parallel, in total it's been taking around 45mins on a local runner
- This PR does **not** cover version tags on docker images at the moment
Co-authored-by: realbigsean <seananderson33@gmail.com>
## Issue Addressed
NA
## Proposed Changes
Presently `master` is stable (and will be sunsetted) which means our docs only update after a release. This PR sets the docs to build on the `unstable` branch, which is equivalent to what what we've always had.
## Additional Info
This does raise the question of whether or not docs should target `stable` or `unstable`, but I'd prefer to maintain current functionality and merge #1966 for now. I think having two versions might be handy, one for stable and one for unstable; I don't imagine this very difficult to achieve.
## Issue Addressed
N/A
## Proposed Changes
Remove the MacOs tests. They routinely fail, causing bors to retry and slowing down the whole merge process.
## Additional Info
N/A
Co-authored-by: Michael Sproul <michael@sigmaprime.io>
## Issue Addressed
Resolves#1100
## Proposed Changes
* Implement the `SafeArith` trait for `Slot` and `Epoch`, so that methods like `safe_add` become available.
* Tweak the `SafeArith` trait to allow a different `Rhs` type (analagous to `std::ops::Add`, etc).
* Add a `legacy-arith` feature to `types` and `state_processing` that conditionally enables implementations of
the `std` ops with saturating semantics.
* Check compilation of `types` and `state_processing` _without_ `legacy-arith` on CI,
thus guaranteeing that they only use the `SafeArith` primitives 🎉
## Additional Info
The `legacy-arith` feature gets turned on by all higher-level crates that depend on `state_processing` or `types`, thus allowing the beacon chain, networking, and other components to continue to rely on the availability of ops like `+`, `-`, `*`, etc.
**This is a consensus-breaking change**, but brings us in line with the spec, and our incompatibilities shouldn't have been reachable with any valid configuration of Eth2 parameters.
This reverts commit 2627463366.
## Issue Addressed
This is a temporary fix for #1589, by reverting #1574. The Docker image needs to be built with `--build-arg PORTABLE=true`, and we could probably integrate that into the multi-arch build, but in the interests of expediting a fix, this PR opts for a revert.
## Issue Addressed
#1512
## Proposed Changes
Use Github Actions to automate the Docker image build, so that we can make a multi-arch image.
## Additional Info
This change will require adding the DOCKER_USERNAME and DOCKER_PASSWORD secrets in Github. It will also require disabling the Docker Hub automated build.
## Issue Addressed
NA
## Proposed Changes
- Refactor the `bls` crate to support multiple BLS "backends" (e.g., milagro, blst, etc).
- Removes some duplicate, unused code in `common/rest_types/src/validator.rs`.
- Removes the old "upgrade legacy keypairs" functionality (these were unencrypted keys that haven't been supported for a few testnets, no one should be using them anymore).
## Additional Info
Most of the files changed are just inconsequential changes to function names.
## TODO
- [x] Optimization levels
- [x] Infinity point: https://github.com/supranational/blst/issues/11
- [x] Ensure milagro *and* blst are tested via CI
- [x] What to do with unsafe code?
- [x] Test infinity point in signature sets
## Proposed Changes
CI is failing on PRs because of a regression in nightly Rust. This change forces `rustup` to install a version of the nightly compiler that can at least build `rustfmt`, which should reduce the frequency of this happening.
## Additional Info
Example failing run: https://github.com/sigp/lighthouse/runs/896694659
Alternatively, we could allow CI to succeed even when `cargo-udeps` fails, but I think we should avoid doing that unless we really have to.
## Issue Addressed
Prevent CI from succeeding when there are warnings. Code can still be built and tested with warnings locally, but CI may fail during a Rust update (which is fine IMO).
## Proposed Changes
* Deny warnings for all stable compiler jobs on CI (excludes `cargo udeps`, which runs under nightly)
* Fix the warnings currently on `master` related to unnecessary `mem::replace`
* Add mac build to CI
* Always return an error for Health when not linux
* Change macos workflow
* Rename macos tests
* Disable health API test on Mac
Co-authored-by: Michael Sproul <michael@sigmaprime.io>
* Fix parallelism bug in exit processing
Also:
* Remove parallelism for all other operations except deposit merkle proofs
* Improve exit tests
* Fix broken attestation test
Closes#1090
* Allow for generating block/pre/post states from some unit tests (#1123)
* Add post-state checks, comments
* Add state_transition_vectors crate
* Integrate new testing crate with CI
* Add readme
* Add additional valid tests
* Remove ExitTests (they were moved to new crate)
* Small test fixes
* Delete incorrect saturating_sub in slash_validator
And clean-up the balance increase/decrease functions to look more like the spec.
Co-authored-by: Paul Hauner <paul@paulhauner.com>
* Add progress on new deposits
* Add deposited command to account manager
* Remove old lcli::helpers mod
* Clean clap_utils
* Refactor lcli deposit contract commands to use IPC
* Make testnet optional for environment
* Use dbg formatting for deploy address
* Add command to generate bootnode enr
* Ensure lcli returns with 1 on error
* Ensure account manager returns 1 on error
* Disallow deposits to the zero address
* Update web3 in eth1 crate
* Ensure correct lighthouse dir is created
* Reduce deposit gas requirement
* Update cargo.lock
* Add progress on new deposits
* Add deposited command to account manager
* Remove old lcli::helpers mod
* Clean clap_utils
* Refactor lcli deposit contract commands to use IPC
* Add command to generate bootnode enr
* Ensure lcli returns with 1 on error
* Ensure account manager returns 1 on error
* Update web3 in eth1 crate
* Update Cargo.lock
* Move lcli out of main install script
* Change --limit to --at-least
* Change --datadir to --validator-dir
* Remove duplication in docs
* Create libp2p instance
* Change logger to stdlog
* test_connection initial commit
* Add gossipsub test
* Delete tests in network crate
* Add test module
* Clean tests
* Remove dependency on discovery
* Working publish between 2 nodes
TODO: Publish should be called just once
* Working 2 peer gossipsub test with additional events
* Cleanup test
* Add rpc test
* Star topology discovery WIP
* build_nodes builds and connects n nodes. Increase nodes in gossipsub test
* Add unsubscribe method and expose reference to gossipsub object for gossipsub tests
* Add gossipsub message forwarding test
* Fix gossipsub forward test
* Test improvements
* Remove discovery tests
* Simplify gossipsub forward test topology
* Add helper functions for topology building
* Clean up tests
* Update naming to new network spec
* Correct ssz encoding of protocol names
* Further additions to network upgrade
* Initial network spec update WIP
* Temp commit
* Builds one side of the streamed RPC responses
* Temporary commit
* Propagates streaming changes up into message handler
* Intermediate network update
* Partial update in upgrading to the new network spec
* Update dependencies, remove redundant deps
* Correct sync manager for block stream handling
* Re-write of RPC handler, improves efficiency and corrects bugs
* Stream termination update
* Completed refactor of rpc handler
* Remove crates
* Correct compile issues associated with test merge
* Build basic tests and testing structure for eth2-libp2p
* Enhance RPC tests and add logging
* Complete RPC testing framework and STATUS test
* Decoding bug fixes, log improvements, stream test
* Clean up RPC handler logging
* Decoder bug fix, empty block stream test
* Add BlocksByRoot RPC test
* Add Goodbye RPC test
* Syncing and stream handling bug fixes and performance improvements
* Applies discv5 bug fixes
* Adds DHT IP filtering for lighthouse - currently disabled
* Adds randomized network propagation as a CLI arg
* Add disconnect functionality
* Adds attestation handling and parent lookup
* Adds RPC error handling for the sync manager
* Allow parent's blocks to be already processed
* Update github workflow
* Adds reviewer suggestions
