Commit Graph

11306 Commits

Author SHA1 Message Date
Ian Norden
139725986e review fixes and fixes for issues ran into in integration 2019-11-06 16:25:45 -06:00
Ian Norden
1d8b8fe824 cli parameter to limit statediffing to select account addresses + test 2019-11-06 16:25:45 -06:00
Ian Norden
cc3d6a9c35 make state diff rlp serializable 2019-11-06 16:25:45 -06:00
Ian Norden
4d908eab6c option to process intermediate nodes 2019-11-06 16:25:45 -06:00
Ian Norden
04a6efb1a1 make proofs and paths optional + compress service loop into single for loop (may be missing something here) 2019-11-06 16:25:45 -06:00
Ian Norden
f4160487f6 refactoring state diff service and adding api which allows for streaming state diff payloads over an rpc websocket subscription 2019-11-06 16:25:45 -06:00
Ian Norden
c8eb1d4d6c refactoring statediff builder and types and adjusted to relay proofs and paths (still need to make this optional) 2019-11-06 16:25:45 -06:00
Ian Norden
5a01aceb98 refactoring/reorganizing packages 2019-11-06 16:25:45 -06:00
Elizabeth
a99042bcf8 Statediff for full node (#6)
* Open a trie from the in-memory database

* Use a node's LeafKey as an identifier instead of the address

It was proving difficult to find look the address up from a given path
with a full node (sometimes the value wouldn't exist in the disk db).
So, instead, for now we are using the node's LeafKey with is a Keccak256
hash of the address, so if we know the address we can figure out which
LeafKey it matches up to.

* Make sure that statediff has been processed before pruning

* Use blockchain stateCache.OpenTrie for storage diffs

* Clean up log lines and remove unnecessary fields from builder

* Apply go fmt changes

* Add a sleep to the blockchain test

* Address PR comments

* Address PR comments
2019-11-06 16:25:45 -06:00
Elizabeth
d29733e1f7 Gracefully exit geth command(#4) 2019-11-06 16:25:44 -06:00
Elizabeth Engelman
5a451cc085 Apply gosimple changes to statediff 2019-11-06 16:25:44 -06:00
Elizabeth Engelman
88197c40b2 Apply goimports to statediff 2019-11-06 16:25:44 -06:00
Elizabeth Engelman
d4d60bdc80 Apply go fmt changes to statediff 2019-11-06 16:25:44 -06:00
Elizabeth
c8d1af4b6a Write state diff to CSV (#2)
* port statediff from 9b7fd9af80/statediff/statediff.go; minor fixes

* integrating state diff extracting, building, and persisting into geth processes

* work towards persisting created statediffs in ipfs; based off github.com/vulcanize/eth-block-extractor

* Add a state diff service

* Remove diff extractor from blockchain

* Update imports

* Move statediff on/off check to geth cmd config

* Update starting state diff service

* Add debugging logs for creating diff

* Add statediff extractor and builder tests and small refactoring

* Start to write statediff to a CSV

* Restructure statediff directory

* Pull CSV publishing methods into their own file

* Reformatting due to go fmt

* Add gomega to vendor dir

* Remove testing focuses

* Update statediff tests to use golang test pkg

instead of ginkgo

- builder_test
- extractor_test
- publisher_test

* Use hexutil.Encode instead of deprecated common.ToHex

* Remove OldValue from DiffBigInt and DiffUint64 fields

* Update builder test

* Remove old storage value from updated accounts

* Remove old values from created/deleted accounts

* Update publisher to account for only storing current account values

* Update service loop and fetching previous block

* Update testing

- remove statediff ginkgo test suite file
- move mocks to their own dir

* Updates per go fmt

* Updates to tests

* Pass statediff mode and path in through cli

* Return filename from publisher

* Remove some duplication in builder

* Remove code field from state diff output

this is the contract byte code, and it can still be obtained by querying
the db by the codeHash

* Consolidate acct diff structs for updated & updated/deleted accts

* Include block number in csv filename

* Clean up error logging

* Cleanup formatting, spelling, etc

* Address PR comments

* Add contract address and storage value to csv

* Refactor accumulating account row in csv publisher

* Add DiffStorage struct

* Add storage key to csv

* Address PR comments

* Fix publisher to include rows for accounts that don't have store updates

* Update builder test after merging in release/1.8

* Update test contract to include storage on contract intialization

- so that we're able to test that storage diffing works for created and
deleted accounts (not just updated accounts).

* Factor out a common trie iterator method in builder
2019-11-06 16:25:44 -06:00
gary rong
b9bac1f384 les: fix and slim the unit tests of les (#20247)
* les: loose restriction of unit tests

* les: update unit tests

* les, light: slim the unit tests
2019-11-06 22:09:37 +01:00
Péter Szilágyi
fc3661f89c
Merge pull request #20248 from karalabe/cht-1.9.7
params: hard-code new CHTs for the 1.9.7 release
2019-11-06 17:54:39 +02:00
Péter Szilágyi
9948724deb
params: hard-code new CHTs for the 1.9.7 release 2019-11-06 17:47:13 +02:00
Péter Szilágyi
c702bd70ed
travis: bump linter to Go 1.13.x 2019-11-05 15:35:51 +02:00
Péter Szilágyi
734e00af9e
travis, build, internal: use own Go bundle for PPA builds (#20240)
* build: bump PPAs to Go 1.13 (via longsleep), keep Trusty on 1.11

* travis, build, vendor: use own Go bundle for PPA builds

* travis, build, internal, vendor: smarter Go bundler, own untar

* build: updated ci-notes with new Go bundling, only make, don't test
2019-11-05 15:32:42 +02:00
Martin Holst Swende
b566cfdffd core/evm: avoid copying memory for input in calls (#20177)
* core/evm, contracts: avoid copying memory for input in calls + make ecrecover not modify input buffer

* core/vm: optimize mstore a bit

* core/vm: change Get -> GetCopy in vm memory access
2019-11-04 11:31:09 +02:00
gary rong
7a6d5d0cce cmd/puppeth: integrate istanbul into puppeth (#19926)
* cmd/puppeth: integrate istanbul into puppeth

* cmd/puppeth: address comment

* cmd/puppeth: use hexutil.Big for fork indicator

* cmd/puppeth: finalize istanbul fork

* cmd/puppeth: fix 2200 for parity, rename is to eip1283ReenableTransition

* cmd/puppeth: fix eip1108

* cmd/puppeth: add blake2f for parity

* cmd/puppeth: add aleth istanbul precompiled

* cmd/puppeth: use hexutil.Big

* cmd/puppeth: fix unit tests

* cmd/puppeth: update testdata
2019-11-04 10:41:29 +02:00
Péter Szilágyi
0ff7380465
Merge pull request #20231 from SamuelMarks/go1.13.4
appveyor: bump to Go 1.13.4
2019-11-04 10:38:10 +02:00
gary rong
0ce5e113be les: rework clientpool (#20077)
* les: rework clientpool
2019-11-02 13:02:35 +01:00
Samuel Marks
86fe283d19
appveyor: bump to Go 1.13.4 2019-11-02 18:54:04 +11:00
gary rong
44b74cfc40 accounts/abi: add internalType information and fix issues (#20179)
* accounts/abi: fix various issues

The fixed issues include:

(1) If there is no return in a call function, unpack should
return nil error

(2) For some functions which have struct array as parameter,
it will also be detected and generate the struct definition

(3) For event, if it has non-indexed parameter, the parameter
name will also be assigned if empty. Also the internal struct
will be detected and generate struct defition if not exist.

(4) Fix annotation generation in event function

* accounts/abi: add new abi field internalType

* accounts: address comments and add tests

* accounts/abi: replace strings.ReplaceAll with strings.Replace
2019-10-31 14:17:51 +01:00
Martin Holst Swende
9278951a62 params, core/forkid: configure mainnet istanbul block 9069K (#20222)
* params: configure mainnet istanbul block 9069K

* core/forkid: add some more test items for mainnet istanbul
2019-10-31 11:04:26 +02:00
Péter Szilágyi
12f2a25d5e
Merge pull request #20225 from karalabe/forkid-eth-handshake-verification-plus
cmd/devp2p, core/forkid: make forkid.Filter API uniform
2019-10-31 10:58:28 +02:00
Péter Szilágyi
8927f7724a
cmd/devp2p, core/forkid: make forkid.Filter API uniform 2019-10-31 10:38:14 +02:00
Péter Szilágyi
93422e9d15
Merge pull request #20140 from karalabe/eth64-handshake-forkid
eth: eth/64 - extend handshake with with fork id
2019-10-30 13:21:25 +02:00
gary rong
5d91acccd5 miner: increase import time allowance (#20217)
Fix the block import unit test which can time out sometimes.
2019-10-30 12:07:30 +01:00
Péter Szilágyi
9641cacea8 core/forkid: add two clauses for more precise validation (#20220) 2019-10-30 12:05:31 +01:00
Péter Szilágyi
64571f9379
eth: eth/64 - extend handshake packet with fork id 2019-10-29 18:04:39 +02:00
Péter Szilágyi
e306304414
Merge pull request #20204 from holiman/fix_downloader_race
eth/downloader: fix data race in downloader
2019-10-29 17:10:44 +02:00
Felix Lange
2c37142d2f cmd/devp2p, p2p: dial using node iterator, discovery crawler (#20132)
* p2p/enode: add Iterator and associated utilities

* p2p/discover: add RandomNodes iterator

* p2p: dial using iterator

* cmd/devp2p: add discv4 crawler

* cmd/devp2p: WIP nodeset filter

* cmd/devp2p: fixup lesFilter

* core/forkid: add NewStaticFilter

* cmd/devp2p: make -eth-network filter actually work

* cmd/devp2p: improve crawl timestamp handling

* cmd/devp2p: fix typo

* p2p/enode: fix comment typos

* p2p/discover: fix comment typos

* p2p/discover: rename lookup.next to 'advance'

* p2p: lower discovery mixer timeout

* p2p/enode: implement dynamic FairMix timeouts

* cmd/devp2p: add ropsten support in -eth-network filter

* cmd/devp2p: tweak crawler log message
2019-10-29 17:08:57 +02:00
Martin Holst Swende
3eca7b5d27
eth/downloader: fix data race in downloader 2019-10-29 14:32:45 +01:00
Michael Forney
b0b277525c core/asm: assembly parser label fixes (#20210)
* core/asm: Fix encoding of pushed labels

EVM uses big-endian byte-order, so to pad a label value to 4 bytes,
zeros must be added to the front, not the end.

* core/asm: Fix PC calculations when a label is pushed

Incrementing PC by 5 is only correct if the label appears after a jump,
in which case there is an implicit push. When it appears after an explicit
push, PC should only be incremented by 4.

* core/asm: Allow JUMP with no argument

This way, a label can be pushed explicitly, or loaded from memory to
implement a jump table.
2019-10-29 13:47:18 +01:00
gary rong
ecdbb402ee trie: remove node ordering slice in sync batch (#19929)
When we flush a batch of trie nodes into database during the state
sync, we should guarantee that all children should be flushed before
parent.

Actually the trie nodes commit order is strict by: children -> parent.
But when we flush all ready nodes into db, we don't need the order
anymore since

    (1) they are all ready nodes (no more dependency)
    (2) underlying database provides write atomicity
2019-10-28 18:50:11 +01:00
Michael Forney
9c81387bef cmd/evm: remove surrounding whitespace in hex input code (#20211)
This way, the output of `evm compile` can be used directly in `evm
--codefile code.txt run`, without stripping the trailing newline first.
2019-10-28 14:55:20 +01:00
Guillaume Ballet
72617a0742 consensus: fix possessives in comments. (#20209) 2019-10-28 09:57:34 +02:00
Martin Holst Swende
db79143a13
clef: resolve windows pipes, fixes #20121 (#20166) 2019-10-24 10:45:07 +02:00
Piotr Dyraga
538f763fdc accounts/abi/bind: take into account gas price during gas estimation (#20189)
The gas price was not passed to the `EstimateGas` function. As a result,
conditional execution paths depending on `tx.gasprice` could be not
correctly processed and we could get invalid gas estimates for contract
function calls.
2019-10-21 10:13:41 +02:00
gary rong
d4bb3798d8 miner: add generate and import unit test (#20111)
This PR adds a new unit test in miner package which will create some blocks from miner and then import into another chain. In this way, we can ensure all blocks generated by Geth miner obey consensus rules.
2019-10-20 12:36:40 +02:00
Marius Kjærstad
08953e42c1 metrics: change links in README.md to https (#20182) 2019-10-20 12:25:25 +02:00
Marius Kjærstad
b9299bbc46 dashboard: change links in README to https (#20181)
Changed http:// to https:// on links in dashboard/README.md
2019-10-18 21:30:53 +02:00
Marius Kjærstad
9a77065948 Changed http:// to https:// on links in log/README.md (#20178)
docs: change http to https on links in log/README.md
2019-10-18 08:51:54 +02:00
Jeffery Robert Walsh
a28093ced4 README: use new miner threads flag instead of legacy minerthreads flag (#20165) 2019-10-17 11:39:13 +03:00
Ross
d5b79e752e p2p/simulations: add node properties support and utility functions (#20060) 2019-10-17 10:07:09 +02:00
Felix Lange
7300365956 p2p/dnsdisc: update to latest EIP-1459 spec (#20168)
This updates the DNS TXT record format to the latest
changes in ethereum/EIPs#2313.
2019-10-16 14:35:24 +03:00
Martin Holst Swende
c476460cb2
params: check fork ordering when initializing new genesis, fixes #20136 (#20169)
prevent users from misconfiguring their nodes so that fork ordering is not preserved.
2019-10-16 13:23:14 +02:00
gary rong
028af3457d cmd/utils: fix command line flag resolve (#20167)
In Geth, we have two sources for configuration:
(1) Config file
(2) Command line flag

Basically geth will first resolve config file and then overwrite
configs with command line flags.

This issue is: geth should only overwrite configs if flags are truly
set. So before we apply any flag to configs, `GlobalIsSet` check
is necessary.
2019-10-15 10:19:20 +02:00