Commit Graph

11896 Commits

Author SHA1 Message Date
Felix Lange
dd28ba378a node: require LocalAppData variable (#19132)
* node: require LocalAppData variable

This avoids path inconsistencies on Windows XP.
Hat tip to @MicahZoltu for catching this so quickly.

* node: fix typo
2019-02-28 13:11:52 +02:00
lash
62d9d63858 swarm/network: WIP consider all nodes for healthy iteration (#19155)
* swarm/network: WIP consider all nodes for healthy iteration

* swarm/network/simulation: extend TestWaitTillHealthy to really check kads are healthy

* cmd/swarm/swarm-snapshot: fixed bugs in snapshot creation binary

* swarm/network/simulation: addressed PR comments

* swarm/network/simulation: defer sim.Clsoe()

* swarm/network/simulation: fixed wrong sim.Close()

* swarm/network/simulation: addressed PR comments

* cmd/swarm/swarm-snapshot: reducing default to 8 nodes, more to 4

* cmd/swarm/swarm-snapshot: extended timeout to 3 mins, or 256 nodes snapshot times out

* swarm/network/simulation: More PR comments
2019-02-28 08:12:50 +01:00
Péter Szilágyi
505a49e689
Merge pull request #19166 from SamuelMarks/go-1.12
Upgrade to Go 1.12
2019-02-27 15:09:59 +02:00
Péter Szilágyi
2b6158a51e
common/fdlimit: fix macos file descriptors for Go 1.12 2019-02-27 14:21:02 +02:00
Samuel Marks
e43bc36226
travis, appveyor, Dockerfile: upgrade to Go 1.12 2019-02-27 14:21:02 +02:00
gary rong
7ebd2fa5db vendor: update leveldb upstream which include a compaction fix (#19163) 2019-02-27 14:10:10 +02:00
Janoš Guljaš
f0233948d2 swarm/chunk: move chunk related declarations to chunk package (#19170) 2019-02-26 16:09:32 +01:00
Péter Szilágyi
b7e0dec6bd
travis, build: switch to NDK 19b, fix gomobile builds (#19171)
* travis, build: switch to NDK 19b, fix gomobile builds

* travis, build: move NDK into its final bundle location

* travis: disable Android build on PRs once again
2019-02-26 16:56:12 +02:00
Péter Szilágyi
6d652ce1b7
Merge pull request #19164 from karalabe/nuke-old-containers
containers/docker: nuke per the 1.8.0 deprecation note
2019-02-26 13:51:09 +02:00
Felföldi Zsolt
c2003ed63b les, les/flowcontrol: improved request serving and flow control (#18230)
This change

- implements concurrent LES request serving even for a single peer.
- replaces the request cost estimation method with a cost table based on
  benchmarks which gives much more consistent results. Until now the
  allowed number of light peers was just a guess which probably contributed
  a lot to the fluctuating quality of available service. Everything related
  to request cost is implemented in a single object, the 'cost tracker'. It
  uses a fixed cost table with a global 'correction factor'. Benchmark code
  is included and can be run at any time to adapt costs to low-level
  implementation changes.
- reimplements flowcontrol.ClientManager in a cleaner and more efficient
  way, with added capabilities: There is now control over bandwidth, which
  allows using the flow control parameters for client prioritization.
  Target utilization over 100 percent is now supported to model concurrent
  request processing. Total serving bandwidth is reduced during block
  processing to prevent database contention.
- implements an RPC API for the LES servers allowing server operators to
  assign priority bandwidth to certain clients and change prioritized
  status even while the client is connected. The new API is meant for
  cases where server operators charge for LES using an off-protocol mechanism.
- adds a unit test for the new client manager.
- adds an end-to-end test using the network simulator that tests bandwidth
  control functions through the new API.
2019-02-26 12:32:48 +01:00
Roc Yu
c2b33a117f graphql: fix typos in comments (#19041) 2019-02-26 12:19:43 +01:00
Péter Szilágyi
a42afa766c
Merge pull request #19169 from karalabe/ppa-bump
build: bump PPA builders to Go 1.11
2019-02-26 12:30:55 +02:00
Péter Szilágyi
647692fbfe
build: bump PPA builders to Go 1.11 2019-02-26 11:30:56 +02:00
Janoš Guljaš
c83ba9e794 swarm/storage/localstore: fix tests for windows os (#19161) 2019-02-26 10:02:05 +01:00
Janoš Guljaš
340a53a98b swarm/pss: fix data race on HandshakeController.symKeyIndex (#19162)
* swarm/pss: fix data race on HandshakeController.symKeyIndex

The HandshakeController.symKeyIndex map was accessed concurrently.
Since insufficient test coverage the race is not detected every time.
However, running TestClientHandshake a 100 times seems to be enough to
reproduce the race.

Note: I've chosen HandshakeController.lock to protect
HandshakeController.symKeyIndex as that was already protected in a few
functions by that lock.

Additionally:
- removed unused testStore
- enabled tests in handshake_test.go as they pass
- removed code duplication by adding getSymKey()

* swarm/pss: fix a data race on HandshakeController.keyC

* swarm/pss: fix data races with on Pss.symKeyPool
2019-02-26 08:17:20 +01:00
Matthew Halpern
badaf43019 les: remove redundant type specifiers (#19091) 2019-02-25 12:49:49 +02:00
Péter Szilágyi
f5f742d1ac
containers/docker: nuke per the 1.8.0 deprecation note 2019-02-25 12:40:33 +02:00
Marius van der Wijden
7d881e45bd rlp: added pooling of streams using sync (#19044)
Prevents reallocation, improves performance
2019-02-25 12:34:08 +02:00
Janoš Guljaš
872370e3bc swarm/network/simulation: do not copy node mutex in UploadSnapshot (#19160) 2019-02-25 10:03:31 +01:00
Matthew Halpern
81babe1509 swarm/*: remove redundant type specifiers (#19089) 2019-02-25 08:58:18 +01:00
Matthew Halpern
4e87b444ca swarm/pss: remove unused function (#19100) 2019-02-24 12:41:22 +01:00
Matthew Halpern
90b6cdaadf cmd,swarm: enforce camel case variable names (#19060) 2019-02-24 12:39:23 +01:00
Janoš Guljaš
64d10c0872 swarm: mock store listings (#19157)
* swarm/storage/mock: implement listings methods for mem and rpc stores

* swarm/storage/mock/rpc: add comments and newTestStore helper function

* swarm/storage/mock/mem: add missing comments

* swarm/storage/mock: add comments to new types and constants

* swarm/storage/mock/db: implement listings for mock/db global store

* swarm/storage/mock/test: add comments for MockStoreListings

* swarm/storage/mock/explorer: initial implementation

* cmd/swarm/global-store: add chunk explorer

* cmd/swarm/global-store: add chunk explorer tests

* swarm/storage/mock/explorer: add tests

* swarm/storage/mock/explorer: add swagger api definition

* swarm/storage/mock/explorer: not-zero test values for invalid addr and key

* swarm/storage/mock/explorer: test wildcard cors origin

* swarm/storage/mock/db: renames based on Fabio's suggestions

* swarm/storage/mock/explorer: add more comments to testHandler function

* cmd/swarm/global-store: terminate subprocess with Kill in tests
2019-02-23 10:47:33 +01:00
Janoš Guljaš
02c28046a0 swarm: Fix localstore test deadlock with race detector (#19153)
* swarm/storage/localstore: close localstore in two tests

* swarm/storage/localstore: fix a possible deadlock in tests

* swarm/storage/localstore: re-enable pull subs tests for travis race

* swarm/storage/localstore: stop sending to errChan on context done in tests

* swarm/storage/localstore: better want check in readPullSubscriptionBin

* swarm/storage/localstore: protect chunk put with addr lock in tests

* swamr/storage/localstore: wait for gc and writeGCSize workers on Close

* swarm/storage/localstore: more correct testDB_collectGarbageWorker

* swarm/storage/localstore: set DB Close timeout to 5s
2019-02-22 23:19:09 +01:00
Ferenc Szabo
d9adcd3a27 travis.yml: add race detector job for Swarm (#19148) 2019-02-22 14:20:21 +01:00
Nguyen Kien Trung
1993227311 .travis.yml: make build independent of repo name (#17607) 2019-02-22 12:07:04 +01:00
Janoš Guljaš
c8da76e63d swarm/shed: fix a deadlock in meter function (#19149) 2019-02-21 20:42:53 +01:00
Janoš Guljaš
836c846812 swarm/network/master: protect SetNextBatch iterator after close (#19147) 2019-02-21 18:33:49 +01:00
Péter Szilágyi
b9808e392f
swarm/version: bump to v0.3.12 unstable 2019-02-21 15:25:42 +02:00
gary rong
7fd0ccaa68 core: remove unnecessary fields in logs, receipts and tx lookups (#17106)
* core: remove unnecessary fields in log

* core: bump blockchain database version

* core, les: remove unnecessary fields in txlookup

* eth: print db version explicitly

* core/rawdb: drop txlookup entry struct wrapper
2019-02-21 15:14:35 +02:00
Martin Holst Swende
8577b5b020 core: more tests for sidechain import, fixes #19105 (#19113)
* blockchain: more tests for sidechain import, fixes #19105

* core/blockchain: rework import of pruned canon blocks and canon-prepended sidechains

* core/blockchain: minor clarity change

* core/blockchain: remove unused method
2019-02-21 12:36:49 +02:00
Enrique Fynn
628a0bde3f common/fdlimit: Fix compilation error in freebsd, Raise() returns uint64 (#19141)
cast limit.Cur (int64) to uint64, fix test and compilation error
2019-02-21 11:16:51 +02:00
Matthew Halpern
fbedf62f3d swarm/storage: fix loop bound for database cleanup (#19085)
The current loop continuation condition is always true as a uint8
is always being checked whether it is less than 255 (its maximum
value). Since the loop starts with the value 1, the loop termination
can be guarranteed to exit once the value overflows to 0.
2019-02-21 06:37:32 +01:00
Janoš Guljaš
9d5e10f5bb cmd/swarm/global-store: use kill instead interrupt in tests (#19142) 2019-02-20 22:58:25 +01:00
Ferenc Szabo
e38b227ce6 Ci race detector handle failing tests (#19143)
* swarm/storage: increase mget timeout in common_test.go

 TestDbStoreCorrect_1k sometimes timed out with -race on Travis.

--- FAIL: TestDbStoreCorrect_1k (24.63s)
    common_test.go:194: testStore failed: timed out after 10s

* swarm: remove unused vars from TestSnapshotSyncWithServer

nodeCount and chunkCount is returned from setupSim and those values
we use.

* swarm: move race/norace helpers from stream to testutil

As we will need to use the flag in other packages, too.

* swarm: refactor TestSwarmNetwork case

Extract long running test cases for better visibility.

* swarm/network: skip TestSyncingViaGlobalSync with -race

As panics on Travis.

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x7e351b]

* swarm: run TestSwarmNetwork with fewer nodes with -race

As otherwise we always get test failure with `network_test.go:374:
context deadline exceeded` even with raised `Timeout`.

* swarm/network: run TestDeliveryFromNodes with fewer nodes with -race

Test on Travis times out with 8 or more nodes if -race flag is present.

* swarm/network: smaller node count for discovery tests with -race

TestDiscoveryPersistenceSimulationSimAdapters failed on Travis with
`-race` flag present. The failure was due to extensive memory usage,
coming from the CGO runtime. Using a smaller node count resolves the
issue.

=== RUN   TestDiscoveryPersistenceSimulationSimAdapter
==7227==ERROR: ThreadSanitizer failed to allocate 0x80000 (524288) bytes of clock allocator (error code: 12)
FATAL: ThreadSanitizer CHECK failed: ./gotsan.cc:6976 "((0 && "unable to mmap")) != (0)" (0x0, 0x0)
FAIL    github.com/ethereum/go-ethereum/swarm/network/simulations/discovery     804.826s

* swarm/network: run TestFileRetrieval with fewer nodes with -race

Otherwise we get a failure due to extensive memory usage, as the CGO
runtime cannot allocate more bytes.

=== RUN   TestFileRetrieval
==7366==ERROR: ThreadSanitizer failed to allocate 0x80000 (524288) bytes of clock allocator (error code: 12)
FATAL: ThreadSanitizer CHECK failed: ./gotsan.cc:6976 "((0 && "unable to mmap")) != (0)" (0x0, 0x0)
FAIL	github.com/ethereum/go-ethereum/swarm/network/stream	155.165s

* swarm/network: run TestRetrieval with fewer nodes with -race

Otherwise we get a failure due to extensive memory usage, as the CGO
runtime cannot allocate more bytes ("ThreadSanitizer failed to
allocate").

* swarm/network: skip flaky TestGetSubscriptionsRPC on Travis w/ -race

Test fails a lot with something like:
 streamer_test.go:1332: Real subscriptions and expected amount don't match; real: 0, expected: 20

* swarm/storage: skip TestDB_SubscribePull* tests on Travis w/ -race

Travis just hangs...

ok  	github.com/ethereum/go-ethereum/swarm/storage/feed/lookup	1.307s
keepalive
keepalive
keepalive

or panics after a while.

Without these tests the race detector job is now stable. Let's
invetigate these tests in a separate issue:
https://github.com/ethersphere/go-ethereum/issues/1245
2019-02-20 22:57:42 +01:00
lash
d36e974ba3 swarm/network: Keep span across roundtrip (#19140)
* swarm/newtork: WIP Span request span until delivery and put

* swarm/storage: Introduce new trace across single fetcher lifespan

* swarm/network: Put span ids for sendpriority in context value

* swarm: Add global span store in tracing

* swarm/tracing: Add context key constants

* swarm/tracing: Add comments

* swarm/storage: Remove redundant fix for filestore

* swarm/tracing: Elaborate constants comments

* swarm/network, swarm/storage, swarm:tracing: Minor cleanup
2019-02-20 14:50:37 +01:00
lash
460d206f30 swarm/network: Use actual remote peer ip in underlay (#19137)
* swarm/network: Logline to see handshake addr

* swarm/network: Replace remote ip in handshake uaddr

* swarm/network: Add test for enode uaddr rewrite method

* swarm/network: Remove redundance pointer return from sanitize

* swarm/network: Obeying the linting machine

* swarm/network: Add panic comment

(travis trigger take 1)
2019-02-20 14:46:00 +01:00
Janoš Guljaš
ba2dfa5ce4 swarm/network/stream: fix a goroutine leak in Registry (#19139)
* swarm/network/stream: fix a goroutine leak in Registry

* swarm/network, swamr/network/stream: Kademlia close addr count and depth change chans

* swarm/network/stream: rename close channel to quit

* swarm/network/stream: fix sync between NewRegistry goroutine and Close method
2019-02-20 14:45:25 +01:00
Dan Kinsley
f49f95e2b0 accounts/abi: mutex lock in TransactionByHash and code cleanup (#19133) 2019-02-20 08:08:54 +01:00
Matthew Halpern
d3ccedc767 p2p/simulations: enforce camel case variable names (#19053) 2019-02-19 19:50:59 +02:00
Péter Szilágyi
2e8a5e5659
core/vm: remove unused constants (#19095) 2019-02-19 19:49:45 +02:00
Matthew Halpern
8af6c9e6a2 eth: extract check for tracing transaction in block file (#19107)
Simplifies the transaction presense check to use a function to
determine if the transaction is present in the block provided
to trace, which originally had a redundant parenthesis and used
a `exist` flag to dictate control flow.
2019-02-19 19:49:24 +02:00
Matthew Halpern
514a9472ad trie: prefer nil slices over zero-length slices (#19084) 2019-02-19 14:50:11 +01:00
Felix Lange
57f959af41 p2p/enode: use localItemKey for local sequence number
I added localItemKey for this purpose in #18963, but then
forgot to actually use it. This changes the database layout
yet again and requires bumping the version number.
2019-02-19 13:29:41 +01:00
Felix Lange
cf147c71d5 p2p/discover: remove unused function 2019-02-19 13:29:19 +01:00
Matthew Halpern
f1537b774c p2p/discover: make maximum packet size a constant (#19061) 2019-02-19 12:27:29 +01:00
Matthew Halpern
bf42535d31 core: remove redundant parentheses (#19106) 2019-02-19 13:25:42 +02:00
Jeremy Schlatter
b5e5b3567c crypto: fix build when CGO_ENABLED=0 (#19121)
Package crypto works with or without cgo, which is great. However, to make it
work without cgo required setting the build tag `nocgo`. It's common to disable
cgo by instead just setting the environment variable `CGO_ENABLED=0`. Setting
this environment variable does _not_ implicitly set the build tag `nocgo`. So
projects that try to build the crypto package with `CGO_ENABLED=0` will fail. I
have done this myself several times. Until today, I had just assumed that this
meant that this package requires cgo.

But a small build tag change will make this case work. Instead of using `nocgo`
and `!nocgo`, we can use `!cgo` and `cgo`, respectively. The `cgo` build tag is
automatically set if cgo is enabled, and unset if it is disabled.
2019-02-19 12:18:37 +01:00
HackyMiner
f7f6a46029 eth, node: use APPDATA env to support cygwin/msys correctly (#17786)
This changes default location of the data directory to use the LOCALAPPDATA
environment variable, resolving issues with remote home directories an improving
compatibility with Cygwin.

Fixes #2239 
Fixes #2237 
Fixes #16437
2019-02-19 12:15:15 +01:00
Felix Lange
d2256244c4 rpc: fixup change to not verify websocket origin (#19128)
This corrects the previous change which broke the build and
was pushed by accident.
2019-02-19 12:58:32 +02:00