Commit Graph

234 Commits

Author SHA1 Message Date
holisticode
695a5cce1e Increase bzz version (#18184)
* swarm/network/stream/: added stream protocol version match tests

* Increase BZZ version due to streamer version change; version tests

* swarm/network: increased hive and test protocol version
2018-11-26 19:34:40 +01:00
Janoš Guljaš
c207edf2a3 swarm: add database abstractions (shed package) (#18183) 2018-11-26 18:49:01 +01:00
lash
1cd007ecae swarm/network: Correct neighborhood depth (#18066) 2018-11-26 17:13:59 +01:00
holisticode
bba5fd8192 Accounting metrics reporter (#18136) 2018-11-26 17:05:18 +01:00
Javier Peletier
2714e8f091 Remove multihash from Swarm bzz:// for Feeds (#18175) 2018-11-26 16:10:22 +01:00
lash
197d609b9a swarm/pss: Message handler refactor (#18169) 2018-11-26 13:52:04 +01:00
Elad
f5e6634fd2 swarm/api: improve not found error msg (#18171) 2018-11-26 12:53:15 +01:00
Janoš Guljaš
93854bbad4 swarm/network/simulation: fix New function for-loop scope (#18161) 2018-11-26 12:39:38 +01:00
Janoš Guljaš
070caec4bd swarm/network/stream: use swarm/mock/mem as mock global store (#18157) 2018-11-21 20:49:13 +01:00
Anton Evangelatov
4c181e4fb9
swarm/state: refactor InmemoryStore (#18143) 2018-11-21 14:36:56 +01:00
Ferenc Szabo
d876f214e5 swarm/storage: move 'running migrations for' log line (#18120)
So that we only see the log message when we actually have to migrate.
2018-11-20 08:30:38 +01:00
Anton Evangelatov
3333fe660f
swarm/storage: speed up garbage collection and rpc tests (#18128) 2018-11-19 12:26:45 +01:00
Elad
51e2e78d26 swarm/api/http: change request served msg log level (#18127) 2018-11-18 10:54:03 +01:00
holisticode
ffe2fc3bc4 Swarm accounting (#18050)
* swarm: completed 1st phase of swap accounting

* swarm: swap accounting for swarm with p2p accounting

* swarm/swap: addressed PR comments

* swarm/swap: ignore ErrNotFound on stateStore.Get()

* swarm/swap: GetPeerBalance test; add TODO for chequebook API check

* swarm/network/stream: fix NewRegistry calls with new arguments

* swarm/swap: address @justelad's PR comments
2018-11-15 23:41:19 +01:00
Janoš Guljaš
324027640b swarm/network/simulation: use simulations.Event instead p2p.PeerEvent (#18098) 2018-11-15 21:06:27 +01:00
lash
a6942b9f25 swarm/storage: Batched database migration (#18113) 2018-11-15 14:57:03 +01:00
Péter Szilágyi
48b4e8069c
params, swarm: begin Geth v1.8.19 and Swarm v0.3.7 cycle 2018-11-14 10:27:30 +02:00
Péter Szilágyi
58632d4402
params, swarm: release Geth v1.8.18 and Swarm v0.3.6 2018-11-14 10:25:19 +02:00
Alexey Sharov
eb8fa3cc89 cmd/swarm, swarm/api/http, swarm/bmt, swarm/fuse, swarm/network/stream, swarm/storage, swarm/storage/encryption, swarm/testutil: use pseudo-random instead of crypto-random for test files content generation (#18083)
- Replace "crypto/rand" to "math/rand" for files content generation
- Remove swarm/network_test.go.Shuffle and swarm/btm/btm_test.go.Shuffle - because go1.9 support dropped (see https://github.com/ethereum/go-ethereum/pull/17807 and comments to swarm/network_test.go.Shuffle)
2018-11-14 09:21:14 +01:00
Ferenc Szabo
c41e1bd1eb swarm/storage: fix garbage collector index skew (#18080)
On file access LDBStore's tryAccessIdx() function created a faulty
GC Index Data entry, because not indexing the ikey correctly.
That caused the chunk addresses/hashes to start with '00' and the last
two digits were dropped. => Incorrect chunk address.

Besides the fix, the commit also contains a schema change which will
run the CleanGCIndex() function to clean the GC index from erroneous
entries.

Note: CleanGCIndex() rebuilds the index from scratch which can take
a really-really long time with a huge DB (possibly an hour).
2018-11-13 15:22:53 +01:00
Ferenc Szabo
8080265f3f swarm/storage: fix access count on dbstore after cache hit (#17978)
Access count was not incremented when chunk was retrieved
from cache. So the garbage collector might have deleted the most
frequently accessed chunk from disk.

Co-authored-by: Ferenc Szabo <ferenc.szabo@ethereum.org>
2018-11-13 07:41:01 +01:00
lash
201a0bf181 p2p/simulations, swarm/network: Custom services in snapshot (#17991)
* p2p/simulations: Add custom services to simnodes + remove sim down conn objs

* p2p/simulation, swarm/network: Add selective services to discovery sim

* p2p/simulations, swarm/network: Remove useless comments

* p2p/simulations, swarm/network: Clean up mess from rebase

* p2p/simulation: Add sleep to prevent connect flakiness in http test

* p2p/simulations: added concurrent goroutines to prevent sleeps on simulation connect/disconnect

* p2p/simulations, swarm/network/simulations: address pr comments

* reinstated dummy service

* fixed http snapshot test
2018-11-12 14:57:17 +01:00
Anton Evangelatov
cf3b187bde swarm, cmd/swarm: address ineffectual assignments (#18048)
* swarm, cmd/swarm: address ineffectual assignments

* swarm/network: remove unused vars from testHandshake

* swarm/storage/feed: revert cursor changes
2018-11-07 20:39:08 +01:00
Mark Vujevits
81533deae5 swarm/network: light nodes are not dialed, saved and requested from (#17975)
* RequestFromPeers does not use peers marked as lightnode

* fix warning about variable name

* write tests for RequestFromPeers

* lightnodes should be omitted from the addressbook

* resolve pr comments regarding logging, formatting and comments

* resolve pr comments regarding comments and added a missing newline

* add assertions to check peers in live connections
2018-11-07 20:33:36 +01:00
Javier Peletier
36ca85fa1c swarm/api: Fix #18007, missing signature should return HTTP 400 (#18008) 2018-11-07 14:49:42 +01:00
holisticode
79c7a69ac8 swarm: Better syncing and retrieval option definition (#17986)
* swarm: Better syncing and retrieval option definition

* swarm/network/stream: better comments

* swarm/network/stream: addressed PR comments
2018-11-07 00:04:18 +01:00
Anton Evangelatov
53eb4e0b0f
swarm/api: unexport Respond methods (#18037) 2018-11-06 12:34:34 +01:00
KimMachineGun
baee850471 swarm: modify context key (#17925)
* swarm: modify context key

* gofmt sctx.go
2018-11-06 09:54:19 +01:00
Ferenc Szabo
54f650a3be swarm: clean up unused private types and functions (#17989)
* swarm: clean up unused private types and functions

Those that were identified by code inspection tool.

* swarm/storage: move/add Proximity GoDoc from deleted private function

The mentioned proximity() private function was deleted in:
1ca8fc1e6fa0ab4ab1aaca06d6fb32e173cd5f2f
2018-10-27 16:18:42 +02:00
holisticode
88b41a9e68 swarm/network/stream: disambiguate chunk delivery messages (retrieval… (#17920)
* swarm/network/stream: disambiguate chunk delivery messages (retrieval vs syncing)

* swarm/network/stream: addressed PR comments

* swarm/network/stream: stream protocol version change due to new message types in this PR
2018-10-21 09:30:41 +02:00
Elad
66debd91d9 swarm/api/http: remove ModTime=now for direct and multipart uploads (#17945) 2018-10-19 16:02:44 +02:00
Elad
aeb733623e swarm/network: disallow historical retrieval requests (#17936) 2018-10-19 10:50:25 +02:00
Attila Gazso
cdf5982cfc swarm: Lightnode mode: disable sync, retrieve, subscription (#17899)
* swarm: Lightnode mode: disable sync, retrieve, subscription

* swarm/network/stream: assign error and check in one line

* swarm: restructured RegistryOption initializing

* swarm: empty commit to retrigger CI build

* swarm/network/stream: Added comments explaining RegistryOptions
2018-10-17 19:22:37 +02:00
Anton Evangelatov
4e693ad5a6
swarm/tracing: disable stdout logging for opentracing (#17931) 2018-10-17 14:46:59 +02:00
Viktor Trón
6566a0a3b8
swarm/network/stream: generalise setting of next batch (#17818)
* swarm/network/stream: generalize SetNextBatch and add Server SessionIndex

* swarm/network/stream: fix a typo in comment

* swarm/network/stream: remove live argument from NewSwarmSyncerServer
2018-10-12 16:26:16 +02:00
lash
dc3c3fb1e1 swarm/storage: Add accessCnt for GC (#17845) 2018-10-12 16:25:38 +02:00
Felix Lange
dcae0d348b
p2p/simulations: fix a deadlock and clean up adapters (#17891)
This fixes a rare deadlock with the inproc adapter:

- A node is stopped, which acquires Network.lock.
- The protocol code being simulated (swarm/network in my case)
  waits for its goroutines to shut down.
- One of those goroutines calls into the simulation to add a peer,
  which waits for Network.lock.

The fix for the deadlock is really simple, just release the lock
before stopping the simulation node.

Other changes in this PR clean up the exec adapter so it reports
node startup errors better and remove the docker adapter because
it just adds overhead.

In the exec adapter, node information is now posted to a one-shot
server. This avoids log parsing and allows reporting startup
errors to the simulation host.

A small change in package node was needed because simulation
nodes use port zero. Node.{HTTP,WS}Endpoint now return the live
endpoints after startup by checking the TCP listener.
2018-10-11 20:32:14 +02:00
Anton Evangelatov
0fe9a372b3
swarm, swarm/storage: lower constants for faster tests (#17876)
* swarm/storage: lower constants for faster tests

* swarm: reduce test size for TestLocalStoreAndRetrieve

* swarm: reduce nodes for dec_inc_node_count
2018-10-09 11:45:42 +02:00
Péter Szilágyi
ff5538ad4c
params, swarm: begin Geth v1.8.18, Swarm v0.3.6 cycle 2018-10-09 10:37:49 +03:00
Péter Szilágyi
8bbe72075e
params, swarm: release Geth v1.8.17 and Swar v0.3.5 2018-10-09 10:35:31 +03:00
holisticode
11d0ff6578 Fix retrieval tests and simulation backends (#17723)
* swarm/network/stream: introduced visualized snapshot sync test

* swarm/network/stream: non-existing hash visualization sim

* swarm/network/stream: fixed retrieval tests; new backend for visualization

* swarm/network/stream: cleanup of visualized_snapshot_sync_sim_test.go

* swarm/network/stream: rebased PR on master

* swarm/network/stream: fixed loop logic in retrieval tests

* swarm/network/stream: fixed iterations for snapshot tests

* swarm/network/stream: address PR comments

* swarm/network/stream: addressed PR comments
2018-10-08 20:28:44 +02:00
Javier Peletier
c5cb214f68 swarm/storage/feed: Expose MaxUpdateDataLength constant (#17858) 2018-10-08 10:57:38 +02:00
Anton Evangelatov
8c63d0d2e4 swarm/storage: extract isValid. correctly remove invalid chunks from store on migration (#17835) 2018-10-04 18:13:48 +03:00
Viktor Trón
e5677114dc
Merge pull request #17796 from epiclabs-io/mru-feeds
swarm/storage/feeds: Renamed MRU to Swarm Feeds
2018-10-03 14:59:41 +02:00
Anton Evangelatov
303b99663e
swarm: schemas and migrations (#17813) 2018-10-03 14:31:59 +02:00
Javier Peletier
de01178c18 swarm/storage/feed: Renamed package 2018-10-03 09:15:28 +02:00
Javier Peletier
696bc9b01c swarm/storage/feeds: renamed vars that can conflict with package name 2018-10-03 09:12:06 +02:00
Javier Peletier
58c0879c2f swarm/storage/feeds: removed capital Feed throughout 2018-10-03 09:12:06 +02:00
Javier Peletier
68b8088cb9 swarm: Changed owners. 2018-10-03 09:12:06 +02:00
Javier Peletier
b6ccc06cda swarm/storage/feeds: Final package rename and moved files 2018-10-03 09:12:06 +02:00