Anton Evangelatov
4b9c3bd39a
swarm/storage: disable open tracing on indices ( #19578 )
2019-05-15 15:06:57 +02:00
Anton Evangelatov
9b0d1b9ab2
swarm/metrics: collect metrics on datadir disk usage ( #19576 )
2019-05-15 12:22:32 +02:00
Elad
db83ba4067
cmd/swarm: skip export test on windows builds ( #19555 )
2019-05-13 10:27:27 +02:00
Anton Evangelatov
c8a77d8604
swarm/metrics: track runtime metrics ( #19557 )
2019-05-13 09:55:11 +02:00
holisticode
7744241886
swarm/network/stream: add pure retrieval test ( #19552 )
2019-05-11 12:55:06 +02:00
Elad
9b1543c282
swarm/network: update syncer metrics
2019-05-10 12:29:27 +02:00
Anton Evangelatov
297fa1855c
swarm/network: measure addPeer and deletePeer to know if Kad rearranged
...
swarm/storage: remove traces for put/get/set (#1389 )
* swarm/storage: remove traces for put/get/set
* swarm/storage: remove Has traces
2019-05-10 12:29:27 +02:00
Elad
84dfaea246
swarm: instrument setNextBatch
...
swarm/storage/localstore: add gc metrics, disable flaky test
2019-05-10 12:29:22 +02:00
Anton Evangelatov
3e9ba57669
swarm/storage: improve instrumentation
...
swarm/storage/localstore: fix broken metric (#1373 )
p2p/protocols: count different messages (#1374 )
cmd/swarm: disable snapshot create test due to constant flakes (#1376 )
swarm/network: remove redundant goroutine (#1377 )
2019-05-10 12:27:04 +02:00
lash
7f753461ca
swarm/pss: disable failing handshake test
2019-05-10 12:26:59 +02:00
Anton Evangelatov
8802b9ce7f
swarm-smoke: add syncDelay flag
...
swarm/network: add want delay timer to syncing (#1367 )
swarm/network: synchronise peer.close() (#1369 )
2019-05-10 12:26:55 +02:00
Elad
ad6c39012f
swarm: push tags integration - request flow
...
swarm/api: integrate tags to count chunks being split and stored
swarm/api/http: integrate tags in middleware for HTTP `POST` calls and assert chunks being calculated and counted correctly
swarm: remove deprecated and unused code, add swarm hash to DoneSplit signature, remove calls to the api client from the http package
2019-05-10 12:26:52 +02:00
Janoš Guljaš
3030893a21
swarm/network: update syncing
2019-05-10 12:26:49 +02:00
Elad
12240baf61
swarm/chunk: add tags data type
...
* swarm/chunk: add tags backend to chunk package
2019-05-10 12:26:45 +02:00
Anton Evangelatov
f8eb8fe64c
cmd/swarm-smoke: check if chunks are at most prox host
...
swarm/network: measure how many chunks a node delivers (#1358 )
2019-05-10 12:26:41 +02:00
Elad
a1cd7e6e92
p2p/protocols, swarm/network: fix resource leak with p2p teardown
2019-05-10 12:26:37 +02:00
Janoš Guljaš
c1213bd00c
swarm: LocalStore metrics
...
* swarm/shed: remove metrics fields from DB struct
* swarm/schunk: add String methods to modes
* swarm/storage/localstore: add metrics and traces
* swarm/chunk: unknown modes without spaces in String methods
* swarm/storage/localstore: remove bin number from pull subscription metrics
* swarm/storage/localstore: add resetting time metrics and code improvements
2019-05-10 12:26:33 +02:00
Anton Evangelatov
993b145f25
swarm/storage/localstore: fix export db.Put signature
...
cmd/swarm/swarm-smoke: improve smoke tests (#1337 )
swarm/network: remove dead code (#1339 )
swarm/network: remove FetchStore and SyncChunkStore in favor of NetStore (#1342 )
2019-05-10 12:26:30 +02:00
Janoš Guljaš
996755c4a8
cmd/swarm, swarm: LocalStore storage integration
2019-05-10 12:26:26 +02:00
Janoš Guljaš
3873a7314d
swarm/network: fix data races in TestInitialPeersMsg test ( #19490 )
...
* swarm/network: fix data races in TestInitialPeersMsg test
* swarm/network: add Kademlia.Saturation method with lock
* swarm/network: add Hive.Peer method to safely retrieve a bzz peer
* swarm/network: remove duplicate comment
* p2p/testing: prevent goroutine leak in ProtocolTester
* swarm/network: fix data race in newBzzBaseTesterWithAddrs
* swarm/network: fix goroutone leaks in testInitialPeersMsg
* swarm/network: raise number of peer check attempts in testInitialPeersMsg
* swarm/network: use Hive.Peer in Hive.PeerInfo function
* swarm/network: reduce the scope of mutex lock in newBzzBaseTesterWithAddrs
* swarm/storage: disable TestCleanIndex with race detector
2019-04-25 21:33:18 +02:00
gluk256
d9403690ec
swarm/pss: Fix flaky TestProxNetwork ( #19471 )
2019-04-19 11:15:17 +02:00
Péter Szilágyi
78ec90717a
swarm/version: bump version due to Geth hotfix release
2019-04-17 15:50:41 +03:00
Javier Peletier
73fc65bb5b
swarm/storage/feed: add context handling/cancellation to Swarm Feeds lookup, fix bad hint lookup bug ( #19353 )
...
* swarm/storage/feed/lookup: Add context handling/forwarding
* swarm/storage/feed/lookup: Add test to catch bad hint
* swarm/storage/feed/lookup: Added context cancellation test
2019-04-16 07:13:02 +02:00
Ferenc Szabo
26b50e3ebe
cmd/swarm: fix resource leaks in tests ( #19443 )
...
* swarm/api: fix file descriptor leak in NewTestSwarmServer
Swarm storage (localstore) was not closed. That resulted a
"too many open files" error if `TestClientUploadDownloadRawEncrypted`
was run with `-count 1000`.
* cmd/swarm: speed up StartNewNodes() by parallelization
Reduce cluster startup time from 13s to 7s.
* swarm/api: disable flaky TestClientUploadDownloadRawEncrypted with -race
* swarm/storage: disable flaky TestLDBStoreCollectGarbage (-race)
With race detection turned on the disabled cases often fail with:
"ldbstore_test.go:535: expected surplus chunk 150 to be missing, but got no error"
* cmd/swarm: fix process leak in TestACT and TestSwarmUp
Each test run we start 3 nodes, but we did not terminate them. So
those 3 nodes continued eating up 1.2GB (3.4GB with -race) after test
completion.
6b6c4d1c27
2019-04-11 12:44:15 +02:00
Péter Szilágyi
b7dd225179
swarm/version: bump Swarm due to Geth hotfix release
2019-04-10 16:13:35 +03:00
Péter Szilágyi
5fc5971438
swarm/version: bump version due to Geth-only hotfix release
2019-04-09 12:19:24 +03:00
Péter Szilágyi
5b947c5004
swarm/version: bump version due to Geth maintenance release
2019-04-08 16:08:22 +03:00
Viktor Trón
0529015091
swarm/network: hive bug: needed shallow peers are not sent to nodes beyond connection's proximity order ( #19326 )
...
* swarm/network: fix hive bug not sending shallow peers
- hive bug: needed shallow peers were not sent to nodes beyond connection's proximity order
- add extensive protocol exchange tests for initial subPeersMsg-peersMsg exchange
- modify bzzProtocolTester to allow pregenerated overlay addresses
* swarm/network: attempt to fix hive persistance test
* swarm/network: fix TestHiveStatePersistance (#1320 )
* swarm/network: remove trace lines from the hive persistance test
* address PR review comments
* swarm/network: address PR comments on TestInitialPeersMsg
* eliminate *testing.T argument from bzz/hive protocoltesters
* add sorting (only runs in test code) on peersMsg payload
* add random (0 to MaxPeersPerPO) peers for each po
* add extra peers closer to pivot than control
2019-04-02 09:15:16 +02:00
lash
2f5b6cb442
swarm/network: Use different privatekey for bzz overlay in sim ( #19313 )
...
* cmd/swarm, p2p, swarm: Enable ENR in binary/execadapter
* cmd/p2p/swarm: Remove comments + config.Enode nomarshal
* p2p/simulations: Remove superfluous error check
* p2p/simulation: Move init enode comment
* swarm, p2p/simulations, cmd/swarm: Use nodekey in binary record sign
* swarm/network, swarm/pss: Dervice bzzkey
* swarm/pss: Remove unused function
* swarm/network: Store swarm private key in simulation bucket
* swarm/pss: Shorten TextProxNetwork shortrunning test timeout
* swarm/pss: Increase prox test timeout
* swarm/pss: Increase timeout slightly on shortrunning proxtest
* swarm/network: Simplify bucket instantiation in servicectx func
* p2p/simulations: Tcpport -> udpport
* swarm/network, swarm/pss: Simplify + correct lock in servicefunc sim
* swarm/network: Cleanup after rebase on extract swarm enode new
* p2p/simulations, swarm/network: Make exec disc test pass
* swarm/network: Prune ye olde comment
* swarm/pss: Correct revised bzzkey method call
* swarm/network: Clarify comment about privatekey generation data
* swarm/pss: Fix syntax errors after rebase
* swarm/network: Rename misleadingly named method
(amend commit to trigger ci - attempt 5)
2019-03-22 21:37:25 +01:00
gluk256
8d04154691
p2p/simulations: wait until all connections are recreated when uploading snapshot ( #19312 )
...
* swarm/network/simulation: test cases refactored
* swarm/pss: minor refactoring
* swarm/simulation: UploadSnapshot updated
* swarm/network: style fix
* swarm/pss: bugfix
2019-03-22 11:20:17 +01:00
lash
09924cbcaa
cmd/swarm, p2p, swarm: Enable ENR in binary/execadapter ( #19309 )
...
* cmd/swarm, p2p, swarm: Enable ENR in binary/execadapter
* cmd/p2p/swarm: Remove comments + config.Enode nomarshal
* p2p/simulations: Remove superfluous error check
* p2p/simulation: Move init enode comment
* swarm/api: Check error in config test
* swarm, p2p/simulations, cmd/swarm: Use nodekey in binary record sign
* cmd/swarm: Make nodekey available for swarm api config
2019-03-22 05:55:47 +01:00
Anton Evangelatov
baded64d88
swarm/network: measure time of messages in priority queue ( #19250 )
2019-03-20 21:30:34 +01:00
gluk256
6e401792ce
swarm/pss: negihbourhood addressing simulation tests ( #19278 )
...
* swarm/pss: fixed bug in pss.process, test added
* swarm/pss: test case updated
* swarm/pss: WaitTillSnapshotRecreated() func added
* swarm/pss: snapshot test updated
* swarm/pss: WaitTillSnapshotLoaded() fixed
* swarm/pss: gofmt applied
* swarm/pss: refactoring, file renamed
* swarm/pss: input data fixed
* swarm/pss: race condition fixed
* swarm/pss: test timeout increased
* swarm/pss: eliminated the global variables
* swarm/pss: tests added
* swarm/pss: comments added
* swarm/pss: comment fixed
* swarm/pss: refactored according to review
* swarm/pss: style fix
* swarm/pss: increased timeout
2019-03-16 08:39:38 +01:00
Janoš Guljaš
f180981273
swarm/shed: add vector uint64 field ( #19279 )
2019-03-15 15:12:45 +01:00
lash
4b4f03ca37
swarm, p2p: Prerequities for ENR replacing handshake ( #19275 )
...
* swarm/api, swarm/network, p2p/simulations: Prerequisites for handshake remove
* swarm, p2p: Add full sim node configs for protocoltester
* swarm/network: Make stream package pass tests
* swarm/network: Extract peer and addr types out of protocol file
* p2p, swarm: Make p2p/protocols tests pass + rename types.go
* swarm/network: Deactivate ExecAdapter test until binary ENR prep
* swarm/api: Remove comments
* swarm/network: Uncomment bootnode record load
2019-03-15 11:27:17 +01:00
Anton Evangelatov
1a3e25e4c1
swarm: tracing improvements ( #19249 )
2019-03-11 11:45:34 +01:00
Janoš Guljaš
9a58a9b91a
swarm/storage/localstore: global batch write lock ( #19245 )
...
* swarm/storage/localstore: most basic database
* swarm/storage/localstore: fix typos and comments
* swarm/shed: add uint64 field Dec and DecInBatch methods
* swarm/storage/localstore: decrement size counter on ModeRemoval update
* swarm/storage/localstore: unexport modeAccess and modeRemoval
* swarm/storage/localstore: add WithRetrievalCompositeIndex
* swarm/storage/localstore: add TestModeSyncing
* swarm/storage/localstore: fix test name
* swarm/storage/localstore: add TestModeUpload
* swarm/storage/localstore: add TestModeRequest
* swarm/storage/localstore: add TestModeSynced
* swarm/storage/localstore: add TestModeAccess
* swarm/storage/localstore: add TestModeRemoval
* swarm/storage/localstore: add mock store option for chunk data
* swarm/storage/localstore: add TestDB_pullIndex
* swarm/storage/localstore: add TestDB_gcIndex
* swarm/storage/localstore: change how batches are written
* swarm/storage/localstore: add updateOnAccess function
* swarm/storage/localhost: add DB.gcSize
* swarm/storage/localstore: update comments
* swarm/storage/localstore: add BenchmarkNew
* swarm/storage/localstore: add retrieval tests benchmarks
* swarm/storage/localstore: accessors redesign
* swarm/storage/localstore: add semaphore for updateGC goroutine
* swarm/storage/localstore: implement basic garbage collection
* swarm/storage/localstore: optimize collectGarbage
* swarm/storage/localstore: add more garbage collection tests cases
* swarm/shed, swarm/storage/localstore: rename IndexItem to Item
* swarm/shed: add Index.CountFrom
* swarm/storage/localstore: persist gcSize
* swarm/storage/localstore: remove composite retrieval index
* swarm/shed: IterateWithPrefix and IterateWithPrefixFrom Index functions
* swarm/storage/localstore: writeGCSize function with leveldb batch
* swarm/storage/localstore: unexport modeSetRemove
* swarm/storage/localstore: update writeGCSizeWorker comment
* swarm/storage/localstore: add triggerGarbageCollection function
* swarm/storage/localstore: call writeGCSize on DB Close
* swarm/storage/localstore: additional comment in writeGCSizeWorker
* swarm/storage/localstore: add MetricsPrefix option
* swarm/storage/localstore: fix a typo
* swamr/shed: only one Index Iterate function
* swarm/storage/localstore: use shed Iterate function
* swarm/shed: pass a new byte slice copy to index decode functions
* swarm/storage/localstore: implement feed subscriptions
* swarm/storage/localstore: add more subscriptions tests
* swarm/storage/localsore: add parallel upload test
* swarm/storage/localstore: use storage.MaxPO in subscription tests
* swarm/storage/localstore: subscription of addresses instead chunks
* swarm/storage/localstore: lock item address in collectGarbage iterator
* swarm/storage/localstore: fix TestSubscribePull to include MaxPO
* swarm/storage/localstore: improve subscriptions
* swarm/storage/localstore: add TestDB_SubscribePull_sinceAndUntil test
* swarm/storage/localstore: adjust pull sync tests
* swarm/storage/localstore: remove writeGCSizeDelay and use literal
* swarm/storage/localstore: adjust subscriptions tests delays and comments
* swarm/storage/localstore: add godoc package overview
* swarm/storage/localstore: fix a typo
* swarm/storage/localstore: update package overview
* swarm/storage/localstore: remove repeated index change
* swarm/storage/localstore: rename ChunkInfo to ChunkDescriptor
* swarm/storage/localstore: add comment in collectGarbageWorker
* swarm/storage/localstore: replace atomics with mutexes for gcSize and tests
* swarm/storage/localstore: protect addrs map in pull subs tests
* swarm/storage/localstore: protect slices in push subs test
* swarm/storage/localstore: protect chunks in TestModePutUpload_parallel
* swarm/storage/localstore: fix a race in TestDB_updateGCSem defers
* swarm/storage/localstore: remove parallel flag from tests
* swarm/storage/localstore: fix a race in testDB_collectGarbageWorker
* swarm/storage/localstore: remove unused code
* swarm/storage/localstore: add more context to pull sub log messages
* swarm/storage/localstore: BenchmarkPutUpload and global lock option
* swarm/storage/localstore: pre-generate chunks in BenchmarkPutUpload
* swarm/storage/localstore: correct useGlobalLock in collectGarbage
* swarm/storage/localstore: fix typos and update comments
* swarm/storage/localstore: update writeGCSize comment
* swarm/storage/localstore: global batch write lock
* swarm/storage/localstore: remove global lock option
* swarm/storage/localstore: simplify DB.Close
2019-03-09 00:06:39 +01:00
Anton Evangelatov
bb55b0fb53
swarm/storage: add comparison towards leveldb.ErrNotFound ( #19243 )
...
* swarm/storage: add comparison towards leveldb.ErrNotFound
* swarm/storage: wrap leveldb ErrNotFound
2019-03-08 17:28:57 +01:00
Anton Evangelatov
2cfe0bed9f
swarm: fix relationship between spans in open tracing ( #19236 )
...
* swarm/network: propagate span with ctx
* swarm/network: try to stop stream.send.request spans on time
* swarm/storage: add chunk ref as a log to netstore.fetcher span
2019-03-08 08:52:25 +01:00
Janoš Guljaš
a6e5c6a2cc
swarm/storage/localstore: fix synchronization in TestDB_gcSize ( #19235 )
2019-03-08 05:59:59 +01:00
Janoš Guljaš
eb199f1fc2
swarm: localstore hasser ( #19230 )
2019-03-07 10:07:54 +01:00
Ferenc Szabo
d45f8d1880
swarm/network: remove *WithServer tests from stream package ( #19223 )
...
These tests never run as the build tag excluded them from the CI
execution. As a results the (dead) code got out of sync with other
parts of Swarm and now they would not even compile. => Removed.
resolves ethersphere/go-ethereum#1238
2019-03-07 09:27:56 +01:00
holisticode
a87776a5fe
swarm/network/stream: Fix flaky tests in GetSubscriptionsRPC test ( #19227 )
...
* swarm/network/stream: fixed timing issues
* swarm/network/stream: only count first iteration of subscriptions
* swarm/network/stream/: fix linter errors
2019-03-07 09:24:28 +01:00
holisticode
81ed700157
Enable longrunning tests to run ( #19208 )
...
* p2p/simulations: increased snapshot load timeout for debugging
* swarm/network/stream: less nodes for snapshot longrunning tests
* swarm/network: fixed longrunning tests
* swarm/network/stream: store kademlia in bucket
* swarm/network/stream: disabled healthy check in delivery tests
* swarm/network/stream: longer SyncUpdateDelay for longrunning tests
* swarm/network/stream: more debug output
* swarm/network/stream: reduced longrunning snapshot tests to 64 nodes
* swarm/network/stream: don't WaitTillHealthy in SyncerSimulation
* swarm/network/stream: cleanup for PR
2019-03-05 12:54:46 +01:00
Janoš Guljaš
216bd2ceba
swarm/storage/localstore: fix testDB_collectGarbageWorker data race ( #19206 )
2019-03-04 22:19:57 +01:00
Anton Evangelatov
f9aa1cd21f
Revert "swarm/network: Use actual remote peer ip in underlay ( #19137 )" ( #19193 )
...
This reverts commit 460d206f30
2019-03-02 08:45:07 +01:00
Janoš Guljaš
b797dd07d2
swarm/shed, swarm/storage/localstore: add LastPullSubscriptionChunk ( #19190 )
...
* swarm/shed, swarm/storage/localstore: add LastPullSubscriptionChunk
* swarm/shed: fix comments
* swarm/shed: fix TestIncByteSlice test
* swarm/storage/localstore: fix TestDB_LastPullSubscriptionChunk
2019-03-02 08:44:22 +01:00
Anton Evangelatov
4e9230ea7a
swarm: enable p2p/discovery and disable dynamic dialling ( #19189 )
2019-03-01 12:20:37 +01:00
holisticode
994326ba00
swarm: new snapshot files ( #19185 )
2019-02-28 22:30:36 +01:00
lash
62d9d63858
swarm/network: WIP consider all nodes for healthy iteration ( #19155 )
...
* swarm/network: WIP consider all nodes for healthy iteration
* swarm/network/simulation: extend TestWaitTillHealthy to really check kads are healthy
* cmd/swarm/swarm-snapshot: fixed bugs in snapshot creation binary
* swarm/network/simulation: addressed PR comments
* swarm/network/simulation: defer sim.Clsoe()
* swarm/network/simulation: fixed wrong sim.Close()
* swarm/network/simulation: addressed PR comments
* cmd/swarm/swarm-snapshot: reducing default to 8 nodes, more to 4
* cmd/swarm/swarm-snapshot: extended timeout to 3 mins, or 256 nodes snapshot times out
* swarm/network/simulation: More PR comments
2019-02-28 08:12:50 +01:00
