Commit Graph

396 Commits

Author SHA1 Message Date
Anton Evangelatov
ebbf3dfafb swarm/shed: add metrics to each shed db (#18277)
* swarm/shed: add metrics to each shed db

* swarm/shed: push metrics prefix up

* swarm/shed: rename prefix to metricsPrefix

* swarm/shed: unexport Meter, remove Mutex for quit channel
2018-12-12 07:51:29 +01:00
Péter Szilágyi
1e190a3b1c
params, swarm: begin Geth v1.9.0 family, Swarm v0.3.9 cycle 2018-12-11 14:23:57 +02:00
Péter Szilágyi
24d727b6d6
params, swarm: release Geth v1.8.20 and Swarm v0.3.8 2018-12-11 14:20:21 +02:00
Elad
bb724080ca cmd/swarm, metrics, swarm/api/client, swarm/storage, swarm/metrics, swarm/api/http: add instrumentation (#18274) 2018-12-11 09:21:58 +01:00
lash
dd98d1da94 swarm/network: Correct ambiguity in compared addresses (#18251) 2018-12-10 14:56:01 +02:00
Janoš Guljaš
661809714e swarm: snapshot load improvement (#18220)
* swarm/network: Hive - do not notify peer if discovery is disabled

* p2p/simulations: validate all connections on loading a snapshot

* p2p/simulations: track all connections in on snapshot loading

* p2p/simulations: add snapshotLoadTimeout variable

* p2p/simulations: ignore control events in snapshot load

* p2p/simulations: simplify event loop synchronization

* p2p/simulations: return already connected error from Load function

* p2p/simulations: log warning on snapshot loading disconnection
2018-12-07 06:51:40 +01:00
Elad
3ac633ba84 swarm/api/http: add resetting timer metrics to requests (#18249) 2018-12-05 11:20:55 +01:00
holisticode
b98d2e9a1c swarm/network/stream: Debug log instead of Warn for retrieval failure (#18246) 2018-12-04 18:29:51 +01:00
lash
085f89172f swarm/pss: Add same api interface for all Send* methods (#18218) 2018-12-01 07:07:18 +01:00
Anton Evangelatov
7e7781ffaa cmd/swarm: add flag for application name (swarm or swarm-private) (#18189)
* cmd/swarm: add flag for application name (swarm or swarm-private)

* cmd/swarm/swarm-smoke: return correct exit code

* cmd/swarm/swarm-smoke: remove colorable

* remove swarm/grafana_dashboards
2018-11-29 18:43:15 +02:00
Péter Szilágyi
a7501d0c41
params, swarm: start Geth v1.8.20 and Swarm v0.3.8 release cycle 2018-11-28 14:44:00 +02:00
Péter Szilágyi
dae82f0985
params, swarm: release Geth v1.8.19 and Swarm v0.3.7 2018-11-28 14:42:37 +02:00
holisticode
695a5cce1e Increase bzz version (#18184)
* swarm/network/stream/: added stream protocol version match tests

* Increase BZZ version due to streamer version change; version tests

* swarm/network: increased hive and test protocol version
2018-11-26 19:34:40 +01:00
Janoš Guljaš
c207edf2a3 swarm: add database abstractions (shed package) (#18183) 2018-11-26 18:49:01 +01:00
lash
1cd007ecae swarm/network: Correct neighborhood depth (#18066) 2018-11-26 17:13:59 +01:00
holisticode
bba5fd8192 Accounting metrics reporter (#18136) 2018-11-26 17:05:18 +01:00
Javier Peletier
2714e8f091 Remove multihash from Swarm bzz:// for Feeds (#18175) 2018-11-26 16:10:22 +01:00
lash
197d609b9a swarm/pss: Message handler refactor (#18169) 2018-11-26 13:52:04 +01:00
Elad
f5e6634fd2 swarm/api: improve not found error msg (#18171) 2018-11-26 12:53:15 +01:00
Janoš Guljaš
93854bbad4 swarm/network/simulation: fix New function for-loop scope (#18161) 2018-11-26 12:39:38 +01:00
Janoš Guljaš
070caec4bd swarm/network/stream: use swarm/mock/mem as mock global store (#18157) 2018-11-21 20:49:13 +01:00
Anton Evangelatov
4c181e4fb9
swarm/state: refactor InmemoryStore (#18143) 2018-11-21 14:36:56 +01:00
Ferenc Szabo
d876f214e5 swarm/storage: move 'running migrations for' log line (#18120)
So that we only see the log message when we actually have to migrate.
2018-11-20 08:30:38 +01:00
Anton Evangelatov
3333fe660f
swarm/storage: speed up garbage collection and rpc tests (#18128) 2018-11-19 12:26:45 +01:00
Elad
51e2e78d26 swarm/api/http: change request served msg log level (#18127) 2018-11-18 10:54:03 +01:00
holisticode
ffe2fc3bc4 Swarm accounting (#18050)
* swarm: completed 1st phase of swap accounting

* swarm: swap accounting for swarm with p2p accounting

* swarm/swap: addressed PR comments

* swarm/swap: ignore ErrNotFound on stateStore.Get()

* swarm/swap: GetPeerBalance test; add TODO for chequebook API check

* swarm/network/stream: fix NewRegistry calls with new arguments

* swarm/swap: address @justelad's PR comments
2018-11-15 23:41:19 +01:00
Janoš Guljaš
324027640b swarm/network/simulation: use simulations.Event instead p2p.PeerEvent (#18098) 2018-11-15 21:06:27 +01:00
lash
a6942b9f25 swarm/storage: Batched database migration (#18113) 2018-11-15 14:57:03 +01:00
Péter Szilágyi
48b4e8069c
params, swarm: begin Geth v1.8.19 and Swarm v0.3.7 cycle 2018-11-14 10:27:30 +02:00
Péter Szilágyi
58632d4402
params, swarm: release Geth v1.8.18 and Swarm v0.3.6 2018-11-14 10:25:19 +02:00
Alexey Sharov
eb8fa3cc89 cmd/swarm, swarm/api/http, swarm/bmt, swarm/fuse, swarm/network/stream, swarm/storage, swarm/storage/encryption, swarm/testutil: use pseudo-random instead of crypto-random for test files content generation (#18083)
- Replace "crypto/rand" to "math/rand" for files content generation
- Remove swarm/network_test.go.Shuffle and swarm/btm/btm_test.go.Shuffle - because go1.9 support dropped (see https://github.com/ethereum/go-ethereum/pull/17807 and comments to swarm/network_test.go.Shuffle)
2018-11-14 09:21:14 +01:00
Ferenc Szabo
c41e1bd1eb swarm/storage: fix garbage collector index skew (#18080)
On file access LDBStore's tryAccessIdx() function created a faulty
GC Index Data entry, because not indexing the ikey correctly.
That caused the chunk addresses/hashes to start with '00' and the last
two digits were dropped. => Incorrect chunk address.

Besides the fix, the commit also contains a schema change which will
run the CleanGCIndex() function to clean the GC index from erroneous
entries.

Note: CleanGCIndex() rebuilds the index from scratch which can take
a really-really long time with a huge DB (possibly an hour).
2018-11-13 15:22:53 +01:00
Ferenc Szabo
8080265f3f swarm/storage: fix access count on dbstore after cache hit (#17978)
Access count was not incremented when chunk was retrieved
from cache. So the garbage collector might have deleted the most
frequently accessed chunk from disk.

Co-authored-by: Ferenc Szabo <ferenc.szabo@ethereum.org>
2018-11-13 07:41:01 +01:00
lash
201a0bf181 p2p/simulations, swarm/network: Custom services in snapshot (#17991)
* p2p/simulations: Add custom services to simnodes + remove sim down conn objs

* p2p/simulation, swarm/network: Add selective services to discovery sim

* p2p/simulations, swarm/network: Remove useless comments

* p2p/simulations, swarm/network: Clean up mess from rebase

* p2p/simulation: Add sleep to prevent connect flakiness in http test

* p2p/simulations: added concurrent goroutines to prevent sleeps on simulation connect/disconnect

* p2p/simulations, swarm/network/simulations: address pr comments

* reinstated dummy service

* fixed http snapshot test
2018-11-12 14:57:17 +01:00
Anton Evangelatov
cf3b187bde swarm, cmd/swarm: address ineffectual assignments (#18048)
* swarm, cmd/swarm: address ineffectual assignments

* swarm/network: remove unused vars from testHandshake

* swarm/storage/feed: revert cursor changes
2018-11-07 20:39:08 +01:00
Mark Vujevits
81533deae5 swarm/network: light nodes are not dialed, saved and requested from (#17975)
* RequestFromPeers does not use peers marked as lightnode

* fix warning about variable name

* write tests for RequestFromPeers

* lightnodes should be omitted from the addressbook

* resolve pr comments regarding logging, formatting and comments

* resolve pr comments regarding comments and added a missing newline

* add assertions to check peers in live connections
2018-11-07 20:33:36 +01:00
Javier Peletier
36ca85fa1c swarm/api: Fix #18007, missing signature should return HTTP 400 (#18008) 2018-11-07 14:49:42 +01:00
holisticode
79c7a69ac8 swarm: Better syncing and retrieval option definition (#17986)
* swarm: Better syncing and retrieval option definition

* swarm/network/stream: better comments

* swarm/network/stream: addressed PR comments
2018-11-07 00:04:18 +01:00
Anton Evangelatov
53eb4e0b0f
swarm/api: unexport Respond methods (#18037) 2018-11-06 12:34:34 +01:00
KimMachineGun
baee850471 swarm: modify context key (#17925)
* swarm: modify context key

* gofmt sctx.go
2018-11-06 09:54:19 +01:00
Ferenc Szabo
54f650a3be swarm: clean up unused private types and functions (#17989)
* swarm: clean up unused private types and functions

Those that were identified by code inspection tool.

* swarm/storage: move/add Proximity GoDoc from deleted private function

The mentioned proximity() private function was deleted in:
1ca8fc1e6fa0ab4ab1aaca06d6fb32e173cd5f2f
2018-10-27 16:18:42 +02:00
holisticode
88b41a9e68 swarm/network/stream: disambiguate chunk delivery messages (retrieval… (#17920)
* swarm/network/stream: disambiguate chunk delivery messages (retrieval vs syncing)

* swarm/network/stream: addressed PR comments

* swarm/network/stream: stream protocol version change due to new message types in this PR
2018-10-21 09:30:41 +02:00
Elad
66debd91d9 swarm/api/http: remove ModTime=now for direct and multipart uploads (#17945) 2018-10-19 16:02:44 +02:00
Elad
aeb733623e swarm/network: disallow historical retrieval requests (#17936) 2018-10-19 10:50:25 +02:00
Attila Gazso
cdf5982cfc swarm: Lightnode mode: disable sync, retrieve, subscription (#17899)
* swarm: Lightnode mode: disable sync, retrieve, subscription

* swarm/network/stream: assign error and check in one line

* swarm: restructured RegistryOption initializing

* swarm: empty commit to retrigger CI build

* swarm/network/stream: Added comments explaining RegistryOptions
2018-10-17 19:22:37 +02:00
Anton Evangelatov
4e693ad5a6
swarm/tracing: disable stdout logging for opentracing (#17931) 2018-10-17 14:46:59 +02:00
Viktor Trón
6566a0a3b8
swarm/network/stream: generalise setting of next batch (#17818)
* swarm/network/stream: generalize SetNextBatch and add Server SessionIndex

* swarm/network/stream: fix a typo in comment

* swarm/network/stream: remove live argument from NewSwarmSyncerServer
2018-10-12 16:26:16 +02:00
lash
dc3c3fb1e1 swarm/storage: Add accessCnt for GC (#17845) 2018-10-12 16:25:38 +02:00
Felix Lange
dcae0d348b
p2p/simulations: fix a deadlock and clean up adapters (#17891)
This fixes a rare deadlock with the inproc adapter:

- A node is stopped, which acquires Network.lock.
- The protocol code being simulated (swarm/network in my case)
  waits for its goroutines to shut down.
- One of those goroutines calls into the simulation to add a peer,
  which waits for Network.lock.

The fix for the deadlock is really simple, just release the lock
before stopping the simulation node.

Other changes in this PR clean up the exec adapter so it reports
node startup errors better and remove the docker adapter because
it just adds overhead.

In the exec adapter, node information is now posted to a one-shot
server. This avoids log parsing and allows reporting startup
errors to the simulation host.

A small change in package node was needed because simulation
nodes use port zero. Node.{HTTP,WS}Endpoint now return the live
endpoints after startup by checking the TCP listener.
2018-10-11 20:32:14 +02:00
Anton Evangelatov
0fe9a372b3
swarm, swarm/storage: lower constants for faster tests (#17876)
* swarm/storage: lower constants for faster tests

* swarm: reduce test size for TestLocalStoreAndRetrieve

* swarm: reduce nodes for dec_inc_node_count
2018-10-09 11:45:42 +02:00
Péter Szilágyi
ff5538ad4c
params, swarm: begin Geth v1.8.18, Swarm v0.3.6 cycle 2018-10-09 10:37:49 +03:00
Péter Szilágyi
8bbe72075e
params, swarm: release Geth v1.8.17 and Swar v0.3.5 2018-10-09 10:35:31 +03:00
holisticode
11d0ff6578 Fix retrieval tests and simulation backends (#17723)
* swarm/network/stream: introduced visualized snapshot sync test

* swarm/network/stream: non-existing hash visualization sim

* swarm/network/stream: fixed retrieval tests; new backend for visualization

* swarm/network/stream: cleanup of visualized_snapshot_sync_sim_test.go

* swarm/network/stream: rebased PR on master

* swarm/network/stream: fixed loop logic in retrieval tests

* swarm/network/stream: fixed iterations for snapshot tests

* swarm/network/stream: address PR comments

* swarm/network/stream: addressed PR comments
2018-10-08 20:28:44 +02:00
Javier Peletier
c5cb214f68 swarm/storage/feed: Expose MaxUpdateDataLength constant (#17858) 2018-10-08 10:57:38 +02:00
Anton Evangelatov
8c63d0d2e4 swarm/storage: extract isValid. correctly remove invalid chunks from store on migration (#17835) 2018-10-04 18:13:48 +03:00
Viktor Trón
e5677114dc
Merge pull request #17796 from epiclabs-io/mru-feeds
swarm/storage/feeds: Renamed MRU to Swarm Feeds
2018-10-03 14:59:41 +02:00
Anton Evangelatov
303b99663e
swarm: schemas and migrations (#17813) 2018-10-03 14:31:59 +02:00
Javier Peletier
de01178c18 swarm/storage/feed: Renamed package 2018-10-03 09:15:28 +02:00
Javier Peletier
696bc9b01c swarm/storage/feeds: renamed vars that can conflict with package name 2018-10-03 09:12:06 +02:00
Javier Peletier
58c0879c2f swarm/storage/feeds: removed capital Feed throughout 2018-10-03 09:12:06 +02:00
Javier Peletier
68b8088cb9 swarm: Changed owners. 2018-10-03 09:12:06 +02:00
Javier Peletier
b6ccc06cda swarm/storage/feeds: Final package rename and moved files 2018-10-03 09:12:06 +02:00
Javier Peletier
83705ef6aa swarm/storage/mru: Renamed rest of MRU references 2018-10-03 09:12:06 +02:00
Javier Peletier
b35622cf3c swarm/storage/mru: Renamed all comments to Feeds 2018-10-03 09:12:06 +02:00
Javier Peletier
f1e86ad9cf swarm/storage/mru: Renamed all identifiers to Feeds 2018-10-03 09:12:06 +02:00
Alexey Sharov
dc5d643bb5 cmd/swarm, swarm: cross-platform Content-Type detection (#17782)
- Mime types generator (Standard "mime" package rely on system-settings, see mime.osInitMime)
- Changed swarm/api.Upload:
    - simplify I/O throttling by semaphore primitive and use file name where possible
    - f.Close() must be called in Defer - otherwise panic or future added early return will cause leak of file descriptors
    - one error was suppressed
2018-10-01 13:39:39 +02:00
Janoš Guljaš
7e1c374dc6 swarm/storage: ensure 64bit hasherStore struct alignment (#17766) 2018-09-28 20:04:56 +02:00
Anton Evangelatov
0ee44e796a swarm/storage: make linter happy 2018-09-28 17:11:13 +02:00
Javier Peletier
2c110c81ee Swarm MRUs: Adaptive frequency / Predictable lookups / API simplification (#17559)
* swarm/storage/mru: Adaptive Frequency

swarm/storage/mru/lookup: fixed getBaseTime
Added NewEpoch constructor

swarm/api/client: better error handling in GetResource()


swarm/storage/mru: Renamed structures.
Renamed ResourceMetadata to ResourceID. 
Renamed ResourceID.Name to ResourceID.Topic

swarm/storage/mru: Added binarySerializer interface and test tools

swarm/storage/mru/lookup: Changed base time to time and + marshallers

swarm/storage/mru:  Added ResourceID (former resourceMetadata)

swarm/storage/mru: Added ResourceViewId and serialization tests

swarm/storage/mru/lookup: fixed epoch unmarshaller. Added Epoch Equals

swarm/storage/mru: Fixes as per review comments

cmd/swarm: reworded resource create/update help text regarding topic

swarm/storage/mru: Added UpdateLookup and serializer tests

swarm/storage/mru: Added UpdateHeader, serializers and tests

swarm/storage/mru: changed UpdateAddr / epoch to Base()

swarm/storage/mru: Added resourceUpdate serializer and tests

swarm/storage/mru: Added SignedResourceUpdate tests and serializers

swarm/storage/mru/lookup: fixed GetFirstEpoch bug

swarm/storage/mru: refactor, comments, cleanup

Also added tests for Topic
swarm/storage/mru: handler tests pass

swarm/storage/mru: all resource package tests pass

swarm/storage/mru: resource test pass after adding
timestamp checking support

swarm/storage/mru: Added JSON serializers to ResourceIDView structures

swarm/storage/mru: Sever, client, API test pass

swarm/storage/mru: server test pass

swarm/storage/mru: Added topic length check

swarm/storage/mru: removed some literals,
improved "previous lookup" test case

swarm/storage/mru: some fixes and comments as per review

swarm/storage/mru: first working version without metadata chunk

swarm/storage/mru: Various fixes as per review

swarm/storage/mru: client test pass

swarm/storage/mru: resource query strings and manifest-less queries


swarm/storage/mru: simplify naming

swarm/storage/mru: first autofreq working version



swarm/storage/mru: renamed ToValues to AppendValues

swarm/resource/mru: Added ToValues / FromValues for URL query strings

swarm/storage/mru: Changed POST resource to work with query strings.
No more JSON.

swarm/storage/mru: removed resourceid

swarm/storage/mru: Opened up structures

swarm/storage/mru: Merged Request and SignedResourceUpdate

swarm/storage/mru: removed initial data from CLI resource create

swarm/storage/mru: Refactor Topic as a direct fixed-length array

swarm/storage/mru/lookup: Comprehensive GetNextLevel tests

swarm/storage/mru: Added comments

Added length checks in Topic
swarm/storage/mru: fixes in tests and some code comments

swarm/storage/mru/lookup: new optimized lookup algorithm

swarm/api: moved getResourceView to api out of server

swarm/storage/mru: Lookup algorithm working

swarm/storage/mru: comments and renamed NewLookupParams

Deleted commented code


swarm/storage/mru/lookup: renamed Epoch.LaterThan to After

swarm/storage/mru/lookup: Comments and tidying naming



swarm/storage/mru: fix lookup algorithm

swarm/storage/mru: exposed lookup hint
removed updateheader

swarm/storage/mru/lookup: changed GetNextEpoch for initial values

swarm/storage/mru: resource tests pass

swarm/storage/mru: valueSerializer interface and tests



swarm/storage/mru/lookup: Comments, improvements, fixes, more tests

swarm/storage/mru: renamed UpdateLookup to ID, LookupParams to Query

swarm/storage/mru: renamed query receiver var



swarm/cmd: MRU CLI tests

* cmd/swarm: remove rogue fmt

* swarm/storage/mru: Add version / header for future use

* swarm/storage/mru: Fixes/comments as per review

cmd/swarm: remove rogue fmt

swarm/storage/mru: Add version / header for future use-

* swarm/storage/mru: fix linter errors

* cmd/swarm: Speeded up TestCLIResourceUpdate
2018-09-28 12:07:17 +02:00
Janos Guljas
a5e6bf7eef Merge branch 'master' into max-stream-peer-servers 2018-09-27 09:43:00 +02:00
Viktor Trón
c3cfdfacd0
Merge pull request #17757 from ethersphere/retrieve-request-ttl-pr
swarm: prevent forever running retrieve request loops
2018-09-27 07:08:58 +02:00
Viktor Trón
4b6824e07b
Merge pull request #17734 from frncmx/fix-dos-attack-invalid-hash-length
swarm/network/stream: fix DoS invalid offered hashes length
2018-09-26 12:44:42 +02:00
Balint Gabor
3f7acbbeb9 swarm: prevent forever running retrieve request loops 2018-09-26 11:34:40 +02:00
Janos Guljas
0d5e1e7bc9 swarm/network/stream: fix a typo in test comment 2018-09-26 11:30:45 +02:00
Alpay Yildirim
26cf866349 [ImgBot] optimizes images (#17741)
*Total -- 171.97kb -> 127.26kb (26%)

/swarm/api/testdata/test0/img/logo.png -- 17.71kb -> 4.02kb (77.29%)
/cmd/clef/sign_flow.png -- 35.54kb -> 20.27kb (42.98%)
/cmd/clef/docs/qubes/qrexec-example.png -- 18.66kb -> 15.79kb (15.4%)
/cmd/clef/docs/qubes/clef_qubes_http.png -- 13.97kb -> 11.95kb (14.44%)
/cmd/clef/docs/qubes/clef_qubes_qrexec.png -- 19.79kb -> 17.03kb (13.91%)
/cmd/clef/docs/qubes/qubes_newaccount-2.png -- 41.75kb -> 36.38kb (12.86%)
/cmd/clef/docs/qubes/qubes_newaccount-1.png -- 24.55kb -> 21.82kb (11.11%)
2018-09-26 10:39:39 +02:00
Janos Guljas
24349144b6 Merge branch 'master' into max-stream-peer-servers 2018-09-25 16:57:31 +02:00
Janos Guljas
7d56602391 swarm/api: fix TestDumpConfig 2018-09-25 16:32:11 +02:00
Felix Lange
30cd5c1854
all: new p2p node representation (#17643)
Package p2p/enode provides a generalized representation of p2p nodes
which can contain arbitrary information in key/value pairs. It is also
the new home for the node database. The "v4" identity scheme is also
moved here from p2p/enr to remove the dependency on Ethereum crypto from
that package.

Record signature handling is changed significantly. The identity scheme
registry is removed and acceptable schemes must be passed to any method
that needs identity. This means records must now be validated explicitly
after decoding.

The enode API is designed to make signature handling easy and safe: most
APIs around the codebase work with enode.Node, which is a wrapper around
a valid record. Going from enr.Record to enode.Node requires a valid
signature.

* p2p/discover: port to p2p/enode

This ports the discovery code to the new node representation in
p2p/enode. The wire protocol is unchanged, this can be considered a
refactoring change. The Kademlia table can now deal with nodes using an
arbitrary identity scheme. This requires a few incompatible API changes:

  - Table.Lookup is not available anymore. It used to take a public key
    as argument because v4 protocol requires one. Its replacement is
    LookupRandom.
  - Table.Resolve takes *enode.Node instead of NodeID. This is also for
    v4 protocol compatibility because nodes cannot be looked up by ID
    alone.
  - Types Node and NodeID are gone. Further commits in the series will be
    fixes all over the the codebase to deal with those removals.

* p2p: port to p2p/enode and discovery changes

This adapts package p2p to the changes in p2p/discover. All uses of
discover.Node and discover.NodeID are replaced by their equivalents from
p2p/enode.

New API is added to retrieve the enode.Node instance of a peer. The
behavior of Server.Self with discovery disabled is improved. It now
tries much harder to report a working IP address, falling back to
127.0.0.1 if no suitable address can be determined through other means.
These changes were needed for tests of other packages later in the
series.

* p2p/simulations, p2p/testing: port to p2p/enode

No surprises here, mostly replacements of discover.Node, discover.NodeID
with their new equivalents. The 'interesting' API changes are:

 - testing.ProtocolSession tracks complete nodes, not just their IDs.
 - adapters.NodeConfig has a new method to create a complete node.

These changes were needed to make swarm tests work.

Note that the NodeID change makes the code incompatible with old
simulation snapshots.

* whisper/whisperv5, whisper/whisperv6: port to p2p/enode

This port was easy because whisper uses []byte for node IDs and
URL strings in the API.

* eth: port to p2p/enode

Again, easy to port because eth uses strings for node IDs and doesn't
care about node information in any way.

* les: port to p2p/enode

Apart from replacing discover.NodeID with enode.ID, most changes are in
the server pool code. It now deals with complete nodes instead
of (Pubkey, IP, Port) triples. The database format is unchanged for now,
but we should probably change it to use the node database later.

* node: port to p2p/enode

This change simply replaces discover.Node and discover.NodeID with their
new equivalents.

* swarm/network: port to p2p/enode

Swarm has its own node address representation, BzzAddr, containing both
an overlay address (the hash of a secp256k1 public key) and an underlay
address (enode:// URL).

There are no changes to the BzzAddr format in this commit, but certain
operations such as creating a BzzAddr from a node ID are now impossible
because node IDs aren't public keys anymore.

Most swarm-related changes in the series remove uses of
NewAddrFromNodeID, replacing it with NewAddr which takes a complete node
as argument. ToOverlayAddr is removed because we can just use the node
ID directly.
2018-09-25 00:59:00 +02:00
Janos Guljas
9e99a0c2b9 cmd/swarm, swarm: add stream peer servers limit 2018-09-24 17:56:00 +02:00
Péter Szilágyi
0ae462fb80
params, swarm: begin Geth v1.8.17, Swarm v0.3.5 cycle 2018-09-24 16:02:07 +03:00
Péter Szilágyi
477eb0933b
params, swarm: release Geth v1.8.16, Swarm v0.3.4 2018-09-24 15:59:21 +03:00
Ferenc Szabo
d3f056bd68 swarm/network/stream: fix DoS invalid hash length (#927) 2018-09-21 12:56:43 +02:00
HackyMiner
6975c72981 all: fix various comment typos (#17591)
* swarm: fixed comment typo
* eth: fixed comment typo
* cmd/puppeth: fixed comment typo
2018-09-19 18:10:40 +02:00
Balint Gabor
f5c7d1c8eb swarm/storage: Implement global timeout for fetcher (#17702) 2018-09-19 16:59:10 +02:00
Janoš Guljaš
1f45ba9bb1 swarm/network: downgrade fetcher unable to request log message severity (#17692) 2018-09-18 14:54:52 +02:00
Balint Gabor
bd58098f2d
swarm: Chunk refactor improvements (#17683)
* swarm/network: Protocol bump (for chunk refactor)

* swarm/network: Increase discovery and stream protocol version too

* swarm/network: Increase priority queue cap
2018-09-18 10:17:13 +02:00
Emil
86a03f97d3 all: simplify s[:] to s where s is a slice (#17673) 2018-09-14 22:07:13 +02:00
Balint Gabor
3ff2f75636
swarm: Chunk refactor (#17659)
Co-authored-by: Janos Guljas <janos@resenje.org>
Co-authored-by: Balint Gabor <balint.g@gmail.com>
Co-authored-by: Anton Evangelatov <anton.evangelatov@gmail.com>
Co-authored-by: Viktor Trón <viktor.tron@gmail.com>
2018-09-13 11:42:19 +02:00
Anton Evangelatov
ff3a5d24d2 swarm/storage: remove redundant increments for dataIdx and entryCnt (#17484)
* swarm/storage: remove redundant increments for dataIdx and entryCnt

* swarm/storage: add Delete to LDBStore

* swarm/storage: wait for garbage collection
2018-09-12 14:39:45 +02:00
Viktor Trón
bfce00385f Kademlia refactor (#17641)
* swarm/network: simplify kademlia/hive; rid interfaces

* swarm, swarm/network/stream, swarm/netork/simulations,, swarm/pss: adapt to new Kad API

* swarm/network: minor changes re review; add missing lock to NeighbourhoodDepthC
2018-09-12 11:24:56 +02:00
Viktor Trón
6dd87483d4 Encryption async api (#17603)
* swarm/storage/encryption: async segmentwise encryption/decryption

* swarm/storage: adapt hasherstore to encryption API change

* swarm/api: adapt RefEncryption for AC to new Encryption API

* swarm/storage/encryption: address review comments
2018-09-11 11:39:02 +02:00
Elad
8b9b149d54 swarm/api/http: bzz-immutable wrong handler bug (#17602) 2018-09-07 15:23:29 +02:00
Elad
70d31fb278 cmd/swarm: added password to ACT (#17598) 2018-09-07 09:56:05 +02:00
Elad
580145e96d swarm/storage: added metrics for db entry count (#17589) 2018-09-06 12:11:38 +02:00
Elad
4c15ffffdd swarm/api/http: added a regression test for resolver bug from #17483 (#17502) 2018-09-06 12:08:39 +02:00
Péter Szilágyi
62e94895da
params, swarm: begin geth v1.8.16 and swarm v0.3.4 cycle 2018-08-29 18:27:43 +03:00
Péter Szilágyi
89451f7c38
params, swarm: release geth v1.8.15 and swarm 0.3.3 2018-08-29 18:24:32 +03:00
Geon Kim
1acefafe22 swarm/api: fix typo (#17500) 2018-08-27 11:51:58 +03:00
Wenbiao Zheng
d1aa605f1e all: remove the duplicate 'the' in annotations (#17509) 2018-08-27 11:49:29 +03:00
Elad
f34f361ca6 swarm/api/http: fixed resolver bug (#17483) 2018-08-22 17:53:33 +03:00
Péter Szilágyi
2993fb519d
params, swarm: begin geth v1.8.15 and swarm v0.3.3 cycle 2018-08-22 11:42:03 +03:00
Péter Szilágyi
316fc7ecfc
params, swarm: release Geth v1.8.14 and Swarm v0.3.2 2018-08-22 11:39:00 +03:00
Geon Kim
522cfc68ff swarm: fix typos (#17473) 2018-08-21 21:13:33 +03:00
Anton Evangelatov
c582667c9b swarm/network: bump bzz protocol version (#17449) 2018-08-21 11:34:40 +03:00
Anton Evangelatov
dcd97c41fa swarm, swarm/network, swarm/pss: log error and fix logs (#17410)
* swarm, swarm/network, swarm/pss: log error and fix logs

* swarm/pss: log compressed publickey
2018-08-21 11:34:10 +03:00
Anton Evangelatov
a8aa89accb swarm/storage: cleanup task - remove bigger chunks (#17424) 2018-08-20 15:10:30 +03:00
Elad
c4078fc805 cmd/swarm: added swarm bootnodes (#17414) 2018-08-20 15:09:50 +03:00
Elad
e8752f4e9f cmd/swarm, swarm: added access control functionality (#17404)
Co-authored-by: Janos Guljas <janos@resenje.org>
Co-authored-by: Anton Evangelatov <anton.evangelatov@gmail.com>
Co-authored-by: Balint Gabor <balint.g@gmail.com>
2018-08-15 17:41:52 +02:00
Anton Evangelatov
97887d98da swarm/network, swarm/storage: validate chunk size (#17397)
* swarm/network, swarm/storage: validate default chunk size

* swarm/bmt, swarm/network, swarm/storage: update BMT hash initialisation

* swarm/bmt: move segmentCount to tests

* swarm/chunk: change chunk.DefaultSize to be untyped const

* swarm/storage: add size validator

* swarm/storage: add chunk size validation to localstore

* swarm/storage: move validation from localstore to validator

* swarm/storage: global chunk rules in MRU
2018-08-14 16:03:56 +02:00
Janoš Guljaš
6d1e292eef Manifest cli fix and upload defaultpath only once (#17375)
* cmd/swarm: fix manifest subcommands and add tests

* cmd/swarm: manifest update: update default entry for non-encrypted uploads

* swarm/api: upload defaultpath file only once

* swarm/api/client: improve UploadDirectory default path handling

* cmd/swarm: support absolute and relative default path values

* cmd/swarm: fix a typo in test

* cmd/swarm: check encrypted uploads in manifest update tests
2018-08-10 16:12:55 +02:00
Elad
3ec5dda4d2 swarm/api/http: added logging to denote request ended (#17371) 2018-08-10 13:49:37 +02:00
Janoš Guljaš
3bcb501c8f swarm/api: close tar writer in GetDirectoryTar to flush and clean (#17339) 2018-08-09 16:15:07 +02:00
Giulio M
a1eb9c7d13 swarm/api/http: fixed list leaf links (#17342) 2018-08-08 09:33:06 +02:00
Janoš Guljaš
00e6da9704 swarm/bmt: ignore data longer then 4096 bytes in Hasher.Write (#17338) 2018-08-07 15:34:33 +02:00
Attila Gazso
9df16f3468 swarm: Added lightnode flag (#17291)
* swarm: Added lightnode flag

Added --lightnode command line parameter
Added LightNode to Handshake message

* swarm/config: Fixed variable naming

* cmd/swarm: Changed BoolTFlag to BoolFlag for SwarmLightNodeEnabled

* swarm/network: Changed logging

* swarm/network: Changed protocol version testing

* swarm/network: Renamed DefaultNetworkID variable to TestProtocolNetworkID

* swarm/network: Bumped protocol version

* swarm/network: Changed LightNode handhsake test to table driven

* swarm/network: Changed back TestProtocolVersion to 5 for now

* swarm/network: Moved the test configuration inside the test function scope
2018-08-07 15:34:11 +02:00
Elad
4bb2dc3d09 swarm/api/http: test fixes (#17334) 2018-08-07 13:40:38 +02:00
Oleg Kovalov
cf05ef9106 p2p, swarm, trie: avoid copying slices in loops (#17265) 2018-08-07 13:56:40 +03:00
Anton Evangelatov
de9b0660ac swarm/README: add more sections to easily onboard developers (#17333) 2018-08-07 12:56:01 +02:00
Andrew Chiw
042191338d swarm/api/http: GET/PUT/PATCH/DELETE/POST multipart form unit tests. (#17277)
httpDo has a verbose option that dumps the HTTP request
2018-08-07 12:00:12 +02:00
Elad
93fe16b0a5 swarm/api/http: refactored http package (#17309) 2018-08-07 11:56:55 +02:00
Javier Peletier
64a4e89504 swarm/storage/mru: HOTFIX - fix panic in Handler.update (#17313) 2018-08-07 11:53:56 +02:00
Janoš Guljaš
a72ba5a55b cmd/swarm, swarm: various test fixes (#17299)
* swarm/network/simulation: increase the sleep duration for TestRun

* cmd/swarm, swarm: fix failing tests on mac

* cmd/swarm: update TestCLISwarmFs skip comment

* swarm/network/simulation: adjust disconnections on simulation close

* swarm/network/simulation: call cleanups after net shutdown
2018-08-06 12:33:22 +03:00
Péter Szilágyi
454382e81a
params, swarm/version: begin Geth v1.8.14, Swarm v0.3.2 cycle 2018-07-31 13:39:22 +03:00
Péter Szilágyi
225171a4bf
params, swarm/version: release Geth v1.8.13, Swarm 0.3.1 2018-07-31 13:35:53 +03:00
holisticode
d6efa69187 Merge netsim mig to master (#17241)
* swarm: merged stream-tests migration to develop

* swarm/network: expose simulation RandomUpNode to use in stream tests

* swarm/network: wait for subs in PeerEvents and fix stream.runSyncTest

* swarm: enforce waitkademlia for snapshot tests

* swarm: fixed syncer tests and snapshot_sync_test

* swarm: linting of simulation package

* swarm: address review comments

* swarm/network/stream: fix delivery_test bugs and refactor

* swarm/network/stream: addressed PR comments @janos

* swarm/network/stream: enforce waitKademlia, improve TestIntervals

* swarm/network/stream: TestIntervals not waiting for chunk to be stored
2018-07-30 22:55:25 +02:00
Anton Evangelatov
273c7a9dc4 swarm/api: remove ioutil.ReadAll for HandleGetFiles (#17276) 2018-07-30 12:19:26 +03:00
Anton Evangelatov
a5d5609e38 build: rename swarm deb package to ethereum-swarm; change swarm deb version from 1.8.x to 0.3.x (#16988)
* build: add support for different package and binary names

* build: bump up copyright date

* build: change default PackageName to empty string

* build, internal, swarm: enhance build/release process

* build: hack ethereum-swarm as a "depends" in deb package

* build/ci: remove redundant variables

* build, cmd, mobile, params, swarm: remove VERSION file; rename Version to VersionMeta;

* internal: remove VERSION() method which reads VERSION file

* build: fix VersionFilePath to Version

* Makefile: remove clean_go_build_cache.sh until it works

* Makefile: revert removal of clean_go_build_cache.sh
2018-07-30 11:56:40 +03:00
Viktor Trón
1064b9e691
Merge pull request #17233 from ethersphere/swarm-readme
swarm: README.md
2018-07-25 05:38:14 +02:00
Anton Evangelatov
14bdcdeab4 swarm/testutil: remove EnableMetrics 2018-07-23 18:45:39 +02:00
Javier Peletier
427316a707 swarm/storage/mru: Client-side MRU signatures (#784)
* swarm/storage/mru: Add embedded publickey and remove ENS dep

This commit breaks swarm, swarm/api...
but tests in swarm/storage/mru pass

* swarm: Refactor swarm, swarm/api to mru changes, make tests pass

* swarm/storage/mru: Remove self from recv, remove test ens vldtr

* swarm/storage/mru: Remove redundant test, expose ResourceHash mthd

* swarm/storage/mru: Make HeaderGetter mandatory + godoc fixes

* swarm/storage: Remove validator prefix for metadata chunk

* swarm/storage/mru: Use Address instead of PublicKey

* swarm/storage/mru: Change index from name to metadata chunk addr

* swarm/storage/mru: Refactor swarm/api/... to MRU index changes

* swarm/storage/mru: Refactor cleanup

* swarm/storage/mru: Rebase cleanup

* swarm: Use constructor for GenericSigner MRU in swarm.go

* swarm/storage: Change to BMTHash for MRU hashing

* swarm/storage: Reduce loglevel on chunk validator logs

* swarm/storage/mru: Delint

* swarm: MRU Rebase cleanup

* swarm/storage/mru: client-side mru signatures

Rebase to PR #668 and fix all conflicts

* swarm/storage/mru:  refactor and documentation

* swarm/resource/mru: error-checking  tests for parseUpdate/newUpdateChunk

* swarm/storage/mru: Added resourcemetadata tests

* swarm/storage/mru: Added tests  for UpdateRequest

* swarm/storage/mru: more test coverage for UpdateRequest and comments

* swarm/storage/mru: Avoid fake chunks in parseUpdate()

* swarm/storage/mru: Documented resource.go extensively

moved some functions where they make most sense

* swarm/storage/mru: increase test coverage for UpdateRequest and

variable name changes throughout to increase consistency

* swarm/storage/mru: moved default timestamp to NewCreateRequest-

* swarm/storage/mru: lookup refactor

* swarm/storage/mru: added comments and renamed raw flag to rawmru

* swarm/storage/mru: fix receiver typo

* swarm/storage/mru: refactored update chunk new/create

* swarm/storage/mru:  refactored signature digest to avoid malleability

* swarm/storage/mru: optimize update data serialization

* swarm/storage/mru: refactor and cleanup

* swarm/storage/mru: add timestamp struct and serialization

* swarm/storage/mru: fix lint error and mark some old code for deletion

* swarm/storage/mru: remove unnecessary variable

* swarm/storage/mru: Added more comments throughout

* swarm/storage/mru: Refactored metadata chunk layout + extensive error...

* swarm/storage/mru: refactor cli parser
Changed resource info output to JSON

* swarm/storage/mru: refactor serialization for extensibility

refactored error messages to NewErrorf

* swarm/storage/mru: Moved Signature to resource_sign.
Check Sign errors in server tests

* swarm/storage/mru: Remove isSafeName() checks

* swarm/storage/mru: scrubbed off all references to "block" for time

* swarm/storage/mru: removed superfluous isSynced() call.

* swarm/storage/mru: remove isMultihash() and ToSafeName functions

* swarm/storage/mru: various fixes and comments

* swarm/storage/mru: decoupled cli for independent create/update
* Made resource name optional
* Removed unused LookupPrevious

* swarm/storage/mru: Decoupled resource create / update & refactor

* swarm/storage/mru: Fixed some comments as per issues raised in PR #743

* swarm/storage/mru: Cosmetic changes as per #743 comments

* swarm/storage/mru: refct request encoder/decoder > marshal/unmarshal

* swarm/storage/mru: Cosmetic changes as per review in #748

* swarm/storage/mru: removed timestamp proof placeholder

* swarm/storage/mru: cosmetic/doc/fixes changes as per comments in #704

* swarm/storage/mru: removed unnecessary check in Handler.update

* swarm/storage/mru: Implemented Marshaler/Unmarshaler iface in Request

* swarm/storage/mru: Fixed linter error

* swarm/storage/mru: removed redundant address in signature digest

* swarm/storage/mru: fixed bug: LookupLatestVersionInPeriod not working

* swarm/storage/mru: Unfold Request creation API for create or update+create
set common time source for mru package

* swarm/api/http: fix HandleGetResource error variable shadowed
when requesting a resource that does not exist

* swarm/storage/mru: Add simple check to detect duplicate updates

* swarm/storage/mru: moved Multihash() to the right place.

* cmd/swarm: remove unneeded clientaccountmanager.go

* swarm/storage/mru: Changed some comments as per reviews in #784

* swarm/storage/mru: Made SignedResourceUpdate.GetDigest() public

* swarm/storage/mru: cosmetic changes as per comments in #784

* cmd/swarm: Inverted --multihash flag default

* swarm/storage/mru: removed Verify from SignedResourceUpdate.fromChunk

* swarm/storage/mru: Moved validation code out of serializer
Cosmetic / comment changes

* swarm/storage/mru: Added unit tests for UpdateLookup

* swarm/storage/mru: Increased coverage of metadata serialization

* swarm/storage/mru: Increased test coverage of updateHeader serializers

* swarm/storage/mru: Add resourceUpdate serializer test
2018-07-23 15:33:33 +02:00
Elad
0647c4de7b swarm/api/http: http package refactoring 1/5 and 2/5 2018-07-23 15:33:32 +02:00
Janoš Guljaš
dcaaa3c804 swarm: network simulation for swarm tests (#769)
* cmd/swarm: minor cli flag text adjustments

* cmd/swarm, swarm/storage, swarm: fix  mingw on windows test issues

* cmd/swarm: support for smoke tests on the production swarm cluster

* cmd/swarm/swarm-smoke: simplify cluster logic as per suggestion

* changed colour of landing page

* landing page reacts to enter keypress

* swarm/api/http: sticky footer for swarm landing page using flex

* swarm/api/http: sticky footer for error pages and fix for multiple choices

* swarm: propagate ctx to internal apis (#754)

* swarm/simnet: add basic node/service functions

* swarm/netsim: add buckets for global state and kademlia health check

* swarm/netsim: Use sync.Map as bucket and provide cleanup function for...

* swarm, swarm/netsim: adjust SwarmNetworkTest

* swarm/netsim: fix tests

* swarm: added visualization option to sim net redesign

* swarm/netsim: support multiple services per node

* swarm/netsim: remove redundant return statement

* swarm/netsim: add comments

* swarm: shutdown HTTP in Simulation.Close

* swarm: sim HTTP server timeout

* swarm/netsim: add more simulation methods and peer events examples

* swarm/netsim: add WaitKademlia example

* swarm/netsim: fix comments

* swarm/netsim: terminate peer events goroutines on simulation done

* swarm, swarm/netsim: naming updates

* swarm/netsim: return not healthy kademlias on WaitTillHealthy

* swarm: fix WaitTillHealthy call in testSwarmNetwork

* swarm/netsim: allow bucket to have any type for a key

* swarm: Added snapshots to new netsim

* swarm/netsim: add more tests for bucket

* swarm/netsim: move http related things into separate files

* swarm/netsim: add AddNodeWithService option

* swarm/netsim: add more tests and Start* methods

* swarm/netsim: add peer events and kademlia tests

* swarm/netsim: fix some tests flakiness

* swarm/netsim: improve random nodes selection, fix TestStartStop* tests

* swarm/netsim: remove time measurement from TestClose to avoid flakiness

* swarm/netsim: builder pattern for netsim HTTP server (#773)

* swarm/netsim: add connect related tests

* swarm/netsim: add comment for TestPeerEvents

* swarm: rename netsim package to network/simulation
2018-07-23 15:33:25 +02:00
lash
f5b128a5b3 swarm/fuse: Hotfix missing parantheses in statement 2018-07-23 15:31:02 +02:00
Viktor Trón
fd982d3f3b swarm/bmt: async section writer interface to BMT (#778)
- AsyncHasher implements AsyncWriter interface
 - add extra level for zerohashes in pool to lookup empty data hash
 - remove unused segment, hash and depth fields from Tree
 - Hash pkg function -> syncHash moved to test
 - add asyncHash helper func to tests using shuffle
 - add TestAsyncCorrectness to tests
 - add BenchmarkBMTAsync to tests
 - refactor benchmarks using subbenchmarks
 - improved comments
 - preinitialise base hashers on the nodes
2018-07-23 15:09:25 +02:00
Anton Evangelatov
4ea2d707f9 gitter: change ethereum/swarm to ethersphere/orange-lounge 2018-07-21 00:47:46 +02:00
Anton Evangelatov
ee8877509a swarm/readme: add link to code review guidelines 2018-07-19 13:27:31 +02:00
Anton Evangelatov
763e64cad8 swarm: readme.md 2018-07-19 12:13:03 +02:00
Anton Evangelatov
d483da766f swarm/network: bump up protocol versions due to wrapped message intro (#17170) 2018-07-13 17:40:55 +02:00
Anton Evangelatov
7c9314f231 swarm: integrate OpenTracing; propagate ctx to internal APIs (#17169)
* swarm: propagate ctx, enable opentracing

* swarm/tracing: log error when tracing is misconfigured
2018-07-13 17:40:28 +02:00
Anton Evangelatov
f7d3678c28 swarm/api/http: http package refactoring 1/5 and 2/5 (#17164) 2018-07-12 15:42:45 +02:00
Anton Evangelatov
b3711af051 swarm: ctx propagation; bmt fixes; pss generic notification framework (#17150)
* cmd/swarm: minor cli flag text adjustments

* swarm/api/http: sticky footer for swarm landing page using flex

* swarm/api/http: sticky footer for error pages and fix for multiple choices

* cmd/swarm, swarm/storage, swarm: fix  mingw on windows test issues

* cmd/swarm: update description of swarm cmd

* swarm: added network ID test

* cmd/swarm: support for smoke tests on the production swarm cluster

* cmd/swarm/swarm-smoke: simplify cluster logic as per suggestion

* swarm: propagate ctx to internal apis (#754)

* swarm/metrics: collect disk measurements

* swarm/bmt: fix io.Writer interface

  * Write now tolerates arbitrary variable buffers
  * added variable buffer tests
  * Write loop and finalise optimisation
  * refactor / rename
  * add tests for empty input

* swarm/pss: (UPDATE) Generic notifications package (#744)

swarm/pss: Generic package for creating pss notification svcs

* swarm: Adding context to more functions

* swarm/api: change colour of landing page in templates

* swarm/api: change landing page to react to enter keypress
2018-07-09 14:11:49 +02:00
lash
1f0f6f0272 swarm/pss: Hide big network tests under longrunning flag (#17074) 2018-06-25 16:11:47 +02:00
Balint Gabor
0a22ae5572
swarm/fuse: Disable fuse tests, they are flaky (#17072) 2018-06-25 14:04:01 +02:00
ethersphere
e187711c65 swarm: network rewrite merge 2018-06-21 21:10:31 +02:00
Felix Lange
0255951587 crypto: replace ToECDSAPub with error-checking func UnmarshalPubkey (#16932)
ToECDSAPub was unsafe because it returned a non-nil key with nil X, Y in
case of invalid input. This change replaces ToECDSAPub with
UnmarshalPubkey across the codebase.
2018-06-12 15:26:08 +02:00
Wuxiang
8f8774cf6d all: fix various typos (#16533)
* fix typo

* fix typo

* fix typo
2018-04-19 16:32:02 +03:00
thomasmodeneis
ba1030b6b8 build: enable goimports and varcheck linters (#16446) 2018-04-18 00:53:50 +02:00
Felix Lange
0fac705ed0
compression/rle: delete RLE compression (#16468) 2018-04-09 13:47:06 +02:00
Anton Evangelatov
746392cfd2 swarm/storage: disable treechunker test while it is flaky (#16254) 2018-03-05 17:23:33 +01:00
Elad Nachmias
b574b57766 swarm: give correct error on 0x hash prefix (#16195)
- added a case error struct that contains information about certain error cases
in which we would like to output more information to the client
- added a validation method that iterates and adds the information that is
stored in the error cases
2018-02-27 15:32:38 +02:00
cooganb
22fc6928d7 swarm: creates Swarm landing page for browser 'localhost:xxxx/' GET request when running Swarm (#15926)
* swarm: began work on GetHandleFile method re: issue #155

* swarm: now able to serve landing page template

* swarm: added landing page template

* swarm: landing page has working input

* swarm: fixed CSS issue in template

* swarm: deleted extra lines

* swarm: deleted time header and made redirect a relative path

* swarm: removed code mistakenly left
2018-02-26 09:56:40 +01:00
Anton Evangelatov
114738982e swarm/metrics: introduce metrics export flag (#16177) 2018-02-23 16:22:16 +01:00
Anton Evangelatov
dcca613a0b swarm: initial instrumentation (#15969)
* swarm: initial instrumentation with go-metrics

* swarm: initialise metrics collection and add ResettingTimer to HTTP requests

* swarm: update metrics flags names. remove redundant Timer.

* swarm: rename method for periodically updating gauges

* swarm: finalise metrics after feedback

* swarm/network: always init kad metrics containers

* swarm/network: off-by-one index in metrics containers

* swarm, metrics: resolved conflicts
2018-02-23 14:19:59 +01:00
Lewis Marshall
b677a07d36 swarm/api/http: Fix using deprecated bzzr scheme (#16152)
Without this, deprecated bzzr requests just return an empty response.

Signed-off-by: Lewis Marshall <lewis@lmars.net>
2018-02-23 14:09:01 +01:00
Janos Guljas
6a9730edaa swarm, cmd/swarm: Merge branch 'master' into multiple-ens-endpoints 2018-02-22 18:51:34 +01:00
Janos Guljas
a3a07350dc swarm, cmd/swarm: Merge branch 'master' into multiple-ens-endpoints 2018-02-22 14:23:17 +01:00
cooganb
4e61ed02e2 swarm: add favicon for Swarm templates served by browser (#15958)
* swarm: added script to HTML header to create favicon addresses #153

* swarm: moved data blob direclty into link tag, removed script

* swarm: added favicon info to other html templates

* swarm: fixing test errors

* swarm: fixing favicon test

* swarm: fixing travis tests

* swarm: added script to HTML header to create favicon addresses #153

* swarm: moved data blob direclty into link tag, removed script

* swarm: added favicon info to other html templates

* swarm: fixing test errors

* swarm: fixing favicon test

* swarm: fixing travis tests
2018-02-15 16:24:20 +02:00
Felix Lange
a5c0bbb4f4
all: update license information (#16089) 2018-02-14 13:49:11 +01:00
Furkan KAMACI
b8caba9709 various: remove redundant parentheses (#15793) 2018-01-03 14:14:47 +02:00
Janoš Guljaš
542d51895f swarm/api: url scheme bzz-hash to get hashes of swarm content (#15238) (#15715)
* swarm/api: url scheme bzz-hash to get hashes of swarm content (#15238)

Update URI to support bzz-hash scheme and handle such HTTP requests by
responding with hash of the content as a text/plain response.

* swarm/api: return hash of the content for bzz-hash:// requests

* swarm/api: revert "return hash of the content for bzz-hash:// requests"

Return hashes of the content that would be returned by bzz-raw
request.

* swarm/api/http: handle error in TestBzzGetPath

* swarm/api: remove extra blank line in comment
2017-12-21 14:47:10 +02:00
Janoš Guljaš
c786f75389 swarm: bzz-list, bzz-raw and bzz-immutable schemes (#15667)
* swarm/api: url scheme bzz-list for getting list of files from manifest

Replace query parameter list=true for listing all files contained
in a swarm manifest with a new URL scheme bzz-list.

* swarm: replaace bzzr and bzzi schemes with bzz-raw and bzz-immutable

New URI Shemes are added and old ones are deprecated, but not removed.
Old Schemes bzzr and bzzi are functional for backward compatibility.

* swarm/api: completely remove bzzr and bzzi schemes

Remove old schemes in favour of bzz-raw and
bzz-immutable.

* swarm/api: revert "completely remove bzzr and bzzi schemes"

Keep bzzr and bzzi schemes for backward compatibility. At least
until 0.3 swarm release.
2017-12-19 10:49:30 +02:00
Janos Guljas
0d6a735a72 swarm/api: implement NoResolverError with information about TLD
MultiResolver needs to provide information about TLD that has
no resolver configured for.
2017-12-18 23:07:48 +01:00
Janos Guljas
c0a4d9e1e6 cmd/swarm, swarm: disable ENS API by default
Specifying ENS API CLI flag, env variable or configuration
field is required for ENS resolving. Backward compatibility is
preserved with --ens-api="" CLI flag value.
2017-12-18 16:22:39 +01:00
Janos Guljas
47a8014559 cmd/swarm: Merge branch 'master' into multiple-ens-endpoints
Fix a conflict in cmd/swarm envVarsOverride function.
2017-12-14 10:36:12 +01:00
Janos Guljas
19982f9467 swarm, cmd/swarm: Merge branch 'master' into multiple-ens-endpoints
Merge with changes that implement config file PR #15548.

Field *EnsApi string* in swarm/api.Config is replaced with
*EnsAPIs []string*.

A new field *EnsDisabled bool* is added to swarm/api.Config for
easy way to disable ENS resolving with config file.

Signature of function swarm.NewSwarm is changed and simplified.
2017-12-13 10:40:39 +01:00
Zach
3da1bf8ca1 all: use gometalinter.v2, fix new gosimple issues (#15650) 2017-12-12 19:05:47 +01:00
holisticode
32516c768e cmd/swarm: add config file (#15548)
This commit adds a TOML configuration option to swarm. It reuses
the TOML configuration structure used in geth with swarm
customized items.

The commit:

* Adds a "dumpconfig" command to the swarm executable which
  allows printing the (default) configuration to stdout, which
  then can be redirected to a file in order to customize it.
* Adds a "--config <file>" option to the swarm executable which will
  allow to load a configuration file in TOML format from the
  specified location in order to initialize the Swarm node The
  override priorities are like follows: environment variables
  override command line arguments override config file override
  default config.
2017-12-11 22:56:06 +01:00
Janos Guljas
1dc19de5da swarm/api: use path.Ext instead strings.LastIndex in MultiResolver.Resolve 2017-12-04 22:56:52 +01:00
Janos Guljas
3732c15faa swarm/api: remove unneeded blank assignement 2017-12-04 22:32:33 +01:00
Janos Guljas
a758b5cf7a swarm/api: initialize map with make to comply with the convention 2017-12-04 22:31:00 +01:00
Janos Guljas
34edbc8868 swarm/api: remove unneeded assignment in MultiResolverOptionWithResolver 2017-12-04 22:29:37 +01:00
ferhat elmas
1d06e41f04 p2p, swarm/network/kademlia: use IsZero to check for zero time (#15603) 2017-12-04 11:07:10 +01:00
Janos Guljas
057af8c5c8 swarm: add CLI --ens-endpoint flag (#14386)
Implement a CLI flag that can be repeated to allow multiple ENS
resolvers for different TLDs.
2017-12-01 11:25:50 +01:00
Péter Szilágyi
f0ac925fa7
Merge pull request #15329 from holisticode/exact-match-fix
swarm/api: bug fix exact match for manifest
2017-11-24 16:23:37 +02:00
ferhat elmas
86f6568f66 build: enable unconvert linter (#15456)
* build: enable unconvert linter

 - fixes #15453
 - update code base for failing cases

* cmd/puppeth: replace syscall.Stdin with os.Stdin.Fd() for unconvert linter
2017-11-10 19:06:45 +02:00
Fabio Barone
03ec3fed2b swarm/api: bug fix exact match for manifest 2017-11-09 10:38:42 -05:00
ferhat elmas
9619a61024 all: gofmt -w -s (#15419) 2017-11-08 11:45:52 +01:00
ferhat elmas
07e8c177e7 core, swarm: typo fixes 2017-10-30 01:23:23 +01:00
Darrel Herbst
89860f4197 swarm/fuse: return amount of data written if the file exists (#15261)
If the file already existed, the WriteResponse.Size was being set
as the length of the entire file, not just the amount that was
written to the existing file.

Fixes #15216
2017-10-09 12:45:30 +02:00
holisticode
1ae0411d41 swarm/api: fixed 404 handling on missing default entry (#15139) 2017-10-06 15:45:54 +02:00
Zahoor Mohamed
d558a595ad swarm/storage: pyramid chunker re-write (#14382) 2017-09-21 22:22:51 +02:00
holisticode
ac193e36ce swarm/api/http: add error pages (#14967) 2017-09-08 20:29:09 +02:00
Fiisio
2dcb22afec swarm/fuse: use Equal instead of Compare (#15097) 2017-09-07 23:10:51 +02:00
Viktor Trón
2bacf36d80 bmt: Binary Merkle Tree Hash (#14334)
bmt is a new package that provides hashers for binary merkle tree hashes on
size-limited chunks. the main motivation is that using BMT hash as the chunk
hash of the swarm hash offers logsize inclusion proofs for arbitrary files on a
32-byte resolution completely viable to use in challenges on the blockchain.
2017-09-05 12:38:36 +02:00
Egon Elbre
133de3d806 swarm: fix megacheck warnings 2017-08-14 18:12:37 +03:00
Lewis Marshall
60c858a529 swarm/api: make api.NewManifest synchronous (#14880)
Previously, NewManifest was asynchronous so subsequent code which tried
to use the returned manifest could error as the manifest was not yet
persisted.
2017-07-31 15:58:19 +02:00
Lewis Marshall
e9b850805e cmd/swarm: support exporting, importing chunk db (#14868) 2017-07-31 13:23:44 +02:00
Lewis Marshall
f4841ff43d swarm/api/http: redirect root manifest requests to include trailing slash (#14806) 2017-07-25 11:51:26 +02:00
Lewis Marshall
c7041fe145 cmd/swarm: Exit if --ethapi is set
The previous attempt to use --ethapi as a fallback if --ens-api is not
set does not work because --ens-api has a default value, and also
setting --ens-api to "" is the suggested way to disable ENS lookups.

Signed-off-by: Lewis Marshall <lewis@lmars.net>
2017-06-30 10:16:03 +01:00
Lewis Marshall
f3359d5e58 cmd/swarm: Support using Mainnet for resolving ENS names
Signed-off-by: Lewis Marshall <lewis@lmars.net>
2017-06-26 12:51:33 +02:00
aron
caa00b7e6d swarm/storage: remove panic on invalid chunk 2017-06-25 16:10:54 +02:00
Felix Lange
98e101ef8e swarm/fuse: use subtests 2017-06-21 09:56:58 +02:00
Felix Lange
50c18e6eb8 swarm/fuse: simplify externalUnmount
The code looked for /usr/bin/diskutil on darwin, but it's actually
located in /usr/sbin. Fix that by not specifying the absolute path.
Also remove weird timeout construction and extra whitespace.
2017-06-21 09:56:58 +02:00
Lewis Marshall
0036e2a747 swarm/dev: add development environment (#14332)
This PR adds a Swarm development environment which can be run in a
Docker container and provides scripts for building binaries and running
Swarm clusters.
2017-06-01 12:52:18 +02:00
Péter Szilágyi
aa73420207
accounts/keystore, crypto: enforce 256 bit keys on import 2017-05-23 14:58:03 +03:00
Lewis Marshall
2a41e76b39 swarm/api: Fix adding paths which exist as manifests (#14482)
Signed-off-by: Lewis Marshall <lewis@lmars.net>
2017-05-22 08:57:03 +02:00
Lewis Marshall
4a2c17b1ab cmd/swarm: Add --httpaddr flag (#14475)
Fixes #14474.

Signed-off-by: Lewis Marshall <lewis@lmars.net>
2017-05-22 08:56:40 +02:00
Lewis Marshall
b0d0fafd68 swarm/api: fix error reporting in api.Resolve (#14464)
Previously, resolve errors were being swallowed and the returned error
was a generic "not a content hash" which isn't helpful.

This updates the Resolve function to fail fast rather than only
returning an error at the end, and also adds test coverage.
2017-05-13 02:02:25 +02:00
Felix Lange
0cc492f815 all: update license information 2017-04-14 10:29:00 +02:00
holisticode
409b61fe3c swarm/api: better name resolver handling (#3754)
Fixes #3608
2017-04-13 11:06:19 +02:00