Commit Graph

395 Commits

Author SHA1 Message Date
Anton Evangelatov
114738982e swarm/metrics: introduce metrics export flag (#16177) 2018-02-23 16:22:16 +01:00
Anton Evangelatov
dcca613a0b swarm: initial instrumentation (#15969)
* swarm: initial instrumentation with go-metrics

* swarm: initialise metrics collection and add ResettingTimer to HTTP requests

* swarm: update metrics flags names. remove redundant Timer.

* swarm: rename method for periodically updating gauges

* swarm: finalise metrics after feedback

* swarm/network: always init kad metrics containers

* swarm/network: off-by-one index in metrics containers

* swarm, metrics: resolved conflicts
2018-02-23 14:19:59 +01:00
Lewis Marshall
b677a07d36 swarm/api/http: Fix using deprecated bzzr scheme (#16152)
Without this, deprecated bzzr requests just return an empty response.

Signed-off-by: Lewis Marshall <lewis@lmars.net>
2018-02-23 14:09:01 +01:00
Janos Guljas
6a9730edaa swarm, cmd/swarm: Merge branch 'master' into multiple-ens-endpoints 2018-02-22 18:51:34 +01:00
Janos Guljas
a3a07350dc swarm, cmd/swarm: Merge branch 'master' into multiple-ens-endpoints 2018-02-22 14:23:17 +01:00
cooganb
4e61ed02e2 swarm: add favicon for Swarm templates served by browser (#15958)
* swarm: added script to HTML header to create favicon addresses #153

* swarm: moved data blob direclty into link tag, removed script

* swarm: added favicon info to other html templates

* swarm: fixing test errors

* swarm: fixing favicon test

* swarm: fixing travis tests

* swarm: added script to HTML header to create favicon addresses #153

* swarm: moved data blob direclty into link tag, removed script

* swarm: added favicon info to other html templates

* swarm: fixing test errors

* swarm: fixing favicon test

* swarm: fixing travis tests
2018-02-15 16:24:20 +02:00
Felix Lange
a5c0bbb4f4
all: update license information (#16089) 2018-02-14 13:49:11 +01:00
Furkan KAMACI
b8caba9709 various: remove redundant parentheses (#15793) 2018-01-03 14:14:47 +02:00
Janoš Guljaš
542d51895f swarm/api: url scheme bzz-hash to get hashes of swarm content (#15238) (#15715)
* swarm/api: url scheme bzz-hash to get hashes of swarm content (#15238)

Update URI to support bzz-hash scheme and handle such HTTP requests by
responding with hash of the content as a text/plain response.

* swarm/api: return hash of the content for bzz-hash:// requests

* swarm/api: revert "return hash of the content for bzz-hash:// requests"

Return hashes of the content that would be returned by bzz-raw
request.

* swarm/api/http: handle error in TestBzzGetPath

* swarm/api: remove extra blank line in comment
2017-12-21 14:47:10 +02:00
Janoš Guljaš
c786f75389 swarm: bzz-list, bzz-raw and bzz-immutable schemes (#15667)
* swarm/api: url scheme bzz-list for getting list of files from manifest

Replace query parameter list=true for listing all files contained
in a swarm manifest with a new URL scheme bzz-list.

* swarm: replaace bzzr and bzzi schemes with bzz-raw and bzz-immutable

New URI Shemes are added and old ones are deprecated, but not removed.
Old Schemes bzzr and bzzi are functional for backward compatibility.

* swarm/api: completely remove bzzr and bzzi schemes

Remove old schemes in favour of bzz-raw and
bzz-immutable.

* swarm/api: revert "completely remove bzzr and bzzi schemes"

Keep bzzr and bzzi schemes for backward compatibility. At least
until 0.3 swarm release.
2017-12-19 10:49:30 +02:00
Janos Guljas
0d6a735a72 swarm/api: implement NoResolverError with information about TLD
MultiResolver needs to provide information about TLD that has
no resolver configured for.
2017-12-18 23:07:48 +01:00
Janos Guljas
c0a4d9e1e6 cmd/swarm, swarm: disable ENS API by default
Specifying ENS API CLI flag, env variable or configuration
field is required for ENS resolving. Backward compatibility is
preserved with --ens-api="" CLI flag value.
2017-12-18 16:22:39 +01:00
Janos Guljas
47a8014559 cmd/swarm: Merge branch 'master' into multiple-ens-endpoints
Fix a conflict in cmd/swarm envVarsOverride function.
2017-12-14 10:36:12 +01:00
Janos Guljas
19982f9467 swarm, cmd/swarm: Merge branch 'master' into multiple-ens-endpoints
Merge with changes that implement config file PR #15548.

Field *EnsApi string* in swarm/api.Config is replaced with
*EnsAPIs []string*.

A new field *EnsDisabled bool* is added to swarm/api.Config for
easy way to disable ENS resolving with config file.

Signature of function swarm.NewSwarm is changed and simplified.
2017-12-13 10:40:39 +01:00
Zach
3da1bf8ca1 all: use gometalinter.v2, fix new gosimple issues (#15650) 2017-12-12 19:05:47 +01:00
holisticode
32516c768e cmd/swarm: add config file (#15548)
This commit adds a TOML configuration option to swarm. It reuses
the TOML configuration structure used in geth with swarm
customized items.

The commit:

* Adds a "dumpconfig" command to the swarm executable which
  allows printing the (default) configuration to stdout, which
  then can be redirected to a file in order to customize it.
* Adds a "--config <file>" option to the swarm executable which will
  allow to load a configuration file in TOML format from the
  specified location in order to initialize the Swarm node The
  override priorities are like follows: environment variables
  override command line arguments override config file override
  default config.
2017-12-11 22:56:06 +01:00
Janos Guljas
1dc19de5da swarm/api: use path.Ext instead strings.LastIndex in MultiResolver.Resolve 2017-12-04 22:56:52 +01:00
Janos Guljas
3732c15faa swarm/api: remove unneeded blank assignement 2017-12-04 22:32:33 +01:00
Janos Guljas
a758b5cf7a swarm/api: initialize map with make to comply with the convention 2017-12-04 22:31:00 +01:00
Janos Guljas
34edbc8868 swarm/api: remove unneeded assignment in MultiResolverOptionWithResolver 2017-12-04 22:29:37 +01:00
ferhat elmas
1d06e41f04 p2p, swarm/network/kademlia: use IsZero to check for zero time (#15603) 2017-12-04 11:07:10 +01:00
Janos Guljas
057af8c5c8 swarm: add CLI --ens-endpoint flag (#14386)
Implement a CLI flag that can be repeated to allow multiple ENS
resolvers for different TLDs.
2017-12-01 11:25:50 +01:00
Péter Szilágyi
f0ac925fa7
Merge pull request #15329 from holisticode/exact-match-fix
swarm/api: bug fix exact match for manifest
2017-11-24 16:23:37 +02:00
ferhat elmas
86f6568f66 build: enable unconvert linter (#15456)
* build: enable unconvert linter

 - fixes #15453
 - update code base for failing cases

* cmd/puppeth: replace syscall.Stdin with os.Stdin.Fd() for unconvert linter
2017-11-10 19:06:45 +02:00
Fabio Barone
03ec3fed2b swarm/api: bug fix exact match for manifest 2017-11-09 10:38:42 -05:00
ferhat elmas
9619a61024 all: gofmt -w -s (#15419) 2017-11-08 11:45:52 +01:00
ferhat elmas
07e8c177e7 core, swarm: typo fixes 2017-10-30 01:23:23 +01:00
Darrel Herbst
89860f4197 swarm/fuse: return amount of data written if the file exists (#15261)
If the file already existed, the WriteResponse.Size was being set
as the length of the entire file, not just the amount that was
written to the existing file.

Fixes #15216
2017-10-09 12:45:30 +02:00
holisticode
1ae0411d41 swarm/api: fixed 404 handling on missing default entry (#15139) 2017-10-06 15:45:54 +02:00
Zahoor Mohamed
d558a595ad swarm/storage: pyramid chunker re-write (#14382) 2017-09-21 22:22:51 +02:00
holisticode
ac193e36ce swarm/api/http: add error pages (#14967) 2017-09-08 20:29:09 +02:00
Fiisio
2dcb22afec swarm/fuse: use Equal instead of Compare (#15097) 2017-09-07 23:10:51 +02:00
Viktor Trón
2bacf36d80 bmt: Binary Merkle Tree Hash (#14334)
bmt is a new package that provides hashers for binary merkle tree hashes on
size-limited chunks. the main motivation is that using BMT hash as the chunk
hash of the swarm hash offers logsize inclusion proofs for arbitrary files on a
32-byte resolution completely viable to use in challenges on the blockchain.
2017-09-05 12:38:36 +02:00
Egon Elbre
133de3d806 swarm: fix megacheck warnings 2017-08-14 18:12:37 +03:00
Lewis Marshall
60c858a529 swarm/api: make api.NewManifest synchronous (#14880)
Previously, NewManifest was asynchronous so subsequent code which tried
to use the returned manifest could error as the manifest was not yet
persisted.
2017-07-31 15:58:19 +02:00
Lewis Marshall
e9b850805e cmd/swarm: support exporting, importing chunk db (#14868) 2017-07-31 13:23:44 +02:00
Lewis Marshall
f4841ff43d swarm/api/http: redirect root manifest requests to include trailing slash (#14806) 2017-07-25 11:51:26 +02:00
Lewis Marshall
c7041fe145 cmd/swarm: Exit if --ethapi is set
The previous attempt to use --ethapi as a fallback if --ens-api is not
set does not work because --ens-api has a default value, and also
setting --ens-api to "" is the suggested way to disable ENS lookups.

Signed-off-by: Lewis Marshall <lewis@lmars.net>
2017-06-30 10:16:03 +01:00
Lewis Marshall
f3359d5e58 cmd/swarm: Support using Mainnet for resolving ENS names
Signed-off-by: Lewis Marshall <lewis@lmars.net>
2017-06-26 12:51:33 +02:00
aron
caa00b7e6d swarm/storage: remove panic on invalid chunk 2017-06-25 16:10:54 +02:00
Felix Lange
98e101ef8e swarm/fuse: use subtests 2017-06-21 09:56:58 +02:00
Felix Lange
50c18e6eb8 swarm/fuse: simplify externalUnmount
The code looked for /usr/bin/diskutil on darwin, but it's actually
located in /usr/sbin. Fix that by not specifying the absolute path.
Also remove weird timeout construction and extra whitespace.
2017-06-21 09:56:58 +02:00
Lewis Marshall
0036e2a747 swarm/dev: add development environment (#14332)
This PR adds a Swarm development environment which can be run in a
Docker container and provides scripts for building binaries and running
Swarm clusters.
2017-06-01 12:52:18 +02:00
Péter Szilágyi
aa73420207
accounts/keystore, crypto: enforce 256 bit keys on import 2017-05-23 14:58:03 +03:00
Lewis Marshall
2a41e76b39 swarm/api: Fix adding paths which exist as manifests (#14482)
Signed-off-by: Lewis Marshall <lewis@lmars.net>
2017-05-22 08:57:03 +02:00
Lewis Marshall
4a2c17b1ab cmd/swarm: Add --httpaddr flag (#14475)
Fixes #14474.

Signed-off-by: Lewis Marshall <lewis@lmars.net>
2017-05-22 08:56:40 +02:00
Lewis Marshall
b0d0fafd68 swarm/api: fix error reporting in api.Resolve (#14464)
Previously, resolve errors were being swallowed and the returned error
was a generic "not a content hash" which isn't helpful.

This updates the Resolve function to fail fast rather than only
returning an error at the end, and also adds test coverage.
2017-05-13 02:02:25 +02:00
Felix Lange
0cc492f815 all: update license information 2017-04-14 10:29:00 +02:00
holisticode
409b61fe3c swarm/api: better name resolver handling (#3754)
Fixes #3608
2017-04-13 11:06:19 +02:00
Zahoor Mohamed
1d1d988aa7 swarm/api: FUSE read-write support (#13872)
- Moved fuse related code in a new package, swarm/fuse
- Added write support
  - Create new files
  - Delete existing files
  - Append to files (with limitations)
- More test coverage
2017-04-12 02:06:02 +02:00
Lewis Marshall
71fdaa4238 swarm/api: refactor and improve HTTP API (#3773)
This PR deprecates the file related RPC calls in favour of an improved HTTP API.

The main aim is to expose a simple to use API which can be consumed by thin
clients (e.g. curl and HTML forms) without the need for complex logic (e.g.
manipulating prefix trie manifests).
2017-04-07 00:22:22 +02:00
Louis Holbrook
0ec1104ba9 cmd/swarm: allow uploading from stdin (#3744)
- intended to be a swarm alternative to termbin.com
- added --stdin flag to swarm executable. if set, swarm will
  read data from stdin and postRaw it.
2017-04-06 14:21:16 +02:00
Lewis Marshall
b319f027a0 cmd/swarm, swarm/api/client: add HTTP API client and 'swarm ls' command (#3742)
This adds a swarm ls command which lists files and directories stored in a
manifest. Rather than listing all files, it uses "directory prefixes" in case there are a
lot of files in a manifest but you just want to traverse it.

This also includes some refactoring to the tests and the introduction of a
swarm/api/client package to make things easier to test.
2017-04-05 00:20:07 +02:00
Felix Lange
105b37f1b4 swarm/api: improve FUSE build constraints, logging and APIs (#3818)
* swarm/api: fix build/tests on unsupported platforms

Skip FUSE tests if FUSE is unavailable and change build constraints so
the 'lesser' platforms aren't mentioned explicitly. The test are
compiled on all platforms to prevent regressions in _fallback.go

Also gofmt -w -s because why not.

* internal/web3ext: fix swarmfs wrappers

Remove inputFormatter specifications so users get an error
when passing the wrong number of arguments.

* swarm/api: improve FUSE-related logging and APIs

The API now returns JSON objects instead of strings.
Log messages for invalid arguments are removed.
2017-03-31 12:11:01 +02:00
Zahoor Mohamed
11e7a712f4 swarm/api: support mounting manifests via FUSE (#3690) 2017-03-23 14:56:06 +01:00
Felix Lange
c213fd1fd8 all: import "context" instead of "golang.org/x/net/context"
There is no need to depend on the old context package now that the
minimum Go version is 1.7. The move to "context" eliminates our weird
vendoring setup. Some vendored code still uses golang.org/x/net/context
and it is now vendored in the normal way.

This change triggered new vet checks around context.WithTimeout which
didn't fire with golang.org/x/net/context.
2017-03-22 20:49:15 +01:00
holisticode
d97dea51ec swarm/network/kademlia: set kademlia log output to debug instead of warn (#3787) 2017-03-18 01:45:00 +01:00
Micah Zoltu
4e4e5fca54 rpc: add support for extended headers in CORS requests (#3783)
Fixes #3762.  Details about parameter:
a62a804a8a/cors.go (L50-L54)
2017-03-16 04:34:08 +01:00
Péter Szilágyi
7e3762fdc6
errs: kill it with fire 2017-02-28 18:01:54 +02:00
Felix Lange
5f7826270c all: unify big.Int zero checks, use common/math in more places (#3716)
* common/math: optimize PaddedBigBytes, use it more

name              old time/op    new time/op    delta
PaddedBigBytes-8    71.1ns ± 5%    46.1ns ± 1%  -35.15%  (p=0.000 n=20+19)

name              old alloc/op   new alloc/op   delta
PaddedBigBytes-8     48.0B ± 0%     32.0B ± 0%  -33.33%  (p=0.000 n=20+20)

* all: unify big.Int zero checks

Various checks were in use. This commit replaces them all with Int.Sign,
which is cheaper and less code.

eg templates:

    func before(x *big.Int) bool { return x.BitLen() == 0 }
    func after(x *big.Int) bool  { return x.Sign() == 0 }

    func before(x *big.Int) bool { return x.BitLen() > 0 }
    func after(x *big.Int) bool  { return x.Sign() != 0 }

    func before(x *big.Int) int { return x.Cmp(common.Big0) }
    func after(x *big.Int) int  { return x.Sign() }

* common/math, crypto/secp256k1: make ReadBits public in package math
2017-02-28 15:09:11 +01:00
Felix Lange
d0eba23af3 all: disable log message colors outside of geth
Also tweak behaviour so colors are only enabled when stderr is a terminal.
2017-02-27 15:33:12 +01:00
Péter Szilágyi
61e6bb1247
eth, les, swarm: fix go vet issues sufraced by log15 2017-02-23 12:16:45 +02:00
Péter Szilágyi
d4fd06c3dc
all: blidly swap out glog to our log15, logs need rework 2017-02-23 12:16:44 +02:00
Péter Szilágyi
3f923f3902
swarm: remove superfluous line break in log statements 2017-02-23 12:00:04 +02:00
Péter Szilágyi
e94dfb78f8
swarm/api/http: fix go vet issue on Go 1.8 2017-02-17 11:48:26 +02:00
Viktor Trón
e23e86921b swarm/network: fix chunk integrity checks (#3665)
* swarm/network: integrity on incoming known chunks
* swarm/network: fix integrity check for incoming chunks
* swarm/storage: imrpoved integrity checking on chunks
* dbstore panics on corrupt chunk entry an prompts user to run cleandb
* memstore adds logging for garbage collection
* dbstore refactor item delete. correct partial deletes in Get
* cmd/swarm: added cleandb subcommand
2017-02-13 13:20:50 +01:00
Maksim
6dd27e7cff swarm/storage: release chunk storage after stop swarm (#3651)
closes #3650
2017-02-08 18:01:12 +01:00
Péter Szilágyi
e5a93bf99a swarm/api/http: add missing copyright header 2017-01-30 15:21:46 +02:00
nolash
7669c5b5ec cmd/swarm, swarm/api: bzzr improve + networkid prio
fixes #3444
fixes #3494
networkid override

Added comments to explain why test against 0 appears twice

* Command line overrides saved config, saved config overrides system default

---

fixes #3476
bzzr get with path

Finally a hopefully clean commit for this PR
Added check for empty path to avoid SIGSEGV in path parser and resolver
Added requested tests for empty path and non-existing manifest.
However signature for StartHTTPServer had changed.
Now it's hacked as so:

	StartHttpServer(api.API, &Server{Addr: "127.0.0.1:8504", CorsString: ""})

* Parse url before resolve when path and ENS is supplied, example
* swarm/api/http proxy server test for retrieval of subpath through get
* Removed nil entry assignment on subtrie leaf in recursive key retrieval
* Cleaned up path-or-no-path condition in proxy server get handler
* swarm: processed with gofmt refers to lash/go-ethereum@90daa7a
* swarm: Added public access method Parse alias to parse
* swarm: processed with gofmt References nolash/go-ethereum@2ec3fd7
* Rename parse to Parse, removed alias
2017-01-27 08:18:13 +01:00
Bas van Kervel
54a65e6d87
cmd,eth,les,internal: remove natspec support 2017-01-17 12:13:50 +01:00
Felix Lange
b9b3efb09f all: fix ineffectual assignments and remove uses of crypto.Sha3
go get github.com/gordonklaus/ineffassign
ineffassign .
2017-01-09 16:24:42 +01:00
Péter Szilágyi
18c77744ff
all: fix spelling errors 2017-01-06 19:44:35 +02:00
Felix Lange
13e3b2f433 logger, pow/dagger, pow/ezp: delete dead code 2017-01-06 18:18:07 +01:00
Felix Lange
f2da6581ba all: fix issues reported by honnef.co/go/simple/cmd/gosimple 2017-01-06 18:18:07 +01:00
Felix Lange
35a7dcb162 all: gofmt -w -s 2017-01-06 15:52:03 +01:00
Maran
f087633efd swarm/api/http: add support for CORS headers (#3388) 2017-01-05 11:57:41 +01:00
Aron Fischer
9a51f5c350 swarm/http: check error returned by reader.Size (#3470) 2016-12-22 00:35:05 +01:00
Aron Fischer
301c0a6303 swarm/storage: call size before seek-from-end (#3469) 2016-12-22 00:34:05 +01:00
Aron Fischer
65f486ff02 swarm/api: check for zero length manifest error (#3468) 2016-12-22 00:32:08 +01:00
zelig
4181046488 swarm/network, cmd/swarm: swarm default network id is 3 (to match Ropsten) 2016-12-13 12:54:13 +01:00
Nick Johnson
671fd94e25 swarm/api: Update ENS root address for Ropsten & prod (#3391) 2016-12-01 19:33:10 +01:00
Viktor Trón
e949a2ed2f cmd/bzzd: swarm daemon fixes (#3359)
* cmd/bzzd: add missing p2p/discovery flags
* cmd/bzzd: fix two bugs crashing bzzd if bootnodes flag given
* cmd/bzzd: make no swap default, renamed flag bzznoswap->bzzswap
* internal/web3ext: correct methods for bzz IPC module
* cmd/bzzd: ethapi param not mandatory. Warning if no blockchain
* cmd/bzzd: correct default IPC modules in help string
* cmd/utils: fix help description for networkId - add Ropsten
* cmd/bzz, swarm/api, swarm/network: add swarm networkId flag
* cmd/bzzd: change nosync flag to sync and BootTFlag
2016-11-28 13:29:33 +01:00
Felix Lange
d1a95c643e Merge pull request #3325 from fjl/p2p-netrestrict
Prevent relay of invalid IPs, add --netrestrict
2016-11-25 13:59:18 +01:00
Jeffrey Wilcke
67e0894d9e common/httpclient, les: removed httpclient 2016-11-25 13:10:44 +01:00
Felix Lange
e46bda5093 swarm/network: prevent relay of invalid IPs 2016-11-22 22:21:18 +01:00
Felix Lange
5cd4430a8d swarm/api/http: reject requests without content-length 2016-11-11 00:15:21 +01:00
Felix Lange
23420ff67a swarm/network: skip TestSyncDb 2016-11-11 00:15:21 +01:00
Felix Lange
ef50e01c5e swarm/api: add bzz prefix to swarm directory 2016-11-11 00:15:20 +01:00
Felix Lange
b8bd9a71c8 all: update license information 2016-11-09 02:51:34 +01:00
Kenji Siu
d85d3c74db internal/debug, internal/ethapi, rpc, swarm/storage: Ran "go fmt" 2016-11-01 18:40:36 +08:00
Péter Szilágyi
81b01f1c2b Merge pull request #3111 from obscuren/gas-price-fork
core, core/vm: added gas price variance table (EIP #150)
2016-10-14 19:32:11 +03:00
Jeffrey Wilcke
64af2aafda core, core/vm: added gas price variance table
This implements 1b & 1c of EIP150 by adding a new GasTable which must be
returned from the RuleSet config method. This table is used to determine
the gas prices for the current epoch.

Please note that when the CreateBySuicide gas price is set it is assumed
that we're in the new epoch phase.

In addition this PR will serve as temporary basis while refactorisation
in being done in the EVM64 PR, which will substentially overhaul the gas
price code.
2016-10-14 18:09:17 +02:00
Daniel A. Nagy
eb2f01aee8 swarm/storage: Allow EOF at the end of the reader in the chunker. Handle the case when Read returns less than length of target slice 2016-10-13 21:23:28 +02:00
aron
a45421baaf swarm/storage: fix chunker when reader is broken
* brokenLimitedReader gives error after half size
* TestRandomBrokenData tests chunker with broken reader
* add blocking quitC (instead of errC) and use errC only for errors
* don't close chunkC in tester Split,
* use quitC to quit chunk storage loop
2016-10-10 23:34:44 +02:00
ΞTHΞЯSPHΞЯΞ
4d300e4dec swarm: plan bee for content storage and distribution on web3
This change imports the Swarm protocol codebase. Compared to the 'swarm'
branch, a few mostly cosmetic changes had to be made:

* The various redundant log message prefixes are gone.
* All files now have LGPLv3 license headers.
* Minor code changes were needed to please go vet and make the tests
  pass on Windows.
* Further changes were required to adapt to the go-ethereum develop
  branch and its new Go APIs.

Some code has not (yet) been brought over:

* swarm/cmd/bzzhash: will reappear as cmd/bzzhash later
* swarm/cmd/bzzup.sh: will be reimplemented in cmd/bzzup
* swarm/cmd/makegenesis: will reappear somehow
* swarm/examples/album: will move to a separate repository
* swarm/examples/filemanager: ditto
* swarm/examples/files: will not be merged
* swarm/test/*: will not be merged
* swarm/services/swear: will reappear as contracts/swear when needed
2016-08-31 16:19:40 +02:00