Commit Graph

560 Commits

Author SHA1 Message Date
Łukasz Magiera
05cdeb80c3 chore: remove redundant import prefixes 2022-06-15 12:06:22 +02:00
Łukasz Magiera
98a48a47f8 storage: Move extern/storage-sealing to storage/pipeline 2022-06-14 19:41:59 +02:00
Łukasz Magiera
6afb43afbb wallet: Move key.go into separate pkg to avoid import loops 2022-06-14 19:21:40 +02:00
Łukasz Magiera
e65fae28de chore: fix imports 2022-06-14 17:00:51 +02:00
Łukasz Magiera
b53db68bda fix lint 2022-05-24 16:22:52 +02:00
Łukasz Magiera
6095aba4dd lib: support http readers passing over jsonrpc 2022-05-24 15:47:23 +02:00
Geoff Stuart
ebf5eb6e28 Update lib/rpcenc/reader.go
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
2022-05-02 09:59:41 -04:00
Geoff Stuart
b2a6d308ba Consolidate nullreader files 2022-05-02 09:59:41 -04:00
Łukasz Magiera
43436f7bc8 rpcenc: Fix double read at eof 2022-04-28 11:50:51 +02:00
Anton Evangelatov
0ed8647b6f extract unixfs filestore into lib 2022-03-21 10:48:52 +01:00
Łukasz Magiera
037e80c70c
gofmt 2022-03-17 12:10:24 +01:00
Łukasz Magiera
1c055fe83b
Merge branch 'master' into test/t.TempDir 2022-03-17 12:06:52 +01:00
Darko Brdareski
79453663b3
feat: Add additional test annotations (#8272)
* Annotate api,proxy_util,blockstore_badger, policy tests

* Annotate splitstore: bsbadger / markset

* Annotate splitstore feature

* Annotate union/timed blockstore tests

* Annotate openrpc, diff_adt tests

* Annotate error,drand,events tests

* Annotate predicates_test

* Fix annotations

* Annotate tscache, gen tests

* Annotate fundmanager test

* Annotate repub and selection tests

* Annotate statetree_test

* Annotate forks_test

* Annotate searchwait_test.go

* Fix duplicated @@ symbols

* Annotate chain stmgr/store tests

* Annotate more (types) tests

* More tests annotated

* Annotate conformance chaos actor tests

* Annotate more integration tests

* Annotate journal system tests

* Annotate more tests.

* Annotate gas,head buffer behaviors

* Fix markset annotations

* doc: test annotations for the markets dagstore wrapper

* Annotate miner_api test in dagstore

* Annotate more test files

* Remove bad annotations from fsrepo

* Annotate wdpost system

* Remove bad annotations

* Renamce "conformance" to "chaos_actor" tests

* doc: stm annotations for blockheader & election proof tests

* Annotate remaining "A" tests

* annotate: stm for error_test

* memrepo_test.go

* Annotate "b" file tests

* message_test.go

* doc: stm annotate for fsrepo_test

* Annotate "c" file tests

* Annotate "D" test files

* message_test.go

* doc: stm annotate for chain, node/config & client

* docs: stm annotate node_test

* Annotate u,v,wl tests

* doc: stm annotations for various test files

* Annotate "T" test files

* doc: stm annotate for proxy_util_test & policy_test

* doc: stm annotate for various tests

* doc: final few stm annotations

* Add mempool unit tests

* Add two more memPool Add tests

* Update submodules

* Add check function tests

* Add stm annotations, refactor test helper

* Annotate api,proxy_util,blockstore_badger, policy tests

* Annotate splitstore: bsbadger / markset

solving merge conflicts

* Annotate splitstore feature

* Annotate union/timed blockstore tests

* Annotate openrpc, diff_adt tests

* Annotate error,drand,events tests

* Annotate predicates_test

* Fix annotations

* Annotate tscache, gen tests

* Annotate fundmanager test

* Annotate statetree_test

* Annotate forks_test

* Annotate searchwait_test.go

* Fix duplicated @@ symbols

* Annotate chain stmgr/store tests

* Annotate more (types) tests

* More tests annotated

* Annotate conformance chaos actor tests

* Annotate more integration tests

* Annotate journal system tests

* Annotate more tests.

* Annotate gas,head buffer behaviors

solve merge conflict

* Fix markset annotations

* Annotate miner_api test in dagstore

* Annotate more test files

* doc: test annotations for the markets dagstore wrapper

* Annotate wdpost system

* Renamce "conformance" to "chaos_actor" tests

* Annotate remaining "A" tests

* doc: stm annotations for blockheader & election proof tests

* annotate: stm for error_test

* Annotate "b" file tests

* memrepo_test.go

* Annotate "c" file tests

* message_test.go

* Annotate "D" test files

* doc: stm annotate for fsrepo_test

* Annotate u,v,wl tests

* message_test.go

* doc: stm annotate for chain, node/config & client

* docs: stm annotate node_test

* Annotate "T" test files

* doc: stm annotations for various test files

* Add mempool unit tests

solve merge conflict

* doc: stm annotate for proxy_util_test & policy_test

* doc: stm annotate for various tests

* doc: final few stm annotations

* Add two more memPool Add tests

* Update submodules

* Add check function tests

solve conflict

* Add stm annotations, refactor test helper

solve merge conflict

* Change CLI test kinds to "unit"

* Fix double merged test

* Fix ccupgrade_test merge

* Fix lint issues

* Add stm annotation to types_Test

* Test vectors submodule

* Add file annotation to burn_test

Co-authored-by: Nikola Divic <divicnikola@gmail.com>
Co-authored-by: TheMenko <themenkoprojects@gmail.com>
2022-03-16 18:37:34 +01:00
Eng Zer Jun
c229842f57
test: use T.TempDir to create temporary test directory
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-03-15 00:21:31 +08:00
Łukasz Magiera
4421bf2fa1 feat: config: Persistent subsystem log level config 2022-03-10 14:44:56 +01:00
Łukasz Magiera
7dcfeee0ea use old go-path 2022-03-03 14:17:25 +00:00
Aayush Rajasekaran
0e2278cc76 Merge branch 'master' into deps/update-ctx-dsbs 2021-12-14 18:08:29 -05:00
hannahhoward
8e376c84cf feat(tracing): setup logging 2021-12-10 13:50:12 -08:00
whyrusleeping
072297e661 WIP: updating to new datastore/blockstore code with contexts 2021-12-09 14:12:25 +02:00
hannahhoward
f3b691d618 feat(tracing): switch to OpenTelemetry
Switch the underlying tracing library to OpenTelemetry, with a bridge to OpenCensus for
compatibility
2021-12-01 16:19:47 -08:00
Łukasz Magiera
30fccaa0bd fix lint 2021-09-02 18:45:18 +02:00
Łukasz Magiera
95b128b7bc chain: Cleanup consensus logic 2021-09-02 18:09:37 +02:00
Steven Allen
1cf556c3a2 feat: expose ChainGetPath on the gateway 2021-08-30 16:43:21 -07:00
Łukasz Magiera
1ba427f638 alerting: Address review 2021-08-26 16:09:18 +02:00
Łukasz Magiera
81b1dd12f8 Simple alert system; FD limit alerts 2021-08-26 15:44:45 +02:00
Łukasz Magiera
8426a62d15 rpcenc: Test early close, add reader.MustRedirect 2021-07-30 13:35:16 +02:00
Łukasz Magiera
555c402ba3 rpcenc: Fix POST read 2021-07-30 13:03:31 +02:00
Łukasz Magiera
0c809d3a5f rpcenc: Support reader redirect 2021-07-30 12:58:35 +02:00
Raúl Kripalani
8413c080e3 add godocs. 2021-07-20 15:43:03 -07:00
Anton Evangelatov
cd4571f0b0 wrap close(wait) with sync.Once to avoid panic 2021-07-20 15:43:03 -07:00
Jakub Sztandera
7dd58efb84 Add quantiles and histogram
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2021-06-18 15:44:36 -07:00
Cory Schwartz
e24f24bc71 Remove log line when tracing is not configured 2021-05-26 10:45:04 -07:00
Cory Schwartz
d50b59b4c7 don't scare the gosec linter 2021-05-25 23:55:46 -07:00
Cory Schwartz
273368ed6a separate tracing environment variables 2021-05-25 01:23:33 -07:00
Łukasz Magiera
e006310c6f sigs: vector tests for bls 2021-04-15 23:02:21 +02:00
Peter Rabbitson
25a77d905d Centralize everything on ipfs/go-log/v2
I am not entirely sure this is right, but everything seems to build...
2021-04-06 15:04:32 +02:00
Łukasz Magiera
b68eed906a fix lint 2021-04-02 12:37:20 +02:00
Łukasz Magiera
411f561db6 backupds: Improve truncated log handling 2021-03-27 10:05:12 +01:00
Łukasz Magiera
924c419e61 backupds: handle log bool in shed 2021-03-24 21:26:24 +01:00
Łukasz Magiera
bb79eba74c backupds: Compact log on restart 2021-03-24 21:16:42 +01:00
Łukasz Magiera
a2ac26c4f6 backupds: Add unit tests 2021-03-12 16:20:27 +01:00
Łukasz Magiera
31154ed294 fix lint 2021-03-11 12:05:32 +01:00
Łukasz Magiera
ea58bee3f5 backupds: Allow larger values in write log 2021-03-11 12:03:24 +01:00
Łukasz Magiera
3f1054daf4 backupds kvlog: Address review 2021-03-10 13:58:41 +01:00
Łukasz Magiera
2b380c96a5 Exit from the kvlog goroutine when closing 2021-03-10 11:24:19 +01:00
Łukasz Magiera
d4a981f327 fix lint 2021-03-09 23:38:28 +01:00
Łukasz Magiera
b4eefd7220 backupds: make sure logfile matches datastore 2021-03-09 23:00:05 +01:00
Łukasz Magiera
9f7d3ed340 gofmt 2021-03-09 22:33:12 +01:00
Łukasz Magiera
ca7e70bf3a Metadata datastore log 2021-03-09 22:33:01 +01:00
Raúl Kripalani
b0cbc932bd consolidate all blockstores in blockstore package. 2021-01-29 20:01:00 +00:00
Łukasz Magiera
6d368179e1 Fix lint 2021-01-25 14:02:58 +01:00
dignifiedquire
3d9eb226fa refactor: switch to filecoin-ffi bls api for bls signatures 2021-01-20 16:19:07 +01:00
Jakub Sztandera
cfcfd5d8fa
Fix race in blockstore test suite
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2021-01-05 20:34:28 +01:00
yaohcn
7c0b6f41d8 fix log format 2020-11-24 19:09:48 +08:00
hannahhoward
218f7c2c5d feat(markets): update markets, extract commp 2020-11-20 13:29:14 -08:00
vyzo
ef444676c2 combine add and remove evts and use a single emitter
to reduce likelihood of races
2020-11-20 16:15:44 +02:00
vyzo
7aec500384 emit event for peer disconnectionsa and act upon them in the blocksync peer tracker 2020-11-20 15:58:27 +02:00
Peter Rabbitson
29817b6617 Remove all Chain GC wrappings/types
There is no practical way to do GC against the chainstore given the current
state of affairs: remove all GC-related types for the time being.
2020-11-12 06:49:53 +01:00
Peter Rabbitson
7be7a9b5e3 Stop referring to github.com/ipfs/go-ipfs-blockstore outside of lib 2020-11-12 06:01:35 +01:00
Raúl Kripalani
72547f9ec3 badger tests: fix fd leak. 2020-11-11 23:12:57 +00:00
Raúl Kripalani
04a866121e badger/AllKeysChan(): respect context. 2020-11-11 23:12:16 +00:00
Aayush Rajasekaran
735c04f999
Merge pull request #4681 from filecoin-project/badger-viewable
move to native badger blockstore; leverage zero-copy View() to deserialize in-place
2020-11-10 18:08:12 -05:00
Raúl Kripalani
bc6965c86f skip callers in badger log for useful line info. 2020-11-10 22:50:53 +00:00
Raúl Kripalani
659ceaabdd fix comment. 2020-11-10 17:51:11 +00:00
Raúl Kripalani
d79ff24014 remove benchmark. 2020-11-10 17:50:21 +00:00
Raúl Kripalani
339391ecfb Revert "(to be reverted) add debug.PrintStack."
This reverts commit f8c3756ac0.
2020-11-10 16:08:41 +00:00
Raúl Kripalani
f8c3756ac0 (to be reverted) add debug.PrintStack. 2020-11-10 15:22:17 +00:00
Raúl Kripalani
360194b29b fix comment. 2020-11-10 15:02:55 +00:00
Raúl Kripalani
6d78de95a1 remove unused and misleading cachebs store. 2020-11-10 13:34:56 +00:00
Raúl Kripalani
577476b8fe import Viewable non-terminal blockstores. 2020-11-10 13:16:24 +00:00
Raúl Kripalani
54bf7c99d7 add the viewable trait to our gazillion blockstores. 2020-11-10 12:40:32 +00:00
少爷
e340996919 delete new context 2020-11-10 17:40:48 +08:00
少爷
de2637b4fa async connect to bootstrappers 2020-11-10 17:23:46 +08:00
Raúl Kripalani
3577300aee Merge branch 'master' into badger-viewable 2020-11-06 19:34:25 +00:00
Raúl Kripalani
7c442e1a98 godocs. 2020-11-06 18:55:13 +00:00
Raúl Kripalani
de4a0725f3 rename files. 2020-11-06 18:32:44 +00:00
Raúl Kripalani
b1aa43741a remove file that sneaked in. 2020-11-05 13:34:41 +00:00
Łukasz Magiera
a1e1b03ca4 Optionally allow bitswap for chainstore 2020-11-03 23:44:44 +01:00
Łukasz Magiera
b0824ada15 bufbs: Get from write blockstore first 2020-11-03 23:02:01 +01:00
Raúl Kripalani
d1ebf3c769 introduce non-pooled StorageKey method. 2020-11-02 13:32:07 +00:00
Raúl Kripalani
7facdf63c9 support legacy keying: base32 multihashes *sigh*. 2020-11-02 12:55:56 +00:00
Raúl Kripalani
72e573d98d fix lint. 2020-11-01 17:09:14 +00:00
Raúl Kripalani
9437136f84 fixup. 2020-11-01 13:01:38 +00:00
Raúl Kripalani
ce27b13076 add a native badger blockstore with View() method. 2020-11-01 12:55:49 +00:00
Łukasz Magiera
c3d00b0ac6
Merge pull request #4650 from jsign/jsign/offlinemode
make IPFS online mode configurable
2020-10-29 22:24:58 +01:00
Ignacio Hagopian
7d0f279353
make IPFS online mode configurable
Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>
2020-10-29 15:16:45 -03:00
Łukasz Magiera
8c60069bb8
Merge pull request #4535 from filecoin-project/feat/clientdeal-cache-commd
client deal: Cache CommD when creating multiple deals
2020-10-27 03:58:25 +01:00
Łukasz Magiera
31a474a1ca Move CommPWriter to lib/ 2020-10-26 15:16:28 +01:00
Jakub Sztandera
9fbd2a50ff
Fix lint
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-10-23 20:50:17 +02:00
Jakub Sztandera
ea69386519
Fix flaky TestTimedBSSimple
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-10-23 20:46:06 +02:00
Jakub Sztandera
3a46c15c4a
Disable blockstore bloom filter
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-10-21 04:22:49 +02:00
Steven Allen
86e30729bf give the TimeCacheBS tests a bit more time
Otherwise, they flake once in a while.
2020-10-14 16:51:26 -07:00
Steven Allen
811f1304e6 test timed cache blockstore 2020-10-14 16:17:55 -07:00
Steven Allen
15fe998c68 add an timed-cache blockstore
This blockstore lets us write to a temporary scratch location where blocks older
than the specified cache time are automatically cleared.
2020-10-14 14:43:58 -07:00
Steven Allen
4b38809c0b in-memory blockstore
Instead of using an in-memory datastore and dealing with the overhead of
computing datastore keys, creating new blocks, etc, use an in-memory blockstore.
2020-10-14 14:43:58 -07:00
Łukasz Magiera
8cdf078999 backup: append checksum 2020-10-06 01:50:54 +02:00
Łukasz Magiera
d7ec5e3618 lotus-miner init restore 2020-10-01 17:55:48 +02:00
Łukasz Magiera
c8a3269c4b shed: utils to read datastore backups 2020-10-01 17:55:47 +02:00
Łukasz Magiera
2dc9a1ee4e lotus-miner backup command 2020-10-01 17:55:47 +02:00
Łukasz Magiera
4d4bab12eb Improve miner sectors list UX 2020-09-30 13:34:05 +02:00