Commit Graph

338 Commits

Author SHA1 Message Date
Rob Quist
767a53f8c3
Make markets logger less noise when doing retrievals
Logging each block causes a lot of useless information in the logs
2022-05-05 11:56:09 +02:00
zenground0
239274cb1d Properly handle output index 2022-04-12 15:34:45 -04:00
Darko Brdareski
79453663b3
feat: Add additional test annotations (#8272)
* Annotate api,proxy_util,blockstore_badger, policy tests

* Annotate splitstore: bsbadger / markset

* Annotate splitstore feature

* Annotate union/timed blockstore tests

* Annotate openrpc, diff_adt tests

* Annotate error,drand,events tests

* Annotate predicates_test

* Fix annotations

* Annotate tscache, gen tests

* Annotate fundmanager test

* Annotate repub and selection tests

* Annotate statetree_test

* Annotate forks_test

* Annotate searchwait_test.go

* Fix duplicated @@ symbols

* Annotate chain stmgr/store tests

* Annotate more (types) tests

* More tests annotated

* Annotate conformance chaos actor tests

* Annotate more integration tests

* Annotate journal system tests

* Annotate more tests.

* Annotate gas,head buffer behaviors

* Fix markset annotations

* doc: test annotations for the markets dagstore wrapper

* Annotate miner_api test in dagstore

* Annotate more test files

* Remove bad annotations from fsrepo

* Annotate wdpost system

* Remove bad annotations

* Renamce "conformance" to "chaos_actor" tests

* doc: stm annotations for blockheader & election proof tests

* Annotate remaining "A" tests

* annotate: stm for error_test

* memrepo_test.go

* Annotate "b" file tests

* message_test.go

* doc: stm annotate for fsrepo_test

* Annotate "c" file tests

* Annotate "D" test files

* message_test.go

* doc: stm annotate for chain, node/config & client

* docs: stm annotate node_test

* Annotate u,v,wl tests

* doc: stm annotations for various test files

* Annotate "T" test files

* doc: stm annotate for proxy_util_test & policy_test

* doc: stm annotate for various tests

* doc: final few stm annotations

* Add mempool unit tests

* Add two more memPool Add tests

* Update submodules

* Add check function tests

* Add stm annotations, refactor test helper

* Annotate api,proxy_util,blockstore_badger, policy tests

* Annotate splitstore: bsbadger / markset

solving merge conflicts

* Annotate splitstore feature

* Annotate union/timed blockstore tests

* Annotate openrpc, diff_adt tests

* Annotate error,drand,events tests

* Annotate predicates_test

* Fix annotations

* Annotate tscache, gen tests

* Annotate fundmanager test

* Annotate statetree_test

* Annotate forks_test

* Annotate searchwait_test.go

* Fix duplicated @@ symbols

* Annotate chain stmgr/store tests

* Annotate more (types) tests

* More tests annotated

* Annotate conformance chaos actor tests

* Annotate more integration tests

* Annotate journal system tests

* Annotate more tests.

* Annotate gas,head buffer behaviors

solve merge conflict

* Fix markset annotations

* Annotate miner_api test in dagstore

* Annotate more test files

* doc: test annotations for the markets dagstore wrapper

* Annotate wdpost system

* Renamce "conformance" to "chaos_actor" tests

* Annotate remaining "A" tests

* doc: stm annotations for blockheader & election proof tests

* annotate: stm for error_test

* Annotate "b" file tests

* memrepo_test.go

* Annotate "c" file tests

* message_test.go

* Annotate "D" test files

* doc: stm annotate for fsrepo_test

* Annotate u,v,wl tests

* message_test.go

* doc: stm annotate for chain, node/config & client

* docs: stm annotate node_test

* Annotate "T" test files

* doc: stm annotations for various test files

* Add mempool unit tests

solve merge conflict

* doc: stm annotate for proxy_util_test & policy_test

* doc: stm annotate for various tests

* doc: final few stm annotations

* Add two more memPool Add tests

* Update submodules

* Add check function tests

solve conflict

* Add stm annotations, refactor test helper

solve merge conflict

* Change CLI test kinds to "unit"

* Fix double merged test

* Fix ccupgrade_test merge

* Fix lint issues

* Add stm annotation to types_Test

* Test vectors submodule

* Add file annotation to burn_test

Co-authored-by: Nikola Divic <divicnikola@gmail.com>
Co-authored-by: TheMenko <themenkoprojects@gmail.com>
2022-03-16 18:37:34 +01:00
Jennifer Wang
8a2a2d065d Merge branch 'releases' into jen/backport 2022-03-15 02:07:35 -04:00
Aayush
bf0470ad09 Revert "fix issues with fil-markets/data-transfer updates"
This reverts commit ca4ee1e8cf.
2022-03-02 15:39:15 -05:00
Masih H. Derkani
aacc246ba3
Merge branch 'master' into feat/cid-to-piece-idx 2022-03-02 14:06:30 +00:00
Łukasz Magiera
e9a6f5f287 Merge remote-tracking branch 'origin/master' into feat/paych-avail-reuse 2022-03-02 12:56:56 +00:00
Masih H. Derkani
769b0f30ef
Address review comments
- Add comment to clarify the reason for loop in testkit
- Trim common prefix in state printed in CLI commands for better
  readability
- Upgrade to a tagged release of `go-fil-markets` that includes indexing
  work; see: https://github.com/filecoin-project/go-fil-markets/pull/673
- Fix typo in CLI usage.
- Add comments to note that it is safe to use fx `OnStart` context when
  starting the provider engine.
- Fix string concatenation in error message formatting.
2022-03-02 11:04:50 +00:00
zenground0
77a954c7c3 Merge branch 'master' into feat/cid-to-piece-idx 2022-02-28 12:57:11 -07:00
Łukasz Magiera
a05a462c8c market utils: Support unixfsnode in TraverseDag 2022-02-22 10:51:57 +01:00
Aarsh Shah
e3f5b7c59f update dagstore top level index db 2022-02-18 14:00:01 +04:00
Masih H. Derkani
2ebc111b70 Use the same host and datatransfer as markets for index provider
Remove the bespoke instantiation of libp2p host and datatransfer manager
for index provider and reuse the existing instances used by markets.

The rationale for reuse is the following:
 1. Separation of host introduces a discovery problem, where without
    gossipsub the index provider endpoint will not be discoverable.
    Using the same host as markets would mean the chain can be used to
    discover addresses, putting less empassis on criticality of
    gossipsub considering its set-up cost and lack of message delivery
    guarantees.

 2. Only a single instance of graphsync/datatransfer can be instantiated
    per libp2p host; therefore, if the host is shared, so should
    datatransfer manager.

 3. it is not clear if the assumptions under which separation was
    decided still hold.
2022-02-16 13:46:44 +00:00
hannahhoward
e1a36058b4 feat(markets): update markets to simplify client adapter 2022-02-15 21:52:06 -08:00
Masih H. Derkani
f15a1edbf2 Protect index provider peer ID before connecting to full node
Add the peer ID of index provider host to the list of protected peers
before connecting to full node. Otherwise, it is possible for the
connection to be reset by full node before we reach the line that adds
the ID to list of protected peers via JsonRPC API.

Relates to:
 - https://github.com/filecoin-project/index-provider/issues/177
2022-02-15 11:49:29 +00:00
Łukasz Magiera
f61eb23f8f api: separate method for paych funding 2022-02-14 19:56:02 +01:00
Łukasz Magiera
10af768c60 Merge commit 'origin/release/v1.15.0~2' into feat/paych-avail-reuse 2022-02-14 19:27:12 +01:00
Aarsh Shah
399a1afd94 Merge remote-tracking branch 'origin/master' into feat/cid-to-piece-idx 2022-02-08 12:37:57 +04:00
Aarsh Shah
eb8296120b changes to mesh creation code 2022-02-04 13:58:47 +04:00
Aarsh Shah
daac7f2988 fix lint 2022-02-04 13:55:26 +04:00
Aarsh Shah
ca46a79144 protect full node connection 2022-02-04 13:04:07 +04:00
Anton Evangelatov
595ad44ee7 refactor indexprovider libp2p host connection to fullnode with meshcreator 2022-02-03 15:44:18 +01:00
Darko Brdareski
e51ce5c508 Merge remote-tracking branch 'upstream/master' into bloxico/system-test-matrix 2022-01-27 10:57:56 +01:00
Łukasz Magiera
8f9e730ad6 paych: Better off-chain errors 2022-01-20 18:19:27 +01:00
Łukasz Magiera
4235a97cf4 retrieval: OffChainRetrieval config 2022-01-20 18:19:27 +01:00
Łukasz Magiera
8b19b84140 paych: option to force off-chain get 2022-01-20 18:19:26 +01:00
Łukasz Magiera
1f2621b574 Make retrieval work with reused channels 2022-01-20 18:16:00 +01:00
Łukasz Magiera
8e46b9ea5d paych: API to pre-fund channels 2022-01-20 18:15:46 +01:00
Aarsh Shah
7508d96327 fix lint 2022-01-20 16:47:15 +04:00
Aarsh Shah
9ee940098d Merge remote-tracking branch 'origin/master' into feat/cid-to-piece-idx 2022-01-20 16:21:48 +04:00
vyzo
ca4ee1e8cf fix issues with fil-markets/data-transfer updates 2022-01-20 11:38:37 +02:00
hannahhoward
92d56d82f4 feat(deps): update markets stack
update go-fil-markets, go-data-transfer 1.13.0, go-graphsync 0.12.0
2022-01-14 17:21:04 -08:00
c r
da6752eccb
feat: #7747 sealing: Adding conf variable for capping number of concurrent unsealing jobs (#7884)
* adding the new variables- now time for logic

* putting parameters into right placeS

* adding unsealing throttle

* fixing linter issues

* removing one last thing...
2022-01-13 12:26:13 -06:00
Aarsh Shah
3a52f05692 green ci 2022-01-12 15:53:15 +04:00
Aarsh Shah
edfc2c215c Merge remote-tracking branch 'origin/main' into feat/cid-to-piece-idx 2022-01-12 12:48:54 +04:00
zenground0
33f2d24f54 Snap Deals Integration
- FSM handles the actual cc upgrade process including error states
- PoSting (winning and window) works over upgraded and upgrading sectors
- Integration test and changes to itest framework to reduce flakes
- Update CLI to handle new upgrade
- Update dependencies
2022-01-10 15:39:38 +05:30
Darko Brdareski
dda1a42a2a Merge branch 'bloxico/system-test-matrix' of https://github.com/filecoin-project/lotus into merge_lotus 2021-12-20 15:48:16 +01:00
Dirk McCormick
839fc8df7d fix: when retrying Add Piece, first seek to start of reader 2021-12-17 14:44:51 +01:00
Aayush Rajasekaran
0e2278cc76 Merge branch 'master' into deps/update-ctx-dsbs 2021-12-14 18:08:29 -05:00
Aayush Rajasekaran
dfb65ed89f Plumb contexts through 2021-12-11 17:04:00 -05:00
Darko Brdareski
0169d0dafd Annotate state feature tests 2021-12-10 16:08:25 +01:00
Łukasz Magiera
74d3ba6b5e fix lint 2021-12-09 18:10:34 +01:00
Łukasz Magiera
13b260e7f7 piecereader: Move closer to storage 2021-12-08 23:20:20 +01:00
Łukasz Magiera
9f6265e0b4 dagstore pieceReader: Fix wrong ErrUnexpectedEOF return in ReadAt 2021-12-03 23:36:36 +01:00
Łukasz Magiera
3969d6b767 dagstore pieceReader: Always read full in ReadAt 2021-12-03 20:14:35 +01:00
Łukasz Magiera
dad9190142 Add metrics to dagstore piecereader 2021-12-03 17:07:14 +01:00
Łukasz Magiera
4bcde2f0ff dagstore pieceReader: Cleanup reader nil check 2021-11-29 15:32:27 +01:00
Łukasz Magiera
05aa860459 Request correct read size with startOffset in pieceProvider 2021-11-27 00:05:45 +01:00
Łukasz Magiera
9110e6f632 dagstore pieceReader: add debug log on stream restart 2021-11-26 20:24:51 +01:00
Łukasz Magiera
331702cd95 Tweak MaxPieceReaderBurnBytes 2021-11-26 18:49:41 +01:00
Łukasz Magiera
abec7dd3bc Fix imports 2021-11-26 18:40:27 +01:00