Łukasz Magiera
58416d0881
storage: Don't depend on specs-storage, move to storiface
2022-06-17 13:36:33 +02:00
Łukasz Magiera
9c4d10ec73
api: handle no-precommit in StateSectorPreCommitInfo gracefully
2022-06-16 15:20:58 +02:00
Łukasz Magiera
2dc1df878d
api: Move StateComputeDataCID to full-node
2022-06-16 13:29:35 +02:00
Łukasz Magiera
05cdeb80c3
chore: remove redundant import prefixes
2022-06-15 12:06:22 +02:00
Łukasz Magiera
28099a3905
storage: Move storage/sealer/stores to storage/paths
2022-06-14 20:25:52 +02:00
Łukasz Magiera
82857e6d5d
chore: rerun gen with moved packages
2022-06-14 20:13:31 +02:00
Łukasz Magiera
a9600b8a6f
storage: Move extern/sector-storage to storage/sealer
2022-06-14 20:03:38 +02:00
Łukasz Magiera
98a48a47f8
storage: Move extern/storage-sealing to storage/pipeline
2022-06-14 19:41:59 +02:00
Łukasz Magiera
717bda63b1
storage: Move control address logic to storage/ctladdr
2022-06-14 19:32:29 +02:00
Łukasz Magiera
dff1bf2868
storage: Move window-post logic into a separate package
2022-06-14 19:27:09 +02:00
Łukasz Magiera
e65fae28de
chore: fix imports
2022-06-14 17:00:51 +02:00
Steven Allen
ddc9425c07
feat: refactor: actor bundling system ( #8838 )
...
1. Include the builtin-actors in the lotus source tree.
2. Embed the bundle on build instead of downloading at runtime.
3. Avoid reading the bundle whenever possible by including bundle
metadata (the bundle CID, the actor CIDs, etc.).
4. Remove everything related to dependency injection.
1. We're no longer downloading the bundle, so doing anything ahead
of time doesn't really help.
2. We register the manifests on init because, unfortunately, they're
global.
3. We explicitly load the current actors bundle in the genesis
state-tree method.
4. For testing, we just change the in-use bundle with a bit of a
hack. It's not great, but using dependency injection doesn't make
any sense either because, again, the manifest information is
global.
5. Remove the bundle.toml file. Bundles may be overridden by
specifying an override path in the parameters file, or an
environment variable.
fixes #8701
2022-06-13 10:51:49 -07:00
Steven Allen
30981d0fdd
feat: refactor: actor bundling system ( #8838 )
...
1. Include the builtin-actors in the lotus source tree.
2. Embed the bundle on build instead of downloading at runtime.
3. Avoid reading the bundle whenever possible by including bundle
metadata (the bundle CID, the actor CIDs, etc.).
4. Remove everything related to dependency injection.
1. We're no longer downloading the bundle, so doing anything ahead
of time doesn't really help.
2. We register the manifests on init because, unfortunately, they're
global.
3. We explicitly load the current actors bundle in the genesis
state-tree method.
4. For testing, we just change the in-use bundle with a bit of a
hack. It's not great, but using dependency injection doesn't make
any sense either because, again, the manifest information is
global.
5. Remove the bundle.toml file. Bundles may be overridden by
specifying an override path in the parameters file, or an
environment variable.
fixes #8701
2022-06-13 10:15:00 -07:00
Łukasz Magiera
1cd94f598d
Merge pull request #8820 from filecoin-project/asr/deprecate-blocksync
...
refactor: remove old BlockSyncProtocolID
2022-06-13 18:36:21 +02:00
vyzo
d4d5ce3021
only enable rcmgr by default in full nodes
...
opt-in with envvar for other nodes, as there are still some issues
with markets related protocols
2022-06-13 12:16:20 -04:00
Geoff Stuart
b7010c9e60
Implement function to migrate actors with only code changes
2022-06-10 15:52:32 -04:00
Geoff Stuart
801c670edd
Remove ChainPutMany
2022-06-10 14:09:05 -04:00
Geoff Stuart
5c0f2c8ae6
Add putObj and putMany to apiBlockstore
2022-06-09 15:13:42 -04:00
Aayush
789f212f6c
refactor: remove old BlockSyncProtocolID
2022-06-08 13:38:56 -04:00
Aayush
b28c11a57d
Merge branch 'feat/nv16'
2022-06-03 14:01:49 -04:00
Aayush
91e69644c2
Merge branch 'feat/nv16' into jen/mergev1153to16
2022-06-03 13:18:27 -04:00
Geoff Stuart
782011b572
Cleanup LoadBundle
2022-06-01 22:22:52 -04:00
Jennifer Wang
2e59d0129d
Merge branch 'release/v1.15.3' into jen/mergev1153to16
2022-05-31 16:33:18 -04:00
vyzo
207ff66b3b
update comment.
...
Co-authored-by: Peter Rabbitson <ribasushi@protocol.ai>
2022-05-31 12:33:08 +03:00
vyzo
bc180c8d78
update comment.
...
Co-authored-by: Peter Rabbitson <ribasushi@protocol.ai>
2022-05-31 12:32:56 +03:00
vyzo
a1e6410178
remove outdated comment
2022-05-31 12:09:04 +03:00
vyzo
a313dcba74
only enable rcmgr by default in full nodes
...
opt-in with envvar for other nodes, as there are still some issues
with markets related protocols
2022-05-31 12:03:03 +03:00
Łukasz Magiera
56bde5adf9
Merge pull request #8692 from filecoin-project/feat/health-endpoints
...
feat: networking: add healthz and livez endpoints
2022-05-30 15:42:06 +02:00
Łukasz Magiera
48aa0131bf
fix: rpc: readd rpc.discover aliases
2022-05-27 17:03:03 +02:00
Aayush
4fa04abb8c
Add disable flag for Skyr upgrade
2022-05-26 18:20:49 -04:00
Geoff Stuart
30e7f89662
Check for manifest CID while loading bundle
2022-05-26 17:30:32 -04:00
Aayush Rajasekaran
06279b5f1a
Merge pull request #8606 from filecoin-project/asr/fix-drand-round
...
Fix: drand: calculation of round from Filecoin epochs
2022-05-25 15:36:53 -04:00
Aayush Rajasekaran
6924a3d5f2
Fix calculation of Drand round from Filecoin epochs
2022-05-25 12:43:52 -04:00
vyzo
bb0e3a6cb0
don't clobber test bundles during migration
2022-05-25 19:16:11 +03:00
Aayush Rajasekaran
c2f0bd3c62
Merge pull request #8668 from filecoin-project/asr/fix-post-test
...
fix wdpost_dispute tests
2022-05-24 14:22:01 -04:00
Aayush
2b847a98ae
fix wdpost_dispute tests
2022-05-24 13:58:30 -04:00
Cory Schwartz
d2299dfbf8
backoff/reconnect loop
2022-05-24 10:24:37 -07:00
Steven Allen
496799f867
build: set NetworkBundle in params file
...
Otherwise, we end up overriding the ldflags.
fixes #8684
2022-05-24 09:42:22 -07:00
Łukasz Magiera
70f3b98574
Fix config doc
2022-05-24 01:33:56 +02:00
Łukasz Magiera
7612860d15
config: Storage.DisallowRemoteFinalize
2022-05-23 23:53:25 +02:00
Łukasz Magiera
16f434c790
config: Plumb Assigner config correctly
2022-05-23 22:02:39 +02:00
Łukasz Magiera
443488b096
lint, docsgen
2022-05-23 22:02:39 +02:00
Łukasz Magiera
5ba8bd3b99
sched: Configurable assigners
2022-05-23 22:02:39 +02:00
Jennifer Wang
5cfedacf9f
make jen
2022-05-23 15:11:46 -04:00
Cory Schwartz
7d55ab0734
make atomic
2022-05-23 12:04:13 -07:00
Cory Schwartz
0a7fbd0fce
Merge remote-tracking branch 'origin/feat/health-endpoints' into feat/health-endpoints
2022-05-23 11:14:34 -07:00
Cory Schwartz
b4852038a1
Update node/health.go
...
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
2022-05-23 11:11:45 -07:00
Łukasz Magiera
389df95b00
config: Update batching comment in all the places
2022-05-23 18:49:01 +02:00
Cory Schwartz
45d15cb8ee
handle error during liveness check
2022-05-23 09:29:11 -07:00
Cory Schwartz
444d0b1b8a
add healthz and livez endpoints
2022-05-20 18:38:17 -07:00
vyzo
91daae2e36
add the network name to the bundle key in the datastore
2022-05-18 19:14:11 +03:00
Aayush
8cca9b1970
Use new go-state-types accessors
2022-05-17 15:21:27 -04:00
vyzo
93417fca7d
Merge pull request #8660 from filecoin-project/fix/issue-8659
...
[nv16] use network bundle names consistent with the network name
2022-05-17 22:13:02 +03:00
vyzo
31c8460116
fix typo
2022-05-17 21:47:01 +03:00
vyzo
ccb2e44e36
make path/url be a map of network name to car uri
2022-05-17 21:45:17 +03:00
vyzo
d949b6c8ef
fix envvar handling
2022-05-17 21:14:49 +03:00
vyzo
daf452a180
flexible bundle loading
2022-05-17 20:38:32 +03:00
vyzo
314b417446
use network bundle names consistent with the network name
2022-05-16 22:02:37 +03:00
vyzo
eee73eed59
use switch for release/path logic
2022-05-16 21:45:12 +03:00
vyzo
5eccdaaab7
fix typo
...
Co-authored-by: raulk <raul@protocol.ai>
2022-05-16 21:38:43 +03:00
vyzo
221e7952ab
fix typo
...
Co-authored-by: raulk <raul@protocol.ai>
2022-05-16 21:38:09 +03:00
vyzo
46a85fc11e
niceties for development bundle loading
2022-05-16 21:09:09 +03:00
vyzo
b4be759b2c
don't store dev bundle release keys in the datastore
...
This is a nicety for development, so that we always load a development bundle to avoid having
to give them distinct names etc. Just call you release "dev" or "dev.xxx..." and put the bundle
in `.lotus/builtin-actors/v8/dev/builting-actors-${network}.{car,sha256sum}` and it will be
unconditionally loaded.
2022-05-16 15:11:10 +03:00
vyzo
68b280b695
add comments in the DI bundle loader
2022-05-12 22:03:18 +03:00
LexLuthr
7e492f23b1
add dagstore register-shard command
2022-05-12 19:46:53 +05:30
vyzo
7be42d9935
refactor to pass the network bundle name through ldflags from build
2022-05-12 16:42:59 +03:00
vyzo
7b6f109401
add retry logic for bundle fetcher
2022-05-12 10:00:20 +03:00
vyzo
26d07fd987
move bundle fetcher to node/bundle instead of chain/actors
2022-05-12 09:51:08 +03:00
vyzo
da8bda3248
fix typo
2022-05-12 09:47:27 +03:00
vyzo
98730b90a3
fix lint
2022-05-11 22:41:56 +03:00
vyzo
74f300290e
dynamic loading of builtin actor bundles
2022-05-11 22:30:07 +03:00
Aayush Rajasekaran
c69b579201
Merge branch 'releases' into asr/merge-releases
2022-05-10 20:02:01 -04:00
Raúl Kripalani
6050d04c7c
Merge tag 'v1.15.2' into feat/nv16-merge-v1.15.2
2022-05-10 14:17:29 +01:00
Jakub Sztandera
5fb69281d8
Estimate gas across the upgrade with per message multipliers
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2022-05-09 17:08:13 +02:00
Aayush Rajasekaran
8282484f65
Gas estimation: Overestimate by 2 around the M1 upgrade
2022-05-06 11:50:43 -04:00
Aayush Rajasekaran
26a780a198
Gas estimation: Refactor the special PayCh collect case
2022-05-06 11:42:26 -04:00
simlecode
962e37e893
add StateGetNetworkParams api
2022-05-05 14:16:30 +08:00
Łukasz Magiera
f9a4a400da
Merge pull request #8557 from filecoin-project/feat/worker-commp
...
sealing: DataCid on workers
2022-04-28 17:45:32 +02:00
Łukasz Magiera
d7c52f8adf
sealing: ComputeDataCid miner API
2022-04-26 21:37:48 +02:00
vyzo
30ed272aa9
fetch testing-fwp bundle
...
use testing-fwp bundle when InsecurePostValidation is true
testing-fwp is testing-fake-proofs
2022-04-26 18:38:52 +03:00
Travis Person
ea4f599233
Make MarketListDeals and DealsList return points for MarketDeal
2022-04-22 23:26:29 +00:00
Masih H. Derkani
d044e30e12
Infer index provider topic from network name by default
...
Index provider integration uses a gossipsub topic to announce changes to
the advertised content. The topic name was fixed to the default topic
which is `/indexer/ingest/mainnet`.
In the case of lotus, the gossipsub validators enforce a list of topics
the instance is permitted to join by setting subscription filter option
when `PubSub` instance is constructed via DI.
Having the fixed topic name meant that any SP starting up a node on a
network other than `mainnet` would have to override the default config
to avoid the node crashing when index provider is enabled.
Instead of a fixed default, the changes here infer the allowed indexer
topic name from network name automatically if the topic configuration is
left empty.
Fixes #8510
2022-04-22 02:46:05 +02:00
Masih H. Derkani
f273a440f0
Infer index provider topic from network name by default
...
Index provider integration uses a gossipsub topic to announce changes to
the advertised content. The topic name was fixed to the default topic
which is `/indexer/ingest/mainnet`.
In the case of lotus, the gossipsub validators enforce a list of topics
the instance is permitted to join by setting subscription filter option
when `PubSub` instance is constructed via DI.
Having the fixed topic name meant that any SP starting up a node on a
network other than `mainnet` would have to override the default config
to avoid the node crashing when index provider is enabled.
Instead of a fixed default, the changes here infer the allowed indexer
topic name from network name automatically if the topic configuration is
left empty.
Fixes #8510
2022-04-21 14:00:53 +01:00
vyzo
adaa02fd26
load testing bundles for tests
2022-04-19 17:33:20 +03:00
Geoff Stuart
49d6c07280
Implemented StateLoockupRobustAddress
2022-04-14 17:41:18 -04:00
vyzo
6bd2d89e30
use memory blockstore for miner manifest loading
2022-04-14 19:34:26 +03:00
vyzo
507b835a9c
need a blockstore for the manifest loader
2022-04-14 19:34:26 +03:00
vyzo
395c772222
make the miner load the manifest so that it can correctly map actors
2022-04-14 19:34:26 +03:00
vyzo
93f1fac76b
simplify DI -- no need for the special key for builtin actor loading
2022-04-14 19:33:24 +03:00
Aayush
339b7db2b7
Integrate FIP 0027
2022-04-14 19:32:56 +03:00
vyzo
36cb76a322
genesis depends on actor bundle
2022-04-14 19:30:47 +03:00
vyzo
5e3112fa95
introduce a marker type for builtin actor loading to make DI work
2022-04-14 19:29:22 +03:00
vyzo
f26d19ed7d
load actor bundle at startup and initialize the manifest CID from the loaded file
2022-04-14 19:29:22 +03:00
Icarus9913
98c861e3c9
fix the IndexProviderConfig comment
2022-04-13 17:44:06 +08:00
Łukasz Magiera
cc7055774d
chore: config: default-disable kvlog
2022-04-12 23:17:13 +02:00
Jiaying Wang
ce3c6472fb
Merge pull request #8470 from filecoin-project/feat/enable-rcmgr-by-default
...
enable rcmgr by default
2022-04-12 16:12:03 +02:00
Łukasz Magiera
cf92776c64
Merge pull request #8468 from filecoin-project/jen/mergeback
...
chore: merge releases back to master
2022-04-12 15:41:36 +02:00
vyzo
3c8aa18f70
enabled rcmgr by default; disable with LOTUS_RCMGR=0
2022-04-12 11:32:37 +03:00
Aayush Rajasekaran
656a33863f
Fixup merge
2022-04-11 20:45:13 -04:00
jennijuju
0dcad9cc18
Merge branch 'releases' into jen/mergeback
2022-04-12 01:25:02 +02:00
Łukasz Magiera
6bef1aeb82
feat: cli: lotus client list-asks --protocols
2022-04-11 19:49:52 +02:00
Masih H. Derkani
0a2429b467
Fix DI of PubSub
in markets by binding dependencies it needs
...
Bind drand and bootstrap peers config so that `PubSub` instantiated by
`ConfigCommon` has all the dependencies it needs when `PubSub` instance
is needed in markets. In ths case, the instance is needed by the index
provider engine to announce new indexing advertisements.
2022-04-07 11:06:29 -04:00
Masih H. Derkani
e37c54f38e
Reuse the market process PubSub
instance in index provider engine
...
The markets process instantiates its own `PubSub` instance with all
validators, peer scoring, etc. set up. Use that instane to join the
indexing topic, otherwise the default topic instantiated by
index-provider internally (via go-legs) has no validators.
2022-04-07 11:06:23 -04:00
Masih H. Derkani
4043923e56
Set index provider options based on lotus config
...
Always set index provider options based on the lotus configuration.
Otherwise, they will have no effect.
2022-04-07 11:05:09 -04:00
Masih H. Derkani
f369d99138
Fix DI of PubSub
in markets by binding dependencies it needs
...
Bind drand and bootstrap peers config so that `PubSub` instantiated by
`ConfigCommon` has all the dependencies it needs when `PubSub` instance
is needed in markets. In ths case, the instance is needed by the index
provider engine to announce new indexing advertisements.
2022-04-06 20:04:39 +01:00
Masih H. Derkani
7e7e88e330
Reuse the market process PubSub
instance in index provider engine
...
The markets process instantiates its own `PubSub` instance with all
validators, peer scoring, etc. set up. Use that instane to join the
indexing topic, otherwise the default topic instantiated by
index-provider internally (via go-legs) has no validators.
2022-04-06 19:39:12 +01:00
Masih H. Derkani
b88307f875
Set index provider options based on lotus config
...
Always set index provider options based on the lotus configuration.
Otherwise, they will have no effect.
2022-04-06 16:08:10 +01:00
Łukasz Magiera
2b3fad1261
fix: node: Fix market node startup
2022-04-01 17:32:55 -04:00
Łukasz Magiera
d95f24b9d2
config: Sane default parallel sector read defaults
2022-03-29 14:52:37 -04:00
Łukasz Magiera
ebd34f1884
feat: storage: Parallel proving checks
2022-03-29 11:45:08 -04:00
Łukasz Magiera
54cb55a7ae
feat: miner: API/CLI to compute window-post
2022-03-28 17:06:51 -04:00
Łukasz Magiera
c951e348b4
feat: config: Move MakeNewSectorForDeals config into the Sealing sectoin
2022-03-26 23:45:48 -04:00
Łukasz Magiera
b78892934b
feat: config: Move MakeNewSectorForDeals config into the Sealing sectoin
2022-03-26 15:50:21 -04:00
Łukasz Magiera
82343a06e7
Merge remote-tracking branch 'origin/master' into feat/post-worker
2022-03-25 16:49:46 -04:00
Łukasz Magiera
7c4d3a457c
Merge pull request #8343 from filecoin-project/feat/more-snap-config
...
feat: sealing: More SnapDeals config knobs
2022-03-25 15:07:00 -04:00
Łukasz Magiera
945b93590d
ping: Match go-ipfs in cmd output
2022-03-21 15:11:19 +01:00
Łukasz Magiera
701d0a111e
Merge branch 'feat/net-ping' of github.com:ychiaoli18/lotus into feat/net-ping
2022-03-21 13:06:11 +01:00
Łukasz Magiera
92e4caa151
Merge remote-tracking branch 'origin/master' into feat/post-worker
2022-03-21 12:09:47 +01:00
Łukasz Magiera
433e9c351d
config: Fix doc on PreferNewSectorsForDeals; gofmt
2022-03-21 12:07:00 +01:00
Łukasz Magiera
97a73f1c29
sealing: More SnapDeals config knobs
2022-03-21 12:04:23 +01:00
Łukasz Magiera
fc34d9ba1c
Merge pull request #8086 from filecoin-project/nonsense/refactor-nodetype
...
refactor: convert RepoType from int to interface
2022-03-21 11:57:39 +01:00
Anton Evangelatov
0ed8647b6f
extract unixfs filestore into lib
2022-03-21 10:48:52 +01:00
Łukasz Magiera
5365ccfdb1
cli: Show separate worker types in miner info
2022-03-18 21:31:15 +01:00
Łukasz Magiera
7b81369c8c
wdpost: Don't attempt to snark with skipped sectors
2022-03-18 19:45:25 +01:00
Anton Evangelatov
0d6493ec3f
resolve merge conflicts with master
2022-03-18 12:28:22 +01:00
Łukasz Magiera
6ddbe41376
Merge remote-tracking branch 'origin/master' into feat/post-worker
2022-03-18 10:54:44 +01:00
Aayush Rajasekaran
2ed2ee1da7
Merge branch 'release/v1.15.1' into feat/fvm
2022-03-17 11:31:56 -04:00
Łukasz Magiera
037e80c70c
gofmt
2022-03-17 12:10:24 +01:00
Łukasz Magiera
1c055fe83b
Merge branch 'master' into test/t.TempDir
2022-03-17 12:06:52 +01:00
Łukasz Magiera
d3cec2f0d1
sealing: Add MakeCCSectorsAvailable config
2022-03-16 22:29:40 +01:00
Łukasz Magiera
a3bdd29615
Merge pull request #8330 from filecoin-project/feat/snap-queue
...
feat: sealing: Sector upgrade queue
2022-03-16 22:24:45 +01:00
Aayush Rajasekaran
90d78f924f
Merge pull request #8293 from filecoin-project/feat/fvm
...
Filecoin Virtual Machine integration
2022-03-16 15:51:56 -04:00
Łukasz Magiera
4ded059a1c
sealing: Add MakeCCSectorsAvailable config
2022-03-16 19:29:47 +01:00
Darko Brdareski
79453663b3
feat: Add additional test annotations ( #8272 )
...
* Annotate api,proxy_util,blockstore_badger, policy tests
* Annotate splitstore: bsbadger / markset
* Annotate splitstore feature
* Annotate union/timed blockstore tests
* Annotate openrpc, diff_adt tests
* Annotate error,drand,events tests
* Annotate predicates_test
* Fix annotations
* Annotate tscache, gen tests
* Annotate fundmanager test
* Annotate repub and selection tests
* Annotate statetree_test
* Annotate forks_test
* Annotate searchwait_test.go
* Fix duplicated @@ symbols
* Annotate chain stmgr/store tests
* Annotate more (types) tests
* More tests annotated
* Annotate conformance chaos actor tests
* Annotate more integration tests
* Annotate journal system tests
* Annotate more tests.
* Annotate gas,head buffer behaviors
* Fix markset annotations
* doc: test annotations for the markets dagstore wrapper
* Annotate miner_api test in dagstore
* Annotate more test files
* Remove bad annotations from fsrepo
* Annotate wdpost system
* Remove bad annotations
* Renamce "conformance" to "chaos_actor" tests
* doc: stm annotations for blockheader & election proof tests
* Annotate remaining "A" tests
* annotate: stm for error_test
* memrepo_test.go
* Annotate "b" file tests
* message_test.go
* doc: stm annotate for fsrepo_test
* Annotate "c" file tests
* Annotate "D" test files
* message_test.go
* doc: stm annotate for chain, node/config & client
* docs: stm annotate node_test
* Annotate u,v,wl tests
* doc: stm annotations for various test files
* Annotate "T" test files
* doc: stm annotate for proxy_util_test & policy_test
* doc: stm annotate for various tests
* doc: final few stm annotations
* Add mempool unit tests
* Add two more memPool Add tests
* Update submodules
* Add check function tests
* Add stm annotations, refactor test helper
* Annotate api,proxy_util,blockstore_badger, policy tests
* Annotate splitstore: bsbadger / markset
solving merge conflicts
* Annotate splitstore feature
* Annotate union/timed blockstore tests
* Annotate openrpc, diff_adt tests
* Annotate error,drand,events tests
* Annotate predicates_test
* Fix annotations
* Annotate tscache, gen tests
* Annotate fundmanager test
* Annotate statetree_test
* Annotate forks_test
* Annotate searchwait_test.go
* Fix duplicated @@ symbols
* Annotate chain stmgr/store tests
* Annotate more (types) tests
* More tests annotated
* Annotate conformance chaos actor tests
* Annotate more integration tests
* Annotate journal system tests
* Annotate more tests.
* Annotate gas,head buffer behaviors
solve merge conflict
* Fix markset annotations
* Annotate miner_api test in dagstore
* Annotate more test files
* doc: test annotations for the markets dagstore wrapper
* Annotate wdpost system
* Renamce "conformance" to "chaos_actor" tests
* Annotate remaining "A" tests
* doc: stm annotations for blockheader & election proof tests
* annotate: stm for error_test
* Annotate "b" file tests
* memrepo_test.go
* Annotate "c" file tests
* message_test.go
* Annotate "D" test files
* doc: stm annotate for fsrepo_test
* Annotate u,v,wl tests
* message_test.go
* doc: stm annotate for chain, node/config & client
* docs: stm annotate node_test
* Annotate "T" test files
* doc: stm annotations for various test files
* Add mempool unit tests
solve merge conflict
* doc: stm annotate for proxy_util_test & policy_test
* doc: stm annotate for various tests
* doc: final few stm annotations
* Add two more memPool Add tests
* Update submodules
* Add check function tests
solve conflict
* Add stm annotations, refactor test helper
solve merge conflict
* Change CLI test kinds to "unit"
* Fix double merged test
* Fix ccupgrade_test merge
* Fix lint issues
* Add stm annotation to types_Test
* Test vectors submodule
* Add file annotation to burn_test
Co-authored-by: Nikola Divic <divicnikola@gmail.com>
Co-authored-by: TheMenko <themenkoprojects@gmail.com>
2022-03-16 18:37:34 +01:00
Aayush
16128a0212
Merge branch 'master' into feat/fvm
2022-03-16 11:57:46 -04:00
Aayush
9ea623e125
Rename the Lotus VM to LegacyVM
2022-03-15 19:40:17 -04:00
Jiaying Wang
ada499ef14
Merge pull request #8314 from filecoin-project/jen/indexsetting
...
feat: config: enable indexer providing by default
2022-03-15 19:33:49 -04:00
Jiaying Wang
c98ef0aaf8
Merge pull request #8317 from filecoin-project/jen/backport
...
chore: backport: release to master
2022-03-15 19:12:20 -04:00
vyzo
9fcafff847
adjust conns to 2x maxconns for duplicate connections
2022-03-15 10:02:27 +02:00
vyzo
53c525f0ed
improve resource manager integration
...
- add opt-in env var to control instantation, until we are comfortable with testing to enble by default.
- adjust default limits if the connection manager high mark is higher than the default inbound conn limit.
2022-03-15 09:43:18 +02:00
Jennifer Wang
8a2a2d065d
Merge branch 'releases' into jen/backport
2022-03-15 02:07:35 -04:00
Jennifer Wang
6240729ee4
enable indexer providing by default
2022-03-14 23:24:52 -04:00
Eng Zer Jun
c229842f57
test: use T.TempDir
to create temporary test directory
...
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-03-15 00:21:31 +08:00
Łukasz Magiera
135aef78d7
Merge remote-tracking branch 'origin/master' into feat/post-worker
2022-03-11 17:04:58 +01:00
Łukasz Magiera
81f0cfdd1e
Merge pull request #8276 from filecoin-project/feat/faster-clientcalccommp
...
feat: api: Make ClientCalcCommP multithreaded
2022-03-10 23:22:48 +01:00
Łukasz Magiera
0d9e34d651
client impl: Fix typo in ClientCalcCommP CopyBuffer error
2022-03-10 22:41:45 +01:00
Łukasz Magiera
4421bf2fa1
feat: config: Persistent subsystem log level config
2022-03-10 14:44:56 +01:00
Łukasz Magiera
9c4f8254ee
post workers: Fix build post-merge
2022-03-10 14:20:07 +01:00
Łukasz Magiera
a715fb3063
feat: Make ClientCalcCommP multithreaded
2022-03-09 23:25:29 +01:00
hannahhoward
49742f8fdc
feat(deps): update to graphsync v0.13.0 with 2.0 protocol
2022-03-09 18:06:35 +00:00
Łukasz Magiera
046a9f8af0
Merge remote-tracking branch 'origin/master' into feat/post-worker
2022-03-09 16:27:03 +01:00
Anton Evangelatov
7cdf6dc680
remove .Type() now that we have singletons for repos
2022-03-07 13:00:26 +01:00
Anton Evangelatov
b9fe61f9da
Merge branch 'master' into nonsense/refactor-nodetype
2022-03-07 11:51:28 +01:00
Anton Evangelatov
5734a39e8e
move DealmakingConfiger and SealingConfiger away from config/types.go
2022-03-03 16:51:47 +01:00
Anton Evangelatov
62e352ef88
move APIInfoEnvVars, RepoFlags, APIFlags under RepoType
2022-03-03 16:45:11 +01:00
Anton Evangelatov
acbfc879aa
config: extract DealmakingConfig and Sealing config from Lotus for Boost ( #8172 )
...
* refactor sealingCfg and dealmakingCfg to be able to abstract Config
* fix error messages
2022-03-03 15:37:23 +01:00
Anton Evangelatov
066d0edf51
add SupportsStagingDeals trait
2022-03-03 15:37:23 +01:00
Anton Evangelatov
949ec65118
drop defConfForType
2022-03-03 15:37:23 +01:00
Anton Evangelatov
f1f878a000
unexport repo types; use a global var for every repo type
2022-03-03 15:37:23 +01:00
Anton Evangelatov
e3edab66e3
refactor RepoType
2022-03-03 15:37:23 +01:00
Łukasz Magiera
7dcfeee0ea
use old go-path
2022-03-03 14:17:25 +00:00
Masih H. Derkani
1bb65189af
Upgrade to go-path v0.2.1
and remove replace
directive
...
The dependency was used in the chain API. Integrated the new fetcher
factory.
2022-03-03 13:29:32 +00:00
Aayush
85886287b4
Revert "use the libp2p connmgr"
...
This reverts commit 826cdb2186
.
2022-03-02 15:39:23 -05:00
Aayush
b5912d1543
Revert "instantiate resource manager in DI"
...
This reverts commit 8d3f98fe38
.
2022-03-02 15:39:22 -05:00
Aayush
e9e28f75e2
Revert "add resource manager NetStat api"
...
This reverts commit d05d5bcb70
.
2022-03-02 15:39:22 -05:00
Aayush
b8747e3afc
Revert "add resource manager Net(Set)Limit api"
...
This reverts commit 554848fe6b
.
2022-03-02 15:39:21 -05:00
Aayush
48e28898a6
Revert "fix lint"
...
This reverts commit 325a4c0e5b
.
2022-03-02 15:29:27 -05:00
Aayush
17ac583508
Revert "close the rcmgr on shutdown"
...
This reverts commit 73ec10a49e
.
2022-03-02 15:29:26 -05:00
Masih H. Derkani
aacc246ba3
Merge branch 'master' into feat/cid-to-piece-idx
2022-03-02 14:06:30 +00:00
Masih H. Derkani
444a2bf355
Integrate the latest index-provider with config to disable announcements
...
Integrate the latest `index-provider` and reflect the changes to engine
configuration. Note that this commit disables announcements of indices
on the network by default as requested for initial merge to master.
Introduce dedicated index provider configuration parameters with
documentation and defaults that match the defaults in index-provider.
Re-generate code as needed.
2022-03-02 13:45:09 +00:00
Łukasz Magiera
e9a6f5f287
Merge remote-tracking branch 'origin/master' into feat/paych-avail-reuse
2022-03-02 12:56:56 +00:00
Masih H. Derkani
769b0f30ef
Address review comments
...
- Add comment to clarify the reason for loop in testkit
- Trim common prefix in state printed in CLI commands for better
readability
- Upgrade to a tagged release of `go-fil-markets` that includes indexing
work; see: https://github.com/filecoin-project/go-fil-markets/pull/673
- Fix typo in CLI usage.
- Add comments to note that it is safe to use fx `OnStart` context when
starting the provider engine.
- Fix string concatenation in error message formatting.
2022-03-02 11:04:50 +00:00
zenground0
77a954c7c3
Merge branch 'master' into feat/cid-to-piece-idx
2022-02-28 12:57:11 -07:00
jennijuju
c6361b391c
Merge branch 'releases' into jen/v15
2022-02-25 21:01:31 -05:00
zenground0
e5de237990
Fix fault tracker to handle snap deals
2022-02-24 17:53:21 -05:00
ZenGround0
d711775cdf
Merge pull request #8177 from filecoin-project/fix/fault-tracker-snap-deals
...
fix:proving:post check sector handles snap deals replica faults
2022-02-24 11:30:10 -07:00
Aayush
fd8cb44e67
fix: client: calculate commps for pieces bigger than 32GB
2022-02-24 10:27:37 -05:00
zenground0
abe04c33c1
Fix fault tracker to handle snap deals
2022-02-23 09:56:47 -07:00
Aarsh Shah
e3f5b7c59f
update dagstore top level index db
2022-02-18 14:00:01 +04:00
Kevin Li
ba72eff3e6
feat: cli/net: implement 'net ping' command
2022-02-17 17:22:52 +08:00
Masih H. Derkani
a09f386537
Remove redundant IndexProvider
config now that host is shared
...
Now that host is shared having separate config for listen and announce
addresses along with graphsync limit makes no sense since all of that is
shared with the markets host and datatransfer manager.
2022-02-16 13:46:44 +00:00
Masih H. Derkani
2ebc111b70
Use the same host and datatransfer as markets for index provider
...
Remove the bespoke instantiation of libp2p host and datatransfer manager
for index provider and reuse the existing instances used by markets.
The rationale for reuse is the following:
1. Separation of host introduces a discovery problem, where without
gossipsub the index provider endpoint will not be discoverable.
Using the same host as markets would mean the chain can be used to
discover addresses, putting less empassis on criticality of
gossipsub considering its set-up cost and lack of message delivery
guarantees.
2. Only a single instance of graphsync/datatransfer can be instantiated
per libp2p host; therefore, if the host is shared, so should
datatransfer manager.
3. it is not clear if the assumptions under which separation was
decided still hold.
2022-02-16 13:46:44 +00:00
Łukasz Magiera
681b90768a
Merge remote-tracking branch 'origin/master' into feat/paych-avail-reuse
2022-02-16 10:06:36 +01:00
vyzo
9c00af1b86
don't track peer ids in rcmgr metrics
2022-02-15 16:12:17 +02:00
Łukasz Magiera
48dea93c7e
Merge pull request #8089 from filecoin-project/feat/rcmgr-metrics
...
Resource Manager Metrics
2022-02-15 14:40:44 +01:00
Aayush
75b8e4c981
fix: sealer: allow enable/disabling ReplicaUpdate tasks
2022-02-14 15:25:52 -05:00
Aayush
56df886b55
fix: sealer: allow enable/disabling ReplicaUpdate tasks
2022-02-14 15:24:13 -05:00
Łukasz Magiera
60b9acc09d
gen
2022-02-14 21:16:41 +01:00
Aayush
eebe784191
fix: sealer: allow enable/disabling ReplicaUpdate tasks
2022-02-14 15:08:28 -05:00
Łukasz Magiera
36a1934845
paychmgr: Fix tests after api changes
2022-02-14 20:16:30 +01:00
Łukasz Magiera
df5bd147fa
paychmgr: Review comments
...
Co-authored-by: Hannah Howard <hannah@hannahhoward.net>
Co-authored-by: dirkmc <dirkmdev@gmail.com>
2022-02-14 20:07:13 +01:00
Łukasz Magiera
f61eb23f8f
api: separate method for paych funding
2022-02-14 19:56:02 +01:00
Łukasz Magiera
10af768c60
Merge commit 'origin/release/v1.15.0~2' into feat/paych-avail-reuse
2022-02-14 19:27:12 +01:00
vyzo
a9ec408844
collect resource manager metrics
2022-02-14 19:46:05 +02:00
Masih H. Derkani
7e7844f169
Fix log message formatting when instantiating index provider
2022-02-11 12:15:21 +00:00
Will
296eab3045
Merge pull request #8045 from gammazero/feat/cid-to-piece-idx
...
Add indexer pubsub message authentication and rate limiting
2022-02-10 13:49:58 -08:00
gammazero
eedcf910eb
change scoring based on review
2022-02-10 10:53:07 -08:00
gammazero
b72cd90c19
Add indexer message scoring
2022-02-10 09:21:21 -08:00
gammazero
3f3d61b043
Service creation takes interface, not implementation
2022-02-10 08:41:18 -08:00
Jennifer Wang
de1f4792cd
Merge remote-tracking branch 'origin/master' into jen/v15
2022-02-09 19:27:34 -05:00
Łukasz Magiera
44fd0e3349
Merge pull request #8008 from filecoin-project/feat/splitstore-sortless-compaction
...
splitstore sortless compaction
2022-02-09 17:17:57 +00:00
Jennifer Wang
6ed97f0d04
Merge branch 'release/v1.14.0' into jen/v15
2022-02-09 03:17:38 -05:00
zenground0
e32704e7f6
Update node doc gen
2022-02-08 12:56:32 -05:00
zenground0
1e524b7210
Review Response
2022-02-08 12:39:35 -05:00
zenground0
5a09b6496d
Actually read config value
2022-02-08 12:39:35 -05:00
zenground0
2aa5911fb9
config gen
2022-02-08 12:39:35 -05:00
zenground0
4d8b912acc
New config for turning off auto deal creation
2022-02-08 12:39:35 -05:00
Masih H. Derkani
cc61650f86
Upgrade to latest index-provider
and set miner ID as extra gossip data
...
Upgrade to the latest `index-provider` which upgrades the go-legs
protocol to allow the inclusion of extra gossip data that may be used
for gossip validation purposes. In the case of lotus gossip message
validators the miner ID is used to verify the sender's peer ID on chain.
Relates to:
- https://github.com/filecoin-project/lotus/pull/8045
2022-02-08 14:32:03 +00:00
gammazero
b2805823ce
Pass to validator the interfaces needed to get miner info
2022-02-08 04:55:59 -08:00
gammazero
1dc6a2fea6
Add indexer pubsub message authentication and rate limiting
2022-02-08 02:53:25 -08:00
Aarsh Shah
399a1afd94
Merge remote-tracking branch 'origin/master' into feat/cid-to-piece-idx
2022-02-08 12:37:57 +04:00
vyzo
1221c0ba3f
make gen
2022-02-06 12:28:21 +02:00
vyzo
03352ea863
make badger the default splitstore markset type
2022-02-06 11:21:16 +02:00
Aarsh Shah
e155eb9fa3
Merge pull request #8016 from filecoin-project/nonsense/add-fullnode-api
...
add indexProvider host to markets
2022-02-04 12:56:36 +04:00
Aarsh Shah
4691b2b809
changes to the indexer message relay PR
2022-02-04 12:15:01 +04:00
Aarsh Shah
3ecf478ff0
Merge pull request #8026 from gammazero/feat/cid-to-piece-idx
...
Lotus chain nodes relay indexer pubsub messages
2022-02-04 12:05:46 +04:00
gammazero
c084130d3e
Lotus chain nodes relay indexer pubsub messages
...
Content providers announce the availability of indexer data using gossip pubsub. The content providers are not connected directly to indexers, so the pubsub messages are relayed to indexers via chain nodes. This PR makes chain nodes relay gossip pubsub messages, on the /indexer/ingest/<netname> topic.
2022-02-03 14:56:21 -08:00
Will Scott
7247f8e41f
add net protect api methods
...
fix #8015
2022-02-03 17:24:49 +01:00
Anton Evangelatov
dde2ab59ae
fixup
2022-02-03 15:53:46 +01:00
Anton Evangelatov
595ad44ee7
refactor indexprovider libp2p host connection to fullnode with meshcreator
2022-02-03 15:44:18 +01:00
Masih H. Derkani
575e6add43
Rename IndexerProvider
to IndexProvider
and add more docs
...
Rename the config section corresponding to indexing to `IndexProvider`
for better readability.
Update existing docs for better clarity and add docs for config
parameters embedded from `index-provider` `Ingest` config library.
2022-02-03 13:10:47 +00:00
Masih H. Derkani
235377585d
Clarify the purpose of MaxSimultaneousTransfers
config
...
Clarify the documentation of `MaxSimultaneousTransfers` to state that
the "transfer" refers to multihash syncing only and not the data itself.
2022-02-03 11:51:01 +00:00
Anton Evangelatov
faa3a79a5f
use NetAddrListener iface
2022-02-02 16:01:19 +01:00
Anton Evangelatov
2790cf35af
change fullnodeApi to listen addrs
2022-02-02 13:06:53 +01:00
Anton Evangelatov
f9b63c5f8c
add idxProvHost to StorageProvider
2022-02-02 13:03:41 +01:00
Anton Evangelatov
db5e4da268
add fullnodeApi to StorageProvider
2022-02-02 13:03:41 +01:00
Łukasz Magiera
98f4d37f1a
Merge remote-tracking branch 'origin/master' into feat/post-worker
2022-01-31 19:50:52 +00:00
Darko Brdareski
e51ce5c508
Merge remote-tracking branch 'upstream/master' into bloxico/system-test-matrix
2022-01-27 10:57:56 +01:00
Darko Brdareski
8c41e17c93
Fix typo in client_test annotations
2022-01-27 10:56:33 +01:00
Masih H. Derkani
5d72ebe0cc
Upgrade index-provider
and update topic name
...
Update the topic name to be consistent with default
configured in storetheindex.
Upgrade to the latest `index-provider`.
2022-01-27 09:48:10 +00:00
Aayush
1b18236f91
feat: sealer: allow users to abort in-flight snap upgrades
2022-01-25 13:01:51 -05:00
Łukasz Magiera
4235a97cf4
retrieval: OffChainRetrieval config
2022-01-20 18:19:27 +01:00
Łukasz Magiera
8b19b84140
paych: option to force off-chain get
2022-01-20 18:19:26 +01:00
Łukasz Magiera
8e46b9ea5d
paych: API to pre-fund channels
2022-01-20 18:15:46 +01:00
Aarsh Shah
b8bf2af3e3
upgrade on master
2022-01-20 16:34:13 +04:00
Aarsh Shah
9ee940098d
Merge remote-tracking branch 'origin/master' into feat/cid-to-piece-idx
2022-01-20 16:21:48 +04:00
Łukasz Magiera
e476cf7968
Merge remote-tracking branch 'origin/master' into feat/post-worker
2022-01-20 13:15:48 +01:00
vyzo
73ec10a49e
close the rcmgr on shutdown
2022-01-20 11:47:39 +02:00
vyzo
325a4c0e5b
fix lint
2022-01-20 11:46:39 +02:00
vyzo
554848fe6b
add resource manager Net(Set)Limit api
2022-01-20 11:36:11 +02:00
vyzo
d05d5bcb70
add resource manager NetStat api
2022-01-20 11:36:11 +02:00
vyzo
8d3f98fe38
instantiate resource manager in DI
2022-01-20 11:36:11 +02:00
vyzo
826cdb2186
use the libp2p connmgr
2022-01-20 11:36:11 +02:00
Łukasz Magiera
b38141601c
Untangle ffi from api
2022-01-18 11:57:04 +01:00
Łukasz Magiera
03fa955e9e
Cleanup imports
2022-01-18 11:37:24 +01:00
Łukasz Magiera
f2496d87c9
post workers: Snapdeals updates
2022-01-18 11:25:04 +01:00
Łukasz Magiera
0c062f9c08
Merge remote-tracking branch 'origin/master' into feat/post-worker
2022-01-18 10:49:42 +01:00
hannahhoward
92d56d82f4
feat(deps): update markets stack
...
update go-fil-markets, go-data-transfer 1.13.0, go-graphsync 0.12.0
2022-01-14 17:21:04 -08:00
vyzo
7e43bfc109
hook the upgrade schedule to splitstore start
2022-01-14 17:14:32 -05:00
zenground0
d6aa17e21f
Snap Deals Integration
...
- FSM handles the actual cc upgrade process including error states
- PoSting (winning and window) works over upgraded and upgrading sectors
- Integration test and changes to itest framework to reduce flakes
- Update CLI to handle new upgrade
- Update dependencies
2022-01-14 17:14:32 -05:00
Łukasz Magiera
4a874eff70
post workers: Cleanup, tests
2022-01-14 14:17:52 +01:00
c r
da6752eccb
feat: #7747 sealing: Adding conf variable for capping number of concurrent unsealing jobs ( #7884 )
...
* adding the new variables- now time for logic
* putting parameters into right placeS
* adding unsealing throttle
* fixing linter issues
* removing one last thing...
2022-01-13 12:26:13 -06:00
Aarsh Shah
8459d75597
update index provider config
2022-01-13 16:15:43 +04:00
Aarsh Shah
b780a64ed9
send markets multiaddrs to the engine
2022-01-13 15:31:15 +04:00
Aarsh Shah
8c8652e7bb
add a new cli
2022-01-12 18:06:48 +04:00
Aarsh Shah
dec79c70a1
fix compilation
2022-01-12 15:03:46 +04:00
Aarsh Shah
21a8f71859
compiles
2022-01-12 14:59:09 +04:00
Aarsh Shah
edfc2c215c
Merge remote-tracking branch 'origin/main' into feat/cid-to-piece-idx
2022-01-12 12:48:54 +04:00
zenground0
33f2d24f54
Snap Deals Integration
...
- FSM handles the actual cc upgrade process including error states
- PoSting (winning and window) works over upgraded and upgrading sectors
- Integration test and changes to itest framework to reduce flakes
- Update CLI to handle new upgrade
- Update dependencies
2022-01-10 15:39:38 +05:30
hannahhoward
1078dff3a9
feat(storageminer): also list data transfers with no id
2021-12-22 13:41:56 -08:00
hannahhoward
8a923fb420
fix(storageminer): fix conditional
2021-12-22 13:41:56 -08:00
hannahhoward
177282d9f9
fix(storageminer): lint/mod-tidy/extra checks
2021-12-22 13:41:55 -08:00
hannahhoward
cddf63efe9
feat(storageminer): add api for transfer diagnostics
...
Add API + CLI for inspecting in depth diagnostics on graphsync transfers with a given peer
2021-12-22 13:41:29 -08:00
Aayush Rajasekaran
1bc987772c
Merge pull request #7818 from filecoin-project/asr/vm-nvfactor
...
Refactor: VM: Remove the NetworkVersionGetter
2021-12-21 12:19:53 -05:00
Darko Brdareski
a54ac2ef79
Remove FS, MEM, GAS leftover test anotations.
2021-12-20 17:00:01 +01:00
Darko Brdareski
86cf17c1f4
Remove bad annotation from gas_test.go
2021-12-20 16:51:09 +01:00
Darko Brdareski
dda1a42a2a
Merge branch 'bloxico/system-test-matrix' of https://github.com/filecoin-project/lotus into merge_lotus
2021-12-20 15:48:16 +01:00
Aayush Rajasekaran
3f4eaa99d5
Refactor: State: Rename stmgr::GetNtwkVersion to GetNetworkVersion
2021-12-17 18:43:39 -05:00
Dirk McCormick
b1734f84b3
feat: retrieval ask CLI command
2021-12-17 15:55:12 +01:00
Łukasz Magiera
a4728d3c72
Merge pull request #7578 from filecoin-project/rvagg/SimultaneousTransfersForStoragePerClient
...
feat(graphsync): allow setting of per-peer incoming requests for miners
2021-12-17 14:27:15 +01:00
vyzo
dd327f0b22
plumb more contexts
2021-12-17 11:42:09 +02:00
Rod Vagg
9e7d9affbe
feat(graphsync): allow setting of per-peer incoming requests for miners
2021-12-17 15:04:16 +11:00
Darko Brdareski
3e32aa896c
Annotate client storage deals feature
2021-12-16 14:36:02 +01:00
vyzo
22ce395130
fix some unit tests
2021-12-14 17:05:10 +02:00
vyzo
716b4a3b09
hook a great context in the sky to satisfy DI; sigh...
2021-12-14 16:57:09 +02:00
vyzo
eb48dc9b68
fix issues with new peerstore constructor signature in DI
2021-12-14 16:21:55 +02:00
vyzo
f157ac9b68
update for new ConnManager constructor
2021-12-14 15:30:29 +02:00
Dirk McCormick
a51b08e386
feat: add IndexerProvider AnnounceAddress config
2021-12-13 18:24:05 +01:00
Masih H. Derkani
6b7b9793ab
Set default index provider ingest config values
...
Set default values for index-provider ingest config.
Leave a TODO to use provider config APIs once defaults exposed.
Run `go mod tidy`
2021-12-13 14:44:56 +00:00
Darko Brdareski
0addca1070
Fix bad annotations
2021-12-13 13:41:04 +01:00
vyzo
84710cf27e
plumb more contexts in lotus
2021-12-13 14:26:59 +02:00
Aayush Rajasekaran
dfb65ed89f
Plumb contexts through
2021-12-11 17:04:00 -05:00
vyzo
5d6398f20e
hook the upgrade schedule to splitstore start
2021-12-03 12:11:54 +02:00
Łukasz Magiera
f39942283b
Merge pull request #7706 from filecoin-project/feat/ret-matchding-cars
...
retrieval: Only output matching nodes, MatchPath dagspec
2021-11-29 22:14:25 +01:00
Łukasz Magiera
d019853687
review: Cleanup some comments
...
Co-authored-by: Peter Rabbitson <ribasushi@protocol.ai>
2021-11-29 21:29:00 +01:00
Łukasz Magiera
5b5e6b9e44
retrieval: DagSpec.MatchPath -> ExportMerkleProof
2021-11-29 21:14:00 +01:00
Łukasz Magiera
61791b90ea
retrieval: Only output matching nodes, MatchPath dagspec
2021-11-29 20:40:55 +01:00
Steven Allen
797147097c
Merge pull request #7689 from filecoin-project/disable-mplex
...
disable mplex stream muxer
2021-11-29 09:46:24 -08:00
Łukasz Magiera
77d75b7739
Merge pull request #7688 from filecoin-project/chore/partret_cleanup_comment_and_flow
...
Cleanup partial retrieval codepaths ( zero functional changes )
2021-11-29 16:39:55 +01:00
Łukasz Magiera
8d955d5f30
dagstore mount: Add random access support
2021-11-26 17:40:53 +01:00
Marten Seemann
cf0faf58dc
remove muxer config via LIBP2P_MUX_PREFS env
2021-11-26 11:01:54 +04:00
Marten Seemann
12dc3baff9
disable mplex stream muxer
2021-11-25 09:38:57 -08:00
Peter Rabbitson
d5b4d93575
Cleanup partial retrieval codepaths ( zero functional changes )
...
- Adjust the ExportRef comment to reflect change from #7673
- Simplify control flow in parseDagSpec() ( read diff under -w )
2021-11-25 10:35:05 +01:00
Łukasz Magiera
0c884734a3
Merge pull request #7673 from filecoin-project/chore/DM-level-tests_plus_merkle-proof-cars
...
Chore/dm level tests plus merkle proof cars
2021-11-24 20:43:39 +01:00
Łukasz Magiera
bd4927b494
retrieval: Cleanup some comments
2021-11-24 20:13:49 +01:00
Peter Rabbitson
46937a1b9f
Add tests cementing the DM-level behavior on simple paths
2021-11-24 15:31:15 +01:00
Łukasz Magiera
78949d6c05
retrieval: Fix deadlock in ClientGetRetrievalUpdates
2021-11-23 18:32:56 +01:00
Łukasz Magiera
407c2ed114
retrieval: Drop the RootSelector hack
2021-11-23 17:42:43 +01:00
Łukasz Magiera
25e89d3a7a
retrieval: require unixfs exports to be aligned on block boundary
2021-11-22 13:29:09 +01:00
Łukasz Magiera
9ea229ed5a
retrieval: fix defult ask
2021-11-22 13:04:12 +01:00
Łukasz Magiera
2c583b03ff
retrieval: Support DM-paths in ls
2021-11-22 12:46:29 +01:00
Łukasz Magiera
74f645a098
retrieval: Fix parseDagSpec root check
2021-11-22 12:46:29 +01:00
Łukasz Magiera
597b72e286
retrieval: Fix lint, cli docsgen
2021-11-22 12:46:29 +01:00
Łukasz Magiera
9c119bfdad
retrieval: Make the ls command work
2021-11-22 12:46:17 +01:00
Łukasz Magiera
450d0687da
retrieval: REST export endpoint
2021-11-22 12:46:17 +01:00
Łukasz Magiera
b83a9b902a
gofmt
2021-11-22 12:46:17 +01:00
Łukasz Magiera
b26906963b
retrieval: Support multi-root export
2021-11-22 12:46:17 +01:00
Łukasz Magiera
b9bd061bdd
fix unixfs selector root node selection
2021-11-22 12:46:04 +01:00
Łukasz Magiera
b868769ec8
more retrieval api work
2021-11-22 12:46:02 +01:00
Łukasz Magiera
89138bab4d
Simplify retrieval APIs
2021-11-22 12:45:42 +01:00
Łukasz Magiera
60ea33b1c7
client: cleanup clientRetrieve
2021-11-22 12:45:42 +01:00
Łukasz Magiera
08e297a217
client: Support json selectors in retrieval
2021-11-22 12:45:42 +01:00
Aayush Rajasekaran
32fc03886d
CLI: Add a lotus multisig cancel command
2021-11-18 19:21:17 -05:00
Aarsh Shah
7655e660f3
integrate store-the-hash
2021-11-18 12:07:12 +04:00
Anton Evangelatov
5454aebf13
CLI tools for the DAGStore inverted index ( #7361 )
...
* cli commands for dagstore
* address comments from Dirk
* rename inverted index to piece index
2021-11-17 15:39:45 +02:00
Dirk McCormick
501308239f
feat: CLI command to announce deal to indexers
2021-11-17 12:16:22 +01:00
Aarsh Shah
8d3017febb
make gen should work
2021-11-17 15:02:11 +04:00
Aarsh Shah
caf3cfa7d8
Merge branch 'feat/cid-to-piece-idx' into feat/indexer-provider
2021-11-17 11:43:49 +04:00
Aarsh Shah
bfee6699d2
update go mod and inject store the hash index in dagstore
2021-11-17 11:42:14 +04:00
Dirk McCormick
627d4fbf74
wip: storage miner index provider
2021-11-16 17:45:42 +01:00
Matija Petrunić
df17ea06f5
Merge branch 'master' into libp2p-pubsub-tracer
2021-11-10 12:47:09 +00:00
Matija Petrunic
59040d4419
Trace reject messages in pubsub
2021-11-10 13:45:40 +01:00
TheMenko
b45924a234
annotated repo_test
2021-11-01 10:49:30 +01:00
TheMenko
fb911a4542
annotated repo_test
2021-11-01 01:24:26 +01:00
TheMenko
9297e1b90d
annotated repo_test
2021-10-31 12:24:07 +01:00
Łukasz Magiera
d2e9d21031
Gather graphsync metrics on provider side as well
2021-10-19 19:45:25 +02:00
Łukasz Magiera
32a855b984
Fix lint
2021-10-19 19:22:32 +02:00
Łukasz Magiera
9a993d25d0
Collect and expose graphsync metrics
2021-10-19 19:20:00 +02:00
Jiaying Wang
afe54ab5f3
Merge pull request #7511 from filecoin-project/frrist/fix-lily
...
fix: support node instantiation in external packages
2021-10-13 20:33:17 -04:00
frrist
6cd6732225
fix: support node instantiation in external packages
...
- implement ChainGetPath on ChainModuleAPI
2021-10-13 15:24:28 -07:00
Anton Evangelatov
6e5ccc87cf
cli: add retry for deals stuck in Publish with no funds
2021-10-11 14:03:55 +02:00
Peter Rabbitson
a40aa9f90b
Normlize selector use within lotus
2021-10-07 11:01:29 +02:00
Peter Rabbitson
f8ac98bf46
Adjust error wording
2021-10-05 18:16:14 +02:00
Peter Rabbitson
5e6aceeb0e
Merge remote-tracking branch 'origin/master' into feat/datamodel-selector-retrieval
2021-10-05 18:15:51 +02:00
Jiaying Wang
cbb147de23
Merge pull request #7405 from filecoin-project/feat/update-graphsync-0.10.0
...
Update to graphsync to v0.10.0, enable seperate storage and retrieval transfer limits
2021-10-05 11:50:59 -04:00
Rod Vagg
97c2cdb8d6
traversals: limit link load count for SelectiveCar use as well
2021-10-05 10:47:49 +02:00
Rod Vagg
43f7fd5e10
traversals: limit maximum number of DAG links to traverse
...
Impacts CommP and graphsync transfers
2021-10-05 10:47:49 +02:00
Dirk McCormick
a4e2fce9ca
feat: update to go-fil-markets v1.13.1
2021-10-05 10:40:01 +02:00
hannahhoward
368d72ebfe
feat(graphsync): update to v0.10.0-rc1
...
also add config changes
2021-10-05 14:13:58 +11:00
Peter Rabbitson
af0d9b650d
Error on corner cases with corresponding negative tests
2021-10-04 23:21:24 +02:00
Peter Rabbitson
2a1094fc17
Merge banch feat/go-fil-markets-v1.13.0 into feat/datamodel-selector-retrieval
2021-10-04 21:56:11 +02:00
Dirk McCormick
16973bc2e8
feat: use go-fil-markets@v1.13.0
2021-10-04 15:41:40 +02:00
Jennifer Wang
38e01cba83
Merge branch 'release/v1.12.0' into jen/12tomaster
2021-10-04 02:38:00 -04:00
Aayush Rajasekaran
f004d036dc
Set BatchPreCommitAboveBaseFee correctly
2021-10-01 14:23:41 -04:00
Łukasz Magiera
6fd9d5f28b
Precommit batch balancer support/config
2021-10-01 14:23:41 -04:00
Anton Evangelatov
809289f5ef
add Dealmaking.StartEpochSealingBuffer config
2021-10-01 17:44:15 +02:00
Łukasz Magiera
a8a9818043
Expose storage states on the metrics endpoint
2021-10-01 14:45:01 +02:00
Łukasz Magiera
7c84e22c3e
Add envconfig docs to the config
2021-09-30 18:23:05 +02:00
Aayush Rajasekaran
c3c46e9097
Fix Drand fetching around null tipsets
2021-09-29 11:57:59 -04:00
Mak Muftic
530df4298e
Fix linter errors
2021-09-29 16:49:03 +02:00
Mak Muftic
ea7673618d
Add unit tests for lotus tracer
2021-09-29 16:39:44 +02:00
Matija Petrunic
53c8e68fba
Remove extra tracer dtypes
2021-09-29 16:23:08 +02:00
Matija Petrunic
181b35dc5e
Extend peer score event with score components
2021-09-29 16:20:43 +02:00
Mak Muftic
7b19beaf5d
Fix linter errors
2021-09-29 13:12:42 +02:00
Mak Muftic
3433fa4a6e
Add stringdocs to config props and generate docs
2021-09-29 12:51:43 +02:00
Jennifer Wang
d7690aa719
Merge branch 'releases' into jen/backport
2021-09-29 03:21:19 -04:00
Mak Muftic
36913f4efc
Fix elasticsearch index setup
2021-09-27 13:28:28 +02:00
Matija Petrunic
6c680e306d
Rename json tracer flag
2021-09-24 16:25:41 +02:00
Mak Muftic
30fc2387be
Merge branch 'mpetrun5/lotus-extended-tracer' of github.com:ChainSafe/lotus into mpetrun5/lotus-extended-tracer
2021-09-24 14:16:31 +02:00
Mak Muftic
9332b9f399
Change json config naming
2021-09-24 13:53:31 +02:00
Mak Muftic
4438c4bd87
Move index name to config only and add default value
2021-09-24 13:43:25 +02:00
Mak Muftic
22bbb113e7
Fix schema creation
2021-09-24 12:58:32 +02:00
Mak Muftic
94a1601ab2
Remove hardcoded url schema
2021-09-24 12:40:33 +02:00
Mak Muftic
e46da5a181
Add cli flags
2021-09-24 12:37:16 +02:00
Łukasz Magiera
8e0314b3a3
Merge pull request #7362 from filecoin-project/chore/update-libp2p
...
Update go-libp2p to v0.15.0
2021-09-23 14:24:12 +01:00
Aayush Rajasekaran
b9bfcc4b69
Randomness: Move getters from ChainAPI to StateAPI
2021-09-21 14:20:15 -04:00
Anton Evangelatov
c7e4217693
fix staging area path buildup
2021-09-21 16:45:00 +02:00
Aarsh Shah
89ac8b09b5
update libp2p
2021-09-21 15:10:04 +04:00
Matija Petrunic
0c390d12f7
Append source auth flag to lotus tracer event
2021-09-21 12:54:07 +02:00
Matija Petrunic
d65d9aa6e0
Send peerID peer score as separate events to enable usage of kibana runtime fields
2021-09-21 12:30:50 +02:00
Mak Muftic
082f70757c
Revert additional logging in elastic client
2021-09-20 12:29:12 +02:00
Mak Muftic
a685ccc955
Remove document id
2021-09-17 15:14:39 +02:00
Mak Muftic
0962e73e3e
Add loger to elastic search client
2021-09-17 14:22:57 +02:00
Matija Petrunic
7d0aefacbb
Change peer score event type to avoid exceeding elastisearch field limit
2021-09-17 13:46:26 +02:00
Matija Petrunic
a91ffa60d6
Set peer id event as string in lotus trace event
2021-09-17 12:11:14 +02:00
Mak Muftic
dc79ea363d
Change index name
2021-09-17 11:35:36 +02:00
ldoublewood
eb2bbfebe3
format code by make gen
2021-09-17 00:20:24 +08:00
Mak Muftic
f09f5c2d11
Change elastic document name
2021-09-16 16:49:47 +02:00
Mak Muftic
8e2b474fc7
Use only one document on elastic
2021-09-16 16:44:35 +02:00
Matija Petrunic
9efa495b0e
Write every trace to new line when using json transport
2021-09-16 16:42:18 +02:00
Matija Petrunic
941a0f7153
Update peer score tracker creation to include lotus tracer
2021-09-16 16:31:36 +02:00
Matija Petrunic
9cdc5261c4
Fix lotus tracer being nil when remote tracer is not configured
2021-09-16 15:53:39 +02:00
Matija Petrunic
d067bc9f01
Move creating and opening file to json transport constructor
2021-09-16 15:22:18 +02:00
Matija Petrunic
e2206147cc
Add source auth token to lotus traces
2021-09-16 12:21:57 +02:00
Mak Muftic
c25a5e06cf
Use config string for elasticsearch client
2021-09-16 12:15:42 +02:00
ldoublewood
3a1fc8e7da
fix bug for CommittedCapacitySectorLifetime
2021-09-16 09:25:02 +08:00
Mak Muftic
5cbd2519da
Remove processing elasticsearch result
2021-09-15 16:20:36 +02:00
Mak Muftic
67c1d633c6
Rename jsonEvent to evt
2021-09-15 16:16:12 +02:00
Mak Muftic
add13c2646
Code cleanup
2021-09-15 16:05:44 +02:00
Mak Muftic
71a2ca9e7c
Add new wrapper structure in tracer transport
2021-09-15 16:03:51 +02:00
Mak Muftic
707faf57b5
Code cleanup
2021-09-15 15:11:34 +02:00
Mak Muftic
e20cd0ef27
Propagate error on creating transport
2021-09-15 14:58:36 +02:00
Mak Muftic
cf06096565
Fix tracer initialization
2021-09-15 14:54:20 +02:00
Mak Muftic
bb85ca031d
Add elasticsearch transport
2021-09-15 14:50:27 +02:00
Matija Petrunic
c064ce4852
Add generated documentation
2021-09-15 14:22:48 +02:00
Matija Petrunic
8a657b90a7
Refactor lotus tracer to behave like lp2p tracer
2021-09-15 13:41:56 +02:00
Matija Petrunic
ab86c5419d
Implement json tracer transport
2021-09-15 13:41:03 +02:00
Matija Petrunic
8e6a01de6d
Add configuration in pubsub to trace locally to json file
2021-09-15 13:40:54 +02:00
Aayush Rajasekaran
1d6db306ec
Randomness: Move getters from ChainAPI to StateAPI
2021-09-15 00:03:13 +02:00
Matija Petrunic
d5a4b0637a
Add lotus tracer transport calls
2021-09-10 15:37:38 +02:00