cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity
16,728 Commits 24 Branches 342 Tags 314 MiB
0ad96858ff
Commit Graph

1345 Commits

Author SHA1 Message Date
Łukasz Magiera
1aed631fe5 sealing: DataCid on workers 2022-04-26 20:36:23 +02:00
Travis Person
ea4f599233 Make MarketListDeals and DealsList return points for MarketDeal 2022-04-22 23:26:29 +00:00
Geoff Stuart
49d6c07280 Implemented StateLoockupRobustAddress 2022-04-14 17:41:18 -04:00
Aayush
339b7db2b7 Integrate FIP 0027 2022-04-14 19:32:56 +03:00
Łukasz Magiera
6bef1aeb82 feat: cli: lotus client list-asks --protocols 2022-04-11 19:49:52 +02:00
Łukasz Magiera
54cb55a7ae feat: miner: API/CLI to compute window-post 2022-03-28 17:06:51 -04:00
Łukasz Magiera
7401fa234e
Merge pull request #7971 from filecoin-project/feat/post-worker 
feat: PoSt workers
 2022-03-25 20:31:49 -04:00
Łukasz Magiera
82343a06e7 Merge remote-tracking branch 'origin/master' into feat/post-worker 2022-03-25 16:49:46 -04:00
Łukasz Magiera
5f9753933a
chore: Remove temp gomock reflect file 2022-03-25 16:46:50 -04:00
Łukasz Magiera
701d0a111e Merge branch 'feat/net-ping' of github.com:ychiaoli18/lotus into feat/net-ping 2022-03-21 13:06:11 +01:00
Łukasz Magiera
6ddbe41376 Merge remote-tracking branch 'origin/master' into feat/post-worker 2022-03-18 10:54:44 +01:00
Łukasz Magiera
f2c0ba1df5
Merge pull request #8329 from filecoin-project/fix/snap-worker-cleanup 
fix: sealing: Remove sector copies from workers after snapdeals
 2022-03-16 18:48:21 +01:00
Darko Brdareski
79453663b3
feat: Add additional test annotations (#8272) 
* Annotate api,proxy_util,blockstore_badger, policy tests

* Annotate splitstore: bsbadger / markset

* Annotate splitstore feature

* Annotate union/timed blockstore tests

* Annotate openrpc, diff_adt tests

* Annotate error,drand,events tests

* Annotate predicates_test

* Fix annotations

* Annotate tscache, gen tests

* Annotate fundmanager test

* Annotate repub and selection tests

* Annotate statetree_test

* Annotate forks_test

* Annotate searchwait_test.go

* Fix duplicated @@ symbols

* Annotate chain stmgr/store tests

* Annotate more (types) tests

* More tests annotated

* Annotate conformance chaos actor tests

* Annotate more integration tests

* Annotate journal system tests

* Annotate more tests.

* Annotate gas,head buffer behaviors

* Fix markset annotations

* doc: test annotations for the markets dagstore wrapper

* Annotate miner_api test in dagstore

* Annotate more test files

* Remove bad annotations from fsrepo

* Annotate wdpost system

* Remove bad annotations

* Renamce "conformance" to "chaos_actor" tests

* doc: stm annotations for blockheader & election proof tests

* Annotate remaining "A" tests

* annotate: stm for error_test

* memrepo_test.go

* Annotate "b" file tests

* message_test.go

* doc: stm annotate for fsrepo_test

* Annotate "c" file tests

* Annotate "D" test files

* message_test.go

* doc: stm annotate for chain, node/config & client

* docs: stm annotate node_test

* Annotate u,v,wl tests

* doc: stm annotations for various test files

* Annotate "T" test files

* doc: stm annotate for proxy_util_test & policy_test

* doc: stm annotate for various tests

* doc: final few stm annotations

* Add mempool unit tests

* Add two more memPool Add tests

* Update submodules

* Add check function tests

* Add stm annotations, refactor test helper

* Annotate api,proxy_util,blockstore_badger, policy tests

* Annotate splitstore: bsbadger / markset

solving merge conflicts

* Annotate splitstore feature

* Annotate union/timed blockstore tests

* Annotate openrpc, diff_adt tests

* Annotate error,drand,events tests

* Annotate predicates_test

* Fix annotations

* Annotate tscache, gen tests

* Annotate fundmanager test

* Annotate statetree_test

* Annotate forks_test

* Annotate searchwait_test.go

* Fix duplicated @@ symbols

* Annotate chain stmgr/store tests

* Annotate more (types) tests

* More tests annotated

* Annotate conformance chaos actor tests

* Annotate more integration tests

* Annotate journal system tests

* Annotate more tests.

* Annotate gas,head buffer behaviors

solve merge conflict

* Fix markset annotations

* Annotate miner_api test in dagstore

* Annotate more test files

* doc: test annotations for the markets dagstore wrapper

* Annotate wdpost system

* Renamce "conformance" to "chaos_actor" tests

* Annotate remaining "A" tests

* doc: stm annotations for blockheader & election proof tests

* annotate: stm for error_test

* Annotate "b" file tests

* memrepo_test.go

* Annotate "c" file tests

* message_test.go

* Annotate "D" test files

* doc: stm annotate for fsrepo_test

* Annotate u,v,wl tests

* message_test.go

* doc: stm annotate for chain, node/config & client

* docs: stm annotate node_test

* Annotate "T" test files

* doc: stm annotations for various test files

* Add mempool unit tests

solve merge conflict

* doc: stm annotate for proxy_util_test & policy_test

* doc: stm annotate for various tests

* doc: final few stm annotations

* Add two more memPool Add tests

* Update submodules

* Add check function tests

solve conflict

* Add stm annotations, refactor test helper

solve merge conflict

* Change CLI test kinds to "unit"

* Fix double merged test

* Fix ccupgrade_test merge

* Fix lint issues

* Add stm annotation to types_Test

* Test vectors submodule

* Add file annotation to burn_test

Co-authored-by: Nikola Divic <divicnikola@gmail.com>
Co-authored-by: TheMenko <themenkoprojects@gmail.com>
 2022-03-16 18:37:34 +01:00
Łukasz Magiera
9f6f94bd02
Merge pull request #8215 from filecoin-project/feat/remove-mark-for-upgrade 
chore:sealing:remove endpoint from cli
 2022-03-16 15:33:21 +01:00
Łukasz Magiera
45b07674e5 stores: http: Support multiple storage IDs in ?keep 2022-03-16 12:28:56 +01:00
Jennifer Wang
8a2a2d065d Merge branch 'releases' into jen/backport 2022-03-15 02:07:35 -04:00
Łukasz Magiera
135aef78d7 Merge remote-tracking branch 'origin/master' into feat/post-worker 2022-03-11 17:04:58 +01:00
Łukasz Magiera
9c4f8254ee post workers: Fix build post-merge 2022-03-10 14:20:07 +01:00
hannahhoward
49742f8fdc feat(deps): update to graphsync v0.13.0 with 2.0 protocol 2022-03-09 18:06:35 +00:00
Łukasz Magiera
046a9f8af0 Merge remote-tracking branch 'origin/master' into feat/post-worker 2022-03-09 16:27:03 +01:00
Aayush
e9e28f75e2 Revert "add resource manager NetStat api" 
This reverts commit d05d5bcb70.
 2022-03-02 15:39:22 -05:00
Aayush
b8747e3afc Revert "add resource manager Net(Set)Limit api" 
This reverts commit 554848fe6b.
 2022-03-02 15:39:21 -05:00
Aayush
bac17061d4 Revert "add examples to docgen" 
This reverts commit 39bf59d372.
 2022-03-02 15:39:21 -05:00
Aayush
7dbf0ced82 Revert "make gen and friends" 
This reverts commit b360c9403f.
 2022-03-02 15:39:14 -05:00
Masih H. Derkani
aacc246ba3
Merge branch 'master' into feat/cid-to-piece-idx 2022-03-02 14:06:30 +00:00
Łukasz Magiera
e9a6f5f287 Merge remote-tracking branch 'origin/master' into feat/paych-avail-reuse 2022-03-02 12:56:56 +00:00
zenground0
a463c90057 remove endpoint from cli 2022-03-01 11:27:03 -07:00
zenground0
77a954c7c3 Merge branch 'master' into feat/cid-to-piece-idx 2022-02-28 12:57:11 -07:00
jennijuju
c6361b391c Merge branch 'releases' into jen/v15 2022-02-25 21:01:31 -05:00
zenground0
e5de237990 Fix fault tracker to handle snap deals 2022-02-24 17:53:21 -05:00
zenground0
abe04c33c1 Fix fault tracker to handle snap deals 2022-02-23 09:56:47 -07:00
Łukasz Magiera
a0cb6094c6 Merge remote-tracking branch 'origin/master' into feat/paych-avail-reuse 2022-02-22 10:55:03 +01:00
Aarsh Shah
e3f5b7c59f update dagstore top level index db 2022-02-18 14:00:01 +04:00
Kevin Li
ba72eff3e6 feat: cli/net: implement 'net ping' command 2022-02-17 17:22:52 +08:00
Łukasz Magiera
384999556c paychmgr: AvailableAmt -> NonReservedAmt 2022-02-16 20:39:43 +01:00
Kevin Li
f677995603 feat: add MsigGetVestingSchedule to gateway api 
expose MsigGetVestingSchedule method to gateway api
 2022-02-16 16:11:46 +08:00
Łukasz Magiera
60b9acc09d gen 2022-02-14 21:16:41 +01:00
Łukasz Magiera
df5bd147fa
paychmgr: Review comments 
Co-authored-by: Hannah Howard <hannah@hannahhoward.net>
Co-authored-by: dirkmc <dirkmdev@gmail.com>
 2022-02-14 20:07:13 +01:00
Łukasz Magiera
f61eb23f8f api: separate method for paych funding 2022-02-14 19:56:02 +01:00
Łukasz Magiera
10af768c60 Merge commit 'origin/release/v1.15.0~2' into feat/paych-avail-reuse 2022-02-14 19:27:12 +01:00
Łukasz Magiera
53e1f75c2c Add FinalizeReplicaUpdate into some more places 2022-02-10 17:27:49 -05:00
Łukasz Magiera
212f5ddb4f wip FinalizeReplicaUpdate 2022-02-10 17:24:26 -05:00
Jennifer Wang
de1f4792cd Merge remote-tracking branch 'origin/master' into jen/v15 2022-02-09 19:27:34 -05:00
Jennifer Wang
6ed97f0d04 Merge branch 'release/v1.14.0' into jen/v15 2022-02-09 03:17:38 -05:00
Łukasz Magiera
09cfad9d71 Add FinalizeReplicaUpdate into some more places 2022-02-08 17:22:41 +01:00
Łukasz Magiera
142ba6660a wip FinalizeReplicaUpdate 2022-02-08 17:22:41 +01:00
Will Scott
7247f8e41f
add net protect api methods 
fix #8015
 2022-02-03 17:24:49 +01:00
Aayush
1b18236f91 feat: sealer: allow users to abort in-flight snap upgrades 2022-01-25 13:01:51 -05:00
Łukasz Magiera
8b19b84140 paych: option to force off-chain get 2022-01-20 18:19:26 +01:00
Łukasz Magiera
8f6f21c94c paych: Print available amounts in paych status 2022-01-20 18:16:00 +01:00
Łukasz Magiera
9715113898 paych: Test pre-funding 2022-01-20 18:15:59 +01:00
Łukasz Magiera
8e46b9ea5d paych: API to pre-fund channels 2022-01-20 18:15:46 +01:00
Aarsh Shah
9ee940098d Merge remote-tracking branch 'origin/master' into feat/cid-to-piece-idx 2022-01-20 16:21:48 +04:00
Łukasz Magiera
e476cf7968 Merge remote-tracking branch 'origin/master' into feat/post-worker 2022-01-20 13:15:48 +01:00
vyzo
b360c9403f make gen and friends 2022-01-20 11:44:01 +02:00
vyzo
39bf59d372 add examples to docgen 2022-01-20 11:36:11 +02:00
vyzo
554848fe6b add resource manager Net(Set)Limit api 2022-01-20 11:36:11 +02:00
vyzo
d05d5bcb70 add resource manager NetStat api 2022-01-20 11:36:11 +02:00
Łukasz Magiera
b38141601c Untangle ffi from api 2022-01-18 11:57:04 +01:00
Łukasz Magiera
f2496d87c9 post workers: Snapdeals updates 2022-01-18 11:25:04 +01:00
Łukasz Magiera
0c062f9c08 Merge remote-tracking branch 'origin/master' into feat/post-worker 2022-01-18 10:49:42 +01:00
Łukasz Magiera
7bd9af7856 address review 2022-01-14 17:14:32 -05:00
zenground0
d6aa17e21f Snap Deals Integration 
- FSM handles the actual cc upgrade process including error states
- PoSting (winning and window) works over upgraded and upgrading sectors
- Integration test and changes to itest framework to reduce flakes
- Update CLI to handle new upgrade
- Update dependencies
 2022-01-14 17:14:32 -05:00
zenground0
d1480c36c0 RemoveData and Decode 
- Unsealing replica update with sector key works and tested
- Sector key generation added and tested
 2022-01-14 17:14:32 -05:00
zenground0
93656e65f8 WIP sector storage and integration test 2022-01-14 17:14:32 -05:00
Łukasz Magiera
4a874eff70 post workers: Cleanup, tests 2022-01-14 14:17:52 +01:00
Aarsh Shah
8c8652e7bb add a new cli 2022-01-12 18:06:48 +04:00
Aarsh Shah
edfc2c215c Merge remote-tracking branch 'origin/main' into feat/cid-to-piece-idx 2022-01-12 12:48:54 +04:00
Łukasz Magiera
960759d22b address review 2022-01-11 17:31:27 +01:00
Łukasz Magiera
e216aefd23 fix make gen 2022-01-10 18:24:00 +01:00
Łukasz Magiera
cf7dd36829 Merge branch 'feat/decoupling-post-worker' of github.com:wusirdcenter/lotus into feat/decoupling-post-worker 2022-01-10 15:06:04 +01:00
zenground0
33f2d24f54 Snap Deals Integration 
- FSM handles the actual cc upgrade process including error states
- PoSting (winning and window) works over upgraded and upgrading sectors
- Integration test and changes to itest framework to reduce flakes
- Update CLI to handle new upgrade
- Update dependencies
 2022-01-10 15:39:38 +05:30
mz-sirius
793b5c7cc3 fix ci err 2022-01-05 21:41:21 +08:00
mz-sirius
3fd55fa56b decoupling winningpost and windowpost from lotus-miner 2022-01-05 01:50:49 +08:00
zl
4172a3c8b7 ExampleValue for a silce is nil 2022-01-04 14:27:10 +08:00
hannahhoward
cddf63efe9 feat(storageminer): add api for transfer diagnostics 
Add API + CLI for inspecting in depth diagnostics on graphsync transfers with a given peer
 2021-12-22 13:41:29 -08:00
Dirk McCormick
b1734f84b3 feat: retrieval ask CLI command 2021-12-17 15:55:12 +01:00
Aayush Rajasekaran
80d5e52923 Merge branch 'master' into next 2021-12-13 13:24:28 -05:00
zenground0
a5be80828a RemoveData and Decode 
- Unsealing replica update with sector key works and tested
- Sector key generation added and tested
 2021-12-03 15:21:06 -05:00
Łukasz Magiera
727765b248 Command to list active sector locks 2021-12-03 12:33:23 +01:00
Łukasz Magiera
6d52d8552b Fix docsgen 2021-11-30 02:06:58 +01:00
Łukasz Magiera
c9a2ff4007 cleanup worker resource overrides 2021-11-30 02:06:58 +01:00
Clint Armstrong
4ef8543128 Permit workers to override resource table 
In an environment with heterogenious worker nodes, a universal resource
table for all workers does not allow effective scheduling of tasks. Some
workers may have different proof cache settings, changing the required
memory for different tasks. Some workers may have a different count of
CPUs per core-complex, changing the max parallelism of PC1.

This change allows workers to customize these parameters with
environment variables. A worker could set the environment variable
PC1_MIN_MEMORY for example to customize the minimum memory requirement
for PC1 tasks. If no environment variables are specified, the resource
table on the miner is used, except for PC1 parallelism.

If PC1_MAX_PARALLELISM is not specified, and
FIL_PROOFS_USE_MULTICORE_SDR is set, PC1_MAX_PARALLELSIM will
automatically be set to FIL_PROOFS_MULTICORE_SDR_PRODUCERS + 1.
 2021-11-30 02:06:58 +01:00
Clint Armstrong
93e4656a27 Use a float to represent GPU utilization 
Before this change workers can only be allocated one GPU task,
regardless of how much of the GPU resources that task uses, or how many
GPUs are in the system.

This makes GPUUtilization a float which can represent that a task needs
a portion, or multiple GPUs. GPUs are accounted for like RAM and CPUs so
that workers with more GPUs can be allocated more tasks.

A known issue is that PC2 cannot use multiple GPUs. And even if the
worker has multiple GPUs and is allocated multiple PC2 tasks, those
tasks will only run on the first GPU.

This could result in unexpected behavior when a worker with multiple
GPUs is assigned multiple PC2 tasks. But this should not suprise any
existing users who upgrade, as any existing users who run workers with
multiple GPUs should already know this and be running a worker per GPU
for PC2. But now those users have the freedom to customize the GPU
utilization of PC2 to be less than one and effectively run multiple PC2
processes in a single worker.

C2 is capable of utilizing multiple GPUs, and now workers can be
customized for C2 accordingly.
 2021-11-30 02:06:58 +01:00
Clint Armstrong
c4f46171ae Report memory used and swap used in worker res 
Attempting to report "memory used by other processes" in the MemReserved
field fails to take into account the fact that the system's memory used
includes memory used by ongoing tasks.

To properly account for this, worker should report the memory and swap
used, then the scheduler that is aware of the memory requirements for a
task can determine if there is sufficient memory available for a task.
 2021-11-30 02:06:58 +01:00
Łukasz Magiera
320b36495d gofmt 2021-11-29 21:39:27 +01:00
Łukasz Magiera
d019853687
review: Cleanup some comments 
Co-authored-by: Peter Rabbitson <ribasushi@protocol.ai>
 2021-11-29 21:29:00 +01:00
Łukasz Magiera
5b5e6b9e44 retrieval: DagSpec.MatchPath -> ExportMerkleProof 2021-11-29 21:14:00 +01:00
Łukasz Magiera
61791b90ea retrieval: Only output matching nodes, MatchPath dagspec 2021-11-29 20:40:55 +01:00
zenground0
7d2b3f05db WIP sector storage and integration test 2021-11-29 10:24:00 -05:00
Peter Rabbitson
d5b4d93575 Cleanup partial retrieval codepaths ( zero functional changes ) 
- Adjust the ExportRef comment to reflect change from #7673
- Simplify control flow in parseDagSpec() ( read diff under -w )
 2021-11-25 10:35:05 +01:00
Łukasz Magiera
407c2ed114 retrieval: Drop the RootSelector hack 2021-11-23 17:42:43 +01:00
Łukasz Magiera
bdac11ade7 retrieval: Update some API comments 2021-11-22 14:19:03 +01:00
Łukasz Magiera
763659b8a3 retrieval: Update lotus-soup 2021-11-22 12:46:29 +01:00
Łukasz Magiera
ec2bfb99bb make gen 2021-11-22 12:46:25 +01:00
Łukasz Magiera
b83a9b902a gofmt 2021-11-22 12:46:17 +01:00
Łukasz Magiera
b26906963b retrieval: Support multi-root export 2021-11-22 12:46:17 +01:00
Łukasz Magiera
b0c043cc2f make gen 2021-11-22 12:46:15 +01:00
Łukasz Magiera
b868769ec8 more retrieval api work 2021-11-22 12:46:02 +01:00
Łukasz Magiera
89138bab4d Simplify retrieval APIs 2021-11-22 12:45:42 +01:00