cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity
21,287 Commits 24 Branches 342 Tags 314 MiB
3b7f71f533
Commit Graph

317 Commits

Author SHA1 Message Date
Raúl Kripalani
a411342f0c export more metrics. 2020-11-04 17:09:01 +00:00
Raúl Kripalani
370ef93db6 chainstore: new ForceHeadSilent to support benchmarks. 2020-11-04 14:39:01 +00:00
Raúl Kripalani
fd1439f284 fix lint. 2020-11-03 22:49:55 +00:00
Łukasz Magiera
a1e1b03ca4 Optionally allow bitswap for chainstore 2020-11-03 23:44:44 +01:00
Raúl Kripalani
87ce2f774d lotus-bench: write separate profiles for import and validation. 2020-11-03 22:40:21 +00:00
Raúl Kripalani
5487356370 fix lint errors. 2020-11-03 21:23:41 +00:00
Raúl Kripalani
18418128c3 lotus-bench: make start tipset walkback start from end tipset. 2020-11-03 18:15:59 +00:00
Raúl Kripalani
581ac5b019 lotus-bench: add ability to specify tipsets. 2020-11-03 18:11:43 +00:00
Raúl Kripalani
842c8ca056 improve lotus-bench; support running validation on a store without a CAR. 2020-11-03 17:35:31 +00:00
zgfzgf
3e7efad0fd modify lint error 2020-11-03 21:28:01 +08:00
zgfzgf
3c2cd506ec modify merge lotus-bench 2020-11-03 19:56:04 +08:00
Raúl Kripalani
f4e13ffd80 actually register prometheus gauge. 2020-11-01 20:01:57 +00:00
Raúl Kripalani
44e34d9606 lotus-bench: native badger SyncWrites=false. 2020-11-01 19:54:47 +00:00
Raúl Kripalani
35ccd73a86 lotus-bench: improve for observability. 2020-11-01 19:52:53 +00:00
Raúl Kripalani
aece624d1c lotus-bench: add support for native badger blockstore. 2020-11-01 18:54:35 +00:00
Raúl Kripalani
c2355b18f9 write heap and allocs profile in lotus-bench. 2020-11-01 18:00:08 +00:00
Łukasz Magiera
660236b224 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-10-23 23:25:35 +02:00
Łukasz Magiera
041762a34f
Merge pull request #4383 from filecoin-project/steb/actor-v2-imports 
Remove most v0 actor imports
 2020-10-22 17:17:56 +02:00
Łukasz Magiera
0832d5a7d5 bench: Fix lint warnings 2020-10-22 15:00:31 +02:00
whyrusleeping
602d2b8ad5 record datastore metrics after successful run 2020-10-22 14:57:06 +02:00
Jakub Sztandera
0341944350 Inject prometheus 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-10-22 14:57:05 +02:00
whyrusleeping
00d1e6b549 add measure datastore wrapper around bench chain datastore 2020-10-22 14:57:04 +02:00
Steven Allen
4e730b5ec8 port to v2 imports 2020-10-21 12:16:23 -07:00
Łukasz Magiera
71b3b9075d Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-10-13 21:33:21 +02:00
zgfzgf
bc1f18b73a lotus bench optimize 2020-10-11 18:55:12 +08:00
zgfzgf
c3a405eac7 add lotus bench print 2020-10-11 18:26:27 +08:00
zgfzgf
e4209d86f6 lotus bench add ticket-preimage 2020-10-11 17:20:55 +08:00
zgfzgf
fcb9dea501 add sum sealing time 2020-10-11 16:31:39 +08:00
zgfzgf
c810b157cf lotus bench delete ix 2020-10-11 11:39:27 +08:00
zgfzgf
c458f6e826 lotus bench replace abi.SectorID with sid 2020-10-11 11:25:19 +08:00
zgfzgf
c99e223cb1 lotus bench change and add print 2020-10-11 10:48:09 +08:00
zgfzgf
5c27d6aeee lotus-bench UnsealPiece SectorID.Number error 2020-10-11 10:12:06 +08:00
Steven Allen
748d2e82a7 unshare the journal 
Motivation:

* Run lotus with the race detector enabled (primary motivation).
* Allow multiple lotus nodes in a process (not a high priority).

Previously, the journal was shared between all lotus instances, but it was
initialized for every new node. This caused safety problems in tests (at a
minimum).

This patch explicitly passes the journal to all services that need it.
 2020-10-09 13:23:07 -07:00
Łukasz Magiera
2cfe22d4e5 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-09-30 20:48:16 +02:00
Jakub Sztandera
7e8c6e5070
Remove statediff, fix lint, go mod tidy 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-09-29 14:49:47 +02:00
Jakub Sztandera
0d914ac1d4
Switch to badger 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-09-29 14:49:47 +02:00
Jakub Sztandera
76db65b1af
Update pebble 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-09-29 14:49:47 +02:00
Jakub Sztandera
ff8c0af8c8
Add only-import option 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-09-29 14:49:47 +02:00
Jakub Sztandera
55c6b88537
Add toggle for badger, flag out gas tracing 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-09-29 14:49:46 +02:00
Jakub Sztandera
0771c23fb0
Use pebble 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-09-29 14:49:46 +02:00
Jakub Sztandera
242a77b391
go mod tidy 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-09-29 14:49:18 +02:00
Jakub Sztandera
35cf69ae64
Disable bloomcache 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-09-29 14:49:18 +02:00
Jakub Sztandera
ebc8489ff1
Add global-profile option 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-09-29 14:49:18 +02:00
Jakub Sztandera
f21c5cbbe2
Add start-at 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-09-29 14:49:18 +02:00
Jakub Sztandera
01386a206c
Update options 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-09-29 14:49:18 +02:00
Jakub Sztandera
3858309368
Add http to import bench 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-09-29 14:49:17 +02:00
Jakub Sztandera
782717948a
Add logs 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-09-29 14:49:17 +02:00
Jakub Sztandera
108fe7823c
Add command to trigger gc 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-09-29 14:49:17 +02:00
Jakub Sztandera
12a0dd3d0a
<3 to linter 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-09-29 14:49:17 +02:00
Jakub Sztandera
1f4d1dcc58
Do not sync 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-09-29 14:49:17 +02:00
Jakub Sztandera
53ab17cf50
Add no import to import-bench 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-09-29 14:49:17 +02:00
Jakub Sztandera
79ba4598d6
Implement cache 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-09-29 14:49:17 +02:00
Jakub Sztandera
96193c2044
Implement bench-cache 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-09-29 14:49:11 +02:00
Łukasz Magiera
cf71f034dc Merge remote-tracking branch 'origin/dev' into feat/async-restartable-workers 2020-09-26 12:35:32 +02:00
Jakub Sztandera
4eec4a0141
Move policy change to seal bench 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-09-25 23:28:12 +02:00
Steven Allen
32a699d6a3 Add some actors policy setters for testing 
Addresses:
* a307e4593a (r491966115)
* a307e4593a (r491966634)

Note: This puts everything into a policy package to avoid a dependency cycle
between the build package, the miner package, and the types package. This is
also why I introduced a GetPreCommitChallengeDelay function and removed the
variable.
 2020-09-23 15:00:52 -07:00
Łukasz Magiera
aa5bd7bc17 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-09-21 22:36:16 +02:00
jennijuju
5c69249ba3 Use window post for window PoSt related log messages. 2020-09-15 21:22:29 -04:00
whyrusleeping
24756ac056 allow specification of repo directory in lotus-bench 2020-09-15 11:55:54 -07:00
Łukasz Magiera
5f08fe7ead Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-09-10 17:30:54 +02:00
Aayush Rajasekaran
d678fe4bfa Fix tests 2020-09-07 15:48:42 -04:00
Aayush Rajasekaran
39755a294a Update to specs v0.9.6 2020-09-07 15:48:41 -04:00
Łukasz Magiera
5d73943929 storage: Fix import cycle 2020-09-06 18:54:00 +02:00
Steven Allen
5733c71c50 Lint everything 
We were ignoring quite a few error cases, and had one case where we weren't
actually updating state where we wanted to. Unfortunately, if the linter doesn't
pass, nobody has any reason to actually check lint failures in CI.

There are three remaining XXXs marked in the code for lint.
 2020-08-20 20:46:36 -07:00
Raúl Kripalani
efdc428d5d keep storage-fsm (renamed to storage-sealing) and sector-storage in extern. 2020-08-17 14:26:18 +01:00
Raúl Kripalani
3c17cd655e integrate extern/sector-storage into lotus proper. 2020-08-16 11:09:58 +01:00
Steven Allen
3ef3f570fb Fix lint errors an broken tests 2020-08-12 23:16:44 +02:00
Aayush Rajasekaran
d2e203c1ce Fix import 2020-08-04 00:45:02 -04:00
Aayush Rajasekaran
5e485a085a
Merge pull request #2637 from filecoin-project/gas/tag-verify-post 
Use scaling VerifyPost cost
 2020-07-28 21:22:23 -04:00
Jakub Sztandera
b488012d5f
Add warn 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-28 22:31:43 +02:00
Jakub Sztandera
da4bc44032
Discard initial very long OnVerifyPost from analisys 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-28 22:31:43 +02:00
Steven Allen
b7a4dbb07f Support inline CIDs 
And use the new CidBuilder from the spec actors.

This patch does not switch over to inline CIDs by default, but paves the way.
 2020-07-23 23:12:32 -07:00
Frank
c38b8b1265 close file 2020-07-23 18:21:13 +08:00
Jakub Sztandera
610ed0dc13
Change OnIpldGet 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-15 21:40:49 +02:00
Jakub Sztandera
c0eb4b39ac
Reorg some gas charges 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-15 20:01:56 +02:00
Łukasz Magiera
8e0d33a1fd Updates for sectors-storge next 2020-07-08 17:23:27 +02:00
Jakub Sztandera
3429941639
Add info about OnActorExecDone 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:47:34 +02:00
Jakub Sztandera
41b775fdd4
Compute time spent in actors 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:47:34 +02:00
Jakub Sztandera
e9d2a3edd9
Limit float print precision 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:47:34 +02:00
Jakub Sztandera
42696372ee
Progress to stderr 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:47:34 +02:00
Jakub Sztandera
39625eada7
Add count 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:47:34 +02:00
Jakub Sztandera
7a7e5a85ad
Print tt from covar 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:47:34 +02:00
Jakub Sztandera
254295b9c3
Readd covariance 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:47:34 +02:00
Jakub Sztandera
4dd79ec6fa
Calculate correlation 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:47:34 +02:00
Jakub Sztandera
9ccd25b072
Update gas values, compute correlation coefficient 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:47:33 +02:00
Jakub Sztandera
5e7f561133
Support complex extra, fix small bug in covariance 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:47:33 +02:00
Jakub Sztandera
29f5f451b3
Add covariance 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:47:33 +02:00
Jakub Sztandera
2caa7164a6
Midway adding covar 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:47:33 +02:00
Jakub Sztandera
21148033e2
Change alg for computing variance 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:47:33 +02:00
Jakub Sztandera
1e17ea3fff
Adjust gas values 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:47:33 +02:00
Jakub Sztandera
ca7e5f5e16
Set gas to 1 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:47:32 +02:00
Jakub Sztandera
3a9f1bf9a2
Revert "Try without recurson" 
This reverts commit 76e3663b2b1f1b58f9097f45124add4fa7718199.

Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:47:32 +02:00
Jakub Sztandera
9094405537
Try without recurson 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:47:32 +02:00
Jakub Sztandera
a6417dc24b
Add stats test and print observation count 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:47:32 +02:00
Jakub Sztandera
1bc9fbca20
Ignore virtual 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:47:32 +02:00
Jakub Sztandera
369ec8221f
Improve edge case handling 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:47:32 +02:00
Jakub Sztandera
052fc35a91
Maybe avoid NaNs 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:47:31 +02:00
Jakub Sztandera
b8ab549bcb
Cleanup lint 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:47:31 +02:00
Jakub Sztandera
4f60c1afe7
If something costs 0 gas, make it 1. 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:47:31 +02:00
Jakub Sztandera
9cd34509ee
Fix error handling of json 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:47:31 +02:00
Jakub Sztandera
01e230e810
Fix ratio 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:46:42 +02:00
Jakub Sztandera
78ee64d9f8
Fix stupid mistake 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:46:42 +02:00
Jakub Sztandera
e5a7cb635c
Siwtch back to float64 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:46:42 +02:00
Jakub Sztandera
49ddcdb4b0
Don't pass pointers 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:46:42 +02:00
Jakub Sztandera
b5ae3c055b
Remove warning 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:46:41 +02:00
Jakub Sztandera
9712bbc854
Decode in workers 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:46:41 +02:00
Jakub Sztandera
3134fcb54a
Streaming mean and variance 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:46:41 +02:00
Jakub Sztandera
87d668390e
Use float32 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:46:41 +02:00
Jakub Sztandera
710286d6e7
Add pprof 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:46:41 +02:00
Jakub Sztandera
04edeccbf4
fix trimming 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:46:41 +02:00
Jakub Sztandera
db5929f4a5
Limit size of expensiveInvocs 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:46:41 +02:00
Jakub Sztandera
e65215f1b5
Make lotus-bench import analyze multithreaded and less RAM hungry 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:46:41 +02:00
Jakub Sztandera
c97720e731
Fix import analyze 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-07-02 18:46:41 +02:00
Jakub Sztandera
f98063d604
In this case I want it to be not really safe 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-06-25 16:58:55 +02:00
Jakub Sztandera
10c4fae8c0
Make gas traces smaller, strip callers in import-bench 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-06-25 16:46:50 +02:00
whyrusleeping
9903eba7fb stream bench import results to disk 2020-06-22 20:08:19 -07:00
Łukasz Magiera
cc179c5270
Merge pull request #2047 from filecoin-project/feat/parallel-sealing-bench 
allow sealing bench to run parallel seals
 2020-06-18 01:28:56 +02:00
Jim Pick
c7e470bd5e Spelling: s/winnnig/winning/ 
It was bugging me. :-)
 2020-06-17 12:49:23 -07:00
Jeromy
d66c70d1e6 add configurable paralellism for sub portions of the sealing process 2020-06-17 11:02:48 -07:00
Jeromy
ec693008d7 allow sealing bench to run parallel seals 2020-06-16 17:39:51 -07:00
Jeromy
f8c4b64782 improve chain import analyze output, add some rough virtual gas charges 2020-06-15 16:05:29 -07:00
Jeromy
01c4726fd5 add some smartness to bench analyze gas output 2020-06-15 13:02:57 -07:00
Łukasz Magiera
daca8651f4 Fix bench 2020-06-15 19:53:21 +02:00
Łukasz Magiera
907364ce67 Update deps, fix more tests 2020-06-15 18:30:49 +02:00
yaohcn
e0a8e993bc bench prove nedd miner-addr flag 2020-06-11 16:35:46 +08:00
Łukasz Magiera
a00df0da90 FFI v27 2020-06-10 01:44:14 +02:00
Łukasz Magiera
28f895cec2 Update sector-storage with better post faults 2020-06-08 23:20:19 +02:00
Łukasz Magiera
3b13397165 gofmt 2020-06-06 00:59:01 +02:00
Łukasz Magiera
163c2b2544 Merge remote-tracking branch 'origin/master' into next 2020-06-06 00:58:24 +02:00
laser
4855045ea2 pass context to paramfetch.GetParams 
Fixes #1209
 2020-06-05 10:28:00 -07:00
Jakub Sztandera
29dbc26dbd
Update cli 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-06-04 01:01:41 +02:00
Łukasz Magiera
64f4e80446 Include build type in version 2020-06-01 20:49:39 +02:00
Łukasz Magiera
f7a71f4be7
Merge pull request #1837 from filecoin-project/feat/secstor-unseal 
Update sector-storage with ReadPiece impl
 2020-05-29 20:36:40 +02:00
Łukasz Magiera
e5c24ff32b Update sector-storage with ReadPiece impl 2020-05-29 19:53:35 +02:00
Jakub Sztandera
b372881e91
Fix more lint warnings 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-05-28 01:15:19 +02:00
Jakub Sztandera
d6615b6286
Cleanup many lint warnings 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-05-27 22:53:20 +02:00
Jeromy
646800b8a8 update to latest ffi 2020-05-22 09:26:14 -07:00
Jeromy
9804bd3c4c add blockstore 'Has' caching 2020-05-16 11:31:14 -07:00
Jeromy
f6589407c1 register secp signatures for lotus bench usage 2020-05-15 13:01:45 -07:00
Łukasz Magiera
ba389598ee Cleanup build package 2020-05-12 19:59:28 +02:00
Łukasz Magiera
fbfab94114 bench: Fix unseal 2020-05-08 01:44:12 +02:00
Łukasz Magiera
75642f393d v26 2020-05-07 01:22:59 +02:00
Jeromy
74510adaaa add flag to choose what to do when looking back tipsets in null rounds 2020-05-05 10:06:05 -07:00
Aayush Rajasekaran
c196570dd8 Use blake2b, instead of sha256, for ticket creation and verification 2020-05-01 22:43:36 -04:00
Łukasz Magiera
db2e1fc2c4 Merge pull request #1591 from filecoin-project/feat/import-analyze 
add command to analyze import bench output
 2020-04-24 02:14:49 +02:00
Jeromy
7ee42e61d5 add command to analyze import bench output 2020-04-23 16:48:54 -07:00
Jeromy
86ccaa1981 properly load in registered proof type when benchmarking existing sectors 2020-04-24 01:16:26 +02:00
Jeromy
a9b62bcc5c small amount of cleanup for lotus bench 2020-04-24 01:16:26 +02:00
Łukasz Magiera
9f9ae155e3 windowed post: Review cleanup 2020-04-21 23:38:33 +02:00
Łukasz Magiera
eba0743d47 bench: --num-sectors option 2020-04-21 21:47:07 +02:00