Commit Graph

236 Commits

Author SHA1 Message Date
zgfzgf
5c27d6aeee lotus-bench UnsealPiece SectorID.Number error 2020-10-11 10:12:06 +08:00
Steven Allen
748d2e82a7 unshare the journal
Motivation:

* Run lotus with the race detector enabled (primary motivation).
* Allow multiple lotus nodes in a process (not a high priority).

Previously, the journal was shared between all lotus instances, but it was
initialized for every new node. This caused safety problems in tests (at a
minimum).

This patch explicitly passes the journal to all services that need it.
2020-10-09 13:23:07 -07:00
Łukasz Magiera
2cfe22d4e5 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-09-30 20:48:16 +02:00
Jakub Sztandera
7e8c6e5070
Remove statediff, fix lint, go mod tidy
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-29 14:49:47 +02:00
Jakub Sztandera
0d914ac1d4
Switch to badger
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-29 14:49:47 +02:00
Jakub Sztandera
76db65b1af
Update pebble
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-29 14:49:47 +02:00
Jakub Sztandera
ff8c0af8c8
Add only-import option
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-29 14:49:47 +02:00
Jakub Sztandera
55c6b88537
Add toggle for badger, flag out gas tracing
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-29 14:49:46 +02:00
Jakub Sztandera
0771c23fb0
Use pebble
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-29 14:49:46 +02:00
Jakub Sztandera
242a77b391
go mod tidy
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-29 14:49:18 +02:00
Jakub Sztandera
35cf69ae64
Disable bloomcache
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-29 14:49:18 +02:00
Jakub Sztandera
ebc8489ff1
Add global-profile option
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-29 14:49:18 +02:00
Jakub Sztandera
f21c5cbbe2
Add start-at
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-29 14:49:18 +02:00
Jakub Sztandera
01386a206c
Update options
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-29 14:49:18 +02:00
Jakub Sztandera
3858309368
Add http to import bench
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-29 14:49:17 +02:00
Jakub Sztandera
782717948a
Add logs
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-29 14:49:17 +02:00
Jakub Sztandera
108fe7823c
Add command to trigger gc
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-29 14:49:17 +02:00
Jakub Sztandera
12a0dd3d0a
<3 to linter
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-29 14:49:17 +02:00
Jakub Sztandera
1f4d1dcc58
Do not sync
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-29 14:49:17 +02:00
Jakub Sztandera
53ab17cf50
Add no import to import-bench
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-29 14:49:17 +02:00
Jakub Sztandera
79ba4598d6
Implement cache
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-29 14:49:17 +02:00
Jakub Sztandera
96193c2044
Implement bench-cache
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-29 14:49:11 +02:00
Łukasz Magiera
cf71f034dc Merge remote-tracking branch 'origin/dev' into feat/async-restartable-workers 2020-09-26 12:35:32 +02:00
Jakub Sztandera
4eec4a0141
Move policy change to seal bench
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-25 23:28:12 +02:00
Steven Allen
32a699d6a3 Add some actors policy setters for testing
Addresses:
* a307e4593a (r491966115)
* a307e4593a (r491966634)

Note: This puts everything into a policy package to avoid a dependency cycle
between the build package, the miner package, and the types package. This is
also why I introduced a GetPreCommitChallengeDelay function and removed the
variable.
2020-09-23 15:00:52 -07:00
Łukasz Magiera
aa5bd7bc17 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-09-21 22:36:16 +02:00
jennijuju
5c69249ba3 Use window post for window PoSt related log messages. 2020-09-15 21:22:29 -04:00
whyrusleeping
24756ac056 allow specification of repo directory in lotus-bench 2020-09-15 11:55:54 -07:00
Łukasz Magiera
5f08fe7ead Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-09-10 17:30:54 +02:00
Aayush Rajasekaran
d678fe4bfa Fix tests 2020-09-07 15:48:42 -04:00
Aayush Rajasekaran
39755a294a Update to specs v0.9.6 2020-09-07 15:48:41 -04:00
Łukasz Magiera
5d73943929 storage: Fix import cycle 2020-09-06 18:54:00 +02:00
Steven Allen
5733c71c50 Lint everything
We were ignoring quite a few error cases, and had one case where we weren't
actually updating state where we wanted to. Unfortunately, if the linter doesn't
pass, nobody has any reason to actually check lint failures in CI.

There are three remaining XXXs marked in the code for lint.
2020-08-20 20:46:36 -07:00
Raúl Kripalani
efdc428d5d keep storage-fsm (renamed to storage-sealing) and sector-storage in extern. 2020-08-17 14:26:18 +01:00
Raúl Kripalani
3c17cd655e integrate extern/sector-storage into lotus proper. 2020-08-16 11:09:58 +01:00
Steven Allen
3ef3f570fb Fix lint errors an broken tests 2020-08-12 23:16:44 +02:00
Aayush Rajasekaran
d2e203c1ce Fix import 2020-08-04 00:45:02 -04:00
Aayush Rajasekaran
5e485a085a
Merge pull request from filecoin-project/gas/tag-verify-post
Use scaling VerifyPost cost
2020-07-28 21:22:23 -04:00
Jakub Sztandera
b488012d5f
Add warn
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-28 22:31:43 +02:00
Jakub Sztandera
da4bc44032
Discard initial very long OnVerifyPost from analisys
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-28 22:31:43 +02:00
Steven Allen
b7a4dbb07f Support inline CIDs
And use the new CidBuilder from the spec actors.

This patch does not switch over to inline CIDs by default, but paves the way.
2020-07-23 23:12:32 -07:00
Frank
c38b8b1265 close file 2020-07-23 18:21:13 +08:00
Jakub Sztandera
610ed0dc13
Change OnIpldGet
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-15 21:40:49 +02:00
Jakub Sztandera
c0eb4b39ac
Reorg some gas charges
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-15 20:01:56 +02:00
Łukasz Magiera
8e0d33a1fd Updates for sectors-storge next 2020-07-08 17:23:27 +02:00
Jakub Sztandera
3429941639
Add info about OnActorExecDone
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:34 +02:00
Jakub Sztandera
41b775fdd4
Compute time spent in actors
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:34 +02:00
Jakub Sztandera
e9d2a3edd9
Limit float print precision
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:34 +02:00
Jakub Sztandera
42696372ee
Progress to stderr
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:34 +02:00
Jakub Sztandera
39625eada7
Add count
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:34 +02:00
Jakub Sztandera
7a7e5a85ad
Print tt from covar
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:34 +02:00
Jakub Sztandera
254295b9c3
Readd covariance
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:34 +02:00
Jakub Sztandera
4dd79ec6fa
Calculate correlation
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:34 +02:00
Jakub Sztandera
9ccd25b072
Update gas values, compute correlation coefficient
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:33 +02:00
Jakub Sztandera
5e7f561133
Support complex extra, fix small bug in covariance
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:33 +02:00
Jakub Sztandera
29f5f451b3
Add covariance
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:33 +02:00
Jakub Sztandera
2caa7164a6
Midway adding covar
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:33 +02:00
Jakub Sztandera
21148033e2
Change alg for computing variance
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:33 +02:00
Jakub Sztandera
1e17ea3fff
Adjust gas values
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:33 +02:00
Jakub Sztandera
ca7e5f5e16
Set gas to 1
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:32 +02:00
Jakub Sztandera
3a9f1bf9a2
Revert "Try without recurson"
This reverts commit 76e3663b2b1f1b58f9097f45124add4fa7718199.

Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:32 +02:00
Jakub Sztandera
9094405537
Try without recurson
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:32 +02:00
Jakub Sztandera
a6417dc24b
Add stats test and print observation count
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:32 +02:00
Jakub Sztandera
1bc9fbca20
Ignore virtual
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:32 +02:00
Jakub Sztandera
369ec8221f
Improve edge case handling
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:32 +02:00
Jakub Sztandera
052fc35a91
Maybe avoid NaNs
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:31 +02:00
Jakub Sztandera
b8ab549bcb
Cleanup lint
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:31 +02:00
Jakub Sztandera
4f60c1afe7
If something costs 0 gas, make it 1.
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:31 +02:00
Jakub Sztandera
9cd34509ee
Fix error handling of json
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:31 +02:00
Jakub Sztandera
01e230e810
Fix ratio
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:46:42 +02:00
Jakub Sztandera
78ee64d9f8
Fix stupid mistake
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:46:42 +02:00
Jakub Sztandera
e5a7cb635c
Siwtch back to float64
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:46:42 +02:00
Jakub Sztandera
49ddcdb4b0
Don't pass pointers
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:46:42 +02:00
Jakub Sztandera
b5ae3c055b
Remove warning
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:46:41 +02:00
Jakub Sztandera
9712bbc854
Decode in workers
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:46:41 +02:00
Jakub Sztandera
3134fcb54a
Streaming mean and variance
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:46:41 +02:00
Jakub Sztandera
87d668390e
Use float32
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:46:41 +02:00
Jakub Sztandera
710286d6e7
Add pprof
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:46:41 +02:00
Jakub Sztandera
04edeccbf4
fix trimming
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:46:41 +02:00
Jakub Sztandera
db5929f4a5
Limit size of expensiveInvocs
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:46:41 +02:00
Jakub Sztandera
e65215f1b5
Make lotus-bench import analyze multithreaded and less RAM hungry
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:46:41 +02:00
Jakub Sztandera
c97720e731
Fix import analyze
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:46:41 +02:00
Jakub Sztandera
f98063d604
In this case I want it to be not really safe
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-06-25 16:58:55 +02:00
Jakub Sztandera
10c4fae8c0
Make gas traces smaller, strip callers in import-bench
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-06-25 16:46:50 +02:00
whyrusleeping
9903eba7fb stream bench import results to disk 2020-06-22 20:08:19 -07:00
Łukasz Magiera
cc179c5270
Merge pull request from filecoin-project/feat/parallel-sealing-bench
allow sealing bench to run parallel seals
2020-06-18 01:28:56 +02:00
Jim Pick
c7e470bd5e Spelling: s/winnnig/winning/
It was bugging me. :-)
2020-06-17 12:49:23 -07:00
Jeromy
d66c70d1e6 add configurable paralellism for sub portions of the sealing process 2020-06-17 11:02:48 -07:00
Jeromy
ec693008d7 allow sealing bench to run parallel seals 2020-06-16 17:39:51 -07:00
Jeromy
f8c4b64782 improve chain import analyze output, add some rough virtual gas charges 2020-06-15 16:05:29 -07:00
Jeromy
01c4726fd5 add some smartness to bench analyze gas output 2020-06-15 13:02:57 -07:00
Łukasz Magiera
daca8651f4 Fix bench 2020-06-15 19:53:21 +02:00
Łukasz Magiera
907364ce67 Update deps, fix more tests 2020-06-15 18:30:49 +02:00
yaohcn
e0a8e993bc bench prove nedd miner-addr flag 2020-06-11 16:35:46 +08:00
Łukasz Magiera
a00df0da90 FFI v27 2020-06-10 01:44:14 +02:00
Łukasz Magiera
28f895cec2 Update sector-storage with better post faults 2020-06-08 23:20:19 +02:00
Łukasz Magiera
3b13397165 gofmt 2020-06-06 00:59:01 +02:00
Łukasz Magiera
163c2b2544 Merge remote-tracking branch 'origin/master' into next 2020-06-06 00:58:24 +02:00
laser
4855045ea2 pass context to paramfetch.GetParams
Fixes 
2020-06-05 10:28:00 -07:00
Jakub Sztandera
29dbc26dbd
Update cli
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-06-04 01:01:41 +02:00