Jakub Sztandera
96193c2044
Implement bench-cache
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-29 14:49:11 +02:00
Łukasz Magiera
cf71f034dc
Merge remote-tracking branch 'origin/dev' into feat/async-restartable-workers
2020-09-26 12:35:32 +02:00
Jakub Sztandera
4eec4a0141
Move policy change to seal bench
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-25 23:28:12 +02:00
Steven Allen
32a699d6a3
Add some actors policy setters for testing
...
Addresses:
* a307e4593a (r491966115)
* a307e4593a (r491966634)
Note: This puts everything into a policy package to avoid a dependency cycle
between the build package, the miner package, and the types package. This is
also why I introduced a GetPreCommitChallengeDelay function and removed the
variable.
2020-09-23 15:00:52 -07:00
Łukasz Magiera
aa5bd7bc17
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-09-21 22:36:16 +02:00
jennijuju
5c69249ba3
Use window post
for window PoSt related log messages.
2020-09-15 21:22:29 -04:00
whyrusleeping
24756ac056
allow specification of repo directory in lotus-bench
2020-09-15 11:55:54 -07:00
Łukasz Magiera
5f08fe7ead
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-09-10 17:30:54 +02:00
Aayush Rajasekaran
d678fe4bfa
Fix tests
2020-09-07 15:48:42 -04:00
Aayush Rajasekaran
39755a294a
Update to specs v0.9.6
2020-09-07 15:48:41 -04:00
Łukasz Magiera
5d73943929
storage: Fix import cycle
2020-09-06 18:54:00 +02:00
Steven Allen
5733c71c50
Lint everything
...
We were ignoring quite a few error cases, and had one case where we weren't
actually updating state where we wanted to. Unfortunately, if the linter doesn't
pass, nobody has any reason to actually check lint failures in CI.
There are three remaining XXXs marked in the code for lint.
2020-08-20 20:46:36 -07:00
Raúl Kripalani
efdc428d5d
keep storage-fsm (renamed to storage-sealing) and sector-storage in extern.
2020-08-17 14:26:18 +01:00
Raúl Kripalani
3c17cd655e
integrate extern/sector-storage into lotus proper.
2020-08-16 11:09:58 +01:00
Steven Allen
3ef3f570fb
Fix lint errors an broken tests
2020-08-12 23:16:44 +02:00
Aayush Rajasekaran
d2e203c1ce
Fix import
2020-08-04 00:45:02 -04:00
Aayush Rajasekaran
5e485a085a
Merge pull request #2637 from filecoin-project/gas/tag-verify-post
...
Use scaling VerifyPost cost
2020-07-28 21:22:23 -04:00
Jakub Sztandera
b488012d5f
Add warn
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-28 22:31:43 +02:00
Jakub Sztandera
da4bc44032
Discard initial very long OnVerifyPost from analisys
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-28 22:31:43 +02:00
Steven Allen
b7a4dbb07f
Support inline CIDs
...
And use the new CidBuilder from the spec actors.
This patch does not switch over to inline CIDs by default, but paves the way.
2020-07-23 23:12:32 -07:00
Frank
c38b8b1265
close file
2020-07-23 18:21:13 +08:00
Jakub Sztandera
610ed0dc13
Change OnIpldGet
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-15 21:40:49 +02:00
Jakub Sztandera
c0eb4b39ac
Reorg some gas charges
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-15 20:01:56 +02:00
Łukasz Magiera
8e0d33a1fd
Updates for sectors-storge next
2020-07-08 17:23:27 +02:00
Jakub Sztandera
3429941639
Add info about OnActorExecDone
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:34 +02:00
Jakub Sztandera
41b775fdd4
Compute time spent in actors
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:34 +02:00
Jakub Sztandera
e9d2a3edd9
Limit float print precision
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:34 +02:00
Jakub Sztandera
42696372ee
Progress to stderr
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:34 +02:00
Jakub Sztandera
39625eada7
Add count
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:34 +02:00
Jakub Sztandera
7a7e5a85ad
Print tt from covar
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:34 +02:00
Jakub Sztandera
254295b9c3
Readd covariance
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:34 +02:00
Jakub Sztandera
4dd79ec6fa
Calculate correlation
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:34 +02:00
Jakub Sztandera
9ccd25b072
Update gas values, compute correlation coefficient
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:33 +02:00
Jakub Sztandera
5e7f561133
Support complex extra, fix small bug in covariance
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:33 +02:00
Jakub Sztandera
29f5f451b3
Add covariance
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:33 +02:00
Jakub Sztandera
2caa7164a6
Midway adding covar
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:33 +02:00
Jakub Sztandera
21148033e2
Change alg for computing variance
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:33 +02:00
Jakub Sztandera
1e17ea3fff
Adjust gas values
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:33 +02:00
Jakub Sztandera
ca7e5f5e16
Set gas to 1
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:32 +02:00
Jakub Sztandera
3a9f1bf9a2
Revert "Try without recurson"
...
This reverts commit 76e3663b2b1f1b58f9097f45124add4fa7718199.
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:32 +02:00
Jakub Sztandera
9094405537
Try without recurson
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:32 +02:00
Jakub Sztandera
a6417dc24b
Add stats test and print observation count
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:32 +02:00
Jakub Sztandera
1bc9fbca20
Ignore virtual
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:32 +02:00
Jakub Sztandera
369ec8221f
Improve edge case handling
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:32 +02:00
Jakub Sztandera
052fc35a91
Maybe avoid NaNs
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:31 +02:00
Jakub Sztandera
b8ab549bcb
Cleanup lint
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:31 +02:00
Jakub Sztandera
4f60c1afe7
If something costs 0 gas, make it 1.
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:31 +02:00
Jakub Sztandera
9cd34509ee
Fix error handling of json
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:47:31 +02:00
Jakub Sztandera
01e230e810
Fix ratio
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:46:42 +02:00
Jakub Sztandera
78ee64d9f8
Fix stupid mistake
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:46:42 +02:00
Jakub Sztandera
e5a7cb635c
Siwtch back to float64
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:46:42 +02:00
Jakub Sztandera
49ddcdb4b0
Don't pass pointers
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:46:42 +02:00
Jakub Sztandera
b5ae3c055b
Remove warning
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:46:41 +02:00
Jakub Sztandera
9712bbc854
Decode in workers
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:46:41 +02:00
Jakub Sztandera
3134fcb54a
Streaming mean and variance
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:46:41 +02:00
Jakub Sztandera
87d668390e
Use float32
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:46:41 +02:00
Jakub Sztandera
710286d6e7
Add pprof
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:46:41 +02:00
Jakub Sztandera
04edeccbf4
fix trimming
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:46:41 +02:00
Jakub Sztandera
db5929f4a5
Limit size of expensiveInvocs
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:46:41 +02:00
Jakub Sztandera
e65215f1b5
Make lotus-bench import analyze multithreaded and less RAM hungry
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:46:41 +02:00
Jakub Sztandera
c97720e731
Fix import analyze
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-07-02 18:46:41 +02:00
Jakub Sztandera
f98063d604
In this case I want it to be not really safe
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-06-25 16:58:55 +02:00
Jakub Sztandera
10c4fae8c0
Make gas traces smaller, strip callers in import-bench
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-06-25 16:46:50 +02:00
whyrusleeping
9903eba7fb
stream bench import results to disk
2020-06-22 20:08:19 -07:00
Łukasz Magiera
cc179c5270
Merge pull request #2047 from filecoin-project/feat/parallel-sealing-bench
...
allow sealing bench to run parallel seals
2020-06-18 01:28:56 +02:00
Jim Pick
c7e470bd5e
Spelling: s/winnnig/winning/
...
It was bugging me. :-)
2020-06-17 12:49:23 -07:00
Jeromy
d66c70d1e6
add configurable paralellism for sub portions of the sealing process
2020-06-17 11:02:48 -07:00
Jeromy
ec693008d7
allow sealing bench to run parallel seals
2020-06-16 17:39:51 -07:00
Jeromy
f8c4b64782
improve chain import analyze output, add some rough virtual gas charges
2020-06-15 16:05:29 -07:00
Jeromy
01c4726fd5
add some smartness to bench analyze gas output
2020-06-15 13:02:57 -07:00
Łukasz Magiera
daca8651f4
Fix bench
2020-06-15 19:53:21 +02:00
Łukasz Magiera
907364ce67
Update deps, fix more tests
2020-06-15 18:30:49 +02:00
yaohcn
e0a8e993bc
bench prove nedd miner-addr flag
2020-06-11 16:35:46 +08:00
Łukasz Magiera
a00df0da90
FFI v27
2020-06-10 01:44:14 +02:00
Łukasz Magiera
28f895cec2
Update sector-storage with better post faults
2020-06-08 23:20:19 +02:00
Łukasz Magiera
3b13397165
gofmt
2020-06-06 00:59:01 +02:00
Łukasz Magiera
163c2b2544
Merge remote-tracking branch 'origin/master' into next
2020-06-06 00:58:24 +02:00
laser
4855045ea2
pass context to paramfetch.GetParams
...
Fixes #1209
2020-06-05 10:28:00 -07:00
Jakub Sztandera
29dbc26dbd
Update cli
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-06-04 01:01:41 +02:00
Łukasz Magiera
64f4e80446
Include build type in version
2020-06-01 20:49:39 +02:00
Łukasz Magiera
f7a71f4be7
Merge pull request #1837 from filecoin-project/feat/secstor-unseal
...
Update sector-storage with ReadPiece impl
2020-05-29 20:36:40 +02:00
Łukasz Magiera
e5c24ff32b
Update sector-storage with ReadPiece impl
2020-05-29 19:53:35 +02:00
Jakub Sztandera
b372881e91
Fix more lint warnings
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-05-28 01:15:19 +02:00
Jakub Sztandera
d6615b6286
Cleanup many lint warnings
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-05-27 22:53:20 +02:00
Jeromy
646800b8a8
update to latest ffi
2020-05-22 09:26:14 -07:00
Jeromy
9804bd3c4c
add blockstore 'Has' caching
2020-05-16 11:31:14 -07:00
Jeromy
f6589407c1
register secp signatures for lotus bench usage
2020-05-15 13:01:45 -07:00
Łukasz Magiera
ba389598ee
Cleanup build package
2020-05-12 19:59:28 +02:00
Łukasz Magiera
fbfab94114
bench: Fix unseal
2020-05-08 01:44:12 +02:00
Łukasz Magiera
75642f393d
v26
2020-05-07 01:22:59 +02:00
Jeromy
74510adaaa
add flag to choose what to do when looking back tipsets in null rounds
2020-05-05 10:06:05 -07:00
Aayush Rajasekaran
c196570dd8
Use blake2b, instead of sha256, for ticket creation and verification
2020-05-01 22:43:36 -04:00
Łukasz Magiera
db2e1fc2c4
Merge pull request #1591 from filecoin-project/feat/import-analyze
...
add command to analyze import bench output
2020-04-24 02:14:49 +02:00
Jeromy
7ee42e61d5
add command to analyze import bench output
2020-04-23 16:48:54 -07:00
Jeromy
86ccaa1981
properly load in registered proof type when benchmarking existing sectors
2020-04-24 01:16:26 +02:00
Jeromy
a9b62bcc5c
small amount of cleanup for lotus bench
2020-04-24 01:16:26 +02:00
Łukasz Magiera
9f9ae155e3
windowed post: Review cleanup
2020-04-21 23:38:33 +02:00
Łukasz Magiera
eba0743d47
bench: --num-sectors option
2020-04-21 21:47:07 +02:00
Jeromy
24b7661837
fix up some tests
2020-04-17 13:44:13 -07:00
Łukasz Magiera
e4641fc7ed
bench: Also bench windowed post
2020-04-14 00:17:37 +02:00
Łukasz Magiera
a2554952d1
Some electionPoSt -> winningPoSt renaming
2020-04-11 00:22:45 +02:00
Łukasz Magiera
613e61222b
mostly wire up WindowedPoSt
2020-04-10 23:07:18 +02:00
Łukasz Magiera
1c75bf30ac
genesis: Try updating for QA power
2020-04-10 22:04:04 +02:00
Łukasz Magiera
b0fe8b8b4c
v25: get bench to almost work
2020-04-10 14:19:06 +02:00
Łukasz Magiera
37e7c512e6
Update ffi (v25 proofs)
2020-04-09 23:17:23 +02:00
Jeromy
beaa3dffab
add a command to lotus-bench to benchmark importing and validating a chain
2020-04-02 16:18:38 -07:00
Łukasz Magiera
95ee1c1cde
Extract sector-storage
2020-03-28 01:32:39 +01:00
Łukasz Magiera
c66d3dfad7
sectorstorage: Untangle from lotus deps
2020-03-27 22:35:33 +01:00
Łukasz Magiera
e050e0307e
Cleanup after dropping sectorbuilder
2020-03-26 20:35:16 +01:00
Łukasz Magiera
4be54a976c
Merge sectorbuilder into sectorstorage
2020-03-26 03:51:06 +01:00
Łukasz Magiera
d43f2a2e3b
workers: Address review
2020-03-22 22:39:06 +01:00
Łukasz Magiera
c57c0e7f55
workers: Memory based scheduling
2020-03-20 23:30:17 +01:00
Łukasz Magiera
5f2caffb51
Fix tests after specs-storage changes
2020-03-18 02:08:11 +01:00
Łukasz Magiera
1f9fdf0c5c
gofmt
2020-03-16 10:43:27 +01:00
Łukasz Magiera
630df050c0
bench: Option to skip commit2
2020-03-15 18:48:53 +01:00
Łukasz Magiera
a962479ce7
Update sectorbuilder to fix the mkdir bug in precommit1
2020-03-06 23:26:22 +01:00
Łukasz Magiera
3cab8bd5d1
gofmt
2020-03-06 20:01:28 +01:00
Łukasz Magiera
7d3e16986c
sectorbuilder type updates
2020-03-06 06:38:59 +01:00
Łukasz Magiera
98dbb2f70f
bench: set correct registered proofs on things
2020-03-01 22:05:13 +01:00
Łukasz Magiera
f4310e8f69
bench: set sector size in saved commit2 inputs
2020-03-01 05:26:10 +01:00
Łukasz Magiera
ee3e631b1a
bench: Prove command
2020-03-01 03:52:46 +01:00
Łukasz Magiera
342e85d378
bench: flag to save commit2 input
2020-02-29 03:31:25 +01:00
Łukasz Magiera
f54a555aa8
Merge remote-tracking branch 'origin/testnet/3' into feat/4stage-seal
2020-02-29 03:21:01 +01:00
whyrusleeping
5b657a2905
pass on proper sector types
2020-02-28 12:52:14 -08:00
Łukasz Magiera
5a4f917c60
4-stage v23 lotus-bench
2020-02-28 19:06:59 +01:00
Łukasz Magiera
4dc35285dd
Minimum sector size is now 2k
2020-02-28 01:48:34 +01:00
whyrusleeping
e4dcb7441b
fixup lotus-bench
2020-02-27 14:23:05 -08:00
Łukasz Magiera
0a9e0acc6f
types: Drop some redundant alias types
2020-02-25 22:09:22 +01:00
Łukasz Magiera
fa2c6605c2
Fix compilation after dep updates
2020-02-11 02:11:58 +01:00
Łukasz Magiera
c0ed0dda8e
specs-actors: Fis some test compilation errors
2020-02-10 20:16:36 +01:00
Łukasz Magiera
a62572bb76
Merge remote-tracking branch 'origin/master' into feat/new-sb-fs
2020-01-31 08:36:47 +01:00
Łukasz Magiera
aefc463586
build: Lazy ParametersJson
2020-01-30 23:50:24 +01:00
Łukasz Magiera
5af64c53b6
initial sectorbuilder FS refactor integration
2020-01-29 00:08:02 +01:00
Łukasz Magiera
533081454a
bench: Don't print unseal results when unsealing is skipped
2020-01-19 15:56:37 +01:00
Łukasz Magiera
a2bcc1fec2
Mostly functional mock sectorbuilder
2020-01-14 03:05:35 +01:00
whyrusleeping
7534ac1e1a
add ability to control seal operations completion time
2020-01-09 12:50:45 -08:00
Łukasz Magiera
8cbcbbd7a4
Merge branch 'feat/testnet2' of github.com:filecoin-project/go-lotus into feat/testnet2
2020-01-08 22:44:54 +01:00
Łukasz Magiera
da38822dfd
paramfetch: bring parameters.json back to this repo
2020-01-08 21:31:35 +01:00
Jakub Sztandera
1ed62628a7
Update go-log to v2
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-01-08 20:22:31 +01:00
laser
20321bb476
handle rebase
2020-01-07 08:18:35 -08:00
hannahhoward
8418464d91
refactor(address): use extracted address library
...
Switch to using extracted address library
2020-01-07 14:53:27 +01:00
laser
6a740a6d67
consume newly-extracted paramfetch
2020-01-02 11:08:49 -08:00
whyrusleeping
83fb2eff98
add a flag for bench to not run unseal
2019-12-19 11:45:15 -08:00
whyrusleeping
faad2485ee
add some notes on troubleshooting GPU usage
2019-12-17 11:59:14 -08:00
Łukasz Magiera
3fda442bb3
Merge pull request #934 from filecoin-project/feat/sb-diskmgr
...
sectorbuilder: Check free space before creating sectors
2019-12-16 21:37:18 +01:00
Łukasz Magiera
e0c6247ac9
Use power of 2 sizes when parsing byte counts
2019-12-16 20:44:48 +01:00
Łukasz Magiera
825e3a0774
sectorbuilder: check free space before creating sectors
2019-12-16 17:16:08 +01:00
Jakub Sztandera
9589fc0762
Add version commit string
...
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-12-11 23:00:39 +01:00
Łukasz Magiera
43f5ba1eb4
Merge pull request #847 from lanzafame/feat/accept-human-sizes
...
accept humanized size values to cli flags
2019-12-11 13:18:05 +01:00
lanzafame
1888b68e38
accept humanized size values to cli flags
2019-12-11 12:06:28 +10:00
Jakub Sztandera
bfdfd4023e
Cleanup math
...
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2019-12-10 20:53:39 +01:00
Łukasz Magiera
0fa20f6074
bench: print bps speeds
2019-12-10 20:02:42 +01:00
whyrusleeping
df52f8981d
Add missing newline
2019-12-10 15:11:21 +01:00
whyrusleeping
384b0bdf49
add option for json output
2019-12-10 15:05:41 +01:00
whyrusleeping
3f1d8f0ed4
perform homedir expansion on input directory
2019-12-10 14:47:49 +01:00
whyrusleeping
ce5c76c6d6
load sector info from pre seal file
2019-12-10 14:22:39 +01:00
whyrusleeping
5d12659266
allow lotus-bench to benchmark existing sectorbuilders
2019-12-10 14:01:17 +01:00
Łukasz Magiera
3daf9103a8
paramfetch: Only pull necessary params
2019-12-04 20:44:15 +01:00
whyrusleeping
22f96e626b
add flag to disable gpu to lotus-bench
2019-12-02 11:25:16 -08:00
Łukasz Magiera
d9840c05db
bench: Measure unseal time
2019-12-01 23:37:53 +01:00
whyrusleeping
0ca92bdca0
track hot and cold times for epost
2019-11-29 10:48:07 -08:00
Łukasz Magiera
aefd432422
actors: Fallback post progress
2019-11-28 13:46:56 +01:00
whyrusleeping
8fae155cf6
Add timings for proof verification
2019-11-27 19:43:52 -06:00
Łukasz Magiera
4a77fdac4c
bench: Fix after filecoin-ffi switch
2019-11-27 13:12:42 +01:00
whyrusleeping
718d681f73
Add lotus-bench for testing hardware specs
2019-11-26 20:47:08 -06:00