Commit Graph

13016 Commits

Author SHA1 Message Date
vyzo
254c489164 fix typo
Co-authored-by: raulk <raul@protocol.ai>
2021-07-25 09:01:34 +03:00
vyzo
33cdc90f13 fix typo
Co-authored-by: raulk <raul@protocol.ai>
2021-07-25 09:01:34 +03:00
vyzo
da66e7a878 fix typo 2021-07-25 09:01:34 +03:00
vyzo
e696a2c0cc fix newline in progres message 2021-07-25 09:01:34 +03:00
vyzo
36b209ca5c compact and gc coldstore after copying 2021-07-25 09:01:34 +03:00
vyzo
92b9d8c895 quiet linter 2021-07-25 09:01:34 +03:00
vyzo
0c68bccbda add splitstore rollback lotus-shed command 2021-07-25 09:01:34 +03:00
whyrusleeping
78f94c4c6b This pulls in forgotten parts properly implementing PR#5988
( previous testing focused exclusively on offline dealflow .cars )

Allows a workflow of:

~$ dd if=/dev/urandom bs=1M count=1 | ~/go-ipfs/cmd/ipfs/ipfs add --pin=false
added QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2 QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2

~$ ~/go-ipfs/cmd/ipfs/ipfs dag export QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2 > test_mib.car

~$ lotus client import --car ~/test_mib.car
Import 2, Root QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2

~$ ~/go/bin/stream-commp -p $(( 256 * 1024 * 1024 )) < test_mib.car
CommP:    54e4e75ddc3fffa8fd33d3ededc06e564603ac0fe62543ec6463d51b553be40b
CommPCid: baga6ea4seaqfjzhhlxod775i7uz5h3pnybxfmrqdvqh6mjkd5rsghvi3ku56icy
Raw bytes:           1049073 bytes
Unpadded piece:    266338304 bytes
Padded piece:      268435456 bytes
CARv1 detected in stream:
Blocks:         5
Roots:          1
    1: QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2

~$ curl http://127.0.0.1:1234/rpc/v0 -X POST -H "Authorization: Bearer $(cat ~/.lotus/token)" -H "Content-Type: application/json" --data '
  { "jsonrpc": "2.0", "id":1, "method": "Filecoin.ClientStartDeal", "params": [
    {
      "Wallet":"t01004",
      "Miner":"t01005",
      "EpochPrice":"0",
      "MinBlocksDuration":518400,
      "Data": {
        "Root":{ "/":"QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2" },
        "PieceCid":{ "/":"baga6ea4seaqfjzhhlxod775i7uz5h3pnybxfmrqdvqh6mjkd5rsghvi3ku56icy" },
        "PieceSize": 266338304
      }
    }
  ] }
'

~$ ~/go/bin/stream-commp -p $(( 128 * 1024 * 1024 )) < test_mib.car
CommP:    ed904105399ed346f6b03844abc14710a1748854c2781824d6bd1100e63b1807
CommPCid: baga6ea4seaqo3ecbau4z5u2g62ydqrflyfdrbilurbkme6ayetll2eia4y5rqby
Raw bytes:           1049073 bytes
Unpadded piece:    133169152 bytes
Padded piece:      134217728 bytes
CARv1 detected in stream:
Blocks:         5
Roots:          1
    1: QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2

~$ curl http://127.0.0.1:1234/rpc/v0 -X POST -H "Authorization: Bearer $(cat ~/.lotus/token)" -H "Content-Type: application/json" --data '
  { "jsonrpc": "2.0", "id":1, "method": "Filecoin.ClientStatelessDeal", "params": [
    {
      "Wallet":"t01004",
      "Miner":"t01005",
      "EpochPrice":"0",
      "ProviderCollateral":"0",
      "MinBlocksDuration":518400,
      "Data": {
        "TransferType": "manual",
        "Root":{ "/":"QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2" },
        "PieceCid":{ "/":"baga6ea4seaqo3ecbau4z5u2g62ydqrflyfdrbilurbkme6ayetll2eia4y5rqby" },
        "PieceSize": 133169152
      }
    }
  ] }
'
{"jsonrpc":"2.0","result":{"/":"bafyreianhjvev3w6q5lteap3h7tkxbe2jaobwlsi7vzbcoobjpicg3foqi"},"id":1}

~$ lotus-miner storage-deals import-data bafyreianhjvev3w6q5lteap3h7tkxbe2jaobwlsi7vzbcoobjpicg3foqi ~/test_mib.car
2021-07-24 18:17:13 +02:00
Peter Rabbitson
8d873e3edc Tests for online and offline dealpadding
Also bump the times on several flaky tests that can not complete in time
on a typical laptop ( and fail half the time on CircleCI )
2021-07-24 17:46:04 +02:00
Raúl Kripalani
8693df4656 fix racy TestSimultanenousTransferLimit. 2021-07-24 12:53:56 +01:00
Peter Rabbitson
2edf7fd25b Rewire itest's StartDeal to take the full API struct
This allows one to use the harness for much more versatile deal conditions
2021-07-24 04:17:42 +02:00
Jennifer Wang
5cb7411c2a docs fix 2021-07-23 19:11:26 -04:00
Jennifer Wang
7832545b7a Rebase onto master 2021-07-23 19:09:28 -04:00
Jakub Sztandera
e615eecbea Skip also set sectors
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2021-07-23 19:01:50 -04:00
Jakub Sztandera
39cc536115 Add sectors vis
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2021-07-23 19:01:48 -04:00
Jennifer Wang
f09c4733ad Fix links in issue templates 2021-07-23 18:39:41 -04:00
Jennifer
d14b84cd11
Merge pull request #6856 from filecoin-project/misc/m1-form
Update issue templates and add templates for M1
2021-07-23 18:35:37 -04:00
Jennifer Wang
816ee0b004 Update issue templates and add templates for M1 2021-07-23 18:32:54 -04:00
Łukasz Magiera
02a37a1e22
Merge pull request #6833 from filecoin-project/feat/splitstore-badger-markset
Splitstore: support on-disk marksets using badger
2021-07-23 19:56:54 +02:00
Łukasz Magiera
5b4d5c2eb4
Merge branch 'master' into feat/splitstore-badger-markset 2021-07-23 19:37:45 +02:00
Łukasz Magiera
964df1a2ec
Merge pull request #6848 from filecoin-project/feat/config-ux
Config UX improvements
2021-07-23 19:35:22 +02:00
Łukasz Magiera
91c6aca7e7 config: Add newlines consistently 2021-07-23 16:59:55 +02:00
Łukasz Magiera
80af6de316 config: Add note on codegen to types.go 2021-07-23 16:02:45 +02:00
Łukasz Magiera
ac2a1bfeab fix lint 2021-07-23 15:53:31 +02:00
Łukasz Magiera
4b8a5068bd cli docsgen 2021-07-23 15:44:52 +02:00
Łukasz Magiera
a26712ebc4 config: Bring some docs from filecoin-docs 2021-07-23 15:40:40 +02:00
vyzo
d6ebead756 add comment about the necessity of FileIO 2021-07-23 16:30:55 +03:00
vyzo
1480c22447 improve README 2021-07-23 16:27:09 +03:00
Łukasz Magiera
ed7f70e68e config doc gen: generate formatted code 2021-07-23 15:18:20 +02:00
Łukasz Magiera
3f4973cb41 config: Fix doc finding logic 2021-07-23 15:17:22 +02:00
Łukasz Magiera
4757b014bc Nicer lotus-miner config commands 2021-07-23 14:57:42 +02:00
Łukasz Magiera
8b1f19e94c Move doc-comment logic to the config pkg 2021-07-23 14:55:19 +02:00
Łukasz Magiera
97f5bb66c7 config: doc struct codegen 2021-07-23 13:55:50 +02:00
Łukasz Magiera
cf65804883 lotus daemon config utils 2021-07-23 13:09:03 +02:00
vyzo
9faa3d5fd7 adjust compaction test with badger to hit the db 2021-07-23 12:47:18 +03:00
vyzo
9d5cf4177b add splitstore compaction test with badger markset 2021-07-23 12:47:18 +03:00
vyzo
dec5e13c85 update README with configuration options 2021-07-23 12:47:18 +03:00
vyzo
a8c0fd1654 tweak badger options 2021-07-23 12:47:18 +03:00
vyzo
4df6073382 some tweaks in badger markset 2021-07-23 12:47:18 +03:00
vyzo
f492f8d450 track in flight writes in badger markste, as they happen outside the lock 2021-07-23 12:47:18 +03:00
vyzo
f75d982c76 remove early occurs check from trackTxnRef
this happens inline, and it might block when using the badger markset
2021-07-23 12:47:18 +03:00
vyzo
2891a31c99 make badger markset concurrent close safe 2021-07-23 12:47:18 +03:00
vyzo
f2b7c3e6f2 reduce scope of exclusive lock in badger markset 2021-07-23 12:47:18 +03:00
vyzo
12c3432b8d document the "badger" markset type option 2021-07-23 12:47:18 +03:00
vyzo
be9530b66e finetune badger options 2021-07-23 12:47:18 +03:00
vyzo
2c26abc551 add badger markset test 2021-07-23 12:47:18 +03:00
vyzo
54a6968327 add badger-backed markset implementation 2021-07-23 12:47:18 +03:00
Łukasz Magiera
db39323390
Merge pull request #6724 from filecoin-project/raulk/merge-gs-fix
fix deal concurrency test failures by upgrading graphsync and others
2021-07-23 11:20:04 +02:00
Łukasz Magiera
d3564cc2a3
Merge branch 'master' into raulk/merge-gs-fix 2021-07-23 10:50:51 +02:00
Jennifer
b79690cef4
Merge pull request #6798 from filecoin-project/misc/issueform
Update issue templates to forms
2021-07-23 01:07:02 -04:00