Commit Graph
13003 Commits
Author SHA1 Message Date
Łukasz Magiera c2575327c0 Merge pull request #6885 from filecoin-project/feat/update-market
update to go-fil-market v1.6.0
2021-07-27 13:00:38 +02:00
Jennifer Wang d24b2acc63 update to go-fil-market v1.6.0 2021-07-27 04:34:35 -04:00
Peter Rabbitson 2ca174482b Merge pull request #6674 from filecoin-project/chore/upgrade_go-multihash
Bump go-multihash, adjust test for supported version
2021-07-26 21:46:10 +02:00
Peter Rabbitson 20c5dd5bdc Merge remote-tracking branch 'origin/master' into chore/upgrade_go-multihash 2021-07-26 21:26:10 +02:00
Peter Rabbitson 7ffe63a358 Merge pull request #6683 from filecoin-project/feat/allow_padding_redux
Fix padding of deals, which only partially shipped in #5988
2021-07-26 21:12:40 +02:00
Peter Rabbitson 39325e5463 Merge branch 'master' into feat/allow_padding_redux 2021-07-26 20:53:27 +02:00
Jakub Sztandera acc38817ec Merge pull request #6862 from filecoin-project/fix/racy-TestSimultanenousTransferLimit
fix racy TestSimultanenousTransferLimit.
2021-07-26 20:33:28 +02:00
Jiaying Wang 2772b54eda Merge branch 'master' into fix/racy-TestSimultanenousTransferLimit 2021-07-26 14:09:05 -04:00
Jiaying Wang 98c8a2abe5 Merge branch 'master' into feat/allow_padding_redux 2021-07-26 13:59:33 -04:00
vyzo ef24593544 Merge pull request #6867 from filecoin-project/fix/splitstore-warmup
Improve splitstore warmup
2021-07-26 19:15:59 +03:00
vyzo c7e94beddc Merge branch 'master' into fix/splitstore-warmup 2021-07-26 18:56:53 +03:00
Peter Rabbitson ed387b43cc Bump go-multihash, adjust test for supported version 2021-07-26 17:50:19 +02:00
Steven Allen 26e961380a Merge pull request #6872 from filecoin-project/asr/block-height-fix
ValidateBlock: Assert that block header height's are greater than parents
2021-07-26 08:44:06 -07:00
vyzo c52c2738ab fix potential underflow 2021-07-26 18:38:42 +03:00
vyzo 18caefc4a1 fix missing object condition; short-circuit walk 2021-07-26 18:34:05 +03:00
vyzo 0fd8fc24e1 load a full finality worth of state during warmup 2021-07-26 18:34:05 +03:00
Aayush Rajasekaran 104626acb7 ValidateBlock: Assert that block header height's are greater than their parents 2021-07-26 11:09:23 -04:00
Łukasz Magiera 11d3f19a5b Merge pull request #6857 from filecoin-project/mg/feat/graceful-lite-api-w-missing-methods
feat: Don't panic when api impl is nil
2021-07-26 14:26:26 +02:00
Mike Greenberg 76a9f4241b feat: Graceful error when api impl is nil 2021-07-26 13:51:37 +02:00
Łukasz Magiera 5048c3f717 Merge pull request #6811 from filecoin-project/feat/splitstore-shed-utils
splitstore shed utils
2021-07-26 13:44:27 +02:00
vyzo 1f6935f8c4 make gen 2021-07-26 12:15:01 +03:00
vyzo 3c409d5023 require admin for running checks on the splitstore. 2021-07-26 12:13:41 +03:00
vyzo 2cfd73c879 manually compute size when badger is being stupid 2021-07-26 09:46:21 +03:00
vyzo 74009bd67f document lotus-shed splitstore utiilities in the README 2021-07-26 08:52:32 +03:00
vyzo 221dc7024f add splitstore info command 2021-07-26 08:47:02 +03:00
vyzo ce528a1293 implement Info in splitstore 2021-07-26 08:45:46 +03:00
vyzo 30e4b405b7 implement BlockstoreSize for badger 2021-07-26 08:43:09 +03:00
vyzo baaa9a7738 add BlockstoreSize trait for reporting size 2021-07-26 08:42:54 +03:00
vyzo 77604db716 make gen 2021-07-26 08:33:25 +03:00
vyzo a0d6fdba33 add ChainBlockstoreInfo APIv1 endpoint 2021-07-26 08:30:07 +03:00
vyzo c00b86e8a8 stop the walk on missing references 2021-07-25 13:42:20 +03:00
vyzo 2dc72d5849 satisfy linter who wants to be a spell checker in comments 2021-07-25 11:47:21 +03:00
vyzo 5285a14d27 write check summary at the end 2021-07-25 11:42:13 +03:00
vyzo 5718da237f handle newlines consistently in check output 2021-07-25 11:30:58 +03:00
vyzo 21bb2bda09 make gen 2021-07-25 11:25:29 +03:00
vyzo 5b2e4d8ad4 add permission tag to ChainCheckBlockstore API 2021-07-25 11:24:06 +03:00
vyzo c99dc3e811 add splitstore check command 2021-07-25 11:24:06 +03:00
vyzo 3d2ae433ee add ChainCheckBlockstore API 2021-07-25 11:14:48 +03:00
vyzo 1918ffda4e implement splitstore check 2021-07-25 10:35:37 +03:00
vyzo e317c831a8 quiet excessive badger logs 2021-07-25 09:07:27 +03:00
vyzo ce6f410f2a add options to control compaction/gc of the coldstore and config rewrite during rollback 2021-07-25 09:01:34 +03:00
vyzoandraulk 254c489164 fix typo
Co-authored-by: raulk <raul@protocol.ai>
2021-07-25 09:01:34 +03:00
vyzoandraulk 33cdc90f13 fix typo
Co-authored-by: raulk <raul@protocol.ai>
2021-07-25 09:01:34 +03:00
vyzo da66e7a878 fix typo 2021-07-25 09:01:34 +03:00
vyzo e696a2c0cc fix newline in progres message 2021-07-25 09:01:34 +03:00
vyzo 36b209ca5c compact and gc coldstore after copying 2021-07-25 09:01:34 +03:00
vyzo 92b9d8c895 quiet linter 2021-07-25 09:01:34 +03:00
vyzo 0c68bccbda add splitstore rollback lotus-shed command 2021-07-25 09:01:34 +03:00
whyrusleeping 78f94c4c6b This pulls in forgotten parts properly implementing PR#5988
( previous testing focused exclusively on offline dealflow .cars )

Allows a workflow of:

~$ dd if=/dev/urandom bs=1M count=1 | ~/go-ipfs/cmd/ipfs/ipfs add --pin=false
added QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2 QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2

~$ ~/go-ipfs/cmd/ipfs/ipfs dag export QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2 > test_mib.car

~$ lotus client import --car ~/test_mib.car
Import 2, Root QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2

~$ ~/go/bin/stream-commp -p $(( 256 * 1024 * 1024 )) < test_mib.car
CommP:    54e4e75ddc3fffa8fd33d3ededc06e564603ac0fe62543ec6463d51b553be40b
CommPCid: baga6ea4seaqfjzhhlxod775i7uz5h3pnybxfmrqdvqh6mjkd5rsghvi3ku56icy
Raw bytes:           1049073 bytes
Unpadded piece:    266338304 bytes
Padded piece:      268435456 bytes
CARv1 detected in stream:
Blocks:         5
Roots:          1
    1: QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2

~$ curl http://127.0.0.1:1234/rpc/v0 -X POST -H "Authorization: Bearer $(cat ~/.lotus/token)" -H "Content-Type: application/json" --data '
  { "jsonrpc": "2.0", "id":1, "method": "Filecoin.ClientStartDeal", "params": [
    {
      "Wallet":"t01004",
      "Miner":"t01005",
      "EpochPrice":"0",
      "MinBlocksDuration":518400,
      "Data": {
        "Root":{ "/":"QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2" },
        "PieceCid":{ "/":"baga6ea4seaqfjzhhlxod775i7uz5h3pnybxfmrqdvqh6mjkd5rsghvi3ku56icy" },
        "PieceSize": 266338304
      }
    }
  ] }
'

~$ ~/go/bin/stream-commp -p $(( 128 * 1024 * 1024 )) < test_mib.car
CommP:    ed904105399ed346f6b03844abc14710a1748854c2781824d6bd1100e63b1807
CommPCid: baga6ea4seaqo3ecbau4z5u2g62ydqrflyfdrbilurbkme6ayetll2eia4y5rqby
Raw bytes:           1049073 bytes
Unpadded piece:    133169152 bytes
Padded piece:      134217728 bytes
CARv1 detected in stream:
Blocks:         5
Roots:          1
    1: QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2

~$ curl http://127.0.0.1:1234/rpc/v0 -X POST -H "Authorization: Bearer $(cat ~/.lotus/token)" -H "Content-Type: application/json" --data '
  { "jsonrpc": "2.0", "id":1, "method": "Filecoin.ClientStatelessDeal", "params": [
    {
      "Wallet":"t01004",
      "Miner":"t01005",
      "EpochPrice":"0",
      "ProviderCollateral":"0",
      "MinBlocksDuration":518400,
      "Data": {
        "TransferType": "manual",
        "Root":{ "/":"QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2" },
        "PieceCid":{ "/":"baga6ea4seaqo3ecbau4z5u2g62ydqrflyfdrbilurbkme6ayetll2eia4y5rqby" },
        "PieceSize": 133169152
      }
    }
  ] }
'
{"jsonrpc":"2.0","result":{"/":"bafyreianhjvev3w6q5lteap3h7tkxbe2jaobwlsi7vzbcoobjpicg3foqi"},"id":1}

~$ lotus-miner storage-deals import-data bafyreianhjvev3w6q5lteap3h7tkxbe2jaobwlsi7vzbcoobjpicg3foqi ~/test_mib.car
2021-07-24 18:17:13 +02:00
Peter Rabbitson 8d873e3edc Tests for online and offline dealpadding
Also bump the times on several flaky tests that can not complete in time
on a typical laptop ( and fail half the time on CircleCI )
2021-07-24 17:46:04 +02:00