cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity
13,270 Commits 24 Branches 342 Tags 314 MiB
221d1a0e3c
Commit Graph

1174 Commits

Author SHA1 Message Date
Raúl Kripalani
221d1a0e3c fix minor issues in migration. 
- aborting the initialization cancels the AcquireShard context, but
  the DAG store would not release the shard (fixed in
  filecoin-project/dagstore)
- off by one error in console output.
 2021-08-05 19:40:02 +01:00
Raúl Kripalani
f19f5269df upgrade deps. 2021-08-05 14:32:47 +01:00
Raúl Kripalani
270bad8144 pull dagstore migration into Lotus and simplify. 
Migration registers all shards with lazy init.

Shards are then initialized as they are retrieved for the
first time, or in bulk through a lotus-shed tool that will
be provided separately.
 2021-08-04 16:59:27 +01:00
Raúl Kripalani
8f5e6a864f upgrade go-fil-markets. 2021-08-04 13:22:24 +01:00
Raúl Kripalani
be24acbfbe go mod tidy. 2021-08-03 23:43:37 +01:00
Raúl Kripalani
c06c8541f9 refactor dagstore + carv2 integration (+). 
- Integrate with config.toml.
- Export DAGStore to DI context in preparation for JSON-RPC
  and cli commands.
- Renames.
 2021-08-03 23:09:28 +01:00
Raúl Kripalani
b63858f210 upgrade to dagstore v0.4.0. 2021-07-30 15:33:01 +01:00
Aarsh Shah
2e30891005 update dagstore deps 2021-07-30 13:31:34 +05:30
Aarsh Shah
8cd305338a dagstore throttle fetch and copy 2021-07-30 13:02:42 +05:30
Aarsh Shah
fcd630e529 call dagstore start 2021-07-30 01:43:34 +05:30
Raúl Kripalani
073150a306 Merge branch 'release/v1.11.1' into feat/replace-multistore-carv2 2021-07-29 18:42:09 +01:00
Aarsh Shah
029ba39fd9 update deps for logging 2021-07-29 12:19:37 -04:00
aarshkshah1992
0daee83735 more logging in data transfer and markets 2021-07-29 12:08:41 -04:00
hannahhoward
5073c7e6cf feat(deps): update to branches with improved logging 
update sub repos with improved logging around data transfer processing
 2021-07-29 04:31:23 -04:00
Raúl Kripalani
94318da0e6 update go-fil-markets; better log. 2021-07-28 17:14:33 +01:00
Raúl Kripalani
5d047c59b1 bubble up better migrator logging. 2021-07-28 16:48:38 +01:00
Raúl Kripalani
b6112c6fc9 wire in copy throttling. 2021-07-28 16:28:27 +01:00
Raúl Kripalani
1250f8009f upgrade go-fil-markets. 2021-07-27 21:15:24 +01:00
Raúl Kripalani
4b70d13a4b update dependencies. 2021-07-27 21:04:59 +01:00
Raúl Kripalani
987fc0b77d Merge branch 'master' into feat/replace-multistore-carv2 2021-07-27 21:02:52 +01:00
Raúl Kripalani
80b334d04f upgrade dagstore to v0.3.1. 2021-07-27 18:24:23 +01:00
Raúl Kripalani
109a8242ae implement conditional throttling in lotus mount. 2021-07-27 17:24:48 +01:00
Jennifer Wang
d24b2acc63 update to go-fil-market v1.6.0 2021-07-27 04:34:35 -04:00
Raúl Kripalani
9bdbbf33cb Merge branch 'master' into feat/replace-multistore-carv2 2021-07-26 22:15:36 +01:00
Peter Rabbitson
20c5dd5bdc Merge remote-tracking branch 'origin/master' into chore/upgrade_go-multihash 2021-07-26 21:26:10 +02:00
Jiaying Wang
98c8a2abe5
Merge branch 'master' into feat/allow_padding_redux 2021-07-26 13:59:33 -04:00
Peter Rabbitson
ed387b43cc Bump go-multihash, adjust test for supported version 2021-07-26 17:50:19 +02:00
aarshkshah1992
50e1dc6b6e bubble up markets and dagstore changes for m1.1 2021-07-26 19:44:59 +05:30
Mike Greenberg
76a9f4241b feat: Graceful error when api impl is nil 2021-07-26 13:51:37 +02:00
whyrusleeping
78f94c4c6b This pulls in forgotten parts properly implementing PR#5988 
( previous testing focused exclusively on offline dealflow .cars )

Allows a workflow of:

~$ dd if=/dev/urandom bs=1M count=1 | ~/go-ipfs/cmd/ipfs/ipfs add --pin=false
added QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2 QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2

~$ ~/go-ipfs/cmd/ipfs/ipfs dag export QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2 > test_mib.car

~$ lotus client import --car ~/test_mib.car
Import 2, Root QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2

~$ ~/go/bin/stream-commp -p $(( 256 * 1024 * 1024 )) < test_mib.car
CommP:    54e4e75ddc3fffa8fd33d3ededc06e564603ac0fe62543ec6463d51b553be40b
CommPCid: baga6ea4seaqfjzhhlxod775i7uz5h3pnybxfmrqdvqh6mjkd5rsghvi3ku56icy
Raw bytes:           1049073 bytes
Unpadded piece:    266338304 bytes
Padded piece:      268435456 bytes
CARv1 detected in stream:
Blocks:         5
Roots:          1
    1: QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2

~$ curl http://127.0.0.1:1234/rpc/v0 -X POST -H "Authorization: Bearer $(cat ~/.lotus/token)" -H "Content-Type: application/json" --data '
  { "jsonrpc": "2.0", "id":1, "method": "Filecoin.ClientStartDeal", "params": [
    {
      "Wallet":"t01004",
      "Miner":"t01005",
      "EpochPrice":"0",
      "MinBlocksDuration":518400,
      "Data": {
        "Root":{ "/":"QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2" },
        "PieceCid":{ "/":"baga6ea4seaqfjzhhlxod775i7uz5h3pnybxfmrqdvqh6mjkd5rsghvi3ku56icy" },
        "PieceSize": 266338304
      }
    }
  ] }
'

~$ ~/go/bin/stream-commp -p $(( 128 * 1024 * 1024 )) < test_mib.car
CommP:    ed904105399ed346f6b03844abc14710a1748854c2781824d6bd1100e63b1807
CommPCid: baga6ea4seaqo3ecbau4z5u2g62ydqrflyfdrbilurbkme6ayetll2eia4y5rqby
Raw bytes:           1049073 bytes
Unpadded piece:    133169152 bytes
Padded piece:      134217728 bytes
CARv1 detected in stream:
Blocks:         5
Roots:          1
    1: QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2

~$ curl http://127.0.0.1:1234/rpc/v0 -X POST -H "Authorization: Bearer $(cat ~/.lotus/token)" -H "Content-Type: application/json" --data '
  { "jsonrpc": "2.0", "id":1, "method": "Filecoin.ClientStatelessDeal", "params": [
    {
      "Wallet":"t01004",
      "Miner":"t01005",
      "EpochPrice":"0",
      "ProviderCollateral":"0",
      "MinBlocksDuration":518400,
      "Data": {
        "TransferType": "manual",
        "Root":{ "/":"QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2" },
        "PieceCid":{ "/":"baga6ea4seaqo3ecbau4z5u2g62ydqrflyfdrbilurbkme6ayetll2eia4y5rqby" },
        "PieceSize": 133169152
      }
    }
  ] }
'
{"jsonrpc":"2.0","result":{"/":"bafyreianhjvev3w6q5lteap3h7tkxbe2jaobwlsi7vzbcoobjpicg3foqi"},"id":1}

~$ lotus-miner storage-deals import-data bafyreianhjvev3w6q5lteap3h7tkxbe2jaobwlsi7vzbcoobjpicg3foqi ~/test_mib.car
 2021-07-24 18:17:13 +02:00
Peter Rabbitson
8d873e3edc Tests for online and offline dealpadding 
Also bump the times on several flaky tests that can not complete in time
on a typical laptop ( and fail half the time on CircleCI )
 2021-07-24 17:46:04 +02:00
Raúl Kripalani
21f9dc8096 update graphsync to an actual release. 2021-07-22 23:47:47 +01:00
raulk
4ffbb03a82
Merge branch 'master' into raulk/merge-gs-fix 2021-07-22 23:37:17 +01:00
ZenGround0
3155609304 Finalize deps + fix after rebase 2021-07-22 09:49:56 -04:00
ZenGround0
56d71a190f Update versions to have more digits 2021-07-22 09:49:47 -04:00
ZenGround0
47b5afa84b Add version six and a half 2021-07-22 09:49:47 -04:00
ZenGround0
f49a8248f0 PriceListByVersion 2021-07-22 09:49:47 -04:00
aarshkshah1992
c8d328d386 fix bugs in the dagstore wrapper 2021-07-22 09:44:14 +05:30
aarshkshah1992
fd879e0426 update ds 2021-07-22 00:21:39 +05:30
aarshkshah1992
27f3170022 update dagstore 2021-07-21 23:38:22 +05:30
aarshkshah1992
5b1241d213 fix dagstore errors 2021-07-21 23:15:39 +05:30
aarshkshah1992
eaee1578dd log more 2021-07-21 22:47:31 +05:30
aarshkshah1992
9e5c5b624b update dagstore logs 2021-07-21 22:22:26 +05:30
aarshkshah1992
640d69f5cb update dagstire 2021-07-21 21:54:28 +05:30
aarshkshah1992
36876c2601 update deps 2021-07-21 20:50:47 +05:30
aarshkshah1992
ea71e4de60 update dagstore fix 2021-07-21 17:42:19 +05:30
aarshkshah1992
6bee14d6fa update dagstore and markets 2021-07-21 15:34:38 +05:30
aarshkshah1992
213dbc6bc8 update dagstore 2021-07-21 14:54:33 +05:30
aarshkshah1992
d7827da5bb update deps 2021-07-21 09:30:33 +05:30
Aarsh Shah
b2ff2edd0c
Merge branch 'master' into feat/replace-multistore-carv2 2021-07-20 22:03:18 +05:30