cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity
21,322 Commits 24 Branches 342 Tags 314 MiB
0d96c6ee46
Commit Graph

1603 Commits

Author SHA1 Message Date
Hector Sanjuan
fa93c23813 Chain ranged export: rework and address current shortcomings 
This commit moderately refactors the ranged export code. It addresses several
problems:
  * Code does not finish cleanly and things hang on ctrl-c
  * Same block is read multiple times in a row (artificially increasing cached
    blockstore metrics to 50%)
  * It is unclear whether there are additional races (a single worker quits
    when reaching height 0)
  * CARs produced have duplicated blocks (~400k for an 80M-blocks CAR or
    so). Some blocks appear up to 5 times.
  * Using pointers for tasks where it is not necessary.

The changes:

  * Use a FIFO instead of stack: simpler implementation as its own type. This
has not proven to be much more memory-friendly, but it has not made things
worse either.
  * We avoid a probably not small amount of allocations by not using
    unnecessary pointers.
  * Fix duplicated blocks by atomically checking+adding to CID set.
  * Context-termination now works correctly. Worker lifetime is correctly tracked and all channels
  are closed, avoiding any memory leaks and deadlocks.
  * We ensure all work is finished before finishing, something that might have
  been broken in some edge cases previously. In practice, we would not have
  seen this except perhaps in very early snapshots close to genesis.

Initial testing shows the code is currently about 5% faster. Resulting
snapshots do not have duplicates so they are a bit smaller. We have manually
verified that no CID is lost versus previous results, with both old and recent
snapshots.
 2023-02-14 21:08:10 +01:00
frrist
21efd481d8 First efficient ranged-export implementation by @frisst 
This first commit contains the first and second implementation stabs (after
primary review by @hsanjuan), using a stack for task buffering.

Known issues: ctrl-c (context cancellation) results in the export code getting
deadlocked. Duplicate blocks in exports. Duplicate block reads from store.

Original commit messages:

works

works against mainnet and calibnet

feat: add internal export api method

- will hopfully make things faster by not streaming the export over the json rpc api

polish: better file nameing

fix: potential race in marking cids as seen

chore: improve logging

feat: front export with cache

fix: give hector a good channel buffer on this shit

docsgen
 2023-02-14 15:41:10 +01:00
Raúl Kripalani
8ff55c027e Merge branch 'release/v1.20.0' into asr/merge-release-into-master 2023-02-13 11:45:09 +00:00
ychiao
3c580403c1 implement itest and handle optional params 2023-02-10 13:33:59 -05:00
Aayush Rajasekaran
693f230fbb Merge branch 'release/v1.20.0' into asr/merge-release-into-master 2023-02-07 11:03:01 -05:00
Łukasz Magiera
6851b6a523 Merge remote-tracking branch 'origin/release/v1.20.0' into fix/eth-orpc-validate 2023-02-06 10:41:52 +01:00
Aayush
f291572b4a Merge branch 'release/v1.20.0' into asr/merge-release-into-master 2023-02-03 09:27:04 -05:00
Łukasz Magiera
a6a2f0dbae gateway: Support all EthModule methods 2023-02-02 13:37:31 +01:00
Ian Davis
245be406d9 Fix merge conflicts 2023-02-01 16:41:01 +00:00
Łukasz Magiera
ad14d71978 eth rpc: Params are optional in eth_subscribe 2023-01-31 11:08:51 +01:00
Łukasz Magiera
1286d76988 gateway: eth_subscribe support 2023-01-31 10:28:12 +01:00
Łukasz Magiera
6491becbe1 rpc: Switch eth_subscribe to reverse calls 2023-01-31 10:28:12 +01:00
Jorropo
f572852d06 chore: all: bump go-libipfs to replace go-block-format 
Includes changes from:
- https://github.com/ipfs/go-block-format/pull/37
- https://github.com/ipfs/go-libipfs/pull/58
 2023-01-26 17:03:18 +01:00
ychiao
ee6af9de61 Eth JSON-RPC: implement web3_clientVersion 2023-01-24 22:56:04 -05:00
Geoff Stuart
5133b9da94 Add EthGetMessageCidByTransactionHash, fix EthGetTransactionHashByCid 2023-01-18 17:48:10 -05:00
Geoff Stuart
a8436074a6 Store mapping from hashes for Ethereum transactions to Filecoin Message Cids 2023-01-16 01:17:56 -05:00
Łukasz Magiera
eadf8b680c address review 2023-01-13 21:55:14 -05:00
Łukasz Magiera
708618d11c feat: gateway: eth_ api support 2023-01-13 20:04:41 +01:00
raulk
014d95454b
address review comments from @magik6k on #9617 (#9997) 2023-01-12 19:11:48 +00:00
ychiao
105a1259bc
Eth JSON-RPC: use ToFilecoinAddress() to get f4 addr and validate v sig + cleanups (#9970) 
Co-authored-by: raulk <raul@protocol.ai>
 2023-01-12 14:55:07 +00:00
Raúl Kripalani
8f4c4268ba cleanup. 2023-01-12 08:50:38 +00:00
ychiao
7029364f4d check non-existent block hash 2023-01-02 17:21:47 +08:00
Aayush Rajasekaran
a4d93d7088
feat: docgen: use real values for Eth types (#9938) 2022-12-27 16:15:22 +00:00
Aayush
a5851a1763 Merge branch 'master' into feat/nv18-fevm 2022-12-23 16:37:26 -05:00
Aayush
f4ffff7334 fix: docgen: correctly check for exported fields 2022-12-23 12:59:12 -05:00
Aayush
9d3e50f140 feat: implement and test the v18 migration 2022-12-22 13:05:00 -05:00
Geoff Stuart
4ae498a72e Fixes 2022-12-16 17:24:46 -05:00
Geoff Stuart
92f3a7b050 Merge branch 'feat/nv18-fevm' into feat/nv18-events 2022-12-16 16:27:14 -05:00
Geoff Stuart
bc7c190db0 Move all eth stuff into ethtypes 2022-12-16 13:48:06 -05:00
Geoff Stuart
442132be70 Move eth types to chain/types/ethtypes 2022-12-14 14:08:38 -05:00
Geoff Stuart
96bb03e1e0 Move EthTxReceipt to api 2022-12-14 14:00:19 -05:00
Geoff Stuart
855c35cb88 Move eth functions out of api 2022-12-14 11:57:01 -05:00
Steven Allen
266492fa1c Merge branch 'feat/nv18-fevm' into feat/nv18-events 2022-12-07 22:16:56 -08:00
Ian Davis
1afc32f650 Move FilterID to types package 2022-12-07 18:34:19 +00:00
Ian Davis
067f08f092 Fix marshal test 2022-12-07 13:53:25 +00:00
Ian Davis
56259c40fc Use EthHash compatible type for subscription and filter IDs 2022-12-07 12:51:03 +00:00
Aayush Rajasekaran
6443e72ad7 Merge branch 'master' into feat/nv18-fevm 2022-11-29 20:16:12 -06:00
Aayush Rajasekaran
15afe384ff Merge branch 'master' into feat/nv18 2022-11-29 10:01:51 -06:00
Ian Davis
0a4efd4820 Merge branch 'main' into id/update-go-version 2022-11-29 14:29:11 +00:00
ychiao
f1493fbee7
eth: rpc: fix sendRawTransaction, transactionIndex, feeHistory (#9736) 
* Eth JSON-RPC API: add transactionIndex

* Eth JSON-RPC API: fix EthFeeHistory and EthSendRawTransaction

* Eth JSON-RPC API: fix parsing block parameter for getBlockByNumber

* fix itest
 2022-11-27 18:44:43 -08:00
Ian Davis
9f85d3dca7 Address simple linter issues 2022-11-24 16:32:27 +00:00
Łukasz Magiera
87e5549af5 sealing: Split unsealed cleanup from Finalize 2022-11-23 18:57:16 +01:00
Łukasz Magiera
211712bf6d itests: Test snapdeals abort cleanup 2022-11-23 18:57:16 +01:00
Steven Allen
176275267b Merge branch 'feat/nv18' into feat/nv18-fevm 2022-11-22 10:32:26 -08:00
Steven Allen
2651a38f89 Merge branch 'master' into feat/nv18 2022-11-22 10:08:54 -08:00
lovedret
e56d4b06df
fix: typos in eth based struct (#9656) 2022-11-21 22:39:03 +00:00
Raúl Kripalani
c474aeee5e fix: Eth JSON-RPC API: return empty bloom on every block. 2022-11-18 19:49:39 +00:00
Raúl Kripalani
ec667d49f4 fix: Eth JSON-RPC API: empty EthBytes serializes to '0x'. 2022-11-18 19:01:30 +00:00
Raúl Kripalani
2ff679c679 fix test compilation. 2022-11-16 22:18:40 +00:00
Raúl Kripalani
23007b0b3b fix: do not hash log values; decode as cbor byte arrays. 2022-11-16 21:45:42 +00:00
Raúl Kripalani
d441425618 eth_getTransactionReceipt: return type & non-zero bloom if logs, rm root. 2022-11-16 21:16:27 +00:00
Raúl Kripalani
a536f533d8 Eth JSON-RPC API: make EthLog#Data a string. 2022-11-16 20:51:28 +00:00
Raúl Kripalani
1ab39a40da simplify by moving receipt constructor logic to API. 2022-11-16 14:15:29 +00:00
Raúl Kripalani
32385a97cd Eth JSON-RPC API: return logs in eth_getTransactionReceipt. 2022-11-16 13:53:27 +00:00
Ian Davis
314fb31886 Fix signature of EthSubscribe 2022-11-16 12:18:25 +00:00
Ian Davis
8134d2f05b Parse block heights as hex 2022-11-16 12:08:48 +00:00
Ian Davis
10357112e5 Add basic itests for ethereum filter api 2022-11-15 18:02:36 +00:00
Raúl Kripalani
846aea57c5 Merge branch 'feat/nv18-fevm' into feat/nv18-events 2022-11-15 12:28:25 +00:00
Shrenuj Bansal
b95d1a6323 Merge branch 'master' into sbansal/nonce-coordination-and-consensus-for-chain-nodes 2022-11-14 17:50:52 -05:00
Geoff Stuart
95af40543a Fix lotus-miner proving check 2022-11-14 15:29:50 -05:00
Raúl Kripalani
a915c703a5 Merge branch 'feat/nv18-events' into feat/nv18-ethevents 2022-11-14 11:46:08 +00:00
Ian Davis
91952f84bd Fix eth types comments 2022-11-14 10:27:06 +00:00
ychiao
8a7367f1c9
api: ethrpc: add missing fields (#9631) 
* api: ethrpc: add missing fields

* fix make gen
 2022-11-12 09:44:52 -08:00
raulk
4e7b8d555e
Merge pull request #9624 from iand/feat/nv18-rcptevent 2022-11-11 20:25:37 +00:00
Raúl Kripalani
9c5d888ec9 docs gen; also fixes lack of *cid.Cid examples elsewhere. 2022-11-11 20:20:28 +00:00
Shrenuj Bansal
2fa21ff091 Merge branch 'master' into sbansal/nonce-coordination-and-consensus-for-chain-nodes 2022-11-11 14:41:38 -05:00
vyzo
7664e0691c fix tx check and test 2022-11-11 05:49:06 +02:00
vyzo
4c76f654b9 eth api: remove bad todo link 2022-11-11 05:20:10 +02:00
vyzo
54f311d94a eth rlp: fix warts 2022-11-11 04:58:15 +02:00
vyzo
18f2a14a00 eth api: fix warts 2022-11-11 04:52:38 +02:00
Ian Davis
7383ecb0ba make gen 2022-11-10 15:04:58 +00:00
Ian Davis
69210d0917 Ethereum compatible actor event API 2022-11-10 15:02:53 +00:00
vyzo
2588b45826 fix gst builtin module refs 
it breaks the gen script in proxy_gen
 2022-11-10 04:46:13 +02:00
vyzo
6d7a33ff70 api: ethrpc: fix unit test 
and skip a non-testing test; printing doesn't constitute testing.
 2022-11-10 02:52:06 +02:00
vyzo
deea6797c5 make gen 2022-11-10 02:52:06 +02:00
vyzo
2b23372ad3 fixup merge artifacts 2022-11-10 02:51:57 +02:00
imxyb
53c7d9c18f fix:add UnmarshalJSON for EthNonce (#9589) 2022-11-09 19:47:44 +02:00
Steven Allen
49da019bd0 fix: allow EthCall.From to be nil (#9556) 2022-11-09 19:47:25 +02:00
Steven Allen
6337d594fa fix: evm: use CREATE not CREATE2 for EVM contract deployment (#9553) 
Also updates the actors to accommodate this change, and fix a bug in
looking up addresses for f4 actors.
 2022-11-09 19:47:23 +02:00
Raúl Kripalani
dc5fc9ded4 fix eth_feeHistory. 2022-11-09 19:47:07 +02:00
Raúl Kripalani
ab5f0216f6 fix message sending logic. 2022-11-09 19:47:07 +02:00
Raúl Kripalani
32617a61df fix eth_feeHistory: add missing param. 2022-11-09 19:47:06 +02:00
Raúl Kripalani
6d5ec6aff8 fix address recovery from pubkey. 2022-11-09 19:46:24 +02:00
Raúl Kripalani
9017e5de52 fix signature validation logic. 2022-11-09 19:46:24 +02:00
Kevin Li
64afdfc642 api: ethrpc: implement eth_feeHistory (#9539) 
Co-authored-by: Raúl Kripalani <raul@protocol.ai>
 2022-11-09 19:46:22 +02:00
Kevin Li
e17e92775c api: ethrpc: fix eth_getCode, eth_getTransactionReceipt (#9530) 2022-11-09 19:40:00 +02:00
Steven Allen
56b238980b feat: use f4 eth addresses wherever possible (#9532) 
Co-authored-by: Raúl Kripalani <raul@protocol.ai>
 2022-11-09 19:39:50 +02:00
Steven Allen
aa0e6c17b9 Eth TX Encoding (#9531) 2022-11-09 19:39:49 +02:00
Melanie Riise
7e7fff1dc5 Eth JSON-RPC API: Add EAM as contract creator (#9528) 2022-11-09 19:33:25 +02:00
Raúl Kripalani
98e534cc77 add support for f4 addresses in the Eth JSON-RPC and signature. 2022-11-09 19:33:23 +02:00
Raúl Kripalani
17e9e97064 Eth JSON-RPC API: support f4 addresses. 2022-11-09 19:32:43 +02:00
Steven Allen
420bd34888 fix(api): eth/id address conversion (#9511) 2022-11-09 19:30:40 +02:00
Kevin Li
e109be4c6a fix lint 2022-11-09 19:30:27 +02:00
vyzo
61de2f4a4a fix eth api issues with evm constructor changes 2022-11-09 19:30:08 +02:00
raulk
ee83dceef0 Eth JSON-RPC API: implement eth_getCode and eth_getStorageAt (#9397) 2022-11-09 19:30:06 +02:00
Kevin Li
6c7fd94aa7 api: ethrpc: implement eth_sendRawTransaction (#9334) 
Co-authored-by: Raúl Kripalani <raul@protocol.ai>
 2022-11-09 19:28:17 +02:00
Kevin Li
b53d5924a1 feat: ethrpc: implement call, estimateGas, getTransactionCount (#9306) 2022-11-09 19:25:40 +02:00
Kevin Li
e5bb5b7430 feat: ethrpc: implement EthBlock and EthTx structs (#9287) 
Co-authored-by: Raúl Kripalani <raul@protocol.ai>
 2022-11-09 19:23:49 +02:00
Kevin Li
4652d8559f api: ethrpc: implement a few ethereum JSON-RPC API methods (#9208) 2022-11-09 19:23:11 +02:00
vyzo
6bf7b0312e make gen 2022-11-09 10:45:11 +02:00
Łukasz Magiera
2c89b3240f retrieval: Support retrievals into remote stores 2022-11-08 09:37:34 +00:00
Aayush
c0b7343e60 Merge branch 'release/v1.18.0' into asr/merge-release-into-master 2022-11-06 14:40:13 -05:00
simlecode
5284f60e83 fix: add fields to ForkUpgradeParams 2022-10-22 23:41:52 +08:00
Shrenuj Bansal
15ed1ee33c Merge branch 'master' into sbansal/nonce-coordination-and-consensus-for-chain-nodes 2022-10-17 22:55:48 -04:00
Łukasz Magiera
dc102f076a sealing pipeline: Get allocationId with StateApi 2022-10-12 13:54:03 -04:00
Aayush Rajasekaran
e838a86df9
Merge pull request #9437 from filecoin-project/gstuart/verifreg-accessors 
feat: add API methods to get allocations and claims
 2022-10-12 13:09:43 -04:00
Łukasz Magiera
2c11f9d265 feat: Add node uptime rpc / output in info command 2022-10-11 10:11:09 +02:00
simlecode
11b15dbcd1 fix: missing permissions tag 2022-10-10 16:54:07 +08:00
Geoff Stuart
e2d5d12e7f Add accessors for allocations and claims maps 2022-10-07 16:41:59 -04:00
Geoff Stuart
b5c5e6627e review fixes 2022-10-06 11:06:21 -04:00
Geoff Stuart
f55dc46a32 Add api for getting allocation 2022-10-06 11:06:21 -04:00
Geoff Stuart
b4c04ad927 update markets 2022-10-06 11:06:21 -04:00
Shrenuj Bansal
f89a682d98 Add Mpool ref to raft state and rearrange some APIs 2022-09-29 10:56:57 +00:00
Shrenuj Bansal
559c2c6d34 Merge branch 'master' into sbansal/nonce-coordination-and-consensus-for-chain-nodes 2022-09-27 16:29:03 +00:00
Shrenuj Bansal
99e7c322eb More wip 2022-09-27 16:08:04 +00:00
Aayush
9c242a8012 address magik supernit 2022-09-27 15:34:01 +00:00
Aayush
8244297b89 chore: refactor: move retry test to API 2022-09-27 15:34:01 +00:00
Aayush
13f3e0aca4 fix: itest: check for closed connection 2022-09-27 15:34:01 +00:00
Aayush
5d28eea968 chore: deps: update to go-jsonrpc 0.1.8 2022-09-27 15:34:01 +00:00
Aayush
27880ece2b feat: support typed errors over RPC 2022-09-27 15:34:01 +00:00
Aayush
3e81c56349 address magik supernit 2022-09-23 11:41:53 -04:00
Aayush
3c195e4d71 chore: refactor: move retry test to API 2022-09-23 11:37:13 -04:00
Aayush
06262868f5 fix: itest: check for closed connection 2022-09-22 13:58:07 -04:00
Aayush
6189932bef chore: deps: update to go-jsonrpc 0.1.8 2022-09-22 11:44:19 -04:00
Aayush
93f94d4ee9 feat: support typed errors over RPC 2022-09-22 11:44:19 -04:00
Aayush
32670e810c chore: refactor: rename NewestNetworkVersion 2022-09-21 15:48:51 -04:00
Łukasz Magiera
859c2606f0 sealing: Address review 2022-09-19 12:13:06 +02:00
Łukasz Magiera
061a990eb8 sector import: RemoteSealingDoneEndpoint 2022-09-19 12:13:06 +02:00
Łukasz Magiera
c17ab761c5 sector import: Remote Commit2 2022-09-19 12:13:06 +02:00
Łukasz Magiera
9f03569cd0 sector import: Remote Commit1 2022-09-19 12:13:06 +02:00
Łukasz Magiera
fef7232c6b sector import: Initial SectorReceive itest 2022-09-19 12:13:06 +02:00
Łukasz Magiera
fbb487ae2b sector import: Plumbing for DownloadSectorData in the sealing system 2022-09-19 12:13:05 +02:00
Łukasz Magiera
2318419291 sector import: Fix missing perm tag on SectorReceive 2022-09-19 12:13:05 +02:00
Łukasz Magiera
5c485c3375 sealing: Drop redundant Piece type 2022-09-19 12:13:05 +02:00
Łukasz Magiera
29135aa77c sector import: Initial api scaffolding 2022-09-19 12:13:03 +02:00
Geoff Stuart
d94cdaaade Fixes node startup 2022-09-14 13:15:17 -04:00
Geoff Stuart
21906b5a63 add api and cli calls for beneficiary withdrawl 2022-09-14 12:26:35 -04:00
Shrenuj Bansal
4be8861f41 Merge branch 'master' into sbansal/nonce-coordination-and-consensus-for-chain-nodes 2022-09-13 17:41:06 -04:00
Shrenuj Bansal
a1f2fdb706 Merge branch 'master' into sbansal/nonce-coordination-and-consensus-for-chain-nodes 2022-09-12 16:23:24 -04:00
Shrenuj Bansal
8f1b1bb1ff WIP: Raft consensus for lotus nodes in a cluster 2022-09-12 16:10:15 -04:00
Aayush
c0bbaebaf4 more fixes 2022-09-09 21:40:15 -04:00
Aayush
92cc24fca2 support fvm gas tracing 2022-09-09 21:40:15 -04:00
Geoff Stuart
642555d0fb Fix discrepancies, run fiximports 2022-09-09 21:40:15 -04:00
Aayush
0efca4d266 introduce v9 actors and nv17 2022-09-09 21:40:15 -04:00
LexLuthr
67d4f905e6
feat: proving: Introduce manual sector fault recovery (#9144) 
* rebase

* fix lint error

* fix errors

* add itest

* make gen after rebase

* apply suggestion from review

* make gen after latest rebase

* gen after rebase

* fix waitgroup

* change as per review

* refactor Recovery Batches

* fix CLI

* gen after rebase

* close the channel
 2022-09-06 21:17:30 +05:30
Łukasz Magiera
08b22edd89 fix make gen 2022-08-29 16:25:30 +02:00
Łukasz Magiera
45d1bd61ce
Merge pull request #9183 from filecoin-project/feat/sectornum-mgmt 
feat: sealing: Use bitfields to manage sector numbers
 2022-08-26 10:59:24 -04:00
Łukasz Magiera
2086b219d2 Don't use go-libp2p-core 2022-08-25 14:20:41 -04:00
Łukasz Magiera
335f5df3ec numassigner: SectorNumReserveCount 2022-08-24 11:25:37 -04:00
Łukasz Magiera
b290dac3cb Merge remote-tracking branch 'origin/master' into feat/sectornum-mgmt 2022-08-23 11:44:43 -04:00