cerc-io/lotus
15
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity
19,545 Commits 24 Branches 342 Tags 314 MiB
a1c27038ed
Commit Graph

1555 Commits

Author SHA1 Message Date
Hector Sanjuan
ec01036871 Ranged export: remove non-internal ranged export method 
Additionally, per feedback:

* Set "admin" permissions
* Remove from v0api
 2023-02-14 21:08:10 +01:00
Hector Sanjuan
1bb698619c Ranged-export: Remove CachingBlockstore 
The improvements in the range-export code lead to avoid reading most blocks
twice, as well as to allowing some blocks to be written to disk multiple times.

The cache hit-rate went down from being close to 50% to a maximum of 12% at
the very end of the export. The reason is that most CIDs are never read twice
since they are correctly tracked in the CID set.

These numbers do not support the maintenance of the CachingBlockstore
code. Additional testing shows that removing it has similar memory-usage
behaviour and about 5 minute-faster execution (around 10%).

Less code to maintain and less options to mess up with.
 2023-02-14 21:08:10 +01:00
Hector Sanjuan
fa93c23813 Chain ranged export: rework and address current shortcomings 
This commit moderately refactors the ranged export code. It addresses several
problems:
  * Code does not finish cleanly and things hang on ctrl-c
  * Same block is read multiple times in a row (artificially increasing cached
    blockstore metrics to 50%)
  * It is unclear whether there are additional races (a single worker quits
    when reaching height 0)
  * CARs produced have duplicated blocks (~400k for an 80M-blocks CAR or
    so). Some blocks appear up to 5 times.
  * Using pointers for tasks where it is not necessary.

The changes:

  * Use a FIFO instead of stack: simpler implementation as its own type. This
has not proven to be much more memory-friendly, but it has not made things
worse either.
  * We avoid a probably not small amount of allocations by not using
    unnecessary pointers.
  * Fix duplicated blocks by atomically checking+adding to CID set.
  * Context-termination now works correctly. Worker lifetime is correctly tracked and all channels
  are closed, avoiding any memory leaks and deadlocks.
  * We ensure all work is finished before finishing, something that might have
  been broken in some edge cases previously. In practice, we would not have
  seen this except perhaps in very early snapshots close to genesis.

Initial testing shows the code is currently about 5% faster. Resulting
snapshots do not have duplicates so they are a bit smaller. We have manually
verified that no CID is lost versus previous results, with both old and recent
snapshots.
 2023-02-14 21:08:10 +01:00
frrist
21efd481d8 First efficient ranged-export implementation by @frisst 
This first commit contains the first and second implementation stabs (after
primary review by @hsanjuan), using a stack for task buffering.

Known issues: ctrl-c (context cancellation) results in the export code getting
deadlocked. Duplicate blocks in exports. Duplicate block reads from store.

Original commit messages:

works

works against mainnet and calibnet

feat: add internal export api method

- will hopfully make things faster by not streaming the export over the json rpc api

polish: better file nameing

fix: potential race in marking cids as seen

chore: improve logging

feat: front export with cache

fix: give hector a good channel buffer on this shit

docsgen
 2023-02-14 15:41:10 +01:00
Raúl Kripalani
8ff55c027e Merge branch 'release/v1.20.0' into asr/merge-release-into-master 2023-02-13 11:45:09 +00:00
ychiao
3c580403c1 implement itest and handle optional params 2023-02-10 13:33:59 -05:00
Aayush Rajasekaran
693f230fbb Merge branch 'release/v1.20.0' into asr/merge-release-into-master 2023-02-07 11:03:01 -05:00
Łukasz Magiera
6851b6a523 Merge remote-tracking branch 'origin/release/v1.20.0' into fix/eth-orpc-validate 2023-02-06 10:41:52 +01:00
Aayush
f291572b4a Merge branch 'release/v1.20.0' into asr/merge-release-into-master 2023-02-03 09:27:04 -05:00
Łukasz Magiera
a6a2f0dbae gateway: Support all EthModule methods 2023-02-02 13:37:31 +01:00
Ian Davis
245be406d9 Fix merge conflicts 2023-02-01 16:41:01 +00:00
Łukasz Magiera
ad14d71978 eth rpc: Params are optional in eth_subscribe 2023-01-31 11:08:51 +01:00
Łukasz Magiera
1286d76988 gateway: eth_subscribe support 2023-01-31 10:28:12 +01:00
Łukasz Magiera
6491becbe1 rpc: Switch eth_subscribe to reverse calls 2023-01-31 10:28:12 +01:00
Jorropo
f572852d06 chore: all: bump go-libipfs to replace go-block-format 
Includes changes from:
- https://github.com/ipfs/go-block-format/pull/37
- https://github.com/ipfs/go-libipfs/pull/58
 2023-01-26 17:03:18 +01:00
ychiao
ee6af9de61 Eth JSON-RPC: implement web3_clientVersion 2023-01-24 22:56:04 -05:00
Geoff Stuart
5133b9da94 Add EthGetMessageCidByTransactionHash, fix EthGetTransactionHashByCid 2023-01-18 17:48:10 -05:00
Geoff Stuart
a8436074a6 Store mapping from hashes for Ethereum transactions to Filecoin Message Cids 2023-01-16 01:17:56 -05:00
Łukasz Magiera
eadf8b680c address review 2023-01-13 21:55:14 -05:00
Łukasz Magiera
708618d11c feat: gateway: eth_ api support 2023-01-13 20:04:41 +01:00
raulk
014d95454b
address review comments from @magik6k on #9617 (#9997) 2023-01-12 19:11:48 +00:00
ychiao
105a1259bc
Eth JSON-RPC: use ToFilecoinAddress() to get f4 addr and validate v sig + cleanups (#9970) 
Co-authored-by: raulk <raul@protocol.ai>
 2023-01-12 14:55:07 +00:00
Raúl Kripalani
8f4c4268ba cleanup. 2023-01-12 08:50:38 +00:00
ychiao
7029364f4d check non-existent block hash 2023-01-02 17:21:47 +08:00
Aayush Rajasekaran
a4d93d7088
feat: docgen: use real values for Eth types (#9938) 2022-12-27 16:15:22 +00:00
Aayush
a5851a1763 Merge branch 'master' into feat/nv18-fevm 2022-12-23 16:37:26 -05:00
Aayush
f4ffff7334 fix: docgen: correctly check for exported fields 2022-12-23 12:59:12 -05:00
Aayush
9d3e50f140 feat: implement and test the v18 migration 2022-12-22 13:05:00 -05:00
Geoff Stuart
4ae498a72e Fixes 2022-12-16 17:24:46 -05:00
Geoff Stuart
92f3a7b050 Merge branch 'feat/nv18-fevm' into feat/nv18-events 2022-12-16 16:27:14 -05:00
Geoff Stuart
bc7c190db0 Move all eth stuff into ethtypes 2022-12-16 13:48:06 -05:00
Geoff Stuart
442132be70 Move eth types to chain/types/ethtypes 2022-12-14 14:08:38 -05:00
Geoff Stuart
96bb03e1e0 Move EthTxReceipt to api 2022-12-14 14:00:19 -05:00
Geoff Stuart
855c35cb88 Move eth functions out of api 2022-12-14 11:57:01 -05:00
Steven Allen
266492fa1c Merge branch 'feat/nv18-fevm' into feat/nv18-events 2022-12-07 22:16:56 -08:00
Ian Davis
1afc32f650 Move FilterID to types package 2022-12-07 18:34:19 +00:00
Ian Davis
067f08f092 Fix marshal test 2022-12-07 13:53:25 +00:00
Ian Davis
56259c40fc Use EthHash compatible type for subscription and filter IDs 2022-12-07 12:51:03 +00:00
Aayush Rajasekaran
6443e72ad7 Merge branch 'master' into feat/nv18-fevm 2022-11-29 20:16:12 -06:00
Aayush Rajasekaran
15afe384ff Merge branch 'master' into feat/nv18 2022-11-29 10:01:51 -06:00
Ian Davis
0a4efd4820 Merge branch 'main' into id/update-go-version 2022-11-29 14:29:11 +00:00
ychiao
f1493fbee7
eth: rpc: fix sendRawTransaction, transactionIndex, feeHistory (#9736) 
* Eth JSON-RPC API: add transactionIndex

* Eth JSON-RPC API: fix EthFeeHistory and EthSendRawTransaction

* Eth JSON-RPC API: fix parsing block parameter for getBlockByNumber

* fix itest
 2022-11-27 18:44:43 -08:00
Ian Davis
9f85d3dca7 Address simple linter issues 2022-11-24 16:32:27 +00:00
Łukasz Magiera
87e5549af5 sealing: Split unsealed cleanup from Finalize 2022-11-23 18:57:16 +01:00
Łukasz Magiera
211712bf6d itests: Test snapdeals abort cleanup 2022-11-23 18:57:16 +01:00
Steven Allen
176275267b Merge branch 'feat/nv18' into feat/nv18-fevm 2022-11-22 10:32:26 -08:00
Steven Allen
2651a38f89 Merge branch 'master' into feat/nv18 2022-11-22 10:08:54 -08:00
lovedret
e56d4b06df
fix: typos in eth based struct (#9656) 2022-11-21 22:39:03 +00:00
Raúl Kripalani
c474aeee5e fix: Eth JSON-RPC API: return empty bloom on every block. 2022-11-18 19:49:39 +00:00
Raúl Kripalani
ec667d49f4 fix: Eth JSON-RPC API: empty EthBytes serializes to '0x'. 2022-11-18 19:01:30 +00:00
Raúl Kripalani
2ff679c679 fix test compilation. 2022-11-16 22:18:40 +00:00
Raúl Kripalani
23007b0b3b fix: do not hash log values; decode as cbor byte arrays. 2022-11-16 21:45:42 +00:00
Raúl Kripalani
d441425618 eth_getTransactionReceipt: return type & non-zero bloom if logs, rm root. 2022-11-16 21:16:27 +00:00
Raúl Kripalani
a536f533d8 Eth JSON-RPC API: make EthLog#Data a string. 2022-11-16 20:51:28 +00:00
Raúl Kripalani
1ab39a40da simplify by moving receipt constructor logic to API. 2022-11-16 14:15:29 +00:00
Raúl Kripalani
32385a97cd Eth JSON-RPC API: return logs in eth_getTransactionReceipt. 2022-11-16 13:53:27 +00:00
Ian Davis
314fb31886 Fix signature of EthSubscribe 2022-11-16 12:18:25 +00:00
Ian Davis
8134d2f05b Parse block heights as hex 2022-11-16 12:08:48 +00:00
Ian Davis
10357112e5 Add basic itests for ethereum filter api 2022-11-15 18:02:36 +00:00
Raúl Kripalani
846aea57c5 Merge branch 'feat/nv18-fevm' into feat/nv18-events 2022-11-15 12:28:25 +00:00
Shrenuj Bansal
b95d1a6323 Merge branch 'master' into sbansal/nonce-coordination-and-consensus-for-chain-nodes 2022-11-14 17:50:52 -05:00
Geoff Stuart
95af40543a Fix lotus-miner proving check 2022-11-14 15:29:50 -05:00
Raúl Kripalani
a915c703a5 Merge branch 'feat/nv18-events' into feat/nv18-ethevents 2022-11-14 11:46:08 +00:00
Ian Davis
91952f84bd Fix eth types comments 2022-11-14 10:27:06 +00:00
ychiao
8a7367f1c9
api: ethrpc: add missing fields (#9631) 
* api: ethrpc: add missing fields

* fix make gen
 2022-11-12 09:44:52 -08:00
raulk
4e7b8d555e
Merge pull request #9624 from iand/feat/nv18-rcptevent 2022-11-11 20:25:37 +00:00
Raúl Kripalani
9c5d888ec9 docs gen; also fixes lack of *cid.Cid examples elsewhere. 2022-11-11 20:20:28 +00:00
Shrenuj Bansal
2fa21ff091 Merge branch 'master' into sbansal/nonce-coordination-and-consensus-for-chain-nodes 2022-11-11 14:41:38 -05:00
vyzo
7664e0691c fix tx check and test 2022-11-11 05:49:06 +02:00
vyzo
4c76f654b9 eth api: remove bad todo link 2022-11-11 05:20:10 +02:00
vyzo
54f311d94a eth rlp: fix warts 2022-11-11 04:58:15 +02:00
vyzo
18f2a14a00 eth api: fix warts 2022-11-11 04:52:38 +02:00
Ian Davis
7383ecb0ba make gen 2022-11-10 15:04:58 +00:00
Ian Davis
69210d0917 Ethereum compatible actor event API 2022-11-10 15:02:53 +00:00
vyzo
2588b45826 fix gst builtin module refs 
it breaks the gen script in proxy_gen
 2022-11-10 04:46:13 +02:00
vyzo
6d7a33ff70 api: ethrpc: fix unit test 
and skip a non-testing test; printing doesn't constitute testing.
 2022-11-10 02:52:06 +02:00
vyzo
deea6797c5 make gen 2022-11-10 02:52:06 +02:00
vyzo
2b23372ad3 fixup merge artifacts 2022-11-10 02:51:57 +02:00
imxyb
53c7d9c18f fix:add UnmarshalJSON for EthNonce (#9589) 2022-11-09 19:47:44 +02:00
Steven Allen
49da019bd0 fix: allow EthCall.From to be nil (#9556) 2022-11-09 19:47:25 +02:00
Steven Allen
6337d594fa fix: evm: use CREATE not CREATE2 for EVM contract deployment (#9553) 
Also updates the actors to accommodate this change, and fix a bug in
looking up addresses for f4 actors.
 2022-11-09 19:47:23 +02:00
Raúl Kripalani
dc5fc9ded4 fix eth_feeHistory. 2022-11-09 19:47:07 +02:00
Raúl Kripalani
ab5f0216f6 fix message sending logic. 2022-11-09 19:47:07 +02:00
Raúl Kripalani
32617a61df fix eth_feeHistory: add missing param. 2022-11-09 19:47:06 +02:00
Raúl Kripalani
6d5ec6aff8 fix address recovery from pubkey. 2022-11-09 19:46:24 +02:00
Raúl Kripalani
9017e5de52 fix signature validation logic. 2022-11-09 19:46:24 +02:00
Kevin Li
64afdfc642 api: ethrpc: implement eth_feeHistory (#9539) 
Co-authored-by: Raúl Kripalani <raul@protocol.ai>
 2022-11-09 19:46:22 +02:00
Kevin Li
e17e92775c api: ethrpc: fix eth_getCode, eth_getTransactionReceipt (#9530) 2022-11-09 19:40:00 +02:00
Steven Allen
56b238980b feat: use f4 eth addresses wherever possible (#9532) 
Co-authored-by: Raúl Kripalani <raul@protocol.ai>
 2022-11-09 19:39:50 +02:00
Steven Allen
aa0e6c17b9 Eth TX Encoding (#9531) 2022-11-09 19:39:49 +02:00
Melanie Riise
7e7fff1dc5 Eth JSON-RPC API: Add EAM as contract creator (#9528) 2022-11-09 19:33:25 +02:00
Raúl Kripalani
98e534cc77 add support for f4 addresses in the Eth JSON-RPC and signature. 2022-11-09 19:33:23 +02:00
Raúl Kripalani
17e9e97064 Eth JSON-RPC API: support f4 addresses. 2022-11-09 19:32:43 +02:00
Steven Allen
420bd34888 fix(api): eth/id address conversion (#9511) 2022-11-09 19:30:40 +02:00
Kevin Li
e109be4c6a fix lint 2022-11-09 19:30:27 +02:00
vyzo
61de2f4a4a fix eth api issues with evm constructor changes 2022-11-09 19:30:08 +02:00
raulk
ee83dceef0 Eth JSON-RPC API: implement eth_getCode and eth_getStorageAt (#9397) 2022-11-09 19:30:06 +02:00
Kevin Li
6c7fd94aa7 api: ethrpc: implement eth_sendRawTransaction (#9334) 
Co-authored-by: Raúl Kripalani <raul@protocol.ai>
 2022-11-09 19:28:17 +02:00
Kevin Li
b53d5924a1 feat: ethrpc: implement call, estimateGas, getTransactionCount (#9306) 2022-11-09 19:25:40 +02:00
Kevin Li
e5bb5b7430 feat: ethrpc: implement EthBlock and EthTx structs (#9287) 
Co-authored-by: Raúl Kripalani <raul@protocol.ai>
 2022-11-09 19:23:49 +02:00