Commit Graph

19371 Commits

Author SHA1 Message Date
Łukasz Magiera
12656b05fe feat: shed: Add a tool for removing sector data 2023-02-16 16:47:13 +01:00
Phi
27d4d02fe5 Update enhancement and feature templates
Update enhancement and feature_request templates:

- Removing unnecessary checklist options.
   - These have been move to the Github New Issue UI to reduce text overhead: 
- Remove deprecated components options
- Add FVM/FEVM component
2023-02-16 13:11:10 +01:00
Phi
d3e715594b chore: github: Update bug_report template
Update bug_report template:

- Removing unnecessary checklist options.
   - Some of these has been move to the Github New Issue UI to reduce overhead
- Remove deprecated components options
- Add FVM/FEVM component
2023-02-16 10:59:45 +01:00
Łukasz Magiera
285dfed338
Merge pull request from filecoin-project/feat/faster-sectors-list
sp cli: make sectors list much faster
2023-02-15 19:59:17 +01:00
Łukasz Magiera
6028d7ba18 sectors list: Explain http mode 2023-02-15 19:41:46 +01:00
Łukasz Magiera
2dd2edc1d3
Merge pull request from filecoin-project/asr/fix-shed
fix: shed: default to LOTUS_PATH for repo
2023-02-15 18:57:15 +01:00
Aayush
4f259d8644 Revert "Standardize path variable"
This reverts commit a9c1caa188.
2023-02-15 11:09:24 -05:00
Aayush Rajasekaran
29a82cdf0f
Merge pull request from filecoin-project/asr/merge-release-into-master
chore: merge release branch into master
2023-02-15 09:47:54 -05:00
Jiaying Wang
6a90d32885
chore: add v1.20.0-rc1 changelog ()
* add v1.20.0-rc1 changelog



Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>
2023-02-14 20:52:32 -05:00
Aayush
75c744a438 Merge branch 'release/v1.20.0' 2023-02-14 20:29:14 -05:00
Aayush Rajasekaran
6b56b7710d
Merge pull request from filecoin-project/jen/v1200rc1prep
build: release: v1.20.0-rc1 prep
2023-02-14 20:02:52 -05:00
jennijuju
39c709f7b3 update butterfly artifects 2023-02-14 19:24:35 -05:00
Aayush
3f063f7cd7 fix: StorageKey is now fixed-size array 2023-02-14 19:04:43 -05:00
jennijuju
45cbbe6c7a update to gst 10-rc3 2023-02-14 18:48:41 -05:00
jennijuju
30fe2c7c69 - add calibration nv18 upgade epoch
- bump version to v1.20.0-rc1
2023-02-14 18:48:23 -05:00
Hector Sanjuan
7ecc00090e range-export: make gen 2023-02-14 21:08:10 +01:00
Hector Sanjuan
a2475bca94 range-export: write files to lotus repo path 2023-02-14 21:08:10 +01:00
Hector Sanjuan
be1614a042 range-export: check serverside that head > tail 2023-02-14 21:08:10 +01:00
Hector Sanjuan
ec01036871 Ranged export: remove non-internal ranged export method
Additionally, per feedback:

* Set "admin" permissions
* Remove from v0api
2023-02-14 21:08:10 +01:00
Hector Sanjuan
1bb698619c Ranged-export: Remove CachingBlockstore
The improvements in the range-export code lead to avoid reading most blocks
twice, as well as to allowing some blocks to be written to disk multiple times.

The cache hit-rate went down from being close to 50% to a maximum of 12% at
the very end of the export. The reason is that most CIDs are never read twice
since they are correctly tracked in the CID set.

These numbers do not support the maintenance of the CachingBlockstore
code. Additional testing shows that removing it has similar memory-usage
behaviour and about 5 minute-faster execution (around 10%).

Less code to maintain and less options to mess up with.
2023-02-14 21:08:10 +01:00
Hector Sanjuan
fa93c23813 Chain ranged export: rework and address current shortcomings
This commit moderately refactors the ranged export code. It addresses several
problems:
  * Code does not finish cleanly and things hang on ctrl-c
  * Same block is read multiple times in a row (artificially increasing cached
    blockstore metrics to 50%)
  * It is unclear whether there are additional races (a single worker quits
    when reaching height 0)
  * CARs produced have duplicated blocks (~400k for an 80M-blocks CAR or
    so). Some blocks appear up to 5 times.
  * Using pointers for tasks where it is not necessary.

The changes:

  * Use a FIFO instead of stack: simpler implementation as its own type. This
has not proven to be much more memory-friendly, but it has not made things
worse either.
  * We avoid a probably not small amount of allocations by not using
    unnecessary pointers.
  * Fix duplicated blocks by atomically checking+adding to CID set.
  * Context-termination now works correctly. Worker lifetime is correctly tracked and all channels
  are closed, avoiding any memory leaks and deadlocks.
  * We ensure all work is finished before finishing, something that might have
  been broken in some edge cases previously. In practice, we would not have
  seen this except perhaps in very early snapshots close to genesis.

Initial testing shows the code is currently about 5% faster. Resulting
snapshots do not have duplicates so they are a bit smaller. We have manually
verified that no CID is lost versus previous results, with both old and recent
snapshots.
2023-02-14 21:08:10 +01:00
Ian Davis
35730637dc fix: tracer: emit raw peer ids for compatibility with libp2p tracer 2023-02-14 16:17:03 +00:00
Łukasz Magiera
7150243cda
feat: ethrpc: Support newPendingTransactions in eth_subscribe () 2023-02-14 16:15:42 +00:00
Jiaying Wang
86b4aeef1a
Merge pull request from filecoin-project/steb/fix-msg-modify
fix: stmgr: copy the message before modifying it
2023-02-14 10:07:24 -05:00
frrist
21efd481d8 First efficient ranged-export implementation by @frisst
This first commit contains the first and second implementation stabs (after
primary review by @hsanjuan), using a stack for task buffering.

Known issues: ctrl-c (context cancellation) results in the export code getting
deadlocked. Duplicate blocks in exports. Duplicate block reads from store.

Original commit messages:

works

works against mainnet and calibnet

feat: add internal export api method

- will hopfully make things faster by not streaming the export over the json rpc api

polish: better file nameing

fix: potential race in marking cids as seen

chore: improve logging

feat: front export with cache

fix: give hector a good channel buffer on this shit

docsgen
2023-02-14 15:41:10 +01:00
raulk
11ac325d1a
Revert "fix: stmgr: check message validity before invoking vm" ()
This reverts commit 2bbd1fbbb6.
2023-02-14 14:06:33 +00:00
Phi-rjan
3879c80e6f
Add link to discussion and FIPs in New Issue
In our current issue flow, we ask a lot checklist questions :

`This is not a question or a support request. If you have any lotus related questions, please ask in the lotus forum.`
`This is not a new feature or an enhancement to the Filecoin protocol. If it is, please open an FIP issue. *`

This information is better displayed in the `New Issue` chooser, to guide users to the relevant place for posting. This file will add two new entries in the issue chooser:
- A entry that links to the Lotus Github Discussion pager for Lotus questions or troubleshooting
- A entry that links to the FIPs repo for Filecoin protocol feature or enhancements
2023-02-14 11:13:55 +01:00
Jiaying Wang
b1e1d2d205
Merge pull request from filecoin-project/steb/update-ffi
chore: ffi: update the FFI
2023-02-13 22:28:16 -05:00
Steven Allen
abeb4fadc4
fix: itest: EthFeeHistory flake () 2023-02-14 02:42:44 +00:00
Steven Allen
2ed92c222d chore: ffi: update the FFI
This:

- Improves error handling when the FFI is passed invalid messages.
- Updates the FVM.
2023-02-13 18:42:29 -08:00
Aayush Rajasekaran
3c4a83916b
Merge pull request from filecoin-project/steb/call-check-message-validity
fix: stmgr: check message validity before invoking vm
2023-02-13 21:29:19 -05:00
snissn
9f2f9154d2
itest for block.difficulty () 2023-02-13 18:29:11 -08:00
ychiao
0d92c746bd
Eth JSON-RPC: populate reward in eth_feeHistory ()
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
Co-authored-by: Raúl Kripalani <raul@protocol.ai>
2023-02-14 01:02:37 +00:00
Steven Allen
a16c54051c
fix: eth: rename yParity to v in transaction return values ()
fixes https://github.com/filecoin-project/ref-fvm/issues/1662
2023-02-13 23:29:13 +00:00
Steven Allen
2bbd1fbbb6 fix: stmgr: check message validity before invoking vm
Otherwise we may, e.g., try to estimate gas on a message to an f4
address before the nv18 migration.

I'm _not_ checking the "prior messages" here as this is just a sanity
check.
2023-02-13 11:01:42 -08:00
Steven Allen
f427c2b566
fix: eth: correctly convert filecoin message <-> eth txn ()
1. We do allow deploying with empty initcode.
2. Make sure that the encoded "code" is non-empty, if specified.

Basically, this makes everything consistent (and it's how I specified it
in the FIP).
2023-02-13 10:33:59 -08:00
Steven Allen
77fbc60cb8 fix: stmgr: copy the message before modifying it 2023-02-13 10:06:59 -08:00
Łukasz Magiera
61f29a84b5
Merge pull request from filecoin-project/asr/merge-release-into-master
chore: merge release/v1.20.0 into master
2023-02-13 15:25:23 +01:00
Łukasz Magiera
03fd703db5
Merge pull request from filecoin-project/mikers/itestsProxy
test: itest: fevm proxy contracts
2023-02-13 14:03:56 +01:00
Mike Seiler
3177f63a17 add test for transparent delegate proxy from openzeppelin 2023-02-13 13:57:18 +01:00
Łukasz Magiera
d108565ea2 Merge remote-tracking branch 'origin/master' into asr/merge-release-into-master 2023-02-13 13:40:32 +01:00
Łukasz Magiera
7486227c27 Merge remote-tracking branch 'origin/release/v1.20.0' into asr/merge-release-into-master 2023-02-13 13:14:09 +01:00
Łukasz Magiera
7b959ab39c
Merge pull request from filecoin-project/steb/fix-eth-get-code-none
fix: eth: correctly handle ethGetCode ege-cases
2023-02-13 13:13:00 +01:00
Łukasz Magiera
a652ffcb62 make gen 2023-02-13 13:09:09 +01:00
Łukasz Magiera
4d20011b78
Merge pull request from filecoin-project/jen/bumpmaster
build: bump version to v1.21.0-dev
2023-02-13 13:05:03 +01:00
Łukasz Magiera
98482a2795
Merge pull request from filecoin-project/jen/ipldcp
backport: deps: update ipld prime
2023-02-13 13:03:56 +01:00
Łukasz Magiera
e6b0ee7e4a
Merge pull request from filecoin-project/jen/ipld
dep: ipld: update ipld prime to v0.20.0
2023-02-13 13:03:44 +01:00
Raúl Kripalani
8ff55c027e Merge branch 'release/v1.20.0' into asr/merge-release-into-master 2023-02-13 11:45:09 +00:00
Jennifer Wang
948a963f15 update ipld prime 2023-02-13 00:06:30 -05:00
Jennifer Wang
e73a038fe8 bump version to v1.21.0-dev 2023-02-13 00:00:07 -05:00