cerc-io/ipld-eth-server
18
0
Fork 1
Code Issues 10 Pull Requests 1 Actions Packages Projects Releases 37 Wiki Activity
273 Commits 12 Branches 52 Tags 80 MiB
59cdaa05e6
Commit Graph

110 Commits

Author SHA1 Message Date
Ian Norden
59cdaa05e6 moving some memory allocation to init function rather than within 
execution cycles and fixing test db tear down so that it drops
checked_headers table instead of individual columns (droping columns
only hides them, they remain in memory and continue to contribute to the
1600 column limit which causes us to eventually exceed it)
 2019-01-07 13:07:31 -06:00
Ian Norden
2cbe6e7a70 method piping 2018-12-21 11:14:54 -06:00
Ian Norden
456c735087 fetch headers and logs in batches across all contracts and events 2018-12-21 10:33:31 -06:00
Ian Norden
d188329661 fetch event logs in batches- all events for a contract address at a given block height are fetched together, different contract events still fetched separately 2018-12-21 10:33:31 -06:00
Ian Norden
e390a97502 finish method polling with hash or address type argument values 
collected from watched events; 'MissingMethodsCheckedEventsIntersection'
method to find headers which have been checked for each event of
interest but methods have not yet been polled at that header; tests for
new features; travis ci go version 1.9 -> 1.11 ; consolidate omniWatcher and lightOmniWatcher into single command with light as default
 2018-12-21 10:33:31 -06:00
Ian Norden
0a59f06cac begin work on: Add checked_headers column for methods that are polled so taht we don’t duplicate; Add batching of method polling so that we arent generating a rediculously large account address list before using it to poll methods (or persist the list in pg?); User passed ABI and other ways to get ABI; Add ability to collect []byte and hashes from events and use them in method polling same manner as addresses; Event filter addrs => only those event’s addresses/hashes are used for polling; Option to persist seen address/hash/bytes lists into pg; Only generate lists of addresses, []byte, or hashes if a method will use them later 2018-12-21 10:33:31 -06:00
Ian Norden
e02b33547d finishing porting omni watcher to work with light sync; split into full, light, 
and shared directories and refactor as much into shared; finish
lightSync omni watcher tests
 2018-11-30 13:33:48 -06:00
Ian Norden
975f13b969 reorganizing omni directory and beginning light watcher work 2018-11-29 20:33:21 -06:00
Ian Norden
817bd76713 refactoring event converter and repository so that event logs are 
persisted as they are converted, preventing build up in memory, and finishing  method polling (for full sync)
 2018-11-29 20:33:21 -06:00
Ian Norden
390a60f7f6 port over lightSync updates from maker repo 2018-11-29 20:33:21 -06:00
Ian Norden
417b18ec6a Edits to address PR issues; change license from apache to AGPL; and work 
towards generic method polling and reposito;y; config settings to
filterevents/methods by account address; refactoring some stuff out of
repo and into converter; remove fetcher and instead call
blockchain's FetchContractData directly; finishing tests
 2018-11-15 12:32:52 -06:00
Ian Norden
b459cf35ed beginning work on method polling; first need to generate list of token holder address in a completely generic/contratc agnostic fashion. Created address retriever that can iterate through any given contract's watched events, finding the inputs/arguments with address type, and generate a list from those values. Edit: Contract objects now cache every event emitted address as its event logs are transformed into the repo to grow a list of contract associated addresses as we go 2018-11-04 21:37:31 -06:00
Ian Norden
e9dbd771e5 tests and fixes for fetcher, parser, retriever, converter, and repository; update cmd and watcher 2018-11-04 01:49:11 -05:00
Ian Norden
8ce75fe5ad Generic watcher that takes a contract address, grabs the contract abi and starting block number, creates custom event filters, and extracts and transforms event data into postgres. Can configure to look at only a subset of events through CLI flag. Building but needs testing. 2018-11-03 14:00:25 -05:00
Ian Norden
9b41000b88 Moved fetcher to generic directory (methods have to remain public since it is in seperate package now), added FetchHash method, created ERC20 and generic getters which call the fetcher with specific contract methods (GetTotalSupply, GetBalance, GetAllowance for ERC20 getter, and GetOwner, GetStoppedStatus, GetStringName, GetHashName, GetStringSymbol, GetHashSymbol, and GetDecimals for generic getter). Getter tests cover all but GetBalance and GetAllowance, and also cover all of the Fetcher methods- but with only nil methodArgs. GetAllowance and GetBalance tests are not working against infura and these are the only contract method calls with arguments passed in so I suspect this might be where the issue lies. Have tested GetBalance using previous version of FetchContractData without the variadic input to the Pack method and it fails with the same error so I don’t think it is due to those changes. 2018-11-03 13:49:23 -05:00
Ian Norden
ccd21f73c9 adjusted FetchContractData to work with variable number of contract method arguments, changed fetcher's FetchSupply method to more general FetchBigInt method and added FetchBool and FetchAddress methods 2018-11-03 13:49:23 -05:00
Rob Mulholand
ba071ef13f Consolidate test doubles 
- Migrate various mocks of core namespaces to shared version in `fakes` pkg
- Err on the side of making test doubles less sophisticated
- Don't pull over mocks of namespaces that are only used in example code
 2018-11-03 13:49:23 -05:00
Rob Mulholand
5fe6394406 Add tests for pkg/geth/blockchain 
- inject dependencies instead of initializing them in the constructor
 2018-11-03 13:49:23 -05:00
Rob Mulholand
05186634bd Add light sync command 
- Only syncs block headers (excludes block bodies, transactions, receipts, and logs)
- Modifies validation window to include the most recent block
- Isolates validation window to the variable defined in the cmd directory (blocks
  have a separate variable defined in the block_repository for determining when
  to set a block as final)
 2018-11-03 13:49:23 -05:00
Maxim Krasilnikov
7b507667f8 Added user and password to db config 2018-11-03 13:49:23 -05:00
Rob Mulholand
3e39ccb9bb Fix missing block numbers lookup 
- Previously, a block was treated as missing if there was a number in
the given range for which there existing a block that did not match
the node's fingerprint. This meant that in a case where we have a block
that does not match the node's fingerprint and also one that does match,
the block would be treated as missing. This led to errors being thrown
when attempting to add a block that already exists.
- These changes treat a block as missing only if we do not already have
a block that matches the number and node fingerprint.
 2018-06-08 11:26:25 -05:00
Rob Mulholand
d5c2ab33fc Sync only missing blocks on cold import 
= Add eth_node_fingerprint to block that can be imitated by both hot and cold imports
- Only sync missing blocks (blocks that are missing or don't share a fingerprint) on cold import
- Set block is_final status after import
 2018-05-07 16:38:04 -05:00
Rob Mulholand
5a5e08bd13 Get head block number 
- Allows us to refuse to sync past current head
- Creates the opportunity to add a flag for syncing all blocks
 2018-05-04 11:03:50 -05:00
Rob Mulholand
5f6bf32ec1 Add cold import script 2018-05-04 10:54:00 -05:00
Rob Mulholand
462f94d84a Associate receipts with blocks instead of transactions 
- Allows us to fetch receipts by block and persist even when
associated transaction is unknown. (The associated transaction
can still be derived from the tx_hash column at query time, but
is an expensive operation to require for inserts).
 2018-05-04 10:52:28 -05:00
Matt K
8a9395819c Get transactions (#45) 
* Make transactions requests in parallel

* Update transaction error handling
 2018-03-27 16:06:12 -05:00
Matt Krump
88210e436a Remove test and travis deps on private dev network 2018-03-22 10:15:18 -05:00
Matt Krump
1b4e57a5b4 Rename node table and node column 2018-03-21 13:24:13 -05:00
Matt Krump
a11fb709b2 Update watcher to use blockchain instead of contract data fetcher 2018-03-12 16:19:06 -05:00
Matt Krump
76dad443ec Update FetchContractData to allow optional args 2018-03-12 14:21:34 -05:00
Matt Krump
5a2bb04670 Remove unused contract methods 2018-03-12 14:21:34 -05:00
Matt K
5a652190d9 Allow Parity as ingest node (#36) 
* Upgrade go-ethereum to v1.8

* Add Node Info for parity nodes

* Upgrade start_private_blockchain to use v1.8
 2018-03-07 15:29:21 -06:00
Matt Krump
06f78e0083 Handle events 
- Adds interfaces for developers to build handlers that update data in
response to log events
- Resolves #29
 2018-03-05 10:01:50 -06:00
Matt K
ed907535e3 Separate DB access into several repos (#28) 
* Separate files for InMemory

* Start using separate repos for collaborating objects

* Before Updating schema

* Separate various repos
 2018-02-12 10:54:05 -06:00
Matt K
605b0a96ae Add graphql server (#27) 
* Add graphql server

* Update Makefile

* Update log_filters constraint

* Add GetLogFilter to repo

* Update travis (use Makefile, go fmt, go vet)

* Add logFilter schema and resolvers

* Add GetWatchedEvent to watched_events_repo

* Add watchedEventLog schema and resolvers
 2018-02-08 10:12:08 -06:00
Matt K
d5852654bb Update table columns (#26) 
* Update block table names

* Update transaction table names
 2018-02-02 16:12:14 -06:00
Matt K
aea9c7b5e2 Separate repositories (#25) 
* Separate Repository into multiple Repositories

* Use struct scan for transactions

* Use struct scan for blocks

* Remove unused block repo methods

* Update naming

* Rename / Cleanup repository related fields
 2018-02-02 15:53:16 -06:00
Matt K
3863bcb614 Events (#24) 
* Add watched events repo
 2018-02-02 13:58:59 -06:00
Matt Krump
aac2239097 Merge old private repo into vulcanize 2018-01-25 18:08:26 -06:00
Matt K
c00b8a5a98 Add Filters (#133) 
* Add LogFilter struct

* Add log_filters table

* Add view for events watching

* Add cmd line "add_filter" to mimic eventual endpoint

* Allow multiple filters in config
 2018-01-23 12:43:35 -06:00
Matt K
3f06c7374b Update transaction value field (#132) 
* Add test for converting big.Int

* Use string as internal representation of transaction value
 2018-01-16 14:25:33 -06:00
Matt Krump
6583ce72b8 Remove log_uc 
* Logs now are attached to receipt, so removing block + index unique
constraint
 2018-01-16 09:44:46 -06:00
Matt Krump
82c39a2c1f Add fk constraint on logs 2018-01-16 08:58:11 -06:00
Matt Krump
9ee13e715d Add topics type 2018-01-15 14:49:29 -06:00
Matt Krump
50f00b80c1 Lowercase log address 2018-01-15 14:46:48 -06:00
Matt Krump
431be46005 Add store logs with receipts 2018-01-15 14:46:32 -06:00
Matt K
a9bea4f492 Allow for multiple Geth nodes (#128) 2018-01-10 15:54:36 -06:00
Matt K
70cfa20c68 Watch contact updates (#127) 
* Downcase all arguments for contact watching

* ABI retrieval from test networks
 2018-01-08 15:59:47 -06:00
Matt K
14e1fc4213 Remove admin api dependency (#126) 2018-01-08 14:19:42 -06:00
Matt K
54c4f0c2fe Add back infura related tests (#123) 2018-01-08 11:41:01 -06:00