Ian Norden
b1bb646ad5
goimports + streamSubscribe command for raw access to the seed node data
2019-12-02 13:24:51 -06:00
Ian Norden
8ccdfd4835
fix streamFilters issue
2019-12-02 13:24:51 -06:00
Ian Norden
1dc46640cf
request handler and response screener for seed node api service
2019-12-02 13:24:51 -06:00
Ian Norden
34393ffb3f
fixes for issues uncovered in integration
2019-12-02 13:24:51 -06:00
Ian Norden
b83c0371d9
pub sub service and api
2019-12-02 13:24:50 -06:00
Ian Norden
5ebe2243d8
goimports -w
2019-12-02 13:24:50 -06:00
Ian Norden
671a324294
update to work with updated state diffing code
2019-12-02 13:24:50 -06:00
Ian Norden
3108957e5f
ipfs fetcher
2019-12-02 13:24:50 -06:00
Ian Norden
15e044403d
work on mocks and unit tests
2019-12-02 13:24:50 -06:00
Ian Norden
d702cb720c
update dependencies to work with update eth-block-extractor
2019-12-02 13:24:49 -06:00
Ian Norden
31a9017c4f
index all cids in a payload in a single atomic tx; misc fixes; comment additions
2019-12-02 13:24:46 -06:00
Ian Norden
1921c06b02
repository - repository for indexing CIDs
2019-12-02 13:24:46 -06:00
Ian Norden
1fba4d0858
publisher - interface for publishing IPLD objects
2019-12-02 13:24:46 -06:00
Ian Norden
d248a1050e
syncer - interface for syncing with a statediff subscription
2019-12-02 13:24:46 -06:00
Ian Norden
a31c7e9ced
payload converter - interface for converting state diff payload into payload for ipld publishing
2019-12-02 13:24:46 -06:00
Ian Norden
35719b5180
indexer - top level interface for syncAndPublish
2019-12-02 13:24:46 -06:00
Ian Norden
c6c070005b
subscribe method for the rpc client
2019-12-02 13:24:46 -06:00
Ian Norden
bc59aa4ed6
readme additions and vendor updates
2019-12-02 13:24:46 -06:00
Ian Norden
db0f024088
review fixes
2019-12-02 11:26:44 -06:00
Ian Norden
a834e55b9f
command edits; external pkg for finding min deployment block; gofmt
2019-12-02 11:22:18 -06:00
Ian Norden
563832422c
add err logging to eth/node
2019-12-02 11:07:43 -06:00
Rob Mulholand
4a98b7942f
Don't return unexported types
2019-10-31 13:42:19 -05:00
Rob Mulholand
e1236b4072
Prefer all caps for initialisms and acronyms
2019-10-31 13:42:19 -05:00
Rob Mulholand
eba868ff00
Drop else when if statement includes return
2019-10-31 13:33:39 -05:00
Rob Mulholand
0ded7bcd62
Prefer fmt.Errorf(s) to errors.New(fmt.Sprintf(s))
2019-10-31 13:33:39 -05:00
Rob Mulholand
234fff83c5
Use testing.toml and vulcanize_testing in tests
...
- remove legacy references to infura.toml and vulcanize_private
2019-10-31 13:10:25 -05:00
Rob Mulholand
e252229b8a
Add constraint to prevent duplicate headers
...
- Disallow inserts of headers with the same number, hash, and node
fingerprint, since it will enable duplicate log fetching for the
same header
2019-10-28 14:57:13 -05:00
Ian Norden
a2d249ca9d
review fixes
2019-10-28 11:40:32 -05:00
Ian Norden
4fbde836d4
log sql.ErrNoRows which I suspect is what is leading to the flaky test
2019-10-28 09:37:21 -05:00
Ian Norden
65808998b3
goimports -w; golinting, remove some unused code
2019-10-28 09:37:21 -05:00
Ian Norden
11b5efbfe3
fix for issue #146 ; mark header checked for contract if it doesnt have
...
any logs at that header but other contracts do; test
2019-10-28 09:34:42 -05:00
Edvard Hübinette
3fff2896aa
Rename geth to eth, signifying client independence ( #161 )
2019-10-28 12:30:24 +01:00
Edvard Hübinette
f7c4a6736d
VDB-919 Generalise converter ( #152 )
...
* Generalise transformer stack to use InsertionModel
* Add tests for event repository
* Restrict accepted values in InsertionModel
* Add call to repository.SetDB
* Improve error propagation/clarity on GetABI()
* Remove maker references in example
* Please golint
* refactor rollback error handling in repository
* Cleaner errors in repository, refactor tests
2019-10-28 11:48:31 +01:00
Rob Mulholand
2800e6df36
Enable contractWatcher without prior headerSync
...
- Previous setup would fail if there were no headers in the db. This
makes sense because we need headers that haven't been checked for
logs to exist so that we can fetch logs for those headers. But it
also prevents us from kicking off the headerSync and contractWatcher
processes concurrently. These changes enable kicking off both
processes at the same time with the idea that we will have unchecked
headers upon transformer execution.
2019-10-04 16:00:13 -05:00
Elizabeth
f6ab9382b2
Address repo updates ( #134 )
...
* Factor out get or create address into one sql string
* Factor out getChecksumAddress method in address repo
* Update address repo methods to not need a receiver
* Move address repository to libraries/shared
2019-10-03 11:17:08 -05:00
Rob Mulholand
4505382590
Increase logging in contract watcher
...
- Focus on header mode
- Add context to errors, trace guard clauses, warn on non-returned
errors
- Give errors distinct names so compiler will recognize if unchecked
- Remove redundant type declarations/fix typos
2019-10-02 15:14:42 -05:00
Rob Mulholand
2ff88de859
Extract helper for converting hex to keccak256 hash
...
- Also prefer crypto.Keccak256Hash(x) to common.BytesToHash(crypto.Keccak256(x))
2019-09-25 16:36:08 -05:00
Elizabeth Engelman
8e166f2a56
Update after rebasing with staging
2019-09-25 16:32:31 -05:00
Elizabeth Engelman
ee244ac6f5
Apply go fmt
2019-09-25 16:32:27 -05:00
Elizabeth Engelman
34f8e16c11
Add a Subscribe method to rpc client interface
2019-09-25 16:32:27 -05:00
Elizabeth Engelman
36533f7c3f
Update vendor directory and make necessary code changes
...
Fixes for new geth version
2019-09-25 16:32:27 -05:00
20ce0ab852
Revert "option to write the plugin .go file and run the plugin migrations without building the .so file so that it can be done from the shell instead to afford complete error messages"
...
This reverts commit 9bb2f27a69
.
2019-09-24 23:54:25 +00:00
Ian Norden
9bb2f27a69
option to write the plugin .go file and run the plugin migrations without building the .so file so that it can be done from the shell instead to afford complete error messages
2019-09-24 18:44:04 -05:00
Rob Mulholand
c21c069b69
Replace magic numbers with variables
2019-09-23 16:13:45 -05:00
Rob Mulholand
eea0b0bdca
Return missing headers after starting block number
...
- Previously, we required that the first missing header match the
configured starting block number. This helps guarantee that we have
the necessary data for method polling in memory, but prevents the
process from moving forward if restarted after the starting block
has already been checked.
2019-09-23 16:13:45 -05:00
Rob Mulholand
37e2673a8d
Do nothing on duplicate log insert
...
- Prevents duplicate key constraint violation from blocking the process
from moving forward on restart.
- If header_id, log_idx, and tx_idx are the same, we can safely do
nothing since it's definitely the same log - a reorg would cause the
original header to be replaced with a new ID.
2019-09-23 16:02:07 -05:00
Rob Mulholand
3466a51b4d
Correctly parse uint8 values in contract watcher
...
- calling string(uint8) => insert errors on numeric columns
2019-09-20 21:37:59 -05:00
Rob Mulholand
749db40b96
Parse bytes32 vals in Contract Watcher
2019-09-20 13:48:43 -05:00
Rob Mulholand
f83e996ab8
Update comments in checked headers repository
2019-09-18 21:39:34 -05:00
Rob Mulholand
2b798e00e0
Cap random fake timestamp
2019-09-13 11:20:13 -05:00
Rob Mulholand
3f9b034c4c
Rename checked_logs => watched_logs
...
- We're logging that a given log has been included in any fetch calls
for checked headers, rather than that we have already checked for
that log
2019-09-10 21:22:14 -05:00
Rob Mulholand
13d503b851
Distinguish between missing and unchecked headers
...
- Missing == not in DB
- Unchecked == logs haven't been fetched
2019-09-10 14:39:45 -05:00
Rob Mulholand
ce91b0d9e6
Simplify checked header repository tests
...
- Use assertions instead of comments to document expectations
- Also randomize the fake timestamp in test data
2019-09-10 13:47:43 -05:00
Rob Mulholand
b9f3b9f946
Reference header sync logs address via foreign key
2019-08-28 10:41:34 -05:00
Rob Mulholand
5ac76eee74
Make check_count a column on public.headers
...
- Don't need to maintain it on public.checked_headers if we're not
adding additional columns to that table
2019-08-28 09:50:17 -05:00
Rob Mulholand
666ea1c325
Update checked headers for new transformers
...
- If a header was marked as checked before a transformer was added to
the watcher, mark all headers since the new transformer's starting
block number as unchecked.
2019-08-28 09:25:14 -05:00
Rob Mulholand
63dabbb051
Extract and delegate logs concurrently
2019-08-28 09:25:13 -05:00
Rob Mulholand
d496dad33c
Decouple log extraction from transformer delegation
...
- limit missing headers results set to 100 so that extraction doesn't
excessively block delegation
- wrap checked headers functions in repository struct
- move storage repository to factory, to correspond with event
repository path
- remove unused files
- reformat sql
- remove line breaks in imports
2019-08-28 09:25:13 -05:00
Rob Mulholand
cb819fa9a6
Write event logs to database before transforming
...
- enables decoupling event extraction/persistence from transformation
- modifies event transformer, converter, and log chunker to accept
payload that includes internal log database ID with log data
- remove alias for transformer pkg as shared_t
- remove unused mock watcher repository
2019-08-28 09:13:44 -05:00
Rob Mulholand
3693ed905f
Rename logs to full sync logs
...
- Enable creating new table for logs used in event watching based on
header sync
2019-08-28 09:11:06 -05:00
Elizabeth Engelman
edc0bdf668
Address more PR feedback
2019-08-23 10:11:37 -05:00
Elizabeth Engelman
1b3786338f
Address PR feedback
2019-08-23 10:11:37 -05:00
Elizabeth Engelman
58c23c6632
Update Copyrights
2019-08-23 10:11:37 -05:00
Elizabeth Engelman
38f6847ff2
Create a HeaderSyncReceiptRepository
2019-08-23 10:11:37 -05:00
Elizabeth Engelman
7e38764618
Rename ReceiptRepository -> FullSyncReceiptRepository
2019-08-23 10:11:37 -05:00
Elizabeth Engelman
4e40e892d2
Updating header_sync_receipts to have FK reference to addresses
2019-08-23 10:11:37 -05:00
Elizabeth Engelman
bcd6d14fcd
Updating full_sync_receipts to have FK reference to addresses
2019-08-23 10:11:37 -05:00
Elizabeth Engelman
258035833b
Add GetAddressById
2019-08-23 10:11:37 -05:00
Elizabeth Engelman
4b61c87b55
Get or create address record in a transaction
2019-08-23 10:11:37 -05:00
Elizabeth Engelman
1373fe83a1
Pass a db into GetOrCreateAddress
2019-08-23 10:11:37 -05:00
Elizabeth Engelman
b13eefbce7
Add AddressRepository
2019-08-23 10:11:37 -05:00
Rob Mulholand
987abd4b2e
Update Geth to 1.9.0
2019-07-23 15:26:18 -05:00
Andrew J Yao
ee77fc6521
Remove injection of SubCommand for logs
2019-07-23 13:11:20 -07:00
Andrew J Yao
92d153b010
Add subCommand name to log fields
2019-07-23 08:09:57 -07:00
Edvard Hübinette
2c092e8d04
[VDB-751 VDB-754] Bugfix null pointer panic and improve logging ( #119 )
...
* VDB-751 VDB-754 Bugfix null pointer panic and improve logging
* Fix typo
2019-07-18 09:21:40 +02:00
Rob Mulholand
0f95267e84
Use goose library for running plugin migrations
...
- Removes dependency on goose binary existing in plugin execution
environment
2019-06-27 13:58:58 -05:00
Rob Mulholand
f0bcf930f4
Separate tmp migrations dir from home in join
...
- enable separator between home and migrations dir
2019-06-27 13:58:58 -05:00
Rob Mulholand
0558792282
Log error on failure to setup plugin migrations
...
- include specific error
2019-06-27 13:58:58 -05:00
Andrew J Yao
11a65c525c
Adds ability to set log level from toml or CLI args
2019-06-27 11:14:11 -07:00
Elizabeth Engelman
fa03716cb2
Address small PR comments
2019-05-10 11:54:15 -05:00
Elizabeth Engelman
e1a0d894a2
Update blockchain method GetHeaderByNumbers -> GetHeadersByNumbers
2019-05-08 13:42:43 -05:00
Elizabeth Engelman
ba81766f6c
Small spelling fix
2019-05-08 13:42:43 -05:00
Rob Mulholand
76ab914bdc
Add license
2019-05-01 12:32:39 -05:00
Rob Mulholand
2d684c5aec
Extract storage diff fetching behind an interface
...
- Replaces directly reading from a CSV
- Simplifies testing
- Should hopefully make it easier to plug in other sources for storage
diffs (e.g. differently formatted CSVs, JSON RPC, etc)
2019-05-01 12:30:36 -05:00
Gabe Laughlin
a344432156
(VDB-560) Rename lightSync to headerSync
2019-05-01 12:12:55 -05:00
Rob Mulholand
3fd6269b78
Don't lookup transactions if no log events
...
- Prevents EOF error on transactions lookup
2019-04-18 09:53:18 -05:00
ana0
964fc8bd22
Merge branch 'staging' into databaseConfig
2019-04-11 14:19:57 -04:00
anon
bd12149594
Database config
2019-04-10 17:56:31 -04:00
Gabe Laughlin
8903b6fb2d
Add timestamp customization for fake header
2019-04-09 17:08:15 -05:00
Ian Norden
9befc76fc6
remove LIMIT 100; method to check continuity of headers returned but doesn't require
...
the set to start at a specific block number; config for
account_transformer; review fixes; update schema
2019-04-05 17:21:59 -05:00
Ian Norden
c1940c3e58
full_sync and light_sync receipt tables and light receipt repository methods and test
2019-04-05 16:36:48 -05:00
Ian Norden
04ad7b7696
Merge pull request #47 from vulcanize/track_uncle_rewards
...
Track uncle rewards + direct balance polling
2019-04-04 15:55:40 -05:00
Ian Norden
5dcf534b2c
review fixes
2019-04-04 15:21:39 -05:00
Ian Norden
72c73c25b7
update vendor dep; constrain to revision of geth with UnpackIntoMap; updates to match
2019-04-03 00:14:02 -05:00
Ian Norden
197f98c93d
going ahead and indexing the entire uncle blocks (one of the issues open on public); finish tests
2019-04-02 14:32:30 -05:00
Ian Norden
fd407825c1
review fixes
2019-04-02 10:10:17 -05:00
Ian Norden
185f4c0e93
adjust block/uncle reward tests and add methods to pkg/geth blockchain and ethclient for direct fetching of eth balances
2019-04-02 10:10:17 -05:00
Ian Norden
9030cff2bd
keep track of who is receiving uncle rewards
2019-04-02 10:10:17 -05:00
Rob Mulholand
f5b32a11b0
Improve transaction syncing test coverage
2019-03-28 14:31:17 -05:00
Rob Mulholand
81dfd12665
Address PR comments
2019-03-28 14:31:17 -05:00
Rob Mulholand
54d46638a8
updates for light sync transactions
2019-03-28 14:31:17 -05:00
Rob Mulholand
79e011aad2
Add light sync transactions table
...
- Foreign key to header instead of block
- Can use same Transaction struct for both
2019-03-28 14:31:17 -05:00
Rob Mulholand
d93006321b
Rename transactions to full_sync_transactions
...
- Table has foreign key to blocks
- Add transaction RLP and transaction index to table
- Enables other tables with standalone transactions or transactions
associated with other data structures (e.g. headers)
2019-03-28 14:31:17 -05:00
Ian Norden
1aa849bcb4
PR fixes; remove all infura token references and setup travis to use encrypted env
...
variable; rest of the ethjson_rpc dependent tests extracted to integration_test
2019-03-21 18:36:51 -05:00
Ian Norden
d3e172d9ab
remove references to infura token except for in infura.toml so that tests will continue to pass on CI
2019-03-21 18:36:49 -05:00
Ian Norden
37fc038605
fixes for review comments
2019-03-21 18:36:49 -05:00
Ian Norden
37e581c7ec
PR#72 from public repo- https://github.com/vulcanize/vulcanizedb/pull/72- also needed to finish pluggin in ENS record transformer
2019-03-21 18:33:56 -05:00
Ian Norden
24879a85aa
mv pkg/omni pkg/contract_watcher
2019-03-21 18:33:56 -05:00
Ian Norden
073378de6d
update © to 2019
2019-03-21 18:33:56 -05:00
Ian Norden
e4e092f542
changes to plugin and commands to accomodate changes
2019-03-21 18:33:56 -05:00
Ian Norden
8174ce4aee
refactor omni code; rename omniWatcher => contractWatcher; use config instead of excessive number of CLI flags
2019-03-21 18:30:48 -05:00
Ian Norden
ff55e3ba7a
TransformerInitializer => EventTransformerInitializer reorganization
2019-03-14 11:59:39 -05:00
Ian Norden
9f8c50e3ab
tests for migration path ordering and errors
2019-03-13 11:42:30 -05:00
Ian Norden
f37c458992
the part that actually runs the migrations in order
2019-03-13 11:42:29 -05:00
Ian Norden
84aa0a7eba
plugin migration order specified in config
2019-03-13 11:42:29 -05:00
Rob Mulholand
1414779d52
Use *sqlx.Tx instead of *sql.Tx
...
- requires using db.Beginx() instead of db.Begin()
- enables calling tx.Get()
2019-03-06 12:43:39 -06:00
Rob Mulholand
7ed4dcd1fa
Remove Tic logic
2019-03-06 12:43:39 -06:00
Rob Mulholand
b6bef49318
Add tests for plugin helpers
...
- Prevents code analysis from showing functions as unused
- Also address other code analysis issues
2019-03-06 12:43:39 -06:00
Rob Mulholand
a806cb2818
Extract slow tests from omni full transformer
2019-03-06 12:43:06 -06:00
Rob Mulholand
34c96654ce
Extract slow tests from omni light transformer
2019-03-06 12:43:06 -06:00
Ian Norden
8bd397e2eb
Merge pull request #29 from vulcanize/fix
...
Fix for Travis issue at https://github.com/vulcanize/mcd_transformers/pull/6
2019-03-06 11:27:43 -06:00
Ian Norden
e1a317c3f6
different errors; readme edit
2019-03-06 00:14:37 -06:00
Ian Norden
5fd9a59027
fix for issue with Travis building plugin from plugin repo
2019-03-05 08:57:54 -06:00
Edvard Hübinette
55b6ad9381
Optimise missing block numbers query ( #23 )
2019-03-05 11:52:38 +01:00
Ian Norden
83f7daf37d
compose and execute as separate commands; update README and add git checkout vendor/github.com/ethereum/go-ethereum/accounts/abi to make build
2019-02-28 18:15:16 -06:00
Ian Norden
708425c4d6
rebase; extract factories and the mocks they are dependent on to
...
libraries/shared; adjust omni test_helpers to drop and recreate
checked_headers table to avoid reaching 1600 column limit after repeated
tests (dropping columns doesn't actually remove them from contributing
to that limit)
2019-02-25 01:34:38 -06:00
Ian Norden
17f4d3dfa5
move maker plugin generator test to mcd_transformers
2019-02-24 21:53:04 -06:00
Ian Norden
3d34a9e7c9
remove maker migrations and convert back to timestamps and fix bug in
...
composeAndExecute command
2019-02-24 21:52:23 -06:00
Ian Norden
b449193b16
addressing review comments; still need to reorg migrations and add helper tests
2019-02-24 21:38:48 -06:00
Ian Norden
2868cf2f73
goose changes; update plugin readme; use rel paths for transformers in config; more commenting in pkg/plugin; fix PluginWriter to work with different transformer types; reopen generator tests
2019-02-24 21:38:48 -06:00
Ian Norden
03a7379617
work on porting storage watcher; watcher type is defined for
...
transformers in the config, storage transformers are exported from
plugin like event transformers
2019-02-24 21:38:47 -06:00
Ian Norden
b51dcb55de
comment out plugin test til mcd_transformers is updated correctly
2019-02-24 21:38:47 -06:00
Ian Norden
a322372713
refactoring plugin generating code
2019-02-24 21:38:47 -06:00
Ian Norden
decc2a3caf
remove pkg/transformers; extract shared files needed to
...
libraries/shared; work on automated db migration management
2019-02-24 21:38:47 -06:00
Ian Norden
55fa9b8364
auto management of dependencies temporarily needed to build plugin and transformer
...
migrations; new test showing it is working with external transformers runs
properly when migrations were manually performed but still need
to test automated migration management
2019-02-24 15:23:35 -06:00
Ian Norden
6c2d895023
composeAndExecute command that loads and executes over arbitrary transformer set exported from a go plugin generated according to config file; test for plugin generation, loading, and execution; work on plugins README
2019-02-24 15:23:35 -06:00
Edvard Hübinette
fa4abe4e5c
Remove spammy and unnecessary error log ( #21 )
2019-02-22 12:56:06 +01:00
Rob Mulholand
745d0e8eda
Use storage value metadata getter in pit repository tests
2019-02-21 15:25:16 -06:00
Rob Mulholand
a5121913f4
Error on missing keys in pit repository
...
- If ilk missing for ilk-related value, fail fast
2019-02-21 15:03:22 -06:00
Rob Mulholand
3d9341ea1d
Reference ilks foreign key from Cat storage repository
2019-02-21 14:33:59 -06:00
Rob Mulholand
dbdd16d3a2
(VDB-380) Create separate table for ilks
...
- reference ilk by foreign key every place it's used
2019-02-21 11:23:00 -06:00
Rob Mulholand
11dd641a84
(VDB-354) Queue unrecognized storage diffs
...
- If we recognize a storage diff as coming from a watched contract but
don't recognize the key, queue it for retrying later (after we've seen
an event that might help us recognize the key)
- Remove unused errs and args
- Panic on unrecognized types (should not happen)
2019-02-20 15:22:39 -06:00
Edvard
fb47a747f6
Fix confusion between nflip and flip
2019-02-20 14:37:55 +01:00
Edvard
56ca45509f
DRY up storage diff repo tests
2019-02-20 14:37:55 +01:00
Edvard
5746de7c97
Add tests for Cat storage diff transformer
...
Update schema after rebase migrations
Fix small issues from review
2019-02-20 14:37:46 +01:00
Edvard
cd6611d2ec
Add Cat storage transformer
2019-02-20 14:37:46 +01:00
Edvard Hübinette
53789c82ab
Merge pull request #12 from vulcanize/VDB-392-fix-io-error-propagation
...
VDB-392 Fix IO error propagation
2019-02-20 13:58:00 +01:00
Edvard
1a2f46b308
Fix small issues from review
2019-02-20 13:10:55 +01:00
Ian Norden
2ecc524224
Merge pull request #9 from vulcanize/VDB322-Vow-storage-diff
...
Vdb322 vow storage diff
2019-02-15 11:04:22 -06:00
Taka Goto
64c7ae1bb1
uppercase Woe and Ash
2019-02-15 06:15:47 -06:00
Rob Mulholand
df521eea7f
Merge pull request #11 from vulcanize/storage-transformers-readme
...
Add storage transformers readme
2019-02-14 14:24:01 -06:00
Edvard
13eacf2081
Update tests for error propagation
2019-02-14 16:04:12 +01:00
Edvard
6cd4e5ea95
Improve I/O error propagation
2019-02-14 16:03:57 +01:00
Rob Mulholand
0378068467
add example contract and instructions for watching its storage
2019-02-13 20:34:30 -06:00
Taka Goto
a8322c7d26
add woe to storage diff vow contract
2019-02-12 21:50:52 -06:00
Rob Mulholand
ee113668a5
Add first draft of storage transformers readme
...
- Explain process of creating a new storage transformer
2019-02-12 15:58:28 -06:00
Rob Mulholand
ac9375ae47
Discard logs created during tests
...
- Make sure output is being set to discard for logrus to reduce clutter in the console
2019-02-12 11:18:39 -06:00
Taka Goto
4fe2ff4104
repository methods:
2019-02-11 16:34:02 -06:00
Taka Goto
c2c21325b5
vow contract storage diffing excluding sin
2019-02-11 15:28:54 -06:00
Edvard Hübinette
f73623789e
Merge pull request #6 from 8thlight/VDB-337-Dockerize-vDB
...
VDB-337 Dockerize vDB
2019-02-11 12:37:56 +01:00
Edvard
9b986c4c14
Nuke OpenRC from docker image, update readme's
2019-02-11 11:27:33 +01:00
Edvard
553fdbfa09
Default config starting block to 0
2019-02-11 11:27:33 +01:00
Edvard
e6e66d58dd
Add container service files and docker README
2019-02-11 11:26:12 +01:00
Rob Mulholand
c05d3b7836
(VDB-280) Consume vat contract storage diffs
2019-02-08 13:36:30 -06:00
Rob Mulholand
dba4e48a3e
Use map for key in storage value metadata
...
- Enables inserting multiple keys for one storage row (e.g. ilk and urn)
2019-02-08 11:41:17 -06:00
Rob Mulholand
09abcb841a
Persist Urns in raw bytes32 format
...
- Enables simple usage in contract storage lookups
2019-02-08 11:41:17 -06:00
Takayuki Goto
0a024d429d
* update transformer to able to recheck headers ( #4 )
...
* update transformer to able to recheck headers
* put cap on rechecking header
* integration test for recheck headers
* use enum for recheck headers param; make recheck cap configurable
* update integration tests with new test config
* update omni pkg with new recheck header column type
* update migration with new migration tool and final tweaks needed to accommodate changes in omni pkg
2019-02-08 10:35:46 -06:00
Rob Mulholand
867f92c431
(VDB-298) Consume Pit contract storage diffs
...
- Continuously parse storage diffs CSV data to read Pit contract state
- Convert ilks in database to raw bytes32 value for use in generating
storage keys dynamically
- Persist storage diffs with block number and hash for validation
2019-02-06 10:31:46 -06:00
Edvard Hübinette
29f2537130
Merge pull request #7 from vulcanize/VDB-347-Squelch-lightSync
...
VDB-347 Squelch lightSync in stdout
2019-02-01 11:02:48 +01:00
Edvard
a1fa21cbec
Remove test based on window buffer stuff
2019-01-31 16:39:29 +01:00
Edvard
8eea1b00ed
Redirect validation window status to logrus instead of stdout
2019-01-31 15:55:28 +01:00
Ian Norden
8749c94994
update license to AGPL-3.0
2019-01-24 14:41:30 -06:00
Ian Norden
ab371a2024
Merge remote-tracking branch 'maker/staging' into staging
2019-01-24 10:22:22 -06:00
Ian Norden
1f95eb6443
Merge branch 'staging' into pub_merge
2019-01-24 09:47:12 -06:00
Taka Goto
c5d513e96d
fix index out of range on lightSync; hopefully the new integration tests will catch them next time :)
2019-01-23 15:25:44 -06:00
Ian Norden
00c2052da3
for some reason an error is thrown when the ipc string is declared in integration_tests_suite_test.go but not when moved to a different file in the integration_tests package; I think this is due to switching to go1.11
2019-01-23 15:23:23 -06:00
Ian Norden
dc87f39889
change omniwatcher database test helpers so that it doesn't interfere with downstream maker tests
2019-01-23 14:55:20 -06:00
Konstantin Zolotarev
65cb6e9092
VDB-205 Fix external configuration values ( #121 )
...
* Updated loading config from /Users/konstantinzolotarev/.vulcanizedb for addreses
* Updated contract addresses loading from env variables.
* Implement getter chain for addresses, configs and transformers
Solves the issue with declaring stuff as vars when the environment file hasn't been loaded yet. Instead, this data is fetched dynamically when the root commands are called.
* Update test data to use constant Kovan addresses
* Decouple integration tests from config file IO
* Add ABI to environment file, convert getter chains for related values
* Decouple tests suites from ABI config file IO
* Add startingBlock to environment and implement getter chain
* Exit when called without config file
* Add missing deployment blocks to staging.toml
* Remove comment
2019-01-23 19:44:09 +01:00
Ian Norden
46b898f690
merge public vulcanizedb/master
2019-01-23 00:37:26 -06:00
Taka Goto
3595771825
use batchElem from rpc client
2019-01-21 14:52:37 -06:00
Takayuki Goto
54d771cb06
Merge branch 'staging' into batch-historical-headers
2019-01-16 15:56:36 -06:00
Taka Goto
24cf0661b0
lowercase address on map to get transformers
2019-01-16 14:24:00 -06:00
Taka Goto
07da9d720b
fix get headers when less than 100 fetched
2019-01-16 10:16:06 -06:00
Taka Goto
074140b9ac
iterate for loop by headers instead of blocknumbers
2019-01-15 16:34:20 -06:00
Ian Norden
aa4e698240
move interface_getter.go from ens_watcher to vDB pkging- used to construct a custom ABI based on results from calls to supportsInterface; also moved fetcher.go from examples to pkg
2019-01-15 15:49:18 -06:00
Takayuki Goto
f1ba21b505
Merge branch 'staging' into batch-historical-headers
2019-01-13 21:29:58 -06:00
yaoandrew
e2b864baa8
Add rollback if GetTicInTx fails and move GetTicInTx to shared/repository.go
2019-01-11 10:07:00 -08:00
Andrew J Yao
c8aa3dc275
Replace standard log with logrus import
2019-01-11 10:07:00 -08:00
yaoandrew
bc8b7a101c
refactor error handling in vat, cat, vow
2019-01-11 10:07:00 -08:00
yaoandrew
525fada3d7
refactor error handling in frob, pit, price, tend
2019-01-11 10:07:00 -08:00
yaoandrew
657aaccff3
refactor error handling in drip, flap, flop
2019-01-11 10:07:00 -08:00
yaoandrew
05351a8ef4
refactor error handling in dent
2019-01-11 10:07:00 -08:00
yaoandrew
08ac7f18a5
refactor error handling in flip_deal
2019-01-11 10:07:00 -08:00
yaoandrew
f655997fde
refactor error handling in pit_vow
2019-01-11 10:07:00 -08:00
yaoandrew
843d30c88e
refactor error handling in flip
2019-01-11 10:07:00 -08:00
yaoandrew
513513e93b
refactor error handling in chop_lump
2019-01-11 10:07:00 -08:00
yaoandrew
78b302eab6
refactor error handling in bite transformer
2019-01-11 10:07:00 -08:00
Edvard Hübinette
ebca338b1e
VDB-302 Sleep when no missing blocks in lightSync ( #129 )
...
* Sleep when no missing blocks in lightSync
* Fix tests and error propagation
* Correct geth.log to logrus
2019-01-11 10:58:41 +01:00
Takayuki Goto
ac430ddf54
Merge branch 'staging' into batch-historical-headers
2019-01-08 19:06:53 -06:00
Taka Goto
aa99c861a8
batch headers by 100s
2019-01-08 16:53:26 -06:00
Ian Norden
59cdaa05e6
moving some memory allocation to init function rather than within
...
execution cycles and fixing test db tear down so that it drops
checked_headers table instead of individual columns (droping columns
only hides them, they remain in memory and continue to contribute to the
1600 column limit which causes us to eventually exceed it)
2019-01-07 13:07:31 -06:00
Edvard
2292c94e99
Merge branch 'staging' into VDB-125-aggregate-log-fetching
2019-01-07 15:25:37 +01:00
Edvard
50cc0c2459
Update tests for watcher and repository
2019-01-07 15:20:05 +01:00
Edvard
833dde62cf
Remove injection of fetcher and repository to watcher
2019-01-07 15:19:31 +01:00
Ian Norden
2cbe6e7a70
method piping
2018-12-21 11:14:54 -06:00
Ian Norden
456c735087
fetch headers and logs in batches across all contracts and events
2018-12-21 10:33:31 -06:00
Ian Norden
d188329661
fetch event logs in batches- all events for a contract address at a given block height are fetched together, different contract events still fetched separately
2018-12-21 10:33:31 -06:00
Ian Norden
e390a97502
finish method polling with hash or address type argument values
...
collected from watched events; 'MissingMethodsCheckedEventsIntersection'
method to find headers which have been checked for each event of
interest but methods have not yet been polled at that header; tests for
new features; travis ci go version 1.9 -> 1.11 ; consolidate omniWatcher and lightOmniWatcher into single command with light as default
2018-12-21 10:33:31 -06:00
Ian Norden
0a59f06cac
begin work on: Add checked_headers column for methods that are polled so taht we don’t duplicate; Add batching of method polling so that we arent generating a rediculously large account address list before using it to poll methods (or persist the list in pg?); User passed ABI and other ways to get ABI; Add ability to collect []byte and hashes from events and use them in method polling same manner as addresses; Event filter addrs => only those event’s addresses/hashes are used for polling; Option to persist seen address/hash/bytes lists into pg; Only generate lists of addresses, []byte, or hashes if a method will use them later
2018-12-21 10:33:31 -06:00
Edvard
e54699c039
Remove todo on chunker non-match (it's basically a filter)
2018-12-14 15:22:46 +01:00
Edvard Hübinette
a3e8633aff
VDB-104 improve lightsync ui ( #125 )
...
* Return PopulateMissingHeaders early if the sync is at the head of the chain
* Squelch logging if no blocks to sync
* Fix broken test
* Refactor repository tests
2018-12-14 12:11:48 +01:00
Edvard
7cf694f5e7
Update transformer documentation
2018-12-14 10:30:08 +01:00
Edvard
35f7f719e6
Swap strings.Builder for bytes.Buffer for go 1.9 support
2018-12-13 16:08:57 +01:00
Edvard
129964f3bc
Add tests for now shared MissingHeaders
2018-12-13 13:58:16 +01:00
Edvard
e5db740ae4
Move MockTransformer to test_data
2018-12-13 13:03:51 +01:00
Edvard
d30fcfed80
Use new config getter on shared.Transformer <3
2018-12-13 12:39:57 +01:00
Edvard
882a6dd7a2
Update tests with new adding of transformers
2018-12-12 15:41:44 +01:00
Edvard
dc43547612
Improve process of adding transformers
2018-12-12 15:41:29 +01:00
Edvard
a86a2ec5c6
Improve checked header column detection
2018-12-12 12:13:23 +01:00
Edvard
4e089c363e
Fixes after merging staging
2018-12-11 15:19:27 +01:00
Edvard
ff47e2e06e
Merge branch 'staging' into VDB-125-aggregate-log-fetching
2018-12-11 15:03:21 +01:00
Edvard
9000c6e9f9
Add test for watcher log delegation
2018-12-11 15:02:32 +01:00
Edvard
5d4239de05
Finish watcher and watcher tests (+fmt)
2018-12-11 13:52:02 +01:00
Edvard
d93817f346
Interface-ify shared repository
2018-12-11 11:35:13 +01:00
Edvard
83593cb5f2
Cleanup fetcher and add test todos
2018-12-10 21:12:55 +01:00
Edvard
2d81861ae5
Add chunker tests
2018-12-10 21:11:25 +01:00
Edvard
bc9fd17793
Fix remaining integration tests
2018-12-10 17:12:51 +01:00
Edvard
3cb8fe60d4
Fix persistHeader not returning proper header ID
2018-12-10 16:58:24 +01:00
Edvard
637623b752
Fix test suites!
2018-12-10 15:40:45 +01:00
Rob Mulholand
670741616e
Use logrus for logging in populating blocks
...
- Removes logging from test output
2018-12-07 11:24:06 -06:00
Rob Mulholand
2a518ad518
(VDB-254) Convert numerical values for Pit.file ilk
...
- convert data to ray when what == "spot" and wad when what == "line"
2018-12-07 11:24:06 -06:00
Rob Mulholand
087af252c9
(VDB-253) Update numerical values for Pit.file debt ceiling
...
- convert Line to wad
2018-12-07 11:24:06 -06:00
Rob Mulholand
4a5765b20d
(VDB-249) Update numerical values for Vat.fold
...
- convert rate to ray
2018-12-07 11:24:06 -06:00
Edvard
e1ba7ac5b4
WIP update tests for aggregate fetching
2018-12-07 18:10:36 +01:00
Edvard
47c75d055b
Update repository tests for absence of MissingHeaders
2018-12-04 17:53:46 +01:00
Edvard
9789648cc2
Delegate log chunks to respective transformers
2018-12-04 17:40:39 +01:00
Edvard
8bebcdc064
Change transformer.Execute to single header
2018-12-04 17:05:34 +01:00
Edvard
38c745e8c3
Refactor fetching out from repositories to log_fetcher and watcher
2018-12-04 16:04:13 +01:00
Ian Norden
e02b33547d
finishing porting omni watcher to work with light sync; split into full, light,
...
and shared directories and refactor as much into shared; finish
lightSync omni watcher tests
2018-11-30 13:33:48 -06:00
Edvard
c26736dc9e
WIP continue on aggregate fetching architecture
2018-11-30 17:28:52 +01:00
Ian Norden
975f13b969
reorganizing omni directory and beginning light watcher work
2018-11-29 20:33:21 -06:00
Ian Norden
817bd76713
refactoring event converter and repository so that event logs are
...
persisted as they are converted, preventing build up in memory, and finishing method polling (for full sync)
2018-11-29 20:33:21 -06:00
Ian Norden
390a60f7f6
port over lightSync updates from maker repo
2018-11-29 20:33:21 -06:00
Elizabeth
8ce901d1af
Convert drip_file_ilk tax field to ray ( #117 )
2018-11-29 14:10:12 -06:00
Edvard
45a087f5d1
WIP: First steps in refactoring to aggregate fetching
2018-11-28 17:00:04 +01:00
Edvard
423fdf01b5
Add log_chunker
2018-11-28 16:59:06 +01:00
Edvard
f7ecab9dc2
Fix typo
2018-11-28 15:39:46 +01:00
Taka Goto
e8be96a4e7
use logrus for logging
2018-11-21 10:14:11 -06:00
Elizabeth
128d20c9bf
Vdb 269- fetch logs by hash ( #122 )
...
* Upgrade geth from 1.8.15 to 1.8.18
* Update vat_tune to use shared repository methods
* Query blockchain by block hash instead of block number range
* Remove hash validation from repositories
* Fix vow flog integration test
* Update README Travis build sticker
* Update constants formatting per go fmt
* Update EthPublicKeyParser.ParsePublicKey to use discv5.PubkeyID method
* Address PR comments
2018-11-21 09:30:00 -06:00
Elizabeth
b19e77e419
Add integration test for Vow.flog ( #114 )
2018-11-15 16:44:49 -06:00
Ian Norden
417b18ec6a
Edits to address PR issues; change license from apache to AGPL; and work
...
towards generic method polling and reposito;y; config settings to
filterevents/methods by account address; refactoring some stuff out of
repo and into converter; remove fetcher and instead call
blockchain's FetchContractData directly; finishing tests
2018-11-15 12:32:52 -06:00
Edvard
7cd512ae91
Remove PFSF label
2018-11-15 12:14:59 -06:00
Rob Mulholand
854236fd99
(VDB-267) Remove pit file stability fee
...
- Method is no longer present on the Pit contract
2018-11-15 12:14:59 -06:00
Edvard Hübinette
40408e8300
VDB-120 include tic in dent and tend ( #118 )
...
* Add constants for TTL and tic for fakeHeader
* Add tic to tend transformer
* Update tests for tend
* Fix string conversion bug in fakes
* Fix tend integration tests after staging rebase
* Add tic to dent transformer
* Update dent tests
* Change integration tests to use hardcoded block timestamp
2018-11-15 12:57:29 +01:00
Edvard
8f4648a9e3
Add support for never-ending transformer
2018-11-14 10:11:16 -06:00
Rob Mulholand
82fd73ba3f
(VDB-68) Verify log block hash matches header block hash
...
- Delete header on conflict to prompt data refresh (cascade deletes all
data associated with that block)
- Derive header hash from rpc payload rather than computing it from data
(prevents hash mismatch between blockchain and cache)
2018-11-13 14:51:39 -06:00
Elizabeth
6a43cd4287
Vdb 207 the rest of the repos ( #111 )
...
* Use shared repository test examples for frob
* Use shared repository methods for frob
* Use shared repository test examples for flop kick
* Use shared repository methods for flop kick
* Use shared repository test examples for flip kick
* Use shared repository methods for flip kick
* Use shared repository test examples for bite
* Use shared repository methods for bite
* Alphabetize check headers constants
2018-11-12 10:50:51 -06:00
Edvard Hübinette
4f295b9aec
VDB-207 refactor init move split toll tune flog ( #110 )
...
* Use shared repo tests for vat_init
* Use shared methods in vat_init repository
* Use shared repo tests for vat_move
* Use shared methods in vat_move repository
* Use shared repo tests for vat_slip
* Use shared methods in vat_slip repository
* Use shared repo tests for vat_toll
* Use shared methods in vat_toll repository
* Use shared methods in vat_tune repository
* Use shared repo tests for vow_flog (+small namespace fix)
* Use shared methods in vow_flog repository
* Go vet + tiny oopsie
2018-11-12 14:17:58 +01:00
Edvard Hübinette
571f300392
Add rendering of available transformer names to continuousLogSync help ( #109 )
2018-11-12 14:01:07 +01:00
Elizabeth
672269effa
VDB 207 refactor more repo tests ( #107 )
...
* Use shared repo tests for vat flux
* Use shared repo methods for vat flux
* Use shared repo test examples for vat fold
* Use shared repository methods for vat fold
* Use shared repo test examples for vat grab
* Use shared repo methods for vat grab
* Use shared repo test examples for vat heal
* Use shared repo methods for vat heal
2018-11-09 14:54:22 -06:00
Elizabeth
c1e10f09fb
Vdb 67 convert numbers to ray and wad ( #105 )
...
* Add method to convert values to ray or wad units
* Convert data to ray or wad for cat_file_chop_lump
* Use shared convert functions in price feed conversion
* Pull common ray/wad values into vars
* Fix after rebase with staging
2018-11-09 14:53:20 -06:00
Rob Mulholand
0b58efb56a
Extract common repository methods
...
- Inject column names to reduce duplication across common behavior
- Extract checked headers column names to constants
2018-11-08 13:08:59 -06:00
Rob Mulholand
8c168ed332
Extract constants package
...
- Reduce clutter in the shared package
2018-11-08 13:08:59 -06:00
Rob Mulholand
8c5192f82f
(VDB-210) Refactor tend repository tests
2018-11-08 13:08:59 -06:00
Rob Mulholand
1c1a9c699a
(VDB-209) Refactor price feed repository tests
2018-11-08 13:08:59 -06:00
Rob Mulholand
af8bbe5ce9
(VDB-208) Refactor pit file repository tests
2018-11-08 13:08:59 -06:00
Rob Mulholand
a1c4d70a77
(VDB-99) Transform Flap deal events
...
- Add flapper address to deal transformer config
2018-11-07 10:52:55 -06:00
Rob Mulholand
9cc9d4d3b6
(VDB-95) Transform Flop dent events
...
- Add flopper address to dent config
2018-11-07 10:52:55 -06:00
Rob Mulholand
52bbedd837
(VDB-98) Transform Flap tend events
...
- Add Flapper contract address to tend transformer config
2018-11-07 10:52:55 -06:00
Rob Mulholand
b2a78f1da4
Remove unused transformer config struct
2018-11-07 10:52:55 -06:00
Rob Mulholand
cb141f7cb7
(VDB-204) Add transformer for Flap kick
2018-11-07 10:52:55 -06:00
Taka Goto
627ae288bd
use self hosted parity kovan for staging
2018-11-06 12:58:45 -06:00
Takayuki Goto
7afd978cf0
Merge pull request #100 from 8thlight/update-integration-tests
...
integration tests hits self hosted remote parity kovan node:
2018-11-06 12:08:48 -06:00
Elizabeth
0ab5ea1b25
Refactor more repo tests ( #101 )
...
* Use shared repository tests for deal
* Small updates to repo tests
* Use shared repository test examples for dent
* Use shared repository test examples for drip_drip
* Use shared repository test examples for drip_file_ilk
* Use shared repository test examples for drip_file_repo
* Use shared repository test examples for drip_file_vow
* Update formatting
2018-11-06 10:51:26 -06:00
Elizabeth
a4abf3781e
Refactor repository tests ( #99 )
...
* Add shared test examples for repositories
- Create
- MissingHeaders
- MarkHeaderChecked
* Use shared repository tests for cat_file_chop_lump
* Use shared repository test for cat_file_flip
* Use shared repository test for cat_file_pit_vow
* Updates to shared repository tests
* Use shared repository tests for deal
* Small updates to repo tests
2018-11-06 10:30:31 -06:00
Elizabeth
bfe4531d7e
Common transformer tests ( #97 )
...
* Move chop lump transformer test to a generic log_note_transformer_test
* Add test cases to transformer test
- cat file flip
- cat file pit vow
- deal
- dent
* Move transformer tests over to transformer factor test file
- drip drip
- drip file ilk/repo/vow
- pit file deb ceiling/ilk/stability fee
- price feeds
* Move LogNoteTransformer test into factory package and remove log specific transformer tests
* Move Transformer test into factory package and remove log event specific transformer tests
* Respond to PR comments
2018-11-06 10:30:11 -06:00
Taka Goto
2cc9826be9
create integration toml file
2018-11-06 09:25:08 -06:00
Ian Norden
b459cf35ed
beginning work on method polling; first need to generate list of token holder address in a completely generic/contratc agnostic fashion. Created address retriever that can iterate through any given contract's watched events, finding the inputs/arguments with address type, and generate a list from those values. Edit: Contract objects now cache every event emitted address as its event logs are transformed into the repo to grow a list of contract associated addresses as we go
2018-11-04 21:37:31 -06:00
Ian Norden
e9dbd771e5
tests and fixes for fetcher, parser, retriever, converter, and repository; update cmd and watcher
2018-11-04 01:49:11 -05:00
Ian Norden
8ce75fe5ad
Generic watcher that takes a contract address, grabs the contract abi and starting block number, creates custom event filters, and extracts and transforms event data into postgres. Can configure to look at only a subset of events through CLI flag. Building but needs testing.
2018-11-03 14:00:25 -05:00
Ian Norden
9b41000b88
Moved fetcher to generic directory (methods have to remain public since it is in seperate package now), added FetchHash method, created ERC20 and generic getters which call the fetcher with specific contract methods (GetTotalSupply, GetBalance, GetAllowance for ERC20 getter, and GetOwner, GetStoppedStatus, GetStringName, GetHashName, GetStringSymbol, GetHashSymbol, and GetDecimals for generic getter). Getter tests cover all but GetBalance and GetAllowance, and also cover all of the Fetcher methods- but with only nil methodArgs. GetAllowance and GetBalance tests are not working against infura and these are the only contract method calls with arguments passed in so I suspect this might be where the issue lies. Have tested GetBalance using previous version of FetchContractData without the variadic input to the Pack method and it fails with the same error so I don’t think it is due to those changes.
2018-11-03 13:49:23 -05:00
Ian Norden
ccd21f73c9
adjusted FetchContractData to work with variable number of contract method arguments, changed fetcher's FetchSupply method to more general FetchBigInt method and added FetchBool and FetchAddress methods
2018-11-03 13:49:23 -05:00
Rob Mulholand
ba071ef13f
Consolidate test doubles
...
- Migrate various mocks of core namespaces to shared version in `fakes` pkg
- Err on the side of making test doubles less sophisticated
- Don't pull over mocks of namespaces that are only used in example code
2018-11-03 13:49:23 -05:00
Rob Mulholand
5fe6394406
Add tests for pkg/geth/blockchain
...
- inject dependencies instead of initializing them in the constructor
2018-11-03 13:49:23 -05:00
Rob Mulholand
05186634bd
Add light sync command
...
- Only syncs block headers (excludes block bodies, transactions, receipts, and logs)
- Modifies validation window to include the most recent block
- Isolates validation window to the variable defined in the cmd directory (blocks
have a separate variable defined in the block_repository for determining when
to set a block as final)
2018-11-03 13:49:23 -05:00
Maxim Krasilnikov
7b507667f8
Added user and password to db config
2018-11-03 13:49:23 -05:00
Takayuki Goto
0d325afbfb
flop kick transformer factory ( #98 )
...
* flop kick transformer factory
* add tests for wrong model/entity
2018-11-02 09:13:10 -05:00
Elizabeth
981393d0a7
Use transformer factory for FlipKick ( #96 )
2018-11-02 09:12:51 -05:00
Taka Goto
3b722c132e
integration tests hits self hosted parity kovan node:
2018-11-01 10:24:03 -05:00
Elizabeth
ab4e904713
Frob transformer factory ( #95 )
...
* Updates to previous transformers
* Use transformer factory for Frob
* Move bite integration test to integrationt est pkg
2018-10-29 15:08:00 -05:00
Rob Mulholand
78b8173b37
Improve test coverage for event method signature lookups
2018-10-26 15:35:18 -05:00
Rob Mulholand
602802e5f4
Don't test order of DB results
...
- reduce flakiness by enabling tests to pass regardless of result order
2018-10-26 15:35:18 -05:00
Rob Mulholand
ed624330f6
Rename Flog pack to Vow flog
...
- enables differentiation if another contract implements flog
- also deletes flog-specific mocks
2018-10-26 15:35:18 -05:00
Rob Mulholand
1b680d2a4b
Use transformer factory for Drip drip
2018-10-26 15:35:18 -05:00
Edvard Hübinette
f3a82e6ba1
VDB-184 Refactor Dent ( #91 )
...
* Factorise dent transformer
* Update dent transformer tests
* Add additional tests to dent repository test
* Update dent to use LogNoteTransformer factory
2018-10-26 13:56:49 -05:00
Rob Mulholand
d1ee501f12
Use transformer factory for Bite ( #89 )
...
* Use transformer factory for Bite
- introduces separate transformer factory for non-LogNote events
- converter includes `ToEntities` for events defined via ABI
* Updates after rebasing with staging
2018-10-26 13:26:10 -05:00
Rob Mulholand
0e5b23993a
Discard log output from Vat heal during tests
2018-10-26 12:02:56 -05:00
Rob Mulholand
45a52a59f9
Add repository tests for Deal
2018-10-26 12:02:56 -05:00
Edvard
0e1675354f
Update tests for deal transformer
2018-10-26 12:02:56 -05:00
Edvard
35a8009410
Refactor deal transformer to use factory
2018-10-26 12:02:56 -05:00
Elizabeth
d522d33531
Update vat heal to use transformer factory ( #88 )
...
* Update vat heal to use transformer factory
2018-10-26 11:40:24 -05:00
Rob Mulholand
5de949f4c5
Add cat file repository tests
2018-10-26 11:36:02 -05:00
Edvard
00bb48bafb
Fix whitespace errors
2018-10-26 11:36:02 -05:00
Edvard
e6e2ed3308
Update cat file transformer tests
2018-10-26 11:36:02 -05:00
Edvard
28ab5f92ab
Factorise cat_file transformers
2018-10-26 11:36:02 -05:00
Rob Mulholand
47a3c35938
Use transformer factory for Price Feed events
2018-10-26 11:25:16 -05:00
Elizabeth
bd428e752a
Update vat grab to use transformer factory ( #85 )
...
* Update vat grab to use transformer factory
2018-10-26 11:02:39 -05:00
Rob Mulholand
745b3a460e
Add test for updating existing checked header row
2018-10-26 10:45:49 -05:00
Rob Mulholand
e37447ae9a
Use transformer factory for Vat fold
2018-10-26 10:45:49 -05:00
Takayuki Goto
0e7f316568
flip kick integration test ( #83 )
...
* flip kick integration test
* use timestamp instead to compare date
* fix after rebase staging
2018-10-26 10:26:22 -05:00
Rob Mulholand
3ec5b4afab
Remove unused param in integration test helper
2018-10-25 10:02:06 -05:00
Rob Mulholand
8831586fba
Extract shared mock converter and repository
2018-10-25 10:02:06 -05:00
Rob Mulholand
f5b9a21a8c
Use transformer factory for Vat slip
2018-10-25 10:02:06 -05:00
Rob Mulholand
1e7ee08af5
Use transformer factory for Vat toll
2018-10-25 10:02:06 -05:00
Rob Mulholand
6d662a6ce7
Add test for can't create Vat tune of wrong type
2018-10-25 10:02:06 -05:00
Rob Mulholand
a0cae99437
Use transformer factory for Vat flux
2018-10-25 10:02:06 -05:00
Edvard Hübinette
be249437c2
VDB-117: Add Vow.flog
transformer ( #80 )
...
* Add DB migrations for flog
* Add Vow addresses and constants
* Add err logging to ABI signature getter, test for flog sig
* Add flog transformer
* Add tests+data for flog transformer
* Prepend DB tables and columns with vow_
* Panic on failure to get method sig
2018-10-25 12:19:46 +02:00
Rob Mulholand
78d746bb5b
Use transformer factory for Vat tune
2018-10-24 15:49:19 -05:00
Elizabeth
5682ee988b
Add log index to various events ( #78 )
...
* Add log_index field to flip kick
* Add log index to flop kick
* Add repo tests for Cat.file
* Add log_index to vat_flux
* Add log_index to vat_slip
* Add log_index to vat_toll
* Add log_index to vat_tune
* Add test to repos for updating checked headers
2018-10-23 16:33:04 -05:00
Takayuki Goto
28f8861f81
Merge pull request #71 from 8thlight/VDB-150-vat-flux-integration-test
...
integration test for vat flux
2018-10-23 16:01:42 -05:00
Rob Mulholand
42d60c3cbd
Add log index to Vat move
2018-10-23 15:25:21 -05:00
Rob Mulholand
4ac896c246
Add log index to Vat init
2018-10-23 15:25:21 -05:00
Rob Mulholand
702f90367a
Add log index to Vat heal
2018-10-23 15:25:21 -05:00
Rob Mulholand
650bbf5874
Add log index to Vat grab
2018-10-23 15:25:21 -05:00
Rob Mulholand
1e74c73591
Add log index to Vat fold
2018-10-23 15:25:21 -05:00
Rob Mulholand
db52955669
Add log index to Tend
2018-10-23 15:25:21 -05:00
Rob Mulholand
e0253cce55
Add log index to Price Feed events
2018-10-23 15:25:21 -05:00
Rob Mulholand
a0a86871b5
Add log index to Frob
2018-10-23 15:25:21 -05:00
Rob Mulholand
b3d00fe3c5
Add log index to Pit file events
2018-10-23 15:25:21 -05:00
Edvard
a0deb5eb62
Add and use transformer labels in shared constants
2018-10-23 11:23:31 +02:00
Edvard
f55449d0e5
Replace GinkgoRandomSeed with proper randomisation
2018-10-23 10:47:01 +02:00
Edvard
794a7e5f99
Nuke some leftover converter interfaces
2018-10-23 10:47:01 +02:00
Edvard
6d9547a888
Fix invalidated test for tend repository
2018-10-23 10:47:01 +02:00
Edvard
7ee3329baf
Privatise DB repository field and add transformer test
2018-10-23 10:47:01 +02:00
Edvard
cb74cbe990
Add cast error checking to refactored transformers
2018-10-23 10:47:01 +02:00
Edvard
b7f8432e65
Fix git whitespace warnings
2018-10-23 10:47:00 +02:00
Edvard
bb204b6b14
Remove unnecessary type washing in factory transformer
2018-10-23 10:45:48 +02:00
Edvard
e7e511764e
Implement skip to next header on empty matchingLogs
2018-10-23 10:45:48 +02:00
Edvard
33f1f62c11
Implement SingleTransformerConfig
2018-10-23 10:45:48 +02:00
Edvard
7d3bf1f0f6
Fixes after rebase
2018-10-23 10:45:48 +02:00
Edvard
70722fc9e3
Add todo for changing to single topic in shared transformer config
2018-10-23 10:45:48 +02:00
Edvard
2a9fafca08
Update drip_file tests
2018-10-23 10:45:48 +02:00
Edvard
dc4eb009c1
Refactor drip_file transformers
2018-10-23 10:45:48 +02:00
Edvard
ccca6e0229
Update old pit file shared config references
2018-10-23 10:43:03 +02:00
Edvard
d9b6603109
Update pit file debt ceiling tests
2018-10-23 10:43:03 +02:00
Edvard
24199cb67e
Refactor pit file debt ceiling transformer
2018-10-23 10:43:03 +02:00
Edvard
0239c4daf6
Fix sneaky missing asterisk
2018-10-23 10:43:03 +02:00
Edvard
399b3b78a0
Update tests for pit file ilk transformer
2018-10-23 10:43:03 +02:00
Edvard
98062a7970
Refactor pit file ilk transfomer
2018-10-23 10:43:03 +02:00
Edvard
837a777c8a
Update transformer name for stability fee
2018-10-23 10:43:03 +02:00
Edvard
3412866182
Update tests for stability_fee
2018-10-23 10:43:03 +02:00
Edvard
2cc303a2f5
Refactor stability_fee transfomer
2018-10-23 10:43:03 +02:00
Edvard
57b3744085
Update tend tests
2018-10-23 10:43:03 +02:00
Edvard
a4df8f348d
Refactor Tend transformer using factory
2018-10-23 10:43:03 +02:00
Edvard
4d36f6200d
Further refactor move and init tests
2018-10-23 10:43:03 +02:00
Edvard
c4d3783f8b
Expand error logging in transformer
2018-10-23 10:43:03 +02:00
Edvard
719ffd5819
Update vat init tests for factory
2018-10-23 10:43:03 +02:00
Edvard
fedbce80f6
Refactor vat init transformer to use factory
2018-10-23 10:43:03 +02:00
Edvard
177d212cc4
Add SetDB test for vat move
2018-10-23 10:43:03 +02:00
Edvard
32654ffabc
Add transformer name to config for logging in factory transformer
2018-10-23 10:43:03 +02:00
Edvard
46d8ea8d97
Snag transformer topics from config in factory
2018-10-23 10:43:03 +02:00
Edvard
155569a1d8
Update vat_move tests
2018-10-23 10:43:03 +02:00
Edvard
af1dfbc489
Implement vat_move transformer using factory
2018-10-23 10:43:03 +02:00
Edvard
15e33db33d
Fix transformer factory
2018-10-23 10:43:03 +02:00
Edvard
1b9e6491da
Add transformer factory
2018-10-23 10:43:03 +02:00
Elizabeth
954fcf2378
integration tests ( #75 )
...
* Deal integration test
* Add LogIndex to CatFileChopLump to update unique constraint
* Add LogIndex to cat_file_pit_vow records
* Add integration tests for Cat.file transformers
* Add log index to Cat.file flip
2018-10-22 13:28:42 -05:00
Rob Mulholand
d87fb14c6b
Add log index to Drip file events
2018-10-22 12:14:43 -05:00
Rob Mulholand
de294ad8ea
Add log index to Drip drip
2018-10-22 12:14:43 -05:00
Rob Mulholand
e5ba0496a6
Add log index to Dent
2018-10-22 12:14:43 -05:00
Rob Mulholand
1aafb7201e
Add log index to Deal
2018-10-22 12:14:43 -05:00
Rob Mulholand
495f39abb8
Add log index to Bite
2018-10-22 12:14:43 -05:00
Rob Mulholand
ea3de51792
Add Flop kick integration test
2018-10-19 14:21:57 -05:00
Rob Mulholand
fa828cf041
Add Tend integration test
2018-10-19 14:21:57 -05:00
Rob Mulholand
cbb5ffa3c7
Add Pit file stability fee integration test
2018-10-19 14:21:57 -05:00
Rob Mulholand
3449a1e246
Add Pit file debt ceiling integration test
2018-10-19 14:21:57 -05:00
Rob Mulholand
a83bf5ed0d
Add Pit file ilk integration test
2018-10-19 14:21:57 -05:00
Rob Mulholand
677e78891d
Add Drip file vow integration test
2018-10-19 14:21:57 -05:00
Rob Mulholand
0ce3e3d829
Add Bite integration test
2018-10-19 14:21:57 -05:00
Rob Mulholand
ddc7ee5143
Add Drip drip integration test
2018-10-19 10:56:28 -05:00
Rob Mulholand
b067ba565e
Add Vat fold integration test
2018-10-19 10:56:28 -05:00
Rob Mulholand
fdbe0b0b44
Add Vat heal integration test
2018-10-19 10:56:28 -05:00
Rob Mulholand
a7b9b825c8
Add Vat init integration test
2018-10-19 10:56:28 -05:00
Rob Mulholand
9b427ea1e9
Add Vat move integration test
2018-10-19 10:56:28 -05:00
Taka Goto
21088eb3a2
integration test for vat flux
2018-10-17 16:25:12 -05:00
Konstantin Zolotarev
b38c29ce43
Merge pull request #64 from 8thlight/contract_addresses
...
Added configuration for contract addresses
2018-10-17 21:34:25 +03:00
Rob Mulholand
130d11d0aa
Add Vat slip transformer
2018-10-17 13:30:35 -05:00
Elizabeth
3baca64903
Update address formatting ( #67 )
...
* Add getSignatures command for list of signatures in constants
* Update VatTune converter's formatting of addresses
and added an integration test
* Move VatTune and PriceFeeds integration tests to a common suite
* Update VatGrab converter for address format
* Update frob converter to fix urn and ilk formatting/decoding
* Update formatting
* Update make test command and add integrationtest command
* Add integration test to travis build
* Update creating headers in integration tests to include Raw and Timestamp fields
2018-10-17 13:04:55 -05:00
Taka Goto
4721a91faa
use before each for transformer test
2018-10-17 12:15:15 -05:00
Taka Goto
a3ca319e9b
remove unnecessary byte slicing
2018-10-17 12:04:11 -05:00
Taka Goto
2f64269ab3
update from pr feedback; update tests and use shared fake header
2018-10-17 11:48:57 -05:00
Taka Goto
f32a4fa2a5
use kovan vat contract address
2018-10-17 10:55:30 -05:00
Taka Goto
2e72a7e940
vat flux transformer
2018-10-17 10:53:53 -05:00
Rob Mulholand
d0226dbce5
Convert Bite event field types to numeric
...
- Enables arithmetic operations when constructing views
- Also removes application-level references to DB ID
2018-10-16 15:27:13 -05:00
Rob Mulholand
d3df685c46
Add timestamp to block header
...
- Enables quickly deriving timestamp from header for log events
2018-10-12 12:00:02 -05:00
Rob Mulholand
5f67161f41
Convert raw header to JSON
...
- Enables parsing values off of header via Postgres migration
2018-10-12 12:00:02 -05:00
Edvard Hübinette
be58dd4ac8
Add vat_move
transformer ( #47 )
...
* Add vat_move transformer base
* Add vat_move migrations
* Add test data for vat_move
* Add vat_move transformer to initialisers
* Add numeric cast to psql insert of Rad
* Add new db schema
* Dependency update
* Expand abbreviation in repository
* Add test suite for vat_move
* Add header checking to transformer and mock repository
* Remove trailing zero in test data
* Fix minor mishaps
* Go fmt nitpicking
* Refactoring in tests
* Add tests covering checked headers stuff (and fix revealed bugs)
* go fmt fixes
* Implement batching behaviour of transformer
* Small fixes after review
* Go fmt
2018-10-12 16:13:13 +02:00
Konstantin Zolotarev
0b281eab47
Added default addresses to config & rename fallback var
2018-10-12 11:11:31 +03:00
Konstantin Zolotarev
245732b7f4
Added configuration for contract addresses
2018-10-11 19:10:53 +03:00
Elizabeth
6c77f369d9
Vat heal ( #56 )
...
* VatHeal Converter
* Add VatHeal repository
* Add VatHeal transformer
* Add VatHeal to continuousLogSync command
* Mark vat_init_checked as true when creating vat init records
* Update urn and v converting
* Return error if Repository.MarkCheckedHeader fails
* Add deleting vat heal from test cleanup method
2018-10-10 11:56:06 -05:00
David Terry
0bb3737027
vat.fold: pull common test setup into BeforeEach blocks
2018-10-10 18:13:10 +03:00