updated dependencies for statediffing geth 1.10.23 (#21 )

* updated dependencies for statediffing geth 1.10.23 * another ref to GenesisBlockForTesting
Make writeStateDiffAt call on missing block and add metrics (#20 )
2022-09-02 14:44:47 -04:00 · 2022-08-23 13:05:50 +05:30 · 2022-08-03 14:51:29 -04:00 · 2022-08-03 14:42:06 -04:00 · 2022-08-03 14:36:47 -04:00 · 2022-08-03 11:37:55 -04:00
31 changed files with 3120 additions and 975 deletions
@@ -0,0 +1,6 @@
+name: Run all tests
+on: [pull_request]
+
+jobs:
+  run-tests:
+    uses: ./.github/workflows/tests.yml
@@ -0,0 +1,91 @@
+name: Test the stack.
+on:
+  workflow_call:
+
+env:
+  STACK_ORCHESTRATOR_REF: "f2fd766f5400fcb9eb47b50675d2e3b1f2753702"
+  GO_ETHEREUM_REF: "v1.10.19-statediff-4.1.0-alpha" # Use the tag, we are going to download the bin not build it.
+  IPLD_ETH_DB_REF: "b59505eab252670c622b42ce60621e9747fb64f9"
+
+jobs:
+  integrationtest:
+    name: Run integration tests
+    env:
+      GOPATH: /tmp/go
+      DB_WRITE: true
+    runs-on: ubuntu-latest
+    steps:
+      - name: Create GOPATH
+        run: mkdir -p /tmp/go
+
+      - uses: actions/setup-go@v3
+        with:
+          go-version: "1.18.0"
+          check-latest: true
+
+      - uses: actions/checkout@v2
+        with:
+          path: "./ipld-eth-db-validator"
+
+      - uses: actions/checkout@v2
+        with:
+          ref: ${{ env.STACK_ORCHESTRATOR_REF }}
+          path: "./stack-orchestrator/"
+          repository: vulcanize/stack-orchestrator
+
+      - uses: actions/checkout@v2
+        with:
+          ref: ${{ env.IPLD_ETH_DB_REF }}
+          repository: vulcanize/ipld-eth-db
+          path: "./ipld-eth-db/"
+
+      - name: Create config file
+        run: |
+          echo vulcanize_test_contract=$GITHUB_WORKSPACE/ipld-eth-db-validator/test/contract >> ./config.sh
+          echo vulcanize_ipld_eth_db=$GITHUB_WORKSPACE/ipld-eth-db/ >> ./config.sh
+          echo genesis_file_path=start-up-files/go-ethereum/genesis.json >> ./config.sh
+          echo db_write=$DB_WRITE >> ./config.sh
+          cat ./config.sh
+
+      - name: Download Geth
+        run: |
+          cd $GITHUB_WORKSPACE/stack-orchestrator/helper-scripts
+          wget https://github.com/vulcanize/go-ethereum/releases/download/${{env.GO_ETHEREUM_REF}}/geth-linux-amd64
+
+      - name: Run docker compose
+        run: |
+          docker-compose  \
+          -f "$GITHUB_WORKSPACE/stack-orchestrator/docker/local/docker-compose-db-sharding.yml" \
+          -f "$GITHUB_WORKSPACE/stack-orchestrator/docker/local/docker-compose-go-ethereum.yml" \
+          -f "$GITHUB_WORKSPACE/stack-orchestrator/docker/local/docker-compose-contract.yml" \
+          --env-file "$GITHUB_WORKSPACE/config.sh" \
+          up -d --build
+
+      - name: Run integration test.
+        run: |
+          cd $GITHUB_WORKSPACE/ipld-eth-db-validator
+          ./scripts/run_integration_test.sh
+
+  unittest:
+    name: Run unit tests
+    runs-on: ubuntu-latest
+    steps:
+      - name: Create GOPATH
+        run: mkdir -p /tmp/go
+
+      - uses: actions/setup-go@v3
+        with:
+          go-version: "1.18.0"
+          check-latest: true
+
+      - name: Checkout code
+        uses: actions/checkout@v2
+
+      - name: Spin up database
+        run: |
+          docker-compose up -d
+
+      - name: Run unit tests
+        run: |
+          sleep 30
+          PGPASSWORD=password DATABASE_USER=vdbm DATABASE_PORT=8077 DATABASE_PASSWORD=password DATABASE_HOSTNAME=127.0.0.1 DATABASE_NAME=vulcanize_testing make test
@@ -0,0 +1,3 @@
+ipld-eth-db-validator
+.vscode
+.idea
@@ -0,0 +1,661 @@
+                                                    GNU AFFERO GENERAL PUBLIC LICENSE
+                                                        Version 3, 19 November 2007
+
+                                  Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
+                                  Everyone is permitted to copy and distribute verbatim copies
+                                  of this license document, but changing it is not allowed.
+
+                                                             Preamble
+
+                                   The GNU Affero General Public License is a free, copyleft license for
+                                 software and other kinds of works, specifically designed to ensure
+                                 cooperation with the community in the case of network server software.
+
+                                   The licenses for most software and other practical works are designed
+                                 to take away your freedom to share and change the works.  By contrast,
+                                 our General Public Licenses are intended to guarantee your freedom to
+                                 share and change all versions of a program--to make sure it remains free
+                                 software for all its users.
+
+                                   When we speak of free software, we are referring to freedom, not
+                                 price.  Our General Public Licenses are designed to make sure that you
+                                 have the freedom to distribute copies of free software (and charge for
+                                 them if you wish), that you receive source code or can get it if you
+                                 want it, that you can change the software or use pieces of it in new
+                                 free programs, and that you know you can do these things.
+
+                                   Developers that use our General Public Licenses protect your rights
+                                 with two steps: (1) assert copyright on the software, and (2) offer
+                                 you this License which gives you legal permission to copy, distribute
+                                 and/or modify the software.
+
+                                   A secondary benefit of defending all users' freedom is that
+                                 improvements made in alternate versions of the program, if they
+                                 receive widespread use, become available for other developers to
+                                 incorporate.  Many developers of free software are heartened and
+                                 encouraged by the resulting cooperation.  However, in the case of
+                                 software used on network servers, this result may fail to come about.
+                                 The GNU General Public License permits making a modified version and
+                                 letting the public access it on a server without ever releasing its
+                                 source code to the public.
+
+                                   The GNU Affero General Public License is designed specifically to
+                                 ensure that, in such cases, the modified source code becomes available
+                                 to the community.  It requires the operator of a network server to
+                                 provide the source code of the modified version running there to the
+                                 users of that server.  Therefore, public use of a modified version, on
+                                 a publicly accessible server, gives the public access to the source
+                                 code of the modified version.
+
+                                   An older license, called the Affero General Public License and
+                                 published by Affero, was designed to accomplish similar goals.  This is
+                                 a different license, not a version of the Affero GPL, but Affero has
+                                 released a new version of the Affero GPL which permits relicensing under
+                                 this license.
+
+                                   The precise terms and conditions for copying, distribution and
+                                 modification follow.
+
+                                                        TERMS AND CONDITIONS
+
+                                   0. Definitions.
+
+                                   "This License" refers to version 3 of the GNU Affero General Public License.
+
+                                   "Copyright" also means copyright-like laws that apply to other kinds of
+                                 works, such as semiconductor masks.
+
+                                   "The Program" refers to any copyrightable work licensed under this
+                                 License.  Each licensee is addressed as "you".  "Licensees" and
+                                 "recipients" may be individuals or organizations.
+
+                                   To "modify" a work means to copy from or adapt all or part of the work
+                                 in a fashion requiring copyright permission, other than the making of an
+                                 exact copy.  The resulting work is called a "modified version" of the
+                                 earlier work or a work "based on" the earlier work.
+
+                                   A "covered work" means either the unmodified Program or a work based
+                                 on the Program.
+
+                                   To "propagate" a work means to do anything with it that, without
+                                 permission, would make you directly or secondarily liable for
+                                 infringement under applicable copyright law, except executing it on a
+                                 computer or modifying a private copy.  Propagation includes copying,
+                                 distribution (with or without modification), making available to the
+                                 public, and in some countries other activities as well.
+
+                                   To "convey" a work means any kind of propagation that enables other
+                                 parties to make or receive copies.  Mere interaction with a user through
+                                 a computer network, with no transfer of a copy, is not conveying.
+
+                                   An interactive user interface displays "Appropriate Legal Notices"
+                                 to the extent that it includes a convenient and prominently visible
+                                 feature that (1) displays an appropriate copyright notice, and (2)
+                                 tells the user that there is no warranty for the work (except to the
+                                 extent that warranties are provided), that licensees may convey the
+                                 work under this License, and how to view a copy of this License.  If
+                                 the interface presents a list of user commands or options, such as a
+                                 menu, a prominent item in the list meets this criterion.
+
+                                   1. Source Code.
+
+                                   The "source code" for a work means the preferred form of the work
+                                 for making modifications to it.  "Object code" means any non-source
+                                 form of a work.
+
+                                   A "Standard Interface" means an interface that either is an official
+                                 standard defined by a recognized standards body, or, in the case of
+                                 interfaces specified for a particular programming language, one that
+                                 is widely used among developers working in that language.
+
+                                   The "System Libraries" of an executable work include anything, other
+                                 than the work as a whole, that (a) is included in the normal form of
+                                 packaging a Major Component, but which is not part of that Major
+                                 Component, and (b) serves only to enable use of the work with that
+                                 Major Component, or to implement a Standard Interface for which an
+                                 implementation is available to the public in source code form.  A
+                                 "Major Component", in this context, means a major essential component
+                                 (kernel, window system, and so on) of the specific operating system
+                                 (if any) on which the executable work runs, or a compiler used to
+                                 produce the work, or an object code interpreter used to run it.
+
+                                   The "Corresponding Source" for a work in object code form means all
+                                 the source code needed to generate, install, and (for an executable
+                                 work) run the object code and to modify the work, including scripts to
+                                 control those activities.  However, it does not include the work's
+                                 System Libraries, or general-purpose tools or generally available free
+                                 programs which are used unmodified in performing those activities but
+                                 which are not part of the work.  For example, Corresponding Source
+                                 includes interface definition files associated with source files for
+                                 the work, and the source code for shared libraries and dynamically
+                                 linked subprograms that the work is specifically designed to require,
+                                 such as by intimate data communication or control flow between those
+                                 subprograms and other parts of the work.
+
+                                   The Corresponding Source need not include anything that users
+                                 can regenerate automatically from other parts of the Corresponding
+                                 Source.
+
+                                   The Corresponding Source for a work in source code form is that
+                                 same work.
+
+                                   2. Basic Permissions.
+
+                                   All rights granted under this License are granted for the term of
+                                 copyright on the Program, and are irrevocable provided the stated
+                                 conditions are met.  This License explicitly affirms your unlimited
+                                 permission to run the unmodified Program.  The output from running a
+                                 covered work is covered by this License only if the output, given its
+                                 content, constitutes a covered work.  This License acknowledges your
+                                 rights of fair use or other equivalent, as provided by copyright law.
+
+                                   You may make, run and propagate covered works that you do not
+                                 convey, without conditions so long as your license otherwise remains
+                                 in force.  You may convey covered works to others for the sole purpose
+                                 of having them make modifications exclusively for you, or provide you
+                                 with facilities for running those works, provided that you comply with
+                                 the terms of this License in conveying all material for which you do
+                                 not control copyright.  Those thus making or running the covered works
+                                 for you must do so exclusively on your behalf, under your direction
+                                 and control, on terms that prohibit them from making any copies of
+                                 your copyrighted material outside their relationship with you.
+
+                                   Conveying under any other circumstances is permitted solely under
+                                 the conditions stated below.  Sublicensing is not allowed; section 10
+                                 makes it unnecessary.
+
+                                   3. Protecting Users' Legal Rights From Anti-Circumvention Law.
+
+                                   No covered work shall be deemed part of an effective technological
+                                 measure under any applicable law fulfilling obligations under article
+                                 11 of the WIPO copyright treaty adopted on 20 December 1996, or
+                                 similar laws prohibiting or restricting circumvention of such
+                                 measures.
+
+                                   When you convey a covered work, you waive any legal power to forbid
+                                 circumvention of technological measures to the extent such circumvention
+                                 is effected by exercising rights under this License with respect to
+                                 the covered work, and you disclaim any intention to limit operation or
+                                 modification of the work as a means of enforcing, against the work's
+                                 users, your or third parties' legal rights to forbid circumvention of
+                                 technological measures.
+
+                                   4. Conveying Verbatim Copies.
+
+                                   You may convey verbatim copies of the Program's source code as you
+                                 receive it, in any medium, provided that you conspicuously and
+                                 appropriately publish on each copy an appropriate copyright notice;
+                                 keep intact all notices stating that this License and any
+                                 non-permissive terms added in accord with section 7 apply to the code;
+                                 keep intact all notices of the absence of any warranty; and give all
+                                 recipients a copy of this License along with the Program.
+
+                                   You may charge any price or no price for each copy that you convey,
+                                 and you may offer support or warranty protection for a fee.
+
+                                   5. Conveying Modified Source Versions.
+
+                                   You may convey a work based on the Program, or the modifications to
+                                 produce it from the Program, in the form of source code under the
+                                 terms of section 4, provided that you also meet all of these conditions:
+
+                                     a) The work must carry prominent notices stating that you modified
+                                     it, and giving a relevant date.
+
+                                     b) The work must carry prominent notices stating that it is
+                                     released under this License and any conditions added under section
+                                     7.  This requirement modifies the requirement in section 4 to
+                                     "keep intact all notices".
+
+                                     c) You must license the entire work, as a whole, under this
+                                     License to anyone who comes into possession of a copy.  This
+                                     License will therefore apply, along with any applicable section 7
+                                     additional terms, to the whole of the work, and all its parts,
+                                     regardless of how they are packaged.  This License gives no
+                                     permission to license the work in any other way, but it does not
+                                     invalidate such permission if you have separately received it.
+
+                                     d) If the work has interactive user interfaces, each must display
+                                     Appropriate Legal Notices; however, if the Program has interactive
+                                     interfaces that do not display Appropriate Legal Notices, your
+                                     work need not make them do so.
+
+                                   A compilation of a covered work with other separate and independent
+                                 works, which are not by their nature extensions of the covered work,
+                                 and which are not combined with it such as to form a larger program,
+                                 in or on a volume of a storage or distribution medium, is called an
+                                 "aggregate" if the compilation and its resulting copyright are not
+                                 used to limit the access or legal rights of the compilation's users
+                                 beyond what the individual works permit.  Inclusion of a covered work
+                                 in an aggregate does not cause this License to apply to the other
+                                 parts of the aggregate.
+
+                                   6. Conveying Non-Source Forms.
+
+                                   You may convey a covered work in object code form under the terms
+                                 of sections 4 and 5, provided that you also convey the
+                                 machine-readable Corresponding Source under the terms of this License,
+                                 in one of these ways:
+
+                                     a) Convey the object code in, or embodied in, a physical product
+                                     (including a physical distribution medium), accompanied by the
+                                     Corresponding Source fixed on a durable physical medium
+                                     customarily used for software interchange.
+
+                                     b) Convey the object code in, or embodied in, a physical product
+                                     (including a physical distribution medium), accompanied by a
+                                     written offer, valid for at least three years and valid for as
+                                     long as you offer spare parts or customer support for that product
+                                     model, to give anyone who possesses the object code either (1) a
+                                     copy of the Corresponding Source for all the software in the
+                                     product that is covered by this License, on a durable physical
+                                     medium customarily used for software interchange, for a price no
+                                     more than your reasonable cost of physically performing this
+                                     conveying of source, or (2) access to copy the
+                                     Corresponding Source from a network server at no charge.
+
+                                     c) Convey individual copies of the object code with a copy of the
+                                     written offer to provide the Corresponding Source.  This
+                                     alternative is allowed only occasionally and noncommercially, and
+                                     only if you received the object code with such an offer, in accord
+                                     with subsection 6b.
+
+                                     d) Convey the object code by offering access from a designated
+                                     place (gratis or for a charge), and offer equivalent access to the
+                                     Corresponding Source in the same way through the same place at no
+                                     further charge.  You need not require recipients to copy the
+                                     Corresponding Source along with the object code.  If the place to
+                                     copy the object code is a network server, the Corresponding Source
+                                     may be on a different server (operated by you or a third party)
+                                     that supports equivalent copying facilities, provided you maintain
+                                     clear directions next to the object code saying where to find the
+                                     Corresponding Source.  Regardless of what server hosts the
+                                     Corresponding Source, you remain obligated to ensure that it is
+                                     available for as long as needed to satisfy these requirements.
+
+                                     e) Convey the object code using peer-to-peer transmission, provided
+                                     you inform other peers where the object code and Corresponding
+                                     Source of the work are being offered to the general public at no
+                                     charge under subsection 6d.
+
+                                   A separable portion of the object code, whose source code is excluded
+                                 from the Corresponding Source as a System Library, need not be
+                                 included in conveying the object code work.
+
+                                   A "User Product" is either (1) a "consumer product", which means any
+                                 tangible personal property which is normally used for personal, family,
+                                 or household purposes, or (2) anything designed or sold for incorporation
+                                 into a dwelling.  In determining whether a product is a consumer product,
+                                 doubtful cases shall be resolved in favor of coverage.  For a particular
+                                 product received by a particular user, "normally used" refers to a
+                                 typical or common use of that class of product, regardless of the status
+                                 of the particular user or of the way in which the particular user
+                                 actually uses, or expects or is expected to use, the product.  A product
+                                 is a consumer product regardless of whether the product has substantial
+                                 commercial, industrial or non-consumer uses, unless such uses represent
+                                 the only significant mode of use of the product.
+
+                                   "Installation Information" for a User Product means any methods,
+                                 procedures, authorization keys, or other information required to install
+                                 and execute modified versions of a covered work in that User Product from
+                                 a modified version of its Corresponding Source.  The information must
+                                 suffice to ensure that the continued functioning of the modified object
+                                 code is in no case prevented or interfered with solely because
+                                 modification has been made.
+
+                                   If you convey an object code work under this section in, or with, or
+                                 specifically for use in, a User Product, and the conveying occurs as
+                                 part of a transaction in which the right of possession and use of the
+                                 User Product is transferred to the recipient in perpetuity or for a
+                                 fixed term (regardless of how the transaction is characterized), the
+                                 Corresponding Source conveyed under this section must be accompanied
+                                 by the Installation Information.  But this requirement does not apply
+                                 if neither you nor any third party retains the ability to install
+                                 modified object code on the User Product (for example, the work has
+                                 been installed in ROM).
+
+                                   The requirement to provide Installation Information does not include a
+                                 requirement to continue to provide support service, warranty, or updates
+                                 for a work that has been modified or installed by the recipient, or for
+                                 the User Product in which it has been modified or installed.  Access to a
+                                 network may be denied when the modification itself materially and
+                                 adversely affects the operation of the network or violates the rules and
+                                 protocols for communication across the network.
+
+                                   Corresponding Source conveyed, and Installation Information provided,
+                                 in accord with this section must be in a format that is publicly
+                                 documented (and with an implementation available to the public in
+                                 source code form), and must require no special password or key for
+                                 unpacking, reading or copying.
+
+                                   7. Additional Terms.
+
+                                   "Additional permissions" are terms that supplement the terms of this
+                                 License by making exceptions from one or more of its conditions.
+                                 Additional permissions that are applicable to the entire Program shall
+                                 be treated as though they were included in this License, to the extent
+                                 that they are valid under applicable law.  If additional permissions
+                                 apply only to part of the Program, that part may be used separately
+                                 under those permissions, but the entire Program remains governed by
+                                 this License without regard to the additional permissions.
+
+                                   When you convey a copy of a covered work, you may at your option
+                                 remove any additional permissions from that copy, or from any part of
+                                 it.  (Additional permissions may be written to require their own
+                                 removal in certain cases when you modify the work.)  You may place
+                                 additional permissions on material, added by you to a covered work,
+                                 for which you have or can give appropriate copyright permission.
+
+                                   Notwithstanding any other provision of this License, for material you
+                                 add to a covered work, you may (if authorized by the copyright holders of
+                                 that material) supplement the terms of this License with terms:
+
+                                     a) Disclaiming warranty or limiting liability differently from the
+                                     terms of sections 15 and 16 of this License; or
+
+                                     b) Requiring preservation of specified reasonable legal notices or
+                                     author attributions in that material or in the Appropriate Legal
+                                     Notices displayed by works containing it; or
+
+                                     c) Prohibiting misrepresentation of the origin of that material, or
+                                     requiring that modified versions of such material be marked in
+                                     reasonable ways as different from the original version; or
+
+                                     d) Limiting the use for publicity purposes of names of licensors or
+                                     authors of the material; or
+
+                                     e) Declining to grant rights under trademark law for use of some
+                                     trade names, trademarks, or service marks; or
+
+                                     f) Requiring indemnification of licensors and authors of that
+                                     material by anyone who conveys the material (or modified versions of
+                                     it) with contractual assumptions of liability to the recipient, for
+                                     any liability that these contractual assumptions directly impose on
+                                     those licensors and authors.
+
+                                   All other non-permissive additional terms are considered "further
+                                 restrictions" within the meaning of section 10.  If the Program as you
+                                 received it, or any part of it, contains a notice stating that it is
+                                 governed by this License along with a term that is a further
+                                 restriction, you may remove that term.  If a license document contains
+                                 a further restriction but permits relicensing or conveying under this
+                                 License, you may add to a covered work material governed by the terms
+                                 of that license document, provided that the further restriction does
+                                 not survive such relicensing or conveying.
+
+                                   If you add terms to a covered work in accord with this section, you
+                                 must place, in the relevant source files, a statement of the
+                                 additional terms that apply to those files, or a notice indicating
+                                 where to find the applicable terms.
+
+                                   Additional terms, permissive or non-permissive, may be stated in the
+                                 form of a separately written license, or stated as exceptions;
+                                 the above requirements apply either way.
+
+                                   8. Termination.
+
+                                   You may not propagate or modify a covered work except as expressly
+                                 provided under this License.  Any attempt otherwise to propagate or
+                                 modify it is void, and will automatically terminate your rights under
+                                 this License (including any patent licenses granted under the third
+                                 paragraph of section 11).
+
+                                   However, if you cease all violation of this License, then your
+                                 license from a particular copyright holder is reinstated (a)
+                                 provisionally, unless and until the copyright holder explicitly and
+                                 finally terminates your license, and (b) permanently, if the copyright
+                                 holder fails to notify you of the violation by some reasonable means
+                                 prior to 60 days after the cessation.
+
+                                   Moreover, your license from a particular copyright holder is
+                                 reinstated permanently if the copyright holder notifies you of the
+                                 violation by some reasonable means, this is the first time you have
+                                 received notice of violation of this License (for any work) from that
+                                 copyright holder, and you cure the violation prior to 30 days after
+                                 your receipt of the notice.
+
+                                   Termination of your rights under this section does not terminate the
+                                 licenses of parties who have received copies or rights from you under
+                                 this License.  If your rights have been terminated and not permanently
+                                 reinstated, you do not qualify to receive new licenses for the same
+                                 material under section 10.
+
+                                   9. Acceptance Not Required for Having Copies.
+
+                                   You are not required to accept this License in order to receive or
+                                 run a copy of the Program.  Ancillary propagation of a covered work
+                                 occurring solely as a consequence of using peer-to-peer transmission
+                                 to receive a copy likewise does not require acceptance.  However,
+                                 nothing other than this License grants you permission to propagate or
+                                 modify any covered work.  These actions infringe copyright if you do
+                                 not accept this License.  Therefore, by modifying or propagating a
+                                 covered work, you indicate your acceptance of this License to do so.
+
+                                   10. Automatic Licensing of Downstream Recipients.
+
+                                   Each time you convey a covered work, the recipient automatically
+                                 receives a license from the original licensors, to run, modify and
+                                 propagate that work, subject to this License.  You are not responsible
+                                 for enforcing compliance by third parties with this License.
+
+                                   An "entity transaction" is a transaction transferring control of an
+                                 organization, or substantially all assets of one, or subdividing an
+                                 organization, or merging organizations.  If propagation of a covered
+                                 work results from an entity transaction, each party to that
+                                 transaction who receives a copy of the work also receives whatever
+                                 licenses to the work the party's predecessor in interest had or could
+                                 give under the previous paragraph, plus a right to possession of the
+                                 Corresponding Source of the work from the predecessor in interest, if
+                                 the predecessor has it or can get it with reasonable efforts.
+
+                                   You may not impose any further restrictions on the exercise of the
+                                 rights granted or affirmed under this License.  For example, you may
+                                 not impose a license fee, royalty, or other charge for exercise of
+                                 rights granted under this License, and you may not initiate litigation
+                                 (including a cross-claim or counterclaim in a lawsuit) alleging that
+                                 any patent claim is infringed by making, using, selling, offering for
+                                 sale, or importing the Program or any portion of it.
+
+                                   11. Patents.
+
+                                   A "contributor" is a copyright holder who authorizes use under this
+                                 License of the Program or a work on which the Program is based.  The
+                                 work thus licensed is called the contributor's "contributor version".
+
+                                   A contributor's "essential patent claims" are all patent claims
+                                 owned or controlled by the contributor, whether already acquired or
+                                 hereafter acquired, that would be infringed by some manner, permitted
+                                 by this License, of making, using, or selling its contributor version,
+                                 but do not include claims that would be infringed only as a
+                                 consequence of further modification of the contributor version.  For
+                                 purposes of this definition, "control" includes the right to grant
+                                 patent sublicenses in a manner consistent with the requirements of
+                                 this License.
+
+                                   Each contributor grants you a non-exclusive, worldwide, royalty-free
+                                 patent license under the contributor's essential patent claims, to
+                                 make, use, sell, offer for sale, import and otherwise run, modify and
+                                 propagate the contents of its contributor version.
+
+                                   In the following three paragraphs, a "patent license" is any express
+                                 agreement or commitment, however denominated, not to enforce a patent
+                                 (such as an express permission to practice a patent or covenant not to
+                                 sue for patent infringement).  To "grant" such a patent license to a
+                                 party means to make such an agreement or commitment not to enforce a
+                                 patent against the party.
+
+                                   If you convey a covered work, knowingly relying on a patent license,
+                                 and the Corresponding Source of the work is not available for anyone
+                                 to copy, free of charge and under the terms of this License, through a
+                                 publicly available network server or other readily accessible means,
+                                 then you must either (1) cause the Corresponding Source to be so
+                                 available, or (2) arrange to deprive yourself of the benefit of the
+                                 patent license for this particular work, or (3) arrange, in a manner
+                                 consistent with the requirements of this License, to extend the patent
+                                 license to downstream recipients.  "Knowingly relying" means you have
+                                 actual knowledge that, but for the patent license, your conveying the
+                                 covered work in a country, or your recipient's use of the covered work
+                                 in a country, would infringe one or more identifiable patents in that
+                                 country that you have reason to believe are valid.
+
+                                   If, pursuant to or in connection with a single transaction or
+                                 arrangement, you convey, or propagate by procuring conveyance of, a
+                                 covered work, and grant a patent license to some of the parties
+                                 receiving the covered work authorizing them to use, propagate, modify
+                                 or convey a specific copy of the covered work, then the patent license
+                                 you grant is automatically extended to all recipients of the covered
+                                 work and works based on it.
+
+                                   A patent license is "discriminatory" if it does not include within
+                                 the scope of its coverage, prohibits the exercise of, or is
+                                 conditioned on the non-exercise of one or more of the rights that are
+                                 specifically granted under this License.  You may not convey a covered
+                                 work if you are a party to an arrangement with a third party that is
+                                 in the business of distributing software, under which you make payment
+                                 to the third party based on the extent of your activity of conveying
+                                 the work, and under which the third party grants, to any of the
+                                 parties who would receive the covered work from you, a discriminatory
+                                 patent license (a) in connection with copies of the covered work
+                                 conveyed by you (or copies made from those copies), or (b) primarily
+                                 for and in connection with specific products or compilations that
+                                 contain the covered work, unless you entered into that arrangement,
+                                 or that patent license was granted, prior to 28 March 2007.
+
+                                   Nothing in this License shall be construed as excluding or limiting
+                                 any implied license or other defenses to infringement that may
+                                 otherwise be available to you under applicable patent law.
+
+                                   12. No Surrender of Others' Freedom.
+
+                                   If conditions are imposed on you (whether by court order, agreement or
+                                 otherwise) that contradict the conditions of this License, they do not
+                                 excuse you from the conditions of this License.  If you cannot convey a
+                                 covered work so as to satisfy simultaneously your obligations under this
+                                 License and any other pertinent obligations, then as a consequence you may
+                                 not convey it at all.  For example, if you agree to terms that obligate you
+                                 to collect a royalty for further conveying from those to whom you convey
+                                 the Program, the only way you could satisfy both those terms and this
+                                 License would be to refrain entirely from conveying the Program.
+
+                                   13. Remote Network Interaction; Use with the GNU General Public License.
+
+                                   Notwithstanding any other provision of this License, if you modify the
+                                 Program, your modified version must prominently offer all users
+                                 interacting with it remotely through a computer network (if your version
+                                 supports such interaction) an opportunity to receive the Corresponding
+                                 Source of your version by providing access to the Corresponding Source
+                                 from a network server at no charge, through some standard or customary
+                                 means of facilitating copying of software.  This Corresponding Source
+                                 shall include the Corresponding Source for any work covered by version 3
+                                 of the GNU General Public License that is incorporated pursuant to the
+                                 following paragraph.
+
+                                   Notwithstanding any other provision of this License, you have
+                                 permission to link or combine any covered work with a work licensed
+                                 under version 3 of the GNU General Public License into a single
+                                 combined work, and to convey the resulting work.  The terms of this
+                                 License will continue to apply to the part which is the covered work,
+                                 but the work with which it is combined will remain governed by version
+                                 3 of the GNU General Public License.
+
+                                   14. Revised Versions of this License.
+
+                                   The Free Software Foundation may publish revised and/or new versions of
+                                 the GNU Affero General Public License from time to time.  Such new versions
+                                 will be similar in spirit to the present version, but may differ in detail to
+                                 address new problems or concerns.
+
+                                   Each version is given a distinguishing version number.  If the
+                                 Program specifies that a certain numbered version of the GNU Affero General
+                                 Public License "or any later version" applies to it, you have the
+                                 option of following the terms and conditions either of that numbered
+                                 version or of any later version published by the Free Software
+                                 Foundation.  If the Program does not specify a version number of the
+                                 GNU Affero General Public License, you may choose any version ever published
+                                 by the Free Software Foundation.
+
+                                   If the Program specifies that a proxy can decide which future
+                                 versions of the GNU Affero General Public License can be used, that proxy's
+                                 public statement of acceptance of a version permanently authorizes you
+                                 to choose that version for the Program.
+
+                                   Later license versions may give you additional or different
+                                 permissions.  However, no additional obligations are imposed on any
+                                 author or copyright holder as a result of your choosing to follow a
+                                 later version.
+
+                                   15. Disclaimer of Warranty.
+
+                                   THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
+                                 APPLICABLE LAW.  EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
+                                 HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
+                                 OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
+                                 THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+                                 PURPOSE.  THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
+                                 IS WITH YOU.  SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
+                                 ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
+
+                                   16. Limitation of Liability.
+
+                                   IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+                                 WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
+                                 THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
+                                 GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
+                                 USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
+                                 DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
+                                 PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
+                                 EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
+                                 SUCH DAMAGES.
+
+                                   17. Interpretation of Sections 15 and 16.
+
+                                   If the disclaimer of warranty and limitation of liability provided
+                                 above cannot be given local legal effect according to their terms,
+                                 reviewing courts shall apply local law that most closely approximates
+                                 an absolute waiver of all civil liability in connection with the
+                                 Program, unless a warranty or assumption of liability accompanies a
+                                 copy of the Program in return for a fee.
+
+                                                      END OF TERMS AND CONDITIONS
+
+                                             How to Apply These Terms to Your New Programs
+
+                                   If you develop a new program, and you want it to be of the greatest
+                                 possible use to the public, the best way to achieve this is to make it
+                                 free software which everyone can redistribute and change under these terms.
+
+                                   To do so, attach the following notices to the program.  It is safest
+                                 to attach them to the start of each source file to most effectively
+                                 state the exclusion of warranty; and each file should have at least
+                                 the "copyright" line and a pointer to where the full notice is found.
+
+                                     <one line to give the program's name and a brief idea of what it does.>
+                                     Copyright (C) <year>  <name of author>
+
+                                     This program is free software: you can redistribute it and/or modify
+                                     it under the terms of the GNU Affero General Public License as published by
+                                     the Free Software Foundation, either version 3 of the License, or
+                                     (at your option) any later version.
+
+                                     This program is distributed in the hope that it will be useful,
+                                     but WITHOUT ANY WARRANTY; without even the implied warranty of
+                                     MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+                                     GNU Affero General Public License for more details.
+
+                                     You should have received a copy of the GNU Affero General Public License
+                                     along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+                                 Also add information on how to contact you by electronic and paper mail.
+
+                                   If your software can interact with users remotely through a computer
+                                 network, you should also make sure that it provides a way for users to
+                                 get its source.  For example, if your program is a web application, its
+                                 interface could display a "Source" link that leads users to an archive
+                                 of the code.  There are many ways you could offer source, and different
+                                 solutions will be better for different programs; see section 13 for the
+                                 specific requirements.
+
+                                   You should also get your employer (if you work as a programmer) or school,
+                                 if any, to sign a "copyright disclaimer" for the program, if necessary.
+                                 For more information on this, and how to apply and follow the GNU AGPL, see
+                                 <http://www.gnu.org/licenses/>.
@@ -3,22 +3,18 @@ BASE = $(GOPATH)/src/$(PACKAGE)
 PKGS = go list ./... | grep -v "^vendor/"

 # Tools
-## Testing library
-GINKGO = $(BIN)/ginkgo
-$(BIN)/ginkgo:
-	go install github.com/onsi/ginkgo/ginkgo

 .PHONY: integrationtest
-integrationtest: | $(GINKGO) $(GOOSE)
+integrationtest: | $(GOOSE)
 	go vet ./...
 	go fmt ./...
-	$(GINKGO) -r test/ -v
+	go run github.com/onsi/ginkgo/ginkgo -r test/ -v

 .PHONY: test
-test: | $(GINKGO) $(GOOSE)
+test: | $(GOOSE)
 	go vet ./...
 	go fmt ./...
-	$(GINKGO) -r validator_test/ -v
+	go run github.com/onsi/ginkgo/ginkgo -r validator_test/ -v

 build:
 	go fmt ./...
@@ -1,75 +1,124 @@
- [Validator-README](#validator-readme)
- [Overview](#overview)
- [Intention for the Validator](#intention-for-the-validator)
-  - [Edge Cases](#edge-cases)
- [Instructions for Testing](#instructions-for-testing)
- [Code Overview](#code-overview)
- [Known Bugs](#known-bugs)
- [Tests on 03/03/22](#tests-on-03-03-22)
-  - [Set Up](#set-up)
-  - [Testing Failures](#testing-failures)
+# ipld-eth-db-validator

-<small><i><a href='http://ecotrust-canada.github.io/markdown-toc/'>Table of contents generated with markdown-toc</a></i></small>
+> `ipld-eth-db-validator` performs validation checks on indexed Ethereum IPLD objects in a Postgres database:
+> * Attempt to apply transactions in each block and validate resultant block hash
+> * Check referential integrity between IPLD blocks and index tables

-# Overview
+## Setup

-This repository contains the validator. The purpose of the validator is to ensure that the data in the Core Postgres database match the data on the blockchain.
+Build the binary:

-# Intention for the Validator
+```bash
+make build
+```

-The perfect scenario for the validator is as follows:
+## Configuration

-1. The validator will have the capacity to perform historical checks for the Core Postgres database. Users can contain these historical checks to specified configurations (block range).
-2. The validator will validate a certain number of trailing blocks, `t`, trailing the head, `n`. Therefore the validator will constantly perform real-time validation starting at `n` and ending at `n - t`.
-3. The validator validates the IPLD blocks in the Core Database; it will update the core database to indicate that the validator validated it.
+An example config file:

-## Edge Cases
+```toml
+[database]
+  # db credentials
+  name     = "vulcanize_public" # DATABASE_NAME
+  hostname = "localhost"        # DATABASE_HOSTNAME
+  port     = 5432               # DATABASE_PORT
+  user     = "vdbm"             # DATABASE_USER
+  password = "..."              # DATABASE_PASSWORD

-We must consider the following edge cases for the validator.
+[validate]
+  # block height to initiate database validation at
+  blockHeight = 1      # VALIDATE_BLOCK_HEIGHT  (default: 1)
+  # number of blocks to trail behind the head
+  trail  = 16         # VALIDATE_TRAIL  (default: 16)
+  # sleep interval after validator has caught up to (head-trail) height (in sec)
+  sleepInterval = 10  # VALIDATE_SLEEP_INTERVAL (default: 10)

- There are three different data types that the validator must account for.
+  # whether to perform a statediffing call on a missing block
+  stateDiffMissingBlock = true # (default: false)
+  # statediffing call timeout period (in sec)
+  stateDiffTimeout = 240 # (default: 240)

-# Instructions for Testing
+[ethereum]
+  # node info
+  # path to json chain config (optional)
+  chainConfig = ""            # ETH_CHAIN_CONFIG
+  # eth chain id for config (overridden by chainConfig)
+  chainID = "1"               # ETH_CHAIN_ID (default: 1)
+  # http RPC endpoint URL for a statediffing node
+  httpPath = "localhost:8545" # ETH_HTTP_PATH

-Follow steps in [test/README.md](./test/README.md)
+[prom]
+  # prometheus metrics
+  metrics = true        # PROM_METRICS    (default: false)
+  http = true           # PROM_HTTP       (default: false)
+  httpAddr = "0.0.0.0"  # PROM_HTTP_ADDR  (default: 127.0.0.1)
+  httpPort = "9001"     # PROM_HTTP_PORT  (default: 9001)
+  dbStats = true        # PROM_DB_STATS   (default: false)

-# Code Overview
+[log]
+  # log level (trace, debug, info, warn, error, fatal, panic)
+  level = "info"  # LOG_LEVEL (default: info)
+  # file path for logging, leave unset to log to stdout
+  file  = ""      # LOG_FILE_PATH
+```

-This section will provide some insight into specific files and their purpose.

- `validator_test/chain_maker.go` - This file contains the code for creating a “test” blockchain.
- `validator_test/validator_test.go` - This file contains testing to validate the validator. It leverages `chain_maker.go` to create a blockchain to validate.
- `pkg/validator/validator.go` - This file contains most of the core logic for the validator.
+* The validation process trails behind the latest block number in the database by config parameter `validate.trail`.

-# Known Bugs
+* If the validator has caught up to (head-trail) height, it waits for a configured time interval (`validate.sleepInterval`) before again querying the database.

-1. The validator is improperly handling missing headers from the database.
-   1. Scenario
-      1. The IPLD blocks from the mock blockchain are inserted into the Postgres Data.
-      2. The validator runs, and all tests pass.
-      3. Users manually remove the last few rows from the database.
-      4. The validator runs, and all tests pass - This behavior is neither expected nor wanted.
+* If the validator encounters a missing block (gap) in the database, it makes a `writeStateDiffAt` call to the configured statediffing endpoint (`ethereum.httpPath`) if `validate.stateDiffMissingBlock` is set to `true`. Here it is assumed that the statediffing node pointed to is writing out to the database.

-# Tests on 03/03/22
+### Local Setup

-The tests highlighted below were conducted to validate the initial behavior of the validator.
+* Create a chain config file `chain.json` according to chain config in genesis json file used by local geth.

-## Set Up
+  Example:

-Below are the steps utilized to set up the test environment.
+  ```json
+  {
+    "chainId": 41337,
+    "homesteadBlock": 0,
+    "eip150Block": 0,
+    "eip150Hash": "0x0000000000000000000000000000000000000000000000000000000000000000",
+    "eip155Block": 0,
+    "eip158Block": 0,
+    "byzantiumBlock": 0,
+    "constantinopleBlock": 0,
+    "petersburgBlock": 0,
+    "istanbulBlock": 0,
+    "clique": {
+      "period": 5,
+      "epoch": 30000
+    }
+  }
+  ```

-1. Run the `scripts/run_integration_test.sh` script.
-   1. First comment outline 130 to 133 from `validator_test/validator_test.go`
-2. Once the code has completed running, comment out lines 55 to 126, 38 to 40, and 42 to 44.
-   1. Make the following change `db, err = setupDB() --> db, _ = setupDB()`
-3. Run the following command: `ginkgo -r validator_test/ -v`
-   1. All tests should pass
+  Provide the path to the above file in the config.

-## Testing Failures
+## Usage

-Once we had populated the database, we tested for failures.
+* Create / update the config file (refer to example config above).

-1. Removing a Transaction from `transaction_cids` - If we removed a transaction from the database and ran the test, the test would fail. **This is the expected behavior.**
-2. Removing Headers from `eth.header_cids`
-   1. If we removed a header block sandwiched between two header blocks, the test would fail (For example, we removed the entry for block 4, and the block range is 1-10). **This is the expected behavior.**
-   2. If we removed the tail block(s) from the table, the test would pass (For example, we remove the entry for blocks 8, 9, 10, and the block range is 1-10). **This is _not_ the expected behavior.**
+* Run validator:
+
+  ```bash
+  ./ipld-eth-db-validator stateValidator --config=<config path>
+  ```
+
+  Example:
+
+  ```bash
+  ./ipld-eth-db-validator stateValidator --config=environments/example.toml
+  ```
+
+## Monitoring
+
+* Enable metrics using config parameters `prom.metrics` and `prom.http`.
+* `ipld-eth-db-validator` exposes following prometheus metrics at `/metrics` endpoint:
+  * `last_validated_block`: Last validated block number.
+  * DB stats if `prom.dbStats` set to `true`.
+
+## Tests
+
+* Follow [Test Instructions](./test/README.md) to run unit and integration tests locally.
@@ -0,0 +1,84 @@
+// VulcanizeDB
+// Copyright © 2022 Vulcanize
+
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU Affero General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU Affero General Public License for more details.
+
+// You should have received a copy of the GNU Affero General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package cmd
+
+import (
+	"github.com/spf13/viper"
+)
+
+const (
+	LOG_LEVEL     = "LOG_LEVEL"
+	LOG_FILE_PATH = "LOG_FILE_PATH"
+
+	PROM_METRICS   = "PROM_METRICS"
+	PROM_HTTP      = "PROM_HTTP"
+	PROM_HTTP_ADDR = "PROM_HTTP_ADDR"
+	PROM_HTTP_PORT = "PROM_HTTP_PORT"
+	PROM_DB_STATS  = "PROM_DB_STATS"
+
+	DATABASE_NAME     = "DATABASE_NAME"
+	DATABASE_HOSTNAME = "DATABASE_HOSTNAME"
+	DATABASE_PORT     = "DATABASE_PORT"
+	DATABASE_USER     = "DATABASE_USER"
+	DATABASE_PASSWORD = "DATABASE_PASSWORD"
+
+	DATABASE_MAX_IDLE_CONNECTIONS = "DATABASE_MAX_IDLE_CONNECTIONS"
+	DATABASE_MAX_OPEN_CONNECTIONS = "DATABASE_MAX_OPEN_CONNECTIONS"
+	DATABASE_MAX_CONN_LIFETIME    = "DATABASE_MAX_CONN_LIFETIME"
+
+	ETH_CHAIN_CONFIG = "ETH_CHAIN_CONFIG"
+	ETH_CHAIN_ID     = "ETH_CHAIN_ID"
+	ETH_HTTP_PATH    = "ETH_HTTP_PATH"
+
+	VALIDATE_BLOCK_HEIGHT            = "VALIDATE_BLOCK_HEIGHT"
+	VALIDATE_TRAIL                   = "VALIDATE_TRAIL"
+	VALIDATE_SLEEP_INTERVAL          = "VALIDATE_SLEEP_INTERVAL"
+	VALIDATE_STATEDIFF_MISSING_BLOCK = "VALIDATE_STATEDIFF_MISSING_BLOCK"
+	VALIDATE_STATEDIFF_TIMEOUT       = "VALIDATE_STATEDIFF_TIMEOUT"
+)
+
+// Bind env vars
+func init() {
+	viper.BindEnv("log.level", LOG_LEVEL)
+	viper.BindEnv("log.file", LOG_FILE_PATH)
+
+	viper.BindEnv("prom.metrics", PROM_METRICS)
+	viper.BindEnv("prom.http", PROM_HTTP)
+	viper.BindEnv("prom.httpAddr", PROM_HTTP_ADDR)
+	viper.BindEnv("prom.httpPort", PROM_HTTP_PORT)
+	viper.BindEnv("prom.dbStats", PROM_DB_STATS)
+
+	viper.BindEnv("database.name", DATABASE_NAME)
+	viper.BindEnv("database.hostname", DATABASE_HOSTNAME)
+	viper.BindEnv("database.port", DATABASE_PORT)
+	viper.BindEnv("database.user", DATABASE_USER)
+	viper.BindEnv("database.password", DATABASE_PASSWORD)
+
+	viper.BindEnv("database.maxIdle", DATABASE_MAX_IDLE_CONNECTIONS)
+	viper.BindEnv("database.maxOpen", DATABASE_MAX_OPEN_CONNECTIONS)
+	viper.BindEnv("database.maxLifetime", DATABASE_MAX_CONN_LIFETIME)
+
+	viper.BindEnv("ethereum.chainConfig", ETH_CHAIN_CONFIG)
+	viper.BindEnv("ethereum.chainID", ETH_CHAIN_ID)
+	viper.BindEnv("ethereum.httpPath", ETH_HTTP_PATH)
+
+	viper.BindEnv("validate.blockHeight", VALIDATE_BLOCK_HEIGHT)
+	viper.BindEnv("validate.trail", VALIDATE_TRAIL)
+	viper.BindEnv("validate.sleepInterval", VALIDATE_SLEEP_INTERVAL)
+	viper.BindEnv("validate.stateDiffMissingBlock", VALIDATE_STATEDIFF_MISSING_BLOCK)
+	viper.BindEnv("validate.stateDiffTimeout", VALIDATE_STATEDIFF_TIMEOUT)
+}
@@ -1,12 +1,31 @@
+// VulcanizeDB
+// Copyright © 2022 Vulcanize
+
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU Affero General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU Affero General Public License for more details.
+
+// You should have received a copy of the GNU Affero General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 package cmd

 import (
+	"fmt"
 	"os"
 	"strings"

 	log "github.com/sirupsen/logrus"
 	"github.com/spf13/cobra"
 	"github.com/spf13/viper"
+
+	"github.com/vulcanize/ipld-eth-db-validator/pkg/prom"
 )

 var (
@@ -31,7 +50,7 @@ func Execute() {
 }

 func initFunc(cmd *cobra.Command, args []string) {
-	logfile := viper.GetString("logfile")
+	logfile := viper.GetString("log.file")
 	if logfile != "" {
 		file, err := os.OpenFile(logfile,
 			os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0666)
@@ -49,6 +68,21 @@ func initFunc(cmd *cobra.Command, args []string) {
 	if err := logLevel(); err != nil {
 		log.Fatal("Could not set log level: ", err)
 	}
+
+	if viper.GetBool("prom.metrics") {
+		log.Info("initializing prometheus metrics")
+		prom.Init()
+	}
+
+	if viper.GetBool("prom.http") {
+		addr := fmt.Sprintf(
+			"%s:%s",
+			viper.GetString("prom.httpAddr"),
+			viper.GetString("prom.httpPort"),
+		)
+		log.Info("starting prometheus server")
+		prom.Serve(addr)
+	}
 }

 func init() {
@@ -57,21 +91,33 @@ func init() {
 	viper.AutomaticEnv()

 	rootCmd.PersistentFlags().StringVar(&cfgFile, "config", "", "config file location")
-	rootCmd.PersistentFlags().String("logfile", "", "file path for logging")
 	rootCmd.PersistentFlags().String("database-name", "vulcanize_public", "database name")
 	rootCmd.PersistentFlags().Int("database-port", 5432, "database port")
 	rootCmd.PersistentFlags().String("database-hostname", "localhost", "database hostname")
 	rootCmd.PersistentFlags().String("database-user", "", "database user")
 	rootCmd.PersistentFlags().String("database-password", "", "database password")
+	rootCmd.PersistentFlags().String("log-file", "", "file path for logging")
 	rootCmd.PersistentFlags().String("log-level", log.InfoLevel.String(), "Log level (trace, debug, info, warn, error, fatal, panic")

-	_ = viper.BindPFlag("logfile", rootCmd.PersistentFlags().Lookup("logfile"))
+	rootCmd.PersistentFlags().Bool("prom-metrics", false, "enable prometheus metrics")
+	rootCmd.PersistentFlags().Bool("prom-http", false, "enable prometheus http service")
+	rootCmd.PersistentFlags().String("prom-httpAddr", "127.0.0.1", "prometheus http host")
+	rootCmd.PersistentFlags().String("prom-httpPort", "9001", "prometheus http port")
+	rootCmd.PersistentFlags().Bool("prom-dbStats", false, "enables prometheus db stats")
+
 	_ = viper.BindPFlag("database.name", rootCmd.PersistentFlags().Lookup("database-name"))
 	_ = viper.BindPFlag("database.port", rootCmd.PersistentFlags().Lookup("database-port"))
 	_ = viper.BindPFlag("database.hostname", rootCmd.PersistentFlags().Lookup("database-hostname"))
 	_ = viper.BindPFlag("database.user", rootCmd.PersistentFlags().Lookup("database-user"))
 	_ = viper.BindPFlag("database.password", rootCmd.PersistentFlags().Lookup("database-password"))
+	_ = viper.BindPFlag("log.file", rootCmd.PersistentFlags().Lookup("log-file"))
 	_ = viper.BindPFlag("log.level", rootCmd.PersistentFlags().Lookup("log-level"))
+
+	_ = viper.BindPFlag("prom.metrics", rootCmd.PersistentFlags().Lookup("prom-metrics"))
+	_ = viper.BindPFlag("prom.http", rootCmd.PersistentFlags().Lookup("prom-http"))
+	_ = viper.BindPFlag("prom.httpAddr", rootCmd.PersistentFlags().Lookup("prom-httpAddr"))
+	_ = viper.BindPFlag("prom.httpPort", rootCmd.PersistentFlags().Lookup("prom-httpPort"))
+	_ = viper.BindPFlag("prom.dbStats", rootCmd.PersistentFlags().Lookup("prom-dbStats"))
 }

 func logLevel() error {
@@ -1,10 +1,28 @@
+// VulcanizeDB
+// Copyright © 2022 Vulcanize
+
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU Affero General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU Affero General Public License for more details.
+
+// You should have received a copy of the GNU Affero General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 package cmd

 import (
 	"context"
 	"fmt"
+	"os"
+	"os/signal"
+	"sync"

-	"github.com/ethereum/go-ethereum/statediff"
 	log "github.com/sirupsen/logrus"
 	"github.com/spf13/cobra"
 	"github.com/spf13/viper"
@@ -31,40 +49,41 @@ func stateValidator() {
 		logWithCommand.Fatal(err)
 	}

-	height := viper.GetUint64("validate.block-height")
-	if height < 1 {
-		logWithCommand.Fatalf("block height cannot be less the 1")
-	}
-	trail := viper.GetUint64("validate.trail")
+	service := validator.NewService(cfg, nil)

-	chainConfigPath := viper.GetString("ethereum.chainConfig")
-	chainCfg, err := statediff.LoadConfig(chainConfigPath)
-	if err != nil {
-		logWithCommand.Fatal(err)
-	}
+	wg := new(sync.WaitGroup)
+	wg.Add(1)
+	go service.Start(context.Background(), wg)

-	srvc := validator.NewService(cfg.DB, height, trail, chainCfg)
-
-	_, err = srvc.Start(context.Background())
-	if err != nil {
-		logWithCommand.Fatal(err)
-	}
-
-	logWithCommand.Println("state validation complete")
+	shutdown := make(chan os.Signal, 1)
+	signal.Notify(shutdown, os.Interrupt)
+	<-shutdown
+	service.Stop()
+	wg.Wait()
 }

 func init() {
 	rootCmd.AddCommand(stateValidatorCmd)

 	stateValidatorCmd.PersistentFlags().String("block-height", "1", "block height to initiate state validation")
-	stateValidatorCmd.PersistentFlags().String("trail", "0", "trail of block height to validate")
+	stateValidatorCmd.PersistentFlags().String("trail", "16", "trail of block height to validate")
+	stateValidatorCmd.PersistentFlags().String("sleep-interval", "10", "sleep interval in seconds after validator has caught up to (head-trail) height")
+	stateValidatorCmd.PersistentFlags().Bool("statediff-missing-block", false, "whether to perform a statediffing call on a missing block")
+	stateValidatorCmd.PersistentFlags().Uint("statediff-timeout", 240, "statediffing call timeout period (in sec)")

-	stateValidatorCmd.PersistentFlags().String("chain-config", "", "path to chain config")
+	stateValidatorCmd.PersistentFlags().String("eth-chain-config", "", "path to json chain config")
+	stateValidatorCmd.PersistentFlags().String("eth-chain-id", "1", "eth chain id")
+	stateValidatorCmd.PersistentFlags().String("eth-http-path", "", "http url for a statediffing node")

-	_ = viper.BindPFlag("validate.block-height", stateValidatorCmd.PersistentFlags().Lookup("block-height"))
+	_ = viper.BindPFlag("validate.blockHeight", stateValidatorCmd.PersistentFlags().Lookup("block-height"))
 	_ = viper.BindPFlag("validate.trail", stateValidatorCmd.PersistentFlags().Lookup("trail"))
+	_ = viper.BindPFlag("validate.sleepInterval", stateValidatorCmd.PersistentFlags().Lookup("sleep-interval"))
+	_ = viper.BindPFlag("validate.stateDiffMissingBlock", stateValidatorCmd.PersistentFlags().Lookup("statediff-missing-block"))
+	_ = viper.BindPFlag("validate.stateDiffTimeout", stateValidatorCmd.PersistentFlags().Lookup("statediff-timeout"))

-	_ = viper.BindPFlag("ethereum.chainConfig", stateValidatorCmd.PersistentFlags().Lookup("chain-config"))
+	_ = viper.BindPFlag("ethereum.chainConfig", stateValidatorCmd.PersistentFlags().Lookup("eth-chain-config"))
+	_ = viper.BindPFlag("ethereum.chainID", stateValidatorCmd.PersistentFlags().Lookup("eth-chain-id"))
+	_ = viper.BindPFlag("ethereum.httpPath", stateValidatorCmd.PersistentFlags().Lookup("eth-http-path"))
 }

 func initConfig() {
@@ -5,7 +5,7 @@ services:
    restart: on-failure
    depends_on:
      - ipld-eth-db
-    image: vulcanize/ipld-eth-db:v4.1.1-alpha
+    image: vulcanize/ipld-eth-db:v4.2.0-alpha
    environment:
      DATABASE_USER: "vdbm"
      DATABASE_NAME: "vulcanize_testing"
@@ -6,8 +6,24 @@
    user     = "vdbm"

 [validate]
-    block-height = 1
-    trail  = 10
+    blockHeight = 1
+    trail  = 16
+    sleepInterval = 10
+    stateDiffMissingBlock = true
+    stateDiffTimeout = 240

 [ethereum]
-    chainConfig = "./chain.json"
+    chainConfig = ""
+    chainID = "1"
+    httpPath = "localhost:8545"
+
+[prom]
+    metrics = true
+    http = true
+    httpAddr = "localhost"
+    httpPort = "9001"
+    dbStats = true
+
+[log]
+    file = ""
+    level = "info"
@@ -1,223 +1,223 @@
 module github.com/vulcanize/ipld-eth-db-validator

-go 1.17
+go 1.18

 require (
-	github.com/ethereum/go-ethereum v1.10.17
+	github.com/ethereum/go-ethereum v1.10.23
 	github.com/jmoiron/sqlx v1.3.5
 	github.com/onsi/ginkgo v1.16.5
 	github.com/onsi/gomega v1.19.0
+	github.com/prometheus/client_golang v1.12.1
 	github.com/sirupsen/logrus v1.8.1
 	github.com/spf13/cobra v1.4.0
 	github.com/spf13/viper v1.11.0
-	github.com/vulcanize/ipfs-ethdb/v4 v4.0.0-alpha
-	github.com/vulcanize/ipld-eth-server/v4 v4.0.1-alpha
+	github.com/vulcanize/ipfs-ethdb/v4 v4.0.7-alpha
+	github.com/vulcanize/ipld-eth-server/v4 v4.1.7-alpha
 )

 require (
 	bazil.org/fuse v0.0.0-20200117225306-7b5117fecadc // indirect
-	github.com/StackExchange/wmi v0.0.0-20180116203802-5d049714c4a6 // indirect
 	github.com/Stebalien/go-bitfield v0.0.1 // indirect
 	github.com/VictoriaMetrics/fastcache v1.6.0 // indirect
-	github.com/benbjohnson/clock v1.1.0 // indirect
+	github.com/alecthomas/units v0.0.0-20210927113745-59d0afb8317a // indirect
+	github.com/benbjohnson/clock v1.3.0 // indirect
 	github.com/beorn7/perks v1.0.1 // indirect
 	github.com/blang/semver/v4 v4.0.0 // indirect
 	github.com/btcsuite/btcd v0.22.1 // indirect
 	github.com/btcsuite/btcd/btcec/v2 v2.2.0 // indirect
+	github.com/btcsuite/btcd/chaincfg/chainhash v1.0.1 // indirect
 	github.com/cenkalti/backoff v2.2.1+incompatible // indirect
-	github.com/cenkalti/backoff/v4 v4.1.1 // indirect
+	github.com/cenkalti/backoff/v4 v4.1.3 // indirect
 	github.com/cespare/xxhash/v2 v2.1.2 // indirect
 	github.com/cheekybits/genny v1.0.0 // indirect
+	github.com/containerd/cgroups v1.0.3 // indirect
+	github.com/coreos/go-systemd/v22 v22.3.2 // indirect
+	github.com/cpuguy83/go-md2man/v2 v2.0.2 // indirect
 	github.com/crackcomm/go-gitignore v0.0.0-20170627025303-887ab5e44cc3 // indirect
 	github.com/cskr/pubsub v1.0.2 // indirect
 	github.com/davecgh/go-spew v1.1.1 // indirect
 	github.com/davidlazar/go-crypto v0.0.0-20200604182044-b73af7476f6c // indirect
 	github.com/deckarep/golang-set v1.8.0 // indirect
 	github.com/decred/dcrd/dcrec/secp256k1/v4 v4.0.1 // indirect
+	github.com/docker/go-units v0.4.0 // indirect
+	github.com/dustin/go-humanize v1.0.0 // indirect
 	github.com/edsrzf/mmap-go v1.0.0 // indirect
+	github.com/elastic/gosigar v0.14.2 // indirect
 	github.com/facebookgo/atomicfile v0.0.0-20151019160806-2de1f203e7d5 // indirect
 	github.com/fjl/memsize v0.0.0-20190710130421-bcb5799ab5e5 // indirect
 	github.com/flynn/noise v1.0.0 // indirect
 	github.com/francoispqt/gojay v1.2.13 // indirect
-	github.com/fsnotify/fsnotify v1.5.1 // indirect
+	github.com/fsnotify/fsnotify v1.5.4 // indirect
 	github.com/gballet/go-libpcsclite v0.0.0-20190607065134-2772fd86a8ff // indirect
 	github.com/georgysavva/scany v0.2.9 // indirect
-	github.com/go-ole/go-ole v1.2.1 // indirect
+	github.com/go-logr/logr v1.2.3 // indirect
+	github.com/go-logr/stdr v1.2.2 // indirect
+	github.com/go-ole/go-ole v1.2.6 // indirect
 	github.com/go-stack/stack v1.8.0 // indirect
 	github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0 // indirect
+	github.com/godbus/dbus/v5 v5.1.0 // indirect
 	github.com/gogo/protobuf v1.3.2 // indirect
 	github.com/golang-jwt/jwt/v4 v4.3.0 // indirect
 	github.com/golang/protobuf v1.5.2 // indirect
 	github.com/golang/snappy v0.0.4 // indirect
 	github.com/google/gopacket v1.1.19 // indirect
 	github.com/google/uuid v1.3.0 // indirect
-	github.com/gorilla/websocket v1.4.2 // indirect
+	github.com/gorilla/websocket v1.5.0 // indirect
 	github.com/hannahhoward/go-pubsub v0.0.0-20200423002714-8d62886cc36e // indirect
-	github.com/hashicorp/errwrap v1.0.0 // indirect
+	github.com/hashicorp/errwrap v1.1.0 // indirect
 	github.com/hashicorp/go-bexpr v0.1.10 // indirect
 	github.com/hashicorp/go-multierror v1.1.1 // indirect
 	github.com/hashicorp/golang-lru v0.5.5-0.20210104140557-80c98217689d // indirect
 	github.com/hashicorp/hcl v1.0.0 // indirect
 	github.com/holiman/bloomfilter/v2 v2.0.3 // indirect
 	github.com/holiman/uint256 v1.2.0 // indirect
-	github.com/huin/goupnp v1.0.3-0.20220313090229-ca81a64b4204 // indirect
+	github.com/huin/goupnp v1.0.3 // indirect
 	github.com/inconshreveable/mousetrap v1.0.0 // indirect
 	github.com/ipfs/bbloom v0.0.4 // indirect
-	github.com/ipfs/go-bitswap v0.4.0 // indirect
+	github.com/ipfs/go-bitfield v1.0.0 // indirect
+	github.com/ipfs/go-bitswap v0.8.0 // indirect
 	github.com/ipfs/go-block-format v0.0.3 // indirect
-	github.com/ipfs/go-blockservice v0.1.7 // indirect
-	github.com/ipfs/go-cid v0.0.7 // indirect
-	github.com/ipfs/go-cidutil v0.0.2 // indirect
-	github.com/ipfs/go-datastore v0.4.6 // indirect
-	github.com/ipfs/go-ds-measure v0.1.0 // indirect
-	github.com/ipfs/go-fetcher v1.5.0 // indirect
-	github.com/ipfs/go-filestore v1.0.0 // indirect
+	github.com/ipfs/go-blockservice v0.4.0 // indirect
+	github.com/ipfs/go-cid v0.2.0 // indirect
+	github.com/ipfs/go-cidutil v0.1.0 // indirect
+	github.com/ipfs/go-datastore v0.5.1 // indirect
+	github.com/ipfs/go-delegated-routing v0.3.0 // indirect
+	github.com/ipfs/go-ds-measure v0.2.0 // indirect
+	github.com/ipfs/go-fetcher v1.6.1 // indirect
+	github.com/ipfs/go-filestore v1.2.0 // indirect
 	github.com/ipfs/go-fs-lock v0.0.7 // indirect
-	github.com/ipfs/go-graphsync v0.8.0 // indirect
-	github.com/ipfs/go-ipfs v0.10.0 // indirect
-	github.com/ipfs/go-ipfs-blockstore v1.0.1 // indirect
+	github.com/ipfs/go-graphsync v0.13.1 // indirect
+	github.com/ipfs/go-ipfs-blockstore v1.2.0 // indirect
 	github.com/ipfs/go-ipfs-chunker v0.0.5 // indirect
-	github.com/ipfs/go-ipfs-config v0.16.0 // indirect
 	github.com/ipfs/go-ipfs-delay v0.0.1 // indirect
-	github.com/ipfs/go-ipfs-ds-help v1.0.0 // indirect
-	github.com/ipfs/go-ipfs-exchange-interface v0.0.1 // indirect
-	github.com/ipfs/go-ipfs-exchange-offline v0.0.1 // indirect
-	github.com/ipfs/go-ipfs-files v0.0.8 // indirect
+	github.com/ipfs/go-ipfs-ds-help v1.1.0 // indirect
+	github.com/ipfs/go-ipfs-exchange-interface v0.2.0 // indirect
+	github.com/ipfs/go-ipfs-exchange-offline v0.3.0 // indirect
+	github.com/ipfs/go-ipfs-files v0.1.1 // indirect
 	github.com/ipfs/go-ipfs-keystore v0.0.2 // indirect
-	github.com/ipfs/go-ipfs-pinner v0.1.2 // indirect
+	github.com/ipfs/go-ipfs-pinner v0.2.1 // indirect
 	github.com/ipfs/go-ipfs-posinfo v0.0.1 // indirect
 	github.com/ipfs/go-ipfs-pq v0.0.2 // indirect
-	github.com/ipfs/go-ipfs-provider v0.6.1 // indirect
-	github.com/ipfs/go-ipfs-routing v0.1.0 // indirect
+	github.com/ipfs/go-ipfs-provider v0.7.1 // indirect
+	github.com/ipfs/go-ipfs-routing v0.2.1 // indirect
 	github.com/ipfs/go-ipfs-util v0.0.2 // indirect
 	github.com/ipfs/go-ipld-cbor v0.0.5 // indirect
-	github.com/ipfs/go-ipld-format v0.2.0 // indirect
-	github.com/ipfs/go-ipld-legacy v0.1.0 // indirect
+	github.com/ipfs/go-ipld-format v0.4.0 // indirect
+	github.com/ipfs/go-ipld-legacy v0.1.1 // indirect
 	github.com/ipfs/go-ipns v0.1.2 // indirect
 	github.com/ipfs/go-log v1.0.5 // indirect
-	github.com/ipfs/go-log/v2 v2.3.0 // indirect
-	github.com/ipfs/go-merkledag v0.4.0 // indirect
+	github.com/ipfs/go-log/v2 v2.5.1 // indirect
+	github.com/ipfs/go-merkledag v0.6.0 // indirect
 	github.com/ipfs/go-metrics-interface v0.0.1 // indirect
-	github.com/ipfs/go-mfs v0.1.2 // indirect
-	github.com/ipfs/go-namesys v0.3.1 // indirect
-	github.com/ipfs/go-path v0.1.2 // indirect
-	github.com/ipfs/go-peertaskqueue v0.4.0 // indirect
-	github.com/ipfs/go-unixfs v0.2.5 // indirect
-	github.com/ipfs/go-unixfsnode v1.1.3 // indirect
+	github.com/ipfs/go-mfs v0.2.1 // indirect
+	github.com/ipfs/go-namesys v0.5.0 // indirect
+	github.com/ipfs/go-path v0.3.0 // indirect
+	github.com/ipfs/go-peertaskqueue v0.7.1 // indirect
+	github.com/ipfs/go-unixfs v0.4.0 // indirect
+	github.com/ipfs/go-unixfsnode v1.4.0 // indirect
 	github.com/ipfs/go-verifcid v0.0.1 // indirect
-	github.com/ipfs/interface-go-ipfs-core v0.5.1 // indirect
-	github.com/ipld/go-codec-dagpb v1.3.0 // indirect
-	github.com/ipld/go-ipld-prime v0.12.2 // indirect
+	github.com/ipfs/interface-go-ipfs-core v0.7.0 // indirect
+	github.com/ipfs/kubo v0.14.0 // indirect
+	github.com/ipld/edelweiss v0.1.4 // indirect
+	github.com/ipld/go-codec-dagpb v1.4.0 // indirect
+	github.com/ipld/go-ipld-prime v0.17.0 // indirect
 	github.com/jackc/chunkreader/v2 v2.0.1 // indirect
-	github.com/jackc/pgconn v1.10.0 // indirect
+	github.com/jackc/pgconn v1.12.1 // indirect
 	github.com/jackc/pgio v1.0.0 // indirect
 	github.com/jackc/pgpassfile v1.0.0 // indirect
-	github.com/jackc/pgproto3/v2 v2.1.1 // indirect
+	github.com/jackc/pgproto3/v2 v2.3.0 // indirect
 	github.com/jackc/pgservicefile v0.0.0-20200714003250-2b9c44734f2b // indirect
-	github.com/jackc/pgtype v1.8.1 // indirect
-	github.com/jackc/pgx/v4 v4.13.0 // indirect
-	github.com/jackc/puddle v1.1.3 // indirect
+	github.com/jackc/pgtype v1.11.0 // indirect
+	github.com/jackc/pgx/v4 v4.16.1 // indirect
+	github.com/jackc/puddle v1.2.1 // indirect
 	github.com/jackpal/go-nat-pmp v1.0.2 // indirect
 	github.com/jbenet/go-temp-err-catcher v0.1.0 // indirect
 	github.com/jbenet/goprocess v0.1.4 // indirect
-	github.com/klauspost/compress v1.11.7 // indirect
-	github.com/klauspost/cpuid/v2 v2.0.9 // indirect
+	github.com/jinzhu/inflection v1.0.0 // indirect
+	github.com/jinzhu/now v1.1.4 // indirect
+	github.com/klauspost/compress v1.15.1 // indirect
+	github.com/klauspost/cpuid/v2 v2.0.12 // indirect
 	github.com/koron/go-ssdp v0.0.2 // indirect
-	github.com/lib/pq v1.10.5 // indirect
-	github.com/libp2p/go-addr-util v0.1.0 // indirect
+	github.com/lib/pq v1.10.6 // indirect
 	github.com/libp2p/go-buffer-pool v0.0.2 // indirect
 	github.com/libp2p/go-cidranger v1.1.0 // indirect
-	github.com/libp2p/go-conn-security-multistream v0.2.1 // indirect
-	github.com/libp2p/go-doh-resolver v0.3.1 // indirect
+	github.com/libp2p/go-doh-resolver v0.4.0 // indirect
 	github.com/libp2p/go-eventbus v0.2.1 // indirect
 	github.com/libp2p/go-flow-metrics v0.0.3 // indirect
-	github.com/libp2p/go-libp2p v0.15.0 // indirect
-	github.com/libp2p/go-libp2p-asn-util v0.0.0-20200825225859-85005c6cf052 // indirect
-	github.com/libp2p/go-libp2p-autonat v0.4.2 // indirect
-	github.com/libp2p/go-libp2p-blankhost v0.2.0 // indirect
-	github.com/libp2p/go-libp2p-circuit v0.4.0 // indirect
-	github.com/libp2p/go-libp2p-connmgr v0.2.4 // indirect
-	github.com/libp2p/go-libp2p-core v0.9.0 // indirect
-	github.com/libp2p/go-libp2p-discovery v0.5.1 // indirect
-	github.com/libp2p/go-libp2p-kad-dht v0.13.1 // indirect
+	github.com/libp2p/go-libp2p v0.20.3 // indirect
+	github.com/libp2p/go-libp2p-asn-util v0.2.0 // indirect
+	github.com/libp2p/go-libp2p-core v0.16.1 // indirect
+	github.com/libp2p/go-libp2p-discovery v0.7.0 // indirect
+	github.com/libp2p/go-libp2p-kad-dht v0.16.0 // indirect
 	github.com/libp2p/go-libp2p-kbucket v0.4.7 // indirect
 	github.com/libp2p/go-libp2p-loggables v0.1.0 // indirect
-	github.com/libp2p/go-libp2p-mplex v0.4.1 // indirect
-	github.com/libp2p/go-libp2p-nat v0.0.6 // indirect
-	github.com/libp2p/go-libp2p-noise v0.2.2 // indirect
-	github.com/libp2p/go-libp2p-peerstore v0.2.8 // indirect
-	github.com/libp2p/go-libp2p-pnet v0.2.0 // indirect
-	github.com/libp2p/go-libp2p-pubsub v0.5.4 // indirect
-	github.com/libp2p/go-libp2p-pubsub-router v0.4.0 // indirect
-	github.com/libp2p/go-libp2p-quic-transport v0.12.0 // indirect
+	github.com/libp2p/go-libp2p-peerstore v0.6.0 // indirect
+	github.com/libp2p/go-libp2p-pubsub v0.6.1 // indirect
+	github.com/libp2p/go-libp2p-pubsub-router v0.5.0 // indirect
 	github.com/libp2p/go-libp2p-record v0.1.3 // indirect
+	github.com/libp2p/go-libp2p-resource-manager v0.3.0 // indirect
 	github.com/libp2p/go-libp2p-routing-helpers v0.2.3 // indirect
-	github.com/libp2p/go-libp2p-swarm v0.5.3 // indirect
-	github.com/libp2p/go-libp2p-tls v0.2.0 // indirect
-	github.com/libp2p/go-libp2p-transport-upgrader v0.4.6 // indirect
-	github.com/libp2p/go-libp2p-xor v0.0.0-20210714161855-5c005aca55db // indirect
-	github.com/libp2p/go-libp2p-yamux v0.5.4 // indirect
-	github.com/libp2p/go-maddr-filter v0.1.0 // indirect
-	github.com/libp2p/go-mplex v0.3.0 // indirect
-	github.com/libp2p/go-msgio v0.0.6 // indirect
-	github.com/libp2p/go-nat v0.0.5 // indirect
-	github.com/libp2p/go-netroute v0.1.6 // indirect
+	github.com/libp2p/go-libp2p-swarm v0.11.0 // indirect
+	github.com/libp2p/go-libp2p-xor v0.1.0 // indirect
+	github.com/libp2p/go-mplex v0.7.0 // indirect
+	github.com/libp2p/go-msgio v0.2.0 // indirect
+	github.com/libp2p/go-nat v0.1.0 // indirect
+	github.com/libp2p/go-netroute v0.2.0 // indirect
 	github.com/libp2p/go-openssl v0.0.7 // indirect
-	github.com/libp2p/go-reuseport v0.0.2 // indirect
-	github.com/libp2p/go-reuseport-transport v0.0.5 // indirect
-	github.com/libp2p/go-sockaddr v0.1.1 // indirect
-	github.com/libp2p/go-stream-muxer-multistream v0.3.0 // indirect
-	github.com/libp2p/go-tcp-transport v0.2.8 // indirect
-	github.com/libp2p/go-ws-transport v0.5.0 // indirect
-	github.com/libp2p/go-yamux/v2 v2.2.0 // indirect
-	github.com/libp2p/zeroconf/v2 v2.0.0 // indirect
-	github.com/lucas-clemente/quic-go v0.23.0 // indirect
+	github.com/libp2p/go-reuseport v0.2.0 // indirect
+	github.com/libp2p/go-yamux/v3 v3.1.2 // indirect
+	github.com/libp2p/zeroconf/v2 v2.1.1 // indirect
+	github.com/lucas-clemente/quic-go v0.27.1 // indirect
 	github.com/magiconair/properties v1.8.6 // indirect
 	github.com/mailgun/groupcache/v2 v2.3.0 // indirect
-	github.com/marten-seemann/qtls-go1-16 v0.1.4 // indirect
-	github.com/marten-seemann/qtls-go1-17 v0.1.0 // indirect
+	github.com/marten-seemann/qtls-go1-16 v0.1.5 // indirect
+	github.com/marten-seemann/qtls-go1-17 v0.1.1 // indirect
+	github.com/marten-seemann/qtls-go1-18 v0.1.1 // indirect
 	github.com/marten-seemann/tcp v0.0.0-20210406111302-dfbc87cc63fd // indirect
 	github.com/mattn/go-colorable v0.1.12 // indirect
 	github.com/mattn/go-isatty v0.0.14 // indirect
 	github.com/mattn/go-runewidth v0.0.9 // indirect
 	github.com/matttproud/golang_protobuf_extensions v1.0.1 // indirect
-	github.com/miekg/dns v1.1.43 // indirect
+	github.com/miekg/dns v1.1.48 // indirect
 	github.com/mikioh/tcpinfo v0.0.0-20190314235526-30a79bb1804b // indirect
 	github.com/mikioh/tcpopt v0.0.0-20190314235656-172688c1accc // indirect
-	github.com/minio/blake2b-simd v0.0.0-20160723061019-3f5f724cb5b1 // indirect
 	github.com/minio/sha256-simd v1.0.0 // indirect
 	github.com/mitchellh/go-homedir v1.1.0 // indirect
 	github.com/mitchellh/mapstructure v1.4.3 // indirect
 	github.com/mitchellh/pointerstructure v1.2.0 // indirect
 	github.com/mr-tron/base58 v1.2.0 // indirect
-	github.com/multiformats/go-base32 v0.0.3 // indirect
+	github.com/multiformats/go-base32 v0.0.4 // indirect
 	github.com/multiformats/go-base36 v0.1.0 // indirect
-	github.com/multiformats/go-multiaddr v0.4.0 // indirect
+	github.com/multiformats/go-multiaddr v0.5.0 // indirect
 	github.com/multiformats/go-multiaddr-dns v0.3.1 // indirect
 	github.com/multiformats/go-multiaddr-fmt v0.1.0 // indirect
-	github.com/multiformats/go-multibase v0.0.3 // indirect
-	github.com/multiformats/go-multicodec v0.3.0 // indirect
-	github.com/multiformats/go-multihash v0.1.0 // indirect
-	github.com/multiformats/go-multistream v0.2.2 // indirect
+	github.com/multiformats/go-multibase v0.1.0 // indirect
+	github.com/multiformats/go-multicodec v0.5.0 // indirect
+	github.com/multiformats/go-multihash v0.2.0 // indirect
+	github.com/multiformats/go-multistream v0.3.3 // indirect
 	github.com/multiformats/go-varint v0.0.6 // indirect
 	github.com/nxadm/tail v1.4.8 // indirect
 	github.com/olekukonko/tablewriter v0.0.5 // indirect
+	github.com/opencontainers/runtime-spec v1.0.2 // indirect
 	github.com/opentracing/opentracing-go v1.2.0 // indirect
+	github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58 // indirect
 	github.com/pelletier/go-toml v1.9.4 // indirect
 	github.com/pelletier/go-toml/v2 v2.0.0-beta.8 // indirect
 	github.com/pganalyze/pg_query_go/v2 v2.1.0 // indirect
 	github.com/pkg/errors v0.9.1 // indirect
 	github.com/pmezard/go-difflib v1.0.0 // indirect
 	github.com/polydawn/refmt v0.0.0-20201211092308-30ac6d18308e // indirect
-	github.com/prometheus/client_golang v1.11.0 // indirect
 	github.com/prometheus/client_model v0.2.0 // indirect
-	github.com/prometheus/common v0.30.0 // indirect
+	github.com/prometheus/common v0.33.0 // indirect
 	github.com/prometheus/procfs v0.7.3 // indirect
-	github.com/prometheus/tsdb v0.7.1 // indirect
+	github.com/prometheus/tsdb v0.10.0 // indirect
+	github.com/raulk/clock v1.1.0 // indirect
+	github.com/raulk/go-watchdog v1.2.0 // indirect
 	github.com/rjeczalik/notify v0.9.1 // indirect
 	github.com/rs/cors v1.7.0 // indirect
+	github.com/russross/blackfriday/v2 v2.1.0 // indirect
 	github.com/segmentio/fasthash v1.0.3 // indirect
-	github.com/shirou/gopsutil v3.21.5+incompatible // indirect
+	github.com/shirou/gopsutil v3.21.11+incompatible // indirect
 	github.com/spacemonkeygo/spacelog v0.0.0-20180420211403-2296661a0572 // indirect
 	github.com/spaolacci/murmur3 v1.1.0 // indirect
 	github.com/spf13/afero v1.8.2 // indirect
@@ -226,48 +226,55 @@ require (
 	github.com/spf13/pflag v1.0.5 // indirect
 	github.com/status-im/keycard-go v0.0.0-20190316090335-8537d3370df4 // indirect
 	github.com/stretchr/objx v0.2.0 // indirect
-	github.com/stretchr/testify v1.7.1 // indirect
+	github.com/stretchr/testify v1.7.2 // indirect
 	github.com/subosito/gotenv v1.2.0 // indirect
-	github.com/syndtr/goleveldb v1.0.1-0.20210819022825-2ae1ddf74ef7 // indirect
+	github.com/syndtr/goleveldb v1.0.1-0.20220614013038-64ee5596c38a // indirect
 	github.com/thoas/go-funk v0.9.2 // indirect
+	github.com/tidwall/gjson v1.14.0 // indirect
+	github.com/tidwall/match v1.1.1 // indirect
+	github.com/tidwall/pretty v1.2.0 // indirect
 	github.com/tklauser/go-sysconf v0.3.6 // indirect
 	github.com/tklauser/numcpus v0.2.2 // indirect
 	github.com/tyler-smith/go-bip39 v1.0.1-0.20181017060643-dbb3b84ba2ef // indirect
-	github.com/vulcanize/eth-ipfs-state-validator/v4 v4.0.0-alpha // indirect
+	github.com/urfave/cli/v2 v2.10.2 // indirect
+	github.com/vulcanize/eth-ipfs-state-validator/v4 v4.0.7-alpha // indirect
+	github.com/vulcanize/go-eth-state-node-iterator v1.1.4 // indirect
+	github.com/wI2L/jsondiff v0.2.0 // indirect
 	github.com/whyrusleeping/base32 v0.0.0-20170828182744-c30ac30633cc // indirect
 	github.com/whyrusleeping/cbor-gen v0.0.0-20210219115102-f37d292932f2 // indirect
 	github.com/whyrusleeping/chunker v0.0.0-20181014151217-fe64bd25879f // indirect
 	github.com/whyrusleeping/go-keyspace v0.0.0-20160322163242-5b898ac5add1 // indirect
-	github.com/whyrusleeping/mdns v0.0.0-20190826153040-b9b60ed33aa9 // indirect
 	github.com/whyrusleeping/multiaddr-filter v0.0.0-20160516205228-e903e4adabd7 // indirect
 	github.com/whyrusleeping/timecache v0.0.0-20160911033111-cfcb2f1abfee // indirect
+	github.com/xrash/smetrics v0.0.0-20201216005158-039620a65673 // indirect
+	github.com/yusufpapurcu/wmi v1.2.2 // indirect
 	go.opencensus.io v0.23.0 // indirect
-	go.opentelemetry.io/otel v0.20.0 // indirect
-	go.opentelemetry.io/otel/metric v0.20.0 // indirect
-	go.opentelemetry.io/otel/trace v0.20.0 // indirect
+	go.opentelemetry.io/otel v1.7.0 // indirect
+	go.opentelemetry.io/otel/trace v1.7.0 // indirect
 	go.uber.org/atomic v1.9.0 // indirect
-	go.uber.org/dig v1.10.0 // indirect
-	go.uber.org/fx v1.13.1 // indirect
-	go.uber.org/multierr v1.7.0 // indirect
-	go.uber.org/zap v1.19.0 // indirect
+	go.uber.org/dig v1.14.0 // indirect
+	go.uber.org/fx v1.16.0 // indirect
+	go.uber.org/multierr v1.8.0 // indirect
+	go.uber.org/zap v1.21.0 // indirect
 	go4.org v0.0.0-20200411211856-f5505b9728dd // indirect
-	golang.org/x/crypto v0.0.0-20220411220226-7b82a4e95df4 // indirect
-	golang.org/x/mod v0.6.0-dev.0.20211013180041-c96bc1413d57 // indirect
-	golang.org/x/net v0.0.0-20220412020605-290c469a71a5 // indirect
+	golang.org/x/crypto v0.0.0-20220829220503-c86fa9a7ed90 // indirect
+	golang.org/x/mod v0.6.0-dev.0.20220106191415-9b9b3d81d5e3 // indirect
+	golang.org/x/net v0.0.0-20220607020251-c690dde0001d // indirect
 	golang.org/x/sync v0.0.0-20210220032951-036812b2e83c // indirect
-	golang.org/x/sys v0.0.0-20220412211240-33da011f77ad // indirect
+	golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a // indirect
 	golang.org/x/text v0.3.7 // indirect
 	golang.org/x/time v0.0.0-20210220033141-f8bda1e9f3ba // indirect
-	golang.org/x/tools v0.1.8 // indirect
-	golang.org/x/xerrors v0.0.0-20220411194840-2f41105eb62f // indirect
+	golang.org/x/tools v0.1.10 // indirect
+	golang.org/x/xerrors v0.0.0-20220517211312-f3a8303e98df // indirect
 	google.golang.org/protobuf v1.28.0 // indirect
 	gopkg.in/ini.v1 v1.66.4 // indirect
 	gopkg.in/natefinch/npipe.v2 v2.0.0-20160621034901-c1b8fa8bdcce // indirect
 	gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 // indirect
-	gopkg.in/urfave/cli.v1 v1.20.0 // indirect
 	gopkg.in/yaml.v2 v2.4.0 // indirect
-	gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect
-	lukechampine.com/blake3 v1.1.6 // indirect
+	gopkg.in/yaml.v3 v3.0.1 // indirect
+	gorm.io/driver/postgres v1.3.7 // indirect
+	gorm.io/gorm v1.23.5 // indirect
+	lukechampine.com/blake3 v1.1.7 // indirect
 )

-replace github.com/ethereum/go-ethereum v1.10.17 => github.com/vulcanize/go-ethereum v1.10.17-statediff-4.0.1-alpha
+replace github.com/ethereum/go-ethereum v1.10.23 => github.com/vulcanize/go-ethereum v1.10.23-statediff-4.2.0-alpha
@@ -1,3 +1,19 @@
+// VulcanizeDB
+// Copyright © 2022 Vulcanize
+
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU Affero General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU Affero General Public License for more details.
+
+// You should have received a copy of the GNU Affero General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 package main

 import "github.com/vulcanize/ipld-eth-db-validator/cmd"
@@ -0,0 +1,157 @@
+// VulcanizeDB
+// Copyright © 2022 Vulcanize
+
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU Affero General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU Affero General Public License for more details.
+
+// You should have received a copy of the GNU Affero General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package prom
+
+import (
+	"database/sql"
+
+	"github.com/prometheus/client_golang/prometheus"
+)
+
+// DBStatsGetter is an interface that gets sql.DBStats.
+type DBStatsGetter interface {
+	Stats() sql.DBStats
+}
+
+// DBStatsCollector implements the prometheus.Collector interface.
+type DBStatsCollector struct {
+	sg DBStatsGetter
+
+	// descriptions of exported metrics
+	maxOpenDesc           *prometheus.Desc
+	openDesc              *prometheus.Desc
+	inUseDesc             *prometheus.Desc
+	idleDesc              *prometheus.Desc
+	waitedForDesc         *prometheus.Desc
+	blockedSecondsDesc    *prometheus.Desc
+	closedMaxIdleDesc     *prometheus.Desc
+	closedMaxLifetimeDesc *prometheus.Desc
+}
+
+// NewDBStatsCollector creates a new DBStatsCollector.
+func NewDBStatsCollector(dbName string, sg DBStatsGetter) *DBStatsCollector {
+	labels := prometheus.Labels{"db_name": dbName}
+	return &DBStatsCollector{
+		sg: sg,
+		maxOpenDesc: prometheus.NewDesc(
+			prometheus.BuildFQName(namespace, connSubsystem, "max_open"),
+			"Maximum number of open connections to the database.",
+			nil,
+			labels,
+		),
+		openDesc: prometheus.NewDesc(
+			prometheus.BuildFQName(namespace, connSubsystem, "open"),
+			"The number of established connections both in use and idle.",
+			nil,
+			labels,
+		),
+		inUseDesc: prometheus.NewDesc(
+			prometheus.BuildFQName(namespace, connSubsystem, "in_use"),
+			"The number of connections currently in use.",
+			nil,
+			labels,
+		),
+		idleDesc: prometheus.NewDesc(
+			prometheus.BuildFQName(namespace, connSubsystem, "idle"),
+			"The number of idle connections.",
+			nil,
+			labels,
+		),
+		waitedForDesc: prometheus.NewDesc(
+			prometheus.BuildFQName(namespace, connSubsystem, "waited_for"),
+			"The total number of connections waited for.",
+			nil,
+			labels,
+		),
+		blockedSecondsDesc: prometheus.NewDesc(
+			prometheus.BuildFQName(namespace, connSubsystem, "blocked_seconds"),
+			"The total time blocked waiting for a new connection.",
+			nil,
+			labels,
+		),
+		closedMaxIdleDesc: prometheus.NewDesc(
+			prometheus.BuildFQName(namespace, connSubsystem, "closed_max_idle"),
+			"The total number of connections closed due to SetMaxIdleConns.",
+			nil,
+			labels,
+		),
+		closedMaxLifetimeDesc: prometheus.NewDesc(
+			prometheus.BuildFQName(namespace, connSubsystem, "closed_max_lifetime"),
+			"The total number of connections closed due to SetConnMaxLifetime.",
+			nil,
+			labels,
+		),
+	}
+}
+
+// Describe implements the prometheus.Collector interface.
+func (c DBStatsCollector) Describe(ch chan<- *prometheus.Desc) {
+	ch <- c.maxOpenDesc
+	ch <- c.openDesc
+	ch <- c.inUseDesc
+	ch <- c.idleDesc
+	ch <- c.waitedForDesc
+	ch <- c.blockedSecondsDesc
+	ch <- c.closedMaxIdleDesc
+	ch <- c.closedMaxLifetimeDesc
+}
+
+// Collect implements the prometheus.Collector interface.
+func (c DBStatsCollector) Collect(ch chan<- prometheus.Metric) {
+	stats := c.sg.Stats()
+
+	ch <- prometheus.MustNewConstMetric(
+		c.maxOpenDesc,
+		prometheus.GaugeValue,
+		float64(stats.MaxOpenConnections),
+	)
+	ch <- prometheus.MustNewConstMetric(
+		c.openDesc,
+		prometheus.GaugeValue,
+		float64(stats.OpenConnections),
+	)
+	ch <- prometheus.MustNewConstMetric(
+		c.inUseDesc,
+		prometheus.GaugeValue,
+		float64(stats.InUse),
+	)
+	ch <- prometheus.MustNewConstMetric(
+		c.idleDesc,
+		prometheus.GaugeValue,
+		float64(stats.Idle),
+	)
+	ch <- prometheus.MustNewConstMetric(
+		c.waitedForDesc,
+		prometheus.CounterValue,
+		float64(stats.WaitCount),
+	)
+	ch <- prometheus.MustNewConstMetric(
+		c.blockedSecondsDesc,
+		prometheus.CounterValue,
+		stats.WaitDuration.Seconds(),
+	)
+	ch <- prometheus.MustNewConstMetric(
+		c.closedMaxIdleDesc,
+		prometheus.CounterValue,
+		float64(stats.MaxIdleClosed),
+	)
+	ch <- prometheus.MustNewConstMetric(
+		c.closedMaxLifetimeDesc,
+		prometheus.CounterValue,
+		float64(stats.MaxLifetimeClosed),
+	)
+}
@@ -0,0 +1,59 @@
+// VulcanizeDB
+// Copyright © 2022 Vulcanize
+
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU Affero General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU Affero General Public License for more details.
+
+// You should have received a copy of the GNU Affero General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package prom
+
+import (
+	"github.com/prometheus/client_golang/prometheus"
+	"github.com/prometheus/client_golang/prometheus/promauto"
+)
+
+const (
+	namespace = "ipld_eth_state_snapshot"
+
+	connSubsystem  = "connections"
+	statsSubsystem = "stats"
+)
+
+var (
+	metrics            bool
+	lastValidatedBlock prometheus.Gauge
+)
+
+func Init() {
+	metrics = true
+
+	lastValidatedBlock = promauto.NewGauge(prometheus.GaugeOpts{
+		Namespace: namespace,
+		Subsystem: statsSubsystem,
+		Name:      "last_validated_block",
+		Help:      "Last validated block number",
+	})
+}
+
+// RegisterDBCollector create metric collector for given connection
+func RegisterDBCollector(name string, db DBStatsGetter) {
+	if metrics {
+		prometheus.Register(NewDBStatsCollector(name, db))
+	}
+}
+
+// SetLastValidatedBlock sets the last validated block number
+func SetLastValidatedBlock(blockNumber float64) {
+	if metrics {
+		lastValidatedBlock.Set(blockNumber)
+	}
+}
@@ -0,0 +1,47 @@
+// VulcanizeDB
+// Copyright © 2022 Vulcanize
+
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU Affero General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU Affero General Public License for more details.
+
+// You should have received a copy of the GNU Affero General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package prom
+
+import (
+	"errors"
+	"net/http"
+
+	"github.com/prometheus/client_golang/prometheus/promhttp"
+	"github.com/sirupsen/logrus"
+)
+
+var errPromHTTP = errors.New("can't start http server for prometheus")
+
+// Serve start listening http
+func Serve(addr string) *http.Server {
+	mux := http.NewServeMux()
+	mux.Handle("/metrics", promhttp.Handler())
+	srv := http.Server{
+		Addr:    addr,
+		Handler: mux,
+	}
+	go func() {
+		if err := srv.ListenAndServe(); err != nil {
+			logrus.
+				WithError(err).
+				WithField("module", "prom").
+				WithField("addr", addr).
+				Fatal(errPromHTTP)
+		}
+	}()
+	return &srv
+}
@@ -1,3 +1,19 @@
+// VulcanizeDB
+// Copyright © 2022 Vulcanize
+
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU Affero General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU Affero General Public License for more details.
+
+// You should have received a copy of the GNU Affero General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 package validator

 import (
@@ -6,21 +22,14 @@ import (
 	"time"

 	"github.com/ethereum/go-ethereum/params"
+	"github.com/ethereum/go-ethereum/rpc"
+	"github.com/ethereum/go-ethereum/statediff"
 	"github.com/ethereum/go-ethereum/statediff/indexer/database/sql/postgres"
 	"github.com/jmoiron/sqlx"
 	"github.com/spf13/viper"
 	"github.com/vulcanize/ipld-eth-server/v4/pkg/shared"
-)

-var (
-	DATABASE_NAME                 = "DATABASE_NAME"
-	DATABASE_HOSTNAME             = "DATABASE_HOSTNAME"
-	DATABASE_PORT                 = "DATABASE_PORT"
-	DATABASE_USER                 = "DATABASE_USER"
-	DATABASE_PASSWORD             = "DATABASE_PASSWORD"
-	DATABASE_MAX_IDLE_CONNECTIONS = "DATABASE_MAX_IDLE_CONNECTIONS"
-	DATABASE_MAX_OPEN_CONNECTIONS = "DATABASE_MAX_OPEN_CONNECTIONS"
-	DATABASE_MAX_CONN_LIFETIME    = "DATABASE_MAX_CONN_LIFETIME"
+	"github.com/vulcanize/ipld-eth-db-validator/pkg/prom"
 )

 var IntegrationTestChainConfig = &params.ChainConfig{
@@ -59,23 +68,37 @@ var TestChainConfig = &params.ChainConfig{
 type Config struct {
 	dbConfig postgres.Config
 	DB       *sqlx.DB
+
+	ChainCfg              *params.ChainConfig
+	Client                *rpc.Client
+	StateDiffMissingBlock bool
+	StateDiffTimeout      uint
+
+	BlockNum, Trail uint64
+	SleepInterval   uint
 }

 func NewConfig() (*Config, error) {
 	cfg := new(Config)
-	return cfg, cfg.setupDB()
+	err := cfg.setupDB()
+	if err != nil {
+		return nil, err
+	}
+
+	err = cfg.setupEth()
+	if err != nil {
+		return nil, err
+	}
+
+	err = cfg.setupValidator()
+	if err != nil {
+		return nil, err
+	}
+
+	return cfg, nil
 }

 func (c *Config) setupDB() error {
-	_ = viper.BindEnv("database.name", DATABASE_NAME)
-	_ = viper.BindEnv("database.hostname", DATABASE_HOSTNAME)
-	_ = viper.BindEnv("database.port", DATABASE_PORT)
-	_ = viper.BindEnv("database.user", DATABASE_USER)
-	_ = viper.BindEnv("database.password", DATABASE_PASSWORD)
-	_ = viper.BindEnv("database.maxIdle", DATABASE_MAX_IDLE_CONNECTIONS)
-	_ = viper.BindEnv("database.maxOpen", DATABASE_MAX_OPEN_CONNECTIONS)
-	_ = viper.BindEnv("database.maxLifetime", DATABASE_MAX_CONN_LIFETIME)
-
 	// DB Config
 	c.dbConfig.DatabaseName = viper.GetString("database.name")
 	c.dbConfig.Hostname = viper.GetString("database.hostname")
@@ -92,7 +115,53 @@ func (c *Config) setupDB() error {
 	if err != nil {
 		return fmt.Errorf("failed to create config: %w", err)
 	}
-
 	c.DB = db
+
+	// Enable DB stats
+	if viper.GetBool("prom.dbStats") {
+		prom.RegisterDBCollector(c.dbConfig.DatabaseName, c.DB)
+	}
+
 	return nil
 }
+
+func (c *Config) setupEth() error {
+	var err error
+	chainConfigPath := viper.GetString("ethereum.chainConfig")
+	if chainConfigPath != "" {
+		c.ChainCfg, err = statediff.LoadConfig(chainConfigPath)
+	} else {
+		// read chainID if chain config path not provided
+		chainID := viper.GetUint64("ethereum.chainID")
+		c.ChainCfg, err = statediff.ChainConfig(chainID)
+	}
+	if err != nil {
+		return err
+	}
+
+	// setup a statediffing client
+	ethHTTP := viper.GetString("ethereum.httpPath")
+	if ethHTTP != "" {
+		ethHTTPEndpoint := fmt.Sprintf("http://%s", ethHTTP)
+		c.Client, err = rpc.Dial(ethHTTPEndpoint)
+	}
+
+	return err
+}
+
+func (c *Config) setupValidator() error {
+	var err error
+	c.BlockNum = viper.GetUint64("validate.blockHeight")
+	if c.BlockNum < 1 {
+		return fmt.Errorf("block height cannot be less the 1")
+	}
+
+	c.Trail = viper.GetUint64("validate.trail")
+	c.SleepInterval = viper.GetUint("validate.sleepInterval")
+	c.StateDiffMissingBlock = viper.GetBool("validate.stateDiffMissingBlock")
+	if c.StateDiffMissingBlock {
+		c.StateDiffTimeout = viper.GetUint("validate.stateDiffTimeout")
+	}
+
+	return err
+}
@@ -1,25 +1,29 @@
-// Copyright 2020 The go-ethereum Authors
-// This file is part of go-ethereum.
-//
-// go-ethereum is free software: you can redistribute it and/or modify
-// it under the terms of the GNU General Public License as published by
+// VulcanizeDB
+// Copyright © 2022 Vulcanize
+
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU Affero General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
-//
-// go-ethereum is distributed in the hope that it will be useful,
+
+// This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
-// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-// GNU General Public License for more details.
-//
-// You should have received a copy of the GNU General Public License
-// along with go-ethereum. If not, see <http://www.gnu.org/licenses/>.
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU Affero General Public License for more details.
+
+// You should have received a copy of the GNU Affero General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.

 package validator

 import (
+	"errors"
+
 	"github.com/ethereum/go-ethereum/ethdb"
 )

+var errNotSupported = errors.New("this operation is not supported")
+
 type database struct {
 	ethDB ethdb.Database
 }
@@ -121,7 +125,7 @@ func (d *database) NewBatchWithSize(size int) ethdb.Batch {
 	return d.ethDB.NewBatchWithSize(size)
 }

-func (d *database) ReadAncients(fn func(ethdb.AncientReader) error) (err error) {
+func (d *database) ReadAncients(fn func(ethdb.AncientReaderOp) error) (err error) {
 	return d.ethDB.ReadAncients(fn)
 }

@@ -133,3 +137,8 @@ func (d *database) Close() error {
 func (d *database) NewSnapshot() (ethdb.Snapshot, error) {
 	return d.NewSnapshot()
 }
+
+// NewSnapshot creates a database snapshot based on the current state.
+func (d *database) AncientDatadir() (string, error) {
+	return "", errNotSupported
+}
@@ -0,0 +1,239 @@
+// VulcanizeDB
+// Copyright © 2022 Vulcanize
+
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU Affero General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU Affero General Public License for more details.
+
+// You should have received a copy of the GNU Affero General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package validator
+
+import (
+	"fmt"
+
+	"github.com/jmoiron/sqlx"
+)
+
+// ValidateReferentialIntegrity validates referential integrity at the given height
+func ValidateReferentialIntegrity(db *sqlx.DB, blockNumber uint64) error {
+	err := ValidateHeaderCIDsRef(db, blockNumber)
+	if err != nil {
+		return err
+	}
+
+	err = ValidateUncleCIDsRef(db, blockNumber)
+	if err != nil {
+		return err
+	}
+
+	err = ValidateTransactionCIDsRef(db, blockNumber)
+	if err != nil {
+		return err
+	}
+
+	err = ValidateReceiptCIDsRef(db, blockNumber)
+	if err != nil {
+		return err
+	}
+
+	err = ValidateStateCIDsRef(db, blockNumber)
+	if err != nil {
+		return err
+	}
+
+	err = ValidateStorageCIDsRef(db, blockNumber)
+	if err != nil {
+		return err
+	}
+
+	err = ValidateStateAccountsRef(db, blockNumber)
+	if err != nil {
+		return err
+	}
+
+	err = ValidateAccessListElementsRef(db, blockNumber)
+	if err != nil {
+		return err
+	}
+
+	err = ValidateLogCIDsRef(db, blockNumber)
+	if err != nil {
+		return err
+	}
+
+	return nil
+}
+
+// ValidateHeaderCIDsRef does a reference integrity check on references in eth.header_cids table
+func ValidateHeaderCIDsRef(db *sqlx.DB, blockNumber uint64) error {
+	err := ValidateIPFSBlocks(db, blockNumber, "eth.header_cids", "mh_key")
+	if err != nil {
+		return err
+	}
+
+	return nil
+}
+
+// ValidateUncleCIDsRef does a reference integrity check on references in eth.uncle_cids table
+func ValidateUncleCIDsRef(db *sqlx.DB, blockNumber uint64) error {
+	var exists bool
+	err := db.Get(&exists, UncleCIDsRefHeaderCIDs, blockNumber)
+	if err != nil {
+		return err
+	}
+	if exists {
+		return fmt.Errorf(ReferentialIntegrityErr, blockNumber, "eth.header_cids")
+	}
+
+	err = ValidateIPFSBlocks(db, blockNumber, "eth.uncle_cids", "mh_key")
+	if err != nil {
+		return err
+	}
+
+	return nil
+}
+
+// ValidateTransactionCIDsRef does a reference integrity check on references in eth.header_cids table
+func ValidateTransactionCIDsRef(db *sqlx.DB, blockNumber uint64) error {
+	var exists bool
+	err := db.Get(&exists, TransactionCIDsRefHeaderCIDs, blockNumber)
+	if err != nil {
+		return err
+	}
+	if exists {
+		return fmt.Errorf(ReferentialIntegrityErr, blockNumber, "eth.header_cids")
+	}
+
+	err = ValidateIPFSBlocks(db, blockNumber, "eth.transaction_cids", "mh_key")
+	if err != nil {
+		return err
+	}
+
+	return nil
+}
+
+// ValidateReceiptCIDsRef does a reference integrity check on references in eth.receipt_cids table
+func ValidateReceiptCIDsRef(db *sqlx.DB, blockNumber uint64) error {
+	var exists bool
+	err := db.Get(&exists, ReceiptCIDsRefTransactionCIDs, blockNumber)
+	if err != nil {
+		return err
+	}
+	if exists {
+		return fmt.Errorf(ReferentialIntegrityErr, blockNumber, "eth.transaction_cids")
+	}
+
+	err = ValidateIPFSBlocks(db, blockNumber, "eth.receipt_cids", "leaf_mh_key")
+	if err != nil {
+		return err
+	}
+
+	return nil
+}
+
+// ValidateStateCIDsRef does a reference integrity check on references in eth.state_cids table
+func ValidateStateCIDsRef(db *sqlx.DB, blockNumber uint64) error {
+	var exists bool
+	err := db.Get(&exists, StateCIDsRefHeaderCIDs, blockNumber)
+	if err != nil {
+		return err
+	}
+	if exists {
+		return fmt.Errorf(ReferentialIntegrityErr, blockNumber, "eth.header_cids")
+	}
+
+	err = ValidateIPFSBlocks(db, blockNumber, "eth.state_cids", "mh_key")
+	if err != nil {
+		return err
+	}
+
+	return nil
+}
+
+// ValidateStorageCIDsRef does a reference integrity check on references in eth.storage_cids table
+func ValidateStorageCIDsRef(db *sqlx.DB, blockNumber uint64) error {
+	var exists bool
+	err := db.Get(&exists, StorageCIDsRefStateCIDs, blockNumber)
+	if err != nil {
+		return err
+	}
+	if exists {
+		return fmt.Errorf(ReferentialIntegrityErr, blockNumber, "eth.state_cids")
+	}
+
+	err = ValidateIPFSBlocks(db, blockNumber, "eth.storage_cids", "mh_key")
+	if err != nil {
+		return err
+	}
+
+	return nil
+}
+
+// ValidateStateAccountsRef does a reference integrity check on references in eth.state_accounts table
+func ValidateStateAccountsRef(db *sqlx.DB, blockNumber uint64) error {
+	var exists bool
+	err := db.Get(&exists, StateAccountsRefStateCIDs, blockNumber)
+	if err != nil {
+		return err
+	}
+	if exists {
+		return fmt.Errorf(ReferentialIntegrityErr, blockNumber, "eth.state_cids")
+	}
+
+	return nil
+}
+
+// ValidateAccessListElementsRef does a reference integrity check on references in eth.access_list_elements table
+func ValidateAccessListElementsRef(db *sqlx.DB, blockNumber uint64) error {
+	var exists bool
+	err := db.Get(&exists, AccessListElementsRefTransactionCIDs, blockNumber)
+	if err != nil {
+		return err
+	}
+	if exists {
+		return fmt.Errorf(ReferentialIntegrityErr, blockNumber, "eth.transaction_cids")
+	}
+
+	return nil
+}
+
+// ValidateLogCIDsRef does a reference integrity check on references in eth.log_cids table
+func ValidateLogCIDsRef(db *sqlx.DB, blockNumber uint64) error {
+	var exists bool
+	err := db.Get(&exists, LogCIDsRefReceiptCIDs, blockNumber)
+	if err != nil {
+		return err
+	}
+	if exists {
+		return fmt.Errorf(ReferentialIntegrityErr, blockNumber, "eth.receipt_cids")
+	}
+
+	err = ValidateIPFSBlocks(db, blockNumber, "eth.log_cids", "leaf_mh_key")
+	if err != nil {
+		return err
+	}
+
+	return nil
+}
+
+// ValidateIPFSBlocks does a reference integrity check between the given CID table and IPFS blocks table on MHKey and block number
+func ValidateIPFSBlocks(db *sqlx.DB, blockNumber uint64, CIDTable string, mhKeyField string) error {
+	var exists bool
+	err := db.Get(&exists, fmt.Sprintf(CIDsRefIPLDBlocks, CIDTable, mhKeyField), blockNumber)
+	if err != nil {
+		return err
+	}
+	if exists {
+		return fmt.Errorf(ReferentialIntegrityErr, blockNumber, "public.blocks")
+	}
+
+	return nil
+}
@@ -0,0 +1,137 @@
+// VulcanizeDB
+// Copyright © 2022 Vulcanize
+
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU Affero General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU Affero General Public License for more details.
+
+// You should have received a copy of the GNU Affero General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package validator
+
+// Queries to validate referential integrity in the indexed data:
+// At the given block number,
+// In each table, for each (would be) foreign key reference, perform left join with the referenced table on the foreign key fields.
+// Select rows where there are no matching rows in the referenced table.
+// If any such rows exist, there are missing entries in the referenced table.
+
+const (
+	CIDsRefIPLDBlocks = `SELECT EXISTS (
+						SELECT *
+						FROM %[1]s
+						LEFT JOIN public.blocks ON (
+							%[1]s.%[2]s = blocks.key
+							AND %[1]s.block_number = blocks.block_number
+						)
+						WHERE
+							%[1]s.block_number = $1
+							AND blocks.key IS NULL
+					)`
+
+	UncleCIDsRefHeaderCIDs = `SELECT EXISTS (
+						SELECT *
+						FROM eth.uncle_cids
+						LEFT JOIN eth.header_cids ON (
+							uncle_cids.header_id = header_cids.block_hash
+							AND uncle_cids.block_number = header_cids.block_number
+						)
+						WHERE
+							uncle_cids.block_number = $1
+							AND header_cids.block_hash IS NULL
+					)`
+
+	TransactionCIDsRefHeaderCIDs = `SELECT EXISTS (
+						SELECT *
+						FROM eth.transaction_cids
+						LEFT JOIN eth.header_cids ON (
+							transaction_cids.header_id = header_cids.block_hash
+							AND transaction_cids.block_number = header_cids.block_number
+						)
+						WHERE
+							transaction_cids.block_number = $1
+							AND header_cids.block_hash IS NULL
+					)`
+
+	ReceiptCIDsRefTransactionCIDs = `SELECT EXISTS (
+						SELECT *
+						FROM eth.receipt_cids
+						LEFT JOIN eth.transaction_cids ON (
+							receipt_cids.tx_id = transaction_cids.tx_hash
+							AND receipt_cids.header_id = transaction_cids.header_id
+							AND receipt_cids.block_number = transaction_cids.block_number
+						)
+						WHERE
+							receipt_cids.block_number = $1
+							AND transaction_cids.tx_hash IS NULL
+					)`
+
+	StateCIDsRefHeaderCIDs = `SELECT EXISTS (
+						SELECT *
+						FROM eth.state_cids
+						LEFT JOIN eth.header_cids ON (
+							state_cids.header_id = header_cids.block_hash
+							AND state_cids.block_number = header_cids.block_number
+						)
+						WHERE
+							state_cids.block_number = $1
+							AND header_cids.block_hash IS NULL
+					)`
+
+	StorageCIDsRefStateCIDs = `SELECT EXISTS (
+						SELECT *
+						FROM eth.storage_cids
+						LEFT JOIN eth.state_cids ON (
+							storage_cids.state_path = state_cids.state_path
+							AND storage_cids.header_id = state_cids.header_id
+							AND storage_cids.block_number = state_cids.block_number
+						)
+						WHERE
+							storage_cids.block_number = $1
+							AND state_cids.state_path IS NULL
+					)`
+
+	StateAccountsRefStateCIDs = `SELECT EXISTS (
+						SELECT *
+						FROM eth.state_accounts
+						LEFT JOIN eth.state_cids ON (
+							state_accounts.state_path = state_cids.state_path
+							AND state_accounts.header_id = state_cids.header_id
+							AND state_accounts.block_number = state_cids.block_number
+						)
+						WHERE
+							state_accounts.block_number = $1
+							AND state_cids.state_path IS NULL
+					)`
+
+	AccessListElementsRefTransactionCIDs = `SELECT EXISTS (
+						SELECT *
+						FROM eth.access_list_elements
+						LEFT JOIN eth.transaction_cids ON (
+							access_list_elements.tx_id = transaction_cids.tx_hash
+							AND access_list_elements.block_number = transaction_cids.block_number
+						)
+						WHERE
+							access_list_elements.block_number = $1
+							AND transaction_cids.tx_hash IS NULL
+					)`
+
+	LogCIDsRefReceiptCIDs = `SELECT EXISTS (
+						SELECT *
+						FROM eth.log_cids
+						LEFT JOIN eth.receipt_cids ON (
+							log_cids.rct_id = receipt_cids.tx_id
+							AND log_cids.header_id = receipt_cids.header_id
+							AND log_cids.block_number = receipt_cids.block_number
+						)
+						WHERE
+							log_cids.block_number = $1
+							AND receipt_cids.tx_id IS NULL
+					)`
+)
@@ -1,10 +1,28 @@
+// VulcanizeDB
+// Copyright © 2022 Vulcanize
+
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU Affero General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU Affero General Public License for more details.
+
+// You should have received a copy of the GNU Affero General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 package validator

 import (
 	"context"
+	"encoding/json"
 	"errors"
 	"fmt"
 	"math/big"
+	"sync"
 	"time"

 	"github.com/ethereum/go-ethereum/common"
@@ -17,32 +35,51 @@ import (
 	"github.com/ethereum/go-ethereum/core/vm"
 	"github.com/ethereum/go-ethereum/params"
 	"github.com/ethereum/go-ethereum/rpc"
+	"github.com/ethereum/go-ethereum/statediff"
 	"github.com/jmoiron/sqlx"
 	log "github.com/sirupsen/logrus"
+
 	ipfsethdb "github.com/vulcanize/ipfs-ethdb/v4/postgres"
 	ipldEth "github.com/vulcanize/ipld-eth-server/v4/pkg/eth"
 	ethServerShared "github.com/vulcanize/ipld-eth-server/v4/pkg/shared"
+
+	"github.com/vulcanize/ipld-eth-db-validator/pkg/prom"
 )

 var (
 	big8  = big.NewInt(8)
 	big32 = big.NewInt(32)
+
+	ReferentialIntegrityErr = "referential integrity check failed at block %d, entry for %s not found"
+	EntryNotFoundErr        = "entry for %s not found"
 )

 type service struct {
 	db              *sqlx.DB
 	blockNum, trail uint64
-	logger          *log.Logger
+	sleepInterval   uint
 	chainCfg        *params.ChainConfig
+
+	stateDiffMissingBlock bool
+	stateDiffTimeout      uint
+	ethClient             *rpc.Client
+
+	quitChan     chan bool
+	progressChan chan uint64
 }

-func NewService(db *sqlx.DB, blockNum, trailNum uint64, chainCfg *params.ChainConfig) *service {
+func NewService(cfg *Config, progressChan chan uint64) *service {
 	return &service{
-		db:       db,
-		blockNum: blockNum,
-		trail:    trailNum,
-		logger:   log.New(),
-		chainCfg: chainCfg,
+		db:                    cfg.DB,
+		blockNum:              cfg.BlockNum,
+		trail:                 cfg.Trail,
+		sleepInterval:         cfg.SleepInterval,
+		chainCfg:              cfg.ChainCfg,
+		stateDiffMissingBlock: cfg.StateDiffMissingBlock,
+		stateDiffTimeout:      cfg.StateDiffTimeout,
+		ethClient:             cfg.Client,
+		quitChan:              make(chan bool),
+		progressChan:          progressChan,
 	}
 }

@@ -76,51 +113,138 @@ func NewEthBackend(db *sqlx.DB, c *ipldEth.Config) (*ipldEth.Backend, error) {
 }

 // Start is used to begin the service
-func (s *service) Start(ctx context.Context) (uint64, error) {
-	api, err := ethAPI(ctx, s.db, s.chainCfg)
+func (s *service) Start(ctx context.Context, wg *sync.WaitGroup) {
+	defer wg.Done()
+
+	api, err := EthAPI(ctx, s.db, s.chainCfg)
 	if err != nil {
-		return 0, err
+		log.Fatal(err)
+		return
 	}

 	idxBlockNum := s.blockNum
-	headBlock, _ := api.B.BlockByNumber(ctx, rpc.LatestBlockNumber)
-	headBlockNum := headBlock.NumberU64()

-	for headBlockNum-s.trail >= idxBlockNum {
-		validateBlock, err := api.B.BlockByNumber(ctx, rpc.BlockNumber(idxBlockNum))
-		if err != nil {
-			return idxBlockNum, err
+	for {
+		select {
+		case <-s.quitChan:
+			log.Infof("last validated block %v", idxBlockNum-1)
+			if s.progressChan != nil {
+				close(s.progressChan)
+			}
+			return
+		default:
+			idxBlockNum, err = s.Validate(ctx, api, idxBlockNum)
+			if err != nil {
+				log.Infof("last validated block %v", idxBlockNum-1)
+				log.Fatal(err)
+				return
+			}
+
+			prom.SetLastValidatedBlock(float64(idxBlockNum))
 		}
+	}
+}

-		stateDB, err := applyTransaction(validateBlock, api.B)
-		if err != nil {
-			return idxBlockNum, err
-		}
+// Stop is used to gracefully stop the service
+func (s *service) Stop() {
+	log.Info("stopping ipld-eth-db-validator process")
+	close(s.quitChan)
+}

-		blockStateRoot := validateBlock.Header().Root.String()
+func (s *service) Validate(ctx context.Context, api *ipldEth.PublicEthAPI, idxBlockNum uint64) (uint64, error) {
+	headBlockNum, err := fetchHeadBlockNumber(ctx, api)
+	if err != nil {
+		return idxBlockNum, err
+	}

-		dbStateRoot := stateDB.IntermediateRoot(true).String()
-		if blockStateRoot != dbStateRoot {
-			s.logger.Errorf("failed to verify state root at block %d", idxBlockNum)
-			return idxBlockNum, fmt.Errorf("failed to verify state root at block")
+	// Check if it block at height idxBlockNum can be validated
+	if idxBlockNum <= headBlockNum-s.trail {
+		blockToBeValidated, err := api.B.BlockByNumber(ctx, rpc.BlockNumber(idxBlockNum))
+		if err != nil {
+			log.Errorf("failed to fetch block at height %d", idxBlockNum)
+			return idxBlockNum, err
 		}

-		s.logger.Infof("state root verified for block %d", idxBlockNum)
+		// Make a writeStateDiffAt call if block not found in the db
+		if blockToBeValidated == nil {
+			err = s.writeStateDiffAt(idxBlockNum)
+			return idxBlockNum, err
+		}

-		headBlock, err = api.B.BlockByNumber(ctx, rpc.LatestBlockNumber)
+		err = ValidateBlock(blockToBeValidated, api.B, idxBlockNum)
 		if err != nil {
+			log.Errorf("failed to verify state root at block %d", idxBlockNum)
 			return idxBlockNum, err
+		}
+
+		log.Infof("state root verified for block %d", idxBlockNum)
+
+		err = ValidateReferentialIntegrity(s.db, idxBlockNum)
+		if err != nil {
+			log.Errorf("failed to verify referential integrity at block %d", idxBlockNum)
+			return idxBlockNum, err
+		}
+		log.Infof("referential integrity verified for block %d", idxBlockNum)
+
+		if s.progressChan != nil {
+			s.progressChan <- idxBlockNum
 		}

-		headBlockNum = headBlock.NumberU64()
 		idxBlockNum++
+	} else {
+		// Sleep / wait for head to move ahead
+		time.Sleep(time.Second * time.Duration(s.sleepInterval))
 	}

-	s.logger.Infof("last validated block %v", idxBlockNum-1)
 	return idxBlockNum, nil
 }

-func ethAPI(ctx context.Context, db *sqlx.DB, chainCfg *params.ChainConfig) (*ipldEth.PublicEthAPI, error) {
+// writeStateDiffAt calls out to a statediffing geth client to fill in a gap in the index
+func (s *service) writeStateDiffAt(height uint64) error {
+	if !s.stateDiffMissingBlock {
+		return nil
+	}
+
+	var data json.RawMessage
+	params := statediff.Params{
+		IntermediateStateNodes:   true,
+		IntermediateStorageNodes: true,
+		IncludeBlock:             true,
+		IncludeReceipts:          true,
+		IncludeTD:                true,
+		IncludeCode:              true,
+	}
+
+	ctx, cancel := context.WithTimeout(context.Background(), time.Duration(s.stateDiffTimeout*uint(time.Second)))
+	defer cancel()
+
+	log.Warnf("making writeStateDiffAt call at height %d", height)
+	if err := s.ethClient.CallContext(ctx, &data, "statediff_writeStateDiffAt", height, params); err != nil {
+		log.Errorf("writeStateDiffAt %d faild with err %s", height, err.Error())
+		return err
+	}
+
+	return nil
+}
+
+// ValidateBlock validates block at the given height
+func ValidateBlock(blockToBeValidated *types.Block, b *ipldEth.Backend, blockNumber uint64) error {
+	stateDB, err := applyTransaction(blockToBeValidated, b)
+	if err != nil {
+		return err
+	}
+
+	blockStateRoot := blockToBeValidated.Header().Root.String()
+
+	dbStateRoot := stateDB.IntermediateRoot(true).String()
+	if blockStateRoot != dbStateRoot {
+		return fmt.Errorf("state roots do not match at block %d", blockNumber)
+	}
+
+	return nil
+}
+
+func EthAPI(ctx context.Context, db *sqlx.DB, chainCfg *params.ChainConfig) (*ipldEth.PublicEthAPI, error) {
 	// TODO: decide network for custom chainConfig.
 	backend, err := NewEthBackend(db, &ipldEth.Config{
 		ChainConfig: chainCfg,
@@ -147,6 +271,16 @@ func ethAPI(ctx context.Context, db *sqlx.DB, chainCfg *params.ChainConfig) (*ip
 	return ipldEth.NewPublicEthAPI(backend, nil, false, false, false)
 }

+// fetchHeadBlockNumber gets the latest block number from the db
+func fetchHeadBlockNumber(ctx context.Context, api *ipldEth.PublicEthAPI) (uint64, error) {
+	headBlock, err := api.B.BlockByNumber(ctx, rpc.LatestBlockNumber)
+	if err != nil {
+		return 0, err
+	}
+
+	return headBlock.NumberU64(), nil
+}
+
 // applyTransaction attempts to apply a transaction to the given state database
 // and uses the input parameters for its environment. It returns the stateDB of parent with applied transa
 func applyTransaction(block *types.Block, backend *ipldEth.Backend) (*state.StateDB, error) {
@@ -4,20 +4,27 @@

 - For running integration tests:

-  - Clone [stack-orchestrator](https://github.com/vulcanize/stack-orchestrator) and [go-ethereum](https://github.com/vulcanize/go-ethereum) repositories.
+  - Clone [stack-orchestrator](https://github.com/vulcanize/stack-orchestrator), [go-ethereum](https://github.com/vulcanize/go-ethereum) and [ipld-eth-db](https://github.com/vulcanize/ipld-eth-db) repositories.

-  - Checkout [v4 release](https://github.com/vulcanize/go-ethereum/releases/tag/v1.10.17-statediff-4.0.1-alpha) in go-ethereum repo.
+  - Checkout [v4 release](https://github.com/vulcanize/ipld-eth-db/releases/tag/v4.2.1-alpha) in ipld-eth-db repo.
+
+    ```bash
+    # In ipld-eth-db repo.
+    git checkout v4.2.1-alpha
+    ```
+
+  - Checkout [v4 release](https://github.com/vulcanize/go-ethereum/releases/tag/v1.10.21-statediff-4.1.2-alpha) in go-ethereum repo.

    ```bash
    # In go-ethereum repo.
-    git checkout v1.10.17-statediff-4.0.1-alpha
+    git checkout v1.10.21-statediff-4.1.2-alpha
    ```

  - Checkout working commit in stack-orchestrator repo.

    ```bash
    # In stack-orchestrator repo.
-    git checkout 42af57a2963bb6ca55fb5fcb16ed75b39fae60f9
+    git checkout f2fd766f5400fcb9eb47b50675d2e3b1f2753702
    ```

 ## Run
@@ -48,12 +55,16 @@
      ```bash
      #!/bin/bash

+      # Path to ipld-eth-server repo.
+      vulcanize_ipld_eth_db=~/ipld-eth-db/
+
      # Path to go-ethereum repo.
      vulcanize_go_ethereum=~/go-ethereum

      # Path to contract folder.
      vulcanize_test_contract=~/ipld-eth-db-validator/test/contract

+      genesis_file_path='start-up-files/go-ethereum/genesis.json'
      db_write=true
      ```

@@ -65,7 +76,7 @@

      ./wrapper.sh \
      -e docker \
-      -d ../docker/latest/docker-compose-db.yml \
+      -d ../docker/local/docker-compose-db-sharding.yml \
      -d ../docker/local/docker-compose-go-ethereum.yml \
      -d ../docker/local/docker-compose-contract.yml \
      -v remove \
@@ -0,0 +1,29 @@
+// SPDX-License-Identifier: AGPL-3.0
+pragma solidity ^0.8.0;
+
+contract Test {
+  address payable owner;
+
+  modifier onlyOwner {
+    require(
+      msg.sender == owner,
+      "Only owner can call this function."
+    );
+    _;
+  }
+
+  uint256[100] data;
+
+  constructor() {
+    owner = payable(msg.sender);
+    data = [1];
+  }
+
+  function Put(uint256 addr, uint256 value) public {
+    data[addr] = value;
+  }
+
+  function close() public onlyOwner {
+    selfdestruct(owner);
+  }
+}
@@ -63,6 +63,49 @@ fastify.get('/v1/sendEth', async (req, reply) => {
    }
 });

+fastify.get('/v1/deployTestContract', async (req, reply) => {
+    const testContract = await hre.ethers.getContractFactory("Test");
+    const test = await testContract.deploy();
+    await test.deployed();
+
+    return {
+        address: test.address,
+        txHash: test.deployTransaction.hash,
+        blockNumber: test.deployTransaction.blockNumber,
+        blockHash: test.deployTransaction.blockHash,
+    }
+});
+
+fastify.get('/v1/putTestValue', async (req, reply) => {
+    const addr = req.query.addr;
+    const index = req.query.index;
+    const value = req.query.value;
+
+    const testContract = await hre.ethers.getContractFactory("Test");
+    const test = await testContract.attach(addr);
+
+    const tx = await test.Put(index, value);
+    const receipt = await tx.wait();
+
+    return {
+        blockNumber: receipt.blockNumber,
+    }
+});
+
+fastify.get('/v1/destroyTestContract', async (req, reply) => {
+    const addr = req.query.addr;
+
+    const testContract = await hre.ethers.getContractFactory("Test");
+    const test = await testContract.attach(addr);
+
+    await test.destroy();
+    const blockNum = await hre.ethers.provider.getBlockNumber()
+
+    return {
+        blockNumber: blockNum,
+    }
+})
+
 async function main() {
    try {
        await fastify.listen(3000, '0.0.0.0');
@@ -72,4 +115,4 @@ async function main() {
    }
 }

-main();
+main();
@@ -0,0 +1,54 @@
+package integration
+
+import (
+	"encoding/json"
+	"fmt"
+	"net/http"
+
+	ethServerIntegration "github.com/vulcanize/ipld-eth-server/v4/test"
+)
+
+type PutResult struct {
+	BlockNumber int64 `json:"blockNumber"`
+}
+
+const srvUrl = "http://localhost:3000"
+
+func DeployTestContract() (*ethServerIntegration.ContractDeployed, error) {
+	ethServerIntegration.DeployContract()
+	res, err := http.Get(fmt.Sprintf("%s/v1/deployTestContract", srvUrl))
+	if err != nil {
+		return nil, err
+	}
+	defer res.Body.Close()
+
+	var contract ethServerIntegration.ContractDeployed
+	decoder := json.NewDecoder(res.Body)
+
+	return &contract, decoder.Decode(&contract)
+}
+
+func PutTestValue(addr string, index, value int) (*PutResult, error) {
+	res, err := http.Get(fmt.Sprintf("%s/v1/putTestValue?addr=%s&index=%d&value=%d", srvUrl, addr, index, value))
+	if err != nil {
+		return nil, err
+	}
+
+	var blockNumber PutResult
+	decoder := json.NewDecoder(res.Body)
+
+	return &blockNumber, decoder.Decode(&blockNumber)
+}
+
+func DestroyTestContract(addr string) (*ethServerIntegration.ContractDestroyed, error) {
+	res, err := http.Get(fmt.Sprintf("%s/v1/destroyTestContract?addr=%s", srvUrl, addr))
+	if err != nil {
+		return nil, err
+	}
+	defer res.Body.Close()
+
+	var data ethServerIntegration.ContractDestroyed
+	decoder := json.NewDecoder(res.Body)
+
+	return &data, decoder.Decode(&data)
+}
@@ -2,43 +2,107 @@ package integration_test

 import (
 	"context"
+	"sync"
 	"time"

 	. "github.com/onsi/ginkgo"
 	. "github.com/onsi/gomega"

 	"github.com/vulcanize/ipld-eth-db-validator/pkg/validator"
+	integration "github.com/vulcanize/ipld-eth-db-validator/test"

 	"github.com/vulcanize/ipld-eth-server/v4/pkg/shared"
-	integration "github.com/vulcanize/ipld-eth-server/v4/test"
+	ethServerIntegration "github.com/vulcanize/ipld-eth-server/v4/test"
 )

-const trail = 0
+const (
+	blockNum               = 1
+	trail                  = 0
+	validatorSleepInterval = uint(5)
+)
+
+var (
+	testAddresses = []string{
+		"0x1111111111111111111111111111111111111112",
+		"0x1ca7c995f8eF0A2989BbcE08D5B7Efe50A584aa1",
+		"0x9a4b666af23a2cdb4e5538e1d222a445aeb82134",
+		"0xF7C7AEaECD2349b129d5d15790241c32eeE4607B",
+		"0x992b6E9BFCA1F7b0797Cee10b0170E536EAd3532",
+		"0x29ed93a7454Bc17a8D4A24D0627009eE0849B990",
+		"0x66E3dCA826b04B5d4988F7a37c91c9b1041e579D",
+		"0x96288939Ac7048c27E0E087b02bDaad3cd61b37b",
+		"0xD354280BCd771541c935b15bc04342c26086FE9B",
+		"0x7f887e25688c274E77b8DeB3286A55129B55AF14",
+	}
+)

 var _ = Describe("Integration test", func() {
 	ctx := context.Background()

-	var contract *integration.ContractDeployed
-	var contractErr error
-	sleepInterval := 5 * time.Second
+	var contract *ethServerIntegration.ContractDeployed
+	var err error
+	sleepInterval := 2 * time.Second
+	timeout := 4 * time.Second
+
+	db := shared.SetupDB()
+	cfg := validator.Config{
+		DB:            db,
+		BlockNum:      blockNum,
+		Trail:         trail,
+		SleepInterval: validatorSleepInterval,
+		ChainCfg:      validator.IntegrationTestChainConfig,
+	}
+	validationProgressChan := make(chan uint64)
+	service := validator.NewService(&cfg, validationProgressChan)
+
+	wg := new(sync.WaitGroup)
+
+	It("test init", func() {
+		wg.Add(1)
+		go service.Start(ctx, wg)
+
+		// Deploy a dummy contract as the first contract might get deployed at block number 0
+		_, _ = ethServerIntegration.DeployContract()
+		time.Sleep(sleepInterval)
+	})
+
+	defer It("test teardown", func() {
+		service.Stop()
+		wg.Wait()
+
+		Expect(validationProgressChan).To(BeClosed())
+	})

 	Describe("Validate state", func() {
-		BeforeEach(func() {
-			// Deploy a dummy contract as the first contract might get deployed at block number 0
-			_, _ = integration.DeployContract()
+		It("performs validation on contract deployment", func() {
+			contract, err = integration.DeployTestContract()
+			Expect(err).ToNot(HaveOccurred())
 			time.Sleep(sleepInterval)

-			contract, contractErr = integration.DeployContract()
-			time.Sleep(sleepInterval)
+			Expect(validationProgressChan).ToNot(BeClosed())
+			Eventually(validationProgressChan, timeout).Should(Receive(Equal(uint64(contract.BlockNumber))))
 		})

-		It("Validate state root", func() {
-			Expect(contractErr).ToNot(HaveOccurred())
+		It("performs validation on contract transactions", func() {
+			for i := 0; i < 10; i++ {
+				res, txErr := integration.PutTestValue(contract.Address, i, i)
+				Expect(txErr).ToNot(HaveOccurred())
+				time.Sleep(sleepInterval)

-			db := shared.SetupDB()
-			srvc := validator.NewService(db, uint64(contract.BlockNumber), trail, validator.IntegrationTestChainConfig)
-			_, err := srvc.Start(ctx)
-			Expect(err).ToNot(HaveOccurred())
+				Expect(validationProgressChan).ToNot(BeClosed())
+				Eventually(validationProgressChan, timeout).Should(Receive(Equal(uint64(res.BlockNumber))))
+			}
+		})
+
+		It("performs validation on eth transfer transactions", func() {
+			for _, address := range testAddresses {
+				tx, txErr := ethServerIntegration.SendEth(address, "0.01")
+				Expect(txErr).ToNot(HaveOccurred())
+				time.Sleep(sleepInterval)
+
+				Expect(validationProgressChan).ToNot(BeClosed())
+				Eventually(validationProgressChan, timeout).Should(Receive(Equal(uint64(tx.BlockNumber))))
+			}
 		})
 	})
 })
@@ -2,6 +2,7 @@ package validator_test

 import (
 	"crypto/ecdsa"
+	"github.com/ethereum/go-ethereum/statediff/test_helpers"
 	"math/big"

 	"github.com/ethereum/go-ethereum/common"
@@ -23,7 +24,7 @@ var (
 	TestBankKey, _  = crypto.HexToECDSA("b71c71a67e1177ad4e901695e1b4b9ee17ae16c6668d313eac2f96dbcda3f291")
 	TestBankAddress = crypto.PubkeyToAddress(TestBankKey.PublicKey) //0x71562b71999873DB5b286dF957af199Ec94617F7
 	TestBankFunds   = big.NewInt(100000000)
-	Genesis         = core.GenesisBlockForTesting(Testdb, TestBankAddress, TestBankFunds)
+	Genesis         = test_helpers.GenesisBlockForTesting(Testdb, TestBankAddress, TestBankFunds)

 	Account1Key, _       = crypto.HexToECDSA("8a1f9a8f95be41cd7ccb6168179afb4504aefe388d1e14474d32c45c72ce7b7a")
 	Account2Key, _       = crypto.HexToECDSA("49a7b37aa6f6645917e7b807e9d1c00d4fa71f18343b0d4122a4d2df64dd6fee")
@@ -0,0 +1,334 @@
+package validator_test
+
+import (
+	"context"
+	"fmt"
+
+	"github.com/ethereum/go-ethereum/params"
+	"github.com/ethereum/go-ethereum/statediff/indexer/interfaces"
+	"github.com/ethereum/go-ethereum/statediff/indexer/mocks"
+	"github.com/jmoiron/sqlx"
+	. "github.com/onsi/ginkgo"
+	. "github.com/onsi/gomega"
+
+	"github.com/vulcanize/ipld-eth-db-validator/pkg/validator"
+	"github.com/vulcanize/ipld-eth-server/v4/pkg/eth/test_helpers"
+	"github.com/vulcanize/ipld-eth-server/v4/pkg/shared"
+)
+
+var _ = Describe("RefIntegrity", func() {
+	var (
+		ctx = context.Background()
+
+		db          *sqlx.DB
+		diffIndexer interfaces.StateDiffIndexer
+	)
+
+	BeforeEach(func() {
+		db = shared.SetupDB()
+		diffIndexer = shared.SetupTestStateDiffIndexer(ctx, params.TestChainConfig, test_helpers.Genesis.Hash())
+	})
+
+	AfterEach(func() {
+		shared.TearDownDB(db)
+	})
+
+	Describe("ValidateHeaderCIDsRef", func() {
+		BeforeEach(func() {
+			tx, err := diffIndexer.PushBlock(test_helpers.MockBlock, test_helpers.MockReceipts, test_helpers.MockBlock.Difficulty())
+			Expect(err).ToNot(HaveOccurred())
+
+			err = tx.Submit(err)
+			Expect(err).ToNot(HaveOccurred())
+		})
+
+		It("Validates referential integrity of header_cids table", func() {
+			err := validator.ValidateHeaderCIDsRef(db, test_helpers.MockBlock.NumberU64())
+			Expect(err).ToNot(HaveOccurred())
+		})
+
+		It("Throws an error if corresponding header IPFS block entry not found", func() {
+			err := deleteEntriesFrom(db, "public.blocks")
+			Expect(err).ToNot(HaveOccurred())
+
+			err = validator.ValidateHeaderCIDsRef(db, test_helpers.MockBlock.NumberU64())
+			Expect(err).To(HaveOccurred())
+			Expect(err.Error()).To(ContainSubstring(validator.EntryNotFoundErr, "public.blocks"))
+		})
+	})
+
+	Describe("ValidateUncleCIDsRef", func() {
+		BeforeEach(func() {
+			tx, err := diffIndexer.PushBlock(test_helpers.MockBlock, test_helpers.MockReceipts, test_helpers.MockBlock.Difficulty())
+			Expect(err).ToNot(HaveOccurred())
+
+			err = tx.Submit(err)
+			Expect(err).ToNot(HaveOccurred())
+		})
+
+		It("Validates referential integrity of uncle_cids table", func() {
+			err := validator.ValidateUncleCIDsRef(db, test_helpers.MockBlock.NumberU64())
+			Expect(err).ToNot(HaveOccurred())
+		})
+
+		It("Throws an error if corresponding header_cid entry not found", func() {
+			err := deleteEntriesFrom(db, "eth.header_cids")
+			Expect(err).ToNot(HaveOccurred())
+
+			err = validator.ValidateUncleCIDsRef(db, test_helpers.MockBlock.NumberU64())
+			Expect(err).To(HaveOccurred())
+			Expect(err.Error()).To(ContainSubstring(validator.EntryNotFoundErr, "eth.header_cids"))
+		})
+
+		It("Throws an error if corresponding uncle IPFS block entry not found", func() {
+			err := deleteEntriesFrom(db, "public.blocks")
+			Expect(err).ToNot(HaveOccurred())
+
+			err = validator.ValidateUncleCIDsRef(db, test_helpers.MockBlock.NumberU64())
+			Expect(err).To(HaveOccurred())
+			Expect(err.Error()).To(ContainSubstring(validator.EntryNotFoundErr, "public.blocks"))
+		})
+	})
+
+	Describe("ValidateTransactionCIDsRef", func() {
+		BeforeEach(func() {
+			tx, err := diffIndexer.PushBlock(test_helpers.MockBlock, test_helpers.MockReceipts, test_helpers.MockBlock.Difficulty())
+			Expect(err).ToNot(HaveOccurred())
+
+			err = tx.Submit(err)
+			Expect(err).ToNot(HaveOccurred())
+		})
+
+		It("Validates referential integrity of transaction_cids table", func() {
+			err := validator.ValidateTransactionCIDsRef(db, test_helpers.MockBlock.NumberU64())
+			Expect(err).ToNot(HaveOccurred())
+		})
+
+		It("Throws an error if corresponding header_cid entry not found", func() {
+			err := deleteEntriesFrom(db, "eth.header_cids")
+			Expect(err).ToNot(HaveOccurred())
+
+			err = validator.ValidateTransactionCIDsRef(db, test_helpers.MockBlock.NumberU64())
+			Expect(err).To(HaveOccurred())
+			Expect(err.Error()).To(ContainSubstring(validator.EntryNotFoundErr, "eth.header_cids"))
+		})
+
+		It("Throws an error if corresponding transaction IPFS block entry not found", func() {
+			err := deleteEntriesFrom(db, "public.blocks")
+			Expect(err).ToNot(HaveOccurred())
+
+			err = validator.ValidateTransactionCIDsRef(db, test_helpers.MockBlock.NumberU64())
+			Expect(err).To(HaveOccurred())
+			Expect(err.Error()).To(ContainSubstring(validator.EntryNotFoundErr, "public.blocks"))
+		})
+	})
+
+	Describe("ValidateReceiptCIDsRef", func() {
+		BeforeEach(func() {
+			tx, err := diffIndexer.PushBlock(test_helpers.MockBlock, test_helpers.MockReceipts, test_helpers.MockBlock.Difficulty())
+			Expect(err).ToNot(HaveOccurred())
+
+			err = tx.Submit(err)
+			Expect(err).ToNot(HaveOccurred())
+		})
+
+		It("Validates referential integrity of receipt_cids table", func() {
+			err := validator.ValidateReceiptCIDsRef(db, test_helpers.MockBlock.NumberU64())
+			Expect(err).ToNot(HaveOccurred())
+		})
+
+		It("Throws an error if corresponding transaction_cids entry not found", func() {
+			err := deleteEntriesFrom(db, "eth.transaction_cids")
+			Expect(err).ToNot(HaveOccurred())
+
+			err = validator.ValidateReceiptCIDsRef(db, test_helpers.MockBlock.NumberU64())
+			Expect(err).To(HaveOccurred())
+			Expect(err.Error()).To(ContainSubstring(validator.EntryNotFoundErr, "eth.transaction_cids"))
+		})
+
+		It("Throws an error if corresponding receipt IPFS block entry not found", func() {
+			err := deleteEntriesFrom(db, "public.blocks")
+			Expect(err).ToNot(HaveOccurred())
+
+			err = validator.ValidateReceiptCIDsRef(db, test_helpers.MockBlock.NumberU64())
+			Expect(err).To(HaveOccurred())
+			Expect(err.Error()).To(ContainSubstring(validator.EntryNotFoundErr, "public.blocks"))
+		})
+	})
+
+	Describe("ValidateStateCIDsRef", func() {
+		BeforeEach(func() {
+			tx, err := diffIndexer.PushBlock(test_helpers.MockBlock, test_helpers.MockReceipts, test_helpers.MockBlock.Difficulty())
+			Expect(err).ToNot(HaveOccurred())
+
+			for _, node := range test_helpers.MockStateNodes {
+				err = diffIndexer.PushStateNode(tx, node, test_helpers.MockBlock.Hash().String())
+				Expect(err).ToNot(HaveOccurred())
+			}
+
+			err = tx.Submit(err)
+			Expect(err).ToNot(HaveOccurred())
+		})
+
+		It("Validates referential integrity of state_cids table", func() {
+			err := validator.ValidateStateCIDsRef(db, test_helpers.MockBlock.NumberU64())
+			Expect(err).ToNot(HaveOccurred())
+		})
+
+		It("Throws an error if corresponding header_cids entry not found", func() {
+			err := deleteEntriesFrom(db, "eth.header_cids")
+			Expect(err).ToNot(HaveOccurred())
+
+			err = validator.ValidateStateCIDsRef(db, test_helpers.MockBlock.NumberU64())
+			Expect(err).To(HaveOccurred())
+			Expect(err.Error()).To(ContainSubstring(validator.EntryNotFoundErr, "eth.header_cids"))
+		})
+
+		It("Throws an error if corresponding state IPFS block entry not found", func() {
+			err := deleteEntriesFrom(db, "public.blocks")
+			Expect(err).ToNot(HaveOccurred())
+
+			err = validator.ValidateStateCIDsRef(db, test_helpers.MockBlock.NumberU64())
+			Expect(err).To(HaveOccurred())
+			Expect(err.Error()).To(ContainSubstring(validator.EntryNotFoundErr, "public.blocks"))
+		})
+	})
+
+	Describe("ValidateStorageCIDsRef", func() {
+		BeforeEach(func() {
+			tx, err := diffIndexer.PushBlock(test_helpers.MockBlock, test_helpers.MockReceipts, test_helpers.MockBlock.Difficulty())
+			Expect(err).ToNot(HaveOccurred())
+
+			for _, node := range test_helpers.MockStateNodes {
+				err = diffIndexer.PushStateNode(tx, node, test_helpers.MockBlock.Hash().String())
+				Expect(err).ToNot(HaveOccurred())
+			}
+
+			err = tx.Submit(err)
+			Expect(err).ToNot(HaveOccurred())
+		})
+
+		It("Validates referential integrity of storage_cids table", func() {
+			err := validator.ValidateStorageCIDsRef(db, test_helpers.MockBlock.NumberU64())
+			Expect(err).ToNot(HaveOccurred())
+		})
+
+		It("Throws an error if corresponding state_cids entry not found", func() {
+			err := deleteEntriesFrom(db, "eth.state_cids")
+			Expect(err).ToNot(HaveOccurred())
+
+			err = validator.ValidateStorageCIDsRef(db, test_helpers.MockBlock.NumberU64())
+			Expect(err).To(HaveOccurred())
+			Expect(err.Error()).To(ContainSubstring(validator.EntryNotFoundErr, "eth.state_cids"))
+		})
+
+		It("Throws an error if corresponding storage IPFS block entry not found", func() {
+			err := deleteEntriesFrom(db, "public.blocks")
+			Expect(err).ToNot(HaveOccurred())
+
+			err = validator.ValidateStorageCIDsRef(db, test_helpers.MockBlock.NumberU64())
+			Expect(err).To(HaveOccurred())
+			Expect(err.Error()).To(ContainSubstring(validator.EntryNotFoundErr, "public.blocks"))
+		})
+	})
+
+	Describe("ValidateStateAccountsRef", func() {
+		BeforeEach(func() {
+			tx, err := diffIndexer.PushBlock(test_helpers.MockBlock, test_helpers.MockReceipts, test_helpers.MockBlock.Difficulty())
+			Expect(err).ToNot(HaveOccurred())
+
+			for _, node := range test_helpers.MockStateNodes {
+				err = diffIndexer.PushStateNode(tx, node, test_helpers.MockBlock.Hash().String())
+				Expect(err).ToNot(HaveOccurred())
+			}
+
+			err = tx.Submit(err)
+			Expect(err).ToNot(HaveOccurred())
+		})
+
+		It("Validates referential integrity of state_accounts table", func() {
+			err := validator.ValidateStateAccountsRef(db, test_helpers.MockBlock.NumberU64())
+			Expect(err).ToNot(HaveOccurred())
+		})
+
+		It("Throws an error if corresponding state_cids entry not found", func() {
+			err := deleteEntriesFrom(db, "eth.state_cids")
+			Expect(err).ToNot(HaveOccurred())
+
+			err = validator.ValidateStateAccountsRef(db, test_helpers.MockBlock.NumberU64())
+			Expect(err).To(HaveOccurred())
+			Expect(err.Error()).To(ContainSubstring(validator.EntryNotFoundErr, "eth.state_cids"))
+		})
+	})
+
+	Describe("ValidateAccessListElementsRef", func() {
+		BeforeEach(func() {
+			indexAndPublisher := shared.SetupTestStateDiffIndexer(ctx, mocks.TestConfig, test_helpers.Genesis.Hash())
+
+			tx, err := indexAndPublisher.PushBlock(mocks.MockBlock, mocks.MockReceipts, mocks.MockBlock.Difficulty())
+			Expect(err).ToNot(HaveOccurred())
+
+			err = tx.Submit(err)
+			Expect(err).ToNot(HaveOccurred())
+		})
+
+		It("Validates referential integrity of access_list_elements table", func() {
+			err := validator.ValidateAccessListElementsRef(db, mocks.MockBlock.NumberU64())
+			Expect(err).ToNot(HaveOccurred())
+		})
+
+		It("Throws an error if corresponding transaction_cids entry not found", func() {
+			err := deleteEntriesFrom(db, "eth.transaction_cids")
+			Expect(err).ToNot(HaveOccurred())
+
+			err = validator.ValidateAccessListElementsRef(db, mocks.MockBlock.NumberU64())
+			Expect(err).To(HaveOccurred())
+			Expect(err.Error()).To(ContainSubstring(validator.EntryNotFoundErr, "eth.transaction_cids"))
+		})
+	})
+
+	Describe("ValidateLogCIDsRef", func() {
+		BeforeEach(func() {
+			tx, err := diffIndexer.PushBlock(test_helpers.MockBlock, test_helpers.MockReceipts, test_helpers.MockBlock.Difficulty())
+			Expect(err).ToNot(HaveOccurred())
+
+			for _, node := range test_helpers.MockStateNodes {
+				err = diffIndexer.PushStateNode(tx, node, test_helpers.MockBlock.Hash().String())
+				Expect(err).ToNot(HaveOccurred())
+			}
+
+			err = tx.Submit(err)
+			Expect(err).ToNot(HaveOccurred())
+		})
+
+		It("Validates referential integrity of log_cids table", func() {
+			err := validator.ValidateLogCIDsRef(db, test_helpers.MockBlock.NumberU64())
+			Expect(err).ToNot(HaveOccurred())
+		})
+
+		It("Throws an error if corresponding receipt_cids entry not found", func() {
+			err := deleteEntriesFrom(db, "eth.receipt_cids")
+			Expect(err).ToNot(HaveOccurred())
+
+			err = validator.ValidateLogCIDsRef(db, test_helpers.MockBlock.NumberU64())
+			Expect(err).To(HaveOccurred())
+			Expect(err.Error()).To(ContainSubstring(validator.EntryNotFoundErr, "eth.receipt_cids"))
+		})
+
+		It("Throws an error if corresponding log IPFS block entry not found", func() {
+			err := deleteEntriesFrom(db, "public.blocks")
+			Expect(err).ToNot(HaveOccurred())
+
+			err = validator.ValidateLogCIDsRef(db, test_helpers.MockBlock.NumberU64())
+			Expect(err).To(HaveOccurred())
+			Expect(err.Error()).To(ContainSubstring(validator.EntryNotFoundErr, "public.blocks"))
+		})
+	})
+
+})
+
+func deleteEntriesFrom(db *sqlx.DB, tableName string) error {
+	pgStr := "DELETE FROM %s"
+	_, err := db.Exec(fmt.Sprintf(pgStr, tableName))
+	return err
+}
@@ -1,4 +1,4 @@
-package validator_test_test
+package validator_test

 import (
 	"io/ioutil"
@@ -7,6 +7,7 @@ import (
 	"github.com/ethereum/go-ethereum/common"
 	"github.com/ethereum/go-ethereum/core"
 	"github.com/ethereum/go-ethereum/core/types"
+	"github.com/ethereum/go-ethereum/rpc"
 	"github.com/ethereum/go-ethereum/statediff"
 	sdtypes "github.com/ethereum/go-ethereum/statediff/types"
 	"github.com/jmoiron/sqlx"
@@ -114,10 +115,20 @@ var _ = Describe("eth state reading tests", func() {

 	Describe("state_validation", func() {
 		It("Validator", func() {
-			srvc := validator.NewService(db, blockHeight, trail, validator.TestChainConfig)
-
-			_, err := srvc.Start(context.Background())
+			api, err := validator.EthAPI(context.Background(), db, validator.TestChainConfig)
 			Expect(err).ToNot(HaveOccurred())
+
+			for i := uint64(blockHeight); i <= chainLength-trail; i++ {
+				blockToBeValidated, err := api.B.BlockByNumber(context.Background(), rpc.BlockNumber(i))
+				Expect(err).ToNot(HaveOccurred())
+				Expect(blockToBeValidated).ToNot(BeNil())
+
+				err = validator.ValidateBlock(blockToBeValidated, api.B, i)
+				Expect(err).ToNot(HaveOccurred())
+
+				err = validator.ValidateReferentialIntegrity(db, i)
+				Expect(err).ToNot(HaveOccurred())
+			}
 		})
 	})
 })
Author	SHA1	Message	Date
Michael	52979bf36b	updated dependencies for statediffing geth 1.10.23 (#21 ) * updated dependencies for statediffing geth 1.10.23 * another ref to GenesisBlockForTesting	2022-09-02 14:44:47 -04:00
prathamesh0	acf8e6f96a	Make `writeStateDiffAt` call on missing block and add metrics (#20 ) * Add chainID flag and make chainConfig optional * Add prometheus metrics to monitor validation progress * Make writeStateDiffAt calls on missing blocks * Update docs and config improvements * Use standard logger * Add copyright to files * Upgrade dependencies	2022-08-23 13:05:50 +05:30
Michael	716978e23c	Merge pull request #19 from vulcanize/geth_1_10_21_wip geth 1.10.21 update	2022-08-03 14:51:29 -04:00
Michael Shaw	e1a92a978e	go-quic not 1.19 ready for both checks	2022-08-03 14:42:06 -04:00
Michael Shaw	8e3bc49407	go-quic not 1.19 ready	2022-08-03 14:36:47 -04:00
Michael Shaw	3804f52aaf	ipld-eth-server missed version update	2022-08-03 11:37:55 -04:00
Michael Shaw	47ef12426d	geth 1.10.21 update	2022-08-03 11:11:22 -04:00
Michael	5c7b15f304	Merge pull request #18 from vulcanize/release-v4.1.1-alpha update for go-ethereum 1.10.20	2022-07-20 17:18:07 -04:00
Michael Shaw	55248fdd0d	update for go-ethereum 1.10.20	2022-07-19 14:57:44 -04:00
prathamesh0	5fb86ba963	Upgrade dependencies and update referential integrity checks (#17 ) * Upgrade dependencies and update referential integrity queries * Update GitHub workflow to run tests	2022-07-13 13:12:59 +05:30
Abdul Rabbani	7ae4886b9a	update geth and CICD refs (#15 ) * update geth and CICD refs * Update stack-orch * Download bin instead of building it * Fix integration Tests * Update DB Ref	2022-06-20 09:37:15 -04:00
nabarun	b11ace0711	Revert changes to perform referential integrity tests (#16 )	2022-06-20 14:20:14 +05:30
Abdul Rabbani prathamesh Ashwin Phatak	d111dd85db	Feature/update go geth sharding (#13 ) * Upgrade Geth, Go and add CICD * update ipld-eth-server version * Track validation progress on a channel * Add integration tests * Setup validator config and update instructions to run tests locally * Update readme and tests * Update test to use v4 Infrastructure * Inlcude the env file * Fix config file write * Update DB configuration Co-authored-by: prathamesh0 <prathamesh.musale0@gmail.com> Co-authored-by: Ashwin Phatak <ashwinpphatak@gmail.com>	2022-06-03 09:28:02 -04:00
prathamesh0	cc935dc97b	Add checks to validate referential integrity in the data (#7 ) * Add checks to validate referential integrity * Add unit tests for referential integrity validation * Use EXISTS in referential integrity validation queries	2022-06-01 11:29:53 +05:30
Ashwin Phatak	effbdc3f58	Merge pull request #9 from deep-stack/pm-merge-perpetual Merge changes to make validation process perpetual	2022-05-31 13:44:36 +05:30
prathamesh	31cc17b70f	Merge changes to make validation process perpetual	2022-05-31 13:09:45 +05:30
prathamesh0	6eeaa2c8bd	Make validation process perpetual (#8 ) * Make validation process perpetual * Get sleep interval from config * Add license * Gracefully shutdown validator on interrupt * Change default trail and sleep values	2022-05-31 11:59:49 +05:30