Documentation update

docs/anatomy.rst

@@ -4,4 +4,4 @@
 Plugin Anatomy
 ==============
 
-.. todo:: fill in disections of archetypal plugins 
+.. todo:: fill in disections of archetypal plugins (Philip: Are you doing this, or am I?)

docs/api.rst

@@ -4,6 +4,9 @@
 API
 ===
 
+PluGeth Hooks
++++++++++++++
+
 Plugins for Plugeth use Golang's `Native Plugin System`_. Plugin modules must export variables using specific names and types. These will be processed by the plugin loader, and invoked at certain points during Geth's operations.
 
 Flags
@@ -62,27 +65,30 @@ Tracers
              alter the
              results of the EVM execution in unpredictable ways. Additopackage main
 
-import (
+
+.. code-block:: go
+
+    import (
       "github.com/openrelayxyz/plugeth-utils/core"
       "gopkg.in/urfave/cli.v1"
-)
+    )
 
-var (
+    var (
       log core.Logger
-)
+    )
 
-type myservice struct{}
+    type myservice struct{}
 
-func (*myservice) Hello() string {
+    func (*myservice) Hello() string {
       return "Hello world"
-}
+    }
 
-func Initialize(ctx *cli.Context, loader core.PluginLoader, logger core.Logger) {
+    func Initialize(ctx *cli.Context, loader core.PluginLoader, logger core.Logger) {
       log = logger
       log.Info("Initialized hello")
-}
+    }
 
-func GetAPIs(node core.Node, backend core.Backend) []core.API {
+    func GetAPIs(node core.Node, backend core.Backend) []core.API {
       defer log.Info("APIs Initialized")
       return []core.API{
         {
@@ -92,30 +98,36 @@ func GetAPIs(node core.Node, backend core.Backend) []core.API {
           Public:    true,
         },
       }
-}
+    }
-package main
 
-import (
+
+.. todo:: Why do we have two copies of this? Also, these are GetAPIs examples, not tracers
+
+.. code-block:: go
+
+    package main
+
+    import (
       "github.com/openrelayxyz/plugeth-utils/core"
       "gopkg.in/urfave/cli.v1"
-)
+    )
 
-var (
+    var (
       log core.Logger
-)
+    )
 
-type myservice struct{}
+    type myservice struct{}
 
-func (*myservice) Hello() string {
+    func (*myservice) Hello() string {
       return "Hello world"
-}
+    }
 
-func Initialize(ctx *cli.Context, loader core.PluginLoader, logger core.Logger) {
+    func Initialize(ctx *cli.Context, loader core.PluginLoader, logger core.Logger) {
       log = logger
       log.Info("Initialized hello")
-}
+    }
 
-func GetAPIs(node core.Node, backend core.Backend) []core.API {
+    func GetAPIs(node core.Node, backend core.Backend) []core.API {
       defer log.Info("APIs Initialized")
       return []core.API{
         {
@@ -125,12 +137,15 @@ func GetAPIs(node core.Node, backend core.Backend) []core.API {
           Public:    true,
         },
       }
-}
+    }
-nally, some objects may be reused acress calls, so data you wish to capture should be copied rather than retianed by reference. 
+
+Internally, some objects may be reused across calls, so data you wish to capture should be copied rather than retained by reference.
 
 LiveTracer
 ----------
 
+.. todo:: Let's leave this out until we can have a more detailed implementation.
+
 * **Name:** LiveTracers
 * **Type:** core.Tracer
 * **Behavior:** This tracer is used for tracing transactions as they are processed within blocks. Note that if a block does not validate, some transactions may be processed that don't end up in blocks, so be sure to check transactions against finalized blocks.
@@ -148,7 +163,7 @@ The GetAPIs function itself will generally be fairly brief, and will looks somet
 
 .. code-block:: go
 
-	``func GetAPIs(stack *node.Node, backend core.Backend) []core.API {
+    func GetAPIs(stack *node.Node, backend core.Backend) []core.API {
       return []rpc.API{
         {
           Namespace: "mynamespace",
@@ -157,7 +172,7 @@ The GetAPIs function itself will generally be fairly brief, and will looks somet
           Public:		true,
         },
       }
-      }``
+    }
 
 The bulk of the implementation will be in the ``MyService`` struct. MyService should be a struct with public functions. These functions can have two different types of signatures:
 
@@ -169,18 +184,280 @@ A very simple MyService might look like:
 
 .. code-block:: go
 
-	``type MyService struct{}
+  type MyService struct{}
 
     func (h MyService) HelloWorld(ctx context.Context) string {
       return "Hello World"
-	  }``
+    }
 
 And the client could access this with an rpc call to
-``mynamespace_helloworld``
+``mynamespace_helloWorld``
+
+Injected APIs
++++++++++++++
+
+In addition to hooks that get invoked by Geth, several objects are injected that give you access to additional information.
+
+Backend Object
+--------------
+
+The ``core.Backend`` object is injected by the ``InitializeNode()`` and ``GetAPI()`` functions. It offers the following functions:
+
+Downloader
+^^^^^^^^^^
+``Downloader() Downloader``
+
+Returns a Downloader objects, which can provide Syncing status
+
+SuggestGasTipCap
+^^^^^^^^^^^^^^^^
+``SuggestGasTipCap(ctx context.Context) (*big.Int, error)``
+
+Suggests a Gas tip for the current block.
+
+ExtRPCEnabled
+^^^^^^^^^^^^^
+``ExtRPCEnabled() bool``
+
+Returns whether RPC external RPC calls are enabled.
+
+RPCGasCap
+^^^^^^^^^
+``RPCGasCap() uint64``
+
+Returns the maximum Gas available to RPC Calls.
+
+RPCTxFeeCap
+^^^^^^^^^^^
+``RPCTxFeeCap() float64``
+
+Returns the maximum transaction fee for a transaction submitted via RPC.
+
+UnprotectedAllowed
+^^^^^^^^^^^^^^^^^^
+``UnprotectedAllowed() bool``
+
+Returns whether or not unprotected transactions can be transmitted through this
+node via RPC.
+
+SetHead
+^^^^^^^
+``SetHead(number uint64)``
+
+Resets the head to the specified block number.
+
+HeaderByNumber
+^^^^^^^^^^^^^^
+``HeaderByNumber(ctx context.Context, number int64) ([]byte, error)``
+
+Returns an RLP encoded block header for the specified block number.
+
+The RLP encoded response can be decoded into a `plugeth-utils/restricted/types.Header` object.
+
+HeaderByHash
+^^^^^^^^^^^^
+``HeaderByHash(ctx context.Context, hash Hash) ([]byte, error)``
+
+Returns an RLP encoded block header for the specified block hash.
+
+The RLP encoded response can be decoded into a `plugeth-utils/restricted/types.Header` object.
+
+CurrentHeader
+^^^^^^^^^^^^^
+``CurrentHeader() []byte``
+
+Returns an RLP encoded block header for the current block.
+
+The RLP encoded response can be decoded into a `plugeth-utils/restricted/types.Header` object.
+
+CurrentBlock
+^^^^^^^^^^^^
+``CurrentBlock() []byte``
+
+Returns an RLP encoded full block for the current block.
+
+The RLP encoded response can be decoded into a `plugeth-utils/restricted/types.Block` object.
 
 
+BlockByNumber
+^^^^^^^^^^^^^
+``BlockByNumber(ctx context.Context, number int64) ([]byte, error)``
+
+
+Returns an RLP encoded full block for the specified block number.
+
+The RLP encoded response can be decoded into a `plugeth-utils/restricted/types.Block` object.
+BlockByHash
+^^^^^^^^^^^
+``BlockByHash(ctx context.Context, hash Hash) ([]byte, error)``
+
+Returns an RLP encoded full block for the specified block hash.
+
+The RLP encoded response can be decoded into a `plugeth-utils/restricted/types.Block` object.
+
+GetReceipts
+^^^^^^^^^^^
+``GetReceipts(ctx context.Context, hash Hash) ([]byte, error)``
+
+Returns an JSON encoded list of receipts for the specified block hash.
+
+The JSON encoded response can be decoded into a `plugeth-utils/restricted/types.Receipts` object.
+
+
+GetTd
+^^^^^
+``GetTd(ctx context.Context, hash Hash) *big.Int``
+
+Returns the total difficulty for the specified block hash.
+
+SubscribeChainEvent
+^^^^^^^^^^^^^^^^^^^
+``SubscribeChainEvent(ch chan<- ChainEvent) Subscription``
+
+Subscribes the provided channel to new chain events.
+
+SubscribeChainHeadEvent
+^^^^^^^^^^^^^^^^^^^^^^^
+``SubscribeChainHeadEvent(ch chan<- ChainHeadEvent) Subscription``
+
+Subscribes the provided channel to new chain head events.
+
+SubscribeChainSideEvent
+^^^^^^^^^^^^^^^^^^^^^^^
+``SubscribeChainSideEvent(ch chan<- ChainSideEvent) Subscription``
+
+Subscribes the provided channel to new chain side events.
+
+SendTx
+^^^^^^
+``SendTx(ctx context.Context, signedTx []byte) error``
+
+Sends an RLP encoded, signed transaction to the network.
+
+GetTransaction
+^^^^^^^^^^^^^^
+``GetTransaction(ctx context.Context, txHash Hash) ([]byte, Hash, uint64, uint64, error)``
+
+Returns an RLP encoded transaction at the specified hash, along with the hash and number of the included block, and the transaction's position within that block.
+
+GetPoolTransactions
+^^^^^^^^^^^^^^^^^^^
+``GetPoolTransactions() ([][]byte, error)``
+
+Returns a list of RLP encoded transactions found in the mempool
+
+GetPoolTransaction
+^^^^^^^^^^^^^^^^^^
+``GetPoolTransaction(txHash Hash) []byte``
+
+Returns the RLP encoded transaction from the mempool at the specified hash.
+
+GetPoolNonce
+^^^^^^^^^^^^
+``GetPoolNonce(ctx context.Context, addr Address) (uint64, error)``
+
+Returns the nonce of the last transaction for a given address, including
+transactions found in the mempool.
+
+Stats
+^^^^^
+``Stats() (pending int, queued int)``
+
+Returns the number of pending and queued transactions in the mempool.
+
+TxPoolContent
+^^^^^^^^^^^^^
+``TxPoolContent() (map[Address][][]byte, map[Address][][]byte)``
+
+Returns a map of addresses to the list of RLP encoded transactions pending in
+the mempool, and queued in the mempool.
+
+SubscribeNewTxsEvent
+^^^^^^^^^^^^^^^^^^^^
+``SubscribeNewTxsEvent(chan<- NewTxsEvent) Subscription``
+
+Subscribe to a feed of new transactions added to the mempool.
+
+GetLogs
+^^^^^^^
+``GetLogs(ctx context.Context, blockHash Hash) ([][]byte, error)``
+
+Returns a list of RLP encoded logs found in the specified block.
+
+SubscribeLogsEvent
+^^^^^^^^^^^^^^^^^^
+``SubscribeLogsEvent(ch chan<- [][]byte) Subscription``
+
+Subscribe to logs included in a confirmed block.
+
+SubscribePendingLogsEvent
+^^^^^^^^^^^^^^^^^^^^^^^^^
+``SubscribePendingLogsEvent(ch chan<- [][]byte) Subscription``
+
+Subscribe to logs from pending transactions.
+
+SubscribeRemovedLogsEvent
+^^^^^^^^^^^^^^^^^^^^^^^^^
+``SubscribeRemovedLogsEvent(ch chan<- []byte) Subscription``
+
+Subscribe to logs removed from the canonical chain in reorged blocks.
+
+
+Node Object
+-----------
+
+The ``core.Node`` object is injected by the ``InitializeNode()`` and ``GetAPI()`` functions. It offers the following functions:
+
+Server
+^^^^^^
+``Server() Server``
+
+The Server object provides access to ``server.PeerCount()``, the number of peers connected to the node.
+
+DataDir
+^^^^^^^
+``DataDir() string``
+
+Returns the Ethereuem datadir.
+
+InstanceDir
+^^^^^^^^^^^
+``InstanceDir() string``
+
+Returns the instancedir used by the protocol stack.
+
+IPCEndpoint
+^^^^^^^^^^^
+``IPCEndpoint() string``
+
+The path of the IPC Endpoint for this node.
+
+HTTPEndpoint
+^^^^^^^^^^^^
+``HTTPEndpoint() string``
+
+The url of the HTTP Endpoint for this node.
+
+WSEndpoint
+^^^^^^^^^^
+``WSEndpoint() string``
+
+The url of the websockets Endpoint for this node.
+
+
+ResolvePath
+^^^^^^^^^^^
+``ResolvePath(x string) string``
+
+Resolves a path within the DataDir.
 
 
 .. _*cli.Context: https://pkg.go.dev/github.com/urfave/cli#Context
 .. _flag.FlagSet: https://pkg.go.dev/flag#FlagSet
 .. _Native Plugin System: https://pkg.go.dev/plugin
+
+Logger
+------
+
+The Logger object is injected by the ``Initialize()`` function. It implements
+logging based on the interfaces of `Log15 <https://github.com/inconshreveable/log15>`_.

docs/build.rst

@@ -49,7 +49,7 @@ Navigate to ``plugethPlugins/packages/hello``. Inside you will see a ``main.go``
 
    $ go build -buildmode=plugin
 
-This will compile the plugin and produce a ``hello.so`` file. Move ``hello.so`` into ``~/.ethereum/plugins`` . In order to use this plugin geth will need to be started with a ``http.api=mymamespace`` flag. Additionally you will need to include a ``--http`` flag in order to access the standard json rpc methods. 
+This will compile the plugin and produce a ``hello.so`` file. Move ``hello.so`` into ``~/.ethereum/plugins`` . In order to use this plugin geth will need to be started with a ``http.api=mynamespace`` flag. Additionally you will need to include a ``--http`` flag in order to access the standard json rpc methods.
 
 Once geth has started you should see that the first ``INFO`` log reads: ``initialized hello`` . A new json rpc method, called hello, has been been appended to the list of available json rpc methods. In order to access this method you will need to ``curl`` into the network with this command:
 
@@ -61,7 +61,7 @@ You should see that the network has responded with:
 
 .. code-block:: shell
 
-   ``{"jsonrpc":"2.0","id":0,"result":"Hello world"}``
+   {"jsonrpc":"2.0","id":0,"result":"Hello world"}
 
 Congradulations. You have just built and run your first Plugeth plugin.
 

docs/contact.rst

@@ -7,4 +7,4 @@ Get in touch with us
 
 We want to hear from you! The best way to reach the PluGeth team is through our Discord server. Drop in, say hello, we are anxious to hear your ideas and help work through any problems you may be having.
 
-**todo: best way to link to our discord?**
+`Join our Discord <https://discord.gg/J3tQMWCVPn>`_

docs/core_restricted.rst

@@ -1,8 +1,11 @@
 .. _core_restricted:
 
-============================================
+=========================
-Core vs Restricted packages in Plugeth-utils
+PluGeth-utils Subpackages
-============================================
+=========================
 
+PluGeth-utils is separated into two main packages: core, and restricted.
 
-.. todo:: need explinations of core vs restircted functionality. 		   what, why, how. 
+The `core` package has been implemented by the Rivet team, and is licensed under the MIT license, allowing it to be used in open source and closed source plugins alike. Nearly all plugins will need to import plugeth-utils/core in order to
+
+The `restricted` package copies code from the go-ethereum project, which means it must be licensed under the LGPL license. If you import plugeth-utils/restricted, you must be sure that your plugin complies with requirements of linking to LGPL code, which will usually require making your source code available to anyone you distribute the plugin to.

docs/plugin_loader.rst

@@ -4,4 +4,59 @@
 Plugin Loader
 =============
 
-.. todo:: breakdown of plugin loader function
+The Plugin Loader is provided to each Plugin through the Initialize()``
+function. It provides plugins with:
+
+
+Lookup
+======
+``Lookup(name string, validate func(interface{}) bool) []interface{}``
+
+Returns a list of values from plugins identified by ``name``, which match the
+provided ``validate`` predicate. For example:
+
+
+.. code-block:: go
+
+    pl.Lookup("Version", func(item interface{}) bool {
+      _, ok := item.(int)
+      return ok
+    })
+
+Would return a list of ``int`` objects named ``Version`` in any loaded plugins.
+This can enable Plugins to interact with each other, accessing values and
+functions implemented in other plugins.
+
+GetFeed
+=======
+``GetFeed() Feed``
+
+Returns a new feed that the plugin can used for publish/subscribe models.
+
+For example:
+
+.. code-block:: go
+
+    feed := pl.GetFeed()
+    go func() {
+      ch := make(chan string)
+      sub := feed.Subscribe(ch)
+      for {
+        select {
+        case item := <-ch:
+          // Do something with item
+        case err := <sub.Err():
+          log.Error("An error has occurred", "err", err)
+          sub.Unsubscribe()
+          close(ch)
+          return
+        }
+      }
+    }()
+
+    feed.Send("hello")
+    feed.Send("world")
+
+
+Note that you can send any type through a feed, but the subscribed channel and
+sent objects must be of matching types.

docs/project.rst

@@ -37,11 +37,47 @@ Plugins are packages which contain premade plugins as well as a location provide
 Dependency Scheme
 -----------------
 
-.. todo:: needs elaboration of dependency scheme
+PluGeth is separated into three packages in order to minimize dependency conflicts. Golang plugins cannot include different versions of the same packages as the program loading the plugin. If plugins had to import packages from PluGeth itself, a plugin build could only be loaded by that same version of PluGeth. By separating out the PluGeth-utils package, both PluGeth and the plugins must rely on the same version of PluGeth-utils, but plugins can be compatible with any version of PluGeth compiled with the same version of PluGeth-utils.
 
+PluGeth builds will follow the naming convention:
 
+.. code-block:: shell
 
+   geth-$PLUGETH_UTILS_VERSION-$GETH_VERSION-$RELEASE
 
+For example:
+
+.. code-block:: shell
+
+   geth-0.1.0-1.10.8-0
+
+Tells us that:
+
+* PluGeth-utils version is 0.1.0
+* Geth version is 1.10.8
+* This is the first release with that combination of dependencies.
+
+Plugin builds will follow the naming convention:
+
+.. code-block:: shell
+
+   $PLUGIN_NAME-$PLUGETH_UTILS_VERSION-$PLUGIN_VERSION
+
+For example:
+
+.. code-block:: shell
+
+   blockupdates-0.1.0-1.0.2
+
+Tells us that:
+
+* The plugin is "blockupdates"
+* The PluGeth-utils version is 0.1.0
+* The plugin version is 1.0.2
+
+When a Geth update comes out, you can expect a release of `geth-0.1.0-1.10.9-0`, which will be compatible with the same set of plugins.
+
+When PluGeth upgrades are necessary, plugins will need to be recompiled. Whenever possible, we will try to avoid forcing plugins to be recompiled for an immediate Geth upgrade. For example, if we have geth-0.1.0-1.10.8, and upgrade PluGeth-utils, we will have a geth-0.1.1-1.10.8, followed by a geth-0.1.1-1.10.9. This will give users time to upgrade plugins from PluGeth-utils 0.1.0 to 0.1.1 while staying on Geth 1.10.8, and when it is time to upgrade to Geth 1.10.9 they can continue using the plugins they were using with geth 1.10.8. Depending on upgrades to Geth, it may not always be possible to maintain compatibility with existing PluGeth versions, which will be noted in release notes.
 
 
 .. _obscures security updates as optimizations: https://blog.openrelay.xyz/vulnerability-lifecycle-framework-geth/

docs/types.rst

@@ -19,15 +19,16 @@ A subcommand redifines the total behavior of Geth and could stand on its own.
 Tracers
 -------
 
-**Tracers vs LiveTracers. 
+Tracers are used to collect information from transaction execution, through Geth's ``debug_traceCall``, ``debug_traceTransaction``, and ``debug_traceBlock``. While standard Geth allows you to specify custom tracers in JavaScript, tracers written as plugins can be made much more performant.
+
+While normal tracers run in the context of an RPC call, Live Tracers run on transactions in the course of Geth's block validation process. The methods for a tracer and live tracer are the same, but while tracers expose information through a RPC calls, live tracers simply provide an opportunity to aggregate information and have no inherent method for providing it to a consumer.
 
-Tracers rely on historic data whereas LiveTracers run concurent with Geth's verification system. LiveTracers are more generic in that they cannot control the way in which they recieve information. 
 
 
 Subscriptions
 -------------
 
-A subscription must take a context.context as an argument and return a channel and an error. Subscriptions require a stable connection and return contant information. Subscriptions require a websocket connection and pass a json argument such as: ``{"jsonrpc":"2.0", "id": 0, "method": "namespace_subscribe", "params": ["subscriptionName", $args...]}``
+A subscription must take a context.context as an argument and return a channel and an error. Subscriptions require a stable connection and return a stream of information. Subscriptions require a websocket connection and pass a json argument such as: ``{"jsonrpc":"2.0", "id": 0, "method": "namespace_subscribe", "params": ["subscriptionName", $args...]}``
 
 .. NOTE:: Plugins are not limited to a singular functionality and can be customized to operate as hybrids of the above archtypes.