Move documentation -> docs, don't commit build

This commit is contained in:
Austin Roberts
2021-10-04 10:04:01 -05:00
parent d27c507112
commit acde69034b
106 changed files with 1 additions and 19715 deletions
+20
View File
@@ -0,0 +1,20 @@
# Minimal makefile for Sphinx documentation
#
# You can set these variables from the command line, and also
# from the environment for the first two.
SPHINXOPTS ?=
SPHINXBUILD ?= sphinx-build
SOURCEDIR = .
BUILDDIR = _build
# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
.PHONY: help Makefile
# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+7
View File
@@ -0,0 +1,7 @@
.. _anatomy:
==============
Plugin Anatomy
==============
.. todo:: fill in disections of archetypal plugins
+186
View File
@@ -0,0 +1,186 @@
.. _api:
===
API
===
Plugins for Plugeth use Golang's `Native Plugin System`_. Plugin modules must export variables using specific names and types. These will be processed by the plugin loader, and invoked at certain points during Geth's operations.
Flags
-----
* **Name:** Flags
* **Type:** `flag.FlagSet`_
* **Behavior:** This FlagSet will be parsed and your plugin will be able to access the resulting flags. Flags will be passed to Geth from the command line and are intended to of the plugin. Note that if any flags are provided, certain checks are disabled within Geth to avoid failing due to unexpected flags.
Subcommands
-----------
* **Name:** Subcommands
* **Type:** map[string]func(ctx `*cli.Context`_, args []string) error
* **Behavior:** If Geth is invoked with ``./geth YOUR_COMMAND``, the plugin loader will look for ``YOUR_COMMAND`` within this map, and invoke the corresponding function. This can be useful for certain behaviors like manipulating Geth's database without having to build a separate binary.
Initialize
----------
* **Name:** Initialize
* **Type:** func(*cli.Context, core.PluginLoader, core.logs )
* **Behavior:** Called as soon as the plugin is loaded, with the cli context and a reference to the plugin loader. This is your plugin's opportunity to initialize required variables as needed. Note that using the context object you can check arguments, and optionally can manipulate arguments if needed for your plugin.
.. todo:: explain that plugin could provide node.Node with
restricted.backend
InitializeNode
--------------
* **Name:** InitializeNode
* **Type:** func(core.Node, core.Backend)
* **Behavior:** This is called as soon as the Geth node is initialized. The core.Node object represents the running node with p2p and RPC capabilities, while the Backend gives you access to a wide array of data you may need to access.
Tracers
-------
* **Name:** Tracers
* **Type:** map[string]TracerResult
* **Behavior:** When calling debug.traceX functions (such as ``debug_traceCall`` and ``debug_traceTransaction``) the tracer can be specified as a key to this map and the tracer used will be the TracerResult specified here. TracerResult objects must match the interface:
.. code-block:: go
// CaptureStart is called at the start of each transaction
CaptureStart(env core.EVM, from core.Address, to core.Address, create bool, input []byte, gas uint64, value *big.Int) {}
// CaptureState is called for each opcode
CaptureState(env core.EVM, pc uint64, op core.OpCode, gas, cost uint64, scope *vm.ScopeContext, rData []byte, depth int, err error) {}
// CaptureFault is called when an error occurs in the EVM
CaptureFault(env core.EVM, pc uint64, op core.OpCode, gas, cost uint64, scope *vm.ScopeContext, depth int, err error) {}
// CaptureEnd is called at the end of each transaction
CaptureEnd(output []byte, gasUsed uint64, t time.Duration, err error) {}
// GetResult should return a JSON serializable result object to respond to the trace call
GetResult() (interface{}, error) {}
.. warning:: Modifying the values passed into tracer functions can
alter the
results of the EVM execution in unpredictable ways. Additopackage main
import (
"github.com/openrelayxyz/plugeth-utils/core"
"gopkg.in/urfave/cli.v1"
)
var (
log core.Logger
)
type myservice struct{}
func (*myservice) Hello() string {
return "Hello world"
}
func Initialize(ctx *cli.Context, loader core.PluginLoader, logger core.Logger) {
log = logger
log.Info("Initialized hello")
}
func GetAPIs(node core.Node, backend core.Backend) []core.API {
defer log.Info("APIs Initialized")
return []core.API{
{
Namespace: "mynamespace",
Version: "1.0",
Service: &myservice{},
Public: true,
},
}
}
package main
import (
"github.com/openrelayxyz/plugeth-utils/core"
"gopkg.in/urfave/cli.v1"
)
var (
log core.Logger
)
type myservice struct{}
func (*myservice) Hello() string {
return "Hello world"
}
func Initialize(ctx *cli.Context, loader core.PluginLoader, logger core.Logger) {
log = logger
log.Info("Initialized hello")
}
func GetAPIs(node core.Node, backend core.Backend) []core.API {
defer log.Info("APIs Initialized")
return []core.API{
{
Namespace: "mynamespace",
Version: "1.0",
Service: &myservice{},
Public: true,
},
}
}
nally, some objects may be reused acress calls, so data you wish to capture should be copied rather than retianed by reference.
LiveTracer
----------
* **Name:** LiveTracers
* **Type:** core.Tracer
* **Behavior:** This tracer is used for tracing transactions as they are processed within blocks. Note that if a block does not validate, some transactions may be processed that don't end up in blocks, so be sure to check transactions against finalized blocks.
The interface for a vm.Tracer is similar to a TracerResult (above), but does not require a ``GetResult()`` function.
GetAPIs
-------
* **Name:** GetAPIs
* **Type:** func(core.Node, core.Backend) []rpc.API
* **Behavior:** This allows you to register new RPC methods to run within Geth.
The GetAPIs function itself will generally be fairly brief, and will looks something like this:
.. code-block:: go
``func GetAPIs(stack *node.Node, backend core.Backend) []core.API {
return []rpc.API{
{
Namespace: "mynamespace",
Version: "1.0",
Service: &MyService{backend},
Public: true,
},
}
}``
The bulk of the implementation will be in the ``MyService`` struct. MyService should be a struct with public functions. These functions can have two different types of signatures:
* RPC Calls: For straight RPC calls, a function should have a ``context.Context`` object as the first argument, followed by an arbitrary number of JSON marshallable arguments, and return either a single JSON marshal object, or a JSON marshallable object and an error. The RPC framework will take care of decoding inputs to this function and encoding outputs, and if the error is non-nil it will serve an error response.
* Subscriptions: For subscriptions (supported on IPC and websockets), a function should have a ``context.Context`` object as the first argument followed by an arbitrary number of JSON marshallable arguments, and should return an ``*rpc.Subscription`` object. The subscription object can be created with ``rpcSub := notifier.CreateSubscription()``, and JSON marshallable data can be sent to the subscriber with ``notifier.Notify(rpcSub.ID, b)``.
A very simple MyService might look like:
.. code-block:: go
``type MyService struct{}
func (h MyService) HelloWorld(ctx context.Context) string {
return "Hello World"
}``
And the client could access this with an rpc call to
``mynamespace_helloworld``
.. _*cli.Context: https://pkg.go.dev/github.com/urfave/cli#Context
.. _flag.FlagSet: https://pkg.go.dev/flag#FlagSet
.. _Native Plugin System: https://pkg.go.dev/plugin
+75
View File
@@ -0,0 +1,75 @@
.. _build:
================
Build and Deploy
================
.. contents:: :local:
Setting up the environment
**************************
.. NOTE:: Plugeth is built on a fork of `Geth`_ and as such requires familiarity with `Go`_ and a funtional `environment`_ in which to build Go projects. Thankfully for everyone Go provides a compact and useful `tutorial`_ as well as a `space for practice`_.
PluGeth is an application built in three seperate repositories.
* `PluGeth`_
* `PluGethUtils`_
* `PluGethPlugins`_
Once all three are cloned into their own directories you are ready to begin. First we need to build geth though the PluGeth project. Navigate to ``plugeth/cmd/geth`` and run:
.. code-block:: shell
$ go get
This will download all dependencies needed for the project. This process will take a moment or two the first time through. Next run:
.. code-block:: shell
$ go build
Once this is complete you should see a ``.ethereum`` folder in your home directory.
At this point you are ready to start downloading local ethereum nodes. In order to do so, from ``plugeth/cmd/geth`` run:
.. code-block:: shell
$ ./geth
.. NOTE:: ``./geth`` is the primary command to build a *Mainnet* node. Building a mainnet node requires at least 8 GB RAM, 2 CPUs, and 350 GB of SSD disks. However, dozens of available flags will change the behavior of whichever network you choose to connect to. ``--help`` is your friend.
Build a plugin
**************
Navigate to ``plugethPlugins/packages/hello``. Inside you will see a ``main.go`` file. From this location run:
.. code-block:: shell
$ go build -buildmode=plugin
This will compile the plugin and produce a ``hello.so`` file. Move ``hello.so`` into ``~/.ethereum/plugins`` . In order to use this plugin geth will need to be started with a ``http.api=mymamespace`` flag. Additionally you will need to include a ``--http`` flag in order to access the standard json rpc methods.
Once geth has started you should see that the first ``INFO`` log reads: ``initialized hello`` . A new json rpc method, called hello, has been been appended to the list of available json rpc methods. In order to access this method you will need to ``curl`` into the network with this command:
.. code-block:: shell
$ curl 127.0.0.1:8545 -H "Content-Type: application/json" --data '{"jsonrpc":"2.0","method":"mynamespace_hello","params":[],"id":0}'
You should see that the network has responded with:
.. code-block:: shell
``{"jsonrpc":"2.0","id":0,"result":"Hello world"}``
Congradulations. You have just built and run your first Plugeth plugin.
.. _space for practice: https://tour.golang.org/welcome/1
.. _tutorial: https://tour.golang.org/welcome/1
.. _environment: https://golang.org/doc/code
.. _Go: https://golang.org/doc/
.. _Geth: https://geth.ethereum.org/
.. _PluGeth: https://github.com/openrelayxyz/plugeth
.. _PluGethUtils: https://github.com/openrelayxyz/plugeth-utils
.. _PluGethPlugins: https://github.com/openrelayxyz/plugeth-plugins
+65
View File
@@ -0,0 +1,65 @@
# Configuration file for the Sphinx documentation builder.
#
# This file only contains a selection of the most common options. For a full
# list see the documentation:
# https://www.sphinx-doc.org/en/master/usage/configuration.html
# -- Path setup --------------------------------------------------------------
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
# import os
# import sys
# sys.path.insert(0, os.path.abspath('.'))
# -- Project information -----------------------------------------------------
project = 'Plugeth'
copyright = '2021, Philip Morlier'
author = 'Philip Morlier, Austin Roberts'
# The full version, including alpha/beta/rc tags
release = 'Austin Roberts'
# -- General configuration ---------------------------------------------------
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
'sphinx.ext.intersphinx',
'sphinx.ext.autodoc',
'sphinx.ext.doctest',
'sphinx.ext.todo',
]
todo_include_todos=True
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
# -- Options for HTML output -------------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
import sphinx_rtd_theme
html_theme = 'sphinx_rtd_theme'
html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
+10
View File
@@ -0,0 +1,10 @@
.. _contact:
====================
Get in touch with us
====================
We want to hear from you! The best way to reach the PluGeth team is through our Discord server. Drop in, say hello, we are anxious to hear your ideas and help work through any problems you may be having.
**todo: best way to link to our discord?**
+8
View File
@@ -0,0 +1,8 @@
.. _core_restricted:
============================================
Core vs Restricted packages in Plugeth-utils
============================================
.. todo:: need explinations of core vs restircted functionality. what, why, how.
+64
View File
@@ -0,0 +1,64 @@
.. _hooks:
============
Plugin Hooks
============
The key to understanding a plugin is understanding its corresponding hook. Broadly, plugin hooks can be thought of as functions which deliver a function signiture to the **plugin loader**. The signature matches that of the function(s) called by the plugin and describes the data that is being captured and how the plugin will deliver said data upon invocation.
The public plugin hook function should follow the naming convention ``Plugin$HookName``. The first argument should be a `` core.PluginLoader``, followed by any arguments required by the functions to be provided by any plugins implementing this hook.
**Public and Private Hook Functions**
Each hook provides both a Public and private version. The private plugin hook function should bear the same name as the public plugin hook function, but with a lower case first letter. The signature should match the public plugin hook function, except that the first argument referencing the PluginLoader should be removed. It should invoke the public plugin hook function on plugins.DefaultPluginLoader. It should always verify that the DefaultPluginLoader is non-nil, log warning and return if the DefaultPluginLoader has not been initialized.
**Invocation**
the private plugin hook function should be invoked, with the appropriate arguments, in a single line within the Geth codebase inorder to minimize unexpected conflicts merging upstream Geth into plugeth.
Selected Hooks
--------------
`StateUpdate`_
************
`Invocation`_
The state update plugin provides a snapshot of the state subsystem in the form of a a stateUpdate object. The stateUpdate object contains all information transformed by a transaction but not the transaction itself.
Invoked for each new block, StateUpdate provides the changes to the blockchain state. root corresponds to the state root of the new block. parentRoot corresponds to the state root of the parent block. destructs serves as a set of accounts that self-destructed in this block. accounts maps the hash of each account address to the SlimRLP encoding of the account data. storage maps the hash of each account to a map of that account's stored data.
.. warning:: StateUpdate is only called if Geth is running with
``-snapshots=true``. This is the default behavior for Geth, but if you are explicitly running with ``--snapshot=false`` this function will not be invoked.
`AppendAncient`_
*************
Invoked when the freezer moves a block from LevelDB to the ancients database. ``number`` is the number of the block. ``hash`` is the 32 byte hash of the block as a raw ``[]byte``. ``header``, ``body``, and ``receipts`` are the RLP encoded versions of their respective block elements. ``td`` is the byte encoded total difficulty of the block.
`NewHead`_
**********
Invoked when a new block becomes the canonical latest block. Note that if several blocks are processed in a group (such as during a reorg) this may not be called for each block. You should track the prior latest head if you need to process intermediate blocks.
`GetRPCCalls`_
************
Invoked when the RPC handler registers a method call. returns the call ``id``, method ``name``, and any ``params`` that may have been passed in.
**I am struggling all of assudden with if this is really this best course of action. Giving all of this reference, seems to beg the -utils vs plugeth conversation and I am not sure that is a smart thing to get into.**
.. _StateUpdate: https://github.com/openrelayxyz/plugeth/blob/develop/core/state/plugin_hooks.go
.. _Invocation: https://github.com/openrelayxyz/plugeth/blob/develop/core/state/statedb.go#L955
.. _AppendAncient: https://github.com/openrelayxyz/plugeth/blob/develop/core/rawdb/plugin_hooks.go
.. _GetRPCCalls: https://github.com/openrelayxyz/plugeth/blob/develop/rpc/plugin_hooks.go
.. _NewHead: https://github.com/openrelayxyz/plugeth/blob/develop/core/plugin_hooks.go#L108
+54
View File
@@ -0,0 +1,54 @@
.. Plugeth documentation master file, created by
sphinx-quickstart on Tue Sep 21 16:08:24 2021.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
=======
PluGeth
=======
PluGeth is a fork of the `Go Ethereum Client Geth`_ that implements a plugin architecture, allowing developers to extend Geth's capabilities in a number of different ways using plugins, rather than having to create additional, new forks of Geth.
.. warning:: Right now PluGeth is in early development. We are
still settling on some of the plugin APIs, and are
not yet making official releases. From an operational
perspective, PluGeth should be as stable as upstream Geth less whatever isstability is added by plugins you might run. But if you plan to run PluGeth today, be aware that furture updates will likely break you plugins.
Table of Contents
*****************
.. toctree::
:maxdepth: 1
:caption: Overview
project
types
anatomy
.. toctree::
:maxdepth: 1
:caption: Tutorials
build
.. toctree::
:maxdepth: 1
:caption: Reference
system_req
plugin_loader
hooks
core_restricted
api
.. toctree::
:maxdepth: 1
:caption: Contact
contact
.. _Go Ethereum Client Geth: https://github.com/ethereum/go-ethereum
+35
View File
@@ -0,0 +1,35 @@
@ECHO OFF
pushd %~dp0
REM Command file for Sphinx documentation
if "%SPHINXBUILD%" == "" (
set SPHINXBUILD=sphinx-build
)
set SOURCEDIR=.
set BUILDDIR=_build
if "%1" == "" goto help
%SPHINXBUILD% >NUL 2>NUL
if errorlevel 9009 (
echo.
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
echo.installed, then set the SPHINXBUILD environment variable to point
echo.to the full path of the 'sphinx-build' executable. Alternatively you
echo.may add the Sphinx directory to PATH.
echo.
echo.If you don't have Sphinx installed, grab it from
echo.http://sphinx-doc.org/
exit /b 1
)
%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
goto end
:help
%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
:end
popd
+7
View File
@@ -0,0 +1,7 @@
.. _plugeth:
=======
PluGeth
=======
**to do: discussion on plugeth repository**
+7
View File
@@ -0,0 +1,7 @@
.. _plugin_hooks:
============
Plugin Hooks
============
.. todo:: guide to writing plugin hooks
+7
View File
@@ -0,0 +1,7 @@
.. _plugin_loader:
=============
Plugin Loader
=============
.. todo:: breakdown of plugin loader function
+7
View File
@@ -0,0 +1,7 @@
.. _plugins:
===============
PluGeth Plugins
===============
**todo: copy on plugeth plugins repo**
+50
View File
@@ -0,0 +1,50 @@
.. _project:
==============
Project Design
==============
Design Goals
The upstream Geth client exists primarily to serve as a client for the Ethereum mainnet, though it also supports a number of popular testnets. Supporting the Ethereum mainnet is a big enough challenge in its own right that the Geth team generally avoids changes to support other networks, or to provide features only a small handful of users would be interested in.
The result is that many projects have forked Geth. Some implement their own consensus protocols or alter the behavior of the EVM to support other networks. Others are designed to extract information from the Ethereum mainnet in ways the standard Geth client does not support.
Creating numerous different forks to fill a variety of different needs comes with a number of drawbacks. Forks tend to drift apart from each other. Many networks that forked from Geth long ago have stopped merging updates from Geth; this makes some sense, given that those networks have moved in different directions than Geth and merging upstream changes while properly maintaining consensus rules of an existing network could prove quite challenging. But not merging changes from upstream can mean that security updates are easily missed, especially when the upstream team `obscures security updates as optimizations`_ as a matter of process.
PluGeth aims to provide a single Geth fork that developers can choose to extend rather than forking the Geth project. Out of the box, PluGeth behaves exactly like upstream Geth, but by installing plugins written in Golang, developers can extend its functionality in a wide variety of ways.
Three Repositories
------------------
PluGeth is an application built in three repositories:
`PluGeth`_
**********
The largest of the three Repositories, PluGeth is a fork of Geth which has been modified to enable a plugin architecture. The Plugin loader, wrappers, and hooks all reside in this repository.
`PluGeth-Utils`_
***************
Utils are small packages used to develop PluGeth plugins without Geth dependencies. For a more detailed analysis of the reasons see **here**
`PluGeth-Plugins`_
*****************
Plugins are packages which contain premade plugins as well as a location provided for storing new custom plugins.
Dependency Scheme
-----------------
.. todo:: needs elaboration of dependency scheme
.. _obscures security updates as optimizations: https://blog.openrelay.xyz/vulnerability-lifecycle-framework-geth/
.. _PluGeth: https://github.com/openrelayxyz/plugeth
.. _PluGeth-Utils: https://github.com/openrelayxyz/plugeth-utils
.. _PluGeth-Plugins: https://github.com/openrelayxyz/plugeth-plugin
+9
View File
@@ -0,0 +1,9 @@
.. _system_req:
===================
System Requirements
===================
System requirements will vary depending on which network you are connecting to. On the Ethereum mainnet, you should have at least 8 GB RAM, 2 CPUs, and 350 GB of SSD disks.
PluGeth relies on Golang's Plugin implementation, which is only supported on Linux, FreeBSD, and macOS. Windows support is unlikely to be added in the foreseeable future.
+34
View File
@@ -0,0 +1,34 @@
.. _types:
======================
Basic Types of Plugins
======================
RPC Methods
-----------
In general these plugins provide new json rpc methods. They will requirre an initialize function that takes a context, loader, and logger as arguments. They will also need a GetAPIs function that takes a node and backend as arguments and returns an API.
.. NOTE:: In order to be made available a flag: ``http.api=<the name of your service>`` will need to be appended to the command line upon starting Geth.
Subcommand
------------
A subcommand redifines the total behavior of Geth and could stand on its own.
Tracers
-------
**Tracers vs LiveTracers.
Tracers rely on historic data whereas LiveTracers run concurent with Geth's verification system. LiveTracers are more generic in that they cannot control the way in which they recieve information.
Subscriptions
-------------
A subscription must take a context.context as an argument and return a channel and an error. Subscriptions require a stable connection and return contant information. Subscriptions require a websocket connection and pass a json argument such as: ``{"jsonrpc":"2.0", "id": 0, "method": "namespace_subscribe", "params": ["subscriptionName", $args...]}``
.. NOTE:: Plugins are not limited to a singular functionality and can be customized to operate as hybrids of the above archtypes.
**todo: this page needs a lot of work**
+7
View File
@@ -0,0 +1,7 @@
.. _utils:
=============
PluGeth Utils
=============
**todo: copy on plugeth utils**