Merge pull request #8878 from ethereum/selecting-yul-optimisation-steps-in-strict-assembly-mode

Selecting Yul optimisation steps in strict assembly mode
This commit is contained in:
chriseth 2020-05-11 16:05:57 +02:00 committed by GitHub
commit 3312150d7f
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
13 changed files with 310 additions and 68 deletions

View File

@ -5,6 +5,7 @@ Language Features:
Compiler Features:
* Commandline Interface: Don't ignore `--yul-optimizations` in assembly mode.

View File

@ -1041,14 +1041,16 @@ Optimization step sequence
--------------------------
By default the Yul optimizer applies its predefined sequence of optimization steps to the generated assembly.
You can override this sequence and supply your own using the `--yul-optimizations` option when compiling
in Solidity mode:
You can override this sequence and supply your own using the ``--yul-optimizations`` option:
.. code-block:: sh
solc --optimize --ir-optimized --yul-optimizations 'dhfoD[xarrscLMcCTU]uljmul'
By enclosing part of the sequence in square brackets (`[]`) you tell the optimizer to repeatedly
The order of steps is significant and affects the quality of the output.
Moreover, applying a step may uncover new optimization opportunities for others that were already
applied so repeating steps is often beneficial.
By enclosing part of the sequence in square brackets (``[]``) you tell the optimizer to repeatedly
apply that part until it no longer improves the size of the resulting assembly.
You can use brackets multiple times in a single sequence but they cannot be nested.
@ -1057,37 +1059,37 @@ The following optimization steps are available:
============ ===============================
Abbreviation Full name
============ ===============================
f `BlockFlattener`
l `CircularReferencesPruner`
c `CommonSubexpressionEliminator`
C `ConditionalSimplifier`
U `ConditionalUnsimplifier`
n `ControlFlowSimplifier`
D `DeadCodeEliminator`
v `EquivalentFunctionCombiner`
e `ExpressionInliner`
j `ExpressionJoiner`
s `ExpressionSimplifier`
x `ExpressionSplitter`
I `ForLoopConditionIntoBody`
O `ForLoopConditionOutOfBody`
o `ForLoopInitRewriter`
i `FullInliner`
g `FunctionGrouper`
h `FunctionHoister`
T `LiteralRematerialiser`
L `LoadResolver`
M `LoopInvariantCodeMotion`
r `RedundantAssignEliminator`
m `Rematerialiser`
V `SSAReverser`
a `SSATransform`
t `StructuralSimplifier`
u `UnusedPruner`
d `VarDeclInitializer`
``f`` ``BlockFlattener``
``l`` ``CircularReferencesPruner``
``c`` ``CommonSubexpressionEliminator``
``C`` ``ConditionalSimplifier``
``U`` ``ConditionalUnsimplifier``
``n`` ``ControlFlowSimplifier``
``D`` ``DeadCodeEliminator``
``v`` ``EquivalentFunctionCombiner``
``e`` ``ExpressionInliner``
``j`` ``ExpressionJoiner``
``s`` ``ExpressionSimplifier``
``x`` ``ExpressionSplitter``
``I`` ``ForLoopConditionIntoBody``
``O`` ``ForLoopConditionOutOfBody``
``o`` ``ForLoopInitRewriter``
``i`` ``FullInliner``
``g`` ``FunctionGrouper``
``h`` ``FunctionHoister``
``T`` ``LiteralRematerialiser``
``L`` ``LoadResolver``
``M`` ``LoopInvariantCodeMotion``
``r`` ``RedundantAssignEliminator``
``m`` ``Rematerialiser``
``V`` ``SSAReverser``
``a`` ``SSATransform``
``t`` ``StructuralSimplifier``
``u`` ``UnusedPruner``
``d`` ``VarDeclInitializer``
============ ===============================
Some steps depend on properties ensured by `BlockFlattener`, `FunctionGrouper`, `ForLoopInitRewriter`.
Some steps depend on properties ensured by ``BlockFlattener``, ``FunctionGrouper``, ``ForLoopInitRewriter``.
For this reason the Yul optimizer always applies them before applying any steps supplied by the user.

View File

@ -4,6 +4,7 @@ planned state of the optimiser.
Table of Contents:
- [Selecting optimisations](#selecting-optimisations)
- [Preprocessing](#preprocessing)
- [Pseudo-SSA Transformation](#pseudo-ssa-transformation)
- [Tools](#tools)
@ -33,6 +34,17 @@ the following transformation steps are the main components:
- [Redundant Assign Eliminator](#redundant-assign-eliminator)
- [Full Function Inliner](#full-function-inliner)
## Selecting optimisations
By default the optimiser applies its predefined sequence of optimisation steps to the generated assembly.
You can override this sequence and supply your own using the `--yul-optimizations` option:
``` bash
solc --optimize --ir-optimized --yul-optimizations 'dhfoD[xarrscLMcCTU]uljmul'
```
There's a [table listing available abbreviations in the optimiser docs](/docs/yul.rst#optimization-step-sequence).
## Preprocessing
The preprocessing components perform transformations to get the program

View File

@ -699,7 +699,7 @@ void CommandLineInterface::createFile(string const& _fileName, string const& _da
string pathName = (p / _fileName).string();
if (fs::exists(pathName) && !m_args.count(g_strOverwrite))
{
serr() << "Refusing to overwrite existing file \"" << pathName << "\" (use --overwrite to force)." << endl;
serr() << "Refusing to overwrite existing file \"" << pathName << "\" (use --" << g_strOverwrite << " to force)." << endl;
m_error = true;
return;
}
@ -719,10 +719,10 @@ bool CommandLineInterface::parseArguments(int _argc, char** _argv)
g_hasOutput = false;
// Declare the supported options.
po::options_description desc(R"(solc, the Solidity commandline compiler.
po::options_description desc((R"(solc, the Solidity commandline compiler.
This program comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. See 'solc --license'
are welcome to redistribute it under certain conditions. See 'solc --)" + g_strLicense + R"('
for details.
Usage: solc [options] [input_file...]
@ -732,9 +732,9 @@ at standard output or in files in the output directory, if specified.
Imports are automatically read from the filesystem, but it is also possible to
remap paths using the context:prefix=path syntax.
Example:
solc --bin -o /tmp/solcoutput dapp-bin=/usr/local/lib/dapp-bin contract.sol
solc --)" + g_argBinary + R"( -o /tmp/solcoutput dapp-bin=/usr/local/lib/dapp-bin contract.sol
Allowed options)",
Allowed options)").c_str(),
po::options_description::m_default_line_length,
po::options_description::m_default_line_length - 23
);
@ -780,20 +780,27 @@ Allowed options)",
)
(
g_argImportAst.c_str(),
"Import ASTs to be compiled, assumes input holds the AST in compact JSON format. "
"Supported Inputs is the output of the --standard-json or the one produced by --combined-json ast,compact-format"
("Import ASTs to be compiled, assumes input holds the AST in compact JSON format. "
"Supported Inputs is the output of the --" + g_argStandardJSON + " or the one produced by "
"--" + g_argCombinedJson + " " + g_strAst + "," + g_strCompactJSON).c_str()
)
(
g_argAssemble.c_str(),
"Switch to assembly mode, ignoring all options except --machine, --yul-dialect and --optimize and assumes input is assembly."
("Switch to assembly mode, ignoring all options except "
"--" + g_argMachine + ", --" + g_strYulDialect + ", --" + g_argOptimize + " and --" + g_strYulOptimizations + " "
"and assumes input is assembly.").c_str()
)
(
g_argYul.c_str(),
"Switch to Yul mode, ignoring all options except --machine, --yul-dialect and --optimize and assumes input is Yul."
("Switch to Yul mode, ignoring all options except "
"--" + g_argMachine + ", --" + g_strYulDialect + ", --" + g_argOptimize + " and --" + g_strYulOptimizations + " "
"and assumes input is Yul.").c_str()
)
(
g_argStrictAssembly.c_str(),
"Switch to strict assembly mode, ignoring all options except --machine, --yul-dialect and --optimize and assumes input is strict assembly."
("Switch to strict assembly mode, ignoring all options except "
"--" + g_argMachine + ", --" + g_strYulDialect + ", --" + g_argOptimize + " and --" + g_strYulOptimizations + " "
"and assumes input is strict assembly.").c_str()
)
(
g_strYulDialect.c_str(),
@ -807,8 +814,8 @@ Allowed options)",
)
(
g_argLink.c_str(),
"Switch to linker mode, ignoring all options apart from --libraries "
"and modify binaries in place."
("Switch to linker mode, ignoring all options apart from --" + g_argLibraries + " "
"and modify binaries in place.").c_str()
)
(
g_argMetadataHash.c_str(),
@ -835,7 +842,7 @@ Allowed options)",
"Set for how many contract runs to optimize. "
"Lower values will optimize more for initial deployment cost, higher values will optimize more for high-frequency usage."
)
(g_strOptimizeYul.c_str(), "Legacy option, ignored. Use the general --optimize to enable Yul optimizer.")
(g_strOptimizeYul.c_str(), ("Legacy option, ignored. Use the general --" + g_argOptimize + " to enable Yul optimizer.").c_str())
(g_strNoOptimizeYul.c_str(), "Disable Yul optimizer in Solidity.")
(
g_strYulOptimizations.c_str(),
@ -933,7 +940,7 @@ Allowed options)",
for (string const& item: boost::split(requests, m_args[g_argCombinedJson].as<string>(), boost::is_any_of(",")))
if (!g_combinedJsonArgs.count(item))
{
serr() << "Invalid option to --combined-json: " << item << endl;
serr() << "Invalid option to --" << g_argCombinedJson << ": " << item << endl;
return false;
}
}
@ -1047,7 +1054,7 @@ bool CommandLineInterface::processInput()
std::optional<langutil::EVMVersion> versionOption = langutil::EVMVersion::fromString(versionOptionStr);
if (!versionOption)
{
serr() << "Invalid option for --evm-version: " << versionOptionStr << endl;
serr() << "Invalid option for --" << g_strEVMVersion << ": " << versionOptionStr << endl;
return false;
}
m_evmVersion = *versionOption;
@ -1064,14 +1071,37 @@ bool CommandLineInterface::processInput()
bool optimize = m_args.count(g_argOptimize);
if (m_args.count(g_strOptimizeYul))
{
serr() << "--optimize-yul is invalid in assembly mode. Use --optimize instead." << endl;
serr() << "--" << g_strOptimizeYul << " is invalid in assembly mode. Use --" << g_argOptimize << " instead." << endl;
return false;
}
if (m_args.count(g_strNoOptimizeYul))
{
serr() << "--no-optimize-yul is invalid in assembly mode. Optimization is disabled by default and can be enabled with --optimize." << endl;
serr() << "--" << g_strNoOptimizeYul << " is invalid in assembly mode. Optimization is disabled by default and can be enabled with --" << g_argOptimize << "." << endl;
return false;
}
optional<string> yulOptimiserSteps;
if (m_args.count(g_strYulOptimizations))
{
if (!optimize)
{
serr() << "--" << g_strYulOptimizations << " is invalid if Yul optimizer is disabled" << endl;
return false;
}
try
{
yul::OptimiserSuite::validateSequence(m_args[g_strYulOptimizations].as<string>());
}
catch (yul::OptimizerException const& _exception)
{
serr() << "Invalid optimizer step sequence in --" << g_strYulOptimizations << ": " << _exception.what() << endl;
return false;
}
yulOptimiserSteps = m_args[g_strYulOptimizations].as<string>();
}
if (m_args.count(g_argMachine))
{
string machine = m_args[g_argMachine].as<string>();
@ -1083,7 +1113,7 @@ bool CommandLineInterface::processInput()
targetMachine = Machine::Ewasm;
else
{
serr() << "Invalid option for --machine: " << machine << endl;
serr() << "Invalid option for --" << g_argMachine << ": " << machine << endl;
return false;
}
}
@ -1099,13 +1129,14 @@ bool CommandLineInterface::processInput()
inputLanguage = Input::Ewasm;
if (targetMachine != Machine::Ewasm)
{
serr() << "If you select Ewasm as --yul-dialect, --machine has to be Ewasm as well." << endl;
serr() << "If you select Ewasm as --" << g_strYulDialect << ", ";
serr() << "--" << g_argMachine << " has to be Ewasm as well." << endl;
return false;
}
}
else
{
serr() << "Invalid option for --yul-dialect: " << dialect << endl;
serr() << "Invalid option for --" << g_strYulDialect << ": " << dialect << endl;
return false;
}
}
@ -1122,7 +1153,7 @@ bool CommandLineInterface::processInput()
"Warning: Yul is still experimental. Please use the output with care." <<
endl;
return assemble(inputLanguage, targetMachine, optimize);
return assemble(inputLanguage, targetMachine, optimize, yulOptimiserSteps);
}
if (m_args.count(g_argLink))
{
@ -1142,7 +1173,7 @@ bool CommandLineInterface::processInput()
m_metadataHash = CompilerStack::MetadataHash::None;
else
{
serr() << "Invalid option for --metadata-hash: " << hashStr << endl;
serr() << "Invalid option for --" << g_argMetadataHash << ": " << hashStr << endl;
return false;
}
}
@ -1534,18 +1565,21 @@ string CommandLineInterface::objectWithLinkRefsHex(evmasm::LinkerObject const& _
bool CommandLineInterface::assemble(
yul::AssemblyStack::Language _language,
yul::AssemblyStack::Machine _targetMachine,
bool _optimize
bool _optimize,
optional<string> _yulOptimiserSteps
)
{
solAssert(_optimize || !_yulOptimiserSteps.has_value(), "");
bool successful = true;
map<string, yul::AssemblyStack> assemblyStacks;
for (auto const& src: m_sourceCodes)
{
auto& stack = assemblyStacks[src.first] = yul::AssemblyStack(
m_evmVersion,
_language,
_optimize ? OptimiserSettings::full() : OptimiserSettings::minimal()
);
OptimiserSettings settings = _optimize ? OptimiserSettings::full() : OptimiserSettings::minimal();
if (_yulOptimiserSteps.has_value())
settings.yulOptimiserSteps = _yulOptimiserSteps.value();
auto& stack = assemblyStacks[src.first] = yul::AssemblyStack(m_evmVersion, _language, settings);
try
{
if (!stack.parseAndAnalyze(src.first, src.second))

View File

@ -56,7 +56,12 @@ private:
/// @returns the full object with library placeholder hints in hex.
static std::string objectWithLinkRefsHex(evmasm::LinkerObject const& _obj);
bool assemble(yul::AssemblyStack::Language _language, yul::AssemblyStack::Machine _targetMachine, bool _optimize);
bool assemble(
yul::AssemblyStack::Language _language,
yul::AssemblyStack::Machine _targetMachine,
bool _optimize,
std::optional<std::string> _yulOptimiserSteps = std::nullopt
);
void outputCompilationResults();

View File

@ -245,7 +245,18 @@ printTask "Running general commandline tests..."
stdoutExpectationFile="${tdir}/output.json"
args="--standard-json "$(cat ${tdir}/args 2>/dev/null || true)
else
inputFile="${tdir}input.sol"
if [[ -e "${tdir}input.yul" && -e "${tdir}input.sol" ]]
then
printError "Ambiguous input. Found both input.sol and input.yul."
exit 1
fi
if [ -e "${tdir}input.yul" ]
then
inputFile="${tdir}input.yul"
else
inputFile="${tdir}input.sol"
fi
stdin=""
stdout="$(cat ${tdir}/output 2>/dev/null || true)"
stdoutExpectationFile="${tdir}/output"

View File

@ -0,0 +1,26 @@
{
"language": "Yul",
"sources":
{
"A":
{
"content": "{ let x := mload(0) sstore(add(x, 0), 0) }"
}
},
"settings":
{
"optimizer": {
"enabled": true,
"details": {
"yul": true,
"yulDetails": {
"optimizerSteps": "dhfoDgvulfnTUtnIf"
}
}
},
"outputSelection":
{
"*": { "*": ["*"], "": [ "*" ] }
}
}
}

View File

@ -0,0 +1,30 @@
{"contracts":{"A":{"object":{"evm":{"assembly":" /* \"A\":17:18 */
0x00
/* \"A\":11:19 */
mload
/* \"A\":38:39 */
0x00
/* \"A\":34:35 */
0x00
/* \"A\":31:32 */
dup3
/* \"A\":27:36 */
add
/* \"A\":20:40 */
sstore
pop
","bytecode":{"linkReferences":{},"object":"bytecode removed","opcodes":"opcodes removed","sourceMap":"sourceMap removed"}},"ir":"object \"object\" {
code {
let x := mload(0)
sstore(add(x, 0), 0)
}
}
","irOptimized":"object \"object\" {
code {
{
let x := mload(0)
sstore(add(x, 0), 0)
}
}
}
"}}},"errors":[{"component":"general","formattedMessage":"Yul is still experimental. Please use the output with care.","message":"Yul is still experimental. Please use the output with care.","severity":"warning","type":"Warning"}]}

View File

@ -0,0 +1 @@
--strict-assembly --optimize --yul-optimizations dhfoDgvulfnTUtnIf

View File

@ -0,0 +1 @@
Warning: Yul is still experimental. Please use the output with care.

View File

@ -0,0 +1,27 @@
object "C_6" {
code {
mstore(64, 128)
if callvalue() { revert(0, 0) }
codecopy(0, dataoffset("C_6_deployed"), datasize("C_6_deployed"))
return(0, datasize("C_6_deployed"))
}
object "C_6_deployed" {
code {
{
mstore(64, 128)
if iszero(lt(calldatasize(), 4))
{
let selector := shift_right_224_unsigned(calldataload(0))
pop(selector)
}
pop(iszero(calldatasize()))
revert(0, 0)
}
function shift_right_224_unsigned(value) -> newValue
{
newValue := shr(224, value)
}
}
}
}

View File

@ -0,0 +1,88 @@
======= strict_asm_optimizer_steps/input.yul (EVM) =======
Pretty printed source:
object "C_6" {
code {
{
mstore(64, 128)
if callvalue() { revert(0, 0) }
codecopy(0, dataoffset("C_6_deployed"), datasize("C_6_deployed"))
return(0, datasize("C_6_deployed"))
}
}
object "C_6_deployed" {
code {
{
mstore(64, 128)
pop(iszero(lt(calldatasize(), 4)))
revert(0, 0)
}
}
}
}
Binary representation:
60806040523415600f5760006000fd5b6010601d60003960106000f3fe608060405260043610155060006000fd
Text representation:
/* "strict_asm_optimizer_steps/input.yul":45:48 */
0x80
/* "strict_asm_optimizer_steps/input.yul":41:43 */
0x40
/* "strict_asm_optimizer_steps/input.yul":34:49 */
mstore
/* "strict_asm_optimizer_steps/input.yul":61:72 */
callvalue
/* "strict_asm_optimizer_steps/input.yul":58:60 */
iszero
tag_1
jumpi
/* "strict_asm_optimizer_steps/input.yul":85:86 */
0x00
/* "strict_asm_optimizer_steps/input.yul":82:83 */
0x00
/* "strict_asm_optimizer_steps/input.yul":75:87 */
revert
/* "strict_asm_optimizer_steps/input.yul":58:60 */
tag_1:
/* "strict_asm_optimizer_steps/input.yul":98:163 */
dataSize(sub_0)
dataOffset(sub_0)
/* "strict_asm_optimizer_steps/input.yul":107:108 */
0x00
/* "strict_asm_optimizer_steps/input.yul":98:163 */
codecopy
/* "strict_asm_optimizer_steps/input.yul":172:207 */
dataSize(sub_0)
/* "strict_asm_optimizer_steps/input.yul":179:180 */
0x00
/* "strict_asm_optimizer_steps/input.yul":172:207 */
return
stop
sub_0: assembly {
/* "strict_asm_optimizer_steps/input.yul":298:301 */
0x80
/* "strict_asm_optimizer_steps/input.yul":294:296 */
0x40
/* "strict_asm_optimizer_steps/input.yul":287:302 */
mstore
/* "strict_asm_optimizer_steps/input.yul":348:349 */
0x04
/* "strict_asm_optimizer_steps/input.yul":332:346 */
calldatasize
/* "strict_asm_optimizer_steps/input.yul":329:350 */
lt
/* "strict_asm_optimizer_steps/input.yul":322:351 */
iszero
/* "strict_asm_optimizer_steps/input.yul":319:321 */
pop
/* "strict_asm_optimizer_steps/input.yul":570:571 */
0x00
/* "strict_asm_optimizer_steps/input.yul":567:568 */
0x00
/* "strict_asm_optimizer_steps/input.yul":560:572 */
revert
}

View File

@ -9,7 +9,7 @@ The input is a set of one or more [Yul](/docs/yul.rst) programs and each sequenc
Optimised programs are given numeric scores according to the selected metric.
Optimisation step sequences are presented in an abbreviated form - as strings of letters where each character represents one step.
The abbreviations are defined in [`OptimiserSuite::stepNameToAbbreviationMap()`](/libyul/optimiser/Suite.cpp#L388-L423).
There's a [table listing available abbreviations in the optimiser docs](/docs/yul.rst#optimization-step-sequence).
### How to use it
The application has sensible defaults for most parameters.
@ -66,14 +66,18 @@ tools/yul-phaser *.yul \
`yul-phaser` can process the intermediate representation produced by `solc`:
``` bash
solc/solc <sol file> \
--ir \
--no-optimize-yul \
--output-dir <output directory>
solc/solc <sol file> --ir --output-dir <output directory>
```
After running this command you'll find one or more .yul files in the output directory.
These files contain whole Yul objects rather than just raw Yul programs but `yul-phaser` is prepared to handle them.
These files contain whole Yul objects rather than just raw Yul programs but `yul-phaser` is prepared to handle them too.
#### Using optimisation step sequences with the compiler
You can tell Yul optimiser to use a specific sequence for your code by passing `--yul-optimizations` option to `solc`:
``` bash
solc/solc <sol file> --optimize --ir-optimized --yul-optimizations <sequence>
```
### How to choose good parameters
Choosing good parameters for a genetic algorithm is not a trivial task but phaser's defaults are generally enough to find a sequence that gives results comparable or better than one hand-crafted by an experienced developer for a given set of programs.