diff --git a/Changelog.md b/Changelog.md index f1a0b741b..6ec21a974 100644 --- a/Changelog.md +++ b/Changelog.md @@ -5,6 +5,7 @@ Language Features: Compiler Features: + * Commandline Interface: Don't ignore `--yul-optimizations` in assembly mode. diff --git a/docs/yul.rst b/docs/yul.rst index c200558b0..2e8bc4a10 100644 --- a/docs/yul.rst +++ b/docs/yul.rst @@ -1041,14 +1041,16 @@ Optimization step sequence -------------------------- By default the Yul optimizer applies its predefined sequence of optimization steps to the generated assembly. -You can override this sequence and supply your own using the `--yul-optimizations` option when compiling -in Solidity mode: +You can override this sequence and supply your own using the ``--yul-optimizations`` option: .. code-block:: sh solc --optimize --ir-optimized --yul-optimizations 'dhfoD[xarrscLMcCTU]uljmul' -By enclosing part of the sequence in square brackets (`[]`) you tell the optimizer to repeatedly +The order of steps is significant and affects the quality of the output. +Moreover, applying a step may uncover new optimization opportunities for others that were already +applied so repeating steps is often beneficial. +By enclosing part of the sequence in square brackets (``[]``) you tell the optimizer to repeatedly apply that part until it no longer improves the size of the resulting assembly. You can use brackets multiple times in a single sequence but they cannot be nested. @@ -1057,37 +1059,37 @@ The following optimization steps are available: ============ =============================== Abbreviation Full name ============ =============================== -f `BlockFlattener` -l `CircularReferencesPruner` -c `CommonSubexpressionEliminator` -C `ConditionalSimplifier` -U `ConditionalUnsimplifier` -n `ControlFlowSimplifier` -D `DeadCodeEliminator` -v `EquivalentFunctionCombiner` -e `ExpressionInliner` -j `ExpressionJoiner` -s `ExpressionSimplifier` -x `ExpressionSplitter` -I `ForLoopConditionIntoBody` -O `ForLoopConditionOutOfBody` -o `ForLoopInitRewriter` -i `FullInliner` -g `FunctionGrouper` -h `FunctionHoister` -T `LiteralRematerialiser` -L `LoadResolver` -M `LoopInvariantCodeMotion` -r `RedundantAssignEliminator` -m `Rematerialiser` -V `SSAReverser` -a `SSATransform` -t `StructuralSimplifier` -u `UnusedPruner` -d `VarDeclInitializer` +``f`` ``BlockFlattener`` +``l`` ``CircularReferencesPruner`` +``c`` ``CommonSubexpressionEliminator`` +``C`` ``ConditionalSimplifier`` +``U`` ``ConditionalUnsimplifier`` +``n`` ``ControlFlowSimplifier`` +``D`` ``DeadCodeEliminator`` +``v`` ``EquivalentFunctionCombiner`` +``e`` ``ExpressionInliner`` +``j`` ``ExpressionJoiner`` +``s`` ``ExpressionSimplifier`` +``x`` ``ExpressionSplitter`` +``I`` ``ForLoopConditionIntoBody`` +``O`` ``ForLoopConditionOutOfBody`` +``o`` ``ForLoopInitRewriter`` +``i`` ``FullInliner`` +``g`` ``FunctionGrouper`` +``h`` ``FunctionHoister`` +``T`` ``LiteralRematerialiser`` +``L`` ``LoadResolver`` +``M`` ``LoopInvariantCodeMotion`` +``r`` ``RedundantAssignEliminator`` +``m`` ``Rematerialiser`` +``V`` ``SSAReverser`` +``a`` ``SSATransform`` +``t`` ``StructuralSimplifier`` +``u`` ``UnusedPruner`` +``d`` ``VarDeclInitializer`` ============ =============================== -Some steps depend on properties ensured by `BlockFlattener`, `FunctionGrouper`, `ForLoopInitRewriter`. +Some steps depend on properties ensured by ``BlockFlattener``, ``FunctionGrouper``, ``ForLoopInitRewriter``. For this reason the Yul optimizer always applies them before applying any steps supplied by the user. diff --git a/libyul/optimiser/README.md b/libyul/optimiser/README.md index 58cc006b4..4bd3d6a36 100644 --- a/libyul/optimiser/README.md +++ b/libyul/optimiser/README.md @@ -4,6 +4,7 @@ planned state of the optimiser. Table of Contents: +- [Selecting optimisations](#selecting-optimisations) - [Preprocessing](#preprocessing) - [Pseudo-SSA Transformation](#pseudo-ssa-transformation) - [Tools](#tools) @@ -33,6 +34,17 @@ the following transformation steps are the main components: - [Redundant Assign Eliminator](#redundant-assign-eliminator) - [Full Function Inliner](#full-function-inliner) +## Selecting optimisations + +By default the optimiser applies its predefined sequence of optimisation steps to the generated assembly. +You can override this sequence and supply your own using the `--yul-optimizations` option: + +``` bash +solc --optimize --ir-optimized --yul-optimizations 'dhfoD[xarrscLMcCTU]uljmul' +``` + +There's a [table listing available abbreviations in the optimiser docs](/docs/yul.rst#optimization-step-sequence). + ## Preprocessing The preprocessing components perform transformations to get the program diff --git a/solc/CommandLineInterface.cpp b/solc/CommandLineInterface.cpp index 537b2096b..1342ef1f6 100644 --- a/solc/CommandLineInterface.cpp +++ b/solc/CommandLineInterface.cpp @@ -699,7 +699,7 @@ void CommandLineInterface::createFile(string const& _fileName, string const& _da string pathName = (p / _fileName).string(); if (fs::exists(pathName) && !m_args.count(g_strOverwrite)) { - serr() << "Refusing to overwrite existing file \"" << pathName << "\" (use --overwrite to force)." << endl; + serr() << "Refusing to overwrite existing file \"" << pathName << "\" (use --" << g_strOverwrite << " to force)." << endl; m_error = true; return; } @@ -719,10 +719,10 @@ bool CommandLineInterface::parseArguments(int _argc, char** _argv) g_hasOutput = false; // Declare the supported options. - po::options_description desc(R"(solc, the Solidity commandline compiler. + po::options_description desc((R"(solc, the Solidity commandline compiler. This program comes with ABSOLUTELY NO WARRANTY. This is free software, and you -are welcome to redistribute it under certain conditions. See 'solc --license' +are welcome to redistribute it under certain conditions. See 'solc --)" + g_strLicense + R"(' for details. Usage: solc [options] [input_file...] @@ -732,9 +732,9 @@ at standard output or in files in the output directory, if specified. Imports are automatically read from the filesystem, but it is also possible to remap paths using the context:prefix=path syntax. Example: -solc --bin -o /tmp/solcoutput dapp-bin=/usr/local/lib/dapp-bin contract.sol +solc --)" + g_argBinary + R"( -o /tmp/solcoutput dapp-bin=/usr/local/lib/dapp-bin contract.sol -Allowed options)", +Allowed options)").c_str(), po::options_description::m_default_line_length, po::options_description::m_default_line_length - 23 ); @@ -780,20 +780,27 @@ Allowed options)", ) ( g_argImportAst.c_str(), - "Import ASTs to be compiled, assumes input holds the AST in compact JSON format. " - "Supported Inputs is the output of the --standard-json or the one produced by --combined-json ast,compact-format" + ("Import ASTs to be compiled, assumes input holds the AST in compact JSON format. " + "Supported Inputs is the output of the --" + g_argStandardJSON + " or the one produced by " + "--" + g_argCombinedJson + " " + g_strAst + "," + g_strCompactJSON).c_str() ) ( g_argAssemble.c_str(), - "Switch to assembly mode, ignoring all options except --machine, --yul-dialect and --optimize and assumes input is assembly." + ("Switch to assembly mode, ignoring all options except " + "--" + g_argMachine + ", --" + g_strYulDialect + ", --" + g_argOptimize + " and --" + g_strYulOptimizations + " " + "and assumes input is assembly.").c_str() ) ( g_argYul.c_str(), - "Switch to Yul mode, ignoring all options except --machine, --yul-dialect and --optimize and assumes input is Yul." + ("Switch to Yul mode, ignoring all options except " + "--" + g_argMachine + ", --" + g_strYulDialect + ", --" + g_argOptimize + " and --" + g_strYulOptimizations + " " + "and assumes input is Yul.").c_str() ) ( g_argStrictAssembly.c_str(), - "Switch to strict assembly mode, ignoring all options except --machine, --yul-dialect and --optimize and assumes input is strict assembly." + ("Switch to strict assembly mode, ignoring all options except " + "--" + g_argMachine + ", --" + g_strYulDialect + ", --" + g_argOptimize + " and --" + g_strYulOptimizations + " " + "and assumes input is strict assembly.").c_str() ) ( g_strYulDialect.c_str(), @@ -807,8 +814,8 @@ Allowed options)", ) ( g_argLink.c_str(), - "Switch to linker mode, ignoring all options apart from --libraries " - "and modify binaries in place." + ("Switch to linker mode, ignoring all options apart from --" + g_argLibraries + " " + "and modify binaries in place.").c_str() ) ( g_argMetadataHash.c_str(), @@ -835,7 +842,7 @@ Allowed options)", "Set for how many contract runs to optimize. " "Lower values will optimize more for initial deployment cost, higher values will optimize more for high-frequency usage." ) - (g_strOptimizeYul.c_str(), "Legacy option, ignored. Use the general --optimize to enable Yul optimizer.") + (g_strOptimizeYul.c_str(), ("Legacy option, ignored. Use the general --" + g_argOptimize + " to enable Yul optimizer.").c_str()) (g_strNoOptimizeYul.c_str(), "Disable Yul optimizer in Solidity.") ( g_strYulOptimizations.c_str(), @@ -933,7 +940,7 @@ Allowed options)", for (string const& item: boost::split(requests, m_args[g_argCombinedJson].as(), boost::is_any_of(","))) if (!g_combinedJsonArgs.count(item)) { - serr() << "Invalid option to --combined-json: " << item << endl; + serr() << "Invalid option to --" << g_argCombinedJson << ": " << item << endl; return false; } } @@ -1047,7 +1054,7 @@ bool CommandLineInterface::processInput() std::optional versionOption = langutil::EVMVersion::fromString(versionOptionStr); if (!versionOption) { - serr() << "Invalid option for --evm-version: " << versionOptionStr << endl; + serr() << "Invalid option for --" << g_strEVMVersion << ": " << versionOptionStr << endl; return false; } m_evmVersion = *versionOption; @@ -1064,14 +1071,37 @@ bool CommandLineInterface::processInput() bool optimize = m_args.count(g_argOptimize); if (m_args.count(g_strOptimizeYul)) { - serr() << "--optimize-yul is invalid in assembly mode. Use --optimize instead." << endl; + serr() << "--" << g_strOptimizeYul << " is invalid in assembly mode. Use --" << g_argOptimize << " instead." << endl; return false; } if (m_args.count(g_strNoOptimizeYul)) { - serr() << "--no-optimize-yul is invalid in assembly mode. Optimization is disabled by default and can be enabled with --optimize." << endl; + serr() << "--" << g_strNoOptimizeYul << " is invalid in assembly mode. Optimization is disabled by default and can be enabled with --" << g_argOptimize << "." << endl; return false; } + + optional yulOptimiserSteps; + if (m_args.count(g_strYulOptimizations)) + { + if (!optimize) + { + serr() << "--" << g_strYulOptimizations << " is invalid if Yul optimizer is disabled" << endl; + return false; + } + + try + { + yul::OptimiserSuite::validateSequence(m_args[g_strYulOptimizations].as()); + } + catch (yul::OptimizerException const& _exception) + { + serr() << "Invalid optimizer step sequence in --" << g_strYulOptimizations << ": " << _exception.what() << endl; + return false; + } + + yulOptimiserSteps = m_args[g_strYulOptimizations].as(); + } + if (m_args.count(g_argMachine)) { string machine = m_args[g_argMachine].as(); @@ -1083,7 +1113,7 @@ bool CommandLineInterface::processInput() targetMachine = Machine::Ewasm; else { - serr() << "Invalid option for --machine: " << machine << endl; + serr() << "Invalid option for --" << g_argMachine << ": " << machine << endl; return false; } } @@ -1099,13 +1129,14 @@ bool CommandLineInterface::processInput() inputLanguage = Input::Ewasm; if (targetMachine != Machine::Ewasm) { - serr() << "If you select Ewasm as --yul-dialect, --machine has to be Ewasm as well." << endl; + serr() << "If you select Ewasm as --" << g_strYulDialect << ", "; + serr() << "--" << g_argMachine << " has to be Ewasm as well." << endl; return false; } } else { - serr() << "Invalid option for --yul-dialect: " << dialect << endl; + serr() << "Invalid option for --" << g_strYulDialect << ": " << dialect << endl; return false; } } @@ -1122,7 +1153,7 @@ bool CommandLineInterface::processInput() "Warning: Yul is still experimental. Please use the output with care." << endl; - return assemble(inputLanguage, targetMachine, optimize); + return assemble(inputLanguage, targetMachine, optimize, yulOptimiserSteps); } if (m_args.count(g_argLink)) { @@ -1142,7 +1173,7 @@ bool CommandLineInterface::processInput() m_metadataHash = CompilerStack::MetadataHash::None; else { - serr() << "Invalid option for --metadata-hash: " << hashStr << endl; + serr() << "Invalid option for --" << g_argMetadataHash << ": " << hashStr << endl; return false; } } @@ -1534,18 +1565,21 @@ string CommandLineInterface::objectWithLinkRefsHex(evmasm::LinkerObject const& _ bool CommandLineInterface::assemble( yul::AssemblyStack::Language _language, yul::AssemblyStack::Machine _targetMachine, - bool _optimize + bool _optimize, + optional _yulOptimiserSteps ) { + solAssert(_optimize || !_yulOptimiserSteps.has_value(), ""); + bool successful = true; map assemblyStacks; for (auto const& src: m_sourceCodes) { - auto& stack = assemblyStacks[src.first] = yul::AssemblyStack( - m_evmVersion, - _language, - _optimize ? OptimiserSettings::full() : OptimiserSettings::minimal() - ); + OptimiserSettings settings = _optimize ? OptimiserSettings::full() : OptimiserSettings::minimal(); + if (_yulOptimiserSteps.has_value()) + settings.yulOptimiserSteps = _yulOptimiserSteps.value(); + + auto& stack = assemblyStacks[src.first] = yul::AssemblyStack(m_evmVersion, _language, settings); try { if (!stack.parseAndAnalyze(src.first, src.second)) diff --git a/solc/CommandLineInterface.h b/solc/CommandLineInterface.h index f6972cf04..446ea8224 100644 --- a/solc/CommandLineInterface.h +++ b/solc/CommandLineInterface.h @@ -56,7 +56,12 @@ private: /// @returns the full object with library placeholder hints in hex. static std::string objectWithLinkRefsHex(evmasm::LinkerObject const& _obj); - bool assemble(yul::AssemblyStack::Language _language, yul::AssemblyStack::Machine _targetMachine, bool _optimize); + bool assemble( + yul::AssemblyStack::Language _language, + yul::AssemblyStack::Machine _targetMachine, + bool _optimize, + std::optional _yulOptimiserSteps = std::nullopt + ); void outputCompilationResults(); diff --git a/test/cmdlineTests.sh b/test/cmdlineTests.sh index 077aa5c19..07e249492 100755 --- a/test/cmdlineTests.sh +++ b/test/cmdlineTests.sh @@ -245,7 +245,18 @@ printTask "Running general commandline tests..." stdoutExpectationFile="${tdir}/output.json" args="--standard-json "$(cat ${tdir}/args 2>/dev/null || true) else - inputFile="${tdir}input.sol" + if [[ -e "${tdir}input.yul" && -e "${tdir}input.sol" ]] + then + printError "Ambiguous input. Found both input.sol and input.yul." + exit 1 + fi + + if [ -e "${tdir}input.yul" ] + then + inputFile="${tdir}input.yul" + else + inputFile="${tdir}input.sol" + fi stdin="" stdout="$(cat ${tdir}/output 2>/dev/null || true)" stdoutExpectationFile="${tdir}/output" diff --git a/test/cmdlineTests/standard_yul_optimiserSteps/input.json b/test/cmdlineTests/standard_yul_optimiserSteps/input.json new file mode 100644 index 000000000..1015496f9 --- /dev/null +++ b/test/cmdlineTests/standard_yul_optimiserSteps/input.json @@ -0,0 +1,26 @@ +{ + "language": "Yul", + "sources": + { + "A": + { + "content": "{ let x := mload(0) sstore(add(x, 0), 0) }" + } + }, + "settings": + { + "optimizer": { + "enabled": true, + "details": { + "yul": true, + "yulDetails": { + "optimizerSteps": "dhfoDgvulfnTUtnIf" + } + } + }, + "outputSelection": + { + "*": { "*": ["*"], "": [ "*" ] } + } + } +} diff --git a/test/cmdlineTests/standard_yul_optimiserSteps/output.json b/test/cmdlineTests/standard_yul_optimiserSteps/output.json new file mode 100644 index 000000000..383ad8557 --- /dev/null +++ b/test/cmdlineTests/standard_yul_optimiserSteps/output.json @@ -0,0 +1,30 @@ +{"contracts":{"A":{"object":{"evm":{"assembly":" /* \"A\":17:18 */ + 0x00 + /* \"A\":11:19 */ + mload + /* \"A\":38:39 */ + 0x00 + /* \"A\":34:35 */ + 0x00 + /* \"A\":31:32 */ + dup3 + /* \"A\":27:36 */ + add + /* \"A\":20:40 */ + sstore + pop +","bytecode":{"linkReferences":{},"object":"bytecode removed","opcodes":"opcodes removed","sourceMap":"sourceMap removed"}},"ir":"object \"object\" { + code { + let x := mload(0) + sstore(add(x, 0), 0) + } +} +","irOptimized":"object \"object\" { + code { + { + let x := mload(0) + sstore(add(x, 0), 0) + } + } +} +"}}},"errors":[{"component":"general","formattedMessage":"Yul is still experimental. Please use the output with care.","message":"Yul is still experimental. Please use the output with care.","severity":"warning","type":"Warning"}]} diff --git a/test/cmdlineTests/strict_asm_optimizer_steps/args b/test/cmdlineTests/strict_asm_optimizer_steps/args new file mode 100644 index 000000000..ade406ee4 --- /dev/null +++ b/test/cmdlineTests/strict_asm_optimizer_steps/args @@ -0,0 +1 @@ +--strict-assembly --optimize --yul-optimizations dhfoDgvulfnTUtnIf diff --git a/test/cmdlineTests/strict_asm_optimizer_steps/err b/test/cmdlineTests/strict_asm_optimizer_steps/err new file mode 100644 index 000000000..014a1178f --- /dev/null +++ b/test/cmdlineTests/strict_asm_optimizer_steps/err @@ -0,0 +1 @@ +Warning: Yul is still experimental. Please use the output with care. diff --git a/test/cmdlineTests/strict_asm_optimizer_steps/input.yul b/test/cmdlineTests/strict_asm_optimizer_steps/input.yul new file mode 100644 index 000000000..b1c756df3 --- /dev/null +++ b/test/cmdlineTests/strict_asm_optimizer_steps/input.yul @@ -0,0 +1,27 @@ +object "C_6" { + code { + mstore(64, 128) + if callvalue() { revert(0, 0) } + codecopy(0, dataoffset("C_6_deployed"), datasize("C_6_deployed")) + return(0, datasize("C_6_deployed")) + } + object "C_6_deployed" { + code { + { + mstore(64, 128) + if iszero(lt(calldatasize(), 4)) + { + let selector := shift_right_224_unsigned(calldataload(0)) + pop(selector) + } + pop(iszero(calldatasize())) + revert(0, 0) + } + + function shift_right_224_unsigned(value) -> newValue + { + newValue := shr(224, value) + } + } + } +} diff --git a/test/cmdlineTests/strict_asm_optimizer_steps/output b/test/cmdlineTests/strict_asm_optimizer_steps/output new file mode 100644 index 000000000..369be348c --- /dev/null +++ b/test/cmdlineTests/strict_asm_optimizer_steps/output @@ -0,0 +1,88 @@ + +======= strict_asm_optimizer_steps/input.yul (EVM) ======= + +Pretty printed source: +object "C_6" { + code { + { + mstore(64, 128) + if callvalue() { revert(0, 0) } + codecopy(0, dataoffset("C_6_deployed"), datasize("C_6_deployed")) + return(0, datasize("C_6_deployed")) + } + } + object "C_6_deployed" { + code { + { + mstore(64, 128) + pop(iszero(lt(calldatasize(), 4))) + revert(0, 0) + } + } + } +} + + +Binary representation: +60806040523415600f5760006000fd5b6010601d60003960106000f3fe608060405260043610155060006000fd + +Text representation: + /* "strict_asm_optimizer_steps/input.yul":45:48 */ + 0x80 + /* "strict_asm_optimizer_steps/input.yul":41:43 */ + 0x40 + /* "strict_asm_optimizer_steps/input.yul":34:49 */ + mstore + /* "strict_asm_optimizer_steps/input.yul":61:72 */ + callvalue + /* "strict_asm_optimizer_steps/input.yul":58:60 */ + iszero + tag_1 + jumpi + /* "strict_asm_optimizer_steps/input.yul":85:86 */ + 0x00 + /* "strict_asm_optimizer_steps/input.yul":82:83 */ + 0x00 + /* "strict_asm_optimizer_steps/input.yul":75:87 */ + revert + /* "strict_asm_optimizer_steps/input.yul":58:60 */ +tag_1: + /* "strict_asm_optimizer_steps/input.yul":98:163 */ + dataSize(sub_0) + dataOffset(sub_0) + /* "strict_asm_optimizer_steps/input.yul":107:108 */ + 0x00 + /* "strict_asm_optimizer_steps/input.yul":98:163 */ + codecopy + /* "strict_asm_optimizer_steps/input.yul":172:207 */ + dataSize(sub_0) + /* "strict_asm_optimizer_steps/input.yul":179:180 */ + 0x00 + /* "strict_asm_optimizer_steps/input.yul":172:207 */ + return +stop + +sub_0: assembly { + /* "strict_asm_optimizer_steps/input.yul":298:301 */ + 0x80 + /* "strict_asm_optimizer_steps/input.yul":294:296 */ + 0x40 + /* "strict_asm_optimizer_steps/input.yul":287:302 */ + mstore + /* "strict_asm_optimizer_steps/input.yul":348:349 */ + 0x04 + /* "strict_asm_optimizer_steps/input.yul":332:346 */ + calldatasize + /* "strict_asm_optimizer_steps/input.yul":329:350 */ + lt + /* "strict_asm_optimizer_steps/input.yul":322:351 */ + iszero + /* "strict_asm_optimizer_steps/input.yul":319:321 */ + pop + /* "strict_asm_optimizer_steps/input.yul":570:571 */ + 0x00 + /* "strict_asm_optimizer_steps/input.yul":567:568 */ + 0x00 + /* "strict_asm_optimizer_steps/input.yul":560:572 */ + revert +} diff --git a/tools/yulPhaser/README.md b/tools/yulPhaser/README.md index abc2a0e47..3bc72b537 100644 --- a/tools/yulPhaser/README.md +++ b/tools/yulPhaser/README.md @@ -9,7 +9,7 @@ The input is a set of one or more [Yul](/docs/yul.rst) programs and each sequenc Optimised programs are given numeric scores according to the selected metric. Optimisation step sequences are presented in an abbreviated form - as strings of letters where each character represents one step. -The abbreviations are defined in [`OptimiserSuite::stepNameToAbbreviationMap()`](/libyul/optimiser/Suite.cpp#L388-L423). +There's a [table listing available abbreviations in the optimiser docs](/docs/yul.rst#optimization-step-sequence). ### How to use it The application has sensible defaults for most parameters. @@ -66,14 +66,18 @@ tools/yul-phaser *.yul \ `yul-phaser` can process the intermediate representation produced by `solc`: ``` bash -solc/solc \ - --ir \ - --no-optimize-yul \ - --output-dir +solc/solc --ir --output-dir ``` After running this command you'll find one or more .yul files in the output directory. -These files contain whole Yul objects rather than just raw Yul programs but `yul-phaser` is prepared to handle them. +These files contain whole Yul objects rather than just raw Yul programs but `yul-phaser` is prepared to handle them too. + +#### Using optimisation step sequences with the compiler +You can tell Yul optimiser to use a specific sequence for your code by passing `--yul-optimizations` option to `solc`: + +``` bash +solc/solc --optimize --ir-optimized --yul-optimizations +``` ### How to choose good parameters Choosing good parameters for a genetic algorithm is not a trivial task but phaser's defaults are generally enough to find a sequence that gives results comparable or better than one hand-crafted by an experienced developer for a given set of programs.