diff --git a/docs/contributing.rst b/docs/contributing.rst index dbfcebe89..571a298f2 100644 --- a/docs/contributing.rst +++ b/docs/contributing.rst @@ -81,26 +81,55 @@ Thank you for your help! Running the compiler tests ========================== -The ``./scripts/tests.sh`` script executes most Solidity tests automatically, -but for quicker feedback, you might want to run specific tests. +Prerequisites +------------- + +Some tests require the `evmone `_ +library, others require `libz3 `_. The test script +tries to discover the location of the ``evmone`` library, which can be located +in the current directory, installed on the system level, or the ``deps`` folder +in the project top level. The required file is called ``libevmone.so`` on Linux +systems, ``evmone.dll`` on Windows systems and ``libevmone.dylib`` on macOS. + +Running the tests +----------------- Solidity includes different types of tests, most of them bundled into the `Boost C++ Test Framework `_ application ``soltest``. Running ``build/test/soltest`` or its wrapper ``scripts/soltest.sh`` is sufficient for most changes. -Some tests require the ``evmone`` library, others require ``libz3``. +The ``./scripts/tests.sh`` script executes most Solidity tests automatically, +including those bundled into the `Boost C++ Test Framework `_ application ``soltest`` (or its wrapper ``scripts/soltest.sh``), +as well as command line tests and compilation tests. -The test system will automatically try to discover the location of the ``evmone`` library +The test system automatically tries try to discover the location of the ``evmone`` library starting from the current directory. The required file is called ``libevmone.so`` on Linux systems, -``evmone.dll`` on Windows systems and ``libevmone.dylib`` on MacOS. If it is not found, the relevant tests -are skipped. To run all tests, download the library from -`Github `_ -and either place it in the project root path or inside the ``deps`` folder. +``evmone.dll`` on Windows systems and ``libevmone.dylib`` on macOS. If it is not found, tests that +use it are skipped. These tests are ``libsolididty/semanticTests``, ``libsolidity/GasCosts``, +``libsolidity/SolidityEndToEndTest``, part of the soltest suite. To run all tests, download the library from +`GitHub `_ +and place it in the project root path or inside the ``deps`` folder. -If you do not have libz3 installed on your system, you should disable the SMT tests: -``./scripts/soltest.sh --no-smt``. +If the ``libz3`` library is not installed on your system, you should disable the +SMT tests by exporting ``SMT_FLAGS=--no-smt`` before running ``./scripts/tests.sh`` or +running ``./scripts/soltest.sh --no-smt``. +These tests are ``libsolidity/smtCheckerTests`` and ``libsolidity/smtCheckerTestsJSON``. + +.. note :: + + To get a list of all unit tests run by Soltest, run ``./build/test/soltest --list_content=HRF``. + +For quicker results you can run a subset of, or specific tests. + +To run a subset of tests, you can use filters: +``./scripts/soltest.sh -t TestSuite/TestName``, +where ``TestName`` can be a wildcard ``*``. + +Or, for example, to run all the tests for the yul disambiguator: +``./scripts/soltest.sh -t "yulOptimizerTests/disambiguator/*" --no-smt``. ``./build/test/soltest --help`` has extensive help on all of the options available. + See especially: - `show_progress (-p) `_ to show test completion, @@ -109,20 +138,10 @@ See especially: .. note :: - Those working in a Windows environment wanting to run the above basic sets without libz3 in Git Bash, you would have to do: ``./build/test/Release/soltest.exe -- --no-smt``. + Those working in a Windows environment wanting to run the above basic sets + without libz3. Using Git Bash, you use: ``./build/test/Release/soltest.exe -- --no-smt``. If you are running this in plain Command Prompt, use ``.\build\test\Release\soltest.exe -- --no-smt``. -To run a subset of tests, you can use filters: -``./scripts/soltest.sh -t TestSuite/TestName``, -where ``TestName`` can be a wildcard ``*``. - -For example, here is an example test you might run; -``./scripts/soltest.sh -t "yulOptimizerTests/disambiguator/*" --no-smt``. -This will test all the tests for the disambiguator. - -To get a list of all tests, use -``./build/test/soltest --list_content=HRF``. - If you want to debug using GDB, make sure you build differently than the "usual". For example, you could run the following command in your ``build`` folder: :: @@ -130,14 +149,11 @@ For example, you could run the following command in your ``build`` folder: cmake -DCMAKE_BUILD_TYPE=Debug .. make -This will create symbols such that when you debug a test using the ``--debug`` flag, you will have access to functions and variables in which you can break or print with. - - -The script ``./scripts/tests.sh`` also runs commandline tests and compilation tests -in addition to those found in ``soltest``. - -The CI runs additional tests (including ``solc-js`` and testing third party Solidity frameworks) that require compiling the Emscripten target. +This creates symbols so that when you debug a test using the ``--debug`` flag, +you have access to functions and variables in which you can break or print with. +The CI runs additional tests (including ``solc-js`` and testing third party Solidity +frameworks) that require compiling the Emscripten target. Writing and running syntax tests --------------------------------