watcher-ts container doesn't build on arm #561

Closed
opened 2023-10-06 17:01:37 +00:00 by dboreham · 4 comments
Owner

Build fails trying to run gyp to compile some native dependency. Usually this happens because there was a pre-built x86 binary that is used and the gyp build is therefore never run, and there's some bug in the gyp build process. The bug is never exercised because on x86 the pre-built binary is fetched.

Tail of the log:

Building watcher-ts
yarn install v1.22.19                                                                                                                                           [1/4] Resolving packages...                                                                                                                                     [2/4] Fetching packages...
warning Pattern ["node-fetch@https://registry.npmjs.org/@achingbrain/node-fetch/-/node-fetch-2.6.7.tgz"] is trying to unpack in the same destination "/usr/local/share/.cache/yarn/v6/npm-node-fetch-2.6.7-1b5d62978f2ed07b99444f64f0df39f960a6d34d/node_modules/node-fetch" as pattern ["node-fetch@2.6.7"]. This could result in non-deterministic behavior, skipping.
[3/4] Linking dependencies...
warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cache > eslint-config-standard@16.0.3" has incorrect peer dependency "eslint@^7.12.1".
warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cache > eslint-plugin-promise@5.2.0" has incorrect peer dependency "eslint@^7.0.0".
warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cli > @apollo/client@3.7.10" has unmet peer dependency "graphql@^14.0.0 || ^15.0.0 || ^16.0.0".
warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cli > @graphql-tools/utils@9.2.1" has unmet peer dependency "graphql@^14.0.0 || ^15.0.0 || ^16.0.0 || ^17.0.0".                                                                                                                                   warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cli > apollo-server-express@3.12.0" has unmet peer dependency "graphql@^15.3.0 || ^16.0.0".
warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cli > graphql-subscriptions@2.0.0" has unmet peer dependency "graphql@^15.7.2 || ^16.0.0".
warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cli > eslint-config-standard@5.3.5" has unmet peer dependency "eslint@>=2.0.0".
warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/graph-node > @nomiclabs/hardhat-waffle@2.0.5" has unmet peer dependency "ethereum-waffle@*".
warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cache > @typescript-eslint/eslint-plugin > tsutils@3.21.0" has unmet peer dependency "typescript@>=2.8.0 || >= 3.2.0-dev || >= 3.3.0-dev || >= 3.4.0-dev || >= 3.5.0-dev || >= 3.6.0-dev || >= 3.6.0-beta || >= 3.7.0-dev || >= 3.7.0-beta".
warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cli > @apollo/client > @graphql-typed-document-node/core@3.2.0" has unmet peer dependency "graphql@^0.8.0 || ^0.9.0 || ^0.10.0 || ^0.11.0 || ^0.12.0 || ^0.13.0 || ^14.0.0 || ^15.0.0 || ^16.0.0 || ^17.0.0".
warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cli > @apollo/client > graphql-tag@2.12.6" has unmet peer dependency "graphql@^0.9.0 || ^0.10.0 || ^0.11.0 || ^0.12.0 || ^0.13.0 || ^14.0.0 || ^15.0.0 || ^16.0.0".
warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cli > apollo-server-express > apollo-server-types@3.8.0" has unmet peer dependency "graphql@^15.3.0 || ^16.0.0".
warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/codegen > @poanet/solidity-flattener > glob-promise@3.4.0" has unmet peer dependency "glob@*".                                                                                                                                                    warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cli > @cerc-io/nitro-node > @cerc-io/nitro-protocol > @typechain/ethers-v5@9.0.0" has unmet peer dependency "typechain@^7.0.0".
warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cli > @cerc-io/nitro-node > @cerc-io/nitro-protocol > @typechain/ethers-v5@9.0.0" has unmet peer dependency "@ethersproject/bytes@^5.0.0".
warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cli > @cerc-io/nitro-node > @cerc-io/nitro-protocol > @typechain/ethers-v5@9.0.0" has unmet peer dependency "@ethersproject/abi@^5.0.0".                                                                                                          [4/4] Building fresh packages...
error /app/node_modules/bufferutil: Command failed.                                                                                                             Exit code: 1
Command: node-gyp-build                                                                                                                                         Arguments:
Directory: /app/node_modules/bufferutil
Output:
gyp info it worked if it ends with ok
gyp info using node-gyp@7.1.2                                                                                                                                   gyp info using node@18.17.1 | linux | arm64                                                                                                                     gyp info find Python using Python version 3.11.6 found at "/usr/bin/python3"
gyp http GET https://nodejs.org/download/release/v18.17.1/node-v18.17.1-headers.tar.gz
gyp http 200 https://nodejs.org/download/release/v18.17.1/node-v18.17.1-headers.tar.gz
gyp http GET https://nodejs.org/download/release/v18.17.1/SHASUMS256.txt
gyp http 200 https://nodejs.org/download/release/v18.17.1/SHASUMS256.txt
(node:257) [DEP0150] DeprecationWarning: Setting process.config is deprecated. In the future the property will be read-only.
(Use `node --trace-deprecation ...` to show where the warning was created)
gyp info spawn /usr/bin/python3
gyp info spawn args [
gyp info spawn args   '/app/node_modules/@npmcli/run-script/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/app/node_modules/bufferutil/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/app/node_modules/@npmcli/run-script/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/root/.cache/node-gyp/18.17.1/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/root/.cache/node-gyp/18.17.1',
gyp info spawn args   '-Dnode_gyp_dir=/app/node_modules/@npmcli/run-script/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/root/.cache/node-gyp/18.17.1/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/app/node_modules/bufferutil',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
Traceback (most recent call last):
  File "/app/node_modules/@npmcli/run-script/node_modules/node-gyp/gyp/gyp_main.py", line 51, in <module>
    sys.exit(gyp.script_main())
             ^^^^^^^^^^^^^^^^^
  File "/app/node_modules/@npmcli/run-script/node_modules/node-gyp/gyp/pylib/gyp/__init__.py", line 670, in script_main
    return main(sys.argv[1:])
           ^^^^^^^^^^^^^^^^^^
  File "/app/node_modules/@npmcli/run-script/node_modules/node-gyp/gyp/pylib/gyp/__init__.py", line 662, in main
    return gyp_main(args)
           ^^^^^^^^^^^^^^
  File "/app/node_modules/@npmcli/run-script/node_modules/node-gyp/gyp/pylib/gyp/__init__.py", line 629, in gyp_main
    [generator, flat_list, targets, data] = Load(
                                            ^^^^^
  File "/app/node_modules/@npmcli/run-script/node_modules/node-gyp/gyp/pylib/gyp/__init__.py", line 150, in Load
    result = gyp.input.Load(
             ^^^^^^^^^^^^^^^
  File "/app/node_modules/@npmcli/run-script/node_modules/node-gyp/gyp/pylib/gyp/input.py", line 3021, in Load
    LoadTargetBuildFile(
  File "/app/node_modules/@npmcli/run-script/node_modules/node-gyp/gyp/pylib/gyp/input.py", line 411, in LoadTargetBuildFile
    build_file_data = LoadOneBuildFile(
                      ^^^^^^^^^^^^^^^^^
  File "/app/node_modules/@npmcli/run-script/node_modules/node-gyp/gyp/pylib/gyp/input.py", line 239, in LoadOneBuildFile
    build_file_contents = open(build_file_path, "rU").read()
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: invalid mode: 'rU' while trying to load binding.gyp
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/app/node_modules/@npmcli/run-script/node_modules/node-gyp/lib/configure.js:351:16)
gyp ERR! stack     at ChildProcess.emit (node:events:514:28)
gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:291:12)
gyp ERR! System Linux 5.15.0-76-generic
gyp ERR! command "/usr/local/bin/node" "/app/node_modules/.bin/node-gyp" "rebuild"
gyp ERR! cwd /app/node_modules/bufferutil
gyp ERR! node -v v18.17.1
gyp ERR! node-gyp -v v7.1.2
gyp ERR! not ok
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
The command '/bin/sh -c echo "Building watcher-ts" &&     yarn && yarn build' returned a non-zero code: 1
Error running build for cerc/watcher-ts
FATAL Error: container build failed and --continue-on-error not set, exiting
Build fails trying to run gyp to compile some native dependency. Usually this happens because there was a pre-built x86 binary that is used and the gyp build is therefore never run, and there's some bug in the gyp build process. The bug is never exercised because on x86 the pre-built binary is fetched. Tail of the log: ``` Building watcher-ts yarn install v1.22.19 [1/4] Resolving packages... [2/4] Fetching packages... warning Pattern ["node-fetch@https://registry.npmjs.org/@achingbrain/node-fetch/-/node-fetch-2.6.7.tgz"] is trying to unpack in the same destination "/usr/local/share/.cache/yarn/v6/npm-node-fetch-2.6.7-1b5d62978f2ed07b99444f64f0df39f960a6d34d/node_modules/node-fetch" as pattern ["node-fetch@2.6.7"]. This could result in non-deterministic behavior, skipping. [3/4] Linking dependencies... warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cache > eslint-config-standard@16.0.3" has incorrect peer dependency "eslint@^7.12.1". warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cache > eslint-plugin-promise@5.2.0" has incorrect peer dependency "eslint@^7.0.0". warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cli > @apollo/client@3.7.10" has unmet peer dependency "graphql@^14.0.0 || ^15.0.0 || ^16.0.0". warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cli > @graphql-tools/utils@9.2.1" has unmet peer dependency "graphql@^14.0.0 || ^15.0.0 || ^16.0.0 || ^17.0.0". warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cli > apollo-server-express@3.12.0" has unmet peer dependency "graphql@^15.3.0 || ^16.0.0". warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cli > graphql-subscriptions@2.0.0" has unmet peer dependency "graphql@^15.7.2 || ^16.0.0". warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cli > eslint-config-standard@5.3.5" has unmet peer dependency "eslint@>=2.0.0". warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/graph-node > @nomiclabs/hardhat-waffle@2.0.5" has unmet peer dependency "ethereum-waffle@*". warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cache > @typescript-eslint/eslint-plugin > tsutils@3.21.0" has unmet peer dependency "typescript@>=2.8.0 || >= 3.2.0-dev || >= 3.3.0-dev || >= 3.4.0-dev || >= 3.5.0-dev || >= 3.6.0-dev || >= 3.6.0-beta || >= 3.7.0-dev || >= 3.7.0-beta". warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cli > @apollo/client > @graphql-typed-document-node/core@3.2.0" has unmet peer dependency "graphql@^0.8.0 || ^0.9.0 || ^0.10.0 || ^0.11.0 || ^0.12.0 || ^0.13.0 || ^14.0.0 || ^15.0.0 || ^16.0.0 || ^17.0.0". warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cli > @apollo/client > graphql-tag@2.12.6" has unmet peer dependency "graphql@^0.9.0 || ^0.10.0 || ^0.11.0 || ^0.12.0 || ^0.13.0 || ^14.0.0 || ^15.0.0 || ^16.0.0". warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cli > apollo-server-express > apollo-server-types@3.8.0" has unmet peer dependency "graphql@^15.3.0 || ^16.0.0". warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/codegen > @poanet/solidity-flattener > glob-promise@3.4.0" has unmet peer dependency "glob@*". warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cli > @cerc-io/nitro-node > @cerc-io/nitro-protocol > @typechain/ethers-v5@9.0.0" has unmet peer dependency "typechain@^7.0.0". warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cli > @cerc-io/nitro-node > @cerc-io/nitro-protocol > @typechain/ethers-v5@9.0.0" has unmet peer dependency "@ethersproject/bytes@^5.0.0". warning "workspace-aggregator-608cdab6-be1f-43b3-a3ce-3264f5e759b8 > @cerc-io/cli > @cerc-io/nitro-node > @cerc-io/nitro-protocol > @typechain/ethers-v5@9.0.0" has unmet peer dependency "@ethersproject/abi@^5.0.0". [4/4] Building fresh packages... error /app/node_modules/bufferutil: Command failed. Exit code: 1 Command: node-gyp-build Arguments: Directory: /app/node_modules/bufferutil Output: gyp info it worked if it ends with ok gyp info using node-gyp@7.1.2 gyp info using node@18.17.1 | linux | arm64 gyp info find Python using Python version 3.11.6 found at "/usr/bin/python3" gyp http GET https://nodejs.org/download/release/v18.17.1/node-v18.17.1-headers.tar.gz gyp http 200 https://nodejs.org/download/release/v18.17.1/node-v18.17.1-headers.tar.gz gyp http GET https://nodejs.org/download/release/v18.17.1/SHASUMS256.txt gyp http 200 https://nodejs.org/download/release/v18.17.1/SHASUMS256.txt (node:257) [DEP0150] DeprecationWarning: Setting process.config is deprecated. In the future the property will be read-only. (Use `node --trace-deprecation ...` to show where the warning was created) gyp info spawn /usr/bin/python3 gyp info spawn args [ gyp info spawn args '/app/node_modules/@npmcli/run-script/node_modules/node-gyp/gyp/gyp_main.py', gyp info spawn args 'binding.gyp', gyp info spawn args '-f', gyp info spawn args 'make', gyp info spawn args '-I', gyp info spawn args '/app/node_modules/bufferutil/build/config.gypi', gyp info spawn args '-I', gyp info spawn args '/app/node_modules/@npmcli/run-script/node_modules/node-gyp/addon.gypi', gyp info spawn args '-I', gyp info spawn args '/root/.cache/node-gyp/18.17.1/include/node/common.gypi', gyp info spawn args '-Dlibrary=shared_library', gyp info spawn args '-Dvisibility=default', gyp info spawn args '-Dnode_root_dir=/root/.cache/node-gyp/18.17.1', gyp info spawn args '-Dnode_gyp_dir=/app/node_modules/@npmcli/run-script/node_modules/node-gyp', gyp info spawn args '-Dnode_lib_file=/root/.cache/node-gyp/18.17.1/<(target_arch)/node.lib', gyp info spawn args '-Dmodule_root_dir=/app/node_modules/bufferutil', gyp info spawn args '-Dnode_engine=v8', gyp info spawn args '--depth=.', gyp info spawn args '--no-parallel', gyp info spawn args '--generator-output', gyp info spawn args 'build', gyp info spawn args '-Goutput_dir=.' gyp info spawn args ] Traceback (most recent call last): File "/app/node_modules/@npmcli/run-script/node_modules/node-gyp/gyp/gyp_main.py", line 51, in <module> sys.exit(gyp.script_main()) ^^^^^^^^^^^^^^^^^ File "/app/node_modules/@npmcli/run-script/node_modules/node-gyp/gyp/pylib/gyp/__init__.py", line 670, in script_main return main(sys.argv[1:]) ^^^^^^^^^^^^^^^^^^ File "/app/node_modules/@npmcli/run-script/node_modules/node-gyp/gyp/pylib/gyp/__init__.py", line 662, in main return gyp_main(args) ^^^^^^^^^^^^^^ File "/app/node_modules/@npmcli/run-script/node_modules/node-gyp/gyp/pylib/gyp/__init__.py", line 629, in gyp_main [generator, flat_list, targets, data] = Load( ^^^^^ File "/app/node_modules/@npmcli/run-script/node_modules/node-gyp/gyp/pylib/gyp/__init__.py", line 150, in Load result = gyp.input.Load( ^^^^^^^^^^^^^^^ File "/app/node_modules/@npmcli/run-script/node_modules/node-gyp/gyp/pylib/gyp/input.py", line 3021, in Load LoadTargetBuildFile( File "/app/node_modules/@npmcli/run-script/node_modules/node-gyp/gyp/pylib/gyp/input.py", line 411, in LoadTargetBuildFile build_file_data = LoadOneBuildFile( ^^^^^^^^^^^^^^^^^ File "/app/node_modules/@npmcli/run-script/node_modules/node-gyp/gyp/pylib/gyp/input.py", line 239, in LoadOneBuildFile build_file_contents = open(build_file_path, "rU").read() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ ValueError: invalid mode: 'rU' while trying to load binding.gyp gyp ERR! configure error gyp ERR! stack Error: `gyp` failed with exit code: 1 gyp ERR! stack at ChildProcess.onCpExit (/app/node_modules/@npmcli/run-script/node_modules/node-gyp/lib/configure.js:351:16) gyp ERR! stack at ChildProcess.emit (node:events:514:28) gyp ERR! stack at ChildProcess._handle.onexit (node:internal/child_process:291:12) gyp ERR! System Linux 5.15.0-76-generic gyp ERR! command "/usr/local/bin/node" "/app/node_modules/.bin/node-gyp" "rebuild" gyp ERR! cwd /app/node_modules/bufferutil gyp ERR! node -v v18.17.1 gyp ERR! node-gyp -v v7.1.2 gyp ERR! not ok info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command. The command '/bin/sh -c echo "Building watcher-ts" && yarn && yarn build' returned a non-zero code: 1 Error running build for cerc/watcher-ts FATAL Error: container build failed and --continue-on-error not set, exiting ```
Member

similar to https://github.com/cerc-io/watcher-ts/issues/420, which i fixed by editing the file.

similar to https://github.com/cerc-io/watcher-ts/issues/420, which i fixed by editing the file.
Member
https://github.com/nodejs/node-gyp/issues/2219 but see this comment:https://github.com/nodejs/node-gyp/issues/2219#issuecomment-1359162118 `node-gyp 7.1.2` is before that fix
iskay commented 2023-10-06 21:32:49 +00:00 (Migrated from github.com)

can we use alpine 3.17 based image (which still uses python 3.10) instead of alpine 3.18 or does that cause other problems?

editing the watcher-ts Dockerfile to FROM node:18.17.1-alpine3.17 let it build for me

can we use alpine 3.17 based image (which still uses python 3.10) instead of alpine 3.18 or does that cause other problems? editing the watcher-ts Dockerfile to `FROM node:18.17.1-alpine3.17` let it build for me
Author
Owner

Sounds like a plan. Although we should avoid alpine in general. That's a bigger discussion though.

Sounds like a plan. Although we should avoid alpine in general. That's a bigger discussion though.
Sign in to join this conversation.
No Milestone
No project
No Assignees
3 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: cerc-io/stack-orchestrator#561
No description provided.