cerc-io/stack-orchestrator
17
1
Fork 2
Code Issues 112 Pull Requests 15 Actions Packages Projects Releases 473 Wiki Activity

Error on using laconic-so commands #471

New Issue
Closed
opened 2023-07-31 05:57:04 +00:00 by nikugogoi · 10 comments
nikugogoi commented 2023-07-31 05:57:04 +00:00 (Migrated from github.com)
Copy Link

  • On trying to use latest SO release for deploying MobyMask, got error with laconic-so version command

    $ laconic-so version
Traceback (most recent call last):
  File "/home/dev/bin/laconic-so/_bootstrap/__init__.py", line 76, in import_string
  File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/cli.py", line 20, in <module>
    from app import build_containers
  File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/app/build_containers.py", line 31, in <module>
    from app.base import get_npm_registry_url
  File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/app/base.py", line 18, in <module>
    from app.deploy import get_stack_status
  File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/app/deploy.py", line 26, in <module>
    from python_on_whales import DockerClient, DockerException
  File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/python_on_whales/__init__.py", line 1, in <module>
    from .client_config import ClientNotFoundError
  File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/python_on_whales/client_config.py", line 10, in <module>
    import pydantic
  File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/pydantic/__init__.py", line 3, in <module>
    import pydantic_core
  File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/pydantic_core/__init__.py", line 6, in <module>
    from ._pydantic_core import (
ModuleNotFoundError: No module named 'pydantic_core._pydantic_core'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/dev/bin/laconic-so/__main__.py", line 3, in <module>
  File "/home/dev/bin/laconic-so/_bootstrap/__init__.py", line 253, in bootstrap
  File "/home/dev/bin/laconic-so/_bootstrap/__init__.py", line 81, in import_string
  File "/home/dev/bin/laconic-so/_bootstrap/__init__.py", line 59, in import_string
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/cli.py", line 20, in <module>
    from app import build_containers
  File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/app/build_containers.py", line 31, in <module>
    from app.base import get_npm_registry_url
  File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/app/base.py", line 18, in <module>
    from app.deploy import get_stack_status
  File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/app/deploy.py", line 26, in <module>
    from python_on_whales import DockerClient, DockerException
  File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/python_on_whales/__init__.py", line 1, in <module>
    from .client_config import ClientNotFoundError
  File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/python_on_whales/client_config.py", line 10, in <module>
    import pydantic
  File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/pydantic/__init__.py", line 3, in <module>
    import pydantic_core
  File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/pydantic_core/__init__.py", line 6, in <module>
    from ._pydantic_core import (
ModuleNotFoundError: No module named 'pydantic_core._pydantic_core'

    Error was thrown on using other laconic-so commands too

  • Checked with other releases of laconic-so:

- On trying to use latest SO release for deploying MobyMask, got error with `laconic-so version` command ``` $ laconic-so version Traceback (most recent call last): File "/home/dev/bin/laconic-so/_bootstrap/__init__.py", line 76, in import_string File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/cli.py", line 20, in <module> from app import build_containers File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/app/build_containers.py", line 31, in <module> from app.base import get_npm_registry_url File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/app/base.py", line 18, in <module> from app.deploy import get_stack_status File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/app/deploy.py", line 26, in <module> from python_on_whales import DockerClient, DockerException File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/python_on_whales/__init__.py", line 1, in <module> from .client_config import ClientNotFoundError File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/python_on_whales/client_config.py", line 10, in <module> import pydantic File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/pydantic/__init__.py", line 3, in <module> import pydantic_core File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/pydantic_core/__init__.py", line 6, in <module> from ._pydantic_core import ( ModuleNotFoundError: No module named 'pydantic_core._pydantic_core' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "/home/dev/bin/laconic-so/__main__.py", line 3, in <module> File "/home/dev/bin/laconic-so/_bootstrap/__init__.py", line 253, in bootstrap File "/home/dev/bin/laconic-so/_bootstrap/__init__.py", line 81, in import_string File "/home/dev/bin/laconic-so/_bootstrap/__init__.py", line 59, in import_string File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1050, in _gcd_import File "<frozen importlib._bootstrap>", line 1027, in _find_and_load File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 688, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 883, in exec_module File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/cli.py", line 20, in <module> from app import build_containers File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/app/build_containers.py", line 31, in <module> from app.base import get_npm_registry_url File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/app/base.py", line 18, in <module> from app.deploy import get_stack_status File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/app/deploy.py", line 26, in <module> from python_on_whales import DockerClient, DockerException File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/python_on_whales/__init__.py", line 1, in <module> from .client_config import ClientNotFoundError File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/python_on_whales/client_config.py", line 10, in <module> import pydantic File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/pydantic/__init__.py", line 3, in <module> import pydantic_core File "/home/dev/.shiv/laconic-so_1d1696dfb73c0ef92795f7b95b3d6bac18c9edb389eb82952720f9e3760cb229/site-packages/pydantic_core/__init__.py", line 6, in <module> from ._pydantic_core import ( ModuleNotFoundError: No module named 'pydantic_core._pydantic_core' ``` Error was thrown on using other laconic-so commands too - Checked with other releases of laconic-so: - The error seems to be introduced in release [v1.1.0-8eed5a7-202307310428](https://github.com/cerc-io/stack-orchestrator/releases/tag/v1.1.0-8eed5a7-202307310428) - Release [v1.1.0-758ec16-202307300438](https://github.com/cerc-io/stack-orchestrator/releases/tag/v1.1.0-758ec16-202307300438) prior to that is working and not throwing the error
dboreham commented 2023-07-31 12:23:50 +00:00
Owner
Copy Link

This doesn't appear to be related to any recent commit. I'm wondering if it might be some build environment thing?

This may be related: https://github.com/pydantic/pydantic/issues/6557

This doesn't appear to be related to any recent commit. I'm wondering if it might be some build environment thing? This may be related: https://github.com/pydantic/pydantic/issues/6557
dboreham commented 2023-07-31 12:26:07 +00:00
Owner
Copy Link

btw, the problematic release works for me (although it does introduce a weird warning message from the same library):

$ ./laconic-so version
/home/david/.shiv/laconic-so_adfaa4046412af789761876fd25d439b3a10be5cb12b0acd910a72a86e5ad6f0/site-packages/pydantic/_internal/_config.py:269: UserWarning: Valid config keys have changed in V2:
* 'allow_population_by_field_name' has been renamed to 'populate_by_name'
  warnings.warn(message, UserWarning)
Version: 1.1.0-8eed5a7-202307310428
btw, the problematic release works for me (although it does introduce a weird warning message from the same library): ``` $ ./laconic-so version /home/david/.shiv/laconic-so_adfaa4046412af789761876fd25d439b3a10be5cb12b0acd910a72a86e5ad6f0/site-packages/pydantic/_internal/_config.py:269: UserWarning: Valid config keys have changed in V2: * 'allow_population_by_field_name' has been renamed to 'populate_by_name' warnings.warn(message, UserWarning) Version: 1.1.0-8eed5a7-202307310428 ```
dboreham commented 2023-07-31 12:35:46 +00:00
Owner
Copy Link

This seems to be some error from a mystery meat transitive dependency. The error's manifestation depends on the version of Python used.

This seems to be some error from a mystery meat transitive dependency. The error's manifestation depends on the version of Python used.
dboreham commented 2023-07-31 13:07:51 +00:00
Owner
Copy Link

Should be fixed in the latest release, please re-test.

Should be fixed in the latest release, please re-test.
nikugogoi commented 2023-07-31 13:17:12 +00:00 (Migrated from github.com)
Copy Link

Tested with latest release and getting same error.

The error's manifestation depends on the version of Python used.

What python version should we use?

Current python version

$ python3 --version
Python 3.10.6
Tested with latest release and getting same error. > The error's manifestation depends on the version of Python used. What python version should we use? Current python version ```bash $ python3 --version Python 3.10.6 ```
dboreham commented 2023-07-31 13:25:17 +00:00
Owner
Copy Link

stack orchestrator supports Python 3.8 and later so 3.10.6 is supported.

stack orchestrator supports Python 3.8 and later so 3.10.6 is supported.
dboreham commented 2023-07-31 13:28:25 +00:00
Owner
Copy Link

It looks like just updating our first order dependencies to the latest wasn't enough to pull in the updated pydantic.
I'll force its version explicitly.

It looks like just updating our first order dependencies to the latest wasn't enough to pull in the updated pydantic. I'll force its version explicitly.
dboreham commented 2023-07-31 16:44:56 +00:00
Owner
Copy Link

The underlying cause is : https://github.com/gabrieldemarmiesse/python-on-whales/pull/455

The underlying cause is : https://github.com/gabrieldemarmiesse/python-on-whales/pull/455
dboreham commented 2023-07-31 18:22:31 +00:00
Owner
Copy Link

Ok, hopefully this one https://github.com/cerc-io/stack-orchestrator/pull/473 should get us going again.

I don't fully understand what happened, but roughly: we use Docker on Whales which is a Python library for docker.
That library uses a library called pydantic, the purpose of which I don't understand fully, but it appears to involve extreme cleverness.
Pydantic people re-wrote their code totally, making pydantic v2.
Yesterday docker on whales people merged a PR that uses pydantic v2 by default.
This appears to produce two problems : spurious warning messages emitted to stdout/stderr from pydantic code; and exceptions thrown if you execute with some version of Python vs the version used to build the shiv package.

The second problem actually sounds pretty ominous because it has the feel that pydantic v2 is not "python-version-neutral". This would be a big problem for us because we are trying to ship a version (minor version) neutral package.

I've been able to force pydantic back to v1 for now, which works around the problem.

Ok, hopefully this one https://github.com/cerc-io/stack-orchestrator/pull/473 should get us going again. I don't fully understand what happened, but roughly: we use Docker on Whales which is a Python library for docker. That library uses a library called pydantic, the purpose of which I don't understand fully, but it appears to involve extreme cleverness. Pydantic people re-wrote their code totally, making pydantic v2. Yesterday docker on whales people merged a PR that uses pydantic v2 by default. This appears to produce two problems : spurious warning messages emitted to stdout/stderr from pydantic code; and exceptions thrown if you execute with some version of Python vs the version used to build the shiv package. The second problem actually sounds pretty ominous because it has the feel that pydantic v2 is not "python-version-neutral". This would be a big problem for us because we are trying to ship a version (minor version) neutral package. I've been able to force pydantic back to v1 for now, which works around the problem.
😕 1
dboreham commented 2023-07-31 18:22:57 +00:00
Owner
Copy Link

Should finally fix the issue: https://github.com/cerc-io/stack-orchestrator/pull/473

Should finally fix the issue: https://github.com/cerc-io/stack-orchestrator/pull/473
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
bug

Something isn't working

  
documentation

Improvements or additions to documentation

  
duplicate

This issue or pull request already exists

  
enhancement

New feature or request

  
feature

   
good first issue

Good for newcomers

  
help wanted

Extra attention is needed

  
in progress

   
invalid

This doesn't seem right

  
question

Further information is requested

  
wontfix

This will not be worked on

  
Copied from Github

An issue or PR manually copied from GitHub.

  
Kind/Breaking

Breaking change that won't be backward compatible

  
Kind/Bug

Something is not working

  
Kind/Documentation

Documentation changes

  
Kind/Enhancement

Improve existing functionality

  
Kind/Feature

New functionality

  
Kind/Security

This is security issue

  
Kind/Testing

Issue or pull request related to testing

  
Priority
Critical
The priority is critical

  
Priority
High
The priority is high

  
Priority
Low
The priority is low

  
Priority
Medium
The priority is medium

  
Reviewed
Confirmed
Issue has been confirmed

  
Reviewed
Duplicate
This issue or pull request already exists

  
Reviewed
Invalid
Invalid issue

  
Reviewed
Won't Fix
This issue won't be fixed

  
Status
Abandoned
Somebody has started to work on this but abandoned work

  
Status
Blocked
Something is blocking this issue or pull request

  
Status
Need More Info
Feedback is required to reproduce issue or to continue work

No Label
bug
documentation
duplicate
enhancement
feature
good first issue
help wanted
in progress
invalid
question
wontfix
Copied from Github
Kind/Breaking
Kind/Bug
Kind/Documentation
Kind/Enhancement
Kind/Feature
Kind/Security
Kind/Testing
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Reviewed
Confirmed
Reviewed
Duplicate
Reviewed
Invalid
Reviewed
Won't Fix
Status
Abandoned
Status
Blocked
Status
Need More Info
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: cerc-io/stack-orchestrator#471
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?