deep-stack/stack-orchestrator
3
0
Fork 0
forked from cerc-io/stack-orchestrator
Code Pull Requests Activity

Defensively handle errors examining app requests. (#922)

Browse Source 
Related to cerc-io/webapp-deployment-status-api#10

There are two issues in that.  One is that the output probably changed recently, whether in the client or server, where no matching record is found by ID (Note this is specific to `laconic record get --id <v>` and does not seem to apply to the similar command to retrieve a record by name, `laconic name resolve <n>`).

Rather than returning `[]` it is now returning `[ null ]`.  This cause us to think there *was* an application record found, and we attempt to treat the `null` entry like an Application object.  That's fixed by filtering out null responses, which is a good precaution for the deployer, though I think it makes sense to ask whether this new behavior by the client/server is correct.  Seems suspicious.

The other issue is that all the defensive checks we had in place to deal with broken/bad AppDeploymentRequests were around the _build_.  This error was coming much earlier, merely when parsing and examining the request to see if it needed to be handled at all.

I have now added similar defensive error handling around that portion of the code.

Reviewed-on: cerc-io/stack-orchestrator#922
Reviewed-by: zramsay <zramsay@noreply.git.vdb.to>
Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com>
Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
This commit is contained in:
Thomas E Lackey 2024-08-14 18:04:31 +00:00 committed by Thomas E Lackey
parent 8576137557
commit 5c275aa622
2 changed files with 33 additions and 24 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
stack_orchestrator/deploy/webapp/deploy_webapp_from_registry.py
View File

@ -274,13 +274,16 @@ def command(ctx, kube_config, laconic_config, image_registry, deployment_parent_
requests_by_name = {} requests_by_name = {}
skipped_by_name = {} skipped_by_name = {}
for r in requests: for r in requests:
status = None
try:
if r.id in previous_requests and previous_requests[r.id].get("status", "") != "RETRY": if r.id in previous_requests and previous_requests[r.id].get("status", "") != "RETRY":
print(f"Skipping request {r.id}, we've already seen it.") print(f"Skipping request {r.id}, we've already seen it.")
continue continue
app = laconic.get_record(r.attributes.application) app = laconic.get_record(r.attributes.application)
if not app: if not app:
print("Skipping request %s, cannot locate app." % r.id) print(f"Skipping request {r.id}, cannot locate app.")
status = "SEEN"
continue continue
requested_name = r.attributes.dns requested_name = r.attributes.dns
@ -302,6 +305,12 @@ def command(ctx, kube_config, laconic_config, image_registry, deployment_parent_
print("Found request %s to run application %s on %s." % (r.id, r.attributes.application, requested_name)) print("Found request %s to run application %s on %s." % (r.id, r.attributes.application, requested_name))
requests_by_name[requested_name] = r requests_by_name[requested_name] = r
except Exception as e:
print(f"ERROR examining request {r.id}: " + str(e))
status = "ERROR"
finally:
if status:
dump_known_requests(state_file, [r], status)
# Find deployments. # Find deployments.
deployments = laconic.app_deployments() deployments = laconic.app_deployments()

2
stack_orchestrator/deploy/webapp/util.py
View File

@ -172,7 +172,7 @@ class LaconicRegistryClient:
name_or_id, name_or_id,
] ]
parsed = [AttrDict(r) for r in json.loads(logged_cmd(self.log_file, *args))] parsed = [AttrDict(r) for r in json.loads(logged_cmd(self.log_file, *args)) if r]
if len(parsed): if len(parsed):
self._add_to_cache(parsed) self._add_to_cache(parsed)
return parsed[0] return parsed[0]