Next.js consumes run-time config at build time #633

New Issue

dboreham · 2023-11-08T21:23:40Z

dboreham commented

2023-11-08 21:23:40 +00:00

This came up trying to package the iglootools app. It seems to be a manifestation of this bug: https://github.com/vercel/next.js/issues/45383

The symptoms are that at build time it fails if a (valid) API key for bugsnag is not provided.
This obviously makes no sense because bugsnag is a thing concerned with run time state,
and has no business being invoked at build time.

Add details here on exactly what's happening and how/if we can work around:

This has more background: https://stackoverflow.com/questions/70764556/whats-the-difference-between-exposing-environment-variables-in-nextjs-through-t

And the bug above continues on this discussion forum where they eventually announce a redesigned process for build/config/pre-render: https://github.com/vercel/next.js/discussions/46544#discussioncomment-7115457

This came up trying to package the [iglootools](https://github.com/snowball-tools/iglootools-home/tree/sdk) app. It seems to be a manifestation of this bug: https://github.com/vercel/next.js/issues/45383 The symptoms are that at build time it fails if a (valid) API key for bugsnag is not provided. This obviously makes no sense because bugsnag is a thing concerned with run time state, and has no business being invoked at build time. Add details here on exactly what's happening and how/if we can work around: This has more background: https://stackoverflow.com/questions/70764556/whats-the-difference-between-exposing-environment-variables-in-nextjs-through-t And the bug above continues on this discussion forum where they eventually announce a redesigned process for build/config/pre-render: https://github.com/vercel/next.js/discussions/46544#discussioncomment-7115457

telackey commented

2023-11-11 00:23:40 +00:00

This seems to be "standard behavior" for next, so that it can pre-render static HTML. This is obviously a bad system though, as it makes it practically impossible to build a generic container which will be deployed with different configuration at runtime.

That is, with the standard next build step, the distinction between build time and runtime has been blurred to such an extent that practically it has to be built at runtime.

Fortunately, there is a partial fix/workaround in the new next experimental-compile and next experimental-generate commands.

With these, the build process gets split in half. The compilation step can be done in advance and does not require any "runtime" environment. The "generate" step still requires runtime environment and needs to be delayed.

We have incorporated this distinction into our SO scripts. Ahead of the compile step, all the environment variables are replaced with placeholder values, a new experimental-compile step is injected into package.json, and executed. When the container is launched, the placeholders are replaced with the true values just before running next experimental-generate && next start.

Unfortunately, the experimental-generate command proved to be quite experimental, and we uncovered a thorny bug that would cause it to hang indefinitely. This bug was dependent on Next.js version, affecting all versions between 13.4.13 and 14.0.1 inclusively. It continues to affect the latest versions if the incremental-cache-server is enabled. (The cache server is disabled by default starting in 14.0.2, released yesterday, which had the happy accident of avoiding the hang, though that is not why they disabled it.)

I opened an issue with Next.js here identifying the issue: https://github.com/vercel/next.js/issues/58274

And also a PR fixing it here: https://github.com/vercel/next.js/pull/58276

In the meantime I added a workaround in the build script which will scrape the output to discover when the generate step has completed, then signal the process to exit.

This seems to be "standard behavior" for next, so that it can pre-render static HTML. This is obviously a bad system though, as it makes it practically impossible to build a generic container which will be deployed with different configuration at runtime. That is, with the standard `next build` step, the distinction between build time and runtime has been blurred to such an extent that practically it has to be built at runtime. Fortunately, there is a partial fix/workaround in the new `next experimental-compile` and `next experimental-generate` commands. With these, the build process gets split in half. The compilation step can be done in advance and does not require any "runtime" environment. The "generate" step still requires runtime environment and needs to be delayed. We have incorporated this distinction into our SO scripts. Ahead of the compile step, all the environment variables are replaced with placeholder values, a new `experimental-compile` step is injected into package.json, and executed. When the container is launched, the placeholders are replaced with the true values just before running `next experimental-generate && next start`. Unfortunately, the `experimental-generate` command proved to be quite experimental, and we uncovered a thorny bug that would cause it to hang indefinitely. This bug was dependent on Next.js version, affecting all versions between 13.4.13 and 14.0.1 inclusively. It continues to affect the latest versions if the incremental-cache-server is enabled. (The cache server is disabled by default starting in 14.0.2, released yesterday, which had the happy accident of avoiding the hang, though that is not why they disabled it.) I opened an issue with Next.js here identifying the issue: https://github.com/vercel/next.js/issues/58274 And also a PR fixing it here: https://github.com/vercel/next.js/pull/58276 In the meantime I added a workaround in the build script which will scrape the output to discover when the generate step has completed, then signal the process to exit.

Sign in to join this conversation.

No Milestone

No project

No Assignees

2 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: cerc-io/stack-orchestrator#633