Skip to content

Conversation

@doorgan
Copy link
Collaborator

@doorgan doorgan commented Dec 24, 2025

Drastically improves Engine load times by incrementally namespacing engine files

Rather than complicating the transformer modules to make them aware of which files should be touched and which shouldn't, we:

  1. Figure out which files are new, change and which should be deleted from the output directory
  2. Delete the files that are flagged for deletion in the output folder
  3. Copy the new+changed files to a temporary folder
  4. Namespace that folder
  5. Copy the contents of the temp folder to the output folder

This achieves the same with the slight overhead of creating and cleaning up a temporary folder.

In my test on my m2 macbook I got the times down from ~10 seconds to 800-1200ms.

@doorgan doorgan force-pushed the doorgan/incremental-namespacing branch from dc807dd to 78011ff Compare December 24, 2025 07:01
@doorgan doorgan linked an issue Dec 24, 2025 that may be closed by this pull request
@doorgan doorgan force-pushed the doorgan/incremental-namespacing branch 3 times, most recently from 551aa61 to 63ded6d Compare December 24, 2025 07:33
@doorgan doorgan marked this pull request as ready for review December 24, 2025 13:06
@doorgan doorgan force-pushed the doorgan/incremental-namespacing branch from 3d00df6 to 5e741b8 Compare December 24, 2025 13:24
@Moosieus
Copy link
Collaborator

I wonder if Briefly is handling Windows' file system properly.

@doorgan
Copy link
Collaborator Author

doorgan commented Dec 26, 2025

It does! It creates something like C:\Users\Dorgan\AppData\Local\Temp/briefly--5764607095903232-YiSQwcVeeltDoQnC53r7/briefly-5764606969264128-2dncVrKcfGsPTZ2JmgPO

@Moosieus
Copy link
Collaborator

Maybe a file permission issue? I'm grasping at straws.

This PR reads solid otherwise.

@mhanberg
Copy link
Member

mhanberg commented Jan 1, 2026

Question, we don't need to namespace everytime during a normal release, correct? meaning you downloaded expert via a package manager, github releases, or are building a specific release from source.

I think in that case, we don't need to namespace every time? Only the first time.

In the case that we are working in development, we can maybe have an env var or just use mix env dev to namespace every time.

I'll still review this tho.

@doorgan
Copy link
Collaborator Author

doorgan commented Jan 1, 2026

@mhanberg we need to do that for nightly since the engine builds are "namespaced" by the expert version(which right now is always 0.1.0)
Unless we append the nightly commit hash to the version?

Even if we do that it's still beneficial to do incremental namespacing, it cuts down iteration time while developing

@mhanberg
Copy link
Member

mhanberg commented Jan 1, 2026

Nightly should already have the commit hash appended to it.

It's locally compiled releases that only have '0.1.0'

@doorgan doorgan force-pushed the doorgan/incremental-namespacing branch from 0ca2621 to 0acdcb2 Compare January 1, 2026 23:56
@doorgan
Copy link
Collaborator Author

doorgan commented Jan 1, 2026

Ah, I didn't realize we do that to version.txt for nightly, that does simplify things. Changing Expert.vsn/0 to read that file instead of the mix config does the trick for published releases

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Skip or incrementally namespace Engine builds

4 participants