Commit Graph
23 Commits
Author SHA1 Message Date
bbassieandClaude Opus 5.5 909f6ab08b Move ModInfo.fromModFile to ModStore.readModInfo
Reading a mod zip needs zip_ops, which pulled yauzl into the web
interface bundle through the data classes. ModStore is node only, so
the zip reading now lives there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 10:17:40 +00:00
BastiaanandClaude Opus 5.5 c28e617b22 Replace jszip with yauzl and yazl (#1011)
* Replace jszip with yauzl and yazl

jszip loads a whole archive into memory to read it and buffers the
output when writing.  Saves and mods can be several hundred megabytes,
so read zips with yauzl, which only reads the central directory and
streams entries on demand, and write them with yazl, which streams the
output.

lib gains a ZipArchive wrapper around yauzl with the lookup helpers the
callers need, findRoot() now takes it instead of a JSZip.  Saves are
patched by streaming every kept entry into a new zip, mod info.json is
read straight from the file, exports are streamed to the controller and
Factorio zip downloads go through a temp file instead of a buffer.

Patching now inflates and deflates each entry again where jszip reused
the compressed data, on a 336 MB synthetic save that is 7 s instead of
0.7 s, with peak memory going from 416 MB to 151 MB.

Closes #1002

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* Copy save entries without recompressing them

yazl inflates and deflates every entry it is given, so patching a save
took ten times longer than with jszip, which reused the compressed
data.  Add a ZipWriter to lib that streams entries out as they are
added and copies entries from a ZipArchive byte for byte, with ZIP64
structures when sizes or offsets need them, and use it in patch().

On a 336 MB synthetic save patching now takes 0.4 s at 143 MB peak RSS
against 0.75 s at 417 MB with jszip.  Factorio loads the result.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* Handle save write errors before the patch loop awaits them

A write error on the temporary save, such as a full disk, rejected the
pipeline promise while addEntry() or addBuffer() was still pending. Nothing
was listening yet, so the host's unhandledRejection handler exited the
process instead of failing the instance start.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Keep the export zip cache local to each export

The cache was module level, so when two instances on one host exported at
once, the first to finish closed archives the other was still reading.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Start writing zips before adding files from disk

yazl reads files passed to addFile() as soon as they are added. An
unreadable file emitted an error while the directory traversal was still
running, before the output pipeline existed, and the process exited on the
uncaught error. build_mod and zipDirectory now start the pipeline first and
reject instead. build_mod also removes the partial zip on failure.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-17 21:44:19 +03:00
Hornwitser 5dcaca4d0b Migrate to ECMAScript Modules
Migrate the usage of CommonJS specific code and imports to ESM.  Roughly
summarised this:
- Changes all require() calls to import statement or expressions.
- Changes usage of module.exports to export statements.
- Replaces require.resolve with import.meta.resolve.
- Replaces conditional imports with conditional subpath imports.
- Replaces module === require.main with import.meta.main.
- Adds the full filename with extention to all relative imports.
- Changes all usage of __dirname and __filename to their import.meta
  equivalents.
- Sets "type": "module" in package.json files.
- Removes usage of "use strict".
- Adds .ts an an alias of .js for the webpack build.

Due to differences in how require and import resolves files the plugin
entrypoints will have to include the filename and extension after this
migration in order to work.
2026-09-11 22:43:57 +02:00
Cooldude2606 198ea76674 Add support for recommended mod dependencies (#946) 2026-07-18 21:57:50 +01:00
Cooldude2606 573274875c Rename to SourceVersion 2026-07-02 00:27:05 +01:00
Cooldude2606 2e27c4f499 Maintain strict tagging of GameVersion 2026-07-02 00:08:09 +01:00
bbassieandClaude Opus 4.8 8029ad3b2d Resolve GameVersion type as a plain-string supertype
Cooldude's GameVersion brand was disjoint from the clean version types in
both directions, forcing safeVersion/safeFactorioVersion companions and
breaking widening from FullVersion at file-name call sites.

Make GameVersion a plain string alias: the raw, un-normalised version as the
game reads it. The stricter template-literal types (FullVersion,
MajorMinorVersion, PartialVersion) now widen into it for free, while a
GameVersion still can't be used where a normalised version is required
without going through normaliseGameVersion/integerGameVersion. Runtime
validation stays in GameVersionSchema.

- Drop safeVersion/safeFactorioVersion; ModInfo.version stays raw (verbatim so
  it matches the mod's file name), integerVersion normalises via the new
  integerGameVersion. factorioVersion reverts to a normalised MajorMinorVersion
  since it is never part of a file name.
- ModRecord.version becomes a GameVersion so mod packs round-trip the exact
  file-name version.
- ModVersionEquality.testVersion / ModVersionRange.testVersion accept a raw
  GameVersion and normalise it; the equality bound stays strictly validated.
- Migration only pads clean partial versions; raw versions are kept verbatim.
- Cover the leading-zeros case: version kept verbatim in the file name while
  comparing equal to its normalised form.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 13:54:06 +00:00
Cooldude2606 523293d4d9 Manual adjustments to GameVersion and ReleaseChannel types
Through the creation of this commit, it was discovered that ModInfo.version must be typed as GameVersion due to file names. This results in wide spread propagation of the unsafe type.  The full impact including on ModRecord and ModVersion.testVersion have no been implemented until a path forward is decided.
2026-06-30 23:39:08 +01:00
bbassieandClaude Opus 4.8 5d20f8fc84 Revert strict ModInfo.jsonSchema and cover the factorio_version error
The strict major.minor jsonSchema did not improve any consuming type (the
factorioVersion field is already typed) and only complicated the fromJSON
signature, so revert it (and drop the now-unused MajorMinorVersionSchema).
Add a test for the unparseable factorio_version branch in fromJSON.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 23:20:37 +00:00
bbassie e27ad5fdf9 Merge remote-tracking branch 'upstream/master' into feature/lenient-mod-version-parsing
# Conflicts:
#	packages/lib/src/data/version.ts
2026-06-29 22:57:48 +00:00
bbassieandClaude Opus 4.8 5b1f7d796d Make ModInfo.jsonSchema strict about the major.minor factorio_version
The stored ModInfo always normalises factorio_version to a major.minor (see
fromJSON), so the serialised schema can use the strict MajorMinorVersionSchema
while infoJsonSchema keeps the lenient GameVersionSchema for raw info.json.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 22:49:59 +00:00
bbassieandClaude Opus 4.8 7c2793752d Add MajorMinorVersion type and dedupe the game version regex
Address review feedback: introduce a MajorMinorVersion type with
isMajorMinorVersion, rename majorMinorVersion to normaliseMajorMinorVersion,
and type ModInfo.factorioVersion as MajorMinorVersion so already-reduced
values are not re-reduced (ModPackViewPage uses mod.info.factorioVersion
directly). Reuse the single gameVersionRegExp in both GameVersionSchema and
normaliseGameVersion instead of duplicating the pattern.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 22:12:36 +00:00
bbassieandClaude Opus 4.8 97a6c32838 Parse mod versions leniently like the game does
The mod portal "ApiVersion" enum was validated strictly when loading a
mod's info.json, which rejected versions the game itself accepts. Versions
are now normalised the same lenient way the game reads them: leading
whitespace is allowed, the major and minor are required, the patch is
optional, any trailing content is ignored, and numbers are read as integers.

This applies to both a mod's version and its factorio_version, validated by
a regex schema rather than the fixed enum. The enum is no longer used to
validate versions at all - the mod portal is the source of truth, so its
error is now surfaced when a version is unsupported. The known-version list
kept for the web interface fallback gains 2.1.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 16:24:43 +00:00
Hornwitser 2e48b728dd Remove fs-extra and klaw (#882)
These packages are obsoleted by the promises API to the built-in Node fs
module.  The fs-extra package is also causing problems with broken APIs
that don't behave the same as Node's fs module, and throwing errors
without code properties on them.
2026-04-09 22:53:13 +02:00
Cooldude2606 aeb0c3f5f7 Fix mod dep parser breaking with spaces 2026-02-08 22:56:51 +00:00
Cooldude2606 d52931a326 Better handling of mod dependencies (#801)
* Add ModDependency and version types

* Add warnings for mod dependencies

* Add warning for wrong factorio version

* Add fix issues button

* Add automatic dependency resolution and download

* Add abilty to check for updates

* Add extra test coverage

* One extra test

* Fix copy paste error
2025-11-07 19:22:17 +00:00
Hornwitser 7625fd82e3 Cache ModInfo to reduce startup time (#606)
Store and use a cache of the loaded ModInfo data to mod-info-cache.json
in the mods directory during startup.  For simplicity this does not
cache mods added to the controller or host after startup, these get
added to the cache on the next startup.
2024-03-06 17:30:45 +00:00
Hornwitser c53781d16c Add time unit suffixes to variables (#561)
Add suffixes like Ms and Ns to variables whos time unit is not in
seconds.  This is aligning with a new code style of putting units
on variables that are not in SI base units.
2024-01-20 13:26:55 +01:00
Hornwitser 867bb0c646 Add ModInfo.id getter
Provide a simple unique identifier for mods consisting of the name
followed by an underscore and then the version.
2023-12-12 00:08:25 +01:00
Hornwitser f12a9926eb Add isDeleted and updatedAt to subscribable types
Prepare for subscriptinos being able to be cached and resumed by adding
updatedAt and isDeleted fields to the subscribable types.
2023-12-08 18:22:34 +01:00
Hornwitser 853866492b Refactor controller mod storage into ModStore
Structure the storage of mods on the controller into into a reusable
component in preparation for the host using the same kind of mod storage
to retrieve and cache the mods it needs to run mod packs on instances.
2023-12-03 19:59:14 +01:00
Hornwitser a66e930997 Use strict type checking 2023-08-12 02:31:17 +02:00
Hornwitser 679c00527f Port remaining lib/data modules to TypeScript 2023-08-11 01:13:38 +02:00