Sigil (0.22.11)

Existing projects build without changes. Run sigil deps install once in each project after upgrading, so it writes the new dependency index.

sigil task runs a project's scripts and its dependencies'

sigil task <name> [args...] runs one of the project's tasks: a step task (a tasks: build pipeline, which takes no arguments) or a script task, a Sigil shell script that receives every argument after its name unchanged. A package declares script tasks as quoted data under provides::

provides: (list
  script-tasks: '((publish-web file: "tasks/publish-web.sgl"
                               description: "Publish a checked site tree")))

A dependency's script tasks run only with its package prefix, as in sigil task substratic:publish-web, and only for dependencies listed in dependencies: or dev-dependencies:. A bare name is always the project's own task, so a dependency can't stand in for one, and sigil build, sigil test and sigil deps install never run a script task. Before a script task runs, one line on stderr says where it comes from, including the locked commit. sigil task --list lists every task.

sgx is sigil task under a short name: sigil cli install, use and upgrade write $SIGIL_HOME/bin/sgx, and sigil completions bash|zsh prints a completion script for both. sigil run -t <task> still works, prints Deprecated: use 'sigil task' instead, and goes away before 1.0.

Upgrading from 0.22.10 or earlier, the older CLI does the switch and doesn't know about sgx, so run sigil cli use 0.22.11 once more after upgrading to create it.

sigil run passes your arguments through

Everything after sigil run's own options now goes to the program unchanged: sigil run app --dry-run -t x passes app --dry-run -t x instead of running task x. The program reads them with (command-line), as a built binary does.

Scripts know their project and can keep secrets from child processes

The shell prelude gains (project-root), the root of the project the script was started in. $, $?, the captures, spawn and pipe stages accept env-unset: ("NAME" ...) to keep a variable such as a token out of a child's environment.

Commands start faster, and installed dependencies can be checked

sigil deps install writes .sigil/deps.index, so commands no longer read every dependency's package.sgl each time they start: an empty script starts about 100 ms faster in a project with many dependencies. sigil deps verify re-hashes each installed dependency and reports anything changed since it was installed. Native builds no longer write into a dependency's shared copy; the generated version_info.c goes to the build's own output. A package.sgl whose sigil: floor is newer than the running Sigil is refused before anything in it runs: this package needs sigil ≥ X; this is Y.

Large numbers, web builds and macros no longer corrupt memory

• number->string on a large integer in a radix below 10 wrote past the end of its buffer, and string->number with a radix above 16 skipped the digits g to z and could abort.
• On the web, bytevector-copy! and other range checks could wrap around and write out of bounds, and make-vector or make-bytevector with a huge length quietly made a tiny one. They raise an error now.
• syntax->datum could lose objects to the garbage collector under memory pressure, so macro-heavy code could read freed memory.
• The garbage collector didn't trace the recent-calls ring behind stack traces, so capturing a stack trace after a collection could copy freed memory into the trace and corrupt the heap later, typically as a crash at exit. The profiler's tables had the same gap under --profile.
• A path containing a NUL character was cut at the NUL, so the file checked and the file opened could differ. It raises an error now.

let no longer mistakes its body for a named let

A let whose body starts with a list of lists, such as (let ((k 5)) ((f k)) 0), was read as a named let and referred to a global k instead of the binding, failing with unbound variable 'k'. It affected every backend.

Errors read the same on every backend

Generic runtime errors now have the kind runtime-error everywhere (the bytecode VM used vm-error), and index errors say "index out of range". Code that catches vm-error by name should catch runtime-error instead.

Fixes

• length of an improper list returned -1 in compiled code, and length or apply on a circular list aborted or hung. Both raise a catchable error now, and list? of a circular list is #f.
• apply accepts continuations and parameter objects, as a direct call does.
• Re-entering a multi-shot call/cc continuation could stop the bytecode VM with a GET_CELL error, and compile-file on a file with several libraries folded them into the first.
• (format "~a" proc) printed compiled procedures as #<object>; it prints #<procedure>.
• format, println and str printed integers of 2^31 and above wrong on the web (4294967297 came out as 1, 4294967295 as -1), and wrote a non-ASCII character as one broken byte on every platform: (format "~a" (integer->char 955)) now prints λ.
• Web apps print non-ASCII text correctly (it came out as replacement characters), keep stdout and stderr in order, and no longer slow down on long output lines.
• sigil_make_bytevector now returns zeroed memory, so C code that builds a bytevector, including an embedder's natives, can't expose old heap contents.
• string-repeat could crash in desktop builds, and a corrupt .sgb file could corrupt memory as it loaded. make-srcloc and make-generated-srcloc raise for positions past 2^31-1.
• Compiling a library in a process that already had a library of the same name loaded bound its exports in a different order, so the same source could compile to different bytecode. Compiled output no longer depends on what else is loaded.

Desktop apps can compile their dependencies natively (opt-in)

Set SIGIL_NATIVE_DEPS=all when you build, and a desktop app's release build compiles the dependency modules it imports to native code, not just its own modules; the rest stays bytecode. Calls into libraries such as sigil-geometry run 1.5 to 2 times faster and stop allocating per argument. It is off by default in this release while games try it. Binaries grow with the code they carry: 11% for Crash The Stack, 31% for Phantom Burn and 36% for Harkfell.

To try it, raise sigil: to 0.22.11, run sigil deps update --upgrade-sigil, and build with SIGIL_NATIVE_DEPS=all. A native app links the runtime its lock names, so the lock must carry 0.22.11's: it fixes a garbage-collector bug that native dependency code exposed.

Known issues

• Quoted list constants in compiled code are rebuilt each time they're evaluated, so two evaluations of one quoted list aren't eq? as they are on the bytecode VM.
• Dependency copies are not yet read-only: older Sigil versions still write into them, so locking them down would break builds with an older CLI.
• (apply apply k args) with a bytecode continuation k doesn't work; (apply k args) does.
• write still prints a compiled procedure as #<object …> while format "~a" now prints #<procedure>.