Environment Variables

User Variables

Sprite’s behavior can be controlled by setting certain environment variables. The following are recognized:

CURRYPATH

A colon-separated list of paths used to search for Curry modules. Sprite silently appends to this the path to its system libraries. A program given as a file (sprite-exec prog.curry, sprite-make --so prog.curry, curry.import_(path, is_sourcefile=True)) has its directory searched first, before the entries of CURRYPATH, as PAKCS searches: the modules beside the program are found without a CURRYPATH that names their directory. A module given by name is searched on CURRYPATH alone.

SPRITE_HOME

The installation tree of Sprite: the directory that holds bin, curry, lib, python, sysconfig, and tools. The launchers under bin set it. When it is not set, the Python package finds the tree from its own location, so import curry works with the package directory on PYTHONPATH.

SPRITE_FRONTEND_WARNINGS

The Curry front end warns on overlapping rules (“Function f is potentially non-deterministic due to overlapping rules”), under which build 0 = ...; build n = ... compiles to a choice whose second alternative is infinite. A run of the front end (an import, a compile from text, sprite-make) reports that warning once per module through the log at the WARNING level, with the text of the front end; the other warnings of the front end stay quiet. The value 0 (or off, no, false) silences the warnings for the process; any other value, or none, leaves them on, and an empty value counts as unset. sprite-make -q silences them as well. The test runner, the test library and the benchmark harness set the variable to 0, so the output of a suite does not depend on which modules a run compiles; the run scripts of the examples whose rules overlap by design set it too. To compile a program without the warnings, say:

SPRITE_FRONTEND_WARNINGS=0 sprite-make --so prog.curry

SPRITE_GC_THRESHOLD

The number of nodes at which the collector of the C++ backend runs. The default is 1048576, about 30 MB of nodes in the block heap of the runtime. The allocator checks the count once per run of free slots, so a collection comes within a few thousand nodes of the threshold. After a collection the threshold is eight times the survivors, but not less than the configured value, so the heap stays within eight times the live nodes. The collector also runs when the live configurations of the scheduler reach one eighth of the node threshold, with the same growth rule: a set function consumed only in part leaves its queue to the collector. A lower value keeps the heap smaller. A higher value runs fewer collections, and it costs time on most programs, because a small heap stays in the cache. The value is read when the runtime library loads. To run the collector every 65536 nodes, say:

SPRITE_GC_THRESHOLD=65536 sprite-exec prog.curry

SPRITE_GC_GROWTH

The growth factor of the collector of the C++ backend: after a collection, the next one runs when the heap reaches this many times the survivors, but not before the threshold. The default is 8. A value of 2 keeps the heap within twice the live nodes and runs about three times as many collections. The value must be at least 2 and is read when the runtime library loads.

The default stays at 8 by a measurement of the memory suite of the benchmarks at 8, 12 and 16 (the dated TODO entry on G8, issue #83). The factor touches a program only when the survivors of a collection exceed the threshold divided by the factor (131,072 nodes at growth 8 and 65,536 at growth 16, with the default threshold), or when the live configurations exceed one eighth of that number, since the configuration threshold follows the same rule. 18 of the 29 programs measured kept their collections and their nodes marked exactly at every factor. On the search programs with a live set near 300 MiB the trade is proportional, as the policy makes it. Growth 16 halves the collections and the nodes marked (QueensSet9 11 to 6 collections, SearchQueens 19 to 10, PermSort 9 to 5). It cuts the seconds of the collector by a quarter to a half, which is 1 to 8 percent of the CPU time of those programs and 1 to 3 percent of their instructions. It costs 43 to 53 percent more peak RSS (QueensSet9 330 to 473 MiB, SearchQueens 307 to 465 MiB, PermSort 314 to 481 MiB), and the bound of the heap becomes sixteen times the live nodes in place of eight. A user with memory to spare buys the saving with SPRITE_GC_GROWTH=16. Growth 12 sits halfway on both sides of the trade.

SPRITE_GC_STRESS

The stress mode of the collector of the C++ backend. With the value 1, the collector runs at every safepoint of the scheduler: after every rewrite step of the outermost evaluation, and a nested set function hands the request outward. A node that no root reaches is then reclaimed at the next step, so a missing root shows up at once, as a wrong value or a crash. Every collection of the mode also runs the heap verifier, which checks every live node, its successors, and every node Python holds against the block heap and stops the process when the heap is inconsistent. The mode costs a collection per step. It is meant for the test suite (see tests/README), not for a program. The value 0 or an empty value turns the mode off, which is the default; another value turns it off with a warning. The value is read when the runtime library loads. To run the unit tests in stress mode, say:

SPRITE_GC_STRESS=1 ./run_tests 'unit_*.py'

A runtime built with the Memory Pool System (make GC=mps, an experiment) reads the three variables above with another meaning: SPRITE_GC_THRESHOLD runs a full collection every so many nodes in addition to the collections MPS schedules itself (unset: none), SPRITE_GC_GROWTH scales the configuration threshold after a full collection, and SPRITE_GC_STRESS runs a full collection at every safepoint, without a heap verifier. SPRITE_GC_MPS_ARENA_MB (256), SPRITE_GC_MPS_NURSERY_KB (8192), and SPRITE_GC_MPS_GEN1_KB (65536) size its arena and its two generations. See src/cyrt/graph/gc/mps.cpp.

SPRITE_GC_REPORT

With the value 1, every collection of the collector of the C++ backend prints one line of key=value pairs on the standard error stream when it ends: its number, whether it ran inside a nested evaluation, the nodes before it and the survivors, the nodes marked (old and young apart), the seconds of its phases, the configurations pushed, the queues and configurations destroyed, the writes into old nodes since the collection before, the threshold it set, and the configurations, queues, sets and blocks alive. The fields are those of sprite-exec --stats (gc_marked and the others, without the prefix), per collection instead of summed. The value 0 or an empty value turns the report off, which is the default; another value turns it off with a warning. The value is read when the runtime library loads.

SPRITE_INTERPRETER_FLAGS

Overrides default flags in Sprite’s Curry interpreter. This can be set to a comma-separated list of colon-separated pairs (without spaces). An item splits at its first colon, so rotation:time:10ms is one item.

See the list of flags for details, and Interpreter flags for the table with the defaults.

For example, to have Sprite generate debug code and print execution traces, set the following:

SPRITE_INTERPRETER_FLAGS=trace:True,debug:True

The backend flag names the backend: cxx, the C++ backend, the default of an installation (configure --with-default-backend).

The interpret flag selects how the C++ backend runs a module without a compiled object: tiered (the default) interprets it and compiles it in the background, new interprets it and never compiles it, all interprets every module, and off compiles every module first (a module imported from an ICurry object at its first use; see the flag in curry.interpreter.flags). An installation without a C++ compiler runs under the default with one notice; see Installing and running without a C++ compiler.

The stack_limit flag sets the number of bytes of C stack the C++ backend lets one evaluation use. The default is 4194304. When an alternative reaches the limit, the backend unwinds to the scheduler and runs the other alternatives. An alternative that cannot proceed within the limit is dropped, and its error is reported after the others have run. A limit larger than the stack of the thread is clamped to that stack, less a margin of 1 MiB. Set the flag to None to disable the guard. To raise the limit, raise the stack size (ulimit -s) as well, and say:

SPRITE_INTERPRETER_FLAGS=stack_limit:16777216 sprite-exec prog.curry

The rotation flag selects how the C++ backend paces the rotation of its queue of alternatives. The scheduler rotates at a safepoint, after a completed rewrite step, so a diverging alternative cannot starve the others. In time mode, the default (time:10ms), a ticker thread sets one byte every quantum and the safepoint polls it; the latency of a waiting alternative is bounded by the quantum whatever a step costs. In step mode (steps:65536) the safepoint counts the steps and rotates every N of them; the schedule then depends on the program alone, so the exact steps and forks of a search and the order of its values reproduce between runs. A deterministic subcomputation never rotates in either mode. In time mode two runs of a non-deterministic goal may print their values in a different order; to pin the order, set step mode:

SPRITE_INTERPRETER_FLAGS=backend:cxx,rotation:steps:65536 sprite-exec prog.curry

Time mode starts its thread at the first evaluation, so a host that forks after an evaluation is multi-threaded, and Python warns on os.fork in such a process; the forked child starts a ticker of its own at its first evaluation. Step mode starts no thread. SPRITE_ROTATION (below) sets the same flag alone.

The typed_expr flag selects the typed builder of curry.expr(). It is True by default. With False, curry.expr converts Python values by their Python type alone and supplies no class dictionaries, as curry.raw_expr() does, and exprtype is ignored. The flag serves bisection:

SPRITE_INTERPRETER_FLAGS=typed_expr:False install/bin/python script.py

SPRITE_LOG_FILE

The file to which logging output is directed. The default, -, directs this to the standard error stream. import curry configures the root logger of Python (logging.basicConfig): a handler on that file or stream with the format %(asctime)s [%(levelname)s] %(message)s, and the level of SPRITE_LOG_LEVEL. A program that configured logging before the import keeps its configuration. The two variables change nothing then. An unknown level name still stops the import with an error.

SPRITE_LOG_LEVEL

Sets the logging verbosity. The default level is WARNING. Supported values are:

CRITICAL Log only critical (usually fatal) problems.
ERROR Log errors.
WARNING Log warnings.
INFO Log information about what Sprite is doing.
DEBUG Log detailed information about everything.

Each of these includes all output from levels listed above it.

SPRITE_ROTATION

The rotation mode of the C++ backend: time:10ms (time mode, the default; the unit may be s, ms, us or ns) or steps:65536 (step mode). It sets the interpreter flag rotation (above) below SPRITE_INTERPRETER_FLAGS. The test runner, the benchmark harness and the CI jobs set steps:65536 through it, so that the counters of a run and the outputs of the tests reproduce; a value in the environment of a run wins. An empty value counts as unset. Other text is an error when the interpreter starts. Time mode starts a thread at the first evaluation, so a host that forks after an evaluation is multi-threaded (Python warns on os.fork); the forked child rotates with a ticker of its own. Step mode starts no thread. To run a search goal with the order of its values pinned, say:

SPRITE_ROTATION=steps:65536 SPRITE_INTERPRETER_FLAGS=backend:cxx sprite-exec prog.curry

Development Variables

Additional environment variables are recognized. These are intended for people developing Sprite itself. If you plan only to compile and run Curry programs with Sprite, then you should not need these.

SPRITE_CACHE_FILE

Names the cache database, an SQLite file. The database stores the output of the Curry front end, the Curry to ICurry step, which takes seconds per module. An entry is keyed by a digest of the module source, of the sources of the modules it imports, of the front-end options, and of the route from Curry to ICurry (SPRITE_CURRY2ICURRY) with its program and flags. The file name is not part of the key. So the same text compiles once on a machine, from any directory and, for a module compiled from a string, under any module name. A change in the text, or in an imported module, misses the cache. A new front end misses it too, and an entry written by one route is never served to the other. An error the front end reports about the program is cached as well; a failure of the environment is not, and neither is a missing module, because the key sees the Curry path only through the source files found in it. An entry holds the ICurry of the module and the two interface files the front end wrote for it (.fint and .icurry); a hit writes all three beside each other, so the types of a module are at hand without a run of the front end.

A file name turns the cache on. The empty string turns caching off. When the variable is not set, the installation decides: configure --cache icurry or --cache all (ENABLE_ICURRY_CACHE in Make.config) turns the cache on at $HOME/.sprite/cache.db. The test drivers set the variable to tests/.cache/icurry.db. The database can also hold Sprite’s in-memory IR of a JSON file (ENABLE_PARSED_JSON_CACHE).

Note

Outside the test drivers, caching is off by default.

SPRITE_CACHE_UPDATE

Specifies cache entries to update. An entry of the ICurry cache is replaced on its own when its inputs change. An update can still be needed for the parsed-JSON cache, or when a tool behind the front end changes without a change to the icurry program itself. It is fine to simply delete the cache file in that case, but this method provides a more conservative option.

Files matching the given pattern are updated in the cache database. For the ICurry cache the pattern is compared with the name of the Curry source file, for the parsed-JSON cache with the name of the JSON file. The pattern is interpreted as a glob unless it begins and ends with ‘/’, as in /pattern/, in which case it is considered a regular expression.

Example:

To update all compressed JSON files, set SPRITE_CACHE_UPDATE='*.json.z' in the environment.

SPRITE_CURRY2ICURRY

Names the route from Curry to ICurry. frontend runs the Curry front end and then Sprite’s own translation from FlatCurry (curry.toolchain.flat2icurry). icurry runs the Curry front end, then the icurry program, when configure was given --with-icurry; both routes need the front end. The default is the choice made by configure (option --curry2icurry), else the front end. Both routes write the same files.

SPRITE_CXX_PCH_ROOT

The directory under which the C++ backend keeps the precompiled form of the runtime header that every generated module includes. The default is the include directory of the installation, where the compiler finds it without an extra flag. When that directory cannot be written, or when the installation lies in a conda environment (a directory with a conda-meta entry above it), the default is a directory of the user’s cache instead: $XDG_CACHE_HOME/sprite/pch/<key>, or ~/.cache/sprite/pch/<key>, where the key is a digest of the real path of the installation. So a package is not written at run time. The cache keeps one directory of about 70 MB per installation and removes none of them; the directory pch may be deleted at any time, and the header is built again on the next compile. Set the variable to the empty string to compile without the precompiled header. The header is built again when it is older than a header the generated code includes (the include closure of cyrt/cyrt.hpp). A header outside the closure never makes it stale.

SPRITE_DEBUG

Enables debugging for Sprite internal errors. The command-line tools sprite-exec and sprite-make normally report unexpected errors tersely. Enabling this allows one to see the full stack trace when Sprite fails. Set this to the value 1 to enable debugging.

SPRITE_DISABLE_SYSLIB_CHECKS

When the variable is set, Sprite skips the check of the system libraries, the Prelude and the other modules of curry/ in the installation. The check finds each of them on the Curry path and confirms that no file earlier on the path shadows it. Without the variable a missing or shadowed system library ends the process with a message that names the paths.

SPRITE_ENABLE_BREAKPOINT

When the variable is set, import curry installs a breakpoint hook of Sprite’s own (sys.breakpointhook), an interactive prompt in the frame of the call, and the built-in pdbtrace, which starts PDB. For the development of Sprite itself.

SPRITE_FORCE_RECOMPILE_CXX

When the variable is set, the C++ backend leaves the generated C++ file and the shared object of a module out of the prerequisites it compares, so every module is generated and compiled again whatever their times say.

SPRITE_PRODUCT_CACHE

The directory of the product cache of the C++ backend (curry.toolchain._productcache). A compile of a module stores its shared object, the generated C++ and the ABI stamp there, under the digest of the stamp and a key of the facts that decide the products: the texts of the module (its source, its ICurry file and its JSON, the input of the code generator) and of the modules it imports, the format of the generated code and the sources of the code generator and of the ICurry reader, the flags of the optimizer and the inline budget, and the route from Curry to ICurry. No part of the key names a tree, and the products name no path: an object names the objects of its imports by their SONAME (sprite-<module>.so.<format>), not by path, and its record names the source relative to the installation or to the object. So an entry serves every tree of one runtime: the same tree after its products were removed or after an edit and its revert, and a second worktree, which compiles nothing the first compiled. sprite-make, the background compile of the tiered mode and the prepare pass of the test runner look in the cache before they compile. A module whose entry is there is neither generated nor compiled: its files are placed beside the source, the object as a hard link when the two lie on one file system, with a stamp for this installation. Under the interpreter flag interpret set to new nothing is placed: that mode interprets a module without a current object and never compiles it. Without the variable, or with the empty string, the cache is off; every product is then compiled in place, as before the cache existed. The test runner and the worktree script name a directory of their own (tests/.cache/products); a developer who wants one across trees names it. A module in the temporary directory of the system is never cached. The cache grows: each change to a runtime header the generated code includes, to the compiler or to the flags starts a new digest directory (about 42 MB for the Curry library), and the toolchain removes nothing; the directory may be deleted at any time. sprite-make prints one line with the counts of a run that restored or stored a product (sprite-make: product cache: 43 restored, 0 stored). The test runner and the test library set the variable to tests/.cache/products unless the environment names it, and the runner prunes that cache after its prepare pass: the digest directories of other runtimes go.

SPRITE_WORKTREE_ROOT

The directory under which scripts/new-worktree.sh puts the install and object trees of a new git worktree; the default is ~/.cache/sprite/worktrees.

SPRITE_WORKTREE_PREPARE

With the value 1, scripts/new-worktree.sh ends with the prepare pass of the test runner on the C++ backend (tests/run_tests --prepare-only), which compiles the shared Curry products of the test pool into the new worktree and stores them in the product cache. The script never copies the compiled products of the main tree: their ABI stamps name the installation of that tree. A product cache the two trees share serves the worktree the products of the main tree.

Test Variables

The test runner (tests/run_tests; section 10 of tests/README) sets the environment of every test file: SPRITE_HOME, SPRITE_CACHE_FILE (tests/.cache/icurry.db), SPRITE_PRODUCT_CACHE (tests/.cache/products) unless the environment names it, SPRITE_INTERPRETER_FLAGS (the backend of the run), SPRITE_ROTATION=steps:65536 unless the environment of the run names a mode, and SPRITE_FRONTEND_WARNINGS=0 unless the environment sets it. Three variables address the tests themselves:

SPRITE_TEST_MAX_VMEM_KB: The cap on the address space of a test process, in KiB, or unlimited. The runner keeps the cap as a backstop behind its memory budget (6 GB, or three times the cap of the file, whichever is larger); the other drivers under tests/ cap every process at 6 GiB.
SPRITE_TEST_FLAGS: Interpreter flags that the CI script .github/scripts/run-tests.sh appends to SPRITE_INTERPRETER_FLAGS for the files of a shard; the nightly job runs the suites under interpret:all and interpret:off through it.
SPRITE_UPDATE_EXPECTED: With the value 1, tests/unit_examples.py writes the output of each run script of examples/ to its expected.out instead of comparing with it; run it once per backend and review the diff.

Build Variables

These are variables of make, read when Sprite is built. Make.config describes them beside the settings of configure.

SPRITE_REBUILD_ICY: make stage derives the JSON of a library module from its committed .icy file and never translates the module again. With SPRITE_REBUILD_ICY=1 it runs the front end and the translation again and writes a new .icy; the committed files are pinned artifacts of the front end of PAKCS 3.4.1 and of icurry 3.1.0, so this is a deliberate step.
DEBUG=1: The debug flavor of the runtime: the assertions on, no optimization. The C++ backend compiles the modules of an interpreter whose flag debug is set in the same flavor.
TRACE=1: Computation tracing in the C++ runtime (SPRITE_TRACE_ENABLED).
COUNTERS=1: The scheduler counters of the C++ runtime (SPRITE_SCHEDULER_COUNTERS), reported by sprite-exec --stats; see sprite-exec.
GC=mps: The Memory Pool System in place of the default collector, an experiment behind a gate; the SPRITE_GC_* variables above change their meaning under it.
GC_WRITE_COUNTERS=1: The counters of the writes into old nodes (SPRITE_GC_WRITE_COUNTERS), the four gc_old_* fields of --stats. They are part of the ABI stamp, so a switch compiles every module again.
JOBS=N, PREFIX=DIR: The job count of the build (configure --jobs) and the directory of make install.