Make numerical scripts compose across lists, vectors, and matrices

.github/workflows/tests.yml

@@ -1,22 +1,29 @@
-name: tests
+name: ci
 
 on:
   push:
-    branches: [ master ]
+    branches: [master]
   pull_request:
-    branches: [ master ]
+    branches: [master]
 
 env:
   CARGO_TERM_COLOR: always
 
 jobs:
-  build:
-
+  test:
     runs-on: ubuntu-latest
-
+    strategy:
+      matrix:
+        features: ["", "rand,nalgebra", "io,rand,nalgebra,metadata"]
     steps:
-      - uses: actions/checkout@v2
-      - name: Build
-        run: cargo build --verbose --all-features
-      - name: Run tests
-        run: cargo test --verbose --all-features
+      - uses: actions/checkout@v4
+      - uses: actions-rs/toolchain@v1
+        with:
+          toolchain: stable
+          override: true
+          components: rustfmt
+      - uses: Swatinem/rust-cache@v2
+      - name: Format
+        run: cargo fmt --all -- --check
+      - name: Test
+        run: cargo test --no-default-features --features "${{ matrix.features }}"

CHANGELOG.md

@@ -0,0 +1,26 @@
+# Changelog
+
+## 0.4.0 — Unreleased
+
+### Numerical workflows
+
+- Add `row`, `col`, and `mat` constructors over ordinary Rhai arrays with explicit orientation and numeric shape validation.
+- Add `dot` as a real scalar inner product of equal-length lists, rows, or columns. Use `mtimes` for matrix multiplication.
+- Accept numeric row and column vectors consistently in statistics, moving and cumulative operations, differences, interpolation, and trapezoidal integration. Mixed INT/FLOAT samples are supported; sequence results remain flat lists.
+- Validate complete matrix shapes and preserve row/column orientation through transpose, concatenation, and related operations.
+- Preserve integer precision in integer statistics, compare extrema numerically, and report malformed numerical input as script errors in the revised paths.
+- Return the fitted `intercept` from `regress`, alongside the existing predictor coefficients, p-values, and standard errors. Validate response shape and length before fitting.
+- Add examples for local CSV calibration and residual diagnostics, matrix inversion, projectile motion, and explicit XOR backpropagation.
+
+### Compatibility and migration
+
+- `regress` fits an intercept automatically. Do not add a column of ones; calculate predictions as `intercept + X * parameters`. Existing result fields retain their meanings, with `intercept` added as a separate field.
+- The Rhai names `transpose`, `horzcat`, `vertcat`, and `mtimes` remain available. Rust callers of matrix helpers should use the new `RhaiMatrix` wrappers or the corresponding `*_from_array` functions for array inputs.
+- New constructors reject empty or ragged shapes. The flat-list identities for `sum`, `prod`, `diff`, and `unique` remain available on empty arrays.
+- `dot` supports real vectors only; it does not implement matrix-axis overloads or complex conjugation. No implicit broadcasting is introduced.
+
+### Build and validation
+
+- Restore minimal-feature builds without the optional linear-algebra backend.
+- Limit Polars to the supported 0.45 API and enable only its CSV feature.
+- Test minimal, numerical, and full/documentation feature configurations in CI.

Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "rhai-sci"
-version = "0.3.0"
+version = "0.4.0"
 edition = "2021"
 authors = ["Chris McComb <ccmcc2012@gmail.com>"]
 description = "Scientific computing in the Rhai scripting language"
@@ -23,7 +23,7 @@ rand = ["randlib"]
 [dependencies]
 rhai = "1.8.0"
 nalgebralib = { version = ">=0.33.2,<1", optional = true, package = "nalgebra" }
-polars = { version = ">=0.45.1,<1", optional = true }
+polars = { version = "0.45.1", default-features = false, features = ["csv"], optional = true }
 url = { version = "2.0.0", optional = true }
 temp-file = { version = "0.2.0", optional = true }
 csv-sniffer = { version = "0.3.1", optional = true }
@@ -35,7 +35,7 @@ linregress = { version = "0.5.0", optional = true }
 [build-dependencies]
 rhai = "1.8.0"
 nalgebralib = { version = ">=0.33.2,<1", optional = true, package = "nalgebra" }
-polars = { version = ">=0.45.1,<1", optional = true }
+polars = { version = "0.45.1", default-features = false, features = ["csv"], optional = true }
 url = { version = "2.0.0", optional = true }
 temp-file = { version = "0.2.0", optional = true }
 csv-sniffer = { version = "0.3.1", optional = true }

README.md

@@ -2,52 +2,71 @@
 [![Crates.io](https://img.shields.io/crates/v/rhai-sci.svg)](https://crates.io/crates/rhai-sci)
 [![docs.rs](https://img.shields.io/docsrs/rhai-sci/latest?logo=rust)](https://docs.rs/rhai-sci)
 
-# About `rhai-sci`
+# rhai-sci
 
-This crate provides some basic scientific computing utilities for the [`Rhai`](https://rhai.rs/) scripting language,
-inspired by languages like MATLAB, Octave, and R. For a complete API reference,
-check [the docs](https://docs.rs/rhai-sci).
+Scientific computing for the [Rhai](https://rhai.rs/) scripting language, inspired
+by MATLAB, Octave, and R. Includes statistics, linear algebra, interpolation,
+integration, and regression.
 
-# Install
+## Quickstart
 
-To use the latest released version of `rhai-sci`, add this to your `Cargo.toml`:
+Add the crate to your `Cargo.toml`:
 
 ```toml
-rhai-sci = "0.2.3"
+rhai-sci = "0.4.0"
 ```
 
-# Usage
-
-Using this crate is pretty simple! If you just want to evaluate a single line of [`Rhai`](https://rhai.rs/), then you
-only need:
+Evaluate a Rhai expression:
 
 ```rust
 use rhai::INT;
 use rhai_sci::eval;
+
 let result = eval::<INT>("argmin([43, 42, -500])").unwrap();
+assert_eq!(result, 2);
 ```
 
-If you need to use `rhai-sci` as part of a persistent [`Rhai`](https://rhai.rs/) scripting engine, then do this instead:
+For a persistent engine, register `SciPackage` as shown in the
+[Rust host example](examples/regression_workflow.rs).
 
-```rust
-use rhai::{Engine, packages::Package, INT};
-use rhai_sci::SciPackage;
+## Numerical workflows
 
-// Create a new Rhai engine
-let mut engine = Engine::new();
+Use `row`, `col`, and `mat` to construct vectors and matrices from ordinary Rhai
+arrays. Statistics accept lists, rows, or columns; sequence results are flat lists.
+Use `mtimes` for matrix multiplication and `dot` for a scalar vector inner product.
 
-// Add the rhai-sci package to the new engine
-engine.register_global_module(SciPackage::new().as_shared_module());
+`regress(X, y)` fits an intercept automatically and returns it separately from the
+predictor coefficients. See the [workflow guide](docs/numerical-workflows.md) for
+shape conventions, predictions, and input validation.
 
-// Now run your code
-let value = engine.eval::<INT>("argmin([43, 42, -500])").unwrap();
+Run the bundled CSV example to fit a model and summarize its residuals:
+
+```bash
+cargo run --example regression_workflow
 ```
 
-# Features
+More examples: [matrix inversion](examples/matrix_inversion.rhai),
+[projectile motion](examples/projectile_motion.rhai), and
+[XOR backpropagation](examples/neural_network_backprop.rhai).
+
+## Features
+
+| Feature | Default | Enables |
+| --- | --- | --- |
+| `io` | On | CSV loading with `read_matrix` |
+| `nalgebra` | On | Matrix operations and regression |
+| `rand` | On | Random values and matrices |
+| `metadata` | Off | Function metadata and Rhai documentation tests |
+
+Disable default features and select only what you need for smaller builds;
+CSV support brings in Polars.
+
+## Reference
+
+[API documentation](https://docs.rs/rhai-sci) ·
+[Changelog](CHANGELOG.md) ·
+[Development checks](docs/numerical-workflows.md#development)
+
+## License
 
-| Feature    | Default  | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
-|------------|----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `metadata` | Disabled | Enables exporting function metadata and is ___necessary for running doc-tests on Rhai examples___.                                                                                                                                                                                                                                                                                                                                                                                                                    |
-| `io`       | Enabled  | Enables the [`read_matrix`](#read_matrixfile_path-string---array) function but pulls in several additional dependencies (`polars`, `url`, `temp-file`, `csv-sniffer`, `minreq`).                                                                                                                                                                                                                                                                                                                                      |
-| `nalgebra` | Enabled  | Enables several functions ([`regress`](#regressx-array-y-array---map), [`inv`](#invmatrix-array---array), [`mtimes`](#mtimesmatrix1-array-matrix2-array---array), [`horzcat`](#horzcatmatrix1-array-matrix2-array---array), [`vertcat`](#vertcatmatrix1-array-matrix2-array---array), [`repmat`](#repmatmatrix-array-nx-i64-ny-i64---array), [`svd`](#svdmatrix-array---map), [`hessenberg`](#hessenbergmatrix-array---map), and [`qr`](#qrmatrix-array---map)) but brings in the `nalgebra` and `linregress` crates. |
-| `rand`     | Enabled  | Enables the [`rand`](#rand) function for generating random FLOAT values and random matrices, but brings in the `rand` crate.                                                                                                                                                                                                                                                                                                                                                                                          |
+Licensed under [MIT](LICENSE-MIT.txt) or [Apache-2.0](LICENSE-APACHE.txt), at your option.

build.rs

@@ -225,3 +225,8 @@ mod functions {
 
 #[cfg(feature = "metadata")]
 pub use functions::*;
+
+#[cfg(feature = "metadata")]
+pub mod matrix {
+    include!("src/matrix/mod.rs");
+}

docs/numerical-workflows.md

@@ -0,0 +1,95 @@
+# Numerical workflows
+
+[Back to the README](../README.md)
+
+## Matrix and vector conventions
+
+Matrices use ordinary Rhai arrays of rows. Constructors make orientation explicit
+and validate numeric values while preserving their INT/FLOAT types:
+
+```typescript
+let values = [1, 2, 3];        // plain Rhai list
+let c = col(values);           // N by 1: [[1], [2], [3]]
+let r = row(values);           // 1 by N: [[1, 2, 3]]
+let A = mat([[1, 2], [3, 4]]); // rectangular numeric matrix
+```
+
+`row` and `col` also convert between vector orientations. Constructors reject empty,
+ragged, or nonnumeric inputs. The returned arrays remain editable; matrix operations
+check their inputs again. General matrix arithmetic does not implicitly broadcast
+or turn a flat list into a row or column.
+
+Use `mtimes` for matrix multiplication and `dot` for a real scalar inner product:
+
+```typescript
+let A = mat([[1, 2], [3, 4]]);
+let x = col([5, 6]);
+let prediction = mtimes(A, x); // [[17.0], [39.0]]
+let energy = dot(x, x);        // 61.0; also accepts lists or row vectors
+let At = transpose(A);
+let augmented = horzcat(A, x);
+let extended = vertcat(A, row([7, 8]));
+```
+
+`dot` accepts equal-length, nonempty vectors in any combination of orientations and
+returns FLOAT. It does not implement MATLAB's matrix/axis overloads or complex
+conjugation. `mtimes` requires matching inner dimensions and returns a matrix,
+including a 1 by 1 matrix for a row times a column.
+
+Statistics, moving/cumulative operations, differences, interpolation, and trapezoidal
+integration accept numeric lists, rows, and columns. Mixed INT/FLOAT values are
+supported. Scalar statistics return scalars; sequence operations return flat lists,
+so use `col` or `row` when feeding those results back into matrix operations.
+Inputs retain their original shape:
+
+```typescript
+let samples = col([1, 2.0, 3]);
+let average = mean(samples);         // 2.0
+let smoothed = movmean(samples, 3);  // [1.5, 2.0, 2.5]
+let area = trapz(row([0, 1, 2]), samples); // 4.0
+```
+
+Empty numeric samples produce a script error where a value is required. The
+flat-list identities `sum([]) == 0`, `prod([]) == 1`, `diff([]) == []`, and
+`unique([]) == []` remain available.
+
+## Regression and predictions
+
+`regress(X, y)` treats rows as observations and fits an intercept automatically.
+Do not add a column of ones. The returned `parameters`, `pvalues`, and
+`standard_errors` correspond to predictor columns in order. The fitted `intercept`
+is returned separately:
+
+```typescript
+let X = col([0, 1, 2]);
+let fit = regress(X, col([1.1, 2.8, 5.1]));
+let linear_part = mtimes(X, col(fit.parameters));
+let first_prediction = fit.intercept + linear_part[0][0];
+```
+
+Earlier releases omitted the fitted intercept from the result. Existing result
+fields are retained; use the new `intercept` field when calculating predictions.
+
+## CSV calibration example
+
+Run from the repository root:
+
+```bash
+cargo run --example regression_workflow
+```
+
+This loads the bundled calibration CSV, constructs two predictor columns, fits a
+linear model, predicts responses, and summarizes residuals. The sample has an
+intercept of 1, slopes of 2 and 0.5, and an RMSE of approximately 0.1414.
+It uses local data and requires the `io` and `nalgebra` features (both on by default).
+Rust hosts that already have data can pass an `observations` array to the same
+[script](../examples/regression_workflow.rhai) without enabling `io`.
+
+## Development
+
+```bash
+cargo fmt --all -- --check
+cargo test --no-default-features
+cargo test --no-default-features --features rand,nalgebra
+cargo test --all-features
+```

examples/data/calibration.csv

@@ -0,0 +1,7 @@
+temperature,load,response
+0,0,1.1
+1,1,3.3
+2,0,5.1
+3,1,7.6
+4,0,8.8
+5,1,11.6

examples/download_and_regress.rhai

@@ -5,8 +5,8 @@ let x = read_matrix(url).transpose();
 // Massage data
 let L = x.len;
 let y = x.drain(|v, i| i == (L-1));
-let x = ones(1, size(x)[1]) + x;
+// regress fits the intercept automatically; no column of ones is needed.
 
 // Do regression and report
 let b = regress(x.transpose(), y.transpose());
-print(b);
+print(b);

examples/matrix_inversion.rhai

@@ -0,0 +1,4 @@
+let m = [[1, 2], [3, 4]];
+let inv_m = inv(m);
+print(inv_m);
+inv_m

examples/matrix_inversion.rs

@@ -0,0 +1,21 @@
+//! Demonstrates computing the inverse of a matrix using rhai-sci.
+
+fn main() {
+    #[cfg(feature = "nalgebra")]
+    {
+        use rhai::{packages::Package, Engine};
+        use rhai_sci::SciPackage;
+
+        // Create a new Rhai engine
+        let mut engine = Engine::new();
+
+        // Add the rhai-sci package to the engine
+        engine.register_global_module(SciPackage::new().as_shared_module());
+
+        // Run the script that inverts a matrix
+        let result = engine
+            .run_file("examples/matrix_inversion.rhai".into())
+            .expect("script should run");
+        println!("{:?}", result);
+    }
+}