Overview
NSL ships with 40 standard library modules. No package manager needed -- everything is built in. Access modules with module.function() or use module.*.
Tensor and GPU
Create and manipulate multi-dimensional arrays. Matrix multiplication automatically uses GPU tensor cores for large matrices.
let a = tensor.random([128, 64])
let b = tensor.random([64, 256])
let c = tensor.matmul(a, b) # GPU-accelerated for dims >= 64
print(tensor.shape(c)) # [128, 256]
let t = tensor.from_list([[1,2],[3,4]])
print(tensor.sum(t)) # 10
print(tensor.mean(t)) # 2.5
let t2 = tensor.transpose(t)
let identity = tensor.identity(3) # 3x3 identity matrix
Key functions: create, zeros, ones, random, from_list, to_list, reshape, matmul, add, sub, mul, dot, transpose, sum, mean, max, min, scale, clone, batched_matmul.
Autograd (grad)
Tape-based reverse-mode automatic differentiation:
let x = grad.variable([1.0, 2.0, 3.0])
let w = grad.variable([0.5, 0.5, 0.5])
let y = grad.mul(x, w)
let loss = grad.sum(y)
grad.backward(loss)
print(grad.get_grad(w)) # [1.0, 2.0, 3.0]
Operations: add, sub, mul, matmul, relu, gelu, sigmoid, tanh, softmax, sum, mean, log, exp, cross_entropy, mse, layer_norm, linear, embedding, dropout, slice, cat, reshape, transpose.
GPU-accelerated: matmul (dims >= 64), relu, gelu, softmax (n >= 4096).
Optimizers (optim)
let opt = optim.adam([w1, w2, b1], 0.001) # Adam with AdamW weight decay
# or
let opt = optim.sgd([w1, w2], 0.01, 0.9) # SGD with momentum
optim.step(opt) # update parameters
optim.zero_grad(opt) # reset gradients
optim.set_lr(opt, 0.0005) # change learning rate
optim.clip_grad_norm([w1, w2], 1.0) # gradient clipping
Tokenizer
BPE, character-level, and word-level tokenizers:
let tok = tokenizer.create("bpe")
tokenizer.train(tok, ["Hello world", "Hello NSL", "world of code"], 256)
let ids = tokenizer.encode(tok, "Hello world")
let text = tokenizer.decode(tok, ids)
tokenizer.save(tok, "my_tokenizer.txt")
let loaded = tokenizer.load("my_tokenizer.txt")
Training Utilities (train)
# DataLoader with shuffle
let loader = train.dataloader(data, labels, 32, true)
while train.has_next(loader)
let batch = train.next_batch(loader)
# batch.data and batch.labels
# Learning rate schedulers
let lr = train.lr_cosine(0.001, step, total_steps, 0.0001)
let lr = train.lr_linear(0.001, step, warmup_steps, total_steps)
let lr = train.lr_step(0.001, step, 100, 0.1)
# Checkpointing
train.save_checkpoint("model.json", [w1, w2], opt, epoch)
let ckpt = train.load_checkpoint("model.json")
# Progress formatting
print(train.progress(epoch, batch, total, loss))
NSD (NSL Structured Data)
Human-friendly text format, 3-8x fewer tokens than JSON:
let config = nsd.parse("name: \"My App\"\nversion: 3\ndebug: false")
print(config.name) # "My App"
let text = nsd.encode({host: "localhost", port: 5432})
nsd.save("config.nsd", {host: "localhost", port: 5432})
let loaded = nsd.load("config.nsd")
# Table utilities
let users = [{name: "Alice", age: 25}, {name: "Bob", age: 30}]
let sorted_users = nsd.sort_by(users, "age")
let grouped = nsd.group_by(users, "name")
let filtered = nsd.where(users, fn(row) => row.age > 25)
Supports: sections, tables, anchors/references, dates, durations, base64, sets, tuples, CSV conversion, binary format.
NCF (Nexus Compact Format)
Binary serialization with LZ77+Huffman compression:
let bytes = ncf.encode({model: weights, config: params})
ncf.save("data.ncf", my_data)
let loaded = ncf.load("data.ncf")
# Indexed random access
ncf.save_indexed("big.ncf", {layer1: w1, layer2: w2, config: cfg})
let w1_only = ncf.load_field("big.ncf", "layer1")
# Signed integrity
let signed = ncf.encode_signed(data, "secret_key")
let valid = ncf.verify_signed(signed, "secret_key")
File I/O
let content = file.read("data.txt")
file.write("output.txt", "hello world")
let files = dir.list("./src")
let exists = path.exists("config.nsd")
HTTP / Networking
let resp = http.get("https://api.example.com/data")
let data = json.parse(resp.body)
let resp = http.post("https://api.example.com/submit", json.stringify(payload))
JSON, XML, YAML
let obj = json.parse('{"key": "value"}')
let text = json.stringify(obj)
let xml_data = xml.parse("<root><item>1</item></root>")
let yaml_data = yaml.parse("key: value")
Python Bridge (py)
Embed CPython and call any installed Python library:
if py.is_available()
py.exec("import numpy as np")
py.exec("arr = np.array([1.0, 2.0, 3.0, 4.0])")
let mean = py.eval("float(arr.mean())")
print(mean) # 2.5
let np = py.import("numpy")
let result = py.call(np, "sqrt", 16.0) # 4.0
py.free(np)
py.set_global("x", [10, 20, 30])
let total = py.eval("sum(x)") # 60
Types auto-convert between NSL and Python (int, float, string, bool, list, dict, null/None).
Other Modules
| Module | Purpose | Key Functions |
|--------|---------|---------------|
| math | Extended math | pi, e, sin, cos, atan2, log2 |
| string | String utilities | pad_left, pad_right, repeat, chars |
| crypto | Cryptography | sha256, hmac, aes_encrypt, bcrypt |
| regex | Pattern matching | match, find_all, replace, split |
| time | Date/time | now, format, parse, sleep |
| db | SQLite | open, query, execute, close |
| path | File paths | join, dirname, basename, ext |
| sys | System info | args, env, exit, platform |
| prediction | Bayesian | enable, record, evaluate, confidence |
| ai | ML utilities | embed, similarity, tokenize, analyze |
| audio | Audio | Recording and playback |
| pdf | PDF files | Read and write PDF |
| gui | UI toolkit | Windows, buttons, layouts |
| zip | Archives | Compress and decompress |
| email | Email | Send and receive |
| debug | Debugging | Breakpoints, inspection |
Memory Module (74 functions)
OS-level memory management with 8 layers:
# Virtual memory
let addr = memory.alloc(4096)
memory.write(addr, "hello world")
let data = memory.read(addr, 11)
memory.free(addr)
# Slab allocator (O(1) alloc/free)
let block = memory.slab_alloc(256)
memory.slab_free(block)
# CPU tensors with AVX-512 alignment
let t = memory.tensor_alloc(1024, 64)
memory.typed_poke(t, 0, 3.14, "float32")
let val = memory.typed_peek(t, 0, "float32")
# Data integrity
memory.zero(addr, size) # NIST SP 800-88
let h = memory.checksum(addr, size) # FNV-1a 64-bit
let ok = memory.verify(addr, size, h)
let e = memory.entropy(addr, size) # Shannon entropy
# GPU bridge
let gpu_ptr = memory.gpu_alloc(4096)
memory.gpu_upload(gpu_ptr, cpu_data, size)
memory.gpu_download(cpu_dest, gpu_ptr, size)
# Semantic memory (via PredictionEngine)
memory.tag(addr, "weights")
let related = memory.query_related("weights")
All 40 modules are available with use module.* or by calling module.function() directly.