NSD: A Human-Friendly Data Format That Saves 3-8x Tokens

NSL TeamFeb 20, 20264 min readDeep Dive

What is NSD?

NSD (NSL Structured Data) is a human-friendly text data format built into NSL. It is designed as a superset of JSON's capabilities with significantly fewer tokens -- 3.5 to 8x reduction for structured data.

File extension: .nsd

Why Not JSON?

JSON is verbose. Every key needs quotes. Every object needs braces. Arrays of objects repeat key names on every row. NSD fixes all of this.

|------------|-------------|------------|----------|

| Config (key-value) | 100 | 28 | 3.6x |

| Table (array of objects) | 200 | 25 | 8.0x |

| Nested config | 150 | 42 | 3.6x |

| Mixed (config + table) | 300 | 60 | 5.0x |

What It Looks Like

Here is a real NSD document:

# Comments with #, //, or /* */
name: "My App"
version: 3
debug: false

[server]                  # TOML-style sections
host: "0.0.0.0"
port: 443

[server.pool]             # dotted paths
min: 5
max: 20

# Indented nested blocks
database:
  host: "localhost"
  port: 5432

# Tables -- this is the killer feature
table users(name, age, active):
  "Alice", 25, true
  "Bob", 30, false
  "Charlie", 28, true

# Anchors & references (dedup)
&timeout: 30
dev_timeout: *timeout
prod_timeout: *timeout

# Spread operator
&defaults: {retries: 3, debug: false}
config: {...*defaults, debug: true}

The same data in JSON would be roughly 4x longer.

Tables: The Killer Feature

NSD tables are first-class. Column names appear once, then rows are just comma-separated values:

table employees(name, department, salary):
  "Alice", "Engineering", 95000
  "Bob", "Design", 85000
  "Charlie", "Engineering", 92000

Compare to JSON:

[
  {"name": "Alice", "department": "Engineering", "salary": 95000},
  {"name": "Bob", "department": "Design", "salary": 85000},
  {"name": "Charlie", "department": "Engineering", "salary": 92000}
]

The NSD version is 8x fewer tokens. For 100-row tables, the savings are massive.

Tables support both positional and named rows:

table users(name, age, active):
  "Alice", 25, true                    # positional
  name: "Bob", age: 30, active: false  # named (order-independent)

Rich Types

NSD supports types that JSON cannot express:

# Dates and durations
created: @2024-01-15
updated: @2024-01-15T14:30:00Z
timeout: @5m30s
uptime: @1d12h

# Sets and tuples
tags: {|"fast", "safe", "simple"|}
point: (10, 20)

# Base64
icon: b64"iVBORw0KGgoAAAANSUhEU..."

# Raw strings
pattern: r"no\escapes\here"

# Number separators
population: 1_000_000
flags: 0xFF_FF

# Special values
ratio: Infinity
unknown: NaN

Table Query Functions

NSD includes SQL-like operations for working with table data:

let data = nsd.load("employees.nsd")

# Filter rows
let engineers = nsd.where(data, fn(row) => row.department == "Engineering")

# Sort by column
let by_salary = nsd.sort_by(data, "salary", true)  # descending

# Group and count
let dept_groups = nsd.group_by(data, "department")
let dept_counts = nsd.count_by(data, "department")

# Select specific columns
let names_only = nsd.select(data, ["name", "department"])

# Get column values
let all_names = nsd.column(data, "name")

NSD API

# Parse and encode
let obj = nsd.parse(text)
let text = nsd.encode(obj)
let compact = nsd.compact(obj)
let pretty = nsd.pretty(obj, 4)  # 4-space indent

# File I/O
nsd.save("config.nsd", data)
let loaded = nsd.load("config.nsd")

# Validation and inspection
let ok = nsd.valid(text)          # true if parseable
let kind = nsd.type(text)         # "document", "list", etc.

# CSV conversion
let from_csv = nsd.from_csv(csv_text)
let to_csv = nsd.to_csv(table_data)

# Binary variant (NSD-B)
nsd.save_binary("data.nsdb", value)
let loaded = nsd.load_binary("data.nsdb")

When to Use NSD vs JSON vs NCF

|--------|----------|------------|---------------|

| NSD | Config files, AI prompts, tables | Low (3-8x less) | Yes |

| NCF | Model weights, large data, RPC | Zero (binary) | No |

NSD is built into NSL -- no imports, no dependencies. Just nsd.parse() and nsd.encode().