Laya: Open Non-Autoregressive Multilingual System 1 Decision Engine Released

Overview

Laya is an open-source, non-autoregressive System 1 decision engine designed to deliver typed choices, scores, and yes/no decisions across more than 100 languages in a single forward pass. Operating on models within the ModernBERT family (ranging from approximately 150M to 421M parameters), the system achieves decision times around 33 ms on standard hardware such as a T4 GPU. It features an intelligent router that detects scripts and languages to select the appropriate checkpoint per request.

Key Features and Extensions

• Multilingual and Long-Document Support: The multilingual checkpoint reads up to 8,192 tokens when configured with `max_len=8192`.
• Vega Integration: Optional Vega extensions (`vegaml=True`) combine a reader with a physics engine to output conformal prediction sets, abstain flags, out-of-distribution flags, and routing decisions without guessing thresholds.
• Fine-Tuning and Calibration: Users can fit and calibrate lightweight decision heads on custom labeled data using reinforcement learning against strictly proper scoring rules (RLCD) without altering the underlying base reader.
• Flexible Deployment: Available via PyPI, GitHub source checkouts, and `uv`, supporting Python 3.10 or newer alongside optional extras for serving, MCP, LangChain, LlamaIndex, CrewAI, ONNX Runtime, and audio/vision inputs.

Practical Implications and Verification

Deployments can leverage built-in command-line tools (`laya`), batch processing with length sorting for performance gains, and a self-contained FastAPI server example for local testing. Users should verify installation using environment-specific checks and consult the official documentation and repository for hardware-specific PyTorch builds and compatibility notes.