The Nervous System for AI

Local-First Intelligence

Command Line Usage

The ogenius CLI provides an interactive chat REPL and an OpenAI-compatible API server, with automatic model downloading from Huggingface.

Quick Install

# Standard install (CPU only)
cargo install ogenius

# Install with Metal (macOS)
brew install cmake
cargo install ogenius --features metal

# Install with CUDA (cross-platform)
cargo install ogenius --features cuda

# Install with Vulkan (cross-platform)
cargo install ogenius --features vulkan

Basic Usage

# Download a model
ogenius download Qwen/Qwen2.5-1.5B-Instruct

# Start interactive chat
ogenius chat --model Qwen/Qwen2.5-1.5B-Instruct

# Run the API & Web Server (defaults to port 8080)
ogenius serve --model Qwen/Qwen2.5-1.5B-Instruct

Library Usage

Add Rusty-Genius to your project's Cargo.toml:

[dependencies]
rusty-genius = { version = "0.1.2", features = ["metal"] }

or use:

cargo add rusty-genius --features metal

Configuration

Control behavior via environment variables or manifest files:

Extension API

Extend the registry by creating manifest.toml in your home directory.

[[models]]
name = "my-local-model"
repo = "Org/Repo-GGUF"
filename = "model.Q4_K_M.gguf"
quantization = "Q4_K_M"

Try It Out

Run these commands in the terminal to test the crates locally:

1. Test Asset Downloader

cargo run -p facecrab --example downloader

2. Test Local Inference

Run the chat example with your preferred hardware acceleration:

cargo run -p rusty-genius --example basic_chat --features metal

cargo run -p rusty-genius --example basic_chat --features cuda

Usage

Initialize the orchestrator, download a model, and start chatting:

use rusty_genius::Orchestrator;
use rusty_genius::core::protocol::{AssetEvent, BrainstemInput, BrainstemOutput, InferenceEvent};
use futures::{StreamExt, sink::SinkExt, channel::mpsc};

#[async_std::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // 1. Core orchestration setup
    let mut genius = Orchestrator::new().await?;
    let (mut input, rx) = mpsc::channel(100);
    let (tx, mut output) = mpsc::channel(100);

    async_std::task::spawn(async move { 
        if let Err(e) = genius.run(rx, tx).await {
            eprintln!("Orchestrator error: {}", e);
        }
    });

    // 2. Select model (downloads and verifies automatically)
    input.send(BrainstemInput::LoadModel(
        "tiny-model".into()
    )).await?;

    // 3. Submit prompt
    input.send(BrainstemInput::Infer {
        prompt: "Once upon a time...".into(),
        config: Default::default(),
    }).await?;

    // 4. Stream results
    while let Some(msg) = output.next().await {
        match msg {
            BrainstemOutput::Asset(a) => match a {
                AssetEvent::Complete(path) => println!("Model ready at: {}", path),
                AssetEvent::Error(e) => eprintln!("Download error: {}", e),
                _ => {}
            },
            BrainstemOutput::Event(e) => match e {
                InferenceEvent::Content(c) => print!("{}", c),
                InferenceEvent::Complete => break,
                _ => {}
            },
            BrainstemOutput::Error(err) => {
                eprintln!("Error: {}", err);
                break;
            }
        }
    }

    Ok(())
}

Inside the Cranium

OS Requirements

Rusty-Genius leverages hardware acceleration via llama-cpp-2. Requirements vary by platform and GPU availability.