LLM Gateway

Deploy your own LLM Gateway to unify multiple providers with DID authentication and usage-based pricing

With Nuwa LLM Gateway, you can unify multiple LLM providers (OpenAI, OpenRouter, LiteLLM) behind a single endpoint with DID authentication and usage-based pricing. The gateway provides provider-first routing, automatic usage tracking, and seamless payment integration.

NPM Package: @nuwa-ai/llm-gateway - Ready to use without building from source.

Before you start: obtain a Service Key for your service DID. See the Service Key guide and use the copied value as SERVICE_KEY.

Quick start

Using NPM Package (Recommended)

1. Install globally

   # Install the published package
   npm install -g @nuwa-ai/llm-gateway
   
   # Or use without installation
   npx @nuwa-ai/llm-gateway --help
   
2. Set environment variables

   # Required
   export SERVICE_KEY=0x...
   export OPENAI_API_KEY=sk-proj-...
   
   # Network configuration
   export ROOCH_NETWORK=test
   
   # Optional pricing configuration
   # export PRICING_MULTIPLIER=1.10  # Range 0~2, e.g. 1.10 = +10%
   # export PRICING_OVERRIDES='{"gpt-4":{"promptPerMTokUsd":25,"completionPerMTokUsd":50}}'
   

   Not sure where SERVICE_KEY comes from? Follow the Service Key guide to generate it via test-id.nuwa.dev.

3. Run the gateway

   # Start with debug logging
   llm-gateway --debug
   
   # Or specify custom port
   llm-gateway --port 3000 --debug
   
   # → serves provider-first routes like /openai/v1/chat/completions
   
4. Verify the gateway

   You can use the Nuwa Login Demo to verify your gateway:

   1. Open the demo and set gateway URL to http://localhost:8080
   2. Connect your wallet and authenticate with DID
   3. Send a chat request to test the integration

Build from Source

1. Clone and build

   For development or customization, you can build from source. Find the full implementation in the Nuwa Monorepo.

   git clone https://github.com/nuwa-protocol/nuwa.git
   cd nuwa/nuwa-services/llm-gateway
   pnpm install
   pnpm build
   
2. Set environment and run

   Set your environment variables (same as above) and run:

   export SERVICE_KEY=...
   export OPENAI_API_KEY=...
   
   pnpm run dev
   

Provider routing

The gateway uses provider-first routing for clear endpoint organization:

OpenAI

# Chat completions
POST /openai/v1/chat/completions

# Responses
POST /openai/v1/responses

OpenRouter

# Chat completions
POST /openrouter/api/v1/chat/completions

LiteLLM

# Chat completions
POST /litellm/chat/completions

Using PaymentChannelHttpClient

For programmatic access with automatic payment handling, use the PaymentChannelHttpClient from the Payment Kit:

Installation & Setup

npm install @nuwa-ai/payment-kit @nuwa-ai/identity-kit

import { PaymentChannelHttpClient } from '@nuwa-ai/payment-kit';
import { IdentityKit } from '@nuwa-ai/identity-kit';

// Initialize identity environment
const env = await IdentityKit.bootstrap({
  method: 'rooch',
  vdrOptions: {
    network: 'test',
  },
});

// Import your Service Key
await env.keyManager.importKeyFromString('0x...');
const signer = env.keyManager.getSigner();

// Create HTTP client
const client = new PaymentChannelHttpClient({
  baseUrl: 'http://localhost:8080',
  signer,
  chainConfig: { network: 'test' },
});

Making Requests

// Chat completion request
const result = await client.post('/openai/v1/chat/completions', {
  model: 'gpt-4',
  messages: [
    { role: 'user', content: 'Hello, how are you?' }
  ],
  stream: false
});

console.log('Response:', result.data);
console.log('Payment info:', result.payment);

// Streaming request
const handle = await client.requestWithPayment(
  'POST', 
  '/openai/v1/chat/completions',
  {
    body: JSON.stringify({
      model: 'gpt-4',
      messages: [{ role: 'user', content: 'Hello!' }],
      stream: true
    }),
    headers: { 'Content-Type': 'application/json' }
  }
);

// Get response immediately
const response = await handle.response;
const reader = response.body?.getReader();

// Payment resolves when stream completes
const payment = await handle.payment;
console.log('Stream payment:', payment);

Error Handling

try {
  const result = await client.post('/openai/v1/chat/completions', {
    model: 'gpt-4',
    messages: [{ role: 'user', content: 'Hello!' }]
  });
  
  if (result.payment) {
    console.log(`Paid ${result.payment.costUsd} picoUSD for request`);
  } else {
    console.log('Free request (no payment required)');
  }
} catch (error) {
  console.error('Request failed:', error);
  
  // Check if it's a payment-related error
  if (error.code === 'INSUFFICIENT_BALANCE') {
    console.log('Need to deposit more funds to payment channel');
  }
}

Pricing configuration

The gateway supports flexible pricing configuration:

• Built-in pricing: Default model pricing for OpenAI models
• Pricing overrides: Custom pricing via PRICING_OVERRIDES environment variable
• Global multiplier: Adjust all costs with PRICING_MULTIPLIER (range 0-2)

Usage tracking

• Automatic tracking: All requests are logged with token counts and USD costs
• Streaming support: Chat Completions automatically inject stream_options.include_usage=true
• Response API: Usage appears in response.completed SSE events
• Access logs: Structured JSON logs to stdout with request details

Common issues

• “SERVICE_KEY is required”: Generate and set your service key from the Service Key guide
• “At least one provider API key is required”: Configure at least one provider API key
• Port conflicts: Use --port <number> to specify a different port

Relevant Docs

Service Key

How to obtain and configure SERVICE_KEY

Payment Kit

PaymentChannelHttpClient for programmatic access

Service Pricing

Configure pricing for your LLM Gateway

Revenue

View balances, withdraw, and audit revenue history