v1.0.0

LLMProxyAS v1.0.0

First public release of LLMProxyAS — an OpenAI-compatible LLM reverse proxy written in Go.

Point any OpenAI SDK or HTTP client at LLMProxyAS. The proxy validates models against an allow-list, injects upstream credentials when needed, and forwards traffic to any OpenAI-compatible provider (OpenAI, Azure OpenAI, Groq, Together, DeepSeek, vLLM, Ollama, and more).

Repository: https://github.com/dev2k6/LLMProxyAS

---

Highlights

• OpenAI-compatible API surface under /v1
• SSE streaming (stream=true) with idle / max stream timeouts
• Model allow-list (allowed_models.json)
• Config-driven upstream (base_url + api_key) — switch providers without code changes
• Server-side API key injection so clients can call the proxy without a provider key
• Tools / function calling and vision (request body forwarded as-is)
• Panic recovery, body size limits, transport timeouts, graceful shutdown
• Docker multi-stage image + Compose stack
• Prebuilt binaries for Windows, Linux, and macOS (amd64 + arm64)

---

What's included

API

Method	Path	Notes
GET	/health	Liveness
GET	/v1/models	Allow-listed models only
GET	/v1/models/{id}	Single model or 404
POST	/v1/chat/completions	Chat, tools, vision, stream
POST	/v1/completions	Legacy completions
POST	/v1/embeddings	Embeddings
POST	/v1/images/*, /v1/audio/*, /v1/moderations	Passthrough

Default listen port: 20182
Client base URL: http://<host>:20182/v1

Configuration

• configs/config.json — upstream URL, API key, timeouts, port, log level
• configs/allowed_models.json — models clients may use
• CLI: -config, -models

Auth model

Hop	Behavior
Client → LLMProxyAS	No proxy API key required
LLMProxyAS → upstream	Forwards client Authorization if present; otherwise uses api_key from config

Production: the proxy does not authenticate callers. Bind to a private network or put a gateway in front.

Resilience

• Panic recovery (OpenAI-shaped JSON errors when possible)
• Request body size limit (default 32 MiB)
• Dial / TLS / response-header timeouts
• Non-stream client timeout
• Stream idle timeout + stream max duration
• Client disconnect cancels upstream
• Graceful shutdown on SIGINT / SIGTERM

---

Downloads

Attach or link the binaries from bin/ (or upload the files below to this Release):

Asset	Platform
LLMProxyAS-windows-amd64.exe	Windows x64
LLMProxyAS-windows-arm64.exe	Windows ARM64
LLMProxyAS-linux-amd64	Linux x64
LLMProxyAS-linux-arm64	Linux ARM64
LLMProxyAS-darwin-amd64	macOS Intel
LLMProxyAS-darwin-arm64	macOS Apple Silicon

Also include (recommended):

• configs/config.example.json
• configs/allowed_models.json
• bin/README.txt (run instructions)

Raw download examples (after tag v1.0.0 is pushed):

https://github.com/dev2k6/LLMProxyAS/releases/download/v1.0.0/LLMProxyAS-windows-amd64.exe
https://github.com/dev2k6/LLMProxyAS/releases/download/v1.0.0/LLMProxyAS-linux-amd64
https://github.com/dev2k6/LLMProxyAS/releases/download/v1.0.0/LLMProxyAS-darwin-arm64

───

Quick start

Binary

# 1. Download the binary for your OS
# 2. Place configs next to it (or pass -config / -models)
# 3. Edit api_key + base_url
# 4. Run (from the folder that contains configs/)

# Windows
.\LLMProxyAS-windows-amd64.exe

# Linux / macOS
chmod +x ./LLMProxyAS-linux-amd64
./LLMProxyAS-linux-amd64

Health check:

curl http://localhost:20182/health

From source

git clone https://github.com/dev2k6/LLMProxyAS.git
cd LLMProxyAS
cp configs/config.example.json configs/config.json
# edit configs/config.json
go run ./cmd/server

Docker

# edit configs/config.json first
docker compose up -d --build
curl http://localhost:20182/health

Client (Python)

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:20182/v1",
    api_key="not-needed",  # use config api_key on the server
)

print(client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}],
).choices[0].message.content)

───

Requirements

• Go 1.22+ (build from source), or a prebuilt binary / Docker
• A valid upstream OpenAI-compatible endpoint and API key (if the provider requires one)

───

Documentation

• README (https://github.com/dev2k6/LLMProxyAS/blob/v1.0.0/README.md)
• SECURITY (https://github.com/dev2k6/LLMProxyAS/blob/v1.0.0/SECURITY.md)
• CONTRIBUTING (https://github.com/dev2k6/LLMProxyAS/blob/v1.0.0/CONTRIBUTING.md)
• CHANGELOG (https://github.com/dev2k6/LLMProxyAS/blob/v1.0.0/CHANGELOG.md)
• License (MIT) (https://github.com/dev2k6/LLMProxyAS/blob/v1.0.0/LICENSE)

───

Security notes

1. Do not commit real api_key values; use config.example.json as a template.
2. Restrict models via allowed_models.json.
3. Do not expose port 20182 publicly without authentication in front of the proxy.

───

Known limitations (v1.0.0)

• No built-in client authentication (API keys / mTLS)
• No rate limiting or multi-upstream failover yet
• No hot-reload of config / allow-list (restart required)
• log_body is reserved and not fully implemented

These are candidates for future releases.

───

License

MIT — see LICENSE (https://github.com/dev2k6/LLMProxyAS/blob/v1.0.0/LICENSE).

Full changelog: this is the initial release (v1.0.0).