SecNex/api-gateway
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

license: Add MIT license

Browse Source
This commit is contained in:
Björn Benouarets
2026-02-05 23:59:17 +01:00
parent 30adf0c701
commit 137be3e1e0
8 changed files with 611 additions and 295 deletions
Download Patch File Download Diff File Expand all files Collapse all files

179
.docs/architecture.md
View File

@@ -5,40 +5,40 @@
The SecNex API Gateway follows a modular architecture with clear separation of concerns. The system is built using the chi/v5 HTTP router and implements a middleware pipeline pattern.
```
┌─────────────────────────────────────────────────────────────┐
│ Client Request │
└─────────────────────────┬───────────────────────────────────┘
┌────────────────────────────────────────────────────────────┐
│ Client Request │
└─────────────────────────┬──────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Gateway (chi Router) │
┌────────────────────────────────────────────────────────────┐
│ Gateway (chi Router) │
│ ┌──────────────────────────────────────────────────────┐ │
│ │ Middleware Pipeline │ │
│ │ ┌────────────┐ ┌────────────┐ ┌──────────────┐ │ │
│ │ │ Request ID │→ │ Real IP │→ │ Logger │ │ │
│ │ └────────────┘ └────────────┘ └──────────────┘ │ │
│ │ Global Middleware Pipeline │ │
│ │ ┌────────────┐ ┌────────────┐ ┌──────────────┐ │ │
│ │ │ Request ID │→ │ Real IP │→ │ Logger │ │ │
│ │ └────────────┘ └────────────┘ └──────────────┘ │ │
│ └──────────────────────────────────────────────────────┘ │
└─────────────────────────┬───────────────────────────────────┘
└─────────────────────────┬──────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Route Handler │
┌────────────────────────────────────────────────────────────┐
│ Route Handler │
│ ┌──────────────────────────────────────────────────────┐ │
│ │ Per-Route Middleware Chain │ │
│ │ ┌────────────┐ ┌─────────┐ ┌──────────────┐ │ │
│ │ │ Strip │→ │ WAF │→ │ Auth │ │ │
│ │ │ Prefix │ │ Filter │ │ Middleware │ │ │
│ │ └────────────┘ └─────────┘ └──────────────┘ │ │
│ │ Per-Route Middleware Chain │ │
│ │ ┌────────────┐ ┌──────────────┐ │ │
│ │ │ Strip │→ │ Reverse │ │ │
│ │ │ Prefix │ │ Proxy │ │ │
│ │ └────────────┘ └──────────────┘ │ │
│ └──────────────────────────────────────────────────────┘ │
└─────────────────────────┬───────────────────────────────────┘
└─────────────────────────┬──────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Reverse Proxy (httputil.ReverseProxy) │
│ │ │
│ ▼ │
│ Backend Service │
└─────────────────────────────────────────────────────────────┘
┌────────────────────────────────────────────────────────────┐
│ Reverse Proxy (httputil.ReverseProxy) │
│ │ │
│ ▼ │
│ Backend Service │
└────────────────────────────────────────────────────────────┘
```
## Component Architecture
@@ -51,18 +51,17 @@ app/
├── config/ # Configuration management
│ ├── config.go # Config interface
│ ├── file.go # File-based config loader
│ └── types.go # Configuration type definitions
│ └── database.go # Database configuration
├── server/ # Core server components
│ ├── gateway.go # Main gateway server
│ ├── routes.go # Route handler creation
│ └── proxy.go # Host-based virtual hosting
│ ├── routes.go # Route registration
│ ├── api.go # API definitions
│ ├── host.go # Host definitions
│ └── target.go # Target (backend) definitions
├── middlewares/ # HTTP middleware
│ ├── auth.go # Authentication middleware
│ └── waf.go # Web Application Firewall
├── handlers/ # Request handlers
│ └── route.go # Route handlers (placeholder)
└── res/ # HTTP responses
└── error.go # Error response utilities
│ ├── host.go # Host logging middleware
│ └── logger.go # Structured logging middleware
└── utils/ # Utility functions
```
### Core Components
@@ -71,25 +70,39 @@ app/
The Gateway is the main server component that:
- Initializes the chi router
- Applies global middleware (request_id, real_ip, logger)
- Applies global middleware (request_id, real_ip, logger, host)
- Configures proxy directors for each API
- Registers route handlers
- Starts the HTTP server
#### Routes (`server/routes.go`)
The Routes component creates handlers for each configured route:
1. Finds the matching API backend target
2. Creates a reverse proxy using `httputil.NewSingleHostReverseProxy`
3. Optionally strips prefix from the request path
4. Applies middleware chain: WAF → Auth
The Routes component handles:
- Creating route handlers from configuration
- Applying strip prefix middleware
- Registering routes with chi router (method-agnostic)
- Connecting routes to API backends
#### Configuration (`config/`)
#### API (`server/api.go`)
Configuration is loaded from YAML and provides:
- Gateway settings (host, port, features)
- Route definitions with security policies
- API backend targets
- Proxy configurations for virtual hosting
API definitions that:
- Link hosts to backend targets
- Implement `http.Handler` interface for proxying
- Delegate requests to the reverse proxy
#### Hosts (`server/host.go`)
Host definitions for:
- Virtual hosting support
- Domain-based routing
- Host header validation
#### Targets (`server/target.go`)
Target (backend) definitions that:
- Store backend service URLs
- Create `httputil.ReverseProxy` instances
- Handle proxy configuration
## Middleware Chain
@@ -100,56 +113,56 @@ Applied to all requests via chi middleware:
| Feature | Description |
|---------|-------------|
| `request_id` | Adds unique request ID to context |
| `real_ip` | Determines the real client IP |
| `logger` | Logs HTTP requests |
| `real_ip` | Determines the real client IP from headers |
| `logger` | Logs HTTP requests with structured JSON output |
| `host` | Logs the host header for each request |
### Per-Route Middleware
Applied in order to each route handler:
Applied to each route handler:
1. **StripPrefix** - Removes specified prefix from request path before proxying
2. **WAF** - Filters requests based on HTTP method
3. **Auth** - Validates authentication credentials
## Request Flow
1. **Client Request** → Gateway receives HTTP request
2. **Global Middleware** → Request ID, Real IP, Logging applied
3. **Route Matching** → Chi matches route pattern (e.g., `/api/v1/dev/*`)
4. **Per-Route Middleware** → StripPrefix → WAF → Auth
2. **Global Middleware** → Request ID, Real IP, Host logging, Logger applied
3. **Route Matching** → Chi matches route pattern (e.g., `/api/v1/*`)
4. **Per-Route Middleware** → StripPrefix (if enabled)
5. **Reverse Proxy** → Request forwarded to backend API
6. **Response** → Backend response returned to client
## Authentication Flow
The authentication middleware supports path-based filtering:
```
┌─────────────────────┐
│ Include Set? │
└──────────┬──────────┘
│
┌─────┴─────┐
│ Yes │ No
▼ ▼
┌─────────┐ ┌─────────────┐
│ Match │ │ Exclude │
│ Include?│ │ Set? │
└────┬────┘ └──────┬──────┘
│ │
┌──┴──┐ ┌──┴──┐
│Yes │No │Yes │No
▼ ▼ ▼ ▼
┌────┐┌────┐ ┌────┐┌────┐
│Auth││Skip│ │Skip││Auth│
└────┘└────┘ └────┘└────┘
```
## Configuration Flow
1. Load `gateway.yaml` via `config.NewFileConfig()`
1. Load `gateway.yaml` via `config.NewFile()`
2. Parse YAML into `Configuration` struct
3. Create Gateway with configuration
4. Create Routes from route definitions and API targets
5. Register handlers with chi router
6. Start HTTP server
3. Create Hosts from configuration
4. Create Targets from configuration
5. Create APIs (linking Hosts to Targets)
6. Create Routes (linking Routes to APIs)
7. Initialize Gateway with all components
8. Configure proxy directors
9. Register routes with chi router
10. Start HTTP server
## Logging
The gateway uses structured JSON logging via `masterlog`:
- **HTTP Request Logging** - method, path, status, duration, host, IP
- **Gateway Events** - startup, route registration, proxy configuration
- **Sensitive Field Pseudonymization** - user_id, email, ip fields are pseudonymized
Example log output:
```json
{
"level": "info",
"msg": "HTTP Request",
"method": "GET",
"path": "/api/v1/users",
"status": 200,
"duration": "45ms",
"host": "localhost:8080",
"ip": "127.0.0.1:52342"
}
```

233
.docs/configuration.md
View File

@@ -12,33 +12,30 @@ gateway:
- request_id
- real_ip
- logger
- host
proxies:
- id: "proxy-id"
host: "example.com"
target: "http://backend:3000"
hosts:
- id: "host-001"
name: "localhost"
domain: "localhost:8080"
targets:
- id: "target-001"
name: "httpbin"
url: "https://httpbin.org"
apis:
- id: "api-id"
target: "https://api.example.com"
- id: "api-001"
host: "host-001"
target: "target-001"
routes:
- id: "route-id"
- id: "route-001"
api: "api-001"
path: "/api/v1/*"
strip_prefix:
enabled: true
prefix: "/api/v1"
security:
auth:
enabled: true
type: "api_key"
header: "X-Api-Key"
path:
include: []
exclude: []
waf:
enabled: true
methods: ["GET", "POST"]
```
## Sections
@@ -61,37 +58,49 @@ Available global features:
|---------|-------------|
| `request_id` | Adds unique request ID to each request |
| `real_ip` | Determines real client IP from headers |
| `logger` | Logs all HTTP requests |
| `logger` | Logs all HTTP requests with structured JSON |
| `host` | Logs the host header for each request |
### Proxies
### Hosts
Virtual hosting configuration for host-based routing.
Virtual hosting configuration for domain-based routing.
| Field | Type | Description |
|-------|------|-------------|
| `id` | string | Unique proxy identifier |
| `host` | string | Domain/host name to match |
| `target` | string | Backend URL to proxy to |
| `id` | string | Unique host identifier (referenced by APIs) |
| `name` | string | Human-readable name |
| `domain` | string | Domain/host name to match |
### Targets
Backend service definitions referenced by APIs.
| Field | Type | Description |
|-------|------|-------------|
| `id` | string | Unique target identifier (referenced by APIs) |
| `name` | string | Human-readable name |
| `url` | string | Backend URL to proxy to |
### APIs
Backend service definitions referenced by routes.
Links hosts to backend targets.
| Field | Type | Description |
|-------|------|-------------|
| `id` | string | Unique API identifier (referenced by routes) |
| `target` | string | Backend URL |
| `host` | string | Host ID to use |
| `target` | string | Target ID to proxy to |
### Routes
Route definitions with security policies.
Route definitions with path patterns and prefix stripping.
| Field | Type | Description |
|-------|------|-------------|
| `id` | string | Unique route identifier (must match API ID) |
| `id` | string | Unique route identifier |
| `api` | string | API ID to use for this route |
| `path` | string | Chi route pattern (e.g., `/api/v1/*`) |
| `strip_prefix` | object | Prefix stripping configuration |
| `security` | object | Security policies (auth, WAF) |
#### Strip Prefix
@@ -100,91 +109,112 @@ Route definitions with security policies.
| `enabled` | boolean | Enable prefix stripping |
| `prefix` | string | Prefix to remove from path |
#### Security
##### Authentication
| Field | Type | Description |
|-------|------|-------------|
| `enabled` | boolean | Enable authentication |
| `type` | string | Auth type (`api_key`, `session`, etc.) |
| `header` | string | Header name to validate |
| `path` | object | Path-based filtering |
##### Auth Path Filtering
| Field | Type | Description |
|-------|------|-------------|
| `include` | array | Paths that require auth (empty = all) |
| `exclude` | array | Paths that skip auth |
**Include/Exclude Logic:**
- If `include` is set → only matching paths require auth
- If `include` is empty → all paths require auth except `exclude`
Wildcards (`*`) are supported in path patterns.
##### WAF (Web Application Firewall)
| Field | Type | Description |
|-------|------|-------------|
| `enabled` | boolean | Enable WAF |
| `methods` | array | Allowed HTTP methods (`["*"]` for all) |
## Example Configurations
### Public API (No Auth)
### Simple Proxy (No Prefix Stripping)
```yaml
gateway:
host: "0.0.0.0"
port: 8080
features:
- logger
hosts:
- id: "host-001"
name: "localhost"
domain: "localhost:8080"
targets:
- id: "target-001"
name: "backend"
url: "https://api.example.com"
apis:
- id: "api-001"
host: "host-001"
target: "target-001"
routes:
- id: "public-api"
path: "/public/*"
strip_prefix:
enabled: true
prefix: "/public"
security:
auth:
enabled: false
waf:
enabled: true
methods: ["GET", "POST"]
- id: "route-001"
api: "api-001"
path: "/api/*"
```
### Protected API with API Key
**Request flow:**
- Client requests: `/api/users/123`
- Backend receives: `/api/users/123`
### Prefix Stripping
```yaml
routes:
- id: "protected-api"
- id: "route-001"
api: "api-001"
path: "/api/v1/*"
strip_prefix:
enabled: true
prefix: "/api/v1"
security:
auth:
enabled: true
type: "api_key"
header: "X-Api-Key"
waf:
enabled: true
methods: ["*"]
```
### Mixed Auth (Path-based)
**Request flow:**
- Client requests: `/api/v1/users/123`
- Gateway strips: `/api/v1`
- Backend receives: `/users/123`
### Multiple Routes
```yaml
routes:
- id: "mixed-api"
- id: "public-route"
api: "api-001"
path: "/public/*"
strip_prefix:
enabled: true
prefix: "/public"
- id: "api-route"
api: "api-001"
path: "/api/v1/*"
strip_prefix:
enabled: true
prefix: "/api/v1"
```
### Multiple Backends
```yaml
hosts:
- id: "host-001"
name: "api-host"
domain: "api.example.com"
- id: "host-002"
name: "admin-host"
domain: "admin.example.com"
targets:
- id: "target-001"
name: "api-backend"
url: "https://api-backend.internal"
- id: "target-002"
name: "admin-backend"
url: "https://admin-backend.internal"
apis:
- id: "api-001"
host: "host-001"
target: "target-001"
- id: "api-002"
host: "host-002"
target: "target-002"
routes:
- id: "route-001"
api: "api-001"
path: "/api/*"
security:
auth:
enabled: true
header: "Authorization"
path:
include: ["/api/admin/*", "/api/users/*/profile"]
exclude: ["/api/health", "/api/public/*"]
waf:
enabled: true
methods: ["*"]
- id: "route-002"
api: "api-002"
path: "/admin/*"
```
## Configuration Loading
@@ -192,7 +222,7 @@ routes:
The gateway loads configuration from a file path relative to the binary:
```go
cfg, err := config.NewFileConfig("../gateway.yaml")
cfg, err := config.NewFile("../gateway.yaml")
```
For Docker deployments, mount the config file:
@@ -201,3 +231,16 @@ For Docker deployments, mount the config file:
volumes:
- ./gateway.yaml:/app/gateway.yaml:ro
```
## Chi Route Patterns
The gateway uses chi/v5 routing patterns. Common patterns:
| Pattern | Matches | Example |
|---------|---------|---------|
| `/api/*` | `/api/` and any subpath | `/api/users`, `/api/users/123` |
| `/api/v1/*` | `/api/v1/` and any subpath | `/api/v1/users` |
| `/users/{id}` | `/users/` with any value | `/users/123` |
| `/files/*` | `/files/` and any subpath | `/files/doc.pdf` |
Note: `/*` matches zero or more path segments.

12
.docs/index.md
View File

@@ -4,7 +4,7 @@ Welcome to the official documentation for the SecNex API Gateway.
## Overview
SecNex API Gateway is a high-performance, configurable API gateway built in Go. It provides reverse proxy capabilities with built-in security features including authentication and web application firewall (WAF) functionality.
SecNex API Gateway is a high-performance, configurable API gateway built in Go. It provides reverse proxy capabilities with path-based routing, prefix stripping, and structured logging.
## Quick Start
@@ -28,11 +28,11 @@ docker compose up -d
## Features
- **Reverse Proxy** Route requests to backend services
- **Authentication** Header-based authentication (API key, session tokens)
- **WAF** Web Application Firewall with method filtering and IP-based access control
- **Path-based Routing** Configurable route patterns with prefix stripping
- **Middleware Pipeline** Extensible middleware chain for custom logic
- **Logging** Structured logging with sensitive field pseudonymization
- **Path-based Routing** Configurable route patterns with chi/v5
- **Prefix Stripping** Remove path prefixes before proxying
- **Host-based Routing** Virtual hosting support
- **Middleware Pipeline** Extensible middleware chain (request ID, real IP, logging)
- **Logging** Structured JSON logging with sensitive field pseudonymization
## Health Check

196
.docs/usage.md
View File

@@ -47,14 +47,13 @@ Response: `OK`
After configuring routes in `gateway.yaml`, make requests to the gateway:
```bash
# Example: Request to a route configured at /api/v1/dev/*
curl http://localhost:8080/api/v1/dev/users
# Example: Request to a route configured at /api/v1/*
curl http://localhost:8080/api/v1/users
# With API key authentication
curl -H "X-Api-Key: your-key-here" http://localhost:8080/api/v1/dev/data
# With session authentication
curl -H "Authorization: Bearer token" http://localhost:8080/api/v1/protected
# All HTTP methods are supported
curl -X POST http://localhost:8080/api/v1/data
curl -X PUT http://localhost:8080/api/v1/data/123
curl -X DELETE http://localhost:8080/api/v1/data/123
```
## Route Examples
@@ -65,6 +64,7 @@ curl -H "Authorization: Bearer token" http://localhost:8080/api/v1/protected
```yaml
routes:
- id: "api"
api: "api-001"
path: "/api/v1/*"
strip_prefix:
enabled: true
@@ -76,65 +76,57 @@ routes:
- Gateway strips: `/api/v1`
- Backend receives: `/users/123`
### Authentication Example
### Multiple Routes Example
**Configuration:**
```yaml
security:
auth:
enabled: true
type: "api_key"
header: "X-Api-Key"
routes:
- id: "public-route"
api: "api-001"
path: "/public/*"
strip_prefix:
enabled: true
prefix: "/public"
- id: "api-route"
api: "api-001"
path: "/api/v1/*"
strip_prefix:
enabled: true
prefix: "/api/v1"
```
**Valid request:**
**Requests:**
```bash
curl -H "X-Api-Key: secret123" http://localhost:8080/api/v1/data
```
# Public route - backend receives /status
curl http://localhost:8080/public/status
**Invalid request (missing header):**
```bash
curl http://localhost:8080/api/v1/data
# Returns: 401 Unauthorized
```
### WAF Method Filtering Example
**Configuration:**
```yaml
security:
waf:
enabled: true
methods: ["GET", "POST"]
```
**Allowed:**
```bash
curl -X GET http://localhost:8080/api/v1/data
curl -X POST http://localhost:8080/api/v1/data
```
**Blocked:**
```bash
curl -X DELETE http://localhost:8080/api/v1/data
# Returns: 403 Forbidden
# API route - backend receives /users
curl http://localhost:8080/api/v1/users
```
## Logging
The gateway uses structured logging via `masterlog`. Logs include:
The gateway uses structured JSON logging via `masterlog`. All HTTP requests are logged with:
- Request ID (if enabled)
- Client IP (if real_ip enabled)
- Request method and path
- Response status
- Response status code
- Request duration
- Host header
**Example log output:**
```json
{
"level": "info",
"msg": "Registering route",
"path": "/api/v1/dev/*"
"msg": "HTTP Request",
"method": "GET",
"path": "/api/v1/users",
"status": 200,
"duration": "45ms",
"host": "localhost:8080",
"ip": "127.0.0.1:52342"
}
```
@@ -147,30 +139,116 @@ The gateway uses structured logging via `masterlog`. Logs include:
lsof -i :8080
```
2. Verify configuration file exists and is valid YAML
2. Verify configuration file exists and is valid YAML:
```bash
cat ../gateway.yaml
```
3. Check logs for detailed error messages
### 401 Unauthorized
### 404 Not Found
- Verify auth header is included
- Check header name matches configuration
- Verify path is not excluded from auth
### 403 Forbidden
- Check WAF method configuration
- Verify HTTP method is allowed
- Verify the route path matches your request
- Check chi route pattern syntax (use `/*` for wildcards)
- Ensure the route is registered (check startup logs)
### Backend connection errors
- Verify API target URL is correct
- Check backend service is running
- Verify network connectivity
- Verify target URL is correct in configuration
- Check backend service is running and accessible
- Verify network connectivity from gateway to backend
- Check DNS resolution if using domain names
### Route not matching
- Ensure request path matches the route pattern exactly
- Remember that `/*` matches zero or more path segments
- Check that multiple routes don't have conflicting patterns
## Performance Considerations
- The gateway uses Go's `httputil.ReverseProxy` for efficient proxying
- HTTP/2 is supported when both client and backend support it
- Keep-alive connections are reused by default
- Consider connection pooling for high-traffic scenarios
- Monitor memory usage with high request volumes
## Common Patterns
### API Versioning
```yaml
routes:
- id: "v1-api"
api: "api-001"
path: "/api/v1/*"
strip_prefix:
enabled: true
prefix: "/api/v1"
- id: "v2-api"
api: "api-001"
path: "/api/v2/*"
strip_prefix:
enabled: true
prefix: "/api/v2"
```
### Microservices Routing
```yaml
targets:
- id: "user-service"
name: "User Service"
url: "https://user-service.internal"
- id: "order-service"
name: "Order Service"
url: "https://order-service.internal"
apis:
- id: "user-api"
host: "host-001"
target: "user-service"
- id: "order-api"
host: "host-001"
target: "order-service"
routes:
- id: "users-route"
api: "user-api"
path: "/users/*"
strip_prefix:
enabled: true
prefix: "/users"
- id: "orders-route"
api: "order-api"
path: "/orders/*"
strip_prefix:
enabled: true
prefix: "/orders"
```
### External API Proxy
```yaml
targets:
- id: "external-api"
name: "External API"
url: "https://api.external.com"
apis:
- id: "external"
host: "host-001"
target: "external-api"
routes:
- id: "external-proxy"
api: "external"
path: "/proxy/external/*"
strip_prefix:
enabled: true
prefix: "/proxy/external"
```
Request to `/proxy/external/users` becomes `/users` on the external API.