Initial implementation of GitHub download proxy service
- Add flexible URL template configuration via environment variables - Implement automatic repository path parsing and extraction - Add 404 response handling for all non-200 status codes - Support both SecNex and GitHub URL formats through URL template - Include comprehensive documentation and Docker support - Add proper Go project structure with .gitignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
35
.gitignore
vendored
Normal file
35
.gitignore
vendored
Normal file
@@ -0,0 +1,35 @@
|
||||
# Binaries for programs and plugins
|
||||
*.exe
|
||||
*.exe~
|
||||
*.dll
|
||||
*.so
|
||||
*.dylib
|
||||
|
||||
# Test binary, built with `go test -c`
|
||||
*.test
|
||||
|
||||
# Output of the go coverage tool, specifically when used with LiteIDE
|
||||
*.out
|
||||
|
||||
# Dependency directories (remove the comment below to include it)
|
||||
# vendor/
|
||||
|
||||
# Go workspace file
|
||||
go.work
|
||||
|
||||
# Binary output
|
||||
gh-download-proxy
|
||||
|
||||
# IDE files
|
||||
.vscode/
|
||||
.idea/
|
||||
*.swp
|
||||
*.swo
|
||||
*~
|
||||
|
||||
# OS files
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
|
||||
# Logs
|
||||
*.log
|
||||
68
CLAUDE.md
Normal file
68
CLAUDE.md
Normal file
@@ -0,0 +1,68 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Project Overview
|
||||
|
||||
This is a Go project named `gh-download-proxy` which appears to be a GitHub download proxy service. The project is currently in initial setup phase with only a `go.mod` file present.
|
||||
|
||||
## Development Setup
|
||||
|
||||
### Go Environment
|
||||
- Go version: 1.25.3
|
||||
- Module name: `gh-download-proxy`
|
||||
|
||||
### Common Commands
|
||||
|
||||
#### Building the project
|
||||
```bash
|
||||
go build ./...
|
||||
```
|
||||
|
||||
#### Running tests
|
||||
```bash
|
||||
go test ./...
|
||||
```
|
||||
|
||||
#### Running a specific test
|
||||
```bash
|
||||
go test -run <TestName> ./...
|
||||
```
|
||||
|
||||
#### Managing dependencies
|
||||
```bash
|
||||
go mod tidy # Clean up dependencies
|
||||
go mod download # Download dependencies
|
||||
```
|
||||
|
||||
#### Running the application (once main.go exists)
|
||||
```bash
|
||||
go run main.go
|
||||
```
|
||||
|
||||
## Project Structure
|
||||
|
||||
Currently minimal structure:
|
||||
- `go.mod` - Go module definition
|
||||
|
||||
As the project develops, typical Go project structure would include:
|
||||
- `main.go` - Main application entry point
|
||||
- `cmd/` - Command-line applications
|
||||
- `internal/` - Internal application code
|
||||
- `pkg/` - Public library code
|
||||
- `go.sum` - Dependency checksums
|
||||
|
||||
## Architecture Notes
|
||||
|
||||
Since this is intended to be a GitHub download proxy, the architecture will likely involve:
|
||||
- HTTP server to handle download requests
|
||||
- GitHub API integration
|
||||
- File caching mechanisms
|
||||
- Proxy functionality to intercept and serve GitHub downloads
|
||||
|
||||
## Development Notes
|
||||
|
||||
- This is a new repository with no commit history yet
|
||||
- No Go source files exist yet - development is just beginning
|
||||
- Consider adding a `README.md` with project documentation
|
||||
- Consider adding `.gitignore` for Go-specific files
|
||||
17
Dockerfile
Normal file
17
Dockerfile
Normal file
@@ -0,0 +1,17 @@
|
||||
FROM golang:1.25.3-alpine3.22 AS builder
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY ./app/go.mod ./app/go.sum ./
|
||||
|
||||
RUN go mod download && go mod verify
|
||||
|
||||
COPY ./app ./.
|
||||
|
||||
RUN go build -o download .
|
||||
|
||||
FROM alpine:latest AS runner
|
||||
|
||||
COPY --from=builder /app/download /app/download
|
||||
|
||||
CMD ["/app/download"]
|
||||
140
README.md
Normal file
140
README.md
Normal file
@@ -0,0 +1,140 @@
|
||||
# GitHub Download Proxy
|
||||
|
||||
A Go-based proxy service that automatically reads repositories and formats paths based on a customizable URL template. Features flexible URL configuration through environment variables.
|
||||
|
||||
## Features
|
||||
|
||||
- Flexible URL template configuration via environment variables
|
||||
- Automatic path parsing and repository name extraction
|
||||
- Proper HTTP header forwarding and status code handling
|
||||
- Comprehensive logging with masterlog
|
||||
- Simple and clean implementation
|
||||
|
||||
## Configuration
|
||||
|
||||
The proxy is configured via environment variables:
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `URL` | `https://git.secnex.io/secnex/%s/raw/branch/main/bin/%s` | URL template with `%s` placeholders for repository and file path |
|
||||
| `PORT` | `:8080` | Port to listen on |
|
||||
|
||||
The `URL` template uses two `%s` placeholders:
|
||||
1. First `%s` - Repository name
|
||||
2. Second `%s` - File path within the repository
|
||||
|
||||
## Usage
|
||||
|
||||
### Starting the server
|
||||
|
||||
```bash
|
||||
# Default configuration (SecNex bin directory)
|
||||
go run ./app
|
||||
|
||||
# With custom port
|
||||
PORT=:9090 go run ./app
|
||||
|
||||
# For GitHub repositories
|
||||
URL="https://github.com/%s/refs/heads/main/%s" go run ./app
|
||||
|
||||
# For SecNex raw files (not bin directory)
|
||||
URL="https://git.secnex.io/secnex/%s/raw/branch/main/%s" go run ./app
|
||||
|
||||
# Custom Git server
|
||||
URL="https://custom.git.server.com/%s/raw/main/%s" go run ./app
|
||||
```
|
||||
|
||||
### Making requests
|
||||
|
||||
Request format: `GET /{repository}/{path/to/file}`
|
||||
|
||||
```bash
|
||||
# Get specific file
|
||||
curl http://localhost:8080/my-repo/path/to/file.txt
|
||||
|
||||
# Get file from repository root
|
||||
curl http://localhost:8080/my-repo/config.yaml
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
### SecNex Bin Directory (Default)
|
||||
```bash
|
||||
# Default configuration
|
||||
go run ./app
|
||||
|
||||
# Request file
|
||||
curl http://localhost:8080/my-app/release/my-app-linux
|
||||
```
|
||||
This proxies to: `https://git.secnex.io/secnex/my-app/raw/branch/main/bin/release/my-app-linux`
|
||||
|
||||
### GitHub Raw Files
|
||||
```bash
|
||||
# Configure for GitHub
|
||||
URL="https://github.com/%s/refs/heads/main/%s" go run ./app
|
||||
|
||||
# Request file
|
||||
curl http://localhost:8080/octocat/Hello-World/README.md
|
||||
```
|
||||
This proxies to: `https://github.com/octocat/Hello-World/refs/heads/main/README.md`
|
||||
|
||||
### Custom GitLab Instance
|
||||
```bash
|
||||
# Configure for custom GitLab
|
||||
URL="https://gitlab.company.com/%s/-/raw/main/%s" go run ./app
|
||||
|
||||
# Request file
|
||||
curl http://localhost:8080/project/repo/src/main.go
|
||||
```
|
||||
This proxies to: `https://gitlab.company.com/project/repo/-/raw/main/src/main.go`
|
||||
|
||||
### Different Branch
|
||||
```bash
|
||||
# Configure for develop branch
|
||||
URL="https://git.secnex.io/secnex/%s/raw/branch/develop/%s" go run ./app
|
||||
```
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
.
|
||||
├── app/
|
||||
│ └── main.go # Main application entry point
|
||||
├── README.md # This documentation
|
||||
├── CLAUDE.md # Development instructions
|
||||
├── go.mod # Go module definition
|
||||
├── go.sum # Dependency checksums
|
||||
├── Dockerfile # Container configuration
|
||||
└── docker-compose.yml # Development setup
|
||||
```
|
||||
|
||||
## Building
|
||||
|
||||
```bash
|
||||
# Build the application
|
||||
go build ./app
|
||||
|
||||
# Build for production
|
||||
go build -o gh-download-proxy ./app
|
||||
|
||||
# Run tests
|
||||
go test ./...
|
||||
```
|
||||
|
||||
## Dependencies
|
||||
|
||||
- Go 1.25.3+
|
||||
- git.secnex.io/secnex/masterlog v0.1.0
|
||||
|
||||
## Running with Docker
|
||||
|
||||
```bash
|
||||
# Build the Docker image
|
||||
docker build -t gh-download-proxy .
|
||||
|
||||
# Run with default configuration
|
||||
docker run -p 8080:8080 gh-download-proxy
|
||||
|
||||
# Run with custom URL
|
||||
docker run -p 8080:8080 -e URL="https://github.com/%s/refs/heads/main/%s" gh-download-proxy
|
||||
```
|
||||
5
app/go.mod
Normal file
5
app/go.mod
Normal file
@@ -0,0 +1,5 @@
|
||||
module gh-download-proxy
|
||||
|
||||
go 1.25.3
|
||||
|
||||
require git.secnex.io/secnex/masterlog v0.1.0
|
||||
2
app/go.sum
Normal file
2
app/go.sum
Normal file
@@ -0,0 +1,2 @@
|
||||
git.secnex.io/secnex/masterlog v0.1.0 h1:74j9CATpfeK0lxpWIQC9ag9083akwG8khi5BwLedD8E=
|
||||
git.secnex.io/secnex/masterlog v0.1.0/go.mod h1:OnDlwEzdkKMnqY+G5O9kHdhoJ6fH1llbVdXpgSc5SdM=
|
||||
150
app/main.go
Normal file
150
app/main.go
Normal file
@@ -0,0 +1,150 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"os"
|
||||
"path"
|
||||
"strings"
|
||||
|
||||
"git.secnex.io/secnex/masterlog"
|
||||
)
|
||||
|
||||
type Config struct {
|
||||
Port string
|
||||
Url string
|
||||
}
|
||||
|
||||
func main() {
|
||||
config := loadConfig()
|
||||
|
||||
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
|
||||
proxyHandler(w, r, config)
|
||||
})
|
||||
|
||||
masterlog.Info("Starting server", map[string]interface{}{
|
||||
"port": config.Port,
|
||||
"url": config.Url,
|
||||
})
|
||||
|
||||
if err := http.ListenAndServe(config.Port, nil); err != nil {
|
||||
masterlog.Error("Error starting server", map[string]interface{}{"error": err})
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
func loadConfig() *Config {
|
||||
config := &Config{
|
||||
Url: getEnv("URL", "https://git.secnex.io/secnex/%s/raw/branch/main/%s"),
|
||||
Port: getEnv("PORT", ":8080"),
|
||||
}
|
||||
|
||||
return config
|
||||
}
|
||||
|
||||
func getEnv(key, defaultValue string) string {
|
||||
if value := os.Getenv(key); value != "" {
|
||||
return value
|
||||
}
|
||||
return defaultValue
|
||||
}
|
||||
|
||||
func proxyHandler(w http.ResponseWriter, r *http.Request, config *Config) {
|
||||
// Only handle GET requests
|
||||
if r.Method != http.MethodGet {
|
||||
http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
|
||||
return
|
||||
}
|
||||
|
||||
// Extract repository and file path from URL
|
||||
// Expected format: /repositoryName/path/to/file
|
||||
cleanPath := path.Clean(r.URL.Path)
|
||||
if cleanPath == "/" || cleanPath == "" {
|
||||
http.Error(w, "Repository name required", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
pathParts := strings.Split(strings.TrimPrefix(cleanPath, "/"), "/")
|
||||
if len(pathParts) < 1 {
|
||||
http.Error(w, "Invalid path format", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
repository := pathParts[0]
|
||||
filePath := strings.Join(pathParts[1:], "/")
|
||||
|
||||
// Log the incoming request
|
||||
masterlog.Info("Incoming request", map[string]interface{}{
|
||||
"method": r.Method,
|
||||
"repository": repository,
|
||||
"file_path": filePath,
|
||||
"original_path": r.URL.Path,
|
||||
})
|
||||
|
||||
// Build target URL based on server type
|
||||
var targetURL string
|
||||
targetURL = fmt.Sprintf(config.Url, repository, filePath)
|
||||
|
||||
// Create HTTP client with timeout
|
||||
client := &http.Client{}
|
||||
req, err := http.NewRequestWithContext(r.Context(), http.MethodGet, targetURL, nil)
|
||||
if err != nil {
|
||||
masterlog.Error("Error creating request", map[string]interface{}{"error": err})
|
||||
http.Error(w, "Failed to create request", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
// Copy headers from original request
|
||||
for key, values := range r.Header {
|
||||
// Skip certain headers that shouldn't be forwarded
|
||||
if strings.ToLower(key) == "host" {
|
||||
continue
|
||||
}
|
||||
for _, value := range values {
|
||||
req.Header.Add(key, value)
|
||||
}
|
||||
}
|
||||
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
masterlog.Error("Error fetching target URL", map[string]interface{}{"error": err, "url": targetURL})
|
||||
http.Error(w, "Not Found", http.StatusNotFound)
|
||||
return
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
// Check if response is successful
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
masterlog.Info("Target returned non-200 status", map[string]interface{}{
|
||||
"url": targetURL,
|
||||
"status_code": resp.StatusCode,
|
||||
})
|
||||
http.Error(w, "Not Found", http.StatusNotFound)
|
||||
return
|
||||
}
|
||||
|
||||
// Copy headers from target response
|
||||
for key, values := range resp.Header {
|
||||
for _, value := range values {
|
||||
w.Header().Add(key, value)
|
||||
}
|
||||
}
|
||||
|
||||
// Set status code to 200
|
||||
w.WriteHeader(http.StatusOK)
|
||||
|
||||
// Copy the response body
|
||||
_, err = io.Copy(w, resp.Body)
|
||||
if err != nil {
|
||||
masterlog.Error("Error copying response", map[string]interface{}{"error": err})
|
||||
// Can't send error response after headers have been written
|
||||
return
|
||||
}
|
||||
|
||||
masterlog.Info("Successfully proxied request", map[string]interface{}{
|
||||
"url": targetURL,
|
||||
"status_code": http.StatusOK,
|
||||
"content_length": resp.ContentLength,
|
||||
})
|
||||
}
|
||||
13
docker-compose.yml
Normal file
13
docker-compose.yml
Normal file
@@ -0,0 +1,13 @@
|
||||
services:
|
||||
app:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
image: git.secnex.io/secnex/secnex-downloader:latest
|
||||
container_name: secnex-downloader
|
||||
restart: always
|
||||
ports:
|
||||
- 8080:8080
|
||||
environment:
|
||||
- URL=https://git.secnex.io/secnex/%s/raw/branch/main/%s
|
||||
- PORT=:8080
|
||||
Reference in New Issue
Block a user