Initial implementation of GitHub download proxy service
- Add flexible URL template configuration via environment variables - Implement automatic repository path parsing and extraction - Add 404 response handling for all non-200 status codes - Support both SecNex and GitHub URL formats through URL template - Include comprehensive documentation and Docker support - Add proper Go project structure with .gitignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
35
.gitignore
vendored
Normal file
35
.gitignore
vendored
Normal file
@@ -0,0 +1,35 @@
|
|||||||
|
# Binaries for programs and plugins
|
||||||
|
*.exe
|
||||||
|
*.exe~
|
||||||
|
*.dll
|
||||||
|
*.so
|
||||||
|
*.dylib
|
||||||
|
|
||||||
|
# Test binary, built with `go test -c`
|
||||||
|
*.test
|
||||||
|
|
||||||
|
# Output of the go coverage tool, specifically when used with LiteIDE
|
||||||
|
*.out
|
||||||
|
|
||||||
|
# Dependency directories (remove the comment below to include it)
|
||||||
|
# vendor/
|
||||||
|
|
||||||
|
# Go workspace file
|
||||||
|
go.work
|
||||||
|
|
||||||
|
# Binary output
|
||||||
|
gh-download-proxy
|
||||||
|
|
||||||
|
# IDE files
|
||||||
|
.vscode/
|
||||||
|
.idea/
|
||||||
|
*.swp
|
||||||
|
*.swo
|
||||||
|
*~
|
||||||
|
|
||||||
|
# OS files
|
||||||
|
.DS_Store
|
||||||
|
Thumbs.db
|
||||||
|
|
||||||
|
# Logs
|
||||||
|
*.log
|
||||||
68
CLAUDE.md
Normal file
68
CLAUDE.md
Normal file
@@ -0,0 +1,68 @@
|
|||||||
|
# CLAUDE.md
|
||||||
|
|
||||||
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||||
|
|
||||||
|
## Project Overview
|
||||||
|
|
||||||
|
This is a Go project named `gh-download-proxy` which appears to be a GitHub download proxy service. The project is currently in initial setup phase with only a `go.mod` file present.
|
||||||
|
|
||||||
|
## Development Setup
|
||||||
|
|
||||||
|
### Go Environment
|
||||||
|
- Go version: 1.25.3
|
||||||
|
- Module name: `gh-download-proxy`
|
||||||
|
|
||||||
|
### Common Commands
|
||||||
|
|
||||||
|
#### Building the project
|
||||||
|
```bash
|
||||||
|
go build ./...
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Running tests
|
||||||
|
```bash
|
||||||
|
go test ./...
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Running a specific test
|
||||||
|
```bash
|
||||||
|
go test -run <TestName> ./...
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Managing dependencies
|
||||||
|
```bash
|
||||||
|
go mod tidy # Clean up dependencies
|
||||||
|
go mod download # Download dependencies
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Running the application (once main.go exists)
|
||||||
|
```bash
|
||||||
|
go run main.go
|
||||||
|
```
|
||||||
|
|
||||||
|
## Project Structure
|
||||||
|
|
||||||
|
Currently minimal structure:
|
||||||
|
- `go.mod` - Go module definition
|
||||||
|
|
||||||
|
As the project develops, typical Go project structure would include:
|
||||||
|
- `main.go` - Main application entry point
|
||||||
|
- `cmd/` - Command-line applications
|
||||||
|
- `internal/` - Internal application code
|
||||||
|
- `pkg/` - Public library code
|
||||||
|
- `go.sum` - Dependency checksums
|
||||||
|
|
||||||
|
## Architecture Notes
|
||||||
|
|
||||||
|
Since this is intended to be a GitHub download proxy, the architecture will likely involve:
|
||||||
|
- HTTP server to handle download requests
|
||||||
|
- GitHub API integration
|
||||||
|
- File caching mechanisms
|
||||||
|
- Proxy functionality to intercept and serve GitHub downloads
|
||||||
|
|
||||||
|
## Development Notes
|
||||||
|
|
||||||
|
- This is a new repository with no commit history yet
|
||||||
|
- No Go source files exist yet - development is just beginning
|
||||||
|
- Consider adding a `README.md` with project documentation
|
||||||
|
- Consider adding `.gitignore` for Go-specific files
|
||||||
17
Dockerfile
Normal file
17
Dockerfile
Normal file
@@ -0,0 +1,17 @@
|
|||||||
|
FROM golang:1.25.3-alpine3.22 AS builder
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
COPY ./app/go.mod ./app/go.sum ./
|
||||||
|
|
||||||
|
RUN go mod download && go mod verify
|
||||||
|
|
||||||
|
COPY ./app ./.
|
||||||
|
|
||||||
|
RUN go build -o download .
|
||||||
|
|
||||||
|
FROM alpine:latest AS runner
|
||||||
|
|
||||||
|
COPY --from=builder /app/download /app/download
|
||||||
|
|
||||||
|
CMD ["/app/download"]
|
||||||
140
README.md
Normal file
140
README.md
Normal file
@@ -0,0 +1,140 @@
|
|||||||
|
# GitHub Download Proxy
|
||||||
|
|
||||||
|
A Go-based proxy service that automatically reads repositories and formats paths based on a customizable URL template. Features flexible URL configuration through environment variables.
|
||||||
|
|
||||||
|
## Features
|
||||||
|
|
||||||
|
- Flexible URL template configuration via environment variables
|
||||||
|
- Automatic path parsing and repository name extraction
|
||||||
|
- Proper HTTP header forwarding and status code handling
|
||||||
|
- Comprehensive logging with masterlog
|
||||||
|
- Simple and clean implementation
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
The proxy is configured via environment variables:
|
||||||
|
|
||||||
|
| Variable | Default | Description |
|
||||||
|
|----------|---------|-------------|
|
||||||
|
| `URL` | `https://git.secnex.io/secnex/%s/raw/branch/main/bin/%s` | URL template with `%s` placeholders for repository and file path |
|
||||||
|
| `PORT` | `:8080` | Port to listen on |
|
||||||
|
|
||||||
|
The `URL` template uses two `%s` placeholders:
|
||||||
|
1. First `%s` - Repository name
|
||||||
|
2. Second `%s` - File path within the repository
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
### Starting the server
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Default configuration (SecNex bin directory)
|
||||||
|
go run ./app
|
||||||
|
|
||||||
|
# With custom port
|
||||||
|
PORT=:9090 go run ./app
|
||||||
|
|
||||||
|
# For GitHub repositories
|
||||||
|
URL="https://github.com/%s/refs/heads/main/%s" go run ./app
|
||||||
|
|
||||||
|
# For SecNex raw files (not bin directory)
|
||||||
|
URL="https://git.secnex.io/secnex/%s/raw/branch/main/%s" go run ./app
|
||||||
|
|
||||||
|
# Custom Git server
|
||||||
|
URL="https://custom.git.server.com/%s/raw/main/%s" go run ./app
|
||||||
|
```
|
||||||
|
|
||||||
|
### Making requests
|
||||||
|
|
||||||
|
Request format: `GET /{repository}/{path/to/file}`
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Get specific file
|
||||||
|
curl http://localhost:8080/my-repo/path/to/file.txt
|
||||||
|
|
||||||
|
# Get file from repository root
|
||||||
|
curl http://localhost:8080/my-repo/config.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
## Examples
|
||||||
|
|
||||||
|
### SecNex Bin Directory (Default)
|
||||||
|
```bash
|
||||||
|
# Default configuration
|
||||||
|
go run ./app
|
||||||
|
|
||||||
|
# Request file
|
||||||
|
curl http://localhost:8080/my-app/release/my-app-linux
|
||||||
|
```
|
||||||
|
This proxies to: `https://git.secnex.io/secnex/my-app/raw/branch/main/bin/release/my-app-linux`
|
||||||
|
|
||||||
|
### GitHub Raw Files
|
||||||
|
```bash
|
||||||
|
# Configure for GitHub
|
||||||
|
URL="https://github.com/%s/refs/heads/main/%s" go run ./app
|
||||||
|
|
||||||
|
# Request file
|
||||||
|
curl http://localhost:8080/octocat/Hello-World/README.md
|
||||||
|
```
|
||||||
|
This proxies to: `https://github.com/octocat/Hello-World/refs/heads/main/README.md`
|
||||||
|
|
||||||
|
### Custom GitLab Instance
|
||||||
|
```bash
|
||||||
|
# Configure for custom GitLab
|
||||||
|
URL="https://gitlab.company.com/%s/-/raw/main/%s" go run ./app
|
||||||
|
|
||||||
|
# Request file
|
||||||
|
curl http://localhost:8080/project/repo/src/main.go
|
||||||
|
```
|
||||||
|
This proxies to: `https://gitlab.company.com/project/repo/-/raw/main/src/main.go`
|
||||||
|
|
||||||
|
### Different Branch
|
||||||
|
```bash
|
||||||
|
# Configure for develop branch
|
||||||
|
URL="https://git.secnex.io/secnex/%s/raw/branch/develop/%s" go run ./app
|
||||||
|
```
|
||||||
|
|
||||||
|
## Project Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
.
|
||||||
|
├── app/
|
||||||
|
│ └── main.go # Main application entry point
|
||||||
|
├── README.md # This documentation
|
||||||
|
├── CLAUDE.md # Development instructions
|
||||||
|
├── go.mod # Go module definition
|
||||||
|
├── go.sum # Dependency checksums
|
||||||
|
├── Dockerfile # Container configuration
|
||||||
|
└── docker-compose.yml # Development setup
|
||||||
|
```
|
||||||
|
|
||||||
|
## Building
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Build the application
|
||||||
|
go build ./app
|
||||||
|
|
||||||
|
# Build for production
|
||||||
|
go build -o gh-download-proxy ./app
|
||||||
|
|
||||||
|
# Run tests
|
||||||
|
go test ./...
|
||||||
|
```
|
||||||
|
|
||||||
|
## Dependencies
|
||||||
|
|
||||||
|
- Go 1.25.3+
|
||||||
|
- git.secnex.io/secnex/masterlog v0.1.0
|
||||||
|
|
||||||
|
## Running with Docker
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Build the Docker image
|
||||||
|
docker build -t gh-download-proxy .
|
||||||
|
|
||||||
|
# Run with default configuration
|
||||||
|
docker run -p 8080:8080 gh-download-proxy
|
||||||
|
|
||||||
|
# Run with custom URL
|
||||||
|
docker run -p 8080:8080 -e URL="https://github.com/%s/refs/heads/main/%s" gh-download-proxy
|
||||||
|
```
|
||||||
5
app/go.mod
Normal file
5
app/go.mod
Normal file
@@ -0,0 +1,5 @@
|
|||||||
|
module gh-download-proxy
|
||||||
|
|
||||||
|
go 1.25.3
|
||||||
|
|
||||||
|
require git.secnex.io/secnex/masterlog v0.1.0
|
||||||
2
app/go.sum
Normal file
2
app/go.sum
Normal file
@@ -0,0 +1,2 @@
|
|||||||
|
git.secnex.io/secnex/masterlog v0.1.0 h1:74j9CATpfeK0lxpWIQC9ag9083akwG8khi5BwLedD8E=
|
||||||
|
git.secnex.io/secnex/masterlog v0.1.0/go.mod h1:OnDlwEzdkKMnqY+G5O9kHdhoJ6fH1llbVdXpgSc5SdM=
|
||||||
150
app/main.go
Normal file
150
app/main.go
Normal file
@@ -0,0 +1,150 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"os"
|
||||||
|
"path"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"git.secnex.io/secnex/masterlog"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Config struct {
|
||||||
|
Port string
|
||||||
|
Url string
|
||||||
|
}
|
||||||
|
|
||||||
|
func main() {
|
||||||
|
config := loadConfig()
|
||||||
|
|
||||||
|
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
proxyHandler(w, r, config)
|
||||||
|
})
|
||||||
|
|
||||||
|
masterlog.Info("Starting server", map[string]interface{}{
|
||||||
|
"port": config.Port,
|
||||||
|
"url": config.Url,
|
||||||
|
})
|
||||||
|
|
||||||
|
if err := http.ListenAndServe(config.Port, nil); err != nil {
|
||||||
|
masterlog.Error("Error starting server", map[string]interface{}{"error": err})
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func loadConfig() *Config {
|
||||||
|
config := &Config{
|
||||||
|
Url: getEnv("URL", "https://git.secnex.io/secnex/%s/raw/branch/main/%s"),
|
||||||
|
Port: getEnv("PORT", ":8080"),
|
||||||
|
}
|
||||||
|
|
||||||
|
return config
|
||||||
|
}
|
||||||
|
|
||||||
|
func getEnv(key, defaultValue string) string {
|
||||||
|
if value := os.Getenv(key); value != "" {
|
||||||
|
return value
|
||||||
|
}
|
||||||
|
return defaultValue
|
||||||
|
}
|
||||||
|
|
||||||
|
func proxyHandler(w http.ResponseWriter, r *http.Request, config *Config) {
|
||||||
|
// Only handle GET requests
|
||||||
|
if r.Method != http.MethodGet {
|
||||||
|
http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Extract repository and file path from URL
|
||||||
|
// Expected format: /repositoryName/path/to/file
|
||||||
|
cleanPath := path.Clean(r.URL.Path)
|
||||||
|
if cleanPath == "/" || cleanPath == "" {
|
||||||
|
http.Error(w, "Repository name required", http.StatusBadRequest)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
pathParts := strings.Split(strings.TrimPrefix(cleanPath, "/"), "/")
|
||||||
|
if len(pathParts) < 1 {
|
||||||
|
http.Error(w, "Invalid path format", http.StatusBadRequest)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
repository := pathParts[0]
|
||||||
|
filePath := strings.Join(pathParts[1:], "/")
|
||||||
|
|
||||||
|
// Log the incoming request
|
||||||
|
masterlog.Info("Incoming request", map[string]interface{}{
|
||||||
|
"method": r.Method,
|
||||||
|
"repository": repository,
|
||||||
|
"file_path": filePath,
|
||||||
|
"original_path": r.URL.Path,
|
||||||
|
})
|
||||||
|
|
||||||
|
// Build target URL based on server type
|
||||||
|
var targetURL string
|
||||||
|
targetURL = fmt.Sprintf(config.Url, repository, filePath)
|
||||||
|
|
||||||
|
// Create HTTP client with timeout
|
||||||
|
client := &http.Client{}
|
||||||
|
req, err := http.NewRequestWithContext(r.Context(), http.MethodGet, targetURL, nil)
|
||||||
|
if err != nil {
|
||||||
|
masterlog.Error("Error creating request", map[string]interface{}{"error": err})
|
||||||
|
http.Error(w, "Failed to create request", http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Copy headers from original request
|
||||||
|
for key, values := range r.Header {
|
||||||
|
// Skip certain headers that shouldn't be forwarded
|
||||||
|
if strings.ToLower(key) == "host" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
for _, value := range values {
|
||||||
|
req.Header.Add(key, value)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
resp, err := client.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
masterlog.Error("Error fetching target URL", map[string]interface{}{"error": err, "url": targetURL})
|
||||||
|
http.Error(w, "Not Found", http.StatusNotFound)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
|
||||||
|
// Check if response is successful
|
||||||
|
if resp.StatusCode != http.StatusOK {
|
||||||
|
masterlog.Info("Target returned non-200 status", map[string]interface{}{
|
||||||
|
"url": targetURL,
|
||||||
|
"status_code": resp.StatusCode,
|
||||||
|
})
|
||||||
|
http.Error(w, "Not Found", http.StatusNotFound)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Copy headers from target response
|
||||||
|
for key, values := range resp.Header {
|
||||||
|
for _, value := range values {
|
||||||
|
w.Header().Add(key, value)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Set status code to 200
|
||||||
|
w.WriteHeader(http.StatusOK)
|
||||||
|
|
||||||
|
// Copy the response body
|
||||||
|
_, err = io.Copy(w, resp.Body)
|
||||||
|
if err != nil {
|
||||||
|
masterlog.Error("Error copying response", map[string]interface{}{"error": err})
|
||||||
|
// Can't send error response after headers have been written
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
masterlog.Info("Successfully proxied request", map[string]interface{}{
|
||||||
|
"url": targetURL,
|
||||||
|
"status_code": http.StatusOK,
|
||||||
|
"content_length": resp.ContentLength,
|
||||||
|
})
|
||||||
|
}
|
||||||
13
docker-compose.yml
Normal file
13
docker-compose.yml
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
services:
|
||||||
|
app:
|
||||||
|
build:
|
||||||
|
context: .
|
||||||
|
dockerfile: Dockerfile
|
||||||
|
image: git.secnex.io/secnex/secnex-downloader:latest
|
||||||
|
container_name: secnex-downloader
|
||||||
|
restart: always
|
||||||
|
ports:
|
||||||
|
- 8080:8080
|
||||||
|
environment:
|
||||||
|
- URL=https://git.secnex.io/secnex/%s/raw/branch/main/%s
|
||||||
|
- PORT=:8080
|
||||||
Reference in New Issue
Block a user