SecNex/masterlog
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

feat(log): Formatted console logging with file output

Browse Source
This commit is contained in:
Björn Benouarets
2025-11-10 05:20:25 +01:00
parent 3e78775b0f
commit 7b3c176d20
12 changed files with 1344 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

580
README.md
View File

@@ -1,2 +1,582 @@
# MasterLog - Logging Library for Golang
A powerful, extensible, and zero-dependency logging library for Go with built-in support for structured logging, colored output, and data pseudonymization.
## Features
- 🎨 **Colored Console Output** - Beautiful, syntax-highlighted logs with automatic terminal detection
- 📝 **Structured Logging** - Support for custom fields and structured data
- 🔒 **Data Pseudonymization** - Built-in HMAC-based deterministic pseudonymization for sensitive data
- 📊 **Multiple Encoders** - Formatted text and JSON encoding out of the box
- 📁 **Flexible Writers** - Console and file writers with easy extensibility
- 🎯 **Zero Dependencies** - Uses only Go standard library
- 🔧 **Extensible Architecture** - Easy to add custom encoders and writers
- ⚡ **Performance** - Efficient logging with minimal overhead
- 🎛️ **Configurable Log Levels** - Trace, Debug, Info, Warn, Error
## Installation
```bash
go get git.secnex.io/secnex/masterlog
```
## Quick Start
```go
package main
import (
"git.secnex.io/secnex/masterlog"
)
func main() {
// Simple logging
masterlog.Info("Hello, World!")
// Logging with fields
masterlog.Info("User logged in", map[string]interface{}{
"user_id": 12345,
"ip": "192.168.1.1",
})
}
```
**Output:**
```
2025-11-10T05:06:02+01:00 INF main.go:9 > Hello, World! go_version=go1.25.3 pid=12345
2025-11-10T05:06:02+01:00 INF main.go:12 > User logged in user_id=12345 ip=192.168.1.1 go_version=go1.25.3 pid=12345
```
## Table of Contents
- [Log Levels](#log-levels)
- [Structured Logging](#structured-logging)
- [Colored Output](#colored-output)
- [Encoders](#encoders)
- [Writers](#writers)
- [Custom Logger Instances](#custom-logger-instances)
- [Data Pseudonymization](#data-pseudonymization)
- [Extending MasterLog](#extending-masterlog)
- [API Reference](#api-reference)
- [Best Practices](#best-practices)
## Log Levels
MasterLog supports five log levels:
- `TRC` (Trace) - Most verbose, for detailed debugging
- `DBG` (Debug) - Debug information
- `INF` (Info) - General informational messages
- `WRN` (Warn) - Warning messages
- `ERR` (Error) - Error messages
### Setting Log Level
```go
// Set level for default logger
masterlog.SetLevel(masterlog.LevelDebug)
// Create logger with specific level
logger := masterlog.New(masterlog.LevelTrace)
logger.Trace("This will be logged")
logger.Debug("This will also be logged")
```
### Logging Methods
```go
masterlog.Trace("trace message")
masterlog.Debug("debug message")
masterlog.Info("info message")
masterlog.Warn("warning message")
masterlog.Error("error message")
```
## Structured Logging
MasterLog supports structured logging with custom fields. Custom fields are displayed before default fields (go_version, pid).
```go
masterlog.Info("User action", map[string]interface{}{
"user_id": 12345,
"action": "purchase",
"amount": 99.99,
"ip": "192.168.1.1",
})
```
**Output:**
```
2025-11-10T05:06:02+01:00 INF main.go:12 > User action user_id=12345 action=purchase amount=99.99 ip=192.168.1.1 go_version=go1.25.3 pid=12345
```
### Field Ordering
Custom fields (user-provided) are always displayed before default fields:
1. Custom fields (user-provided)
2. Default fields (go_version, pid)
## Colored Output
MasterLog automatically detects if output is going to a terminal and applies colors accordingly. Colors are never written to files.
### Color Scheme
- **Timestamp**: Dark gray
- **Log Level**:
- `TRC`: Gray
- `DBG`: Cyan
- `INF`: Green
- `WRN`: Yellow
- `ERR`: Red
- **File:Line**: Dark gray
- **Message**: White
- **Field Keys**: Turquoise
- **Field Values**: White
### Example Output
When logging to a terminal, you'll see beautifully colored output. When redirecting to a file or non-terminal, colors are automatically disabled.
```go
masterlog.Info("Colored output example", map[string]interface{}{
"key": "value",
})
```
## Encoders
Encoders determine the format of log entries. MasterLog provides two built-in encoders:
### FormattedEncoder
The default encoder that produces human-readable formatted text.
```go
logger := masterlog.New()
// FormattedEncoder is used by default
logger.Info("Formatted log")
```
**Output:**
```
2025-11-10T05:06:02+01:00 INF main.go:12 > Formatted log go_version=go1.25.3 pid=12345
```
### JSONEncoder
Produces JSON-formatted log entries, perfect for log aggregation systems.
```go
logger := masterlog.New()
logger.AddEncoder(&masterlog.JSONEncoder{})
logger.Info("JSON log", map[string]interface{}{
"key": "value",
})
```
**Output:**
```json
{"timestamp":"2025-11-10T05:06:02+01:00","level":"INF","file":"main.go","line":12,"message":"JSON log","fields":{"key":"value","go_version":"go1.25.3","pid":12345}}
```
**Note:** JSON encoder output is automatically excluded from console output when using the default logger to keep console logs readable.
### Multiple Encoders
You can use multiple encoders simultaneously:
```go
logger := masterlog.New()
logger.AddEncoder(&masterlog.JSONEncoder{})
// Now logs will be written in both formatted and JSON formats
logger.Info("Dual format log")
```
## Writers
Writers determine where log entries are written. MasterLog provides two built-in writers:
### ConsoleWriter
Writes to stdout (default). Automatically detects if output is a terminal.
```go
logger := masterlog.New()
// ConsoleWriter is used by default
logger.Info("Console log")
```
### FileWriter
Writes log entries to a file.
```go
logger := masterlog.New()
fileWriter, err := masterlog.NewFileWriter("app.log")
if err != nil {
log.Fatal(err)
}
defer fileWriter.Close()
logger.AddWriter(fileWriter)
logger.Info("File log")
```
### Multiple Writers
You can write to multiple destinations simultaneously:
```go
logger := masterlog.New()
// Add file writer
fileWriter, _ := masterlog.NewFileWriter("app.log")
defer fileWriter.Close()
logger.AddWriter(fileWriter)
// Logs will be written to both console and file
logger.Info("Multi-destination log")
```
## Custom Logger Instances
Create custom logger instances with specific configurations:
```go
// Create logger with custom level
logger := masterlog.New(masterlog.LevelDebug)
// Add JSON encoder
logger.AddEncoder(&masterlog.JSONEncoder{})
// Add file writer
fileWriter, _ := masterlog.NewFileWriter("custom.log")
defer fileWriter.Close()
logger.AddWriter(fileWriter)
logger.Info("Custom logger example")
```
## Data Pseudonymization
MasterLog includes built-in support for deterministic pseudonymization of sensitive data using HMAC-SHA256. This allows you to:
- Protect sensitive information in logs
- Maintain traceability (same input always produces same pseudonymized output)
- Comply with data protection regulations
### Basic Usage
```go
// Create pseudonymizer with secret
pseudonymizer := masterlog.NewPseudonymizerFromString("your-secret-key")
logger := masterlog.New()
logger.SetPseudonymizer(pseudonymizer)
// Mark fields as sensitive
logger.AddSensitiveFields("user_id", "email", "ip", "ssn")
// Log with sensitive data
logger.Info("User logged in", map[string]interface{}{
"user_id": 12345, // Will be pseudonymized
"email": "user@ex.com", // Will be pseudonymized
"ip": "192.168.1.1", // Will be pseudonymized
"action": "login", // Not sensitive, remains unchanged
})
```
**Output:**
```
2025-11-10T05:06:02+01:00 INF main.go:15 > User logged in user_id=a1b2c3d4 email=e5f6g7h8 ip=i9j0k1l2 action=login go_version=go1.25.3 pid=12345
```
### Deterministic Behavior
The same input always produces the same pseudonymized output, allowing you to trace the same entity across multiple log entries:
```go
logger.Info("User action 1", map[string]interface{}{
"user_id": 12345, // Pseudonymized to: a1b2c3d4
})
logger.Info("User action 2", map[string]interface{}{
"user_id": 12345, // Pseudonymized to: a1b2c3d4 (same as above)
})
```
### Configuration Options
#### Using Environment Variables
For production, use environment variables to store the secret:
```go
// Read secret from environment variable
pseudonymizer := masterlog.NewPseudonymizerFromEnv("LOG_PSEUDONYMIZER_SECRET")
logger.SetPseudonymizer(pseudonymizer)
```
#### Custom Hash Length
Adjust the length of pseudonymized values (default: 8 hex characters):
```go
pseudonymizer := masterlog.NewPseudonymizerFromString("secret")
pseudonymizer.SetHashLength(16) // Use 16 hex characters instead of 8
```
#### Managing Sensitive Fields
```go
// Add single field
logger.AddSensitiveField("user_id")
// Add multiple fields
logger.AddSensitiveFields("email", "ip", "ssn", "phone")
// Remove field (if needed)
pseudonymizer := masterlog.NewPseudonymizerFromString("secret")
pseudonymizer.RemoveSensitiveField("phone")
```
### Security Considerations
1. **Secret Management**: Store the pseudonymization secret securely (environment variables, secrets manager)
2. **Secret Rotation**: Changing the secret will produce different pseudonymized values
3. **Hash Length**: Longer hashes provide better uniqueness but are less readable
4. **Field Selection**: Only mark truly sensitive fields to maintain log usefulness
## Extending MasterLog
MasterLog is designed to be easily extensible. You can create custom encoders and writers.
### Custom Encoder
```go
type CustomEncoder struct{}
func (e *CustomEncoder) Encode(entry masterlog.Entry) ([]byte, error) {
// Your custom encoding logic
return []byte("custom format"), nil
}
// Usage
logger := masterlog.New()
logger.AddEncoder(&CustomEncoder{})
```
### Custom Writer
```go
type CustomWriter struct {
// Your writer fields
}
func (w *CustomWriter) Write(data []byte) error {
// Your custom writing logic
return nil
}
func (w *CustomWriter) Close() error {
// Cleanup logic
return nil
}
func (w *CustomWriter) SupportsColors() bool {
return false // or true if your writer supports colors
}
// Usage
logger := masterlog.New()
logger.AddWriter(&CustomWriter{})
```
## API Reference
### Package-Level Functions
#### Logging Functions
```go
func Trace(message string, fields ...map[string]interface{})
func Debug(message string, fields ...map[string]interface{})
func Info(message string, fields ...map[string]interface{})
func Warn(message string, fields ...map[string]interface{})
func Error(message string, fields ...map[string]interface{})
```
#### Configuration Functions
```go
func SetLevel(level Level)
func AddWriter(writer Writer)
func AddEncoder(encoder Encoder)
func SetPseudonymizer(pseudonymizer *Pseudonymizer)
func AddSensitiveField(fieldName string)
func AddSensitiveFields(fieldNames ...string)
```
### Logger Methods
```go
type MasterLogger struct {
// ...
}
func New(level ...Level) *MasterLogger
func (l *MasterLogger) SetLevel(level Level)
func (l *MasterLogger) AddWriter(writer Writer)
func (l *MasterLogger) AddEncoder(encoder Encoder)
func (l *MasterLogger) SetPseudonymizer(pseudonymizer *Pseudonymizer)
func (l *MasterLogger) AddSensitiveField(fieldName string)
func (l *MasterLogger) AddSensitiveFields(fieldNames ...string)
func (l *MasterLogger) Trace(message string, fields ...map[string]interface{})
func (l *MasterLogger) Debug(message string, fields ...map[string]interface{})
func (l *MasterLogger) Info(message string, fields ...map[string]interface{})
func (l *MasterLogger) Warn(message string, fields ...map[string]interface{})
func (l *MasterLogger) Error(message string, fields ...map[string]interface{})
```
### Pseudonymizer
```go
type Pseudonymizer struct {
// ...
}
func NewPseudonymizer(secret []byte) *Pseudonymizer
func NewPseudonymizerFromString(secret string) *Pseudonymizer
func NewPseudonymizerFromEnv(envVar string) *Pseudonymizer
func (p *Pseudonymizer) SetHashLength(length int) error
func (p *Pseudonymizer) AddSensitiveField(fieldName string)
func (p *Pseudonymizer) AddSensitiveFields(fieldNames ...string)
func (p *Pseudonymizer) RemoveSensitiveField(fieldName string)
func (p *Pseudonymizer) IsSensitive(fieldName string) bool
func (p *Pseudonymizer) Pseudonymize(value interface{}) string
func (p *Pseudonymizer) PseudonymizeFields(fields map[string]interface{}) map[string]interface{}
```
### Writers
```go
func NewConsoleWriter() *ConsoleWriter
func NewFileWriter(filename string) (*FileWriter, error)
```
### Encoders
```go
func NewFormattedEncoder(useColors bool) *FormattedEncoder
// JSONEncoder has no constructor, use &masterlog.JSONEncoder{}
```
## Best Practices
### 1. Use Structured Logging
Always use structured fields instead of string concatenation:
```go
// ❌ Bad
masterlog.Info(fmt.Sprintf("User %d logged in from %s", userID, ip))
// ✅ Good
masterlog.Info("User logged in", map[string]interface{}{
"user_id": userID,
"ip": ip,
})
```
### 2. Set Appropriate Log Levels
Use log levels appropriately:
- `TRC`: Detailed trace information (usually disabled in production)
- `DBG`: Debug information (disabled in production)
- `INF`: General information about application flow
- `WRN`: Warning conditions that don't stop execution
- `ERR`: Error conditions that need attention
### 3. Pseudonymize Sensitive Data
Always pseudonymize sensitive information:
```go
logger.AddSensitiveFields("user_id", "email", "ip", "ssn", "credit_card")
```
### 4. Use Environment Variables for Secrets
Never hardcode secrets:
```go
// ❌ Bad
pseudonymizer := masterlog.NewPseudonymizerFromString("hardcoded-secret")
// ✅ Good
pseudonymizer := masterlog.NewPseudonymizerFromEnv("LOG_PSEUDONYMIZER_SECRET")
```
### 5. Separate Logs by Environment
Use different configurations for development and production:
```go
logger := masterlog.New()
if os.Getenv("ENV") == "production" {
// Production: JSON to file, no colors
logger.AddEncoder(&masterlog.JSONEncoder{})
fileWriter, _ := masterlog.NewFileWriter("/var/log/app.log")
logger.AddWriter(fileWriter)
// Enable pseudonymization
pseudonymizer := masterlog.NewPseudonymizerFromEnv("LOG_SECRET")
logger.SetPseudonymizer(pseudonymizer)
logger.AddSensitiveFields("user_id", "email", "ip")
} else {
// Development: Formatted output with colors
logger.SetLevel(masterlog.LevelDebug)
}
```
### 6. Close File Writers
Always close file writers when done:
```go
fileWriter, err := masterlog.NewFileWriter("app.log")
if err != nil {
return err
}
defer fileWriter.Close()
```
### 7. Use Custom Logger Instances
Create separate logger instances for different components:
```go
var (
apiLogger = masterlog.New(masterlog.LevelInfo)
dbLogger = masterlog.New(masterlog.LevelDebug)
authLogger = masterlog.New(masterlog.LevelWarn)
)
```
## Examples
See the [example](example/) directory for complete working examples.
## License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
## Contributing
Contributions are welcome! Please feel free to submit a pull request.
## Support
Please feel free to open an issue if you have any questions or suggestions. You can also send an email to support@secnex.io.