MasterLog - Logging Library for Golang

A powerful, extensible, and zero-dependency logging library for Go with built-in support for structured logging, colored output, and data pseudonymization.

Features

• 🎨 Colored Console Output - Beautiful, syntax-highlighted logs with automatic terminal detection
• 📝 Structured Logging - Support for custom fields and structured data
• 🔒 Data Pseudonymization - Built-in HMAC-based deterministic pseudonymization for sensitive data
• 📊 Multiple Encoders - Formatted text and JSON encoding out of the box
• 📁 Flexible Writers - Console and file writers with easy extensibility
• 🎯 Zero Dependencies - Uses only Go standard library
• 🔧 Extensible Architecture - Easy to add custom encoders and writers
• ⚡ Performance - Efficient logging with minimal overhead
• 🎛️ Configurable Log Levels - Trace, Debug, Info, Warn, Error

Installation

go get git.secnex.io/secnex/masterlog

Quick Start

Basic Usage

package main

import (
    "git.secnex.io/secnex/masterlog"
)

func main() {
    // Simple logging using the global logger
    masterlog.Info("Hello, World!")
    
    // Logging with fields
    masterlog.Info("User logged in", map[string]interface{}{
        "user_id": 12345,
        "ip":      "192.168.1.1",
    })
}

Output:

2025-11-10T05:06:02+01:00 INF main.go:9 > Hello, World! go_version=go1.25.3 pid=12345
2025-11-10T05:06:02+01:00 INF main.go:12 > User logged in user_id=12345 ip=192.168.1.1 go_version=go1.25.3 pid=12345

Configuring the Global Logger

Configure the global logger once at application startup:

package main

import (
    "git.secnex.io/secnex/masterlog"
)

func main() {
    // Configure global logger at startup
    masterlog.SetLevel(masterlog.LevelDebug)
    
    // Add file writer
    fileWriter, _ := masterlog.NewFileWriter("app.log")
    defer fileWriter.Close()
    masterlog.AddWriter(fileWriter)
    
    // Add JSON encoder
    masterlog.AddEncoder(&masterlog.JSONEncoder{})
    
    // Configure pseudonymization
    pseudonymizer := masterlog.NewPseudonymizerFromEnv("LOG_SECRET")
    masterlog.SetPseudonymizer(pseudonymizer)
    masterlog.AddSensitiveFields("user_id", "email", "ip")
    
    // Now all package-level functions use the configured global logger
    masterlog.Info("Application started")
    masterlog.Debug("Debug message")
}

Benefits:

• Configure once, use everywhere
• No need to pass logger instances around
• Simple and convenient for most use cases

Table of Contents

• Log Levels
• Structured Logging
• Colored Output
• Encoders
• Writers
• Custom Logger Instances
• Data Pseudonymization
• Extending MasterLog
• API Reference
• Best Practices

Log Levels

MasterLog supports five log levels:

• TRC (Trace) - Most verbose, for detailed debugging
• DBG (Debug) - Debug information
• INF (Info) - General informational messages
• WRN (Warn) - Warning messages
• ERR (Error) - Error messages

Setting Log Level

// Set level for default logger
masterlog.SetLevel(masterlog.LevelDebug)

// Create logger with specific level
logger := masterlog.New(masterlog.LevelTrace)
logger.Trace("This will be logged")
logger.Debug("This will also be logged")

Logging Methods

masterlog.Trace("trace message")
masterlog.Debug("debug message")
masterlog.Info("info message")
masterlog.Warn("warning message")
masterlog.Error("error message")

Structured Logging

MasterLog supports structured logging with custom fields. Custom fields are displayed before default fields (go_version, pid).

masterlog.Info("User action", map[string]interface{}{
    "user_id": 12345,
    "action":  "purchase",
    "amount":  99.99,
    "ip":      "192.168.1.1",
})

Output:

2025-11-10T05:06:02+01:00 INF main.go:12 > User action user_id=12345 action=purchase amount=99.99 ip=192.168.1.1 go_version=go1.25.3 pid=12345

Field Ordering

Custom fields (user-provided) are always displayed before default fields:

1. Custom fields (user-provided)
2. Default fields (go_version, pid)

Colored Output

MasterLog automatically detects if output is going to a terminal and applies colors accordingly. Colors are never written to files.

Color Scheme

• Timestamp: Dark gray
• Log Level:
  • TRC: Gray
  • DBG: Cyan
  • INF: Green
  • WRN: Yellow
  • ERR: Red
• File:Line: Dark gray
• Message: White
• Field Keys: Turquoise
• Field Values: White

Example Output

When logging to a terminal, you'll see beautifully colored output. When redirecting to a file or non-terminal, colors are automatically disabled.

masterlog.Info("Colored output example", map[string]interface{}{
    "key": "value",
})

Encoders

Encoders determine the format of log entries. MasterLog provides two built-in encoders:

FormattedEncoder

The default encoder that produces human-readable formatted text.

logger := masterlog.New()
// FormattedEncoder is used by default
logger.Info("Formatted log")

Output:

2025-11-10T05:06:02+01:00 INF main.go:12 > Formatted log go_version=go1.25.3 pid=12345

JSONEncoder

Produces JSON-formatted log entries, perfect for log aggregation systems.

logger := masterlog.New()
logger.AddEncoder(&masterlog.JSONEncoder{})
logger.Info("JSON log", map[string]interface{}{
    "key": "value",
})

Output:

{"timestamp":"2025-11-10T05:06:02+01:00","level":"INF","file":"main.go","line":12,"message":"JSON log","fields":{"key":"value","go_version":"go1.25.3","pid":12345}}

Note: JSON encoder output is automatically excluded from console output when using the default logger to keep console logs readable.

Multiple Encoders

You can use multiple encoders simultaneously:

logger := masterlog.New()
logger.AddEncoder(&masterlog.JSONEncoder{})
// Now logs will be written in both formatted and JSON formats
logger.Info("Dual format log")

Writers

Writers determine where log entries are written. MasterLog provides two built-in writers:

ConsoleWriter

Writes to stdout (default). Automatically detects if output is a terminal.

logger := masterlog.New()
// ConsoleWriter is used by default
logger.Info("Console log")

FileWriter

Writes log entries to a file.

Using with global logger (recommended):

// Add file writer to global logger
fileWriter, err := masterlog.NewFileWriter("app.log")
if err != nil {
    log.Fatal(err)
}
defer fileWriter.Close()

masterlog.AddWriter(fileWriter)
masterlog.Info("File log")  // Uses global logger

Using with custom logger:

logger := masterlog.New()
fileWriter, err := masterlog.NewFileWriter("app.log")
if err != nil {
    log.Fatal(err)
}
defer fileWriter.Close()

logger.AddWriter(fileWriter)
logger.Info("File log")

Multiple Writers

You can write to multiple destinations simultaneously:

logger := masterlog.New()

// Add file writer
fileWriter, _ := masterlog.NewFileWriter("app.log")
defer fileWriter.Close()
logger.AddWriter(fileWriter)

// Logs will be written to both console and file
logger.Info("Multi-destination log")

Global Logger vs Custom Instances

MasterLog provides two ways to use logging:

Global Logger (Recommended for Most Cases)

The global logger is pre-configured and ready to use. Configure it once at startup:

func main() {
    // Configure global logger at startup
    masterlog.SetLevel(masterlog.LevelDebug)
    
    // Add file writer
    fileWriter, _ := masterlog.NewFileWriter("app.log")
    defer fileWriter.Close()
    masterlog.AddWriter(fileWriter)
    
    // Add JSON encoder
    masterlog.AddEncoder(&masterlog.JSONEncoder{})
    
    // Use package-level functions anywhere in your code
    masterlog.Info("This uses the global logger")
    masterlog.Debug("No need to pass logger instances around")
}

Advantages:

• Simple and convenient
• No need to pass logger instances
• Configure once, use everywhere
• Perfect for most applications

Custom Logger Instances

Create custom logger instances when you need different configurations for different components:

// Create logger with custom level
logger := masterlog.New(masterlog.LevelDebug)

// Add JSON encoder
logger.AddEncoder(&masterlog.JSONEncoder{})

// Add file writer
fileWriter, _ := masterlog.NewFileWriter("custom.log")
defer fileWriter.Close()
logger.AddWriter(fileWriter)

logger.Info("Custom logger example")

Use Cases:

• Different log levels for different components
• Separate log files for different modules
• Different encoders for different outputs
• Component-specific pseudonymization rules

Replacing the Global Logger

You can also replace the global logger with a custom instance:

// Create and configure a custom logger
customLogger := masterlog.New(masterlog.LevelTrace)
customLogger.AddEncoder(&masterlog.JSONEncoder{})

// Replace the global logger
masterlog.SetDefaultLogger(customLogger)

// Now all package-level functions use your custom logger
masterlog.Info("Uses the custom logger")

Getting the Global Logger

Access the global logger instance directly if needed:

globalLogger := masterlog.GetDefaultLogger()
globalLogger.AddWriter(fileWriter)

Data Pseudonymization

MasterLog includes built-in support for deterministic pseudonymization of sensitive data using HMAC-SHA256. This allows you to:

• Protect sensitive information in logs
• Maintain traceability (same input always produces same pseudonymized output)
• Comply with data protection regulations

Basic Usage

// Create pseudonymizer with secret
pseudonymizer := masterlog.NewPseudonymizerFromString("your-secret-key")
logger := masterlog.New()
logger.SetPseudonymizer(pseudonymizer)

// Mark fields as sensitive
logger.AddSensitiveFields("user_id", "email", "ip", "ssn")

// Log with sensitive data
logger.Info("User logged in", map[string]interface{}{
    "user_id": 12345,           // Will be pseudonymized
    "email":   "user@ex.com",   // Will be pseudonymized
    "ip":      "192.168.1.1",   // Will be pseudonymized
    "action":  "login",         // Not sensitive, remains unchanged
})

Output:

2025-11-10T05:06:02+01:00 INF main.go:15 > User logged in user_id=a1b2c3d4 email=e5f6g7h8 ip=i9j0k1l2 action=login go_version=go1.25.3 pid=12345

Deterministic Behavior

The same input always produces the same pseudonymized output, allowing you to trace the same entity across multiple log entries:

logger.Info("User action 1", map[string]interface{}{
    "user_id": 12345,  // Pseudonymized to: a1b2c3d4
})

logger.Info("User action 2", map[string]interface{}{
    "user_id": 12345,  // Pseudonymized to: a1b2c3d4 (same as above)
})

Configuration Options

Using Environment Variables

For production, use environment variables to store the secret:

// Read secret from environment variable
pseudonymizer := masterlog.NewPseudonymizerFromEnv("LOG_PSEUDONYMIZER_SECRET")
logger.SetPseudonymizer(pseudonymizer)

Custom Hash Length

Adjust the length of pseudonymized values (default: 8 hex characters):

pseudonymizer := masterlog.NewPseudonymizerFromString("secret")
pseudonymizer.SetHashLength(16)  // Use 16 hex characters instead of 8

Managing Sensitive Fields

// Add single field
logger.AddSensitiveField("user_id")

// Add multiple fields
logger.AddSensitiveFields("email", "ip", "ssn", "phone")

// Remove field (if needed)
pseudonymizer := masterlog.NewPseudonymizerFromString("secret")
pseudonymizer.RemoveSensitiveField("phone")

Security Considerations

1. Secret Management: Store the pseudonymization secret securely (environment variables, secrets manager)
2. Secret Rotation: Changing the secret will produce different pseudonymized values
3. Hash Length: Longer hashes provide better uniqueness but are less readable
4. Field Selection: Only mark truly sensitive fields to maintain log usefulness

Extending MasterLog

MasterLog is designed to be easily extensible. You can create custom encoders and writers.

Custom Encoder

type CustomEncoder struct{}

func (e *CustomEncoder) Encode(entry masterlog.Entry) ([]byte, error) {
    // Your custom encoding logic
    return []byte("custom format"), nil
}

// Usage
logger := masterlog.New()
logger.AddEncoder(&CustomEncoder{})

Custom Writer

type CustomWriter struct {
    // Your writer fields
}

func (w *CustomWriter) Write(data []byte) error {
    // Your custom writing logic
    return nil
}

func (w *CustomWriter) Close() error {
    // Cleanup logic
    return nil
}

func (w *CustomWriter) SupportsColors() bool {
    return false  // or true if your writer supports colors
}

// Usage
logger := masterlog.New()
logger.AddWriter(&CustomWriter{})

API Reference

Package-Level Functions

Logging Functions

func Trace(message string, fields ...map[string]interface{})
func Debug(message string, fields ...map[string]interface{})
func Info(message string, fields ...map[string]interface{})
func Warn(message string, fields ...map[string]interface{})
func Error(message string, fields ...map[string]interface{})

Configuration Functions

func SetLevel(level Level)
func AddWriter(writer Writer)
func AddEncoder(encoder Encoder)
func SetPseudonymizer(pseudonymizer *Pseudonymizer)
func AddSensitiveField(fieldName string)
func AddSensitiveFields(fieldNames ...string)
func SetDefaultLogger(logger *MasterLogger)
func GetDefaultLogger() *MasterLogger

Logger Methods

type MasterLogger struct {
    // ...
}

func New(level ...Level) *MasterLogger
func (l *MasterLogger) SetLevel(level Level)
func (l *MasterLogger) AddWriter(writer Writer)
func (l *MasterLogger) AddEncoder(encoder Encoder)
func (l *MasterLogger) SetPseudonymizer(pseudonymizer *Pseudonymizer)
func (l *MasterLogger) AddSensitiveField(fieldName string)
func (l *MasterLogger) AddSensitiveFields(fieldNames ...string)
func (l *MasterLogger) Trace(message string, fields ...map[string]interface{})
func (l *MasterLogger) Debug(message string, fields ...map[string]interface{})
func (l *MasterLogger) Info(message string, fields ...map[string]interface{})
func (l *MasterLogger) Warn(message string, fields ...map[string]interface{})
func (l *MasterLogger) Error(message string, fields ...map[string]interface{})

Pseudonymizer

type Pseudonymizer struct {
    // ...
}

func NewPseudonymizer(secret []byte) *Pseudonymizer
func NewPseudonymizerFromString(secret string) *Pseudonymizer
func NewPseudonymizerFromEnv(envVar string) *Pseudonymizer
func (p *Pseudonymizer) SetHashLength(length int) error
func (p *Pseudonymizer) AddSensitiveField(fieldName string)
func (p *Pseudonymizer) AddSensitiveFields(fieldNames ...string)
func (p *Pseudonymizer) RemoveSensitiveField(fieldName string)
func (p *Pseudonymizer) IsSensitive(fieldName string) bool
func (p *Pseudonymizer) Pseudonymize(value interface{}) string
func (p *Pseudonymizer) PseudonymizeFields(fields map[string]interface{}) map[string]interface{}

Writers

func NewConsoleWriter() *ConsoleWriter
func NewFileWriter(filename string) (*FileWriter, error)

Encoders

func NewFormattedEncoder(useColors bool) *FormattedEncoder
// JSONEncoder has no constructor, use &masterlog.JSONEncoder{}

Best Practices

1. Use Structured Logging

Always use structured fields instead of string concatenation:

// ❌ Bad
masterlog.Info(fmt.Sprintf("User %d logged in from %s", userID, ip))

// ✅ Good
masterlog.Info("User logged in", map[string]interface{}{
    "user_id": userID,
    "ip":      ip,
})

2. Set Appropriate Log Levels

Use log levels appropriately:

• TRC: Detailed trace information (usually disabled in production)
• DBG: Debug information (disabled in production)
• INF: General information about application flow
• WRN: Warning conditions that don't stop execution
• ERR: Error conditions that need attention

3. Pseudonymize Sensitive Data

Always pseudonymize sensitive information:

logger.AddSensitiveFields("user_id", "email", "ip", "ssn", "credit_card")

4. Use Environment Variables for Secrets

Never hardcode secrets:

// ❌ Bad
pseudonymizer := masterlog.NewPseudonymizerFromString("hardcoded-secret")

// ✅ Good
pseudonymizer := masterlog.NewPseudonymizerFromEnv("LOG_PSEUDONYMIZER_SECRET")

5. Separate Logs by Environment

Use different configurations for development and production:

logger := masterlog.New()

if os.Getenv("ENV") == "production" {
    // Production: JSON to file, no colors
    logger.AddEncoder(&masterlog.JSONEncoder{})
    fileWriter, _ := masterlog.NewFileWriter("/var/log/app.log")
    logger.AddWriter(fileWriter)
    
    // Enable pseudonymization
    pseudonymizer := masterlog.NewPseudonymizerFromEnv("LOG_SECRET")
    logger.SetPseudonymizer(pseudonymizer)
    logger.AddSensitiveFields("user_id", "email", "ip")
} else {
    // Development: Formatted output with colors
    logger.SetLevel(masterlog.LevelDebug)
}

6. Close File Writers

Always close file writers when done:

fileWriter, err := masterlog.NewFileWriter("app.log")
if err != nil {
    return err
}
defer fileWriter.Close()

7. Use Custom Logger Instances

Create separate logger instances for different components:

var (
    apiLogger    = masterlog.New(masterlog.LevelInfo)
    dbLogger     = masterlog.New(masterlog.LevelDebug)
    authLogger   = masterlog.New(masterlog.LevelWarn)
)

Examples

See the example directory for complete working examples.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Contributing

Contributions are welcome! Please feel free to submit a pull request.

Support

Please feel free to open an issue if you have any questions or suggestions. You can also send an email to support@secnex.io.