Architecture

LogXide delivers high performance through its native Rust implementation, providing Python applications with fast logging while maintaining a familiar API.

Core Architecture

┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│   Python API    │    │   Rust Core      │    │   I/O Output    │
│                 │    │                  │    │                 │
│ ┌─────────────┐ │    │ ┌──────────────┐ │    │ ┌─────────────┐ │
│ │ PyLogger    │ │───▶│ │ LogRecord    │ │───▶│ │ Files       │ │
│ │ Methods     │ │    │ │ Creation     │ │    │ │ Streams     │ │
│ └─────────────┘ │    │ └──────────────┘ │    │ │ HTTP/OTLP   │ │
│                 │    │                  │    │ └─────────────┘ │
│ ┌─────────────┐ │    │ ┌──────────────┐ │    │                 │
│ │ basicConfig │ │───▶│ │ Direct       │ │    │                 │
│ │ flush()     │ │    │ │ Handler Call │ │    │                 │
│ └─────────────┘ │    │ └──────────────┘ │    │                 │
└─────────────────┘    └──────────────────┘    └─────────────────┘

Message Processing Flow

1. Python Call → LogXide PyLogger methods via PyO3
2. Record Creation → Rust LogRecord with full metadata (logger name, level, timestamp, thread info)
3. Handler Dispatch → Each handler's emit() is called (non-blocking for stream/HTTP/OTLP, synchronous for file and rotating-file handlers)
4. Output → Formatted messages written to files/streams/HTTP endpoints

Key Components

PyO3 Integration (src/lib.rs, src/py_logger.rs, src/py_handlers.rs)

• Python bindings exposing Logger, Handler, and Formatter types
• addHandler() accepts LogXide's Rust native handlers (fast path) and standard Python logging.Handler subclasses (run once on the Python side, without the fast-path GIL release). As of 0.2.0, Rust-backed handlers are dispatched exactly once and never leak to unrelated loggers.

Core Types (src/core.rs)

• LogRecord — Rust struct holding log metadata (name, level, message, timestamp, thread info, extras)
• Logger — Core logger with level filtering and handler dispatch
• LoggerManager — Hierarchical logger registry with parent-child relationships

Fast Logger (src/fast_logger.rs)

• Lock-free implementation using atomic operations
• Optimized for high-performance scenarios where mutex contention is a concern
• Uses AtomicU8 for fast level checking

FastLoggerWrapper (logxide/fast_logger_wrapper.py)

Python-side optimization wrapper that intercepts logging calls before they cross the PyO3 boundary:

Python call → FastLoggerWrapper.info() → level check (Python) → [skip if disabled]
                                                               → [delegate to Rust if enabled]

• 2-5x speedup for disabled log calls by avoiding:
  • PyObject creation for messages
  • PyTuple/PyDict packaging for args/kwargs
  • PyO3 boundary crossing overhead
• Caches getEffectiveLevel() on the Python side and invalidates on setLevel()/addHandler()/removeHandler()
• Transparent delegation: all non-hot-path attributes fall through to the underlying Rust PyLogger via __getattr__

Handlers (src/handler.rs)

All handlers implement the synchronous Handler trait:

pub trait Handler: Send + Sync {
    fn emit(&self, record: &LogRecord);
    fn flush(&self);
}

Handler	Description	I/O Strategy
StreamHandler	stdout/stderr output	crossbeam 채널 + 백그라운드 스레드 (논블로킹)
FileHandler	File output	동기 직접 write (Mutex<BufWriter>)
RotatingFileHandler	Auto-rotating files	동기 직접 write + size-based rotation
HTTPHandler	HTTP log shipping	crossbeam 채널 + 백그라운드 스레드 (배치)
OTLPHandler	OpenTelemetry OTLP	crossbeam 채널 + 백그라운드 스레드 (Protobuf)
MemoryHandler	In-memory capture	동기 Vec::push (Mutex)
NullHandler	Discards all logs	Zero overhead

Non-blocking Handlers (Stream/HTTP/OTLP)

StreamHandler, HTTPHandler, OTLPHandler use the channel + background thread pattern:

Logger → emit() → crossbeam-channel sender → Background thread → I/O output

• emit() formats the message and sends it to a bounded crossbeam-channel. The send itself doesn't block the caller, but the queue has finite capacity: when the worker can't keep up, the handler applies its overflow policy and records can be dropped (counted in get_metrics() as queue_dropped). StreamHandler defaults to drop_newest; HTTPHandler/OTLPHandler default to block, which back-pressures the producer up to the handler's flush timeout and drops only if that timeout expires. The GIL-released fast path is the one case where a block send waits indefinitely.
• A dedicated background thread performs actual I/O
• HTTP/OTLP handlers additionally batch records before sending

Synchronous Handlers (File/RotatingFile)

FileHandler, RotatingFileHandler use direct synchronous writes:

Logger → emit() → Mutex<BufWriter<File>> → write + conditional flush

• emit() acquires a Mutex lock and writes directly to BufWriter
• Level-based flush: records at ERROR or above trigger immediate flush()
• Simpler and faster for single-thread-dominant workloads

Formatters (src/formatter.rs)

• PythonFormatter (RustFormatter from Python): %(name)s percent-style fields, optional zero-pad/width, strftime-style datefmt, and a per-second default asctime cache
• {}/$ styles exist only as Python classes (compat_handlers.py); handlers using them dispatch through Python instead of the Rust formatter
• Direct ANSI color support: Support for terminal coloring placeholders (%(ansi_level_color)s, %(ansi_reset_color)s) is natively absorbed into PercentStyle (RustFormatter/Formatter), eliminating the runtime overhead of delegating to a separate ColorFormatter wrapper.

Filters (src/filter.rs)

• Name-based filtering matching Python's logging.Filter
• Hierarchical name matching (e.g., "myapp" matches "myapp.database")

Logger Names (src/fast_logger.rs)

• Each FastLogger stores its name as a single Arc<str> clone instead of a String per record.
• No message-text caching: the dynamic log message body (text) is not cached. The former dead string-cache module (src/string_cache.rs) was removed outright; its remaining useful behavior lives on as this one-field Arc<str> name.

Handler Architecture

┌─────────────────────┐
│   PyLogger          │
│   (per-logger       │
│    handler list)    │
├─────────────────────┤
│ ┌─────────────────┐ │
│ │ FileHandler     │ │─── Mutex<BufWriter> → 동기 직접 write
│ └─────────────────┘ │
│ ┌─────────────────┐ │
│ │ StreamHandler   │ │─── crossbeam-channel → Background thread → stderr/stdout
│ └─────────────────┘ │
│ ┌─────────────────┐ │
│ │ HTTPHandler     │ │─── crossbeam-channel → Background thread → HTTP batch
│ └─────────────────┘ │
│ ┌─────────────────┐ │
│ │ OTLPHandler     │ │─── crossbeam-channel → Background thread → OTLP Protobuf
│ └─────────────────┘ │
└─────────────────────┘

Each logger maintains its own handler list. When logger.addHandler() is called with a Rust handler, it is stored in the logger's local handler list. Global handlers configured via basicConfig() are also supported.

Handler routing and formatting (0.2.0)

Handlers route by backend kind, and each is dispatched exactly once:

• Structured sinks (HTTPHandler, OTLPHandler) serialize the record in Rust (JSON / protobuf), so extra fields are preserved.
• Text-sink wrappers (FileHandler, StreamHandler, RotatingFileHandler, MemoryHandler) format the line via their Python emit() override, which is what makes formatted output and pytest capture behave correctly.
• Foreign Python handlers run once on the Python side.

A Rust-backed handler attached to one logger no longer double-emits or leaks records to unrelated loggers, and removeHandler() / clear_handlers() / close() tear down routing and the background worker together.

GIL Scope

LogXide releases the GIL for the Rust dispatch only on the fast path: a record that hits no Python filter, no Python handler, and no caller-info field. On that path, field extraction happens under the GIL and the subsequent Rust dispatch (formatting + I/O for Rust-native handlers) runs with the GIL released. A %-args call re-acquires the GIL inside emit() to run % formatting, and any Python handler/filter or caller-info collection also holds the GIL. On current CPython GIL builds, producer throughput does not scale linearly across threads because the fast path shares a handler mutex and sink I/O is serialized; free-threaded builds need separate verification.

Thread Safety

• Mutex-protected handlers — parking_lot::Mutex for handler state
• Thread-safe logger registry — DashMap for concurrent logger access
• Atomic level checks — AtomicU8 for fast level filtering without locks
• Background-thread I/O — stream/HTTP/OTLP writes run on a worker thread that does not hold Python's GIL

Memory Management

• Rust ownership prevents memory leaks
• Arc-based sharing for handlers and formatters across loggers
• BufWriter with 64KB buffers reduces syscall overhead for file handlers
• Arc<str> logger name shared per logger instead of per-record string clones

Comparison with Standard Logging

Aspect	Python logging	LogXide
Implementation	Pure Python	Native Rust via PyO3
Handler calls	Python method dispatch	Direct Rust function calls
String formatting	Python string operations	Rust native formatting
Thread safety	Global lock (_lock)	Per-handler mutexes
I/O	Python file objects	Direct OS I/O (bypasses GIL)
Custom handlers	Unlimited (Python subclasses)	Rust native handlers (fast path); Python subclasses also accepted, run once on the Python side
subclassing	Full support	Not supported

Dependencies

Crate	Purpose
pyo3	Python-Rust bindings
chrono	Timestamp formatting
parking_lot	Fast mutexes
dashmap	Concurrent logger map
crossbeam-channel	Stream/HTTP/OTLP handler channels
ureq	HTTP requests (HTTPHandler)
serde / serde_json	JSON serialization
prost / opentelemetry-proto	OTLP Protobuf encoding

Format string parsing

The active formatter hot path uses a single-pass O(N) parser built once at formatter construction (itoa handles numeric fields); it does not invoke the regex crate for %(field)s field extraction, and the regex crate is not a dependency of this crate.