Writing Logs to Files in Node.js with the LogLayer

Writing Logs to Files in Node.js with LogLayer Writing logs to files is a crucial aspect of application monitoring and debugging in production environments. In this article, we'll explore how to effectively manage file-based logging in Node.js applications using LogLayer and the Log File Rotation Transport, a powerful and flexible logging library. What is LogLayer? LogLayer is a modern, TypeScript-first logging library that provides a clean and intuitive API for structured logging, and it acts as a layer on top of your favorite logging library like winston or pino. Why LogLayer for File Logging? LogLayer provides a robust solution for file-based logging with several advantages: Automatic log file rotation based on size or time Support for log compression Batching capabilities for improved performance Flexible log formatting Automatic cleanup of old log files Getting Started First, install the necessary packages: npm install loglayer @loglayer/transport-log-file-rotation serialize-error Basic File Logging Setup Here's a basic example of setting up file logging: import { LogLayer } from "loglayer"; import { LogFileRotationTransport } from "@loglayer/transport-log-file-rotation"; import { serializeError } from "serialize-error"; const logger = new LogLayer({ errorSerializer: serializeError, transport: [ new LogFileRotationTransport({ filename: "./logs/app.log" }), ], }); // Start logging! logger.withMetadata({ port: 3000, }).info("Application started"); logger .withError(new Error("Something went wrong")) .error("An error occurred"); Adding Context and Metadata LogLayer provides several ways to enrich your logs with additional information: Using Context Context is persistent data that applies to all subsequent log entries: import { hostname } from "node:os"; const logger = new LogLayer({ transport: [ new LogFileRotationTransport({ filename: "./logs/app.log" }), ], }); // Add context that will be included in all subsequent logs const contextLogger = logger.withContext({ hostname: hostname(), environment: process.env.NODE_ENV, version: process.env.npm_package_version }); contextLogger.info("Server starting"); // Will include context contextLogger.error("Connection failed"); // Will include same context Using Metadata Metadata is one-time data that only applies to the current log entry: // Add metadata for a single log entry logger.withMetadata({ requestId: "123", duration: 45, statusCode: 200 }).info("Request processed"); // Different metadata for different log entries logger.withMetadata({ userId: "user-456", action: "login" }).info("User logged in"); // Combine context and metadata const userLogger = logger.withContext({ service: "user-service" }); userLogger.withMetadata({ duration: 123, status: "success" }).info("Operation completed"); Static Data in Transport You can also configure static data at the transport level: const logger = new LogLayer({ transport: [ new LogFileRotationTransport({ filename: "./logs/app.log", staticData: { hostname: hostname(), environment: process.env.NODE_ENV, version: process.env.npm_package_version } }), ], }); This will produce logs like: { "level": "info", "message": "Request processed", "timestamp": "2024-01-17T12:34:56.789Z", "hostname": "prod-server-1", "environment": "production", "version": "1.0.0", "requestId": "123", "duration": 45, "statusCode": 200 } Advanced Configuration Daily Log Rotation For applications that need to rotate logs daily: const logger = new LogLayer({ transport: [ new LogFileRotationTransport({ filename: "./logs/app-%DATE%.log", frequency: "daily", dateFormat: "YMD", compressOnRotate: true, maxLogs: "7d" // Keep logs for 7 days }), ], }); Size-based Rotation For high-volume applications, rotating based on file size: const logger = new LogLayer({ transport: [ new LogFileRotationTransport({ filename: "./logs/app.log", size: "10M", // Rotate when file reaches 10 MB maxLogs: 5, // Keep last 5 log files compressOnRotate: true }), ], }); Performance Optimization with Batching For high-throughput applications, enable batching to reduce disk I/O: const logger = new LogLayer({ transport: [ new LogFileRotationTransport({ filename: "./logs/app.log", batch: { size: 1000, // Write after 1000 logs are queued timeout: 5000 // Or after 5 seconds, whichever comes first } }), ], }); Best Practices Use Rotation: Always configure log rotation to prevent disk space issues: new LogFileRotationTransport({ filename: "./logs/app-%DATE%.log", frequen

Mar 2, 2025 - 22:08

Writing Logs to Files in Node.js with the LogLayer

Writing Logs to Files in Node.js with LogLayer

Writing logs to files is a crucial aspect of application monitoring and debugging in production environments. In this article, we'll explore how to effectively manage file-based logging in Node.js applications using LogLayer and the Log File Rotation Transport, a powerful and flexible logging library.

What is LogLayer?

LogLayer is a modern, TypeScript-first logging library that provides a clean and intuitive API for structured logging, and it acts as a layer on top of your favorite logging library like winston or pino.

Why LogLayer for File Logging?

LogLayer provides a robust solution for file-based logging with several advantages:

Automatic log file rotation based on size or time
Support for log compression
Batching capabilities for improved performance
Flexible log formatting
Automatic cleanup of old log files

Getting Started

First, install the necessary packages:

npm install loglayer @loglayer/transport-log-file-rotation serialize-error

Basic File Logging Setup

Here's a basic example of setting up file logging:

import { LogLayer } from "loglayer";
import { LogFileRotationTransport } from "@loglayer/transport-log-file-rotation";
import { serializeError } from "serialize-error";

const logger = new LogLayer({
  errorSerializer: serializeError,
  transport: [
    new LogFileRotationTransport({
      filename: "./logs/app.log"
    }),
  ],
});

// Start logging!
logger.withMetadata({
  port: 3000,
}).info("Application started");

logger
  .withError(new Error("Something went wrong"))
  .error("An error occurred");

Adding Context and Metadata

LogLayer provides several ways to enrich your logs with additional information:

Using Context

Context is persistent data that applies to all subsequent log entries:

import { hostname } from "node:os";

const logger = new LogLayer({
  transport: [
    new LogFileRotationTransport({
      filename: "./logs/app.log"
    }),
  ],
});

// Add context that will be included in all subsequent logs
const contextLogger = logger.withContext({
  hostname: hostname(),
  environment: process.env.NODE_ENV,
  version: process.env.npm_package_version
});

contextLogger.info("Server starting");  // Will include context
contextLogger.error("Connection failed");  // Will include same context

Using Metadata

Metadata is one-time data that only applies to the current log entry:

// Add metadata for a single log entry
logger.withMetadata({
  requestId: "123",
  duration: 45,
  statusCode: 200
}).info("Request processed");

// Different metadata for different log entries
logger.withMetadata({
  userId: "user-456",
  action: "login"
}).info("User logged in");

// Combine context and metadata
const userLogger = logger.withContext({ service: "user-service" });

userLogger.withMetadata({
  duration: 123,
  status: "success"
}).info("Operation completed");

Static Data in Transport

You can also configure static data at the transport level:

const logger = new LogLayer({
  transport: [
    new LogFileRotationTransport({
      filename: "./logs/app.log",
      staticData: {
        hostname: hostname(),
        environment: process.env.NODE_ENV,
        version: process.env.npm_package_version
      }
    }),
  ],
});

This will produce logs like:

{
  "level": "info",
  "message": "Request processed",
  "timestamp": "2024-01-17T12:34:56.789Z",
  "hostname": "prod-server-1",
  "environment": "production",
  "version": "1.0.0",
  "requestId": "123",
  "duration": 45,
  "statusCode": 200
}

Advanced Configuration

Daily Log Rotation

For applications that need to rotate logs daily:

const logger = new LogLayer({
  transport: [
    new LogFileRotationTransport({
      filename: "./logs/app-%DATE%.log",
      frequency: "daily",
      dateFormat: "YMD",
      compressOnRotate: true,
      maxLogs: "7d" // Keep logs for 7 days
    }),
  ],
});

Size-based Rotation

For high-volume applications, rotating based on file size:

const logger = new LogLayer({
  transport: [
    new LogFileRotationTransport({
      filename: "./logs/app.log",
      size: "10M", // Rotate when file reaches 10 MB
      maxLogs: 5, // Keep last 5 log files
      compressOnRotate: true
    }),
  ],
});

Performance Optimization with Batching

For high-throughput applications, enable batching to reduce disk I/O:

const logger = new LogLayer({
  transport: [
    new LogFileRotationTransport({
      filename: "./logs/app.log",
      batch: {
        size: 1000,    // Write after 1000 logs are queued
        timeout: 5000  // Or after 5 seconds, whichever comes first
      }
    }),
  ],
});

Best Practices

Use Rotation: Always configure log rotation to prevent disk space issues:

   new LogFileRotationTransport({
     filename: "./logs/app-%DATE%.log",
     frequency: "daily",
     maxLogs: "30d",
     compressOnRotate: true
   })

Enable Batching for High Volume: For applications with high log volume:

   new LogFileRotationTransport({
     filename: "./logs/app.log",
     batch: {
       size: 1000,
       timeout: 5000
     }
   })

Separate Logs by Concern: Use different transports for different types of logs:

   const logger = new LogLayer({
     transport: [
       new LogFileRotationTransport({
         filename: "./logs/app.log"
       }),
       new LogFileRotationTransport({
         filename: "./logs/errors.log",
         levelMap: {
           error: "ERROR",
           fatal: "FATAL"
         }
       })
     ],
   });

Use Context and Metadata Appropriately:

   // Use context for values that remain constant
   const serviceLogger = logger.withContext({
     service: "user-service",
     version: "1.0.0"
   });

   // Use metadata for request-specific information
   serviceLogger.withMetadata({
     requestId: "req-123",
     duration: 45,
     statusCode: 200
   }).info("Request completed");

Error Handling

Proper error serialization is crucial for debugging:

import { serializeError } from "serialize-error";

const logger = new LogLayer({
  errorSerializer: serializeError,
  transport: [
    new LogFileRotationTransport({
      filename: "./logs/app.log"
    }),
  ],
});

try {
  throw new Error("Something went wrong");
} catch (error) {
  logger.withError(error).error("Operation failed");
}

Conclusion

LogLayer provides a powerful and flexible solution for file-based logging in Node.js applications. With features like automatic rotation, compression, and batching, it helps you maintain clean and efficient logs while ensuring you don't miss important information for debugging and monitoring.

For more detailed information about LogLayer's file rotation transport, check out: