Skip to content

Centralized Log

The documentation applies to:✅ v0.8.0

Distributed Systems problem

Let's take a look in the problem when we develop Distributed Systems

Distributed System Log

Popular solution for this problem above is we need to use Correlation ID which helps us to gather all logs from service to service. Gateway will add X-Correlation-ID in HTTP Request and down it to all calling services. Then later you can use this ID for querying in a Centralized Log. So there are our solution:

Correlation Id

Frequency of log

Log, likes as software's heartbeat, exposes your software weakness. In distributed systems, log will be sent everytime to Centralized Log. Because you shouldn't allow to lose any log in the transaction.

However, LET Portal is designed for small systems, so it is very important in term of Performance when all services always send the log into Centralized Log. It may become unpredictable in Production with many requests come in and out. So that LET Portal reduces a frequency of log by changing from Active mode to Passive mode.

  • Active Mode: Each service always send log to Centralized Log server
  • Passive Mode: Service sends log to Centralized Log server based on condition. If nothing matches, service keeps log in local storage (Disk, Database). Later, admin needs to trigger Gather Logs action to get all logs in local storage per service

Passive Mode

Serilog

LET Portal chooses Serilog for writing because it is flexible data. You can open Files\Shared\v1.0\appsettings.json for Serilog Configuration.

{
"Serilog": {
    "Using": [ "Serilog.Sinks.Console", "Serilog.Sinks.MongoDB", "Serilog.Sinks.File" ],
    "MinimumLevel": "Information",
    "WriteTo": [
      {
        "Name": "Console"
      },
      {
        "Name": "MongoDBCapped",
        "Args": {
          "databaseUrl": "mongodb://localhost:27017/logs",
          "collectionName": "servicelogs"
        }
      }
    ],
    "Enrich": [ "FromLogContext" ]
  }
}

Serilog will store log in MongoDB as you see. In fact, it is a centralized log database, so that means all service logs will be written in one place. That's our limitation in this time.

Limitation

We are trying to decouple a MongoDB by file. However, this way requires more effort than writing directly in Database. Beside, MongoDB is good way to store the log because it is schemaless.

Next, if you want to use Serilog to write, you need to inject IServiceLogger<T> into your class constructor

 public class ChartsController : ControllerBase
    {
        // Use this logger for writing log
        private readonly IServiceLogger<ChartsController> _logger;

        public ChartsController(
            IServiceLogger<ChartsController> logger)
        {
            _logger = logger;
        }

        public IActionResult Get()
        {
            // Simple log
            _logger.Info("New message");

            // Log with object
            _logger.Info("Found chart: {@result}", new Chart{ Name = "abc" });
            return Ok();
        }
    }

Enable Notify Log

First, you need to enable Serilog in Startup.cs

services.AddLetPortal(Configuration, options =>
            {
                options.EnableMicroservices = true;
                options.EnableSerilog = true;
            })

Second, you add UseLetPortal in Configure method in Startup.cs.

app.UseLetPortal(options =>
  {
      options.EnableCheckUserSession = true;
      options.EnableCheckTraceId = true;
      options.EnableWrapException = true;
      options.SkipCheckUrls = new string[] { 
          // Your ignore urls, for example: "api/accounts/login"
      };
  });

Third, you can enable Notify Log based on HTTP Response Statuscode in LoggerOptions. See Add New Service

{
  "LoggerOptions": {
    "NotifyOptions": {
      "Enable": true, // Enable Notify Log, it will push log to Service Management
      "StatusCodes": [ 500 ], // Condition Statuscode
      // Condition for specific url and status code
      "Urls": [
        {
          "Enable" : true,
          "UrlPath": "api/accounts/login",
          "StatusCodes": [ 200, 204, 500]
        }
      ]
    }
  }
}

Query Log

You can open LET Portal, go to Service Logs page.

Centralized Log UI