Setting up Serilog in ASP.NET Core – Detailed Beginner Guide

This article will cover what is Serilog along with its concepts and how to implement Serilog in ASP.NET Core. ASP.NET Core has a built-in logging API that supports a wide variety of logging providers. Serilog is one of the most popular logging frameworks which can be plugged into .NET Core based applications.

Logging providers work by storing logs in some destination like files or databases. The in-built logging API is available in Microsoft.Extensions.Logging namespace and works mostly all in-built and third-party logging providers.

Serilog provides libraries for logging application information into the console, files, etc. Serilog supports structured logging which is a very popular format when it comes to reading these log information using some tool.

Need for logging

Logging should be an integral part of every application. There will always need to troubleshoot application issues for which logs will be required to identify the root cause of the issue being reported.

When something breaks in production then the first thing we do is try and get information about what went wrong and our primary source of information is logs during that point in time. So it becomes very important to log all error events by default and if needed there should be an option to selectively enable additional logs for debugging purposes.

Logging is also required for logging of performance indicators i.e. when we need some profiling to be done for our code to identify slow working code which needs some improvements.

Requirements for a good logging framework

We should be able to fix errors by simply looking at the logs i.e. without the need to simulate errors in development/UAT or asking users what did they do? For this, all the details should be available in logs i.e. a sequence of events along with data used for the requests.

From logs, we should be able to understand which errors are occurring on the production as there can be some errors that users might not be reporting but they are occurring on the production i.e. to identify all the exceptions occurring in our application and over time ensure that our application logs are error-free.

We should be able to prioritize defect fixing based on the criticality of the exceptions logged in logs. This is sample data.

From performance indicators being logged, we should be able to identify which transactions/requests needs performance improvements.

Log data should be easily accessible, readable, efficient & usable i.e. log should be structured which will allow the log reader to query data from the log store using named values.

You should not write sensitive information like passwords, user data, etc. to the logs.

What is Structured Logging?

Structured logging is essential for log data to be efficient & usable. Consider we have an application running on multiple servers and on each server we have a text-based file logging. Now during problem or exception times, it will be difficult to go to each server and manually read those files to identify the issue. This is not only time consuming and there can be mistakes in scanning the log files manually as there are chances are skipping errors if log files are very huge.

Consider you are writing a performance counter i.e. time taken by one of your important requests to log file now manually reading these performance counters for each request and identifying average or max/min is difficult instead it should be possible to query the logs on this performance counter and generate a report out of it to understand it’s throughput during peak loads.

Structured logging makes this querying of log files possible which makes it both an efficient and usable way of logging data. Serilog in ASP.NET Core provides a structured logging framework.

Have a look at the code line for the logging performance counter shown below

Log.Information("Time taken by order request is {timeTaken}", profiler.timeTaken);

Here {timeTaken} is the named property and profiler.timeTaken is the value for that property. Named properties should have values as these named properties and their values will be stored in the configured data store that allows them to be queried. In Serilog in ASP.NET Core structured logging, it will be possible to query logs using this named property {timeTaken}

Datastores i.e. Sinks that are not capable of storing structured data will save this information as a string.

Introduction to Serilog

There are many third-party providers for logging in to ASP.NET Core and one of these is Serilog in ASP.NET Core. Serilog is a popular third party logging provider that plugs into ASP.NET Core and uses .NET Core standard API for logging. Since Serilog supports ASP.NET Cores default logging APIs it can receive log events from ASP.NET Core framework libraries as well.

Serilog in ASP.NET Core is very easy to set up and integrate. Serilog provides a structured logging framework and supports a wide variety of sinks to log to console, files, azure, etc.

Serilog is extensible which allows developers to extend the functionality of Serilog by adding their own custom codes as per their requirements.

Serilog makes it easier to configure multiple logging data stores to store log information. Serilog was built with the goal of structured logging as the target.

Introduction to Serilog Sinks

Serilog Sinks are used to writing log events in various formats. Sinks are another name for data store using which we specify where we want to store log information for our application. The most common and popular sinks are consoles or files which are easy to configure. There are other sinks as well which support structured logging and allow the saving of name properties and their values.

Sinks are configured along with Serilog configuration at the start of the project in the Startup class. Here is the list of some common Sinks supported by Serilog in ASP.NET Core

• Serilog.Sinks.ApplicationInsights
• Serilog.Sinks.AzureTableStorage
• Serilog.Sinks.Console
• Serilog.Sinks.Elasticsearch
• Serilog.Sinks.ElmahIo
• Serilog.Sinks.Email
• Serilog.Sinks.File
• Serilog.Sinks.GoogleCloudLogging
• Serilog.Sinks.InMemory
• Serilog.Sinks.Log4Net
• Serilog.Sinks.Loggly
• Serilog.Sinks.MongoDB
• Serilog.Sinks.NLog
• Serilog.Sinks.PostgreSQL
• Serilog.Sinks.RabbitMQ
• Serilog.Sinks.RollingFile
• Serilog.Sinks.Seq
• Serilog.Sinks.SignalR
• Serilog.Sinks.SQLite
• Serilog.Sinks.Stackify

Serilog in ASP.NET Core provides sinks for writing log events to storage in various formats.

Default logging in ASP.NET Core

When you create ASP.NET Core MVC or API application by default in the program.cs it will add an entry to the CreateDefaultBuilder function which will be invoked on application start and this will register the default logger along with other stuff. The out of box logging level for this default logging will be configured in the appsettings.json file.

To demonstrate this default logging provided out of the box in ASP.NET Core I have created a project of the type ASP.NET Core Web API as shown below.

Below is the default code generated in the program.cs file which shows code for CreateHostBuilder.

public class Program
{
    public static void Main(string[] args)
    {
        CreateHostBuilder(args).Build().Run();
    }

    public static IHostBuilder CreateHostBuilder(string[] args) =>
        Host.CreateDefaultBuilder(args)
            .ConfigureWebHostDefaults(webBuilder =>
            {
                webBuilder.UseStartup<Startup>();
            });
}

Below is the out of box configuration for default logging in the appsettings.json file

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft": "Warning",
      "Microsoft.Hosting.Lifetime": "Information"
    }
  },
  "AllowedHosts": "*"
}

By default, this logging configuration will write logs to the console window. To check these logs in the console window you will have to run the application in Visual studio under the Kestrel web server instead of IIS Express. For this, you will have to select the name of the application options under the launch dropdown in Visual Studio as shown below

To use this default logger we can simply add a logger object to the constructor of the controller and start logging using this object. In default WetherForecastController.cs this logger object is injected through constructor using dependency injection container. We will make use of this logger object and write the information log when the Get method is called as shown in the code below.

[ApiController]
[Route("[controller]")]
public class WeatherForecastController : ControllerBase
{
    private static readonly string[] Summaries = new[]
    {
        "Freezing", "Bracing", "Chilly", "Cool", "Mild", "Warm", "Balmy", "Hot", "Sweltering", "Scorching"
    };

    private readonly ILogger<WeatherForecastController> _logger;

    public WeatherForecastController(ILogger<WeatherForecastController> logger)
    {
        _logger = logger;
    }

    [HttpGet]
    public IEnumerable<WeatherForecast> Get()
    {
        _logger.LogInformation("In Weather Forecast Get Method");
        var rng = new Random();
        return Enumerable.Range(1, 5).Select(index => new WeatherForecast
        {
            Date = DateTime.Now.AddDays(index),
            TemperatureC = rng.Next(-20, 55),
            Summary = Summaries[rng.Next(Summaries.Length)]
        })
        .ToArray();
    }
}

Code Line 20 – Using logger object writes information “In Weather Forecast Get Method” to log data store which is the console in this case. This log info will be written when the Get method on WeatheForecastController is invoked.

Now let’s run the application and call the Get method of WeatheForecastController to write the information log to the console window as shown below

Let’s add our custom controller to simulate an exception condition & see how that error gets logged to the console window. We will add MathController that has the potential to generate an exception in the divide method when the number is divided by 0 as shown below

[Route("[controller]")]
[ApiController]
public class MathController : ControllerBase
{
    private readonly ILogger<MathController> _logger;

    public MathController(ILogger<MathController> logger)
    {
        _logger = logger;
    }

    [HttpGet]
    public decimal Divide(decimal a, decimal b)
    {
        try
        {
            return (a / b);
        }
        catch (Exception ex)
        {
            _logger.LogInformation("Error in Divide Method - Value of a is {a}", a);
            _logger.LogInformation("Error in Divide Method - Value of b is {b}", b);
            _logger.LogError(ex, "Error in Divide Method");
            return 0;
        }
    }
}

Now let’s run the code and call the divide method in the math controller using values 5 & 0 then it should generate an exception Divide by Zero & details should be written to the console window as shown below

Log Levels

Log levels are basically a type of log entry that is made into the log data store. Whenever we write logs we specify the log type i.e. level of the log. You can even say that it is a grouping of the logs by the level & level that defines the priority of the logs in which they should be addressed.

It is always a good practice for the developer to use the appropriate log level while writing the logs. By using this log level we can control the size of logs that should be written for an application as these log levels follow a hierarchy i.e. log levels can be used to control/filter the logs that should be written.

When we try to write a Log of type/level Warning then it will be written only if the log level enabled is either equal to Warning or greater than Warning i.e. Information or Debug or Trace/Verbose.

Here is the list of log levels that are available in Serilog in ASP.NET Core

• Verbose/Trace – tracing information and debugging; generally only switched on in unusual situations
• Debug – internal control flow and diagnostic state dumps to facilitate pinpointing of recognized problems
• Information – events of interest or that have relevance to outside observers; the default enabled minimum logging level
• Warning – indicators of possible issues or service/functionality degradation
• Error – indicating a failure within the application or connected system
• Fatal/Critical – critical errors causing complete failure of the application

General Hierarchy – Verbose > Debug > Information > Warning > Error > Fatal. The enabled log level will print logs for the enabled level & for all levels on its right.

Implement Serilog in ASP.NET Core

Here is a quick & short video on the implementation of Serilog in ASP.NET Core

Install required NuGet Packages for Serilog & Sinks.

Below are Nuget packages which will be installed for the demonstration of Serilog in ASP.NET Core.

• Serilog.AspNetCore
• Serilog.Settings.Configuration
• Serilog.Sinks.File
• Serilog.Sinks.Console

Now we have installed the required packages let’s configure Serilog in ASP.NET Core

Configure Serilog in ASP.NET Core

Now let’s replace the default logging with Serilog which will write logs to the console window.

In appsettings.json remove the default logging settings and replace them with Serilog settings as shown below

{
  "AllowedHosts": "*",

  "Serilog": {
    "MinimumLevel": "Information",
    "Override": {
      "Microsoft.AspNetCore": "Warning"
    },
    "WriteTo": [
      {
        "Name": "Console"
      }
    ]
  }
}

In the startup.cs add code to read these Serilog settings in the constructor as shown below

public Startup(IConfiguration configuration)
{
    Configuration = configuration;

    Log.Logger = new LoggerConfiguration()
        .ReadFrom.Configuration(configuration)
        .CreateLogger();
}

The above code will create Serilog Logger with the settings specified in the appsettings.json file and also add sinks as per the configuration specified. By specifying the configuration in the appsettings.json file we will be able to change the configuration or add/remove sinks without code changes.

Now change the code in the program.cs to specify Host Builder to use Serilog in ASP.NET Core instead of default logger.

public static IHostBuilder CreateHostBuilder(string[] args) =>
    Host.CreateDefaultBuilder(args)
        .UseSerilog()
        .ConfigureWebHostDefaults(webBuilder =>
        {
            webBuilder.UseStartup<Startup>();
        });

After making the above changes run the application & check the logs written to the console. Also, simulate the error condition again by calling the divide method in the math controller using values 5 & 0. Now logs from Serilog in ASP.NET Core should be like as shown below

So far we covered how to replace the default logger with Serilog. Now let’s add one more Sink File i.e. along console logs will also be written to the file.

Configure Serilog logging to file

Let’s configure Serilog in ASP.NET Core to make changes to the appsettings.json file in the Serilog settings block to include one more sink i.e. File which will create Serilog logger with a file sink to write logs details to the file as well.

{
  "AllowedHosts": "*",

  "Serilog": {
    "MinimumLevel": "Information",
    "Override": {
      "Microsoft.AspNetCore": "Warning"
    },
    "WriteTo": [
      {
        "Name": "Console"
      },
      {
        "Name": "File",
        "Args": {
          "path": "Serilogs\AppLogs.log"
        }
      }
    ]
  }
}

As shown above, the File sink requires an argument where we need to specify the log file name along with the path of the file. This path can be an absolute or relative path. Here I have specified a relative path that will create folder serilogs in the application folder and write to the file AppLogs.log in that folder.

After running the application & generating error conditions will write logs to the console & file as well. Here is the screenshot of the logs written to the file.

Configure Serilog Structured Logging

To configure structured logging using Serilog in ASP.NET Core we will make use of the File sink and write logs to files in the JSON format. We will make use of JSON formatter to write log data to the file. For this, we will make changes to the appsettings.json file in the Serilog settings block to include one more sink i.e. File and add JSON Formatter as a parameter to the settings.

{
  "AllowedHosts": "*",

  "Serilog": {
    "MinimumLevel": "Information",
    "Override": {
      "Microsoft.AspNetCore": "Warning"
    },
    "WriteTo": [
      {
        "Name": "Console"
      },
      {
        "Name": "File",
        "Args": {
          "path": "Serilogs\AppLogs.log"
        }
      },
      {
        "Name": "File",
        "Args": {
          "path": "Serilogs\AppJSONLogs.log",
          "formatter": "Serilog.Formatting.Json.JsonFormatter, Serilog"
        }
      }
    ]
  }
}

After running the application & generating error conditions will write logs to the console, text file & JSON formatted file as well. Here is the screenshot of the logs written to a JSON formatted file.

How to use Serilog Enrichers

Serilog in ASP.NET Core uses enrichers to handle the information that is part of the log entry. Some additional information like Machine Name, Assembly Name, HTTP Request Path, etc. is available with Serilog Enrichers which can be included as part of each log entry.

You can also define your own custom enrichment for fields like session id, user, cookie, etc. that you want to be part of the log entry for better analysis of the log entries that are stored.

To add fields from enrichers in our application logs we need to add the NuGet packages mentioned below in our project to get details from the environment of the process and thread.

• Serilog.Enrichers.Environment
• Serilog.Enrichers.Process
• Serilog.Enrichers.Thread

After adding the above NuGet packages configure enricher in Serilog setting in appsettings.json file as shown below

{
  "AllowedHosts": "*",

  "Serilog": {
    "MinimumLevel": "Information",
    "Override": {
      "Microsoft.AspNetCore": "Warning"
    },
    "WriteTo": [
      {
        "Name": "Console"
      },
      {
        "Name": "File",
        "Args": {
          "path": "Serilogs\AppLogs.log"
        }
      },
      {
        "Name": "File",
        "Args": {
          "path": "Serilogs\AppLogs.json",
          "formatter": "Serilog.Formatting.Json.JsonFormatter, Serilog"
        }
      }
    ],
    "Enrich": [
      "FromLogContext",
      "WithMachineName",
      "WithProcessId",
      "WithThreadId"
    ]
  }
}

After the above changes run the application & generate error conditions to write logs to console, text file & JSON formatted file as well. Here is the screenshot of the comparison of logs (without enrichers & with enrichers) written to a JSON formatted file.

You can use Serilog.Enrichers.AspnetcoreHttpcontext NuGet Package to add properties from HttpContext to the log entry. You can try this out and if you did any help around this then feel free to reach out to me by making use of the comments section below.

Log Message Output Templates

Text-based sinks like Console or File which write log entries as plain text support property outputTemplate which allows us to control how the logs event data is formatted. We can modify the format of the log events written by Sink to the Console or File.

For this, we need to specify the property outputTemplate with the value that contains the properties for the format in which the entry is to be written. Supported properties in outputTemplate in Serilog in ASP.NET Core are as follows

• Message – log event’s message, rendered as plain text
• Exception – full exception message and stack trace, formatted across multiple lines
• Level – log event level, formatted as the full level name
• NewLine – property with the value of System.Environment.NewLine.
• Properties – event property values that don’t appear elsewhere in the output
• TimeStamp – event’s timestamp

For example, we can specify outputTemplate for Console Sink as shown below

{
  "Name": "Console",
  "Args": {
    "outputTemplate": "{Timestamp:HH:mm:ss.fff zzz} [{Level}] {Message}{NewLine}{Exception}",
  }
}

After making the above changes run the application & check the logs written to the console. Also, simulate the error condition again by calling the divide method in the math controller using values 5 & 0. Now logs from Serilog in ASP.NET Core should be like as shown below

Log Filtering

So far we have seen how to write log entries into the configured sinks. We have been logging all the entries to the configured sinks without applying any filters. Now we will see how to filter different log entries so that we have control over what is being written to log files.

If you are using more than one sink then you will be able to specify different filtering criteria for each sink.

For a demonstration of filtering what I have done is added one sink File and in that file, I am filtering i.e. excluding all the log events which have been by SourceContext starting with Microsoft. i.e. all the log events from namespace starting with Microsoft. have been excluded from the new File Sink.

For Log filters to work you have to add NuGet Package Serilog.Expressions. After adding this package we will configure a new file sink in Serilog settings in the appsettings.json file as shown below

{
  "Name": "Logger",
  "Args": {
    "configureLogger": {
      "WriteTo": [
        {
          "Name": "File",
          "Args": {
            "path": "Serilogs\api.log",
            "outputTemplate": "{Timestamp:HH:mm:ss.fff zzz} [{Level}] [{SourceContext}] {Message}{NewLine}{Exception}"
          }
        }
      ],
      "Filter": [
        {
          "Name": "ByExcluding",
          "Args": {
            "expression": "StartsWith(SourceContext, 'Microsoft.')"
          }
        }
      ]
    }
  }
}

After making the above changes run the application & check the logs written to the new file sink. Also, simulate the error condition again by calling the divide method in the math controller using values 5 & 0. Now logs from Serilog in ASP.NET Core in the new filtered sink should be like as shown below

Summary

Logging should be part of every application. ASP.NET Core Structured logging needs to be configured before using it in your application. ASP.NET Core Structured logging has inbuilt log providers and also many third-party log providers are supported.

Serilog in ASP.NET Core is a third party structured log provider that is supported in .NET Core logging. Serilog in ASP.NET Core provides basic diagnostic logging that supports a wide variety of sinks & enrichers to decorate log entries with additional information.

Source Code Download

Here you can download the complete source code for this article demonstrating how to implement Serilog in ASP.NET Core

https://github.com/procodeguide/ProCodeGuide.Samples.Serilog

References: https://docs.microsoft.com/en-us/aspnet/core/fundamentals/logging/?view=aspnetcore-3.1 & https://serilog.net/

You can also check my other Article on ASP.NET Core Caching: https://procodeguide.com/programming/aspnet-core-caching/

Hope you found this article useful. Please support the Author