Contents
In the previous article, I introduced how to create Api Controller for CURD. In this time, we will take a look how to use the Serilog in the .Net core!
1. Serilog Introduction
1.1 What is Serilog?
Serilog is a popular logging framework for .NET applications. Here is a quick introduction to Serilog:
- Serilog is a structured logging library that allows writing log messages in JSON format. This makes the logs easy to read by machines/computers.
It provides a flexible logging API that allows configuring and integrating with different storage backends like files, databases, etc.
Serilog provides useful enrichers to add useful info like timestamps, thread ids, etc to log messages.
It has support for structured logging, which means logging objects directly instead of just text messages. This provides more useful logging info.
1.2. Why Serilog?
Structured logging – Serilog logs are structured as JSON which makes them easy to consume. This is useful for log aggregation/analysis.
Multiple sinks – Serilog supports writing logs to multiple sinks like file, database, Azure services etc. This makes it flexible to route and store logs.
Enrichment – Serilog provides enrichers to augment log events with useful info like timestamps, thread ids, machine names etc.
Strongly typed – Serilog is strongly typed, so you get compile time safety while logging events and objects.
Customizable and extensible – Serilog is highly customizable via configuration. You can write custom sinks and enrichers.
Framework independent – Serilog can be used outside ASP.NET too, for example in console apps, services etc.
Decoupled architecture – The logging pipeline is decoupled from the logging API surface. This makes it flexible.
Structured logging – Serilog supports structured logging i.e. logging objects rather than just text messages.
High performance – Serilog is optimized for performance and allocates less objects.
Popular and mature – Serilog is very popular, well-supported and mature logging library.
Do you want to be a good trading in cTrader? >> TRY IT! <<
2. Using Serilog
2.1 Install
We can easy to install Serilog from NuGet, for good to use it, I will recommend to install below packages
Serilog.AspNetCore
Serilog.Settings.Configuration
Serilog.Sinks.MSSqlServer
Serilog.Filters.Expressions
if you don’t want to save log to database, then don’t need to install Serilog.Sinks.MSSqlServer, just install the above to the Api project.
2.2. Setup Serilog
There are two ways for setup the Serilog, but for now, I will show you how to setup in Program.cs first
Add the below codes in Program.cs, this is the base setting, it will print all of the logs in console
builder.Host.UseSerilog((context, logConfig) => logConfig
.ReadFrom.Configuration(context.Configuration)
.WriteTo.Console());
But, since Serilog is so powerful, of course, We will not satisfy these basic usage! There are some requirements what I want to do:
1) We want to split the debug and error log to difference log file. So we can use the log level for filter
//for debug log
.WriteTo.Logger(l => l.Filter.ByIncludingOnly(e => e.Level == LogEventLevel.Warning))
//for error log
.WriteTo.Logger(l => l.Filter.ByIncludingOnly(e => e.Level == LogEventLevel.Error))
There are 6 logging levels can be use:
a) Verbose – Anything and everything you might want to know about a running block of code. For the most detailed logs. Useful for troubleshooting.
b) Debug – Internal system events that aren’t necessarily observable from the outside. For debugging information. Logs that are useful during development.
c) Information – The lifeblood of operational intelligence – things happen. For tracking general flow of the application. Usually high volume logs.
d) Warning – Service is degraded or endangered. For logs that indicate a potential problem. But the application can continue running.
e) Error – Functionality is unavailable, invariants are broken or data is lost. For exceptions or errors that require investigation.
f) Fatal – If you have a pager, it goes off when one of these occurs. For severe errors that cause application crash or failure.
The logging levels in Serilog have an order of precedence. Each level includes all the levels above it. So logs written at Error level will also include Fatal, Warning, Information etc.
2) Serilog support structure logging, so I want to write the log to a json file, so that we can trace more details information.
.WriteTo.File(new JsonFormatter(), "./logs/debug-logs.json")
for save to the json format, we need to use JsonFormatter under Serilog.Formatting.Json namespace
3) We also want to group all of the log by day, that’s mean there is one log file per day, so update the above code as below
.WriteTo.File(new JsonFormatter(), "./logs/debug-logs-.json",
rollingInterval: RollingInterval.Day)
use the rollingInterval for set the interval, and the RollingInterval support below
a) Infinite: The log file will never roll; no time period information will be appended to the log file name.
b) Year: Roll every year. Filenames will have a four-digit year appended in the pattern: yyyy
c) Month: Roll every calendar month. Filenames will have yyyMM appended
e) Day: Roll every day. Filenames will have yyyyMMdd appended
f) Hour: Roll every hour. Filenames will have yyyyMMddHH appended
g) Minute: Roll every minute. Filenames will have yyyyMMddHHmm appended
4) In the end, show all of the debug message in console. we can use the restrictedToMinimumLevel in console
.WriteTo.Console(restrictedToMinimumLevel: LogEventLevel.Debug)
Ok, we combine all together, so the complete codes should be like below
builder.Host.UseSerilog((context, logConfig) => logConfig
.ReadFrom.Configuration(context.Configuration)
.WriteTo.Logger(l => l.Filter.ByIncludingOnly(e => e.Level == LogEventLevel.Warning)
.WriteTo.File(new JsonFormatter(), "./logs/debug-logs-.json",
rollingInterval: RollingInterval.Day)
)
.WriteTo.Logger(l => l.Filter.ByIncludingOnly(e => e.Level == LogEventLevel.Error)
.WriteTo.File(new JsonFormatter(), "./logs/error-logs-.json",
rollingInterval: RollingInterval.Day)
)
.WriteTo.Console(restrictedToMinimumLevel: LogEventLevel.Debug)
);
The setup is done, and then we can use it in our controller.
2.3. Using Serilog in controller
Add the below in UserController
protected readonly ILogger<User> _logger;
public UserController(IUserRepository userRepository, ILogger<User> logger)
{
this._userRepository = userRepository;
this._logger = logger;
}
and we can call the _logger for write log in anywhere, for example, we write the debug log when get all users, so add the code in GetUsers() method
[HttpGet("users")]
public async Task<ActionResult<IEnumerable<User>>> GetUsers()
{
...
_logger.LogDebug("LogDebug ================ GetUsers ");
...
}
That’s ok, let’s try to have a look the result! We can call the GetUsers() from Swagger UI, and you will find the below debug log file has been generated and also show the log in console
I have format the json file, so looks good now, but there is a problem that you will also find there are a lot of other debug messages that what we want to care about, can we just write the messages what we write in the controller?
Yes, we can! We can use the Serilog.Filters.Expressions library to filter only the messages what we want. For this case, we can find there are some properties in the log
{
"Timestamp": "2023-07-28T10:25:12.9690530+08:00",
"Level": "Debug",
"MessageTemplate": "LogDebug ================ GetUsers ",
"Properties": {
"SourceContext": "MyDemo.Core.Data.Entity.User",
"ActionId": "56d17f70-be16-4491-a7fe-8832462da7f8",
"ActionName": "MyDemo.Controllers.UserController.GetUsers (MyDemo)",
"RequestId": "0HMSF27B8QJ5M:00000004",
"RequestPath": "/api/users",
"ConnectionId": "0HMSF27B8QJ5M"
}
}
so we can filter these properties to identify which log we need, for example we can filter the SourceContext start with MyDemo, that will make sure only show the logs from our project
//create a filter query, the query syntax like the SQL
var filterExpr = "@Properties['SourceContext'] like 'MyDemo%'";
//use the filter
builder.Host.UseSerilog((context, logConfig) => logConfig
.ReadFrom.Configuration(context.Configuration)
.MinimumLevel.Debug()
.Filter.ByIncludingOnly(filterExpr) //use the filter here
.WriteTo.Logger(l => l.Filter.ByIncludingOnly(e => e.Level == LogEventLevel.Debug)
.WriteTo.File(new JsonFormatter(), "./logs/debug-logs-.json",
rollingInterval: RollingInterval.Day)
)
);
after that, we can find that only the message what we write in the log file
And we try to throw an exception to see whether will generate the error log file
public async Task<ActionResult<IEnumerable<User>>> GetUsers()
{
var apiResult = new ApiResult<IEnumerable<User>>();
_logger.LogDebug("LogDebug ================ GetUsers ");
try
{
apiResult.Data = await _userRepository.GetAllAsync();
//throw an exception
throw new Exception("Testing error");
return Ok(apiResult);
}
catch (Exception ex)
{
//log the error
_logger.LogError(ex, "GetUsers Exception");
apiResult.Success = false;
apiResult.Message = ex.Message;
return StatusCode(500, apiResult);
}
}
after that, we can find the error log as below
2.4. Config Serilog in appsettings.json
The other way for using Serilog is config within appsettings.json. As you know, appsettings.json is the .Net core common setting file, we put the Serilog config here will be better for maintain, we can change the settings in runtime and don’t need to publish the project again. Ok, let’s do it!
1) First, we can add the simple and base config
"Serilog": {
"MinimumLevel": "Debug",
"WriteTo": [
{
"Name": "Console",
"Args": {
"outputTemplate": "===> {Timestamp:HH:mm:ss} [{Level}] {Message}{NewLine}{Exception}"
}
}
]
}
this config only use the Serilog and write log in console with the specify format, and we need to add more functions
2) Create a debug log file and only show the message for MyDemo project, we need to add the using library for sinks and filters
"Serilog": {
"Using": [ "Serilog.Sinks.File", "Serilog.Filters.Expressions" ],
"MinimumLevel": "Debug",
...
}
add the name of Logger (this is the fixed name, equal .WriteTo.Logger in program) and set the filter, in the end, set where should write the log file and formatting:
{
"Name": "Logger",
"Args": {
"configureLogger": {
"Filter": [
{
"Name": "ByIncludingOnly",
"Args": {
"expression": "@Level = 'Debug' and @Properties['SourceContext'] like 'MyDemo%' "
}
}
],
"WriteTo": [
{
"Name": "File",
"Args": {
"path": "./logs/debug-.log",
"outputTemplate": "[{Timestamp:yyyy-MM-dd HH:mm:ss.fff} {Level:u3}] {Message:lj}{NewLine}",
"rollingInterval": "Day"
}
}
]
}
}
}
the above config is exactly the same with the settings in Program.cs
1) Create another config for error log, that will be similar with above
{
"Name": "Logger",
"Args": {
"configureLogger": {
"Filter": [
{
"Name": "ByIncludingOnly",
"Args": {
"expression": "@Level = 'Error'"
}
}
],
"WriteTo": [
{
"Name": "File",
"Args": {
"path": "./logs/error-.log",
"outputTemplate": "[{Timestamp:yyyy-MM-dd HH:mm:ss.fff} {Level:u3}] {Message:lj}{NewLine}{Exception}",
"rollingInterval": "Day"
}
}
]
}
}
}
with above config, you will find generate a simple *.log file only with a message what you write in controller and there is no other addition information with json format, if you want the full json format, you can add the formatter and remove the outputTemplate
"WriteTo": [
{
"Name": "File",
"Args": {
"path": "./logs/debug-.json",
"rollingInterval": "Day",
"formatter": "Serilog.Formatting.Json.JsonFormatter, Serilog"
}
}
]
after that, we also need to update the Program.cs , remove the other Serilog settings, we only need to register Serilog here
builder.Host.UseSerilog((context, logConfig) => logConfig
.ReadFrom.Configuration(context.Configuration));
2.5. Log the Database SQL Statement
Sometime we also want to know what’s SQL statements have been executed, this is very helpful for tracing the data issue. We still use the appsettings.json for setup it, it’s need to add more more Logger item
actually, that’s very easy to do, just need to change the filter expression will be ok, if you want to log the SQL statements, just filter the properties event name with Microsoft.EntityFrameworkCore.Database.Command.CommandExecuting, so the config item should be as below
{
"Name": "Logger",
"Args": {
"configureLogger": {
"Filter": [
{
"Name": "ByIncludingOnly",
"Args": {
"expression": "@Properties['EventId']['Name'] = 'Microsoft.EntityFrameworkCore.Database.Command.CommandExecuting'"
}
}
],
"WriteTo": [
{
"Name": "File",
"Args": {
"path": "./logs/sql-log-.sql",
"outputTemplate": "--[{Timestamp:yyyy-MM-dd HH:mm:ss.fff} {Level:u3}]{NewLine}--{Message:lj}{NewLine}{NewLine}",
"rollingInterval": "Day"
}
}
]
}
}
}
after that, try to get users and create an user, you will find there is more than a sql log file as below
3. Summarize
We have introduce Serilog with details, you can setup it by coding in Program.cs file, or config it in the appsettings.json. For easy maintain I will suggest to set the config in appsettings.json. And we tried to create 3 difference log files with the Serilog filter function, that’s very powerful and helpful for create difference logs what you need!
We have done a very base Asp.Net Core Api project with these articles, so in the next time, I will introduce how to create the client side project with Angular 🙂
