Log Analytics is a tool in the Azure portal where you can query, analyze and visualize data in Azure Monitor logs.
Using the Application Insights feature in the API, requests are inserted into Log Analytics by default, however, request data such as body and headers are not, so I chose to write a small middleware that solves this problem.
Implementation
The source code below is the middleware I wrote to capture and save information from requests in Azure Log Analytics.
Firstly, to access the request information, it’s necessary to use the IHttpContextAccessor interface, and we will use it instead of the context received as an argument in InvokeAsync for good practice reasons.
The class has five methods that I created that will be responsible for processing our log.
GetHeaders: Returns all headers from an HTTP request in key-value format.
GetRawJson: Reads the request data stream and returns a JSON string. It’s called only on PUT and POST operations.
GetTenant: This method gets the information from a single header.
Serialize: Serialize data into JSON string format.
Enrich: Add all collected data as features of the current HTTP request, it will put the information in the log analytics. You can read more about how the request features work here.
using System.Text;
using Microsoft.ApplicationInsights.DataContracts;
using Microsoft.Extensions.Primitives;
using Newtonsoft.Json;
namespace Sample.Middlewares
{
public class RequestLoggingMiddleware : IMiddleware
{
private readonly IHttpContextAccessor context;
public RequestLoggingMiddleware(IHttpContextAccessorcontext)
{
this.context = context;
}
public async Task InvokeAsync(HttpContext http, RequestDelegate
next)
{
if (context.HttpContext is not null)
{
HttpRequestrequest = context.HttpContext.Request;
string? body = null;
if ((request.Method==HttpMethods.Post ||
request.Method==HttpMethods.Put) &&
request.Body.CanRead)
{
body = await GetRawJson(request, Encoding.UTF8);
}
Dictionary<string, string> pairs=new()
{
{ "RequestBody", body },
{ "Method", request.Method },
{ "Tenant", GetTenant(request) },
{ "Route", Serialize(request.RouteValues) },
{ "Query", Serialize(request.Query) },
{ "Headers", Serialize(GetHeaders(request)) }
};
Enrich(pairs);
}
await next.Invoke(http);
}
private void Enrich(Dictionary<string, string> pairs)
{
foreach (KeyValuePair<string, string> item in pairs)
{
context.HttpContext.Features.Get<RequestTelemetry>
().Properties[item.Key] = item.Value;
}
}
private static string GetTenant(HttpRequestrequest)
{
bool hasTenant = request.Headers.TryGetValue("x-request-
tenant", out StringValues values);
if (!hasTenant)
return null;
return values.First();
}
private static Dictionary<string, string> GetHeaders
(HttpRequestrequest)
{
Dictionary<string, string> requestHeaders = new
Dictionary<string, string>();
foreach (KeyValuePair<string, StringValues> header in
request.Headers)
{
requestHeaders.Add(header.Key, header.Value);
}
return requestHeaders;
}
private static async Task<string> GetRawJson(HttpRequest request,
Encoding encoding = null)
{
request.EnableBuffering();
string response = await new StreamReader(request.Body)
.ReadToEndAsync();
request.Body.Position=0;
return response;
}
private static string Serialize(object item)
{
return JsonConvert.SerializeObject(item, Formatting.Indented);
}
}
}Now we need to enable some features in the startup class.
using Sample.Analytics.API.Middlewares;
var builder = WebApplication.CreateBuilder(args);
// Add services to the container.
builder.Services.AddHttpContextAccessor();
// Telemetry
if (!string.IsNullOrWhiteSpace(builder.Configuration["InstrumentationKey"]))
{
builder.Services.AddApplicationInsightsTelemetry
(builder.Configuration["InstrumentationKey"]);
}
builder.Services.AddScoped<RequestLoggingMiddleware>();
builder.Services.AddControllers();
builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();
var app = builder.Build();
// Configure the HTTP request pipeline.
if (app.Environment.IsDevelopment())
{
app.UseSwagger();
app.UseSwaggerUI();
}
app.UseMiddleware<RequestLoggingMiddleware>();
app.UseHttpsRedirection();
app.UseAuthorization();
app.MapControllers();
app.Run();AddHttpContextAccessor: Access the current HTTP request context.
AddApplicationInsightsTelemetry: Application Insights through the instrumentation key, monitors the API and directs telemetry data to an Application Insights resource and logs to Azure Log Analytics.
UseMiddleware: Adds a custom middleware class. The RequestLoggingMiddleware was added by dependency injection as scoped, which means that every time an HTTP request reaches the endpoint it will be triggered.
In Appsettings.json it’s necessary to put the instrumentation key of the application insights resource created in Azure.
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft.AspNetCore": "Warning"
}
},
"InstrumentationKey": "YOUR_INSTRUMENTATION_KEY"
}I won’t go into the details of how to create an Application Insights resource, but if you have doubts, there’s a link to the official documentation that might help below. Link - Create a new Azure Application Insights resource — Azure Monitor I also added a simple endpoint to validate the implementation for testing purposes.
After executing a request in the swagger, we can notice that when it passes through the middleware, before executing the Enrich method it’s possible to see all the captured information that will be present as custom dimensions in Log Analytics.
In the Azure portal, we can see all requests to our API.
Below there’s a quick demo of a query that searches for a specific value that was sent in the request body, we can see that all data that is sent is there.
There’s also the recently released HTTP Logging Middleware for .NET 6, which would make implementation a lot easier, and it wouldn’t be necessary to write one from scratch, however, at the time of writing this article, there’s no way to send the data generated by the official middleware to Azure Log Analytics. Link - HTTP Logging in .NET Core and ASP.NET Core Having observability is of utmost importance in distributed cloud systems and microservices. This is because observability can provide a 360-degree view of a system and allow you to determine when, why, and how an atypical event happened, in addition to enabling the prevention of incidents.
The source code of the article is available on my GitHub:
Source: Medium - Lucas Diogo
The Tech Platform