Introduction

It’s the beginning of December and Christmas time is almost upon us 🎅! That means that Santa Claus is busy preparing all sorts of gifts for people all around the world. But how does Santa Claus know what people want 🤔?

Well, it’s 2023, so the likely answer is that he uses some sort of monitoring application that tracks what people do on the internet, like on a webshop for example! This way he can see what people are interested in and what they want for Christmas! 🎁

There are many different monitoring applications out there, but Santa Claus probably uses Azure Application Insights to get the job done. With its powerful monitoring and logging capabilities, Santa can keep track of who’s been naughty and who’s been nice, all in real-time. Plus, with its easy-to-use interface, even the elves can get in on the action and help Santa make sure that every child gets exactly what they want for Christmas!

However, there is a problem. Apparently someone has been sending invalid logs and telemetry to the monitoring application, which is causing havoc in the North Pole! Santa needs our help to fix this problem before it’s too late!

The problem

It turns out that most browser monitoring tools, including Azure Application Insights, aren’t 100% secure by default. This is not to say that Azure Application Insights is a bad choice; it’s a fundamental problem for lots of tools. To send logs and telemetry to a monitoring application, you need to store an API key of some sorts and send it along with your logs and telemetry. This way your monitoring application knows that the incoming telemetry belongs to your application.

For Azure Application Insights, this API key is a connection string that can look like this: InstrumentationKey=00000000-0000-0000-0000-000000000000;IngestionEndpoint=https://westeurope-5.in.applicationinsights.azure.com/;LiveEndpoint=https://westeurope.livediagnostics.monitor.azure.com/

Storing this securely isn’t difficult for a back-end application because the API key can be hidden away from prying eyes. However, for a front-end application, this is a different story. The API key needs to be stored in the front-end application, which means that the user has access to it.

It’s common knowledge that secrets should not be stored in the client because it’s easy to extract them. In our case, a bad actor (let’s call him The Grinch) could simply open the browser’s devtools and look for the secret, or even simply inspect the HTTP requests that are being sent to Azure Application Insights and find the connection string there.

And now that The Grinch knows what the connectionstring is, he could integrate it in his own application to send incorrect/fake telemetry to Santa’s Azure Application Insights resource 😱! Processing all of this extra telemetry costs money and can make monitoring the real application more difficult because of all the noise. We want to prevent this from happening!

So, how do we fix this?

The solution

The solution is simple: we need to hide the connection string from the user. This way, The Grinch can’t use it to send incorrect telemetry to Santa’s Azure Application Insights resource.

Microsoft recommends using Microsoft Entra ID-based authentication to further secure your Azure Application Insights resource, but this is not possible in the browser. You can find a lot of demand for this online; I recently found this GitHub issue that asks how to hide this connection string in the browser, but no solutions had been posted yet. That motivated me to write this post, because there is a way!

The core of the solution is to use a reverse proxy. A reverse proxy sits between the client and the server and forwards requests from the client to the real server. That doesn’t sound all that useful until you make use of the 2 benefits that this approach brings:

1. You can hide the real endpoint from the client because the proxy forwards the traffic to the real URL.
2. You can modify the incoming HTTP request before sending the modified version to the real URL.

With that setup in mind, we can solve this problem as follows:

1. Modify Azure Application Insights to send telemetry to the URL where the reverse proxy runs instead of the real Azure Application Insights URL.
2. Make the connection string contain a placeholder for the instrumentation key (like TEMPINSTRUMENTATIONKEY) instead of the real value.
3. When the reverse proxy is activated, replace the incoming HTTP request’s instrumentation key with the real value.
4. Forward the request to Azure Application Insights 🚀!

The Grinch doesn’t have have access to the configuration of the reverse proxy, so he can’t obtain the real connection string anymore! The only thing he will see is a connection string that looks like TEMPINSTRUMENTATIONKEY, which is useless to him. No longer can he build his own application to forge incorrect telemetry to send to Santa’s Azure Application Insights resource! 🎉

Now that we know how to solve the problem, it’s time to implement to write some code! The solution is split into 2 parts. First, we’ll create the reverse proxy. Then, we’ll configure Azure Application Insights to send telemetry to this reverse proxy.

Setting up the reverse proxy

Like we discussed, we’ll need to set up a reverse proxy to handle the telemetry requests by replacing the connection string and forwarding the request to Azure Application Insights. There are loads of great reverse proxy tools available, like Traefik, Nginx, HAProxy, Caddy and more. However, I’ll be using YARP today.

YARP is a reverse proxy that is built by Microsoft and is easy to use. It’s open-source and can be integrated in loads of environments. It’s written in C# and has great performance! I’ve used YARP in multiple projects, including one where I implemented the solution that I’m showcasing today.

I would like this blog post to stay short, so I won’t describe all the YARP features I’m using here. If you want to learn more, you can find YARP’s excellent documentation here.

We’ll be building a small ASP.NET Core application that runs YARP to handle the telemetry requests today.

The entire program is pretty small and looks like this:

Application Code

using System.Text;
using Yarp.ReverseProxy.Transforms;
using Yarp.ReverseProxy.Transforms.Builder;

// To test this application, configure a front-end application to POST telemetry to /track.
var builder = WebApplication.CreateBuilder(args);

// Grab the reverse proxy configuration from the configuration system.
// In our case, this is appsettings.json and appsettings.Development.json
builder.Services.AddReverseProxy()
    .LoadFromConfig(builder.Configuration.GetSection("ReverseProxy")) 
    .AddTransforms(transformBuilder =>
    {
        if (transformBuilder.Route.RouteId == "trackRoute")
        {
            // This replaces the connection string placeholder with the real value that is stored in this secured service.
            transformBuilder.AddAppInsightsReplaceConnectionStringTransform(
                builder.Configuration.GetConnectionString("ApplicationInsights")
            );
        }
        // You can add other transforms here if you use YARP to route calls to other services.
    });

var app = builder.Build();

app.UseHttpsRedirection();
app.MapReverseProxy(); // This adds YARP to the request pipeline
app.Run();

public static class YarpTransformExtensions
{
    public static void AddAppInsightsReplaceConnectionStringTransform(this TransformBuilderContext context, string connectionString)
    {
        context.AddRequestTransform(async context =>
        {
            var httpContext = context.HttpContext;

            var requestBody = "";

            // Read the request body so we can modify it later
            using (var sr = new StreamReader(httpContext.Request.Body, Encoding.UTF8, detectEncodingFromByteOrderMarks: true, leaveOpen: true))
            {
                requestBody = await sr.ReadToEndAsync();
            }

            // This is where the magic happens; the placeholder is replaced with the real value
            var replacedContent = requestBody.Replace("TEMPINSTRUMENTATIONKEY", connectionString);

            // Set the new requestBody in the HTTP Request and recalculate the content-length.
            httpContext.Request.Body = new MemoryStream(Encoding.UTF8.GetBytes(replacedContent));
            context.ProxyRequest.Content!.Headers.ContentLength = httpContext.Request.Body.Length;

            // Required for YARP
            context.ProxyRequest.Headers.Host = null;

            // Remove some private headers that should not be forwarded to Microsoft Servers.
            context.ProxyRequest.Headers.Remove("Authorization");
            context.ProxyRequest.Headers.Remove("Cookie");
        });
    }
}

Configuration

Finally, we need to configure YARP. This is done in this part of appsettings.json. If you want to learn more about .NET configuration, take a look at my related blog post!

{
  // Other settings are omitted to keep things brief..

  "ConnectionStrings": {
    "ApplicationInsights": "<from-a-secure-location>"
  },

  "ReverseProxy": {
    "Routes": {
      "trackRoute": {
        "ClusterId": "trackCluster", // Requests that land on this route will be sent to the "trackCluster" below
        // You can uncomment this (and configure it to your liking) 
        // if you configure this service to use authentication/authorization
        // "AuthorizationPolicy": "default", 
        "Match": {
          "Path": "/track", // YARP listens to this endpoint and forwards traffic to the cluster mentioned above.
          "Methods": [ "POST" ]
        },
        "Transforms": [
          { "PathRemovePrefix": "/track" }
        ]
      }
    },
    "Clusters": {
      "trackCluster": {
        "Destinations": {
          "destination1": {
            "Address": "https://dc.services.visualstudio.com/v2/track" // The Application Insights URL where the telemetry is sent to
          }
        }
      }
    }
  }
}

You can configure your front-end to talk to talk to the /track endpoint of this service and everything should be handled for you!

Adding more features

You could turn the code above into its own service and run that together with your back-end and front-end applications. However, I feel like creating an entire service to forward telemetry requests is overkill. To improve upon this, you could turn this application into a BFF (Back-end for front-end). If you’re looking for a more feature complete implementation of the BFF pattern, I recommend you check out Duende BFF or one of Damien Bowden’s many secure BFF implementations.

Configuring the front-end

The Application Insights JavaScript SDK provides plugins for React, Angular and React Native that contain some extra framework integrations out of the box. The framework agnostic base version should work well enough for lots of other frameworks, though you could always look for a community-provided plugin for your framework of choice if you feel the need to do so. Personally I’ve always used the “base” version without plug-ins, but I thought it was worth mentioning!

I’ll only be focusing on configuring the correct Application Insights settings because initialization and use of Application Insights can vary wildly depending on your use case. I recommend taking a look at some samples or other resources if you’re looking for more information on setting up Azure Application Insights in the browser.

Regardless of your framework, your approach to the solution should look the same. The connectionString, disableInstrumentationKeyValidation and endpointUrl properties are most important, but I recommend that you take a look at the other properties you can configure.

const applicationInsights = new ApplicationInsights({
  config: {
    /* The instrumentation key will be replaced in the reverse proxy with the real value.
     * If your reverse proxy runs on the same domain as your client, you can leave the IngestionEndpoint as-is.
     * However, if it runs on a different domain, you should update IngestionEndpoint to point to the reverse proxy.
     * If you run into incorrect path issues with that approach
     * please take a look at https://github.com/microsoft/ApplicationInsights-JS/issues/2197.
     */
    connectionString: 'InstrumentationKey=TEMPINSTRUMENTATIONKEY;IngestionEndpoint=/',
    
    // The SDK shouldn't throw an error about the invalid instrumentation key as it will be replaced in the reverse proxy.
    disableInstrumentationKeyValidation: true,

    // Send telemetry to the reverse proxy that listens on /track so we don't need to expose the connection string to the front-end.
    endpointUrl: 'track'
  },
});
applicationInsights.loadAppInsights();

You can now run your front-end and reverse proxy side by side. If you set everything up correctly, you’ll see that the telemetry is sent to your reverse proxy. The reverse proxy should then replace the TEMPINSTRUMENTATIONKEY instrumentation key with the securely set-up real value and forward it to Azure Application Insights.

What about other monitoring tools?

I only talked about Azure Application Insights in this blog post, but this solution can be applied to other monitoring tools as well. As long as you can configure the monitoring tool to send telemetry to a different URL, you can use this solution to hide your API key by moving it to a reverse proxy.

Feel free to share your setup in the comments below!

Finishing up

I want to thank Matthew Groves and Calvin Allen for organising the C# Advent calendar. Dustin Moris organised the .NET Advent calendar and I’m just as grateful for this as well! Writing this blog post has been a lot of fun I hope you enjoyed the Christmas spin I put on it.

Thanks to turtlewarrior for the Christmas cursor!

If you have any questions or comments, let me know in the comments below! Finally, I want to wish you all a Merry Christmas and a Happy New Year! 🎄🎉