Azure Monitor and Logging

The following is from Azure Developer Training lab for AZ-203

Azure Monitor overview

Note: Consolidation of monitoring services into Azure Monitor

Log Analytics and Application Insights have been consolidated into Azure Monitor to provide a single integrated experience for monitoring Azure resources and hybrid environments.

Overview

The following diagram gives a high-level view of Azure Monitor. At the center of the diagram are the data stores for metrics and logs which are the two fundamental types of data use by Azure Monitor. On the left are the sources that collect telemetry from different monitored resources and populate the data stores. On the right are the different functions that Azure Monitor performs with this collected data such as analysis, alerting, and streaming to external systems.

 

Monitoring data platform

All data collected by Azure Monitor fits into one of two fundamental types, metrics and logs. Metrics are numerical values that describe some aspect of a system at a particular point in time. They are lightweight and capable of supporting near real-time scenarios.Logs contain different kinds of data organized into records with different sets of properties for each type. Telemetry such as events and traces are stored as logs in addition to performance data so that it can all be combined for analysis.

Log data collected by Azure Monitor is stored in Log Analytics which includes a rich query language to quickly retrieve, consolidate, and analyze collected data. You can create and test queries using the Log Analytics page in the Azure portal and then either directly analyze the data using these tools or save queries for use with visualizations or alert rules.

 

What data does Azure Monitor collect?

Azure Monitor can collect data from a variety of sources. You can think of monitoring data for your applications in tiers ranging from your application, any operating system and services it relies on, down to the platform itself. Azure Monitor collects data fromeach of the following tiers:

  • Application monitoring data: Data about the performance and functionality of the code you have written, regardless of its platform.
  • Guest OS monitoring data: Data about the operating system on which your application is running. This could be running in Azure, another cloud, or on-premises.
  • Azure resource monitoring data: Data about the operation of an Azure resource.
  • Azure subscription monitoring data: Data about the operation and management of an Azure subscription, as well as data about the health and operation of Azure itself.
  • Azure tenant monitoring data: Data about the operation of tenant-level Azure services, such as Azure Active Directory.

As soon as you create an Azure subscription and start adding resources such as virtual machines and web apps, Azure Monitor starts collecting data. Activity Logs record when resources are created or modified. Metrics tell you how the resource is performing andthe resources that it’s consuming.

Extend the data you’re collecting into the actual operation of the resources by enabling diagnostics and adding an agent to compute resources. This will collect telemetry for the internal operation of the resource and allow you to configure different data sources tocollect logs and metrics from Windows and Linux guest operating system.

Add an instrumentation package to your application, to enable Application Insights to collect detailed information about your application including page views, application requests, and exceptions. Further verify the availability of your application by configuringan availability test to simulate user traffic.

Insights

Monitoring data is only useful if it can increase your visibility into the operation of your computing environment. Azure Monitor includes several features and tools that provide valuable insights into your applications and other resources that they depend on.Monitoring solutions and features such as Application Insights and Container Insights provide deep insights into different aspects of your application and specific Azure services.

Application Insights

Application Insights monitors the availability, performance, and usage of your web applications whether they’re hosted in the cloud or on-premises. It leverages the powerful data analysis platform in Azure Monitor to provide you with deep insights into your application’s operations and diagnose errors without waiting for a user to report them. Application Insights includes connection points to a variety of development tools and integrates with Visual Studio to support your DevOps processes.

Azure Monitor for containers

Azure Monitor for containers is a feature designed to monitor the performance of container workloads deployed to managed Kubernetes clusters hosted on Azure Kubernetes Service (AKS). It gives you performance visibility by collecting memory and process ormetrics from controllers, nodes, and containers that are available in Kubernetes through the Metrics API. Container logs are also collected. After you enable monitoring from Kubernetes clusters, these metrics and logs are automatically collected for you through a containerized version of the Log Analytics agent for Linux.

Azure Monitor for VMs

Azure Monitor VM insights monitors your Azure virtual machines (VM) at scale by analyzing the performance and health of your Windows and Linux VMs, including their different processes and interconnected dependencies on other resources and external processes. The solution includes support for monitoring performance and application dependencies for VMs hosted on-premises or another cloud provider.

Monitoring solutions

Monitoring solutions in Azure Monitor are packaged sets of logic that provide insights for a particular application or service. They include logic for collecting monitoring data for the application or service, queries to analyze that data, and views for visualization.Monitoring solutions are available from Microsoft and partners to provide monitoring for various Azure services and other applications.

 

Responding to critical situations

In addition to allowing you to interactively analyze monitoring data, an effective monitoring solution must be able to proactively respond to critical conditions identified in the data that it collects. This could be sending a text or mail to an administrator responsible for investigating an issue. Or you could launch an automated process that attempts to correct an error condition.

Alerts

Alerts in Azure Monitor proactively notify you of critical conditions and potentially attempt to take corrective action. Alert rules based on metrics provide near real time alerting based on numeric values, while rules based on logs allow for complex logic across data from multiple sources.

Alert rules in Azure Monitor use action groups, which contain unique sets of recipients and actions that can be shared across multiple rules. Based on your requirements, action groups can perform such actions as using webhooks to have alerts start external actions or to integrate with your ITSM tools.

Autoscale

Autoscale allows you to have the right amount of resources running to handle the load on your application. It allows you to create rules that use metrics collected by Azure Monitor to determine when to automatically add resources to handle increases in load and also save money by removing resources that are sitting idle. You specify a minimum and maximum number of instances and the logic for when to increase or decrease resources.

 

Sources of data in Azure Monitor

Monitoring data in Azure comes from a variety of sources that can be organized into tiers, the highest tiers being your application and any operating systems and the lower tiers being components of Azure platform. This is illustrated in the following diagram withdescriptions below.

 

  • Azure Tenant: Telemetry related to your Azure tenant is collected from tenant-wide services such as Azure Active Directory.
  • Azure platform: Telemetry related to the health and operation of Azure itself includes data about the operation and management of your Azure subscription. It includes service health data stored in the Azure Activity log and audit logs from Azure ActiveDirectory.
  • Guest operating system: Compute resources in Azure, in other clouds, and on-premises have a guest operating system to monitor. With the installation of one or more agents, you can gather telemetry from the guest into the same monitoring tools as the Azureservices themselves.
  • Applications: In addition to telemetry that your application may write to the guest operating system, detailed application monitoring is done with Application Insights. Application Insights can collect data from applications running on a variety of platforms.The application can be running in Azure, another cloud, or on-premises.
  • Custom sources: Azure Monitor can collect log data from any REST client using the Data Collector API. This allows you to create custom monitoring scenarios and extend monitoring to resources that don’t expose telemetry through other sources.

 

Monitoring and Logging

Application Insights

Application Insights is an extensible Application Performance Management (APM) service for web developers on multiple platforms. Use it to monitor your live web application. It will automatically detect performance anomalies. It includes powerful analyticstools to help you diagnose issues and to understand what users actually do with your app.

What does Application Insights monitor?

Application Insights is aimed at the development team, to help you understand how your app is performing and how it’s being used. It monitors:

  • Request rates, response times, and failure rates – Find out which pages are most popular, at what times of day, and where your users are. See which pages perform best. If your response times and failure rates go high when there are more requests, thenperhaps you have a resourcing problem.
  • Dependency rates, response times, and failure rates – Find out whether external services are slowing you down.
  • Exceptions – Analyse the aggregated statistics, or pick specific instances and drill into the stack trace and related requests. Both server and browser exceptions are reported.
  • Page views and load performance – reported by your users’ browsers.
  • AJAX calls from web pages – rates, response times, and failure rates.
  • User and session counts.
  • Performance counters from your Windows or Linux server machines, such as CPU, memory, and network usage.
  • Host diagnostics from Docker or Azure.
  • Diagnostic trace logs from your app – so that you can correlate trace events with requests.
  • Custom events and metrics that you write yourself in the client or server code, to track business events such as items sold or games won.

How Application Insights works

You install a small instrumentation package in your application, and set up an Application Insights resource in the Microsoft Azure portal. The instrumentation monitors your app and sends telemetry data to the portal. (The application can run anywhere – itdoesn’t have to be hosted in Azure.)

Note: The impact on your app’s performance is very small. Tracking calls are non-blocking, and are batched and sent in a separate thread.

You can instrument not only the web service application, but also any background components, and the JavaScript in the web pages themselves.

In addition, you can pull in telemetry from the host environments such as performance counters, Azure diagnostics, or Docker logs. You can also set up web tests that periodically send synthetic requests to your web service.

All these telemetry streams are integrated in the Azure portal, where you can apply powerful analytic and search tools to the raw data.

Overview of alerts in Microsoft Azure

This lesson describes what alerts are, their benefits, and how to get started using them.

What are alerts in Microsoft Azure?

Alerts proactively notify you when important conditions are found in your monitoring data. They allow you to identify and address issues before the users of your system notice them.

Note: This lesson discusses the unified alert experience in Azure Monitor, which now includes Log Analytics and Application Insights. The previous alert experience and alert types are called classic alerts. You can view this older experience and older alert type byclicking on View classic alerts at the top of the alert page.

Overview

The diagram below represents the flow of alerts.

Alert rules are separated from alerts and the action that are taken when an alert fires.

Alert rule – The alert rule captures the target and criteria for alerting. The alert rule can be in an enabled or a disabled state. Alerts only fire when enabled.

The key attributes of an alert rule are:

  • Target Resource – Defines the scope and signals available for alerting. A target can be any Azure resource. Example targets: a virtual machine, a storage account, a virtual machine scale set, a Log Analytics workspace, or an Application Insights resource.For certain resources (like Virtual Machines), you can specify multiple resources as the target of the alert rule.
  • Signal – Signals are emitted by the target resource and can be of several types. Metric, Activity log, Application Insights, and Log.
  • Criteria – Criteria is combination of Signal and Logic applied on a Target resource. Examples:
    • Percentage CPU > 70%
    • Server Response Time > 4 ms
    • Result count of a log query > 100
  • Alert Name – A specific name for the alert rule configured by the user
  • Alert Description – A description for the alert rule configured by the user
  • Severity – The severity of the alert once the criteria specified in the alert rule is met. Severity can range from 0 to 4.
  • Action – A specific action taken when the alert is fired. For more information, see Action Groups.

What you can alert on

You can alert on metrics and logs. These include but are not limited to:

  • Metric values
  • Log search queries
  • Activity Log events
  • Health of the underlying Azure platform
  • Tests for web site availability

Manage alerts

You can set the state of an alert to specify where it is in the resolution process. When the criteria specified in the alert rule is met, an alert is created or fired, it has a status of New. You can change the status when you acknowledge an alert and when you close it.All state changes are stored in the history of the alert.

The following alert states are supported.

State Description
New The issue has just been detected and has not yet been reviewed.
Acknowledged An administrator has reviewed the alert and started working on it.
Closed The issue has been resolved. After an alert has been closed, you can reopen it by changing it to another state.

Alert state is different and independent of the monitor condition. Alert state is set by the user. Monitor condition is set by the system. When an alert fires, the alert’s monitor condition is set to fired. When the underlying condition that caused the alert to fire clears,the monitor condition is set to resolved. The alert state isn’t changed until the user changes it.

Alerts experience

The default Alerts page provides a summary of alerts that are created within a particular time window. It displays the total alerts for each severity with columns that identify the total number of alerts in each state for each severity. Select any of the severities toopen the All Alerts page filtered by that severity.

It does not show or track older classic alerts. You can change the subscriptions or filter parameters to update the page.

Create an alert rule

Alerts can be authored in a consistent manner regardless of the monitoring service or signal type. All fired alerts and related details are available in single page.

You create a new alert rule with the following three steps:

  1. Pick the target for the alert.
  2. Select the signal from the available signals for the target.
  3. Specify the logic to be applied to data from the signal.

This simplified authoring process no longer requires you to know the monitoring source or signals that are supported before selecting an Azure resource. The list of available signals is automatically filtered based on the target resource that you select. Also basedon that target, you are guided through defining the logic of the alert rule automatically.

Application Insights for web pages

Find out about the performance and usage of your web page or app. If you add Application Insights to your page script, you get timings of page loads and AJAX calls, counts and details of browser exceptions and AJAX failures, as well as users and sessioncounts. All these can be segmented by page, client OS and browser version, geo location, and other dimensions. You can set alerts on failure counts or slow page loading. And by inserting trace calls in your JavaScript code, you can track how the differentfeatures of your web page application are used.

Add the SDK script to your app or web pages

Application Insights can be used with any web pages – you just add a short piece of JavaScript. If your web service is Java or ASP.NET, you can integrate telemetry from your server and clients.

<!– To collect user behavior analytics about your application, insert the following script into each page you want to track. Place this code immediately before the closing </head> tag, and before any other scripts. Your first data will appear automatically in just a few seconds. –> <script type=”text/javascript”> var appInsights=window.appInsights||function(a){ function b(a){c[a]=function(){var b=arguments;c.queue.push(function(){c[a].apply(c,b)})}}var c={config:a},d=document,e=window;setTimeout(function(){var b=d.createElement(“script”);b.src=a.url||”https://az416426.vo.msecnd.net/scripts/a/ai.0.js”,d.getElementsByTagName(“script”)[0].parentNode.appendChild(b)});try{c.cookie=d.cookie}catch(a){}c.queue=[];for(var f=[“Event”,”Exception”,”Metric”,”PageView”,”Trace”,”Dependency”];f.length;)b(“track”+f.pop());if(b(“setAuthenticatedUserContext”),b(“clearAuthenticatedUserContext”),b(“startTrackEvent”),b(“stopTrackEvent”),b(“startTrackPage”),b(“stopTrackPage”),b(“flush”),!a.disableExceptionTracking){f=”onerror”,b(“_”+f);var g=e[f];e[f]=function(a,b,d,e,h){var i=g&&g(a,b,d,e,h);return!0!==i&&c[“_”+f](a,b,d,e,h),i}}return c }({ instrumentationKey:”<your instrumentation key>” }); window.appInsights=appInsights,appInsights.queue&&0===appInsights.queue.length&&appInsights.trackPageView(); </script>

Insert the script just before the </head> tag of every page you want to track. If your website has a master page, you can put the script there. For example in an ASP.NET MVC project, you’d put it in View\Shared\_Layout.cshtml.

The script contains the instrumentation key that directs the data to your Application Insights resource.

Detailed configuration

There are several parameters you can set, though in most cases, you shouldn’t need to. For example, you can disable or limit the number of Ajax calls reported per page view (to reduce traffic). Or you can set debug mode to have telemetry move rapidly throughthe pipeline without being batched.

To set these parameters, look for this line in the code snippet, and add more comma-separated items after it:

})({ instrumentationKey: “…” // Insert here });

The available parameters include:

// Send telemetry immediately without batching. // Remember to remove this when no longer required, as it // can affect browser performance. enableDebug: boolean, // Don’t log browser exceptions. disableExceptionTracking: boolean, // Don’t log ajax calls. disableAjaxTracking: boolean, // Limit number of Ajax calls logged, to reduce traffic. maxAjaxCallsPerView: 10, // default is 500 // Time page load up to execution of first trackPageView(). overridePageViewDuration: boolean, // Set dynamically for an authenticated user. accountId: string,

Application Insights for .NET console applications

To get started:

  • In the Azure portal, create an Application Insights resource. For application type, choose General.
  • Take a copy of the Instrumentation Key. Find the key in the Essentials drop-down of the new resource you created.
  • Install latest Microsoft.ApplicationInsights package.
  • Set the instrumentation key in your code before tracking any telemetry (or set APPINSIGHTS_INSTRUMENTATIONKEY environment variable). After that, you should be able to manually track telemetry and see it on the Azure portal

    TelemetryConfiguration.Active.InstrumentationKey = ” *your key* “; var telemetryClient = new TelemetryClient(); telemetryClient.TrackTrace(“Hello World!”);

  • Install latest version of Microsoft.ApplicationInsights.DependencyCollector package – it automatically tracks HTTP, SQL, or some other external dependency calls.

You may initialize and configure Application Insights from the code or using ApplicationInsights.config file. Make sure initialization happens as early as possible.

Note: Instructions referring to ApplicationInsights.config are only applicable to apps that are targeting the .NET Framework, and do not apply to .NET Core applications.

Using config file

By default, Application Insights SDK looks for ApplicationInsights.config file in the working directory when TelemetryConfiguration is being created

// Reads ApplicationInsights.config file if present TelemetryConfiguration config = TelemetryConfiguration.Active;

You may also specify path to the config file.

using System.IO; TelemetryConfiguration configuration = TelemetryConfiguration.CreateFromConfiguration(File.ReadAllText(“C:\\ApplicationInsights.config”)); var telemetryClient = new TelemetryClient(configuration);

For more information, see configuration file reference.

You may get a full example of the config file by installing latest version of Microsoft.ApplicationInsights.WindowsServer package. Here is the minimal configuration for dependency collection that is equivalent to the code example.

<?xml version=”1.0″ encoding=”utf-8″?> <ApplicationInsights xmlns=”http://schemas.microsoft.com/ApplicationInsights/2013/Settings”> <InstrumentationKey>Your Key</InstrumentationKey> <TelemetryInitializers> <Add Type=”Microsoft.ApplicationInsights.DependencyCollector.HttpDependenciesParsingTelemetryInitializer, Microsoft.AI.DependencyCollector”/> </TelemetryInitializers> <TelemetryModules> <Add Type=”Microsoft.ApplicationInsights.DependencyCollector.DependencyTrackingTelemetryModule, Microsoft.AI.DependencyCollector”> <ExcludeComponentCorrelationHttpHeadersOnDomains> <Add>core.windows.net</Add> <Add>core.chinacloudapi.cn</Add> <Add>core.cloudapi.de</Add> <Add>core.usgovcloudapi.net</Add> <Add>localhost</Add> <Add>127.0.0.1</Add> </ExcludeComponentCorrelationHttpHeadersOnDomains> <IncludeDiagnosticSourceActivities> <Add>Microsoft.Azure.ServiceBus</Add> <Add>Microsoft.Azure.EventHubs</Add> </IncludeDiagnosticSourceActivities> </Add> </TelemetryModules> <TelemetryChannel Type=”Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.ServerTelemetryChannel, Microsoft.AI.ServerTelemetryChannel”/> </ApplicationInsights>

Configuring telemetry collection from code

  • During application start-up create and configure DependencyTrackingTelemetryModule instance – it must be singleton and must be preserved for application lifetime.

    var module = new DependencyTrackingTelemetryModule(); // Prevent Correlation Id to be sent to certain endpoints. // You may add other domains as needed. module.ExcludeComponentCorrelationHttpHeadersOnDomains.Add(“core.windows.net”); //… // enable known dependency tracking, note that in future versions, we will extend this list. // please check default settings in // https://github.com/Microsoft/ApplicationInsights-dotnet-server/blob/develop/Src/DependencyCollector/NuGet/ApplicationInsights.config.install.xdt#L20 module.IncludeDiagnosticSourceActivities.Add(“Microsoft.Azure.ServiceBus”); module.IncludeDiagnosticSourceActivities.Add(“Microsoft.Azure.EventHubs”); //…. // initialize the module module.Initialize(configuration);

  • Add common telemetry initializers

    // stamps telemetry with correlation identifiers TelemetryConfiguration.Active.TelemetryInitializers.Add(new OperationCorrelationTelemetryInitializer()); // ensures proper DependencyTelemetry.Type is set for Azure RESTful API calls TelemetryConfiguration.Active.TelemetryInitializers.Add(new HttpDependenciesParsingTelemetryInitializer());

  • For .NET Framework Windows app, you may also install and initialize Performance Counter collector module.

Full example

using Microsoft.ApplicationInsights; using Microsoft.ApplicationInsights.DependencyCollector; using Microsoft.ApplicationInsights.Extensibility; using System.Net.Http; using System.Threading.Tasks; namespace ConsoleApp { class Program { static void Main(string[] args) { TelemetryConfiguration configuration = TelemetryConfiguration.Active; configuration.InstrumentationKey = “removed”; configuration.TelemetryInitializers.Add(new OperationCorrelationTelemetryInitializer()); configuration.TelemetryInitializers.Add(new HttpDependenciesParsingTelemetryInitializer()); var telemetryClient = new TelemetryClient(); using (InitializeDependencyTracking(configuration)) { // run app… telemetryClient.TrackTrace(“Hello World!”); using (var httpClient = new HttpClient()) { // Http dependency is automatically tracked! httpClient.GetAsync(“https://microsoft.com”).Wait(); } } // before exit, flush the remaining data telemetryClient.Flush(); // flush is not blocking so wait a bit Task.Delay(5000).Wait(); } static DependencyTrackingTelemetryModule InitializeDependencyTracking(TelemetryConfiguration configuration) { var module = new DependencyTrackingTelemetryModule(); // prevent Correlation Id to be sent to certain endpoints. You may add other domains as needed. module.ExcludeComponentCorrelationHttpHeadersOnDomains.Add(“core.windows.net”); module.ExcludeComponentCorrelationHttpHeadersOnDomains.Add(“core.chinacloudapi.cn”); module.ExcludeComponentCorrelationHttpHeadersOnDomains.Add(“core.cloudapi.de”); module.ExcludeComponentCorrelationHttpHeadersOnDomains.Add(“core.usgovcloudapi.net”); module.ExcludeComponentCorrelationHttpHeadersOnDomains.Add(“localhost”); module.ExcludeComponentCorrelationHttpHeadersOnDomains.Add(“127.0.0.1”); // enable known dependency tracking, note that in future versions, we will extend this list. // please check default settings in https://github.com/Microsoft/ApplicationInsights-dotnet-server/blob/develop/Src/DependencyCollector/NuGet/ApplicationInsights.config.install.xdt#L20 module.IncludeDiagnosticSourceActivities.Add(“Microsoft.Azure.ServiceBus”); module.IncludeDiagnosticSourceActivities.Add(“Microsoft.Azure.EventHubs”); // initialize the module module.Initialize(configuration); return module; } } }

Application Insights in classic Windows desktop apps

Applications hosted on premises, in Azure, and in other clouds can all take advantage of Application Insights. The only limitation is the need to allow communication to the Application Insights service.

To send telemetry to Application Insights from a Classic Windows application

  1. In the Azure portal, create an Application Insights resource. For application type, choose ASP.NET app.
  2. Take a copy of the Instrumentation Key. Find the key in the Essentials drop-down of the new resource you just created.
  3. In Visual Studio, edit the NuGet packages of your app project, and add Microsoft.ApplicationInsights.WindowsServer. (Or choose Microsoft.ApplicationInsights if you just want the bare API, without the standard telemetry collection modules.)
  4. Set the instrumentation key either in your code:TelemetryConfiguration.Active.InstrumentationKey = ” your key “;or in ApplicationInsights.config (if you installed one of the standard telemetry packages):<InstrumentationKey>your key</InstrumentationKey>If you use ApplicationInsights.config, make sure its properties in Solution Explorer are set to Build Action = Content, Copy to Output Directory = Copy.
  5. Use the API to send telemetry.
  6. Run your app, and see the telemetry in the resource you created in the Azure Portal.

Example code

public partial class Form1 : Form { private TelemetryClient tc = new TelemetryClient(); … private void Form1_Load(object sender, EventArgs e) { // Alternative to setting ikey in config file: tc.InstrumentationKey = “key copied from portal”; // Set session data: tc.Context.User.Id = Environment.UserName; tc.Context.Session.Id = Guid.NewGuid().ToString(); tc.Context.Device.OperatingSystem = Environment.OSVersion.ToString(); // Log a page view: tc.TrackPageView(“Form1”); … } protected override void OnClosing(CancelEventArgs e) { stop = true; if (tc != null) { tc.Flush(); // only for desktop apps // Allow time for flushing: System.Threading.Thread.Sleep(1000); } base.OnClosing(e); }

Application Map: Triage Distributed Applications

Application Map helps you spot performance bottlenecks or failure hotspots across all components of your distributed application. Each node on the map represents an application component or its dependencies; and has health KPI and alerts status. You canclick through from any component to more detailed diagnostics, such as Application Insights events. If your app uses Azure services, you can also click through to Azure diagnostics, such as SQL Database Advisor recommendations.

What is a Component?

Components are independently deployable parts of your distributed/microservices application. Developers and operations teams have code-level visibility or access to telemetry generated by these application components.

  • Components are different from “observed” external dependencies such as SQL, EventHub etc. which your team/organization may not have access to (code or telemetry).
  • Components run on any number of server/role/container instances.
  • Components can be separate Application Insights instrumentation keys (even if subscriptions are different) or different roles reporting to a single Application Insights instrumentation key. The preview map experience shows the components regardless of howthey are set up.

Composite Application Map

You can see the full application topology across multiple levels of related application components. Components could be different Application Insights resources, or different roles in a single resource. The app map finds components by following HTTP dependencycalls made between servers with the Application Insights SDK installed.

This experience starts with progressive discovery of the components. When you first load the application map, a set of queries are triggered to discover the components related to this component. A button at the top-left corner will update with the number ofcomponents in your application as they are discovered.

On clicking Update map components, the map is refreshed with all components discovered until that point. Depending on the complexity of your application, this may take a minute to load.

If all of the components are roles within a single Application Insights resource, then this discovery step is not required. The initial load for such an application will have all its components.

One of the key objectives with this experience is to be able to visualize complex topologies with hundreds of components.

Click on any component to see related insights and go to the performance and failure triage experience for that component.

Set cloud_RoleName

Application Map uses the cloud_RoleName property to identify the components on the map. The Application Insights SDK automatically adds the cloud_RoleName property to the telemetry emitted by components. For example, the SDK will add a website name or service role name to the cloud_RoleName property. However, there are cases where you may want to override the default value. To override cloud_RoleName and change what gets displayed on the Application Map:

.NET

using Microsoft.ApplicationInsights.Channel; using Microsoft.ApplicationInsights.Extensibility; namespace CustomInitializer.Telemetry { public class MyTelemetryInitializer : ITelemetryInitializer { public void Initialize(ITelemetry telemetry) { if (string.IsNullOrEmpty(telemetry.Context.Cloud.RoleName)) { //set custom role name here telemetry.Context.Cloud.RoleName = “RoleName”; } } } }

Load your initializer

In ApplicationInsights.config:

<ApplicationInsights> <TelemetryInitializers> <!– Fully qualified type name, assembly name: –> <Add Type=”CustomInitializer.Telemetry.MyTelemetryInitializer, CustomInitializer”/> … </TelemetryInitializers> </ApplicationInsights>

An alternate method is to instantiate the initializer in code, for example in Global.aspx.cs:

using Microsoft.ApplicationInsights.Extensibility; using CustomInitializer.Telemetry; protected void Application_Start() { // … TelemetryConfiguration.Active.TelemetryInitializers.Add(new MyTelemetryInitializer()); }

Node.js

var appInsights = require(“applicationinsights”); appInsights.setup(‘INSTRUMENTATION_KEY’).start(); appInsights.defaultClient.context.tags[“ai.cloud.role”] = “your role name”; appInsights.defaultClient.context.tags[“ai.cloud.roleInstance”] = “your role instance”;

Alternate method for Node.js

var appInsights = require(“applicationinsights”); appInsights.setup(‘INSTRUMENTATION_KEY’).start(); appInsights.defaultClient.addTelemetryProcessor(envelope => { envelope.tags[“ai.cloud.role”] = “your role name”; envelope.tags[“ai.cloud.roleInstance”] = “your role instance” });

Client/browser-side JavaScript

appInsights.queue.push(() => { appInsights.context.addTelemetryInitializer((envelope) => { envelope.tags[“ai.cloud.role”] = “your role name”; envelope.tags[“ai.cloud.roleInstance”] = “your role instance”; }); });

Create custom KPI dashboards

You can create multiple dashboards in the Azure portal that each include tiles visualizing data from multiple Azure resources across different resource groups and subscriptions. You can pin different charts and views from Azure Application Insights to createcustom dashboards that provide you with complete picture of the health and performance of your application. This lesson walks you through the creation of a custom dashboard that includes multiple types of data and visualizations from Azure ApplicationInsights.

Create a new dashboard

A single dashboard can contain resources from multiple applications, resource groups, and subscriptions. To create a new dashboard:

  1. On the dashboard pane, select New dashboard.
  2. Type a name for the dashboard.
  3. Have a look at the Tile Gallery for a variety of tiles that you can add to your dashboard. In addition to adding tiles from the gallery you can pin charts and other views directly from Application Insights to the dashboard.
  4. Locate the Markdown tile and drag it on to your dashboard. This tile allows you to add text formatted in markdown which is ideal for adding descriptive text to your dashboard.
  5. Add text to the tile’s properties and resize it on the dashboard canvas.
  6. Click Done customizing at the top of the screen to exit tile customization mode.

Add health overview

You can add Application Insights tiles from the Tile Gallery, or you can pin them directly from Application Insights screens. This allows you to configure charts and views that you’re already familiar with before pinning them to your dashboard. Start by adding thestandard health overview for your application. This requires no configuration and allows minimal customization in the dashboard.

  1. Select your Application Insights resource on the home screen.
  2. In the Overview pane, click the pushpin icon to add the tile to the last dashboard that you were viewing.
  3. In the top right a notification will appear that your tile was pinned to your dashboard. Click Pinned to dashboard in the notification to return to your dashboard or use the dashboard pane.
  4. That tile is now added to your dashboard. Select Edit to change the positioning of the tile. Click and drag the it into position and then click Done customizing. Your dashboard now has a tile with some useful information.

Add custom metric chart

The Metrics panel allows you to graph a metric collected by Application Insights over time with optional filters and grouping. Like everything else in Application Insights, you can add this chart to the dashboard. This does require you to do a little customizationfirst.

  1. Select your Application Insights resource in the home screen.
  2. Select Metrics.
  3. An empty chart has already been created, and you’re prompted to add a metric. Add a metric to the chart and optionally add a filter and a grouping. The example below shows the number of server requests grouped by success. This gives a running view ofsuccessful and unsuccessful requests.
  4. Select Pin to dashboard on the right. This adds the view to the last dashboard that you were working with.
  5. In the top right a notification will appear that your tile was pinned to your dashboard. Click Pinned to dashboard in the notification to return to your dashboard or use the dashboard blade.
  6. That tile is now added to your dashboard.

Add Analytics query

Azure Application Insights Analytics provides a rich query language that allows you to analyze all of the data collected Application Insights. Just like charts and other views, you can add the output of an Analytics query to your dashboard.

Since Azure Applications Insights Analytics is a separate service, you need to share your dashboard for it to include an Analytics query. When you share an Azure dashboard, you publish it as an Azure resource which can make it available to other users andresources.

  1. At the top of the dashboard screen, click Share.
  2. Keep the Dashboard name the same and select the Subscription Name to share the dashboard. Click Publish. The dashboard is now available to other services and subscriptions. You can optionally define specific users who should have access to thedashboard.
  3. Select your Application Insights resource in the home screen.
  4. Click Analytics at the top of the screen to open the Analytics portal.
  5. Type the following query, which returns the top 10 most requested pages and their request count:

    requests | summarize count() by name | sort by count_ desc | take 10

  6. Click Run to validate the results of the query.
  7. Click the pin icon and select the name of your dashboard. The reason that this option has you select a dashboard unlike the previous steps where the last dashboard was used is because the Analytics console is a separate service and needs to select from allavailable shared dashboards.

View activity logs to audit actions on resources

Through activity logs, you can determine:

  • what operations were taken on the resources in your subscription
  • who initiated the operation (although operations initiated by a backend service do not return a user as the caller)
  • when the operation occurred
  • the status of the operation
  • the values of other properties that might help you research the operation

The activity log contains all write operations (PUT, POST, DELETE) performed on your resources. It does not include read operations (GET). For a list of resource actions, see Azure Resource Manager Resource Provider operations. You can use the audit logsto find an error when troubleshooting or to monitor how a user in your organization modified a resource.

Activity logs are retained for 90 days. You can query for any range of dates, as long as the starting date is not more than 90 days in the past.

You can retrieve information from the activity logs through the portal, PowerShell, Azure CLI, Insights REST API, or Insights .NET Library.

PowerShell

  1. To retrieve log entries, run the Get-AzureRmLog command. You provide additional parameters to filter the list of entries. If you do not specify a start and end time, entries for the last hour are returned. For example, to retrieve the operations for aresource group during the past hour run:

    Get-AzureRmLog -ResourceGroup ExampleGroup

    The following example shows how to use the activity log to research operations taken during a specified time. The start and end dates are specified in a date format.

    Get-AzureRmLog -ResourceGroup ExampleGroup -StartTime 2015-08-28T06:00 -EndTime 2015-09-10T06:00

    Or, you can use date functions to specify the date range, such as the last 14 days.

    Get-AzureRmLog -ResourceGroup ExampleGroup -StartTime (Get-Date).AddDays(-14)

  2. Depending on the start time you specify, the previous commands can return a long list of operations for the resource group. You can filter the results for what you are looking for by providing search criteria. For example, if you are trying to research how aweb app was stopped, you could run the following command:

    Get-AzureRmLog -ResourceGroup ExampleGroup -StartTime (Get-Date).AddDays(-14) | Where-Object OperationName -eq Microsoft.Web/sites/stop/action

    Which for this example shows that a stop action was performed by someone@contoso.com.

    Authorization : Scope : /subscriptions/xxxxx/resourcegroups/ExampleGroup/providers/Microsoft.Web/sites/ExampleSite Action : Microsoft.Web/sites/stop/action Role : Subscription Admin Condition : Caller : someone@contoso.com CorrelationId : 84beae59-92aa-4662-a6fc-b6fecc0ff8da EventSource : Administrative EventTimestamp : 8/28/2015 4:08:18 PM OperationName : Microsoft.Web/sites/stop/action ResourceGroupName : ExampleGroup ResourceId : /subscriptions/xxxxx/resourcegroups/ExampleGroup/providers/Microsoft.Web/sites/ExampleSite Status : Succeeded SubscriptionId : xxxxx SubStatus : OK

  3. You can look up the actions taken by a particular user, even for a resource group that no longer exists.

    Get-AzureRmLog -ResourceGroup deletedgroup -StartTime (Get-Date).AddDays(-14) -Caller someone@contoso.com

  4. You can filter for failed operations.

    Get-AzureRmLog -ResourceGroup ExampleGroup -Status Failed

  5. You can focus on one error by looking at the status message for that entry.

    ((Get-AzureRmLog -Status Failed -ResourceGroup ExampleGroup -DetailedOutput).Properties[1].Content[“statusMessage”] | ConvertFrom-Json).error

    Which returns:

    code message —- ——- DnsRecordInUse DNS record dns.westus.cloudapp.azure.com is already used by another public IP.

Azure CLI

To retrieve log entries, run the az monitor activity-log list command.

az monitor activity-log list –resource-group <group name>

REST API

The REST operations for working with the activity log are part of the Insights REST API. To retrieve activity log events, see List the management events in a subscription.

Monitor availability and responsiveness of any web site

After you’ve deployed your web app or web site to any server, you can set up tests to monitor its availability and responsiveness. Azure Application Insights sends web requests to your application at regular intervals from points around the world. It alerts you ifyour application doesn’t respond, or responds slowly.

You can set up availability tests for any HTTP or HTTPS endpoint that is accessible from the public internet. You don’t have to add anything to the web site you’re testing. It doesn’t even have to be your site: you could test a REST API service on which you depend.

There are two types of availability tests:

  • URL ping test: a simple test that you can create in the Azure portal.
  • Multi-step web test: which you create in Visual Studio Enterprise and upload to the portal.

You can create up to 100 availability tests per application resource.

If you have already configured Application Insights for your web app, open its Application Insights resource in the Azure portal.

Or, if you want to see your reports in a new resource, go to the Azure portal, and create an Application Insights resource.

Create a URL ping test

Open the Availability blade and add a test.

  • URL: Can be any web page you want to test, but it must be visible from the public internet. The URL can include a query string. So, for example, you can exercise your database a little. If the URL resolves to a redirect, we follow it up to 10 redirects.
  • Parse dependent requests: If this option is checked, the test requests images, scripts, style files, and other files that are part of the web page under test. The recorded response time includes the time taken to get these files. The test fails if all these resourcescannot be successfully downloaded within the timeout for the whole test. If the option is not checked, the test only requests the file at the URL you specified.
  • Enable retries: If this option is checked, when the test fails, it is retried after a short interval. A failure is reported only if three successive attempts fail. Subsequent tests are then performed at the usual test frequency. Retry is temporarily suspended until thenext success. This rule is applied independently at each test location. We recommend this option. On average, about 80% of failures disappear on retry.
  • Test frequency: Sets how often the test is run from each test location. With a default frequency of five minutes and five test locations, your site is tested on average every minute.
  • Test locations are the places from where our servers send web requests to your URL. Our minimum number of recommended test locations is five in order to insure that you can distinguish problems in your website from network issues. You can select up to 16locations.Note: We strongly recommend testing from multiple locations with a minimum of five locations. This is to prevent false alarms that may result from transient issues with a specific location. In addition we have found that the optimal configuration isto have the number of test locations be equal to the alert location threshold + 2. Enabling the “Parse dependent requests” option results in a stricter check. The test could fail for cases which may not be noticeable when manually browsing the site.
  • Success criteria:Test timeout: Decrease this value to be alerted about slow responses. The test is counted as a failure if the responses from your site have not been received within this period. If you selected Parse dependent requests, then all the images, stylefiles, scripts, and other dependent resources must have been received within this period.HTTP response: The returned status code that is counted as a success. 200 is the code that indicates that a normal web page has been returned.Content match: a string,like “Welcome!” We test that an exact case-sensitive match occurs in every response. It must be a plain string, without wildcards. Don’t forget that if your page content changes you might have to update it.
  • Alert location threshold: We recommend a minimum of 3/5 locations. The optimal relationship between alert location threshold and the number of test locations is alert location threshold = number of test locations – 2, with a minimum of five test locations.

Multi-step web tests

You can monitor a scenario that involves a sequence of URLs. For example, if you are monitoring a sales website, you can test that adding items to the shopping cart works correctly.

To create a multi-step test, you record the scenario by using Visual Studio Enterprise, and then upload the recording to Application Insights. Application Insights replays the scenario at intervals and verifies the responses.

  • You can’t use coded functions or loops in your tests. The test must be contained completely in the .webtest script. However, you can use standard plugins.
  • Only English characters are supported in the multi-step web tests. If you use Visual Studio in other languages, please update the web test definition file to translate/exclude non-English characters.