When a .NET application misbehaves in production, traditional debugging isn’t an option. You can’t simply pause a live service or attach a full debugger without disrupting users. This is where production diagnostics comes into play. In .NET 8, developers have a suite of powerful tools that bridge the gap between development-time debugging and runtime diagnostics, enabling you to analyze applications on the fly with minimal impact on availability. Key among these tools are dotnet-monitor for on-demand telemetry, dotnet-counters for real-time metrics, and dotnet-dump for deep dive memory analysis. Together, these tools empower .NET developers of all levels to identify performance bottlenecks, memory leaks, and runtime issues in live environments—without taking the application down.

This article provides a comprehensive guide to mastering these diagnostic tools in .NET 8. We’ll start with an introduction to production diagnostics and why it’s critical. Then we’ll dive into each tool: dotnet-monitor for capturing logs, traces, metrics, and more via a convenient service; dotnet-counters for lightweight performance monitoring; and dotnet-dump for collecting and inspecting memory dumps. Along the way, we’ll discuss when and why to use each tool and best practices for monitoring live applications without pausing or impacting your services. Code examples, CLI usage, and sample outputs are included to illustrate each concept in a practical way. By the end, you’ll have a solid grasp of how to move from interactive debugging to robust diagnostics in production mastering the art of keeping .NET 8 applications healthy and performant in the wild.Production Diagnostics vs. Traditional Debugging in .NET 8

Diagnosing issues in a production environment is a different ballgame from debugging on a developer’s machine. In development, you might use interactive debuggers, breakpoints, and step-by-step code execution to inspect behavior. In production, however, you typically cannot pause the process or attach a debugger without risking downtime. Production diagnostics is about gathering insights from a running application in situ, under real workloads, with minimal overhead. .NET 8 brings improved diagnostics and observability features – building on the EventPipe infrastructure – that make it easier than ever to monitor and inspect apps on the fly:. The goal is to find memory leaks, CPU spikes, slowdowns, or crashes in a live app using specialized tools that don’t interfere with the app’s normal operation.

Key challenges in production diagnostics include capturing data safely (e.g. grabbing a memory dump or performance trace without crashing the app) and interpreting that data to pinpoint problems. Historically, on Windows one might use tools like Performance Monitor, ETW traces, or crash dumps via WinDbg. In the cross-platform .NET Core era, Microsoft introduced a standardized diagnostic IPC mechanism (EventPipe) and a suite of cross-platform CLI tools. These tools dotnet-counters, dotnet-trace, dotnet-dump and eventually dotnet-monitor leverage EventPipe to collect information from a running .NET process without needing a traditional debugger or profiler attached. This means even on Linux or in a Docker container, you can inspect a .NET 8 app’s performance and health in real time.

Production diagnostics in .NET 8 is also proactive. Rather than waiting for a catastrophic crash, you can continuously monitor metrics (like CPU, memory, request rate) and automatically trigger actions (like collecting a dump) when certain conditions are met. This is where tools like dotnet-monitor shine, offering always-on monitoring and rule-based collection of diagnostics. Before diving into each tool, let’s set the stage with a scenario: Imagine your .NET 8 web API is occasionally slowing down or consuming excessive memory in production. With the tools we cover below, you could do the following without stopping the service:

• Use dotnet-counters to watch live metrics (CPU%, GC heap size, exception rate) to see if something obvious correlates with slowdowns.
• Run dotnet-monitor as a sidecar or background process to capture detailed traces or logs when the issue occurs, or even configure it to automatically collect a dump when memory usage exceeds a threshold.
• If a severe issue is detected, use dotnet-dump to capture a memory dump of the running process at that moment, then analyze it to find root causes (e.g. which objects are filling the heap, or what threads are stuck).

All of this can be done on a live app with minimal interruption. Now, let’s explore each tool in detail, starting with the all-in-one diagnostic Swiss Army knife: dotnet-monitor.Overview of dotnet-monitor: The Diagnostics Swiss Army Knife

dotnet-monitor is a tool designed to make production diagnostics easy by exposing a simple REST API to collect diagnostic information from your .NET 8 application:. Think of dotnet-monitor as a diagnostic endpoint or sidecar that runs alongside your app. Instead of fumbling with multiple tools or attaching debuggers, you run dotnet-monitor and then use HTTP APIs (or the built-in CLI commands) to request the data you need. It can capture a variety of artifacts on-demand: memory dumps, GC dumps, performance traces, logs, and metrics, all from a running process. In fact, the tool’s name hints at its purpose – “monitor” your dotnet application. This is particularly powerful in containerized or cloud environments (Kubernetes, etc.), where dotnet-monitor is often run as a sidecar container in the same pod as your app to provide diagnostic access without altering the app container itself.

Let’s break down how to get started with dotnet-monitor and what it offers:Installing and Running dotnet-monitor

dotnet-monitor is distributed as a .NET global tool and as a Docker container image. You can install it globally on any machine that has the .NET 8 SDK or runtime. To install the latest version as a global tool, use the .NET CLI:

dotnet tool install --global dotnet-monitor

This will add the dotnet-monitor command to your PATH (you might need to restart your shell). Alternatively, if you prefer to use it in a containerized setup, you can pull the official Docker image (which as of .NET 8 is provided as an ultra-small “chiseled” Ubuntu image for minimal footprint:

docker pull mcr.microsoft.com/dotnet/monitor:8.0

Once installed, running dotnet-monitor is straightforward. In the simplest case, you can just execute it with the default “collect” command:

dotnet monitor collect

This will start the diagnostic monitor. By default, it will:

• Listen on an HTTPS port (52323 by default) for the REST API endpoints, and an HTTP port (52325) for metrics. You can override these with --urls and --metricUrls options if needed.
• Require authentication via an API key for safety. When you first run it, dotnet-monitor may generate a random API key (and print it to the console) unless you disable auth (not recommended in production) with --no-auth:.. You can also generate a persistent key with dotnet monitor generatekey and configure it via environment variables or config files.
• Automatically discover any running .NET processes that it has permission to monitor. By default, it runs in “connect” mode, meaning it will try to connect to the diagnostic IPC endpoints of already running .NET processes on the same machine. Optionally, you could run it in “listen” mode on a specific --diagnostic-port for scenarios like container-to-container attachment, but that’s an advanced setup.

When dotnet-monitor is running, it will output some log information to indicate it’s ready (such as the URLs it’s listening on). At this point, your diagnostic microservice is up and running alongside your app. You can then start hitting its endpoints to gather information. Here’s a look at a typical setup with an app and dotnet-monitor side by side:Example: Running dotnet-monitor alongside an application. The tool (top-right terminal) is listening on default URLs (52323 for HTTPS API, 52325 for HTTP metrics). Using curl or a browser, you can hit endpoints like /processes to list target processes (bottom-right). The application itself (left) continues running normally. hanselman.com hanselman.com

In the screenshot above, we see dotnet-monitor listing the processes it can monitor. Let’s explore the available HTTP API endpoints and what they do.Using the dotnet-monitor HTTP API for Diagnostics

The power of dotnet-monitor lies in its HTTP API. This API provides a unified way to trigger various diagnostic actions. The major endpoints include:

• /processes – Get a list of discoverable .NET processes and their IDs (useful if multiple processes are running; in a container with one process, you can often skip this and use PID 1 by default).
• /dump/{pid?} – Capture a memory dump of the specified process (or the only process, if PID not given). This is equivalent to using dotnet-dump collect, but done through the monitor’s API.
• /gcdump/{pid?} – Capture a GC dump (a lightweight dump focused on managed heap usage, smaller than a full dump).
• /trace/{pid?} – Capture a performance trace (CPU profiles, events) for a process for a given duration, without needing to attach a profiler.
• /logs/{pid?} – Stream logs from the application. You can optionally filter by category or level. Logs are returned as an NDJSON (newline-delimited JSON) event stream.
• /metrics – Produce a one-time snapshot of metrics in Prometheus exposition format (which can be scraped by Prometheus, should you use that for monitoring). This covers a standard set of runtime metrics like CPU, working set, GC stats, etc., plus any custom metrics your app is exposing via EventCounter or Meter.
• /livemetrics – Stream live metrics in a more real-time fashion (this endpoint provides a continuous event stream of metrics, updated at short intervals, which you might view in a tool or console).
• /info – Get info about the dotnet-monitor tool itself (version, etc.).
• /operations – Check status of asynchronous operations or cancel them. For example, a trace or dump might be collected asynchronously; this endpoint lets you query progress.

All these endpoints make dotnet-monitor a one-stop service for diagnostics. You can interact with them using any HTTP client. For example, on a machine with curl and assuming no auth for simplicity (don’t disable auth in production without ensuring network security!), you could do:

# List processes curl -s http://localhost:52325/processes | jq . # Get a dump of process 1234 curl -o myapp_dump.dmp http://localhost:52323/dump/1234 # Stream logs for 60 seconds (adjust the JSON as needed for filters, and note using HTTPS if auth is enabled) curl -X POST "https://localhost:52323/logs?pid=1234&durationSeconds=60" ^ -H "Accept: application/x-ndjson" -H "Content-Type: application/json" ^ -d "{ \"filterSpecs\": { \"Microsoft.AspNetCore.Server.Kestrel\": \"Warning\" } }" 

In the logs example above (split with ^ for readability in Windows curl usage), we request 60 seconds of logs from the Kestrel web server category at Warning level. The response would be a stream of JSON objects representing log events. The Accept: application/x-ndjson header tells dotnet-monitor we want NDJSON streaming. The output might look like:

{"Timestamp":"2025-09-22T14:30:01Z","LogLevel":"Warning","EventId":13,"EventName":"ConnectionAborted","Category":"Microsoft.AspNetCore.Server.Kestrel","Message":"Connection id \"0HM123ABC\" aborted by the client."} {"Timestamp":"2025-09-22T14:30:05Z","LogLevel":"Warning","EventId":0,"EventName":"ApplicationError","Category":"Microsoft.AspNetCore.Diagnostics","Message":"An unhandled exception has occurred while executing the request.","Exception":"System.NullReferenceException: Object reference not set to an instance of an object. at MyApp.Controllers.HomeController.Get() ..."} 

As you can see, this is extremely valuable for grabbing logs or traces from a live app on demand. Similarly, hitting /metrics would return a plaintext document of metrics in a format that Prometheus can scrape. For example, you’d see lines like:

# HELP process_cpu_usage_cpu_percent CPU usage of the process. # TYPE process_cpu_usage_cpu_percent gauge process_cpu_usage_cpu_percent 35.7 # HELP dotnet_gc_heap_size_bytes Total GC heap size in bytes. # TYPE dotnet_gc_heap_size_bytes gauge dotnet_gc_heap_size_bytes 104857600 

The above is just an illustration – actual metrics output covers many aspects of the runtime (and even custom metrics from EventCounter/Meter in your code). .NET 8 has enhanced metric support with the System.Diagnostics.Metrics API, and dotnet-monitor can expose those too, meaning you can plug into OpenTelemetry or Prometheus easily. Real-Time Telemetry and Automated Collection (Triggers in dotnet-monitor)

One of the killer features of dotnet-monitor is the ability to set up collection rules, or triggers, to automate diagnostics. This moves you from a reactive posture (“I manually took a dump when I noticed a problem”) to a proactive one (“the tool will automatically collect data when certain conditions occur”). Using a JSON configuration, you can tell dotnet-monitor to watch for specific events or metrics and then perform actions like capturing a dump or trace, subject to certain limits.

For example, consider a high CPU scenario. You might want dotnet-monitor to detect when CPU usage stays above 80% for over a minute, and then collect a *triage dump* (a lightweight heap dump) but at most once per hour (to avoid flooding with dumps). This can be expressed as a collection rule in the config file (usually dotnet-monitor-config.json or via environment variables). Here’s a simplified example of such a rule:

{ "CollectionRules": { "HighCpuRule": { "Filters": [ { "Key": "ProcessName", "Value": "MyApp", "MatchType": "Exact" } ], "Trigger": { "Type": "EventCounter", "Settings": { "ProviderName": "System.Runtime", "CounterName": "cpu-usage", "GreaterThan": 80, "SlidingWindowDuration": "00:01:00" } }, "Limits": { "ActionCount": 1, "ActionCountSlidingWindowDuration": "1:00:00" }, "Actions": [ { "Type": "CollectDump", "Settings": { "Type": "Triage", "Egress": "myStorage" } } ] } } } 

In plain English, this rule says: *for any process named “MyApp”, if its CPU usage (as reported by the runtime’s System.Runtime:cpu-usage counter) exceeds 80% for a continuous 1 minute period, then collect a triage dump. Only do this action once per hour maximum.* The dump would be egressed to a storage provider named “myStorage” (which you would configure separately, e.g. an Azure Blob Storage connection or a local folder). This kind of automation is incredibly useful for catching intermittent issues that might happen at 2 AM when no one is watching. With .NET 8 and dotnet-monitor, your app can effectively capture its own diagnostic evidence when things go awry, so you have it ready for analysis later.

Besides EventCounter triggers, dotnet-monitor supports other trigger types like ASP.NET request duration, manual trigger via HTTP, or custom events. The key takeaway is that dotnet-monitor provides both an on-demand diagnostics buffet and an automated watchdog, all in one tool.

We’ve seen how comprehensive dotnet-monitor is. However, sometimes you may not need a full-blown sidecar or all the features it offers. If all you require is a quick look at some performance counters in real time, the next tool, dotnet-counters, is perfect for that.Using dotnet-counters for Lightweight Real-Time Metrics

While dotnet-monitor gives you broad diagnostic powers, dotnet-counters is a more focused tool for ad-hoc performance monitoring. It’s great for getting a live readout of key metrics when you suspect something is off, or as a first-level investigation before diving deeper. The tool connects to a running .NET process and displays a set of performance counters that update at a chosen interval. You can think of it as analogous to the Windows Performance Monitor, but for .NET Core/.NET processes and usable on any platform via the command line.

dotnet-counters can show you things like CPU usage, GC heap size, allocations/sec, number of exceptions thrown, thread pool usage, and any custom counters your application exposes via the EventCounter API or the newer Meter API (which underpins OpenTelemetry metrics). It’s an excellent way to see *right now* what your app is doing.Installing dotnet-counters and Getting Started

Like the other diagnostic CLI tools, dotnet-counters is installed as a global tool. Install it with:

dotnet tool install --global dotnet-counters

After installation, you get the dotnet-counters command. If you run it with --help, you’ll see it has a few subcommands: typically monitor, collect, and ps:

• dotnet-counters ps – Lists the running .NET processes that you can attach to, similar to a “process list” (it will show the process ID, name, and in newer versions, the command line of each process).
• dotnet-counters monitor – Connects to a process and displays chosen counters continuously, refreshing on an interval (default 1 sec). This is the live monitoring mode.
• dotnet-counters collect – Attaches to a process and records counter values over time, saving them to a file (CSV or JSON format) for later analysis. Useful if you want to chart or analyze metrics after the fact, or integrate with other tools.

In most cases, dotnet-counters monitor is the go-to for a quick glance at what’s happening. Let’s try an example. Suppose you have a .NET 8 application running (could be a web app or any process). First, find its Process ID (PID). You might use dotnet-counters ps to get this:

$ dotnet-counters ps 7192 MyApp.dll dotnet MyApp.dll 23100 dotnet C:\Program Files\dotnet\dotnet.exe

The output above (format may vary by OS) shows a .NET process with PID 7192 running MyApp.dll. Once you have the PID, you can attach the monitor. Let’s watch some essential counters for this process:

dotnet-counters monitor --process-id 7192 --refresh-interval 3

This will start printing a selection of counters from the default provider (which is System.Runtime, covering general .NET runtime stats) every 3 seconds. The first time you run it, you’ll see a list of all counters being tracked and their values updating. The output might look like this:

System.Runtime: % Processor Time (cpu-usage) 45 Working Set (MB) 200 GC Heap Size (MB) 150 Gen 0 GC Count 5 Gen 1 GC Count 3 Gen 2 GC Count 1 Time in GC (%) 2 # of Exceptions / sec 10 ThreadPool Threads Count 20 Monitor Lock Contention Count / sec 0 Press p to pause, r to resume, q to quit. 

Every few seconds, the numbers will update (the above is an illustrative snapshot). Here we can see, for example, that our app is using ~45% CPU, about 200 MB of working set, 150 MB of GC heap, and throwing 10 exceptions per second (which might be a red flag!). The GC has occurred a few times in each generation, and 2% of time is spent in GC – all useful info at a glance.

You can customize which counters to display. By default, if you don’t specify, it shows the main ones from System.Runtime (and ASP.NET Core counters if it detects an ASP.NET app). If you want to focus, say, only on garbage collection counters, you could do:

dotnet-counters monitor -p 7192 --counters System.Runtime[gc-heap-size,gc-fragmentation]

This uses the --counters argument where you can specify `provider[counter1,counter2,…]`. You can also monitor multiple providers at once (for example, add `Microsoft.AspNetCore.Hosting` to get Kestrel request counters, etc.). To discover available counters, you could run `dotnet-counters list` (in some versions) or check documentation for built-in providers.

One neat trick: dotnet-counters can even launch a process for you and monitor it from startup. For instance, if you want to see metrics from the very beginning (maybe an app initialization issue), you can:

dotnet-counters monitor -- refresh-interval 1 -- MyApp.exe arg1 arg2

This will start MyApp.exe with the given args and attach the counters immediately. It’s useful for short-lived processes or capturing startup metrics. Note that when running on Linux/macOS, ensure the user running dotnet-counters has permission to the target process (typically the same user or root); otherwise, attach will fail. Similarly, on those OSes, the diagnostic communication uses a temporary socket file – the TMPDIR environment must be shared if you’re doing something like running dotnet-counters outside a container to target a process inside a container (an advanced scenario solved by using the --diagnostic-port option or running the tool in the same container).

Why use dotnet-counters when dotnet-monitor can also display metrics? The main reasons are simplicity and zero config. dotnet-counters is quick and ideal for spot-checking performance. It’s also interactive on the console, which some engineers prefer during development or troubleshooting sessions. The overhead of dotnet-counters is very low – it’s tapping into EventCounters which are designed to be lightweight. You can run it against production systems to observe behavior in real time with confidence that it won’t perturb the system significantly.

For example, if your web app is sluggish, running dotnet-counters might reveal high CPU or frequent GCs or a surge in exceptions. That insight can then guide your next steps (maybe collect a trace or dump if needed). In practice, many developers use dotnet-counters as a first response: it’s the quickest way to answer “what’s my app doing right now?” before proceeding to deeper analysis.

Speaking of deeper analysis, sometimes you encounter issues that require capturing the entire state of the process – such as a memory leak, a deadlock, or an unexpected crash. For those scenarios, let’s turn to dotnet-dump, the tool for memory dumps and post-mortem debugging.Collecting and Analyzing Memory Dumps with dotnet-dump

dotnet-dump is the .NET CLI tool that allows you to capture a process dump and analyze it, all without a GUI debugger. It’s essentially a way to do crash dump debugging on any platform. On Windows, this is akin to creating a dump (like with Task Manager or ProcDump) and then opening it in WinDbg or Visual Studio. On Linux/macOS, it fills an especially important role because you might not have access to WinDbg, and the dumps are in core dump format – dotnet-dump lets you inspect those .NET Core dumps in a cross-platform manner.

Notably, dotnet-dump is not a live debugger, it doesn’t attach in the same way a debugger does. Instead, it takes a snapshot of the process memory (the dump) and then you can analyze that snapshot. This means it’s non-intrusive in the sense that after the dump is taken, the process continues to run (unless it crashed on its own). The act of taking a dump will pause the process briefly, but once the dump is written, the process resumes normal operation. This is great for production diagnostics because you can capture a dump of a hung or problematic process and then restore service (if it was hung, often capturing a dump doesn’t unhang it, but you might choose to restart after capturing for analysis). The key point: you get all the info you need (memory contents, call stacks, etc.) without needing the app to be in debug mode.Installing dotnet-dump and Capturing a Dump

As with our other tools, install dotnet-dump via the global tool mechanism:

dotnet tool install --global dotnet-dump

Once installed, you’ll have the dotnet-dump command. The two primary subcommands are collect (to create a dump) and analyze (to open a dump in the analysis REPL). There’s also a ps command similar to the others, which lists processes that can be dumped.

To capture a dump, you need the process ID or name. Let’s say our app’s PID is 7192 again. The basic command is:

dotnet-dump collect --process-id 7192

When you run this, dotnet-dump will by default capture a full heap dump (the default type is “Full”, which includes all memory pages). You can specify --type Heap for a heavy dump excluding module images, or Mini for a lighter dump with just stack and thread info. Full dumps can be very large (as big as the process’s memory usage), so sometimes a heap dump is sufficient for debugging most .NET issues and is smaller. For example:

dotnet-dump collect -p 7192 --type Heap --output "/dumps/myapp_heap.dmp"

The tool will print progress as it writes the dump. You might see output like:

Writing full dump to file ./core_20250922_151012 Written 150233472 bytes (36633 pages) to core file Complete

On Windows, it would save a .dmp file (e.g. dump_20250922_151012.dmp), and on Linux/macOS, a core dump file (as named above). The CLI output confirms the size and path. During this process, the target process was briefly paused. Be aware that if the process is very large (e.g. using many GB of memory), writing a full dump can take time and potentially fill disk space. In container environments, also ensure that writing the dump won’t exceed the container’s memory limit – because as the docs note, capturing a dump may cause the OS to page in all memory, which if hitting a cgroup limit could OOM kill the container. In production, a strategy is to capture dumps during off-peak times or under controlled conditions, and to have sufficient disk space for them.

Also note, similar to counters, you need appropriate permissions: the dump command must run as the same user or a user with permission to the process (on Linux, ptrace permissions apply; you might need to set proc/sys/kernel/yama/ptrace_scope to 0 or run as root if dumping another user’s process). In many cases, if you’re in a Docker container with the app, you’d exec into it as root to run dotnet-dump, or use a diagnostic sidecar approach. On Windows, if you’re an admin or the same user, you can dump the process.Analyzing a Dump with dotnet-dump (SOS Debugger)

Once you have a dump file, how do you analyze it? You have a few options:

• Open it in Visual Studio (on Windows) or WinDbg to inspect, if those are available. This is a GUI approach – sometimes convenient, but not always possible on Linux servers.
• Use dotnet-dump analyze, which is a cross-platform way to inspect the dump using SOS debugging commands in a REPL shell.

The second approach is what we’ll focus on, as it doesn’t require any heavy IDE and works wherever you can run the CLI. To analyze, run:

dotnet-dump analyze myapp_heap.dmp

This drops you into an interactive prompt (it’ll load the dump first, which can take a moment). You’ll see something like:

Loading core dump: myapp_heap.dmp Ready to process analysis commands. Type 'help' to see available commands. %

Now you have a % (or >) prompt where you can issue SOS commands. SOS (Son of Strike) is the debugging extension for .NET that lets you inspect managed memory, threads, exceptions, etc. In dotnet-dump, the SOS commands are built-in (no need to prefix with ! as in WinDbg):. If you type help, you’ll see a list of commands to use. Here are some of the most commonly used ones and scenarios:

• clrstack – Prints the managed call stacks of all threads in the dump. This is essential to see what each thread was doing. If your app was hung, you might find a thread stuck on a lock or long operation in this output.
• dumpheap -stat – Lists all types in the .NET heap and how much memory they occupy, sorted by size. Great for identifying memory leaks (e.g. see which objects are most numerous or biggest).
• dumpheap -type SomeType – Lists all instances of a given type, if you suspect a particular object is leaking or want to inspect them.
• gcroot <object_address> – For a given object address from dumpheap, this traces the GC roots to see why the object is still alive (what’s holding a reference to it). This helps answer “why wasn’t this object garbage collected?”.
• dumpobj <object_address> – Inspects the field values of a given object instance.
• threads – Lists the threads in the process with their IDs and some info (managed thread ID, etc.). You might use this to then decide which thread to examine with clrstack or other commands.
• pe (Print Exception) – Shows details about exceptions. For example, pe on its own shows the last thrown exception on each thread (if any) or you can provide an exception object address to get details. If your process crashed due to an exception, this is how you’d see what happened.
• clrthreads – More detailed thread info including the call stack location where each thread is (useful to identify deadlocks or waiting threads).

As a quick example, suppose we suspect a memory leak. In the dotnet-dump analysis shell, we could do:

% dumpheap -stat Statistics: MT Count TotalSize Class Name 0x7f8e62123456 500 80000000 MyNamespace.LargeObject 0x7f8e62121000 1500 36000000 System.String 0x7f8e620FF000 10000 16000000 MyNamespace.LeakyCacheItem ... etc ...

This shows that MyNamespace.LeakyCacheItem has 10,000 instances taking up 16 MB, which might be suspicious. We can pick one and find out why it’s still around:

% dumpheap -type MyNamespace.LeakyCacheItem Address MT Size 0x7f8e40010008 0x7f8e620FF000 160 0x7f8e40010110 0x7f8e620FF000 160 ... (many addresses) ... % gcroot 0x7f8e40010008

The gcroot result might show something like:

Object 0x7f8e40010008 is rooted in: static field MyNamespace.Cache._items (System.Collections.Generic.List<LeakyCacheItem>) -> object 0x7f8e50020090 System.Collections.Generic.List<LeakyCacheItem> -> fields: _items 0x7f8e50020100 LeakyCacheItem[]

Aha – it indicates our LeakyCacheItem instances are being held alive by a static cache in our application that never gets cleared (a typical memory leak pattern). This kind of insight is only possible with a memory dump analysis.

For a hanging app example, you could run clrstack and might see one thread’s stack trace showing it’s waiting on a lock or stuck in a long-running function – giving you a clue what code is responsible. Or using pe to find what exception was thrown that caused a crash. All this without needing a live debugging session.

One thing to be aware of: dotnet-dump analyze uses SOS which depends on having the matching DAC (Debugging Assistance Components) for the runtime of the app you dumped. Usually, dotnet-dump will handle loading the correct ones (especially if the same SDK is installed). If you ever hit an analysis issue (commands not working), ensure you have a compatible runtime or use sos Update or dotnet-sos tool to get the latest SOS. But typically on a given machine, if you dump a .NET 8 app and have .NET 8 SDK, it should work out of the box.

In summary, dotnet-dump is your go-to for post-mortem debugging and deep dives: it lets you freeze time and pour over the state of your application at your leisure, answering questions that logs or metrics sometimes can’t (like exactly which objects are in memory, or seeing the precise code path on each thread). It’s an indispensable tool when facing complex issues in production that aren’t easily reproducible in dev.When and Why to Use Each Tool

Now that we’ve covered dotnet-monitor, dotnet-counters, and dotnet-dump, you might wonder how they overlap and when to pick one over the other. All three are part of .NET’s diagnostic toolbox, but each has its strengths and ideal use cases. Here’s a quick comparison and guideline:dotnet-monitor: Use this when you want a comprehensive, always-available diagnostics endpoint for an app, especially in production or staging environments. It’s great if you need on-demand access to multiple types of data (logs, metrics, dumps, traces) or want to automate collection via rules. For example, in a Kubernetes cluster, run it as a sidecar to each .NET app for a full diagnostics service. It does require some setup (managing the tool or container, handling authentication keys, etc.), but once running, it’s extremely powerful. Choose dotnet-monitor if you’re in a scenario where you need to frequently or remotely get diagnostics, or you want to minimize direct interaction with the app container (just call the REST API). It’s also the only one of the three that can proactively collect data based on triggers, making it ideal for catching intermittent issues automatically. dotnet-counters: Use this for quick, real-time health checks and performance monitoring. It’s the simplest to use on the fly. If you suspect high CPU, memory usage, or thread pool exhaustion, dotnet-counters will confirm it in seconds. It’s also useful in development or load testing scenarios to observe how the system behaves (e.g., watch GC heap during a stress test). Because it’s lightweight, you might even run it in production briefly to diagnose an active incident (just remember to turn it off after, as you typically wouldn’t keep it running continuously; for continuous metrics, you’d integrate with something like Prometheus or use dotnet-monitor /metrics). Think of dotnet-counters as your stethoscope – immediate feedback on system vitals. dotnet-dump: Use this when you need a deep forensic analysis of an application’s state. This is typically after you detect a serious problem – e.g., memory leak, unresponsive process, or a crash. It’s the tool to capture evidence (the dump) that you or a specialist can analyze. In some cases, you might use dotnet-dump routinely in production as part of incident response: for example, if an app is hung, first collect a dump, then restart the app to restore service, and later analyze the dump to find what caused the hang. It’s also useful for debugging crashes after the fact (if you configure your environment to produce core dumps on crash, you can load them with dotnet-dump analyze). Choose dotnet-dump if the question you need to answer can’t be solved by metrics or logs alone, and requires inspecting objects, call stacks, or detailed error info offline.

Importantly, these tools are not mutually exclusive – they complement each other. In a single troubleshooting episode, you might use dotnet-counters to notice that “memory usage keeps climbing over time,” then use dotnet-monitor to trigger a GC dump or full dump when memory crosses a threshold, and finally use dotnet-dump to analyze that dump for the leak cause. Or use counters to observe a spike, monitor to capture a trace during the spike, and dump to examine post-mortem. The .NET diagnostics ecosystem is designed such that these tools work together and even build on the same underlying technologies (EventPipe events, SOS debugging APIs, etc.), which is why moving between them is relatively smooth.

Another consideration: cross-platform and environment compatibility. All three tools work on Windows, Linux, and macOS. If you’re in a constrained environment (like only SSH access to a Linux server, no UI), dotnet-counters and dotnet-dump are lifesavers. dotnet-monitor also runs headless and can be operated via web calls, which is very ops-friendly. .NET 8 even extends support to Native AOT apps for EventPipe-based tools (counters, traces, monitor) as long as you enable it, meaning even if you’ve ahead-of-time compiled your app, you can still get metrics and diagnostics, narrowing the gap between fully managed and AOT scenarios.

In summary, use the right tool for the job: Monitor for breadth and automation, Counters for quick insight, Dump for deep dive. When in doubt and if it’s feasible, start with counters (to gauge what’s wrong broadly), then escalate to dumps or advanced collection via monitor if needed.Best Practices for Production Monitoring (Without Pausing Your Services)

Collecting diagnostics in production requires careful balance. Here are some best practices to ensure you get the information you need without undue impact on your running services:Use the least intrusive tool first: If a simple metric will do, don’t start with a full memory dump. For instance, to check if high CPU is an issue, grabbing a quick counter is far gentler than recording a full trace. All the tools above are designed to minimize overhead, but their impact varies. dotnet-counters monitoring of a few counters has negligible overhead. dotnet-monitor streaming logs or metrics is generally low overhead (it’s using the same mechanisms under the hood), though capturing extensive traces or very frequent dumps will cost more. Memory dumps have the largest impact momentarily (pausing the app and consuming memory/disk). So, start small and escalate only as needed. Be mindful of resource limits: As noted, if your app is containerized with a strict memory limit, a full dump could push it over the edge learn.microsoft.com . In such cases, consider using heap dumps or GC dumps (which are smaller). Also ensure you have disk space for any artifacts you collect. If using dotnet-monitor, configure an appropriate egress (e.g. cloud storage) so that large dumps or traces are offloaded and don’t fill the local disk. Automate triggers for known issues: If your app historically has certain failure modes (e.g. sudden memory spikes or deadlocks under high load), use dotnet-monitor’s collection rules to automatically gather data when it happens next devblogs.microsoft.com . This way you don’t have to scramble at 3 AM; the data will be waiting for you. Just ensure your trigger conditions are well-chosen (not too sensitive, to avoid false positives, and with limits to avoid repeated dumps in a short span). Security and access control: When running dotnet-monitor in production, always keep authentication enabled (the default) or run it behind a secure network boundary. The data exposed (memory dumps, logs) can be highly sensitive. Use strong, rotated API keys or integrate with existing auth mechanisms if possible. On the flip side, if you’re giving developers access to production diagnostics, ensure they understand the data sensitivity. Practice in non-production first: Get familiar with these tools in a staging environment. Practice taking dumps and analyzing them, so when a real incident happens, you’re comfortable with the workflow. For example, deliberately leak memory in a test app, use dotnet-counters to see it, dotnet-dump to capture, and analyze to find the leak (much like the scenario in Tess Ferrandez’s blog example) tessferrandez.com . This builds confidence and skill in using the tools effectively. Clean up after diagnostics: Don’t leave diagnostic processes or files lingering longer than needed. If you run dotnet-counters or a dotnet-monitor session temporarily, close it once done (they do use some resources, albeit small). Delete old dump files or trace files from servers to free space (or archive them securely if needed for compliance). In automated rules, set appropriate limits (like one dump per hour as in the example) to avoid spam. Essentially, treat diagnostic data with lifecycle management: collect, analyze, archive or purge. Monitor the monitors: It sounds meta, but keep an eye on the overhead introduced by diagnostics themselves. For dotnet-monitor in production, ensure it’s not consuming too much CPU or memory. It’s generally lightweight until you ask it to do heavy lifting. If you stream very high volumes of logs through it, that could consume CPU/bandwidth – consider sampling or filtering logs to just what you need. If you find the diagnostics are impacting performance, dial back the frequency of collection or the amount of data being captured. Stay updated: The .NET diagnostics tools evolve. .NET 8 has brought improvements (like better support for Native AOT and more counters/metrics). Newer versions might add features or fix bugs (for example, dotnet-monitor’s trigger capabilities were expanded in .NET 6 and beyond). Keep your diagnostic tools updated along with the runtime to benefit from enhancements. The official docs and GitHub repositories (dotnet/diagnostics) often have useful guidance and even troubleshooting FAQs for these tools learn.microsoft.com .

By following these practices, you can gather invaluable diagnostic information from production systems while minimizing any adverse effects. The whole design of the .NET diagnostics suite is geared towards being as transparent as possible – so developers can keep services running and users happy, even while investigating issues under the hood.Conclusion

Modern .NET applications demand robust diagnostics, especially as they run at scale in cloud and container environments. In this article, we’ve journeyed from traditional debugging to production diagnostics, exploring how .NET 8 empowers us with tools like dotnet-monitor, dotnet-counters, and dotnet-dump. These tools help demystify what’s happening inside a running application – whether it’s watching real-time performance metrics, capturing snapshots of memory, or streaming logs and traces on demand.

We learned that dotnet-monitor acts as a diagnostics server, providing easy endpoints to fetch dumps, traces, metrics, and more from live apps. It shines in production by enabling both on-demand and automatic collection of diagnostic artifacts, all without modifying your application code. dotnet-counters, on the other hand, is our quick and nimble friend – perfect for observing live metrics and getting a pulse on app health in real time. And when deeper issues arise, dotnet-dump gives us the ability to freeze the app’s state and dissect it at our leisure, uncovering the root causes of memory leaks, crashes, and hung threads with powerful SOS commands.

Crucially, we emphasized a production-oriented mindset: using these tools to diagnose issues without pausing or disrupting services. By strategically applying the right tool at the right time – counters for quick checks, monitor for broader telemetry, dumps for forensic analysis – you can resolve incidents faster and with minimal customer impact. We also covered best practices to ensure that monitoring itself remains low-impact and secure, such as using triggers, staying mindful of overhead, and safeguarding diagnostic data.

Whether you are a seasoned engineer or new to .NET development, mastering these diagnostic techniques will elevate your ability to maintain and troubleshoot applications in real-world conditions. .NET 8’s diagnostic features, as we’ve seen, are both developer-friendly and production-safe. They represent the shift in our industry towards observability – not just writing code, but understanding its behavior in the wild. So the next time a bug or performance issue sneaks into your production system, you’ll know how to go from debugging to diagnostics, using dotnet-monitor, counters, and dumps to illuminate exactly what’s happening under the hood.

Armed with these tools and knowledge, you can confidently keep your .NET 8 applications running smoothly, and turn those dreaded “it works on my machine” moments into “I see what’s wrong in production and I know how to fix it.” Happy diagnosing!