2 MUST USE features for dotnet test debugging

Transcript

Hi, my name is Stu and in this video you're going to discover two different debugging superpowers that come with dotnet test. These help you with two different types of crashes that happen with dotnet tests, which are test hangs and test crashes. To save you time recreating some of the code here today and some of the examples and snippets, members of my Patreon have access to the source code as part of this video. If you're interested in this, there's a link in the description below.

Test crashes often happen at the worst possible time, and they can be really annoying to figure out what's going wrong. The test case that I've got here causes the test process to crash with a simple stack overflow exception. This is to show you the different options that we have with dotnet test. The reality is though, each crash is different, although the steps here should largely be applicable, at least initially.

Dotnet test contains an optional argument called blame crash. When we add this argument, two files get generated in the event of a crash. The first is a sequence file and the second is a memory dump. These files are placed in the test results directory under a special folder. There is a unique GUID which represents the test invocation ID.

The sequence file helps determine the order in which the tests are run. This helps us find any hidden dependencies between tests that we may have. It could also be a starting point for you to begin your investigation about which tests caused the crash itself. If we open this file, we can see that our test cases being invoked, what parameters they had, and whether or not they completed. For completed tests, the sequence file may also contain some logs if they are output by the tests.

The memory dump helps to see information about the current state of the memory, current threads, and more at the time of the crash. To analyze this file, we need the dotnet dump utility which can be installed via.

dotnet tool install -g dotnet-dump. Looking at memory dumps is a vast and complicated subject in all honesty and probably outside of the scope of this video, but there are however a few key commands which you can use to start isolating and start to debug your crashes. The first one is CLR stack. This shows you all managed stacks for all CLR threads. CLR threads shows you the display of the threads that are running at the time of the crash, and you can also use help and then the command name to produce more detailed help information around the command.

If you stick around to the end of this video, I'm going to show you how to add blame crash to every single test invocation that you have without adding it to the command line.

Now we've got the basics of how to diagnose test crashes, we now need to look at test hangs. Test hangs are often caused by a number of issues such as long-running tests or deadlocks. Dotnet test contains a slightly different parameter for dealing with test hangs. In fact, it actually has multiple parameters.

The first one that we're going to look at is blame hang. This puts the CLI into a diagnostic mode much like it does for blame crash. It's important to note that you can run both of these options in parallel. This differs to blame crash in the sense that it only triggers when there is a timeout on a test.

We can control the test timeout value by using the blame hang timeout argument that we pass into the CLI. What this argument does is tells the CLI that we want a timeout of X period to be run for each of our tests. If any test breaches that period of time, then a memory dump and sequence file is generated. We can set the value such as 1h for hours, 1m for minutes, 1s for seconds, or the default value which is in milliseconds or ms.

The next argument that we might want to set is the blame hang dump type, and this can be either full, mini, or none. This parameter basically allows you to control the type of memory dump that's created. A full memory dump will take the entire process, but a mini dump may be smaller but still offers some valuable information. My personal default is to use the full type because I'm not sure exactly what I'm going to need before I debug. The obvious point is that a full dump is a lot larger than a mini dump file.

Both the dump type and the timeout value both implicitly imply the blame hang argument which we would normally supply. This means that we don't need to supply the blame hang argument. We can just supply the timeout or type of dump that we want.

When we look at our test case, we create a very simple deadlock to cause a hang. As we execute the tests with the parameters specified, the process will exit quickly and generate two files in the same way as the crash parameters. These files are the sequence file which shows us the test execution order and the dump file generated when the timer is reached. We can analyze both files in the same way that we can with the crash variations. It's important to realize that the sequence file is only ever generated when there is a hang or a crash. On a successful completion of a test run, the sequence file is not generated.

Long time viewers of this channel know that I love to put things inside the csproj so that everyone gets the same experience without them having to remember all these complex parameters. Whilst diagnosing an issue at work, I actually discovered that the VS Test CLI uses certain project parameters that we can set and get the same experience. So let's take a look at how that gets done.

The key properties that we are interested in are VSTest Blame, which is equivalent to including the --blame parameter. We can set this to true. VSTest Blame Crash, which is equivalent to the blame crash parameter. We can set that to true when we want to have crash dumps. VSTest Blame Hang, which is equivalent to the blame hang parameter, can also be set to true for when we want the hang dump files. VSTest Blame Hang Dump Type, which is equivalent to the CLI command blame hang dump type, can be set to full, mini, or none. And lastly, VSTest Blame Hang Timeout, which is equivalent to the blame hang timeout. You can set this to the desired timeout as you have before.

If we execute our tests again now by running.

dotnet test, we'll be able to see that our tests crash and the correct files are generated in exactly the same way as they were before. This time we haven't supplied any CLI arguments.

If you enjoyed this video, consider subscribing to the YouTube channel for more content like this.