SonarQube with GitHub Actions and .NET Core 5.x

GitHub Actions are a great devops tool. As you’re upgrading projects to .NET 5, however, you may run into issues with code coverage and static code analysis. I did. I’ll show you today how to get SonarQube working with GitHub Actions and .NET Core 5.x.

Preface

If you’re here, you probably started with the official SonarCloud GitHub Action. That’s where I started. I quickly learned, however, this doesn’t work with .NET 5. The logs gave me the following message:

WARN: Your project contains C# files which cannot be analyzed with the scanner you are using. To analyze C# or VB.NET, you must use the Scanner for 5.x or higher, see https://redirect.sonarsource.com/doc/install-configure-scanner-msbuild.html

At the time of writing, the GitHub Action only works for .NET Core 3.x and below. That means we need to get creative. Documentation is lacking, unfortunately, so I pieced it together from multiple sources.

In my specific use-case I’m working with .NET 5 API projects, MSTest UnitTest projects (though XUnit suffer the same issue), and MSTest integration tests w/WebApiFactory implementations.

SonarQube GitHub Actions

So without further ado, let’s go! Let’s begin with a sample pull-request workflow:

on:
  workflow_dispatch:
  pull_request:
    branches: 
      - develop
      - release/**
      - feature/**

jobs:
  validate-changes:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2.2.0
        with:
          # Disabling shallow clone is recommended for improving relevancy of sonarqube reporting
          fetch-depth: 0

      - name: Setup dotnet
        uses: actions/setup-dotnet@v1
        with:
          dotnet-version: '5.0.x'

      - name: Install dependencies
        run: dotnet restore

      - name: Sonarqube Begin
        run: | 
          dotnet tool install --global dotnet-sonarscanner
          dotnet sonarscanner begin /o:someorg /k:somekey /d:sonar.login=${{ secrets.SONAR_TOKEN }} /s:$GITHUB_WORKSPACE/SonarQube.Analysis.xml

      - name: Build
        run: dotnet build

      - name: Test with the dotnet CLI
        run: dotnet test --settings coverlet.runsettings --logger:trx
        env:
          ASPNETCORE_ENVIRONMENT: Development

      - name: Sonarqube end
        run: dotnet sonarscanner end /d:sonar.login="${{ secrets.SONAR_TOKEN }}"
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

This workflow file does the following steps:

  • Does a full checkout of the repository (as recommended by Sonar)
  • Sets up .NET 5 on the image
  • Installs dependencies for your solution
  • Starts the SonarQube scan phase (more on this later)
  • Builds your solution
  • Runs tests for your solution (more on this later)
  • Ends the SonarQube scan phase

Starting the SonarQube Scan Phase

In this phase, we install the dotnet-scanner as a global tool. Next comes the meat. You will notice that we pass a couple of required parameters and two optional parameters. Everything contained in the settings file (SonarQube.Analysis.xml) could be passed as parameters but I prefer a settings file. The other settings are the organization key (used by SonarCloud), project key, and login. Please also note that in my case the analysis file is in the root GITHUB_WORKSPACE folder where it checked out your repository.

Now let’s dig into the SonarQube.Analysis.xml file. You can get an ultra-basic sample from the SonarScanner for .NET documentation page or a slightly better sample from their GitHub page. For brevity, I’m only including the properties I override.

<?xml version="1.0" encoding="utf-8" ?>
<SonarQubeAnalysisProperties  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://www.sonarsource.com/msbuild/integration/2015/1">

  <Property Name="sonar.host.url">https://sonarcloud.io</Property>
  <Property Name="sonar.exclusions">./BuildScripts/**,./DatabaseMigrations/**</Property>
  <Property Name="sonar.cs.vstest.reportsPaths">**/*.trx</Property>
  <Property Name="sonar.cs.opencover.reportsPaths">**/coverage.opencover.xml</Property> 
</SonarQubeAnalysisProperties>

Now let’s go over the properties here. I set the sonar host url to SonarCloud. Next, I set a few exclusion paths. After that, I set the path for report transaction files as generated by dotnet test. Lastly, I set the coverage paths also as generated by dotnet test. Please note that test transaction and coverage reports are not generated by default. We’ll go over that soon.

Generating data for analysis

Now that our scanner is running, we need to build and test our solution or project. SonarScanner detects the build and test events and uses it to run static code analysis. These are represented in my sample workflow with the dotnet build and dotnet test steps. Running the test step isn’t quite sufficient, however, since it won’t generate code coverage reports or metrics.

Getting code coverage

I previously showed you how to configure the paths for code coverage detection. But how do we generate these? If you refer back to the command, you’ll notice I have a couple of extra parameters dotnet test --settings coverlet.runsettings --logger:trx in it.

When you generate an MSTest or xUnit test project in Visual Studio, they both include the coverlet.collector nuget package. The default coverage report format for coverlet, however, is a coverage.cobertura.xml file. SonarScanner does not recognize this format for .NET. To generate that format, however, you would run dotnet test --collect:"XPlat Code Coverage" (see coverlet’s GitHub page for more info).

The --logger:trx command generates a trx logger results file (see docs). SonarScanner uses this to collect test count metrics.

SonarScanner supports a few test coverage formats (see their documentation). Specifically for C#, however, they support vscoveragexml, dotcover, opencover, and deprecated ncover3. Luckily, coverlet can generate reports using the opencover format. Referring back to my dotnet test command, you’ll note that I pass in a settings file. You can read more about that here but look below for a sample. I place this in my root folder adjacent to the SonarScanner.Analysis.xml file.

<?xml version="1.0" encoding="utf-8" ?>
<RunSettings>
  <DataCollectionRunSettings>
    <DataCollectors>
      <DataCollector friendlyName="XPlat Code Coverage">
        <Configuration>
          <Format>opencover</Format>
        </Configuration>
      </DataCollector>
    </DataCollectors>
  </DataCollectionRunSettings>
</RunSettings>

Ending the scan

This section is pretty minimal. I’m only calling it out to describe what happens behind the scenes. When I run the dotnet sonarscanner end /d:sonar.login="${{ secrets.SONAR_TOKEN }}" command in an action step, this instructs SonarScanner to complete the analysis. The scanner uses the folders as defined in our SonarQube.Analysis.xml file previously. It parses and uploads the test logger transaction files and test results in opencover format.

You may have noticed an oddity with this step. That is, I’m setting the GITHUB_TOKEN environment variable even though I don’t specifically use it. SonarScanner has SCM detection built-in. When running it as part of a GitHub Actions workflow, it complained that I was missing this variable and caused the scan to fail. It uses it to upload the status to the PR.

SonarScanner status uploaded to the PR

Conclusion

Getting SonarQube GitHub Actions working with .NET Core 5.x takes a few extra steps. We can get full code coverage via coverlet by exporting the reports in opencover format.

Other options

While building my workflow I did run across a non-official scanner action. It may work for you. I still ran into issues with it which led me to my own solution.

Credits

Photo by Kowon vn on Unsplash