Autograding Model

Java library that autogrades projects based on a configurable set of metrics. Currently, you can select from the following metrics:

• Test statistics (e.g., number of failed tests)

• Code coverage (e.g., line coverage percentage)

• Mutation coverage (e.g., survived mutations' percentage)

• Static analysis warnings (e.g., number of SpotBugs warnings)

This library internally uses the Java libraries analysis-model and coverage-model to read and parse build reports. These results are then aggregated and evaluated. These libraries are the fundament for the following tools:

• Jenkins autograding plugin: Show the autograding results in Jenkins' UI

• GitHub autograding action: Show the autograding results in GitHub pull requests

• GitLab autograding action: Show the autograding results in GitLab merge requests

For each metric you can define the impact on the overall score, and the individual scoring criteria using a JSON configuration:

{
  "tests": {
    "name": "Tests",
    "tools": [
      {
        "id": "test",
        "name": "Tests",
        "pattern": "**/target/*-reports/TEST*.xml"
      }
    ],
    "passedImpact": 0,
    "skippedImpact": -1,
    "failureImpact": -5,
    "maxScore": 100
  },
  "analysis": [
    {
      "name": "Style",
      "id": "style",
      "tools": [
        {
          "id": "checkstyle",
          "name": "CheckStyle",
          "pattern": "**/target/checkstyle-result.xml"
        },
        {
          "id": "pmd",
          "name": "PMD",
          "pattern": "**/target/pmd.xml"
        }
      ],
      "errorImpact": -1,
      "highImpact": -1,
      "normalImpact": -1,
      "lowImpact": -1,
      "maxScore": 100
    },
    {
      "name": "Bugs",
      "id": "bugs",
      "icon": "bug",
      "tools": [
        {
          "id": "spotbugs",
          "name": "SpotBugs",
          "sourcePath": "src/main/java",
          "pattern": "**/target/spotbugsXml.xml"
        }
      ],
      "errorImpact": -3,
      "highImpact": -3,
      "normalImpact": -3,
      "lowImpact": -3,
      "maxScore": 100
    }
  ],
  "coverage": [
    {
      "name": "Code Coverage",
      "tools": [
        {
          "id": "jacoco",
          "name": "Line Coverage",
          "metric": "line",
          "sourcePath": "src/main/java",
          "pattern": "**/target/site/jacoco/jacoco.xml"
        },
        {
          "id": "jacoco",
          "name": "Branch Coverage",
          "metric": "branch",
          "sourcePath": "src/main/java",
          "pattern": "**/target/site/jacoco/jacoco.xml"
        }
      ],
      "maxScore": 100,
      "missedPercentageImpact": -1
    },
    {
      "name": "Mutation Coverage",
      "tools": [
        {
          "id": "pit",
          "name": "Mutation Coverage",
          "metric": "mutation",
          "sourcePath": "src/main/java",
          "pattern": "**/target/pit-reports/mutations.xml"
        }
      ],
      "maxScore": 100,
      "missedPercentageImpact": -1
    }
  ]
}