Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: add aggregate test coverage collection of gax within the showcase and gax modules #1430

Merged
merged 22 commits into from
Mar 16, 2023
Merged

chore: add aggregate test coverage collection of gax within the showcase and gax modules #1430

Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
d7c919a
chore: add aggregate test coverage collection for showcase and gax
mpeddada1 Mar 2, 2023
135022c
add profile to collect test coverage
mpeddada1 Mar 2, 2023
d2be02f
add fmt-maven-plugin
mpeddada1 Mar 2, 2023
22cda89
add maven compiler version property
mpeddada1 Mar 2, 2023
cd0de15
skip fmt
mpeddada1 Mar 2, 2023
cddb9f2
add version to fmt plugin
mpeddada1 Mar 2, 2023
c116da1
add jar plugin to showcase
mpeddada1 Mar 2, 2023
208cfff
add jar plugin to gapic-showcase
mpeddada1 Mar 2, 2023
02ef0cb
empty commit to rerun builds
mpeddada1 Mar 2, 2023
7f720b8
empty commit
mpeddada1 Mar 3, 2023
68557fe
fix syntax in dep declaration
mpeddada1 Mar 3, 2023
57ddf93
remove spaces
mpeddada1 Mar 3, 2023
fcc5257
add README for coverage-report module
mpeddada1 Mar 6, 2023
45a0987
replace monorepo with repsitory in README
mpeddada1 Mar 6, 2023
47cf349
add profile to specify modules
mpeddada1 Mar 10, 2023
d86667f
Merge branch 'main' of github.com:googleapis/gapic-generator-java int…
mpeddada1 Mar 15, 2023
42f13c0
undo workaround
mpeddada1 Mar 15, 2023
f813b69
modify unit test step and clean up README
mpeddada1 Mar 15, 2023
47fc1db
try calling mvn install before bazelisk
mpeddada1 Mar 15, 2023
879784b
undo mvn compile replacement
mpeddada1 Mar 15, 2023
5db0d69
install mvn modules before integration tests
mpeddada1 Mar 15, 2023
31365de
add enableTestCoverage parameter and fix showcase compilation
mpeddada1 Mar 16, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 2 additions & 5 deletions .github/workflows/ci-maven.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ jobs:
- run: java -version
- name: Unit Tests
run: |
mvn install --batch-mode --no-transfer-progress -Dcheckstyle.skip \
mvn test --batch-mode --no-transfer-progress -Dcheckstyle.skip \
suztomo marked this conversation as resolved.
Show resolved Hide resolved
-Dfmt.skip
- run: bazelisk version
- name: Integration Tests
Expand Down Expand Up @@ -71,10 +71,7 @@ jobs:
set -x
export JAVA_HOME=$JAVA11_HOME
export PATH=${JAVA_HOME}/bin:$PATH

# Why not compile? It's because the process needs to package jar so
# that gapic-generator-java module can use testlib modules of gax.
mvn verify --batch-mode --no-transfer-progress -Dcheckstyle.skip \
mvn compile --batch-mode --no-transfer-progress -Dcheckstyle.skip \
-DskipTests -Dfmt.skip
- name: Run test with Java 8 for all modules except gapic-generator-java
shell: bash
Expand Down
23 changes: 23 additions & 0 deletions coverage-report/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
## Coverage Report
mpeddada1 marked this conversation as resolved.
Show resolved Hide resolved

This module gathers aggregated jacoco test coverage metrics across the `gax-java` and `showcase` modules. The purpose of
the metrics is to provide insights into how much of GAX code is being exercised by showcase and GAX tests and where
(unit tests versus integration tests). They will also provide information on any change in coverage observed
as showcase tests continue to be added to the repository.

### Unit Test Coverage
In order to view aggregate unit test coverage of GAX in both `gax-java` and `showcase`:

1. At the root of the repository, run `mvn clean test -DenableTestCoverage`.
2. The metrics can be found at `gapic-generator-java/coverage-report/target/site/jacoco-aggregate/index.html`

![Screenshot 2023-03-03 at 6 32 50 PM](https://user-images.githubusercontent.com/66699525/222854612-787b4dde-f9a3-469a-8227-8f46dc0a4a20.png)

### Integration Test Coverage

In order to view aggregate integration test coverage of GAX in both `gax-java` and `showcase`:

1. At the root of the repository, run `mvn clean verify -DskipUnitTests -DenableTestCoverage -Penable-integration-tests`.
2. The metrics can be found at `gapic-generator-java/coverage-report/target/site/jacoco-aggregate/index.html`

![Screenshot 2023-03-03 at 6 36 26 PM](https://user-images.githubusercontent.com/66699525/222854973-f8a96f01-abc1-4e6b-9ab8-99b5e50dec6a.png)
81 changes: 81 additions & 0 deletions coverage-report/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
<?xml version='1.0' encoding='UTF-8'?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.google.cloud</groupId>
<artifactId>coverage-report</artifactId>
<packaging>pom</packaging>
<version>0.0.1-SNAPSHOT</version>
<name>Jacoco Aggregrate Test Coverage Report</name>

<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<checkstyle.skip>true</checkstyle.skip>
<clirr.skip>true</clirr.skip>
<enforcer.skip>true</enforcer.skip>
<maven.compiler.release>8</maven.compiler.release>
<fmt.skip>true</fmt.skip>
<sonar.coverage.jacoco.xmlReportPaths>
${project.basedir}/coverage-report/target/site/
jacoco-aggregate/jacoco.xml
</sonar.coverage.jacoco.xmlReportPaths>
</properties>

<dependencies>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>gapic-showcase</artifactId>
<version>0.0.1-SNAPSHOT</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.google.api</groupId>
<artifactId>gax</artifactId>
<version>2.23.3-SNAPSHOT</version> <!-- {x-version-update:gax:current} -->
</dependency>
<dependency>
<groupId>com.google.api</groupId>
<artifactId>gax-grpc</artifactId>
<version>2.23.3-SNAPSHOT</version> <!-- {x-version-update:gax:current} -->
</dependency>
<dependency>
<groupId>com.google.api</groupId>
<artifactId>gax-httpjson</artifactId>
<version>0.108.3-SNAPSHOT</version> <!-- {x-version-update:gax:current} -->
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.8</version>
<executions>
<execution>
<id>unit-tests-report-aggregate</id>
<goals>
<goal>report-aggregate</goal>
</goals>
<phase>test</phase>
</execution>
<execution>
<id>integration-tests-report-aggregate</id>
<goals>
<goal>report-aggregate</goal>
</goals>
<phase>integration-test</phase>
</execution>
</executions>
</plugin>
<plugin>
<groupId>com.coveo</groupId>
<artifactId>fmt-maven-plugin</artifactId>
<version>2.9</version>
</plugin>
</plugins>
</build>


</project>
83 changes: 82 additions & 1 deletion gapic-generator-java-pom-parent/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,8 +95,89 @@
</plugins>
</build>
</profile>
</profiles>

<profile>
<id>test-coverage</id>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description does a great job clarifying how this is meant to be used, but that may get lost in obscurity once the PR is merged. Could you add some information in an appropriate .md to explain the different expected workflows + report options?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, done. I've added a README to the coverage-report module.

<activation>
<property>
<name>enableTestCoverage</name>
</property>
</activation>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M8</version>
<configuration>
<!-- Excludes integration tests and smoke tests when unit tests are run -->
<excludes>
<exclude>**/*SmokeTest.java</exclude>
<exclude>**/IT*.java</exclude>
</excludes>
<reportNameSuffix>sponge_log</reportNameSuffix>
<argLine>${surefire.jacoco.args}</argLine>
<skipTests>${skipUnitTests}</skipTests>
</configuration>
</plugin>

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>3.0.0-M8</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
<configuration>
<forkedProcessTimeoutInSeconds>3600</forkedProcessTimeoutInSeconds>
<reportNameSuffix>sponge_log</reportNameSuffix>
<includes>
<include>**/IT*.java</include>
<include>**/*SmokeTest.java</include>
</includes>
<argLine>${failsafe.jacoco.args}</argLine>
</configuration>
</plugin>
</plugins>
</pluginManagement>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.8</version>
<executions>
<execution>
<id>unit-test-execution</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<propertyName>surefire.jacoco.args</propertyName>
</configuration>
</execution>
<execution>
<id>integration-test-execution</id>
<phase>pre-integration-test</phase>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<propertyName>failsafe.jacoco.args</propertyName>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>

<repositories>
<repository>
<id>google-maven-central-copy</id>
Expand Down
13 changes: 13 additions & 0 deletions pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,19 @@
</plugins>
</build>
</profile>

<profile>
<id>activate-test-coverage</id>
<activation>
<property>
<name>enableTestCoverage</name>
</property>
</activation>
<modules>
<module>showcase</module>
<module>coverage-report</module>
</modules>
</profile>
</profiles>

</project>
2 changes: 1 addition & 1 deletion showcase/gapic-showcase/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<modelVersion>4.0.0</modelVersion>
<groupId>com.google.cloud</groupId>
<artifactId>gapic-showcase</artifactId>
<version>0.0.1-SHAPSHOT</version>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>GAPIC Showcase Client</name>
<description>
Expand Down