Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDFS-17570 Respect Non-Default HADOOP_ROOT_LOGGER when HADOOP_DAEMON_ROOT_LOGGER is not specified in Daemon mode #6922

Open
wants to merge 3 commits into
base: trunk
Choose a base branch
from

Conversation

VicoWu
Copy link

@VicoWu VicoWu commented Jul 4, 2024

Description of PR

the -Dhadoop.root.logger will use the value of HADOOP_ROOT_LOGGER.
In most cases, hadoop administrator tends to setup HADOOP_ROOT_LOGGER directly, it is effective in non-daemon mode and most hadoop administrator are not even aware of the existence of HADOOP_DAEMON_ROOT_LOGGER.
So when he started up HADOOP as daemon mode, no matter how he set up HADOOP_ROOT_LOGGER, the -Dhadoop.root.logger is not changed because in Daemon mode, the HADOOP_ROOT_LOGGER is set up by HADOOP_DAEMON_ROOT_LOGGER which is in fact a default value. The code is as below:

  if [[ "${HADOOP_DAEMON_MODE}" != "default" ]]; then
    HADOOP_ROOT_LOGGER="${HADOOP_DAEMON_ROOT_LOGGER}"
    if [[ "${HADOOP_SUBCMD_SECURESERVICE}" = true ]]; then
      HADOOP_LOGFILE="hadoop-${HADOOP_SECURE_USER}-${HADOOP_IDENT_STRING}-${HADOOP_SUBCMD}-${HOSTNAME}.log"
    else
      HADOOP_LOGFILE="hadoop-${HADOOP_IDENT_STRING}-${HADOOP_SUBCMD}-${HOSTNAME}.log"
    fi
  fi

This makes log setup and troubleshooting becomes very time consuming before we finally figure out the reason and the best practice without reading the code.

My change is:
In daemon mode

  • If user has customize(changed) HADOOP_ROOT_LOGGER but no HADOOP_DAEMON_ROOT_LOGGER, then we will respect HADOOP_ROOT_LOGGER as the final -Dhadoop.root.logger value, instead of setting up HADOOP_ROOT_LOGGER with the default HADOOP_DAEMON_ROOT_LOGGER and print a warning
  • In other cases, we will use HADOOP_DAEMON_ROOT_LOGGER(no matter it is default or customized), we will setup the HADOOP_ROOT_LOGGER with this customized HADOOP_DAEMON_ROOT_LOGGER

How was this patch tested?

Testing Case Expected Result
Customized HADOOP_LOGLEVEL Respect HADOOP_LOGLEVEL
Without customizing anything Respect default HADOOP_DAEMON_ROOT_LOGGER in daemon mode
Customize the HADOOP_DAEMON_ROOT_LOGGER without customizing HADOOP_ROOT_LOGGER Respect customized HADOOP_DAEMON_ROOT_LOGGER
Customize the HADOOP_ROOT_LOGGER but not customize HADOOP_DAEMON_ROOT_LOGGER Respect customized HADOOP_ROOT_LOGGER with a warning
Customize both HADOOP_ROOT_LOGGER and HADOOP_DAEMON_ROOT_LOGGER Respect customized HADOOP_DAEMON_ROOT_LOGGER
  1. Customize the HADOOP_LOGLEVEL, should respect the HADOOP_LOGLEVEL and HADOOP_DAEMON_ROOT_LOGGER

    export HADOOP_LOGLEVEL=DEBUG
    root@rccd101-6a:/home/hadoop/current-hadoop# ps -ef|grep NameNode
    hadoop   2852464       1 45 04:04 pts/1    00:00:09 /usr/lib/jvm/java-11-openjdk-amd64/bin/java -Dproc_namenode -Djava.net.preferIPv4Stack=true -Dhdfs.audit.logger=INFO,NullAppender -Dlog4j.configuration=file:/home/hadoop/current-hadoop/etc/hadoop/log4j-namenode.properties -XX:+UseG1GC -Xmx4096m -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=1029 -Dhadoop.security.logger=ERROR,RFAS -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005 -Dyarn.log.dir=/home/hadoop/hadoop-3.2.4/../logs -Dyarn.log.file=test-host-name.dev.corp.com.log -Dyarn.home.dir=/home/hadoop/hadoop-3.2.4 -Dyarn.root.logger=INFO,console -Djava.library.path=/home/hadoop/hadoop-3.2.4/lib/native -Dhadoop.log.dir=/home/hadoop/hadoop-3.2.4/../logs -Dhadoop.log.file=test-host-name.dev.corp.com.log -Dhadoop.home.dir=/home/hadoop/hadoop-3.2.4 -Dhadoop.id.str=hadoop -Dhadoop.root.logger=DEBUG,RFA -Dhadoop.policy.file=hadoop-policy.xml org.apache.hadoop.hdfs.server.namenode.NameNode
    
  2. Without customizing anything, should respect default HADOOP_DAEMON_ROOT_LOGGER

    root@rccd101-6a:/home/hadoop/current-hadoop# ps -ef|grep NameNode
    hadoop   2855220       1 43 04:05 pts/1    00:00:07 /usr/lib/jvm/java-11-openjdk-amd64/bin/java -Dproc_namenode -Djava.net.preferIPv4Stack=true -Dhdfs.audit.logger=INFO,NullAppender -Dlog4j.configuration=file:/home/hadoop/current-hadoop/etc/hadoop/log4j-namenode.properties -XX:+UseG1GC -Xmx4096m -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=1029 -Dhadoop.security.logger=ERROR,RFAS -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005 -Dyarn.log.dir=/home/hadoop/hadoop-3.2.4/../logs -Dyarn.log.file=test-host-name.dev.corp.com.log -Dyarn.home.dir=/home/hadoop/hadoop-3.2.4 -Dyarn.root.logger=INFO,console -Djava.library.path=/home/hadoop/hadoop-3.2.4/lib/native -Dhadoop.log.dir=/home/hadoop/hadoop-3.2.4/../logs -Dhadoop.log.file=test-host-name.dev.corp.com.log -Dhadoop.home.dir=/home/hadoop/hadoop-3.2.4 -Dhadoop.id.str=hadoop -Dhadoop.root.logger=INFO,RFA -Dhadoop.policy.file=hadoop-policy.xml org.apache.hadoop.hdfs.server.namenode.NameNode
  3. Customize the HADOOP_DAEMON_ROOT_LOGGER without customizing HADOOP_ROOT_LOGGER, should respect customized HADOOP_DAEMON_ROOT_LOGGER

    export HADOOP_DAEMON_ROOT_LOGGER=TRACE,RFA
     root@rccd101-6a:/home/hadoop/current-hadoop# ps -ef|grep NameNode
     hadoop   2857093       1 99 04:06 pts/1    00:00:07 /usr/lib/jvm/java-11-openjdk-amd64/bin/java -Dproc_namenode -Djava.net.preferIPv4Stack=true -Dhdfs.audit.logger=INFO,NullAppender -Dlog4j.configuration=file:/home/hadoop/current-hadoop/etc/hadoop/log4j-namenode.properties -XX:+UseG1GC -Xmx4096m -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=1029 -Dhadoop.security.logger=ERROR,RFAS -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005 -Dyarn.log.dir=/home/hadoop/hadoop-3.2.4/../logs -Dyarn.log.file=test-host-name.dev.corp.com.log -Dyarn.home.dir=/home/hadoop/hadoop-3.2.4 -Dyarn.root.logger=INFO,console -Djava.library.path=/home/hadoop/hadoop-3.2.4/lib/native -Dhadoop.log.dir=/home/hadoop/hadoop-3.2.4/../logs -Dhadoop.log.file=test-host-name.dev.corp.com.log -Dhadoop.home.dir=/home/hadoop/hadoop-3.2.4 -Dhadoop.id.str=hadoop -Dhadoop.root.logger=TRACE,RFA -Dhadoop.policy.file=hadoop-policy.xml org.apache.hadoop.hdfs.server.namenode.NameNode
     root     2857504 2745542  0 04:07 pts/1    00:00:00 grep --color=auto NameNode
  4. Customize the HADOOP_ROOT_LOGGER but not customize HADOOP_DAEMON_ROOT_LOGGER, should respect customized HADOOP_ROOT_LOGGER with a warning

    export HADOOP_ROOT_LOGGER=TRACE,console
    root@rccd101-6a:/home/hadoop/current-hadoop# sudo -u hadoop bin/hdfs --daemon stop namenode;
    Using customized HADOOP_ROOT_LOGGER as final hadoop.root.logger. Recommending using HADOOP_LOGLEVEL or customize HADOOP_DAEMON_ROOT_LOGGER for logging in daemon mode.
    root@rccd101-6a:/home/hadoop/current-hadoop# ps -ef|grep NameNode
    hadoop   2874729       1 99 04:16 pts/1    00:00:07 /usr/lib/jvm/java-11-openjdk-amd64/bin/java -Dproc_namenode -Djava.net.preferIPv4Stack=true -Dhdfs.audit.logger=INFO,NullAppender -Dlog4j.configuration=file:/home/hadoop/current-hadoop/etc/hadoop/log4j-namenode.properties -XX:+UseG1GC -Xmx4096m -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=1029 -Dhadoop.security.logger=ERROR,RFAS -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005 -Dyarn.log.dir=/home/hadoop/hadoop-3.2.4/../logs -Dyarn.log.file=test-host-name.dev.corp.com.log -Dyarn.home.dir=/home/hadoop/hadoop-3.2.4 -Dyarn.root.logger=INFO,console -Djava.library.path=/home/hadoop/hadoop-3.2.4/lib/native -Dhadoop.log.dir=/home/hadoop/hadoop-3.2.4/../logs -Dhadoop.log.file=test-host-name.dev.corp.com.log -Dhadoop.home.dir=/home/hadoop/hadoop-3.2.4 -Dhadoop.id.str=hadoop -Dhadoop.root.logger=TRACE,console -Dhadoop.policy.file=hadoop-policy.xml org.apache.hadoop.hdfs.server.namenode.NameNode
  5. Customize both HADOOP_ROOT_LOGGER and HADOOP_DAEMON_ROOT_LOGGER, should respect customized HADOOP_DAEMON_ROOT_LOGGER

    export HADOOP_DAEMON_ROOT_LOGGER=TRACE,RFA
    export HADOOP_ROOT_LOGGER=TRACE,console
    root@rccd101-6a:/home/hadoop/current-hadoop# ps -ef|grep NameNode
    hadoop   2877521       1 61 04:18 pts/1    00:00:07 /usr/lib/jvm/java-11-openjdk-amd64/bin/java -Dproc_namenode -Djava.net.preferIPv4Stack=true -Dhdfs.audit.logger=INFO,NullAppender -Dlog4j.configuration=file:/home/hadoop/current-hadoop/etc/hadoop/log4j-namenode.properties -XX:+UseG1GC -Xmx4096m -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=1029 -Dhadoop.security.logger=ERROR,RFAS -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005 -Dyarn.log.dir=/home/hadoop/hadoop-3.2.4/../logs -Dyarn.log.file=test-host-name.dev.corp.com.log -Dyarn.home.dir=/home/hadoop/hadoop-3.2.4 -Dyarn.root.logger=INFO,console -Djava.library.path=/home/hadoop/hadoop-3.2.4/lib/native -Dhadoop.log.dir=/home/hadoop/hadoop-3.2.4/../logs -Dhadoop.log.file=test-host-name.dev.corp.com.log -Dhadoop.home.dir=/home/hadoop/hadoop-3.2.4 -Dhadoop.id.str=hadoop -Dhadoop.root.logger=TRACE,RFA -Dhadoop.policy.file=hadoop-policy.xml org.apache.hadoop.hdfs.server.namenode.NameNode

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

@VicoWu VicoWu marked this pull request as draft July 4, 2024 03:25
@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 10m 8s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 shelldocs 0m 0s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 💚 mvninstall 32m 43s trunk passed
+1 💚 mvnsite 0m 48s trunk passed
+1 💚 shadedclient 19m 42s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 30s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 0m 41s the patch passed
+1 💚 shellcheck 0m 2s No new issues.
-1 ❌ shadedclient 2m 13s patch has errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 0m 55s /patch-unit-hadoop-common-project_hadoop-common.txt hadoop-common in the patch failed.
+1 💚 asflicense 0m 23s The patch does not generate ASF License warnings.
69m 16s
Reason Tests
Failed TAP tests hadoop_basic_init.bats.tap
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/1/artifact/out/Dockerfile
GITHUB PR #6922
Optional Tests dupname asflicense mvnsite unit codespell detsecrets shellcheck shelldocs
uname Linux 72b4515ca60b 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / e65cb38
TAP logs https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/1/artifact/out/patch-hadoop-common-project_hadoop-common.tap
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/1/testReport/
Max. process+thread count 552 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/1/console
versions git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@VicoWu VicoWu marked this pull request as ready for review July 4, 2024 07:51
@VicoWu VicoWu marked this pull request as draft July 4, 2024 07:51
@VicoWu VicoWu changed the title refine HADOOP_ROOT_LOGGER and HADOOP_DAEMON_ROOT_LOGGER logic HDFS-17570 Respect Non-Default HADOOP_ROOT_LOGGER when HADOOP_DAEMON_ROOT_LOGGER is not specified in Daemon mode Jul 4, 2024
@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 21s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 shelldocs 0m 0s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 💚 mvninstall 35m 33s trunk passed
+1 💚 mvnsite 0m 53s trunk passed
+1 💚 shadedclient 23m 12s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 34s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 0m 46s the patch passed
+1 💚 shellcheck 0m 2s No new issues.
+1 💚 shadedclient 23m 19s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 1m 2s hadoop-common in the patch passed.
+1 💚 asflicense 0m 24s The patch does not generate ASF License warnings.
87m 23s
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/2/artifact/out/Dockerfile
GITHUB PR #6922
Optional Tests dupname asflicense mvnsite unit codespell detsecrets shellcheck shelldocs
uname Linux fd88dc2efd10 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 13dcc40
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/2/testReport/
Max. process+thread count 557 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/2/console
versions git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 18s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 shelldocs 0m 0s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 💚 mvninstall 32m 34s trunk passed
+1 💚 mvnsite 0m 47s trunk passed
+1 💚 shadedclient 19m 44s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 29s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 0m 44s the patch passed
+1 💚 shellcheck 0m 2s No new issues.
+1 💚 shadedclient 19m 29s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 0m 59s hadoop-common in the patch passed.
+1 💚 asflicense 0m 26s The patch does not generate ASF License warnings.
76m 39s
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/3/artifact/out/Dockerfile
GITHUB PR #6922
Optional Tests dupname asflicense mvnsite unit codespell detsecrets shellcheck shelldocs
uname Linux fbf8e0079eba 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 6a82029
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/3/testReport/
Max. process+thread count 552 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/3/console
versions git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@VicoWu VicoWu marked this pull request as ready for review July 22, 2024 02:23
@VicoWu VicoWu force-pushed the refine-daemon-root-log-level branch from 6a82029 to 83659c7 Compare July 22, 2024 02:24
@VicoWu
Copy link
Author

VicoWu commented Jul 22, 2024

@aw-was-here @aajisaka Would you please take a look for this?

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 4m 1s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 shelldocs 0m 0s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 💚 mvninstall 36m 47s trunk passed
+1 💚 mvnsite 0m 47s trunk passed
+1 💚 shadedclient 21m 43s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 30s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 0m 41s the patch passed
+1 💚 shellcheck 0m 2s No new issues.
+1 💚 shadedclient 22m 14s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 0m 55s hadoop-common in the patch passed.
+1 💚 asflicense 0m 22s The patch does not generate ASF License warnings.
89m 27s
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/4/artifact/out/Dockerfile
GITHUB PR #6922
Optional Tests dupname asflicense mvnsite unit codespell detsecrets shellcheck shelldocs
uname Linux 5da9cb7bc23f 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 83659c7
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/4/testReport/
Max. process+thread count 552 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/4/console
versions git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@github-actions github-actions bot added the HDFS label Jul 25, 2024
@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 21s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 shelldocs 0m 0s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 13m 43s Maven dependency ordering for branch
+1 💚 mvninstall 23m 9s trunk passed
+1 💚 compile 10m 46s trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚 compile 10m 9s trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚 checkstyle 2m 13s trunk passed
+1 💚 mvnsite 1m 56s trunk passed
+1 💚 javadoc 1m 31s trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚 javadoc 1m 56s trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚 spotbugs 3m 40s trunk passed
+1 💚 shadedclient 28m 17s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 21s Maven dependency ordering for patch
+1 💚 mvninstall 1m 19s the patch passed
+1 💚 compile 10m 50s the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚 javac 10m 50s the patch passed
+1 💚 compile 9m 55s the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚 javac 9m 55s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 2m 19s the patch passed
+1 💚 mvnsite 1m 45s the patch passed
+1 💚 shellcheck 0m 2s No new issues.
+1 💚 javadoc 1m 19s the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚 javadoc 1m 40s the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚 spotbugs 3m 47s the patch passed
+1 💚 shadedclient 28m 51s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 16m 20s hadoop-common in the patch passed.
-1 ❌ unit 207m 16s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch passed.
+1 💚 asflicense 0m 47s The patch does not generate ASF License warnings.
383m 43s
Reason Tests
Failed junit tests hadoop.hdfs.server.namenode.TestAddOverReplicatedStripedBlocks
hadoop.hdfs.server.namenode.TestDefaultBlockPlacementPolicy
hadoop.hdfs.server.namenode.TestReencryption
hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks
hadoop.hdfs.server.balancer.TestBalancer
hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
hadoop.hdfs.TestReconstructStripedFile
hadoop.hdfs.TestErasureCodingMultipleRacks
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/5/artifact/out/Dockerfile
GITHUB PR #6922
Optional Tests dupname asflicense mvnsite unit codespell detsecrets shellcheck shelldocs compile javac javadoc mvninstall shadedclient spotbugs checkstyle
uname Linux 4dbaabf14a14 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / cf8ead0
Default Java Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/5/testReport/
Max. process+thread count 4188 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/5/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@VicoWu VicoWu force-pushed the refine-daemon-root-log-level branch from cf8ead0 to 0535f2d Compare July 26, 2024 04:09
@github-actions github-actions bot removed the HDFS label Jul 26, 2024
@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 19s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 shelldocs 0m 0s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 💚 mvninstall 36m 30s trunk passed
+1 💚 mvnsite 0m 51s trunk passed
+1 💚 shadedclient 24m 51s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 29s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 0m 40s the patch passed
+1 💚 shellcheck 0m 2s No new issues.
+1 💚 shadedclient 25m 0s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 0m 56s hadoop-common in the patch passed.
+1 💚 asflicense 0m 20s The patch does not generate ASF License warnings.
91m 26s
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/6/artifact/out/Dockerfile
GITHUB PR #6922
Optional Tests dupname asflicense mvnsite unit codespell detsecrets shellcheck shelldocs
uname Linux 5068e7a51e84 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 0535f2d
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/6/testReport/
Max. process+thread count 551 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/6/console
versions git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants