run_sys_tests doesn't run under Python-3 and ncar_pylib

[ekluzek]

Brief summary of bug

run_sys_tests fails when running with python-3 and the ncar_pylib environment.

General bug information

CTSM version you are using: ctsm1.0.dev035-7-g0b2970e (will be ctsm1.0.dev036)
Does this bug cause significantly incorrect results in the model's science? No
Configurations affected: run_sys_tests on cheyenne with python3 and ncar_pylib

Details of bug

It fails in the use of subprocess which is a python module for PBS qsub. It seems to work fine with either python-2.7 or python-3.6 on their own, as well as python-2.7 with ncar_pylib, but fails when you combine python-3.6 with ncar_pylib.

Important details of your setup / configuration so we can reproduce the bug

you have to both load python/3 AND run the ncar python environment...

Currently Loaded Modules Matching: python

1. python/3.6.8

[(NPL) ] erik/ctsm_cisofix> which ncar_pylib
ncar_pylib: aliased to source /gpfs/u/apps/ch/opt/python/3.6.8/gnu/7.3.0/bin/ncar_pylib.csh

Important output or errors that show the problem

[(NPL) ] erik/ctsm_cisofix> ./run_sys_tests -s clm_short --skip-compare --skip-generate
Testroot: /glade/scratch/erik/tests_0424-103123ch

Current hash: 0b2970e Change ciso test so that you don't have to exclude all the carbon isotope fields

cisofix...ekluzek/cisofix

?? cases/
?? run_sys_test.log

manage_externals status:
Processing externals description file : Externals.cfg
Processing externals description file : Externals_CLM.cfg
Processing externals description file : Externals_CISM.cfg
Checking status of externals: clm, fates, ptclm, cism, source_cism, rtm, mosart, cime,
./cime
clean sandbox, on ctsm/ctsm1.0/cime5.7.9/n03
./components/cism
clean sandbox, on release-cesm2.0.04
./components/cism/source_cism
clean sandbox, on release-cism2.1.03
./components/mosart
clean sandbox, on release-cesm2.0.00
./components/rtm
clean sandbox, on release-cesm2.0.00
./src/fates
clean sandbox, on fates_s1.8.1_a3.0.0_rev2
./tools/PTCLM
clean sandbox, on PTCLM2_180611

Traceback (most recent call last):
File "./run_sys_tests", line 22, in
main(cime_path=cime_path)
File "/gpfs/fs1/work/erik/ctsm_cisofix/python/ctsm/run_sys_tests.py", line 72, in main
extra_create_test_args=args.extra_create_test_args)
File "/gpfs/fs1/work/erik/ctsm_cisofix/python/ctsm/run_sys_tests.py", line 151, in run_sys_tests
dry_run=dry_run)
File "/gpfs/fs1/work/erik/ctsm_cisofix/python/ctsm/run_sys_tests.py", line 464, in _run_test_suite
dry_run=dry_run)
File "/gpfs/fs1/work/erik/ctsm_cisofix/python/ctsm/run_sys_tests.py", line 490, in _run_create_test
dry_run=dry_run)
File "/gpfs/fs1/work/erik/ctsm_cisofix/python/ctsm/joblauncher/job_launcher_base.py", line 89, in run_command
self.run_command_impl(command, stdout_path, stderr_path)
File "/gpfs/fs1/work/erik/ctsm_cisofix/python/ctsm/joblauncher/job_launcher_qsub.py", line 29, in run_command_impl
(out, err) = qsub_process.communicate(' '.join(command))
File "/glade/u/apps/ch/opt/python/3.6.8/gnu/7.3.0/lib/python3.6/subprocess.py", line 863, in communicate
stdout, stderr = self._communicate(input, endtime, timeout)
File "/glade/u/apps/ch/opt/python/3.6.8/gnu/7.3.0/lib/python3.6/subprocess.py", line 1519, in _communicate
input_view = memoryview(self._input)
TypeError: memoryview: a bytes-like object is required, not 'str'

[billsacks]

I think we may need universal_newlines in the call to subprocess.Popen. I'll try that out.

[ekluzek]

Dick Valent of CISL consulting is looking into it. Here's the CISL ticket for it 287972.

[billsacks]

My sense is that this is a legitimate problem with run_sys_tests using recent python versions. See #696 for a fix. Please go ahead and merge that PR if you feel it fixes the problem.