Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing to build c packages on z/OS #215

Closed
pitmanst opened this issue Oct 2, 2023 · 0 comments · Fixed by pypa/setuptools#4298
Closed

Failing to build c packages on z/OS #215

pitmanst opened this issue Oct 2, 2023 · 0 comments · Fixed by pypa/setuptools#4298

Comments

@pitmanst
Copy link
Contributor

pitmanst commented Oct 2, 2023

Hi. Currently, distutils/setuptools does not provide the ability to build C/C++ packages on z/OS Unix System Services, and there's a few changes / additions that are needed to make it work:

  1. On z/OS, to use the functions provided by a shared library you need to specify the side deck file that is created with the shared library. Without this, you'll hit an error declaring all of these functions unresolved. For example, trying to build the PyPI package hello-c-extension will hit the following errors when trying to pip install it:
IEW2456E 9207 SYMBOL PyModule_Create2 UNRESOLVED.  MEMBER COULD NOT BE INCLUDED
         FROM THE DESIGNATED CALL LIBRARY.
IEW2456E 9207 SYMBOL PyLong_FromLong UNRESOLVED.  MEMBER COULD NOT BE INCLUDED
         FROM THE DESIGNATED CALL LIBRARY.
IEW2665S 40FF MODULE *NULL*  IS NON-EXECUTABLE AND WAS NOT SAVED BECAUSE
         STORENX=NEVER.
  1. There's multiple different supported c compilers available on z/OS, but none are installed by default. Along side this, the different compilers accept different flags / different styles of flags. For example:

To specify changing the c standard being used to c99:
IBM z/OS XL C/C++: -qlanglvl=c99
IBM XL C/C++ V2.4.1 for z/OS 2.4 and 2.5: -qlanglvl=c99 or -std=c99
IBM Open XL C/C++ 1.1 for z/OS and IBM C/C++ For Open Enterprise Languages on z/OS 2.0: -std=c99

So the flags for each compiler need to be set depending on which compiler the user is attempting to use.

  1. On z/OS, there's two ways that a user can install Python on their system. The first is through an installer to the predefined/prefixed location. The second method is simply a pax archive, which can be extracted and used at any location. In the former, there's no issue. In the latter, distutils will pick up the expected installation path rather than where it actually is.

I do have a PR which fixes these issues which I'll be creating shortly.

pitmanst added a commit to pitmanst/distutils that referenced this issue Oct 2, 2023
jaraco pushed a commit to pitmanst/distutils that referenced this issue Feb 12, 2024
jaraco pushed a commit to pitmanst/distutils that referenced this issue Feb 12, 2024
@jaraco jaraco closed this as completed in a55a441 Feb 12, 2024
jaraco added a commit that referenced this issue Feb 12, 2024
Add support for z/OS compilers; Fixes #215
clrpackages pushed a commit to clearlinux-pkgs/pypi-setuptools that referenced this issue Apr 15, 2024
…version 69.5.0

Anderson Bravalheri (3):
      Improve test_rfc822_escape, capturing interoperability requirements
      Improve TestMetadata, capturing interoperability requirements
      Fix interoperability of rfc822_escape with stblib's email library

Avasam (1):
      Allow mypy on PyPy (jaraco/skeleton#111)

Bartosz Sławecki (1):
      Add link to blog entry from jaraco/skeleton#115 above CI build matrix.

DWesl (2):
      CI: Install git on Cygwin CI runner
      CI: Try to fix Cygwin tox configuration.

Dimitri Papadopoulos (1):
      Update URLs in documentation: http:// → https://

Dustin Ingram (6):
      Support PEP 625
      Add news fragment
      Revert changes to distutils
      Try monkeypatching right before we use it instead
      Linting
      Fix canonicalization

Harmen Stoppels (1):
      GNU: use -Wl,-rpath,<dir> instead of -Wl,-R<dir>

Jason R. Coombs (96):
      Copy 'missing_compiler_executable from Python 3.12 and customize it for compatibility with distutils.
      Remove build and dist from excludes. It appears they are not needed and their presence blocks the names of packages like 'builder' and 'distutils'. Ref pypa/distutils#224.
      Mark this function as uncovered.
      Also disable the check
      Remove pin on inflect as it's insufficient to avoid the Rust dependency.
      Remove Sphinx pin. Ref sphinx-doc/sphinx#11662.
      👹 Feed the hobgoblins (delint).
      Extracted method for resolving python lib dir.
      👹 Feed the hobgoblins (delint).
      Update more tests to match the new expectation.
      Rely on always_iterable to conditionally extend the lib_opts.
      Restore integration test with Setuptools
      Include deps from the base config in diffcov.
      Enable complexity check and pycodestyle warnings. Closes jaraco/skeleton#110.
      Extract a method for customizing the compiler for macOS.
      Convert comment to docstring; update wording.
      Create a fixture to patch-out compiler customization on macOS.
      Utilize the fixture for disabling compiler customization on macOS for cxx test. Closes #231.
      Limit mutating global state and simply rely on functools.lru_cache to limit the behavior to a single invocation.
      Use 'extend-select' to avoid disabling the default config. Ref jaraco/skeleton#110.
      In test_build_ext, expose Path objects and use a path builder to build content. Fixes some EncodingWarnings. Ref pypa/distutils#232.
      In support, specify encoding. Ref pypa/distutils#232.
      In test_build_py, rely on tree builder to build trees. Ref pypa/distutils#232.
      Specify encoding in util.byte_compile. Ref pypa/distutils#232.
      Rely on tree builder in test_build_scripts. Ref pypa/distutils#232.
      Rely on Path object to replace the suffix, open the file, and count the lines. Ref pypa/distutils#232.
      Fix EncodingWarnings in test_core. Ref pypa/distutils#232.
      Ran pyupgrade for Python 3.8+ followed by ruff format.
      Rely on tree builder in test_dir_util. Ref pypa/distutils#232.
      Rely on tree builder and path objects. Ref pypa/distutils#232.
      Remove reliance on TempdirManager in test_file_util.
      Rely on tmp_path fixture directly.
      👹 Feed the hobgoblins (delint).
      Rely on tree builder. Ref pypa/distutils#232.
      Specify encoding in test_install. Ref pypa/distutils#232.
      Re-use write_sample_scripts in test_install_scripts. Ref pypa/distutils#232.
      Use Path objects in test_register. Ref pypa/distutils#232.
      Specify encoding in test_sdist. Ref pypa/distutils#232.
      Fix EncodingWarning in test_spawn. Ref pypa/distutils#232.
      Fix EncodingWarnings in test_sdist. Ref pypa/distutils#232.
      Rely on tree builder. Ref pypa/distutils#232.
      Ran pyupgrade for Python 3.8+ followed by ruff format.
      Suppress diffcov error.
      Suppress more diffcov errors.
      Address EncodingWarning in ccompiler. Ref pypa/distutils#232.
      Fix EncodingWarnings in distutils/command/config.py. Ref pypa/distutils#232.
      Fix EncodingWarnings in distutils/config.py. Ref pypa/distutils#232.
      Fix EncodingWarnings in sdist.py. Ref pypa/distutils#232.
      Fix EncodingWarnings in text_file.py. Ref pypa/distutils#232.
      Fix EncodingWarnings in dist.py. Ref pypa/distutils#232.
      Fix EncodingWarning in cygwinccompiler. Ref pypa/distutils#232.
      Fix EncodingWarning in file_util. Ref pypa/distutils#232.
      Suppress EncodingWarnings in pyfakefs. Ref pypa/distutils#232. Workaround for pytest-dev/pyfakefs#957.
      Replaced deprecated cgi module with email module. Ref pypa/distutils#232.
      Fix exception reference in missing_compiler_executable. Ref pypa/distutils#225. Closes pypa/distutils#238.
      Satisfy EncodingWarning by passing the encoding.
      Re-enable ignoring of temporary merge queue branches. Closes jaraco/skeleton#103.
      Fetch unshallow clones in readthedocs. Closes jaraco/skeleton#114.
      Move Python 3.11 out of the test matrix.
      Configure pytest to support namespace packages. Ref pytest-dev/pytest#12112.
      Revert "Suppress EncodingWarnings in pyfakefs. Ref pypa/distutils#232. Workaround for pytest-dev/pyfakefs#957."
      🧎‍♀️ Genuflect to the types.
      Fix ruff.toml syntax and suppress emergent failure.
      Move implementation to monkey.patch.
      Update readme to reflect current state.
      Apply ruff --select UP safe fixes.
      Apply ruff --select UP unsafe fixes.
      👹 Feed the hobgoblins (delint).
      👹 Feed the hobgoblins (delint).
      👹 Feed the hobgoblins (delint).
      Remove unreachable branch
      Extract method for comparing prerelease. Satisfies complexity check.
      Re-organize for brevity.
      Rely on None==None and handle two cases together.
      Refresh RangeMap from jaraco.collections 5.0.1.
      Ruff fixes B007.
      👹 Feed the hobgoblins (delint).
      Fix B026 by moving star arg ahead of keyword arg.
      Extract 'make_iterable' for upload and register commands, avoiding masking loop input variable (B020).
      Fix pointless comparison (B015).
      Remove Python 3.7 compatibility from build_ext
      Remove Python 3.7 compatibility from test_sysconfig.
      Move comment nearer the skip directive. Update wording.
      Add news fragment.
      Omit distutils from coverage checks.
      Fix EncodingWarnings in tools.finalize.
      Bump version: 69.2.0 → 69.3.0
      Refresh unpinned vendored dependencies.
      Ensure that 'backports' is included on older Pythons
      Exclude vendored packages and tools from coverage checks.
      Bump version: 69.3.0 → 69.4.0
      Update to packaging 24
      Remove attempt to canonicalize the version. It's already canonical enough.
      Bump version: 69.3.0 → 69.3.1
      Bump version: 69.4.0 → 69.4.1
      Bump version: 69.4.1 → 69.5.0

Lisandro Dalcin (1):
      Fix accumulating flags after compile/link

Marcel Telka (1):
      Add mypy.ini to MANIFEST.in

Steve Dower (2):
      Fixes pypa/distutils#219 Use sysconfig.get_config_h_filename() to locate pyconfig.h
      Also use sysconfig.get_config_h_filename() to implement distutils.sysconfig version

Steven Pitman (1):
      Add support for z/OS compilers; Fixes pypa/distutils#215
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant