Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BLD: Use conda(-forge) compilers #38084

Merged
merged 2 commits into from
Nov 26, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 34 additions & 7 deletions doc/source/development/contributing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@ Creating a development environment
----------------------------------

To test out code changes, you'll need to build pandas from source, which
requires a C compiler and Python environment. If you're making documentation
requires a C/C++ compiler and Python environment. If you're making documentation
changes, you can skip to :ref:`contributing.documentation` but you won't be able
to build the documentation locally before pushing your changes.

Expand Down Expand Up @@ -195,6 +195,13 @@ operations. To install pandas from source, you need to compile these C
extensions, which means you need a C compiler. This process depends on which
platform you're using.

If you have setup your environment using ``conda``, the packages ``c-compiler``
and ``cxx-compiler`` will install a fitting compiler for your platform that is
compatible with the remaining conda packages. On Windows and macOS, you will
also need to install the SDKs as they have to be distributed separately.
These packages will be automatically installed by using ``pandas``'s
``environment.yml``.

**Windows**

You will need `Build Tools for Visual Studio 2017
Expand All @@ -206,12 +213,33 @@ You will need `Build Tools for Visual Studio 2017
scrolling down to "All downloads" -> "Tools for Visual Studio 2019".
In the installer, select the "C++ build tools" workload.

You can install the necessary components on the commandline using
`vs_buildtools.exe <https://aka.ms/vs/16/release/vs_buildtools.exe>`_:

.. code::

vs_buildtools.exe --quiet --wait --norestart --nocache ^
--installPath C:\BuildTools ^
--add "Microsoft.VisualStudio.Workload.VCTools;includeRecommended" ^
--add Microsoft.VisualStudio.Component.VC.v141 ^
--add Microsoft.VisualStudio.Component.VC.v141.x86.x64 ^
--add Microsoft.VisualStudio.Component.Windows10SDK.17763

To setup the right paths on the commandline, call
``"C:\BuildTools\VC\Auxiliary\Build\vcvars64.bat" -vcvars_ver=14.16 10.0.17763.0``.

**macOS**

Information about compiler installation can be found here:
To use the ``conda``-based compilers, you will need to install the
Developer Tools using ``xcode-select --install``. Otherwise
information about compiler installation can be found here:
https://devguide.python.org/setup/#macos

**Unix**
**Linux**

For Linux-based ``conda`` installations, you won't have to install any
additional components outside of the conda environment. The instructions
below are only needed if your setup isn't based on conda environments.

Some Linux distributions will come with a pre-installed C compiler. To find out
which compilers (and versions) are installed on your system::
Expand Down Expand Up @@ -243,11 +271,10 @@ Let us know if you have any difficulties by opening an issue or reaching out on
Creating a Python environment
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Now that you have a C compiler, create an isolated pandas development
environment:
Now create an isolated pandas development environment:

* Install either `Anaconda <https://www.anaconda.com/download/>`_ or `miniconda
<https://conda.io/miniconda.html>`_
* Install either `Anaconda <https://www.anaconda.com/download/>`_, `miniconda
<https://conda.io/miniconda.html>`_, or `miniforge <https://github.com/conda-forge/miniforge>`_
* Make sure your conda is up to date (``conda update conda``)
* Make sure that you have :ref:`cloned the repository <contributing.forking>`
* ``cd`` to the pandas source directory
Expand Down
3 changes: 3 additions & 0 deletions environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@ dependencies:
- asv

# building
# The compiler packages are meta-packages and install the correct compiler (activation) packages on the respective platforms.
- c-compiler
- cxx-compiler
- cython>=0.29.21

# code checks
Expand Down
5 changes: 4 additions & 1 deletion scripts/generate_pip_deps_from_conda.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@

import yaml

EXCLUDE = {"python"}
EXCLUDE = {"python", "c-compiler", "cxx-compiler"}
RENAME = {"pytables": "tables", "pyqt": "pyqt5", "dask-core": "dask"}


Expand Down Expand Up @@ -48,6 +48,9 @@ def conda_package_to_pip(package):

break

if package in EXCLUDE:
return

if package in RENAME:
return RENAME[package]

Expand Down