Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python 3 release - will be 0.7.0 #594

Closed
mikepurvis opened this issue Jan 21, 2020 · 95 comments · Fixed by #648
Closed

Python 3 release - will be 0.7.0 #594

mikepurvis opened this issue Jan 21, 2020 · 95 comments · Fixed by #648

Comments

@mikepurvis
Copy link
Member

Would be great to have a released version of catkin_tools which works well for Python 3. Myself and some colleagues have been using this from source in Py3 virtualenvs for a few weeks and not discovered anything horrific; are there any objections before we ask @wjwwood to pull the lever?

@timonegk
Copy link
Member

I am also using a source build of catkin_tools with python3 without any problems and would appreciate a new release.

@sloretz
Copy link
Contributor

sloretz commented Mar 27, 2020

A new release would be appreciated on Debian Buster and Ubuntu Focal. It looks like it depends on python3-trollius, but that package doesn't exist on those platforms. Looks like that has been fixed on master already in #574

Package: python3-catkin-tools
Priority: optional
Section: python
Installed-Size: 786
Maintainer: William Woodall <william@osrfoundation.org>
Architecture: all
Version: 0.4.5-1
Depends: python3-catkin-pkg (>= 0.2.9), python3-pkg-resources, python3-trollius, python3-yaml, python3:any (>= 3.2~), python3-setuptools
Conflicts: python-catkin-tools
Filename: pool/main/p/python3-catkin-tools/python3-catkin-tools_0.4.5-1_all.deb
SHA512: bb6d05238d8a18685c2370b70cbe0f95edf433044ae1f8c59c1fd96742e9557cc620acb1d8094c3ce43452576415075c86404866fecce59eac4d8e46bff2a26c
Size: 313732
SHA256: 6abf1d25c38cef549d3dc1c3abd3f927fd5d71633a904c70563ddc34a1931d6c
SHA1: 5d040daee76c9e90893c68641656651ecaa8772f
MD5sum: b36033eecbf8f1050bd6277ba0105c02
Description: Command line tools for working with catkin.
 Provides command line tools for working with catkin.

@timonegk
Copy link
Member

I guess #597 should be closed first, I suggested a fix in #601.

@DLu
Copy link
Contributor

DLu commented Apr 23, 2020

👍

@tykurtz
Copy link

tykurtz commented May 14, 2020

Cutting a new release would be greatly appreciated. Running 0.4.5 on python3 #510 and #558 (comment) are still open issues which were fixed by switching to master.

@nim65s
Copy link

nim65s commented May 28, 2020

PyPI release is also broken for Python 3.8, because trollius defines async, which is now a reserved keyword:

Traceback (most recent call last):
  File "/home/nim/.local/bin/catkin", line 5, in <module>
    from catkin_tools.commands.catkin import main
  File "/home/nim/.local/lib/python3.8/site-packages/catkin_tools/commands/catkin.py", line 28, in <module>
    from catkin_tools.common import is_tty
  File "/home/nim/.local/lib/python3.8/site-packages/catkin_tools/common.py", line 23, in <module>
    import trollius as asyncio
  File "/home/nim/.local/lib/python3.8/site-packages/trollius/__init__.py", line 21, in <module>
    from .base_events import *
  File "/home/nim/.local/lib/python3.8/site-packages/trollius/base_events.py", line 42, in <module>
    from . import tasks
  File "/home/nim/.local/lib/python3.8/site-packages/trollius/tasks.py", line 565
    def async(coro_or_future, loop=None):
        ^
SyntaxError: invalid syntax

The workaround is:
pip install -U "git+https://github.com/catkin/catkin_tools.git#egg=catkin_tools"

@facontidavide
Copy link

It would be great to get the new release on all the LTS version of ROS ;)
There are some recent changes that affect me, soooo 😉

@davetcoleman
Copy link
Contributor

This is blocking MoveIt releasing into Noetic :-/

@gavanderhoorn
Copy link
Contributor

While a new release would be nice, you should be able to install catkin_tools using pip3. See my comment here: #603 (comment) and the linked ROS Answers question.

@dirk-thomas
Copy link
Contributor

This is blocking MoveIt releasing into Noetic :-/

Out of curiosity: what feature of catkin_tools does MoveIt (I assume the CI?) need which can't be satisfied by catkin_make_isolated or perhaps colcon?

@davetcoleman
Copy link
Contributor

Thanks @gavanderhoorn, that did the trick (for now).

@dirk-thomas We've used catkin_tools for years, both in the CI and as the recommended user approach to building from source. As you know, it has a lot of nice features compared to catkin_make

I don't know enough about colcon at this point to say why we aren't using that, except it didn't exist until more recently.

Are you suggesting we should, for ROS 1 Noetic, switch fully to colcon? Can you look at our source install instructions and provide example replacement commands please?

We should also probably move this to a standalone moveit issue, if this is what you are proposing, rather than spam this issue.

@dirk-thomas
Copy link
Contributor

I was just asking because catkin_tools isn't actively being maintained and I was just curious why you use it and none of the other options. I don't mind what tool MoveIt uses in CI or in the instructions.

Just from a brief look at the instructions I didn't see any special features being used which aren't available in any of the other tools (building, ignoring specific packages, passing CMake build type.

I only noticed that MoveIt use wstool for cloning the repositories. As of Noetic the official instructions are using vcstool instead - for the same reason that wstool is not actively being maintained.

@gavanderhoorn
Copy link
Contributor

gavanderhoorn commented Jun 2, 2020

I was just asking because catkin_tools isn't actively being maintained

it may not be actively maintained, but Colcon is also not recommended for ROS 1 use. At least it wasn't last time I checked. At the very least it doesn't support a devel space, something many users use in their daily work.

This leaves us with plain catkin_make or catkin_make_isolated. Both don't come near the user experience of catkin_tools, even if it is unmaintained.

@facontidavide
Copy link

My two cents: a LOT of people use catkin_tools. One thing is: "we don't want to support any further catkin_tools to focus on something else (Colcon?)", but 0.45 was release 13 months ago and few commits have been done in the midtime.

Can we just have a release to make those changes available to everybody?

@dirk-thomas
Copy link
Contributor

a LOT of people use catkin_tools

That is certainly the case. It doesn't change the fact though that nobody seems to be willing to actively maintain it. Afaik @mikepurvis is currently the only person putting any effort in it.

Can we just have a release to make those changes available to everybody?

Maybe @mikepurvis can clarify what the pending items are which need to happen before the next release. Then anyone interested in a sooner release can consider to help on those.

@davetcoleman
Copy link
Contributor

davetcoleman commented Jun 2, 2020

@dirk-thomas at a high level: we've switched to vcstool and colcon for MoveIt 2 (@henningkayser), but I'd very much rather stick to the classic ROS 1 tools for this last ros 1 release rather than putting a lot of work into migrating into a new tool set.

Our MoveIt 1 master branch is also suppose to be backwards compatible with Melodic, as we move to Noetic, another reason to stay with these tools.

@wjwwood
Copy link
Member

wjwwood commented Jun 2, 2020

I'm going to try to make a release today or failing that this week at @davetcoleman's request, but I simply don't have the bandwidth to keep this going. So we need someone new to step up if you all are going to continue to use it.

@davetcoleman
Copy link
Contributor

Thanks @wjwwood !!

@wjwwood
Copy link
Member

wjwwood commented Jun 2, 2020

So we need someone new to step up if you all are going to continue to use it.

And by this I mean we need people to help @mikepurvis and others, who are still active. I was thinking more the future.

@mikepurvis should I release as-is? I'm going to check into the reported issue with Python 3.8 first, but is there anything else that should block a release?

@wjwwood
Copy link
Member

wjwwood commented Jun 3, 2020

I misunderstood the report about 3.8, I see now that it was already fixed on master. Ok, I'll assume master is good to go (I tried it out on 3.8) and release it.

@wjwwood
Copy link
Member

wjwwood commented Jun 3, 2020

catkin_tools 0.5.0 is released, but there's still an issue with the deb version. I'm looking into that now... Also, I see a new commit on master, so I'll probably do a 0.5.1 release as soon as I figure out why trollius is still a deb depend.

@roehling
Copy link
Contributor

roehling commented Jun 3, 2020

@wjwwood: Is this related to the following error I received:

Setting up python-catkin-tools (0.5.0-1) ...
  File "/usr/lib/python2.7/dist-packages/catkin_tools/execution/executor.py", line 73
    yield from lock
             ^
SyntaxError: invalid syntax

dpkg: error processing package python-catkin-tools (--configure):
 installed python-catkin-tools package post-installation script subprocess returned error exit status 101

@aroller
Copy link

aroller commented Jun 3, 2020

@wjwwood: Is this related to the following error I received:

Setting up python-catkin-tools (0.5.0-1) ...
  File "/usr/lib/python2.7/dist-packages/catkin_tools/execution/executor.py", line 73
    yield from lock
             ^
SyntaxError: invalid syntax

dpkg: error processing package python-catkin-tools (--configure):
 installed python-catkin-tools package post-installation script subprocess returned error exit status 101

I just received the same error when installing to a ros:melodic docker container on a mac.

@wjwwood
Copy link
Member

wjwwood commented Jun 3, 2020

I’m aware of the issues and I’m working on a fix. Sorry for the disruption.

@moriarty
Copy link

moriarty commented Jun 3, 2020

Traceback (most recent call last):
  File "/usr/bin/catkin", line 11, in <module>
    load_entry_point('catkin-tools==0.5.0', 'console_scripts', 'catkin')()
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 480, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2693, in load_entry_point
    return ep.load()
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2324, in load
    return self.resolve()
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2330, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python2.7/dist-packages/catkin_tools/commands/catkin.py", line 28, in <module>
    from catkin_tools.common import is_tty
  File "/usr/lib/python2.7/dist-packages/catkin_tools/common.py", line 24, in <module>
    import asyncio
ImportError: No module named asyncio

@wjwwood
Copy link
Member

wjwwood commented Jun 3, 2020

I've released 0.6.0 which rolls back to the state of 0.4.5 to fix everyone. I'll make a 0.7.0 release when I'm sure it won't break everything again. It may take a few minutes to propagate.

@varunagrawal
Copy link

So we need someone new to step up if you all are going to continue to use it.

And by this I mean we need people to help @mikepurvis and others, who are still active. I was thinking more the future.

@mikepurvis should I release as-is? I'm going to check into the reported issue with Python 3.8 first, but is there anything else that should block a release?

I am volunteering to help. I can start by doing some PR reviews and we can take it from there?

@timonegk
Copy link
Member

Hi @varunagrawal, thanks for the offer!
The problem with the release is that currently everything is ready, the binaries are built and tested, but @wjwwood has to release them into the ROS Package Server. I wrote to him and he said he would not have time until the current ROS 2 release slows down. Since the release is this Sunday, I hope that he finds time for catkin_tools very soon.

@varunagrawal
Copy link

Alright, sounds good. If anything changes, please do let me know.

taketwo added a commit to taketwo/vim-ros that referenced this issue May 19, 2021
catkin_tools is not fully functional in Python 3 (see: catkin/catkin_tools#594), thus we use Python 2.
toomuchcookies added a commit to toomuchcookies/moveit_tutorials that referenced this issue Jun 30, 2021
fresh install ran into the problem myself and found the solution burried here: catkin/catkin_tools#594
rhaschke added a commit to rhaschke/moveit_tutorials that referenced this issue Jun 30, 2021
which is required additionally to python3-catkin-tools due to a wrongly
configured Debian dependency.
See catkin/catkin_tools#594.
@MatthijsBurgh
Copy link
Contributor

When will this be released on pip and/or apt?

@wolfv
Copy link

wolfv commented Jul 20, 2021

0.7.1 is on conda-forge :) conda-forge/catkin_tools-feedstock#18

@timonegk
Copy link
Member

Yes, thanks for the reminder. I just released the new version for pip. I've also been trying to get @wjwwood to update the apt package on packages.ros.org, but I have been repeatedly put off for months, so I assume that that's not going to happen anytime soon.
Therefore, I opened a Launchpad PPA where you can get the apt package, if you prefer that to pip. The launchpad is available here: https://launchpad.net/~catkin-tools/+archive/ubuntu/ppa, and it can be added with sudo add-apt-repository ppa:catkin-tools/ppa, and then the new version is available with a sudo apt update && sudo apt install python3-catkin-tools.
The package version is 0.7.1 instead of 0.7.0 for technical reasons (i.e. I messed up the first upload to launchpad).

@wjwwood
Copy link
Member

wjwwood commented Aug 1, 2021

I was trying to release onto ROS's packages.ros.org repository (as apt) and also on PyPi for pip, but I'm getting an error in the setup.py:

% python3 setup.py sdist bdist_wheel
running sdist
running egg_info
creating catkin_tools.egg-info
writing catkin_tools.egg-info/PKG-INFO
writing dependency_links to catkin_tools.egg-info/dependency_links.txt
writing entry points to catkin_tools.egg-info/entry_points.txt
writing requirements to catkin_tools.egg-info/requires.txt
writing top-level names to catkin_tools.egg-info/top_level.txt
writing manifest file 'catkin_tools.egg-info/SOURCES.txt'
reading manifest file 'catkin_tools.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'catkin_tools.egg-info/SOURCES.txt'
running check
creating catkin_tools-0.7.1
creating catkin_tools-0.7.1/catkin_tools
creating catkin_tools-0.7.1/catkin_tools.egg-info
creating catkin_tools-0.7.1/catkin_tools/commands
creating catkin_tools-0.7.1/catkin_tools/execution
creating catkin_tools-0.7.1/catkin_tools/jobs
creating catkin_tools-0.7.1/catkin_tools/jobs/cmake
creating catkin_tools-0.7.1/catkin_tools/jobs/commands
creating catkin_tools-0.7.1/catkin_tools/notifications
creating catkin_tools-0.7.1/catkin_tools/notifications/resources
creating catkin_tools-0.7.1/catkin_tools/notifications/resources/linux
creating catkin_tools-0.7.1/catkin_tools/notifications/resources/osx
creating catkin_tools-0.7.1/catkin_tools/notifications/resources/osx/catkin build.app
creating catkin_tools-0.7.1/catkin_tools/notifications/resources/osx/catkin build.app/Contents
creating catkin_tools-0.7.1/catkin_tools/notifications/resources/osx/catkin build.app/Contents/MacOS
creating catkin_tools-0.7.1/catkin_tools/notifications/resources/osx/catkin build.app/Contents/Resources
creating catkin_tools-0.7.1/catkin_tools/notifications/resources/osx/catkin build.app/Contents/Resources/en.lproj
creating catkin_tools-0.7.1/catkin_tools/spaces
creating catkin_tools-0.7.1/catkin_tools/verbs
creating catkin_tools-0.7.1/catkin_tools/verbs/catkin_build
creating catkin_tools-0.7.1/catkin_tools/verbs/catkin_clean
creating catkin_tools-0.7.1/catkin_tools/verbs/catkin_config
creating catkin_tools-0.7.1/catkin_tools/verbs/catkin_create
creating catkin_tools-0.7.1/catkin_tools/verbs/catkin_env
creating catkin_tools-0.7.1/catkin_tools/verbs/catkin_init
creating catkin_tools-0.7.1/catkin_tools/verbs/catkin_list
creating catkin_tools-0.7.1/catkin_tools/verbs/catkin_locate
creating catkin_tools-0.7.1/catkin_tools/verbs/catkin_profile
creating catkin_tools-0.7.1/completion
copying files to catkin_tools-0.7.1...
copying MANIFEST.in -> catkin_tools-0.7.1
copying README.md -> catkin_tools-0.7.1
copying setup.py -> catkin_tools-0.7.1
copying catkin_tools/__init__.py -> catkin_tools-0.7.1/catkin_tools
copying catkin_tools/argument_parsing.py -> catkin_tools-0.7.1/catkin_tools
copying catkin_tools/common.py -> catkin_tools-0.7.1/catkin_tools
copying catkin_tools/config.py -> catkin_tools-0.7.1/catkin_tools
copying catkin_tools/context.py -> catkin_tools-0.7.1/catkin_tools
copying catkin_tools/metadata.py -> catkin_tools-0.7.1/catkin_tools
copying catkin_tools/resultspace.py -> catkin_tools-0.7.1/catkin_tools
copying catkin_tools/terminal_color.py -> catkin_tools-0.7.1/catkin_tools
copying catkin_tools/utils.py -> catkin_tools-0.7.1/catkin_tools
copying catkin_tools.egg-info/PKG-INFO -> catkin_tools-0.7.1/catkin_tools.egg-info
copying catkin_tools.egg-info/SOURCES.txt -> catkin_tools-0.7.1/catkin_tools.egg-info
copying catkin_tools.egg-info/dependency_links.txt -> catkin_tools-0.7.1/catkin_tools.egg-info
copying catkin_tools.egg-info/entry_points.txt -> catkin_tools-0.7.1/catkin_tools.egg-info
copying catkin_tools.egg-info/requires.txt -> catkin_tools-0.7.1/catkin_tools.egg-info
copying catkin_tools.egg-info/top_level.txt -> catkin_tools-0.7.1/catkin_tools.egg-info
copying catkin_tools/commands/__init__.py -> catkin_tools-0.7.1/catkin_tools/commands
copying catkin_tools/commands/catkin.py -> catkin_tools-0.7.1/catkin_tools/commands
copying catkin_tools/execution/__init__.py -> catkin_tools-0.7.1/catkin_tools/execution
copying catkin_tools/execution/controllers.py -> catkin_tools-0.7.1/catkin_tools/execution
copying catkin_tools/execution/events.py -> catkin_tools-0.7.1/catkin_tools/execution
copying catkin_tools/execution/executor.py -> catkin_tools-0.7.1/catkin_tools/execution
copying catkin_tools/execution/io.py -> catkin_tools-0.7.1/catkin_tools/execution
copying catkin_tools/execution/job_server.py -> catkin_tools-0.7.1/catkin_tools/execution
copying catkin_tools/execution/jobs.py -> catkin_tools-0.7.1/catkin_tools/execution
copying catkin_tools/execution/stages.py -> catkin_tools-0.7.1/catkin_tools/execution
copying catkin_tools/jobs/__init__.py -> catkin_tools-0.7.1/catkin_tools/jobs
copying catkin_tools/jobs/catkin.py -> catkin_tools-0.7.1/catkin_tools/jobs
copying catkin_tools/jobs/cmake.py -> catkin_tools-0.7.1/catkin_tools/jobs
copying catkin_tools/jobs/output.py -> catkin_tools-0.7.1/catkin_tools/jobs
copying catkin_tools/jobs/utils.py -> catkin_tools-0.7.1/catkin_tools/jobs
copying catkin_tools/jobs/cmake/python.cmake -> catkin_tools-0.7.1/catkin_tools/jobs/cmake
copying catkin_tools/jobs/cmake/python_install_dir.cmake -> catkin_tools-0.7.1/catkin_tools/jobs/cmake
copying catkin_tools/jobs/commands/__init__.py -> catkin_tools-0.7.1/catkin_tools/jobs/commands
copying catkin_tools/jobs/commands/cmake.py -> catkin_tools-0.7.1/catkin_tools/jobs/commands
copying catkin_tools/jobs/commands/make.py -> catkin_tools-0.7.1/catkin_tools/jobs/commands
copying catkin_tools/notifications/__init__.py -> catkin_tools-0.7.1/catkin_tools/notifications
copying catkin_tools/notifications/impl.py -> catkin_tools-0.7.1/catkin_tools/notifications
copying catkin_tools/notifications/resources/linux/catkin_icon.png -> catkin_tools-0.7.1/catkin_tools/notifications/resources/linux
copying catkin_tools/notifications/resources/linux/catkin_icon_red.png -> catkin_tools-0.7.1/catkin_tools/notifications/resources/linux
copying catkin_tools/notifications/resources/osx/catkin build.app/Contents/Info.plist -> catkin_tools-0.7.1/catkin_tools/notifications/resources/osx/catkin build.app/Contents
copying catkin_tools/notifications/resources/osx/catkin build.app/Contents/PkgInfo -> catkin_tools-0.7.1/catkin_tools/notifications/resources/osx/catkin build.app/Contents
copying catkin_tools/notifications/resources/osx/catkin build.app/Contents/MacOS/catkin build -> catkin_tools-0.7.1/catkin_tools/notifications/resources/osx/catkin build.app/Contents/MacOS
copying catkin_tools/notifications/resources/osx/catkin build.app/Contents/Resources/AppIcon.icns -> catkin_tools-0.7.1/catkin_tools/notifications/resources/osx/catkin build.app/Contents/Resources
copying catkin_tools/notifications/resources/osx/catkin build.app/Contents/Resources/catkin_blue_logo_only.icns -> catkin_tools-0.7.1/catkin_tools/notifications/resources/osx/catkin build.app/Contents/Resources
copying catkin_tools/notifications/resources/osx/catkin build.app/Contents/Resources/en.lproj/Credits.rtf -> catkin_tools-0.7.1/catkin_tools/notifications/resources/osx/catkin build.app/Contents/Resources/en.lproj
copying catkin_tools/notifications/resources/osx/catkin build.app/Contents/Resources/en.lproj/InfoPlist.strings -> catkin_tools-0.7.1/catkin_tools/notifications/resources/osx/catkin build.app/Contents/Resources/en.lproj
copying catkin_tools/notifications/resources/osx/catkin build.app/Contents/Resources/en.lproj/MainMenu.nib -> catkin_tools-0.7.1/catkin_tools/notifications/resources/osx/catkin build.app/Contents/Resources/en.lproj
copying catkin_tools/spaces/__init__.py -> catkin_tools-0.7.1/catkin_tools/spaces
copying catkin_tools/spaces/build.py -> catkin_tools-0.7.1/catkin_tools/spaces
copying catkin_tools/spaces/devel.py -> catkin_tools-0.7.1/catkin_tools/spaces
copying catkin_tools/spaces/install.py -> catkin_tools-0.7.1/catkin_tools/spaces
copying catkin_tools/spaces/log.py -> catkin_tools-0.7.1/catkin_tools/spaces
copying catkin_tools/spaces/source.py -> catkin_tools-0.7.1/catkin_tools/spaces
copying catkin_tools/verbs/__init__.py -> catkin_tools-0.7.1/catkin_tools/verbs
copying catkin_tools/verbs/catkin_shell_verbs.bash -> catkin_tools-0.7.1/catkin_tools/verbs
copying catkin_tools/verbs/catkin_build/__init__.py -> catkin_tools-0.7.1/catkin_tools/verbs/catkin_build
copying catkin_tools/verbs/catkin_build/build.py -> catkin_tools-0.7.1/catkin_tools/verbs/catkin_build
copying catkin_tools/verbs/catkin_build/cli.py -> catkin_tools-0.7.1/catkin_tools/verbs/catkin_build
copying catkin_tools/verbs/catkin_build/color.py -> catkin_tools-0.7.1/catkin_tools/verbs/catkin_build
copying catkin_tools/verbs/catkin_clean/__init__.py -> catkin_tools-0.7.1/catkin_tools/verbs/catkin_clean
copying catkin_tools/verbs/catkin_clean/clean.py -> catkin_tools-0.7.1/catkin_tools/verbs/catkin_clean
copying catkin_tools/verbs/catkin_clean/cli.py -> catkin_tools-0.7.1/catkin_tools/verbs/catkin_clean
copying catkin_tools/verbs/catkin_clean/color.py -> catkin_tools-0.7.1/catkin_tools/verbs/catkin_clean
copying catkin_tools/verbs/catkin_config/__init__.py -> catkin_tools-0.7.1/catkin_tools/verbs/catkin_config
copying catkin_tools/verbs/catkin_config/cli.py -> catkin_tools-0.7.1/catkin_tools/verbs/catkin_config
copying catkin_tools/verbs/catkin_create/__init__.py -> catkin_tools-0.7.1/catkin_tools/verbs/catkin_create
copying catkin_tools/verbs/catkin_create/cli.py -> catkin_tools-0.7.1/catkin_tools/verbs/catkin_create
copying catkin_tools/verbs/catkin_env/__init__.py -> catkin_tools-0.7.1/catkin_tools/verbs/catkin_env
copying catkin_tools/verbs/catkin_env/cli.py -> catkin_tools-0.7.1/catkin_tools/verbs/catkin_env
copying catkin_tools/verbs/catkin_init/__init__.py -> catkin_tools-0.7.1/catkin_tools/verbs/catkin_init
copying catkin_tools/verbs/catkin_init/cli.py -> catkin_tools-0.7.1/catkin_tools/verbs/catkin_init
copying catkin_tools/verbs/catkin_list/__init__.py -> catkin_tools-0.7.1/catkin_tools/verbs/catkin_list
copying catkin_tools/verbs/catkin_list/cli.py -> catkin_tools-0.7.1/catkin_tools/verbs/catkin_list
copying catkin_tools/verbs/catkin_locate/__init__.py -> catkin_tools-0.7.1/catkin_tools/verbs/catkin_locate
copying catkin_tools/verbs/catkin_locate/cli.py -> catkin_tools-0.7.1/catkin_tools/verbs/catkin_locate
copying catkin_tools/verbs/catkin_profile/__init__.py -> catkin_tools-0.7.1/catkin_tools/verbs/catkin_profile
copying catkin_tools/verbs/catkin_profile/cli.py -> catkin_tools-0.7.1/catkin_tools/verbs/catkin_profile
copying completion/_catkin -> catkin_tools-0.7.1/completion
copying completion/catkin.bash -> catkin_tools-0.7.1/completion
Writing catkin_tools-0.7.1/setup.cfg
creating dist
Creating tar archive
removing 'catkin_tools-0.7.1' (and everything under it)
running bdist_wheel
running build
running build_py
creating build
creating build/lib
creating build/lib/catkin_tools
copying catkin_tools/argument_parsing.py -> build/lib/catkin_tools
copying catkin_tools/__init__.py -> build/lib/catkin_tools
copying catkin_tools/resultspace.py -> build/lib/catkin_tools
copying catkin_tools/utils.py -> build/lib/catkin_tools
copying catkin_tools/context.py -> build/lib/catkin_tools
copying catkin_tools/metadata.py -> build/lib/catkin_tools
copying catkin_tools/config.py -> build/lib/catkin_tools
copying catkin_tools/common.py -> build/lib/catkin_tools
copying catkin_tools/terminal_color.py -> build/lib/catkin_tools
creating build/lib/catkin_tools/spaces
copying catkin_tools/spaces/build.py -> build/lib/catkin_tools/spaces
copying catkin_tools/spaces/__init__.py -> build/lib/catkin_tools/spaces
copying catkin_tools/spaces/install.py -> build/lib/catkin_tools/spaces
copying catkin_tools/spaces/devel.py -> build/lib/catkin_tools/spaces
copying catkin_tools/spaces/source.py -> build/lib/catkin_tools/spaces
copying catkin_tools/spaces/log.py -> build/lib/catkin_tools/spaces
creating build/lib/catkin_tools/jobs
copying catkin_tools/jobs/__init__.py -> build/lib/catkin_tools/jobs
copying catkin_tools/jobs/cmake.py -> build/lib/catkin_tools/jobs
copying catkin_tools/jobs/utils.py -> build/lib/catkin_tools/jobs
copying catkin_tools/jobs/catkin.py -> build/lib/catkin_tools/jobs
copying catkin_tools/jobs/output.py -> build/lib/catkin_tools/jobs
creating build/lib/catkin_tools/notifications
copying catkin_tools/notifications/__init__.py -> build/lib/catkin_tools/notifications
copying catkin_tools/notifications/impl.py -> build/lib/catkin_tools/notifications
creating build/lib/catkin_tools/verbs
copying catkin_tools/verbs/__init__.py -> build/lib/catkin_tools/verbs
creating build/lib/catkin_tools/commands
copying catkin_tools/commands/__init__.py -> build/lib/catkin_tools/commands
copying catkin_tools/commands/catkin.py -> build/lib/catkin_tools/commands
creating build/lib/catkin_tools/execution
copying catkin_tools/execution/job_server.py -> build/lib/catkin_tools/execution
copying catkin_tools/execution/__init__.py -> build/lib/catkin_tools/execution
copying catkin_tools/execution/jobs.py -> build/lib/catkin_tools/execution
copying catkin_tools/execution/stages.py -> build/lib/catkin_tools/execution
copying catkin_tools/execution/controllers.py -> build/lib/catkin_tools/execution
copying catkin_tools/execution/io.py -> build/lib/catkin_tools/execution
copying catkin_tools/execution/events.py -> build/lib/catkin_tools/execution
copying catkin_tools/execution/executor.py -> build/lib/catkin_tools/execution
creating build/lib/catkin_tools/jobs/commands
copying catkin_tools/jobs/commands/__init__.py -> build/lib/catkin_tools/jobs/commands
copying catkin_tools/jobs/commands/cmake.py -> build/lib/catkin_tools/jobs/commands
copying catkin_tools/jobs/commands/make.py -> build/lib/catkin_tools/jobs/commands
creating build/lib/catkin_tools/verbs/catkin_locate
copying catkin_tools/verbs/catkin_locate/__init__.py -> build/lib/catkin_tools/verbs/catkin_locate
copying catkin_tools/verbs/catkin_locate/cli.py -> build/lib/catkin_tools/verbs/catkin_locate
creating build/lib/catkin_tools/verbs/catkin_config
copying catkin_tools/verbs/catkin_config/__init__.py -> build/lib/catkin_tools/verbs/catkin_config
copying catkin_tools/verbs/catkin_config/cli.py -> build/lib/catkin_tools/verbs/catkin_config
creating build/lib/catkin_tools/verbs/catkin_profile
copying catkin_tools/verbs/catkin_profile/__init__.py -> build/lib/catkin_tools/verbs/catkin_profile
copying catkin_tools/verbs/catkin_profile/cli.py -> build/lib/catkin_tools/verbs/catkin_profile
creating build/lib/catkin_tools/verbs/catkin_env
copying catkin_tools/verbs/catkin_env/__init__.py -> build/lib/catkin_tools/verbs/catkin_env
copying catkin_tools/verbs/catkin_env/cli.py -> build/lib/catkin_tools/verbs/catkin_env
creating build/lib/catkin_tools/verbs/catkin_init
copying catkin_tools/verbs/catkin_init/__init__.py -> build/lib/catkin_tools/verbs/catkin_init
copying catkin_tools/verbs/catkin_init/cli.py -> build/lib/catkin_tools/verbs/catkin_init
creating build/lib/catkin_tools/verbs/catkin_list
copying catkin_tools/verbs/catkin_list/__init__.py -> build/lib/catkin_tools/verbs/catkin_list
copying catkin_tools/verbs/catkin_list/cli.py -> build/lib/catkin_tools/verbs/catkin_list
creating build/lib/catkin_tools/verbs/catkin_clean
copying catkin_tools/verbs/catkin_clean/__init__.py -> build/lib/catkin_tools/verbs/catkin_clean
copying catkin_tools/verbs/catkin_clean/color.py -> build/lib/catkin_tools/verbs/catkin_clean
copying catkin_tools/verbs/catkin_clean/clean.py -> build/lib/catkin_tools/verbs/catkin_clean
copying catkin_tools/verbs/catkin_clean/cli.py -> build/lib/catkin_tools/verbs/catkin_clean
creating build/lib/catkin_tools/verbs/catkin_create
copying catkin_tools/verbs/catkin_create/__init__.py -> build/lib/catkin_tools/verbs/catkin_create
copying catkin_tools/verbs/catkin_create/cli.py -> build/lib/catkin_tools/verbs/catkin_create
creating build/lib/catkin_tools/verbs/catkin_build
copying catkin_tools/verbs/catkin_build/build.py -> build/lib/catkin_tools/verbs/catkin_build
copying catkin_tools/verbs/catkin_build/__init__.py -> build/lib/catkin_tools/verbs/catkin_build
copying catkin_tools/verbs/catkin_build/color.py -> build/lib/catkin_tools/verbs/catkin_build
copying catkin_tools/verbs/catkin_build/cli.py -> build/lib/catkin_tools/verbs/catkin_build
creating build/lib/catkin_tools/jobs/cmake
copying catkin_tools/jobs/cmake/python.cmake -> build/lib/catkin_tools/jobs/cmake
Traceback (most recent call last):
  File "setup.py", line 144, in <module>
    cmdclass={'install': PermissiveInstall},
  File "/usr/local/lib/python3.6/dist-packages/setuptools/__init__.py", line 145, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.6/dist-packages/wheel/bdist_wheel.py", line 192, in run
    self.run_command('build')
  File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/lib/python3.6/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.6/dist-packages/setuptools/command/build_py.py", line 53, in run
    self.build_package_data()
  File "/usr/local/lib/python3.6/dist-packages/setuptools/command/build_py.py", line 126, in build_package_data
    srcfile in self.distribution.convert_2to3_doctests):
TypeError: argument of type 'NoneType' is not iterable

@timonegk did you run into this at all? I'm using this setup:

% lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 18.04.5 LTS
Release:	18.04
Codename:	bionic

% python3 --version
Python 3.6.9

% python3 -c 'import setuptools; print(setuptools.__version__)'
40.8.0

@tykurtz
Copy link

tykurtz commented Aug 1, 2021

Jumping to (maybe) help move this along.

Would the following docker image help?

FROM ubuntu:18.04

RUN apt-get update && apt-get install -y \
    build-essential                      \
    wget                                 \
    python3-dev                          \
    python3-setuptools                   \
    python3-wheel                        \
 && rm -rf /var/lib/apt/lists/*

RUN wget https://github.com/catkin/catkin_tools/archive/refs/tags/0.7.1.tar.gz && \
    tar xf 0.7.1.tar.gz && \
    rm -rf 0.7.1.tar.gz

RUN cd catkin_tools-0.7.1 && \
    /usr/bin/python3.6 setup.py sdist bdist_wheel

I couldn't reproduce your issue, but you should be able to build the artifact.

@jspricke
Copy link

jspricke commented Aug 1, 2021

I was trying to release onto ROS's packages.ros.org repository (as apt) and also on PyPi for pip, but I'm getting an error in the setup.py:

% python3 setup.py sdist bdist_wheel

[...]

copying catkin_tools/jobs/cmake/python.cmake -> build/lib/catkin_tools/jobs/cmake
Traceback (most recent call last):
File "setup.py", line 144, in
cmdclass={'install': PermissiveInstall},
File "/usr/local/lib/python3.6/dist-packages/setuptools/init.py", line 145, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.6/dist-packages/wheel/bdist_wheel.py", line 192, in run
self.run_command('build')
File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/lib/python3.6/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.6/dist-packages/setuptools/command/build_py.py", line 53, in run
self.build_package_data()
File "/usr/local/lib/python3.6/dist-packages/setuptools/command/build_py.py", line 126, in build_package_data
srcfile in self.distribution.convert_2to3_doctests):
TypeError: argument of type 'NoneType' is not iterable

@wjwwood note the /usr/local/python3.6 in the traceback, you have a modified Python installed breaking the command.
Using the Ubuntu Python it works fine, as Tyler reported.

@wjwwood
Copy link
Member

wjwwood commented Aug 1, 2021

Actually I don't have a custom python installation. But I do have an updated setuptools and usr local lib is the right place for non-apt dependencies. It might be an issue that it doesn't work with newer versions of setuptools which is something I don't want to ignore. It's recommended to use the latest version when releasing to pypi. Thanks for the suggestions though, I'll look into it more tomorrow.

@timonegk
Copy link
Member

timonegk commented Aug 1, 2021

I did not run into this problem. (And I did already successfully release the package on PyPi.)

@wjwwood
Copy link
Member

wjwwood commented Aug 2, 2021

The package version is 0.7.1 instead of 0.7.0 for technical reasons (i.e. I messed up the first upload to launchpad).

@timonegk you probably already realize this, but next time you release, do at least 0.7.3 otherwise your workaround will prevent you from putting the "real" 0.7.2 on the PPA. I thought there was a -N you could append to the version to solve issues like this (packaging issues), because using a future version number causes conflicts with the source repository.

It might be an issue that it doesn't work with newer versions of setuptools which is something I don't want to ignore.

In case anyone else ran into this problem...

It turned out to be a bug in the version of setuptools I was using. I narrowed it down to a few changes, but in the end just updating setuptools fixed it.

I was surprised at this issue (and I still think it has to do with something "special" we're doing in our setup.py) because I had just released osrf_pycommon version 1.0.0 which also is the first Python3-only release of that, and I didn't have this problem.


0.7.1 should now be available via packages.ros.org too. I really am sorry it took so long, but finally I got the Python3 only version sufficiently tested and released for osrf_pycommon which was a prerequisite for this last week. And I spent some time this weekend convincing myself that I wasn't going to disrupt existing uses of out catkin tools apt packages, like I accidentally did last time.

From now on, updating the version should be quick (now that the Python3 only is not new), so just ping me on an issue if you want a new one @timonegk. I actually had quite a few small issues to resolve when releasing the debs for osrf_pycommon version 1.0.0, so it wasn't very straightforward the first time, but I don't expect problems in the future.

I will also still follow up with someone about maybe just pulling from your PPA, rather than us making and uploading debs. That becomes even easier for me, since all we will do is run a build farm job on demand after you update the PPA to pull in new stuff. But I can do manual releases until then.


Despite the testing, if issues come up, please let me know and I'll fix it as quick as I can (now that it's live).

@wjwwood
Copy link
Member

wjwwood commented Aug 2, 2021

The package version is 0.7.1 instead of 0.7.0 for technical reasons (i.e. I messed up the first upload to launchpad).

Actually I misunderstood you, I see you've already fixed that by doing a 0.7.1 release locally. Sorry, my bad :)

Still I think, in the future, you may be able to fix that by putting something like 0.7.0-2 on the PPA.

@jspricke
Copy link

jspricke commented Aug 2, 2021

Still I think, in the future, you may be able to fix that by putting something like 0.7.0-2 on the PPA.

@wjwwood A -2 can only change files in the debian/ directory. If I got it correct, the actual fix was in the stdeb.cfg which is part of the upstream project: 7a8c239, so a 0.7.1 was the right thing to do.

@wjwwood
Copy link
Member

wjwwood commented Aug 2, 2021

Hmm, the stdeb.cfg is only used as part of the release and isn't installed. so I would have thought that only affects stuff in the debian folder, but maybe not.

@jspricke
Copy link

jspricke commented Aug 2, 2021

@wjwwood 'installed' does not matter, point is what is in the source package and where is it versioned. I just had a look at the PPA and the stdeb.cfg is indeed not part of the source package but as the debian/ directory is generated from it, it was still the right way to make the package reproducible.

@timonegk
Copy link
Member

timonegk commented Aug 2, 2021

I uploaded 0.7.0 with the wrong stdeb.cfg, and after the fix I was not able to upload it with the same version, so I had to increase it to 0.7.1.

@wjwwood
Copy link
Member

wjwwood commented Aug 4, 2021

Sorry, I didn't mean to imply that they way @timonegk did it was wrong, that's certainly not the case. I agree having the tagged version be consistent with what is released is a good thing. And version numbers are essentially free.

I was just pointing out that if you need to do so (just information that might be useful) you can adjust the "debian version", stdeb has this option: https://github.com/astraw/stdeb/blob/3e6522cefa240be365ebdc0dc0c7923ba7118999/stdeb/util.py#L84 and therefore 0.7.0-1 could become 0.7.0-2 if the mistake was purely packaging related. That would let you upload to the ppa without a new release. Like if you accidentally used an old version of stdeb the first time and the generated debian/ folder was wrong but you didn't notice until you uploaded it, or some other packaging only issue came up.

It's kind of peculiar (compared to other packaging processes) that we include this packaging information in the source repository. But I agree that keeping the stdeb.cfg correct in the source repository and tagged with the release, since we do include it.

@mikepurvis
Copy link
Member Author

Having the packaging info in-source I think kind of implies that it should be a 3.0 (native) rather than 3.0 (quilt) format package, eg a version number with no debian increment at all. But I expect that that would be even weirder to tools like stdeb than just including the increment and rarely/never using it.

mintar added a commit to mintar/moveit that referenced this issue Aug 18, 2021
This was a workaround for
catkin/catkin_tools#594. It is no longer
necessary since python3-catkin-tools 0.7 added the proper dependency.
rhaschke pushed a commit to moveit/moveit that referenced this issue Aug 18, 2021
This was a workaround for catkin/catkin_tools#594,
which is no longer needed because python3-catkin-tools 0.7 added the required dependency.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.