env.sh's new shell when no arguments are given lacks same env changes if .bashrc changes env vars

[dirk-thomas]

@wjwwood as discussed env.sh could do a better job of explaining that running or sourcing it without further arguments is not a valid usecase, as the resulting env is not what can be expected.

[dirk-thomas]

Why would the resulting environment not be correct?

[dirk-thomas]

Invoking env.sh without arguments has a very valid purpose:
it create a subshell with the environment which the user can leave again by calling "exit".
It actually states that explicitly when you invoke it that way...

[wjwwood]

I think he might be talking about CMI's?

https://github.com/ros/catkin/blob/groovy-devel/python/catkin/builder.py#L322

Does this one do anything without arguments?

[tkruse]

Sorry, I should have explained more. It was late, I was in a hurry, and i thought William knew what I mean. This relates to catkin_make_isolated, and the effect I noticed in #367

To reproduce, in a groovy underlay catkin workspace (after running catkin_make_isolated):

devel_isolated/genmsg/env.sh env | sort > /tmp/env.orig
devel_isolated/genmsg/env.sh 
env | sort > /tmp/env.other
exit
diff -iB /tmp/env.orig /tmp/env.other 

This gives me differences for CMAKE_PREFIX_PATH, CPATH, LD_LIBRARY_PATH, PATH, PKG_CONFIG_PATH, PYTHONPATH

[wjwwood]

I knew, I hadn't talked to dirk about it. (I did not know that other env.sh's worked like that)

[wjwwood]

I can tell you that env.sh generated by CMI (at least for third party packages) does not work unless the command is passed to it. I don't know if this is a failing of CMI or actually a bug in the env.sh generation, or something implicit to your environment.

[wjwwood]

I also updated the ticket title, the use of 'useless' was pretty unhelpful.

[dirk-thomas]

I would suggest that CMI generates the same files as catkin (env.sh, setup.sh|bash|zsh) and also uses the templates available in catkin instead of generating them separately. Then the semantic and functionality is the same.

[wjwwood]

@dirk-thomas I agree, but at the time of implementation we considered this, I think there was some problem with reusing the catkin templates, but I can't remember what that was now. (possibly the templates are generated with CMake not python?)

[dirk-thomas]

Yes, the templates are .in files but expanding them in Python should be not difficult. All except setup.sh are very simple and should reuse the existing files. setup.sh will either only redirect to an already generated setup.sh in one of the package folders or be generated for non-catkin as the env is currently.

[wjwwood]

I have opened a new ticket:

#371

That ticket addresses the problem on inconsistency of env.sh files generated for non-catkin packages.

@tkruse's original example indicates that genmsg's env.sh does not behave correctly, I'd have to test that locally to confirm, but that might still be a bug or something.

[tkruse]

Regarding the new issue title, I believe that env.sh does generate a new
shell (else exit would close the terminal I believe), but that shell just
does not have the expected env.

On Thu, Feb 28, 2013 at 9:36 PM, William Woodall
notifications@github.comwrote:

I have opened a new ticket:

#371 #371

That ticket addresses the problem on inconsistency of env.sh files
generated for non-catkin packages.

@tkruse https://github.com/tkruse's original example indicates that
genmsg's env.sh does not behave correctly, I'd have to test that locally to
confirm, but that might still be a bug or something.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/370#issuecomment-14256055
.

[wjwwood]

I see, you can change it to something more appropriate then.

[dirk-thomas]

I can neither reproduce it with the currently released catkin version nor with latest groovy-devel. The only diff in the environment is OLDPWD, SHLVL and _. Marking it as closed for now.

Please post a reproducible command sequence and post the actual and expected output (than the issue will get reopened).

[tkruse]

Given that you did not get the same results as me, I tried guessing the difference, and I found a likely one. My .bashrc changes those env variables that I mentioned.

So a prepending entry like this in the .bashrc (or any othe file processed for new shells I guess):

export PATH=/usr/lib/jvm/default-java/bin:${PATH}

Will cause the PATH to be different in the diff:

-PATH=/tmp/catkin_ws/devel_isolated/genmsg/bin:
+PATH=/usr/lib/jvm/default-java/bin:/tmp/catkin_ws/devel_isolated/genmsg/bin:

and a replacing entry like this:

export CPATH=/home/kruset/local/ubuntu-amd64/include

will cause this difference:

-CPATH=/tmp/catkin_ws/devel_isolated/genmsg/include:/tmp/catkin_ws/devel_isolated/catkin/include
+CPATH=/home/kruset/local/ubuntu-amd64/include

The complete steps to reproduce are to make the above changes to the bashrc, then to perform these steps:

mkdir catkin_ws
cd catkin_ws/
mkdir src
cd src
git clone git://github.com/ros/catkin.git
git clone https://github.com/ros/genmsg
cd ..
src/catkin/bin/catkin_make_isolated 
devel_isolated/genmsg/env.sh env | sort > /tmp/env.orig
devel_isolated/genmsg/env.sh 
env | sort > /tmp/env.other
exit
diff -u /tmp/env.orig /tmp/env.other 

[dirk-thomas]

When you invoke env.sh without arguments a new shell will be spawn which will source your .bashrc. If you invoke it with arguments no new shell will be spawn and therefore assignments from your .bashrc will not be effective instead the current environment is just carried over. Is that the scenario you describe?

[tkruse]

Yes, I believe that is what happens. What should happen is either when
invoked without arguments, first the shell should be created (including
bashrc and such) and then the env should be changed, or users must be
warned that any setting in their bashrc can disturb the env of a new shell.

On Sat, Mar 2, 2013 at 8:35 PM, Dirk Thomas notifications@github.comwrote:

When you invoke env.sh without arguments a new shell will be spawn which
will source your .bashrc. If you invoke it with arguments no new shell will
be spawn and therefore assignments from your .bashrc will not be effective
instead the current environment is just carried over. Is that the scenario
you describe?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/370#issuecomment-14334114
.

[dirk-thomas]

Just to clarify: that behavior is not catkin_make(_isolated) specific. This has been they way it works since some ROS releases.

I don't know a way to implement what you suggest - spawning a new shell and performing a command like sourcing the setup in the interactive shell after .bashrc etc has been evaluated. Please suggest a way to perform that (at best a generic solution which works for the different supported shells) or suggest a formulation which you think is necessary to describe the existing behavior.

[tkruse]

I am only aware of an env.sh that was introduced in fuerte, which to me does not mean "many" releases ago. And yes, the one in /opt/ros/fuerte suffers the same problem and should also be fixed.

I don't know of any way to implement this neither, in which case I would prefer env.sh not to offer this functionality at all.
I guess most releases of ROS came without any such function precisely because this is not possible, in general, i.e. a bad idea to start with,

For bash though, I found this though:
http://stackoverflow.com/questions/7192575/run-bash-command-in-new-shell-and-stay-in-new-shell-after-this-command-executes
Maybe someone will see this and have a greate idea for Bourne shells.

In any case, if it is decided to keep this flawed behavior for the convenience of a few hackers, I would phrase a warning like:

Entering environment at '/home/kruset/groovy_underlay/devel_isolated/genmsg', type 'exit' to leave
Warning: if your shell startup files (e.g. .bashrc) modify variables, then this environment may not be valid.