Docker: Add support for amazonlinux (i.e. AWS Lambda)

[christianklotz]

Installing ortools using easy_install works great on an EC2 instance but in combination with AWS Lambda which requires the application package to be put together using virtualenv, copying all dependencies into the project root. So far this has left me unsuccessful getting ortools to run on AWS Lambda.

All other packages I'm using are installed using pip and work as expected. Doing so for ortools however fails with the following message, I suspect because the native libraries are not provided for that architecture?

$ pip install ortools
Collecting ortools
  Could not find a version that satisfies the requirement ortools (from versions: )
No matching distribution found for ortools

Do you know a way to install ortools using pip, from source or otherwise to get it to work on AWS Lambda?

[dlahlou]

Hello Christian,

I guess you're trying to install or-tools using pip on linux. Unfortunately
this is not possible because we couldn't build or-tools wheel for linux
since it is not pure Python, and PyPI "currently only allows uploads of
platform wheels for Windows and OS X, NOT linux". You can check this link
https://packaging.python.org/distributing/ for more details.

I hope this helps.

Regards.

On Fri, Oct 14, 2016 at 8:53 PM, Christian Klotz notifications@github.com
wrote:

Installing ortools using easy_install works great on an EC2 instance but
in combination with AWS Lambda which requires the application package to be
put together using virtualenv, copying all dependencies into the project
root. So far this has left me unsuccessful getting ortools to run on AWS
Lambda.

All other packages I'm using are installed using pip and work as
expected. Doing so for ortools however fails with the following message,
I suspect because the native libraries are not provided for that
architecture?

$ pip install ortools
Collecting ortools
Could not find a version that satisfies the requirement ortools (from versions: )
No matching distribution found for ortools

Do you know a way to install ortools using pip, from source or otherwise
to get it to work on AWS Lambda?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#259, or mute the thread
https://github.com/notifications/unsubscribe-auth/ASwpfJrQOTrd4lh73vFepAgfCWV0NDNMks5qz8-1gaJpZM4KXVIi
.

driss

[christianklotz]

Thank you for your response. I played around with it on Linux and got or-tools to run smoothly on an EC2 instance using the normal installation process using python setup.py.

The tricky bit seems to be the packaging for Lambda as it requires all project dependencies on project root which works fine for most parts but doesn't seem to play well either with non-pip packages or is related to the native libraries.

I've also put up a SO post in case you have any ideas :)

[ryan-lane]

Hi there. Pypi allows uploading of linux wheels, and another google project is doing it, if you want to follow them as an example:

https://github.com/google/protobuf

https://pypi.python.org/pypi/protobuf/3.2.0

[jcruelty]

Hi, we see that a linux wheel exists now which is great. Could you please publish this to pypi?

[berendberendsen]

@dlahlou would it be possible to publish the wheel to pypi? Running or-tools as a lambda function would be great

[lperron]

Actually, pure python linux wheel are OK. the manylinux are really hard to do. Laurent Perron | Operations Research | lperron@google.com | (33) 1 42 68 53 00 2017-05-16 23:16 GMT+02:00 Amar Pai <notifications@github.com>:

…

Hi, we see that a linux wheel exists now which is great. Could you please publish this to pypi? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#259 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKj17UX9lxBDWSmLD6xrEd3f7QeQOLvOks5r6hIVgaJpZM4KXVIi> .

[tunamonster]

I managed to get ortools working on AWS Lambda. In addition to the usual procedure for including libraries, I had to do some chmodding, so Lambda could read the files. Repo with files and instructions: https://github.com/tunamonster/aws_lambda_ortools

[berendberendsen]

Great stuff @tunamonster - works like a charm. Thanks for the clear instructions as well

[lperron]

Yes, really nice. Thanks Laurent Perron | Operations Research | lperron@google.com | (33) 1 42 68 53 00 2017-07-04 3:44 GMT-07:00 berendberendsen <notifications@github.com>:

…

Great stuff @tunamonster <https://github.com/tunamonster> - works like a charm — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#259 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKj17RPc9trsMtGsYRMSj-hEtnArY9JDks5sKheYgaJpZM4KXVIi> .

[christianklotz]

Thanks @tunamonster, I've taken your instructions and turned them into a build and packaging script using Docker. This way you can skip setting up an EC2 instance and just build everything right for Lambda within the Docker container.

See https://github.com/christianklotz/or-tools-lambda for more details.

I hope this makes it a little easier for anyone who just wants to run their code on Lambda without worrying too much about provisioning machines 🚀.

Please let me know if you got any feedback. There might be room for improvement still since it feels a bit weird to copy egg contents manually instead of leaving it up to the installers to use the correct target directories. Also, once or-tool's pypi wheel uploads include support for different Linux distributions, this workaround should no longer be necessary.

[Mizux]

Hi,

Please note we now provide wheel package for manylinux distro on pypi...

[lperron]

Unless told otherwise, I believe the manylinux packages will solve this problem.
Thus I am closing this issue.