Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[librdkafka] Add new port #5921

Merged
merged 17 commits into from
Jul 2, 2019
Merged

[librdkafka] Add new port #5921

merged 17 commits into from
Jul 2, 2019

Conversation

myd7349
Copy link
Contributor

@myd7349 myd7349 commented Apr 3, 2019

@myd7349 myd7349 marked this pull request as ready for review April 3, 2019 10:18
@Rastaban
Copy link
Contributor

Rastaban commented Apr 4, 2019

I see failures on Linux and UWP. Are those expected?

@Rastaban Rastaban mentioned this pull request Apr 4, 2019
Closed
@myd7349 myd7349 changed the title [rdkafka] Add new port [WIP][rdkafka] Add new port Apr 4, 2019
@myd7349
Copy link
Contributor Author

myd7349 commented Apr 4, 2019

@Rastaban I haven't tested it yet. :-)
Maybe I will test it after the holiday.

@myd7349 myd7349 changed the title [WIP][rdkafka] Add new port [WIP][librdkafka] Add new port Apr 7, 2019
@myd7349
Copy link
Contributor Author

myd7349 commented Apr 7, 2019

Known issues:

  1. x86-windows

    1> Creating library E:/vcpkg/buildtrees/rdkafka/src/v1.0.0-5c63a98714/examples/build/Debug/rdkafka_example_cpp.lib and object E:/vcpkg/buildtrees/rdkafka/src/v1.0.0-5c63a98714/examples/build/Debug/rdkafka_example_cpp.exp
    1>rdkafka_example.obj : error LNK2001: unresolved external symbol "public: static int const RdKafka::Topic::PARTITION_UA" (?PARTITION_UA@Topic@RdKafka@@2HB)
    1>rdkafka_example.obj : error LNK2001: unresolved external symbol "public: static __int64 const RdKafka::Topic::OFFSET_BEGINNING" (?OFFSET_BEGINNING@Topic@RdKafka@@2_JB)
    1>rdkafka_example.obj : error LNK2001: unresolved external symbol "public: static __int64 const RdKafka::Topic::OFFSET_END" (?OFFSET_END@Topic@RdKafka@@2_JB)
    1>rdkafka_example.obj : error LNK2001: unresolved external symbol "public: static __int64 const RdKafka::Topic::OFFSET_STORED" (?OFFSET_STORED@Topic@RdKafka@@2_JB)
    1>E:\vcpkg\buildtrees\rdkafka\src\v1.0.0-5c63a98714\examples\build\Debug\rdkafka_example_cpp.exe : fatal error LNK1120: 4 unresolved externals

@NancyLi1013
Copy link
Contributor

Update the latest test results from the current CI system:

x64-uwp master test notes
librdkafka Fail New Port
arm-uwp master test notes
librdkafka Fail New Port

Please help confirm if they are expected. Thanks.

@myd7349 myd7349 mentioned this pull request Apr 8, 2019
Merged
@myd7349
Copy link
Contributor Author

myd7349 commented Apr 8, 2019 •
edited
Loading

Known issues:

1. x86-windows
   > 1>   Creating library E:/vcpkg/buildtrees/rdkafka/src/v1.0.0-5c63a98714/examples/build/Debug/rdkafka_example_cpp.lib and object E:/vcpkg/buildtrees/rdkafka/src/v1.0.0-5c63a98714/examples/build/Debug/rdkafka_example_cpp.exp
   > 1>rdkafka_example.obj : error LNK2001: unresolved external symbol "public: static int const RdKafka::Topic::PARTITION_UA" (?PARTITION_UA@Topic@RdKafka@@2HB)
   > 1>rdkafka_example.obj : error LNK2001: unresolved external symbol "public: static __int64 const RdKafka::Topic::OFFSET_BEGINNING" (?OFFSET_BEGINNING@Topic@RdKafka@@2_JB)
   > 1>rdkafka_example.obj : error LNK2001: unresolved external symbol "public: static __int64 const RdKafka::Topic::OFFSET_END" (?OFFSET_END@Topic@RdKafka@@2_JB)
   > 1>rdkafka_example.obj : error LNK2001: unresolved external symbol "public: static __int64 const RdKafka::Topic::OFFSET_STORED" (?OFFSET_STORED@Topic@RdKafka@@2_JB)
   > 1>E:\vcpkg\buildtrees\rdkafka\src\v1.0.0-5c63a98714\examples\build\Debug\rdkafka_example_cpp.exe : fatal error LNK1120: 4 unresolved externals

This seems a bug in the upstream repo: confluentinc/librdkafka#2274

@grdowns grdowns self-assigned this Apr 12, 2019
@grdowns grdowns added the wip label Apr 12, 2019
@webfolderio
Copy link
Contributor

This seems a bug in the upstream repo

It's fixed

@myd7349
Copy link
Contributor Author

myd7349 commented Apr 17, 2019 •
edited
Loading

I will give those features a test as soon as possible.

@myd7349
Copy link
Contributor Author

myd7349 commented May 1, 2019

confluentinc/librdkafka#2307

@myd7349 myd7349 changed the title [WIP][librdkafka] Add new port [librdkafka] Add new port May 7, 2019
@myd7349
Copy link
Contributor Author

myd7349 commented May 7, 2019 •
edited
Loading

OK. I have tested all features with this little project https://github.com/myd7349/Ongoing-Study/tree/master/cpp/CMake/vcpkg/librdkafka_pr_2274. These triplets are tested:

  • x86-windows
  • x86-windows-static
  • x64-windows
  • x64-windows-static
  • x64-linux

I have removed the WIP tag.

@myd7349
Copy link
Contributor Author

myd7349 commented May 7, 2019

I have gave UWP support a try, but failed: https://github.com/edenhill/librdkafka/files/3152143/install-x86-uwp-dbg-out.log
Maybe we can add UWP support in the future.

@webfolderio
Copy link
Contributor

Maybe we can add UWP support in the future.

Yes, go without UWP.

@NancyLi1013
Copy link
Contributor

@myd7349, thanks for your all efforts on this port. The port only failed on arm64-windows now except for x64-uwp and arm-uwp. Have you tested on arm64-windows?
The error log on arm64-window is as follows:
arm64-windows.zip

@myd7349
Copy link
Contributor Author

myd7349 commented May 8, 2019 •
edited
Loading

Hi @NancyLi1013 I don't have ARM toolchain installed on my PC, so I am not able to fix it currently.

@JackBoosY
Copy link
Contributor

Hi @myd7349, can you use this patch to solve the problem of building errors under ARM/ARM64?
fix-builderror-in-arm.zip

Thanks.

@myd7349
Copy link
Contributor Author

myd7349 commented May 9, 2019

Hi @JackBoosY Thanks!
I will give it a try after work.

@myd7349 myd7349 mentioned this pull request May 11, 2019
Closed
7 tasks
@myd7349
Copy link
Contributor Author

myd7349 commented May 11, 2019 •
edited
Loading

Hi! @JackBoosY I am not sure whether it will work out by defining get/put_unaligned* as get/put_unaligned_direct since I saw these commits in snappy_compact.h. So I used another patch which I thought might be safer. Thanks for your help anyway.
I have create an issue here for tackling this: confluentinc/librdkafka#2324

@myd7349
Copy link
Contributor Author

myd7349 commented May 21, 2019 •
edited
Loading

Hi! Can anyone tell me whether the ARM patch work or not? If it works, maybe I can create a PR in the upstream.
@Rastaban @NancyLi1013 @JackBoosY Thanks in advance!

@myd7349 myd7349 changed the title [librdkafka] Add new port [WIP][librdkafka] Add new port May 21, 2019
JackBoosY
JackBoosY reviewed May 24, 2019
View reviewed changes
Copy link
Contributor

@JackBoosY JackBoosY left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When using the patch, we do not need to write the path, but we must use the file name + suffix.

ports/librdkafka/portfile.cmake Outdated Show resolved Hide resolved
@myd7349
Copy link
Contributor Author

myd7349 commented May 25, 2019

Hi! @JackBoosY I have fixed the patch filename. Does it work for ARM/ARM64 now? Thanks!

@JackBoosY
Copy link
Contributor

It seems that this port does not support uwp.
Except for uwp, other triplets are built successfully.

@myd7349
Copy link
Contributor Author

myd7349 commented May 27, 2019

@JackBoosY Thanks!
BTW, it seems that I can see the CI log now:
#6618
#6628
https://dev.azure.com/vcpkg/public/_build?definitionId=10&view=buildsHistory

@grdowns
Copy link
Contributor

grdowns commented Jun 11, 2019

/azp run

@grdowns
Copy link
Contributor

grdowns commented Jul 1, 2019

/azp run

@myd7349
Merge branch 'master' into rdkafka-init
086c727
@myd7349
[librdkafka] Update to 1.1.0
11708d2 
[librdkafka] Use vcpkg_check_features

[librdkafka] Remove default feature
@myd7349 myd7349 changed the title [WIP][librdkafka] Add new port [librdkafka] Add new port Jul 2, 2019
@grdowns
Copy link
Contributor

grdowns commented Jul 2, 2019

Thanks for the new port, @myd7349!

@grdowns grdowns merged commit 59995fa into microsoft:master Jul 2, 2019
@grdowns grdowns removed the wip label Jul 2, 2019
@myd7349 myd7349 deleted the rdkafka-init branch July 2, 2019 23:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JackBoosY JackBoosY JackBoosY left review comments

Assignees

@grdowns grdowns

@JackBoosY JackBoosY

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@myd7349 @Rastaban @NancyLi1013 @webfolderio @JackBoosY @grdowns