Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GH-494] ci: Build and test on multiple GNU/Linux architectures #577

Conversation

rettichschnidi
Copy link
Contributor

@rettichschnidi rettichschnidi commented Apr 5, 2021

Foundation for coverage of non-amd64 GNU/Linux architectures in our CI. Not using qemu-user directly on Ubuntu 20.04 as it is missing support for certain architectures which are supported by Debian (armv6/armel, mipsel, and others).

Some unit tests where failing on aarch64, s390x and ppc64le before. Fixed it by rewriting utils_textToFloat(). See commit messages for details.

Please note:

  • Due to incompatibilities of CMake 3.16, which is bundled with Debian Buster, and QEMU, which executes the Docker containers with the foreign architectures, updating to CMake version 3.19 or newer is needed. As KitWare only offers prebuilt binaries for x86_64 and aarch64, we need to build CMake from source, which takes a long time (~90 minutes). Luckily, the resulting docker image will be cached and subsequent runs (on the same GitHub account) run much faster.

At a later point, I intend to also add GNU/Linux on MIPSel and, if possible, ARMv5. Before this can happen however, those architectures must be added to https://github.com/uraimo/run-on-arch-action first.

@rettichschnidi rettichschnidi force-pushed the gardena/rs/multiple-architectures branch from b45f60e to 39fc4f7 Compare April 6, 2021 08:23
@rettichschnidi rettichschnidi marked this pull request as draft April 6, 2021 10:30
@rettichschnidi rettichschnidi changed the title [GH-494] ci: Build and test on ARMv6 architecture [GH-494] ci: Build and test on multiple GNU/Linux architectures Apr 6, 2021
@rettichschnidi rettichschnidi force-pushed the gardena/rs/multiple-architectures branch 4 times, most recently from fb5d75c to a3ee8f0 Compare April 6, 2021 16:37
@rettichschnidi rettichschnidi marked this pull request as ready for review April 6, 2021 18:38
@rettichschnidi rettichschnidi force-pushed the gardena/rs/multiple-architectures branch from a3ee8f0 to 3d2b300 Compare April 7, 2021 08:28
@rettichschnidi
Copy link
Contributor Author

Rebased on most recent master, knowing this will break the build on aarch64, s390x and ppc64le. Its just too good an example of why this PR makes sense. :)

Will submit a fix later on.

@rettichschnidi rettichschnidi marked this pull request as draft April 7, 2021 08:31
This also fixes the CMake build on big endian architectures and the unit
tests on aarch64, s390x, ppc64le.

This is more complex than it should be because...
- Ubuntu 20.04 does not support many architectures, Debian does (e.g.
  ARMv5, MIPSel)
- CMake versions before 3.19 are affected by
  https://gitlab.kitware.com/cmake/cmake/-/issues/20568

Signed-off-by: Reto Schneider <code@reto-schneider.ch>
A buffer filled with a float string followed by non-numeric data is a
valid input.

Signed-off-by: Reto Schneider <code@reto-schneider.ch>
@rettichschnidi rettichschnidi force-pushed the gardena/rs/multiple-architectures branch from 3d2b300 to de4c697 Compare April 9, 2021 08:21
@rettichschnidi rettichschnidi marked this pull request as ready for review April 9, 2021 08:21
@rettichschnidi rettichschnidi force-pushed the gardena/rs/multiple-architectures branch 2 times, most recently from 81840e8 to b0c0ede Compare April 9, 2021 08:31
Leveraging the C99 implementation strtod() simplifies and corrects the
code, at the cost of one additional memory allocation.

Some additional tests have been added to cover the corner cases. However,
as we can not (yet) instrument lwm2m_malloc(), memory exhaustion is not
tested.

Signed-off-by: Reto Schneider <code@reto-schneider.ch>
@rettichschnidi rettichschnidi force-pushed the gardena/rs/multiple-architectures branch from b0c0ede to 99c78f0 Compare April 9, 2021 08:34
@rettichschnidi
Copy link
Contributor Author

rettichschnidi commented Apr 9, 2021

Rebased on most recent master, knowing this will break the build on aarch64, s390x and ppc64le. Its just too good an example of why this PR makes sense. :)

Will submit a fix later on.

Done. No longer a draft, reviews wanted 😄

Copy link
Contributor

@tuve tuve left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine to me, but i find it a bit odd that the function is named utils_textToFloat and then we do strtod and populate a a double. Maybe it is nitpicking but shouldn't yo rename the function also (this will of course affect everywhere it is used).

@rettichschnidi
Copy link
Contributor Author

Looks fine to me, but i find it a bit odd that the function is named utils_textToFloat and then we do strtod and populate a a double. Maybe it is nitpicking but shouldn't yo rename the function also (this will of course affect everywhere it is used).

I think the current API needs a lot of rework, including an upfront discussion (#544?), but I'd rather not do this as part of this issue.

@sbernard31 sbernard31 merged commit 99c78f0 into eclipse-wakaama:master Apr 12, 2021
@rettichschnidi rettichschnidi deleted the gardena/rs/multiple-architectures branch April 12, 2021 15:33
}

/* Must have a decimal digit first after optional sign */
if (i >= length || !isdigit(buffer[i])) return 0;
char *const buffer_c_str = lwm2m_malloc(length + 1);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realize this has already been merged, but I would prefer to see utils_textToFloat operate without the dynamic memory allocation. Dynamic memory adds a possible failure that is very real on a constrained device.

Copy link
Contributor Author

@rettichschnidi rettichschnidi Apr 19, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would have preferred it too, but could not see a realistic way to get the conversion working properly (i.e. passing the existing unit tests) on all architectures.

If you do not feel comfortable using this code in your product, please let me know and I will give it a 2nd try to come up with a different solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants