Update bundled musl to 1.2.5 #125692

est31 · 2024-05-29T02:54:30Z

Update the bundled musl library from 1.2.3 to the 1.2.5 release from February 29, 2024.

Releases page

Prior update: #107129

r? wesleywiser

try-job: dist-x86_64-musl
try-job: dist-i586-gnu-i586-i686-musl
try-job: dist-x86_64-linux

Kobzol · 2024-05-29T06:49:55Z

Let's try on CI.

@bors try

bors · 2024-05-29T06:51:05Z

⌛ Trying commit 2173f48 with merge 14bf8cb...

Update bundled musl to 1.2.5 [Releases page](https://musl.libc.org/releases.html) Prior update: rust-lang#107129 r? `@wesleywiser` try-job: dist-x86_64-musl try-job: dist-i586-gnu-i586-i686-musl

bors · 2024-05-29T09:44:50Z

☀️ Try build successful - checks-actions
Build commit: 14bf8cb (14bf8cb2ba148468f901fb4ff5126efd3bec469c)

When using the linux-musl target for rust, the file creation time cannot be retrieved, as the current version does not support it yet ( This will be fixed with [0]). In the meantime, we parse the datetime from the filename and use that as a fallback. Fixes: tokio-rs#2999 [0]: rust-lang/rust#125692

szokeasaurusrex · 2024-07-03T13:02:33Z

Hello, are there any plans to merge this PR anytime soon? Doing so might unblock rust-cross/rust-musl-cross#152, which could resolve getsentry/sentry-cli#1929.

Kobzol · 2024-07-03T13:08:35Z

The release notes look relatively harmless. I'll nominate this for t-compiler discussion.

Kobzol · 2024-07-03T13:13:22Z

This looks like a rather small update, that could nevertheless unblock some projects.

@rustbot label +I-compiler-nominated

apiraino · 2024-07-11T10:05:57Z

Discussed in t-compiler triage meeting on Zulip.

cc @wesleywiser

@rustbot label -I-compiler-nominated

wesleywiser · 2024-07-12T03:05:26Z

Looking at the symbol tables for libc.a in musl 1.2.3 vs 1.2.5, I see the following missing symbols:

Symbol	Provided in `libc` crate	Notes
`aio_cancel64`		Not widely used
`aio_error64`		Not widely used
`aio_fsync64`		Not widely used
`aio_read64`		Not widely used
`aio_return64`		Not widely used
`aio_write64`		Not widely used
`aio_suspend64`		Not widely used
`lio_listio64`		Not widely used
`alphasort64`		Not widely used
`readdir64`	✅
`readdir64_r`	✅
`scandir64`		Not widely used
`versionsort64`		Not widely used
`creat64`	✅
`open64`	✅
`openat64`	✅
`posix_fadvise64`	✅
`posix_fallocate64`	✅
`ftw64`		Not widely used
`fallocate64`	✅
`getdents64`		⚠️ Some use in ecosystem
`prlimit64`	✅
`sendfile64`	✅
`getrlimit64`	✅
`lockf64`		Not widely used
`ntw64`		Not widely used
`setrlimit64`	✅
`mmap64`	✅
`glob64`		⚠️ Some use in ecosystem
`globfree64`		⚠️ Some use in ecosystem
`__fxstat64`		Not widely used
`__fxstatat64`		Not widely used
`__lxstat64`		Not widely used
`__xstat64`		Not widely used
`fstat64`	✅
`fstatat64`	✅
`lstat64`	✅
`stat64`	✅
`fstatfs64`	✅
`fstatvfs64`	✅
`statfs64`	✅
`statvfs64`	✅
`fgetpos64`	✅
`fopen64`	✅
`freopen64`	✅
`fseeko64`	✅
`fsetpos64`	✅
`ftello64`	✅
`tmpfile64`	✅
`mkostemp64`		Not widely used
`mkostemps64`		Not widely used
`mkstemp64`		Not widely used
`mkstemps64`		Not widely used
`ftruncate64`	✅
`lseek64`	✅
`pread64`	✅
`preadv64`	✅
`pwrite64`	✅
`pwritev64`	✅
`truncate64`	✅

libc began providing these symbols in 0.2.145 (rust-lang/libc#2935).

I think we should definitely do a crater run to help gauge how much impact losing the LFS64 symbols will be.

wesleywiser · 2024-07-12T19:11:20Z

The try build artifacts seem to have timed out. Let's go again:

@bors try

bors · 2024-07-12T19:12:32Z

🔒 Merge conflict

This pull request and the master branch diverged in a way that cannot be automatically merged. Please rebase on top of the latest master branch, and let the reviewer approve again.

How do I rebase?

Assuming self is your fork and upstream is this repository, you can resolve the conflict following these steps:

git checkout update_musl_1.2.5 (switch to your branch)
git fetch upstream master (retrieve the latest master)
git rebase upstream/master -p (rebase on top of it)
Follow the on-screen instruction to resolve conflicts (check git status if you got lost).
git push self update_musl_1.2.5 --force-with-lease (update this PR)

You may also read Git Rebasing to Resolve Conflicts by Drew Blessing for a short tutorial.

Please avoid the "Resolve conflicts" button on GitHub. It uses git merge instead of git rebase which makes the PR commit history more difficult to read.

Sometimes step 4 will complete without asking for resolution. This is usually due to difference between how Cargo.lock conflict is handled during merge and rebase. This is normal, and you should still perform step 5 to update this PR.

Error message

Auto-merging src/doc/rustc/src/platform-support.md
CONFLICT (content): Merge conflict in src/doc/rustc/src/platform-support.md
Automatic merge failed; fix conflicts and then commit the result.

Update bundled musl to 1.2.5 Update the bundled musl library from 1.2.3 to the 1.2.5 release from February 29, 2024. [Releases page](https://musl.libc.org/releases.html) Prior update: rust-lang#107129 `r? wesleywiser` try-job: dist-x86_64-musl try-job: dist-i586-gnu-i586-i686-musl

bors · 2024-07-13T01:53:21Z

⌛ Trying commit 1b6a1f8 with merge 65efa13...

bors · 2024-07-13T03:24:29Z

☀️ Try build successful - checks-actions
Build commit: 65efa13 (65efa137c3954425fa4cef841f977caf3d87f6f7)

wesleywiser · 2024-07-15T15:59:43Z

We also need an x86_64-unknown-linux-gnu host compiler so I added try-job: dist-x86_64-linux to the PR description.

@bors try

bors · 2024-07-15T16:00:52Z

⌛ Trying commit 1b6a1f8 with merge 115cea8...

Update bundled musl to 1.2.5 Update the bundled musl library from 1.2.3 to the 1.2.5 release from February 29, 2024. [Releases page](https://musl.libc.org/releases.html) Prior update: rust-lang#107129 `r? wesleywiser` try-job: dist-x86_64-musl try-job: dist-i586-gnu-i586-i686-musl try-job: dist-x86_64-linux

[DO NOT MERGE] Build musl targets for crater run Using to test rust-lang#125692 try-job: dist-x86_64-musl try-job: dist-i586-gnu-i586-i686-musl try-job: dist-x86_64-linux r? `@ghost`

bors · 2024-07-15T17:51:37Z

☀️ Try build successful - checks-actions
Build commit: 115cea8 (115cea8e821b9c40aef33b4644037c0963561120)

wesleywiser · 2024-07-15T22:16:10Z

@craterbot run mode=build-and-test name=musl_upgrade_1_2_5 start=try#52500f34dbfa5ef09f94c5cb770491fbd834bcb6+target=x86_64-unknown-linux-musl end=try#115cea8e821b9c40aef33b4644037c0963561120+target=x86_64-unknown-linux-musl

craterbot · 2024-07-15T22:16:17Z

👌 Experiment musl_upgrade_1_2_5 created and queued.
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

craterbot · 2024-07-21T09:28:20Z

🚧 Experiment musl_upgrade_1_2_5 is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

craterbot · 2024-07-23T12:58:52Z

🎉 Experiment musl_upgrade_1_2_5 is completed!
📊 11788 regressed and 91 fixed (482394 total)
📰 Open the full report.

⚠️ If you notice any spurious failure please add them to the blacklist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

wesleywiser · 2024-07-23T15:27:52Z

Looked at ~20 random regressions and they were all due to getrandom using an older version of libc which calls open64.

I think we need to consider building musl with -D_LARGEFILE64_SOURCE to get the LFS64 symbols back. That will give time for newer versions of the libc crate to propagate through the ecosystem.

Skgland · 2024-07-23T15:55:09Z

Is this already fixed in a released libc version? I noticed one of my repos with getrandom v0.2.6 and libc v0.2.126 in there.
Those are only some patch versions behind latest getrandom v0.2.15 and libc v0.2.155, so I would expect running cargo update to be able to get the lockfile to latest. Is there someway I could test whether updating the lock file would resolve the problem?

est31 · 2024-07-24T00:40:35Z

Is this already fixed in a released libc version?

Yes, it's fixed in 0.2.145, also mentioned in the post above. 0.2.145 was released on June 4, 2023, so more than one year ago, but I suppose not everyone runs cargo update regularly.

I think we need to consider building musl with -D_LARGEFILE64_SOURCE to get the LFS64 symbols back.

Yeah, agreed. To verify beyond your sample of 20 crates, I downloaded the regressed crates logs from the crater report and checked the regressions. There is only 146 regressions which do not contain "undefined reference". I checked 3-4 of them and they seemed all noise like out of space errors.

$ rg -g "*115cea8e821b9c40aef33b4644037c0963561120*" -l 'undefined reference' | wc -l
11642
$ rg -g "*115cea8e821b9c40aef33b4644037c0963561120*" --files-without-match 'undefined reference' | wc -l
146

Statistics of which functions were noted as missing:

$ rg -g "*115cea8e821b9c40aef33b4644037c0963561120*" 'undefined reference to' | sed 's/^.* undefined reference to//;s/ symbol //' | sort | uniq -c | sort -rn
  13041  `open64'
    176 'open64@@GLIBC_2.2.5'
     57  `fopen64'
     41  `mmap64'
     38  `fstat64'
     22  `fseeko64'
     15 'readdir64_r@@GLIBC_2.2.5'
     11  `tmpfile64'
     11  `posix_fadvise64'
     11  `mkstemp64'
     11  `ftello64'
      5  `lseek64'
      3  `setrlimit64'
      3  `readdir64_r'
      3  `getrlimit64'
      3  `fallocate64'
      2  `statvfs64'
      2  `readdir64'
      2  `openat64'
      1  `statfs64'
      1  `sendfile64'
      1  `pread64'
      1 'mkstemp64@@GLIBC_2.2.5'
      1  `lstat64'
      1 'lseek64@@GLIBC_2.2.5'
      1  `__fxstat64'
      1  `ftruncate64'
      1 'fstat64@@GLIBC_2.33'

not an expert but given they all contain 64 in their names, they seem to be from LFS64.

So next step is probably me figuring out how to pass -D_LARGEFILE64_SOURCE to the musl build...

est31 · 2024-07-24T00:51:46Z

@bors try

Update bundled musl to 1.2.5 Update the bundled musl library from 1.2.3 to the 1.2.5 release from February 29, 2024. [Releases page](https://musl.libc.org/releases.html) Prior update: rust-lang#107129 `r? wesleywiser` try-job: dist-x86_64-musl try-job: dist-i586-gnu-i586-i686-musl try-job: dist-x86_64-linux

bors · 2024-07-24T00:52:56Z

⌛ Trying commit 974ed62 with merge 497c108...

bors · 2024-07-24T02:43:48Z

☀️ Try build successful - checks-actions
Build commit: 497c108 (497c108e484bd2f2c090cc4ab84ab4c59703cecd)

est31 · 2024-07-24T06:16:01Z

For trying out the try build, I'm using https://github.com/kennytm/rustup-toolchain-install-master . command:

rustup-toolchain-install-master -t x86_64-unknown-linux-musl -- 115cea8e821b9c40aef33b4644037c0963561120
rustup-toolchain-install-master -t x86_64-unknown-linux-musl -- 497c108e484bd2f2c090cc4ab84ab4c59703cecd

with that, one can check the custom toolchains via:

cargo +115cea8e821b9c40aef33b4644037c0963561120 build --target x86_64-unknown-linux-musl
cargo +497c108e484bd2f2c090cc4ab84ab4c59703cecd build --target x86_64-unknown-linux-musl

apparently though, neither works :(.

I see the flag added to CFLAGS though here...

est31 · 2024-07-24T06:34:37Z

Ahhh apparently the -D_LARGEFILE64_SOURCE flag needs to be set by the application that wants to use these functions, not when compiling musl itself, see link here.

The actual linkable symbols were removed by this commit, first included in 1.2.4. The commit also adjusted the dynamic linker to account for that removal, but as we don't use it in this instance, that adjustment can't be leveraged.

I wonder if we should patch musl to make the symbols linkable again?

rustbot assigned wesleywiser May 29, 2024

rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels May 29, 2024

est31 force-pushed the update_musl_1.2.5 branch from c4a778e to 2173f48 Compare May 29, 2024 02:54

est31 added relnotes Marks issues that should be documented in the release notes of the next release. O-musl Target: The musl libc labels May 29, 2024

est31 marked this pull request as ready for review May 29, 2024 20:48

kaffarell mentioned this pull request Jun 6, 2024

tracing-appender: log rotation doesn't work with musl target tokio-rs/tracing#2999

Open

kaffarell mentioned this pull request Jun 7, 2024

appender: add fallback to file creation date tokio-rs/tracing#3000

Open

messense mentioned this pull request Jun 13, 2024

Upgrade musl to 1.2.5 rust-cross/rust-musl-cross#152

Open

rustbot added the I-compiler-nominated Nominated for discussion during a compiler team meeting. label Jul 3, 2024

rustbot removed the I-compiler-nominated Nominated for discussion during a compiler team meeting. label Jul 11, 2024

wesleywiser removed the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jul 12, 2024

wesleywiser mentioned this pull request Jul 15, 2024

[DO NOT MERGE] Build musl targets for crater run #127768

Closed

craterbot added S-waiting-on-crater Status: Waiting on a crater run to be completed. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 15, 2024

craterbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-crater Status: Waiting on a crater run to be completed. labels Jul 23, 2024

est31 added 2 commits July 24, 2024 02:51

Update bundled musl to 1.2.5

905cafc

add -D_LARGEFILE64_SOURCE to CFLAGS for compatibility

974ed62

est31 force-pushed the update_musl_1.2.5 branch from 1b6a1f8 to 974ed62 Compare July 24, 2024 00:51

usamoi mentioned this pull request Sep 16, 2024

remove ctime for musl environment kamiyaa/joshuto#565

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update bundled musl to 1.2.5 #125692

Update bundled musl to 1.2.5 #125692

est31 commented May 29, 2024 •

edited by wesleywiser

Loading

Kobzol commented May 29, 2024

bors commented May 29, 2024

bors commented May 29, 2024

szokeasaurusrex commented Jul 3, 2024

Kobzol commented Jul 3, 2024 •

edited

Loading

Kobzol commented Jul 3, 2024

apiraino commented Jul 11, 2024

wesleywiser commented Jul 12, 2024 •

edited

Loading

wesleywiser commented Jul 12, 2024

bors commented Jul 12, 2024

bors commented Jul 13, 2024

bors commented Jul 13, 2024

wesleywiser commented Jul 15, 2024

bors commented Jul 15, 2024

bors commented Jul 15, 2024

wesleywiser commented Jul 15, 2024

craterbot commented Jul 15, 2024

craterbot commented Jul 21, 2024

craterbot commented Jul 23, 2024

wesleywiser commented Jul 23, 2024

Skgland commented Jul 23, 2024 •

edited

Loading

est31 commented Jul 24, 2024 •

edited

Loading

est31 commented Jul 24, 2024

bors commented Jul 24, 2024

bors commented Jul 24, 2024

est31 commented Jul 24, 2024

est31 commented Jul 24, 2024 •

edited

Loading

Update bundled musl to 1.2.5 #125692

Are you sure you want to change the base?

Update bundled musl to 1.2.5 #125692

Conversation

est31 commented May 29, 2024 • edited by wesleywiser Loading

Kobzol commented May 29, 2024

bors commented May 29, 2024

bors commented May 29, 2024

szokeasaurusrex commented Jul 3, 2024

Kobzol commented Jul 3, 2024 • edited Loading

Kobzol commented Jul 3, 2024

apiraino commented Jul 11, 2024

wesleywiser commented Jul 12, 2024 • edited Loading

wesleywiser commented Jul 12, 2024

bors commented Jul 12, 2024

bors commented Jul 13, 2024

bors commented Jul 13, 2024

wesleywiser commented Jul 15, 2024

bors commented Jul 15, 2024

bors commented Jul 15, 2024

wesleywiser commented Jul 15, 2024

craterbot commented Jul 15, 2024

craterbot commented Jul 21, 2024

craterbot commented Jul 23, 2024

wesleywiser commented Jul 23, 2024

Skgland commented Jul 23, 2024 • edited Loading

est31 commented Jul 24, 2024 • edited Loading

est31 commented Jul 24, 2024

bors commented Jul 24, 2024

bors commented Jul 24, 2024

est31 commented Jul 24, 2024

est31 commented Jul 24, 2024 • edited Loading

est31 commented May 29, 2024 •

edited by wesleywiser

Loading

Kobzol commented Jul 3, 2024 •

edited

Loading

wesleywiser commented Jul 12, 2024 •

edited

Loading

Skgland commented Jul 23, 2024 •

edited

Loading

est31 commented Jul 24, 2024 •

edited

Loading

est31 commented Jul 24, 2024 •

edited

Loading