[pull] master from ruby:master #364

Add a new API rb_profile_thread_frames(), which is essentialy a per-thread version of rb_profile_frames(). While the original rb_profile_frames() always returns results about the current active thread obtained by GET_EC(), this new API takes a Thread to be profiled as an argument. This should come in handy when profiling I/O-bound programs such as webapps, since this new API allows us to learn about Threads performing I/O (which do not have the GVL). Profiling worker threads (such as Sidekiq workers) may be another application. Implements [Feature #10602] Co-authored-by: Mike Perham <mike@perham.net>

Currently, we do not have an amd64 job that runs with YJIT disabled and runs the full check task. Most amd64 jobs implicitly have YJIT enabled, even though --enable-yjit was not specified. This makes it easier to determine whether a problem is YJIT-related or not by reviewing CI logs.

For static analysis, it’s better to keep unused literal nodes. If simply change `block_append` to fall through, both "unused literal ignored" and "possibly useless use of a literal in void context" warnings are shown for the same line. But it’s verbose then remove "unused literal ignored" warning. This kind of optimization is already implemented on compile.c. `compile_block` calls `iseq_compile_each0` with `popped = 1` when NODE_BLOCK has next.

ruby/prism@7dbb8c7e3e

ruby/prism@6e5556dbdf

ruby/prism@fa108b8626

…he cache rubygems/rubygems@8d51390ca4

This commit implements compilation for the PostExeuctionNode by using the ScopeNode to create child iseqs where appropriate.

There is a handful of call sites where we may transition to OBJ_TOO_COMPLEX_SHAPE if we just ran out of shapes, but that weren't handling it properly.

We weren't taking in to account that objects with generic IV tables could go "too complex" in the IV set code. This commit takes that in to account and also ensures FL_EXIVAR is set when a geniv object transitions to "too complex" Co-Authored-By: Jean Boussier <byroot@ruby-lang.org>

On 32-bit systems, we must store the shape ID in the gen_ivtbl to not lose the shape. If we directly store the ST table into the generic ivar table, then we lose the shape. This makes it impossible to determine the shape of the object and whether it is too complex or not.

Useful if your plugin introduces new methods to the DSL, so that Gemfiles can easily abort if the plugin hasn't loaded yet rubygems/rubygems@b733055c6e

it was a performance improvement only, but it causes failures in unrelated tests rubygems/rubygems@b4149cb9bf

`remove_shape_recursive` wasn't considering that if we run out of shapes, it might have to transition to SHAPE_TOO_COMPLEX. When this happens, we now return with an error and the caller initiates the evacuation.

…d macro

Prior to this commit, KeywordParameterNode included both optional and required keywords. With this commit, it is split in two, with `OptionalKeywordParameterNode`s no longer having a value field. ruby/prism@89084d9af4

… RequiredKeywordParameterNode ruby/prism@c31f61e898

ruby/prism@e13f2e4590

ruby/prism@2a11bfee76

ruby/prism@8062849d0d

ruby/prism@cfd4431c3e

ruby/prism@c2b7724d91

ruby/prism@3abd09c803

ruby/prism@085da4feb9

ruby/prism@ca9a660f52

ruby/prism@dfdcc98a3c

ruby/prism@2b6e661bbc

ruby/prism@bec5ca37a0

ruby/prism@ff1d2ec579

ruby/prism@97b3cc0af8

ruby/prism@88336e7d9f

ruby/prism@26934263b7

ruby/prism@52a0d80a15

ruby/prism@76af64eb21

ruby/prism@16e0579044

ruby/prism@e2e29bef2d

ruby/prism@857f72e1f4

ruby/prism@271f3bf628

ruby/prism@69323d3df4

ruby/prism@9c648ce615

ruby/prism@84e22c5581

ruby/prism@83f737ce17

ruby/prism@e327449db6

ruby/prism@77d4056766

ruby/prism@ec419422f8

String#chars returns an array instead of an enumerator since Ruby 2.0.

`rb_file_open_str` calls `FilePathValue`, and the converted result is not used in this function.

Generic names like Node and NodeList should be namespaced properly.

`mswin` platform links to `vcruntimeXXX.dll` but `mingw-ucrt` doesn't.

… from rubygems_integration.rb ioquatix/bake#15 (comment)

There is a duplicated check for the object is too complex.

This commit makes every initial size pool shape a root shape and assigns it a capacity of 0.

This commit changes class ivars to respect the capacity transition in shapes rather than growing the capacity independently.

Unfortunately, 84dfa0f introduced a bug where we were no longer testing the "popped" case because the "; 1" meant to be appended to the source was no longer functioning as intended. This commit re-introduces the popped case, and comments out all now failing tests.

It's possible for us to parse a constant that is invalid in the current encoding. To fix this, we wrap the intern function in an rb_protect to ensure we always put a valid symbol into the AST. ruby/prism@e78398554d

If the required capacity would fit in an embded string, returns one. This can reduce malloc churn for code that use string buffers.

Wait for the worker thread to finish.

Co-authored-by: Adam Hess <HParker@github.com>

Previously emitting a call node with an argument followed by another node would cause the argument to be mistakenly omitted from the argument list causing a stack underflow. ``` PRISM: ************************************************** -- raw disasm-------- 0000 putself ( 0) 0001 send <calldata:puts, 1>, nil ( 0) * 0004 pop ( 0) 0005 putobject 1 ( 0) 0007 leave ( 0) --------------------- ```

…_generate in ppc64le. We observed that this test randomly fails in the ruby/ruby Travis ppc64le case. This commit is to pend the test_generate if the assertion for the generated file's modified time fails in a ppc64le environment. Note that I didn't use the word "Travis CI" or Travis CI specific environment variables such as `TRAVIS` and `TRAVIS_CPU_ARCH`[1] in the code. Because I wanted to prioritize the rdoc's independence from the ruby/ruby. [1] https://docs.travis-ci.com/user/environment-variables/#default-environment-variables ruby/rdoc@42cdad1cf2

It was assuming only objects can be complex.

Now I expect that Travis ppc64le always passes without any random failures by the commit 1f1b9b0 that is a workaround to pass the `test/rdoc/test_rdoc_generator_json_index.rb` in Travis ppc64le case.

When an inline cache misses, it is very likely that the stale shape_id and the current instance shape_id have a close common ancestor. For example if the instance variable is sometimes frozen sometimes not, one of the two shape will be the direct parent of the other. Another pattern that commonly cause IC misses is "memoization", in such case the object will have a "base common shape" and then a number of close descendants. In addition, when we find a common ancestor, we store it in the inline cache instead of the current shape. This help prevent the cache from flip-flopping, ensuring the next lookup will be marginally faster and more generally avoid writing in memory too much. However, now that shapes have an ancestors index, we only check for a few ancestors before falling back to use the index. So overall this change speeds up what is assumed to be the more common case, but makes what is assumed to be the less common case a bit slower. ``` compare-ruby: ruby 3.3.0dev (2023-10-26T05:30:17Z master 701ca07) [arm64-darwin22] built-ruby: ruby 3.3.0dev (2023-10-26T09:25:09Z shapes_double_sear.. a723a85235) [arm64-darwin22] warming up...... | |compare-ruby|built-ruby| |:------------------------------------|-----------:|---------:| |vm_ivar_stable_shape | 11.672M| 11.679M| | | -| 1.00x| |vm_ivar_memoize_unstable_shape | 7.551M| 10.506M| | | -| 1.39x| |vm_ivar_memoize_unstable_shape_miss | 11.591M| 11.624M| | | -| 1.00x| |vm_ivar_unstable_undef | 9.037M| 7.981M| | | 1.13x| -| |vm_ivar_divergent_shape | 8.034M| 6.657M| | | 1.21x| -| |vm_ivar_divergent_shape_imbalanced | 10.471M| 9.231M| | | 1.13x| -| ``` Co-Authored-By: John Hawthorn <john@hawthorn.email>

I think the "1+$(nproc)" is too much.

… options ruby/prism@99e81619de

ruby/prism@8582d372a3

ruby/prism@f0aa8ad93b

ruby/prism@5a2252e3ac

ruby/prism@4b538af20f

ruby/prism@84229529d7

ruby/prism@33cc75a4b7

ruby/prism@13fa262669

ruby/prism@81a9b2817f

ruby/prism@4f3a3e3ec1

This commit changes generic ivars to respect the capacity transition in shapes rather than growing the capacity independently.

We were previously holding a type field on Comment to tell what kind of comment it was. Instead, let's just use actual classes for this. ruby/prism@e76830ca6e

ruby/prism@fbb30216ca

Use GCC version 11.4.0 (gcc-11 package) instead of the default GCC 11.3.0 to align with the GCC used in the RubyCI s390x server below. ``` $ /usr/bin/gcc --version gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 Copyright (C) 2021 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. $ ls -l /usr/bin/gcc lrwxrwxrwx 1 root root 6 Aug 5 2021 /usr/bin/gcc -> gcc-11* $ dpkg -S /usr/bin/gcc-11 gcc-11: /usr/bin/gcc-11 ``` Ubuntu Jammy gcc-11: https://packages.ubuntu.com/jammy-updates/gcc-11

ruby/prism@61a306cd5c

When a begin node is popped it only needs to putnil if that nil is going to be the return value, otherwise it can successfully be optimised out.

ruby/prism@145a67d1c3

The version information is useful to report issues to Ruby and RubyGems projects.

Add a RubyGems issue ticket link. The issue happend in Travis arm32 at once.

…ibutes ruby/prism@0d0d039f8e

* YJIT: implement two-step call threshold Automatically switch call threshold to a larger value for larger, production-sized apps, while still allowing smaller apps and command-line programs to start with a lower threshold. * Update yjit/src/options.rs Co-authored-by: Alan Wu <XrXr@users.noreply.github.com> * Make the new variables constants * Check that a custom call threshold was not specified --------- Co-authored-by: Alan Wu <XrXr@users.noreply.github.com>

It's crashing inside the bug reporter after a crash, so not sure why it's crashing. It's not really useful for maintaining RJIT to flag this test failure, so let's just ignore it until we figure out why it fails. https://github.com/ruby/ruby/actions/runs/6752729246/job/18358439166

ruby/stringio@3f90a0d619

ruby/stringio@4400bf3380

first_column and last_column return byte positions, but existing implementations did not consider multibyte.

No encodings are guaranteed in C compilers, and other than UTF-8 encodings may be assumed in some platforms, e.g., CP932 on Windows Japanese edition, and may result in compilation errors.

ruby/etc@e4c71e5996

Looking at building_ruby.md it is unclear that you are able to build Ruby on Windows. To fix that a reference has been added to the windows.md file.

LibreSSL seems not to support `scrypt`. https://rubyci.s3.amazonaws.com/openbsd-current/ruby-master/log/20231105T233005Z.fail.html.gz ``` 1) OpenSSL::KDF.scrypt creates the same value with the same input ERROR NoMethodError: undefined method `scrypt' for module OpenSSL::KDF ```

ruby/net-http@c1c5638014

ruby/net-http@4be99c204c

ruby/rdoc@9c14229876

`Kernel#require` converts feature name objects that have the `to_path` method such as `Pathname`, but had used the original object on error and had resulted in an unexpected `TypeError`.

ruby/un@0d6669bff0

ruby/un@1f636a6239

ruby/benchmark@81af12352d

ruby/singleton@3c7fb5f258

ruby/shellwords@fc63ac832d

ruby/yaml@f83b0404d5

ruby/uri@b50d37f7a1

Compilation now works for MultiWriteNodes and MultiTargetNodes, with nesting on MultiWrites. See the tests added in this commit for example behavior.

character attributes (ruby/reline#552) * Reine::Face * fix test_yamatanooroti * Define singleton methods to make accessors to attributes of a face * s/display/foreground/ * s/default/default_style/ && s/normal_line/default/ && s/enhanced_line/enhanced/ * fix typo * FaceConfig.new now takes keyword arguments * Update lib/reline/face.rb Co-authored-by: Stan Lo <stan001212@gmail.com> * Update test/reline/test_face.rb Co-authored-by: Stan Lo <stan001212@gmail.com> * Fix to correspond to frozen_string_literal * Face::FaceConfig -> Face::Config * ref ruby/reline#552 (review) * delete unused ivar * ref ruby/reline#552 (comment) * insert "\e[0m" into all SGR * tiny fix * ESSENTIAL_DEFINE_NAMES ref ruby/reline#552 (comment) * Change to Hash-accessor style - Reline::Face[:completion_dialog].enhanced -> Reline::Face[:completion_dialog][:enhanced] - Reline::Face.configs shows all defined values * Cache array method call in local variable * Tests for Face configuration variations * resolve ruby/reline#552 (review) * amend to * check invalid SGR parameter in :style * The order of define values should be preserved * Update test/reline/test_face.rb Co-authored-by: Stan Lo <stan001212@gmail.com> * Update test/reline/test_face.rb Co-authored-by: Stan Lo <stan001212@gmail.com> * Add methods: load_initial_config and reset_to_initial_config. And teardown in tests * omission in amending "style: :default" to "style: :reset" * refs ruby/reline#598 * Fix link * amend method name * Update lib/reline/face.rb Co-authored-by: ima1zumi <52617472+ima1zumi@users.noreply.github.com> --------- ruby/reline@fdc1d3b1e5 Co-authored-by: Stan Lo <stan001212@gmail.com> Co-authored-by: ima1zumi <52617472+ima1zumi@users.noreply.github.com>

(ruby/reline#600) * Test Reline::Face without mocking Because `test-unit-rr` is not a default gem, using it would break CRuby's CI. * Add ruby-core workflow ruby/reline@d2189ac436

It wasn't supposed to be sync-ed in: 2dd32e7

following up on #8832

This reverts commit de82439.

After a pthread for getaddrinfo is detached, we cannot predict when the thread will exit. It would lead to a segfault by setting pthread_setaffinity to the terminated pthread. I guess this problem would be more likely to occur in high-load environments. This change detaches the pthread after pthread_setaffinity is called. [Feature #19965]

@nobu

pointed by @nobu

Do not use `pthread_attr_setaffinity_np` if `sched_getcpu()` exceeds `CPU_SETSIZE`. (Using `CPU_ALLOC()` would be more appropriate.)

`$=` has been obsolete and has no effect since 1.9. ruby/English@121939695a

ruby/pp@6e086e6df9

ruby/prettyprint@5ea64f43fa

ruby/optparse@acbf6e3e12

ruby/weakref@f763384744

ruby/tsort@2b3fe78646

ruby/rinda@f6d76b3927

ruby/open3@0db9a0afe7

ruby/tmpdir@8bbd87c8ac

ruby/mutex_m@635da25a09

ruby/pstore@6d5c0833e7

ruby/observer@6c978e6196

ruby/getoptlong@f49629dfaa

ruby/find@192237e766

ruby/tempfile@d6ddf7881e

ruby/net-protocol@2d3c4b43a8

ruby/date@50e18d2684

ruby/securerandom@12c4ef3ea1

ruby/delegate@d4bdf89328

ruby/timeout@3e42aa4d84

ruby/timeout@a65e49cc31

ruby/time@8dba14997c

ruby/logger@0996f90650

Follow up ruby/bigdecimal#89. `BigDecimal.new` has already been removed. This PR replaces `BigDecimal.new` with `Kernel.BigDecimal` in the documentation, following the message below: > BigDecimal.new is deprecated; use Kernel.BigDecimal method instead. ruby/bigdecimal@26d84ba ruby/bigdecimal@dd52adf3b2