Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve block loading PHP performance #2252

Closed
wants to merge 25 commits into from

Conversation

aristath
Copy link
Member

This PR improves the PHP performance of blocks:

  • Adds a new Grunt task to convert block.json files to block-json.php
  • Uses the new block-json.php files in the register_block_type_from_metadata() function

Note: To test this patch, you'll need to run npm run build, or npm run grunt blockJson2PHP to generate the PHP files from JSON.

Trac ticket: https://core.trac.wordpress.org/ticket/55005


This Pull Request is for code review only. Please keep all other discussion in the Trac ticket. Do not merge this Pull Request. See GitHub Pull Requests for Code Review in the Core Handbook for more details.

Gruntfile.js Outdated
@@ -1370,6 +1371,17 @@ module.exports = function(grunt) {
}
} );

grunt.registerTask( 'blockJson2PHP', 'Copies block.json file contents to block-json.php.', function() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

block.json files are copied with webpack:

const blockMetadataFiles = {
'widgets/src/blocks/legacy-widget/block.json': 'wp-includes/blocks/legacy-widget/block.json',
'widgets/src/blocks/widget-group/block.json': 'wp-includes/blocks/widget-group/block.json',
...blockFolders.reduce( ( files, blockName ) => {
files[ `block-library/src/${ blockName }/block.json` ] = `wp-includes/blocks/${ blockName }/block.json`;
return files;
} , {} ),
};

I think it would be better to convert them to PHP files in one go. At the moment, you would have both block.json and block-json.php files in the same folder where the former becomes unused.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could easily remove the JSON files, I'm just not sure if it's something we want to do or not.

  • I was thinking that maybe we want to use the .json files instead of the .php files when WP_DEBUG and/or SCRIPT_DEBUG is true
  • The concept in this patch was to make this change as minimal as possible, avoiding any backwards-compatibility issues etc, just in case a plugin is accessing the JSON files for core blocks. Is that a valid worry, or do you think it's an edge-case? 🤔

Also, I'm not that familiar with webpack so using grunt was a crutch, the only way I knew how to do this 😅

@gziolo
Copy link
Member

gziolo commented Jan 31, 2022

How more performant does this become when we replace JSON files with PHP files? It's maybe 50-60 files to process.

@aristath
Copy link
Member Author

How more performant does this become when we replace JSON files with PHP files? It's maybe 50-60 files to process.

Hmmm I was looking for a way to run some PHP profiling when using @wordpress/env but so far haven't found a solution. Any ideas?

@aristath
Copy link
Member Author

aristath commented Jan 31, 2022

Not exactly a concrete result, but it does confirm that there is indeed an improvement.
I used this code:

add_action( 'init', function() {
	$start = microtime( true );
	for ( $i = 0; $i < 1000; $i++ ) {
		register_block_type_from_metadata( '/var/www/src/wp-includes/blocks/button' );
		register_block_type_from_metadata( '/var/www/src/wp-includes/blocks/buttons' );
		register_block_type_from_metadata( '/var/www/src/wp-includes/blocks/cover' );
		register_block_type_from_metadata( '/var/www/src/wp-includes/blocks/calendar' );
	}
	$end = microtime( true );
	$diff = $end - $start;

	echo 'Execution time: ' . $diff;
});

The results of doing something like that are notoriously unreliable, so I kept refreshing my page for ~50 times before and after the patch, and kept the best result for both pre/post-patch.

Prior to the patch, the best result I got was ~6.38s (max: 12)
After the patch, it dropped to ~4.23s (max: 5)

It's clear that the function now becomes faster (33% faster by my results), but I have no metric to see what the actual impact would be on a WP site... so I'll theorize a bit:
When I run npm run grunt blockJson2PHP, it generates 71 files so I'm assuming that's how many times the function will run on a normal site. My loop runs the function 4000 times (4 calls x 1000 loops), so 56.34 times more than WP Core would. The difference of my pre/post runs was 2.15s (6.38s - 4.23s), so dividing that by 56.34 I get 0.038s, or 38ms.
That sounds like a lot so my results are probably false, but even if it's just a tenth of that, it's still 4ms per-page-load on 40% of the web 😅

@gziolo
Copy link
Member

gziolo commented Jan 31, 2022

Maybe we could find a way to combine all those PHP files for core into a single file if we are at 71 files already. It should further speed up the processing. You could write a file that returns key-value pairs where the path to the block directory is the key, and the metadata is the value. It would be more work, as you would have to check this array for core blocks but maybe it's worth it.

@aristath
Copy link
Member Author

Hmmm it would make sense... and we'd only have to load a single file. Great idea 👍

@aristath
Copy link
Member Author

aristath commented Feb 1, 2022

Maybe we could find a way to combine all those PHP files for core into a single file if we are at 71 files already. It should further speed up the processing. You could write a file that returns key-value pairs where the path to the block directory is the key, and the metadata is the value. It would be more work, as you would have to check this array for core blocks but maybe it's worth it.

Applied the changes. It's marginally faster, but it makes more sense since we're only reading a single file instead of performing 70 filesystem requests. If the filesystem is relatively slow, then this should further improve the performance 👍

@aristath aristath force-pushed the fix/55005 branch 2 times, most recently from 61bbe6f to 688e7a5 Compare February 1, 2022 13:26
@johnbillion
Copy link
Member

It would be great to pretty print the block-json.php file so subsequent updates to the file produce a more readable diff, but it looks like json2php doesn't support pretty printing.

Copy link

@stokesman stokesman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this initiative 🙌. I think it'd be really great to implement it a way that lets third parties do the same. For example, if register_block_type_from_metadata were modified to accept string|array for the first argument so the metadata could be passed in that way. I'm suggesting this because I've been working on build tooling for my own blocks and wanted do this same thing. I've made it work but I have to largely re-implement register_block_type_from_metatdata.

Gruntfile.js Show resolved Hide resolved
Copy link
Member

@mukeshpanchal27 mukeshpanchal27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look solid. Great if it merge if core so it call only single file. Left one small change that we should end the PHP tag.

Gruntfile.js Show resolved Hide resolved
@SergeyBiryukov
Copy link
Member

Thanks for the PR! I've attempted to rebase it to see if that would resolve the test failures, but it did not.

It appears that there are two kinds of failures here:

  1. The "Ensure version-controlled files are not modified or deleted" job in E2E, JavaScript, and NPM tests reports some differences in the wp-includes/blocks/blocks-json.php file. When inspecting the differences, it looks like they have to do with this line being present in a few blocks before the change:

    'fontSize' => array('type' => 'string', 'default' => 'small')
    
    • core/comment-date
    • core/comment-edit-link
    • core/comment-reply-link
    • core/comment-template

    and not being present after the change.

  2. Some PHPUnit test failures (see below). Reading through the PR, it's not quite clear to me what exactly causes these. Hopefully this should provide some details to help with further debugging for someone more familiar with the code 🙂

There were 3 errors:

1) Tests_Blocks_Render::test_do_block_output with data set #12 ('core__cover.html', 'core__cover.server.html')
Undefined index: useFeaturedImage

/var/www/src/wp-includes/blocks/cover.php:17
/var/www/src/wp-includes/class-wp-block.php:256
/var/www/src/wp-includes/blocks.php:1027
/var/www/src/wp-includes/blocks.php:1065
/var/www/tests/phpunit/tests/blocks/render.php:[225](https://github.com/WordPress/wordpress-develop/actions/runs/3083280374/jobs/4984017435#step:17:226)
phpvfscomposer:///var/www/vendor/phpunit/phpunit/phpunit:51
/var/www/vendor/bin/phpunit:118

2) Tests_Blocks_Render::test_do_block_output with data set #33 ('core__latest-comments.html', 'core__latest-comments.server.html')
Undefined index: displayDate

/var/www/src/wp-includes/blocks/latest-comments.php:124
/var/www/src/wp-includes/class-wp-block.php:256
/var/www/src/wp-includes/blocks.php:1027
/var/www/src/wp-includes/blocks.php:1065
/var/www/tests/phpunit/tests/blocks/render.php:225
phpvfscomposer:///var/www/vendor/phpunit/phpunit/phpunit:51
/var/www/vendor/bin/phpunit:118

3) Tests_Blocks_Render::test_render_latest_comments_on_password_protected_post
Undefined index: displayAvatar

/var/www/src/wp-includes/blocks/latest-comments.php:65
/var/www/src/wp-includes/class-wp-block.php:256
/var/www/src/wp-includes/blocks.php:1027
/var/www/src/wp-includes/blocks.php:1065
/var/www/tests/phpunit/tests/blocks/render.php:329
phpvfscomposer:///var/www/vendor/phpunit/phpunit/phpunit:51
/var/www/vendor/bin/phpunit:118

--

There were 5 failures:

1) Tests_Blocks_Render::test_do_block_output with data set #8 ('core__column.html', 'core__column.server.html')
File '/var/www/tests/phpunit/includes/../data/blocks/fixtures/core__column.html' does not match expected value
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
 
-<div class="wp-container-1 wp-block-column">
+<div class="wp-block-column">
 	
 	<p>Column One, Paragraph One</p>
 	
 	
 	<p>Column One, Paragraph Two</p>
 	
 </div>

/var/www/tests/phpunit/tests/blocks/render.php:[240](https://github.com/WordPress/wordpress-develop/actions/runs/3083280374/jobs/4984017435#step:17:241)
phpvfscomposer:///var/www/vendor/phpunit/phpunit/phpunit:51
/var/www/vendor/bin/phpunit:118

2) Tests_Blocks_Render::test_do_block_output with data set #9 ('core__columns.html', 'core__columns.server.html')
File '/var/www/tests/phpunit/includes/../data/blocks/fixtures/core__columns.html' does not match expected value
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
 
 <div class="wp-container-1 wp-block-columns has-3-columns">
 	
-	<div class="wp-container-1 wp-block-column">
+	<div class="wp-block-column">
@@ @@
 	
-	<div class="wp-container-1 wp-block-column">
+	<div class="wp-block-column">
 		
 		<p>Column Two, Paragraph One</p>
 		
 		
 		<p>Column Three, Paragraph One</p>
 		
 	</div>
 	
 </div>

/var/www/tests/phpunit/tests/blocks/render.php:240
phpvfscomposer:///var/www/vendor/phpunit/phpunit/phpunit:51
/var/www/vendor/bin/phpunit:118

3) Tests_Blocks_RenderReusableCommentTemplate::test_rendering_comment_template
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-<ol class="wp-block-comment-template"><li id="comment-40" class="comment even thread-even depth-1"><div class="wp-block-comment-author-name"><a rel="external nofollow ugc" href="http://example.com/author-url/" target="_self" >Test</a></div><div class="wp-block-comment-content"><p>Hello world</p></div></li></ol>
+<ol class="wp-block-comment-template"><li id="comment-40" class="comment even thread-even depth-1"><div class="wp-block-comment-author-name">Test</div><div class="wp-block-comment-content"><p>Hello world</p></div></li></ol>

/var/www/tests/phpunit/tests/blocks/renderCommentTemplate.php:[285](https://github.com/WordPress/wordpress-develop/actions/runs/3083280374/jobs/4984017435#step:17:286)
phpvfscomposer:///var/www/vendor/phpunit/phpunit/phpunit:51
/var/www/vendor/bin/phpunit:118

4) Tests_Blocks_RenderReusableCommentTemplate::test_rendering_comment_template_nested
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-<ol class="wp-block-comment-template"><li id="comment-41" class="comment even thread-even depth-1"><div class="wp-block-comment-author-name"><a rel="external nofollow ugc" href="http://example.com/author-url/" target="_self" >Test</a></div><div class="wp-block-comment-content"><p>Hello world</p></div><ol><li id="comment-42" class="comment odd alt depth-2"><div class="wp-block-comment-author-name"><a rel="external nofollow ugc" href="http://example.com/author-url/" target="_self" >Test</a></div><div class="wp-block-comment-content"><p>Hello world</p></div><ol><li id="comment-44" class="comment even depth-3"><div class="wp-block-comment-author-name"><a rel="external nofollow ugc" href="http://example.com/author-url/" target="_self" >Test</a></div><div class="wp-block-comment-content"><p>Hello world</p></div></li></ol></li><li id="comment-43" class="comment odd alt depth-2"><div class="wp-block-comment-author-name"><a rel="external nofollow ugc" href="http://example.com/author-url/" target="_self" >Test</a></div><div class="wp-block-comment-content"><p>Hello world</p></div></li></ol></li></ol>
+<ol class="wp-block-comment-template"><li id="comment-41" class="comment even thread-even depth-1"><div class="wp-block-comment-author-name">Test</div><div class="wp-block-comment-content"><p>Hello world</p></div><ol><li id="comment-42" class="comment odd alt depth-2"><div class="wp-block-comment-author-name">Test</div><div class="wp-block-comment-content"><p>Hello world</p></div><ol><li id="comment-44" class="comment even depth-3"><div class="wp-block-comment-author-name">Test</div><div class="wp-block-comment-content"><p>Hello world</p></div></li></ol></li><li id="comment-43" class="comment odd alt depth-2"><div class="wp-block-comment-author-name">Test</div><div class="wp-block-comment-content"><p>Hello world</p></div></li></ol></li></ol>

/var/www/tests/phpunit/tests/blocks/renderCommentTemplate.php:391
phpvfscomposer:///var/www/vendor/phpunit/phpunit/phpunit:51
/var/www/vendor/bin/phpunit:118

5) Tests_Blocks_RenderReusableCommentTemplate::test_rendering_comment_template_unmoderated_preview
Should include unapproved comments when filter applied
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-<ol class="wp-block-comment-template"><li id="comment-47" class="comment even thread-even depth-1"><div class="wp-block-comment-author-name"><a rel="external nofollow ugc" href="http://example.com/author-url/" target="_self" >Test</a></div><div class="wp-block-comment-content"><p>Hello world</p></div></li><li id="comment-48" class="comment odd alt thread-odd thread-alt depth-1"><div class="wp-block-comment-author-name">Visitor</div><div class="wp-block-comment-content"><p><em class="comment-awaiting-moderation">Your comment is awaiting moderation.</em></p>Hi there! My comment needs moderation.</div></li></ol>
+<ol class="wp-block-comment-template"><li id="comment-47" class="comment even thread-even depth-1"><div class="wp-block-comment-author-name">Test</div><div class="wp-block-comment-content"><p>Hello world</p></div></li><li id="comment-48" class="comment odd alt thread-odd thread-alt depth-1"><div class="wp-block-comment-author-name">Visitor</div><div class="wp-block-comment-content"><p><em class="comment-awaiting-moderation">Your comment is awaiting moderation.</em></p>Hi there! My comment needs moderation.</div></li></ol>

/var/www/tests/phpunit/tests/blocks/renderCommentTemplate.php:510
phpvfscomposer:///var/www/vendor/phpunit/phpunit/phpunit:51
/var/www/vendor/bin/phpunit:118

src/wp-includes/blocks.php Outdated Show resolved Hide resolved
@aristath
Copy link
Member Author

I rebased too and also ran npm run build again to regenerate the block-json.php file.
The test failures I see don't make any sense and I don't see how they could be related to this PR...
@gziolo do you have any idea what the issue might be here? 🤔

src/wp-includes/blocks.php Outdated Show resolved Hide resolved
@gziolo
Copy link
Member

gziolo commented Sep 21, 2022

It looks like the latest change resolved all the issues. The failing CI check on Windows is a known issue that we are trying to address before WordPress 6.1 Beta 1.

In the future, we could go even one step further and come up with a way to register multiple blocks at once by passing a similar array as in the block-json.php file.

// Try to get metadata from the static cache for core blocks.
$metadata = false;
if ( str_starts_with( $file_or_folder, ABSPATH . WPINC ) ) {
$core_block_name = str_replace( ABSPATH . WPINC . '/blocks/', '', $file_or_folder );
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you considered using the same paths passed to the register_block_type as keys in the generated file? This way, you could do a simple check:

if ( ! empty( $core_blocks_meta[ $file_or_folder ] ) ) {
    $metadata = $core_blocks_meta[ $file_or_folder ];
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aren't those paths absolute - which means they'd change from site to site? 🤔

Copy link
Member

@gziolo gziolo Sep 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking you could set a variable to use in keys for the generated PHP file:

$dir = ABSPATH . WPINC . '/blocks/';

Maybe, it's too much work in a bit different place 😄

@mukeshpanchal27
Copy link
Member

@aristath I think it's good to set 6.1 milestone as it's ready for review and merge.

phpcs.xml.dist Outdated Show resolved Hide resolved
@SergeyBiryukov
Copy link
Member

Thanks for the PR! Merged in r54276.

@gziolo
Copy link
Member

gziolo commented Sep 21, 2022

I’m sure that @azaozz will be delighted when hee discovers that this optimization landed. Thank you so much for taking it to the finish line!

@aristath aristath deleted the fix/55005 branch September 22, 2022 07:54
@justlevine
Copy link

I'm not seeing any hooks allowing devs to load their own block-json.php. Is that because

  • there is an existing hook we can tap into (if so, what is it),
  • intentional design (if so, is there an alternative method for custom block registration to reap similar performance improvements), or
  • an oversight (if so is there already a ticket, and/or will it be included in 6.1)?

@aristath
Copy link
Member Author

replied on your comment on https://make.wordpress.org/core/2022/10/07/improved-php-performance-for-core-blocks-registration/#comment-43894 to keep the discussion in a single place and not fragmented 👍

@CreativeDive
Copy link

CreativeDive commented Apr 25, 2023

hey @aristath I saw that you added improvements to avoid loading the block.json file for each block type. But I noticed that this is not available for custom blocks. Personally I have more than 50 custom blocks and it would be very helpful to make the same improvements for custom blocks.

All we need is just a filter and everyone can create their own logic to load block data more efficiently.

Changing the existing part ...

if ( str_starts_with( $file_or_folder, ABSPATH . WPINC ) ) {
		$core_block_name = str_replace( ABSPATH . WPINC . '/blocks/', '', $file_or_folder );
		if ( ! empty( $core_blocks_meta[ $core_block_name ] ) ) {
			$metadata = $core_blocks_meta[ $core_block_name ];
		}
	}

... to this ...

if ( str_starts_with( $file_or_folder, ABSPATH . WPINC ) ) {
		$core_block_name = str_replace( ABSPATH . WPINC . '/blocks/', '', $file_or_folder );
		if ( ! empty( $core_blocks_meta[ $core_block_name ] ) ) {
			$metadata = $core_blocks_meta[ $core_block_name ];
		}
	} else {
		$metadata = apply_filters( 'custom_blocks_meta', $metadata, $file_or_folder ); 
	}

... and that's all. Is this something that can be extended and should I open a new issue for it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
10 participants