Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Migrate to Playwright from Cypress #2471

Draft
wants to merge 74 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
e8b5768
Remove CircleCI config.
kopepasah Nov 24, 2022
c146b24
Cleanup current actions.
kopepasah Nov 24, 2022
a358106
Rename old actions.
kopepasah Nov 24, 2022
890d4cb
Add .nvmrc file.
kopepasah Nov 24, 2022
9cdaa83
Add localization workflow.
kopepasah Nov 24, 2022
e75dce4
Add create artifact workflow, and initial release workflow.
kopepasah Nov 24, 2022
337339e
Comment out default branch code until ready.
kopepasah Nov 24, 2022
6bab74d
Install dev on artifact.
kopepasah Nov 24, 2022
9093d2c
Introduce validation-coding-standards.yml (#2452)
EvanHerman Dec 2, 2022
293b6e3
Merge branch 'master' into feature/github-actions-migration
kopepasah Dec 6, 2022
c1bc713
GitHub Actions Migration - Add Initial Tests (#2457)
kopepasah Dec 6, 2022
e931b80
Try running close in beforeEach.cy.visit.onLoad()
kopepasah Dec 6, 2022
3df550d
Add accordion test.
kopepasah Dec 6, 2022
b569438
Try just alert block.
kopepasah Dec 6, 2022
3be7831
Run with author.
kopepasah Dec 6, 2022
ca50f25
Fix toolbar options not showing.
kopepasah Dec 6, 2022
e9e10dc
Change when the save is reloaded.
kopepasah Dec 6, 2022
7694be0
Try waiting 5 seconds.
kopepasah Dec 6, 2022
47bc557
Run headed.
kopepasah Dec 6, 2022
f1099c5
Switch back to old method
kopepasah Dec 6, 2022
ff884b9
Add Playwright.
kopepasah Jan 20, 2023
1ab174e
Ignore playwright files.
kopepasah Jan 20, 2023
769ce49
Add Playwright config.
kopepasah Jan 20, 2023
8679ebe
Add playwright global setup.
kopepasah Jan 20, 2023
152e047
Change testMatch and add alert test.
kopepasah Jan 20, 2023
6f31593
Merge branch 'master' into feature/github-actions-migration-WPEX-2662
kopepasah Jan 20, 2023
5db384a
Merge branch 'master' into feature/github-actions-migration
kopepasah Jan 20, 2023
0b21f5d
Fix global setup to use correct context.
kopepasah Jan 20, 2023
b6f87d4
Ignore recoreded videos from Playwright.
kopepasah Jan 20, 2023
2815122
Update Playwright.
kopepasah Jan 20, 2023
9d276ca
Update config.
kopepasah Jan 20, 2023
1a730bd
Group test and run in parallel.
kopepasah Jan 20, 2023
c98554f
Finish up accordion e2e test.
kopepasah Jan 23, 2023
c45c5b9
Merge branch 'master' into feature/github-actions-migration
kopepasah Jan 23, 2023
f86ffc5
Merge branch 'feature/github-actions-migration' into feature/github-a…
kopepasah Jan 23, 2023
bcd2baa
Add click-to-tweet test.
kopepasah Jan 23, 2023
585b3d8
Revert changes for Cypress.
kopepasah Jan 23, 2023
37a2a09
Only run the alert test for now.
kopepasah Jan 23, 2023
f1f0f87
Readd to block check in Cypress.
kopepasah Jan 23, 2023
abc35d8
Wait for wp.data to exist.
kopepasah Jan 23, 2023
68668ae
Lets try calling here.
kopepasah Jan 23, 2023
b8f33a4
Revert "Lets try calling here."
kopepasah Jan 24, 2023
29992f1
Revert "Wait for wp.data to exist."
kopepasah Jan 24, 2023
491207b
Add check for login for goTo()
kopepasah Jan 24, 2023
35a77fd
Use wp.data object instead of window.
kopepasah Jan 25, 2023
bccff1d
Run test using yarn.
kopepasah Jan 25, 2023
858b4b9
Try with one retry.
kopepasah Jan 25, 2023
283b3eb
Add accordion spec.
kopepasah Jan 25, 2023
a04f1ad
Import data on tests-cli.
kopepasah Jan 26, 2023
2d5ad31
Run from bin.
kopepasah Jan 26, 2023
25dd479
Reverse order.
kopepasah Jan 26, 2023
a9ff562
Let’s try this.
kopepasah Jan 26, 2023
b26b56f
Add another test.
kopepasah Jan 26, 2023
26e7a19
Try with all tests.
kopepasah Jan 26, 2023
73d9e8e
Add additional tests.
kopepasah Jan 26, 2023
c3f80df
Remove record key.
kopepasah Jan 26, 2023
831bef0
Go back to 0 retries.
kopepasah Jan 26, 2023
b02da25
Make record key not required.
kopepasah Jan 26, 2023
1e28717
WP version.
kopepasah Jan 26, 2023
20f0c85
Change name.
kopepasah Jan 26, 2023
9d0f4b8
Set wp version.
kopepasah Jan 26, 2023
0e76fd3
Does it pass on 5.9?
kopepasah Jan 26, 2023
7bcf93e
Input on tests-cli.
kopepasah Jan 26, 2023
49c088d
Let’s fast fail.
kopepasah Jan 26, 2023
b3318c4
Run in non test.
kopepasah Jan 26, 2023
a90d3ed
Remove fast fail.
kopepasah Jan 26, 2023
74c5545
Try this.
kopepasah Jan 26, 2023
5b401e5
Add firefox.
kopepasah Jan 27, 2023
51eca58
Remove firefox.
kopepasah Jan 27, 2023
f59ffce
Merge branch 'feature/github-actions-migration' into feature/github-a…
kopepasah Jan 27, 2023
95de0d5
Merge branch 'master' into feature/github-actions-migration-WPEX-2662
kopepasah Jan 27, 2023
86bd6bd
Remove superfluous job.
kopepasah Jan 27, 2023
c5ded8f
Merge branch 'master' into feature/github-actions-migration-WPEX-2662
kopepasah Feb 3, 2023
892bba2
Merge branch 'master' into feature/github-actions-migration-WPEX-2662
AnthonyLedesma Feb 14, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions .dev/tests/playwright/global-setup.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
/**
* Global setup file for Playwright tests, and is executed once before all tests
* are run.
*/
const { chromium, expect } = require( '@playwright/test' );

module.exports = async () => {
const browser = await chromium.launch();
const context = await browser.newContext( {
baseURL: 'http://localhost:8889',
recordVideo: {
dir: '.dev/tests/playwright/videos',
},
} );

const page = await context.newPage();

// Log in to WordPress.
await page.goto( '/wp-login.php' );
await page.fill( '#user_login', 'admin' );
await page.fill( '#user_pass', 'password' );
await page.click( '#wp-submit' );
await page.goto( '/wp-admin/post-new.php?post_type=post' );

// Wait for wp.data to be available.
await page.waitForFunction( () => window.wp.data !== undefined );

// Ensure the Welcome Guide is disabled.
if ( await page.evaluate( `wp.data.select( 'core/edit-post' ).isFeatureActive( 'welcomeGuide' )` ) ) {
await page.evaluate( `wp.data.dispatch( 'core/edit-post' ).toggleFeature( 'welcomeGuide' )` );
}

await page.reload();

// Expect the Welcome Guide to be hidden.
await expect( page.locator( '.edit-post-welcome-guide' ) ).toBeHidden();

// Save signed-in state to 'storageState.json'.
await page.context().storageState( { path: '.dev/tests/playwright/storage-state.json' } );
await context.close();
await browser.close();
};
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,8 @@ languages/*.po
cypress.env.json
.wp-env.json
.wp-env.override.json
/test-results/
/playwright-report/
/playwright/.cache/
.dev/tests/playwright/videos/
.dev/tests/playwright/storage-state.json
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@
"@godaddy-wordpress/eslint-config": "^0.6.0",
"@godaddy-wordpress/stylelint-config": "^0.6.0",
"@jest/core": "^27.4.5",
"@playwright/test": "^1.29.2",
"@react-google-maps/api": "^2.7.0",
"@testing-library/jest-dom": "^5.16.1",
"@testing-library/react": "^12.1.5",
Expand Down
56 changes: 56 additions & 0 deletions playwright.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
// @ts-check
const { devices } = require( '@playwright/test' );

const config = {
expect: {
timeout: 5000,
},
forbidOnly: !! process.env.CI,
fullyParallel: true,
globalSetup: require.resolve( './.dev/tests/playwright/global-setup' ),
outputDir: './.dev/tests/playwright/test-results/',
projects: [
{
name: 'chromium',
use: {
...devices[ 'Desktop Chrome' ],
},
},

{
name: 'firefox',
use: {
...devices[ 'Desktop Firefox' ],
},
},

/**
* WordPress version 6.1+ throws and error with the version of Webkit
* that Playwright currently utilizes. Likely this will be resolved when
* Gutenberg finishes migrating to Playwright, and then we can use the
* wordpress/e2e-test-utils-playwright package from Core.
*/
// {
// name: 'webkit',
// use: {
// ...devices[ 'Desktop Safari' ],
// },
// },
],

reporter: process.env.CI ? 'dot' : [ [ 'html', { open: 'never' } ] ],
retries: process.env.CI ? 2 : 0,
testDir: './src',
testMatch: '**/src/**/**/*.playwright.js',
timeout: 30 * 1000,
use: {
actionTimeout: 0,
baseURL: 'http://localhost:8889',
headless: true,
storageState: './.dev/tests/playwright/storage-state.json',
trace: 'on-first-retry',
video: 'on-first-retry',
},
};

module.exports = config;
8 changes: 8 additions & 0 deletions src/blocks/accordion/test/accordion.cypress.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,14 @@ import * as helpers from '../../../../.dev/tests/cypress/helpers';

describe( 'Block: Accordion', () => {
beforeEach( () => {
cy.visit( Cypress.env( 'testURL' ) + '/wp-admin/post-new.php?post_type=post', {
onLoad: ( contentWindow ) => {
if ( !! contentWindow.wp.data.select( 'core/edit-post' ).isFeatureActive( 'welcomeGuide' ) ) {
contentWindow.wp.data.dispatch( 'core/edit-post' ).toggleFeature( 'welcomeGuide' );
}
},
} );

helpers.addBlockToPost( 'coblocks/accordion', true );
} );

Expand Down
74 changes: 74 additions & 0 deletions src/blocks/accordion/test/accordion.playwright.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
import { test, expect } from '@playwright/test';

const blockName = 'coblocks/accordion';

test.describe.parallel( 'Accordion', () => {
test.beforeEach( async ( { page, browserName }, info ) => {
await page.goto( '/wp-admin/post-new.php?post_type=post' );

await page.waitForFunction( () => window.wp.data !== undefined );

await page.evaluate( `wp.data.dispatch( 'core/editor' ).editPost( { title: '${ browserName } - ${ info.title }' } )` );
await page.evaluate( `wp.data.dispatch( 'core/block-editor' ).insertBlock( wp.blocks.createBlock( '${ blockName }' ) )` );
await page.evaluate( `wp.data.dispatch( 'core/editor' ).savePost()` );
} );

test( 'saves with no title or content', async ( { page } ) => {
await expect( page.locator( `[data-type="${ blockName }"]` ) ).toBeVisible();
await page.reload();
await expect( page.locator( `[data-type="${ blockName }"]` ) ).toBeVisible();
} );

test( 'saves with custom title', async ( { page } ) => {
const title = 'GoDaddy';
const titleLocator = '.wp-block-coblocks-accordion-item__title';

await page.type( titleLocator, title );

await expect( page.locator( titleLocator ) ).not.toBeEmpty();

await expect( page.locator( titleLocator ) ).toContainText( title );

await page.evaluate( `wp.data.dispatch( 'core/editor' ).savePost()` );
await page.reload();

await expect( page.locator( titleLocator ) ).toContainText( title );
} );

test( 'saves with custom content', async ( { page } ) => {
const clientId = await page.evaluate( `wp.data.select( 'core/block-editor' ).getBlocks().shift().innerBlocks.shift().clientId` );
const contentLocator = '.wp-block-coblocks-accordion-item__content .wp-block-paragraph:first-child';
const contentText = 'Domain Names, Websites, Hosting & Online Marketing Tools';

await page.type( '.wp-block-coblocks-accordion-item__title', 'GoDaddy' );
await page.evaluate( `wp.data.dispatch( 'core/block-editor' ).insertBlocks( wp.blocks.createBlock( 'core/paragraph', { content: '${ contentText }' } ), 0, '${ clientId }' )` );

await expect( page.locator( contentLocator ) ).toBeVisible();
await expect( page.locator( contentLocator ) ).not.toBeEmpty();

await expect( page.locator( contentLocator ) ).toContainText( contentText );

await page.evaluate( `wp.data.dispatch( 'core/editor' ).savePost()` );
await page.reload();

await page.locator( '.wp-block-coblocks-accordion-item__title' ).focus();
await expect( page.locator( contentLocator ) ).toContainText( contentText );
} );

test( 'appends additional accordion item', async ( { page } ) => {
const accordionItemsLocator = '.wp-block-coblocks-accordion [data-type="coblocks/accordion-item"]';
await expect( page.locator( '.coblocks-block-appender' ) ).toBeVisible();

await page.locator( '.coblocks-block-appender' ).click();

let accordionItems = await page.locator( accordionItemsLocator );
await expect( accordionItems ).toHaveCount( 2 );

await page.evaluate( `wp.data.dispatch( 'core/editor' ).savePost()` );
await page.reload();

accordionItems = await page.locator( accordionItemsLocator );
await expect( accordionItems ).toHaveCount( 2 );
} );
} );

10 changes: 10 additions & 0 deletions src/blocks/alert/test/alert.cypress.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,16 @@
import * as helpers from '../../../../.dev/tests/cypress/helpers';

describe( 'Test CoBlocks Alert Block', function() {
beforeEach( () => {
cy.visit( Cypress.env( 'testURL' ) + '/wp-admin/post-new.php?post_type=post', {
onLoad: ( contentWindow ) => {
if ( !! contentWindow.wp.data.select( 'core/edit-post' ).isFeatureActive( 'welcomeGuide' ) ) {
contentWindow.wp.data.dispatch( 'core/edit-post' ).toggleFeature( 'welcomeGuide' );
}
},
} );
} );

/**
* Test that we can add a alert block to the content, not add any text or
* alter any settings, and are able to successfully save the block without errors.
Expand Down
42 changes: 42 additions & 0 deletions src/blocks/alert/test/alert.playwright.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
import { test, expect } from '@playwright/test';

const blockName = 'coblocks/alert';

test.describe.parallel( 'Alert', () => {
test.beforeEach( async ( { page, browserName }, info ) => {
await page.goto( '/wp-admin/post-new.php?post_type=post' );

await page.waitForFunction( () => window.wp.data !== undefined );

await page.evaluate( `wp.data.dispatch( 'core/editor' ).editPost( { title: '${ browserName } - ${ blockName } - ${ info.title }' } )` );
await page.evaluate( `wp.data.dispatch( 'core/block-editor' ).insertBlocks( wp.blocks.createBlock( '${ blockName }' ) )` );
await page.evaluate( `wp.data.dispatch( 'core/editor' ).savePost()` );
} );

test( 'saves with no title or text', async ( { page } ) => {
await expect( page.locator( `[data-type="${ blockName }"]` ) ).toBeVisible();
await page.reload();
await expect( page.locator( `[data-type="${ blockName }"]` ) ).toBeVisible();
} );

test( 'saves with custom title and text', async ( { page } ) => {
const alertTitle = 'GoDaddy';
const alertText = 'Domain Names, Websites, Hosting & Online Marketing Tools';

await page.type( '.wp-block-coblocks-alert__title', alertTitle );
await page.type( '.wp-block-coblocks-alert__text', alertText );

await expect( page.locator( `.wp-block-coblocks-alert__title` ) ).not.toBeEmpty();
await expect( page.locator( `.wp-block-coblocks-alert__text` ) ).not.toBeEmpty();

await expect( page.locator( `.wp-block-coblocks-alert__title` ) ).toContainText( alertTitle );
await expect( page.locator( `.wp-block-coblocks-alert__text` ) ).toContainText( alertText );

await page.evaluate( `wp.data.dispatch( 'core/editor' ).savePost()` );
await page.reload();

await expect( page.locator( `.wp-block-coblocks-alert__title` ) ).toContainText( alertTitle );
await expect( page.locator( `.wp-block-coblocks-alert__text` ) ).toContainText( alertText );
} );

} );
55 changes: 55 additions & 0 deletions src/blocks/click-to-tweet/test/click-to-tweet.playwright.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
import { test, expect } from '@playwright/test';

const blockName = 'coblocks/click-to-tweet';

test.describe.parallel( 'Click to Tweet', () => {
test.beforeEach( async ( { page, browserName }, info ) => {
await page.goto( '/wp-admin/post-new.php?post_type=post' );

await page.waitForFunction( () => window.wp.data !== undefined );

await page.evaluate( `wp.data.dispatch( 'core/editor' ).editPost( { title: '${ browserName } - ${ info.title }' } )` );
await page.evaluate( `wp.data.dispatch( 'core/block-editor' ).insertBlock( wp.blocks.createBlock( '${ blockName }' ) )` );
await page.evaluate( `wp.data.dispatch( 'core/editor' ).savePost()` );
} );

test( 'saves with no content', async ( { page } ) => {
await expect( page.locator( `[data-type="${ blockName }"]` ) ).toBeVisible();
await page.reload();
await expect( page.locator( `[data-type="${ blockName }"]` ) ).toBeVisible();
} );

test( 'saves with custom content', async ( { page } ) => {
const text = 'GoDaddy - Domain Names, Websites, Hosting & Online Marketing Tools';
const textLocator = '.wp-block-coblocks-click-to-tweet__text';

await page.type( textLocator, text );

await expect( page.locator( textLocator ) ).not.toBeEmpty();

await expect( page.locator( textLocator ) ).toContainText( text );

await page.evaluate( `wp.data.dispatch( 'core/editor' ).savePost()` );
await page.reload();

await expect( page.locator( textLocator ) ).toContainText( text );
} );

test( 'can add tweet via', async ( { page } ) => {
const via = 'godaddy';
const viaLocator = '.wp-block-coblocks-click-to-tweet__via';

await page.type( viaLocator, via );

await expect( page.locator( viaLocator ) ).not.toBeEmpty();

await expect( page.locator( viaLocator ) ).toHaveValue( via );

await page.evaluate( `wp.data.dispatch( 'core/editor' ).savePost()` );
await page.reload();

await page.locator( '.wp-block-coblocks-click-to-tweet__text' ).focus();
await expect( page.locator( viaLocator ) ).toHaveValue( via );
} );
} );

19 changes: 16 additions & 3 deletions yarn.lock
Original file line number Diff line number Diff line change
Expand Up @@ -1583,6 +1583,14 @@
"@nodelib/fs.scandir" "2.1.5"
fastq "^1.6.0"

"@playwright/test@^1.29.2":
version "1.29.2"
resolved "https://registry.yarnpkg.com/@playwright/test/-/test-1.29.2.tgz#c48184721d0f0b7627a886e2ec42f1efb2be339d"
integrity sha512-+3/GPwOgcoF0xLz/opTnahel1/y42PdcgZ4hs+BZGIUjtmEFSXGg+nFoaH3NSmuc7a6GSFwXDJ5L7VXpqzigNg==
dependencies:
"@types/node" "*"
playwright-core "1.29.2"

"@pmmmwh/react-refresh-webpack-plugin@^0.5.2":
version "0.5.6"
resolved "https://registry.npmjs.org/@pmmmwh/react-refresh-webpack-plugin/-/react-refresh-webpack-plugin-0.5.6.tgz"
Expand Down Expand Up @@ -2184,9 +2192,9 @@
integrity sha512-HUAiN65VsRXyFCTicolwb5+I7FM6f72zjMWr+ajGk+YTvzBgXqa2A5U7d+rtsouAkunJ5U4Sb5lNJjo9w+nmXg==

"@types/node@*":
version "17.0.33"
resolved "https://registry.npmjs.org/@types/node/-/node-17.0.33.tgz"
integrity sha512-miWq2m2FiQZmaHfdZNcbpp9PuXg34W5JZ5CrJ/BaS70VuhoJENBEQybeiYSaPBRNq6KQGnjfEnc/F3PN++D+XQ==
version "18.11.18"
resolved "https://registry.yarnpkg.com/@types/node/-/node-18.11.18.tgz#8dfb97f0da23c2293e554c5a50d61ef134d7697f"
integrity sha512-DHQpWGjyQKSHj3ebjFI/wRKcqQcdR+MoFBygntYOZytCqNfkd2ZC4ARDJ2DQqhjH5p85Nnd3jhUJIXrszFX/JA==

"@types/node@^14.14.31":
version "14.18.18"
Expand Down Expand Up @@ -10655,6 +10663,11 @@ pkg-dir@4.2.0, pkg-dir@^4.1.0, pkg-dir@^4.2.0:
dependencies:
find-up "^4.0.0"

playwright-core@1.29.2:
version "1.29.2"
resolved "https://registry.yarnpkg.com/playwright-core/-/playwright-core-1.29.2.tgz#2e8347e7e8522409f22b244e600e703b64022406"
integrity sha512-94QXm4PMgFoHAhlCuoWyaBYKb92yOcGVHdQLoxQ7Wjlc7Flg4aC/jbFW7xMR52OfXMVkWicue4WXE7QEegbIRA==

plur@^4.0.0:
version "4.0.0"
resolved "https://registry.npmjs.org/plur/-/plur-4.0.0.tgz"
Expand Down