Time to introduce first UI tests

[derberg]

context: #1087

We need automation to run on PR level some automated UI tests.

The first test would be simple: evaluating if the certain list of id is present in HTML after rendering on a client-side.

More complicated it to setup infra for test:

• should we use Puppeteer or Playwritgh or Cypress
• or maybe we should use some external test runner like https://www.browserstack.com/ but of course choose the one that is for free for Open Source

Thoughts?

[arunavabasucom]

Can I work on this ??

[derberg]

definitely. but before any implementation please describe what is your plan, what tools you wanna use

[akshatnema]

Introduce UI tests and refactoring in the website

We need automation tests to run on the PR level which checks the overall build and codebase of the changes made in the PR. It should cover all the aspects of testing the website and the scripts running inside it. The idea of this project is not only to add UI tests, but also to refactor some parts in code to effectively create tests on it.

What are UI tests?

User Interface testing, also known as UI testing or GUI testing, tests the application’s visual elements to validate proper functionality and expected performance. It ensures that UI functions and application components have no defects, such as icons, radio buttons, text boxes, toolbars, color, fonts, checkboxes, windows, and menus. The primary aspects of UI testing are functionality, performance, usability, compliance, and visual design of the software application to make sure the application works successfully and satisfactorily.

Why do we need this?

We do need these tests because of following reasons:

• we have no tests to test the components and the functions inside it.
• more and more scripts are added to the repository
• no tests, means nothing forces us to write clearer and testable small functions.
• soon website maintenance/contribution will become hard as more and more addition of code will make Duplication and inconsistency inside the codebase.

Which tests you have to focus on?

We already have Neltify build tests and Lighthouse tests for each PR build, but the website wants more deep-down tests like automated Unit testing, Integration testing, etc. so maintainers need not to review each component individually for all the basic checks. Also, make sure you add one more test to check the IDs mentioned with each link or not. With more and more contributions in the website repository, it is now becoming difficult for reviewers to test each PR individually and with all tests. So, these types of tests are becoming important for the repository to remain maintainable in broad aspect.

What contributor has to do?

This project consists of first, a plan that specifies the tools for testing and the structure of tests that are needed in the repository. And then, requires the implementation of the tests with the proper refactoring of code as needed. Here are the points you need to be noted:

• Understand the codebase and list which type of tests are needed for the website.
• List the appropriate tools and logic on which tests will be built in the repository, with proper reason and research on it.
• Refactor the codebase according to your requirements such that it fit with the tests you want to create.
• Implement the various tests, both in UI and the automated scripts, to check each function's functionality appropriately.
• Integrate your tests either with GitHub Actions or Netlify or both, so that they are applied on each PR build

This issue is now being submitted as a GSoC project idea and will accept proposals for contribution from the contributors. The Mentors for this project are @akshatnema and @derberg.

[vasucp1207]

@akshatnema, I want to start working on this issue. Here is my plan for writing the test,
I started with unit tests as they can helps to catch bugs early in the development cycle and ensure that each component is functioning as expected.
The folder structure would be,

website
|__tests
   |__unitTest
      |__componentName
         |__componentName.test.js

I am considering using vitest or jest for writing the unit tests, know me your view about this then I can raise a PR with a basic test to check that I am doing it right or not.

[akshatnema]

Hey @vasucp1207, thanks for showing interest in the issue. Regarding your approach, I won't think it is much more scalable, as there are many components in the website and more to be added soon. So we can't add so many unit tests in the repo regarding each component. So, you should think an approach such that it should test all the pages and components used inside it in a go.

But please make sure, you don't initialize the PR right now as we are considering this idea as part of the upcoming Mentorship program. And you have to make the proposal regarding the approach you will use to solve this issue.

[vasucp1207]

Hey @vasucp1207, thanks for showing interest in the issue. Regarding your approach, I won't think it is much more scalable, as there are many components in the website and more to be added soon. So we can't add so many unit tests in the repo regarding each component. So, you should think an approach such that it should test all the pages and components used inside it in a go.

But please make sure, you don't initialize the PR right now as we are considering this idea as part of the upcoming Mentorship program. And you have to make the proposal regarding the approach you will use to solve this issue.

Yeh, I understand then we can write integration tests for simple components and end-to-end tests for components that required human interaction or page navigations, cypess is an easy choice for end-to-end tests.

[reachaadrika]

Hey , I am Aadrika Bhargava a software engineer based in India , with an experience in developing pixel perfect user interfaces . I have an expertise in frontend technologies like React and Next and I am interested for this project . Is there any chat channel I could join and move ahead .
Also a quick question , we need tests for the UI components only , or also tests to check PRs and workflows too ?

[akshatnema]

Is there any chat channel I could join and move ahead .

Hey @reachaadrika you can chat with all the mentors or maintainers here in issue also. There is no specific chat channel we have specified so far. You can also ask your queries at https://asyncapi.com/slack-invite.

we need tests for the UI components only , or also tests to check PRs and workflows too ?

Yepp we do need to check PRs and branches too to test all the components in the website.

[reachaadrika]

I can write tests for the UI Components , I have some doubts for how to do it for PR Tests and all , also key points for proposal ?

[akshatnema]

I can write tests for the UI Components , I have some doubts for how to do it for PR Tests and all , also key points for proposal ?

You can make a js script to run all the tests inside repo and then, we can run this script on every pull request opened using Github Actions. Also, regarding the proposal, you need to create your plan first that how you will implement testing in the website, according to the codebase of the repo. And then, plan your timeline accordingly inside contributing period.

[reachaadrika]

Also for the time being Can I create some miscellaneous issues along side ?
Any specific testing library we will be using ? I have react Testing library in my mind

[akshatnema]

Also for the time being Can I create some miscellaneous issues along side ?

Yes, you can create other issues in the repo if you find some. We welcome new issues and appreciate that new contributors work in order to resolve these issues and contribute to our projects.

Any specific testing library we will be using ? I have react Testing library in my mind

As such, we don't have any library constraint issue for testing. It's entirely your plan on how you want to make testing scripts on the website.

[imabp]

Hey folks, we are glad to see your interest in this idea.

As this is my first message, quickly introducing, I am Abir, and I will be mentoring along with Akshat for this idea.
that also means.. When @akshatnema is not merging your PR, ping me up 😉..

So serious things...here are some of the steps we are going to follow on before deciding to work on this, so make sure, you have included them in your proposals 😁

1. Choosing a Testing Library

   • This is the most crucial step, because, this will create the foundation
   • What are the criteria to choose a testing library?
     • Fast and Reliable
     • Flake Detection and Handling
       • Test flakiness, is somewhat we will be dealing in this project.
       • Flaky test cases, are those test cases, which sometimes passes/ and sometimes not.
     • User flows
       • User flows help to create mandatory test scenarios to pass for a code.

2. Draft out cases which are required to be tested.

   • Identify the scenarios which require testing.
   • Categorise them into critical, medium, low risk.

3. Examples

• We want to see more ideas, suggestions and libraries from folks, before we decide on things.
• Meanwhile, till we get a new library, Cypress (https://cypress.io) do suite our requirements.

All the best for your ideas. And have a great day as well.

[vaibhav-1508]

Hey there everyone! This issue looks interesting and I would like to contribute. Can I straight away make PRs for automatic testing or do I need to submit a proposal first?

[Harish-b-03]

Heyyyyyy Everyone 👋..... Helloo @akshatnema @imabp 😁

I am Harish, a react developer, CS student. I have already contributed to AsyncAPI, opened 4-5 PRs over the last few months and I would like to work on this project.

I will get some ideas and plans, and will get back to you.

Thank you 😀

[akshatnema]

Can I straight away make PRs for automatic testing or do I need to submit a proposal first?

@vaibhav-1508 You have to submit the proposal first as this issue is under the GSoC program.

[vaibhav-1508]

Can I straight away make PRs for automatic testing or do I need to submit a proposal first?

@vaibhav-1508 You have to submit the proposal first as this issue is under the GSoC program.

Ooh 😅
So how do I get started with that?

[vaibhav-1508]

Btw, I am Vaibhav Tripathi, B. Tech sophomore from IIT Varanasi, India. I found the projects interesting so I'm planning to contribute to the same for GSoC

[akshatnema]

So how do I get started with that?

You should prepare a document stating your plan and schedule to complete this project inside GSoC. You can see GSoC proposals as many are available on the Internet.

[vaibhav-1508]

So how do I get started with that?

You should prepare a document stating your plan and schedule to complete this project inside GSoC. You can see GSoC proposals as many are available on the Internet.

I mean do I need to contribute first anywhere before making the proposal?😅

[akshatnema]

I mean do I need to contribute first anywhere before making the proposal?:sweat_smile:

Not necessary, but if you want, you are welcomed to contribute to the repositories.

[reachaadrika]

@akshatnema & @imabp I tried to run this website on my local system but I was not able to do that , Shall I paste the error here ?

[akshatnema]

@reachaadrika yeah, you can ask here. Just make sure that you are using node v16+ (not v18) and npm v8.1+ versions in your laptop.

[reachaadrika]

Okay I was on node version 14.7.4 , I will upgrade it to 17 and npm versions are also old . I will upgrade them and then work

[Devanshu-17]

@derberg @imabp Hey, I am Devanshu Mahapatra, a Postman Student Expert and an AIML enthusiast. I am really excited to work on this, as this would be the first time that I will be contributing to an Open Source Organisation.

I have checked the async website and their code. And as per @derberg above,

We need automation to run on PR level some automated UI tests.

The first test would be simple: evaluating if the certain list of id is present in HTML after rendering on a client-side.

I think we can use puppeteer for the test codes and integrate it with Jest for running the tests.

Or, we can use Cypress and integrate with PR level automation using tools such as GitHub Actions or Netlify.

If you don't mind, can you please suggest, what would be best here.

[reachaadrika]

Can we include UI Changes to the website what I have thought off some and other use cases that have been identified or the focus has to be on testing only ?

[akshatnema]

Can we include UI Changes to the website what I have thought off some and other use cases that have been identified or the focus has to be on testing only?

@reachaadrika Sorry, this gsoc idea only focuses on testing so for other cases, you have to make other issues and work on it. Feel free to make issues on the website repository and contribute to it.

[akshatnema]

If you don't mind, can you please suggest, what would be best here.

@Devanshu-17 Sorry, we can't suggest any specific testing libraries as then it can be a case that we are referring to one specific library only rather than researching and finding out the best one. So, I suggest everyone to first go through all the libraries and find the best one out of them 😄

[reachaadrika]

@akshatnema Thank you , I will keep that in mind !

[AhmadSalikFazely]

Hello, my name is Ahmad Salik Fazely and i am interested in contributing to the project that involves creating automation tests for UI and scripts, and refactoring the code to make it more testable and maintainable. I have experience in software testing methodologies and techniques, defect tracking and resolution test planning and execution, test automation using tools and frameworks such as Selenium and Cucumber, and knowledge of programming languages like Java.
I find this project interesting because it addresses the need for automation tests to run on the PR level which checks the overall build and codebase of the changes made in the PR. I am excited to contribute to the project by adding UI tests and refactoring parts of the code to effectively create tests.
I am a hardworking and dedicated individual who is eager to learn and contribute to the project. I am open to feedback and suggestions, and will strive to meet the project requirements and goals. I am also available to communicate frequently and participate in regular meetings to ensure the success of the project.
I am open to learning new technologies and tools that may be required for the project, and will work on improving my skills in software testing and test automation. I am also interested in learning more about agile development methodology and how it can be implemented in the project.
I am flexible with work preferences and will adapt to the team's preferred tools and workflow. I usually use tools such as JIRA, Git, and Jenkins for project management and version control. I also prioritize keeping myself organized by creating clear goals and schedules, and regularly reviewing and adjusting my progress.
I am open to discussing and finalizing the schedule with the team based on the project requirements and timelines. However, some possible milestones and deliverables could include completing an initial assessment of the codebase and identifying areas for refactoring and adding UI tests, creating a test automation framework, and executing and reviewing test cases on a regular basis.

By the way I had two questions.(I already read all cooments but i couldn't find my answer)
Can you give me an overview of the existing test suite and the specific areas that need additional testing coverage?
Are there any particular UI testing tools or frameworks that the team has already considered or prefers to use?

[akshatnema]

Can you give me an overview of the existing test suite and the specific areas that need additional testing coverage?

There is no test suite present in the website so this issue target to make the tests in the website from scratch.

Are there any particular UI testing tools or frameworks that the team has already considered or prefers to use?

No, we don't have any special preference for the UI testing tools or frameworks. The contributors has to decide what they will use in their plan.

[reachaadrika]

Where should I submit the draft proposal for review ? Should I share the link on your personal chats on slack ?
@akshatnema @imabp

[akshatnema]

Where should I submit the draft proposal for review ? Should I share the link on your personal chats on slack ?

Yeah, you can DM any one of us on Slack.

[akshatnema]

Hey @vasucp1207 @reachaadrika @vaibhav-1508 @Harish-b-03 @Devanshu-17 @AhmadSalikFazely, kindly register for the program and submit your proposals for the idea if you haven't done it till now.

[akshatnema]

You can get reviews of your proposal too before submitting it on the GSoC website. You can either DM your proposal to me or @imabp on Slack for the review or can post it on any channel os AsyncAPI Slack for the reviews.

[reachaadrika]

@akshatnema @imabp shared a draft with you guys , let me know the suggestions , if any : )

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity 😴

It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.

There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.

Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.

Thank you for your patience ❤️

[akshatnema]

This issue is on track with GSoC PRs.

[robinbisht20]

provide a simple code structure for UI testing using Puppeteer. This example checks if a list of elements with specific IDs is present on a web page.
First, install Puppeteer:

npm install puppeteer

Now, write the test script:

```javascript
const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();

  // Navigate to the URL you want to test
  await page.goto('https://example.com');

  // List of element IDs to check
  const elementIds = ['element1', 'element2', 'element3'];

  // Check if each element with the given ID exists
  const results = await Promise.all(
    elementIds.map(async (id) => {
      const element = await page.$(`#${id}`);
      return {
        id,
        exists: !!element,
      };
    })
  );

  // Output the results
  console.log('Element existence results:');
  results.forEach((result) => {
    console.log(`${result.id}: ${result.exists ? 'Exists' : 'Not Found'}`);
  });

  await browser.close();
})();

[akshatnema]

@robinbisht20 This issue has been completed under GSoC 2023 by @reachaadrika. The issue is still open because the README PR has not been merged till now. So, kindly contribute to the existing issues related to tests.