Automated Usability Testing Tools

 Automated usability testing refers to the use of tools to evaluate a product or service, assessing the degree of difficulty with which a user in a given context can effectively, efficiently, and satisfactorily achieve a specific goal.

There are numerous automated tools available that provide a wide range of functions that can detect flaws or issues when it comes to usability. These tools are used in a variation of usability testing, which gives insight that can be beneficial for the development team in that they can learn how to:

1. Identify the usability issue.

2. Save time and resources.

3. Gain a competitive advantage.

4. Enhance the user experience.

5. Improve brand reputations.

 

To understand the important contributions of these automated usability tools, two tools, Google Lighthouse, and GTMetrix, would be used in an experiment to test the usability of three different websites (GRA, GTT, GPL) by using human testers of varying degrees of computing skill ranging from low to high.

Google Lighthouse and GTMetrix were chosen because they are freely available versions that provide essential insights, making them practical choices for developers on a budget.

These tools offer detailed reports on performance, accessibility, SEO, and best practices, allowing developers to address issues efficiently. By utilizing these tools, developers can ensure their websites are optimized for usability and overall user satisfaction. Finally, they are accessible to users with various web browsers, ensuring that developers can be consistent with their testing across different platforms.

 

Tests to be conducted:

·         Initiate Google Lighthouse and GTmetrix to analyze the websites' performance and identify improvement areas.

·         Using GTmetrix and Google Lighthouse to identify best practices on the websites.

·         Run Google Lighthouse and GTmetrix Check the SEO of the websites.

·         Run Google Lighthouse and GTmetrix to assess the accessibility of the websites.

 


Methodology:

  • Provide a brief overview of Google Lighthouse and GTmetrix and their features.
  • Ask participants to navigate to the designated websites.
  • Ask participants to run Google Lighthouse and GTmetrix on the selected website and note down their initial observations.
  • Ask participants to use Google Lighthouse and GTmetrix to analyze the websites based on the given tasks.
  • Conduct a brief interview with participants to gather feedback on their overall experience with Google Lighthouse and GTmetrix.
  • Explore their perceptions of the tool's usability, clarity of results, and any difficulties faced.
  • Ask participants to rate the ease of completing tasks using Google Lighthouse and GTmetrix.

 

Analysis:

It was observed that it was the easiest method of generating the necessary reports with the use of Google Lighthouse, with the preferred method being the use of the extension in the different browsers used. The number of participants in this experiment was 5, and they were selected at random based on their availability at the time of the experiment. Each of the participants was familiar with the use of a computer and a web browser, albeit to varying degrees of proficiency. However, this was not an issue, as each participant was mostly able to complete the designated tasks with the two usability tools. It was observed that the easiest method of completing the assigned tasks was with the use of Google Lighthouse, with the preferred method being the use of the extension tab in the web browser. With a few clicks of the mouse, each participant was able to complete the tasks. However, this proved somewhat difficult with the use of GTmetrix. Out of the 3 websites used in this experiment, one was unsuccessful in generating any reports due to an analysis error of unknown origin. Each participant experienced this error regardless of the web browser used, and at the end of the experiment, no one was able to rectify the issue. During this experiment, it was uncovered that each participant was able to generate the same reports using the same tool; however, when comparing the results of the two tools, it was observed that the results differed vastly in key areas. This, in turn, caused a question to be raised about the reliability and accuracy of the tools being used for data analysis. Further investigation is needed to determine the cause of these discrepancies and ensure the validity of future experiments. In all of the experiments, it was found that Google Lighthouse highlighted numerous errors in all three websites; this differed for GTmetrix, with it failing to generate a report due to an analysis error for one website and another with a passing grade. This discrepancy raises concerns about the consistency and effectiveness of these tools for website analysis. It is crucial to address these issues to make informed decisions based on accurate data.

In conclusion, by comparing the analyses of these two usability tools, the discrepancies uncovered valid concerns about the reliability of these tools for making informed decisions regarding web optimization. As such, it is essential to conduct additional research to enhance the reliability and consistency of automated web analysis tools, Only then can organizations and individuals make informed decisions based on accurate and trustworthy data to effectively implement web optimization strategies.

 

Google Lighthouse analysis


 





 

 GTmetrix Analysis




 



 





 



References


Administrator, C. (2023, August 24). Automating usability testing in a CI/CD process. Quality Engineering across entire SDLC | ShiftSync Community. https://shiftsync.tricentis.com/testing-strategies-methodologies-42/automating-usability-testing-in-a-ci-cd-process-486

Dumas J.S, Redish J. (n.d.). A Practical Guide to Usability Testing. Jed Brubaker – Associate Professor of Information Science. https://www.jedbrubaker.com/wp-content/uploads/2013/03/Dumas-99.pdf

Fiora T. W. Au, Simon Baker, Ian Warren, Gillian Dobbie. (n.d.). Automated Usability Testing Framework. https://crpit.scem.westernsydney.edu.au/confpapers/CRPITV76Au.pdf. https://crpit.scem.westernsydney.edu.au/confpapers/CRPITV76Au.pdf

Sparkbox. (n.d.). Google lighthouse review | Automated accessibility testing tools. Sparkbox - Web Design & Development Partner and IT Consultants. https://sparkbox.com/foundry/lighthouse_chrome_website_accessibility_audit_website_accessibility_checker

Usability testing 101. (n.d.). Nielsen Norman Group. https://www.nngroup.com/articles/usability-testing-101/

Usability: An introduction to and literature review of usability testing for educational resources in radiation oncology. (n.d.). PubMed Central (PMC). https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9574480/

 

Comments