Automated Usability Testing Tools
Automated usability testing refers to the use of tools to evaluate a product or service, assessing the degree of difficulty with which a user in a given context can effectively, efficiently, and satisfactorily achieve a specific goal.
There are numerous automated tools available that
provide a wide range of functions that can detect flaws or issues when it comes
to usability. These tools are used in a variation of usability testing, which
gives insight that can be beneficial for the development team in that they can
learn how to:
1. Identify the usability issue.
2. Save time and resources.
3. Gain a competitive advantage.
4. Enhance the user experience.
5. Improve brand reputations.
To understand the important contributions of these
automated usability tools, two tools, Google Lighthouse, and GTMetrix, would be
used in an experiment to test the usability of three different websites (GRA,
GTT, GPL) by using human testers of varying degrees of computing skill ranging
from low to high.
Google Lighthouse and GTMetrix were chosen because
they are freely available versions that provide essential insights, making them
practical choices for developers on a budget.
These tools offer detailed reports on performance,
accessibility, SEO, and best practices, allowing developers to address issues
efficiently. By utilizing these tools, developers can ensure their websites are
optimized for usability and overall user satisfaction. Finally, they are
accessible to users with various web browsers, ensuring that developers can be
consistent with their testing across different platforms.
Tests to be conducted:
·
Initiate
Google Lighthouse and GTmetrix to analyze the websites' performance and identify improvement areas.
·
Using GTmetrix and Google Lighthouse to
identify best practices on the websites.
·
Run Google Lighthouse and GTmetrix Check
the SEO of the websites.
·
Run Google Lighthouse and GTmetrix to
assess the accessibility of the websites.
Methodology:
- Provide
a brief overview of Google Lighthouse and GTmetrix and their features.
- Ask
participants to navigate to the designated websites.
- Ask
participants to run Google Lighthouse and GTmetrix on the selected website
and note down their initial observations.
- Ask
participants to use Google Lighthouse and GTmetrix to analyze the websites
based on the given tasks.
- Conduct
a brief interview with participants to gather feedback on their overall
experience with Google Lighthouse and GTmetrix.
- Explore
their perceptions of the tool's usability, clarity of results, and any
difficulties faced.
- Ask
participants to rate the ease of completing tasks using Google Lighthouse
and GTmetrix.
Analysis:
It was observed that it was the easiest method of
generating the necessary reports with the use of Google Lighthouse, with the
preferred method being the use of the extension in the different browsers used.
The
number of participants in this experiment was 5, and they were selected at
random based on their availability at the time of the experiment. Each of the
participants was familiar with the use of a computer and a web browser, albeit
to varying degrees of proficiency. However, this was not an issue, as each
participant was mostly able to complete the designated tasks with the two
usability tools. It was observed that the easiest method of completing the
assigned tasks was with the use of Google Lighthouse, with the preferred method
being the use of the extension tab in the web browser. With a few clicks of the
mouse, each participant was able to complete the tasks. However, this proved
somewhat difficult with the use of GTmetrix. Out of the 3 websites used in this
experiment, one was unsuccessful in generating any reports due to an analysis
error of unknown origin. Each participant experienced this error regardless of
the web browser used, and at the end of the experiment, no one was able to
rectify the issue. During this experiment, it was uncovered that each
participant was able to generate the same reports using the same tool; however,
when comparing the results of the two tools, it was observed that the results
differed vastly in key areas. This, in turn, caused a question to be raised
about the reliability and accuracy of the tools being used for data analysis.
Further investigation is needed to determine the cause of these discrepancies
and ensure the validity of future experiments. In all of the experiments, it
was found that Google Lighthouse highlighted numerous errors in all three
websites; this differed for GTmetrix, with it failing to generate a report due
to an analysis error for one website and another with a passing grade. This
discrepancy raises concerns about the consistency and effectiveness of these
tools for website analysis. It is crucial to address these issues to
make informed decisions based on accurate data.
In conclusion, by comparing the analyses of these two
usability tools, the discrepancies uncovered valid concerns about the
reliability of these tools for making informed decisions regarding web
optimization. As such, it is essential to conduct additional research to enhance the reliability and consistency of automated web analysis
tools, Only then can organizations and individuals make informed decisions
based on accurate and trustworthy data to effectively implement web
optimization strategies.
Google Lighthouse analysis
GTmetrix
Analysis
References
Administrator, C. (2023, August 24). Automating
usability testing in a CI/CD process. Quality Engineering across entire
SDLC | ShiftSync Community. https://shiftsync.tricentis.com/testing-strategies-methodologies-42/automating-usability-testing-in-a-ci-cd-process-486
Dumas J.S, Redish J. (n.d.). A
Practical Guide to Usability Testing. Jed Brubaker – Associate Professor
of Information Science. https://www.jedbrubaker.com/wp-content/uploads/2013/03/Dumas-99.pdf
Fiora T. W. Au, Simon Baker, Ian Warren,
Gillian Dobbie. (n.d.). Automated Usability Testing Framework.
https://crpit.scem.westernsydney.edu.au/confpapers/CRPITV76Au.pdf. https://crpit.scem.westernsydney.edu.au/confpapers/CRPITV76Au.pdf
Sparkbox. (n.d.). Google lighthouse
review | Automated accessibility testing tools. Sparkbox - Web Design
& Development Partner and IT Consultants. https://sparkbox.com/foundry/lighthouse_chrome_website_accessibility_audit_website_accessibility_checker
Usability testing 101. (n.d.).
Nielsen Norman Group. https://www.nngroup.com/articles/usability-testing-101/
Usability: An introduction to and
literature review of usability testing for educational resources in radiation
oncology. (n.d.). PubMed Central (PMC). https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9574480/
Comments