Publishers of technology books, eBooks, and videos for creative people

Home > Articles > Web Design & Development > Usability

  • Print
  • + Share This
This chapter is from the book

This chapter is from the book

Creating a QA Plan

You have known since the beginning of your redesign project that you would need to QA your site and that you would need a plan for it. Chances are, however, the extent of your QA plan is a budgetary/scheduling line that looks something like this: QA = 12 hours. Or 5 hours. Or 20+ hours. That budgetary line depends on the scope of your project, client expectations, and the expertise of your team.

Reassess your QA plan. Keep in mind that complicated frame sets, intricate HTML templates, light scripting, and links all need to be QA tested. There are essentially three levels of QA: light/informal, semiformal, and formal. Make the decision as to the level of QA your project requires.

Basic/Standard and Adv. Testing Procedures Table

Quality assurance testing can employ several procedures, most of which typically are used both in software development and for testing websites and web applications. In all testing situations, the extent of testing varies widely depending on technical complexity and the detail of the test plan.

A core plan for running quality assurance shows resources, time allotted, the extent of QA expectations, who is involved, criteria for acceptance, and what the development team and the client are each responsible for prior to site launch. Running QA should involve, at the very least, two complete run-throughs: first to generate a comprehensive bug list and second to go back over that bug list and make certain that the cited bugs have been fixed. For informal QA, this basic plan should suffice. For semiformal and formal plans, this core plan is expanded on accordingly.

Test Usability During QA

QA testing and usability testing are similar in approach and scope but different in expertise and goal. At times, however, the two overlap, especially when technical errors and complications (checked for during QA) affect a user's ability to move successfully through a site (checked for through usability testing). In fact, usability testing can sometimes be considered a type of QA.

While you are QA testing your site for errors, technical glitches, and cross-browser compatibility, we strongly suggest you also conduct one-on-one usability testing (also called "verification testing" at this stage). Why? To ensure that your site works from the user's point of view. Naming and labeling must be clear. Navigation must be intuitive and easy to follow. Your site might be clean and free from bugs, but if it isn't easy to use, the chances are it won't get used and will fail.

Conversely, the redesign might be easy to use (congratulations!), but if you have broken links and spelling errors, users won't get very far. Moreover, they will have a poor impression of the site and the company. Make a bug-free and user-friendly site your prelaunch goal. We recommend both QA and usability testing prior to launch. For more on usability testing, see Chapter 8.

Every test plan or testing situation will contain different criteria for acceptance. Each site will need to check functionality against requirements and across browsers, platforms, and operating systems, from simple pop-up windows and submission of forms to complex login procedures and e-commerce ordering systems. As the web continues to evolve from basic HTML to a functional, application- driven environment, more and more attention needs to be allocated to ensuring integration success.

QA & Servers

Prior to a site going live, the production team should test on both the staging/development server and then again when the site is moved over to the actual server environment where eventually it will be live. When the site is moved over, the testing environment needs to be exactly the same as the live environment. This means that the folders, file structure, and server-side scripts must be correctly in place; otherwise, many of the scripts and CGI elements may not work properly.

The Problem With Frames

If your site contains frames, expect QA to take at least twice as long. Nested frames? Even longer. As a rule, the more frames you have, the more QA is needed. Moreover, frames thwart search engines (see Phase 5: Launch and Beyond). Frames, while appropriate and good for some situations (for example, portfolios, maintaining several levels of navigation, and so on), are so problematic that most often they are simply not worth it. We recommend no frames unless absolutely necessary.

Informal testing is very basic and is doable by the development team. Formal usually entails hiring an outside, trained team. Semiformal is, logically, in between the two. Most sites with a development budget under $30,000 can usually get away with informal testing. Sites with complex functionality and an application layer normally include formal or at least semiformal testing in their workflow.

Include the Client

For informal testing, clients should also participate in the QA process in the same fashion as the team members: checking the site and submitting a sheaf of printouts with errors clearly indicated as well as browser and platform types noted. For any level of testing, the client should proof the content. Only the client will be able to truly know if content is in the wrong place or is incorrect. The client should be treated as (and should hopefully act as) a partner and not a finger pointer.

Light/Informal QA

For informal QA processes, the QA lead or the project manager coordinates and tracks all planned tests and assigns team members to sections of the site, individual browsers, browser versions, and platforms. The assigned team member then goes through the site and compiles and lists all bugs for the HTML production team to fix. An easy way of doing this involves printing out pages that have errors and clearly indicating each error on the printout. Note that these printouts are only complete and helpful if the browser and platform is noted on the printout that notes the bug. Without knowing the browser and platform, it is difficult to re-create the error and therefore fix it.

A Core QA Plan

  • Summary of overall goals for QA including methodology, schedule, and resource allocation.

  • List of specific browsers, platforms, and operating systems being tested.

  • List of desired connection speeds being tested.

  • List of any specific paths or functions that need to be tested.

  • A plan for bug tracking (using a web-based program or Excel spreadsheet or printouts).

  • A plan for confirming that fixes have been made prior to launch.

  • Any stated assumptions (known risks) to protect the team if all fixes cannot be caught prior to launch. These should be listed in the Details and Assumptions section (in Phase 1) of the project plan or contract and be signed off on prior to the final site being delivered or launched.

  • A plan for fixing bugs that cannot be resolved prior to launch. Who is to handle them, how will any additional costs be identified, and so on.

The project manager also tracks the "bug list," which, in informal testing, is really no more than a stack of printouts with bugs noted. A big red checkmark through the noted bug indicates that it has been addressed, and an accompanying initial indicating "Fixed" or "Deferred" with a date helps track the fixes.

Usually, for small- to medium-size sites (under $30,000 budgets) with very little technical complexity, this informal process is a perfectly adequate method. Informal testing is also referred to as "ad hoc" or "guerilla testing" in that it has no formal test plan or approach. Testers are just "banging" on the site, looking for bugs to slay.

Test Beds

The bank of computers (set up in the testing area) that reflects the target browsers, platforms, and connection speeds of the audience is often called a "test bed." It is difficult to list every combination of browser and platform; at least use the main ones [6.14]. Even testing a smaller, representative group will result in catching many errors on the site. Test beds are common for semiformal and formal QA. Often for informal testing, the various browsers and platforms are not in the same location.

Figure 6.14. A chart like this one will help track all of the platform/browser configurations of the target audience. It can reflect the test bed setup. This sample audience does not include users on 3.0 browsers or UNIX platforms.

Semiformal QA

If your project requires more than "guerilla testing," yet your budget will not accommodate formal testing with an outside company, the perfect middle ground is semiformal testing. Stepping up from informal to semiformal testing involves more time, expertise, and planning — and if possible, the addition of a trained QA lead and a test bed setup. A semiformal test plan should contain a one- to two-page overview that highlights the scope, timing, and goals of the QA testing process.

Formal QA

Planning for formal QA testing requires experience, time, budget, and most of all, attention to detail — minute detail. The biggest difference between semiformal and formal QA is the level of test planning, the cost, the generation of documentation, and the degree of expertise.

Bug Tracking Tools

Although you cannot substitute automated software systems for actual QA testing with humans, there are many available tools that can aid in the process. For complete HTML validator testing, links, spelling, load time, and more, try http://www.netmechanic.com. Fees range from $35 to $200 for testing up to 400 HTML pages.

Other online tools? They are plentiful. Try http://www.scrubtheweb.com to help check your META information. http://www.w3.org/People/Raggett/tidy will help you clean up your HTML. For an excellent bug-tracking tool, visit http://www.alumni.caltech.edu/~dank/gnats.html. Want to learn more about bugs? Go to http://www.mozilla.org/bugs. Mozilla itself is handy for QA as well.

Formal QA uses a comprehensive bug-tracking system and a fully trained QA staff (yes, staff) to test requirements and pages against specified browsers and platforms. It includes test plans, tools, use cases, a test bed, and reports. To illustrate the extensiveness of the formal testing process, consider this example of a typical formal QA plan: Identify at least 10 different paths through a site and test each path on three platforms (MAC, WIN, UNIX), with each platform hosting three browsers (IE, Netscape, AOL), with each browser having several versions (3.0 through 6.0, note that Netscape skipped 5.0) — all needing to be tested. This example now has approximately 450 different tests (10 x3 x3 x5) for the defined paths. Overwhelming? Yes. Impossible? No. Impossible in an informal setting? Yes. Recommended for large sites with a significant backend engineering and extensive functionality? Absolutely.

Bug Reporting

Reporting a bug is easy. Reporting bugs in a way that is meaningful, reproducible, detailed, and solution oriented is a challenge. Here's the old, serviceable, good-for-informal-testing way: Print the page out, note the browser/platform, circle the error, fix the bug, and then check the error as fixed (or deferred if the bug can't be fixed yet). Here's another (and maybe better) way: Use some kind of tracking device — even an Excel spreadsheet will suffice — although you can only have one person working with the file at a time. Whatever your tracking method, make certain to note the following information:

  • Browser type/platform type.
  • Operating system.
  • Description of problem (one line).
  • Detailed description.
  • URL of page.
  • Severity of problem.
  • Can the error be reproduced?
  • + Share This
  • 🔖 Save To Your Account

Peachpit Promotional Mailings & Special Offers

I would like to receive exclusive offers and hear about products from Peachpit and its family of brands. I can unsubscribe at any time.

Overview


Pearson Education, Inc., 221 River Street, Hoboken, New Jersey 07030, (Pearson) presents this site to provide information about Peachpit products and services that can be purchased through this site.

This privacy notice provides an overview of our commitment to privacy and describes how we collect, protect, use and share personal information collected through this site. Please note that other Pearson websites and online products and services have their own separate privacy policies.

Collection and Use of Information


To conduct business and deliver products and services, Pearson collects and uses personal information in several ways in connection with this site, including:

Questions and Inquiries

For inquiries and questions, we collect the inquiry or question, together with name, contact details (email address, phone number and mailing address) and any other additional information voluntarily submitted to us through a Contact Us form or an email. We use this information to address the inquiry and respond to the question.

Online Store

For orders and purchases placed through our online store on this site, we collect order details, name, institution name and address (if applicable), email address, phone number, shipping and billing addresses, credit/debit card information, shipping options and any instructions. We use this information to complete transactions, fulfill orders, communicate with individuals placing orders or visiting the online store, and for related purposes.

Surveys

Pearson may offer opportunities to provide feedback or participate in surveys, including surveys evaluating Pearson products, services or sites. Participation is voluntary. Pearson collects information requested in the survey questions and uses the information to evaluate, support, maintain and improve products, services or sites; develop new products and services; conduct educational research; and for other purposes specified in the survey.

Contests and Drawings

Occasionally, we may sponsor a contest or drawing. Participation is optional. Pearson collects name, contact information and other information specified on the entry form for the contest or drawing to conduct the contest or drawing. Pearson may collect additional personal information from the winners of a contest or drawing in order to award the prize and for tax reporting purposes, as required by law.

Newsletters

If you have elected to receive email newsletters or promotional mailings and special offers but want to unsubscribe, simply email ask@peachpit.com.

Service Announcements

On rare occasions it is necessary to send out a strictly service related announcement. For instance, if our service is temporarily suspended for maintenance we might send users an email. Generally, users may not opt-out of these communications, though they can deactivate their account information. However, these communications are not promotional in nature.

Customer Service

We communicate with users on a regular basis to provide requested services and in regard to issues relating to their account we reply via email or phone in accordance with the users' wishes when a user submits their information through our Contact Us form.

Other Collection and Use of Information


Application and System Logs

Pearson automatically collects log data to help ensure the delivery, availability and security of this site. Log data may include technical information about how a user or visitor connected to this site, such as browser type, type of computer/device, operating system, internet service provider and IP address. We use this information for support purposes and to monitor the health of the site, identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents and appropriately scale computing resources.

Web Analytics

Pearson may use third party web trend analytical services, including Google Analytics, to collect visitor information, such as IP addresses, browser types, referring pages, pages visited and time spent on a particular site. While these analytical services collect and report information on an anonymous basis, they may use cookies to gather web trend information. The information gathered may enable Pearson (but not the third party web trend services) to link information with application and system log data. Pearson uses this information for system administration and to identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents, appropriately scale computing resources and otherwise support and deliver this site and its services.

Cookies and Related Technologies

This site uses cookies and similar technologies to personalize content, measure traffic patterns, control security, track use and access of information on this site, and provide interest-based messages and advertising. Users can manage and block the use of cookies through their browser. Disabling or blocking certain cookies may limit the functionality of this site.

Do Not Track

This site currently does not respond to Do Not Track signals.

Security


Pearson uses appropriate physical, administrative and technical security measures to protect personal information from unauthorized access, use and disclosure.

Children


This site is not directed to children under the age of 13.

Marketing


Pearson may send or direct marketing communications to users, provided that

  • Pearson will not use personal information collected or processed as a K-12 school service provider for the purpose of directed or targeted advertising.
  • Such marketing is consistent with applicable law and Pearson's legal obligations.
  • Pearson will not knowingly direct or send marketing communications to an individual who has expressed a preference not to receive marketing.
  • Where required by applicable law, express or implied consent to marketing exists and has not been withdrawn.

Pearson may provide personal information to a third party service provider on a restricted basis to provide marketing solely on behalf of Pearson or an affiliate or customer for whom Pearson is a service provider. Marketing preferences may be changed at any time.

Correcting/Updating Personal Information


If a user's personally identifiable information changes (such as your postal address or email address), we provide a way to correct or update that user's personal data provided to us. This can be done on the Account page. If a user no longer desires our service and desires to delete his or her account, please contact us at customer-service@informit.com and we will process the deletion of a user's account.

Choice/Opt-out


Users can always make an informed choice as to whether they should proceed with certain services offered by Adobe Press. If you choose to remove yourself from our mailing list(s) simply visit the following page and uncheck any communication you no longer want to receive: www.peachpit.com/u.aspx.

Sale of Personal Information


Pearson does not rent or sell personal information in exchange for any payment of money.

While Pearson does not sell personal information, as defined in Nevada law, Nevada residents may email a request for no sale of their personal information to NevadaDesignatedRequest@pearson.com.

Supplemental Privacy Statement for California Residents


California residents should read our Supplemental privacy statement for California residents in conjunction with this Privacy Notice. The Supplemental privacy statement for California residents explains Pearson's commitment to comply with California law and applies to personal information of California residents collected in connection with this site and the Services.

Sharing and Disclosure


Pearson may disclose personal information, as follows:

  • As required by law.
  • With the consent of the individual (or their parent, if the individual is a minor)
  • In response to a subpoena, court order or legal process, to the extent permitted or required by law
  • To protect the security and safety of individuals, data, assets and systems, consistent with applicable law
  • In connection the sale, joint venture or other transfer of some or all of its company or assets, subject to the provisions of this Privacy Notice
  • To investigate or address actual or suspected fraud or other illegal activities
  • To exercise its legal rights, including enforcement of the Terms of Use for this site or another contract
  • To affiliated Pearson companies and other companies and organizations who perform work for Pearson and are obligated to protect the privacy of personal information consistent with this Privacy Notice
  • To a school, organization, company or government agency, where Pearson collects or processes the personal information in a school setting or on behalf of such organization, company or government agency.

Links


This web site contains links to other sites. Please be aware that we are not responsible for the privacy practices of such other sites. We encourage our users to be aware when they leave our site and to read the privacy statements of each and every web site that collects Personal Information. This privacy statement applies solely to information collected by this web site.

Requests and Contact


Please contact us about this Privacy Notice or if you have any requests or questions relating to the privacy of your personal information.

Changes to this Privacy Notice


We may revise this Privacy Notice through an updated posting. We will identify the effective date of the revision in the posting. Often, updates are made to provide greater clarity or to comply with changes in regulatory requirements. If the updates involve material changes to the collection, protection, use or disclosure of Personal Information, Pearson will provide notice of the change through a conspicuous notice on this site or other appropriate way. Continued use of the site after the effective date of a posted revision evidences acceptance. Please contact us if you have questions or concerns about the Privacy Notice or any objection to any revisions.

Last Update: November 17, 2020