import test blog

Most people now realize that usability testing is not only an effective and accurate way of answering performance questions, it can also answer questions regarding the way end users interface with the website or software under test. This is vitally important as the way in which users interface with software often determines the marketability of that software and poor website design can result in lower sales and damage to the reputation of a company. Usability testing offers the opportunity to discover how well a website or application achieves its goals. It can provide insight into such things as navigation, business processes and user behavior, as well as answer a large number of other questions.

There are many ways to find the answers you need, it all depends on how you ask the questions, and different methods can reveal different information. Here are a few of the usability testing methods and the kind of information they reveal, along with their advantages and disadvantages.

1. Direct Observation

Sometimes referred to as Contextual Analysis or Contextual Inquiry, this method consists of directly observing the user in the actual environment in which the user operates, in other words, "in context." The user is directly observed using the website or software in question. Observation is then followed up by interviewing the user in a one on one setting in order to answer questions the researcher may have.

Purpose: The purpose of this method is to clarify how the user interacts with the website or software to make certain that the item under test matches user requirements.

Advantage: This method can give the researcher insight into how the user pursues his or her goals through interaction with the website or application, as well as indicate any area where the item under test may not be performing well in a real world environment.

Disadvantage: This method is particularly susceptible to interference from the researcher. The temptation may be to try to guide the user in an attempt to "improve" the user's workflow. This should be avoided. Direct observation should be a generally passive process.

2. Card Sorting

This is a useful psychological tool that is best administered during the development stage. Its purpose is to determine how users categorize the information that will be present in the application or on the website. Users are given a set of cards with relevant information on them and asked to sort the cards in a way that makes sense to them. A variation of this test is where users are given categories and asked to sort the cards by category.

Purpose: To match user requirements as closely as possible during the development cycle.

Advantage: This is very good for developing information architecture for a website or application.

Disadvantage: The results are limited to what those who develop the test think is important, so things can get missed.

3. Usability Audit

This is the evaluation of user interface based on the rules developed by Jakob Nielsen in the 1990's and which have been added to and modified since. They include asking the following questions, all of which should be answered with " yes."

Does the system keep the user informed regarding system operations?

Does the system present information in a natural, understandable and logical order?

Does the system follow established platform conventions?

Does the system provide the user with a way of correcting mistakes that does not require extended dialogue or interaction?  In other words, does it support undo and redo?

Is the system designed to minimize user error?

Is the system flexible and efficient?

Is the system designed to present information in an easy to understand way?

Are error messages clear and easy to understand?  Do they present the problem accurately and offer possible solutions?

Does the system lighten the load on the user by not requiring heavily detailed actions?

Are system instructions easily visible and easy to retrieve when needed?

Is the user required to do more actions than should be logically necessary to accomplish a task?

Are help and documentation functions available when needed?

Purpose: To create an interface that enables users to easily understand their options and intuitively know how to work with the system to achieve their goals. Usability audit ensures that the system serves the user and not the other way around.

All of the above questions are based on certain interface design principles and there is really no disadvantage or downside to following established principles, since they are fundamental to all websites and applications.  They have been hard learned over a period of years.  One need only go back to the overblown websites of the 1990s to understand how far interface design has come.  The gaudy animation packed websites of that era often set the record for confusion and generally poor usability.

These days, designers are dedicated to bringing users all the tools and data needed for every step of a given process, and to do so in a way that's easy to understand and use.  This is why designs are user tested for functionality even after aesthetic changes.  Impressing the user with pizzazz is no longer important.  What matters is helping the user to make his or her own decisions and get the job done.

The modern website or application is designed to perform the most work possible while demanding the least information necessary. This can get tricky, as applications can sometimes be too helpful, slowing the user down by rendering assistance by performing a function that the user didn't ask for or want. The user is then forced to deal with the results of the unwanted assistance before returning to task.

Ultimately, good interface is a matter of balance. Failing to anticipate the user's needs can also have considerable repercussions for websites and applications.  You should never force a user to go hunting for information or for tools.  A quality interface will render any assistance possible and provide what the user will need to accomplish any task, without taking control away from the user.

The purpose of functional testing is to assure that the software fulfills the requirements of the stakeholders.  It is not user testing, but rather the testing of each action the program must take.  Test case scenarios are developed because simply testing one action at a time is not enough.  Instead, the program must be tested based on how it is intended to perform in the real world.

Functional testing is sometimes called black box testing because the tester isn't concerned with the actual code, but rather with the program specifications.  The tester is testing the program, rather than the code.

Functional testing scenarios should be developed as early as possible during the process of specifying requirements.  This enables functional testing of modules as they are delivered.

Functional testing has a wide application. It can be applied to different types of program behavior and can drill down to modular level or a expand outward to system testing.  It can validate that key customer journeys are functioning properly by uncovering any issues that might be hidden.  It can also validate that there are no issues after a major software release.

While setting up test data can be time consuming and functional testing can sometimes have a low bug detection rate, it does provide rapid feedback on the state of an application at almost any point in the development cycle.  It also allows for precise coverage of those test cases that are executed and provides guidance to testers who are operating further down the line.

Functional testing is extremely useful with regard to complex business applications and those applications that have had formal internal testing completed and now require regression executions.

Functional testing describes what the system does, as opposed to what the system should do.  It is in the difference between those two points that functional testing has its purpose.

Lean software development is a subset of agile development and is based on the Toyota production system.  Its purpose is to speed the development cycle by following several principles.  The first two are, eliminating waste, which is self explanatory, and refining learning, which basically means learn as you go and don’t assume you know everything about the development cycle you are working on.

Another two principles are, making decisions as late as possible while delivering as fast as possible, along with conceptual integrity, which means that all elements of the application must be obviously a part of the same design. A coherent design makes it much easier to resolve unanticipated issues.  And finally, there is operating from the big picture down rather than working upward and outward from the details.  These principles all sound pretty good.  They increase development and delivery speed while making troubleshooting easier. Unfortunately, lean methodology, like any other method, has its advantages and disadvantages.

Some of the advantages include the elimination of superfluous activity, thereby saving time and money.  It also enables more functionality to be delivered in a shorter period of time and empowers the development team in the decision-making process thereby improving motivation to do the best job possible.

Lean methodology is also easily scalable, which makes it a good alternative to more conventional software development methods that were primarily designed for large projects.  lean also fits in well with agile and devops because it is applicable across team boundaries and tends to integrate teams and optimize cooperation.

However, lean methodology isn’t perfect.  It is heavily team dependent, which means that you'd better have assembled an excellent team with a high skill level.  Since the overall development team has so much responsibility spread over a number of smaller subteams, it can be relatively easy to lose focus.  Also, lean development requires excellent documentation, especially with regard to business requirements.  This means that any area that is poorly documented can be underdeveloped or developed incorrectly.

Nevertheless, the advantages of lean software development generally outweigh the disadvantages. This is especially true when it comes to upgrades and additions. So, make sure you have an excellent team at your disposal and let them go lean.

Today the costs of testing is rising and so is the need for better quality products. Many of the customers I spoke with were frustrated by the rising cost of testing. Best practices in an organisation is one tester for every three engineers. However, many customers have seen this rise to 1:1 ratio. Frustration also exists due to the difficulty in measuring the return of software testing, making their job even more difficult to justify budgets.

At Bugwolf we are not about putting bums on seats, but about delivering real and measurable return on investment. We’ve built a patent pending model which allows customers to measure every engagement with us and and provide much better visibility both in terms of value and cost savings delivered. When we start a testing cycle with a client, we collectively agree on what it costs them to remediate defects in production. At the end of every cycle, and after we have spent some time analysing and curating the results, we then apply a value to the defects we find.  

And this means, customers get a clear way of measuring the spend they make with us. This also helps them at each budget review when management are looking for ways to reduce costs and move budgets from software quality to other departments.

DevOps simply means the cross department integration between Development, the department creating the code, and Operations, the department using that code. Of course, it’s a little more complicated than that. There are a number of stakeholders along the way. The idea began in 2008 with a discussion between Patrick Debois and Andrew Clay Shafer concerning the concept of agile infrastructure. However, the idea only started to spread in 2009 with the advent of the first DevOpsDays event held in Belgium. What began as a desire to bring more efficiency to software development has evolved into a feedback oriented system designed to modify every aspect of software development from coding all the way through the various stakeholders to deployment.

DevOps is lean thinking blended with agile philosophy. The revolutionary aspect of DevOps is that it crosses a traditional line by merging software development with the environment in which it is developed. This is more than just an attempt at efficiency, it is a cultural change. This cultural change is then made possible by a series of tools that automate such processes as code development and review through continuous integration, while allowing for version control. In fact, it is the automation that makes the agile development cycle possible by merging different functions from software testing to deployment.

Actually, the birth of DevOps isn’t all that surprising. IT has a history of becoming more and more relevant to more and more people all the time. What was once walled off in it’s own department within corporations and universities, has now spread across the world and into everyone’s pocket. DevOps takes the next logical step in this evolution by unifying, or at least linking, development and operations to create a spectrum of activity in which everyone can participate.

The key to successful software development begins with well defined requirements. Changing requirements are a very common cause of problems that can create buggy software which winds up in the hands of end users. While it's impossible to never make changes, the need to alter requirements usually arises from poor product definition in the first place. The more accurate the original requirements, the less need there will be for change later on. There are many good ways to develop accurate requirements, such as prototyping, usability studies or running workshops or development camps where stakeholders get together to help define the product.

Good project planning must be followed up by good project management. Goals should be precisely and clearly set. It also helps to have a clear vision statement that sets out priorities and exactly what the product is intended to do. Start with a feasibility study before budgeting the whole project. This prevents overshooting or undershooting the mark and developing a product no one wants or has unnecessary functions that drive up cost, development and testing time.

Management must also go beyond the technical to encompass other areas of expertise, such as project tracking, risk assessment and accounting. Data collection from software testers, UX testing and the other areas involved in the project must be carefully collated and understood. Corporate should not pressure project managers with unrealistic expectations. Managers should also make sure that each link in the development chain meets its goals and that no problems are passed down the chain. Each link must be responsible for its own area.

Expectations should be realistic. There will be different and sometimes conflicting requirements when multiple stakeholders are involved. So, there should be a pragmatic consensus on what release 1.0 will do and what will be added to later releases. Otherwise, you run the risk of trying to create an application that's supposed to sing, dance, play the harp and do the laundry, all at the same time.

And finally, do whatever you can to ensure that software testing is as thorough as possible. Quality testing is your ultimate guarantee of a successful product.

Digital technology gives us the potential to be more informed and educated than any previous generation; if we take advantage of the tools at our disposal.

We can be more informed about government and we can engage government in an ongoing real time dialogue. And we are the first generation to have that capability. Government can be more informed about the needs and desires of its citizens and citizens can be more aware of government services.

Government can also be more easily informed about us. That isn't always a good thing. The same tools that give us the ability to convey and collect information, also give government the same ability. The domestic spying, secret wiretapping and other surveillance that has come to light over the last few years has demonstrated that governments are not above spying on their own citizens for debatable reasons.

Big data can be used to profile people on everything from religion to sexual preference and more ominously "political reliability", whatever that means. It all depends on what the agenda of the people doing the profiling is.

We humans value our privacy as well as our lives. We expect government to protect us, but to stay within rational guidelines. We like how big data enhances our ability to search on the Internet, but we would not want companies to use that data to manipulate us. Ensuring that no power, public or private, goes too far is an important part of what it means to be civilized.

However IT, like all material advances, is without a conscience. How it is used depends on the people using it. The pessimist will say that digital technology is a new means of slavery and the optimist will say that it is the key to building a better and more enlightened civilization. Actually, it's both. It all depends on what we demand and what we will tolerate. As always, the quality of our future depends less on the quality of our technology, than on the quality of our culture.  

The GUI was the first real breakthrough, it liberated the user’s mind. People no longer had to conform their thinking to the machine’s requirements. Yesterday, it was the limitations of the machine that determined the user's actions. Today, it is the user's psychology that determines the machine’s actions. Interface design has advanced by keeping human factors in mind, especially when doing usability testing. The Graphical User Interface was a massive increase in usability. Function went from something hidden behind the command line that required arcane runes to unlock, to a simple screen graphic.

Interface protocols are the reason why modern applications can perform quite well in the hands of people who have no understanding of the thousands of lines of code that initiate the functions they take for granted. For the majority of users, the interface is the program.

Successful interface design is based on a user centered approach that not only takes the user into account, such as education level and psychology, but also the work environment and even the culture in which the user operates. Modelling the interface to reflect what the user is already accustomed to, such as the famous trash can icon on the desktop, has become standard practice. It is extremely useful to use conceptual models that exist in the real world. This will render the application much easier to comprehend. People are a lot better at recognising things than recalling information, if there is a way to facilitate recognition, then use it.

Computers are an increasingly important part of society. There may well come a day when nothing will function without them. This means that usability will become even more important. And the ultimate interface may one day blur the line between human and machine.

Unfortunately, this phase often gets cut short, and not only for financial reasons. The end users are usually stretched to the limit just running their organisation as it is and can’t afford the time to test new software. This is why it’s important to have a dedicated team of professional testers in the UAT phase.

UAT is vital for the successful deployment of any application. Failure to do proper user acceptance testing can result in turning a business’s customers into user acceptance testers. This is a very fast way to create a PR problem that may take years to rectify and can ruin a company’s reputation, even if it is performing quite well outside the framework of that single malfunctioning application.

Failure to do proper user acceptance testing can also result in legal risk, missed deadlines, wasted resources and added cost. This is why the major purpose of UAT is to determine how the software will interface within a given company before that application is actually implemented. Properly done, user acceptance testing protects the organisation from a whole host of potential problems.

User acceptance testing is also important because failure to do it places an unnecessary burden on system developers who, while they may be experts in development, are not familiar with the vagaries of running an organisation or the day to day difficulties that the software they are developing must account for.

UAT is concerned with both software performance and human behaviour. It ensures that the software interfaces with human behaviour and thought processes in a realistic manner. This increases both employee and client satisfaction. Quality software builds confidence in the company that uses it, while it makes running the organisation easier. Smooth client interaction attracts more customers and the company’s reputation rises. The customers get better service and the company gets more customers. Ultimately, everyone wins.