import test blog

Fragmented Platforms

When I started Bugwolf I was frustrated by the quality of products which were being released. I also knew first hand how difficult and time consuming and frustrating it can be thoroughly test digital products. The market is fragmented especially when you’re talking the Android platform. The environment is becoming even more challenging with the obsession to connect everything to the internet and how the cloud accelerates everything including competition.

We originally went out to market focused on security testing as a crowdsourced model until one day a client asked if we could apply the same Bugwolf model to usability and functional testing. We also discovered that even though the concept of crowdsourcing was a positive one when engaging clients, many of the large enterprises found it difficult to get the concept past security due to concerns of having unknowns accessing their assets.

Adapt Quickly 

By working closely with customers and understanding these challenges early in the lifecycle of the company, we were able to change our service, product, and the way we communicate with clients. The more clients we met the more objections we were able to overcome until we got to a point where we no longer hear a no from customers. We also discovered we were solving so many other problems which we didn’t even know existed until engaging clients.

Once we got to product market fit our business has gone from strength to strength. We have secured a number of global brands who have become repeat customers. We have been able to deliver significant value to these customers. We are expanding our team to manage this growth and have made some significant hires in the last few months to support the success to come. We are now focused on scaling the business, and as always on our customers.

There was a period, from the 1970s until the early 2000s when their core architecture could at least keep pace. However, these legacy systems were becoming outdated by the mid 90s as IT changed the focus of the financial services industry. Things were becoming more customer centered and the need for flexible multi-channel processing and integration was increasing. Online banking was growing and so was the need to address big data and cloud based platforms.

Now, the concept of digital services operating through multiple channels has taken hold, which requires a massive change in IT architecture to create platforms that are scalable, adaptable and economically feasible.

The core banking platforms of the future will focus on agility in order to provide fast customer centered solutions. Banks must be flexible enough to adapt to changing market circumstances. This may mean that financial institutions will be forced to reduce overhead by outsourcing back office functions to contractors and concentrating on automation in order to compete with technology services companies like Google and online startups who either already have a technological edge or a low overhead, or both.  

Fortunately, service oriented architecture enables banks to mix existing functions into innovative products and services that can address changing customer demands, while analytics enables a leveraging of customer behavior and preferences through many different channels. The key to future development lies in being able to see the gaps between the desired results and the present situation.

And so the future of banking involves leveraging disruption as a driver of innovation, while minimizing risk through the accurate prediction of future scenarios, which is something that digital technology can also help with.

Problems with functional testing caused by testing in isolation

There is a tendency to run functional tests in isolation.  Operations such as printing and input or output are tested in isolation rather than as part of a series.  It is sometimes forgotten, or at least under emphasized, that various functions happen together as a sequence of steps.  Testing each element individually in isolation doesn't demonstrate that all elements can work smoothly together.

Inadequate configuration testing

The combination of the application and third party software is often under tested.  The main reason for this is that configuration testing can get expensive, as it requires the maintenance of different operating systems and applications needed as part of the testing process.  This can cause configuration testing to be underutilized in an effort to save money.

Dropping beta testing altogether

There have been a number of examples of attempting to reduce testing costs by turning customers into beta testers.  While this may be workable if customers know what they're getting into, it's not what should be done first.  It places too much emphasis on user experience when users are not professional testers.  The result is constant rework and patching that eventually turns users into regression testers.  This goes around and around, causing more and more trouble and expense, as well as bad public relations.  Ultimately it becomes more expensive than beta testing with trained testers. If you are going to bring in customers, bring them in properly by presenting them with functional software that needs as little patching as possible.

Beta testing using the wrong customer base

Just as beta testing can be underutilized, it is also possible to recruit the wrong customers.  While it's okay to invite customers into the beta testing phase, remember that these may not represent your target niche.  People who are eager to download new software don't usually represent the majority.  Most people would prefer to wait until the application has been proven.  While inviting beta testers is certainly useful, it should be remembered that the kind of people who download beta versions are much more tolerant of bugs. They may not even report minor glitches that the more pragmatic general public might consider annoying or even cause for rejecting the application entirely.

Not aiming for quality testers

Whether beta testing is done by customers or not, there is no substitute for trained UX testers. Testers should be able to understand possible problems and special cases, and should be experienced enough to spot subtle interactions that programmers might miss.  Even then, test designs should be reviewed by more than one person.  There is no substitute for a second set of eyes.

All of these issues can be overcome by maintaining quality communication between developers and testers.  Efficient testing depends on building an efficient working relationship between testers and the rest of the development team.  Ultimately, testers and developers need to work together if the application is to be successful. That requires clear and concise communication from the first documentation right through to the final user experience.

It is especially beneficial when dealing with complex applications.  While some applications are so simple that the user has only one or two routes to follow, others are complicated indeed.  The greater the number of actions available, the greater the number of possible outcomes and the greater the chance of failure.  In such a situation, writing acceptance tests for the many possible scenarios may be time consuming, but it does provide considerable confidence and contributes greatly to quality assurance.

User acceptance testing exists to provide assurance that a software product will meet the requirements of stakeholders and prospective users.  Its purpose is to measure how closely an application matches the planned design.  However, even in these days of agile testing, UAT can get pushed to the end of the development cycle where money is tight and the temptation is to cut back on expenses.

User acceptance testing can also point out bugs which have been missed in previous testing.  Too many bugs found means that previous testing wasn't sufficient or efficient.  The quality of testing depends a great deal on certain criteria, such as the quality of the testers, the available budget and the amount of time allotted for testing.  All of these can have beneficial or adverse effects on the final product.

Perhaps the single biggest way in which user acceptance and user experience testing saves money is in the combination of quality assurance and public relations.  It improves quality assurance by finalizing performance while eliminating the need for excessive regression tests.  And it improves public relations by making sure that buggy software doesn't end up in the hands of end users, creating a nightmare of patching that end users can then complain about on social media.  All in all, user acceptance testing is not only worth the money, it's worth spending a little extra if necessary.

Ad hoc testing can also be described as intuitive testing, the purpose of which is to find what has been overlooked and may cause unanticipated problems in the future.  Even so, ad hoc testing doesn't mean testing based solely on wild hunches.  The better the documentation at the beginning of a project the stronger the domain knowledge will be as the project develops and the more efficient ad hoc testing will become.

Software testing is a dynamic process, it involves both the experience and skill of the tester, a good knowledge of user requirements and an understanding of how users will interact with the application as well as the environment that application will operate in.

Despite the fact that testing time is usually short and can be shortened even more by delays in development, it is vitally important to leave at least some time at the end of the project for ad hoc testing.  Ad hoc testing may even be required in the field after deployment, depending on the circumstances.  In that case, it will probably be primarily deployed for the purpose of fine tuning compliance with field requirements.

The unstructured nature of ad hoc testing makes it a type of behavioral or black box testing.  Consequently such testing requires a considerable level of skill and an in depth understanding of the system under test.

However, ad hoc doesn't mean undisciplined.  Such testing is not random but always aligned with development objectives.  Its purpose is to find holes in the testing process where bugs may be hiding and to increase confidence in the final product.   

In the world of software testing, formal processes don't always yield the best results.  Also, user acceptance testing cannot be guaranteed to find bugs that were overlooked in functional testing.  Ultimately, the human mind is required.  This requires skilled testers who can find bugs by whatever means they deem appropriate.  Ad hoc testing is a skill that greatly improves the chances of delivering a successful product, especially in critical situations.

Now, however, things are starting to change and more tech-centric and self-service models are coming into play.

According to a recent report published by Deloitte, general insurance in the next five to ten years is going to look radically different. In their report, they highlight nine key areas of digital growth and disruption, ranging from seemingly innocuous developments through to previously unfathomable product lines. Here's the full list in no particular order:

1. Price comparison websites
2. Mobile internet transactions
3. Cyber risk insurance
4. Telematics-based services
5. Value comparison websites
6. Social brokers
7. Peer-to-peer insurance
8. Sharing economy insurance
9. Self-driving cars

Digital disruption is very real 

The old model of agent driven sales has started to go the way of the dodo, as more tech savvy customers use the Internet to get the best deal for themselves.

Of course, agents still have their part to play for now, but one can't help but conjure up comparisons between insurance today and the travel or banking industries a decade ago.

That's just the tip of the iceberg. Digital disruption seems to be affecting all facets of the industry, for instance:

• Connected cars allow companies to customise auto insurance coverage based on data collected by sensors that track a person's driving style.
   
• The current auto insurance business model will also be severely impacted as more self driving cars become mainstream.
   
• The sector will see the entrance of tech and mobile companies that have all the customer data they need to bypass traditional brokers. For instance, O2, a mobile operator in the UK now offers car insurance while Baidu does the same in China. 
   
• Drones are now used to survey inaccessible and sometimes risky areas, especially in the aftermath of natural disasters so that claims adjusters and risk engineers don’t have to expose themselves to unnecessary risk.
   
• Travel insurance companies are using sensors in airline operations to cut down on lost baggage incidents and reduce claims.
   
• Robots like the military grade Scio Surveyor 7 are now used to survey unsafe structures, confined areas or hazardous environments. 
   
• Other operating models, like peer-to-peer insurance where people team up to absorb risk, are also seeing a lot of interest from big players who are backing insurance startups like Lemonade. 

Software is eating the insurance industry

Insurance companies can no longer look at technology as a supporting function. Rather than being a delivery channel, it's now the core product, and not being able to tackle software bugs can be catastrophic.

According to one estimate, software failure in the financial sector caused losses of $521 million for a single incident. 

While software failures and demands on IT increase, insurance CIOs are staring at the spectre of stagnant and sometimes shrinking budgets, according to KPMG. In the report, 62% of CIOs said they expected their IT budget to remain the same or decrease over the coming twelve months. 

Perhaps paradoxically, the same report indicated that improving operational efficiencies and delivering business intelligence are the top two expectations from CIOs and digital directors in insurance companies.

If both are to come to fruition, companies will need to adopt newer ways to cut costs, from using virtual assistants and Facebook messenger bots to making the process of getting a new policy as easy as buying on Amazon.

One way to achieve these outcomes is artificial intelligence. According to Accenture:

• 82% of insurers think that AI driven automation will be incorporated into every business process in the next 5 years.
• 82% of insurers are increasing their investments in embedded AI.
• 35% of insurers have saved 15% over the last 2 years by automating various processes. 
• 79% of insurance executives think that AI is going to radically change how companies process data and interact with customers.

One thing is for certain: insurance companies will have to operate like tech companies, adopting agile methodologies to roll out products with shorter time-to-market.

But it’s not enough to build software solutions or automate processes.

Software will have to function reliably, and in the rush to push out new products while keeping the project under budget there’s a risk that quality might suffer because of lack of adequate testing. 

The implications of poor quality software in insurance

Naturally the nature of a software bug can vary dramatically, but could include things like:

• Users not being able to complete key processes online.
• Unsecure handling of confidential data. 
• Claims and applications being delayed or lost.
• Incorrect or invalid product recommendations.
• System crashes at critical times (e.g. after a natural disaster).

The implications of any of these bugs may include consequences such as:

• Disgruntled stakeholders.
• High support burdens.
• Negative publicity.
• Poor ratings and reviews.
• Financial implications.
• Lost business.
• Damaged reputation.

Of course, not all software bugs are catastrophic. Some are so innocuous they’re never even identified. But be warned, they all cost you money, customers and opportunities. Studies show that the cost of resolving bugs after they hit production is 30X greater than remediating the bug prior to launch.

In 2016 alone, it’s estimated that software bugs resulted in 315 years, six months, two weeks, six days, 16 hours, and 26 minutes worth of downtime, lost productivity, and lack of service.

The stakes of not testing code are higher when algorithms are buggy. When so many processes are automated, a tiny bug hidden in a subroutine somewhere deep within the codebase can wreak havoc and deny valid claims, lead to PR crisis, and open companies to the risk of fines or litigation.

The public implosion of the first iteration of Healthcare.gov in the US, which cost the federal government  $174 million, should be a cautionary tale about the importance of testing code. If you're unfamiliar with the story, here are a few key points that illustrate the scope of the catastrophe:

• Testing times were cut to a week down from months in an attempt to fast track launch
• When the website went live many pop-ups still had placeholder lorem ipsum text
• Testing didn’t change the schedule of the project, and all the issues uncovered weren’t fixed before the project went live
• There were no manual processes to fall back on in case the automated process of enrolment didn’t work
• Security testing was faked, and the site leaked personal information like name, email, social security number, and addresses.

For a heavily regulated sector like insurance, security is an especially sensitive issue, and lack of security testing can literally put you out of business.

According to Accenture data:

• 49% of insurers report that the number of security or privacy breaches doubled in the last two years.
• 78% of insurers say that they are not prepared to handle the risks as they move into the digital domain.
• 85% of insurers understand that without trust, their entire business will collapse.

In modern SDLC, testing can’t be pushed off towards the end. With Agile methodologies increasingly being adopted by the industry, testing has to be a continuous process. 

However traditional testing processes are slow and time consuming, and in the interest of saving time most people compromise and test just before release. That, of course doesn’t work as more time and money is spent on fixing the bugs. 

A vicious cycle then sets in, and the project ends up either being delayed or gets shipped while it’s still buggy.

A new way to deliver dramatically higher quality software

Bugwolf takes an innovative approach to software testing, promoting a proactive, rather than reactive, solution to the problem.

Our unique model has made us the UAT solution of choice for digital leaders at NAB, Australia Post, Treasury Wine Estates and many more.

We transform software testing into competitive software challenges that accelerate digital releases, lower customer support calls and reduce defect costs.

During challenges, professional testers compete to uncover as many bugs as possible within a set timeframe (usually six hours).

Our suite of testing tools allows us to provide deep cross-device coverage and detailed video bug reports with audio commentary.

It’s not uncommon for us to uncover one hundred or more bugs in a single six hour challenge. 

Key takeaway

The once-placid world of insurance is going to experience digital disruption as new players and radically different operating models emerge.

To stay in the game insurance companies will have to change how they operate, and embrace a tech driven model which focuses on defect-free software.

Every digital leader has a responsibility to ensure their organisation doesn’t become the next software testing disaster story... That is the only way to curve the cost of software bugs on your organisation and the wider insurance industry.

What Next?

If you are new to Bugwolf and would like to learn more about how we help insurance companies accelerate testing and improve software quality, the quickest and easiest way to find out more is to Request A Demo by clicking HERE.

Change is one of the few constants in the world of software testing. 

An organisation’s ability to stay on the front foot and adapt to evolving development environments and consumer expectations has implications that stretch far beyond the IT department. 

Businesses and their customers are experiencing the fastest and most radical tech changes in history. Nowadays IoT, AI, big data and VR have far surpassed experimental applications and continue to enter the mainstream. 

We compiled the following infographic to share our predictions on how software testing will evolve in 2017 and beyond. Please share your feedback and your own predictions in the comments section below.

At Bugwolf we proactively work with you to ensure our severity guidelines are consistent with your own internal definitions. The result is faster test cycles and more efficient remediation.

We've put together an infographic outlining our approach to managing severity guidelines. If you’d like to find out more about our approach - and see Bugwolf applied to one of your assets at no cost - we'd love to hear from you.

How To Attract The Best Software Testing Talent

Software testers are the gatekeepers for digital quality in your organisation. Their role is effectively to protect your brand, reputation, customer experience and bottomline results. So it shouldn't come as a surprise that the best software testers are highly sought after. 

Bugwolf is the world's first and only gamified user acceptance testing platform. Every year we receive thousands of applications to join our team... but only the top 1% make the cut. Suffice to say, we know a thing or two about attracting the world's best testing talent.

In this infographic we explore what makes top software testers tick, how you can attract the interest of top candidates, and the common assumptions decision makers make that undermine your recruitment efforts.