import test blog

Handoffs have always existed, they happen when one person or group’s output is handed to the next person or group along the development chain. For example, analysts hand off to designers who hand off to coders who then hand their written code to testers. This looks easy, but it often isn’t. Each handoff requires bringing the next group up to speed on what has been done. Unfortunately, people get busy with the next thing to do and tend to hand off with a minimum of explanation. This doesn’t just happen in software development. Finding the time for training is a major problem in many organizations. The unwillingness to brief the next link on the chain can create an incomplete handoff that results in delays and buggy software.

There are several relay points between the software developer and the user. These points breakdown into certain categories. These are the people who build the software, who distribute it, who test it as well as those who support software, train users and the users themselves. And this isn't everybody. There are many people who are responsible for the development cycle and not everyone has their hands directly on the code, there are numerous managerial positions as well. Each one of these individuals needs information and perhaps access to the application under development.

The standard development process basically begins with the delineation of requirements, proceeds through the build stage and on to quality assurance and testing.

Modern innovation is driven by the pipeline of continuous delivery made possible through automation that assists definition, building, and testing by uniting the development team into a collaborative whole that can rapidly move product to market. Yet, no matter how innovative software might be, its value can only be realized in the hands of the user, who then provides the final feedback.

When it comes to outlets for gambling, national lotteries tend to be regarded as reliable and trustworthy, but when technical problems are mismanaged, the deck can be stacked against players as badly as at any seedy casino. Last week, the UK Gambling Commission slammed National Lottery operator Camelot with a £1.15 million fine for "serious control and governance failings."

The UKGC announced the penalty on Thursday, the culmination of an investigation that started in December 2016 after multiple reports of issues with Camelot, including a mobile app that failed to correctly identify users' winning tickets with its QR code scanner.

In February 2017, the investigation was put on hold while Camelot implemented a new "Operational Excellence Program," but the investigation resumed later that year and expanded to take new findings into account.

In the end, the investigation identified the following failures:

1. The misleading mobile app

2. Publishing incomplete Lotto results online

3. A direct debit error that prevented the processing of Lotto sales in May 2016

4. Various security and postal control issues

It has been reported that five winners missed out on claiming million-pound prizes due to Camelot's errors in posting Lotto results, out of dozens of prize results that were omitted from publication. Camelot attempted unsuccessfully to identify and locate all of the winning ticket holders.

The UKGC reported that Camelot "engaged positively" with them during the course of the investigation, and commended them for their efforts to update and improve their internal procedures and controls to prevent problems from recurring in the future.

Camelot has been penalised by the UKGC in the past, most recently in 2016 for paying out fraudulent jackpot claims and posting incorrect Lotto results, and in 2014 for calculating a jackpot prize amount incorrectly.

In a statement, a spokesperson for Camelot promised to make a payment to National Lottery Good Causes in lieu of a penalty and said: "We accept the outcome of the Gambling Commission's investigation...While we have always sought to run The National Lottery to the highest possible standards, we accept that, at the time of these incidents, our standards in certain areas weren't as rigorous as they should have been and for that we're sorry."

Last week, thousands of shoppers in the United Kingdom were hit with duplicate debit card charges after a technical glitch caused a major payment processor to run some transactions twice. The incident happened on Wednesday, August 29, affecting payments processed by Cardnet.

Customers took to social media to report and complain about the double charges, while businesses did their best to notify customers who were affected by the glitch. Shoppers at Tesco and other stores where the glitch occurred received receipts that appeared to show the correct amount being charged, but when they reviewed their bank statements later, they could see that the transaction had been run twice.

Cardnet, which is partly owned by Lloyds Banking Group and First Data, handles more than 1.2 billion transactions each year for shops, pubs, and restaurants in the UK. In a statement published on their website, Cardnet said: "Cardnet sincerely apologises for the issue and the inconvenience caused, we continue to work closely with all parties to resolve this issue swiftly."

Lloyds also addressed the situation, stating that "a small number of transactions" were duplicated and that "action has been taken to rectify this situation as quickly as possible." The estimated that fewer than 5% of all transactions processed by Cardnet on August 29 were duplicated.

While it was mostly Visa debit cards that were impacted, the glitch hit customers from all banks. When the transactions were run, customer banks correctly set aside the correct amounts from the cardholders' accounts, but when the merchant banks attempted to "settle" and collect the money later, the glitch caused many of them to collect the transacted amount twice.

In their statement, Cardnet said that cardholders who were double charged should expect to see refunds in their accounts no later than Tuesday, September 4. They also promised to resolve the issue for merchants, whose ledgers could be thrown out of balance by the duplicated amounts.

All UK cardholders who made any debit or credit card transactions on August 29 have been advised to check their statements carefully to ensure that they haven't been charged with any duplicated transactions.

Want to learn more? Check out our Definitive Guide To User Acceptance Testing (UAT).

The first challenge is quality assurance.  The tendency is to think that the testing team are the people responsible for QA. This is where the challenge comes in. The reality is that quality assurance is the responsibility of both development and testing. Both are actually parts of the same team. Shoving all quality control off on testing can cause bad blood between two elements that should be working together. It must be remembered that development can't always account for everything and testing cannot increase quality, but only assure a minimum level. Development and testing must meet the challenge of working together from day one of the project.

Another challenge is recognizing usability problems as bugs. It's very easy to dismiss certain types of usability problems because they don't immediately stop software function. These problems may be something simple like printouts that are not as good as they could be or data presented in a non-optimum fashion. Such problems appear more as inconveniences that must be lived with, when they should be red flags. The key to overcoming this challenge is to put the user first at all times while using experienced testers who aren't shy about reporting usability problems.

The third challenge is knowing when to use automation and when to test manually. There is a strong drive to push automation, as it is faster and can be less expensive under certain circumstances. However, automation can’t cover everything and there are times when there is no substitute for a skilled tester. Generally, the best time to start automated testing is after the software has been stabilized, to some extent, by manual testing.

There are a number of myths about testing that have to be overcome. A prominent myth is that software testing is not a specialized activity and can be done by the development team. However, testing and development are two different mindsets. There is a big difference between building a software application and intentionally trying to crash that application. These two mindsets don’t go together. Building and testing belong in different magisterium. Nor can software be tested by just anyone.

Putting testing in perspective is perhaps the single biggest challenge. No amount of testing can guarantee to find all the bugs in any system. This is why testing often continues beyond release. How long testing continues and how often retesting is done depends on the complexity and purpose of the software.

Testing is a vital part of the software development pipeline and it is the best way to assure long run performance which requires high quality skilled testers who can meet its many challenges.

Software testing ensures that software meets the design and development guidelines, performs its functions accurately and within the required time frame while responding correctly to all inputs.  Software testing moves from testing modules as they are developed, onto testing the integration of these modules as an increasingly completed application, leading to final integration and establishing that the completed software is sufficiently usable, was properly in its intended environments and delivers the results desired by the stakeholders involved.

Software testing itself follows a basic pattern that consists of a number of different phrases.  Each of these phases exists to move testing operations forward.  In a sense, they mirror the software development cycle in that effective software testing begins not at the end of the development cycle but at the beginning as requirements are established.  From there, the software testing team works in parallel with the software development team, moving forward as the development cycle matures.

And so software testing is not separate from the development cycle, but is in reality an integral part of it.  It establishes the validity of the development cycle at various points within that cycle and so ensures that development is on the right track and successfully moving forward.  Each phase of the software testing lifecycle enters at a different point in the software development cycle.

The software testing lifecycle is composed of six basic phases.  These are:

Requirement analysis

It begins immediately after the requirements for the new application are determined.  The quality assurance team goes over the documented requirements to determine what the functions of the application are that will be tested, as well as nonfunctional elements that require testing.  This may require interfacing with certain stakeholders regarding certain parts of the application in order to get a clear understanding of exactly how the application is intended to function and the environment it is intended to function in.  The deliverables of this phase are an automation feasibility report, if required, and an accurate list of the testable requirements.

Building a testing strategy

This is perhaps the most important part of the software testing lifecycle.  It is where the amount of cost and effort in testing is estimated based on a testing plan that details what tests will be run at what points in the development cycle.  This includes the documentation of the testing plan itself as well as testing effort estimation and the delineation of an operational time frame.

The testing strategy includes, but is not limited to, assigning responsibilities as well as defining what the control procedures will be and preparing test schedules.  It will also contain a testing overview that will define the testing environment, give an estimation of the necessary resources and resource planning, and also define the scope and objectives of testing as well as listing what types of tests will be conducted.  The deliverables of this phase are testing plan and effort estimation documents.

Development of test cases

Once planning and testing strategies have been laid out, the next step is to develop test cases based on the requirements documentation.  An automation feasibility report should have been issued during the testing strategy phase.  This will determine how many tests can be automated and how many must be manually done.  This phase includes preparing data that will be used in executing test cases, and preparing automated scripts when necessary, as well as ensuring that there are no missing or ill defined requirements that could cause unnecessary testing or inhibit the development of accurate tests.  The deliverables here are testing data, test cases and the necessary testing automation scripts.

Setup of the testing environment

An accurate testing environment is vital to the software testing lifecycle.  Setting up the testing environment consists of compiling a list of the hardware and software needed to meet the testing requirements.  This is then followed by setting up the hardware and software and engaging in smoke tests to gauge the testing environment.  The deliverables of testing environment setup are an environment ready to begin testing and the results of the smoke tests.

Testing operations

Requirements have been analyzed, a test strategy has been developed, test cases have been written, and setup is now complete. Testing can begin.  Test cases are then marked as passed or failed.  Those tests that fail are reported to the development team through the established bug tracking system and the bugs themselves are labeled for correction and additional testing.  Ideally, failed test cases should indicate the nature of the bug or bugs.   This means that sufficient test case scenarios must be developed in order to cover possible operational flaws.  And defect reports and test case execution reports are then presented as deliverables.

End of testing cycle

When all tests have been run the testing cycle is then evaluated by the testing team members based on the criteria that had been laid out for cycle completion.  This includes the vital business objectives and software operation and depends heavily on the quality of the testing involved, as well as the quality of the testers involved.  The end of the testing cycle is the time for learning what went right and what went wrong.  Such knowledge improves future testing capabilities by maximizing successful actions and minimizing unsuccessful actions.  It is more than just the deliverables of the test closure report and testing metrics, it also helps build the platform from which the next testing cycle for the next product will be launched.

With this change comes significant changes to humankind. From the way we communicate and connect with one another, to the speed in which we access information to make more informed decisions - these all have significant impact to not only the way we live, but also the economy.

New challenges emerge from these changes, the likes of which have not been encountered in history. The shifting of wealth across regions in developing and developed countries is one such challenge. Another, is the impact of when the software we rely on in our daily lives doesn't work.

This shift motivated Cambridge University to conduct a study and release a report which stated software bugs cost the economy $312 billion (USD) annually in 2013, and this amount continues to grow to $1 trillion in next 5-10 years as software becomes essential in everything we do.

We address this is problem by making quality assurance more fun and engaging, increasing testing coverage and delivering a better digital experiences, all of which lead to a significant reduction in the cost of quality.

In more recent times we’ve seen an increase in on-demand consumer based services, such as Uber, TaskRabbit and HomeJoy. According to a study conducted by software company Intuit, more than 40% of the US workforce will be freelance by 2020, largely due to mobile technology.

At Bugwolf, we have recognised the change which is taking place and we’ve built a SaaS platform and vetted marketplace of on-demand elite software testers. We are not about putting bums on seats, but about providing talented teams of quality assurance professionals that can be scaled up and down as needed, with no lengthy contracts.

By giving our clients access to an on-demand contingent workforce, they reduce the cost of their testing by up to 50% - plus testers have more flexibility about when and where they want to work. This creates a significantly more enjoyable work option than traditional employment models, leading to improved satisfaction and better results for our customers.

References: 40% of America’s workforce will be freelancers by 2020

Research data from the late 1990’s is still alive and well circulated with its share of detractors. Some claim the results are based on multiple variables and others claim the results only pertain to non-Agile development projects. For example, current numbers reported by Bradley Jones in Code Guru 'The Cost of Bugs' indicate that the average cost of fixing defects in the following phases:

Development $937.00
QA $7,136.00
Production $14,102.00

Based on these figures, it is 14 times more expensive to fix production bugs. These numbers don’t measure the impact on business reputation or loss based on negative end user experience.

Re-work costs for resources, scheduling, planning

Regardless of whether Agile or non-Agile methodology is used costs still increase the longer a defect resides in the code. Defects don’t age well in that the severity impact of the bug is greater the longer it resides in the code. Costs increase because developers have to re-code, re-check in and rebuild a release. The release has to be re-deployed to multiple test servers, re-tested and re-deployed to production. Additionally, all the automated unit and regression tests are re-executed and any other manual tests re-executed. It’s all time, money and customer impact.

If the defect is severe or causes significant re-design then the architects, product managers and documentation resources are also part of the re-work.

The endless circle – quick fix in production yields more defects

Production defects tend to replicate as well. For example, the development team makes a quick fix for a simple defect found by customer support immediately after a release.  No problem, simple code fix, re-build, re-deploy, and re-test. Then, the QA testers find additional defects caused by the quick fix to the code.  Now, upper management gets involved to pull back the release or re-fix, re-deploy, and re-test. It’s a continuous loop and it’s expensive.

The exact development phase the defect is found is irrelevant. The important part is that defects are found prior to deployment into production. The faster they are found and fixed, the less continuous looping occurs after a release and the more new development work is done. If the entire development team is chasing and fixing defects continuously there’s little time to design and build new code. 

So why the rapid rise of the CDO’s?

It seems like almost all major businesses and governments have a ‘ digital transformation’ agenda and are making it one of their key strategic pillars.  This is occuring across the the business landscape from Telecos to banking. And over 50% of companies who do not yet have a digital transformation strategy, plan on adopting one within the next year.

The CDO is filling a gap that often exists between traditional CMO / brand marketers that have learnt their trade in the days in the age of 5 P’s, award winning TV ads and a heavy reliance on ad agencies, and CIO’s who were sometimes deemed to be great technology strategist, but lacked the ability to implement complex enterprise wide projects on time and on budget.

The CDO is appointed on the grounds that they are part of the rare breed that understand all forms of digital marketing ( mobile responsive sites, apps, programmatic media buying, social etc ) they understand the next generation of customers ( digital natives ) and are credible with hard marking IT teams ( are fluent in latest tech ). They are expected to a record of executing on-time, every-time, and be an inspiring public speaker, who can take staff, the media and shareholders along their transformational vision.  Based on this lofty brief, you can see why the demand for CDO’s outweighs the supply.   

Why will this role need to evolve or run the risk of being less relevant?

• The next generation of CMO’s will be digital natives, and have an intimate understanding of digital ecosystem

• A recent salesforce survey predicted that by 2021 75% of marketing budgets will be on digital with only 25% on traditional - so all CMO’s will need to be CDO’s.
• Eventually organisations will complete their digital transformation and be fully digitised businesses, making chief digital officer seem like the CEO, an outcome that most CEO’s will be keen to prevent    

• Continued convergence of CMO’s and CIO’s; a battle that is still be played out

​Conclusion

The Chief Digital Officer has the potential to suffer the same fate as the ‘Change Agent’ or ‘Change Manager’ of the past. Eventually ‘change management’  skills became a prerequisite of all executives and senior managers, which resulted in the demand for this specialised role to declined over time. Senior Managers that didn’t have these ‘Change Management’  skills faced extinction and the same will apply with digital skills. Based on this logic, It is likely that we will see a resurgence of CMO’s and CIO’s  as the next generation reclaim the digital agendas. Fast forward five to ten years and  digital transformations plans of most companies will be complete, making the seemingly special word  ‘digital’ superfluous at the CXO level.

Cameron Buckley