Adaptive SystemsSoftware testing
Your application works when you try it. We find out whether it works when your customers do, and give you the evidence either way.
When a test matters most
Most tests are asked for at one of these four moments.
It was built quickly
Software can now be built in days, and testing is the part that gets left out.
An application put together fast usually works along the one path its builder tried, with sensible input, on one browser.
We try the other paths, the odd input and the other browsers, and show you which ones break.
It was built in-house
The person who built it knows how it is meant to be used.
Your customers do not. They paste the wrong thing, press Back, lose their connection half-way and open it on an old phone.
We use it the way they will, and report what happens.
Someone else built it
You are responsible for software you did not write.
A contractor has moved on, or a member of staff has left, and nobody is sure what the application does in every case.
We find out what it does and where it breaks, and tell you what fixing it would take.
Customers are about to rely on it
A fault costs least before anyone depends on the application.
Before a launch, before you take payments through it, or before a larger customer signs, you need to know where it stands.
A test before that point gives you a list of what to fix, in order, and a record of what worked on the day.
What we test
A typical test covers five areas. Which of them, and how deeply, is agreed with you before we start.
Function
Every feature in scope is tested against a written specification.
If you have one, we check it against the application and fill any gaps. If not, we write one once we have looked at the application and asked you about it. You agree it before testing starts, so we both know what counts as working.
The test then goes past the usual path, into the combinations that rarely get tried. You learn which features work, which fail, and under what conditions.
Robustness
We test how the application behaves when things go wrong.
That covers invalid and unexpected input, repeated actions, several people working at once, and connections that fail part-way through.
The report shows where it copes, where it fails, and whether anything is lost when it does.
Accessibility
We test against WCAG 2.2 AA, the standard many public bodies are expected to meet.
Automated checks are followed by testing by hand: with a keyboard alone, a screen reader, enlarged text and high-contrast settings.
Each failure is reported with the affected users and a recommended fix.
Browsers and devices
Your users arrive on whatever they own.
Each screen is checked in the current versions of the major browsers, on desktop and on phones, at widths from a small phone to a wide desktop.
If your users are unusual, such as a warehouse full of old tablets, we can include what they use in the scope.
Outcomes
We check what was stored and sent, not only what the screen reported.
For each action we confirm the result behind the screen: the record saved correctly, the message delivered, the payment taken once.
We also check that the same records agree wherever they appear: in lists, reports and exports.
What the report contains
A test ends with a report, and the report is the product. This is what is in it.
The one-page summary
One page at the front says where the application stands.
It lists the problems that matter most, in plain language, and what fixing them would involve.
It is written for whoever decides what happens next.
Each fault, reproduced
Each problem is written up so that anyone can reproduce it.
It states what we did, on which device and browser, what happened, and what should have happened.
Each one comes with its evidence, usually a screenshot or a recording.
Ratings
Each problem is rated by how much harm it can do and how often it would happen.
Faults that could lose a customer's work, take a payment twice or show one customer another's details are rated highest.
The ratings give your developers an order to work in.
What to do about each
Each problem comes with a suggested fix your developer can act on.
Where several problems share a cause, we say so, since one fix may cover them all.
If you would rather we made the fixes, we quote for them, with a retest to confirm them.
A record of what was tested
The report says exactly what was covered, on what, and when.
It lists the features, devices and browsers tested, and anything left out of the scope.
It describes the application as it was on the dates tested, so you know when a larger change calls for another test.
Questions about testing
We built it ourselves, quickly, with modern tools. Can you still test it?
Yes. We test what the application does, whatever it was built with and whoever built it. Applications put together quickly are the ones that gain most from a test.
We have no written description of what it should do. Is that a problem?
No, most applications built quickly have none. We write one once we have looked at the application and asked you about it, so that we agree on what each feature should do and for whom. A test needs that to tell a fault from intended behaviour, and it stays useful to anyone who changes the application later.
Our developers already test. Why use someone else?
Developers test that the software does what they intended. A tester who did not build it has no such expectations and tries things the developers did not think of, so the two find different problems.
How do you keep our code and findings confidential?
We sign a mutual confidentiality agreement with you before you share anything. Findings go only to you, in an encrypted file, never in the body of an email. Afterwards we keep only our record of the work, encrypted and with your data removed, and we never name you or describe your system as an example without your written agreement.
Will testing disrupt our live system?
We prefer to test a copy. Where testing must take place on a live system, we agree on safe test accounts, permitted actions and measures to protect production data before starting, and ask you to take a backup beforehand.
Customers' details go through it. Does a test cover how they are handled?
Yes, as far as the application's behaviour goes. We check that a record is stored as the person entered it, shown only to the people meant to see it, can be corrected when they ask, and is gone once it should have been deleted, each of them something the Privacy Act 2020 expects. Whether the application is protected well enough against someone trying to get in requires a security review, and we can quote the two together.
Is this the same as a security review?
No. Testing asks whether the application works for its users, and a security review asks whether it can be abused. The two overlap, and if you want both we quote them together, which saves repeating the set-up.
Can you test again after we have made fixes?
Yes. Once you know how much has changed, we quote a retest to match.
Can you fix what you find?
Yes. We quote for the fixes and retest them; your own developer or IT provider can make them instead. Ongoing maintenance and support may be available for applications we test. It is scoped separately, and where an IT provider referred you to us, we discuss it with them first.
Do you work with organisations outside Auckland?
Yes, anywhere in New Zealand. We test from Auckland in your working hours and go through the report with you on a call, or in person if you are nearby.
How small a job will you take on?
Small jobs are welcome. A test of the few screens your customers rely on most can be quoted on its own. We scope each job to what you need, and if a smaller one would answer your question, we say so.
What does a test cost?
It depends on the size of the job: the number of screens and features, the kinds of user, and how many browsers and devices matter to you. Ask for a quote and we will send a written scope and price.
Find out whether it works
Tell us what the application does and who uses it. We will tell you what we would test and send you a quote.