A team uploads their APK to a scanner, gets forty findings back, and forwards the PDF asking how worried to be. Usually the answer is: not very, and also more than this report suggests. Both at once, because the tool measured the things it can measure.
What scanners genuinely do well
- Manifest configuration: debuggable and allowBackup flags, exported components, cleartext traffic permissions.
- Dependency versions with known CVEs, including the native libraries teams forget are there.
- Pattern-matched secrets: API keys, private key blocks, hardcoded URLs.
- Missing hardening: no obfuscation, no pinning configuration, weak crypto primitives used by name.
- Regression checking. Running one on every build is genuinely worth it.
That list is real value, it costs almost nothing, and it belongs in your pipeline. The problem is what people conclude from a clean result.
What no scanner can reach
- Authorisation. A tool has no idea that order 1183 belongs to a different customer. Proving that requires two accounts and someone who understands the domain.
- Business logic. Whether a cancelled subscription still unlocks premium content is a question about your rules, not your code patterns.
- Runtime behaviour. What the app does under instrumentation — pinning bypassed, root detection hooked, a debugger attached — is dynamic, device-specific, and outside static analysis entirely.
- The backing API, which is where most severe mobile findings actually are. The scanner analysed a client; the vulnerability is in the server that trusts it.
- Chained impact. Three findings a tool rates medium can combine into full account takeover, and nothing automated will tell you that they do.
The false-positive tax
The other half of the problem is what scanners report that does not matter. A typical report includes weak-crypto findings in a vendored analytics library, insecure random used to pick a UI animation, and exported activities that a launcher requires. Triaging that noise costs a developer a day, and the day after that they stop reading the reports.
Which is how a genuine finding, on page nine, gets ignored — the failure mode is not the tool being wrong, it is the tool being tiring.
A workable split
| Question | Automated | Manual |
|---|---|---|
| Is a dependency vulnerable? | Yes, continuously | No |
| Is a secret compiled into the build? | Mostly | Confirms and finds obfuscated ones |
| Is allowBackup set correctly? | Yes | No need |
| Can user A read user B’s data? | No | Yes, with two accounts |
| Does pinning fail open? | No | Yes, under instrumentation |
| Can a deeplink skip authentication? | Rarely | Yes |
| Does the API trust a client-supplied role? | No | Yes |
| Do three mediums chain into a critical? | No | Yes — this is the job |
How to use both
- Run the scanner on every build, and fix what it finds that is real. Tune it until the noise stops.
- Test authorisation with two accounts on every new endpoint, as part of the pull request.
- Book a manual assessment before any release that changes authentication, adds payments, or exposes new data.
- Treat a clean scanner report as evidence that the obvious problems are gone — which makes the manual testing cheaper, because those hours go into logic instead of hygiene.