Before an app goes public, I wanted it probed the way a stranger would (with no credentials, no source code, and no inside knowledge) by a tester that physically cannot become a foothold if it's ever turned against me. So I built one: an isolated machine that audits a single app from the outside, proves each weakness with a single harmless request, and hands back a plain-English report. Its first run on my own portfolio found a wide-open chatbot endpoint, a leaky test server, and photos that quietly carried GPS coordinates and a camera serial number, the kind of thing a code scan never catches. Every finding was independently reproduced before I touched anything.
The challenge
Auditing your own work has a blind spot: you already know how it's built, so you test what you know is there and miss what an outsider would stumble onto. Reading your own source finds a different, smaller set of problems than probing the running app blind. I wanted the outsider's view (the position a real attacker actually starts from), but with three constraints that are easy to state and hard to hold to:
- It had to be a true outsider. No credentials, no source, no map of the environment. If the tester knows the internals, it stops being a black-box test.
- It had to be safe. A security test that deletes data or hammers a service to prove a point has done real harm to make a point. The tester needed a hard "prove it, don't exploit it" ceiling.
- It could never become a liability itself. A machine whose whole job is adversarial probing is exactly the machine you don't want to have a path into anything else. It had to be walled off, with no standing access and no automatic route back into my systems.
Approach
I built a dedicated tester as a deliberately constrained box: no stored credentials, no copy of any source, and no knowledge of the wider environment. It's walled off so that even a full compromise of the tester reaches nothing else. Its findings come back on a drive I physically carry, not over the network, so the isolation is a property of the design rather than a promise.
On top of that machine sits a fixed methodology, enforced as its operating rules:
- Map, then probe by class. First it learns the shape of the app the way any visitor can: the pages, the scripts the browser is handed, the endpoints that code actually calls. Then it works through one class of weakness at a time: exposed secrets, authentication gaps, injection, transport and headers, leftover files, third-party code, and what the site's own content gives away (added after this first run).
- Prove with a single request; never exploit. One record is the ceiling for demonstrating a hole. No bulk pulls, no writes, no deletes, no pivoting to another target. If proving a finding would require abuse, the finding is written up instead of pursued.
- Try to destroy every finding before believing it. Each candidate is reproduced a second time and argued against: what innocent explanation fits the same result? Anything that doesn't survive is dropped without a hedge, and the report says how many were dropped, because that number is part of an honest result.
- Verify again, independently, before acting. Nothing the tester reports is acted on until a separate step reproduces it from scratch against the live app. Only what survives that second pass becomes a real, dated, immutable finding.
The reports are written for a decision-maker, not a security specialist: every finding leads with what someone could actually do, how bad it is, and what fixes it, before any technical detail.
Results
The first target was my own portfolio site. The tester looked at nine classes of weakness, surfaced five findings worth acting on, and dropped six candidates that didn't hold up (an "exposed financial dashboard," for instance, turned out to be sandbox demo data). The independent second pass reproduced all five. Highlights:
- The chatbot's back-end answered anyone. The site's "ask the dogs" chat box posts to an endpoint that would reply to any caller, not just requests coming from the site, so a stranger could script it or embed it on their own page. Fixed by closing the endpoint to outside callers and no longer leaking which AI model answered.
- The test server was handing out more than the site. The copy used for testing exposed folder listings and a leftover build file. A housekeeping issue, but exactly the kind that turns into a real leak the moment something sensitive lands in a served folder.
- The photos were talking. This is the finding a purely technical scan misses. The profile photo carried the camera's body serial number, and a follow-up sweep found that every phone photo of the dogs carried GPS coordinates (where each shot was taken). Nothing in the code was wrong; the assets were leaking. Fixed by stripping metadata from every public image and adding a tool that does it for any new one.
- The writing itself was a map. My own case studies, read together, described my setup in more operational detail than a portfolio needs. No hard secrets, but a reminder that prose is part of the attack surface too. (This very write-up was kept deliberately abstract for that reason.)
The lesson
The value wasn't any single bug: this run found no way in, no data to steal, and no code to run. The value was the discipline: separating what an outsider can see from what an insider knows, refusing to believe a finding until it's been reproduced and argued against twice, and treating content and photo metadata as part of the attack surface rather than just the code. The geotagged photos are the clearest proof of that last point: the most privacy-sensitive thing on the site wasn't in any file a developer would think to check.