I'm going to tell you something big cybersecurity doesn't want you to hear: the pentest you paid for probably wasn't a pentest.

I've lost count of how many times I've disclosed critical vulnerabilities to a company, only to hear: "That's weird, we just had a pentest done and they said we were clean." At this point, hearing that a company recently passed a security assessment is almost a leading indicator that I'm about to find something bad.

What You Think You're Buying

When a company pays for a penetration test, they expect a skilled professional to manually examine their application and find the vulnerabilities that automated tools miss.

What they often get is someone running automated scanning software against their infrastructure and packaging the results into a professional-looking report. Or a junior analyst who clicks around for an afternoon and calls it done. Either way, the result is the same. I'll be honest: I've never actually seen one of these big-firm reports myself. I'm sure they look great. I'm sure they have severity ratings, color-coded risk matrices, and executive summaries that make a client feel confident.

But I know what they don't contain, because I keep finding the proof. They don't flag the missing authorization checks, the authentication bypasses, the places where any user can access any other user's data. Whatever these reports look like on the inside, the companies that receive them walk away believing they're secure, and they aren't.

Three Companies. Three "Clean" Assessments. Three Disasters.

"We had a pentest done. They didn't find anything."

A technology platform that works with Fortune 500 automotive companies. I decided to take a look. Within minutes, I found their entire API was unauthenticated, with the schema publicly readable. That means anyone could see every available operation and then execute any of them without logging in. Customer data, financial analytics, internal business data, chatbot conversation histories with customer details, all accessible to anyone.

I reported it through responsible disclosure. Only then did I learn the company had recently paid a well-known UK firm for a security assessment that found no significant issues. They even had a renewal contract planned with the same firm.

When the company's head of engineering saw what I found compared to what the UK firm had reported, he called the firm's results "disgusting." His exact word. The API schema was publicly readable, which means even an automated scanner should have flagged it. Which means the UK firm either never tested the API at all, or something went very wrong.

"Independently Reviewed" by a Named Security Firm

A platform used by the film industry to manage sensitive digital assets, including unreleased films, and hundreds of physical devices at client locations. Their website prominently stated that a named, independent security firm "routinely reviews and audits" their platform. They marketed "bullet-proof infrastructure."

I found that most of their API endpoints had any authorization checks. Any user could access any organization's data. With a regular account. The server rarely validated anything. I basically had full access to most things: uploaded assets, screeners, internal configurations, user permissions etc.

This platform had been "independently reviewed" and "routinely audited." That firm missed the fact that authorization didn't exist on basically every endpoint, amateur hour.

SOC 2 Type II Compliant, Five-Minute Vulnerability

A platform used by major Fortune 500 media companies to manage unreleased content. SOC 2 Type II compliant, meaning an independent auditor had certified their security controls over a sustained period.

Their API had a straightforward flaw: the endpoint for adding content to a list didn't check if the user was authorized to access it. Content IDs were sequential numbers. Create a list, add IDs 1 through 10,000, and the list endpoint returned full metadata, download links, release dates, everything. The funny part? Requesting the content directly still returned a 403 Forbidden. They had authorization on one endpoint but forgot about the other.

This takes about five minutes of manual testing to find. SOC 2 Type II certified. Sequential IDs. No authorization check protecting some of the most valuable unreleased content in the industry.

Why This Keeps Happening

Automated scanners can't test business logic. If the firm relies on scanning tools, those tools find outdated software, missing headers, and known vulnerabilities. They cannot understand that User A shouldn't access User B's data. The most critical vulnerability classes I find are invisible to automated tools.

The business model rewards volume, not depth. Your "pentest" gets assigned to a junior analyst who either runs automated tools and formats the output, or spends a few hours clicking around without understanding the application. The firm bills tens of thousands either way.

Compliance doesn't guarantee security. SOC 2, PCI DSS, and similar frameworks focus on processes and controls, not on whether someone actually tried to break into your application. A company can pass every audit and still have critical vulnerabilities, because the audit wasn't designed to find them.

Nobody verifies the work. When a pentest firm says "no critical findings," who questions it? The company doesn't have the expertise to evaluate thoroughness. The firm gets paid whether they find anything or not.

The brand name is the product. I told one CEO I work with that big pentest vendors don't do thorough work. He agreed. He said he'd hire me to get the actual results, and then hire the big firm to get the badge. When enterprise clients ask for a "third-party pentest," they expect a well-known company name on the report. The actual quality is secondary to the logo on the cover page.

What You Should Do

Ask what methodology your pentest firm uses. If they can't clearly explain their manual testing process, that's a red flag.

Demand evidence of manual testing. A good report should contain application-specific findings that could only come from someone who understood your business logic. If everything in the report looks generic, it probably is.

Supplement big-firm pentests with independent researchers. Use the big firm for the compliance checkbox if you must, but don't rely on their report as your actual security baseline.

Treat "no critical findings" with suspicion, not relief. Every application most likely has some vulnerabilities. The question is whether your tester found them.

Kaeden is an independent security researcher and penetration tester based in Tokyo. For inquiries, reach out at whitehat@pentester.ca.