Check your own app — free, in about a minute.
The free mini-scan checks your live homepage for TLS/HTTPS basics, response security headers, cookie flags, exposed sensitive files, Firebase/Supabase backend rules, and the WordPress plugin versions your site exposes. You get a score and the finding titles — no account, no card.
Run the free mini-scan →A founder told us this week, after running our free scan on his launch: asking Claude Code to analyze his website from a security perspective "produces similar results." He's not wrong about the overlap — and he's exactly right about the question worth asking. So here it is, answered honestly: what does an AI code review actually cover, and what does it structurally never see?
Use it. An LLM reading your repository is fast, it's already in your workflow, and for a real class of problems it's the right tool:
If you ship AI-built apps and you're not running that pass, start there. The argument below is not "AI review is bad." It's that a code review and a deployment check are different instruments, and one of them is pointed at a thing the other cannot see.
An LLM review reads the code you hand it, in a chat, at a moment in time. Your users don't visit your repository. They visit the deployment — and the deployment is a different artifact.
Production config isn't in the repo. Security headers, cookie flags, TLS settings, redirect behavior: in modern stacks these are set by the host, the CDN, a proxy, or an environment flag — not by the code. A repository can contain a perfect header configuration that production never applies, because the app is deployed somewhere that ignores it, a proxy strips it, or the config only loads in one environment. We see the mirror image constantly: sites whose code looks careful and whose live homepage sends no HSTS, no CSP, and no frame protection at all. The review reads intent. The live site is the fact.
Live versions aren't in the repo either. On a WordPress site, the plugin and core versions that matter live in the deployment, and they drift every time someone clicks "update" — or doesn't. Vulnerability data moves daily; a scanner can match the versions actually running against it the day a CVE drops. PreShipIt's checker currently matches 29 plugin CVEs plus a WordPress Core check that way. Concrete example: the WordPress Core upgrade-window RCE (CVE-2026-119977) — we shipped a check for it in October, within days of disclosure. On November 3, 2026, it entered CISA's Known Exploited Vulnerabilities catalog — the federal list of flaws being exploited in the wild right now. An AI assistant's training data lags disclosures by months, and nothing about a chat session re-checks your site the day a new one lands.
Some apps aren't yours to hand over. Agencies and freelancers ship apps they can't paste into a chatbot — a client's codebase, under NDA, or simply never shared. An outside-in check needs a URL. That's the whole input.
A chat isn't proof. Ask the same model the same question twice and you'll get two overlapping, different answers — fine for brainstorming, useless as a verification artifact. A scan produces a deterministic score (ours is a published rule: 100 minus 15 per High, 7 per Medium, 3 per Low) and a shareable result page you can hand to a client, an investor, or a launch reviewer. They can open the link. They can't re-run your chat.
A chat ends. The deployment keeps changing — new plugins, new config, new CVEs. PreShipIt Monitor re-scans on a schedule and flags what changed. The review you ran in October says nothing about November.
A live outside-in scan has its own blind spots, and we'd rather name them than let the comparison do it for us. It cannot read your business logic. Its active probing (our Tier 2) is deliberately bounded and deterministic — login rate-limit behavior, injection canaries, sequential-ID checks — not an open-ended penetration test. Our severity scale tops out at High. And a clean scan describes exposure at a moment in time, not a guarantee. The two instruments overlap in the middle and cover different edges: repo review for the code, live scan for the deployment. Serious teams run both.
If you take one step: check the deployment, not just the code. The free mini-scan at preshipit.com/free-scan reads your live homepage — headers, cookies, exposed files, backend rules, live plugin versions — and gives you a score and a shareable link in about a minute. Keep asking Claude about your code. Just don't mistake the review for the site your users actually touch.
Related: Is Your Vibe-Coded App Safe to Ship? — the launch-day checklist this comparison grows out of. The Security Headers Check — the production-config gap, header by header.
The PreShipIt Brief — a short weekly email for people shipping vibe-coded apps: the week's important vulnerabilities and launch-safety checks, in plain English.
New and free. Double opt-in: we email you a confirmation link before anything else arrives. Unsubscribe any time.
Check your own app — free, in about a minute.
The free mini-scan checks your live homepage for TLS/HTTPS basics, response security headers, cookie flags, exposed sensitive files, Firebase/Supabase backend rules, and the WordPress plugin versions your site exposes. You get a score and the finding titles — no account, no card.
Run the free mini-scan →← Back home · Run a free mini-scan · Scan my app — $29 · See a sample report