Check your own app — free, in about a minute.
The free mini-scan checks your live homepage for TLS/HTTPS basics, response security headers, cookie flags, exposed sensitive files, Firebase/Supabase backend rules, and the WordPress plugin versions your site exposes. You get a score and the finding titles — no account, no card.
Run the free mini-scan →AI coding assistants write code that works. They also write code with security holes at a remarkably consistent rate — study after study finds that 30–40% of AI-generated code contains vulnerabilities. This isn't because the models are bad at programming. It's because of how they're built and what they're optimized for.
Understanding why the models get security wrong helps you catch it. Here's what's going on under the hood, and the vulnerability categories it produces.
LLMs are trained on enormous corpora of public code: GitHub repos, Stack Overflow answers, tutorials, blog posts. A lot of that code is fine. A lot of it is also quick-and-dirty: tutorial code that skips auth "for simplicity," Stack Overflow answers from 2014 using MD5, sample apps with hardcoded credentials. The model doesn't distinguish between production-hardened code and a weekend hack — it learns the statistical shape of "code that answers this kind of question," and insecure patterns are extremely common in that shape.
When you ask for a login form, the model's strongest associations are with the thousands of login-form tutorials it has seen — most of which never mention rate limiting, and some of which store passwords in plaintext "to keep the example simple."
The model's job is to produce code that satisfies your prompt. Your prompt describes the happy path: "build a signup flow," "add a file upload," "make an admin dashboard." Nothing in that prompt asks for the adversarial cases — what happens when someone who isn't a legitimate user hits these endpoints. So the model doesn't generate them. Auth checks, input validation, rate limits, and error handling are all things you have to ask for explicitly, because the default output is the demo, not the defense.
Security best practice moves. The model may confidently recommend an approach that was fine in 2019 and deprecated since: a hashing algorithm that's now too fast, a TLS configuration that's now weak, a library version with a known CVE. It states these with the same confidence as everything else, which makes them harder to spot than an obvious bug.
A human security review starts with "who wants to break this and what do they get?" The model never asks that question. It doesn't consider that your "internal" admin tool is on the public internet, that sequential IDs are enumerable, or that your error messages are a reconnaissance goldmine — unless you put those concerns in the prompt.
These are the patterns that fall out of the causes above, over and over:
The model needs a value to make the code run, so it invents or pastes one inline. API keys, database passwords, and JWT signing secrets end up in source files, committed to git, and — worst case — bundled into frontend JavaScript.
// What the model writes to "make it work"
const stripe = require('stripe')('sk_live_51H...');
const DB_PASSWORD = "changeme123";
Real-world consequence: any secret in client-side code or git history is public. Rotate it the moment you find it — there is no "it was only there briefly."
The single most common serious flaw in AI-generated apps. The model builds the feature (view invoice, edit profile, admin panel) and either skips the ownership check or implements it only in the UI.
// The model generates the endpoint...
app.get('/api/invoices/:id', async (req, res) => {
const invoice = await db.invoices.findById(req.params.id);
res.json(invoice); // ...but never checks: does this user own this invoice?
});
Real-world consequence: IDOR — change the ID in the URL and read other users' data. Test every endpoint that takes an ID with two accounts and swapped IDs.
The model usually reaches for parameterized queries — until the query gets complicated. Dynamic search filters, sort columns, and admin reporting tools are where string concatenation creeps back in:
// "Clever" dynamic sorting, straight from the model
const query = `SELECT * FROM products ORDER BY ${req.query.sort} ${req.query.dir}`;
One request with ?sort=(SELECT password FROM users) and you're having a very bad day. Same story for command injection wherever user input reaches exec() or template rendering without escaping.
Ask for password storage and you'll sometimes get bcrypt — and sometimes get this:
// Seen in the wild from AI assistants
const hash = crypto.createHash('md5').update(password).digest('hex');
MD5 for passwords, homegrown "encryption," JWTs signed with weak or hardcoded secrets, tokens generated with Math.random() instead of a CSPRNG. The model knows the names of the right tools but doesn't reliably reach for them.
As covered in the happy-path section: the model builds the login form, not the lockout. Credential stuffing, password spraying, and enumeration attacks against AI-built auth flows are trivially easy because nothing slows the attacker down.
Detailed errors help developers debug, so the model leaves them on. Stack traces, ORM errors with table names, framework version banners — each one narrows an attacker's search.
The model pins whatever version was current in its training data, which may be months or years old, and never checks for CVEs. AI-generated package.json files routinely include dependencies with published critical vulnerabilities on day one.
None of this means you shouldn't use AI to build. It means you should treat AI-generated code the way you'd treat code from a fast, tireless junior developer who has never been on call for a security incident: review the security-relevant parts yourself, every time.
Practical rules that follow from everything above:
The models are getting better, but their incentives haven't changed: they optimize for code that satisfies the prompt, not code that survives contact with adversaries. Until that changes, the security review is your job — or you automate the outside-in part of it.
Manually probing every endpoint the AI generated takes real time — and the hole you miss is the one that matters. PreShipIt scans your app from the outside for $29, one time: paste your URL, confirm you own it, and get a plain-English prioritized report of what an unauthenticated visitor can actually reach. It doesn't log in, review your source code, or guarantee you're secure — it just shows you what's exposed before strangers do.
Get your pre-ship inspection at preshipit.com
The PreShipIt Brief — a short weekly email for people shipping vibe-coded apps: the week's important vulnerabilities and launch-safety checks, in plain English.
New and free. Double opt-in: we email you a confirmation link before anything else arrives. Unsubscribe any time.
Check your own app — free, in about a minute.
The free mini-scan checks your live homepage for TLS/HTTPS basics, response security headers, cookie flags, exposed sensitive files, Firebase/Supabase backend rules, and the WordPress plugin versions your site exposes. You get a score and the finding titles — no account, no card.
Run the free mini-scan →← Back home · Run a free mini-scan · Scan my app — $29 · See a sample report