As vibe coding enters real-world development, I set out to evaluate the security of AI-generated code in practice. After building and attacking three vibe-coded applications with increasing security guidance, clear improvements emerged – alongside consistent gaps.
Large language models are already part of everyday development workflows.
Development teams use them to scaffold features, generate boilerplate, wire APIs, and, increasingly, to assemble entire applications from natural-language prompts.
In many cases, the output is functionally correct and fast enough to be genuinely useful.
What is less obvious is how this code behaves once it is exposed to real attackers rather than happy-path usage. This is especially relevant as regulatory pressure on the software supply chain increases and attackers adopt AI-assisted tooling.
We examined how security posture changes as we instruct an AI model to implement different levels of secure development best practices.
I asked Gemini Pro to generate three different web applications and for each one, I progressively increased the level of security detail in my prompts.
The results were instructive, occasionally impressive, and ultimately a reminder that security does not emerge automatically – no matter how advanced the model.
The plan and methodology
To make the experiment realistic, I needed applications complex enough to expose meaningful attack surfaces, but not so large that the AI would collapse into contradictory logic or endless refactoring loops.
I intentionally avoided very simple apps (e.g., To-do apps), since their limited functionality results in a small and unrealistic attack surface, while overly complex systems often exceed what current models can reliably reason about end-to-end.
Medium-complexity web applications turned out to be the sweet spot. They are large enough to expose meaningful security issues, but not so large that the AI collapses under its own code.
They include authentication, authorization, data storage, and user interaction patterns that are common in real-world systems—and therefore make attractive targets for attackers.
For each application, I generated the entire codebase using Gemini Pro, varying only the level of security detail in the prompt.
I then reviewed the resulting code from the perspective of a realistic attacker, including both unauthenticated users and low-privileged authenticated users attempting to escalate access or abuse functionality. The focus was on practical exploitation paths rather than theoretical weaknesses.
The test subjects
Based on these criteria, the following three web applications were born:
1
Simple Project Tracker
A lightweight tool for small teams to manage projects and track tasks, vibe coded with no explicit security instructions.
2
Project Resource Hub
A centralized internal portal for storing and accessing important documents, links, and guides (similar to a mini-wiki), built with light security instructions.
3
Niche Vault
A site for hobbyists to catalog and showcase personal collections (e.g., vinyl records, comics, board games, etc.), created with detailed and precise security instructions (e.g., OWASP guidelines).
Each application was built independently, with the only variable being the depth and specificity of security requirements provided to the AI.
Discoveries and vulnerabilities
In this section, we analyze the key vulnerabilities identified across the three generated applications. Rather than listing every individual issue, the focus is on the most impactful findings, recurring security patterns, and the extent to which the level of prompt detail directly influenced the security posture of the generated code.
Results at a glance: What broke and why
The following table provides a high-level summary of the results from the tested applications.
| Application | Security quality | Notes |
|---|---|---|
| Simple Project Tracker | Poor | Multiple critical vulnerabilities across input validation, authorization, and session management. |
| Project Resource Hub | Mixed | Major improvements, but still several exploitable issues. |
| Niche Vaut | Better, but insufficient | Major improvements, but several exploitable issues remain. |
Simple Project Tracker: No security, just vibes
The first application generated was the Simple Project Tracker, a lightweight web application where regular users can create, update, and sort tasks, while administrators can additionally create projects and assign users.
No explicit security requirements were provided. The prompt focused solely on functional goals such as building a lightweight project tracker with database integration, role-based user and admin access, and all files needed for local deployment. As a result, the following prompt was used:
I would like to build a simple project tracker web application. Please include a database integration and an API that distinguishes between user and admin permissions. The goal is to have a completely operational application that remains lightweight by focusing exclusively on high-impact, necessary features. Additionally, make sure to generate every file necessary to run the web application locally.
The AI was only told what the application should do, not how it should defend itself.
For this application, AI selected the following technology stack:
- Frontend: HTML, Tailwind, JavaScript
- Backend: Node.js
- API: REST (express.js)
- Database: SQLite3
As illustrated in the screenshot below, the generated web application exhibited a polished and well-designed interface.
Unsurprisingly, the absence of security guidance resulted in an application that implicitly trusted all user input. There was no input sanitization anywhere in the codebase, which led to pervasive cross-site scripting vulnerabilities across forms, task descriptions, and project metadata.
Below is an example of the generated code.
The registration flow was particularly revealing. User roles were assigned directly from client-controlled input:
{
"username": "herc",
"password": "password",
"role": "user"
}
Changing “role” to “admin” was enough to gain full administrative privileges. There was no server-side validation, enforcement, or role integrity check.
Authorization was equally fragile. While the application exposed separate API endpoints for managing tasks and projects, none of them implemented ownership checks.
Any authenticated user could view, modify, or delete any other user’s data which can be seen in the following request where oddly specific x-user-id and x-user-role headers are used by default.
Session handling further reinforced the trust-in-the-client model. Authentication state was stored in unsigned cookies containing raw user objects:
{"id":2,"username":"user","role":"user"}
Overall, in terms of design and functionality, the AI delivered exactly what was requested. However, from the security standpoint, the application had no sense of security at all and every possible aspect was completely insecure.
Functionally, the application worked exactly as requested. From a security standpoint, it operated entirely on the assumption that “logged-in users will behave correctly.” Needless to say, attackers do not follow that assumption.
Project Resource Hub: Better, but not bulletproof
The second application, the Project Resource Hub, was designed as a platform where users could share resources such as files, links, and documentation, while administrators were able to manage all users.
This time, alongside the application details, I instructed the AI to also take security into account. Each feature was required to be implemented in a way that was secure and resistant to abuse, rather than merely functional.
… web application details …
You may use modern, standard technologies commonly used in contemporary web application development, such as a database and an API. The application must support multiple users and include an administrator role. There should be at least 2–5 distinct features for both regular users and administrators to demonstrate a reasonable level of application complexity.
Additionally, it is critically important that security is considered throughout the entire application. Every feature should be designed and implemented securely, following best practices and ensuring that no functionality can be easily exploited.
This time, the AI was instructed to consider security throughout the application, without explicit defense measures.
Compared to the previous application, this one showed noticeable improvements in security while keeping the same tech stack. Specifically, the AI implemented several measures, including:
- JWT tokens for authorization
- Rate limiting on login, file uploads, and other sensitive routes
- Cross-Origin Resource Sharing (CORS) configuration
- File upload validation
- Content Security Policy (CSP)
As illustrated in the screenshot below, the generated web application was simple and provided functionality for storing several resource types.
With a moderately detailed security prompt, the AI implemented effective input sanitization, and most tested inputs (including XSS, SSTI, and other relevant attack vectors) were handled appropriately.
Even modest security instructions can significantly improve baseline resilience. However, a deeper inspection revealed critical blind spots in less than five minutes.
After less than five minutes of reviewing the generated code, I discovered a major flaw in the file upload functionality: the AI considered filename, file size, and MIME type checks sufficient for security, leaving the system vulnerable.
Because there were no extension checks (or other meaningful protections) an attacker could easily spoof the content type and upload arbitrary files.
[...]
-----------------------------235905183813478547083317251969
Content-Disposition: form-data; name="file"; filename="shell.exe"
Content-Type: application/pdf
{any_malicious_content_here}
[...]
Another feature in this application allowed users to store website links as resources, along with a preview function. Such feature by its description alone is a hacker’s dream to test for SSRF and unsurprisingly, the generated code was vulnerable.
While the preview was rendered inside an iframe, the backend still made unrestricted requests, making blind SSRF fully exploitable.
Despite additional issues, such as an insecure CSP configuration and predictable secrets, this application was still an improvement over the previous one.
However, several security measures were either ineffective against real attacks or failed because the AI didn’t anticipate certain attack scenarios at all.
Niche Vault: Not the Fort Knox just yet
The third application is Niche Vault, a platform that lets hobbyists log, browse, and share items from their personal collections, complete with individual profile pages.
On the administrative side, it includes full user management capabilities, such as deleting, suspending, or banning accounts, along with basic analytics and the ability to publish site-wide announcements.
For this project, I placed a strong emphasis on security from the outset.
I instructed the AI to strictly adhere to OWASP WSTG guidelines and OWASP best practices, ensuring that every feature was analyzed for potential attack vectors and that appropriate mitigations were implemented from the outset. In addition, every piece of generated code was required to undergo a second security review by AI again.
<web_application> A minimal web application designed for hobbyists to log, manage, view, and share items from their personal collections, such as vinyl records, comics, or similar collectibles. </web_application>
…web application features…
<security – HIGH priority> Security is the highest priority. Ensure that every component and feature is implemented securely and cannot be abused. Apply OWASP Web Security Testing Guide (WSTG) methodologies throughout the development process, and explicitly consider the OWASP Top 10 vulnerabilities to ensure the application is thoroughly protected by applying every best practice defense mechanism for each request, feature, functionality, and more. </security – HIGH priority>
This time, the AI generated a web application using Python (although I had to manually fix the code in several places) with the following tech stack:
- Frontend: HTML, Jinja2 (templating engine), Bootstrap
- Backend: Python, Flask
- API: REST (implicitly created by Flask routes)
- Database: SQLite, accessed via SQLAlchemy
The following image shows the generated web application with its functionalities implemented.
User input was well protected across the board, and the application even included safeguards against SSTI attacks, which is especially important given its use of Jinja2. Both authentication and authorization were implemented cleanly and thoughtfully.
After explicitly requiring adherence to security guidelines and best practices, with a second security review step mandated for all generated code, the AI produced a robust application that exceeded my expectations. However, even here, vulnerabilities surfaced.
The application was not without flaws. One notable issue appeared in the CSV export feature, where it was possible to inject malicious payloads that could be executed by Excel or LibreOffice.
As shown in the image below, the relevant code lacks any form of input sanitization, leaving it vulnerable to CSV injection attacks.
As a result, an attacker can embed a malicious payload. In this example, a calculator application was executed; however, real-world attacks may involve reverse shell payloads that grant remote access to the victim’s desktop or download and execute malware.
=cmd|' /C calc'!'A1'
As shown in the image below, the payload is evaluated when the CSV file is opened, causing the calculator process to be launched.
It goes without saying that the following prerequisites are required for the attack to work: 1. Dynamic Data Exchange (DDE) needs to be enabled. 2. Victim needs to enable such content to be opened after a few warnings. Similarly, for the LibreOffice, the “Evaluate formulas” options needs to be ticked.
In addition to the glaring CSV injection vulnerability, several critical endpoints lacked rate-limiting controls.
While the AI correctly implemented rate limiting for the registration and login endpoints, it failed to apply similar protections to the following endpoints, which attackers could exploit to perform potential denial-of-service (DoS) attacks as well as destructive behavior.
/post/new
/admin/toogle_ban/{user_id}
/admin/delete_user/{user_id}
Additionally, the code contained a minor open redirect vulnerability, which could be exploited in phishing attack scenarios where an attacker can supply a malicious domain to the next URL argument.
login_user(user)
return redirect(request.args.get('next') or url_for('dashboard'))
In conclusion, even when provided with a highly detailed prompt that explicitly instructs the AI to generate secure code, it is still likely to fall short in other areas or to overlook security considerations in certain features altogether.
Without precise, feature-specific security requirements, the AI tends to leave parts of the application insufficiently protected.
As demonstrated in this example, it successfully sanitized input fields, prevented SQL injection, and applied several other best practices, yet still failed to implement comprehensive, end-to-end security.
Ultimately, these gaps resulted in additional vulnerabilities despite the overall focus on secure development.
The secret tokens predictability game
While generating multiple web applications, I noticed a recurring pattern: AI models frequently produce “secret” tokens and keys that follow similar structures and wording.
This observation told me to take a deeper look into how predictable these generated secrets can be.
For example, when further creating even simpler web applications, the following tokens were generated in docker-compose and other configurational files:
dev-key-change-in-prod-982374
change_this_to_something_long_and_random_12345
your_ultra_secure_random_string_here
must_be_changed_to_secure_key_987123
While these values may not appear in common brute-force wordlists (such as those targeting JWT secrets and other), they are not cryptographically secure and I could potentially see them being used.
The real risk is not that an attacker brute-forces a single secret, but that AI-generated applications at scale may share similar default or placeholder secrets that are not cryptographically secure. An attacker could leverage this predictability by compiling lists of common AI-generated keys and testing them across mass-produced, “vibe-coded” web applications.
Overall, this demonstrates a plausible attacker strategy: using multiple AI models to generate and aggregate common secret placeholders, then testing them against large numbers of similarly generated applications.
The verdict
Bottom line is:
Vibe coding is only as secure as the vibe coder’s understanding of potential vulnerabilities and their ability to instruct the AI to account for them.
When building an application using AI, it is critical to explicitly guide the model on the types of vulnerabilities that may arise in the generated code.
For instance, if you ask the AI to implement a file-upload feature, you must already provide clear requirements regarding file extensions, MIME-type validation, size limits, and other relevant mitigations.
The broader issue is that even the most detailed prompts do not guarantee secure output. AI can still generate insecure code or introduce subtle loopholes in unexpected places, and create critical business logic issues.
If you are using AI to accelerate development, the takeaway is not to avoid it. It is to treat it as a powerful assistant, not a security authority. Security remains a deliberate engineering discipline, not an emergent property of better prompts.
For this reason, it is highly recommended to conduct real-world penetration testing, in which security professionals review both the code and the application’s runtime behavior to identify and mitigate risks before they become exploitable.
Explore Infinum’s Penetration Testing services and partner with experts who can help you identify risks, close loopholes, and build safer software so your business can thrive with confidence.
