GenAI Is Accelerating Software Development — But Is Security Keeping Up? - Part 1

Generative AI is changing how software is designed, written, tested, and deployed. It can dramatically improve developer productivity, but the same acceleration can also multiply security weaknesses at a scale traditional review processes were never designed to handle.

Introduction

Generative AI has rapidly become part of the modern software-development lifecycle.

Developers increasingly use AI assistants to:

• Generate application code
• Create APIs and backend services
• Build infrastructure-as-code
• Produce database queries
• Generate tests
• Refactor existing applications
• Troubleshoot errors
• Create CI/CD configurations
• Build authentication workflows
• Integrate third-party services
• Generate entire proof-of-concept applications from natural-language descriptions

The productivity benefits are substantial.

A developer who previously spent several hours building a feature may now be able to produce a functional implementation in minutes.

Small engineering teams can build products that would previously have required significantly more developers. Startups can rapidly prototype ideas. Security teams can automate repetitive tasks. Engineers can move from concept to working software faster than ever.

But there is another side to this acceleration.

The volume of software being produced is increasing faster than many organizations’ ability to review, understand, and secure it.

And this creates a fundamental security problem:

AI can generate code at machine speed, while security validation still frequently happens at human speed.

This gap may become one of the defining application-security challenges of the GenAI era.

---

The New Software Development Model

Traditional software development generally followed a relatively predictable workflow:

Developers understood much of the code because they had written it.

Today, that workflow is changing.

A modern development process increasingly looks like:


The amount of code a developer personally writes can therefore decrease while the amount of code the developer is responsible for increases.

That distinction matters enormously for security.

---

Productivity Is Not the Problem

The argument should not be that organizations should stop using AI for software development.

That would be unrealistic and counterproductive.

Generative AI provides enormous benefits.

Faster Development

AI assistants can quickly produce boilerplate, helper functions, APIs, schemas, test cases, documentation, and infrastructure configurations.

Lower Barrier to Software Creation

People who previously required significant software-development experience can now build functional applications.

This enables innovation.

It also creates a new category of developers who may understand the business logic of what they are building without necessarily understanding every security implication of the generated implementation.

Faster Prototyping

Organizations can validate ideas much more quickly.

An application prototype that previously required weeks may now be created in days—or sometimes hours.

Developer Assistance

Experienced developers also benefit enormously from AI.

AI can help:

• Explain unfamiliar code
• Refactor complex logic
• Generate unit tests
• Translate between programming languages
• Produce documentation
• Identify basic implementation mistakes
• Accelerate debugging

The technology is not inherently insecure.

The challenge is what happens when software-generation capability scales much faster than security governance.

---

The Security Scaling Problem

Consider a simplified example.

Imagine an engineering team traditionally produces:

10,000 lines of new or modified code per month

Security teams review that code using:

• Peer review
• SAST
• Dependency scanning
• Penetration testing
• Manual code review

Now introduce AI coding assistants.

The same organization might be able to produce:

50,000
100,000
or even hundreds of thousands of lines

during a comparable period.

The exact numbers are less important than the relationship.

Development capacity increases.

But did the organization simultaneously increase:

Security reviewers ×5?

Threat modeling ×5?

Manual code review ×5?

Penetration testing ×5?

Architecture review ×5?

Usually not.

This creates what we can call the AI Security Review Gap.

As the distance between those lines grows, so does the amount of code reaching production without meaningful security scrutiny.

AI-Generated Code Is Still Software

There is another dangerous assumption appearing in development environments:

“The AI generated it, therefore it probably knows the correct implementation.”

An LLM does not understand security in the same way an experienced security engineer does.

It generates statistically plausible output based on training data and context.

And training data contains both:

Secure code
+
Average code
+
Legacy code
+
Deprecated patterns
+
Insecure examples
+
Stack Overflow snippets
+
Tutorial code
+
Vulnerable open-source implementations

The result can look extremely professional while containing subtle weaknesses.

That combination is particularly dangerous.

Badly written code attracts attention.

AI-generated code may instead be:

• Clean
• Well formatted
• Documented
• Syntactically correct
• Architecturally plausible
• Functional

while still containing exploitable security assumptions.

---

The Problem of Security Hallucination

Most discussions around LLM hallucination focus on incorrect facts.

In software development, there is another useful concept:

security hallucination.

An AI assistant may produce a solution that looks secure without actually creating the intended security property.

For example, generated code may:

• Validate input but miss an alternate encoding
• Implement authentication but omit authorization
• Use encryption with inappropriate key handling
• Add JWT validation but fail to verify critical claims
• Create SQL parameterization in one path while leaving another vulnerable
• Introduce server-side request functionality without destination restrictions
• Trust client-controlled object identifiers
• Generate unsafe deserialization logic
• Expose excessive API functionality
• Disable verification temporarily during troubleshooting
• Hard-code credentials during prototyping
• Misconfigure cloud permissions

The code might work perfectly.

That is exactly the problem.

Functional correctness and security correctness are not the same thing.

---

Vulnerability Multiplication at Scale

Historically, one developer might unknowingly introduce one insecure coding pattern into one application.

AI changes the potential multiplier.

If an insecure pattern is accepted by a coding assistant, it can potentially be reproduced:

One insecure pattern
        ↓
AI recommendation
        ↓
Multiple functions
        ↓
Multiple repositories
        ↓
Multiple projects
        ↓
Organization-wide exposure

The security problem therefore becomes less about a single vulnerable function and more about propagation.

This becomes even more significant when organizations create:

• Internal coding copilots
• Organization-specific RAG systems
• AI development templates
• Fine-tuned coding models
• Agentic development pipelines
• Automated code-generation platforms

A flawed internal pattern may become part of the organization’s AI knowledge source.

Once that happens, AI may repeatedly recommend the same insecure architecture.

---

And Then We Add GenAI Into the Application Itself

So far we have discussed AI writing applications.

But organizations are simultaneously putting AI inside applications.

This introduces an entirely new attack surface.

Modern GenAI applications may contain

Traditional application-security boundaries become significantly more complicated.

The model may consume information from:

• Users
• Databases
• Documents
• Websites
• Email
• API responses
• Images
• Audio
• Vector databases
• MCP servers
• Other agents
• Persistent memory

And its outputs may trigger actions against other systems.

Take Action

If your organization is:

• Developing applications using AI coding assistants
• Building GenAI-enabled products
• Implementing RAG
• Deploying AI agents
• Integrating MCP
• Allowing AI access to internal systems
• Using AI to generate significant amounts of production code

then traditional application-security testing alone may no longer represent your complete attack surface.

LeetProtect can help assess the application, the AI layer, the code, the architecture, and the attack paths connecting them.

The objective is simple:

Build with AI at speed. Test it with the mindset of an attacker before someone else does.

---

References

1. OWASP GenAI Security Project, OWASP Top 10 for LLM Applications 2026
   https://genai.owasp.org/resource/owasp-genai-llm-top-10-2026/

---

Author: Moe Khalilov
LeetProtect