GitHub Copilot replicating vulnerabilities, insecure code

GitHub Copilot could be creating unintended security issues for customers, according to new research from Snyk.

In a blog post Thursday, Snyk explained that generative AI-powered coding assistants such as GitHub Copilot, which use large language models to suggest code completions to development teams, have a limited understanding of software and merely imitate learned patterns based on the training data.

"It's important to recognize that generative AI coding assistants, like Copilot, don't understand code semantics and, as a result, cannot judge it," wrote Randall Degges, Snyk's head of developer relations and community, in the blog post. "Essentially, the tool mimics code that it previously saw during its training."

As a result, these tools, which have become increasingly popular among developers, can reproduce security issues from customers' existing codebases and open source projects. While coding assistants can be enormously helpful to developers in terms of saving time and increasing productivity, Degges said they also carry significant risk.

"Put simply, when Copilot suggests code, it may inadvertently replicate existing security vulnerabilities and bad practices present in the neighbor files," he wrote. "This can lead to insecure coding practices and open the door to a range of security vulnerabilities."

The blog post described examples in which Snyk researchers used GitHub Copilot's neighboring tabs feature to access files from Snyk's integrated development environment. They asked GitHub Copilot to create SQL queries that match the user input, and the responses included good code.

The researchers then introduced a vulnerable snippet of code into a neighboring tab, creating a new SQL query in the project. When they ran the same request again, GitHub Copilot's response replicated the vulnerable code. "We've just gone from one SQL injection in our project to two, because Copilot has used our vulnerable code as context to learn from," Degges wrote.

GitHub Copilot is billed as a tool that can "improve code quality and security," according to the company's website. GitHub launched a new vulnerability filtering system for the AI coding assistant last week, which is designed to make the tool's code suggestions more secure.

Degges noted in the blog post that the more secure a customer's existing codebase is, the less likely GitHub Copilot is to produce code suggestions with vulnerabilities. However, the tool can amplify existing security debt within a customer's codebase, making it even less secure.

Snyk urged development teams to conduct manual reviews of the code generated by tools such as GitHub Copilot, and to have SAST guardrails and policies in place to identify and fix any issues.

In an interview with TechTarget Editorial, Degges emphasized that GitHub Copilot is a valuable tool and one that he himself uses. However, context is extremely important for secure software development, and he urged organizations to apply reviews and safeguards for AI-generated code because the tools lack that context. "AI coding assistants are amazing, but they have the exact same problems normal human developers have," he said.

Degges also noted that in his experience, most developers probably aren't aware that AI coding assistants can easily replicate existing security issues from users' codebases and open source projects.

"The truth is that large language models today and the AI explosion of the last year and a half are built around generative AI, and in those scenarios, the responses are based on a statistical model," he said. "In this case, there's no underlying knowledge of the actual code. It's all based on probability."

A GitHub spokesperson sent the following statement to TechTarget Editorial:

Security is everyone's responsibility, and GitHub welcomes third-party research to explore the impact of AI tools for software development, like GitHub Copilot. From offering tools like Dependabot for free, to requiring 2FA for all GitHub contributors, to bringing AI and security overview capabilities to GitHub Advanced Security users, we work hard to help our communities build and use trusted and secure code.

To ship secure software, teams must employ safeguards at multiple stages of the SDLC -- from in-editor assistive tools like GitHub Copilot to code reviews by experienced engineers. Code analysis tools like GitHub Advanced Security also play a critical role in identifying and resolving vulnerabilities before they move into production -- and sometimes even mitigating vulnerabilities found in production. Regardless of the tool used, teams cannot and should not depend on any single tool to guarantee the security of their software.

In the code editor, GitHub Copilot leverages a variety of security measures to remove sensitive information in code, block insecure coding patterns, and detect vulnerable patterns in incomplete fragments of code. Specifically, GitHub Copilot applies an AI-based vulnerability prevention system that blocks insecure coding patterns in real-time to make GitHub Copilot suggestions more secure. Our model targets the most common vulnerable coding patterns, including hardcoded credentials, SQL injections, and path injections. This, combined with GitHub Advanced Security's code scanning, secret scanning, and dependency management features, provides an end-to-end experience for developers to secure their software.

Whether writing code by hand, copy-pasting from an adjacent project file, or evaluating a GitHub Copilot suggestion, developers should always exercise caution and sound judgment. Our experiments have shown that GitHub Copilot suggests code of the same or better quality than the average developer. However, we can't guarantee that all suggestions are bug-free. Like any programmer, GitHub Copilot may sometimes suggest insecure code. We recommend taking the same precautions you take with the code written by your engineers (linting, code scanning, IP scanning, etc.).

Rob Wright is a longtime reporter and senior news director for TechTarget Editorial's security team. He drives breaking infosec news and trends coverage. Have a tip? Email him.