etc

How Is AI Changing Bug Bounties? (from DEF CON 34)

How Is AI Changing Bug Bounties? (from DEF CON 34)
DEF CON 34, Las Vegas

Notes from the “Navigating AI-Assisted Submissions” panel talk

Hello, this is Seokchan Yoon writing.

At DEF CON 34’s Bug Bounty Village, I attended a panel talk titled “Navigating AI-Assisted Submissions.” I sought out the session because I wanted to hear directly from platform operators about what their triage teams have been dealing with since AI entered the workflow.

The panel talk brought together representatives from five competing bug bounty platforms:

  • HackerOne — Tony Lee
  • Synack — Eddie Rios
  • Bugcrowd — Michael Skelton
  • Intigriti — Alexander Wren
  • YesWeHack — Selim Jaafar

What struck me first was that none of the five companies treated AI itself as the root of the problem. In fact, they saw real value in AI helping researchers find serious vulnerabilities faster than before. The trouble begins after discovery. Validation and remediation still move at human speed, while AI has dramatically accelerated the rate at which findings arrive. Much of the panel talk therefore focused on the consequences of that gap: burnout, report quality, and rewards.

One line from the panel talk captured the situation particularly well:

AI is not a problem. It's just a new reality for us as an industry.

AI is not a passing trend that the bug bounty industry can simply wait out. It is the new environment the industry will have to keep adapting to.

1/ More AI Slop, More Submissions, and Triage Burnout

The moderator began by asking how triage had changed over the past year. The first answer was not submission volume or AI slop. It was burnout. Triage teams are physically and mentally exhausted just trying to keep up with the growing queue of reports.

During the panel talk, the speakers shared several internal figures:

  • Total submissions have roughly doubled year over year.
  • The percentage of valid reports has not changed significantly.
  • Confirmed Critical vulnerabilities have roughly tripled year over year in recent months.

The important metric here is not the percentage of valid reports, but the number of them. Even if the ratio of valid to invalid reports remains unchanged, doubling the overall submission volume still means twice as many reports for triage teams to review. AI has increased not only low-quality submissions but genuine vulnerabilities as well, so there is no corresponding reduction in workload.

Public data shows a similar pattern. In a May 2026 report, HackerOne said submission volume had risen by 76% over 12 months and peaked in March 2026. The proportion of valid reports remained relatively steady throughout the surge. Elastic likewise reported that submissions, which had typically ranged from 600 to 850 per year, had already exceeded 1,390 in the first half of 2026. Bugcrowd said its triage queue grew by 334% in just three weeks.

The averages from large platforms tell only part of the story. Smaller open source projects have had a very different experience. curl, for example, said that roughly one in six reports was valid through early 2025, but by the end of the year that figure had fallen to one in every 20 to 30 reports. The project cited this burden when it ended its bug bounty program on January 31, 2026. The same increase in submissions affects a large platform with dedicated staff very differently from a small open source project.

The panelists did not place all the responsibility on researchers. Researchers need to validate vulnerabilities before submitting them, but platforms must also help valid reports reach their customers effectively. Programs, meanwhile, need to be specific about the vulnerabilities they want—and those they do not. Researchers, platforms, and programs all need to adapt to the post-AI environment.

2/ Another Form of AI Slop: Needlessly Long Reports

The next topic was how to define AI slop. The moderator asked whether Low- or Medium-severity vulnerabilities that would have been valuable a few years ago still carry the same value today. As AI makes it possible to find large numbers of similar issues, some programs have stopped accepting Low and Medium reports altogether.

One panelist questioned whether slop was even the right word. Slop implies garbage, but not every report arriving today is wrong. In many cases, the vulnerability is valid, yet an LLM has buried it under so much unnecessary explanation that the actual finding is difficult to locate. These reports are less like garbage and more like needlessly inflated reports.

Another panelist added that the definition of slop varies by program. Some programs want to review every Medium vulnerability, while others will close an entire class of them. Instead of imposing one universal standard, platforms need to help each program communicate the vulnerabilities it wants and its current business priorities. A program should go beyond listing scope and exclusions; sharing the bug classes and areas it currently cares about can reduce unnecessary submissions.

When asked whether Low and Medium findings should continue to receive rewards, the answer ultimately came down to budget. Bug bounty programs compete with other security solutions for funding. If AI reduces the cost of discovering a certain class of vulnerability, the reward for that work is likely to fall as well.

Google has already moved in this direction. In 2026, it removed monetary rewards and credits from the lower tiers of its OSS VRP and changed the rules to prioritize vulnerability classes that are difficult for AI to find. Later in the panel talk, one speaker compared AI to a factory that mass-produces bread. Google’s policy reflects the same shift in value.

Three Types of AI Slop

Taken together, the panelists’ comments described three broad types of AI slop currently reaching platforms.

The first is the hallucinated report. The vulnerability does not actually exist, but the description makes it sound Critical. Instead of the researcher demonstrating that the issue works, the triage team must prove that it does not. The higher the cost of a false judgment for a program, the more time this kind of validation requires.

The second is the overly long report. The vulnerability itself is valid, but the explanation is so long that the actual impact and reproduction steps are hard to find. The details that matter are buried in dozens of paragraphs seemingly written to satisfy developers, security teams, and product managers all at once.

The third is the automatically submitted report. Someone with little hacking experience hears that they can “run an agent and make money,” then connects an automated tool directly to a live program. One panelist described a person who created 50 to 60 accounts; one of those accounts submitted nothing but robots.txt.

Bugcrowd has announced policy changes targeting accounts that repeatedly submit invalid reports or engage in submission farming. HackerOne has also announced that identity verification will become mandatory across its bug bounty programs beginning August 14, 2026. AI slop is no longer just a community debate—it is changing platform policy.

3/ How AI Is Flattening the Voice of Vulnerability Reports

One of the observations I found most interesting was how AI is changing the voice of vulnerability reports.

One panelist compared reports from a year ago with those arriving today. Researchers used to center their reports on the attack they could carry out with a vulnerability. Now, many feed their findings into Claude or ChatGPT and ask it to write the report. Different researchers consequently produce documents with nearly identical language and structure, while the actual attack scenario that the program owner wants to understand is often missing.

In the past, each report revealed something of the researcher’s personality and thought process. Today, reports from many different researchers can feel as though they were all written by the same AI. The point was simple: triage teams should not feel as if every researcher they encounter is the same chatbot.

A long report is not necessarily a good report. In the past, however, writing at length at least required a corresponding investment of human time and effort. Today, an LLM can generate dozens of paragraphs from a single request to “make it detailed.” The panelists were not telling researchers to avoid AI. They were saying that researchers must remain in control of what goes into the report.

Their shared recommendations were straightforward:

  1. Put the real-world impact at the beginning of the report.
  2. Keep only the steps and evidence needed to reproduce the issue.
  3. Remove explanations the triage team already knows, such as generic definitions of the vulnerability class.
  4. Review any severity rating selected by AI.
  5. Read the entire report yourself before submitting it.

Letting AI choose severity has created its own problems. In one case, a researcher found an RCE and obtained a shell, then complained that the report had not been prioritized. The report-generation tool, however, had classified it as Informational. The opposite is becoming more common too: findings with limited impact are labeled Critical. High and Critical reports receive expedited review, but once their rating is lowered during validation, they return to the general queue. Inflating severity provides researchers with little practical benefit.

Someone also suggested that platforms publish standard report templates or instructions researchers could provide to AI. One panelist said their team was already building a tool that specifies the minimum evidence required for each vulnerability class, with plans to release it on GitHub and develop it into an industry standard. The goal was to establish a shared baseline for quality before every platform builds its own report-writing AI.

AI has also brought clear benefits. Researchers who are not native English speakers can use it for translation and editing, significantly lowering the language barrier. It also reduces the time needed to write an English report after finding a serious vulnerability. What concerned the panelists was not the use of AI itself, but the decision to outsource the researcher’s judgment to it.

4/ Dividing Responsibility Between Researchers and Platforms

The moderator framed the next question from the perspective of an “anchor hacker”—a researcher who has worked on the same program for a long time. Even these researchers are often expected to explain the entire context of a customer’s environment with every finding, as if addressing someone who has never seen it before. Could a platform that knows the customer better take some of that burden away?

The platforms’ answer was clear. A researcher’s job is to analyze the target, find a vulnerability, reproduce it, and demonstrate that it is real. When appropriate, the researcher can also show exploitability within safe boundaries. Translating that result into language the customer understands—and helping researchers and program owners speak from the same context—is the platform’s responsibility.

That did not mean researchers could submit an AI-generated draft without reviewing it. One panelist offered a simple test: if your attention starts to drift while reading the report, or you do not want to finish it, it is not a good report. A researcher cannot send a document to a triage team if even its author has not read it to the end.

AI slop can continue long after the initial submission. After a customer lowers the severity, for example, a researcher may use AI to produce hundreds of paragraphs arguing about IDOR. The panelists said they did not want to debate an AI. They wanted to speak directly with the researcher and reach a conclusion both sides could understand.

At that point, another panelist joked that perhaps the platform’s AI and the researcher’s AI should simply be left to fight each other. Someone countered that if similar models handle both writing and review, the result might be little different from saying, “My friend wrote it, so it must be fine.” The exchange was playful, but it highlighted a real risk: the bias that can arise when AI-generated output is evaluated only by another AI.

The panelists were not asking researchers to become exceptional writers. If someone submits 100 reports overnight, it is unlikely that a human carefully reviewed each one. In that situation, the researcher should slow down and reread the reports from the customer’s or triager’s perspective. They also suggested going beyond asking AI to find vulnerabilities: use a separate agent to challenge the finding and try to disprove it.

I take a similar approach in my own LLM-based vulnerability discovery pipeline. I use a dedicated validation agent, and if a finding cannot pass that agent’s checks, I do not treat it as a security issue.

For the final report, put the impact first and delete anything that does not help explain the impact or reproduce the issue. The panelists also recommended studying a program deeply enough to find High and Critical vulnerabilities rather than mass-submitting Low and Medium findings with AI. Researchers who want AI to retain their own voice can provide examples of reports they wrote themselves. The amount of explanation should ultimately depend on whether the intended reader is a security team, a developer, or a nontechnical stakeholder.

The panelists added that even this advice has a short shelf life. Had they been asked the same questions three months earlier, their answers might have been different; now, the situation seems to change every three days. The platforms do not have a finished answer either. They are working it out alongside the community.

5/ New Bug Classes Made Practical by AI

The first half of the panel talk focused on AI slop and the difficulties facing triage teams. The second half took a more optimistic turn, exploring work that AI has newly made possible.

The first opportunity is vulnerability research at a scale that humans could not previously manage by hand. In the past, testing every endpoint in a new program for authorization flaws took a great deal of time. Those vulnerabilities might therefore trickle in over the course of a year. Now, hundreds can be found in the first week after a program opens. The time required to examine the same surface has effectively fallen from a year to a week.

The second opportunity lies in the gap between extensive product documentation and actual implementation. Many companies build systems around the documentation of services such as Atlassian, yet no person can manually verify that every documented behavior matches reality. Agents can now compare documentation and implementation at scale, turning discrepancies into a new source of vulnerabilities.

The growing use of AI-generated code creates opportunities for researchers as well. Modern services increasingly contain code written by LLMs with nondeterministic output, along with hastily connected third-party services. That may be unsettling for developers, but for AppSec and bug bounty researchers it means the attack surface keeps expanding. The panelists expect both the vulnerabilities we look for and the methods we use to find them to change.

Factory Bread and the Artisan Loaf

One panelist compared AI to a factory that mass-produces bread, and researchers to artisans who bake it by hand.

Factory-made bread can be supplied cheaply to many people. In much the same way, AI will quickly produce large numbers of vulnerabilities that are easy to find. That does not eliminate the value of bread made by a skilled artisan. There will still be demand for what a factory cannot make.

Let AI bake a million loaves. Your job is to find the one thing AI could not. That is the masterpiece.

For a hacker, that masterpiece might be an access-control vulnerability AI missed because it did not know where to look. It might be a flaw visible only after connecting several distant clues, or a business-logic vulnerability that requires years of familiarity with the program.

The panelists then shared a fitting example. A French hacking group had researched the same program for four or five years. After introducing AI into its workflow, the team found five Critical vulnerabilities in two weeks—issues that both the team and other researchers had missed throughout the previous five years.

This was less a case of AI replacing expertise than of AI helping researchers make better use of expertise they had already built. Five years of program knowledge, combined with AI’s ability to explore, uncovered vulnerabilities that had previously gone unnoticed. A researcher using the same tool without that underlying understanding would have struggled to produce the same result.

The panelists said that one person can now conduct much deeper research in a short period of time. Vulnerabilities that once appeared only every few weeks are arriving more frequently, and complex issues are being discovered faster. That is welcome news for researchers, but a heavy burden for the defensive teams that must validate and remediate their findings.

They also mentioned areas that AI cannot yet handle directly, such as hardware. An AI cannot disassemble equipment or solder components. Researchers who expand from the web into mobile, firmware, or hardware may find fields that remain less crowded. I took this as a reminder that when AI handles repetitive work, researchers can use the time it gives back to learn something new.

6/ Three Practical Questions from the Q&A

Collaboration Between Competing Platforms

The first question asked whether competing platforms such as HackerOne, Bugcrowd, and Intigriti actually collaborate on the problem of AI slop.

The panelists said their companies obviously compete at a business level, but their operations and community teams share a considerable amount of information. One speaker joked, “Let the sales teams do the competing.” That got a laugh. The teams meet at events like this one to exchange examples and responses, and when one platform finds an effective approach, the others learn from it.

Unlike sales organizations, operations and community teams are relatively removed from revenue targets and share the goal of solving problems for the community. That may be why five competing companies could speak so candidly about the same issues on one stage.

AI Dependence and the Researcher’s Skills

The second question raised the concern that continued reliance on AI could weaken researchers’ skills.

The panelists responded that the role humans must play has actually become clearer in the AI era. AI can carry out an attack, but it cannot determine what is reasonable or permitted. This year, an agent compromised a target that a person would have known not to touch, while the researcher running the tool did not realize what actions it had taken.

When platforms contact people involved in incidents like these, they usually find no deliberate intent to violate the code of conduct. The researchers simply did not understand the extent of the requests and noise their tools were generating. Once the problem was explained, they apologized and adjusted their workflows. Whether the tool is a new AI model or a traditional vulnerability scanner, the user remains responsible for controlling it.

One panelist described March 2026 as an inflection point, when AI’s hacking capability visibly improved and submission volume surged. HackerOne’s public data likewise shows submissions peaking that month. Bugcrowd announced its policy for handling slop on March 10, and HackerOne stopped accepting new submissions to the Internet Bug Bounty on March 27.

Others noted that the industry has seen similar shifts before. When vulnerability scanners first became widespread, users submitted every detection as a vulnerability. Platforms such as Hack The Box and TryHackMe lowered the barrier to entering security, but also increased competition. AI is changing the skills researchers need in the same way, and those who keep learning will adapt to the new environment.

Unilateral Severity Downgrades After Triage

The final question was not about AI, but about rewards. A researcher with seven years of experience said they were increasingly seeing Critical reports receive the same rating from triage, only for the customer to lower them to B2 or B3 and stop communicating. Support would say the customer’s decision was final, while reconnecting with the triage team that had already made its assessment was difficult.

The platforms began by acknowledging that they did not have a clear solution. As submission volume has surged, triage teams, support teams, and program managers have all become busier. Communication with researchers has suffered as a result.

In principle, the same vulnerability should receive the same result regardless of who reviews it. In practice, the outcome can vary with the customer’s context and judgment. The way a customer interprets a submission may affect the rating; conversely, internal mitigations unknown to the researcher may make the actual risk lower than it appears.

Another panelist said companies are only now recognizing that the increase in submissions is not temporary. Programs previously handled by two people may now require three, so organizations are beginning to reallocate staff and redesign their processes. Some customers do habitually downgrade findings, but a well-run program has a clear incentive to reward vulnerabilities according to their actual risk. The Q&A ended on that note.

Public data suggests this is more than a matter of researcher frustration. HackerOne reported a 21-fold increase in validated but unresolved vulnerabilities and a 25-fold increase in unresolved Critical vulnerabilities. The remediation rate for Critical findings fell from more than 83% to less than 40%. Discovery is accelerating far faster than remediation.

7/ When Validation and Remediation Become Harder Than Discovery

After listening to the panel talk, I came away thinking that AI’s biggest effect on bug bounties has been to expose the bottlenecks after a vulnerability is found.

Finding a serious vulnerability used to be the hardest part. AI now enables more people to uncover complex vulnerabilities at greater speed. Reproducing those findings, determining their real-world impact, communicating them to customers, fixing them, and assigning appropriate rewards still move at the pace of people and organizations. That helps explain why submissions have doubled while the backlog has grown 21-fold.

AI has not merely increased low-quality reports. The number of confirmed Critical vulnerabilities has risen as well, and issues that nobody found in five years have been uncovered in two weeks. Researchers spend less time writing reports in English, and can now attempt research that was previously impractical because of its scale.

What matters, then, is not whether a researcher uses AI, but how much judgment they hand over to it. AI can handle repetitive exploration, translation, and first drafts. The researcher still needs to define the boundaries of an attack, reproduce the vulnerability, assess its severity and real-world impact, and read the final report.

I would summarize the final advice from the panel talk in one sentence:

If I would not do it myself, I should not ask my AI to do it either.

As someone who also uses LLMs to find vulnerabilities, I left the session thinking less about how to produce more findings and more about how far I can trust those findings—and how I should validate them.

References

Read next