Addressing the use of AI in cybersecurity training

In previous posts, I've often talked about how training providers and academic institutes need to level themselves up to impart skills at a higher cognitive level, so I won't revisit these points. Instead, I'd like to address our use of AI, as a training provider.

The training industry, both academic and professional, has not been spared from the influence of AI

A trend I'm seeing is the increase of AI-enabled clanking training in varying shapes and forms. Educators seem to be pressured into including AI in their materials, sometimes just for the "how do you do, fellow kids" value. Then we have the opposite, where nearly all of the training has been generated by AI. Fundamentally, as customers, we apply the same "expectation" on these vendors, similar to how we prefer our art not to be slop (unless clearly advertised and priced as slop). And again, similar to AI-generated art that's passed off as real human work, the undiscerning consumer might be deceived into buying a product that is AI-generated. That being said, if a training provider indicates that the training is AI-generated, and you knowingly want it and pay for it, that's on you. :)

We use AI to generate useful slop projects to help in training, not replace training in any way

What about our training at counterShell? Currently we do use AI very selectively to fill in the gaps that are directly inconsequential for the training experience, but provide an added dimension that is peripheral to the training. Sound confusing? At least it's not slop. For example, the fundamental Blue-Team technical lessons and takeaways in a hand-crafted training range are 100% human generated. However, the training range comprises of simulated users that are driven by AI, to add realism to the training environment the Blue-Team learner operates in. Another example is in the development of deliberately vulnerable applications. A human (usually me) will craft the vulnerability chain, and the intended series of exploitable bugs that the learner is expected to find. However, it often does not make sense to present the chain on its own, especially if the chain is in a webapp (and there's no webapp). This is where AI comes in, to dress up the vulnerabilities and create a proper app that could plausibly have such vulnerabilities in them.

The chained vulnerabilities here involve websockets and backend SSRF. Perfectly feasible for a (clanked) web-based chat app.

Ultimately, my principle towards AI use in cybersecurity has always been that AI is a tool, just like automated scanners are tools, just like fuzzers are tools. In training development, my views are similar. AI is a tool that can help to drive home the lessons and the training we conduct. AI does not produce the lessons, nor the training experience, and certainly cannot (and should not) be a substitute for the trainer. For anything else that supports the training, we consider using AI to handle the menial tasks that are not directly related to the delivery of the training.

What about CTFs?

Separately, we're also very aware that the old "traditional" model of CTFs is pretty much dead at this point. Participants are going to pay for AI subscriptions to work on CTF problems, and CTF creators are likely to also use AI to generate CTF problems. Ultimately, it descends into a pay-to-win model. We do have some ideas, however, to break this modality and essentially bring back more of the human ingenuity and decision-making skills to the forefront. Or at least, make it a token-losing battle to solely and blindly employ AI to solve challenges. And what better way to achieve these objectives, than by using AI to generate the anti-AI tooling. Perhaps this might be the future of software development, to have deliberate anti-AI tricks and roadblocks, just like how we currently have anti-RE techniques built into legit software and malware alike.

Don't believe me? Come to a SHELLgym and take a look at what we do. It's still free to attend. :)