An agent skill is a software install wearing a text file’s clothing. Here’s the checking routine I use, sized to what your AI can touch.
I installed a skill this week that teaches Claude to watch YouTube videos. Claude can’t do that on its own, and someone on the internet had built a tidy fix: download the video, pull frames, grab the captions, hand it all to the AI. Free, public, well documented.
If “skill” is a new word for you, here’s the whole concept: a skill is a small packet you download that teaches your AI assistant a new trick. Inside are written instructions plus a few small programs. You drop it in a folder, your AI reads it, and now your AI knows how to do the thing. Anyone on the internet can write one. Hold onto that last sentence, because it’s the entire post.
Before installing it, I read the skill’s instruction file end to end. And I could not tell you whether it was malicious.
That’s not a confession of incompetence; I work with AI systems every day, and advising businesses on them is my job. From the outside, there is simply no way to tell whether an instruction like “proceed silently” means polished design or something being hidden. So before this skill touched my working setup, I ran it through a checking process I’d built for exactly this moment. The process caught things. That’s the story.
A skill is software. Treat the install like one.
Somewhere around 2004, everyone learned not to run a stranger’s .exe. It took a decade of worms, a thousand IT trainings, and at least one relative’s computer riddled with toolbars, but the flinch got installed. An email attachment ending in .exe now triggers a small, healthy fear in basically every working adult.
An agent skill is the same object in a friendlier costume: a folder of instructions you invite your AI to follow, with whatever access your AI already has. If your AI can read your files, so can the skill. If your AI is connected to your email, your calendar, your CRM, the skill’s author is now writing stage directions for something that can touch all of it. And if your team adopted AI tools before you had a policy, they’ll adopt skills the same way: quietly, helpfully, without asking.
The install hides all of this. Think about installing an app on your phone: the phone stops you and asks, “Can this app see your photos? Your location?” You get a moment to say no.
Installing a skill has no moment like that. You drop the folder in, your AI reads it, and the skill is live, with access to everything your AI already touches. Nobody asked you anything.
And it goes one step further. Most skills need extra software to do their job, and the one I vetted came with its own helper program that downloads those extras quietly, without showing you what it’s fetching. Nothing about that is malicious; it’s meant to be convenient. But notice the design choice: it could have shown you the list and waited for your okay. It just didn’t think you needed to see it.
And the stakes aren’t hypothetical. Browse a public skills directory and you’ll find, next to the PDF-mergers and meeting-summarizers, skills that hand an AI a full hacking toolkit — one popular open-source agent autonomously runs more than 150 security tools built for exactly that work. It installs the same way my video-watcher does: same folder, same prompt. Security researchers have already demonstrated compromising Claude Code with deliberately malicious skills. What I haven’t seen is anyone turn that alarm into something a business owner can actually do: a simple checking routine that works without a security team, even if you can’t read a line of code. That’s the gap this post is for.
Why “just ask your AI to check it” fails
The obvious move is to hand the skill to your AI and ask, “is this safe?” I tried the equivalent, and then noticed the structural problem.
Every skill’s instruction file is written as directions addressed to “you,” the AI reading it. The format cannot tell the difference between an AI reviewing the skill and an AI running it. A malicious skill doesn’t need to hack anything to attack its reviewer: the attack is the text, and the reviewer reads the text. If the AI doing the review is connected to your email or files, you’ve hand-delivered the hostile instructions to the exact thing they were written to steer.
My first version of the process had this hole. I’d isolated where the skill could run, but not what my reviewing AI would read. Version two fixed it with a rule that sounds paranoid and isn’t: the AI with access to my systems never reads the skill’s raw files. It only reads reports about them.
Three layers: the more it reads, the less it can touch
Layer 1 is deliberately dumb. Before any AI reads anything, a basic computer command lists every file in the skill’s folder, including hidden ones, which your computer normally keeps out of sight. (For the technically inclined: it’s find, a tool older than the web. Everyone else: you’ll never need to type it.) No AI involved, because a list-maker can’t be sweet-talked.
This step produced the best catch of the whole exercise. My AI reviewer had carefully read twelve files and reported on each one. The complete list said the skill contained eighteen. Six were sitting in hidden folders, and one was set to run automatically every time a session starts — exactly the file you’d most want reviewed. All six turned out to be harmless, but that’s not the point: a reviewer can only judge what lands on its desk, and nothing about “twelve files reviewed” tells you the real number was eighteen. The dumbest tool in the stack caught what the smartest one couldn’t.
Layer 2 is an isolated reader. An AI with no access to anything of mine reads each file and reports what it finds. It can be lied to, but it cannot leak, because there’s nothing to take. (There’s a free, no-setup version of this idea in the checklist at the end.) It flagged the finding that changed the install: a leftover settings file on your computer could silently switch the skill from “runs privately on my machine” to “sends audio to an outside company,” overriding my explicit decision not to allow that. The author almost certainly meant it as a convenience. As shipped, it’s a switch that flips itself. The same layer flagged that a video’s captions get handed to your AI word for word, as if they were part of your conversation — which makes a subtitle track a way for a stranger to slip instructions to your assistant. It’s the same family of trick as the websites that show your AI a different page than they show you: the text your AI reads is a delivery route.
Layer 3 is judgment. Me and my everyday AI, reading the reports, never the raw files. A human has to decide what the evidence means, because some of it is genuinely ambiguous. “Proceed silently” is fine design when the thing kept quiet is a status message. It’s concealment when it’s an upload. The words are identical; only looking at what goes quiet tells you which one you’re holding.
The verdict, and the uncomfortable good news
The skill passed, conditionally: delete the one line of code that hunted for saved keys, never give it account keys at all, install its extra software by hand so its own installer never runs, and treat everything it pulls off the internet as untrusted.
Here’s what makes this bigger than one skill: this was a good one. Named author. A public change log whose security claims checked out against the actual code. Documentation that matched behavior everywhere I looked. And it still shipped a default that would have silently overridden my security decision, because shipped defaults serve the author’s convenience, not your exposure. If the honest ones need trimming before install, the install prompt deserves your flinch every time. It’s the same rule I run my own content system on: a human gate on everything that ships, no exceptions for the well-behaved.
The part you can actually do
You are not going to build a three-layer review bench, and you don’t need to. (I’ve also written about who checks your AI once it’s allowed to act — that’s the after-install question; today is the gate before it.) The real lesson is that the human’s job was never malware detection. The reports turned the skill into questions any business owner can answer, and I answered every one without reading a line of Python:
Does what it can touch match the job it does? A video-watcher needs the internet and a temporary folder. If a video-watcher wants your email, that’s not a feature, that’s a finding.
Where does data go, and am I okay with that? This skill could send audio to an outside transcription service. Not evil, but that’s my decision to make, not a default’s.
How careful you need to be depends on what your AI can reach. If it has no connections and touches nothing but a scratch folder, a careful read is a reasonable floor. If it reads your email, your client files, or anything that moves money, you’ve earned the fuller routine. For everyone, the floor is this:
- Quarantine first. Download the skill to a folder your AI doesn’t read. Don’t install it just to look at it.
- Count what’s in the box. Look at every file, including hidden ones (on a Mac, Cmd-Shift-period in Finder shows them). If the description mentions five files and the folder holds eleven, that’s your first question. Anything set to run by itself deserves an explanation first.
- Read the instruction file with the two questions. What it can touch versus the job it does. Where data goes versus where you’re comfortable with it going. You’re checking fit, not hunting malware.
- Ask an AI that’s holding nothing. Open a fresh chat with every connector and integration turned off, so it has no access to your files, email, or accounts (use a temporary chat if your AI keeps memory). Upload the skill’s files and ask what it does: where data gets sent, what runs automatically, what stays silent. A flag is strong evidence; a clean report is weak evidence. An AI holding nothing can still be fooled, but it can’t lose anything of yours while it’s being fooled.
- Install extras yourself. The skill’s own installer is never the first thing that runs.
- No account keys by default, and distrust what it fetches. Anything the skill pulls from the internet — captions, titles, page text — lands in front of your AI looking exactly like instructions. Point it only at sources you trust.
Twenty years of security pain taught everyone to hesitate before a stranger’s .exe. The install prompt for an AI skill has earned the same hesitation; it just hasn’t collected its decade of horror stories yet. You can wait for those, or you can borrow the flinch now. The flinch is the feature.
Sources and Further Reading
- Skill Issues: Compromising Claude Code with malicious skills & agents — Part 1 — Reversec Labs, May 2026
- Top Claude Skills for Cybersecurity, Hacking, and Vulnerability Scanning — Snyk, 2026
- hexstrike-ai listing — SkillsLLM directory (basis for the 150+ tools figure)