perfecXion/intentguard-legal
Text Classification • Updated
perfecxion.ai PRO
perfecXion
AI & ML interests
Security
Recent Activity
upvoted an article about 12 hours ago
We Fact-Checked Our Own Security Dataset. Here's What We Fixed. reacted to scthornton's post with 🧠 about 12 hours ago
SecureCode update: we went back and fact-checked our own security dataset and corrected what didn't hold up.
The original claim was "complete incident grounding, every example ties to a documented CVE." An adversarial re-audit found that it was overstated: many CVEs were misattributed, and many "incidents" were representative scenarios carrying invented statistics. So we fixed it.
- Grounding: re-verified every reference. Removed 802 misattributed CVEs on the web side, corrected or honestly relabeled the incident narratives, and confirmed the AI/ML conversation CVEs are real (EchoLeak CVE-2025-32711, EmailGPT CVE-2024-5184, and others).
- Fix-correctness: reviewed whether each "secure" example actually eliminates the vulnerability. Removed 28 that did not (a "secure" secret scanner whose entropy check always returned zero, an Angular example still using bypassSecurityTrustHtml, and more).
- Leakage: re-split so near-duplicates stay on one side. Test contamination went from 11.6% to zero.
- Viewer, schema, and metadata: rebuilt as parquet under a shared schema. All three viewers are live.
- Models: retrained the whole family on the corrected data so the fix reaches the weights, not just the cards. Now ten open models (3B to 26B), including two new Gemma 4 variants, refreshed locally on a DGX Spark GB10. The paper (arXiv:2512.18542) was revised to match.
Counts moved from 2,185 to 2,372 unified (web 1,625 + AI/ML 747). A slightly smaller, fully-checked dataset beats a larger one you have to take on faith. Full writeup and links in the article.
Datasets: scthornton/securecode, scthornton/securecode-web, scthornton/securecode-aiml
reacted to scthornton's post with ❤️ about 12 hours ago
SecureCode update: we went back and fact-checked our own security dataset and corrected what didn't hold up.
The original claim was "complete incident grounding, every example ties to a documented CVE." An adversarial re-audit found that it was overstated: many CVEs were misattributed, and many "incidents" were representative scenarios carrying invented statistics. So we fixed it.
- Grounding: re-verified every reference. Removed 802 misattributed CVEs on the web side, corrected or honestly relabeled the incident narratives, and confirmed the AI/ML conversation CVEs are real (EchoLeak CVE-2025-32711, EmailGPT CVE-2024-5184, and others).
- Fix-correctness: reviewed whether each "secure" example actually eliminates the vulnerability. Removed 28 that did not (a "secure" secret scanner whose entropy check always returned zero, an Angular example still using bypassSecurityTrustHtml, and more).
- Leakage: re-split so near-duplicates stay on one side. Test contamination went from 11.6% to zero.
- Viewer, schema, and metadata: rebuilt as parquet under a shared schema. All three viewers are live.
- Models: retrained the whole family on the corrected data so the fix reaches the weights, not just the cards. Now ten open models (3B to 26B), including two new Gemma 4 variants, refreshed locally on a DGX Spark GB10. The paper (arXiv:2512.18542) was revised to match.
Counts moved from 2,185 to 2,372 unified (web 1,625 + AI/ML 747). A slightly smaller, fully-checked dataset beats a larger one you have to take on faith. Full writeup and links in the article.
Datasets: scthornton/securecode, scthornton/securecode-web, scthornton/securecode-aiml
Organizations
