CmdCaliper: A Semantic-Aware Command-Line Embedding Model and Dataset for Security Research
Sian-Yao Huang, Chenglin Yang, Che-Yu Lin, Chun‐Ying Huang · 2024
This research addresses command-line embedding in cybersecurity, a field obstructed by the lack of comprehensive datasets due to privacy and regulation concerns.We propose the first dataset of similar command lines, named CyPHER 1 , for training and unbiased evaluation.The training set is generated using a set of large language models (LLMs) comprising 28,520 similar command-line pairs.Our testing dataset consists of 2,807 similar command-line pairs sourced from authentic command-line data.In addition, we propose a command-line embedding model named CmdCaliper, enabling the computation of semantic similarity with command lines.Performance evaluations demonstrate that the smallest version of Cmd-Caliper (30 million parameters) suppresses state-of-the-art (SOTA) sentence embedding models with ten times more parameters across various tasks (e.g., malicious command-line detection and similar command-line retrieval).Our study explores the feasibility of data generation using LLMs in the cybersecurity domain.Furthermore, we release our proposed command-line dataset, embedding models' weights and all program codes to the public.This advancement paves the way for more effective command-line embedding for future researchers.