1{2 "document_id": "lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb",3 "family": "public_qasper",4 "split": "dev",5 "text": "Introduction\nMicroblogging such as Twitter and Weibo is a popular social networking service, which allows users to post messages up to 140 characters. There are millions of active users on the platform who stay connected with friends. Unfortunately, spammers also use it as a tool to post malicious links, send unsolicited messages to legitimate users, etc. A certain amount of spammers could sway the public opinion and cause distrust of the social platform. Despite the use of rigid anti-spam rules, human-like spammers whose homepages having photos, detailed profiles etc. have emerged. Unlike previous \"simple\" spammers, whose tweets contain only malicious links, those \"smart\" spammers are more difficult to distinguish from legitimate users via content-based features alone BIBREF0 .\nThere is a considerable amount of previous work on spammer detection on social platforms. Researcher from Twitter Inc. BIBREF1 collect bot accounts and perform analysis on the user behavior and user profile features. Lee et al. lee2011seven use the so-called social honeypot by alluring social spammers' retweet to build a benchmark dataset, which has been extensively explored in our paper. Some researchers focus on the clustering of urls in tweets and network graph of social spammers BIBREF2 , BIBREF3 , BIBREF4 , BIBREF5 , showing the power of social relationship features.As for content information modeling, BIBREF6 apply improved sparse learning methods. However, few studies have adopted topic-based features. Some researchers BIBREF7 discuss topic characteristics of spamming posts, indicating that spammers are highly likely to dwell on some certain topics such as promotion. But this may not be applicable to the current scenario of smart spammers.\nIn this paper, we propose an efficient feature extraction method. In this method, two new topic-based features are extracted and used to discriminate human-like spammers from legitimate users. We consider the historical tweets of each user as a document and use the Latent Dirichlet Allocation (LDA) model to compute the topic distribution for each user. Based on the calculated topic probability, two topic-based features, the Local Outlier Standard Score (LOSS) which captures the user's interests on different topics and the Global Outlier Standard Score (GOSS) which reveals the user's interests on specific topic in comparison with other users', are extracted. The two features contain both local and global information, and the combination of them can distinguish human-like spammers effectively.\nTo the best of our knowledge, it is the first time that features based on topic distributions are used in spammer classification. Experimental results on one public dataset and one self-collected dataset further validate that the two sets of extracted topic-based features get excellent performance on human-like spammer classification problem compared with other state-of-the-art methods. In addition, we build a Weibo dataset, which contains both legitimate users and spammers.\nTo summarize, our major contributions are two-fold:\nIn the following sections, we first propose the topic-based features extraction method in Section 2, and then introduce the two datasets in Section 3. Experimental results are discussed in Section 4, and we conclude the paper in Section 5. Future work is presented in Section 6.\nMethodology\nIn this section, we first provide some observations we obtained after carefully exploring the social network, then the LDA model is introduced. Based on the LDA model, the ways to obtain the topic probability vector for each user and the two topic-based features are provided.\nObservation\nAfter exploring the homepages of a substantial number of spammers, we have two observations. 1) social spammers can be divided into two categories. One is content polluters, and their tweets are all about certain kinds of advertisement and campaign. The other is fake accounts, and their tweets resemble legitimate users' but it seems they are simply random copies of others to avoid being detected by anti-spam rules. 2) For legitimate users, content polluters and fake accounts, they show different patterns on topics which interest them.\nLegitimate users mainly focus on limited topics which interest him. They seldom post contents unrelated to their concern.\nContent polluters concentrate on certain topics.\nFake accounts focus on a wide range of topics due to random copying and retweeting of other users' tweets.\nSpammers and legitimate users show different interests on some topics e.g. commercial, weather, etc.\nTo better illustrate our observation, Figure. 1 shows the topic distribution of spammers and legitimate users in two employed datasets(the Honeypot dataset and Weibo dataset). We can see that on both topics (topic-3 and topic-11) there exists obvious difference between the red bars and green bars, representing spammers and legitimate users. On the Honeypot dataset, spammers have a narrower shape of distribution (the outliers on the red bar tail are not counted) than that of legitimate users. This is because there are more content polluters than fake accounts. In other word, spammers in this dataset tend to concentrate on limited topics. While on the Weibo dataset, fake accounts who are interested in different topics take large proportion of spammers. Their distribution is more flat (i.e. red bars) than that of the legitimate users. Therefore we can detect spammers by means of the difference of their topic distribution patterns.\nLDA model\nBlei et al.blei2003latent first presented Latent Dirichlet Allocation(LDA) as an example of topic model.\nEach document $i$ is deemed as a bag of words $W=\\left\\lbrace w_{i1},w_{i2},...,w_{iM}\\right\\rbrace $ and $M$ is the number of words. Each word is attributable to one of the document's topics $Z=\\left\\lbrace z_{i1},z_{i2},...,z_{iK}\\right\\rbrace $ and $K$ is the number of topics. $\\psi _{k}$ is a multinomial distribution over words for topic $k$ . $\\theta _i$ is another multinomial distribution over topics for document $i$ . The smoothed generative model is illustrated in Figure. 2 . $\\alpha $ and $W=\\left\\lbrace w_{i1},w_{i2},...,w_{iM}\\right\\rbrace $0 are hyper parameter that affect scarcity of the document-topic and topic-word distributions. In this paper, $W=\\left\\lbrace w_{i1},w_{i2},...,w_{iM}\\right\\rbrace $1 , $W=\\left\\lbrace w_{i1},w_{i2},...,w_{iM}\\right\\rbrace $2 and $W=\\left\\lbrace w_{i1},w_{i2},...,w_{iM}\\right\\rbrace $3 are empirically set to 0.3, 0.01 and 15. The entire content of each Twitter user is regarded as one document. We adopt Gibbs Sampling BIBREF8 to speed up the inference of LDA. Based on LDA, we can get the topic probabilities for all users in the employed dataset as: $W=\\left\\lbrace w_{i1},w_{i2},...,w_{iM}\\right\\rbrace $4 , where $W=\\left\\lbrace w_{i1},w_{i2},...,w_{iM}\\right\\rbrace $5 is the number of users. Each element $W=\\left\\lbrace w_{i1},w_{i2},...,w_{iM}\\right\\rbrace $6 is a topic probability vector for the $W=\\left\\lbrace w_{i1},w_{i2},...,w_{iM}\\right\\rbrace $7 document. $W=\\left\\lbrace w_{i1},w_{i2},...,w_{iM}\\right\\rbrace $8 is the raw topic probability vector and our features are developed on top of it.\nTopic-based Features\nUsing the LDA model, each person in the dataset is with a topic probability vector $X_i$ . Assume $x_{ik}\\in X_{i}$ denotes the likelihood that the $\\emph {i}^{th}$ tweet account favors $\\emph {k}^{th}$ topic in the dataset. Our topic based features can be calculated as below.\nGlobal Outlier Standard Score measures the degree that a user's tweet content is related to a certain topic compared to the other users. Specifically, the \"GOSS\" score of user $i$ on topic $k$ can be calculated as Eq.( 12 ):\n$$\\centering \\begin{array}{ll} \\mu \\left(x_{k}\\right)=\\frac{\\sum _{i=1}^{n} x_{ik}}{n},\\\\ GOSS\\left(x_{ik}\\right)=\\frac{x_{ik}-\\mu \\left(x_k\\right)}{\\sqrt{\\underset{i}{\\sum }\\left(x_{ik}-\\mu \\left(x_{k}\\right)\\right)^{2}}}. \\end{array}$$ (Eq. 12)\nThe value of $GOSS\\left(x_{ik}\\right)$ indicates the interesting degree of this person to the $\\emph {k}^{th}$ topic. Specifically, if $GOSS\\left(x_{ik}\\right)$ > $GOSS\\left(x_{jk}\\right)$ , it means that the $\\emph {i}^{th}$ person has more interest in topic $k$ than the $\\emph {j}^{th}$ person. If the value $GOSS\\left(x_{ik}\\right)$ is extremely high or low, the $\\emph {i}^{th}$ person showing extreme interest or no interest on topic $k$ which will probably be a distinctive pattern in the fowllowing classfication. Therefore, the topics interested or disliked by the $\\emph {k}^{th}$0 person can be manifested by $\\emph {k}^{th}$1 , from which the pattern of the interested topics with regarding to this person is found. Denote $\\emph {k}^{th}$2 our first topic-based feature, and it hopefully can get good performance on spammer detection.\nLocal Outlier Standard Score measures the degree of interest someone shows to a certain topic by considering his own homepage content only. For instance, the \"LOSS\" score of account $i$ on topic $k$ can be calculated as Eq.( 13 ):\n$$\\centering \\begin{array}{ll} \\mu \\left(x_{i}\\right)=\\frac{\\sum _{k=1}^{K} x_{ik}}{K},\\\\ LOSS\\left(x_{ik}\\right)=\\frac{x_{ik}-\\mu \\left(x_i\\right)}{\\sqrt{\\underset{k}{\\sum }\\left(x_{ik}-\\mu \\left(x_{i}\\right)\\right)^{2}}}. \\end{array}$$ (Eq. 13)\n$\\mu (x_i)$ represents the averaged interesting degree for all topics with regarding to $\\emph {i}^{th}$ user and his tweet content. Similarly to $GOSS$ , the topics interested or disliked by the $\\emph {i}^{th}$ person via considering his single post information can be manifested by $f_{LOSS}^{i}=[LOSS(x_{i1})\\cdots LOSS(x_{iK})]$ , and $LOSS$ becomes our second topic-based features for the $\\emph {i}^{th}$ person.\nDataset\nWe use one public dataset Social Honeypot dataset and one self-collected dataset Weibo dataset to validate the effectiveness of our proposed features.\nSocial Honeypot Dataset: Lee et al. lee2010devils created and deployed 60 seed social accounts on Twitter to attract spammers by reporting back what accounts interact with them. They collected 19,276 legitimate users and 22,223 spammers in their datasets along with their tweet content in 7 months. This is our first test dataset.\nOur Weibo Dataset: Sina Weibo is one of the most famous social platforms in China. It has implemented many features from Twitter. The 2197 legitimate user accounts in this dataset are provided by the Tianchi Competition held by Sina Weibo. The spammers are all purchased commercially from multiple vendors on the Internet. We checked them manually and collected 802 suitable \"smart\" spammers accounts.\nPreprocessing: Before directly performing the experiments on the employed datasets, we first delete some accounts with few posts in the two employed since the number of tweets is highly indicative of spammers. For the English Honeypot dataset, we remove stopwords, punctuations, non-ASCII words and apply stemming. For the Chinese Weibo dataset, we perform segmentation with \"Jieba\", a Chinese text segmentation tool. After preprocessing steps, the Weibo dataset contains 2197 legitimate users and 802 spammers, and the honeypot dataset contains 2218 legitimate users and 2947 spammers. It is worth mentioning that the Honeypot dataset has been slashed because most of the Twitter accounts only have limited number of posts, which are not enough to show their interest inclination.\nEvaluation Metrics\nThe evaluating indicators in our model are show in 2 . We calculate precision, recall and F1-score (i.e. F1 score) as in Eq. ( 19 ). Precision is the ratio of selected accounts that are spammers. Recall is the ratio of spammers that are detected so. F1-score is the harmonic mean of precision and recall.\n$$precision =\\frac{TP}{TP+FP}, recall =\\frac{TP}{TP+FN}\\nonumber \\\\ F1-score = \\frac{2\\times precision \\times recall}{precision + recall}$$ (Eq. 19)\nPerformance Comparisons with Baseline\nThree baseline classification methods: Support Vector Machines (SVM), Adaboost, and Random Forests are adopted to evaluate our extracted features. We test each classification algorithm with scikit-learn BIBREF9 and run a 10-fold cross validation. On each dataset, the employed classifiers are trained with individual feature first, and then with the combination of the two features. From 1 , we can see that GOSS+LOSS achieves the best performance on F1-score among all others. Besides, the classification by combination of LOSS and GOSS can increase accuracy by more than 3% compared with raw topic distribution probability.\nComparison with Other Features\nTo compare our extracted features with previously used features for spammer detection, we use three most discriminative feature sets according to Lee et al. lee2011seven( 4 ). Two classifiers (Adaboost and SVM) are selected to conduct feature performance comparisons. Using Adaboost, our LOSS+GOSS features outperform all other features except for UFN which is 2% higher than ours with regard to precision on the Honeypot dataset. It is caused by the incorrectly classified spammers who are mostly news source after our manual check. They keep posting all kinds of news pieces covering diverse topics, which is similar to the behavior of fake accounts. However, UFN based on friendship networks is more useful for public accounts who possess large number of followers. The best recall value of our LOSS+GOSS features using SVM is up to 6% higher than the results by other feature groups. Regarding F1-score, our features outperform all other features. To further show the advantages of our proposed features, we compare our combined LOSS+GOSS with the combination of all the features from Lee et al. lee2011seven (UFN+UC+UH). It's obvious that LOSS+GOSS have a great advantage over UFN+UC+UH in terms of recall and F1-score. Moreover, by combining our LOSS+GOSS features and UFN+UC+UH features together, we obtained another 7.1% and 2.3% performance gain with regard to precision and F1-score by Adaboost. Though there is a slight decline in terms of recall. By SVM, we get comparative results on recall and F1-score but about 3.5% improvement on precision.\nConclusion\nIn this paper, we propose a novel feature extraction method to effectively detect \"smart\" spammers who post seemingly legitimate tweets and are thus difficult to identify by existing spammer classification methods. Using the LDA model, we obtain the topic probability for each Twitter user. By utilizing the topic probability result, we extract our two topic-based features: GOSS and LOSS which represent the account with global and local information. Experimental results on a public dataset and a self-built Chinese microblog dataset validate the effectiveness of the proposed features.\nFuture Work\nIn future work, the combination method of local and global information can be further improved to maximize their individual strengths. We will also apply decision theory to enhancing the performance of our proposed features. Moreover, we are also building larger datasets on both Twitter and Weibo to validate our method. Moreover, larger datasets on both Twitter and Weibo will be built to further validate our method.",6 "char_len": 15179,7 "document_hash": "1d2699adbcc941fc6fd3d27246e6c2818d60ec53c12224ac2427fb15fee44682",8 "questions": [9 {10 "question_id": "q00",11 "qtype": "qasper",12 "question": "LDA is an unsupervised method; is this paper introducing an unsupervised approach to spam detection?"13 }14 ]15}