Yong Qian: The application of affective computing in Taobao UGC
First of all, I would like to share with you the application background of UGC emotion computing on Taobao. In Taobao APP, UGC comes from a wide range of sources. When you buy a product, you will see other users' comments on the product. The number of new comments can reach tens of millions every day. In the short video business scenario of Taobao APP, millions of comments will be generated, and the scale of comments in the live broadcast room will be even larger. In addition, when the user enters some search scenarios, we can show the user's opinion of the target product. For example, when we are searching for the product of "lipstick", we will present some opinions or feelings of users who have bought the product. In addition, we also show a summary view to help users understand the attitude of most users who have bought the product, and through it, users can make better shopping decisions and learn the advantages and disadvantages of the product more quickly. Generally, we need to do the following three things for UGC: commodity attribute word extraction, emotion word extraction and emotion word polarity analysis. Next, we make a definition of UGC emotion computing task. 2. Definition of UGC Emotion Computing Task Let's first talk about what attributes are. An attribute can be defined as a part or a part of a commodity. The battery of the laptop, for example, is itself a component and a part of the laptop, so we can define it as a property. The commodity itself also has some inherent attributes, such as the price of the commodity, material and so on. A property can also be a component of a product or a property of a part, such as the battery life of a laptop. We also need to extract emotional words from the comments. Emotional words are some words that users express subjective emotional tendency to an attribute. For example, in comments, users describe the long life of notebook battery. The sentence expresses the positive emotion of users' battery life. The task of analyzing the user's emotional tendency corresponds to judging the part of speech of the user's emotional word output to the attribute. In the emotion computing task, we pay more attention to the positive, negative and neutral emotions. In the comments apparel goods, for example, the user said "received goods feel good fabric, it's good to try it on, zip is not so good, wear comfortable, effect is very good also, worth buying", and then according to the < attribute words, emotional words, emotional attributes > this three parts, we can analyze the < cloth, good, positive > a triad. Because fabric is part of clothing, and the sentiment words correspond to "nice," the sentiment bias is clearly positive. For "it's good to try on", the sentence only contains the emotional word "good", so it describes the effect of trying on, so the effect of trying on is an implicit attribute, and the corresponding emotional tendency should be positive. There is also a user description "zip is not very easy to pull", which corresponds to a part of clothing, emotional word "not very easy to pull" obviously expresses some negative or derogatory emotional tendencies, so it is a negative emotional polarity. Then, "wearing is very comfortable", "wearing" is an upper body effect of clothing, which is a corresponding attribute, and its corresponding emotional word is "very comfortable", which is a positive emotional tendency. Then "effect" is also an attribute of clothing, and the emotional word corresponds to "also good." For the emotional word "worth buying", it describes whether to buy back, is the buyback attribute of clothing, "worth" as an emotional word expresses a positive emotional tendency. 3. Problems & Challenges facing UGC Emotion Computing In terms of UGC emotion computing tasks, the challenges we face are largely due to the characteristics of the business itself. First of all, different categories have huge differences in the attributes of the domain and the way of expressing emotions, namely, long-tail cases are included. For example, taste is an attribute of food, but not of cooking utensils. In addition, for the same emotional word, there will be great differences in the emotional tendency in different categories or fields. For example, "very dry" has opposite emotional tendencies for garlic and fruit. Many expressions of user evaluation opinions contain rich contents. Differences in fields and different proportions of annotated data in different fields of the same expression pose great challenges to the generalization ability of supervised models. In addition, there are some negative comments on the business scene of Taobao, but its proportion in the overall data is small, which may reach 10:1 or 20:1 in different fields. It is difficult to label samples in these extreme cases, and data imbalance will lead to low classification accuracy of some negative samples. Finally, cross-disciplinary issues are also a big challenge. For example, "low sound" is a positive emotion for refrigerators and a negative tendency for sound. The UGC sentiment analysis system designed by us will be introduced in general. We chose supervised training because we couldn't achieve the performance requirements we needed to go online without supervision. We will feed UGC paragraphs together into the attribute-affective word extraction model. UGC clips may come from users' comments on products, users' comments in the live broadcast room, users' comments on short videos, q&A area, etc. The extraction model then breaks them down into triples (attribute, emotion word, emotion polarity). Firstly, the extraction model will extract attribute and emotion words respectively. Then a matching task is carried out, that is, which attribute corresponds to which emotion word. There may be independent emotional words in UGC, which represent that they describe an implicit attribute. In addition, after properties of emotional extraction, we will have more properties - to emotional words, but to express the same attributes and emotional matching distribution in different users' comments, we need to put the same semantic properties - emotion to describe using the same point of view, and in the final convenient view summary and presentation. For the above reasons, we need to normalize attributes & emotions. Specifically, normalization will complete the implicit attributes of the previously generated attribute-emotion pair, and cluster the emotional words with the same attributes or semantics. After normalization, we will have a classification model that identifies attribute-affective pairs for affective tendencies. For the online presentation, we will use an opinion generation module and an opinion aggregation module. Its function is to summarize the words expressing the same point of view and generate a unified point of view for display under the commodity dimension. Of course, in the whole system, we will continuously dig out the samples with poor performance in the model for manual bidding through active learning, and then continuously improve the effect of the model. The viewpoint extraction model is explained in detail below. The input of the model is a comment. Backbone is based on RoBERTa. Continuous training of RoBERTa with e-commerce reviews enables the model to be better expressed in specific domains. RoBERTa then passes through a BiLSTM layer in order to obtain a more abstract sequence feature. The original intention of the intermediate Linear layer is to make the model pay more attention to the characteristics of the current domain, because domain-specific attributes and emotional polarity are very common in taobao scenes and difficult to obtain by traditional methods. For example, the same word may or may not be an attribute word in different fields. Therefore, in order to better describe such words, we need an expert network to control the characteristics of words in a certain field. We borrowed from the MMOE and added the shared property to it. For example, in the scene of e-commerce, the price of goods, the service of merchants, the speed of logistics, etc. The advantage of adding common attributes is that for attributes with rare annotations in some fields, if annotations are sufficient in other fields, the field can make good use of common attributes to express their features. In addition, we also added the feature of dynamic sharing and adopted the attention mechanism to make the model have the ability of feature screening in the current field. Finally, the three features are spliced together to solve the label continuity problem through feature mapping and CRF layer to get the final output. After a lot of experiments, we get that the F1 value is 0.8493 under the basic model of Bert+BiLSTM+CRF, the F1 value is increased to 0.8546 after Backbone is replaced by RoBERTa with continuous learning mechanism, and further increased to 0.8662 after joining the domain private expert network. After joining the domain common network, the F1 value continues to increase to 0.8666. Finally, the F1 value of the model with attention mechanism can reach 0.8668. Of course, in the offline experiment, the data scale is small. In the online experiment of the real model, we sampled the case. Found after joining the optimization performance more obvious ascension into the emotion of knowledge in the process of the training mode of using knowledge into emotional reason is in the process of the training model to extract property level emotion classification tasks and tasks are directly dependent on knowledge related to emotional expression, such as properties, emotional words), in the preliminary training general knowledge in this field will be directly to the promotion of downstream tasks. If we rely entirely on annotation data to train the model, limited by the high annotation cost, the triplet training sample size we have (attribute, emotive word, emotive tendency) is small and there are long-tail expressions, which leads to poor model fitting effect. The types of knowledge used in our pre-training include general knowledge of emotion and specific knowledge of e-commerce. General emotional knowledge includes emotional words, part of speech, emotional polarity of word level, negative words that reverse emotional polarity, degree adverbs that express emotional intensity, adjectives and so on. This knowledge can be obtained through open source and daily business related sentiment thesaurus or POSTAG. For e-commerce domain specific knowledge includes commodity attribute, commodity emotion word, commodity attribute-emotion word collocation, and commodity name (entity). In addition, we also added contextual knowledge, such as the category to which the product belongs, category, CPV/CTV system of the product, product title, etc. Such knowledge can be obtained by unsupervised mining or direct mining of existing models. At present, our model integrates emotional knowledge by adding knowledge-related embedding and mask of emotional word level, so that the model can pay more attention to the knowledge we add after pre-training, so as to significantly improve downstream tasks. Specifically, we add category embedding on the basis of Bert's traditional embedding and realize the integration of emotional word knowledge through sentiment masking. Sentiment masking has a similar idea to Bert. We hope that the model can obtain the expression of emotion word knowledge by recovering attribute-emotion word pairing or emotion word masking. In addition, users actually have an overall emotional tendency towards a product, which is very helpful for judging the emotional tendency of attributes. Therefore, we add the overall emotional polarity of the sentence to the objective function of pre-training. Finally, the loss function of our model is composed of six parts, corresponding to the predicted loss of attribute-affective word pair, the loss function of Bert's traditional pre-training task, the predicted loss of affective word, the predicted loss of commodity category, the predicted loss of sentence overall emotional polarity and the loss function of POSTAG task. Through the experiment, we found that the FORMULA 1 value was improved in most cases by gradually adding the trick proposed before. The macro-F1 value of Bert base model is 0.9306, but after adding all embedding and masking techniques, the final macro-F1 value of the model can reach 0.9543. We also tried to add an emotion map to the pre-training model. The idea of mapping came from the fact that the pre-training model did not work well for some long-tailed cases in the field, and the inclusion of maps enabled those long-tailed cases to match similar expressions. These long tail samples may not be sufficiently labeled in the current field, but there are sufficient samples labeled in other fields. In this case, they can be better expressed by drawing. This approach can also be seen as data enhancement.
Comments
There are no comments for this story
Be the first to respond and start the conversation.