"Star With The" Cool! Ali AI Fake Technology Allows Piracy Nowhere To Hide!

Ali used AI to crack down on fake products, and new research results came out.

This time, they focused their attention on the piracy of clothing, and the difficulty reached the maximum. Pirates plagiarized the process, modified the collar, sleeves and other details, such as Taobao on all kinds of "star with the same", the same can be quickly locked.

The idea behind this is that based on the clothing regional expression retrieval model, the similarity and learning of clothing in the image are studied and measured. The research results have been included in CVPR 2020 and have been selected as Oral papers.

Ali safety Turing laboratory said the work The original protection platform will be used in Ali. Taobao, Tmall and other Ali business platform on-line, providing the ability to detect infringement.

　　 Meticulous plagiarism is one foot high, how high is the fake road?

As far as clothing industry is concerned, although counterfeiting has been continuous, piracy plagiarism is still widespread. And from online to offline, plagiarism is becoming more and more tricky, and the difficulty of counterfeiting increases year by year. At present, there are only three categories of plagiarism in the field of clothing.

The first category concentrates on Image embezzlement Up. Pirates usually use or modify the commercial maps of genuine brands without authorization, such as adding watermark to their stores, or doing some image processing (inversion, scaling, splicing, etc.). This kind of infringement plagiarism cost is very low, but it is easy to be locked by the image retrieval system of the platform, and then quickly "governance".

The second category is Creative embezzlement The bad merchants directly copy the original product design and originality of the original merchants, and make the same or imitation money. The cost of such infringement is slightly higher, but the same item retrieval algorithm based on commodity similarity measure can recall and manage them.

The third category is embezzlement. Some parts of clothing are modified. It is like washing the manuscript, such as changing the design of the neckline, or the layout of the print on the chest, or even changing the style of the clothing. But as shown below, it is still plagiarizing the style and design elements of the genuine brand clothing (the left side is genuine, the right side is pirated), and even sells it as "star money".

This kind of piracy has the highest cost and is not easily locked by traditional algorithms based on the same commodity. Under normal circumstances, the electronic business platform can only be found through manual audit, the cost of counterfeiting is very high.

Is there a way to enable the system to automatically lock such plagiarism? This is the latest research direction of Ali safety Turing laboratory.

Previously, they proposed a clothing copyright algorithm to locate plagiarism based on attribute aware fine grained similarity learning method, which was captured by AAAI2020.

Now they have another one. The new idea, based on the clothing regional expression retrieval model, carries out regionalization similarity learning and measurement of clothing in the image, so as to achieve more effective anti-counterfeiting.

　 Search algorithm of "pirated clothing image" to sleeves and collar accurately

The definition of pirated clothing is to plagiarize the original design and style of clothing, and modify it in one or two regions to avoid the existing clothing samples screened by the same clothing retrieval model.

Ali's researchers divided the clothing in the image into five areas, including collar, chest, waist and two sleeves, and experimentation in four kinds of clothing (short sleeved T-shirt, long sleeved blouse, jacket and dress).

In terms of algorithm design, they put forward a kind of The regional attention mechanism guided by key points of clothing.

First of all, the key points of clothing are predicted by the key points of clothing, which are the key points in clothing images, such as necklines, cuffs, shoulders, armpits and so on.

The number and distribution of key points for each type of clothing vary somewhat, with about 25-40 points per item. According to these key points, the algorithm can partition multiple areas of clothing pictures, such as collar, sleeve, chest, waist area, etc.

Based on the idea of ROI Pooling, regional integration information is decoupled into multiple regionalized feature expressions to independently learn and measure the similarity of features.

At the same time, the key points of clothing combined with the expression of regionalization can be used as a attention mechanism. In the image retrieval network, the weight of key parts is improved, and the weight of non key parts is reduced to enhance the discriminant power of the model to the key parts.

The key point of clothing estimation is that branch and image retrieval branches use the same HR-Net backbone network, and its multilevel parallel structure keeps multi-resolution features while maintaining high resolution.

In the selection of loss function, the mean value loss function is adopted in the estimation of the key points, and the Triplet loss function is adopted in the retrieval branch. The loss function value is no longer the characteristic triplet difference of the whole picture range, but the cumulative result of the difference of the regional characteristics.

The framework of the method is shown in the following figure: the network can be divided into the clothing critical point estimation branch and the clothing retrieval branch. The retrieval network includes two output forms: the same clothing retrieval and the pirated clothing retrieval.

Based on the analysis of the sample of the platform infringement, the Alibaba researchers found that different categories of clothing are easily different from the pirated area. Therefore, it is not enough to decouple the similarity measurement of the clothing image features. It is also necessary to set the differentiated weights for each area of each garment and carry out weighted regional similarity calculation to recall more pirated clothing samples. Ben.

To this end, they set up a dataset named "Fashion Plagiarism Dataset" based on platform pirated clothing data. The dataset contains query images of each original garment corresponding to gallery's "pirated clothing" images, and the data cover four types of short sleeved T-shirts, long sleeved tops, coats and dresses.

In this dataset, Fine Tune training is performed on the pre trained network on the Deepfashion2 dataset, and the Coordinate Ascent algorithm is used to iteratively optimize the weights of different clothing categories in order to reduce the loss function value.

The loss function of "piracy clothing" retrieval training process is also based on the Triplet loss function design. Finally, the pirated retrieval network after training can recall the pirated clothing samples in the green box of Output based on the Input clothing image in the above picture.

What is the effect of counterfeiting? Do not lose or even surpass the previous SOTA.

A kind of

In the experimental part of the paper, Alibaba researchers first evaluated the retrieval ability of the pirated clothing image of the algorithm on "Fashion Plagiarism Dataset".

In addition to the methods proposed in the paper, they also set two methods for comparison: one is the traditional retrieval method, using the same loss function of backbone network and Triplet, but does not contain regionalization feature learning and expression mechanism; the other is the regionalized feature representation mechanism, but the regional weights are obtained by training non Fine Tune, and the evaluation index is mAP.

From the results of the table, we can see that the method used in the paper has achieved the best results in all garment categories.

In addition to the above "piracy clothing retrieval" evaluation experiment, they also conducted clothing key point estimation and the same clothing image retrieval task experiment on the Deepfashion series dataset.

In the estimation of the key points of clothing, Alibaba researchers evaluated the most complex Deepfashion2 data sets. Compared with the existing Match-RCNN, CPN and Simple-Baseline methods, the clothing critical point estimation model achieved the highest mAP results in each subset.

In the same clothing retrieval experiment, they used FashionNet, Match-RCNN, PCB and other methods as contrast, and conducted experiments on Deepfashion and Deepfashion2 respectively.

Among them, Deepfashion mainly focuses on In-shop retrieval scenarios, while Deepfashion2 is for Consumer-to-shop scenarios. The evaluation indexes were Top-N recall and Top-N accuracy respectively.

As shown in the following figure, Ali's method achieves similar results with the SOTA method on the Deepfashion dataset, and the result on the Deepfashion2 dataset is much better than the existing baseline method.

From Ali safety Turing Laboratory

A total of 5 researchers participated in the study. They were from the Alibaba, Zhejiang Gongshang University and the Alibaba / Zhejiang University frontier technology joint research center.

The first author is Yining Lang, second author Yuan He, and third author Fan Yang from Ali safety Turing laboratory. Xue Hui, director of the safety Turing laboratory in Ali, is the author of the article. Zhejiang Gongshang University Jianfeng Dong is also one of the authors of the paper.

Ali safety Turing laboratory was formally established in 2016. It was formerly an Ali security foundation algorithm team. It mainly engaged in the development of AI system for safety and risk. The core technologies include computer vision, Natural Language Processing, biometrics, graphic calculation and anomaly detection and analysis. As of 2018, more than 50 patents have been applied.

At present, they are still recruiting large numbers of recruits, including research interns and formal researchers. If you are interested, you can pay close attention to them.