3.1. Research Framework

Research outline is shown in Figure 1, which includes sales prediction exploiting social media data and sales data in the following steps:

1. Data Collection—To accomplish the task, Twitter and sales data spanning six months were collected from Italian fashion brand.

2. Data Preprocessing and Sentiment extraction—In this step, text mining technique is used for cleaning tweets. These cleaned tweets were then used for sentiment extraction using an enhanced NRC lexicon data and NB classifier. Classification results into three sentiment indicators Positive (S_p), Negative (S_n) and Neutral (S_ne).

3. Examine correlation—Extracted sentiments from earlier stage were aggregated weekly and the relationship between the sentiments of the current week and sales of the upcoming week was examined. This step was carried out to investigate relationship between tweets and sales.

4. SMBF Model—After exploring the relationship between tweet sentiments and sales, this model is designed to forecast garment sales based on historical sales and sentiments from social media. To attain this, FSIS model was built to identify the uncertainty of sentiments in terms of corrective coefficient (V_s), which represents the sales variation factor with actual sales and this is implemented on Exponential Forecast (EF) to predict the final forecast from SMBF model.

5. Performance Validation—The model performance was validated using forecasting measure, Mean Absolute Performance Error (MAPE).

Figure 1

Research framework.

3.2. Data Collection

Data used in this work was collected for one of the Fashion apparel brands, spanning time period of six months from April 2018 and September 2018. This study uses two data forms, first, Twitter data, which were collected using tweet API by creating tweet app to get authentication key [22]. Open source tool “Rstudio” and its package “TwitterR” were used to request API to Twitter using Oauth access [20]. Semantic keywords with hashtag “Brand name” were applied on tweet retrieval string. Collected tweets attributes is shown in the Table 1 and total number of tweets collected for six months interval was 838 as shown in Table 2.

In the next step, sales data for the same interval of time as the Twitter data was collected, which accounts for the daily sales of garments.

3.3. Twitter Data Preprocessing

Twitter data collected in previous step was in an unstructured form as it contained many duplicates, urls, punctuation, etc. It was necessary to clean and normalize tweet for analysis. As the first step of preprocessing, manual screening of tweets were performed to confirm whether or not it belongs to the fashion apparel brand that we targeted. While exploring the tweets, synonyms of selected brand name were found in some tweets. Moreover, some of the tweets were also related to politics, games and some other issues, and therefore such tweets were removed. Secondly, as only text is required for sentiment analysis, first attribute shown in the Table 1 was chosen. Text mining was applied to remove duplicates, whitespaces, hashtags, urls, stop words, retweet entities and numbers [20]. As a result, the total count of tweets reduced to 313, which is 37.35 % of the actual collected data.

3.4. Algorithm Formalization for Sentiment Extraction (Or Mining?) from Tweets

This section discusses the methodology applied for the classification of tweets. Lexical approach was used to extract sentiments from tweets [23]. NRC lexicon dictionary [24] was used, which contains 5636 words labeled with “Positive” and “Negative.” For the tweet classification task, a probabilistic classifier “NB” algorithm [25] was used as it is the effective classification model for text classification. Moreover, it is highly scalable and it allocates the class labels of a finite set to feature vectors [23]. The algorithmic representation of NB is shown in Eq. (1).

• PX|Y is the posterior probability of a target class given attribute.

• PY|X is the likelihood, a probability of attribute given class.

• PX is the prior probability of class.

• PY is the prior probability of predictor.

For the mathematical formulation of this research task Twitter dataset T is considered, which is a set of tweets t and can be represented as T=t1,t2,t3……tn. It was converted into corpus using text mining as explained in the Section 3.3. This corpus is a structured form of text, and is applicable for mathematical operations. Hence, each tweet could be considered as a set of words, i.e.,t=w1,w2,w3……wn. Similarly, Lexical labeled dictionary or dataset L[24] is composed of two attributes: words W′ and sentiment class C, and it is represented as L=W′, C, where, W′=w′1,w′2,…w′n is the set of words in lexicon dictionary and C=c1, c2, c3……cn is labeled sentiment class in lexicon dictionary. The sample of the lexicon dataset is shown in Table 3. Lexicon dictionary is labeled with two classes such as “Positive” and “Negative.”

For each given tweet t in Twitter dataset T, NB computes posterior probability of classes of sentiments, c∈C, where C=positive,  negative and assigns the predicted class C^, which has the maximum posterior probability to the word of a given tweet. Accordingly, NB algorithm is represented in Eq. (1) can be rewritten as shown in Eq. (2).

Further, from Eq. (2), denominator is dropped as P(t) will remain same for all sentiment class and it will have an impact on “argmax” [26]. Therefore, it can be transformed as shown in Eq. (3).

Therefore, Eq. (3) can be rewritten as below

This model follows the assumptions: position of the features (words) are not taken into account and the feature probabilities Pwn|Cn are independent of the given class c∈C [26]. “Prior” and “Likelihood” probability values are calculated for the occurrence of the most probable class and score is assigned.

Furthermore, formalization of the above equations was implemented for the ease of applying it in programming language illustrated in the Figure 2. It is assumed that the algorithm will calculate the maximum posterior probability for the occurrence of a word in a given tweet that is present in a lexical dataset. Therefore, Pc is calculated, which is a prior probability of sentiment class and it can be calculated using the below Eq. (5) and likelihood using Eq. (6).

• N=Total words in lexicon dictionary

• Nc=counts of words from lexicon dictionay for each class c

• (w,c)=number of ocuurences of w in a Tweet t

• (w′,c)=occurence of w′ in lexicon dictionary L labelled with class c

Figure 2

Algorithm for assigning score to the word class.

If the tweet does not have the identical word that exists in the lexical dictionary, then the likelihood in Eq. (6) will be zero as shown in Eq. (7) and this cannot be conditioned, therefore, Laplace smoothing [26] is introduced by adding 1 illustrated in Eq. (8).

While applying this equation in programming, a problematic condition could ascend when count of terms increases. As we know that each probability value of term Pw|c will vary between 0 and 1, and if we multiply them, the product result will start approaching towards zero value. This leads to the problem in the programming language where representation of extremely small number is given as “double” or “long double” type that relies on standard data type “floating,” which is eventually fixed number and its precision reduces with miniscule numbers. The common way to combat this problem is to write the probabilities in “log-probabilities.” This transformation is considered due to number of reasons [27];

• The value of probability lies between 0 and 1, whereas log-probability can lie between –∞ and 0.

• The log-probabilities are easily comparable between smaller and larger numbers, that argmax does in probabilities for example m<n, then lnm<lnn.

• Arithmetic works well, i.e., logarithm of the multiplication of the variable is the addition of the logarithms, i.e.,

Logarithmic representation of the Eq. (4) is represented below comprehensively with the final Eq. (9).

Algorithm presented in Figure 2 assigns score to a word according to its frequency that matches with the words in a lexical dataset. The final score is the absolute of addition of logarithmic calculation of prior and likelihood as shown in Eq. (10). So, the algorithm assigns the class score for all words in a tweet. And, we are interested to know if the given tweet, which is as a whole sentence or statement, is whether positive or negative. Therefore, a new metrics “Best fit” is used for categorizing the tweets as “Positive” or “Negative.”

“Best fit” metrics for a given tweet is calculated as the ratio of the scores of positive and negative class as illustrated in Eq. (11). Classes for a given tweet will be assigned based of the conditions of score shown in Table 4. That is, if the value of “Best fit” score is greater than 1, then the tweet is positive; if it is less than 1 then it will be categorized as “Negative” tweet; and if the score is equal to 1, then it will be categorized as “Neutral” tweet.

3.5. Forecasting Model

This research study uses Exponential forecasting (EF) as a base model for forecasting the sales quantity. Exponential smoothing is a well-known forecasting method and was introduced by [28] and [29]. It can be used as an alternative to group of ARIMA models [30]. EF model equation is shown in the Eq. (12):

• Ft+1 = Forecast for the time t+1

• α = Smoothing constant 0≤α≤1

• At = Actual data

• Ft = Forecast data for time t

Sales data of fashion garment industry was aggregated weekly and an EF was performed. “Corrective Coefficient (V_s)” was calculated, which represents the variation in sales as shown in Eq. (13). V_s is used as an output variable for FSIS, mapping the impact of tweets optimizing the predicting ability of the forecast model.

3.6. Modelling Tweet’s Sentiment Uncertainty for Predicting Sales Using Fuzzy Theory

This section discusses, in detail, the process involved in modelling tweet sentiments. This section is divided into two sub-sections briefing the details of model design and defines linguistic variables for analyzing the impact of Tweet sentiments. Section 3.6.1 outlines the architecture of FSIS model and section 3.6.2 explains the fuzzy linguistic variables for the FSIS model.

The uncertainty is represented by fuzzy sets in classical theory and each element in a set hold a membership degree [31–33]. Fuzzy set can be denoted as a pair U,m, where U is a set and m is a membership function of a fuzzy set, which assigns the value between 0 and 1 and it can be denoted as U→0,1 [34]. Therefore, for a fuzzy set P=U,m, membership function m=μA maps the function between 0 to 1. For a finite set U=a1,…,an, it can be represented as ma1/a1,…,ma/an. Let a∈U, then α will not be considered in the fuzzy set U,m if ma=0, will be fully considered if ma=1 and it will be partly considered if  0<mα<1 [35]. It follows the mathematical operations of complement, intersection and union. For the fuzzy set P and Q, P,Q⊆U, u∈U, complement operation can be represented as μPu=1−μPu, intersection operation can be represented as μP∩Qu=minμPu,μQu and Union of two fuzzy sets can be shown as μP∪Qu=maxμPu,μQu. Fuzzy set has “Commutative,” “Associative,” “Distributive,” “Identity” and “Transitive” properties. Triangular membership [36] is popular to solve engineering optimization problems due to its ability to provide solution instantly. Degree of Membership function for a fuzzy set P can be illustrated in Fig. 3 and Eq. (14) defined by the lower and upper limit, a and b where, a<b<c.

Figure 3

Triangular membership.

3.6.2. Defining universal and fuzzy set for sentiments for FSIS

For formalizing and modeling twitter data with garment sales performance, we used intelligent fuzzy technique as discussed in the Section 3.6.1. FSIS constitutes three fuzzy sentiment input variables I as S_p(Positive), S_n(Negative), S_ne(Neutral)and Output O as V_s (Corrective coefficient). Triangular membership function is assigned to input variables in the range between [0 1] and output variable in the range [−25 25]. Aggregated weekly sentiment data were used for all three input variables to feed into model. The key objective in modelling sentiments, viz., S_p(Positive); S_n (Negative); and S_ne(Neutral)retrieved from tweets with fuzzy inference is to capture human emotions on social media weekly and to forecast the sales variation based on these consumer emotions. Human emotions are uncertain and they change from time to time. So, for example, maybe in one week, sentiment “Positive” could be high and it could lead to a positive effect on sales of upcoming week. Similarly, if it is “Negative,” then it could lead to a negative impact on sales of upcoming week. The designs of the linguistic variables for sentiments are taken as “Low,” “Medium” and “High.” Thus, I is a set of sentiments, i.e.,  InputI=Sp, Sn, Sne and membership sets for I is defined in three linguistic variables as “Low,” “Medium” and “High” for each input variables and can be written as Sp=Low, Medium, High, Sn=Low, Medium, High, Sne=Low, Medium, High, and for output variable OuputO=Vs, membership sets is defined as Vs=Low, Medium, High., where, V_s is a corrective coefficient and it is calculated as a difference between actual sales and EF sales as shown in Eq. (13).

Triangular membership function for the input variable in set I is defined between limit [0 1] and the output variable in the limit [−25 25], which is a corrective coefficient in percentage of sales and it will predict sales variation from −25% to 25%. (Table 5 describes) the membership variables and values for the I and O respectively are shown in Table 5. Figures 4 and 5 depict the triangular membership function for variables I and O.

Input/Membership	Low	Medium	High
(Sp)w	[0 0 0.5]	[0 0.5 1]	[0.5 1 1]
(Sn)w	[0 0 0.5]	[0 0.5 1]	[0.5 1 1]
(Sne)w	[0 0 0.5]	[0 0.5 1]	[0.5 1 1]

Output/Membership	Low	Medium	High
(Vs) w+1	[−25 −25 0]	[−25 0 25]	[0 25 25]

Table 5

Membership function for input variable I (Input) and O (Output).

Figure 4

Triangular membership for input variables (S_p, S_n, S_ne).

Figure 5

Triangular membership for output variables (Vs).

Calculation of the membership function value for the Input variable “Sp=Low, Medium, High” is illustrated below:

 μSp Low=0.5−x0.5      0≤x≤0.5

  μSp Medium=x0.5      0≤x≤0.5   and  1−x0.5      0.5≤x≤1  

  μSp High=x−0.50.5      0.5≤x≤1

Similarly, membership function values is calculated for S_n and S_ne. Besides, the membership function value for corrective coefficient “Vs=Low, Medium, High” is calculated as below:

 μVs Low=0−x0−−25      −25≤x≤0

μSp Medium=x−−250−−25      −25≤x≤0    and25−x25−0      0≤x≤25

μSp High=x−2525−0      0≤x≤25

Considering the impacts of the weekly sentiments, fourteen conditional rules are defined by taking into account human knowledge as shown in Table 5. These rules are defined as

“Rule i: IF condition i THEN action I”

To demonstrate the functioning mechanism of rules consider first row of the Table 6, which states that if S_p = High, S_n = low and S_ne = Medium, then V_s = High, i.e., if the aggregated weekly positive sentiment is high, negative sentiment is low, and the neutral sentiments is medium, then the sales performance will be high and its values will lie between 0 % to 25 % range, meaning there is an incremental variation in sales than the previous week. Similarly, expected sales variation decision will be taken by the model considering the inference of input variables and defined rules. The final score of the output, i.e., defuzzification of the fuzzy set and the crisp number is calculated by the centroid method using Eq. (15). Architecture of FSIS is depicted in Figure 6.

Table 6

Rules defined for Fuzzy Sentiment Impact on Sales (FSIS).

Figure 6

Social Media Based Forecasting (SMBF) model.

3.7. SMBF Model

SMBF model is a time series model, which takes into account the impact of sentiments on sales using FSIS. Parameters of the FSIS are optimized to improve the forecast ability of the model and it is implemented on the exponential time series forecasting and the formula its forecast estimate is given in Eq. (16), where V_s is corrective coefficient, calculated using Eq. (13) and EF is sales volume forecasted by the EF model as explained in section 3.5.

• SMBF = sales Forecast from fuzzy time series model

• V_s = corrective coefficient predicted by the FSIS model

• EF = sales forecasted by Exponential Model

FSIS model parameters, i.e., “Input,” “Output” and “Rules” defined in Section 3.6.2 were optimized using the local optimization method known as “pattern search method” [38], which is useful for the fast convergence. The model was tuned for 500 iterations, and as a result, model was optimized. Optimized parameters for “Input” and “Output” variables are shown in Table 7 and optimized rules are shown in the Table 8.

3.8. Validation Method and Performance Measure (MAPE)

Mean Absolute Percentage Error (MAPE) is a performance metrics for evaluating the performance of the forecasting models and it is denoted as in Eq. (17).

• At = actual sales at time t

• Ft = forecasted sales at time t

4.1. Tweet Classification

Tweets were assigned sentiment class as explained in Section 3.4. Logarithmic values of “Prior” and “Likelihood” were calculated of the lexicon dataset [24] and it is shown in the Table 9. The total count of the words labeled “Positive” is 2312 and “Negative” is 3324 in lexical dataset. (To see in detail) Detailed mechanism (of how the) of algorithm is explained in section 3.4. (Let us take one) Consider, for example, in a tweet, “style made genuine matching lining,” the word “genuine” exists in the lexical database as “Positive,” therefore, the score for this positive word in a tweet will be calculated using Eq. (10). As there is only one positive word that matches with the words in lexical dictionary and not with the negative word in a tweet, the score for the“Positive” class will be calculated by using logarithmic value in Table 9 as “0.891062 + (1*7.74586823) = 8.636929”; and for the “Negative” class, it is “0.528006 + (0*8.1088924156) = 0.528006,” and the final score for the best fit will be calculated using Eq. (11) and the result is shown in Table 10. In Table 10, “5” indicates the tweet number in a Tweeter data T. In this case, as the score is more than 1, the assigned class for this tweet is “Positive. ” Similarly, all tweets were assigned scores and classified as “Positive,” “Negative” and “Neutral” based on their “Best Fit” score. Overall classification of cleaned tweets is illustrated in the Figure 7, and it can be seen that about 45% tweets were classified as “Positive”; 27% tweets were classified as “Negative” and approximately 28% of tweets were classified as “Neutral.” The classification results indicate that the chosen fashion brand has quite a positive outlook from customer perspectives as only 27% of total tweets were negative.

Table 9

Log prior and log likelihood of lexical dictionary.

Table 10

Result interpretation for a tweet has one positive word.

Figure 7

Classification of tweets for fashion brand.

4.2. Correlation Test

The main aim of this study was to investigate if the customer sentiments from tweets collected in the current week have any influence on the upcoming week’s sales. For this, assumption was made and it was verified using Pearson correlation [39]. Following assumptions were made to check the correlation between sentiments indicators S_p, S_n, S_ne and sales volume S_v, where S_p= aggregated “Positive” tweets for a week “w,” S_n = aggregated “Negative” tweets for a week “w,” S_ne= aggregated Neutral tweets for a week “w,” S_v= Change in volume of sales for week w + 1.

• If the correlation between S_p and S_v is positive, then the sales will increase.

• If the correlation between the S_n and S_v is negative the sales will be decrease.

• If the correlation between the S_ne and S_v is positive the sales will be increase.

Correlation values, as shown in the Table 11, indicate that there is a moderate positive correlation between S_ne and S_v that is 32%, weak positive correlation between S_p and S_v with a value of 22% and negative correlation between S_n and S_v with −13%. This result clearly shows that if there is more number of positive tweets and neutral tweets, it will have positive effect on sales, whereas if there is more number of negative tweets, it will have negative effect on sales.

4.3. SMBF Model Performance

In this research FSIS is combined and implemented on EF as explained in Section 3.7 and final forecast was achieved by SMBF using Eq. (16). Model performance of SMBF was evaluated by calculating MAPE values as explained in Section 3.8. MAPE values for EF model and SMBF model is shown in the Table 12. This clearly shows that adding the impact of the sentiments by FSIS model on EF model, exhibit the forecasting capability of model. It can be observed that the performance of the SMBF is better than EF, which solely relies on the historical sales information, whereas SMBF model counts on both historical sales data and social media data. This result illustrates that by adding the features of social media in our forecasting model, the performance of the model can be improved significantly.

The forecasted sales by EF and SMBF models were plotted to compare it with actual sales as shown in Figure 8, which clearly shows that forecast achieved by

Figure 8

Forecasted sales byExponential Forecast (EF) model and Social Media Based Forecasting (SMBF) model with actual sales.

SMBF model is significantly better than EF. The prediction of the SMBF model was very close to the actual as shown in Figure 8. As the collected data was during summer season, there is a peak in sales for week 27 and 34 while there is a slight drop in week 36. This could be attributed to the promotional or sales discount effect on the sales. These factors are not considered in this study. However, model SMBF tried to capture the trend to some extent.