Imported from Medium: https://medium.com/@aruny/how-deep-learning-helped-us-predict-service-computation-time-e041ada65dbb
This might not be the very import problem in general however, there was a need to predict the computation time (actually computation time category like fast [0–2 sec), medium [2–4 sec) or slow [4+ sec)) for one of our service. The service takes short spoken/written text and does some natural language understanding and returns the meaning extracted from the spoken/written text, typical but non-trivial NLU task.
I am not going to discuss how we did this, in fact thats not my primary work. The problem which I am going to discuss here is how can we predict in advance how fast the service will perform above task of natural language understanding in a given environment.
Initial Naive Approach
We initially thought that the computation time could be the function of number of words, sentence or paragraph, so we get hold of our production logs of input text and computation time, we hand coded our features (feature engineering in ML world) and created a dataset something like below:
(#words, #sentences, #messages, some more feature.) -> computation time class
Where computation time class could be 0(fast), 1(medium) and 2 (slow)
We divided the dataset into train and test. Trained our SVM classifier with train set and computed accuracy on test set. And we did miserably bad at prediction. Almost 100% of prediction was of class 0. Though the class 0 accuracy was 90+%, but almost 0% accuracy on class 1 and 2, so the model was almost predicting class 0 whatever is the input, this is because the dataset itself was highly skewed (90+% )towards class 0. So the overall (on all classes) accuracy was disappointing. Then we tried duplicating data of class 0 and 1 to make their distribution similar to class 0, but that didn’t helped either. This lead us to do some analysis on data itself and we tried plotting feature vs. computation time graph and we found that they were unrelated. That gave us the clue that the computation time is not a function of features like #words and #sentences/messages. Its not co-related at all. However we also didn’t know what other attribute could be the potential feature, though we had some knowledge that internally the computation time may depends on what noun, verbs/activity being mentioned in the text. However we do not know them(presence of noun/verbs etc.) upfront, thats the job on service itself. Even if we know, we were not sure if thats the only feature that might impact the computation time.
Initially we thought its a limitation of SVM as a classifier, so we also tried a simple feed forward neural network with same feature set and on the same dataset however it was not so good either, few % more than SVM but still doing mistake on class 1 and 2.
Enter “End to End” Deep Learning
Given we are sure that it would be impossible to identify all the features ourself which might impact the computation time, we had to do something with our original input text as a whole, without worrying about features. I discussed the issue with my colleagues Pinaki and Hari around, and it turns out that doing end-to-end deep learning with neural network might help us to do a better job of classification. Letting the network figure out hidden features that might be impacting the computation time. So it was decided that we will give whole text (will discuss later how we did that) as an input to our network and computation time classes as an output, and let it learn all the nuisances of the data. And hey, it did to some extent the accuracy jumped from earlier 40–44% to 55–57%, so definitely it has learned some aspect of data but still doing mistake when it comes to predicting class 1 and class 2. This leads to validation of other hypothesis that skewed dataset may skew the prediction as well. So like we did for SVM classifier, we carried out data synthesis by duplicating records from class 1 and class 2 to make its distribution same as class 0. And retrained the same network. And we were shocked by the result the network become 93+% accurate (jumped from earlier 5–6% ) on class 1 and 2 and 80+% on class 0 (reduced from 90+%). This was just awesome. This what we wanted to achieve (80+%) on all classes. This when we realised the power of end-to-end deep learning and more than that the significance of data distribution in training class and data synthesis in general.
I have used the term end-to-end deep learning, thats just a name given to the way to model our problem. Which is to model the problem in such a way that there is just input and final expected output, one need not worry about intermediate representation or custom feature as people used to earlier in traditional machine learning. Actually in practice its easier to carry out E2E learning given one has lot of data and infrastructure to run. In absence of any of above two, one need to resort to feature engineering though in many problem its not possible to do it. Like speech recognition and this problem itself.
Key Learning
- Think of end-to-end learning whenever possible. Which can be applied whenever you do not know what influences the outcome and you have lots of data for network to learn for. Actually lot of data is subjective, in fact in our problem we did it with just 40k records. But people say more the better.
- Look for data skewness if your classifier is biased, you can create artificial data (in our case simply duplicating) and try your network. It some cases you need to hand craft data as well, I have seen the same in other problems (using LSTM) that my colleagues where solving.
But before I finish, I promised above that I will tell you how we mapped input text as a network input. As you know the network only understand numbers which we organise as array or vector. So we needed a way to represent input (words in our case) as a real value. The simple way to represent the word is using its one-hot approach. i.e. You define a vector of size or vocabulary of dataset and you set 1 for index representing a given word and rest to 0. E.g. if our vocabulary consists of just 4 words say, “I” “love” “deep” “learning” then we can represent them as below:
“I” : [1,0,0,0]
“love”: [0,1,0,0]
“deep”: [0,0,1,0]
“learning”: [0,0,0,1]
Of course this may be very practical given vocabulary can go of size 10s of thousand, but the bigger problem is such representation doesn’t hold any semantic of words. The better representation would embedding like word2vec made famous by Thomas in this paper or Stanford’s Glove. In our case we used Glove and it worked well.
That’s it!, For me its beginning of exciting journey of applying maths and computer science together to solve real life problem. Hope you will find learning and applying DL equally exciting.