# 2020-07-01 DS FAQ
-
  Q: "
  As per our discussion last night I've cleaned the target variable, developed an initial model (I used a logistic regression rather than a linear regression) and loaded the text in to the dataframe. Do you have any recommendations for NLP models or EDA I can examine before our next meeting? See the notebook here: https://github.com/HechtYehuda/Hate-speech-modeler/blob/master/Hate%20speech%20modeler.ipynb  #email #student #YeHe"
  A: "
  Excellent! Good choice on logistic regression.
  1. First, I would create an new numerical feature based on the text column. Something like the length of the text (number of characters) or the number words.
  2. Retrain your model and with both your features and see if accuracy improves.
  3. Then look for the presence or absence of a hateful word in each string and make that a new binary feature. retrain and get an accuracy.
  4. Then look at using CountVectorizer to do that automatically on all the words in your texts.
  5. Then use TfidfVectorizer to divide that count by the number of documents each word is in.
  6. At this point you'll get great accuracy. But it will be overfitting. So you'll need to do your train test split to see how badly it does on a test set.
  7. Once you have a training set and test set and a TFIDF vectorizer tuned up (playing with min_df and max_df to maximize testset accuracy) then you'll need to read up on more advanced techniques to reduce dimensionality for NL data.    #email #student #YeHe"
-
  Q: "The [shap package](https://github.com/slundberg/shap) produces nice feature maps for image classification deep learning. The `DeepExplainer.image_plot()` overlays red pixels for regions where the filter (convolutional layer output) was negatively correlated with the target class. It uses blue to indicate positive correlation with the target class. But my rectangular images seem to be misaligned with the red and blue overlay of feature importances from shap."
  A: "The shap package may have a bug or you may need to rotate your images before passing them to the image_plot(). `DeepExplainer.image_plot() works fine on the MNIST dataset which is square. So try clipping your images to make them square before passing into shap."
-
  Q: My git commits aren't appearing on my repository on gitlab.  #student
  A: What error messages do you see?  #teacher #socratic
  Q1: "When I type `git push` it just says \"Everything up-to-date\"" #student
  A1: What does `git status` say?
  Q2: "When I type `git status` it shows the file I changed in red and says the changes are not yet staged to be committed" #student
  A2: How do you do what it says and stage those changes to be committed? #socratic #teacher
  Q3: Using the `git add` command or using the `--all` or `-m`
-
  Q: Should I use RandomForestRegressor or RandomForestClassifier for my 911 response time machine learning model? #student
  A: What is the difference between a _classifier_ and a _regressor_? #teacher #socratic
  Q1: My problem has a variety of categorical and numerical features. A regressor uses those features to predict the probability of a variable falling into a particular bucket or class or category. But my model is predicting a numerical quantity, the response time in minutes. It's not a probability, so should I use a classifier? #student
  A1: You're probably thinking of a famous classifier called LogisticRegression. The _LogisticRegression_ name is unfortunate, because it does predict a continuous value, the probability of being in a particular class. But it's purpose and use is as a classifier. It predicts this class (or category, or bucket) in which a particular example belongs. So your problem that is predicting a numerical quantity like time in minutes is not a classifier. #teacher #socratic
  Q2: So I should use a RandomForestRegressor to predict the response time for 911 calls? #student
  A2: Yes, whenever your target variable is a continuous numerical value, a regressor is the right kind of model to use. #teacher
-
  Q: When do I reject the null hypothesis and accept the alternative hypothesis that I proposed at the beginning of the project?
  A: When the P-value for your results is below 0.05 (5%) you can usually reject the null hypothesis. This means that your results are statistically significant and not likely to be caused by chance and your hypothesis represents a probable relationship between your feature variable and your target variable proposed in your hypothesis.
-
  Q: What does a good data story look like?  #student
  A: What kind of story does your audience like to hear? Try making it interesting by setting the stage with a mystery, a plot of your data that looks interesting or unusual or boring. Then say something that interests you about it. Talk about surprising relationships between variables or unusual shapes or values or outliers in the plots. Look for patterns in the plots. First describe those patterns. Then make some guesses at what causes those patterns and how they might be useful in understanding the real world, which is what your stakeholders and the audience for your data story cares about.  #teacher #socratic
-
  Q: How do I come up with a problem statement for data science?  #student
  A: "Start with the data. What columns do you have? What kinds of variables are there in each column? Categorical? Numerical? If you have a categorical variable and a numerical variable you can start with the question (problem statement) \"What is the mean of the numerical variable within each of the categories for the categorical variable?\" #teacher"
-
  Q: If your model to predict the prices of something has a standard error of $42, how often are your predictions to underestimate the price by more than $42?
  A: 16% of the time your predictions will be lower than 1 sigma ($42) below the true price. This can be calculated using your understanding of the 68, 95, 99.7 rule. 100% minus 68%/2 plus 50% is 100 - 84 which is 16%.
-
  Q: What is a good exercise to get up to speed on Data Science?
  A: Load a DataFrame of home prices and calculate some mean and average prices in various categories of homes.
-
  Q: When you are interviewing a Data Scientist what do you look for?
  A: I like to have the candidate talk about their thinking as they solve a data science problem for me verbally. I will describe a dataset and a problem I'm having at work and ask how they would solve it. I look for good data science habbits, like asking me questions about the data, and an agile software development and data science approach. I look for them to pay attention to the both test set error and the training set error to make decisions about what to do next after each new feature engineering or preprocessing step.
-
  Q: When should I use lemmatization on a sentiment analysis or other NLP problem?  #student
  A: When it improves your test set performance metric like F1 score or accuracy.  #teacher
  Q1: Does lemmatization reduce or increase the number of features in your dataset?  #teacher #socratic
  A1: It reduces the number of features, so it can help with overfitting.
-
  Q: About regularization. Why does reducing the values of coefficients from the optimal solution cause the accuracy to go up?
  A: Regularization doesn't improve the accuracy for all of your data. It will always make your training set accuracy worse. And sometimes it can even make your test set accuracy worse. But in some situations it will help your model avoid paying too much attention to your training set and trying to fit it too closely (called over fitting). This will sometime help it pay attention to the features that are most important in the test set. So sometimes it may help close the gap between your training set accuracy and your test set accuracy by increasing your test set accuracy and reducing your training set accuracy.
-
  Q: I'm starting to learn Data Science and python software development. What operating system should I install on my laptop?
  A: If you chose a Linux operating system, you'll never have to learn how to use another one. Linux based OSes (Operating Systems) like Ubuntu, Mint, Centos, and Alpine will always be here. And most of the servers and supercomputers you use for Data Science will be Linux-based. Plus you'll keep manipulative, deceptive, extractive corporations and their closed APIs and expensive products out of your life. Embrace the penguin! You won't regret it.
-
  Q: I'm doing NLP sentiment analysis on restaurant inspection reports. My logistic regression on "number of demerits" (a count between 0 and 60) is giving me 100% R^2 score on the training set and 50% score on the test set. A Linear Regression is giving me only 30% score on the training set and -25% on the test set.  #student #noquestionmark
  A: First, the logistic regression is converting the number of demerits to a bool, so it's only predicting whether or not there are nonzero demerits for a particular restaurant. It's usually not a good sign if a binary classifer is only getting 50% accuracy on the training set. Next you have a severe overfitting problem for both the LogisticRegression and the LinearRegression. What causes overfitting?  #teacher #socratic
  Q1: "Too many features or too many degrees of freedom in the model (such as depth of trees for decision trees and RandomForest).  How do I fix overfitting?  #student"
  A1: "Dimension reduction or feature selection. Try PCA first. Then you may need to try more powerful models that do feature engineering automatically. Deep Learning models may be your only option for this kind of NLP problem.  #teacher"
-
  Q: "When trying to generate a confidence interval around my RMSE, should I use a bootstrapping method? So, keep resampling my predictions and getting the RMSE?  #student"
  A: "To calculate a basic confidence interval using just the normal distribution you can do some math and thinking about the information you already have about your model's error. RMSE tells you a lot about your model's error and the probability of it being wrong by various amounts. You can use the standard deviation of your error all by itself (standard deviation of error is also called standard error or RMSE) to calculate a confidence interval. Once you have that you only need need to put 3 numbers in a sentence about a confidence interval.

    The sentence looks like this: You can be `confidence_value` % that the error in the home price predictions will be between `min_error` and `max_error` .

    1. The confidence value is the probability that the error will be within that range or interval (between the numbers you calculate in 2 and 3 below)2. A minimum error (most negative error, or largest underestimate of home price)3. A maximum error (most positive error, or largest overestimate of home price)
    You get to choose any confidence value you like for step 1 above . Most people like to use the number 68%, 95% or 99.7%. Do you remember the 68-95-99.7 rule? What does it mean? Can you use it to construct a confidence interval? Visualize the standard normal distribution curve in your mind. Think about those % confidence values and which parts of the area under that curve that they represent. You can even plot your own histogram of your errors. You can then plot your standard deviation (RMSE) value on that plot, just like you did in the statistics exercises in the curriculum.
    #procedure #recipe #teacher #approach"
-
  Q: How is a recommendation engine different from a normal Data Science predictive model?
  A: Most Data Science problems involve fitting a model, making predictions with that model, then evaluating thos prediction to see how well they're doing at solving your problem. A recommendation engine is trying to predict some action or value associated with your customer likeing or consuming a product or bit of content. So there may be a bit more feature engineering involved to get your data into this form. In the end you just need a table that contains features and a target variable (like Likes, clicks, or whatever). Once you get it in that form, it's just machine learning and data science as usual.
-
  Q: I have a model that uses financial reports to predict maximum 30-day % drawdown exceeding some threshold in the subsequent 2 years. It gets good recall (80%) and OK precision (60%) as well as F1 score. I think I can improve recall by creating an ensemble model that first predicts whether the max drawdown will be 0% or something very small and then using that to filter out some of the False positives to improve the model precision.
  A: That sounds like a very hand-crafted algorithm. You will likely have more success by turning the problem into a regression or ordinal classification.

