A Wind Power Forecasting Problem
- Kaggle project title: Global Energy Forecasting Competition "A wind power forecasting problem: predicting hourly power generation up to 48 hours ahead at 7 wind farms"
- URL: https://www.kaggle.com/c/GEF2012-wind-forecasting
- Programming Language: R
- Class: Machine Learning with Prof. Andrew Ng
Evaluation: Initial models
- Goal: predict power at 7 wind farms.
- Inputs: 48 hour look ahead forecast for wind speed, wind direction, and the zonal and meridional wind components at each of the farms
- Outputs: for each of the 7 wind farms, normalized wind power measurements between zero and one.
- Model Identification and Training period: Sept. 1st, 2009 to Sept. 30th, 2010
- Evaluation period: Oct. 1st, 2010 to Dec. 31st, 2010
- In Evaluation period: Repeated 72 hour pattern. First 36 hours: we are given the generated wind power from 0 to 1. Remaining 48 hours: Missing data of generated wind power, to be predicted by our model.
Evaluation: Improved models
- Use two time series prediction models: ARIMA and locally weighted regression.
- ARIMIA model: trains itself based on the hourly generated power over all of the training set
- Locally Weighted Regression model: gives more regression weight to the training instances that are similar to the 36 hour given data in the prediction intervals. But, this often leads to huge error in the prediction.
- ARIMA model predicts better than Locally Weighted Regression model, even though ARIMA model tries to average out the prediction.
- Improved these models by adding wind speed features that we chose in our feature selection procedure through correlation to the generated power.
- New ARIMAX and Locally Weighted Regression with Wind Speed models which had a quite reasonable performance improvement over ARIMA and Locally Weighted Regression and beat the benchmark method by almost a factor of 2.
Trust and Helpfulness in Amazon Reviews
- Reference 1: C. Danescu-Niculescu-Mizil, G. Kossinets, J. Kleinberg, and L. Lee, “How opinions are received by online communities: a case study on Amazon.com helpfulness votes,” in Proceedings of the 18th international conference on World wide web, pp. 141–150, ACM, 2009.
- Reference 2: G. Wang, S. Xie, B. Liu, and P. S. Yu, “Review graph based online store review spammer detection,” in Data Mining (ICDM), 2011 IEEE 11th International Conference on, pp. 1242– 1247, IEEE, 2011.
- Programming Language: Python
- Class: Social & Information Network Analysis (now called Machine Learning with Graphs) with Prof. Jure Leskovec
- Data: “Web data: Amazon reviews” dataset available via the course website. The full dataset contains 34,686,770 reviews.
- We focused on a particular dataset referred to as the “Fine Foods” dataset. The “Fine Foods” subset contains 568,454 reviews from 256,059 users.
- Each review has a number of positive ratings and negative ratings called helpfulness votes.
Using Review Helpfulness Votes To Obtain a Helpfulness Metric for Users
- Motivation: Amazon has over 34 million reviews. While many reviews are helpful, there are also fake reviews written to skew product ratings: Book authors may have fake reviews say how great their book is, and manufacturers may write to fake reviews trashing a competitor’s product. Though users can vote a review as either helpful or unhelpful, previous ratings on a product influence how a user votes: If high variance in review star ratings (reviewers disagree), then polarizing reviews (1 or 5 stars) are voted as most helpful. If low variance in review star ratings (reviewers agree), then average reviews (near product mean) are voted as most helpful.
- Problem Statement: How do we tell which reviews are authentic and identify fraudulent reviewers?
- Use graph based algorithm to determine the honesty of users’ reviews and trustworthiness of the user performing the reviews.
- We modify the algorithm for our purposes to use a single store and apply it to the "Fine Foods" Amazon review data. We analyze the user trust ratings computed from the algorithm with the user helpfulness metric in order to classify fraudulent users.
Algorithm To Compute User Trust
- Motivation: We want to determine the helpfulness rating of a user from all number of positive helpful and negative helpful (unhelpful) votes that all his/her reviews receive. Suppose a user wrote 2 reviews, the first one has 95 positive votes and 5 negative votes, the second has 1 positive votes and 3 negative votes.
- Method 1 of User Helpfulness calculation: sum total of helpful votes only / sum total of both helpful and unhelpful votes. The user’s rating is 96/104 where the range is 0.0 to 1.0.
- Method 2 of User Helpfulness calculation: sum total of helpful votes only - sum total of unhelpful votes only. The user described above now has a helpfulness rating of +88 where the range is negative infinity to positive infinity.
- Using Method 1, users with a small number of total votes achieve perfect helpfulness score. In Method 2, we achieve the desired quality where most users now have ratings close to a mean (zero), and just a few outliers have a large positive or negative score. Method 2 solves the problem of rating users who have few helpfulness votes in their reviews.
Conclusion: Relating User Helpfulness (from votes) with User Trust (from our algorithm)
- User trust (-1 distrust to +1 trust) Users are trusted if their reviews are honest, users are untrusted if their reviews are dishonest.
- Item reliability (-1 unreliable to +1 reliable) Items are reliable if they have high scores by trusted users, items are unreliable if they have low scores by trusted users.
- Review honesty (-1 dishonest to +1 honest) First compute review agreement: it is high when review agrees with majority trusted opinion. Review agreement is low when it disagrees with majority trusted opinion. Second compute review honesty: Normalize review agreement and take into account item reliability.
- User trust, product reliability and review honesty are computed through iterative updates: Each variable is initially set 1 and updated many times in a loop that interdependently computes their values.
- When just outlier values are considered, there is a strong correlation between user trust (computed from our algorithm) and user helpfulness (calculated from the second method). For example, 87% of users with helpfulness ratings less than -100 have trustworthiness ratings less than -0.9. So our algorithm using pure graph analysis could be helpful in fraudulent user detection.
- Additionally, a bot that rates every product 5 stars out of 5 will fool our algorithm into thinking it is a helpful user. This is due to the skewed ratings in the data where the products, fine foods, are most commonly reviewed as 5 stars and second most commonly reviewed as 4 stars. Textual analysis of review text, such as searching for common spam phrases, would likely improve our results.