In honor of the Sentiment Analysis Symposium this week in San Francisco (you are going to be there, right?), here's a summary of best practices for tuning sentiment. These will work for any sentiment analysis system, but you should use ours.
Because it's the best, 'natch.
1) 2 datasets: Gather a set of documents and split it in half.
2) 2 people: Have 2 people tag each dataset for sentiment, and have 2 people participate in the process of scoring sentiment bearing phrases, that way you can mitigate the risk of slanting the tuning too much towards one person's biases.
3) 2 tests: After re-modelling or modifying the sentiment scores of sentiment bearing phrases, test against the half of our dataset that you did not use for step one. Check to see how well it agrees with the tagging that the two of you assigned. Then, for your second test, run it against the first half to make sure that you didn't make things worse. You probably didn't, but "mistakes can be made".
Hope to see you in San Francisco in a few days...