Tea Companies Rating Their Own Teas

I’m still trying to figure out if there’s a way I can rate my teas higher than 100. Just kidding.

…sorry, can you repeat that? I can’t understand what you’re trying to say.

Also, I wasn’t talking about 52teas, but we can include them in this discussion.

Sure, let me see if I can put it a different way… 52teas is different from most other companies in that they make their own blends, it’s not like they purchase a tea from a farmer or import it and sell it as is. So they have the chance to tinker w/ it to find the perfect blend, just as one of us may blend teas we have in our cupboard.

Right…well, most tea companies make their own blends and flavor the “plain” teas they buy from their farmers, so I’m not sure how 52teas is any different (other than the fact that they are smaller than, say, Harney and Sons or Mighty Leaf).

Why do you think the fact that they flavor their own products makes it okay for them to rate their own teas that they sell for profit 100, skewing the ranking system?

Because it’s not my job to tell them what to think of their teas. While everyone’s taste is different, I’d think a company would want to sell teas that at least they think are 100 worthy because they can only speak for their tastes. If you want to talk about skewing the ranking system, let’s talk about the way the ratings are calculated. Personally, that bothers me a lot more.

Sorry, I’m still not quite understanding your argument…why is it okay for companies to rate their own teas as 100, thus skewing the “unbiased” ratings on a website that is unaffiliated with any specific tea company?

I’m not asking for companies to say “hey this blend of ours sucks and isn’t as good as our other one”. I also would assume that most companies love their own tea blends, or else they wouldn’t sell them. Still, how does that excuse their co-opting the unbiased, populist Steepster review system for their own financial gain?

Because I would hope they are rating it as such because they honestly think it’s worthy of the score that they give it and not because of where they get it from (themselves). But the ratings are skewed by the formula used to come up w/ the tea’s rating anyway so it really doesn’t bother me.

Cofftea, I’m still not understanding you…why does the fact that a company likes their teas (a given, especially considering that they are the ones making it and they are the ones that profit off of its higher ratings) excuse it from abusing the ratings system? It’s great that you “hope that they are rating it because they honestly think it’s worthy of the score” but what on earth does that have to do with the fact that they are abusing a system designed to be a reflection of the opinions of unbiased, real tea drinkers?

I agree with your suggestion.

Enforcing it might be difficult, as there will always be those people masquerading as individuals to uprate a certain company’s teas, but Steepsterites aren’t stupid and can usually figure out who those people are and report them.

Here is an idea for the overloads. Have all ratings not reflect the outlying ratings. For instance. if a tea has ratings that range from 100 to 0. There are 5 ratings, one = 100. 3 = 75, 1 = 70 and 1 = 5. the ratings of 100 and 5 are dropped and the rating is then calculated. This way when users look at a rating you don’t see the complete lovers and you also don’t see the haters either you see normal range. Granted not perfect.

Yes, I agree. I would still like them to add tasting notes about their teas (as often there is a lot more to be said about a tea than what is on the back of the packet), but it would make sense to disable them from rating.

I like your suggestion Miss Sweet…that way, companies can still promote their teas and get publicity for them without abusing the system and deceiving the people who look at Steepster as a reflection of the reviews of real people tasting teas.

Jade Teapot, I disagree. Personally, I’d be very offended if my ratings of teas I love and dispise weren’t counted. People may not agree w/ my opinion, and that’s ok- I don’t expect them too, but it’s still my opinion and my opinion matters. And I think people would get smart quick anyway by just giving the teas a score of 99 to avoid that. Before they do that, I think they should redo the formula they use to get the “averages”. But that’s just me.

Shanti – I agree that Steepsterites are smart :) but I don’t think that tea companies that abuse the system are deceiving anyone but themselves. I think we are all savvy enough to spot them and personally, I would never buy a tea from a company that engages in that sort of behavior. Let them dig their own grave I say :) I think we are all smart enough to see which companies give thoughtful reviews and ratings. The ratings are just one of many guides that we use to choose our teas anyway – I would hope that no one would click that link above and choose teas solely on what it says.

JacquelineM, I agree. The Steepster guys are also quick to act on less than noble behavior. I guess I’m not following the right people, because I haven’t seen this happen. Which is fine by me:)

I know that most people take more things into consideration than the rating, but it would be nice to make the rating system a bit more honest.

Yeah, that’s very true Jacqueline :) Although I have to admit, when I’m looking for a tea to add to my order from a company (to get to the free shipping limit, or something) I look at the teas reviewed from that company on Steepster. And I click on the ones that show up on the higher rating end and read the individual tasting notes, but the skewed ratings still affect my tea purchasing.

Yeah, the idea of people on Steepster being sleeper agents for companies is really scary. As 52teas mentioned below, there’s no way to know that Jane Doe is not really working for Company X and reviewing their teas well and the other teas badly.

Hi Cofftea I understand where your comming from and do agree in some cases. I just put an idea based on the thought that there will be a top % and bottom % that may not apply. That way the score may reflect the 90% population. If 100% of votes were at 100 then the tea would be a 100 But if 90% of users voted 100 and 10% voted a 5 you would still be a 100. assuming that the ouside the norm is an anomaly

Jonas – I see it as what do we gain and what do we lose by setting more rules here – I guess I feel like by setting more rules we lose more autonomy. I’m willing to take a few unsavory things here and there in order to keep freedom. For example – if tea companies were unable to post ratings, I would lose seeing the Harney & Sons The Store ratings, which I find VERY valuable.

JacquelineM- I completely agree. Not to make more work for the steepster guys, but I think that they can take action if something should arise- especially if one of us sends them a quick PM. They’ve been great about it before. I guess I’m different in that I don’t assume or expect bad things from others. I, too, love seeing H&S TS ratings and tasting notes. Though I wish they’d include more “if only” points. A score of 85 is very good- but I like to know what makes it not a 100, no matter who is drinking it.

Hey JacquelineM, I too love H&S’s account here, and I’m not saying that they shouldn’t be allowed to have accounts. I think they shouldn’t be able to assign a numerical rating to the tea, but they should be allowed to give their tasting notes. They could even say “I would rate this as an 87” in their tasting note. And I would argue that having a bunch of biased reviews here is the opposite of an autonomy, at least in regard to the autonomous, non-preferential ranking of tea. Not to mention, not every tea company has the resources to continuously promote their teas and to counter their own teas being demoted by unsavory accounts. And in the end, even if the companies do come in an promote their teas, we still have an inaccurate tea ranking on our hands that isn’t reflective of the feelings of the Steepster community.

Well said!

Hey 52teas, this makes way more sense to me than a certain other Steepster user’s comments upthread and is actually IMO a valid reason why you rate your own teas.

I think the underlying problem, which you seem to acknowledge, is that companies are allowed to rate their teas at all. You (as a tea company) shouldn’t have to worry about other companies upvoting all of their teas and downvoting yours. That is exactly why I propose that all companies be banned from rating teas.

I also don’t know how we’d make sure Jane Doe isn’t just working for a company and sabotaging others, other than by keeping an eye on people’s reviews and reporting suspicious accounts to Jason and team. I know that when I see a really glowing review or a really horrible review, I check the person’s profile to see what their take on other teas are. I don’t know if others do that. Anyway, by checking the person’s profile, it should be pretty easy to tell when a person is always praising one company and insulting another, and should thus be reported.

Huh, I didn’t think about it like that – I guess a “company” seems more faceless than an individual, but of course you would rate your own teas highly otherwise you wouldn’t sell them.

Oh, I just thought I should add, I think it’s fine for companies to add tasting notes about their teas as long as they don’t skew the ratings. You guys know your tea best, and you’d be able to promote your teas without deceiving people or abusing the system.

I know that you’re not rating them that high because you want to sell more, but because you actually like it. The problem is that virtually all companies will like their own teas.

I thinks it’s good that we can see that you are the one rating, but the problem is that this gives companies with a Steepster account an advantage. If a company doesn’t have one, they can’t rate their teas (I’m assuming that their ratings will be honest) and the rating might be a bit lower than if they did.

Jonas, I think you’re hitting on the same point I was trying to make. The answer isn’t that every company should be allowed to rate their teas because other companies do it—it should be to prevent companies from rating their teas at all. That way, companies without Steepster accounts won’t be disadvantaged, and the ratings system will remain more accurate for Steepster users.

That’s what I was trying to say, but I can see my post was a bit confusing.

@52teas, what I would like to see from you somewhere- either on your site or here… not sure where would be best… is the story behind each of your weekly blends: selection of the flavors, trial and error of the blends/formulas, etc. As a tea blender myself (w/ just tea, not other ingredients), I’m sure it’s quite an interesting process!=D

I’m in full agreement with 52teas. It would be silly for us not to have a presence on steepster if others were here. Especially if those other companies were rating our teas with ridiculously unfair ratings. I am new to online tea forums, and so am just getting accustomed to the rules and ettiquette. As such I will make it clear every time I write my own reviews/ratings, etc. I am quite literally a two person company, one in Taiwan sourcing products directly, and one in Canada selling and marketing. We’re small, so we use all free exposure available to us. But I started this company because I’m a purist, i.e. not blending teas because there are so many naturally occurring flavours and aromas and energy. That said, I won’t rate all of my teas 100 simply because within my company I believe I have some teas that have stronger chi energy than others. And I’ve had even better ones in Taiwan that I don’t sell. I don’t sell them to the mass market because they will be extinct in less than two years, and there’s already a high enough demand for it in Asia. So for me, a rating of 100 would go to the 70 year old sheng, wet-stored pu erh I had at a tea master’s shop in Asia for the depth of flavour, aroma and overwhelming energy it held. Actually, I’d try to give it 110. So I suppose for me the ratings are relative to my own tea experience, and for most steepsters I see this as being true. It will be a subjective reflection not only of the tea being reviewed but also past teas each individual has had. I will continue to rate my teas as I see them relative to my past experience with tea (and I’ve had both some truly excellent teas and some truly abysmal stuff). I rate my teas and sell what I do because I believe they are better than most of what’s available in North America. I know that sounds rather self-promotional, but I believe in my tea.

Yeah, I mentioned that in my comments, but that still doesn’t really change anything…first of all, there’s nothing stopping companies from making fake accounts right now when there’s no tea regulation, so I’m not sure the number of fake accounts would increase if companies were banned altogether. If anything, the penalty for getting caught (shaming and deletion) would be more severe than it is to try doing that now, so that would in theory dissuade companies from doing so.
Second, I mentioned this earlier: “I know that when I see a really glowing review or a really horrible review, I check the person’s profile to see what their take on other teas are. I don’t know if others do that. Anyway, by checking the person’s profile, it should be pretty easy to tell when a person is always praising one company and insulting another, and should thus be reported.” So yeah, it’s not fool proof, but as long as people keep a look out for fake seeming accounts, we could keep them off Steepster. Once again, we currently could have both fake accounts and company accounts. If company accounts were banned, I’m not that the number of fake accounts would rise.
Third, if a company is caught faking an account, that sends a strong message to the buyer base to avoid that company. I know I stay the heck away from the companies on Steepster that say every one of their teas is 100.

From what I understand, we currently are only allowed to rate a tea once. The rating for all of your previous tasting notes changes each time you change your rating.

Shanti, that’s correct. But to me, that confuses me. I’d rather we be able to give a tea one general rating, but then also be able to have the option (not requirement) to give each tasting note a different rating. As it is now, the only way to know if a particular preparation method is good or bad is to read the entire note. I like to know a tea is good, but I like to know the ways that it’s good or bad (temperatures, parameters, additives, etc).

Cofftea, I too was a little miffed by why they only allow us to use one rating. I guess it makes sense because there’s no other way to calculate the overall rating for the tea. I guess the idea of the “ratings” is not just for the single tasting note, but for your opinion of the tea overall. So, for example, I know that some people will raise their rating of a tea because they find themselves drinking it very often. It’s not that the specific cup was tastier than it usually is, but that their opinion of the tea overall has changed. Similarly, if a tea is really finicky and you don’t like it because of that, one cup might taste great and the others bad, and your overall rating would reflect that “hey, I like this tea when it’s good, but usually it’s hard to get right so I’m going to give this a lower rating than I would to a tea that always tastes good”.

I forgot to add, the only alternatives I can see are:
Option 1) Count all of the individual tasting note ratings when calculating the total rating. Why that’s bad: say the first 10 times you made a tea it was horrible, but the last 2 times you’ve made it it was the best tea you’ve ever had. The score would be really low and wouldn’t reflect the fact that you were preparing it wrong/your tastes changed/the tea changed and would just show a very low overall score.
Option 2) Have the most recent tasting note score by the reviewer be counted in the calculation of the overall score. Why that’s bad: If we’re rating each note as you suggested, reflecting the way we prepared it that time and how we would rate that specific preparation, then the overall score would only be representative of your preparation of the latest cup you drank, not your feelings for the tea in general. For example, say you usually make Tea X plain, and it’s delicious each time, and your tea log ratings reflect that. Then, one day, you make it using milk and boiling water, and it comes out horrible, and your rating for that tasting note is very low reflecting your feelings about that one cup. Then, the overall ranking for the tea would go down just because of the most recent cup you drank was awful, not because the tea is awful.
Option 3) Let users choose both a ranting for the specific tasting note, and a rating for the tea overall. Why that’s bad: This one is just confusing and seems like too much work. When you look at the dashboard, you’d see two numbers, which might confuse you, especially if the person doesn’t explain why their tasting note is ranked one way and their overall is ranked another way. Calculation wise, this would be the most accurate and the most complete archival history of your changing opinion of the tea, but it also isn’t quite as user friendly or simple as just choosing a single rating. When rating a tea, it’s a lot more work to have to parse out “well this is what I feel about the tea overall at this moment, and this is how I feel about this specific cup of tea…wait, do I feel the same? Do I feel different? Should I uprate the overall because this one might have been a dud? Vice versa?” It takes away from the simplicity of use and design.

What confuses me is having to think “ok… this is not an actual average… this is based on some algorithm that I don’t understand”… and the fact that this isn’t obvious- I can’t even tell you where I found that information. If they’re going to have an “average” score- or something that appears to be as such, it should be just that- an average. I happen to think this is the least complicated and most straight forward, but that is again, just me. There’s an entire thread devoted to ratings so this is really getting off topic. I love the flow of conversation.:)

Thanks guys! Now I understand the rating system better. And oops in my original post, I meant to say sellers allowed to “rate” their own tea, not “drink”. I don’t think the seller’s one-person opinion counts much, and I guess the steepster system can manage to detect people who register multiple accounts with a purpose of giving false rating.

See, normally I’d agree and say one person’s opinion doesn’t matter much, but the truth is it does. The reason I started this post is because the highest ranked teas on Steepster all have ratings of 100 from the tea companies themselves. When it comes down to it, the company’s ratings does have a big impact on what teas get ranked highest and, ultimately, what teas people buy.

Oh I see! I am still exploring the steepster. Now I realize when you search for top-rated teas, it could be a problem if a top rated teas are rated by only a few people, including the seller. Then probably the system can set a rating threshold for the top-rated tea (like at least 10 or 20 ratings to be listed among the top-rated teas)? If problem is severe to certain degree, maybe it’s better off that tea seller’s ratings are not counted. And meantime, to prevent multiple accounts registered to purposefully help a seller (well I still doubt more than few tea companies would spend time on this but just in case), probably the system can only count a user’s rating after s/he finishes xx numbers of tasting notes and reaches xx days of user history. Just some random thoughts.

denisend, I was specifically referring the large number of teas from the California Tea House that only have 2 to 4 reviews on them, and have 100-rated reviews from the company itself skewing them. I’m not saying their bad teas; I’m just saying that it’s unfair that, for example, a tea with only 3 review, one of which is the actual company, can be the second and third best teas on Steepster, while other teas that have a lot of really high scores from real Steepster users (such as Dawn from Simple Leaf, Pina Colada Honeybush from 52teas, and Almond Cookie from Specialteas) get bumped down. Indeed, California Tea House takes up six spots on the “best teas” list, due to the company’s ratings of 100.

The other thing is, is there any way to see who has rated a tea? For example, we can see who has written tasting notes, but not who has rated a tea…so a company could easily create an account and uprate their teas without anyone knowing. How is that fair? How is that helpful at all to the community?

Hmmm…. I did just do a test, and it seems that if you post a number with no words, it won’t show up on the individual tea’s page (just on your own page).

I guess what I’m trying to get at is that my decision making process (as I outlined above) is guided by reading the reviews, not by a single number. I don’t ever browse by number, especially across companies (I might look at a single company’s ratings when I need a bit more to qualify for free shipping or I’m already placing an order, but I’m actually more likely to sort by “most popular” rather than “highest rating” since I want to read people’s experiences (and, as you note, some of the highest rated don’t have many reviews)).

More thoughts…. California Tea House’s Yunnan Golden Buds (currently #2) has been rated 4 times, once by the company. It hasn’t had a rating below 90. How many teas on this site does THAT apply to?

CTH’s Tea Tongue, on the other hand, has 3 numbers, but only one log, and it doesn’t have much detail. I’d say that it doesn’t belong there. But, again, I’m reading the page and making my decisions based on that.

If you read the page that I linked explaining the system, teas that don’t have many ratings shouldn’t be able to be listed very high in the system, so as the tea gets more ratings, the 1 rating from the company should be downweighted (as long as companies don’t come on here downrating other company’s teas – THAT I have a problem with).

Perhaps the algorithm needs to be tweaked if it’s going to be used as you (and perhaps others) seem to be using it.

This was pretty much what I was trying to get at. Let’s not forget that the primary motivation for a company to be on Steepster is to promote their products (not a bad thing, just something that needs to be taken into consideration).

I think you make a lot of good points here.

If your 1/5 of the thread reading did not include my previous post, I would urge you to please read it, and I will try not to rehash what I said there here.

I have rated some of our teas 100. Early on, admittedly to get some attention for our teas (and because i felt they deserved it; though your argument about all of our teas being “the best” sounds logical, I can tell you that all of the teas we offer are created to yield the best, most balanced and awesome cuppa we can create, so if I don’t think they merit 100, we scrap it and start again, so to ME they are all 100s or we wouldn’t offer them, but now I am repeating myself a bit), more recently (and much less frequently) I have rated the teas out of a sense (I clearly remember thinking this, though I imagine you might doubt me) that it was expected when I wrote a tasting note, and again, I can’t see not rating them 100 when that is what I think.

In any event, I certainly won’t be rating any more after the uproar of this thread. I have no interest in being considered sleazy, though (as stated in my earlier post) I feel as though I have plenty of justification for rating my own teas.

I will also say (very briefly, and hope that it doesn’t get me in trouble) that while the system may be flawed slightly due to tea companies rating their own teas, it is equally flawed by individuals’ ratings. I am referring to a few things:

1) Everyone’s perception is different—Our Malted Genmaicha (http://steepster.com/teas/52teas/6498-malted-genmaicha) has ratings of 100, 93, 79 and 9 (excluding my rating of 100). Someone had a BAD experience with this tea. Does that mean it should have a mediocre score even though other people who tried it really LOVED it? Even with my 100, the tea only rates 72. I don’t know what happened with the individual who gave this tea a 9 (Actually, I do know part of the story, but what I’m saying is it doesn’t matter), but that 9 hurt our score way more than my 100 helped. I’m not begrudging anyone’s right to dislike our tea; if that user’s experience was a 9, then, sad as it may be to me, it’s a 9. All I’m saying is that putting too much stock in these scores is crazy; here’s a tea that 3 out of 4 people thought was really good, but one person thinks it’s HORRIBLE and it winds up with a rating of average.

Well, I was going to make some other points, but I just had a thought:
What if, instead of scores, Steepster allowed everyone to create top ten lists? My top ten flavored green teas, oolongs, or chais? Steepster could then assign a point value to each position 1-10 and compile a larger list based on the points each tea earned. Eh, that probably wouldn’t work, and it would almost certainly give the big brands a boost because they are just more prevalent. I don’t know. I’m just thinking out loud, and I fell on the ice today and hit my back on the curb and I am on some heavy duty pain killers and getting sleepy and dopey, so I’m going to shut up now. Goodnight.

I personally don’t think the system is flawed or anything but it is the nature of this website that might seem a bit unfair to some people. Since the people giving ratings on this website aren’t “officially” tea experts or dedicated tea review writers, it is normal to have all sorts of ratings here, whether it is due to the person’s mistake in preparing the tea or the quality of the tea itself. Also, there aren’t that many teas that have more than 10 ratings (only 2% of the total, and yes I did some math here…), so not much can be done about that one “off” rating since it is his/her opinion of that tea after all. So unless a tea gets plenty of ratings so that the score averages out a bit, there really is nothing that can be done about the seemingly unfair rating. I think a good solution to this problem would be to include a like/dislike/OK system along with the numerical score.

52teas, unless one of the steepster guys tells you otherwise and you’d like to, keep logging your teas. My only suggestion (to all companies) would be to log teas other than the ones you make as well. Yes I know people may read negative intentions into that, but I’m not like that. I take a company only logging their teas to possilbly mean “Our teas are so good that I’ll only drink them”. I guess both have down sides. It goes back to “I can only please one person a day, today is not your day- and tomorrow doesn’t look good either”. So my advice being given, I’d say as long as you aren’t breaking the rules of steepster, do what you pleases you.

Thanks, Cofftea, but I think I will probably steer clear of rating our teas in the future. (I haven’t rated many of them recently anyway).

As far as “Our teas are so good that I’ll only drink them,” I had to laugh at this. Seriously, if you could see our shop… We have over 200 varieTEAs of Zoomdweebie’s Teas, a new 52teas blend to create EVERY WEEK and now a new line of Man Teas. I literally don’t have TIME to drink anyone else’s tea.

Truly, I’m not sure I would trust myself to be unbiased in reviewing anyone else’s tea, and even if I was completely unbiased, I imagine someone would get their feathers ruffled by it anyway, even if it was a good review. Somehow I just imagine they would. :)

Well yeah if you don’t trust yourself, it’s best to avoid that. Drink more Zoomdweebie’s stuff! There hasn’t been any logged til this week and I didn’t even know you had a website for Zooomdweebies… but of course now I have a crap load of more teas I want… sigh… grrr…

teaplz, while I absolutely agree that a person shouldn’t tell you that your opinion is wrong no one should whether they represent a company or not and if they do they should be repremanded no matter what so their affiliation doesn’t have anything to do w/ it IMO.

While I understand what you’re saying, Cofftea, I have to respectfully disagree on one point.

People might be intimidated by a company’s presence, even if the company doesn’t post anything on here. You might be more polite about a tea if you’re buddy-buddy with the representative. You might not be honest, or you might “soften” a blow, as you would for a friend. If a company even passively questions you, it’s a very different thing than an individual doing so. A company is a very different thing than an individual.

To be honest I don’t get that, but I’ll just nod and smile and respect your opinion:)

I absolutely agree with you, 52teas. I would rather use the term “tea sellers” than “tea companies”, because when some people talk about a tea company, I guess they have in their mind a money swallowing machine :P

I always believe in world of tea (well let me rephrase, in the world of loose leaf, fine tea), small businesses eventually would dominate. Most small businesses are run by people who are first of all tea drinkers and tea lovers.

Whether a tea seller should rate their own tea, and how should the system manage it, I believe it’s a technical question and opinion discussion, not a integrity or ethical question.

If multiple accounts are registered to give false testimonial, or a user gives extremely low rating to harm a company, I believe that’s severe integrity and ethical problem.

hah, look at all the tea companies scramble to explain themselves! Maybe this isn’t an issue of integrity, but it might be an issue of pride, which neither of you have. To me when I see a company giving out almost perfect (or blatantly perfect) scores to all of their teas it looks like what it is- a sad, transparent attempt to keep ratings in their favor to push a low quality product.

I’ll have to agree with the other posters who already said it, tea companies who rate their own teas (whether they think they’re being impartial or not) is a sleazy practice and isn’t fooling anyone. And though some people may not care, I think it does a big disservice to your teas when you can’t present them honestly.

And I can understand wanting to have a presence in the community to promote your brand, but you can do that in other ways than skewing the ratings.

I think this is a simple conflict of interest. I don’t think you can assume to know why people rate teas the way they do; but we strive to make the site flexible enough to accommodate a wide variety of tea drinkers.

I have spoken directly with many of the tea companies on Steepster and can guarantee they are not “sleazy” and are in no way trying to deceive people to keep them from noticing their “low quality product”. If they didn’t have high quality products we believe Steepster users would enjoy, I wouldn’t be talking with them and I have denied meetings because of this very concern.

You do hit on a good point though, that we are means for smaller companies to establish a presence with the community and promote their brand. And as we move forward we’ll work to make it easier and more straightforward for companies to do this.