Style:    Language:

Author Topic: Cton skill calculation  (Read 1627 times)

0 Members and 1 Guest are viewing this topic.

Offline Mu_

  • Warrior
  • **
  • Posts: 81
Cton skill calculation
« on: December 11, 2009, 11:08:18 PM »
Can someone explain how is the cton skill calculated currently? It seems that skill gains in cton are reduced by a large factor (someone said 1/4 but I think this is more).

I won a cton the other day vs. 10 other people. 11 player cton... can you imagine how likely you are to win something like that, with all the variables in a game of civ? One of the players had higher skill than I, and several had comparable skill.

All i got was 10 (TEN) skill points. How is this possible? Teamers always get between +/-5 to +/- 15, while winning a very low odds game like 11-player cton gives you 10 points??

This means there is very little incentive to play ctons once you reach somewhere around 1650 - 1700 skill. At that level, 2nd place gets you a loss many times. This I kind of understand, you play vs. lower ranked players, you should be able to win 1st. But then when I do... I get 10 points??

Please advice or comment. I think cton skill calculation should be revised a bit. What is the current formula?

Thank you,
Mu_

The game in question is listed below:

CTON: (ID: #4102) Final Standings - 1. Mu_ (85) [+10], 2. ArmoredSkeleton (101) [+8], 3. negotiator (323) [+12], 4. SirPartyMan (305) [+7], 5. Bopper (163) [+2], 6. BrIMz_LOw (78) [-16], 7. DustyDragoon (167) [-4], 8. Stif1 (558) [-2], 9. jslayton (626) [-1], 10. BladerJustodian (564) [-25], 11. AdminDummyAccount (455) [-32].


Offline Spyro

  • Archer
  • ***
  • Posts: 177
  • Gender: Male
Re: Cton skill calculation
« Reply #1 on: December 11, 2009, 11:51:06 PM »
This means there is very little incentive to play ctons once you reach somewhere around 1650 - 1700 skill. At that level, 2nd place gets you a loss many times.

As far as I know that is completely the opposite since you even win a cton if you are 2nd or 3rd place in most cases, assuming more than 5 players playing it. I know some cton players playing only ctons cause of that fact.

Though, I can't help you with the calculation formula since I don't know it either  8) 8) 8)
There is only ONE truth out there. That truth will be revealed in any case. The only question is the time.

Offline SirPartyMan

  • Head Tournament Director
  • Global Moderator
  • Ironclad
  • *****
  • Posts: 1197
Re: Cton skill calculation
« Reply #2 on: December 12, 2009, 02:45:03 AM »
I remember reading several emails from SLaughter about this subject. I  know the determination of who wins and loses is based on skill level entering the game.  For example 5 people won and 6 people lost skill points in the game cited.  #5 Bopper got a win because #6 is higher skilled than him. #6 was a loser because #7 had a worse skill level. 

SPM

Offline MiroMcDuck

  • Settler
  • Posts: 17
Re: Cton skill calculation
« Reply #3 on: December 12, 2009, 04:05:27 AM »
I'd like to know the formula too. When I win a cton, I get like +2 or +3 points. When I'm second I lose maybe -5 points. Now that's a little demotivating. However this far I understand - there is a big skill leap, so I'm supposed to win no matter what.
But then I noticed funny fact which I dont understand - when I win 3-player cton, I get more points than when I win 7-player cton (containing those 2 people I played previously in that 3-cton). I assume that the formula somehow computes average skill of all opponents and then makes a win against that skill. If it really works like that, then it's very wrong.

Offline Zhenon

  • Clan Killer
  • Civ4 - Admin
  • Great Engineer
  • ****
  • Posts: 2876
  • Gender: Male
Re: Cton skill calculation
« Reply #4 on: December 12, 2009, 04:10:01 AM »
Yeah I don't play ctons any more.  They're too risky once your skill gets past 1700.  You pretty much lose skill unless you get 1st place and then if you make 1st, you get like 3 skill points.

Why is it okay to farm 1v1s and clan teamers, but not ctons?  Anc SS is inherently unfarmable (someone always dies to warrior or something).

Now that we have a different league staff, perhaps we can discuss changing cton site settings again....
« Last Edit: December 12, 2009, 04:11:44 AM by Zhenon »
[PPP]jobe_c4f: you want my mod zhenon? that's rather pathetic, what you're doing O_o
[PPP]jobe_c4f: i feel just sad for humaniy seing it

"ok see you in civ5 bitches, you'll know me because I'm the guy who doesn't need to change his name."
-- 1p0r1g1n41g

Offline CanuckSoldier

  • Owner/Global Admin/Operator
  • Global Admin
  • Commando
  • ******
  • Posts: 6735
  • Gender: Male
    • Civilization Players Leagues
Re: Cton skill calculation
« Reply #5 on: December 12, 2009, 04:51:27 AM »
The only thing I can adjust about Ctons is the "CTON balancing multiplier
(higher value = less skill gain/loss)" in the Admin panel.  Right now it is set at 0.4.  But like the general RD value, this is a double edged sword.  It will not only make you gain more when you win, it will make you lose more skill when you lose.

Really this is a realistic result of this being a skill based league, the problem with Ctons of course is that they are more effected by random factors than team games or 1v1 games, and their is no real way for us to compensate for the luck factor in Ctons.

But yes if you are a 1700+ player and like ctons you are going to have to only play in "elite" ctons of high skill players, or except the fact that you have to win a cton if you play in a mixed bag(skill wise) or you will lose points even if you come in 2nd as the Glicko system expects that if you are 200 points better than every other player that you should win hands down.

Of course team games and ctons don't have this issue as Team skill is averaged, so while you may have 1700 points your teams average might be 1600 and as long as the other team is around the same average then you are still going to win or lose an average amount of points.  Totally unbalanced(skill wise) teamers are rare, while in ctons this happens alot.

I'll try and dig up an old post of Slaughters to explain the math behind the cton skill calculations, we had several large discussions at the time the league started here.  But as far as I'm concerned the cton glicko system is working as it was designed, we are just seeing the leagues general skill level improve across the board, and the luck factor is showing itself to be more of an issue now that it was at the start, but there really is no solution for that, that we can code.

CS
Owner/Global Admin/Operator
Civilization Players Leagues
www.civplayers.com

[CC]Bantams

  • Guest
Re: Cton skill calculation
« Reply #6 on: December 12, 2009, 08:07:49 AM »
total bs why waste 3 hours playing a Cton to gain nothing from being 3rd or 4th no wonder so many ppl withdrew wasnt the league built around Ctons and not anc ss Teamers?

eg

- CTON: (ID: #3717) Final Standings - 1. lekar (67) [+7], 2. Don_Stason (175) [+8], 3. DiVerGenius (387) [+6], 4. Bantams (221) [+-0], 5. kazimo (549) [+-0], 6. cashy111 (595) [-6], 7. PetetheCivilized (120) [-15].
« Last Edit: December 12, 2009, 08:09:48 AM by Bantams »

Offline [KWT]fed1943

  • Scout
  • *
  • Posts: 39
Re: Cton skill calculation
« Reply #7 on: December 12, 2009, 11:00:11 AM »
So, you play not to have fun, pass your free time,learn, even to win a game,
but to win points for ranking.
That's a choice.

Offline Mu_

  • Warrior
  • **
  • Posts: 81
Re: Cton skill calculation
« Reply #8 on: December 12, 2009, 11:29:48 PM »
Fed,

It doesn't work that way man... The league exist for three reasons: to have games without quitters, to have fun and to see if/when you are improving. All three are important and should be attended to. You may not care about rank and skill - I care only because it shows whether I am improving or not. Most of us are competitive enough for this aspect to matter a bit. No, I do not spend hours looking at my scores and skills, but I do like to be rewarded for a nice win.

Offline Mu_

  • Warrior
  • **
  • Posts: 81
Re: Cton skill calculation
« Reply #9 on: December 12, 2009, 11:32:30 PM »
Canuck,

Is the modified 0.4 regardless of how many players are in the game? Why is it that Miro got more points when he played vs. 2 players, than when he played vs. 6 players? He said that the 2 players in the small game were ALSO in the large game, so unless the modifier depends on the number of people, the situation he described cannot happen.

I am good at math - recovering that conversation would help a lot.

Offline Zhenon

  • Clan Killer
  • Civ4 - Admin
  • Great Engineer
  • ****
  • Posts: 2876
  • Gender: Male
Re: Cton skill calculation
« Reply #10 on: December 12, 2009, 11:38:32 PM »
Did you calculate his RD as a factor? 
[PPP]jobe_c4f: you want my mod zhenon? that's rather pathetic, what you're doing O_o
[PPP]jobe_c4f: i feel just sad for humaniy seing it

"ok see you in civ5 bitches, you'll know me because I'm the guy who doesn't need to change his name."
-- 1p0r1g1n41g

Offline Mu_

  • Warrior
  • **
  • Posts: 81
Re: Cton skill calculation
« Reply #11 on: December 13, 2009, 07:17:11 AM »
True, that could be the only reason why - except if the modifier depends on the number of players in the cton.

Offline Mu_

  • Warrior
  • **
  • Posts: 81
Re: Cton skill calculation
« Reply #12 on: December 13, 2009, 08:13:35 AM »
As a statistician, I have thought about this problem and propose a new method of cton skill calculation. I started a thread to this effect and posted all aspects I could think about it. Please read it and provide comments - maybe we even get it implemented if it is liked by people.

The new method is based on the average skill of the cton game, similar to what is done with teamers today. Let it be know that the ELO/RD system is still used to calculate basic skill gain/loss, but the modifiers are changed to provide a, in my opinion, fairer score system for ctons.

Offline SirPartyMan

  • Head Tournament Director
  • Global Moderator
  • Ironclad
  • *****
  • Posts: 1197
Re: Cton skill calculation
« Reply #13 on: December 13, 2009, 09:23:24 AM »
Here is one of Slaughter's original emails explaining his implementation of the Glicko rating system we use:

"OK, I finally finished the Glicko rating system, it is now fully working for every game style (I left out epics since they are basically like ctons – if we decide to use Glicko I will put it in, saves a lot of coding). For every game being reported the system now calculates the new skill with both the Ante system AND the Glicko system, both values are stored in the database then and can be compared easily (I put in a new section where both values are compared for the actual standings). We can track how each system is performing during beta and select the better for the final version … since the Glicko uses more information than the Ante system it needs some more fine tuning so it works best, the values to determine are r = the initial skill after registration (r is set to 1500 atm); RD = the initial rating deviation (RD is set to 300 atm); c = the RD volatility multiplier and t = the inactivity period modifier (for now c = 60, t = 1). Last two are needed to calculate a new RD value after a player was inactive for a longer period, t should be a value which describes how many rating periods a player was inactive (at the moment a rating period is set to 1 day, so for each day being inactive t is increased by 1) and c is a constant that will be multiplied (simply spoken) by t to determine what is added to RD to simulate the deviation volatility. The value of the constant c can be derived by a formula from the starting RD and the rating period, so basically we only need to set the starting values and the rating period.

 

That are lots of values though, let’s have a look at an example to make it more clear:

A Player has a skill of 1500 at start with a RD set to 300. A simple explanation would be that the system now assumes that this player’s skill lies somewhere between 1200 (= 1500-300) and 1800 (= 1500+300) skill since the player can’t be rated exactly because the system did not yet compare this player to any other players. After his first report the player’s skill and RD values will now change, no matter if he wins or loses, the RD value will ALWAYS decrease since the system ‘learned’ something about the player’s performance and has at least 1 more comparison to another player. The more games a player plays and the more frequent he does it, the smaller will be the RD value and the more exact will be the actual skill value. Basically the rule is, the greater the RD value the bigger the skill gain or loss. Let’s assume that in our example the player plays against a player with 1900 skill, if he wins it’s really hard to say if it was an equal matching or not, it might have been actually 1200 vs. 1900 skill or only 1800 vs. 1900 skill, so the player will gain a lot of skill for that win and the RD value will be decreased. Let’s say he gained 200 skill and his new RD value is now 250, so for the system his real skill will now be somewhere between 1450 and 1950, still not really exact but better than before. With a decreasing RD value the skill gain/loss will decrease too until the actual skill value becomes stable and does a good job representing the player’s performance. So in the beginning players will have big jumps or drops in skill until the system learns more about those players’ performance and can rate them good enough … now it comes to the volatility of the RD value, where c and t enter the scene.

Let’s say our player did play regularly for quite some time and now has a skill of 1800 and a RD value of 60 which means that his ‘real’ skill lies between 1740 and 1860, so the skill value does represent the player’s performance somewhat exact now – but suddenly the player stops playing because of unknown reasons and becomes inactive for let’s say 7 days (wives can be very annoying :p). After returning and playing his first match, the system now recalculates his RD value before it actually calculates the gain/loss of skill for this match because it assumes that a player will perform a bit worse after a break without ‘training’. Now with a rating period set to 1 day the t value for this player would be t = 7. Putting this t and c = 60 into the new RD calculation it will give 255 as the player’s new RD value, so the system assumes that the player’s skill now is more imprecise by 55 after the break and will go into the skill calculation with 1800 skill and RD = 255.

 

Back to determining the best values, the starting RD value will also be the highest value that can result from a recalculation after inactivity, that means that a player who was inactive for a really long time is only as bad to rate as a new player. Also the rank drop for being inactive needs to compensate the greater gain of skill after inactivity caused by a higher RD value, we don’t want players to exploit the system by determining a good value to stay inactive just to gain more skill for their next match.

I’m not sure about the best values yet, but my calculations showed that the system did perform well starting with 1500 skill, RD = 300 and with c = 60, t = 1 day – but I think we will get a better view once testing started. Since the RD volatility is calculated with a Pythagoras like formula we might need to change the skill drop from percentage (like it’s now for the Ante system) to a slowly increasing exponential function if we use Glicko for rating. It’s also possible to change values after the system did run for some time and ratings become stable, it will not change much at all but lowering the initial RD value for new players will then prevent too heavy fluctuations, but that’s something to be seen later. It might also be an idea to have a minimal RD value which ensures that there’s always some movement in the league, some players with 3+ games a day may get really low RD values so their skill gain/loss will become somewhat stagnant unless they play players with a big gap in skill. For now all values are hard-coded, but if you like to have the possibility of changing values regarding the rating system I will make them changeable for the headadmin in the league settings later."

Hope this helps.

SPM

Offline SirPartyMan

  • Head Tournament Director
  • Global Moderator
  • Ironclad
  • *****
  • Posts: 1197
Re: Cton skill calculation
« Reply #14 on: December 13, 2009, 09:24:59 AM »
Here's another email of his explaining the logic he implemented;

"As you might have already read in the forum thread people are arguing about the Ante skill system being useful and precise or not. We knew it had some glitches and it won’t satisfy each member, at most players who think they are skilled above average since the system favors the less skilled players more. They mostly pointed out the weaknesses with extreme values and skill differences within a team, at least some did provide more or less suitable improvements, but nothing what really would eliminate all ‘imbalances’.

I’m not really convinced though that the system will fail in practice, I guess in the long run the ‘bad draws’ will be minimized and it will perform quite good and stable – IMHO the skill distribution will emerge as a steep Gaussian bell-curve with only a few extremes outside of a standard deviation range. Too bad my math is not really good enough to make a practical model of the Microsoft TrueSkill System (for anyone who is interested, more info at http://research.microsoft.com/mlp/apg/trueskill.aspx - check out the details, FAQ and the provided technical report); but at least they do provide links to other rating systems – and there is one that is really similar to this TrueSkill System, namely Glicko (resp. Glicko-2); but it’s not that complicated and all needed functions are provided in written form (at MS they did not give all needed information but only described how they would work in theory – I guess a good mathematician may be able to derive those functions and missing constants from the given papers, but not me ;).

 

The Glicko system does provide not only a rating value, but additionally a rating deviation value that describes an uncertainty in the actual rating, that means that the system can’t really say if a player’s rating is exact or not, but that it lies within a range of 2x the rating deviation around the actual rating value. The final rating calculation takes into account, that if you play versus a player with a high deviation value you will not gain/lose as much skill than playing against a player with a small deviation value, because the range where the ‘real’ rating lies in is too big to say if you really played versus a much better rated player or not (or even a lower rated one). Also there is provided a rating volatility value which describes the degree of expected fluctuation in a player's rating and thus is an indicator of a constant performance over a series of games (somewhat similar to a streak, but not that vulnerable to erratic performances). In addition to those values there is another variant in calculations, a time factor of inactivity which will increase your deviation value before the actual rating calculation, it represents that the rating will become inexact over a longer period of inactivity (a common example would be a player not playing civ for 2 months and then doing his first game, with changing popularity in game settings and style that player will not perform that good with a high probability). For more information about the Glicko system have a look at http://math.bu.edu/people/mg/glicko/glicko2.doc/example.html.

 

The extended Glicko-2 system also provides something like a ‘series of games’ where normally unconnected games will be summed up to a single calculation, this would perfectly fit for a CTON or FFA type game. The only thing that it does not cover are teamers, but with the basic model of the MS TrueSkill System I think I will find a way to incorporate them into the Glicko system. Currently I’m working on this and will post again if I find a suitable solution, for now I’ve already made an implementation of the original Glicko system to work with the league system. I also made the skill calculation interchangeable, so if we hit the beta phase we might try and test both, the Ante and the Glick system, parallel and see which is performing better - for now I will keep the Ante system for use in the league and make the Glicko system the default rating system with my next server update after I found a solution for the teamers.

 

NumberOneMercury also said that he is an advanced mathematician and that he might come up with a better rating system, if he shows some results we can even consider trying a third rating system during beta – at least I will implement it and give it a try on some test runs. For now I think I will not post anything about the Glicko system in forums to avoid another quest for weaknesses before even finishing the implementation."

Best, SPM

 

League