[SM Plugin] Emp Stats

Discussion in 'Coding' started by Mikleo, May 13, 2017.

  1. Mikleo

    Mikleo Member

    Messages:
    151
    Likes Received:
    89
    Trophy Points:
    0
    At the moment there seems to be a bug with the mmr system where some players are getting infinitely high or low mmrs, most players are hovering around 1000 mmr where they should. I probably wont be able to get time to fix this till the weekend. But please make sure you are not running an old version of empstats. Send me any error logs and there is also a log in addons/sourcemod/logs/empstats.txt which basically logs everything the mmr system is doing which would be helpful to have.

    And one thing I didn't mention about the teambalance is that you will be free to join whatever team you want if you have played for less than 8 hours. However your mmr will still be counted so other players will need to join the team with the new players. So team balance will not immediately apply to new players or returning vets not registered on the system.

    Edit: Nope Still Happening
     
    Last edited: May 20, 2017
  2. Mikleo

    Mikleo Member

    Messages:
    151
    Likes Received:
    89
    Trophy Points:
    0
    0.24
    fixed issue with crazy MMRs
    Tested and adjusted teambalance feature
    Added an option which prevents players being assigned to the team they didn't select when the teams are uneven. es_teambalance_blockunevenswitch, disabled by default.
    fixed a few issues with predict command
     
    Neoony likes this.
  3. Mikleo

    Mikleo Member

    Messages:
    151
    Likes Received:
    89
    Trophy Points:
    0
    0.25
    Fixed a bug where time_played and mmr were calculated from server start instead of after the comm vote ends.
    Teambalance is now enabled By default
     
  4. Mikleo

    Mikleo Member

    Messages:
    151
    Likes Received:
    89
    Trophy Points:
    0
    Understandably there have been some people commenting how an MMR system with many players on each team will be less accurate than games like LOL or Dota that use 5 player teams.
    Yes MMR systems are generally more accurate the more games you play and the fewer players in each team. Empires has longer games, which does mean less games, probably less than 12 big games a weekend.

    My main point to this is that the system doesn't need to be perfect or incredibly accurate. Server owners have a margin to play with and adjust as they like. Although there has been very few games played in the system you can see a trend emerging in the ratings. Yes there are players that have got lucky and others who have got unlucky with their results, that's just variance. The chance that a team with an average rating of 1000 has equal skill level as a team with an average of 1100 is not very high if the players in those teams have played enough games. In effect the higher team numbers makes this possibility much less likely. Yes those ratings may not be completely precise or accurate, but its unlikely that the 1100 rating team will be worse than the 1000 rating team and if that is the case the system will adjust all their ratings by a large degree.

    Teambalance is just a fallback that stops the MMR difference of both teams getting too high. By default it is set to 100(64% win chance), but server owners can specify whatever margin they want.

    The reason I have set it as default is that I want to test it when it's fresh on my mind. Teambalance wouldn't have applied to a single game over the last weekend. The average MMR of each team was still very close to 1000. It probably wont apply to a single game this weekend either, unless somehow the very highest and lowest rated players went into an 8 player game on different teams, in which case it would be stacked. And server owners can set higher margins which would make it even more unlikely that teambalance will affect equally skilled teams.

    There are two different complaints that It will probably get.

    One complaint will be stronger players should be able to stack the team that is losing. I disagree per my earlier post.

    Another complaint will be I couldn't join the 'weaker team'. This could be because you have a lower than average mmr for the players on the server, which means the system wants you to join the stronger team. The system wants new players or low MMR players to join the stronger team. Or it could be that MMR is inaccurate because there isn't enough data on a lot of the players. In which case the server owner can make the margin higher if they want.
     
    Last edited: May 22, 2017
  5. Mikleo

    Mikleo Member

    Messages:
    151
    Likes Received:
    89
    Trophy Points:
    0
    0.26
    Bug fixes
    Players now have commander ELO rating. Worth around 1.5x normal player rating in terms of team MMR. This is not used in team balancing, mainly used to make the player MMR system more accurate.
     
    Neoony likes this.
  6. Paradox

    Paradox I am a gigantic asshole who loses people's hard wo

    Messages:
    6,920
    Likes Received:
    146
    Trophy Points:
    0
    Your elo system should only be adjusting elos when games become bigger than 10-15 v 10-15.
    Heres why.

    Plenty of players dont even join empires on low pop sizes -> When they play they wont have a properly calibrated elo to work with making the system have way too much variance.
    The people who do play on low pop sizes will have an inflated MMR.

    As you said before Empires works differently from other games cause of larger games and less games in general. I agree with all the former. But I think having your elo rating active all the time is a mistake. It should only be active when games are being played seriously.

    The only downside to this is that actually getting enough games to calibrate properly, but I think a proper slow calibrated MMR is better than a fast calibrated bad MMR.


    Also I really think MMR should be made publicly visible. Not how you calculate it ( people will try to cheat their way up ), but at what. People tend to play more serious and want to win more if they get scored on it.

    This happend in dota when they introduced mmr. People playing unranked tried way less than people in ranked matches.


    ps anyone saying they wont be playing more serious or a majority wont be playing more serious can stfu. Even if 10 percent of the player base plays more serious thatll give better games.
     
  7. Neoony

    Neoony Member

    Messages:
    1,332
    Likes Received:
    89
    Trophy Points:
    0
    Think its kinda too late to not disclose how its calculated.
    Source is already public.
    But I agree it shouldnt be available. To reduce abusing it. Might also affect overall game and how people play it.

    Many games usually dont disclose that.

    With it being public, there is gonna be always somebody trying to abuse it...and then wait until he says his findings to the public xD

    So yeah, cant disagree on making the source/calculations private.
    But yeah, it might be too late.
     
  8. Paradox

    Paradox I am a gigantic asshole who loses people's hard wo

    Messages:
    6,920
    Likes Received:
    146
    Trophy Points:
    0
    Quick delete all evidence!
     
    Neoony likes this.
  9. Mikleo

    Mikleo Member

    Messages:
    151
    Likes Received:
    89
    Trophy Points:
    0
    You're right about players who play in low pop matches having an inflated mmr. Mainly because these games have a lot of new players more than these games not being serious. The average MMR is 1000 right now which is the same as a new player. Naturally the average MMR will inflate because a lot of new players leave the game and ELO is a zero sum game. So in the future 1000 will be quite a bit below average MMR. So playing against nubs in the future will yield quite a bit less benefit than it has now. Currently there is a minimum of 8 players before stats get recorded. I may have a higher level for MMR, possibly 16. But I really don't want to go too high because I don't want to restrict the sample size too much.

    As for making MMR visible, I would only consider it if there was no way to exploit the system. Lets talk about a few ways that the system could be exploited.

    1. Stacking, Even though the teambalance system can restrict the ELO difference to a certain degree, It isn't a standard matchingmaking system that designates teams for you. Some players may decide to join teams together , kindof like prebuilts. Players can have outside knowledge about player lag and players who they think the system has rated inaccurately and they can use this knowledge to try to stack teams. Stacking can artificially inflate your MMR , so if MMR was visible, it would be akin to encouraging it.

    2. Team Leaving/disconnecting. I think maybe I could solve this by using the period between joining and the end of the game to adjust ratings. But that would be less accurate and there are other technical issues involved.

    Ghosting/exploiting would become more common as well.

    And when the system has ways to exploit it, players are not trying to play the game to win they are trying to play the game to preserve their MMR.

    Generally hiding how its calculated would be difficult. Firstly I wouldn't provide a plugin to server owners without the source. It's an issue of trust that they can see what code is being run on their server. Then server admins would still know and could use it to abuse it, or they could easily discuss it.

    The reason why the source is public is that I never intended MMR to be public. I see no way of creating a system that couldn't be exploited without creating a full matchingmaking system similar to Dota etc. This system would also have to place restrictions on spectator and a lot of other game elements to try and prevent the game being exploited. It would never work without these things.

    Effectively you would have to create a "Ladder" server which had proper matchmaking and had a bunch of restrictions.
     
    Last edited: May 27, 2017
    D.D.D. Destroyer and Neoony like this.
  10. Paradox

    Paradox I am a gigantic asshole who loses people's hard wo

    Messages:
    6,920
    Likes Received:
    146
    Trophy Points:
    0
    Then instead of having MMR be public make it so your ranking is public. Like 46/231 etc so theres an incentive. Otherwise you invented a system to balance games that you say yourself are super hard to balance because of the multiplicity of factors involved.

    I think ranking systems and mmr systems are great incentives for people to play " better or more serious".


    also: I dont think it matters as much in a small player base game like empires that theres exploitability, if you see players exploiting you can just mark them as exploiters and everyone will know cause its not like he can go to another server with all other players that dont know him. It's like saying you dont want to go on an adventure cause adventure entails risk. Thats a pessimistic way of viewing it. be optimistic think of all the good things that can come out.
     
    D.D.D. Destroyer likes this.
  11. Mikleo

    Mikleo Member

    Messages:
    151
    Likes Received:
    89
    Trophy Points:
    0
    I'm probably going to change the MMR system so that the ratings are adjusted in different way from the chance to win.

    I totally believe in the chance to win being about percentage of time each players has been on each team calculating the average team MMR.

    But I think adjusting ratings should have the different properties:

    Ratings not adjusted for players who join a certain amount after game start. At the moment players who join a team who are winning get inflated ratings and players who join teams who are losing get deflated ratings. I think ratings shouldn't be adjusted at all for players joining later in the game(like 10% of game time) to prevent these problems.

    Secondly I think ratings should have 100% adjustment for playing over 50% of the game time. So it would go like this.
    <20%: 0% adjustment
    20%-50%: 0-100% adjustment.
    >50% 100% adjustment.

    This means players leaving games early won't have reduced adjustment and everyone on the team will have the same adjustment whether they leave before the end or not. So you can leave early, maybe you got a bad commander or something, that's fine, but once you've played in the game a significant period, your committed, leaving the game early wont save your MMR.

    So you might be thinking this fixes most of the problems associated with exploiting the system. So its OK to make MMR public? Not really, pregame stacking, prebuilts, abusing the systems lack of knowledge on certain players etc still exist.

    However with the autodraft system based on MMR about to be released. It may be possible in the future to have a separate ranked MMR which is visible and is based on ranked(autodraft) games.
     
  12. Sgt.Security

    Sgt.Security Member

    Messages:
    3,083
    Likes Received:
    115
    Trophy Points:
    0
    I am suggesting two things.

    1) Mid-game quitters should gain slightly less points/lose slightly more points when his team wins/loses.

    This is NOT a punishment. The people who quits on a regular basis should have a lower winrate.

    Sometimes, a team is losing but they can still make a comeback, ragequitters can actually kill this last hope.
    This will give ragequitter "a lighter weight".

    There's a problem with this but you are already awared of it: If people quit to spec when the round is basically over, they would lose more points but they shouldn't.

    2) The factors for LIVE winrate prediction.

    They are fully experience based, they are the factors I personally use to predict the result of the game.
    I would use 5 factors, you can give each of the factors different weight, I am giving you the weight that I'd use.

    1: MMR. 25%
    You are already using this so no explanation needed.

    2: Number of "objects" on minimap. 25%
    Map control.
    If we look into this, it's actually "the number of tanks/infantry(players) + the number of buildings"

    If one team has 50 objects and the other team has 60 objects, you probably can't extract much out of this.
    But if one team has 50 objects and the other team has like 100 objects, it's pretty safe to say the team with 100 objects has more map control, and thus a higher winrate.

    3: Resources gain. 15%
    Map control as well, this one should be straightforward.

    But resources gain is much less solid than the number of objects on minimap, so if this factor is given with a heavier weight, you might see winrate fluctuation.

    4: Both team's average score. 20%.
    In most of the matches, the team with higher average score is usually the winning team.

    My scoreboard screenshots support this statement, it's pretty rare that the victorious team has lower average score.

    A lot of actions bring you points, it's a pretty good indication of one team's efficiency.

    5: Both team's current ticket 15%
    I am not looking at kills because kills can be revived.
    But used ticket is a "definite result" and can't be recovered.

    We've had a lot of comebacks because the winning team is out of ticket.
     
    Last edited: Jun 4, 2017
  13. Mikleo

    Mikleo Member

    Messages:
    151
    Likes Received:
    89
    Trophy Points:
    0
    Basing the prediction on anything other than ELO is against the entire point of an ELO based system. Well I could use a million different factors to predict who is going to win and it could be accurate 100% of the time, but I dont give a flying fuck about that. The whole point is that the system should be wrong some of the time so that the ELO's can be adjusted and learn. When the predict function predicts something its not telling you who it thinks will win based on all the information it has its hands, but on the ELO of the players that have been on each team so far.This way the system can learn and adjust the ELO of players to try to match the results of the games. The predict function is just a utility to help me test and to judge how you are doing against how well the system expects your teams ELO to perform. It is a chance based system so that if the system says BE had a 60% chance to win we would expect that 40% of the time NF would be winning when it made that prediction.

    Your system wouldent be able to teach the system anything. Maybe it would teach the system how often a team would win if they had the most buildings or how often a team would win if the players had the most score or the most tickets. But what an ELO system tells you is how often your team is expected to win if your team has certain players vs certain other players.
     
    Last edited: Jun 4, 2017
  14. Sgt.Security

    Sgt.Security Member

    Messages:
    3,083
    Likes Received:
    115
    Trophy Points:
    0
    Then we are talking about two different things, really.

    You are talking about a winrate prediction for your ELO system to correct itself.

    I am talking about a winrate prediction that's just....winrate prediction, it displays the current situation as a number.
     
  15. Mikleo

    Mikleo Member

    Messages:
    151
    Likes Received:
    89
    Trophy Points:
    0
    Generally that is not something I'm interested in developing, as you said yourself,experienced players can see the way the game will go better than any system will ever do. So there is really no point in such a system. The predict function isn't a main function, just a small utility. It supposed to tell you something about the players in each team, the state of the map/tickets is a fairly obvious thing that everyone can see, where as the team balance is a more difficult thing for a player to grasp. At the start I considered making it available just for admins and it might go that way in the future. . If you or anyone wants to create a predict function which factors in stuff that's fine by me, but its not something that gets my juices flowing.

    The next version is introducing optional natives that allow other plugins to get the stats from empstats such as MMR. This will be used in the draft plugin to autodraft players or autodraft squads.
     
    Last edited: Jun 4, 2017
  16. Sgt.Security

    Sgt.Security Member

    Messages:
    3,083
    Likes Received:
    115
    Trophy Points:
    0
    That's not what I said.
    Hell, I'd say if you managed to implement actual winrate prediction, it WILL predict the winrate better than I am.
    Knowing how to play Go doesn't mean you can defeat AlphaGo.
    You really overestimated players's abilities at judging the current situation.

    But yeah, it's a little misunderstanding, your predict function does look like something that's supposed to predict the actual winrate, good to know that it isn't.
    Anyway, suggestion posted, it's there if you find it useful in future.
     
    Last edited: Jun 4, 2017
  17. Mikleo

    Mikleo Member

    Messages:
    151
    Likes Received:
    89
    Trophy Points:
    0
    0.3
    Fixed a MAJOR bug in 0.27 which meant that players on NF didn't have their MMRs adjusted at all.
    Changed the way the predict function works. See OP for details.
    Added higher limits for updating ratings >16 players.
    Other plugins can now use empstats as an optional dependency and get stats on players. look at the include file for details.
     
    Neoony likes this.
  18. Mikleo

    Mikleo Member

    Messages:
    151
    Likes Received:
    89
    Trophy Points:
    0
    0.31
    Fixed a bug where time played wasn't getting reinitialised. I think at the moment players time stats are around 4x what they should be, I will divide the stats when both servers have updated.

    A big change in how data is reported. All stats are now reported at the end of the game instead of when each client disconnects.

    Many changes to how MMR is calculated.

    1. Players who join after the game starts will not have their MMR adjusted.

    2. On the losing team if players have played over 50% of the game they will get a 100% adjustment. They will also receive a -10% adjustment no matter what.

    3 On the winning team players need to have played for at least 50% of the game to gain any adjustment.

    So this means if you leave the game at exactly 50% of the way through you will not gain anything if your team wins and will lose everything if your team loses.

    The way your teams expected chance to win is calculated is still exactly the same as previously. These changes only effect the adjustments.

    This kinda breaks down the zero sum nature of the system. Which I found out was being very unfair to some players in the current version anyway so i decided to scrap. This means I have to watch out for deflation in the system as it is slightly more likely for the average player to lose MMR under this system than gain it. New players joining and a slight inflation adjustment should compensate for this.
     
    Last edited: Jun 10, 2017
  19. Xyaminou

    Xyaminou Member

    Messages:
    1,286
    Likes Received:
    133
    Trophy Points:
    0
    Might need more than that, the other day I got +6 hours in 10 or 15 minutes.
     
  20. Mikleo

    Mikleo Member

    Messages:
    151
    Likes Received:
    89
    Trophy Points:
    0
    Yea it is pretty bugged. I might need to reset it or load just the time stats from a backup.
     

Share This Page