Assume that when a new bot is released, every battle contains that bot, then the probability of meeting that bot is 1 instead of 1 / (n - 1), which is highly biased.

To fix this, we have two options — mutate our current pairing systems to get unbiased score online, or to reset the entire meleerumble periodically.

Since the score of new bots are unbiased, all we need to do for an unbiased score is to ignore (n - 2) / (n - 1) biased battles randomly when calculating the score of an old bot. However this approach takes much more battles.

A more practical way is is, when bot A is added, for each battle, select another bot B randomly, and run melee battles containing those two bots as usual. A battle containing A, B and other 8 bots should yield 45 pairings, but only those matching (A, *) or (B, *) is taken into account. This produces 17 parings.

This scheme does not affect the pairings of the new bot itself at all, which is already unbiased; And for an old bot, the probability of being chosen as B is 1 / (n - 1), therefore the probability of a battle with A present being taken into account for old bots is 1 / (n - 1), the same as the unbiased one.

Your second approach seems good, it will need patches on the client side so that if a priority pairing needs to happen it only uploads the battles which contain one of the priority bots. This filter will work fine for the 1v1 rumble as well.

This should check participants list more often preventing retired bots from being battled and submitted nullifying smart battles. And on unstable network it won't upload 80000 duplicate battles for a single pairing any more. (See pulsar.PulsarNano 0.2.4 and ad.last.Bottom 1.0)

I suspect we should separate roborumble version and robocode version (since they are weakly dependent) and get roborumble bug fixes quicker than robocode, since roborumble bugs are generally more serious, and roborumble fixes has nothing to do with the rest of robocode users.

Currently fixes in roborumble takes almost a year to be actually deployed.

I'll upgrade all my clients later and upload the same bot of mine with different version number to see the impact on score...

The melee part is already discussed, but why the 1v1 rumble needs a wipe as well? And how long it should take to reload the rumble? Or should we run the current rumble as a backup in parallel until the reload completes? (so that new players can still submit their bots)

The rumble clients load version number dynamically, maybe we should tweak this to append Java version after version number as well to prevent running and uploading with Java 10 incidentally.

And since we have rolling average, the accumulated effect of bad configurations should be fixed automatically (after a lot of battles).

A patch for melee priority battle is harder than I thought, as the data structure (currently, very bad, string) needs a redesign for the extra 1 bit of information. And almost every line of code needs a rewrite as it depends on the implementation details of the (not extensible) data structure.

Done! After some refactoring, the rumble client should upload only pairings containing the prioritized bot, or containing the predetermined random bot when running prioritized battles.

The only flaw is that when the rumble server returns prioritized pairings, etc. A and B, if B is not evenly distributed, then B will get biased battles (meet more A). Is literumble using this feature currently? How not evenly distributed is bot B returned in this case?

Bot B is randomly selected from missing pairs, or if pairings are complete then randomly selected, with weighting biased towards lower number of battle pairs. It should be ok (and regardless, much much better than the current situation)

ScalarN scores APS 0% and survival 0% (it was all 100%/100% before recent 1-2 days) against some bots with APS lower than 50, after 1.9.3.3 is allowed. However I've been testing my bot with 1.9.3.2 and 1.9.3.3 since the first day and nothing strange happens.

After more investigation, I thought that those strange score may come from some one who incidentally clicked roborumble.bat without proper configuration which however I never reproduced.

Using the latest robocode version as accepted one and allowing Anonymous uploads is risky as robocode has a lot of downloads per week and anyone who incidentally clicked roborumble.bat contributes without verifying configuration.

Maybe we should also modify roborumble client to add a field usually called "User Agent" and upload OS/Java Version etc. so that strange scores should be more trackable.

Another option is to disable Anonymous uploads when using the latest version of robocode, but this would only prevent incident roborumble runs and wouldn't verify user agent.

Potentially I could add something on the literumble side, yes. Also, if there are less uploads for melee then it can send back priority battles on every pair, or at least more often. Right now it only sends back priority on a fraction of the uploads.

The codesize utility currently uses BCEL 5.2 which cannot handle Java 6, 7, 8 features properly.
And the roborumble client has a logical bug that treats codesize calc failure as codesize 0 and allowing it to participant in even nano. The correct logic should be treating codesize calc failure as infinity, only allowing it to participant in MegaRumble.

I will definitely have to fix codesize as I have not touched it for many years. :O

Also notice that it will be faster to write to me a mail at fnl (at) users.sourceforge.net in order to have me fix as many issues related to RoboRumble as possible (this has a high priority to me). Just tell me to look on this discussion, if all details are already provided. This way you'll not have to wait for me updating Robocode, Codesize, RoboRumble etc. :-)
And thank you for providing me with patches for RoboRumble. It is a great help to all of us.