Changes to Map Testing 2
Posted: Wed Jun 20, 2018 8:52 pm
Changes to Map Testing 2
As a follow up to the latest changes to map testing, I'm now announcing further changes:
We decided to move the whole testing process over to Discord!
Why?
While forum activity has steadily declined, interaction on Discord has always gone up (recently hitting 1000 members). After reviewing both platforms, we've made the following conclusions:
- Discord allows discussions to take place in more dynamic fashion. Instead of writing long posts, which may be perceived as burdensome, mappers and testers are able to have a conversation in a more natural way.
I've noticed that a lot players were disinclined to write a forum post after testing a map; as opposed to giving short feedback on Discord and then continuing to discuss it afterwards if anything was unclear seemed like no problem.
Also, internal disucussions on Discord made things move forward way faster than on the forum (regarding to tester discussions). - Users can be notified, enabling faster responses. If testers wanted to test with a mapper, they would have to look out for them in-game, often without success. Now this can be handled over Discord. Whether it's giving each other a ping, or arranging it beforehand. Also, users have direct access to staff, allowing them to stay informed about their map's current state.
- Since internal testing related discussions were previously on Discord already, it makes sense to merge them into the normal testing process now. This gives mappers more insight and takes some work away from me (I don't have to act as the staff's voice).
- There are some other minor advantages, such as uploading maps to test servers direclty via Discord. Makes life for testers a little easier, but also makes it visiable to mappers and users whether the map on the servers is up-to-date or not.
With the move to Discord, we also want to take the oppertunity to introduce a rating system for maps. It's true that most Map Release Squad members are by far no experts, hence we will move away from making decisions based on a (more or less) simple "yes" or "no". Instead maps are rated on a scale from 0-49 within the scope of criteria. This aims to yield more fair and comprehensible 7 decisions.
Users may assign a score of 0-10 to each of the 4 main critera based on how well the map meets them. Points are:
In addition to the main criteria, users may also give points to each of the following criteria based on their subjective perception. They may be scored from 0-3 points, which are:
The determining scores are calculated like this:
There will still be a certain amount of maps per week choosen to be evaluate (based on their waiting time), but with the rating system also other maps can be evaluated alongside them. Requirements are that the map is in testing for more than a week and that at least 5 votes were submited. (Discord exlcusive)
Note: Required scores might be pushed up later.
Users may assign a score of 0-10 to each of the 4 main critera based on how well the map meets them. Points are:
0-3: Poor, 4-5: Fair, 6-7: Good, 8-10: Excellent
- Balance: How well are parts balanced in terms of difficulty and playtime throughout the map?
- Flow: How do individual parts and the map as a whole flow? Are parts well connected? Is it well speedrunnable?
- Creativity: Are parts unique or the approach original? Is there a variety of different parts?
- Playability: How well is the map playable? Are parts at a reasonable failiness? Is there enough space? Do parts work properly, are they implemented in a good way and well thought-out?
In addition to the main criteria, users may also give points to each of the following criteria based on their subjective perception. They may be scored from 0-3 points, which are:
0: Poor, 1: Fair, 2: Good, 3: Excellent
- Fun: Is it exciting to play the map? Would it be played not only to gain points?
- Design: How visually appealing is it? How innovative and creative is it? Are gametile markings clearly distinguishable?
- Bonus: Is there anything exceptional to the map that doesn't fit to the other criteria?
The determining scores are calculated like this:
score ≈ total score / number of votes
. A map needs an overall score of 20 points and fairly well results in each main criteria (4 points) to be released. Only ratings by Map Release Squad members count towards it, as otherwise people with conflicts of interest or bias could influence the decision. However, we encourage everyone to take part in votes. It helps us get an idea about how the community thinks; and if you prove to reward reasonable points and show that you have a clear understanding of the criteria, you will be added to the team. (Discord exlcusive)There will still be a certain amount of maps per week choosen to be evaluate (based on their waiting time), but with the rating system also other maps can be evaluated alongside them. Requirements are that the map is in testing for more than a week and that at least 5 votes were submited. (Discord exlcusive)
Note: Required scores might be pushed up later.
Overall these changes are mostly about making testing more efficient. Improved communication and easy-to-use concepts should result in a smoother experience for everyone involved.
However, this won't happen over night. We'll still have to handle all of the maps waiting in forum currently. If you have a map in testing and want to help making the transition faster, feel free to submit your map on Discord (it won't necessarily be prefered over other maps tho, but it will land there sooner or later anyways).
New maps can only be submitted on Discord as of now.
More information about how the testing on Discord functions and how you can submit maps and ratings can be found on Discord (Map Testing -> #info-testing). Everything was tested, but it's still possible that there are flaws here and there. If you encounter any bugs or have suggestions, please report them.
If you aren't on DDNet's Discord server yet, feel free to join. Don't hesitate, Discord is lightweight, has a good mobile app and can be used in the browser.
If you aren't on DDNet's Discord server yet, feel free to join. Don't hesitate, Discord is lightweight, has a good mobile app and can be used in the browser.
We plan to push further improvements in the future (e.g. automated queue for waiting maps, more private testing, merger between testers and map release squad). If you have any questions or feedback, feel free to share it here.