Changes to Map Testing 2

More serious discussions, information about official DDNet servers, important announcements.
jao
Testing Team: Team Leader
Posts: 1206
Joined: Wed Jul 06, 2016 1:18 pm

Changes to Map Testing 2

Post by jao » Wed Jun 20, 2018 8:52 pm

Changes to Map Testing 2


As a follow up to the latest changes to map testing, I'm now announcing further changes:
We decided to move the whole testing process over to Discord!

Why?
While forum activity has steadily declined, interaction on Discord has always gone up (recently hitting 1000 members). After reviewing both platforms, we've made the following conclusions:
  • Discord allows discussions to take place in more dynamic fashion. Instead of writing long posts, which may be perceived as burdensome, mappers and testers are able to have a conversation in a more natural way.
    I've noticed that a lot players were disinclined to write a forum post after testing a map; as opposed to giving short feedback on Discord and then continuing to discuss it afterwards if anything was unclear seemed like no problem.
    Also, internal disucussions on Discord made things move forward way faster than on the forum (regarding to tester discussions).
  • Users can be notified, enabling faster responses. If testers wanted to test with a mapper, they would have to look out for them in-game, often without success. Now this can be handled over Discord. Whether it's giving each other a ping, or arranging it beforehand. Also, users have direct access to staff, allowing them to stay informed about their map's current state.
  • Since internal testing related discussions were previously on Discord already, it makes sense to merge them into the normal testing process now. This gives mappers more insight and takes some work away from me (I don't have to act as the staff's voice).
  • There are some other minor advantages, such as uploading maps to test servers direclty via Discord. Makes life for testers a little easier, but also makes it visiable to mappers and users whether the map on the servers is up-to-date or not.
With the move to Discord, we also want to take the oppertunity to introduce a rating system for maps. It's true that most Map Release Squad members are by far no experts, hence we will move away from making decisions based on a (more or less) simple "yes" or "no". Instead maps are rated on a scale from 0-49 within the scope of criteria. This aims to yield more fair and comprehensible 7 decisions.

Users may assign a score of 0-10 to each of the 4 main critera based on how well the map meets them. Points are: 0-3: Poor, 4-5: Fair, 6-7: Good, 8-10: Excellent

  • Balance: How well are parts balanced in terms of difficulty and playtime throughout the map?
  • Flow: How do individual parts and the map as a whole flow? Are parts well connected? Is it well speedrunnable?
  • Creativity: Are parts unique or the approach original? Is there a variety of different parts?
  • Playability: How well is the map playable? Are parts at a reasonable failiness? Is there enough space? Do parts work properly, are they implemented in a good way and well thought-out?

In addition to the main criteria, users may also give points to each of the following criteria based on their subjective perception. They may be scored from 0-3 points, which are: 0: Poor, 1: Fair, 2: Good, 3: Excellent

  • Fun: Is it exciting to play the map? Would it be played not only to gain points?
  • Design: How visually appealing is it? How innovative and creative is it? Are gametile markings clearly distinguishable?
  • Bonus: Is there anything exceptional to the map that doesn't fit to the other criteria?

The determining scores are calculated like this: score ≈ total score / number of votes. A map needs an overall score of 20 points and fairly well results in each main criteria (4 points) to be released. Only ratings by Map Release Squad members count towards it, as otherwise people with conflicts of interest or bias could influence the decision. However, we encourage everyone to take part in votes. It helps us get an idea about how the community thinks; and if you prove to reward reasonable points and show that you have a clear understanding of the criteria, you will be added to the team. (Discord exlcusive)

There will still be a certain amount of maps per week choosen to be evaluate (based on their waiting time), but with the rating system also other maps can be evaluated alongside them. Requirements are that the map is in testing for more than a week and that at least 5 votes were submited. (Discord exlcusive)

Note: Required scores might be pushed up later.

Overall these changes are mostly about making testing more efficient. Improved communication and easy-to-use concepts should result in a smoother experience for everyone involved.
However, this won't happen over night. We'll still have to handle all of the maps waiting in forum currently. If you have a map in testing and want to help making the transition faster, feel free to submit your map on Discord (it won't necessarily be prefered over other maps tho, but it will land there sooner or later anyways).
New maps can only be submitted on Discord as of now.

More information about how the testing on Discord functions and how you can submit maps and ratings can be found on Discord (Map Testing -> #info-testing). Everything was tested, but it's still possible that there are flaws here and there. If you encounter any bugs or have suggestions, please report them.
If you aren't on DDNet's Discord server yet, feel free to join. Don't hesitate, Discord is lightweight, has a good mobile app and can be used in the browser.

We plan to push further improvements in the future (e.g. automated queue for waiting maps, more private testing, merger between testers and map release squad). If you have any questions or feedback, feel free to share it here.

User avatar
lola
User
Posts: 354
Joined: Thu Jan 04, 2018 10:23 pm
Location: France

Re: Changes to Map Testing 2

Post by lola » Wed Jun 20, 2018 9:50 pm

jao discord is not lightweight it slows down my laptop! it's much heavy than ddnet.

User avatar
Fňokurka oo7
User
Posts: 1194
Joined: Thu Feb 19, 2015 1:33 am
Clan: ( . Y . )

Re: Changes to Map Testing 2

Post by Fňokurka oo7 » Wed Jun 20, 2018 10:03 pm

So I have to use Discord in order to get my map released?
------------------------------------------------------------
Some people just need a high-five.
In the face.
With a chair.
Made of steel.
Multiple times.

User avatar
Ravie
Testing Team: Tester
Posts: 368
Joined: Sat Feb 04, 2017 5:04 am
Player profile: Ravie
Mapper profile: Ravie
Clan: TOP

Re: Changes to Map Testing 2

Post by Ravie » Wed Jun 20, 2018 10:04 pm

Fňokurka oo7 wrote:
Wed Jun 20, 2018 10:03 pm
So I have to use Discord in order to get my map released?
Yep.
Image

User avatar
Fňokurka oo7
User
Posts: 1194
Joined: Thu Feb 19, 2015 1:33 am
Clan: ( . Y . )

Re: Changes to Map Testing 2

Post by Fňokurka oo7 » Wed Jun 20, 2018 10:06 pm

Ravie wrote:
Wed Jun 20, 2018 10:04 pm
Fňokurka oo7 wrote:
Wed Jun 20, 2018 10:03 pm
So I have to use Discord in order to get my map released?
Yep.
Doesn't sound good.
------------------------------------------------------------
Some people just need a high-five.
In the face.
With a chair.
Made of steel.
Multiple times.

User avatar
Ravie
Testing Team: Tester
Posts: 368
Joined: Sat Feb 04, 2017 5:04 am
Player profile: Ravie
Mapper profile: Ravie
Clan: TOP

Re: Changes to Map Testing 2

Post by Ravie » Wed Jun 20, 2018 10:10 pm

Fňokurka oo7 wrote:
Wed Jun 20, 2018 10:06 pm
Ravie wrote:
Wed Jun 20, 2018 10:04 pm
Fňokurka oo7 wrote:
Wed Jun 20, 2018 10:03 pm
So I have to use Discord in order to get my map released?
Yep.
Doesn't sound good.
Nope.
Image

User avatar
meew'
User
Posts: 513
Joined: Mon Jul 20, 2015 7:17 pm
Player profile: meew'
Clan: Möhre

Re: Changes to Map Testing 2

Post by meew' » Wed Jun 20, 2018 10:26 pm

dont forget to tell them they instantly get removed from your retard squad as soon as they dont agree with your opinion
better give map testing back to rayb so ddnet might stay alive due to proper map releases instead of mappers sucking jaos unqualified dick to get their trash released
911 mapping police

aaa
Community Team: Team Leader
Posts: 368
Joined: Mon May 05, 2014 7:44 pm

Re: Changes to Map Testing 2

Post by aaa » Wed Jun 20, 2018 10:46 pm

the problem is not that you disagreed but rather your lack of basic social skills as just shown by your post, you can't communicate without being rude/insulting..
that obviously makes you completely unfit for any actual group discussion

User avatar
hi_leute_gll
User
Posts: 2525
Joined: Mon Jul 21, 2014 1:15 am
Player profile: hi_leute_gll
Mapper profile: hi_leute_gll
Clan: iMTG

Re: Changes to Map Testing 2

Post by hi_leute_gll » Thu Jun 21, 2018 12:24 am

jao wrote:
Wed Jun 20, 2018 8:52 pm
While forum activity has steadily declined, interaction on Discord has always gone up (recently hitting 1000 members).
The Discord community isn't growing. 90% of these 1000 members are dead accounts, which are just on the server, because they some when clicked on it. They don't participate at all. The active core there isn't that big, there are always the same few people posting.
jao wrote:
Wed Jun 20, 2018 8:52 pm
Discord allows discussions to take place in more dynamic fashion. Instead of writing long posts, which may be perceived as burdensome, mappers and testers are able to have a conversation in a more natural way.
Discord also allows you to spam discussions away. Everything you write there is lost the day later, while in a forum you can easily search for older topics and continue discussions a few days later if something on the topic changed. In Discord the discussion is lost for ever and you can hardly revive it. Just compare this snippet to the rest of the thread. You will see a massive difference in the quality of discussion, but the same amount of people participating.
jao wrote:
Wed Jun 20, 2018 8:52 pm
I've noticed that a lot players were disinclined to write a forum post after testing a map; as opposed to giving short feedback on Discord and then continuing to discuss it afterwards if anything was unclear seemed like no problem.
If they aren't in the mood to write proper testing-reports they probably won't add much to the testing process anyway. We already see this in the forum when people write one liner without any content. It is rather the opposite: Discord will discourage people to write good testing-reports due to a lack of utility for such kind of postings.
jao wrote:
Wed Jun 20, 2018 8:52 pm
Also, internal disucussions on Discord made things move forward way faster than on the forum (regarding to tester discussions).
DDNet's communication was always split into forum and chat-tool (Skype earlier, now Discord). Both platforms are used for what the single platform is best at. So if you do internal discussions in your testing group/release squad, it is totally fine if you choose Discord as the best tool, but that doesn't mean that the conclusion of such discussions and testing-reports need to be posted in Discord as well.

The same applies for user notifications and such.
jao wrote:
Wed Jun 20, 2018 8:52 pm
Since internal testing related discussions were previously on Discord already, it makes sense to merge them into the normal testing process now.
If you make private discussions public then testers can't speak freely in these discussions anymore, since in a public chat they need to choose polite terms to not make a mapper mad. This will just make the discussions less productive, or testers will open private chats their self.
jao wrote:
Wed Jun 20, 2018 8:52 pm
There are some other minor advantages, such as uploading maps to test servers direclty via Discord. Makes life for testers a little easier, but also makes it visiable to mappers and users whether the map on the servers is up-to-date or not.
I have to admit, that's the only point that is an argument.


About your rating-vote-system:
This can be done in the forum, too. No argument for Discord.
If people can vote public, but only the votes of the testing group do count, you will be quickly in a situation where you e.g. have to explain a score of 4.5 by your testing group and a score of 7.9 by mappers who asked all their friends to vote. Except I don't get what's the difference between people voting a 4 and saying "it's good enough for release". Also tester already say which parts of a map they like and which not, so the votes don't add any new information. They are also by far not representative and will be mostly unqualified, in opposite to a report by an experienced tester, especially since in the release squad, as you said your self, most "members are by far no experts". (With that you even argue that the release squad should be removed do to missing qualifications.) The process of voting via complicated commands is hilarious.


Conclusion:
You brought in a single small argument (map-uploads) why the testing should be moved to Discord. To achieve that you make things more complicated (e.g., threads will be archived via files -> bad accessibility) and remove the advantages of a forum (filing, post history, posts stay up longer, ...). So all in all you make the testing worse, more complicated and reduce the quality even more.

EDIT:
Just saw the info-post on the Discord-channel:
"If you want to share a lot images to share bugs or suggestions, upload them on an external website of your choice (e.g. https://imgur.com/)."
So you have to use a 3rd-Party website to be able to report bugs. Is this the way the testing will be more efficient?
.hi._.leute._.gll. | iMTGmember

User avatar
Prankster
User
Posts: 75
Joined: Wed May 16, 2018 9:04 pm
Location: Europe
Player profile: Prankster
Mapper profile: Prankster
Clan: BlockMark

Re: Changes to Map Testing 2

Post by Prankster » Thu Jun 21, 2018 1:32 am

I like hi_leute_gll, nice to see someone using his brain and taking time to take care of DDNet. You gave some pretty good arguments. I think the forum shouldn't be dismissed, even if you move the testing system to discord, I think a thread should still be made on the forum. The thread would be good to track the current state of the map and post the final verdict there. I think the conversation is cleaner if it is held at the forum, also it's easier to track the votes imo. I don't think discord would help to fix the map faster, testers still need to collect the bugs and send it, the difference is that they can send it immediately during testing. But it doesn't always mean the mapper receives it faster and even if he does, he might not be able to fix it immediately. And then in the end it is just as fast as the forum one, or even slower since it isn't as clean and easy to follow as a forum criticism. I like the forum-system. However, tbh when I first read this post I also thought that this is a good idea, especially because it is more dynamic, but after reading hi_leute_gll's post, I had to realize that it isn't as good as I thought and he is pretty much right. I don't really want to add more things, since he broke it down pretty well.
Imo you could try to run the two system simultaneously and see which one works better/which one you like more. Also, I find it less likely to get banned on forum, so it's a +1 reason for doing it here, my discord state is much more unstable.
It's just a prank bro

Post Reply

Who is online

Users browsing this forum: Q í and 4 guests