Contact puzzlemix
Play free puzzles         Create free account         Instructions         Discussion forum         Log in    

Show only:

Click on column headings to sort.

Get
Brained Up, daily online brain training site
BrainedUp.com
daily online brain training
Cutting-edge brain training created by Dr Gareth Moore
Think faster, better,
and improve your mental capabilities

Sudoku Xtra, written by Dr Gareth Moore
Sudoku Xtra 24
logic puzzle compendium
Printed puzzles from Dr Gareth Moore

   

Back to the forum

Rating the puzzles

12 comments (Add new comment)
Posted 15th Aug 2012 at 23:01
Elisabeth
Following the points made under Odd Pair 53, I was interested that you have recently introduced a column in the rankings giving our rating of each puzzle, Gareth. A good idea.
Rating is very subjective, but my own assessment, which is only for numerical puzzles, is based on whether I could solve without any aids and in what time. Easy, therefore, for no aids and completed in fairly short time, roughly under 10-15 mins.and, yes, a narrow band. Then it's more tricky. I use moderate if I solve without aids but it takes more thought and time to complete e.g. killer jigsaws, whereas if I have to scrap the puzzle several times or start guessing and using major aids then it becomes hard.
Ratings when major aids are used are awkward because if used to best advantage every puzzle can become easy!! On the other hand I am only using them because puzzle was too difficult without.
I am surprised that numbers 1-10 were not successful, since we ought to be reasonably comfortable with them! Have you ever tried using just 1-5?
Posted 19th Aug 2012 at 17:22
Elisabeth
No-one seems interested in this topic and I am still not sure on what grounds people rate puzzles .It is much too subjective to quantify. e.g. I personally wouldn't rate a puzzle as easy if I had spent over an hour on it! Or perhaps they stopped to have a meal and didn't switch off?? Obviously best to leave well alone since there is no response.
Posted 21st Aug 2012 at 00:29
gareth Administrator Daily subscriber
There was a long discussion a year or so ago about this, before I changed to the new system. The old 1-10 definitely didn't work (based on information gathered over 100,000s of ratings), and indeed it was clear that 1-5 wouldn't work either.

So let's just repeat what was posted under Odd Pair 53 in the Sets section:

The currently displayed ratings are: Easy < Mild < Moderate < Tricky < Hard < Fiendish

Players can rate puzzles easy, moderate or hard and the others above are based on equal division of the ratings count into sixths once it is averaged. In fact I think it's clear I should bias this to make the easy band narrower and the hard band wider.

Until several months ago puzzles were rated on a scale of 1 to 10, but 99% of users never selected anything other than 1, 2 or 3 when rating a puzzle - or occasionally 10. The 4, 5, 6, 7, 8, 9 were not used, essentially. So I simplified it to easy, moderate, hard.

Other sites such as Youtube have switched to an even simpler "good" or "bad" (thumbs up/thumbs down, for example) - it turns out the only way to get useful ratings is to make the choice very simple and then use the maths of it to work out what it really means on the basis of many people choosing.

There is definitely a big psychological part in the easy/moderate/hard selection that varies depending on the player, before you even consider varying experience and expectations. (And of course relative difficulties depend on what you're comparing against too).

One thing would be to try pictures instead of words, e.g. stars or happy/unhappy faces - but it isn't obvious what the best choice would be. Stars really need a caption, so if you label 1 star easy and 3 stars hard that hasn't helped; and faces may not be so clear in meaning.

Any ideas, anyone?

Posted 22nd Aug 2012 at 16:55
Elisabeth
Thanks for the above, Gareth. A further point is that I notice that the ratings for those playing without any aids is often 'higher' than those using them, which shows up in your Maths when giving the ratings in the separate categories. This to me is the crux of the matter as those in the first ranking column are aiming for something different and, as someone commented elsewhere, they persevered as they did not want to be relegated to rank 2. Give up and use the aids and the puzzle becomes easier.
I often would like something between Easy and Moderate (like Mild) but I can see you want to keep it simple.
Posted 23rd Aug 2012 at 10:03
Last edited by kocos 23rd Aug 2012 at 10:06
kocos Daily subscriber
I find recent puzzles easier than they were one year ago or two.

With stars I would feel that I rate the puzzle based on how much I liked it, instead of how hard it was. The named rating helps everyone, to see what is the rating about.

Posted 23rd Aug 2012 at 10:47
gareth Administrator Daily subscriber
Yes, that's my feeling about stars as well. As you say, the meaning would be confused.

In terms of difficulty, the average sudoku puzzle has actually got harder over the past year after I allowed them to include slightly tougher logic - perhaps you have got better at solving them too? :) On the other hand there are more smaller puzzles.

If there were three ratings, not on a drop-down menu, all in a row: Easy - Medium - Hard ...and you had to click on one to highlight it, I wonder if that would change things. It's possible some people just select the first option since it's the quickest way they know to get the 'Close window' link to appear.

Posted 25th Aug 2012 at 21:00
MathTeacherGuy Daily subscriber
If I require aids to solve a puzzle, then (to my reasoning) it is not easy.

If I require aids and multiple do-overs then it is hard. Rating a puzzle hard is painful for anyone who is very competitive.

Sometimes the type of puzzle influences my rating. Kakuro is generally very simple for me, so if a puzzle proves challenging I will rate it as hard. Slitherlink, for reasons I still don't get, usually confounds me and I nearly always use aids. I won't rate a Slitherlink as hard because I figure it is "just me." If I do one without aids then it is easy.

I take time to solve into consideration mainly if I worked on the puzzle continuously. For most puzzles where I see average times greater than ten minutes, I work on the puzzle a bit at a time, put it in the background, and work on other tasks until I have another few minutes. A puzzle with over an hour of actual work time is hard, but some puzzles that take 3 hours are moderate because they have only been worked on for ten or fifteen minutes. Some other sites give the ability to minimize or hide the puzzle while turning off the timer, which takes more time on PuzzleMix, where I have to save, close, and re-open later.

Sometimes I would prefer to have Easy, Medium, and Hard with the option of Medium-Easy or Medium-Hard.

Now if I could just be consistent about my own guidelines, I could offer advice to others ... but sadly it is not so.

Posted 27th Aug 2012 at 09:22
kocos Daily subscriber
Actually I try to use Gareth's guideline at rating. That was:
Easy: solution without had to stop thinking.
Moderate: had to stop thinking at least once.
Hard: Felt stuck at a certain moment.

(I use solution class modifying aids only if already screwed up.)

Posted 7th Sep 2012 at 10:13
Elisabeth
I am still confused by this. If I were to go by the guidelines as quoted by kocos then many more would be at least moderate if not hard. I agree with MathTeacherGuy that if I have to use any aids then it is not, by definition, easy. Also I, too, sometimes want a 4th option so that Moderate can be either nearer to easy or to hard. It would be interesting to hear how those using aids decide on their rating. I hope we don't just pick the first option just to close down quickly! We don't need to make a choice, but how about putting hard first and see what happens!:)
Posted 7th Sep 2012 at 13:33
Ritulia
Elisabeth, I'm one of those using aids regularly. :) But I consider them minor - fill-in-valid, auto-remove, remove-now... They don't affect my efforts, just take away the tedious-boring tasks. And since I don't care much about the time but rather the actual solving, I have no qualms about rating many such puzzles as Easy. :)
Posted 7th Sep 2012 at 21:29
Elisabeth
Glad to hear from you Ritulia.The aids you quote are as you say only those which take away the tedius bits, I am wondering more about those who use the major aids. I agree, speed is not everything, although I like to have a reasonably respectable time! It is virtuously impossible to devise a system that suits everybody as we are judging it from our own viewpoint, but it makes for an interesting discussion!
Posted 10th Sep 2012 at 13:31
kocos Daily subscriber
I guess the most important part of rating is the consitancy. If everyone would rate puzzles in a way, that harder puzzles do never get easier rating, then the rating will tell you something by definition.
I guess the second most important part would be the spread. To use each rating level with almost the same frequency. I doubt anyone fulfills this requirement. Neither do I.
Add new comment
Add a comment
Your comment:
Post comment

Back to the forum

©Brained Up Ltd/Gareth Moore 2005-2018 - email gareth@puzzlemix.com - publishers please visit Any Puzzle Media - our privacy policy - registered in England & Wales no. 8642393