Tell us what’s happening:
I will be glad if I missed something but I believe the following to be correct.
In the test_module.UnitTests.test_hat_draw() method are two tests. The first one asks for an exact response when the results are random. The code below is the actual test method.
The test code from the forkable project
class UnitTests(unittest.TestCase): # ... def test_hat_draw(self): hat = prob_calculator.Hat(red=5, blue=2) actual = hat.draw(2) expected = ['blue', 'red'] self.assertEqual(actual, expected, 'Expected hat draw to return two random items from hat contents.') actual = len(hat.contents) expected = 5 self.assertEqual(actual, expected, 'Expected hat draw to reduce number of items in contents.') # ...
Because lists are ordered it requires exactly that the first random ball drawn is blue and the second random ball is red. In a correctly formed program the first value named 'expected’above will only occasionally be [‘blue’, ‘red’] (2/7 * 5/6) or around 24% of the time. That means (ignoring the ‘contributor’ to the problem described below) a correctly formed program will fail around 76% of the time.
Possible contributor to the problem
The problem is made worse (guaranteeing repeated results in a given system) by the line at the top of the file
prob_calculator.random.seed(95). A hard coded seed will generate the same sequence of random numbers (from for example
random.randint using the same arguments repeatedly)in a given system repeatedly. That means if tests are run in the same sequence (for example a single unit test repetaedly) and it passes on a system (as determined by the repeated ‘random’ number sequence), it will continue to pass 100% of the time and the same if it fails. The default seed is generated from the current system time. That problem is removed by removing that line of code allowing random to use the default seed.
All of the above was theory. My actual code seems to confirm.
When running the tests from main and using the uncommented line
prob_calculator.random.seed(95) in test_modules, my actual results in the tests
UnitTests.test_prob_experiment() never vary and the tests both pass every time, but they shouldn’t.
If I comment it out and allow the default random seed, the actual results vary in both methods (as they should) and sometimes they succeed sometimes fail. The test_hat_draw should pass about 25% of the time. The test_prob_experiment seems to pass about 1/2 the time or more. So the full run from main should succeed about 13% of the time. I got actual 7 out of 50.
After all that, I am waiting for someone to point out the elephant in the room that I misunderstood. If you can confirm or disagree, or made similar observations please post.
Your browser information:
User Agent is:
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36.
Challenge: Probability Calculator
Link to the challenge: