Suspected Error in Test Module for Python Probability Calculator

Python
#1

Hi everyone,

I just completed the fifth challenge in the Scientific Computing with Python Certification and I think there is an error in the test_module.py script.
Specifically, the first test in the function test_prob_experiment checks whether the probability computed by the student is almost equal (delta=0.01) to the hard coded value expected=0.272.
The test kept failing for me since my implemenation spits out 0.252 everytime given the seed set in the test_module. Since all other tests passed I did the math on paper for the experiment and the exact probability for the tested outcome is 0.263. I ran my code with 2 million iterations which gave me that exact value as well.

So I see two problems with the test_module. One, the expected value is off and two, a maximal deviation of 0.01 might be too low given the huge amout of possible implenations and the comparitively low number of iterations for the experiment.

Find my code here.

Please let me know if I am mistaken. Apart from that, thanks to everyone who makes this project possible! I’ve recommended it many times and still love it myself.
Cheers

#2

I believe that the test suite is expecting a different method of drawing the balls that, when used with the random seed chosen in the test suite, yields 0.272. This has been a consistent problem with this challenge.

You can flag yet another GitHub Issue against this project.

#3

The difference between your value and the expected is down to your draw() method. You shuffle and pop the array for each draw, which yields 0.252. When I did the project, I used random.choice() and got the expected probability (0.273). I tested the same method in your code and got the expected probability too. If you shuffle just once before the loop, it yields 0.261. Since each selection method uses a different number of random numbers in the same sequence, the selection algorithm will affect the calculated probability,

So, your code is correct, it’s just not in agreement with the test and its delta. It’s clear from these three methods of random choice that the delta is too small for the number of iterations. At 10,000 iterations with the set random seed, I get

./main.py 
# random.choice()
Probability: 0.269
# one shuffle
./main.py 
Probability: 0.2677
# shuffle every draw
./main.py 
Probability: 0.266

so the expected value could be set to 0.263 with the same delta and still work at 10,000 draws.

As far as your math goes, I concur that the actual value is 0.263, unless I miscounted, which is always possible.

1 Like