Help with Pytest results

Hello,

I have recently completed coding for the arithmetic_arranger.py problem and when I run the code using custom inputs, the output looks correct. However, when I run the code along with the test cases, I get a message,

=============== 6 failed, 4 passed in 0.42s ================

I have never used pytest before and I have no idea how do I understand why is my code fails and worse why does it passes the 4 test cases. Please help me in understanding the output from the pytest module.

In case you would like to check my code, here is the replit link.

the test suite shows specifics for each failed test.

This is all the output when you run the replit:

All the output (Click to show/hide)
 python main.py
   32      3801      45      123    
+ 698    -    2    + 43    +  49    
-----    ------    ----    -----    

  32         1      9999      523    
+  8    - 3801    + 9999    -  49    
----    ------    ------    -----    
  40     -3800     19998      474    

====================================== test session starts =======================================
platform linux -- Python 3.8.13, pytest-7.1.1, pluggy-1.0.0 -- /nix/store/06c55y0c5yzx5gx4l6k0pp6071zf1y5i-python3-3.8.13/bin/python
cachedir: .pytest_cache
rootdir: /home/runner/CWihNEf45Ci
collected 10 items                                                                               

test_module.py::test_template[test_two_problems_arrangement1] <- ../boilerplate-arithmetic-formatter/test_module.py FAILED [ 10%]
test_module.py::test_template[test_two_problems_arrangement2] <- ../boilerplate-arithmetic-formatter/test_module.py FAILED [ 20%]
test_module.py::test_template[test_four_problems_arrangement] <- ../boilerplate-arithmetic-formatter/test_module.py FAILED [ 30%]
test_module.py::test_template[test_five_problems_arrangement] <- ../boilerplate-arithmetic-formatter/test_module.py FAILED [ 40%]
test_module.py::test_template[test_too_many_problems] <- ../boilerplate-arithmetic-formatter/test_module.py PASSED [ 50%]
test_module.py::test_template[test_incorrect_operator] <- ../boilerplate-arithmetic-formatter/test_module.py PASSED [ 60%]
test_module.py::test_template[test_too_many_digits] <- ../boilerplate-arithmetic-formatter/test_module.py PASSED [ 70%]
test_module.py::test_template[test_only_digits] <- ../boilerplate-arithmetic-formatter/test_module.py PASSED [ 80%]
test_module.py::test_template[test_two_problems_with_solutions] <- ../boilerplate-arithmetic-formatter/test_module.py FAILED [ 90%]
test_module.py::test_template[test_five_problems_with_solutions] <- ../boilerplate-arithmetic-formatter/test_module.py FAILED [100%]

============================================ FAILURES ============================================
_________________________ test_template[test_two_problems_arrangement1] __________________________

arguments = [['3801 - 2', '123 + 49']]
expected_output = '  3801      123\n-    2    +  49\n------    -----'
fail_message = 'Expected different output when calling "arithmetic_arranger()" with ["3801 - 2", "123 + 49"]'

>   ???
E   AssertionError: Expected different output when calling "arithmetic_arranger()" with ["3801 - 2", "123 + 49"]
E   assert '  3801      123    \n-    2    +  49    \n------    -----    \n' == '  3801      123\n-    2    +  49\n------    -----'
E     -   3801      123
E     +   3801      123    
E     ?                ++++
E     - -    2    +  49
E     + -    2    +  49    
E     ?                ++++
E     - ------    -----
E     + ------    -----    
E     ?                +++++

/home/runner/boilerplate-arithmetic-formatter/test_module.py:77: AssertionError
_________________________ test_template[test_two_problems_arrangement2] __________________________

arguments = [['1 + 2', '1 - 9380']]
expected_output = '  1         1\n+ 2    - 9380\n---    ------'
fail_message = 'Expected different output when calling "arithmetic_arranger()" with ["1 + 2", "1 - 9380"]'

>   ???
E   AssertionError: Expected different output when calling "arithmetic_arranger()" with ["1 + 2", "1 - 9380"]
E   assert '  1         1    \n+ 2    - 9380    \n---    ------    \n' == '  1         1\n+ 2    - 9380\n---    ------'
E     -   1         1
E     +   1         1    
E     ?              ++++
E     - + 2    - 9380
E     + + 2    - 9380    
E     ?              ++++
E     - ---    ------
E     + ---    ------    
E     ?              +++++

/home/runner/boilerplate-arithmetic-formatter/test_module.py:77: AssertionError
_________________________ test_template[test_four_problems_arrangement] __________________________

arguments = [['3 + 855', '3801 - 2', '45 + 43', '123 + 49']]
expected_output = '    3      3801      45      123\n+ 855    -    2    + 43    +  49\n-----    ------    ----    -----'
fail_message = 'Expected different output when calling "arithmetic_arranger()" with ["3 + 855", "3801 - 2", "45 + 43", "123 + 49"]'

>   ???
E   AssertionError: Expected different output when calling "arithmetic_arranger()" with ["3 + 855", "3801 - 2", "45 + 43", "123 + 49"]
E   assert ('    3      3801      45      123    \n'\n '+ 855    -    2    + 43    +  49    \n'\n '-----    ------    ----    -----    \n') == ('    3      3801      45      123\n'\n '+ 855    -    2    + 43    +  49\n'\n '-----    ------    ----    -----')
E     -     3      3801      45      123
E     +     3      3801      45      123    
E     ?                                 ++++
E     - + 855    -    2    + 43    +  49
E     + + 855    -    2    + 43    +  49    
E     ?                                 ++++
E     - -----    ------    ----    -----
E     + -----    ------    ----    -----    
E     ?                                 +++++

/home/runner/boilerplate-arithmetic-formatter/test_module.py:77: AssertionError
_________________________ test_template[test_five_problems_arrangement] __________________________

arguments = [['11 + 4', '3801 - 2999', '1 + 2', '123 + 49', '1 - 9380']]
expected_output = '  11      3801      1      123         1\n+  4    - 2999    + 2    +  49    - 9380\n----    ------    ---    -----    ------'
fail_message = 'Expected different output when calling "arithmetic_arranger()" with ["11 + 4", "3801 - 2999", "1 + 2", "123 + 49", "1 - 9380"]'

>   ???
E   AssertionError: Expected different output when calling "arithmetic_arranger()" with ["11 + 4", "3801 - 2999", "1 + 2", "123 + 49", "1 - 9380"]
E   assert ('  11      3801      1      123         1    \n'\n '+  4    - 2999    + 2    +  49    - 9380    \n'\n '----    ------    ---    -----    ------    \n') == ('  11      3801      1      123         1\n'\n '+  4    - 2999    + 2    +  49    - 9380\n'\n '----    ------    ---    -----    ------')
E     -   11      3801      1      123         1
E     +   11      3801      1      123         1    
E     ?                                         ++++
E     - +  4    - 2999    + 2    +  49    - 9380
E     + +  4    - 2999    + 2    +  49    - 9380    
E     ?                                         ++++
E     - ----    ------    ---    -----    ------
E     + ----    ------    ---    -----    ------    
E     ?                                         +++++

/home/runner/boilerplate-arithmetic-formatter/test_module.py:77: AssertionError
________________________ test_template[test_two_problems_with_solutions] _________________________

arguments = [['3 + 855', '988 + 40'], True]
expected_output = '    3      988\n+ 855    +  40\n-----    -----\n  858     1028'
fail_message = 'Expected solutions to be correctly displayed in output when calling "arithmetic_arranger()" with ["3 + 855", "988 + 40"] and a second argument of `True`.'

>   ???
E   AssertionError: Expected solutions to be correctly displayed in output when calling "arithmetic_arranger()" with ["3 + 855", "988 + 40"] and a second argument of `True`.
E   assert ('    3      988    \n'\n '+ 855    +  40    \n'\n '-----    -----    \n'\n '  858     1028    \n') == '    3      988\n+ 855    +  40\n-----    -----\n  858     1028'
E     -     3      988
E     +     3      988    
E     ?               ++++
E     - + 855    +  40
E     + + 855    +  40    
E     ?               ++++
E     - -----    -----
E     + -----    -----    
E     ?               ++++
E     -   858     1028
E     +   858     1028    
E     ?               +++++

/home/runner/boilerplate-arithmetic-formatter/test_module.py:77: AssertionError
________________________ test_template[test_five_problems_with_solutions] ________________________

arguments = [['32 - 698', '1 - 3801', '45 + 43', '123 + 49', '988 + 40'], True]
expected_output = '   32         1      45      123      988\n- 698    - 3801    + 43    +  49    +  40\n-----    ------    ----    -----    -----\n -666     -3800      88      172     1028'
fail_message = 'Expected solutions to be correctly displayed in output when calling "arithmetic_arranger()" with five arithmetic problems and a second argument of `True`.'

>   ???
E   AssertionError: Expected solutions to be correctly displayed in output when calling "arithmetic_arranger()" with five arithmetic problems and a second argument of `True`.
E   assert ('   32         1      45      123      988    \n'\n '- 698    - 3801    + 43    +  49    +  40    \n'\n '-----    ------    ----    -----    -----    \n'\n ' -666     -3800      88      172     1028    \n') == ('   32         1      45      123      988\n'\n '- 698    - 3801    + 43    +  49    +  40\n'\n '-----    ------    ----    -----    -----\n'\n ' -666     -3800      88      172     1028')
E     -    32         1      45      123      988
E     +    32         1      45      123      988    
E     ?                                          ++++
E     - - 698    - 3801    + 43    +  49    +  40
E     + - 698    - 3801    + 43    +  49    +  40    
E     ?                                          ++++
E     - -----    ------    ----    -----    -----
E     + -----    ------    ----    -----    -----    
E     ?                                          ++++
E     -  -666     -3800      88      172     1028
E     +  -666     -3800      88      172     1028    
E     ?                                          +++++

/home/runner/boilerplate-arithmetic-formatter/test_module.py:77: AssertionError
==================================== short test summary info =====================================
FAILED test_module.py::test_template[test_two_problems_arrangement1] - AssertionError: Expected...
FAILED test_module.py::test_template[test_two_problems_arrangement2] - AssertionError: Expected...
FAILED test_module.py::test_template[test_four_problems_arrangement] - AssertionError: Expected...
FAILED test_module.py::test_template[test_five_problems_arrangement] - AssertionError: Expected...
FAILED test_module.py::test_template[test_two_problems_with_solutions] - AssertionError: Expect...
FAILED test_module.py::test_template[test_five_problems_with_solutions] - AssertionError: Expec...
================================== 6 failed, 4 passed in 0.19s ===================================
 

Here you should be specifically interested in the part that starts with the FAILURES header, in there you can see each failed test under its own header. For example this is the first failed test:

_________________________ test_template[test_two_problems_arrangement1] __________________________

arguments = [['3801 - 2', '123 + 49']]
expected_output = '  3801      123\n-    2    +  49\n------    -----'
fail_message = 'Expected different output when calling "arithmetic_arranger()" with ["3801 - 2", "123 + 49"]'

>   ???
E   AssertionError: Expected different output when calling "arithmetic_arranger()" with ["3801 - 2", "123 + 49"]
E   assert '  3801      123    \n-    2    +  49    \n------    -----    \n' == '  3801      123\n-    2    +  49\n------    -----'
E     -   3801      123
E     +   3801      123    
E     ?                ++++
E     - -    2    +  49
E     + -    2    +  49    
E     ?                ++++
E     - ------    -----
E     + ------    -----    
E     ?                +++++

/home/runner/boilerplate-arithmetic-formatter/test_module.py:77: AssertionError

At the beginning it describes the test:

arguments = [['3801 - 2', '123 + 49']]
expected_output = '  3801      123\n-    2    +  49\n------    -----'
fail_message = 'Expected different output when calling "arithmetic_arranger()" with ["3801 - 2", "123 + 49"]'

It says what it expects, what’s the function being called with, and the fail emssage in plain English.

The line starting with AssertionError repeat the test description.
The line starting with assert put on the same line your output and the expected, first your output then the expected.
And below that there is a diff view that confront the two line by line:
The lines starting with - are the expected, the lines starting with + are your code, and the lines starting with ? point out the difference.
If in the line with ? there is a + it points to a character in your code that is not present in the expected output.
If it seems to not point out to anything, the characters could be spaces or new lines characters.

1 Like

Thank you Ilenia. This explanation helped me fix my code. It passes all test cases now. Thank you very much!!!