Do I need to upgrade codepen to built more project?
No you don’t need to upgrade. You can store unlimited pens as a free account. Unless, if you want to private your projects then you should upgrade.
As mentioned, the free version of codepen allows only one project but, an unlimited number of pens. For this reason freeCodeCamp suggests the use of pens to do the projects for the RWD certification. Note: the examples are in pens, and the test suite is also in a pen you can fork.
Note that FCC and codepen do not use the term project in the same way. FCC’s projects can all be done by creating codepen pens.