How to include _id in JSON response (Exercise Tracker)

DragonOsman · March 13, 2021, 8:52pm

What’s happening:

I want to know two things:

How to include the _id field in the JSON response sent from an Express route. I ask because I can’t modify the _id manually (as it tells me when I try to set it manually).
How to fix this error:

TypeError: Cannot convert object to primitive value
    at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\server.js:18:22
    at Layer.handle [as handle_request] (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\express\lib\router\layer.js:95:5)
    at next (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\express\lib\router\route.js:137:13) 
    at Route.dispatch (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\express\lib\router\route.js:112:3)
    at Layer.handle [as handle_request] (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\express\lib\router\layer.js:95:5)
    at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\express\lib\router\index.js:281:22        
    at Function.process_params (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\express\lib\router\index.js:335:12)
    at next (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\express\lib\router\index.js:275:10) 
    at jsonParser (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\body-parser\lib\types\json.js:101:7)
    at Layer.handle [as handle_request] (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\express\lib\router\layer.js:95:5)

My code so far

const express = require("express");
const app = express();
const cors = require("cors");
require("dotenv").config();
const path = require("path");
const bodyParser = require("body-parser");
const mongoose = require("mongoose");

mongoose.connect(process.env.DB_URI, { useNewUrlParser: true, useUnifiedTopology: true });

const Schema = mongoose.Schema;
const userSchema = new Schema({
  username: { type: String, unique: true } // can't have someone register more than once
});

const User = mongoose.model("user", userSchema);

app.use(cors());
app.use(express.static("public"));
app.get("/", (req, res) => {
  res.sendFile(path.join(`${__dirname}`, "/views/index.html"));
});

app.use(bodyParser.urlencoded({ extended: false }));
app.use(bodyParser.json());

app.post("/api/exercise/new-user", (req, res) => {
  const username = req.body.username;

  User.find({}, (err, users) => {
    if (err) {
      console.log(err);
    }

    const user = new User({
      username: username
    });

    User.findOne({ username: username }, (err, foundUser) => {
      if (err) {
        console.log(err);
      }

      if (!foundUser) {
        user.save((err, user) => {
          if (err) {
            console.log(err);
          }
          console.log(`user ${user.username} saved to database!`);
        });
      }
    });

    res.json({
      username: username,
      _id
    });
  });
});

const listener = app.listen(process.env.PORT || 3000, () => {
  console.log("Your app is listening on port " + listener.address().port);
});

Any help is appreciated. Thanks in advance.

Note: I’m trying to complete the second user story:

You can POST to /api/exercise/new-user with form data username to create a new user. The returned response will be an object with username and _id properties.

My browser information:

User Agent is: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.72 Safari/537.36 Edg/89.0.774.45.

Challenge: Exercise Tracker

Link to the challenge:

phptuts · March 13, 2021, 9:03pm

I think you need to put res.json( in the User.findOne call back. That way you will have access to the user id. My guess is that it’s in the foundUser argument.

Good luck

DragonOsman · March 13, 2021, 9:47pm

I took out its declaration because of the error saying I can’t modify it. I want to know how to get around the error and/or correctly set the _id field.

My app.post callback code was originally like this:

app.post("/api/exercise/new-user", (req, res) => {
  const username = req.body.username;

  User.find({}, (err, users) => {
    if (err) {
      console.log(err);
    }

    const id = users.length;

    const user = new User({
      username: username,
      _id: id
    });

    User.findOne({ username: username }, (err, foundUser) => {
      if (err) {
        console.log(err);
      }

      if (!foundUser) {
        user.save((err, user) => {
          if (err) {
            console.log(err);
          }
          console.log(`user ${user.username} saved to database!`);
        });
      }
    });

    res.json({ user });
  });
});

And when I try to submit a user with this code, I get this error in the console now (was telling me before that I can’t modify the _id manually):

Error: user validation failed: _id: Cast to ObjectId failed for value "0" at path "_id"
    at ValidationError.inspect (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\error\validation.js:47:26)
    at internal/per_context/primordials.js:23:32
    at formatValue (internal/util/inspect.js:783:19)
    at inspect (internal/util/inspect.js:337:10)
    at formatWithOptionsInternal (internal/util/inspect.js:2016:40)
    at formatWithOptions (internal/util/inspect.js:1898:10)
    at console.value (internal/console/constructor.js:323:14)
    at console.log (internal/console/constructor.js:358:61)
    at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\server.js:50:21
    at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\model.js:4870:16
    at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\helpers\promiseOrCallback.js:16:11
    at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\model.js:4893:21
    at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\model.js:500:16
    at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\kareem\index.js:247:48
    at next (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\kareem\index.js:168:27)
    at next (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\kareem\index.js:170:9) {
  errors: {
    _id: CastError: Cast to ObjectId failed for value "0" at path "_id"
        at ObjectId.cast (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\schema\objectid.js:281:11)
        at ObjectId.SchemaType.applySetters (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\schematype.js:1091:12)
        at model.$set (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\document.js:1280:20)
        at model.$set (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\document.js:1024:16)
        at model.Document (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\document.js:148:12)
        at model.Model (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\model.js:105:12)
        at new model (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\model.js:4713:15)
        at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\server.js:37:18
        at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\model.js:4870:16
        at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\model.js:4870:16
        at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\helpers\promiseOrCallback.js:24:16
        at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\model.js:4893:21
        at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\query.js:4400:11
        at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\kareem\index.js:136:16
        at processTicksAndRejections (internal/process/task_queues.js:75:11) {
      stringValue: '"0"',
      messageFormat: undefined,
      kind: 'ObjectId',
      value: 0,
      path: '_id',
      reason: Error: Argument passed in must be a single String of 12 bytes or a string of 24 hex characters
          at new ObjectID (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\bson\lib\bson\objectid.js:59:11)
          at castObjectId (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\cast\objectid.js:25:12)
          at ObjectId.cast (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\schema\objectid.js:279:12)
          at ObjectId.SchemaType.applySetters (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\schematype.js:1091:12)
          at model.$set (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\document.js:1280:20)
          at model.$set (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\document.js:1024:16)
          at model.Document (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\document.js:148:12)
          at model.Model (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\model.js:105:12)
          at new model (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\model.js:4713:15)
          at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\server.js:37:18
          at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\model.js:4870:16       
          at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\model.js:4870:16       
          at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\helpers\promiseOrCallback.js:24:16
          at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\model.js:4893:21       
          at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\query.js:4400:11       
          at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\kareem\index.js:136:16
    }
  },
  _message: 'user validation failed'
}
E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\helpers\promiseOrCallback.js:19
            throw error;
            ^

TypeError: Cannot read property 'username' of undefined
    at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\server.js:52:36
    at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\model.js:4870:16
    at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\helpers\promiseOrCallback.js:16:11
    at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\model.js:4893:21
    at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\model.js:500:16
    at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\kareem\index.js:247:48
    at next (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\kareem\index.js:168:27)
    at next (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\kareem\index.js:170:9)
    at Kareem.execPost (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\kareem\index.js:218:3)   
    at _handleWrapError (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\kareem\index.js:246:21) 
    at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\kareem\index.js:273:14
    at _next (E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\kareem\index.js:95:14)
    at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\kareem\index.js:508:38
    at processTicksAndRejections (internal/process/task_queues.js:75:11)
Emitted 'error' event on Function instance at:
    at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\model.js:4872:13
    at E:\programming\visual_studio_code\boilerplate-project-exercisetracker\node_modules\mongoose\lib\helpers\promiseOrCallback.js:16:11
    [... lines matching original stack trace ...]
    at processTicksAndRejections (internal/process/task_queues.js:75:11)

and I get this JSON object in the browser as the response:

{"user":{"_id":"604d31e8174c225d144b3bbf","username":"DragonOsman"}}

It has to be {"username": <username>, "_id": <id>} doesn’t it?

So yeah, please help me out here.

Sky020 · March 13, 2021, 9:52pm

Hey there,

Why are you trying to modify the _id? One is automatically created for you. Use it. This is the really nice thing about working with MongoDB.

DragonOsman · March 13, 2021, 9:55pm

When I’m initially creating the document, the _id shouldn’t be set yet since the document doesn’t even exist yet, right? So how would I be able to get the _id from MongoDB?

Sky020 · March 13, 2021, 9:57pm

user.save((err, user) => {
          if (err) {
            console.log(err);
          }
          console.log(`user ${user.username} saved to database!`);
        });

This is why you get a Document back from the .save callback.

DragonOsman · March 13, 2021, 9:58pm

So I can get the _id inside that callback and use it for the JSON response, then?

Sky020 · March 13, 2021, 9:59pm

~~No need. The document (what you called user) contains the created object as it is in the DB - with the _id.~~

EDIT:

Yes

DragonOsman · March 13, 2021, 10:10pm

So to do this, what do I do here with this?

    const user = new User({
      username: username,
      _id: id
    });

Here’s my current code in app.post:

app.post("/api/exercise/new-user", (req, res) => {
  const username = req.body.username;

  User.find({}, (err, users) => {
    if (err) {
      console.log(err);
    }

    const id = users.length;

    const user = new User({
      username: username,
      _id: id
    });

    User.findOne({ username: username }, (err, foundUser) => {
      if (err) {
        console.log(err);
      }

      if (!foundUser) {
        user.save((err, user) => {
          if (err) {
            console.log(err);
          }

          console.log(`user ${user.username} saved to database!`);

          res.json({
            username: user.username,
            _id: user._id
          });
        });
      }
    });
  });
});

What else do I need to do here?

jsdisco · March 13, 2021, 10:39pm

It looks like you’re doing far too much actually. You don’t have to .find({}) all documents in the collection first to get the length of the collection, you don’t have to set the ._id manually. That’s a property that mongoose puts on all your documents automatically, you really don’t have to worry about that. I don’t know if you’re using MongoDB Compass, but it’s a great tool to quickly check the content of your database. You’ll see that all documents have that ._id by default.

You only have to check if that username is already taken, like you do with findOne({username}). Then save them to the database if their username isn’t already taken.

Now when you save a new user with .save, it returns the new user, with the ._id already present. You really only have to res.json(theUserThatSaveReturned).

DragonOsman · March 14, 2021, 11:48am

Like this, then?

app.post("/api/exercise/new-user", (req, res) => {
  const username = req.body.username;

  const user = new User({
    username: username
  });

  User.findOne({ username: username }, (err, foundUser) => {
    if (err) {
      console.log(err);
    }

    if (!foundUser) {
      user.save((err, user) => {
        if (err) {
          console.log(err);
        }
        console.log(`user ${user.username} saved to database!`);
      });
    }

    res.json({
      username: user.username,
      _id: user._id
    });
  });
});

I’ll move on to the next user story if this is good. I tested it and I get back the expected JSON response.

Sky020 · March 14, 2021, 12:03pm

What if this fails? You should only respond with the user, if this succeeds.

DragonOsman · March 14, 2021, 12:20pm

If it fails, the if (err) code will run, right? So if that code doesn’t run, that means the document was successfully added to the database. Or is that a misunderstanding?

DragonOsman · March 14, 2021, 12:31pm

For the user story

You can make a GET request to api/exercise/users to get an array of all users. Each element in the array is an object containing a user's username and _id.

would a JSON response like this

{"users":[{"_id":"604df36384b54c25bc43b475","username":"DragonOsman","__v":0}]}

pass the test or do I need to have _id after username? This is my code for the GET request:

app.get("/api/exercise/users", (req, res) => {
  User.find({}, (err, foundUsers) => {
    if (err) {
      console.log(err);
    }

    res.json({
      users: foundUsers
    });
  });
});

Sky020 · March 14, 2021, 12:57pm

That is correct, but you still try to respond with this, even when there is an error:

Then, as no user was saved, your app will crash, and the response will fail. This hint is, move the response inside the .save callback.

This response is incorrect, because this is not

You are responding with an object. Use the example app, to get an example of the response:
https://exercise-tracker.freecodecamp.rocks/api/exercise/users

Order does not matter.

DragonOsman · March 14, 2021, 1:37pm

Like this for the GET request, then?

app.get("/api/exercise/users", (req, res) => {
  User.find({}, (err, foundUsers) => {
    if (err) {
      console.log(err);
    }

    res.json(foundUsers);
  });
});

Sky020 · March 14, 2021, 1:44pm

That looks about right. Only you will be able to ensure foundUsers is an array. Do some testing.

Something you should be keeping in mind throughout these projects is proper error handling. What if I ask for /users, but there are none? What if the database fails to return on a query? What if there is some connection problem between the mongodb/mongoose driver and the database?

You are getting there, but the best way for you to understand is to experiment, and search through Google/documentation.

DragonOsman · March 14, 2021, 1:48pm

How about this?

app.get("/api/exercise/users", (req, res) => {
  User.find({}, (err, foundUsers) => {
    if (err) {
      console.log(err);
      res.json({ error: err });
    }

    if (foundUsers.length === 0) {
      res.json({ error: "No users in database" });
    }

    res.json(foundUsers);
  });
});

This is the JSON response now:

[{"_id":"604df36384b54c25bc43b475","username":"DragonOsman","__v":0}]

Sky020 · March 14, 2021, 2:31pm

That looks decent.

I believe that will pass the test, but, for generality, following the user story, you should probably not include _v in the response, because it does not ask for it.

DragonOsman · March 14, 2021, 3:01pm

I tried this:

app.get("/api/exercise/users", (req, res) => {
  const users = [];
  User.find({}, (err, foundUsers) => {
    if (err) {
      console.log(err);
      res.json({ error: err });
    }

    if (foundUsers.length === 0) {
      res.json({ error: "No users in database" });
    }

    for (const user in foundUsers) {
      users.push({
        username: user.username,
        _id: user._id
      });
    }
  });

  res.json(users);
});

But I get an empty array in the response. I’d originally used of but that had the same result. What did I do wrong?