Mongoose subdocuments and populations

Hello.

I am working on this CodeSandbox Node server, and I’m struggling with mongoose.
Basically I cannot figure out how to reference things with each other:

  • each item should be in a room;
  • each room should contain an array of items;
  • each item comes from a supplier;
  • each supplier provided an array of items.

In theory what I wrote there should work, but when I post something on the /item /add endpoint, the objects array in the room and in the supplier do not get updated. How can I solve this?

Hello!

Is your project running? I’m not sure it is. Every request I tested returns 502, bad gateway, which (usually) means that the project is not running.

No, it is not. I don’t think it’s supposed to though, there is no MongoDB to connect to on Codesandbox sadly. Maybe I’m missing something though

Okay sorry, now it is running. Of course, there is still no MongoDB running, since I didn’t commit that on Github

I’ve ran some tests and it’s working for me.

It doesn’t work if you post something directly to the items route without previously add the required room and supplier. You must create the room(s) and supplier(s) first by calling /suppliers/add and /rooms/add first. If you want them to be automatically created, add the logic for the upsert (you have unique indexes which will make it harder).

Here are the test I ran using the Firefox Plugin RESTer (which you can download from here) or you could use Postman.

Copy the file contents (rester-tests.json) to a file on your computer and then import it to RESTer or Postman.

First of all, thank you very much for your kindness.
I’m not sure I got it. The trials I did with this server running with Postman were:

  • add a room by manually inserting the fields in endpoint /rooms/add
  • grabbing its _id field and copying it
  • do the same with suppliers
  • passed a POST request on /items/add with the body:
{
    "name": "Test",
    "description": "Test description",
    "room": "5f7cb72264b2112d10c7080e",
    "supplier": "5f7b11897370f304c04279d8",
    "invoiceNumber": "1234567890",
    "purchaseDate": "2020-11-02",
    "inUse": true
}

This does not crash, but if I send a GET request on /items all I get is:

[
    {
        "description": "Test description",
        "invoiceNumber": "1234567890",
        "purchaseDate": "2020-11-02T00:00:00.000Z",
        "_id": "5f7dae75f65c63378877f6c0",
        "name": "Test2",
        "room": {
            "objects": [],
            "_id": "5f7cb72264b2112d10c7080e",
            "__v": 0
        },
        "supplier": "5f7b11897370f304c04279d8",
        "inUse": true,
        "createdAt": "2020-10-07T12:03:01.422Z",
        "updatedAt": "2020-10-07T12:03:01.422Z",
        "__v": 0
    }
]

The room gets referenced and the supplier does not; furthermore, even if the room gets referenced, the expected behavior is for the item _id to appear inside the array, also when I do a GET request on /rooms. I’m not sure what I’m doing wrong :sweat_smile:

I see :stuck_out_tongue:.

That’s just because you’re not populating the supplier:

Item.find(query)
    .populate('room') // You could add .populate('supplier') and should work.
    .then((items) => res.json(items))
    .catch((err) => res.status(400).json(err));

furthermore, even if the room gets referenced, the expected behavior is for the item _id to appear inside the array

This is a problem though, because you’ll be creating circular references which, among other problems, decrease performance.

On the other hand, if you populate the objects of the room when you query the room, that’s another impact on performance unless you’re absolutely sure there will never be a lot of objects. Instead, you should paginate the results and/or limit how much your API users can read.

If you really need to populate the item.room.objects, you can: populate('room.objects'). And, in your GET /rooms, you should also populate the objects, which you’re not:

Room.find(query) // populate('objects')
    .then((rooms) => res.json(rooms))
    .catch((err) => console.log(err));

I hope it helps :slight_smile:! Otherwise let me know,

Regards!

Thank you very much! I will try.
Thing is, I’m really new to backend, so everything I’m doing, I don’t actually know if it’s the smartest way to do it :sweat_smile: only way to learn is doing though.

What I actually care is that every object has a room, so that when I will do the frontend, the end user should be able to see in which room is an object, but also which objects are contained inside the selected room; so I should have an array of items inside the room schema, right?

No worries, it’s understandable :slight_smile:.

What I actually care is that every object has a room, so that when I will do the frontend, the end user should be able to see in which room is an object, but also which objects are contained inside the selected room; so I should have an array of items inside the room schema, right?

What I do is have references from the many (the Item) side to the one (the Room) side only. So, when I need to query for the Items of a specific room, I query:

Item.find({ room._id: roomId }).then(items => res.json(items));

Assuming you’re viewing (on the UI) the current room, you should already have access to the room id on the front end, so you would request: GET /room/room_id/items, replacing room_id with the actual _id. Your route would be:

router.route('/:roomId/items', (req, res) => {
  const { roomId } = req.params;
  req.query.roomId = roomId;
  return  itemsController.getItem(req, res);
});

That way you could paginate the results and not necessarily retrieve every Item for a specific room for every request.

Again, your approach is correct if you don’t expect thousands of objects per room.

By the way, whenever you save an Item, you must save the generated _id to the Room.objects, otherwise it will not populate:

newItem
    .save()
    .then(savedItem => {
      console.log('Saved item:', savedItem);
      return Room.findByIdAndUpdate(room, {
        $push: {
          objects: savedItem._id
        }
      });
    })
    .then(() => res.json("Item added!"))
    .catch((err) => res.status(400).json("Error: " + err));

The same applies to the Supplier: you must push the _id to the array.

2 Likes

So by this you mean defining only the room as a reference in Item and not the other way around, leaving only the name field in Room?

Well no, hopefully the objects in a room are not even a hundred :joy:
Wow, I have to say that this mongoose is really a complex fella :sweat_smile:

1 Like

So by this you mean defining only the room as a reference in Item and not the other way around, leaving only the name field in Room?

Yep, exactly that :slight_smile::

// This is the simplified mongoose Schema
item = {
  name: String,
  room: { type: ObjectID, ref: 'Room' },
  // Other properties
}

room = { name: String }

Well no, hopefully the objects in a room are not even a hundred

In that case, don’t worry about it and leave it as it is.

Check out what I did on my project (cloned from yours). This would be the result from GET /rooms:

[
    {
        "objects": [
            {
                "description": "Lorem ipsum...",
                "invoiceNumber": "",
                "purchaseDate": "2020-10-07T00:00:00.000Z",
                "_id": "5f7dd5fcdf3f2800c5f7fb2f",
                "name": "Item 1",
                "room": "5f7dd46ddf3f2800c5f7fb2c",
                "supplier": "5f7dd5d4df3f2800c5f7fb2e",
                "inUse": false,
                "createdAt": "2020-10-07T14:51:40.274Z",
                "updatedAt": "2020-10-07T14:51:40.274Z",
                "__v": 0
            },
            {
                "description": "Lorem ipsum...",
                "invoiceNumber": "",
                "purchaseDate": "2020-10-07T00:00:00.000Z",
                "_id": "5f7de9d1c22cf603bb0fc92a",
                "name": "Item 4",
                "room": "5f7dd46ddf3f2800c5f7fb2c",
                "supplier": "5f7dd5d4df3f2800c5f7fb2e",
                "inUse": false,
                "createdAt": "2020-10-07T16:16:17.465Z",
                "updatedAt": "2020-10-07T16:16:17.465Z",
                "__v": 0
            },
            {
                "description": "Lorem ipsum...",
                "invoiceNumber": "",
                "purchaseDate": "2020-10-07T00:00:00.000Z",
                "_id": "5f7dea6631bea903e1f3820b",
                "name": "Item 2",
                "room": "5f7dd46ddf3f2800c5f7fb2c",
                "supplier": "5f7dd5d4df3f2800c5f7fb2e",
                "inUse": false,
                "createdAt": "2020-10-07T16:18:46.787Z",
                "updatedAt": "2020-10-07T16:18:46.787Z",
                "__v": 0
            }
        ],
        "_id": "5f7dd46ddf3f2800c5f7fb2c",
        "name": "Room X",
        "__v": 0
    }
]

Thank you very much, you’ve been very kind! I’ll give it a whirl :smile:

1 Like

Sorry to bother again, but I updated the promise with

newItem
    .save()
    .then(savedItem => {
      console.log('Saved item: ', savedItem);
      Room.findByIdAndUpdate(room, {
        $push: {
          objects: savedItem._id
        }
      });
      Supplier.findByIdAndUpdate(supplier, {
        $push: {
          objects: savedItem._id
        }
      })
    })
    .then(() => res.json("Item added!"))
    .catch((err) => res.status(400).json("Error: " + err));

and now the supplier pushes in the id of the room :sweat_smile:

Try this instead:

newItem
    .save()
    .then(async savedItem => {
      console.log('Saved item:', savedItem);
      const update = {
        $push: {
          objects: savedItem._id
        }
      };
      const savedRoom = await Room.findByIdAndUpdate(room, update, { new: true });
      console.log('Saved room:', savedRoom);
      const savedSup = await Supplier.findByIdAndUpdate(supplier, update, { new: true });
      console.log('Saved supplier:', savedSup);
    })
    .then(() => res.json("Item added!"))
    .catch((err) => res.status(400).json("Error: " + err));

Aaaaaaah right, I didn’t think about async. I’m learning more this afternoon than I did in the whole month of September :joy:
One last thing: when I delete all the items, references don’t get actually removed. I can’t access the objects array to set its length to 0;

Item.deleteMany()
    .then(item => {
      console.log('Emptied objects.');
      item.room.objects.length = 0;
    })
    .then(() => res.status(200).json("All items deleted."))
    .catch((err) => res.status(400).json("Error: " + err));

This doesn’t work, since it returns undefined

LOL! That’s great though :slight_smile:.

One last thing…

That’s because you have to query the room collection, find the specific room, and then remove the item from that collection. It’s, basically, the same issue as with the inserts.

Take a look at this question on StackOverflow and try to adapt it to your models.

I also recommend you to take the courses on mongodb university, they will teach you the basic operations of the driver that you can then apply through mongoose. There’s also the documentation for the different drivers (the layer below mongoose).

Let me know how it goes :slight_smile:.

1 Like