I give up for now... anybody available to write a sample code?

[Elindo]

I have about 2 months trying to get something done, and I can’t get it… doesn’t mean I won’t learn, it means this part I need it sooner than later while I get some more classes on this topic…

I need a sample code for pagination and global filtering for a collection from MongDB Atlas… and I need to present it on the fronted on a table structure with , and the options of next and previous page. I would like to do the filtering and pagination from the backend, and just present the results on the frontend.

The chart/s I am working can easily reach 700-800 items, maybe less, maybe more… I’ll be looking to throw about 100 items per page… if the user can change the number of items per page is great, but I can just leave set at 100…

Then… I also found this from MongoDB Atlas:

> #### WARNING
> 
> Using `skip()` with large values (more than a few pages of results) is NOT recommended as the server scans the skipped range of documents in order to find the start of the requested document range. See [cursor.skip behaviour](https://docs.mongodb.com/manual/reference/method/cursor.skip/#behavior).

Anyway… where could I find somebody with the power of knowledge to help me with this thing???

And where can I get the knowledge to get it done by myself next time?

[Elindo]

BTW… I am doing this in REACT… Anybody working on MERN?

[JeremyLT]

This forum isn’t for requesting that others write code for you.

[Elindo]

Cool… do you know any other forums on this topic?

[ingabd]

The strategy is more or less like this.
You can send http request to your server with additional req.query to control the page size / items per page. Set it to default 100 is a good idea if no input from user. Node server will receive this queries then use them to construct query to database but with additional options: limit, size, skip; which basically tell Mongo how many data to be sent back to server, instead of send back all of them. Your server will receive chunk of data with its properties (for example: 100 items, current page 1, total page 9). then server will send this data to your client. In Mongoose this option will be argument no. 3 in this method: Model.find({}, {}, options, callback), but be sure to checkout Mongoose or Mongo doc for more detail (I don’t know which one you use but these two are different: Mongoose is an ORM, and additional layer to Mongo for easier interaction with Mongo db, Mongo is the db itself and it already has methods for interactions, be sure not to confuse between the two when you dive into their docs). Or instead of using options you can also use method chaining, like skip() that you mentioned, or limit(), sort(), select(). There are more than 1 way to tell computer to do something.

Then in your React client you can have some logic to show this chunk of data. Your client only know how many data and how many pages there are after it receive response from your server so you must put some logic to handle this.

Same with filtering, you add some filter keyword in req.query, then server will use this to construct options for its db query (you can have exact similarity or regular expression, there are many options).

some implementation tutorials:

Pagination Using MongoDb and React

How to add Pagination using Node and Mongo | ObjectRocket

or just find some others, I think you sometimes need to combine from several tutorials to get solution that works for your case,

Pagination itself is a standard practice because you don’t want to send 10 million data from database to server to your client, it’s too heavy, hence there are many tutorials or Q/A about it. Google is our friend here. Hope this help.

[Elindo]

Cool thanks… I’ll take a look at it…

[Elindo]

Oh look at that… there is something called reactstrap and it has a pagination helper… with a pagination link…

Mongoose has a pagination helper too… but didn’t see the link to next page options…

[Elindo]

Also… not using axios?

There is too many things to learn…

getUsers = (currentPage) => {
    const queryParams = {};
    queryParams["page"] = currentPage; //Page Number
    queryParams["pagination"] = 8; //Number Of records on Page
    this.dataRequest('http://localhost:5000/getUsers', 'POST', queryParams)
      .then(data => {
        console.log("Data FEtched ", data)
        this.setState({
          users: data.users
        })
      })
      .catch(err => {
        console.log("Error In Fetching Users ", err)
      })
  }

[Elindo]

Saayy…

I suppose this would be frontend pagination? given the data size, would it have problems slowing down the computer?

I must say it looks simpler than trying to do it from the backend…

Also… would reactstrap offer global filtering along with their pagination?

[ingabd]

You are correct, there are some options regarding pagination: server side or client side, which one is better? I would answer server side because by specifying the chunk of data in the server and ask only required data from the database, you will only send that chunk of data through the network, in case of client side, you still send all of data through the network, but only show some of them in the client, kinda negating the strength of pagination itself from network bandwidth point of view,

So why there is client side pagination? some possible explanation is to serve different client by one server. For example, in browser client it is possible to serve 50 items in 1 page, but when you browse from your phone, it is more logical to only show like 10-20 items per page, due to limited real estate in your screen, this is where reactstrap come to play, it’s just my view though may be other have better explanation from their experiences,

You are also correct noticing the axios thing, there are lot of web stacks out there, to do 1 thing you have 2/3/4 package options, for example for http request you can use fetch, axios, or jQuery’s ajax, I think there are some more, here in the tutorial he just define his own method with fetch inside,

And to do http request from Node.js server you can use axios, or some others, but you cant use fetch because it is only in browser, Node.js does not have it, it has node-fetch package though,

You use MERN,
M for Mongo, other options are Cassandra, or in relational we have MySQL, PostgreSQL,
E for Express, other option is Adonis.js which looks like Laravel PHP, they who come from PHP will find familiar situation by using Adonis instead of Express,
R for React, other options are Vue, Angular, or jQuery, or vanilla JavaScript,
N for Node, if you use other language like Golang you will use similar like Node but for Golang,

I think it is easy to lost in the forest, but know that these different tools/stacks serve the same objective, and sometimes the differences are marginal, no need to master all of them just master the basic and you can easily and readily read other docs when you need to,

“Take things that work for you”, this one is from Pragmatic Programmer.

[Elindo]

Say, on this section of the data below… what is the reason of the first route? Seems like the second route is required from the front end to get the data, but not sure what is the first route doing?

Also… if I want to implement a filed filter or global filter… I suppose I would put it on the second route?

var express = require('express');
var router = express.Router();
var userss = require('../models/user_model');

router.get("/:id?", function (req, res) {
    //Pagination For number Of receords on page
    if (req.params.id) {
        //Case For Counting Number OF Users
        userss.find({})
        .count()
        .then(data => {
            res.status(200).send({
                "cnt" : data
            })
        })
        .catch(err => {
           res.status(400).send({
               "err" : err
           })
        })
    }
})

router.post("/",function(req,res){
    const pagination = req.body.pagination ? parseInt(req.body.pagination) : 20;
    //PageNumber From which Page to Start 
    const pageNumber = req.body.page ? parseInt(req.body.page) : 1;
    userss.find({})
        //skip takes argument to skip number of entries 
        .sort({"id" : 1})
        .skip((pageNumber - 1) * pagination)
        //limit is number of Records we want to display
        .limit(pagination)
        .then(data => {
            res.status(200).send({
                "users": data
            })
        })
        .catch(err => {
            res.status(400).send({
                "err": err
            })
        })
})

module.exports = router;

[Elindo]

Would this work for filtering to include it in the paginations?

const filters = {I could include some indexing code from MogoDB Atlas here}

router.post("/",function(req,res){
   const filters = {I could include some indexing code from MogoDB Atlas here}
    const pagination = req.body.pagination ? parseInt(req.body.pagination) : 20;
    //PageNumber From which Page to Start 
    const pageNumber = req.body.page ? parseInt(req.body.page) : 1;
    userss.find({filters})
        //skip takes argument to skip number of entries 
        .sort({"id" : 1})
        .skip((pageNumber - 1) * pagination)
        //limit is number of Records we want to display
        .limit(pagination)
        .then(data => {
            res.status(200).send({
                "users": data
            })
        })
        .catch(err => {
            res.status(400).send({
                "err": err
            })
        })
})

module.exports = router;

The filter code would come from a code taken from MongoDB Atlas indexing for the collection I am working with…

Could I just import mongodb from mongodb? and it would work OK?

Then… I would still need to transfer the filter section to the frontend… which I think I have a code to do it… but not sure yet if MongoDB and Mongoose know what I would be talking about…

For filtering I havent found any options from Mongoose… most of everything on filtering comes from MongoDB Atlas…

[ingabd]

Allright I think I miss one thing here, he uses second route POST to send pagination data via req.body, because you are not recommended to send any req.body in GET request, it works, but is it nice? you use POST but you don’t post anything, instead you just getting data so you should use GET instead, POST should return res.status(201) meaning adding data, but he set res.status(200), don’t know for sure though may be other have better answer regarding this,

it is better to use GET, because you only querying data, not modifying or adding data, the pagination data (page, size, etc) better be included in http request via req.query, just like I explained earlier,

it is a possibility to use paginated GET via POST because you want to use req.body, but it kinda looks ugly, right? I don’t think it is recommended in REST API guidance,

answer for your question, yes, if you want to use POST like this tutorial, you will include your filter in second route, but better to use GET and include your filter in req.query,

and your Mongo vs Mongoose question, you definitely can do lot of things with Mongo as you can with Mongoose, but I think it is better to stick with only one tool? I don’t know about this may be other have better answer,

that’s not how you define the filters, object always need key: value pair, I think your filters will cause error lol, you might find guidance from search bar tutorial because filter and search bar are the same thing,