Please help, I'm going nuts (expressjs, mongoose and callbacks)

So I’m working on the URL Shortener Microservice, but I’m having a REALLY hard time in figuring out how to use callbacks in asynchronous code. I’ve tried to read other forum posts and related articles but I’m more confused than ever. I’d have a lot of questions, but atm what I’m trying to figure out is how to return a value from some asynchronous code.

Consider the following:

// a function that tries to get the number of documents for a specific mongoose model:
const docsCount = (model, done) => {
   model.estimatedDocumentCount((err, count) => {
      if(err) return done(err);
      done(null, count)
   })
}

I’m using that function inside the definition of an Express POST endpoint:

app.post('/api/shorturl', (req, res, next) => {
   const inputUrl = req.body.url;
   let currentCount; // I want to update this variable
   // check url validity
   if (!testUrl(inputUrl)) {
      // not valid, respond with json
      res.json({ url: 'invalid' })
   } else {
      // valid url
      docsCount(urlShortner, function(err, count) {
         if(err) {
            console.log(err)
         } else {
            currentCount = count
            console.log(`successfully logs currentCount, but comes second: ${currentCount}`)
         }
      })
   }
   console.log(`currentCount is undefined here, and comes first: ${currentCount}`)
   next()
})

Inside the else statement I have access to the updated currentCount variable, but outside it, it is undefined.
I know that the second console.log() doesn’t wait for the docsCount to finish before executing…still, I’ can’t figure out how to access the updated currentCount value outside the docsCount function.

Suggestions?
Thanks

1 Like

I find a way after reading this article about Async/Await in Expressjs, but I’d really love to know how to do that with callbacks. I feel as confused as I was when I first encountered recursion.

So what do you want to do next after you updated the currentCount variable?
Because working with callbacks is calling callback inside callback, so called “callback hell”.

It is a hell indeed… I managed to do what I wanted making the handler in the app.post() definition asychronous. This way I could also get rid of the docsCount function of my first post.
Now, I can get data with mongoose just prepending await to the methods:

// POST requests
app.post('/api/shorturl', async (req, res, next) => {
   const inputUrl = req.body.url;
   // check url validity
   if (!testUrl(inputUrl)) {
      // not valid, respond with json
      res.json({ url: 'invalid' })
   } else {
      // valid url, check if it's an entry in the database
      const urlSearch = await urlShortner.find({ original_url: inputUrl })
      // not an entry, create and save new document and respond with json
      if (urlSearch.length === 0) {
         const docsCount = await urlShortner.estimatedDocumentCount()
         const newDoc = new urlShortner({
            original_url: inputUrl,
            short_url: docsCount + 1
         })
         await newDoc.save()
         console.log('url not found, created and saved new document')
         res.json({
            original_url: inputUrl,
            short_url: docsCount + 1
         })
      } else {
         // url matches an entry in database
         res.json({ original_url: inputUrl, short_url: urlSearch[0].short_url })
      }
   }
})

I should probably add errors handlers now.

To be honest, I don’t think I could come up with a solution if had to use only callbacks, but I’d still want to grasp how it would be done, at least.

So looking at your async/await code I’d say with callbacks it’ll look something like this:

app.post("/api/shorturl", (req, res, next) => {
  const inputUrl = req.body.url;
  // check url validity
  if (!testUrl(inputUrl)) {
    // not valid, respond with json
    return res.json({ url: "invalid" });
  }
  // valid url
  if (URLSearchParams.length === 0) {
    docsCount(urlShortner, function (err, count) {
      if (err) {
        return res.json({ url: "invalid" });
      }
      const newDoc = new urlShortner({
        original_url: inputUrl,
        short_url: count + 1,
      });

      newDoc.save(function (error, data) {
        res.json({
          original_url: inputUrl,
          short_url: count + 1,
        });
      });
    });
  } else {
    // url matches an entry in database
    res.json({ original_url: inputUrl, short_url: urlSearch[0].short_url });
  }
});

The more callbacks you have the more nested it becomes. For example, if after save you’d want to do another operation it’d add another level.

1 Like

Thank you so much, @jenovs,
I’ll have a look tomorrow.

The only thing I’m not sure of is how you get the value of docsCount in short_url: docsCount + 1, but let me have a look after some rest…now it would be pointless :expressionless:

Yeah, it should be count. I updated my post. But it’s mostly a pseudocode anyway.

Ok, I almost got it.
I looked at your code, did a few mods and it works. Really appreciate your help, thank you.
But it is an absolute nightmare having to write code like that.

Basically (correct me if I’m wrong), if you need to do something with a value that comes from an asynchronous function, you need to do that within that function call…or you can use a callback (which doesn’t really simplify things, nor makes your code easier to understand).

The other (newer) options are promises and async/await, which is just syntactic sugar for promises.

Correct.

You probably mean to extract the code into separate function, because this:

asyncFunction(args, function(result) {
  // do stuff with result
})

and this:

function handleData(data) {
  // do stuff with data
}

asyncFunction(args, handleData)

are the same. They’re both callbacks.

1 Like

Thant’s what I meant.

Although I’m under the impression that the term ‘callback’ generally refers to a (regular) function invoked after an asynchronous operation has been completed.

Or is this a pointless distinction?

Yes, that’s what callback is. But it doesn’t matter if you provide a pre-defined function or define it inline. Both will be invoked after async op has been completed.

Yeah, I get it.

Well, thanks again for your help