I need help in URL Shortener Microservice, please

Tell us what’s happening:
I have been struggling with this URL Shortener project for a while. :worried:
Here is my replit link

I am trying to get this Replit code running on the console, but it will not run the preview of the URL Shortener.

I used my same MONGO_URI code from the first cluster I created in MongoDB Atlas.

I put the MONGO_URI code into the Secrets (Environment Variable) file. Did I put it in the server.js file correctly? If not, how do I put in my mongodb in this project? I also installed the latest versions of mongoose & mongodb by typing in npm install mongoose and npm install mongodb in Replit Console. These were automatically updated in the package.json file within the dependencies array.

Code I am working on the server.js in Replit starter project

const express = require('express');
const mongo = require('mongodb');
const mongoose = require('mongoose');
const bodyParser = require('body-parser');
const cors = require('cors');
const dns = require('dns');

const app = express();
const mongo_key = process.env.MONGO_URI;
const Schema = mongoose.Schema;

// BASIC CONFIGURATION
const port = process.env.PORT || 3000;

// I added my MONGO_URI code in this project. Is this the proper way to add mongodb?
mongoose.connect(process.env.MONGO_URI);
// const mySecret = process.env['MONGO_URI']
// mongoose.connect(process.env.MONGO_URI, {useNewUrlParser: true, useUnifiedTopology: true} );
 
const urlSchema = new Schema({
  id: Number,
  url: String,
});

const urlModel = mongoose.modl('Url', urlSchema);

app.use(cors());
app.use(bodyParser());
app.use('/public', express.static(`${process.cwd()}/public`));

app.get('/', function(req, res) {
  res.sendFile(process.cwd() + '/views/index.html');
});

// Your first API endpoint...
// app.get('/api/hello', function(req, res) {
//    res.json({ greeting: 'hello API' });
//  });


app.post("/api/shorturl/new", function (req, res) {
  let original = req.body;
  let theData;

  let urlRegex = /https:\/\/www.|http:\/\/www./g;
  dns.lookup(req.body.url.replace(urlRegex, ''), (err, address, family) => {
    if(err) {
      res.json({"error": err});
    } else {
      onComplete();
    }
  });
  function onComplete(){
    urlModel.find()
            .exec()
            .then(docs => {
              theData = docs;
              var doc = new urlModel({"id": theData.length, "url": req.body.url });
              theData = theData.filter((obj) => obj["url"] === req.body.url);

              if ( theData.length === 0 ) {
                doc.save()
                .then(result => {
                  res.json(result);
                })
                .catch(err = {
                  
                  res.json({"error": err});
                });
              } else {
                res.json({"error": `URL already in database as ${theData[0].id}` });
              }
            }) 
            .catch(err => {
              
              res.json({"error": err});
            });
         };
});

app.get("/api/shorturl", function (req, res) {
  urlModel.find()
          .exec()
          .then(docs => {
            res.json(docs);
          })
          .catcj(err => {
            
            res.json({"error": err});
          });
});

app.get("/api/shorturl/:short", function (req, res) {
  console.log(req.params.short);
  let short = req.params.short;
  urlModel.find({"id": short}).exec()
  .then(docs => {
    res.redirect(docs[0]["url"]);
  })
  .catch(err => {
    
    res.json({"error": err});
  })
});


app.listen(port, function() {
  console.log(`Listening on port ${port}`);
});
  • Are there any syntax errors I made?
  • Am I missing any specific codes?
  • Do I have to arrange the code in a certain order?
  • Did I do something wrong with the mongodb code, like did I put it in wrong?
  • Do I only have to work on the server.js in Replit? Are there other parts of Replit I need to work on?

I appreciate any help for this. Thank you

Your browser information:

User Agent is: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36

Challenge: URL Shortener Microservice

Link to the challenge:

You made a typo here

It supposed to be model.

But also, you created this variable here

but it doesn’t look like you are using it.

You can either delete that variable or use it in your mongoose.connect.

mongoose.connect(mongo_key, {useNewUrlParser: true, useUnifiedTopology: true});

I just looked at the starter code and it looks like you deleted the dotenv package.

You need to add this at the top of your file.

require('dotenv').config();

Hope that helps!

Ok, I corrected my code for the mongo_key and added that require('dotenv').config(); code you mentioned.

Now I am facing some issue in my Replit Console. The console keeps showing me this syntax error in one line,

res.json({ "error": err });
   ^

It says SyntaxError: Unexpected toke '.'

Why is this not working for me & how can I fix it?
What am I doing wrong in the res.json code?

Updated server.js code in Replit

require('dotenv').config(); // Add this code on top
const express = require('express');
const mongo = require('mongodb');
const mongoose = require('mongoose');
const bodyParser = require('body-parser');
const cors = require('cors');
const dns = require('dns');
const app = express();
const mongo_key = process.env.MONGO_URI;
const Schema = mongoose.Schema;

// BASIC CONFIGURATION
const port = process.env.PORT || 3000;

// I added my MONGO_URI code in this project
mongoose.connect(mongo_key, {useNewUrlParser: true, useUnifiedTopology: true});
 
const urlSchema = new Schema({
  id: Number,
  url: String,
});

const urlModel = mongoose.model('Url', urlSchema);

app.use(cors());
app.use(bodyParser());
app.use('/public', express.static(`${process.cwd()}/public`));

app.get('/', function(request, response) {
  response.sendFile(process.cwd() + '/views/index.html');
});

// Your first API endpoint...
// app.get('/api/hello', function(req, res) {
//    res.json({ greeting: 'hello API' });
//  });

app.post("/api/shorturl/new", function (request, response) {
  let original = request.body;
  let theData;

  let urlRegex = /https:\/\/www.|http:\/\/www./g;
  dns.lookup(request.body.url.replace(urlRegex, ''), (error, address, family) => {
    if(error) {
      response.json({"error": error});
    } else {
      onComplete();
    }
  });
  function onComplete(){
    urlModel.find()
            .exec()
            .then(docs => {
              theData = docs;
              const doc = new urlModel({ "id": theData.length, "url": request.body.url });
              theData = theData.filter((obj) => obj["url"] === request.body.url);

              if ( theData.length === 0 ) {
                doc.save()
                .then(result => {
                  response.json(result);
                })
                .catch(error = {
                  
                  response.json({ "error": err }); //* SyntaxError is shown on Console *//
                });
              } else {
                response.json({"error": `URL already in database as ${theData[0].id}` });
              }
            }) 
            .catch(error => {
              
              response.json({"error": error});
            });
         };
});

app.get("/api/shorturl", function (request, response) {
  urlModel.find()
          .exec()
          .then(docs => {
            response.json(docs);
          })
          .catcj(error => {
            
            response.json({"error": error});
          });
});

app.get("/api/shorturl/:short", function (request, response) {
  console.log(request.params.short);
  let short = request.params.short;
  urlModel.find({"id": short}).exec()
  .then(docs => {
    response.redirect(docs[0]["url"]);
  })
  .catch(error => {
    
    response.json({"error": error});
  })
});


app.listen(port, function() {
  console.log(`Listening on port ${port}`);
});

thank u for responding

I am little confused by this error.
I don’t see res.json in your code.

I see this part

But why is the error message referring to res?

Can you share your replit link?
Maybe it will be easier to see that way

Oh, sorry about that. I know that req and res are the shortened version of request and response. I can change them back. later

Ok I can show you my Replit code. Do you have Replit? I can share this with you by inviting you via email.

Does that sound ok with you?

You don’t have to do all of those steps to share your replit link.

The easiest way is to share the url at the top of the browser.

Then if people want to fork the project and play around with the code they can.

Sharing your url will also help others in the forum see what’s go on.

Ok, you mean like this?
Thank you for your patience and helping me.

Yep that works.

You need to delete your mongo uri out of the sample.env file.
Because right now, I have access to it.


I caught the first error which is this.
I missed this earlier.

I think you meant to write an arrow function here

.catch(err =>

But now I get a new error.
const urlModel = mongoose.modl(‘Url’, urlSchema);

You need to spell model correctly.

Once I fix those, then I get messages for deprecation warnings.
it is referring to this code here

app.use(bodyParser());

You can change that to app.use(express.json()); and that should get rid of the warning message

It is also good practice to test and make sure your database is actually connected.
Underneath your mongoose.connect you can add this code which checks if the database was successfully connected.
If so, then you will get a success message in your console.
If not, then you will get an error message.

// check for proper connection
const database = mongoose.connection;
database.on('error', console.error.bind(console, 'connection error: '));
database.once('open', () => {
    console.log('mongo database connected')
});

Ok, I corrected some of the codes and removed the MONGO_URI from the sample.env file.

I am able to run Replit and see its preview.

I am now just dealing with its 3 test requirements.

1. You can POST a URL to /api/shorturl and get a JSON response with original_url and short_url properties. Here’s an example: { original_url : 'https://freeCodeCamp.org', short_url : 1}

2. When you visit /api/shorturl/<short_url> , you will be redirected to the original URL.

3. If you pass an invalid URL that doesn’t follow the valid http://www.example.com format, the JSON response will contain { error: 'invalid url' }

I can let you look through my Replit code. Is the server.js file they only part that I work on this URL Shortener project?

As for the actual test requirements, I haven’t done this project so I wouldn’t be much help there.

But I did update your original post to include the replit link so more experienced devs can help you out.

Also, cleaning up those earlier errors will make it easier for experienced devs to jump in and assist you with the actual project requirements.

Yeah the server is fine.
That’s what most people do for these projects .

I think for larger projects you build outside of a class, you will tend to create multiple files in different folders.

I am still learning this stuff too but I found breaking up the code for larger projects into folders like routes, controllers, and models helps with the organization of the code.

But for now, I would just leave everything in the server.

I see. To be honest, this API Back-end coding is just too much for me… I prefer Front-end Web coding.

I really appreciate your help. I wish you a good day. :slight_smile:

1 Like

I can relate to that.
I spent the majority of my first year primarily focusing and learning front end.

It’s only been the last couple of months were I started diving deeper into backend so I can get better at it.

But someone else will come along and lend their expertise to your project :grinning:

1 Like

Let’s just look at the POST route because once that works, the rest are much easier.

  1. You don’t need mongo and mongoose. Since you are using mongoose exclusively, delete mongo. This is not causing any problems, however.
  2. If you are going to use express.json() instead of body-parser (they are the same, so it doesn’t matter) you need two lines, just like in the exercises:
app.use(express.urlencoded({extended: false}));
app.use(express.json());

The first decodes and the second parses and leaves the JSON payload in req.body.

  1. Make sure your API endpoints are correct. Your POST route currently is configured at /api/shorturl/new. The spec wants it at /api/shorturl. Once you fix these three, you should be running well enough to fixably fail the tests.
  2. Log all your route inputs. That means add
  console.log(req.body);
  console.log(req.params);
  console.log(req.query);

at the beginning of your routes. For the POST route, only the first will print anything and it should display the URL that was POSTed to the route. Note the lack of www. in the test URLs…

  1. Log all your responses before you send them and make sure you return them. So, instead of
res.json({ "error": err });

use

console.log({ "error": err });
return res.json({ "error": err });

for every response.

Now, run your project, go to the fCC test page in another tab, open the browser console (clear it if necessary) and run the tests. You should see some errors in the browser console (all they mean now is that the tests are failing, which we know). Switch back to your repl.it project and look at that console. You should have plenty of output. Scroll back to the top (helps to clear the console before running the tests), which should be something like:

{
  url: 'https://boilerplate-project-urlshortener-2-kk.jeremyagray.repl.co/?v=1631150533042'
}
{}
{}
{
  error: Error: getaddrinfo ENOTFOUND https://boilerplate-project-urlshortener-2-kk.jeremyagray.repl.co/?v=1631150533042
      at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:66:26) {
    errno: 'ENOTFOUND',
    code: 'ENOTFOUND',
    syscall: 'getaddrinfo',
    hostname: 'https://boilerplate-project-urlshortener-2-kk.jeremyagray.repl.co/?v=1631150533042'
  }
}

So, you can see your route inputs (first three objects) then the error that is your real problem from dns.lookup() whining that it can’t find the hostname because it still has the protocol (https://) and some URL parameters (/?v=1631150533042). So if you want to use dns.lookup() for validation, you are going to have to fix your URL processing so that you only get the hostname to pass to dns.lookup() either by fixing your URL regular expression (not just www.!) or by doing something else. This current failure to validate the URL is preventing you from saving URLs in the database, and therefore preventing the GET tests from working.

You can never log too much unless you’ve found the problem.

Hello Jeremy,

I made changes to my code according to your advice.
I was able to get my Replit code to run and show its console log the error message like you showed me.

I can see the route inputs from the console.

error: Error: getaddrinfo ENOTFOUND ftp:/[john-doe.org](http://john-doe.org)
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:66:26) {
errno: 'ENOTFOUND',
code: 'ENOTFOUND',
syscall: 'getaddrinfo',
hostname: 'ftp:/[john-doe.org](http://john-doe.org)'
}
}

I need help understanding what you mean by the dns.lookup()
How should I fix that?
What changes should I make in the code below?

let urlRegex = /https:\/\/www.|http:\/\/www./g;
// I need to fix something with my dns.lookup()
dns.lookup(req.body.url.replace(urlRegex, ''), (err, address, family) => {
if(err) {
console.log({ "error": err });
return res.json({ "error": err });
} else {
onComplete();
}
});

Thank you for contacting me

You’re calling dns.lookup() (documentation) to verify the URLs, after you do some pre-processing with the regular expression. dns.lookup() only accepts a hostname as its argument, so if a URL comes in as https://www.cnn.com/?page=2 for instance, you need to process that URL so that you only use www.cnn.com (the hostname) as the hostname argument for dns.lookup() (you need to remove the protocol, https, the separator ://, and any parameters, /page=2, in this example).

Your regular expression, if you choose to continue using it, needs work because not all valid URLs have www. as part of their hostname. You can’t replace with it either, because that would remove www. from hostnames that have it, possibly resulting in an invalid hostname.

You can also verify URLs for this project without using dns.lookup() with either a simple regular expression or some string processing. You really just need to make sure that the protocol and separator are correct for the tests in this project.

1 Like

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.