My URL shortener microservice not working: help, please

Hi, I finished the challenged and it is working ok when I test it, but it is not passing the official testing. Any help, please?

Your project link(s)

solution: https://replit.com/@ranran2121/boilerplate-project-urlshortener

Your browser information:

User Agent is: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36

Challenge: URL Shortener Microservice

Link to the challenge:

Hello there,

This is a common issue:

var urlDns= url.split('//')[1];
  dns.lookup(urlDns, (err) => {

The first argument for dns.lookup is expected to be only the hostname.

Hope this helps

you mean without the “www”?

Without most of it :wink:

https://www.freecodecamp.org/learn/some-other-route/
            |   hostname   |

Remember to test your app with some weird URLs. Also, if you are completely stuck, look in the network tab of the devtools to see what requests the tests are making, and what url they are testing with.

Hope this clarifies

i am afraid this is not clear at all, but I would lolve to understand it in order to see what url I am testing ^-^

Sure, I will make a quick guide of how I debug this. However, your app appears to have crashed - cannot start it.

that’s weid because it works for me (that is what I get when I submit the htts://www.freecodecamp.com)

and few minutes ago it was working when I was pasting “/+the url id”
but my laptop and connections are crap :frowning:

Steps:

  1. Open devtools (differs from browser to browser - search how to for yours)

  2. Navigate to the network tab

  3. Submit your project link
    You should see this:

  4. Click on a request (probably makes sense to see the one that failed - red):

Look at the Form Data (bottom right) - that is being passed by the freeCodeCamp tests in a request to your app.

At this stage in the curriculum (and your web development journey), the devtools should be your best friend. There are many excellent YouTube videos going over how best to use it, and I highly recommend you watch some of them. They immensely help me.

Hope this helps

is it a nice way to tell me I am a complete idiot?
well, at this point of the curriculum I can’t remember much of any hint about development tools. So far I had to browse other tutorials since from React onwards I struggled and online there is just TOO MUCH and it is easy to get totally lost, as I am .
(btw, sometimes it is just a language matter, since most of us are not english native speakers).
And sometimes the hints are confusing, like this:
" you can POST a URL to /api/shorturl and get a JSON response with original_url and short_url properties. Here’s an example: { original_url : '**https://freeCodeCamp.org**', short_url : 1}" whereas the json object in the example project is {"original_url":"*https://www.freecodecamp.com*","short_url":56}

going back to the challenge, in the "post " trunk, a part from the this ftp:/etc that should be recognized as an invalid one by DSN (I am going to fix this) it looks like working ( I mean, it is not crashing the app).
But what about inputs like "http://www.imontagnini.it/monte-greco"… does the challenge want also this type of input?

for the “get” part, if I replace the res.redirect(data.url) with res.json(data.url) I can see the correct url to redirect to ("http://www.imontagnini.it"). However the app crashes if I use res.redirect(data.url)
So for my understanding the problem is not the string for the DSN testing but the “post” part…
but probably I am too stupid for this…

Not at all. Sorry, for any misunderstanding. All I meant to say is this is the perfect time to get to know the devtools. There is not necessarily any expectation for anyone following the curriculum to use/know it, but, in my experience, it helped me get through. As such, I recommend it.

The app should work like a simple URL Shortener. So, the tests might not test for every permutation of URL, but there is somewhat of an expectation (from an end user’s perspective) how this microservice might work. That is:

  • I (the user) pass in a URL I want shortened
  • I get a response telling me what that shortened version is
  • I pass in the shortened version, and am redirected to what I originally passed in
  let completeUrl = Url.findById(urlID, (err, data)=> {
    if (err) {
      console.log("error" +urlID);
      res.json({error :"invalid url"});
    }
    //res.json(data.url);
    res.redirect(data.url);       
  })

Here are some notes on this section of code:

  • I am not sure why it is assigning anything to completeUrl as completeUrl does not get used.
  • If you encounter an error (err), you respond with an error - this makes sense. However, the callback function is not returned from. So, even if there is an error, res.redirect(data.url) is called. And, as there is an error (err), data is undefined. So, data.url crashes the app.

Hope this helps

1 Like

Many of us had the same issue.

 var url = req.body.url;
  var urlDns= url.split('//')[1];
  dns.lookup(urlDns, (err) => {

You are getting more than the hostname. I was doing the similar thing, thinking we’re getting something like http://www.mycompany.com. But the passed href could be something like http://www.mycompany.com/api/v2/id=2000. So if you pass url.split(’//’)[1] you getting more than the hostname. I solved the problem by creating an URL object from req.body.url and extracted necessary properties from the URL object to check for the valid format and address.

ok, I am working on that
but in this function do I have to check also for the http/https vs other invalid format like the ftp:/john-doe.org?
wheres for something like “Anello di Monte Greco e Toppe del Tesoro | I montagnini”, I should pass just the domain (“imontagnini.it”), correct?
but when assigning the shorten url I have to consider the complete address in order to redirect the page to the designated one, correct?
if so, what is then the purpose of the dsn lookup, if i had to do all of “his job” cleaning the input?

Here are some notes on this section of code:

  • I am not sure why it is assigning anything to completeUrl as completeUrl >does not get used.
  • If you encounter an error ( err ), you respond with an error - this makes sense. >However, the callback function is not returned from. So, even if there is an error, >res.redirect(data.url) is called. And, as there is an error ( err ), data is >undefined. So, data.url crashes the app.

so should just add and “else {redirect]” after the if(err) condition?
and get rid of that var?

Yes, that should work. Although, there are many ways to handle dealing with errors in Express apps.

Personally, I just return from the callback. This way, I am always sure no code after is run. This is not necessary, but what I do most of the time.

ok I made the changes and it is not crashing anymore (i tried with the else and with return and in both cases it works).
I made a question before, regarding “preparing” the url for the dsn: what is the purpose of the DSN lookup if it is checking ONLY the domain?
and regarding the weird test data ("ftp:/ etc.org), it should be disregarded as invalid in the DSN lookup part, correct?
I have another question regarding the app.post I wrote:

dns.lookup(urlDns, (err) => {
    if (err) {
      res.json({
        error: 'invalid URL at DNS'        
      })
    } else {
      var urlRecord = new Url({url: url});
      urlRecord.save(function(err, data){
        if(err) return res.json({error: 'invalid URL'});
        else return res.json({original_url: url, short_url: data._id.toString()});
      })
    }
  })    

so far I have followed the example code from the lessons, but here I have the error check twice, can you help me explaining the difference between the two checks? I mean, if the lookup already recognized an invalid url, which kind of check ifs the urlRecord.save callback function is performing?

So far, you have done well with dns.lookup. However,

Not necessarily. I believe that example was used, because dns.lookup does not catch it as invalid. So, you need to write some of your own logic to test if a URL satisfies the http/https criterion (user story).


For the most part, you would not use dns.lookup to check if a domain exists. Instead, you would use it to get all the other information provided in the callback:DNS | Node.js v16.2.0 Documentation

Good question. There are two main reasons for getting an error:

  1. dns.lookup(urlDns, (err) => {
  • No domain found / failed to look up urlDns
  1. urlRecord.save(function(err, data){
  • Failed to save record - Tried to save, but the operation failed.

For 2, you technically should not be returing the message 'invalid URL', because that is not the reason for the error. Instead, you could return something mentioning the write operation to the database failed (along with the error message err).

Hope this helps

yes, thank you : that was clarifying a lot.
I am now working for disregarding thing like the “ftp” stuff.
As you amy see from my code, I prepared a function that extracts and returns the domain and I then call this function inside the app.post to pass the domain for DNS.
But now I have to check for the “ftp”, so it should be better to perform all the splitting INSIDE the app.post so I can take advantage of the splitting, correct?
Otherwise, if I keep the function outside, I should then return an array and use the items inside the app.post for something like this

if (arr[0]!=https or http) -> invalid url
else (dns.lookup) -> continue with the existing code

In your opinion, which way is better?

UPDATE: i have completed the challenge. Despite it was working on my side, it was still failing the last test (the fake address). I found out the problem looking through the navigation bar in the dev too ;).
But I still would appreciate your opinion on WHERE to manipulate the input string. Thank you :slight_smile:

Well done, on passing!

I did this project a while ago, and might do it different, if I were to redo. However, this is what I did:

app.route('/api/shorturl/new').post((req,res)=> {
  // Find all URLs in database (do not create duplicate entries)
  URL.find({}, (err, docs) => {
    if (err) return console.log("Error in database");
    if (docs.includes(req.body.url)) {
      // Return doc, as it exists in database already
      return ...
    }
    // Check if URL matches http|https, and return the hostname
    // If no match, return a URL that causes dns.lookup to fail.
    const matchedURL = ...
    dns.lookup(matchedURL, (err) => {
      // If error, return invalid address
      // If no error, create record to store in database,
      //  and return document.
    });
  });
});

So, I did not check for ftp...., because that is not a requirement of the user-stories. The requirement is not to ignore ftp..., the requirement is to include http|https. So, in summary, the logic is:

  • Ensure URL matches https?
  • If not, return error (per the user-story)
  • If no error, return the required object

Hope this helps

i will definitely consider your code for future needs…
at the moment I have an issue with the find() methods in DB and I opened a new post, may i ask you to have a look and help me there, too?
this is the link to the post

Thank you :slight_smile: