URL Shortener Microservice - Retrieving URL from Database and Shortening It?

What’s happening:

I need to know how to send the GET request to retrieve the URL from the database after the POST request to send it there. Also, should I do app.get in the callback for app.post?

And how do I get the shortened URL?

My code so far

require("dotenv").config();
const express = require("express");
const cors = require("cors");
const path = require("path");
const bodyParser = require("body-parser");
const app = express();
const mongoose = require("mongoose");
const dns = require("dns");

mongoose.connect(process.env.DB_URI, { useNewUrlParser: true, useUnifiedTopology: true });

const Schema = mongoose.Schema;
const urlSchema = new Schema({
  urlCode: String,
  longUrl: String,
  shortUrl: String
});

const Url = mongoose.model("url", urlSchema);

// Basic Configuration
const port = process.env.PORT || 3000;

app.use(cors());

app.use("/public", express.static(`${process.cwd()}/public`));

app.get("/", (req, res) => {
  res.sendFile(path.join(`${process.cwd()}`, "/views/index.html"));
});

// Your first API endpoint
app.get("/api/hello", (req, res) => {
  res.json({ greeting: "hello API" });
});

app.listen(port, () => {
  console.log(`Listening on port ${port}`);
});

// to parse POST request body
app.use(bodyParser.urlencoded({ extended: false }));
app.use(bodyParser.json());

let originalUrl;
app.post("/api/shorturl/new", (req, res) => {
  originalUrl = req.body.url;
  const url = new Url({
    original_url: originalUrl
  });

  dns.lookup(originalUrl, (err) => {
    if (!err) {
      const saveUrlInDatabase = (done) => {
        url.save((err, url) => {
          if (err) {
            return done(err);
          }
          console.log(`url ${url} saved to database`);
          return done(null, url);
        });
      };

      saveUrlInDatabase((err, data) => {
        if (err) {
          console.log(err);
        }
        if (!data) {
          console.log("Missing done() argument");
        }
      });
    }

    app.get((req, res) => {
      const retrieveUrl = (url, done) => {
        Url.find({ original_url: url }, (err, url) => {
          if (err) {
            done(err);
          }
          done(null, url);
        });
      };

      retrieveUrl(originalUrl, (err, data) => {
        if (err) {
          console.log(err);
        }
        if (!data) {
          console.log("Missing done() argument");
        }
      });
    });
  });
});

Am I passing done correctly? And when do I use res.json to send the JSON response?

My browser information:

User Agent is: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.182 Safari/537.36 Edg/88.0.705.81.

Challenge: URL Shortener Microservice

Link to the challenge:

Basic concept is to post the url to the db

On the server side (or client side I suppose), assign a new shortened url for it

Save both in the DB, and return the shortened value to the front end

On the GET request, lookup the Shortened url In the DB, send the original back to front end

Can’t remember if you can redirect from server off the top of my head, but otherwise something like this:

https://www.w3schools.com/howto/howto_js_redirect_webpage.asp

You didn’t say if calling app.get inside app.post's callback is okay and if I’m passing done correctly. Please tell me that stuff as well.

Also, am I using dns.lookup correctly?

Thanks for the info you did give me, though.

Hello there,

Not quite. dns.lookup accepts a domain as its first argument. However, you are passing the full URL to it.

Again, not quite. Remember, all you are doing when you write app.get/app.post is defining a route. It does not make sense to define one inside the other. In fact, I would imagine this would cause some very unexpected behaviour.

Hope this clarifies

How would the app.get callback know about the URL and Mongoose document I want to use, then?

What do I do to shorten the URL?

This is what the get request looks like: /api/shorturl/<short_url>
So, the URL wanted is in the short_url parameter.

This can be done in many ways. To keep mine simple, I just used the number of documents in my database. That is, every document in the db had a number related to the its position in the db.

I think you might be making this much more complicated than need be. I suggest you sit back from your keyboard for a few minutes just paying attention to the user stories, and simplifying the workflow.

Keep it up, and I am sure it will click :+1:

So I need a URL parameter? The POST request is to /api/shorturl/new and the GET request is to /api/shorturl/<short_url>? Or are they both supposed to be to the latter?

I’m thinking I’ll just use the first 2 or so digits of the _id. Or would that be a bad idea?

You do not need a URL parameter. You have a URL parameter. If you are confused by the syntax, then I think this lesson will help:

This would be a bad idea, because there could be duplicates.

Then I should probably assign a number to each document based on order of creation. I was already thinking of that from the beginning but I don’t know how to do it.

For the GET request, I just have to get the original URL from the database and make a redirect to it, right?

Then given this code:

app.post("/api/shorturl/new", (req, res) => {
  const originalUrl = req.body.url;
  const domain = new URL(originalUrl).hostname;
  const url = new Url({
    original_url: originalUrl,
    short_url: 
  });

  dns.lookup(domain, (err) => {
    if (err) {
      res.json({ error: err });
    }

    const saveUrlInDatabase = (urlDoc, done) => {
      urlDoc.save((err, url) => {
        if (err) {
          return done(err);
        }
        console.log(`url ${url} saved to database`);
        return done(null, url);
      });
    };

    saveUrlInDatabase(url, (err, data) => {
      if (err) {
        console.log(err);
      }
      if (!data) {
        console.log("Missing done() argument");
      }
    });
  });
});

app.get("/api/shorturl/:short_url", (req, res) => {
  const retrieveOriginalUrl = (url, done) => {
    Url.find({ original_url }, (err, url) => {
      if (err) {
        done(err);
      }
      done(null, url);
    });
  };

  retrieveOriginalUrl(req.params.short_url, (err, data) => {
    if (err) {
      console.log(err);
    }
    if (!data) {
      console.log("Missing done() argument");
    }
  });
});

how do I get the original URL from here on the GET request? I’m not clear on that. And aside from the short_url key’s value, which I don’t have yet, how is my POST code so far?

Also, where in the POST request part should I send the response with the original and short URLs in it? At the bottom of the dns.lookup callback?

Correct.

Remember, the user story is:

If you pass an invalid URL that doesn’t follow the valid http://www.example.com format, the JSON response will contain { error: 'invalid url' }

How do you know what err will hold?


You have the original URL in your database…you need to find it, when someone says “take me to short_url 4”, then you find what “short_url 4” represents in your database, and redirect.

Anywhere… Provided there are no errors in the request (and your logic), and you have the data to send, you can just res.json it. Logically, you have steps to follow:

  1. Accept URL in POST request
  2. Ensure URL is of the correct format (what dns.lookup is doing)
  3. Save URL to database with associated shortened version
  4. Respond with what you saved it as.

Remember, try not to overcomplicate things :slightly_smiling_face:

Hope this helps

How do I get the position of a document in the database? I’m sorry if this is a noob question.

To be honest. I am not sure. My understanding is that you cannot for Non-relational databases. Could very-well be wrong.

You should not need this, though.

I’m just trying to ask you how you did that:

I thought I should try this too but how do I do it? Do I make an array and use the array indices?

What I did was something like this (not actual code, but you get the just):

const short_url = Model.find({}).length; // number of documents in db
const newDoc = new Model({short_url, original_url}); // create new doc

newDoc.save(cb); // Save to db

Then, to get the original URL:

const givenShortURL = req.params.short_url; // From GET req
Model.findOne({short_url: givenShortURL}, cb); // Find and redirect in cb

What am I still doing wrong here?

// to parse POST request body
app.use(bodyParser.urlencoded({ extended: false }));
app.use(bodyParser.json());

app.post("/api/shorturl/new", (req, res) => {
  const originalUrl = req.body.url;
  const domain = new URL(originalUrl).hostname;
  const shortUrl = Url.find({ original_url: originalUrl }).length;
  const url = new Url({
    original_url: originalUrl,
    short_url: shortUrl
  });

  dns.lookup(domain, (err) => {
    if (err) {
      res.json({ error: "invalid url" });
    }

    const saveUrlInDatabase = (urlDoc, done) => {
      urlDoc.save((err, url) => {
        if (err) {
          return done(err);
        }
        console.log(`url ${url} saved to database`);
        return done(null, url);
      });
    };

    saveUrlInDatabase(url, (err, data) => {
      if (err) {
        console.log(err);
      }
      if (!data) {
        console.log("Missing done() argument");
      }
    });

    res.json({
      original_url: originalUrl,
      short_url: shortUrl
    });
  });
});

app.get("/api/shorturl/:short_url", (req, res) => {
  let doc;
  const retrieveOriginalUrl = (url, done) => {
    doc = Url.findOne({ short_url: req.params.short_url }, (err, url) => {
      if (err) {
        done(err);
      }
      done(null, url);
    });
  };

  retrieveOriginalUrl(req.params.short_url, (err, data) => {
    if (err) {
      console.log(err);
    }
    if (!data) {
      console.log("Missing done() argument");
    }
  });
  console.log(doc);
});

I still don’t seem to be getting the URL into the database. What’s logged by me doing console.log(url ${url} saved to database); is url { _id: 60454ad3bcdcdb05cc267529, __v: 0 } saved to database which shows neither the short URL nor the long one.

Any help is appreciated. Thanks.

Edit: The response I get in my browser for the GET request doesn’t have the short URL in it either. Just the original one.

There are a few issues here.

  1. Model.find does not return an array
  2. Model.find is asynchronous. (same goes for all db operations)

You need to be working with promises, when you do these operations.


What is the point of this:

You are getting there, but really take the time to work through this. Take a break, if you need one. Otherwise, you could end up introducing more bugs than you resolve.

retrieveOriginalUrl should get the long URL from the database. Did I do it wrong?

For the short URL, I tried this:

let shortUrl;
  Url.find({ original_url: originalUrl })
    .then(data => console.log(data))
    .then(res => {
      console.log(res);
    })
    .catch(err => console.log(err));

I get undefined for one of those console.log calls.

Let us walk through this.

  1. Url.find returns a promise of some data
  2. Once the promise resolves, the first .then gets it in its callback
  3. Now, once data => console.log(data) resolves, the return is sent to the next .then
  4. console.log() always returns undefined
  5. res => console.log(res) - res is defined as the return of the previous .then which happens to be undefined

I suggest you go through this section of the curriculum, for a refresher on promises, and arrow functions:

Hope this helps

I tried this as well for the code I showed earlier:

Url.find({ original_url: originalUrl })
    .then(data => data)
    .then(res => {
      for (const elem of res) {
        console.log(elem);
      }
      return res;
    })
    .catch(err => console.log(err));

This still doesn’t work.

I also need to know why the response returned from the POST request didn’t have a short url even when I didn’t have that commented out here:

const url = new Url({
    original_url: originalUrl//,
    // short_url: shortUrl
  });