Trying to fetch response from Goodreads API

Oh, I should point out the new version expects a GET request, in case you missed it.

@JacksonBates

Yeah I set it to GET on Postman and every time I send for a response, I get the 500 error plus the json response:

{
    "success": false,
    "message": "Text data outside of root node.\nLine: 0\nColumn: 16\nChar: ."
}

The terminal doesn’t seem to be showing anything about the 500 error. It just keeps saying “Example app listening on port 3000!”

Do you have a GitHub repo for this code?

I don’t have a repo of the code on Github, no.

It’s pretty hard to debug without seeing the code.

I googled the specific error, but every result I found suggested a different cause - so all I can suggest without seeing the code is go through every search result for the error methodically until you find the one relevant to your use case.

It seems like an error that happens reasonably often with XML from what I can tell - but pinning down the cause without the code to verify it is hard.

I see. So I went ahead and created a repo with the code (https://github.com/Dusch4593/book-finder-app). The problem seems to be coming from the backend.js, inside the relay route.

@JacksonBates

I think I found the cause of the 500 error!

The .env file wasn’t in the same directory as the backend.js file. I tested it on Postman and it returned the JSON with "success": true when the environment file was in the same directory as the running backend file (they are both inside a file called src.

Would I also need to put the gitignore file inside the src folder in order for .env to be ignored or does it not matter?

1 Like

You can just add src/.env to the .gitignore if the file is in a different folder.

Okay so at this point, I’m trying to display the GET data to the front end but I can’t figure out how to get that data from backend.js to app.js. Is there a way to do that?

Keep in mind I have a completely separated front end and back end…So, my back end could be on Heroku, while my front end could be served from GH pages, or Netlify for example.

For this demo, I had my backend running locally on port 3000 and my front end running on 8080.

I had to make a change to the backend to allow CORS access from anywhere - you would change this behaviour depending on your needs.

With all that said, here is a very rough demo. It uses Hooks and Async, which you may not be comfortable, but it should be enough to demonstrate that you basically hit the new endpoint the way you would have hit the GoodReads original API:

The important stuff is in the Component.js file.

@JacksonBates

So I went and tried to implement the front end and back end (easier to say ‘fullstack’?) and I’m running into the CORS access problem, despite experimenting with making changes in the backend such as:

Here’s what the front end from 8080 looks like:

Where in the backend would changes need to be made to allow CORS access?

Did you see the CORS stuff I added to my backend, in app.js?

var cors = require("cors");
... 
// Allow CORS from any origin
app.use(cors());

(I omitted some code from the above example, but the full thing is in the backend repo I linked to above)

That did the trick!

Thank you so, so, so much @JacksonBates for your patience and kindness in helping me with this. Thanks to you, I now have a little bit of experience with the backend aspect of web development and I’m going to come back to this thread as I keep working on this project and other projects that involve working with APIs.

1 Like

Glad to help, and congratulations for sticking with it - that’s the hard part.

Just wanted to suggest, instead of creating a back-end server which would just do one task, but run full time which is expensive. A cheaper approach would be using function like AWS Lambda or Google Cloud Functions which are executed only when they are invoked and their price is calculated only on the time it took to execute them.

Great! Could you show an example in code of how that replaces an expressjs backend?

Sorry for the late reply, yes ill try to add an example in on weekends :slight_smile:

I’ll talk about the AWS lambda functions because all the other functions providers (Google, Salesforce) will employ a similar process.
I took a reference from your repository JacksonBate/example-goodreads-api-relay for app.js, the below code is just like writing a node module.

let responseHeaders = {
    "Access-Control-Allow-Origin": `http://${process.env.APP_DOMAIN}`,
    "Access-Control-Allow-Credentials": true,
};

exports.functionFoo = async (event, context) => {
    try {
        // This uses string interpolation to make our search query string
        // it pulls the posted query param and reformats it for goodreads
        const searchString = `q=${event.queryStringParameters.q}`;

        // It uses node-fetch to call the goodreads api, and reads the key from .env
        const response = await fetch(
            `https://www.goodreads.com/search/index.xml?key=${process.env.GOODREADS_API_KEY}&${searchString}`,
        );
        //more info here https://www.goodreads.com/api/index#search.books
        const xml = await response.text();

        // Goodreads API returns XML, so to use it easily on the front end, we can
        // convert that to JSON:
        const json = convert.xml2json(xml, { compact: true, spaces: 2 });

        // The API returns stuff we don't care about, so we may as well strip out
        // everything except the results:
        const results = JSON.parse(json).GoodreadsResponse.search.results;


        return {
            "statusCode": 200,
            "headers": responseHeaders,
            "body": results,
            "isBase64Encoded": false
        };
    } catch (err) {
        return {
            "statusCode": 500,
            "headers": responseHeaders,
            "body": err.message,
            "isBase64Encoded": false
        };
    }
}

A few points to note here are, the query params or body params passed while calling the lambda function URL are to be taken from argument event (similar to the request in express). The lambda functions should return the response in a specific format or else it fails. CORS needs to be enabled in your case as its a different domain application. For this, I would recommend using the serverless framework and for CORS refer https://serverless.com/blog/cors-api-gateway-survival-guide/ for deployment.

An example of serverless.yml

service: your-app-name
provider:
  name: aws
  runtime: nodejs10.x
  stage: dev
  environment:
    NODE_PATH: "./:/opt/node_modules"
functions:
  functionFoo:
    handler: handler.functionFoo 
    events:
     - http:
         path: foo
         method: get
         cors: true

The lambda functions are called using a url generated using api gateway in AWS lambda (https://docs.aws.amazon.com/apigateway/latest/developerguide/getting-started-with-lambda-integration.html). But if you use the serverless framework, everything would be easier. a bit of a learning curve but definately worth it.

Thanks for following up. I look forward to trying this out when I get a chance :slight_smile:

yes sure, let me know if you need any help.