Hello. I’m getting data from an API and writing it to a database, and I’m serving up a webpage that displays the data in various forms.
I’m using Node.js, Express, and Mongoose.
At the moment, I’ve got one back-end file, server.js, that does everything. I want to split the reading and writing, so that I can let the database continue to grow while I fiddle with the other functionality.
I’m sure there’s a standard approach for doing this, but I’m not sure what to Google! Do I just set up two separate projects that deal with the same database, with two server.js files to run independently? Can you set up the same schemas in each without causing problems?
Hello!
Which kind of data you are collecting from the api? I mean, if you want to just get the data and update them from time to time you can use a single server (not necessariliy one file though ) with a ‘section’ used to fetch new data and load them to a db (you can also create a dashboard to interact with those commands) and a ‘main area’ where you have a standard rest api, or whatever you decide to structure it ^^
Do I just set up two separate projects that deal with the same database, with two server.js files to run independently?
You can go this way too^^
Can you set up the same schemas in each without causing problems?
Thanks for replying. I’m using setInterval to call the API every minute - I’m grabbing streaming data for a radio station - and write new data to the database. And every hour I’m writing total listening figures to another collection in the database.
At the moment, because everything’s in one file, if I want to make some changes, I have to stop the regular function calls.
Ah, if that’s the case i don’t know how to help you: you’d need some sort of event dispatched when the original api updates (and if i understand correctly you’re not the owner of the api so you can’t do much on that side) ^^
To be honest i do not think setInterval is such a bad solution though: i mean, ideally the original api might do the work but if you’re working on their data collected in real-time (their ones) well, your options are limited and i can’t think to any better solution …
I was chatting to a friend about this last night. He thinks I just need to go down the two project route, with both back-end files accessing the database. The ‘reading’ one won’t need to have the schema setup code, only the ‘writing’ one will - which I guess is kind of obvious.
) with a ‘section’ used to fetch new data and load them to a db (you can also create a dashboard to interact with those commands) and a ‘main area’ where you have a standard rest api, or whatever you decide to structure it ^^