Hi!
There is something that I couldn’t decide in any way. So I want to ask the freeCodeCamp community.
My team built an application. It contains user authorization and authentication. Also, it contains post, put and get routes. In the backend of the application, Express.js, MongoDB, and RabbitMQ is used. In the front end, React is used.
When a user sends a post request via the front end to the backend in order to add a new record to the list which is rendered current page, Express.js is handling this HTTP request, and, when the request is received, a RabbitMQ publisher is publishing a message to the message broker, and a consumer, which is running permanently and cares about the messages published by the in the message queue publisher, is saving this message to MongoDB.
Here is what I couldn’t decide in any way: does this scenario make sense to manage this kind of process or is it better to use just HTTP traffic instead of adding AMQP/RabbitMQ?
If you have a microservices architecture it’s a good idea to split concerns into different services and, in your case, to use RabbitMQ as an asynchronous communication channel.
Otherwise, if you don’t need a microservices architecture (because it adds complexity), you could simply handle it in the Express app. In this way, you could also make the saving synchronous within the create request, so that the React client could immediately update the list.
Hi @Giles . First of all, thank you for your concern.
We built the app using microservice architecture. So as you mentioned, RabbitMQ is a good choice for us. And when a consumer saves a message to the database, we use socket.io to listen to database changes and render the latest state of the database.
This scenario makes some complexity, yes. But as you predict, the application will have so many features and we want these features can be handled, deployed, coded separately.
The reason behind I asked this question is to make sure that whether we are in the right way or not, and your response is helpful for us. Thank you!
On the other hand, I think it is good to discuss and search for other perspectives if there will be any.
Do you have any suggestions to read about this kind of structure we built?
I agree it’s always a good idea to look for advice. I do it all the time.
Maybe you could look into GraphQL? I think it’s a better choice when building a complex set of APIs for front-end applications. It also supports subscriptions that you can use to notify database changes (it relies on WebSocket).
Did you consider Kafka instead of RabbitMQ for microservices communication?
Is your application deployed in some cloud provider? Maybe you could consider deploying it in a Kubernetes cluster.
We will deploy the application to a cloud provider after the development process ends.
Actually, I already implemented GraphQL to the app but this side is going slowly for now. Because I have some other questions for this matter. Let me ask then:
Do we need Socket.io if we use GraphQL?
Do we need mongoose if we use GraphQL?
We didn’t look for Kafka. Honestly, I don’t know the essential difference between Kafka and RabbitMQ. Can you please mention the difference in a nutshell?
Do we need Socket.io if we use GraphQL?
Do we need mongoose if we use GraphQL?
No, not directly. I assume you would use an external library to implement that, like Apollo. In that case, that library provides hooks and in those hooks, you will make the queries to your database. I don’t know about Socket.IO but I guess the Apollo server will handle it for you.
We didn’t look for Kafka. Honestly, I don’t know the essential difference between Kafka and RabbitMQ. Can you please mention the difference in a nutshell?
I never used Kafka but I see it’s very popular in microservices architectures, so I guess there are some advantages in using that.
We will deploy the application to a cloud provider after the development process ends.
If you’re not worried about being tied to your chosen cloud provider you could look into some provider’s specific services. For instance, on AWS you could use AppSync for implementing the GraphQL APIs or SNS for inter-service communication.
Thank you so much @Giles! Your suggestions were very helpful for us!
I was really confused and couldn’t take one more step because I was afraid to mix things up, but now I feel that some things become clear.
No, it’s a query language, it has nothing to do with web sockets. You can use websockets to push updates – in graphql, these would be subscriptions (something listening for changes, push rather than pull) – but you don’t need websockets at all to do that. You can poll, or you can use server sent events, or you can use the message broker you originally posted about.
Mongoose allows you to write schemas for your Mongo database, has nothing to do with graphql. It is there to provide structure to what is otherwise unstructured data.
Are you handling in the region of a trillion events a day? That’s the usecase for Kafka, that’s why it exists, LinkedIn had to figure out a way to deal with a firehose (so up to 10 million messages a second). A billion a day is still probably a reasonable usecase, a million a day maybe, but that really won’t need anything special most of the time.
What is the usecase overall here? Because splitting stuff into seperate complex services makes things very complicated. Unless you have a good reason to not do it (and I don’t know the situation, so I don’t know), an Express app backed by Mongo with a React frontend, all in one place, is fairly simple and will work just fine, splitting things out into services at an early stage is generally a very bad idea
The application will include calculations of people’s activities who are saved to the database, and these calculations are going to be very varied. It also will contain user authentication, user authorization, users’ profile pages, and user activities, besides an integrated chat application - but we will code chat application after a big while. And the tender spot is that it will be integrated with Machine Learning and AI processes in the future.
Actually, integration with Machine Learning and AI processes is the main reason why we are using microservice architecture. Thus, developers from different technologies and disciplines can do their jobs separately.
I tried to explain what the use case will be but if it is needed, I can explain further.
but I see it’s very popular in microservices architectures, so I guess there are some advantages in using that.