I have a Mongodb database with React based front-end and Node based back-end with approximately 100000 documents in the collection. I have already done indexing and I am already using Redis. I have implemented a search functionality to find all the relevant documents with ‘title’ property containing the search query and display them to the user. I am also using pagination displaying 20 results per page and there are various filters which a user can apply on the result.
Till now what I have done is when a user searches something, a text search on the database is performed. All the results of the search are then scored based on their relevance depending on the search query and they are sorted in the order of the scores and returned to the client.
One thing I could do is for every search retrieve all the data from the database and score them and sort them based on scores and return only 20 results to be displayed on the first page but then every time the page is changed or a filter is applied, I will have to do the complete thing again which will take a long time for something as simple as filtering.
I would like to know how do I go about it to get the data faster on the client-side while not affecting the speed of filtering and changing pages by a lot.
A suggestion of additional or a different tech stack is also welcome. Thank you.