How to design API that display and filter through millions of products.REST API Filters & Pagination

will this work for mongodb?

Hi teacher thanks❤

In backend for offset and limit pagination we can use db limit and offset parameter while creating a query to fetch that many rows and this quite inefficient in certain scenarios.

Keyset pagination which is quite efficient but can’t directly query a particular page. Either go forward or backward in rows with a particular limit.

Hi,your explanation is good.

I have issue with pagination. I using one azure api to get all items. In azure portal, 500 items are there but I am getting 200 items while querying api. I applied limit and offset as well but not working. Please help.

Thank u

Wow terrific to the point with clear and clean message.

Great. Wonderful explanation.......

😊

Nice explanation but only for a very primitive audience to understand. It does not contain any technical details on how it works or implemented . If you could add more details on server side services to map to your URI examples that would be helpful.

Thanks !!

Thank you for this informative video on designing an API that can display and filter through millions of products! As someone who's worked with APIs before, I can definitely appreciate the importance of filters and pagination for improving performance and user experience. One plugin that I highly recommend for filtering large datasets is Expertrec. Their API search engine allows for advanced filtering capabilities, including custom facets and search parameters, which can help users find the products they're looking for quickly and easily. It's a great investment for any business dealing with a large product catalog. Thanks again for sharing your expertise!

Clear and crisp explanation but I've one question. Do we need to hit db for every request, Is it a good idea ? If not then what are the other ways we fetch x amount to data in once(certain threshold) and serve in 20-20 chunks ?

What is there is no pagination.. the data is loaded based on the user scrolling the screen..

You did not discuss this scenario: if client request for 100 items and server max-limit is 20, how client will be able to request items 20 to 39 in the next request? How client will set its offset ?
I am assuming client logic should have offset=lastoffset+numberOfItemsReceived

You did a great job with this video, thank you

how does this compare to graphQL?

Nice. Any video on cursor vs offset based pagination

Mam have you created video on implementation?

hey i was asked a question where i told about offset for pagination, but interviewer told the data is added on api all the time, so when i call the next page , i might get the same data in the next offset as well coz new data was appended by time i clicked on next page, so how to handle such cases???

greatly explained

you're super underrated

Does api works to show products which are on low prices due to price error listed on amazon or flipkart?

Explaination is excellent please provide practical video if possible

Found 2 channels useful so far .. Ur channel and another.. kudos

Mam'm how could I understand my frontend before making backend server. If I created a registration page and login page are on different pages then it's important to make the same thing on the backed side. So, I will have to login and registration page (register, update, compare password pages) are on the different pages or same page in backed side.

Your explainations are the best, right to the point, well illustrated, real world examples. Thank you.

Didn't work.. getting complete result

amazing video thank you!

mam you explain awesome. Thanks

This explanation is so valuable thank you for that , also I would like to suggest if possible to do a video demo with a limited paging server response for example : API like Zoho, or deluge code based API's , how to use the explained methods on a study case like this

just one word Awesome.

Nic mam

Super :)

This is nice topic. Sometime I feel that pure REST approach is not suitable for business need. It mainly revolve around single entity but business application has complex need like to filter. Let's say for example Employee Belong to specific company and organization and there are multiple attribute associated with that and we have to pass multiple filter attribute. I feel that sometime pragmatic approach will make more sense and expose end-point like /search / filters etc will make more sense.

CDN can you share

by your presentation we easily got the concepts, thanks to your editor😅😅

Designing the APIs are fine, but how to implement such things? What is the right approach. Let's assume that I am building an ecommerce site and where from the database I want top 20 filtered items, how to do that in code? Thanks for the awesome content.

PS: All theory and no code, makes jhony a dull boy 😉

Wonderful explanation

please cover sorting also

Hi
I have a quick question.
So if i have 50 orders fetched from api and are available in dom. Now i want to implement search the order by id. In this case shall I do a internal filter in the dataset or shall I fetched filtered data from api?
What would be the best approach to do it?

This is really wonderful explanation. I know its not in scope of this series but out of curiosity would you mind throwing some light on how backing code for pagination can be implemented? Will it use some sort of sorted cache from where it returns results based on offset and max limit or is there a better way to get this done?

Some information on how the backend queries will manage limit and offset would have been helpful to tie in the solution end to end.

Love this video.
Can you please create a video dedicated to API Security? How do you secure an API?

One more way to reduce payload is sending it back as a stream, can you share use case for the same?
Could it be video/audio streaming?

can we send a json data in filter if filtering data is complex?
I currently POST and json body for it currently

actually i learned these terms pagination, limit, offset, filters while learning DBMS and i think they're more related to that that rest Api

Amazing explanation

It's clear ..but there is more link to fetch more data in some pagination APIs

Critical: what if you request, say first 10 (sorted by latest) orders and then 1 new order received. Now if you request next page then 1 order (1st of second page) will be repeated because thats 11th order now (not 10th anymore). How can we fix that.

Thanks for posting this one