paginate operator

[sergdort]

Name and description

Paginate operator

• provides pagination behavior

Motivation for inclusion

It was highly inspired by this tricky behavior in the RxExample 👍

The use case is pretty common (from my perspective), so I want to introduce operator which will provide this behavior

Proposed implementation

extension ObservableType {
   func paginate<O: ObservableType>(nextPageTrigger: O,
                 hasNextPage: E -> Bool,
                 nextPageFactory: E -> Observable<E>) -> Observable<E> {
      return flatMap { page -> Observable<E> in
         if !hasNextPage(page) {
            return Observable.just(page)
         }
         return [
            Observable.just(page),
            Observable.never().takeUntil(nextPageTrigger),
            nextPageFactory(page)
         ].concat()
      }
   }
}

Example of use

   func paginateItems(batch: Batch = Batch.initial,
                      endPoint: EndPoint,
                      nextBatchTrigger: Observable<Void>) -> Observable<Page<[T]>> {

      let params = paramsProvider.pagingListParamsForBatch(batch)
      return httpClient
         .request(.GET, endPoint,
            parameters: paramsProvider.defaultParams + params,
            encoding: .URL,
            headers: nil)
         .map(PagingParser<T>.parse)
         .paginate(nextBatchTrigger,
                   hasNextPage: { (page) -> Bool in
                     return page.batch.next().hasNextPage
         }) { [weak self] (page) -> Observable<Page<[T]>> in
            return self?.paginateItems(page.batch.next(),
                                       endPoint: endPoint,
                                       nextBatchTrigger: nextBatchTrigger) ?? Observable.empty()
      }
   }

P.S.

Still not sure about naming and whether it's could be applied for all scenarios

Wondering what you guys think about its inclusion?

[fpillet]

I'm not sure about this one. They may be as many pagination scenarios as there are applications using RxSwift -- deriving a generic operator may not be enough. While I think that pagination may be too specific in many cases to be generic, we could maybe explore ways to reverse the action by providing a static function like:

extension Observable {
  static public func paginate(requestFactory : (Int) -> Observable<E>, nextBatchTrigger : Observable<Void>)
      -> Observable<E>
 }

There is a lot of boilerplate in the operator you're proposing, and if we can't manage to remove it then we may be failing at offering a useful operator. The solution I'm proposing above (at this stage I don't even suggest an implementation - just exploring the idea from a user's perspective) only requires the user to provide a factory function that returns an observable which provides one page of data (we pass an index, it returns an observable -- when the observable completes, all the data has been received for this page).

The nextBatchTrigger is tricky as it could fire while a previous page request is ongoing. Not sure yet how we should handle this - I guess, flatMap all the data but we should discuss behavior before proceeding with an implementation.

[sergdort]

Yes, sure I agree that, there could be a lot of scenarios , and this was my concern :)

But, what I was thinking about that, that generic E means not final type

• For example some API provides response with current batch and data. So potential structure could be like this

struct Batch {
   let offset: Int
   let limit: Int
   let total: Int
   let count: Int
}
struct UsersPage {
    let batch: Batch
    let users: [User]
}

so in this case it would be Observable< UsersPage > and in the end if some implementation expects [User] , it always possible to apply map operator

• Some API provides next request url, like Twitter API

So in this case instead of batch could be nextUrl

• In case of GitHub Example it could be Observable<SearchRepositoryResponse>

So hasNextPage would be :

 { state in 
    switch state {
           case .Repositories(repositories: _, nextUrl: let nextUrl):
                  return nextUrl != nil
           default:
                  return false
    }
}

• In case some SQL query you can always structure it with Batch since you need limit and offset in order to make new query

Sure, it's not a full list of potential use cases, but what I'm pretty sure that you can always create some type, so you will be able on previous result determine whether it is possible to have next results or not.

What do you think guys, am I missing something? :)

[pgherveou]

Hey guys,
just found out about this issue, I recently created this pager pod https://github.com/pgherveou/rxpager
I changed the code to a plain static function using your suggestions @sergdort
hopefully it can be useful for someone else!

[pgherveou]

@sergdort I refactored code here https://github.com/pgherveou/rxpager to build something similar from what you suggested, I am happy to send you guys a PR or keep it as it is in a separate pod

[freak4pc]

I'm closing this for now due to inactivity - If you have any more comments on this subject, feel free to comment or ping me directly and I'll re-open it :) Thanks !