Laravel Repository Pattern Demystified

Using Repository Pattern In Laravel 5 – Eloquent Relations And Eager Loading

Preface

Before we go to the main topic of the article, I’ll give you a short heads up for some design problems you may face. Recently one of my clients complained that some pages open very slowly. When I say very, I mean incredibly slow. So I’ve decided to debug that page and what I saw shocked me. Query section was showing that on that page was executed an staggering 16500+ queries !!

I found troubling part of the code responsible for that. It was 3 foreach loops, querying an attribute and it’s related attributes. It worked fine, until there was about 5.5k items in the database. Basically, here is what’s happening:

So if  $main_object = MainObject::all();  returns 5.5k results, first  foreach  will run 5.5k times,  second also 5.5k and third same as much. By using ORM developers very often make the mistake of writing very inefficient database queries, and ORM makes them even more difficult to spot. This problem is known as N+1 problem. And I guess previous developer was unaware of that. To avoid this problem we use eager loading.

What Is Eager Loading?

To put it simple, eager loading is a method of doing everything when asked. It’s also exactly opposite from lazy loading when we execute tasks when needed. Eager loading helps us to avoid common performance pitfalls, as you saw in my example above.  You’ll understand it better through an example so let’s imagine the following situation:

Stores Model

we have an Enhanced Entity Relationship model (EER) with three entities each related to another. We can read the EER as follows: Each member can have many stores, but one store belongs to only one member. Each store can have many products, but one product belongs to only one store.

Next step is to create an Eloquent models for these entities.

Member:

Store:

and Product:

Imagine that you are building an application that will allow your users to create their own store. Of course, like in every other store, users are able to create multiple products. Also, we want to create one page that will display all stores and top products from each store. Something like this:

Store List Mockup

You may end up with something like this in your controller:

in the View you want to display that data:

And the result is: N Plus One Problem

For this example, I’ve seeded the database with 5 members, 3 stores and 4 products. First query is to get all stores from the database and that is +1 part of N+1 problem. In this specific case, N represents a number of stores returned from the first query, since that many times we’ll execute  select *from  query on the products and members table. Since we have 3 stores, that means we’ll make 3 queries on the members table and 3 additional queries on the products table. In total we executed 3+3+1 queries.

Now imagine what would happen if we had 5000 or 10000 stores? You would have 10-20k queries executed every time when some user visits that page. And what if you have 10k or 100k visits in 24h? Nightmare! It’s clear now that this approach is a performance killer. No matter what DB you use, how powerful your server is it will always reach the tipping point where your powerful hardware will no longer be your ally. You can try to improve performance by caching these queries, using Redis for example. And it will work just fine, but just temporarily. That way you only delay the inevitable end, which will cost you a lot of money, time and probably you’ll lose some client(s) or your userbase will be reduced significantly.

Here eager loading come to rescue. Using eager loading in Laravel is pretty simple. The relations you want to be eager loaded you specify in the  with  method:

Now, instead executing 7 queries, by using eager loading we reduced number of queries to just 3:

N Plus One Problem

And it’ll be just three queries even if there is a 10k store entries. As you can see, wise use of eager loading may drastically increase the performance of your application. To actually have an improvement, we also need to have an index on the id fields in the members and products table. With a ton of records, executing  in( '1', '2', ... )  on non-indexed fields can take a while.

After this quick introduction into eager loading, let’s see how we can use the relations with repositories.

Extending Repository Class

I’ll show you one way how you can use relations in the concrete repository classes. Here is an example of end result:

As you can see, we have a  with  method where we can chain our model relations. This method will be similar to the Laravel’s Query Builder  with  method.

Now we need to attach each provided relation to the model:

And that’s it,  only thing left is to update our  all()  repository method (and any other you want) to use eager loading:

As I have already mentioned, you can add multiple relations within the  with()  method. Here is an example of StoresController :

In the View you can display data however you want, for testing purposes this is just enough:

As expected, we now have only three queries:

Eager Loading

Conclusion

By using eager loading you can improve your application performance. Sometimes as your application grows, even eager loading is not enough to keep up top performance. In the next tutorial I’ll show you how you can decorate repositories to cache the queries for better performance.

 

Follow me

Mirza Pasic

Full Stack Developer at OLX
Web Developer. Geek. Systematic. Dreamer
Follow me

Published by

Mirza Pasic

Web Developer. Geek. Systematic. Dreamer

  • codextends

    Wouldn’t be easier to set the $with attribute on the eloquent model ?

    • How do you mean? In the model you define the relation, and later you attach that relation to the query builder instance.

      • codextends

        Eloquent as a $with attribute which defines relations that eager load on each query.

      • Defining the relation method but then also an attribute containing the relationships that are to be eagerly loaded. I have written about my take on this here http://blog.dannyweeks.com/web-dev/repositories-in-laravel-sharing-my-base-repository

        As a side note what do you use to view the sql queries in the browser?

        • Yes, I already mentioned, it’s just one way of doing eager loading. I hoped people will come out with other ideas and best practices 😉 Thanks.

          I use debugbar:
          https://github.com/barryvdh/laravel-debugbar

          • codextends

            Sorry my 1st reply wasn’t really accurate as you can choose the relations to load in your repository on a case by case basis. Still, I would be very verbose in the repository, as its role is to provide an easy and understable interface to your storage layer. Something like getStoreWithProductsAndMembers(), which is quite long, but it’s very clear what it does. Remember your goal is to provide a repository class that can be easily swapped out. The more you assume about underlying implementation, the harder it will get to adapt to another storage. In that case, what happen if you have to adapt to an ORM that have no eagerloading ? Things could get really tough. As a rule of thumb, base repository class should be really thin, then extended to adapt the application need, not the other way around. Hope I have been a little more constructive than my first hasty comment 🙂

  • Dixit

    Nice article. I am getting the following error

    [2015-03-28 17:32:04] production.ERROR: exception ‘SymfonyComponentDebugExceptionFatalErrorException’ with message ‘Call to undefined method AppRepositoriesBrandRepository::newQuery()’ in /Users/appnox/blog/app/Repositories/Eloquent/Repository.php:63
    Stack trace:

    • Sorry I forgot to put that method in the code examples:

      public function newQuery() {
      $this->model = $this->model->newQuery();

      return $this;
      }

      • Dixit

        This works!! Thanks a lot 🙂

  • leonel

    Thanks dude, this is gold.

  • Stuart Hannig

    Was this included in your composer package Bosnadev/Repositories?

  • How do you do joins? Do you create a criteria for each join? what about if you need where clauses for that joins?

  • I got this error:
    Undefined property: AppRepositoriesXxxxxRepository::$with
    What’s happen?

    • I have some bad news… You need to change this:
      if(!is_null($this->with))

      for this

      if(is_set($this->with))

      Because when i’m not using ‘with’ in controller method, that if sentence throws a ugly error

      • OK. Thanks, I’ll check it and update it in the next version..

  • Great tutorial! Thanks.

  • Cherif BOUCHELAGHEM

    Why you name it Repository? do you mean DDD repository? if yes this has nothing to do with repository is just a decoration for active record

    • I haven’t named it “repository” – it is implementation of the
      repository pattern. If you look at this as a “decoration” for AR, then I
      strongly suggest you to learn to distinct those two patterns. They are
      very similar.

      Here is a good start:
      http://moleseyhill.com/blog/2009/07/13/active-record-verses-repository/

      • Cherif BOUCHELAGHEM

        Repository pattern is a collection of POPOs “Plain Old PHP Objects” AKA entities not collection of active record objects, as I know Eloquent is an implementation of Active Record, I read Evans book, active record is an anti pattern for domain modeling, Repository works fine with Data mapper instead of Active Record which violates SRP principle in first place, I think the article you advice also wrong you can understand more here http://www.infoq.com/books/domain-driven-design-quickly
        and here http://www.mehdi-khalili.com/orm-anti-patterns-part-1-active-record/

        • I guess if you look at that so strictly it should return plain objects. But, for me it’s good enough to abstract repeating logic and separate database logic from business logic.
          Who wants POPO, just implement RepositoryInterface way you want.
          I haven’t present here anything new. I just described what I have learned from other people and their experience. Almost every article I have read about repositories used Eloquent inside it. Some of the authors did indeed warn about some of the consequences by doing that.
          But like I said, if you look at that so strictly, then you shouldn’t use PHP at all. It’s bad by design. But as a platform it’s (probably) the best for the web.

          • Cherif BOUCHELAGHEM

            Ok, thanks for your time and the infos 🙂

          • No, thank you.. I like comments like yours, which push you to read and learn something new 🙂 Not just typical comment “great article” etc. 🙂

  • oaattia

    Great Article :0

  • Janzen Zarzoso

    public function all($columns = array(‘*’)) {
    $this->applyCriteria();
    $this->newQuery()->eagerLoadRelations();
    return $this->model->get($columns);
    }

    I don’t think you can apply the $this->newQuery() to the Repository class unless it extends the Eloquent model, no?

  • Luís Gonçalves

    This question is related to this article, but a bit off, nonetheless i would appreciate if you could help @mirzap:disqus and the rest of the ppl too 🙂

    Using your example, could you demonstrate how to create a Store, since it depends on the Member?
    Imagine you would be getting the data of the Store from a form, how would you validate that info (probably a form request), pass it to the controller and make use of whatever repos? How would you chain these steps?

    Another situation, making use of the Store. Imagine that it has a relation with StoreContacts, ie, a store can have multiple types of contacts (name of the person and email). And when you create a Store it is mandatory that you need to create at least one contact. How do you proceed?

    Could you give repo, controller, views, etc examples for the create method?

    Thanks

  • Rifqi Ruhyattamam

    Hi I really like how you solved applying the eager loading with repository.
    But I have a dumb question. As far as I know the definition of repository pattern is to abstract the data mapper layer from the database layer, which means for instance, in the future if you want to use MongoDB instead of Eloquent, you just simply change the Repository from Eloquent to MongoDB instead of going back and forth changing here and there.

    I don’t think all databases have “with” as what Eloquent does. Then in the future if I want to change to another storage option, which I’m not sure it does have “with”, how do I overcome with that?