My Blog

Simple open source package for Laravel RESTful APIs

Hi everyone

In my last post about creating simple search and filtering for Laravel I said I was considering creating a package to encapsulate this functionality.

Well, I got this done a few weeks ago and have now released it onto Composer (Packagist) as kya/laravel-rest

You can find the repository on GitHub here, and the documentation is in the Wiki.

There is space on there for issues and things, of course. So if you have any feature requests do let me know, the current release is v0.1.

I know it’s going to be useful for me, as I’ve used it a couple of times already; I hope it proves to be of some use for you, too.

Speak to you all soon, JTC

Model Search and Filtering in Laravel

Hi guys, as I have written this functionality a few times at this point, and I have been asked how to do it a couple of times, too. That is Laravel searching and filtering, I am going to develop a simple composer package which I can use to allow users to search and filter Laravel (Eloquent) Models using PHP traits which can be used on Laravel controllers.

Assumptions

I am going to work on the assumption that you are comfortable with basics.

I shall also assume you’re fairly comfortable with the Illuminate HTTP Request object, a Controller, and Eloquent Model.

Concept

So the first thing we want to tackle, as a concept, is that the best way, in my experience, is to start with a query, and then modify it as required. This allows us to optionally do almost anything, but equally allows us to do nothing, and simply return an unfiltered set if we want to.

This point would be true outside of Eloquent too, even if working with vanilla SQL.

Starting Off

To start with you want something like this within your Controller.

public function index(Request $request)
{
// Of course you can change this if you don't want to select everything
$query = YourModel::select('*');
}

Now we have the $query object which we can work with as and where we want to.

Filtering Records

Excluding deleted/is deleted, as we’ll cover that later on.

Let’s say you now want to filter results on author_id just to keep things simple.

if(!empty($request->author)){
$query->where('author_id', '=', $request->author);
}

Of course take into account everything here, you may need to do sanitisation or validation or anything else in there.

Multiple Filters, Save Lines

If you have many fields on which you wish to do this I usually find something like this is better

$fields = ['author_id', 'category_id'];
foreach($fields as $field){
if(!empty($request->$field)){
$query->where($field, '=', $request->$field);
}
}

That example assumes the exposure of your field names, you could always use an associative array so you could do “author” maps to “author_id” and so on, if you were so inclined.

Fluffy Search

Now you might want to do a fluffy search, taking the assumption of posts with content and an author name, and a category name (stored within the model) you could do something like this

if(!empty($request->search)){
$searchFields = ['title','content','author_name','category_name'];
$query->where(function($query) use($request, $searchFields){
$searchWildcard = '%' . $request->search . '%';
foreach($searchFields as $field){
$query->orWhere($field, 'LIKE', $searchWildcard);
}
})
}

Handling Trash

You probably want something which allows you to see soft deleted models, if you’re using them. I do something like this

if($request->trash == true){
// Retrieve soft deleted models only, you can only do this if you're using soft deletion on this model
$query->onlyTrashed();
}

Ordering and Limiting

Now to handle ordering and pagination – which Eloquent beautifully does for us, all we need to do is let the user modify it if they want to.

// Set the query ordering
$query->order($request->orderBy ?? 'updated_at', $request->order ?? 'DESC');

$perPage = $request->per_page ?? 25;

// It is a good idea to make sure we cap this
if($perPage > $maximumPerPage){
$perPage = $maximumPerPage;
}

// Final part - get the results
$results = $query->paginate($perPage);

// You will now need to do something with your results, and return some kind of Response - this could be a JSON response or adding the results to the data returned to the view, depending on your context

There’s an assumption here that our default order is updated at, last updated first, and that if not specified, we want to return 25 pages.

Of course you can store this information wherever you want.

In Summary, and Further Reading

Rather than creating a really complicated set of rules to handle all the things you may need from your API calls, start with a query, and then optionally modify that query.

Further Reading

An introduction to Checksums

Hi everyone

This is a quick introduction into checksums, and practically how to use them, at the request of someone through the Ask Johno section of the blog.

I know what a checksum is, but I’m not sure how to implement or use one (in PHP) to achieve what I need.

Anonymous Asker

They go on to explain what they are trying to achieve, which is essentially to verify a file hasn’t changed (in this case it’s the HTTP document sent by an API) before doing something.

Firstly, a checksum is a very small data snippet (datum) which represents something larger.

In my experience there are 2 main reasons to use one, these are;

  1. To verify a change in something larger
  2. To verify that a piece of data matches from that which was sent by its originator (checksum on an API payload)

I will very briefly cover both of these with PHP examples.

Note; I am using MD5 for simplicity, but depending on your requirements this likely won’t be the best option for your use case.

Edit: Please see Further Reading at the bottom of this article for more information about hashing

Using a Checksum to Detect Changes

Now I don’t know what we’ve got, but whatever it is; we are going to need a string representation, 2 main ways of getting this:

$stringRepresentation = serialize($myThingToCheck);

or

$stringRepresentation = json_encode($myThingToCheck);

Personally, I would advise PHP’s serialize because that will work with, and instantiate objects. However, the choice is yours depending on your use case, JSON is smaller than a PHP serialization.

Now that we have a string, we need to make it small and easy to check. Something like this will work fine;

$checksum = md5($stringRepresentation);

Now to check for changes, we just need the last checksum that we stored that something happened on.

if($checksum != $oldChecksum){
echo 'Something has changed';
}

So that covers how to use a checksum to detect changes in pieces of data, which is useful if you’re having to poll for changes.

Using a Checksum to Verify Validity

This is something I’ve noticed a couple of times, particularly when working in the financial industry, and around certain payment gateways.

It usually looks something like this; but it does change per API integration so be aware of that and follow their own documentation.

$secret = 'my_secret_api_key';

$payload = [
'foo' => 'Bar',
'another' => 'Thing',
'datetime' => '2018-12-25 00:00:00'
];

$jsonPayload = json_encode($payload);

$checksum = md5($jsonPayload . $secret);

$payload['checksum'] = $checksum;

// Do the rest of your stuff here, including sending the payload etc.

One advantage of this approach is that you never expose the API key in plain text.

If you were to want to verify on the API so that you’re the provider, rather than integration, you would simply do these steps in the opposite order, so it would look something like this

// Assuming you've done everything you need, and now have the $payload array/object back

$secret = 'the_secret_youre_expecting';

$checksumProvided = $payload['checksum'];
unset($payload['checksum'];
$checksum = md5(json_encode($payload) . $secret);

if($checksum != $checksumProvided){
// If the checksums don't match, in theory the secret key provided was incorrect
}

I think that about covers this topic, as a brief introduction to checksums, and how they’re often used in PHP.

Further Reading

PHP: hash() for more information about the best ways to hash data

I have deliberately not gotten into the discussion over hashing algorithms, as it really is out of the scope of this article, and indeed a whole book could be written on the topic alone.

Thanks to u/artemix-org and u/BradChesney79 on Reddit for suggesting this edit.

What is a memory leak? A quick analogy

This is something that came up in conversation, some friends and I were discussing deploying code, that runs in the background, to production environments.

One of the things I raised was what can happen with daemon processes, should you have a very small inefficiency, given enough time to run (usually by the time it gets to production) it can, and will, destroy live servers.

I then realised that, at this point in the conversation, a description of what a memory leak is, had become an appropriate thing to explain.

Anybody who knows me, knows I love an analogy; so this is the analogy I gave, to give a really simple explanation as to what a memory leak is:

Every morning, you go to a fast food drive through.

You order a meal, eat it, and throw the paper bag with some leftovers into the passenger footwell.

At the end of the day, you arrive at home. You pick up the bag of rubbish, and put it into your bin. Without realising it, you drop a single french fry in the car.

In development and testing you run this same process 50 times, dropping a single french fry each time. The fries are not visible, they’re under the passenger seat, or with the momentum of the car have ended up in the back.

When you go live, the process runs more frequently, and instead of a single meal you’re buying 10 at a time.

Very quickly, those single french fries culminate in an unusable car, because you can’t fit in a Honda Civic if it has 1,000,000 festering french fries inside.

Matt “Johno the Coder” Johnson, on a cold winter morning

So there it is, a quick explanation of what a memory leak is, in an easy to understand analogy.

I’ve been asked to do walk throughs on practical implementations on daemons and a few other topics, so I am going to write those up soon.

Practically, what might it look like?

Imagine your daemon script looks something like this…

// Store the jobs that have been processed
$jobsProcessed = [];

// This is a daemon script, it needs to run, forever
while(true){

// This is just for demonstration purposes!
$job = getNewJob();

// Do whatever you need to, to handle the job

// Let's store the job we've processed
$jobsProcessed[] = $job;

}

This all looks fairly innocent right. In testing there are probably, at tops, a few thousand test jobs. In production, when this is running forever, that very small array, can become very big. That’s the bit that could cause a server to topple.

For reference, if you do need to keep this information, store it somewhere, anywhere, else. A log file is usually a good shout (as long as you’re periodically cleaning out your log files), perhaps a database (I’d recommend a MyISAM table for this, as you’re dumping a whole load of plain text data). If you keep this information in a variable in your script, it’ll hold in memory, which is exactly where you don’t want it.

So there it is, a quick and easy analogy, with an (overly) simplified example of what it might look like.

Beware, the Anti Pattern!

In this article I am going to cover the application of patterns within your… application. In short, I am going to show you how to use design patterns in a logical manner.

Patterns are always sometimes awesome!

The first thing here is that all design patterns have a purpose, every design pattern has its place. Whilst I’m talking about design patterns the same can be said of development methodologies, database designs, and really any other kind of concept.

The proper application of design patterns can take a frustrating piece of software, and make it easy to maintain, or hyper-secure, or crazy performant.

The bad application of a design pattern will do the exact opposite.

But, why would you ever not use a pattern?

Well, when it’s not the right time to use it. Every pattern has its place, but not every pattern should be used everywhere, it’s a bad idea.

Don’t take my word for it, let’s look at some practical examples and experiences that really demonstrate what I am saying here.

I’m going to be deliberately controversial here, and I’m going to pick stuff that all developers seem to love and then prove where it will ruin your application.

I’ve heard of Patterns, what is an Anti Pattern?

An anti-pattern is a pattern. Subjectively an anti-pattern is when you take a pattern and either apply it in the wrong circumstances, or implement it badly.

The consequence of the application of an, otherwise great, pattern, causes adverse impacts (usually for maintainability, or for performance etc). Now, it is an anti-pattern.

Model View Controller (MVC)

Well all love MVC, right? I mean, what’s not to love? CodeIgniter, CakePHP, Laravel, Angular, and Joomla all follow this pattern. It is arguably one of the most used design patterns of recent times.

For good reason, it’s awesome! Because it’s awesome, developers are pretty darn impassioned about using it everywhere, and they are right, in the vast majority of cases.

So when, I hear you ask, would this not be a good idea? When could it be an anti-pattern?

What about if I am writing a daemon, which is going to continuously monitor the usage of the mounted hard drives on a server and, in certain scenarios, send an email to a system administrator?

Don’t need models – we’re not handling any data. We don’t have anything, not even a CLI output, so no need for views. Realistically there’s not a controller it’s a standalone script. MVC would be a bit overkill, in this instance, wouldn’t it?

That’s a bit of a drastic example though, let’s look at something more subtle.

Single Responsibility Principle

Ah, right. Let’s get controversial then, shall we?

A class should only have one reason to change

Single Responsibility Principle, SOLID Principles

Everyone loves this, and likes to really preach about it. It is crazy controversial, widely adopted, and I personally think that it’s a good idea.

The Single Responsibility Principle is like the best song in the world, that you hear 1,000 times per day on every music channel and radio station. It is best practice, yes. But sometimes, it’s okay to break it!

Oh Gosh, quick, get the smelling salts and a wet flannel, they’ve feinted!

The whole point of the SOLID principles are to make well structured, easy to maintain code.

So, time for a real-life example. I have a class, it is an Eloquent Model. It is responsible for some mission-critical, core functionality. This class has a method within it. This method is 150 lines long (probably 50 lines of code, once you take out empty lines and code comments).

This method, in of itself, could (and maybe even should) be abstracted into it’s own class. For some context it is a static method, responsible for fetching records, based on a set of arguments.

Every part of the Single Responsibility Principle dictates this method should be abstracted to its own class, perhaps even a set of classes.

Internally, I have had this code reviewed, and to quote the developers who have checked (and indeed worked on) it, it is “exceptionally easy to follow” and “super easy to add and change the functionality”.

It is, essentially, a set of if statements. Based on the outcome of those if statements, the Eloquent Query is modified, then returning either limited, or paginated, results.

So, I hear you scream, “why won’t you abstract it?!” – well I could. But following conversations with the development team, the code would actually be harder to follow if I were to abstract and refactor it.

In this instance, the code can be easily found, easily changed, and is incredibly stable.

Following Single Responsibility to the letter, this time, would be an anti-pattern. Rather than making life easier, it would make life more difficult. Abstraction can be bad! If it has no performance, functional, or security perks. It makes the code more difficult to maintain and follow; then abstracting it would not make any sense. Except to follow a pattern for the sake of following a pattern, at this point, it becomes an anti-pattern.

Dependency Injection

Oh God. I’ve been here before. It is bad. I remember the flames, vividly. As the Reddit Hellfire engulfed my computer. Just kidding, but this one does really evoke emotional reactions.

Just quickly and very simply, dependency injection is parsing an object (dependency) into another object on which the latter depends. Thus, injecting the dependency. It usually looks (forgetting containers and autowiring) something like this.

class Calculator{

public function __construct(
AdditionServiceProvider $addition,
SubtractionServiceProvider $subtraction
){
$this->additionService = $addition;
$this->subtractionService = $subtraction;
}
}

The point here is that I can control the services that the Calculator is using. So if I were, at a later date, wanting to swap out my AdditionServiceProvider for AcmeIncAdditionServiceProvider (assuming they implement the same interface, or extend the same base class) then I could, and the rest of the class would work as expected.

However, I have, several times, seen things like this.

class PaymentsIncorporatedWrapper{

public function __construct(
PaymentsInc\Payer $payer,
PaymentsInc\Refunder $refunder
){
$this->payer = $payer;
$this->refunder = $refunder;
}

}

Right, this makes sense, doesn’t it? Wire up the payer and the refunder, then drop them into your Wrapper. As I said above, take the dependency injection container side of it; whenever I want to do something with the PaymentsIncorporatedWrapper I have to do something like

$payer = new PaymentsInc\Payer($apiKey, $somethingElse);
$refunder = new PaymentsInc\Refunder($apiKey, $somethingElse);
$wrapper = new PaymentsIncorporatedWrapper($payer, $refunder);

That’s a kind of annoying amount of code to write, to instantiate a class. “But the container does that for you!” I hear you scream. Yes, yes it will.

But why? You don’t know, at the time of instantiation, that I need the refunder. I might just be querying a payment. Why do I need the refunder? I don’t. Ah, maybe this is an anti-pattern.

Also, this set of classes is specific to the PaymentsInc integration. So it’s not like I’m going to swap in another payment provider (otherwise this would all make sense).

In this instance, when I couldn’t possibly want to swap anything else in/out, perhaps this would make more sense?

class PaymentsIncFactory{

public static function getPayer() : PaymentsInc\Payer
{
$factory = new static();
return $factory->getPayer();
}

private function getApiKey() : string
{ ... }

private function getSomethingElse() : string
{ ... }

public function getPayer() : PaymentsInc\Payer
{ ... }

...

}

class PaymentsIncoporatedWrapper{

public function takePayment(float $amount)
{
$payer = PaymentsIncFactory::getPayer();
$payer->takePayment($amount);
}

}

Anybody who is passionate about Dependency Injection will argue this is wrong, and they will probably cite Unit Testing as the reason. But, to my knowledge, unit testing isn’t justification for using Dependency Injection.

In fact, I have implemented both Unit Testing, and Test Driven Development, without unnecessarily using Dependency Injection. Of course, Dependency Injection was used, just only where it was truly necessary.

And the point of those tales was….

Just because a pattern is the best thing since sliced bread, doesn’t mean you should apply it liberally, everywhere, without thinking about it.

In the above I’ve taken three of the most beloved patterns we possess, and given you three good places where perhaps those pattern were best not applied.

So think about the patterns you’re using, never blindly use it because someone on [insert social website or influencer here] said is is amazing.

The key, as with all things, is to genuinely understand the pattern, its application, its benefits, and its constraints. And then think, and make a decision, about whether it makes sense to apply it in your use case.

Further Reading / Sources

  1. Model View Controller (MVC) – Wikipedia
  2. Single Responsibility Principle – Wikipedia
  3. Dependency Injection – Wikipedia