Benchmarking and performance concerns should be one of the last things you address while building your application, but it seems as though, in the PHP community especially, it’s often one of the first things novice developers think about.
Any PHP developer who’s been in the community for a while has heard preposterous claims like “use single quotes (‘) for strings instead of double quotes (“), because it’s faster”. That is, faster over the 100,000 or so iterations it took the tester to generate a number sufficiently large enough to justify the claim, with a particular version of PHP, in a particular development environment in which it was tested.
These articles are then followed up with other posts that only serve to perpetuate and solidify the original and invalid performance claims. Articles like these are like a plague to the PHP community, spreading around and steering novice developers in the wrong direction, concerning them with the wrong things. Chris Vincent recently released PHPBench.com – a website that benchmarks chunks of related PHP code against each other to compare the speeds. His goal was just a simple discovery of what code is actually fastest, and although it does yield a few interesting results, I have to wonder if it even matters. If the tests prove anything at all, it’s that the specific syntax you choose doesn’t matter. Why? Because it shows that newer PHP versions (he runs it on PHP 5.2+ where most of the previous articles were based on PHP 4.x) have been corrected and optimized to fix any performance discrepencies that there may have been in the past. In the “double (“) vs. single (‘) quotes” test, Chris concludes:
“In today’s versions of PHP it looks like this argument has been satisfied on both sides of the line. Lets all join together in harmony in this one!”
Similarly, the differences in the “echo vs. print” and “for vs. while” tests are so close in speed (and again remember, this is over 1,000 iterations) that you’re just better off using which ever one is a better fit the for job you’re doing.
Optimize for Performance AFTER You Code
The true way to optimize your code to run faster is to worry about optimization after you code. This is, of course, because the real bottlenecks in your code can only be identified after the code is already written. When put into simplified steps, the development and optimization process would then look something like this:
- Write code
- Run it for a while until you (or your users) start to experience noticeable slowdowns or you begin to reach the capacity of the hardware you’re running on
- Profile your code to identify where the real bottlenecks are
- Re-factor and make adjustments to your code to fix those bottlenecks
- Go back to step 2
Now that’s not to say that performance concerns should be completely discarded when writing your code – experienced developers will automatically make code and design decisions based on their experience that will have a beneficial impact on performance while they’re writing the code. This just means that it should never be your primary focus when developing your application, and that you should never be overly concerned about it until it’s actually a problem.
The Impact on Your Code
So the real difference here is that when you listen to benchmarks and have performance as a high priority immediately, you end up making bad design decisions and waste time worrying and doing things that may not even affect performance at all in the end. So instead, focus on your users, and do what you can to make their lives easier, no matter what you think ahead of time the performance impact may be. If it becomes a problem in the future, you can turn your focus to the specific problem and fix your code where the problems actually are.
Conclusion
Put aside your worries about performance and just build it. Would you rather have 10,000 users using a website with performance problems that you have to fix or 100 users using a website that is half as useful but runs 10% faster because you spent all your time focusing on performance “tips” when they may not even have an impact on performance at all?
Yes indeed. Functionality and usability should come first. Once everything is working properly and you have fixed all bugs, you can start thinking about how to make your code faster. But first build the application
I don’t totally agree. Of course, things like PHPBench.com are barely useful since they’re about micro-optimization. And as you pointed out, micro-optimization is bad. And it’s not only bad early.
For some projects, performance is never a concern. But for the others, actual profiling doesn’t have to wait until your project is almost completed.
Profiling reports can be generated automatically, just like for unit testing. These reports can reveal architectural problems that can become very hard to improve later in the project (for bigger projects anyway). They can give you a summary of the biggest problems. With that kind of tool you could just build your application and know *directly* what is going to be a problem performance-wise.
The problem is that this kind of tool doesn’t exist yet. It could be easily integrated in unit testing tools; after all, you could very well process an xdebug profiling output automatically (xdebug_get_profiler_filename) and display the biggest issues in your application, if you couple it with unit tests and/or (better) selenium tests to ensure every parts of the application are tested. You could also retrieve information about the efficiency of the queries performed (both time and EXPLAIN output), and add a data generator to the mix to make sure those queries still perform well with millions of rows.
If the report is automated and integrated with unit testing then you don’t have to profile yourself, you win a lot of time and can optimize early, and avoid architectural problems later on in your project.
Now if only someone could add this functionality to PHPUnit, early optimization would become a cure.
@Loïc Hoguin
I do agree with you that micro-optimization isn’t ONLY bad early, but I’m not sure about automatic profiling. It seems like even that little bit of performance profiling waving in your face every time you run tests would get you off track in your development.
I guess the main point of the post is to indicate that focusing on performance – micro-benchmarking or anything else – too early on will de-rail your development. You start thinking about shaving milliseconds here and there instead of focusing on getting your project done and moving forward.
Yes, I see what you mean. Focusing on performance is bad, unless you work on slashdot, youtube, twitter.. where (real, not micro) optimization can save you the cost of a few servers. Virtually all the time, focusing on performance is bad.
But it should not be forgotten either, and that’s what an automatic profiling would do. It would only pinpoint the bottlenecks of your application. With that report you could choose what to do before it’s too late. Someone experienced (like the project leader) could use it to detect a bad database design that would be hard to change when your project is feature-complete, for example. It’s worth it for that kind of scenarios. Everything that you can optimize later should wait. But sometimes you can’t optimize later.
That said, since there’s no tool to automate profiling yet, my opinion is completely arguable. And since there’s nothing yet, all developers should follow your advice.
You might be surprised to learn that software performance engineering does not constitute just benchmarking (and tuning).
Performance engineering that involves the creation of models via simulated or actual benchmarks (system execution) or trace executions (software execution) can be of enormous benefit as one learns about the dynamic software execution and not just the code in front of the screen and how one thinks it is called and executed.
There are a number of performance engineering related activities that should be conducted on any real-world application though with varying degrees depending on the levels of risk and cost implications.
eXtensible Software Performance Engineering
http://www.jinspired.com/solutions/xpe/index.html
Any activity that increases ones knowledge of how a software executes (frequency, resource usage and not just timing) is of benefit to the organization beyond the narrow and time limited world of a developer.
William
While obsession with performance is generally bad, I do believe that benchmarking should be a constant part of the development process, specially on applications with limited execution scope such as php-based web applications.
Early benchmarking will allow you to take decisions on the algorithms used and how well they will perform on production scenarios. Profilers are a great help but they won’t give you information regarding the actual usage of “external” resources(memory, file, database, uri-based resources)on the production system.
“Profilers are a great help but they won’t give you information regarding the actual usage of “external” resources(memory, file, database, uri-based resources)on the production system.”
Standard profilers yes but most next generation (especially ones designed for cloud computing) are contextual and multi-language and multi-resource solutions.
Metering the Cloud
http://www.jinspired.com/products/jxinsight/meteringthecloud.html
William
@William Louth
I think you missed the point of the post (did you even read it?). I’m not trying to explain what software performance engineering is, and no I’m not “surprised to learn” what it involves. All this article is trying to point out is that focusing on performance early and placing a high priority on it right away can cause you to lose focus on building your actual application, which is where most of your focus should be in the early start-up stage.
Please don’t try to use this blog as an advertisement for your products.