Book review - PHPStorm Starter (Packt Instant)

The nice people at Packt recently asked me to review on of their new books from the INSTANT series, a focussed line of books concentrating on one single topic. Although PHPStorm can become a big one, this book keeps things straight.

I am talking about: Instant PhpStorm Starter

The target group for this book are beginners and I mean real beginners. After the typical "how to download" and "how to install" and The author creates a short introduction of about five steps to bootstrap the reader and get her/him ready to rock. This intro is quite OK, as far as you have never used anything else than notepad.exe before. Most of the stuff is very common and feels a bit like a transcript of the manual.

The chapter "Top features you need to know about" is mostly about code templates, file templates and shorthand operations in the editor. Those hints are useful and even a power user might feel surprised to find the one or the other little trick that she/he did not know about.

The part about the "High-level programming operations for the PHP language" sounds better than it (sadly) really is. Accessing the documentation, instance diagrams and the autoformatter do not count as high level programming functions for me. One thing I use every day in PHPStorm is missing: The function-, file- and symbol search

The last big chapter is about using a VCS with PHPStorm. This is another chapter that I feel unsure about. I am a very intense console user when it comes to VCS, esp. git. This makes it hard for me to judge if this part of the book is helpful for beginners. I think it might be, esp. if they come from the eclipse world. From my point of view, the VCS chapter should have been replaced with a chapter about the plugin architecture of PHPStorm, a careful selection of helpful plugins and some more "navigating through your code" hints. The hint for the structure- and hierarchy view is totally missing.

When I try to imagine the point of view of someone who comes e.g. from the eclipse world, I must say, that it would have been of my very interest, that PHPStorm can handle eclipse project files (this is because IntelliJ Idea, the Java IDE from JetBrains, is a commercial competitor of the eclipse Java Development Tools) very well and that I could use both at the same time without forcing me to choose one over the other.

Conclusion:
If you never used an IDE before you will learn a lot of new things from this book. If you are an experienced IDE user of any flavor, this book will show you some low-level tricks in PHPStorm, but the, from my personal point of view, important things like code navigation are missing.. If you used any other JetBrains IDE product before (like RubyMine, PyCharm, WebIDE, etc), most of this book is useless for you.

MySQL analysis with AnalyzeMySQL

I recently joined the scalability team at trivago and my first job was to get into the actual MySQL schema to discover which parts can be improved regarding performance and space consumption.

trivago's database consists of about 230 tables, many of them having a history of more than four years, some of them up to 7 years. A few of them do have some historic flaws, like having a signed bigint as primary key (because nobody could imagine what space will be needed in a few years), others do have indexes that are not longer used, etc. There are many tasks to do until you get to know the database scheme very well.

In order to support my work I started to write a litte tool, which was meant to perform some basic tasks on the scheme. The first one was to see what col definitions are much bigger than the actual maximum value it keeps.

What came out of this approach was a small framework-ish script collection that parses the table structures and gives programatic access to it. The framework is designed to use a plugin structure to achieve a simple extensibility.

I'll try to write another post to explain the creation of such a plugin as soon as possible. In the meantime I would like to invite you to have a look at the (yet unfinished and unpolished) code and provide me some feedback.

You can find the code on github: https://github.com/xenji/AnalyzeMySQL

Visualizing trivago's search traffic on a heatmap

While evaluating a message queue for trivago's architecture I wrote a litte demo to present RabbitMQ in one of our internal meetings. The demo should display live searches from the trivago platform and display them on a map using a heatmap layer. The demo should show that the MQ is able to cope the amount of traffic that would hit the queues and this type of technology solves some of our current near realtime problems.

As I was not allowed to touch the live systems code for it, the data needed to come from somewhere else. The accesslog seemed a proper source of information as our scribe logger delivers it in near-realtime. We use two different parameters to determine the location the visitor wants to search for, one is an integer, the other one is a list of integers. A "nice to have" feature is to show the visitors location. The scripts show this feature already, but the screen recording just shows the destinations. Log parsing? AWK to the rescue!

To resolve the IDs to a geo coordinate we did some pre-caching using Redis and a simple ID => geo construct.

The AWK script writes it's result to the STDOUT where a simple node.js scripts gets the content, resolves the geo data from Redis and passes the result to the RabbitMQ exchange. The script already shows the capability to use the geo-lite module and resolve the senders location via geoip lookup.

After sending it to the exchange, the messages are collected from the queue by a simple node.js server daemon. This daemon does only a simple job and pushes the messages from the RabbitMQ via broadcast to all connected websocket clients.

The frontend is as simple as the rest of the stack. On the client-side we use a bit of jQuery, the wonderful gmap3 library and socket.io.

The only limitation with this small toy is the limitation that the browser gives us. I've tried to raise the number of simultaneous data points displayed at the same time, but my MacBook Air (late 2012) went hot on more than 5000 points at the same time. This is related to the interaction with the event-list that controls the heatmap layer.

The average message rate in the video was between 300 and 600 messages per second but my screen recorder is just capable of 10fps, sorry ;).

trivago search destinations on a heatmap from Mario Mueller on Vimeo.

Using tunnelbroker with Apple's Airport Extreme

Inspired by the IPv6 episode from ChaosRadioExpress I tried tunnelbroker.net as a my 6in4 provider, because Unity Media cannot get native IPv6 running for long-term customers (ridiculous: new customers get it without any problem). tunnelbroker enables you to build a static tunnel, which is quite problematic if you have a DSL connection or something similar that changes it's IPv4 frequently. I use my Raspberry-PI, running on ArchLinux ARM, together with this litte ruby script to keep my public IP updated.

X2EZWJNRVT4Q

Release process - from startup to professional

The release process is one of the most underestimated processes in a growing company. Most startups choose between two versions of release management: none or fully automatic. The latter one mostly fits ruby/rails driven startups due the availablity of capistrano or git based hosting companies. The "none" variant means, in our case, deployment via svn checkouts on the server(s). You could say this is release management too, but if you are honest to yourself ...

Startups tend to grow and so you will encounter some point in time when you need to scale the processes within the startup. The release process is one of them, as you get more servers, more developers and more features per development cycle (I tend not to say sprint or release cycle for getting the agile and classically managed in one boat). In addition to the management complexity the software complexity will grow with every feature you do. Phrases like rollback, dependency management and multi-environment come up in tech meetings and the "management by in-flight magazine" comes up with continous deployment to lower the time to market dramatically (sorry, no link or reference for the anti-pattern - the original article seems to be gone, but I guess you can imagine what it means? The CEO or Product Developer reads something in a magazin while traveling and enters the tech department to announce the immediate resource allocation on the new hot topic XYZ without any preperation or even sense). I tend to call this DDD (dream driven development) resulting in RDD (rage driven development) on the tech side.

When you come from a web agency (like me) and start designing a new process for such a company you might tend to rely on a classic release management process. Starting from the way how feature tasks are assigned up to the moment where you build and deploy the piece of software on the server cluster. I have been proven wrong since I stated the attempt to build such a process. The new goal should be to mix up the two worlds, aiming for maximum flexibility within a given level of process security. Cutting it into smaller pieces to take the changes step by step makes it easier for the developers and for the management.

I'll do another post about choosing a software stack to support the release management in a few days.

Indexing octopress article in elasticsearch

I was not satisfied with the simple search plugin that came with octopress (although it is suitable for many blogs), so I started looking for a simple search solution without any branding and with all the features I wanted. I came across Solr, but it was oversized for my concerns. I did not want to create a complex scheme and push the documents with brackground jobs or something similar. This is where elasticsearch came into play. The very dynamic nature fitted very well and the easy to use http interface was exactly what I needed.

I've hacked a simple indexer plugin and a simple javascript file to query the index and display the results in a badly styled overlay (I'm no CSS guy at all).

Here comes the plugin, which is a simple jekyll generator. I does not check for anything, but if you are able to blog with octopress you should be able to cope with the sources and customize them to fit your needs. You'll need to add those two config entries for your index url (which should be protected by http-basic auth) and your index name.

The javascript is also dead simple. You'll need a jsonp request if your elasticsearch server is hosted on another domain or port (which is very likely). The script is nothing crazy. Most of the code is rather bad, using jQuery selectors directly and not using any scope related stuff. The code might work better as a jQuery plugin or something more reusable. Anyway, it works for me as a first step. Feel free to enhance it and drop me a line over twitter so I can link to your solution.

bind function missing in iOS 5 JavaScript

While developing a new, jQuery based, frontend, we encountered some strange behavior while testing a release of the app on the iPad 2 running iOS 5.1.1. Between a bunch of commits the iPad's JavaScript just stopped to do anything. Not a single event was fired, nothing happend.

The debugging tools on the iPad are not that great and so was the debugging process. We found out, that the Konquerer Browser uses the same engine as iOS and the bug was reproducable in Konqueror, too. Git bisect came into play and we found the commit that killed the iPad. We changed the modenizr full build to a custom slim build. This kicked out the "Touch Event" part of the build and this was the reason for all the trouble. The "Touch Event" part defines the bind() function if it is missing in the JavaScript engine. We borrowed the function from modenizr and made the iPad work again.

Sidenote: iOS 6 beta 4, which has a new JavaScript Engine, works without any problems.