Deploying Symfony2 apps with magephp and file system acls

Releases still seem to be the most frightening point in the chain of development events towards finishing a release. I recently had a discussion with a coworker about the best way to deploy a php app, specifically the rights management on the server. We are using magephp for our deployments, which does the same thing as capistrano back in the days, but is way cooler, because it's written in php and installable via composer.

Based on the discussion, here is my attempt so solve all problems and bring peace to the world:

Our deployment root should be e.g. /srv/www. In this directory, we have eventually a domain dir (which is the reversed DNS name, like com.xenji, but in all cases an application dir, which matches the subdomain it is hosted on (like blog). In this application dir we have the default mage release folder structure: current (a symlink), releases and our own addition "shared". We've borrowed this from capistrano, as we needed something to put the logs and shared config files into.

The final structure looks like this:

lrwxrwxrwx   deployer nobody current -> releases/20140305105721
drwxr-xr-x  deployer nobody releases
drwxr-xr-x   nobody   nobody shared

So far, great. We have a structure everybody can remember, all cool. But wait! Nobody? Deployer? Shared is only writable for the server process? Let's peek into the folders:

The releases folder looks quite normal:

drwxr-xr-x 9 deployer nobody 4.0K May  1 16:15 20140501141458
drwxr-xr-x 9 deployer nobody 4.0K May  1 16:30 20140501142942

The shared folder look a bit different in terms of rights.

$> ls -lah shared/app/
drwxr-xr-x  3 nobody nobody 4.0K May  1 10:03 .
drwxr-xr-x  3 nobody nobody 4.0K May  1 10:03 ..
drwxrwxr-x+ 2 nobody nobody 4.0K May  1 10:03 logs

The logs folder is linked into the release folder right before the commands for that release start (generate caches, etc). You might have noticed the plus sign on the right of the folders permission bits. This is the indicator for "There are additional ACLs". You can fetch them with:

$> getfacl logs
# file: logs
# owner: nobody
# group: nobody
user::rwx
user:deployer:rwx
group::r-x
mask::rwx
other::r-x

The group nobody does not have any rights on that folder, but only the deployer user has (in addition to the webserver) the permission to write the logs folder. The same thing is on the app/cache folder. This solves the conflict between the delpoy user and the webserver user.
The application folder looks like this: 

...
-rw-r--r--  1 deployer nobody  90K Apr  7 15:28 bootstrap.php.cache
drwxrwxr-x+ 3 deployer nobody 4.0K May  1 16:30 cache
-rw-r--r--  1 deployer nobody 1.8K Feb  7 18:08 check.php
drwxr-xr-x  3 deployer nobody 4.0K May  1 16:30 config
-rwxr-xr-x  1 deployer nobody  867 Feb  7 18:08 console
-rw-r--r--  1 deployer nobody   13 Feb  7 18:08 .htaccess
lrwxrwxrwx  1 deployer nobody   49 May  1 16:30 logs -> /srv/www/some/path/shared/app/logs
...

The original author of Magallanes seems to have stopped the active development on the project, so we've added plenty of features to our fork of Magallanes, check it out on github: https://github.com/freshcells/Magallanes. You will find the ACL task, a new deployment strategy via remote cache (similar to the one you might know from capistrano) and more.

A usecase for yield in PHP

With the arrival of PHP 5.5 some time ago came the new yield keyword as part of the language spec. I knew yield from other languages like ruby or python, but I never had a usecase for implementing it myself. One of my actual projects at freshcells now brought me to the point where I do not want to miss it again.
The task was to parse a 70 megabyte file of XML data and collect about 330k integer IDs from it. Each ID gets passed to another worker over a message queue. So far this is a good job for XMLReader and my first attempt was to do it in a classic "push everyhing to an array" way. The result was a runtime ~25 seconds including the file read, but with a memory footprint of about 180 megabyte.
As my primary intention was the distribution of the received IDs via RabbitMQ, so I did not need the whole array of IDs at once. This was the point where yield came into play. The change from the buffer version to the yield version was small, so it was worth a try.
The resulting method looks like this:

public function parse($xmlContent)
{
    $xmlReader = $this->xmlReader;
    $xmlReader->XML($xmlContent, $this->encoding, $this->options);
    while ($xmlReader->read() && $xmlReader->name !== 'data');

    while ($xmlReader->next('data')) {
        if ($xmlReader->nodeType === XML_ELEMENT_NODE) {
            yield (new WrapperObject($this->encoding, $this->options))->parse($xmlReader->readOuterXml());
        }
    }
}

The calling code is as easy as it could be:

foreach ($this->parser->parse($xmlListResult) as $id) {
    // do something
}

The result was amazing. 86 megabytes footprint, ~15 seconds runtime. Hope you find some useful ways to bring yield into your own application, too.

The triangle of performance?

I'm thinking a lot about the adoption of a common project management pattern into the world of web performance. The iron triangle, about which  Jeff Atwood already wrote back in 2006,  is something

we all know about from our jobs. Now my question is, if there are other areas of our IT world, where a triangle choice pattern can be found. One, rather unfinished, example I found in the perspective of performance is the relation between data liveness, the response time and the load of the server. The relation is not perfect, as the "Server load" - "Liveness" does not fully oppose the "Response time", but it might be a start for a discussion, if there are such patterns and where they reside. Do you have other examples? I'd like to hear them.

using facebook's React with require.js

I recently stumbled upon facebook's React library, which is a Javascript library for building reusable frontend components. Even if this lib is only at version 0.3.x it behaves very stable, it is fast and is fun to code. I'm a big fan of require.js, so I tried to use React within the require.js eco system. It was not as hard as expected and here are some examples and some thoughts about it.

This is a menu component I use for a twitter bootstrap based project. The eventbus is based on EventEmitter2.

The fact I like most about using require.js together with React is, that you can write a component in one require.js module definition and just export the visible component part. The second advantage is the naming. If you use JSX you can influence the tag names and do some kind of vendor prefixing for your required tags.

In the end, the construction of the menu looks like this:

update:
The way to get things working was quite easy. React is available via Bower, so just add it to your bower.json file:

Afterwards, add it to your require.js config. I decided to change my bower install path to /public/js/components, my application scripts reside within /public/js/app.

After settings things up like this, the previously mentioned examples work like a charm.

Book review - PHPStorm Starter (Packt Instant)

The nice people at Packt recently asked me to review on of their new books from the INSTANT series, a focussed line of books concentrating on one single topic. Although PHPStorm can become a big one, this book keeps things straight.

I am talking about: Instant PhpStorm Starter

The target group for this book are beginners and I mean real beginners. After the typical "how to download" and "how to install" and The author creates a short introduction of about five steps to bootstrap the reader and get her/him ready to rock. This intro is quite OK, as far as you have never used anything else than notepad.exe before. Most of the stuff is very common and feels a bit like a transcript of the manual.

The chapter "Top features you need to know about" is mostly about code templates, file templates and shorthand operations in the editor. Those hints are useful and even a power user might feel surprised to find the one or the other little trick that she/he did not know about.

The part about the "High-level programming operations for the PHP language" sounds better than it (sadly) really is. Accessing the documentation, instance diagrams and the autoformatter do not count as high level programming functions for me. One thing I use every day in PHPStorm is missing: The function-, file- and symbol search

The last big chapter is about using a VCS with PHPStorm. This is another chapter that I feel unsure about. I am a very intense console user when it comes to VCS, esp. git. This makes it hard for me to judge if this part of the book is helpful for beginners. I think it might be, esp. if they come from the eclipse world. From my point of view, the VCS chapter should have been replaced with a chapter about the plugin architecture of PHPStorm, a careful selection of helpful plugins and some more "navigating through your code" hints. The hint for the structure- and hierarchy view is totally missing.

When I try to imagine the point of view of someone who comes e.g. from the eclipse world, I must say, that it would have been of my very interest, that PHPStorm can handle eclipse project files (this is because IntelliJ Idea, the Java IDE from JetBrains, is a commercial competitor of the eclipse Java Development Tools) very well and that I could use both at the same time without forcing me to choose one over the other.

Conclusion:
If you never used an IDE before you will learn a lot of new things from this book. If you are an experienced IDE user of any flavor, this book will show you some low-level tricks in PHPStorm, but the, from my personal point of view, important things like code navigation are missing.. If you used any other JetBrains IDE product before (like RubyMine, PyCharm, WebIDE, etc), most of this book is useless for you.

MySQL analysis with AnalyzeMySQL

I recently joined the scalability team at trivago and my first job was to get into the actual MySQL schema to discover which parts can be improved regarding performance and space consumption.

trivago's database consists of about 230 tables, many of them having a history of more than four years, some of them up to 7 years. A few of them do have some historic flaws, like having a signed bigint as primary key (because nobody could imagine what space will be needed in a few years), others do have indexes that are not longer used, etc. There are many tasks to do until you get to know the database scheme very well.

In order to support my work I started to write a litte tool, which was meant to perform some basic tasks on the scheme. The first one was to see what col definitions are much bigger than the actual maximum value it keeps.

What came out of this approach was a small framework-ish script collection that parses the table structures and gives programatic access to it. The framework is designed to use a plugin structure to achieve a simple extensibility.

I'll try to write another post to explain the creation of such a plugin as soon as possible. In the meantime I would like to invite you to have a look at the (yet unfinished and unpolished) code and provide me some feedback.

You can find the code on github: https://github.com/xenji/AnalyzeMySQL

Visualizing trivago's search traffic on a heatmap

While evaluating a message queue for trivago's architecture I wrote a litte demo to present RabbitMQ in one of our internal meetings. The demo should display live searches from the trivago platform and display them on a map using a heatmap layer. The demo should show that the MQ is able to cope the amount of traffic that would hit the queues and this type of technology solves some of our current near realtime problems.

As I was not allowed to touch the live systems code for it, the data needed to come from somewhere else. The accesslog seemed a proper source of information as our scribe logger delivers it in near-realtime. We use two different parameters to determine the location the visitor wants to search for, one is an integer, the other one is a list of integers. A "nice to have" feature is to show the visitors location. The scripts show this feature already, but the screen recording just shows the destinations. Log parsing? AWK to the rescue!

To resolve the IDs to a geo coordinate we did some pre-caching using Redis and a simple ID => geo construct.

The AWK script writes it's result to the STDOUT where a simple node.js scripts gets the content, resolves the geo data from Redis and passes the result to the RabbitMQ exchange. The script already shows the capability to use the geo-lite module and resolve the senders location via geoip lookup.

After sending it to the exchange, the messages are collected from the queue by a simple node.js server daemon. This daemon does only a simple job and pushes the messages from the RabbitMQ via broadcast to all connected websocket clients.

The frontend is as simple as the rest of the stack. On the client-side we use a bit of jQuery, the wonderful gmap3 library and socket.io.

The only limitation with this small toy is the limitation that the browser gives us. I've tried to raise the number of simultaneous data points displayed at the same time, but my MacBook Air (late 2012) went hot on more than 5000 points at the same time. This is related to the interaction with the event-list that controls the heatmap layer.

The average message rate in the video was between 300 and 600 messages per second but my screen recorder is just capable of 10fps, sorry ;).

trivago search destinations on a heatmap from Mario Mueller on Vimeo.

Using tunnelbroker with Apple's Airport Extreme

Inspired by the IPv6 episode from ChaosRadioExpress I tried tunnelbroker.net as a my 6in4 provider, because Unity Media cannot get native IPv6 running for long-term customers (ridiculous: new customers get it without any problem). tunnelbroker enables you to build a static tunnel, which is quite problematic if you have a DSL connection or something similar that changes it's IPv4 frequently. I use my Raspberry-PI, running on ArchLinux ARM, together with this litte ruby script to keep my public IP updated.

X2EZWJNRVT4Q

Release process - from startup to professional

The release process is one of the most underestimated processes in a growing company. Most startups choose between two versions of release management: none or fully automatic. The latter one mostly fits ruby/rails driven startups due the availablity of capistrano or git based hosting companies. The "none" variant means, in our case, deployment via svn checkouts on the server(s). You could say this is release management too, but if you are honest to yourself ...

Startups tend to grow and so you will encounter some point in time when you need to scale the processes within the startup. The release process is one of them, as you get more servers, more developers and more features per development cycle (I tend not to say sprint or release cycle for getting the agile and classically managed in one boat). In addition to the management complexity the software complexity will grow with every feature you do. Phrases like rollback, dependency management and multi-environment come up in tech meetings and the "management by in-flight magazine" comes up with continous deployment to lower the time to market dramatically (sorry, no link or reference for the anti-pattern - the original article seems to be gone, but I guess you can imagine what it means? The CEO or Product Developer reads something in a magazin while traveling and enters the tech department to announce the immediate resource allocation on the new hot topic XYZ without any preperation or even sense). I tend to call this DDD (dream driven development) resulting in RDD (rage driven development) on the tech side.

When you come from a web agency (like me) and start designing a new process for such a company you might tend to rely on a classic release management process. Starting from the way how feature tasks are assigned up to the moment where you build and deploy the piece of software on the server cluster. I have been proven wrong since I stated the attempt to build such a process. The new goal should be to mix up the two worlds, aiming for maximum flexibility within a given level of process security. Cutting it into smaller pieces to take the changes step by step makes it easier for the developers and for the management.

I'll do another post about choosing a software stack to support the release management in a few days.

Indexing octopress article in elasticsearch

I was not satisfied with the simple search plugin that came with octopress (although it is suitable for many blogs), so I started looking for a simple search solution without any branding and with all the features I wanted. I came across Solr, but it was oversized for my concerns. I did not want to create a complex scheme and push the documents with brackground jobs or something similar. This is where elasticsearch came into play. The very dynamic nature fitted very well and the easy to use http interface was exactly what I needed.

I've hacked a simple indexer plugin and a simple javascript file to query the index and display the results in a badly styled overlay (I'm no CSS guy at all).

Here comes the plugin, which is a simple jekyll generator. I does not check for anything, but if you are able to blog with octopress you should be able to cope with the sources and customize them to fit your needs. You'll need to add those two config entries for your index url (which should be protected by http-basic auth) and your index name.

The javascript is also dead simple. You'll need a jsonp request if your elasticsearch server is hosted on another domain or port (which is very likely). The script is nothing crazy. Most of the code is rather bad, using jQuery selectors directly and not using any scope related stuff. The code might work better as a jQuery plugin or something more reusable. Anyway, it works for me as a first step. Feel free to enhance it and drop me a line over twitter so I can link to your solution.

bind function missing in iOS 5 JavaScript

While developing a new, jQuery based, frontend, we encountered some strange behavior while testing a release of the app on the iPad 2 running iOS 5.1.1. Between a bunch of commits the iPad's JavaScript just stopped to do anything. Not a single event was fired, nothing happend.

The debugging tools on the iPad are not that great and so was the debugging process. We found out, that the Konquerer Browser uses the same engine as iOS and the bug was reproducable in Konqueror, too. Git bisect came into play and we found the commit that killed the iPad. We changed the modenizr full build to a custom slim build. This kicked out the "Touch Event" part of the build and this was the reason for all the trouble. The "Touch Event" part defines the bind() function if it is missing in the JavaScript engine. We borrowed the function from modenizr and made the iPad work again.

Sidenote: iOS 6 beta 4, which has a new JavaScript Engine, works without any problems.

Hetzner vServer on ArchLinux (german)

This post is written in german language, as most people who are interested in it are customers of the german hosting provider "Hetzner".

Lange Suche

Ich bin erst seit Kurzem eine Fan von ArchLinux. Vorher war ich bei SuSE, Fedora, Debian, Ubuntu, Gentoo (Stage 1) und PC-BSD. Das letzte Ubuntu Release, 12.04, hat mich enttäuscht und dazu getrieben mir eine echte Alternative zu suchen. Ich war vorher viel im Gentoo und BSD Umfeld unterwegs und mochte den Ansatz der vollkommenen Anpassbarkeit sehr. Der Schritt zu ArchLinux war damit nicht so weit entfernt. Doch nun begann die Suche erst wirklich. Einen Hosting Provider, am liebsten mit deutschem Standort, und vertretbaren Preisen war nicht so schnell zu finden. AWS, Linode und Rackspace bieten ArchLinux direkt an, jedoch sind alle mit 19,99 USD für 512Mbyte RAM vServer mehr einfach viel zu teuer und wenig attraktiv.

Leider bieten viele deutsche Hosting Provider nur Virtuozzo-basierte Lösungen an, so z. B. Strato und Hosteurope. Die Shared-Kernel Architektur verbietet es dann irgendwas Anderes als die angebotenen Distributionen zu verwenden.

Hetzner to the rescue

Hetzner war der erste (und soweit ich weiss auch einzige) Anbieter mit KVM/QEMU Architektur. Zusätzlich bietet Hetzner seinen Kunden an, den Server im sog. Rescue System auszuliefern. Damit bekommt man ein schlankes Debian gebootet, von dem man seinen Server wunderbar bearbeiten kann.

Leider stimmen die Tutorials auf https://wiki.archlinux.de/title/Arch_Linux_auf_einem_Root-Server und http://www.uninformativ.de/?ndo=single&newsid=138 nicht mehr zu 100%, so dass ich meine Erfahrung hier noch mal schnell zusammen schreibe.

Der Arch Linux Wiki Artikel arbeitet mit einem Software Raid, was meiner Meinung nach für einen vServer wenig Sinn macht, und einer pacman basierten Installation. Hetzner verfügt jedoch über eine Web-KVM Konsole, die es uns erlaubt eine normale Installation durchzuführen. Die Installationsanweisung auf uninformativ geht eben diesen Weg.

Das uninformativ Tutorial kann man bis zur Installation von grub gut durcharbeiten. Leider ist der klassische grub unter dem aktuellen Rescue System von Hetzner nicht mehr verfügbar. Da die aktuelle netinstall Version von Arch keine alte Version von grub mehr supportet und die Version 0.9 nun nur noch im AUR verfügbar ist, sollte man hier besser auf syslinux setzen oder sich mit der manuellen Einrichung von grub2 anfreunden.

Der Rest läuft wie im Artikel von uninformativ beschrieben.

Stateful web applications pt.2

This is a followup of my first article about stateful web applications, which you can find here, you may want to read it first.

Push state - poll state

We are still talking about stateful web applications. In my previous article I've talked about a challenge - response method to keep client and server side in sync. This method had some drawbacks but was simple to achieve.

The push state - poll state method is more complex concerning the setup, but way simple when it comes to the development part. Instead of using a single, serial channel of communication, we break up things and use two separate channels for our needs. This simple change gives us the possibility to acquire more change state requests as we could to with the sequential version (meaning the challenge - response method).

The setup

You will need something that accepts a high number of concurrent request. I would not recommend an Apache httpd for that. I've tried node.js and it worked. (nginx might fit as well, depending on your software stack.) This server, let's call it "change-bucket", gets all the change state requests from all clients via PUT request. It responds with some 2xx state, maybe 204 or 201. This is just the info, that the put-request reached the server. As you might imagine, this might lead to an eventual consistency state. The change-bucket passed the change request to a shared queue, which could be a redis server or something like ApacheMQ, RabbitMQ, or similar. The queue server maintains one list per application (of which multiple per user can exists side by side). By the way: redis is awsome for such a job, because it supports list operations natively - even blocking ones - and produces less overhead than a real message queue. The last thing you need is something like memcache. You can use redis or a real memcache server - depending on your setup. The key is: it must be fast and at least persistent as long as it has a power supply.

Now, that we have all change requests of the user in a queue, we need to get them out and change the server-side application state and reflect those changes to the client. This is also a point in this setup, where you can decide which way to go.

We will now switch perspectives and take a view from the client side now. This helps us to identifiy the point where those two ways will come together again.

The client

The client need a poll mechanism. You know, something like setTimeout() with a propper tail recursion. I would not recommend using call(), as it is of lower priority in the most JavaScript Engines. The intervall depends on your technology stack, but I recommend to make it possible to set it to 250ms or less. This enables you to fake a real-time feeling on the customer's side. This poll request must go somewhere and it needs to receive something. What does it receive? The full application state, where does it go? Now we reached the other side of our two way split.

Path 1 - blocking list operations

This way is of moderate complexity and lacks of scalable performance. You will need only one additional piece for your architecture puzzle. You need an endpoint for the client to ask for the latest application state and you will need to make this endpoint fetch the stack of change requests from redis. This fetch operation must be blocking to prevent a concurrency issue. The concurrency issue comes from requesting the same endpoint twice. The first requests has not persisted the new application state yet and the second request loads the old state from the memcache server. The last one who persists the state wins. In this case we would get a not recoverable, inconsistent state. Another problem: you will need to solve the unsuccessful put-requests in the change-bucket with e.g. a retry loop in the client.

Path2 - everything non-blocking

In this setup you will need another, daemon like instance to solve the problem of concurrency. We need to decouple the polling process from persisting the new state. We could do this by a cronjob or a node.js server or something completly different. The important aspect is, that this cronjob just takes data from the queue and merges it into the new state. The polling mechanism may see the same state twice, but that is OK as far as we got rid of our concurrency issues.

Conclusion

We've build a circle, a round-trip for our application state. You might now have the feeling, that this is a shit load of overhead regarding the task of "just" persisting an application state, but that is OK. It is a rather complex setup with it's own issues and I must admit, that it works only in theory at the moment.

Future

One of the big advantages of this setup is the possibility to scale it. If the polling mechnism reached it's scale, you can easily exchange the polling with persistent socket connections. If you do so, you might not even need a single change-bucket, but you could use the same socket to do both parts of the communication over a single architectual piece.

Stateful web applications pt.1

Preface

In modern web application we mostly use two sided setups. One is a client - which will be a browser in this case - the other is some sort of server with some sort of application. The languages used on the server side do not matter at all, but the client side language will be JavaScript.

It is all about state

The application has a state. This might mean some sort of dataset which the client works on or something which is brought to the client by the server and is fully or partly persisted to enable the client further requests in the same data context. This topic is highly connected to caching those datasets and working on those cached results rather than working on live datasets.

HTTP is stateless, which means that we must rely on other techniques to synchronize the server and the client state. You might call for (web-)sockets now, but not all companies are able to cope with a change of their infrastructure to use sockets for the number of users that are concurrently browsing the app.

Challenge - Response

One way of getting this done is a challenge - response communication protocoll between the server and the client. The client always adds a token to identifiy itself on the server side. In addition, when the client sends a request it adds a ticket id to it. The server processes the response either in realtime or asynchronous. When doing it in realtime, the server responds directly to the client request. The async request must trigger polling mechanism where the server answers at some time with the requested ticket id. This type of communication works well, as long as your requests do not depend on each other. If you need state changes in a certain order, you need to add some kind of counter to it. The server needs to keep up with the counter and cannot process counter number 5 before counter number 4 was processed successfully.

This might lead to a deadlock situation, where counter 4 is lost in space for some reason and the server got stuck at holding back anything above counter 4.

Another drawback might be the danger to slip into some sort of interpretation of the client state. The server gives you a bunch of values to set some of the states that were asked for in the last response. You might be forced to start guessing the state of former components based on the partial response.

Too abstract? Ok, here is an example. You open some sort of info box and it's content must be loaded from a server via an ajax call. The response comes back and you display your text. This box is one of three in a left column UI element. The UI concept says, that not more than one box should be open at the same time. OK, what happens next is common sense at the moment, but from my point of view it is a great danger. The client decides, that, on opening another box, the actual box has to be closed. This state never reaches the server. The client interpreted, that he must close it, because another box want be gain the state "open".

Lets get one step further. We have another, dependent view element that also requires a server request. The answer to that request could possibly open one of the three boxes. As the server's answer enforces the client to do that, the actual state of the client (which is unknown to the server) gets lost. The client need to interpred the result again and concludes, that the active open box must be closed to achieve the requirements of the former server response.

Where is the drawback now, you might ask. Try to let the client send a server request based on data that cannot be interpreted correctly be the client:

• You have a entity to show via an id.


• You need to show meta data based on the id.


• You have a default setting on the root level of your application ("/")


• You do not want to show the meta-data to the default id, but another one

If you just deliver the id, the client is not able to decide if it's default or not. The amount of "please client: if default do that, if not do this" raises and will kill you/your app some day.

The next article will be about using a "Push / Poll State Sync" approach.

Building plugins for PHPStorm sucks - somehow

've started building my own plugins for Symfony2 on PHPStorm a while ago and yes, they make awesome IDEs.
If you ever wondered why there are so few plugins - compared to Eclipse - you might want to have a look at the available documentation for developing plugins in IntelliJ Idea based products. Sorry guys, but this is crap. There is no real API documentation anywhere but in the source code offered in form of the IntelliJ Community Edition. Yes, this might lead you to the assumption, that most of it is usable as a good starting point. Nope.

You might know the concept of extension points from Eclipse. It is an awesome concept and does also find it's place in Jetbrains' IDEs. The lack of documentation, which extension point is intended to be used for what kind of feature makes it nearly impossible to get started without asking rather "dumb" querstions in their forums.

I've tried to create a "Clickable Routes" plugin for supporting Symfony2 developers. It should enable you to click on a route string whithin a twig template's path() call. I've tried to use the same PsiReferenceContributor stuff which I successfully used before for my "Clickable Views" plugin. Unfortunatly twig is a template language and therefore handled differently in PHPStorm. The tokens of the template are not delivered to the contributor's extension point. I had to find this out myself as my post in Jetbrains' devnet was left unanswered by their developers.

This means I just can provide clickable routes in PHP files, but not in twig files. This sucks.

fish - my new favorite shell

Thanks to @hochchristoph, I found my new default shell for my ubuntu systems. It's "fish". You can find it here: http://ridiculousfish.com/shell/.

And for the git users using __git_ps1... Here is my ~/.config/fish/config.fish, based on the info I found here.

PHPStorm and Symfony2

I've been working with Symfony 2 a lot these days and I saw the wonderful work, which Robert did with his Eclipse plugins. I've started to write a bunch of plugins for PHPStorm, which - I hope so - will ease the way of developing Symfony 2 apps in PHPStorm.

You can find them here: http://xenji.github.com/phpstorm-symfony2-plugin/

What is so bad about ... PHP

or web technology? I hear so many people ranting these days. About PHP, about MongoDB and so many other things. I got bored about it, as anything said about it is not new. PHP has been the way it is for over a decade, so why do you fuck with it ... right now? It's the same thing with Java, but Java has prooven itself as an, so called, enterprise technology, which lifts it's status above criticism? PHP has been there, most likely before ruby got famous and before things like django came up to the status at which they reside now. All of them are good in solving problems we never had before they came up. We built websites without MVC, without Doctrine, Symfony2, Rails or ActiveRecord or Document Databases. Don't get me wrong, I like the NoSQL movement! But as it says: "not only" - means for me "in both directions". The thing is, that we've reached a scale, which needs fresh ideas. Rails was a good one, django and Symfony2, too. Esp. Symfony2 showed the PHP world how you can use PHP in an enterprise manner. All you people ranting about PHP, look at this framework, tell me if it does not solve the same kind of problem, which you solve with Rails or whatever tool you use! What you like or dislike is not important for solving a problem. Use the right tool for your problem. These words are old, maybe older than me. They still count. By the way: PHP moved to github. That is your chance to contribute and change the things you do not like - instead of just complaining about them.

GitLab - a private github-like solution

I'm always looking for new solutions in the private code and project hosting category. I recently stumbled upon GitLab, which seems to me like an awesome piece of software. The feature set feel a bit like github, which is very cool. The installation is a bit tricky, as you need to set up a full ruby on rails stack. The requirements talk about ruby 1.9.2 or greater, which demands a custom build on Debian 6 (which my private server run on) or the use of rvm. rvm made some trouble when I tried to use it so I switched to the custom build. The rest of the installation manual runs like a charm, really just copy and paste.

I took the nginx version using the unix socket. The only glitch in the manual is the name of the unix socket in the upstream {} block. This is set to /tmp/gitlab.socket, but it means /home/gitlab/gitlab/tmp/gitlab.socket.


One of the key features, from my point of view, is the permission system. It is not very fine grained, but educated enough to visualize e.g. customer-client relationships or help you to manage your university courses when you give your students some programming homework.


To get an impression of the UI and the features you can take a look at their homepage and their demo. The project's code is hosted on github. 

Xubuntu 12.04 on my Sony Vaio VPCS13V9E

This is my installation report about installing the new Xubuntu 12.04 beta 2 on my Sony Vaio VPCS13V9E/B. This is the fourth attempt to install a different linux distribution on my Vaio. Before Xubuntu 12.04 came Linux Mint 12, which was a total failure, and Fedora Core 16. The SELinux seemed contra-productive for a developer notebook to me. Ubuntu 10.10 and 11.04 were total failures, too. The EDID, needed for the graphics, were not present and had to be extracted on a windows machine. The text-based installation was the only way to cope with the problem.

Now comes (X)ubuntu 12.04, my personal highlight so far. I've used the default installation image, downloaded from the official website. I decided to use a classic CD, the attempt to use an usb key instead failed because it just did not boot from it.

The installation went fine and without problems, even the graphics were good to go. After the reboot I found a system with well recognized hardware. WLAN, Bluetooth, network, sound, touchpad with multitouch (two-finger-scroll) and nvidia graphics were all good. Even the HDMI port and the sound over HDMI worked properly from the start.

The problems left:

• UMTS - The Qualcom Gobi 2000 is always a problem
• Keyboard backlight
• Adaptive screen brightness
• Screen brightness controls

The Gobi 2000 problem is easy to solve, as long as you have the windows drivers at hand. The gobi card has to be initialized with the proper firmware for your carrier (mine is O2 germany, which I will use as example). If you have a windows dual boot, just copy the folder C:\Program Files (x86)\QUALCOMM to an usb key. Otherwise download the firmware from the Sony support website and install it via wine. In the QUALCOMM folder you'll find a folder named "Images", which contains  numbered subfolders (0-12) and a folder named UMTS. Now you have to find out which number is correct for your service provider (Mine is 6, I just google'd it and found it on the first result page). Create the folder /lib/firmware/gobi if it does not exist and copy the content of the UMTS folder and the content of the folder with you corresponding provider number to /lib/firmware/gobi.
The next step is to install the gobi-loader, which enables the system to load the firmware into your UMTS card. Use "apt-get install gobi-loader" to install it. Afterwards we need to make sure, that the kernel loads the modules on boot, which we need to get the dialup connection ready. Do a "sudo vim /etc/modules" and add two new lines "qcserial" and "usb_wwan" at the end of the file. Save it and leave vim. To test it without rebooting your notebook, put the hardware switch to it's off state and do a "sudo modprobe qcserial" and "sudo modprobe usb_wwan". Then turn it back on and type "dmesg" in your shell. You should see the QUALCOMM lines in the output. Go to your network settings. You should now be able to create a new mobile broadband connection according to your carrier settings. For O2 germany you need to discard the settings that are in the preset and enter "internet" as APN.
Please note that it will always take some time for the gobi loader to get the firmware into the card's memory. This means, that it will take 10-15 seconds for the connection to appear in the network manager.

The keyboard backlight is eady to solve, too. Just edit your /etc/rc.local and add this line somewhere before the "exit 0" line:
echo 1 | sudo tee /sys/devices/platform/sony-laptop/kbd_backlight
This will enable the backlight. If you want to turn it off for any reason just change the "echo 1" to "echo 0".


The screen brightness stuff is still without a real solution. I'v read articles on the web that enable the brightness controls via the fn-shortcuts, but they are made of a bunch of custom scripts, which I did not check yet. The adaptive brightness has no solution known to me. If you find one, please leave a comment.

Follow up (April, 8th 19:32):
I've found a similar article dealing with another VAIO notebook on an older Ubuntu version (9.04). Unfortunately the hint regarding nvclock does not work on newer nvidia cards - they are not supported yet.