Welcome!

Apache Authors: Carmen Gonzalez, Liz McMillan, Elizabeth White, Pat Romanski, Christopher Harrold

Related Topics: Agile Computing, Linux Containers, Open Source Cloud, Eclipse, Machine Learning , Apache

Agile Computing: Article

TraceView’s Oboe Symfony No. 1

The adoption of Symfony has already let it pull in some beautiful pieces of architecture

Dvořák’s Symphony No. 9 requires two oboes. Instrumenting Symfony only required one liboboe!

Here at AppNeta we’re still recovering from an immensely productive DrupalCon Portland 2013. Our partnership with Acquia and our continuing support for the contrib TraceView Drupal module made sponsoring the event a natural decision. We’re glad we made the long flight out West, as there was a lot of interest in TraceView. In fact, on the plane ride back to Boston I brought back not only three Voodoo Dozens but a stack of business cards two inches high. (I’ll leave it to you to guess which one ran out first!)

But for as much fun as I had exploring Portland’s unique culture, my favorite part of the con took place well before it. I spent a lot of time coding up improvements for TraceView’s Drupal support based on customer feedback. My personal goal was to submit enough patches to the TraceView Drupal module to warrant a version bump before DrupalCon, and thanks to the hard work of module maintainer Peter Drake, we were able to get there with room to spare. If you’re running Drupal 7, check out the changelog, and then go install it on your site!

The TraceView Drupal module now supports full-stack application tracing using drupal_http_request, including via the Apache Solr module.

I was able to code up more than just improvements in the months leading up to DrupalCon, though. If you stopped by our booth, you’ll have heard all about this, but we have a second important release to announce: we have a beta Symfony bundle and a (very!) preliminary port of our module to Drupal 8! While I had initially planned to just work on Drupal 6 and 7 improvements for TraceView, the announcement that Drupal 8′s SCOTCH and WSCII initiatives had resulted in the incorporation of Symfony components into Drupal 8 prompted me to take another look at what I’d lumped into ‘yet another PHP MVC framework’. It quickly became clear that I’d horribly misjudged Symfony. Drupal 8 hasn’t quite hit code freeze yet, but the adoption of Symfony has already let it pull in some beautiful pieces of architecture.

My favorite architectural decision in Symfony is its robust sub-request framework because of the degree of support that it lends to one of my favorite technologies, Edge Side Includes (ESI). Many large-scale sites leverage CDNs to deliver cached pages and static assets, but they don’t cope with logged-in users well because the content served to them is so heavily customized. ESI allows you to offload mostly cacheable pages to a CDN and only request the uncacheable portions of the page from the origin server, but it’s currently underutilized because most web frameworks can’t render just a portion of a page. In Symfony, though, any component of a page (such as a block or a sidebar) can be built as a sub-request for little additional developer effort. A year from now, large-scale Drupal 8 sites may entirely bypass common performance pain points by taking an ‘ESI Everywhere’ approach and leaving the CDN servers to do the work of assembling a dynamic page from a static cache entry.

Symfony had caught my interest, and my first instinct towards interesting software is to hack on it. That meant I had to create a Symfony site, so naturally, I chose Drupal 8 as a starting point. The codebase is still in flux, but it’s stable enough to install and it already makes use of several core Symfony components. All that I had to do to get an environment running was spin up an Amazon EC2 server, set up a LAMP stack and install TraceView (by far the fastest part of the process). The Drupal 8 installation required installing the trunk version of Drush, but after that I was only a `drush qd` away from a functioning Drupal 8 site.

I hit a wall fairly quickly, since like most Drupal developers I’d never tried writing Symfony code or using Composer to manage packages. Instead of getting frustrated, I took a step back and began to read articles about both Symfony in its own right and how it’s being leveraged to rewrite Drupal. I made heavy use of the Drupal core issue queue and the Symfony API documentation, but I also turned to blogs written by Drupal and Symfony developers. Of particular note, chx wrote an introduction to Symfony in two articles as well as an article that helped me fix my first Drupal 8 performance issue:

Every request to the Drupal 8 server triggered well over a hundred `file_get_contents` calls.

Before writing any integration code, I pulled up some Symfony traces because I was curious what they would look like. Even without code modification, TraceView picked up on Drupal 8 making a huge number of `file_get_contents` calls on each request. While individually fast, the calls often added up to more than 100ms; worse yet, that many disk reads has severe negative implications for scalability. At my last job, a sudden increase in traffic took down our entire distributed filesystem just by repeatedly fetching a single JavaScript file!

When I examined the `file_get_contents` calls in TraceView and realized they were all fetching module-related files, I immediately made the connection to Symfony’s dependency injection container (DIC) (one of its major design paradigms) thanks to another post by chx about how the DIC is stored on the filesystem. The filesystem calls were due to Drupal not finding the DIC on the filesystem and instead recompiling it using a dependency-checking mechanism similar to a package manager but made more complicated by the ability of classes to define additional compiler passes and other overrides. Like most problems I run into on Linux, I was able to trace it back to incorrect permissions:  because I had run Drush as a normal user, I had changed the permissions on the relevant directory and prevented the webserver from writing the compiled DIC so that it could be loaded on future requests. One `chown` later and my testbed site was running more than five times faster.

I kept an open mind to Symfony as I continued to read, but coming from the perspective of a Drupal developer, I was quite surprised to hear that the familiar hook-based event system has already been partially replaced with Symfony’s event listeners (bringing it more in line with some other PHP frameworks that we support). It’s new territory for me, but I can understand the rationale as there are some serious advantages to the switch. Unlike hooks, events don’t need to check every module on the site for an implementation; they can also have their propagation stopped if no further work is necessary. This means that developers will have much finer control over how modules respond to events, rather than having to use unusual hooks simply because they get called at the ‘right’ point in page loading (I’m looking at you, `hook_footer`!). Interestingly, a total switchover has been postponed until Drupal 9, which means that these overlapping systems will both remain in place for the foreseeable future.

The differences between Drupal hooks and Symfony event listeners were quite relevant to building out TraceView support, so I spent a lot of time reading about them. It’s there that I stumbled across another excellent addition to Drupal 8: the `terminate` kernel event! This import from Symfony is the replacement for the recently removed `hook_exit`, with the key difference being that it can be run after a response is returned to a browser. This lets expensive cache setting operations be performed in a way that won’t harm end user performance, emails be fired off only when the request is complete, or disk writes take place after an upload confirmation screen has been shown to the user.

It took about a month to get up to speed on Drupal 8 and Symfony to a degree where I felt ready to stop reading and start writing. That left me with about a month to go before DrupalCon, and in that time I was able to turn out not only the Drupal 7 module improvements but a really awesome DrupalCon booth demo!

The top tier of Drupalception included both our beta TraceView Drupal 8 module and our TraceView Symfony bundle.

I couldn’t think of any better way to show that we’re serious about our Drupal support than to demonstrate every version of Drupal we support at once, and so I built Drupalception: a Drupal 8 + Symfony AWS instance using cURL (via Guzzle) to call into our Drupal 7 demo site, which in turn uses drupal_http_request to call into our Drupal 6 demo site. TraceView catches the entire stack as a single trace even when blocking calls are nested fifteen layers deep!

But wait, aren’t I skipping a step – how did I get from reading a bunch of blog articles about Drupal 8 in Symfony to a functioning Drupal 8 module and Symfony bundle? Well, that’s a story in its own right – and for now, one that you’ll have to wait for another blog post for! Until next time, sign up for our 14 day trial and try out firsthand our support for Drupal 6, 7, or even 8 (or any other stack we support, for that matter).

Related Articles

Drupal Performance Profiling With Tracelytics

Headnet Improves Drupal Performance and Reliability With TraceView

Case Study: Penton Media Improves its Drupal Application Performance

More Stories By James Meickle

James started as a hobbyist web developer, even though his academic background is in social psychology and political science. Lately his interests as a professional Drupal developer have migrated towards performance, security, and automation. His favorite languages is Python, his favorite editor is Sublime, and his favorite game is Dwarf Fortress.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


IoT & Smart Cities Stories
The platform combines the strengths of Singtel's extensive, intelligent network capabilities with Microsoft's cloud expertise to create a unique solution that sets new standards for IoT applications," said Mr Diomedes Kastanis, Head of IoT at Singtel. "Our solution provides speed, transparency and flexibility, paving the way for a more pervasive use of IoT to accelerate enterprises' digitalisation efforts. AI-powered intelligent connectivity over Microsoft Azure will be the fastest connected pat...
There are many examples of disruption in consumer space – Uber disrupting the cab industry, Airbnb disrupting the hospitality industry and so on; but have you wondered who is disrupting support and operations? AISERA helps make businesses and customers successful by offering consumer-like user experience for support and operations. We have built the world’s first AI-driven IT / HR / Cloud / Customer Support and Operations solution.
Codete accelerates their clients growth through technological expertise and experience. Codite team works with organizations to meet the challenges that digitalization presents. Their clients include digital start-ups as well as established enterprises in the IT industry. To stay competitive in a highly innovative IT industry, strong R&D departments and bold spin-off initiatives is a must. Codete Data Science and Software Architects teams help corporate clients to stay up to date with the mod...
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throug...
Druva is the global leader in Cloud Data Protection and Management, delivering the industry's first data management-as-a-service solution that aggregates data from endpoints, servers and cloud applications and leverages the public cloud to offer a single pane of glass to enable data protection, governance and intelligence-dramatically increasing the availability and visibility of business critical information, while reducing the risk, cost and complexity of managing and protecting it. Druva's...
BMC has unmatched experience in IT management, supporting 92 of the Forbes Global 100, and earning recognition as an ITSM Gartner Magic Quadrant Leader for five years running. Our solutions offer speed, agility, and efficiency to tackle business challenges in the areas of service management, automation, operations, and the mainframe.
The Jevons Paradox suggests that when technological advances increase efficiency of a resource, it results in an overall increase in consumption. Writing on the increased use of coal as a result of technological improvements, 19th-century economist William Stanley Jevons found that these improvements led to the development of new ways to utilize coal. In his session at 19th Cloud Expo, Mark Thiele, Chief Strategy Officer for Apcera, compared the Jevons Paradox to modern-day enterprise IT, examin...
With 10 simultaneous tracks, keynotes, general sessions and targeted breakout classes, @CloudEXPO and DXWorldEXPO are two of the most important technology events of the year. Since its launch over eight years ago, @CloudEXPO and DXWorldEXPO have presented a rock star faculty as well as showcased hundreds of sponsors and exhibitors! In this blog post, we provide 7 tips on how, as part of our world-class faculty, you can deliver one of the most popular sessions at our events. But before reading...
DSR is a supplier of project management, consultancy services and IT solutions that increase effectiveness of a company's operations in the production sector. The company combines in-depth knowledge of international companies with expert knowledge utilising IT tools that support manufacturing and distribution processes. DSR ensures optimization and integration of internal processes which is necessary for companies to grow rapidly. The rapid growth is possible thanks, to specialized services an...
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throug...