Opbeat is joining forces with Elastic – Learn more about Elastic APM

The changing face of web development

Recently Marco Arment noted that it was scary how little he’d diverged in his ten years as a web developer. I disagree. Looking over the past ten or fifteen years, the scary thing is how much constant change there has been within the role of a Web Developer. Yes, we are still just sending text over HTTP to browsers, but everything else has changed multiple times over.

The HTML Programmer

In the beginning there was HTML, and web developers hard-coded markup with font attributes and all. Then developers would fire up a browser to see how the site looked before uploading the result to an FTP server. Development to deployment in no time.

One of the first books I read about web development argued passionately for building something it called database-backed web sites. Apparently, static HTML wasn’t the future, and instead it would be more efficient and scalable to model our data first — and then dynamically create pages using that data and a scripting language.

The Backend Programmer

That marked the first transition, not only in web development, but also what it meant to be a web developer. Instead of building every page manually, pages could be dynamically generated in Perl, PHP or VBScript leaning on relational databases (PostgreSQL came to life in the late nineties, and MySQL went open-source in 2000).

Within this period, the foundational tools such as scripting languages, web servers and databases matured – and the rapid evolution of the tool box is the first great example of how tools change our work days, even if the title stays the same.

Fifteen years on, this shift seems very trivial, and it probably was. With that in mind though, it’s worth traveling back in time to see Steve Jobs in 1996 talk about the magic of server-side web development with databases and dynamic objects.

The Framework Programmer

Ruby on Rails is the best example of how a web framework changed everything about web development, but it’s only one of many. Python programmers have become Django programmers. PHP programmers have become Drupal programmers. JavaScript programmers have become jQuery programmers.

These frameworks marked a huge evolution in abstracting the developer tool box – and illustrated the power of open source to ignite new vibrant communities of developers. And it sure as hell marked the time when it became socially unacceptable to build and deploy using your own framework. Probably a good thing.

The Service Programmer

These switches between each previous development paradigm have been marked by adding another layer in the stack. Well-focused, well-maintained, well-documented open-source projects solved common problems, and in turn made it easier for web developers to solve increasingly complex tasks in less and less time.

The most recent paradigm shift is similarly about abstracting away common problems, however it doesn’t do so in code – it does it in services. Before we started getting help from frameworks and open source, we would use that to create our Death Star of a web site that essentially did everything by itself. User login, search indexing, media scaling, comment management, page caching, parsing incoming mail and all things in between. Now, there are great services for all the plumbing which allow us to focus only on the parts specific to our own projects.

Just as we have been able to choose between frameworks, we now get to select between building or buying – and a core skill for any web developer becomes to make this choice and integrate third-party services into ones own code. (Andreas touched on this well in his article about the subject on this blog some weeks ago).

This isn’t just another thing to learn, it requires a different way of thinking from the ground up. It challenges the developers as they’re building. It challenges employers as they’re hiring. And frankly, it changes our marketplace and our customer base, because we need to think about our services being used alongside other ones as just one part of the puzzle.

The age of the devop

This is all interesting as a history lesson from a guy who’s been doing the same thing for quite a long time – but the really fun bit is how each changing paradigm alters the expectations of us. And it challenges the professional web dev to keep expanding his tool box. Do you know HTML? Abso-fucking-lutely! PHP, Python and Perl? JavaScript, CSS and Ajax? SQL, Rails, HAML, SASS? Node, nginx, Varnish? CSS Media Queries, jQuery, Bootstrap? Heroku, S3, CloudFront? Puppet, Fabric, git? SPDY, WebGL, WebSockets? Kinda, yeah. But it’s become a ridiculous list to name-check let alone one to have an opinion about and work within. And of course it touches every part of the stack from prototype to production.

While web software stack have become decidedly more varied, hardcore ops seems to have become easy. The illusion is that there is no hardware involved and anyone can just power up AWS, Rackspace or Azure. These two things in combination shift the balance and has the developer taking over a bigger and bigger piece of operations – in many cases all of it.

Web developers have also become omnipresent devops. You build it, you run it.

To me, fifteen years have changed everything but our title. That’s extremely exciting for two reasons.

First, it changes what we get to build. The available tools and especially the available services let us create much more ambitious apps much more quickly. And we can cherry pick from the entire stack which parts to build ourselves and which parts to farm out. Whenever we choose the latter option, a great, dedicated team will focus on expanding the road map for just that particular service. An doing it well.

Secondly, it changes the tools we get to build for each other. Because the shift from system to service means that there are a lot of great services waiting to be built.

This history lesson is also my round-about way of telling you why I’m excited about Opbeat – because it’s a tool that is sorely needed in the age of the devop. Today, developers are deploying code across several different services and servers, all integrating with each other and third-party services. Meanwhile, they’re responsible for uptime and stability. That’s a pretty important job, which requires a control center. Before Opbeat we didn’t have the build-or-buy option and needed to roll our own (and, of course, we have!). That is kinda nice to farm out to a great, dedicated team…

About the author
Steffen Tiedemann Christensen is the founder of 23 and investor in Opbeat. Follow him on Twitter.