Trends in Ruby Analyzing Rubygems Stats for 2015

2015 is behind us and the market is flooded with new languages and frameworks. A lot of them seem promising, and in the same time a lot of developers are talking about the death of Ruby and Rails.

Rubygems is the official package manager for Ruby and we decided to crunch up some stats for 2015 and see what’s what.

For analyzing the Rubygems database we got ahold of the database dumps. We didn’t get the database dumps through some shady backchannels, Rubygems.org provides official database dumps on a weekly basis. The Rubygems API is quite simple, so the dumps are actually a convenient way of giving people the ability to do analysis without having to hammer the API.

Rubygems hands out minimal statistics on their stats page such as:

This is all well and good, but we wanted to get more detailed insights and find out what’s happening in the Ruby market. Check out what we discovered.

The number of gems created in the 5 years prior to 2015 was on a rise. Back in 2014, there were 27.000 gems created – that’s a lot! Unfortunately, 2015 didn’t manage to keep the trend of growth and was even a bit worse than 2013.

At first, nine out of ten would probably say that the reason for the downfall is that there’s a lot of people leaving Ruby and going for Node, Go, Elixir and similar languages.

However, I would say that’s not the primary reason. The reason is the maturity of the platform. As a Ruby/Rails developer, I can’t remember the last time I needed to build a gem to support a functionality in an app. Chances are that thousands of developers have already encountered a similar problem and there’s a well tested gem for doing what I need.

I also checked out how many gem versions are being released and the trend looks similar to the first one. However, the decline in 2015 was a bit smaller.

Unfortunately, the statistics for downloads prior to August 2014 aren’t available so you will only be able to see the state of downloads for the last 17 months.

We can see that unlike the number of gems created, the downloads are actually skyrocketing. Back in August 2014 there was a total of 141M downloads, while in December of 2015 the downloads were already at 262M. That’s an 85% increase in less than a year and a half.

Let’s be honest, in 90% of cases Ruby is used in the context of Rails. The most popular Ruby framework is on the rise, with a 50% increase in a year and half.

When we analyzed this data, we were kind of worried – what if a large amount of those downloads are coming from legacy apps that are still stuck on Rails 3? So, to verify, I also made two other graphs related to different Rails versions.

Rails 4 was by far the most downloaded version in 2015. At the beginning of the year it was downloaded twice as much as Rails 3, and at the end of the year the ratio moved up to 6 to 1 in favor of Rails 4.

I also looked at downloads for previous major versions, but the numbers are really low – around 10K downloads per month at the end of the year for both Rails 1 and Rails 2.

Rails 4.2 was released at the end of 2014 and its adoption was quite fast. At the beginning of the year 4.1 was still downloaded more often, but by the end Rails 4.2 was at 1 million downloads per month.

These graphs give us the impression that Rails actually really skyrocketed in usage with the release of 4.2.

Sinatra, Ruby’s famous micro web-framework was the 2nd most used framework. I was pleasantly surprised with Sinatra’s downloads per month – it’s being downloaded about 30% more than in the summer of 2014. For some reason I had a feeling that it wouldn’t go up as much.

And it’s also not that far behind Rails – last July, at its peak, Sinatra was downloaded 1 million times.

Note that the biggest spike was in March when the only 2015 release of Sinatra happened. However, its download numbers stayed high since then. These numbers can be ambiguous – they show that Ruby devs tend to update their dependencies, but it also seems that Sinatra is probably being used more than it was before. Anyways, it’s a good thing.

There are other Ruby frameworks for building web apps, but they aren’t nearly as popular as Rails or Sinatra. The three that are most interesting are Padrino, Volt and Lotus, but none of them have hit major version 1.

Padrino, Sinatra’s advanced brother with more structure and Rails-like generators isn’t finding its users yet.

Volt, the reactive web framework for writing Ruby both on the server and in the browser, is still under heavy development and lacks features such as SQL database support, so it isn’t actively used yet.

Lotus, a cleaner and object-oriented alternative to Rails developed by Luca Guidi still isn’t used nearly enough and isn’t stable. However, I’m finding it to be the most promising alternative to Rails for the future. I love how it’s learning from architecture pitfalls Rails has and embracing out of the box multi-application support, views, templates and cleaner controllers.

It’s interesting that Postgres and MySQL adapters had a similar amount of downloads until about a year ago when Postgres took off. I thought the usage statistics for Postgres in Rails were higher than those for MySQL since a long time ago, but it seems it’s a relatively recent trend.

Mongo’s usage is mostly consistent, with a few occasional spikes, while other NoSQL databases are used reasonably less and I haven’t added them to the list.

I also didn’t add the Sqlite adapter since it comes natively with Rails and its download statistics aren’t really related to its usage.

I didn’t expect to see Thin at the top of this list. It seems that Puma & Thin are experiencing a breakthrough. One thing that you’ll see on this graph that isn’t common with others is that usage per gem is declining – particularly for Passenger.

However, I wouldn’t take the info I just gave for granted – Passenger can be installed with Rubygems and downloaded as a standalone Unix package, so these stats don’t represent its real number of downloads. No traffic spike is visible in the stats since the release of a new version at the end of 2014 (codenamed Raptor).

Puma is probably downloaded more than ever due to the fact Heroku listed it as the recommended app server.

Also, I’m not sure whether Thin’s download stats truly reflect its usage as much they reflect the fact that, in some cases, Thin is a dependency for other popular gems such as Rack and Faye and an app server used in development instead of Webrick.

Jeremy Evans has been working real hard on Sequel, the cleaner alternative to ActiveRecord and I’m really glad to see it was used a whole lot in 2015. Sequel provides a flexible interface with which you never have to write custom SQL queries. It’s faster, supports basics like left joins out of the box (unlike ActiveRecord) and is very well maintained (0 issues).

The thing I fear most about using Sequel in Rails is gem compatibility – there’s too much stuff that evolved around ActiveRecord, but hopefully we’ll be using it in the near future. Read more about Sequel here.

I haven’t included usage for ActiveRecord since, being a hard dependency, it pretty much reflects Rails usage.

Minitest certainly owes a large number of its downloads to ActiveSupport, since it is ActiveSupport’s direct dependency (and a direct dependency of a lot of other famous gems such as Sinatra). So it’s natural that as Rails & Sinatra are getting more downloads, so is Minitest.

On the other hand, the RSpec DSL is also growing and got a 100% increase in just a year and a half.

One thing is for sure – as total downloads pile up, so do the testing libraries’ downloads increase. Keep testing, Rubyists (and hope those spec/test folders aren’t empty!).

I thought that Delayed Job’s usage would be way lower than it is, but it seems that there’s still a whole lot of developers that don’t need really fast background processing and ignore Sidekiq and Resque because of their dependency on Redis (like we do). However, it seems that Sidekiq’s lower memory consumption and faster processing has put it on the top of the chart, lately being used twice as much as the other two.

If somebody were to ask me what to use for authorization, I would definitely say Pundit. Pundit uses a nice policy-based approach to doing authorization using DRY principles without having a file bloated with all the authorization rules.

Although it more than doubled in usage over the last year and a half, CanCanCan is still heavily used and has tripled in usage.

The database infrastructure of rubygems.org consists of a Postgres database and a Redis store – Postgres holds data about gems, gem versions and users while Redis is in charge of holding all the information about daily and total downloads per gem and per gem version.

Getting these stats wasn’t easy – Postgres and Redis data aren’t in sync so we had to do that manually. The Postgres dump is not that large, only 150MB, but the unzipped Redis dump is almost 4GB. Redis needs to load all of that data into memory and when loaded, it’s somewhere around 23GB.

I did the analysis on my Mid 2013 MacBook Air with an Intel i5 processor and 4GB of RAM. I probably should have taken a Digital Ocean / Amazon instance with more RAM to crunch this data. Let’s see why:

By reading up on these stats we can see that the Ruby community is still steadily growing and that the ecosystem is quite mature. Depending on how you look at it, that can be either good or bad.

For the language butterfly that always wants to play with the latest piece of shiny tech – that can be bad. For the seasoned professional that just wants his technology stack to be stable and battle-tested – that can be good.

Nevertheless, I have a feeling that the level of innovation in the Ruby community is decreasing and that the ecosystem is plateauing.

For the last couple of years lots of people from the Ruby community went off to build various other languages and frameworks – Jose Valim built Elixir, Chris McCord built the Phoenix framework, Yehuda Katz created Ember, Steve Klabnik joined the Rust core team.

Yehuda and Steve also created the JSON API spec, developers from Manastech built the Crystal programming language and so on.

Nobody should get religiously tied to a technology and I’m really looking forward to new languages and frameworks being built. However, a lot of time is going to pass until a new framework reaches the mature state that Rails has today.