Jul 17 11

Gaining performance boost with Rails, Ruby 1.9, PostgreSQL and concurrency sauce

by serge

Recently I was trying to resolve performance issue in a back-end Rails app. The challenge was to reduce response time of one computational heavy back-end request. Target app is using PostgreSQL 9 for the database and running on Ruby 1.9 MRI. Logic for that particular request naturally breaks down into two parts. The first part is search, which is a combination of different filters applied together in order to reduce available dataset. There are filters that run SQL queries and filters making HTTP requests to other services (3rd party and our own). The second part is all about computing additional data based on search results. The fact that search filters can be made independent of each other was a clear hint for adding concurrency to the search step.

But first, I was trying to squeeze out as much performance as I can from single-threaded code itself. Ruby garbage collector is getting quite bad rap these days, but I didn’t want to go to the dark side right from the beginning and mess with reducing allocations or controlling GC cycles manually. My first targets were SQL queries.

read more…

Jun 14 11

There is nothing new except what is forgotten

by serge

Isn’t it fun that we had “blog in 15 minutes” video for many years now, but industry standard blog engine is written in “fugly” PHP?

There are always interesting things happening in Ruby and Rails ecosystem. Our community embraces change, even when change means redefining conventional wisdom. alias_method_chain has stopped being considered a good practice, and it was only beginning. Today we refer to classics of SOLID design principles, while people continue revisiting patterns and practices that have made Ruby stand out as highly dynamic runtime. Take method_missing as another example: it can be harmful just like any other powerful technique. What’s next? Dynamic dependency loading? (It’s evil indeed)

Our understanding of MVC is changing as well. Frontends aren’t what they were before. Driven by desire to match platform native user experience, people find ways to do crazy and beautiful things.

I can’t help myself but rant play with some fundamentals too, so I’d like to focus on OO design and SOLID theme in particular. I don’t think that all five principles are equally applicable to Ruby because of dynamic typing. SRP (Single Responsibility Principle) may be the only one that can be taken as it is. OCP (Open Closed Principle) makes sense as well, but given Ruby open classes, needs an updated statement (i.e. what is meant by modification and extension).

LSP (Liskov Substitution Principle) is much more curious to think of in Ruby. Why would anyone want to consider such boring thing as this:

Let q(x) be a property provable about objects x of type T. Then q(y) should be true for objects y of type S where S is a subtype of T.

when Ruby offers such wonderful and fun version as Duck Typing:

When I see a bird that walks like a duck and swims like a duck and quacks like a duck, I call that bird a duck.

I’m being sarcastic here, of course I don’t think that Duck Typing can substitute LSP. Have a look at behavioral conditions section in the wikipedia article for reasons. The fact that LSP is attracting attention right now can be pretty telling. I’m excited to witness it. Community is finally seeking for some grownup approaches, great! However, the question is: doesn’t that get us closer to some limitations of our beloved highly dynamic language?

Rails code is rapidly evolving, the same is true for the community and ideas behind it. The fact that such static typing rituals as DIP (Dependency Inversion Principle) and ISP (Interface Segregation Principle) start being considered may mean that within the current pace of growth, it becomes more and more of a challenge to manage complexity. It is well known that in order to successfully “Duck Type” one’s way through code, he or she should often mind more details.

Rituals are what we hate in Java and C++, it’s what gets in our way when we want to take the shortest path from an idea to it’s expression. But as codebase becomes bigger and more complex, rituals become inevitable. I think everyone who has got past “15 minutes blog app” state and the euphoria of “I’ll put together a dozen of gems and my CMS is done, than add some more and here is my CRM” knows that in order to build (and scale) apps beyond the framework, some set of carefully picked rituals is very important. Often the very same set that was in the way at the beginning. An extreme example of building beyond framework is Twitter and their reasons behind switching to Scala.

The fact that you can call any private method you like, or access any instance variable, or reopen class and do whatever you want is fun. But doesn’t that lead to less time spent on thinking about fundamentals like encapsulation? How often do we have thoughts like “should this method that I write be public, private or protected” ?

Ruby is famous for its well thought OO paradigm. It has been influenced by Smalltalk design, which seems to mean a lot in the world of OO. The reality however is that de-facto standard attr_accessor doesn’t contribute at all to good design. And even if we put some opinionated decision and use attr_reader instead, doesn’t that still feel like we’re pretending encapsulation? For example if we expose an array trough attr_reader :x, we can’t do direct assignment, but what about x elements? Being able to ensure immutability on language level is good for VM performance optimizations and very important for various concurrency patterns. There are plenty of opportunities to break encapsulation. Of course being able to do something doesn’t necessarily mean having to do that, which brings us to certain aspects of code discipline. But, isn’t it all about rituals again? Implicit rituals unfortunately.

I heard a phrase at Railsconf 2011, that we end up writing more tests because of interface specs (Lint), and it’s ok to do so. It may be true in that particular case, but generally, is it okay to write more code especially when it does the work that interpreter or runtime can do better?

Aug 6 10

Quickly Copy Current File Path in TextMate

by serge

I work with remote teams most of the time. And wile I do respect co-location practices, from my experience a phone and an instant messenger together can serve as a pretty good communication channel as well. Given this, I often need to get someone’s attention to a particular piece of code, so that we’re looking at the same file in order to discuss it. One way to achieve that is to paste file path into a chat room. But in order to paste, I need to get that path into the clipboard first. There are some some things like this that we just have to accept.

I’ve found this beautiful article on the topic. It has nice explanation of different ways to access current file location in TextMate. I’ve chosen the last option for myself and created a custom bundle, but did a small change in script however. Having file path relative to the file’s project directory is more convenient for me, I don’t want to distract people with details of my home folder organization. Here’s my version of the script:

 
FILEPATH=${TM_FILEPATH/#$TM_PROJECT_DIRECTORY\//}
 
echo -n "$FILEPATH" | pbcopy && echo "Path copied to clipboard: $FILEPATH"

You can create it yourself using CiarĂ¡n’s guide and this code snippet, or just grab it from github: http://github.com/bgipsy/files-tmbundle

Jul 30 10

Two Things I Avoid

by serge

There are two things that smell bad to me: when database data leaks into code and when app code leaks into database. I’m not talking about SQL views, stored procedures, foreign keys, and other well known holy war stuff. I’d like to bring up some of “misplaced constants” problems.

read more…

May 19 10

Using Models In Rails Migrations

by serge

Every once in a while we may want to do something like this:

 
class CreateUsers < ActiveRecord::Migration
  def self.up
    create_table :users do |t|
      t.string :name, :null => false
      t.string :email, :null => false
      t.string :hashed_password, :null => false
      t.string :salt, :null => false
      t.timestamps
    end
 
    add_index :users, :email, :unique => true
 
    User.create!(:name => 'Default User',
      :email => 'user@example.com', :password => 'demo',
      :password_confirmation => 'demo')
  end
 
  def self.down
    drop_table :users
  end
end

Does it look good? If the answer is yes, then the rest of this small post worth reading.

read more…

Nov 28 09

Postgres: Thread safe insert/update

by serge

I don’t like stored procedures, but sometimes creating a materialized view can be reasonable. Business logic doesn’t belong in database procedural code, but, I guess, everyone has a few stories about how easily performance considerations overcome design principles. I think I had just one materialized view that I consider nice in my carrier until now. It was maintaining aggregated view of one quite heavy table. That materialized view was used instead of queries like SELECT ... GROUP BY event_time::date It was long time ago, but up to present day I think that it was performance optimization in its purest form. No business logic was involved.

I’d like to share a trick that served me well in those triggers and I think it was not the last time. Every once in a while we need to create a row in a table if it isn’t there yet or update it otherwise. It can be implemented as easy as it sounds, with solid feeling of accomplishment, until one day you face process races.
read more…

Oct 27 09

Factory Pattern – Ruby Style

by serge

Recently I was working on a framework for front end sites. The framework is packaged as a rails plugin, but unlike regular plugins it contains large blocks of logic. Framework models and controller classes are used to build a reusable abstraction layer above one particular API. A front end application then can subclass framework controller template classes to incorporate shared UI patterns in it’s own controllers. At the same time such applications introduce some variations in user experience. That’s the reason behind having many of them instead of one. And now I’m quizzed to find a nice way to make all the generic framework code play nicely with app specific extensions.

read more…

Oct 5 09

The Dark Side of Duck Typing

by serge

On the Origin of Species

What walks like duck and quacks like a duck is a duck. That what duck typing is about. It makes development easier. We don’t need interfaces and abstract classes. Many object oriented patterns look much simpler because of such dynamic typing. They come more naturally in ruby code and often we don’t even notice them until we see explicit names like Visitor, Decorator, etc.

After I’ve got used with such a powerful tool, it was too easy to step into dangerous territory without even noticing. Sometimes things do walk and quack like ducks all the time… well, almost all the time. And sometimes in a pack of ducks there are some species that do quack (and that makes you feel safe), but that doesn’t make them ducks.

read more…

Jun 3 09

Adding foreign key constraints in rails migrations

by serge

More often than not models have associations. And foreign key constraints are sometimes used for keeping them in good shape. The role of such constraints can be indispensable or trivial, depending on how your database is managed.

Here’s a rails plugin to add foreign keys for associations. It doesn’t have any dependencies. And it’s code is simple so it can be read in one take. These are the two things in plugins world that I appreciate the most.

In migrations, when a table is created, it recognizes columns which are foreign keys by _id suffix. Then it adds foreign key declarations. Referred table name is guessed from column’s name according to rails conventions. This can be disabled by adding :references => false option.

If a column should refer to a table which name can’t be inferred that way, you can explicitly specify it by using a symbol or a string.

create_table :comments do |t|
  t.text :body
  t.integer :commented_post_id, :references => :posts
end

will produce

CREATE TABLE comments(
  id SERIAL PRIMARY KEY,
  body TEXT,
  commented_post_id INTEGER REFERENCES posts
);

And with strings you can do all the things that you shouldn’t want to do:

create_table :comments do |t|
  t.text :body
  t.integer :commented_post_id, :references => 'whatever(strange_pk) ON DELETE RESTRICT'
end

will become

CREATE TABLE comments(
  id SERIAL PRIMARY KEY,
  body TEXT,
  commented_post_id INTEGER REFERENCES whatever(strange_pk) ON DELETE RESTRICT
);

That can be handy when dealing with nasty legacy schema.

And you can add foreign key constraints for already created tables:

add_foreign_key_constraint(:posts, :comment_id, :comments, :p_id)
ALTER TABLE posts ADD FOREIGN KEY (comment_id) REFERENCES comments(p_id);

You get the idea. For more examples please see spec/migration_spec.rb here

So the next thing you may want after you’ve packed your database with foreign key constraints is to get rid of them. In your tests. Rails disables referential integrity checks before loading fixtures and enables them after that. So missing references in fuxtures are ok. And if you need to disable foreign key constraints for certain table you can always make a call.

remove_all_foreign_key_constraints('comments')

Another option is to automatically drop all constraints for tables that are referenced by test suite fixtures statement. That is what another branch with_fixture_patch of the plugin contains. I’m still considering if it can be useful…

To install run

./script/plugin install git://github.com/bgipsy/fk_constraints.git

from your app root directory.