From Reading Spree

Having worked with the Spree gem earlier this year, I was curious to see how things had progressed with my favorite e-commerce store.

The look of the site itself spreecommerce.com has much improved. I’m glad to know that Sean took my suggestion for the name. It really rings well. I’m also happy to see my website amongst those listed in the ‘Featured’ category, although it was a major team effort.

Anyway, getting on to today’s discovery, I was reading through the source code of the various vendor/plugins, hoping to learn or discover a few cool things. Boy, I wasn’t dissappointed!

First off, the awesomenestedset gem. It’s a simple enough concept, to which you can find explained with nice graphics at developer.com. I had read about that data structure before, and reading that article, it became obvious that using this technique for a navigation structure makes all the sense in the world.

Secondly, a minor but fairly important point I did not know about sqlite. It is possible to establish a database entirely in memory. For small datasets, especially those that resolve around test, this is an incredibly valuable technique to speed up an application or testing.

Third, I came upon this declaration, or similar forms a couple of time through different gems.

class_inheritable_accessor :calculators

Fourth, the unloadable keyword, which is probably not something the average developer will need on a day-to-day basis, but which helps to make constant unloadable (not dependent on refreshing the server).

I recommend to anyone interested in Rails to read more about this from raulparolari.com. He explains very clearly this topic. In short, a class inheritable accessor is similar to an class instance variable, whose major difference is that its default value is given to inheriting children.

Reading through the Spree source code is absolutely fantastic exercise for anyone who wants to be a better Rails programmer. There are plenty of highly qualified people who have brought their insights into this project. Unlike Rails itself, it is possible to read all of the code in one day.

From learning how to pack a gem, to using jeweler for your application, test structure with shoulda and rspec, jazz up your application with DHTML with Prototype (script.aculo.us) and jQuery, SASS and compass. It’s all there for the intermediate Rails developer. On top of it all, you get to be part of one of Rails’ fastest growing project!

Leave a comment »

No stack error; how I found ObjectSpace

This is probably the strangest error I’ve yet met in Rails. I was trying out functional testing in Rails 2.3.4, with a generated scaffold. After a few changes to the model and played around with the fixtures, I banged out a ‘rake’ command. The reply I got was really strange.

Loaded suite /opt/local/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake/rake_test_loader
Started
EEEEEEE
Finished in 0.125619 seconds.

1) Error:
test_should_create_user_story(UserStoriesControllerTest):
NoMethodError: undefined method `key?' for #{String:0x27c7ac0}

2) Error:
test_should_destroy_user_story(UserStoriesControllerTest):
NoMethodError: undefined method `key?' for #{String:0x27c40b4}

3) Error:
test_should_get_edit(UserStoriesControllerTest):
NoMethodError: undefined method `key?' for #{String:0x27c0798}

4) Error:
test_should_get_new(UserStoriesControllerTest):
NoMethodError: undefined method `key?' for #{String:0x27bce7c}

5) Error:
test_should_get_this_index(UserStoriesControllerTest):
NoMethodError: undefined method `key?' for #{String:0x27b9574}

6) Error:
test_should_show_user_story(UserStoriesControllerTest):
NoMethodError: undefined method `key?' for #{String:0x27b5c6c}

7) Error:
test_should_update_user_story(UserStoriesControllerTest):
NoMethodError: undefined method `key?' for #{String:0x27b2350}

Small liberties have been taken with the shell output, since WordPress doesn’t like that combination of angle brackets.

This is a very strange error to me; I’m used to being puzzled by bugs, looking through the stack trace and figuring out which function is to blame. But in this case, there was no stack trace at all!

I googled the exact error message and found someone who had a similar issue when going with rspec. Another individual, who was trying to help the first, commented ‘there should be a stack trace’.

I decided to use my good friend ‘ruby-debug’ and started going right after the class declaration of the UserStoriesController.

I wasn’t sure if I was searching at the right place. At first, I was in blankslate.rb. From there, I jumped around a while until I got to the test loader, and then entered optparse. I must have press ‘enter’ a good 500 times before I left the key depressed, speeding through a blur of jumping around in the class. When finally I left optparse, I found myself in another more obscure part of ruby, where I was looped about probably for a good five minutes. I decided to look at what this loop was doing.

I entered into irb and punched in:


ObjectSpace.each_object do |this_one|
puts this_one
end

I saw a brief flurry of code and the return of that iteration was something like 74000. Curious, I scrolled up and saw ‘everything’.

This is actually a bit scary. Every single bit of object imaginable is somewhere in this large space.

I encourage people to try this out from within Rails. I’m sure the exercise will be quite staggering.

There was simply no way I was going to make it through reading that huge pile of objects. I decided simply to type ‘continue’ in the debugger and go back to my previous, esoteric error message.

I tried google again, this time with the `key?’, exactly as per the Ruby output. I found another post on google groups where someone with that error message was asking for help. Looking at that trace, someone suggested the fix was to be made in a yml file.

And so that was it. A broken, not properly space yml file, which made Rails go nuts and swallowed my error message.

I’m wondering why Rails doesn’t include a fixtures/yml tester, to ensure that the syntax of the fixtures correct? Given that white space are counted, I don’t think I should rely on my attentiveness to recall to check on the yamls.

Leave a comment »

Revisiting seed data in Rails

I’m glad to announce the coming release of a new Rails plugin, Seed Me.

It was developed with the intent to fix one of Rails’ flaws when it comes to migrations; it is possible to define precisely what the data structure of an application is going to look like, but it lacks data itself.

This plugin is intented to fill that gap, thus allowing Rails developers to create migrations and the data that go with them. This, to me, is a more natural system then the Rails seeds which is defined at some point.

The difference is more than minor. Seed Me will allow a better tracking of all the data that go into an application, similarly to how migrations document the intention of the data abstraction. At the end of the day, a living system which contains all essential data is a God-sent to any tester or quality assurance specialist.

The other difference is that the Seed Me plugin is better suited to take existing, important application data and commit it into a duplicatable data strategy. If Seed Me data migrations allow a tracking of intentions, then any Seed that isn’t linked to a migration is necessarily ‘core application’.

The team is expecting to release this during the coming week. Stay tuned.

Leave a comment »

Of Smith, China and the USA

I recently re-read the great classic from Adam Smith, The Wealth of Nations. As I was going through the first chapter, I could not help but retrace the steps that the United States had taken to become the foremost super-power of the world.

The key factor of wealth, according to Smith, is specialization, the act by which one man is devoted to a single task. Smith often talked about the act of making nails; where one man could make a few dozens a day, a team, each focused on their specific tasks could make many more.

Thus, the wealth of the United States was realized in the specialization, to the highest degree possible, of its laboring force. The same process is now underway in China, which is turning its vast trove of illiterate laborers into a wealth of specialized workers.

The key gamble of the leaders of the United States is that the process by which China is enriching itself (and the world in general) by specializing its labour force will not interfere with the United States own growth.

I believe it is impossible to occupy the position of a super power and not have a specialized workforce. Some will point to the current trend in the services industry as evidence that the specialization is continuing, but I see the evidence of two levels of services, the low, unspecialized labor of burger flippers and hair dresser, and the high-tech services which account for most of the growth in raw productivity of the United States.

Of course, it is well known that China has cheapen its currency to achieve the goal of attracting capital investments and infrastructure building. The proper response of the United States, all those years ago should have been to reprice China’s currency. Such a maneuver would be difficult, if not impossible both politically and economically at this point. The population of the United States has grown lazy and accustomed to consuming what wealth there is. Maintaining an unbalanced currency helped in this process.

It will be seen now if the gamble of going for the high-tech was worth it.

My view is that it isn’t, simply because the majority of any population is incapable of working in such a mentally demanding environment. Thus menial, physical labor is perfectly acceptable for a large segment of the population. This is not to be understood as a caste system, but more broadly, it is a caste system of our own making, whereby the habits and entertainment we maintain, maintain us. For the beer drinking TV watcher who spends no time in formal education, what hope is he in a society that has turned its back on physical labor? It is, then, for those whose inclination takes them in this direction that industry needs to be taken care of.

Also, logically, how can such an advanced, high-tech society be built, but upon solid foundation of cheaply made, high-quality goods, whose cheapness and quality are dependent upon the specialization of the labor that made it? The common availability of such goods makes it more likely that neighboors will build likewise cheap and high quality goods, a virtuous cycle of sorts.

Finally, there is a very distinction between the physical and mental realms, as far as the labor itself is concerned. The act of mental creation is often quickly replicated; computers make it easy to automate systems and copy information once a clear path as been established, leading to thousands of copies of actions in a very short time. Meanwhile, in the physical realm, such shortcuts usually do not exist, simply because all physical process have a tendency to break down. Resources to make things are usually more expensive than in the mental realm also.

Yet, there is a strange inversion; for if in the physical realm, we have figured out how to proceed with specialization and manufacturing chains and robots, our mental models are far more primitive. How many software projects of large scales have ballooned all out of proportions? The UK health software system, or the USA replacement system for the airport guidance system have both been monumental and costly mistakes. This is because we are still working as mental artisans.

While in theory, we could have specialization in the mental realms, in practice it is often much more difficult to do so. The average high tech worker will have a dozen tools or so as part of his workflow. Email, web sites, internal tools, wordprocessing software, mathematical modeling tools, spreadsheet software, text processing, programming software, database systems, contact management, expenses accounts and many more. It is then impossible to have any speciality in such a workflow. If one could spend five to ten years with, say, a word processing tool, one could become a formidable specialist. Encouragingly, businesses and corporations have been resisting the so-called ‘upgrade treadmill’, which might in turn help overloaded workers acquire more specialization. But with the sheer amount of existing and new tools to help workers in their jobs, the possibility of specialization remains elusive.

The main reason that this is so was hinted above. Whatever can be clearly mentally conceived is quickly resolved, leaving us with a morass of unclear ideas. This is why large software projects run afoul of estimates. We simply have too many unclear ideas of the complexity that we have created. And until we can come to streamline and specialize our thoughts, it shall remain so.

Leave a comment »

Selenium testing best practices

 

During the last two months, I’ve been doing Selenium development. I’ve been somewhat mixed in feelings about the Selenium platform.

Prefer CSS3 selectors

In theory, Selenium makes it fast and easy to create new tests. With a Firefox plugin available, practically anyone can take a page and record a test for it.

In practice, however, there is catch. Most of the selectors of Selenium, when recorded using the Firefox plugin use Xpath to operate their magic. While using an Xpath is not a problem in itself, it is a technology with which fewer developers are familiar, then say, CSS. More aggravating to me is that the Xpath that is implemented is incomplete. I haven’t checked thoroughly on what the differences are between Selenium’s Xpath and the W3 specifications, but there are enough that I will often write a valid Xpath query with another Firefox plugin called Xpath Checker that fails with Selenium. The solution often is to rewrite the selector using CSS3, which seems to work better in most circumstances.  The thing that saves Xpath is its great flexibility when it comes to, say, selecting a tag based on a value somewhere else on the page.

Get GIGO with it

Not all the problems we’ve been having with Selenium come from it.

I’ve spent a week or two to get a meaningful, representative data set, from which to operate Selenium. The, of course crucially important in order to get non GIGO results.

The problem in our case is that it is not something that had been done since the project inception. While migrations help a great deal with the structure and fixtures help with seeding some values, our system is complex enough that it is not realistic to simply rely on fixtures.

The first approach was to take an existing data dump from a live system. While this was quite workable, it was also hard to maintain the test data, the Selenium tests and the application up to date and sufficiently seperate from one another.

The second approach we took was to include fixtures on top of the dump. While this was convenient to quickly resolve issues with a particular test, we rapidly found out that this caused unit and functional test which already relied on certain value to fail.

The third approach was to start from scratch and completely rebuild the database, in order to have a perfect blank, or master from which to evolve. In practice, the system’s data was complex enough that when there was missing data, it was often difficult and time consuming to isolate the origin of the error and correct it.

The current approach I feel is the best one so far. It is simply to take the previous, start-from-scratch approach and automate it entirely, thereby introducing better traceability into the test data process and shortening the time required to fix initial data errors.

The lessons I take from this are:

  • Since you’ll be agile and testing, plan your data creation strategy from the beginning.
  • Make sure that the data creation process is as automated as possible, so that when large changes to the application come about, it is possible to quickly make Selenium follow along
  • Involve domain experts. Often, creating a blank slate of the application can be daunting, especially in a large system that has evolved without such a system. Domain experts are your best bet into what need to be included in the creation of initial data.
  • Maintain the strategy. While it would be nice to have tests on the data that the strategy creates, this is a bit of a chicken and the egg problem. So, instead of testing directly the data creation strategy, reexamine it in thought when one of its dependent test fails.

You’ll probably want to have your strategy using Rake or Raven depending on your platform and preferences. Even using Rake on DotNet and learning a new tool is better then relying on manual efforts to create that valuable data.

Leave a comment »

From Here to there and back again

Well it was a short and sweet trip to Quebec city. I must admit that my first impressions were not all that great. I was very tired and it seemed to me that there was noise everywhere. Then Giulio showed up and explained to me that I was in the core of downtown (well, one of them, as I got further information later to the effect that there are two downtown in Quebec City) and this was rush hour. As we traveled through the city, I realized how much the place share with its european ancestry, the sweetly adorned houses going up and down hills, all freshly and brightly painted. The noise quickly faded away and the footsteps of people became one of the only thing we could hear.

We took some time to walk around then went to the Musee de la Civilisation, where Dragons from all the countries of the world where on display. The most impressive and intricate where the Japanese and Chinese, for whom the beast had heavenly traits that emperors would duplicate for themselves. In the courts, the nobles would be dressed in robes with dragons, use a dragon letter opener, use a censor with a dragon curled on top, carry jewelry of jade with dragon while the Emperor would sit on a throne, large and impressive with, again, a dragon watching over him. I could not help but think that I wanted one of those.

We then saw St-Foy, the most active suburb of Quebec City. I didn’t like the shopping mall atmosphere, long, unwalkable roads and shiny but soulless buildings that have cropped up there. We left soon enough.

We then walked on the Plains of Abraham, where the battle for the destiny of Quebec was waged. The sharpness of the cliff there made the climb and ultimate victory of the British all that more impressive.

We then had another person join us. By that time, it was getting late, so we ate in a great tasting restaurant where the service was less then stellar. We then walked all around the Vieux Quebec, which, under the sway of night, shone of orange glows, small intimate atmosphere. The trees and plants, the cobblestone road, the gorgeous ancient architecture, everything about the Vieux Quebec speaks of luxury. As the days get shorter until the 400 birthday of the City, the pride of the habitants in the beauty of the place is tangible. I wish I will be back next year to see Quebec in its full glory.

But for now it’s back to Montreal, on the way to Toronto.

Leave a comment »

And the trip begins to Quebec City

For the longest time, it seemed as though complications would keep piling up. I will blame my own indecision for these emergent difficulties. To be fair, there are many people I know here in Montreal that have been supportive of me, and me, in return, have been very supportive of.

Two days ago, I took a strong resolution that no matter what obstacle would show up (and show up they did), I would go out there, escape from the Island.

So with the beginning of this voyage, I hope I will be able to find things worthy of being talked about, stories of the people I met and all the normal extraordinary that accompanies traveling.

I suppose there are worst ways to begin then to listen to a live concert of Strauss’ Also Sprach Zarathustra.

Hope for me good times up there in the Capital of the Quebec Nation and I will hope the best for you.

Leave a comment »

Ron Paul at Google Interview

Ron Paul is the hottest candidate on the internet for the next presidential election. There are many interviews where Paul explains bits and pieces of his philosophy. None have been as good or as substantial as the extended dialogue he gave at Google.

If you like what you see, consider checking out the interview at BreakForNews.com where he talks more about healthcare topics.

Leave a comment »

Blogging

Blogging is hard. In fact, writing is hard, but I would postulate that writing successful blogs is even harder. Perhaps it requires a certain way of thinking, a handful of habits that make or break a blog. I don’t think I will ever get the hang of it, mostly because most of the content in blogs is borderline mentally deficient. An endless series of fragmented information, pointed opinions about political realities, top ten lists, blogging seems to me in many ways related to community television.

I compared a bit of writing I did last night, under the circumstances I prefer, that is pen and paper, to some things I had written here as a placeholder for this post. Under the influence of the computer monitor, my thoughts seem to perish. I cannot say why that is. I suspect the eternal white glow, while ok at emulating the look of paper, carries none of the personality paper has, while tiring the eyes and brain. Where is the tactile feel? Where is the fear of making mistakes, with it’s rush of having completed a flawless sentence?

On a blog, everything is set to maximum intensity. Slowly rising thoughts rarely carry the day when comparing them against the lastest sensation, the lastest marketing ploy. I cannot see myself following this path. I do not wish to pursue people, spamming search engines, participating in endless promotion ploys and plotting to make them want me. That’s a perverted kind of writing. I do use pronouns in the first person, but I never conceive of thoughts of writing before making sure that I become a mirror or as mirror-like as I can manage. As I sway under idealistic ideas, writing should leave a permanent mark upon the reader, as if he found something he knew and lost and now, after much toil, cherishes it vastly more.

After all, who am I? Am I not another human, with human ideas and desires? I’ve often glimpsed great ideas or the occasional transcendant thoughs and that is what is not human in me. I do hope to help those few out there, that have tired with the world of sensation and gratification, to show them what they could gain, following their own tune, help them recognize some of the stops along the way.

That’s why I’ll blog. Some of you, I hope, will enjoy the style and stay for the thought, much like a great view and a good restaurant go splendidly well together.

Leave a comment »

Welcome, oh noble traveller!

I’ve just taken possession of these pages. I hope to have more time to write here in a short while and keep you abreast of changes, travels and thoughts as they become relevant. Despair not that there is little to be seen; much is moving under the surface.

Until then, explore your world to the limit of your choices. And click on my favorite links.

Leave a comment »

Follow

Get every new post delivered to your Inbox.