Busy, Busy

Water under the bridge

Realized been so busy actually working not really had much of say on the blog. So whats the I learned.

1) Jruby v.s. Ruby ~ same performance but 2Gb extra ram eaten be Java
2) Hate complex java web service soap security layers
3) Mocking and getting solid contracts between systems on integration really import

For a rails application the change from a installable source code, to a packaged and documented linux rpm that can be installed by none rails people is a lot of work. Still have to for a sales man proof windows msi installation  of whole suite. Have got close may times, but the proof bit really hard.

Rails 3.x

Afraid 3x versions of biorails and a lots of changes in development team since last post. At present looking at  5.x plan and move up to rails 3.x. Learned the from last time and let 3.x , drivers ,server etc really settle before looking at moving up.

Key change will be move some on the 3500 automated tests from test unit to rspec, thanks too Chris. Also learn a lots about factory girl to build nests of objects for testing. Key problem here has been testing complex components like automated cascaded of requests which need lots of data.

Still looking at whether to build on fly from factory or have a test database populator. 

More 2.3.2 Upgrade issues

Lots of ActiveRecord::StaleObjectError errors

Ok took some time clearing these. The basic problem was with large complex trees of objects resulting in multiple paths destroying the same objects. For example a 3 level process <-- context <-- parameters. As a handy shortcut process has a collection of parameters and contexts. The delete was cascading down via both process->context->parameter and process->parameter. This resulted in a attempt to delete a record twice.

Methods of fixing these problems:-

• look carefully and change see if :delete_all raw delete can be used for dependent records.
• look a collection and see if :destroy options can be removed
• monkey patch to activerecord/lib/active_record/locking/optimistic.rb line 126 so only errors if real problem with stale record, allow duplicate deletes.
  

My little patch of Active Record to the last few problems as the following:-

unless affected_rows == 1
raise ActiveRecord::StaleObjectError, "Attempted to delete a stale object"
end

Changed to add test for already delete record and say which object actually had the problem :-

if affected_rows != 1 and self.class.exists?(self.id)
raise ActiveRecord::StaleObjectError, "Attempted to delete a stale object #{self.class}[#{self.id}]"
end

Rails 2.0 -> 2.3.2 Upgrade

General the upgrade was not a major problems most of the application work. Have about 25 or 2500 tests failing, mainly with stale record on delete for lots of controller and unit tests of destroying object. The main issues where as follows:-

Cleanup
Had a few problems with plugins causing problems after update of rails:-

• plugin/engines needed to be deleted,now part of rails
  
• plugin/arts needed to be deleted

Also had a few renames of Dependencies.load_paths to ActiveSupport::Dependencies.load_paths from the 2.1+ days to sort out

Testing
This caused a number of problem due to widespread use of the Test::Unit::TestCase in the code. These had to be changed a number of large search and replaces in the test tree.

• unit tests replace Test::Unit::TestCase with ActiveSupport::TestCase
• function tests Test::Unit::TestCase with ActiveController::TestCase

Database Migrations
Needed a little hack to move development database to rails 2.3.2. Needed to fill the schema_migrations with old style version numbers as follows:-

1.upto(391){|i|DataConcept.connection.execute "insert into schema_migrations (version) values(#{i})"}

Todo List

• Need to work on destroy test failures, to see if bad code or test under 2.3.2
  
• gloc plugin does support new internationalization API will have to be changed
• Need to test modified action web services plugin with 2.3.2
• Need to check with ruby 1.9.1

Starting on Biorails 4.0

Boy been a busy year of delivery and going into production on a number of sites. Have not had time to write for ages. Now starting the majors upgrades to Biorails 4.0 cycle. Currently looking and ruby 1.9.1 and rails 2.3.2 updates.

First three rails things todo are:-

1) Update to Rails 2.3
2) Update Action Web services to work
3) Update Oracle drivers to support bulk inserts
4) Check gems on ruby 1.9.1

On the application side

1) Redo the unstructured data management. Time to make the product a better authoring tools
2) Support a lot better printable output in Open Office,word and PDF formats
3) Enhance the recipes to allow creation of a multiple level tree of items from a recipe
4) Time for UI make over again

Synergy on ubuntu 8.04 , freeze up fixed.

Synergy has been my long term tool for sharing keyboard and mouse between my development and build machines. Unluckily with ubuntu 8.04 there has been a lots of 4-5sec freeze up problems. In the end moved to 2x keyboards and mice again.

Now this is fixed with some open googling and some source patching, as follows.

1) download 1.3.1 source code from http://synergy2.sourceforge.net/
See http://downloads.sourceforge.net/synergy2/synergy-1.3.1.tar.gz

tar -xzf synergy-1.3.1.tar.gz

2) Patch with fix from lanchpad

See. https://bugs.launchpad.net/ubuntu/+source/linux/+bug/194029

wget http://launchpadlibrarian.net/13995863/synergy_diffs
patch -p1 < synergy_diffs

3) Install needed extra libraries (XTest)

sudo apt-get install libxtst-dev

4) rebuild and install

cd synergy-1.3.1
./configure -prefix=/usr
make
sudo make install

Restart synergy all works smoothly again. Can get space for coffee cups back again.


Note: Plus 12hours some problems with synergy going wild and eating 100% cpu so not a full solution looks like I have to review whats happening. Not much betters solution the the run with sudo method I used before.

Biorails 3.0

Ok have merged and release 3.0 code onto core. The http://biorails.org/svn/biorails/core/trunk contains 3.0 code release now. All that is needed is lots of web site updates and packaged installations of open source version. Afraid followed the money can concentrated on enterprise version for last few months.

Have done packaged windows and linux installation of biorails for enterprise version. Now need to do packaged open version on windows for people to play with.

Getting to the full up in 5 minutes up and working experience is a hard for a web application. Almost there now with windows thanks to sqlite as a database. So think first release with a standalone demo version on windows.
Some more work needed for linux versions as apache front end still needs to be hand edited.

Oracle 11g /Ubuntu update

Ok found the magic option to speed up oracle 11g for my development box.

emctl stop dbconsole

Now converted back to simply database server its works a lot better. Afraid after some time looking a performance stats found could not find my application in top SQL . The enterprise console performance stats only showed enterprise console eating all the resources.

Database on its own appears to be comparable but a little larger then XE. Afraid think in case of oracle, enterprise may still mean crew of 100's and something the site of a aircraft carrier to run it at full potential. But that what its aimed at the 1Tb+ database not developer/test systems.

Think I may go back to XE one tested everything works with 11g as expected.
Fun next as time to see how stacks up against Sqlite, Mysql and Postgres in running the all the my test suites.