Recently I had the “pleasure” of upgrading from CMSMS 1.9.3 to 1.10.3.

• Downloaded the cmsmadesimple-1.10.3-full.tar.gz
• Unzipped it overwriting some of the existing files from the older version (1.9.3) [tar -xvf cmsmadesimple-1.10.3-full.tar.gz -C my_existing_site_installation_folder]
• Ran the upgrade script by opening http://my-site.com/install/upgrade.php

I was constantly getting stuck at step 3, it was complaining:

Fatal error: Call to undefined method cms_config :: save () in /install/lib/classes/CMSUpgradePage3.class.php on line 30

Digging around a little bit realized cms_config is no longer available.

Then tried downloading cmsmadesimple-1.9.4.3-full.tar.gz

Luckily this time I was able to go past step 3 without any problem.

So now I was on version 1.9.4.3, but I wanted to get to 1.10.3. So

• As per their advice, upgraded all my modules to the latest version
• Downloaded cmsmadesimple-1.10.3-full.tar.gz,
• Copied its contents
• Tried to run the upgrade script.

Everything went fine, it even updated my database scheme to version 35 successfully. But then when I hit continue on step 6, it was stuck there for ever. Eventually came back with Internal Error 500. Looking at the log file, all I could see is

“2012/07/28 06:28:35 [error] 23816#0: *3319000 upstream timed out (110: Connection timed out) while reading response header from upstream”

Turns out that in 1.10, the CMSMS dev team broke a whole bunch of backward compatibility. In Step 6 of the upgrade, it tries to upgrade and install installed modules. But during this process it just conks out.

Then I tried to uninstall all my modules and run the upgrade script. Abra-kadabra the upgrade went just fine.

• Then I had to go in and install those modules again.
• Also had to update most of the modules to the latest version which is compatible with 1.10.
• And restore the data used by the modules.

Only had I known all of this, it could have saved me a few hours of my precious life.

P.S: Just when I finished all of this, I saw the CMSMS dev team released the latest stable version 1.11

Over the last 6 months, I’ve been blessed with various pharma hacks on almost all my site.

(http://agilefaqs.com, http://agileindia.org, http://sdtconf.com, http://freesetglobal.com, http://agilecoachcamp.org, to name a few.)

This is one of the most clever hacks I’ve seen. As a normal user, if you visit the site, you won’t see any difference. Except when search engine bots visit the page, the page shows up with a whole bunch of spammy links, either at the top of the page or in the footer. Sample below:

Clearly the hacker is after search engine ranking via backlinks. But in the process suddenly you’ve become a major pharma pimp.

There are many interesting things about this hack:

• 1. It affects all php sites. WordPress tops the list. Others like CMS Made Simple and TikiWiki are also attacked by this hack.
• 2. If you search for pharma keywords on your server (both files and database) you won’t find anything. The spammy content is first encoded with MIME base64 and then deflated using gzdeflate. And at run time the content is eval’ed in PHP.

This is how the hacked PHP code looks like:

If you inflate and decode this code it looks like:

• 3. Well documented and mostly self descriptive code.
• 4. Different PHP frameworks have been hacked using slightly different approach:
  • In WordPress, the hackers created a new file called wp-login.php inside the wp-includes folder containing some spammy code. They then modified the wp-config.php file to include(‘wp-includes/wp-login.php’). Inside the wp-login.php code they further include actually spammy links from a folder inside wp-content/themes/mytheme/images/out/’.$dir’
  • In TikiWiki, the hackers modified the /lib/structures/structlib.php to directly include the spammy code
  • In CMS Made Simple, the hackers created a new file called modules/mod-last_visitor.php to directly include the spammy code.
    Again the interesting part here is, when you do ls -al you see: 

    -rwxr-xr-x 1 username groupname 1551 2008-07-10 06:46 mod-last_tracker_items.php

    -rwxr-xr-x 1 username groupname 44357 1969-12-31 16:00 mod-last_visitor.php

    -rwxr-xr-x 1 username groupname 668 2008-03-30 13:06 mod-last_visitors.php

    In case of WordPress the newly created file had the same time stamp as the rest of the files in that folder

How do you find out if your site is hacked?

• 1. After searching for your site in Google, check if the Cached version of your site contains anything unexpected.

• 2. Using User Agent Switcher, a Firefox extension, you can view your site as it appears to Search Engine bot. Again look for anything suspicious.

• 3. Set up Google Alerts on your site to get notification when something you don’t expect to show up on your site, shows up.

• 4. Set up a cron job on your server to run the following commands at the top-level web directory every night and email you the results:
  • mysqldump your_db into a file and run
  • find . | xargs grep “eval(gzinflate(base64_decode(“

If the grep command finds a match, take the encoded content and check what it means using the following site: http://www.tareeinternet.com/scripts/decrypt.php

If it looks suspicious, clean up the file and all its references.

Also there are many other blogs explaining similar, but different attacks:

• Check Your Sites With Google
• WebShell Hack – An Update
• Understanding and cleaning the Pharma hack on WordPress
• WordPress Exploit: We Been Hit By Hidden Spam Link Injection
• Spam Link Injection Hacked (and How I Hopefully Fixed It)
• Google Cloaking Hack Targeting WordPress & How to Fix It
• How to Diagnose and Remove the WordPress Pharma Hack

Hope you don’t have to deal with this mess.

I really like CMS Made Simple. Its pretty neat if you want to set up a website using a CMS software.

However, after being hacked last week, I just tried upgrading to the latest version (1.9.4) of CMS Made Simple. I was happy to see the upgrade script ran fine and everything seemed to work. However there were couple of things that were broken.

I’m documenting them here, hoping it might be useful to others.

1) The old config.php file contained the following property

#If you're using the internal pretty url mechanism or mod_rewrite, would you like to
#show urls in their hierarchy?  (ex. http://www.mysite.com/parent/parent/childpage)
$config['use_hierarchy'] = true;

However in the latest version, when they migrate the old version of config.php, this property is dropped.

Some plugins like Blogs Made Simple rely on this property for creating pretty URLs for RSS feeds.

2) We’ve written some code which looks up the current page id in the $gCms variable.

We had used the following code to figure out the page id:

$smarty->_tpl_vars['gCms']->variables['page_id'])

However in the latest version of CMSMS this does not work. Instead had to change it to:

$smarty->_tpl_vars['page_id'])

At Freeset, we have two CMS Made Simple instances set up. One for development and another one for live environment. We do all our stuff on dev and then run the following shell script to deploy the latest changes from dev to live. This is a great example of “Do the Simplest Thing that could Possibly Work“.

#!/bin/bash
mysqldump --no-create-db --add-drop-table --skip-add-locks --skip-disable-keys --quick 
-u{db.user.name} -p{db.password} {live.db.name} --tables cms_module_blogs_commentprops 
cms_module_blogs_comments cms_module_blogs_comments_seq
 | mysql -u{db.user.name} -p{db.password} {dev.db.name} && mysqldump --opt -u{db.user.name} 
-p{db.password} {dev.db.name} | mysql -u{db.user.name} -p{db.password} {live.db.name}
 && rsync -rpl --delete --exclude=config.php {dev.deployment.dir}/ {live.deployment.dir}/

Over the last year, I’ve been helping (part-time) Freeset build their ecommerce website. David Hussman introduced me to folks from Freeset.

Following is a list of random topics (most of them are Agile/XP practices) about this project:

• Project Inception: We started off with a couple of meetings with folks from Freeset to understand their needs. David quickly created an initial vision document with User Personas and their use cases (about 2 page long on Google Docs). Naomi and John from Freeset, quickly created some screen mock-ups in Photoshop to show user interaction. I don’t think we spent more than a week on all of this. This helped us get started.
• Technology Choice: When we started we had to decide what platform are we going to use to build the site. We had to choose between customer site using Rails v/s using CMS. I think David was leaning towards RoR. I talked to folks at Directi (Sandeep, Jinesh, Latesh, etc) and we thought instead of building a custom website from scratch, we should use a CMS. After a bit of research, we settled on CMS Made Simple, for the following reasons
  • We needed different templates for different pages on the site.
  • PHP: Easiest to set up a PHP site with MySQL on any Shared Host Service Provider
• Planning: We started off with an hour long, bi-weekly planning meetings (conf calls on Skype) on every Saturday morning (India time). We had a massively distributed team. John was in New Zealand. David and Deborah (from BestBuy) were in US. Kerry was in UK for a short while. Naomi, Kelsea and other were in Kolkatta and I was based out of Mumbai. Because of the time zone difference and because we’re all working on this part time, the whole bi-weekly planning meeting felt awkward and heavy weight. So after about 3 such meetings we abandoned it. We created a spreadsheet on Google Docs, added all the items that had high priority and started signing up for tasks. Whenever anyone updated an item on the sheet, everyone would be notified about the change.
• User Stories: We started off with User Persona and Stories, but soon we just fell back to simple tasks on a shared spreadsheet. We had quite a few user related tasks, but just one liner in the spread sheet was more than sufficient. We used this spreadsheet as a sudo-backlog. (by no means we had the rigor to try and build a proper backlog).
• Short Releases: We (were) only working on production environment. Every change made by a developer was immediately live. Only recently we created a development environment (replica of production), on which we do all our development. (I asked John from Freeset, if this change helped him, he had mixed feelings. Recently he did a large website restructuring (added some new section and moved some pages around), and he found the development environment useful for that. But for other things, when he wants to make some small changes, he finds it an over kill to make changes to dev and then sync it up with production. There are also things like news, which makes sense to do on the production server. Now he has to do in both places). So I’m thinking may be, we move back to just production environment and then create a prod on demand if we are plan to make big changes.
• Testing: Original we had plans of at least recording or scripting some Selenium tests to make sure the site is behaving the way we expected it to. This kind of took a back seat and never really became an issue. Recently we had a slight set back when we moved a whole bunch of pages around and their link from other parts of the site were broken. Other than that, so far, its just been fine.
• Evolutionary Design: Always believed in and continue to believe in “Do the Simplest, Dumbest, thing that could Possibly work“. Since we started, the project had taken interesting turns, we used quite a lot of different JavaScript libraries, hacked a bit of PHP code here and there. All of this is evolving and is working fine.
• Usability: We still have lots of usability and optimization issues on our site. Since we don’t have an expert with us and we can’t afford one, we are doing the best we can with what we have on hand. We are hoping we’ll find a volunteer some day soon to help us on this front.
• Versioning: We explored various options for versioning, but as of today we don’t have any repository under which we version our site (content and code). This is a drawback of using an online CMS. Having said that so far (been over a year), we did not really find the need for versioning. As of now we have 4 people working on this site and it just seems to work fine. Reminds me of YAGNI. (May be in future when we have more collaborators, we might need this).
• Continuous Integration: With out Versioning and Testing, CI is out of question.
• Automated Deployment: Until recently we only had one server (production) so there was no need for deployment. Since now we have a dev and a prod environment, Devdas and I quickly hacked a simple shell scrip (with mysqldump & rsync) that does automated deployment. It can’t get simpler than this.
• Hosting: We talked about hosting the site on its own slice v/s using an existing shared host account. We could always move the site to another location when our existing, cheap hosting option will not suit our needs. So as of today, I’m hosting the site under one of my shared host account.
• Rich Media Content: We questioned serving & hosting rich media content like videos from our site or using YouTube to host them. We went with YouTube for the following reasons
  • We wanted to redirect any possible traffic to other sites which are more tuned to catering high bandwidth content
  • We wanted to use YouTube’s existing customer base to attract traffic to our site
  • Since we knew we’ll be moving to another hosting service, we did not want to keep all those videos on the server which then will have to be moved to the new server
• Customer Feedback: So far we have received great feedback from users of this site. We’ve also seen a huge growth in traffic to our site. Currently hovering around 1500 hits per day. Other than getting feedback from users. We also look at Google Analytics to see how users are responding to changes we’ve made and so on.
• We don’t really have/need a System Metaphor and we are not paying as much attention to refactoring. We have some light conventions but we don’t really have any coding standards. Nor do we have the luxury to pair program.
• Distributed/Virtual Team: Since all of us are distributed and traveling, we don’t really have the concept of site. Forget on-site customer or product owner.
• Since all of this is voluntary work, Sustainable pace takes a very different meaning. Sometimes what we do is not sustainable, but that’s the need of the hour. However all of us really like and want to work on this project. We have a sense of ownership. (collective ownership)
• We’ve never really sat down and done a retrospective. May be once in a while we ask a couple of questions regarding how something were going.

Overall, I’ve been extremely happy with the choices we’ve made. I’m not suggesting every project should be run this way. I’m trying to highlight an example of what being agile really means.