Leveraging Drupal: Getting your site done right - Workshop Central

Links

Hi there, my name is Victor Kane. I am a website application and development process engineer and I have written Leveraging Drupal: Getting your site done right to help people develop website applications using the free, Open Source Drupal CMS Web Application Framework without having to go through the steep learning curve I had to follow.

You can:

I am going to be working my way through the book, too, right along with you. And I will do the following:

General Feedback

I will be very grateful if you would leave general feedback on Leveraging Drupal on this page. Thanks!

Chapter 1 - Keeping It Simple

At the time of writing this chapter is available free: grab it (PDF) and share your opinion here!

Have you started your meme map yet? For more info on meme maps see http://oreillynet.com/pub/a/495 (We explain why we need them in Chapter 1).

Show yours here!

Chapter 2 - Taking Baby Steps

Creating an initial development environment, whipping up the initial prototype of the on-line Literary Workshop (website application to be developed chapter by chapter right through the book.

Install Drupal with CVS!

Installation

Very often, the wonderful Drupal security team comes up with a new security release of Drupal core itself.

When that happens, you want to upgrade relatively quickly (after waiting for 24 hours to make sure no-one is having problems...) you don't want to just copy the new version over the old, since it is possible that a single file is split into two, for example, or one is removed, and you don't want conflicting code hanging around in your file system. And you certianly don't want to be futzing around deleting stuff manually.

You want a clean, accurate and fast install every time.

Clean Original "Virgin" Install

(This assumes that nothing has ever been installed before. For a site running on a Drupal coreĀ  not originally installed with CVS see the next section.)

To do that, make sure you carry out original install of Drupal core via CVS, then it is just a matter of a simple CVS update command specifying the new Drupal release.

To checkout a specific Drupal release (here 6.8) to a new specific sub-directory which will become Drupal document root, here "www", execute the following:

$ cvs -z6 -d:pserver:anonymous:anonymous@cvs.drupal.org:/cvs/drupal co -d www -r DRUPAL-6-8 drupal

Executing this statement on the command line will create a new Drupal document root just below the current directory, called "www". If I omit "-d www", then it will create a new Drupal document root called "drupal".

Just visit drupal.org homepage to see which is the current release, and which releases you should be upgrading to.

What if I am running a site where Drupal core was not checked out with CVS?

Well, a lot of us have sites we wish Drupal core had been installed via CVS, so we could just carry out the upgrade instructions in the following section. However, if the Drupal core was simply untarred from an installation tarball or FTP'd into place, you can neither carry out the upgrade, nor can you safely checkout anything (CVS won't overwrite stuff, and some new files will come down, leading to a big mess and unpredictable results).

You are going to have to futz around deleting the existing core very carefully, and then do the checkout into a blank space - but at least it will be the last time! The following would typically be the procedure to follow:

  1. Make a complete backup of the site (including SQL dump of the database). That way if this procedure doesn't work for you, you have a safe path backwards.
  2. Especially keep a handy copy .htaccess, robots.txt, default/settings.php and any other core file you might have modified or which needs special handling in your situation.
  3. Log in as user #1 (full permissions).
  4. Physically delete all of Drupal core (the subdirectories, and the document root files like update.php, etc., but excluding any custom or asset files, and excluding the directories under sites that contain contribution and custom modules and themes).
  5. Go to one step above document root, and check out the current stable version of Drupal from CVS as shown above.
  6. Make sure your custom information is present in default/settings.php
  7. Run the spanking new update.php

The ideal with default/settings.php would be:

  1. Move default/settings.php to a safe place, out of the Drupal file system after step 4.
  2. Merge in your custom settings.php content (especially the database access info) to a copy of the new settings.php (Drupal 5.x) or copy the new default.settings.php to a new settings.php and merge your custom settings into that file. Then run update.php

Upgrading to new release

However you got here, your Drupal core is a working copy of a CVS tagged release of Drupal. So you can now enjoy the benefits of being able to upgrade with a simple checkout statement for the remaining life of the site!

This is to be executed within document root! That is, if I had created a new Drupal install at document root "www", I would change directory to www and then executing the following will upgrade to Drupal 6.9:

$ cvs update -dPr DRUPAL-6-9

If you omit "r{release tag}", in this case "DRUPAL-6-9", you will check out HEAD, which you won't be wanting to do on a production site!

Simple, clean, efficient, easily reversible: you can't beat it.

My own version of Tim Millwood's "Streamline Drupal CVS checkout" script

I just changed two things (I am on Ubuntu with bash shell) in relation to Tim Millwood's excellent bash script (see http://millwoodonline.co.uk/streamline-drupal-cvs-checkout ).

1) Put the functions in .bashrc instead of .profile so I could see the changes immediately (.profile gets loaded via login, not just by opening a new terminal screen).

2) Modified the function getmod() to be able to check out contrib mods to a sensible directory. Since the drupal checkout creates sites/all but not sites/all/modules or similar, so there is no CVS meta info if I create the sites/all/modules directory and try the function getmod there. It won't work. So I modified getmod() so that I could execute all commands right from the Drupal document root directory. That way, I could do the following

~/tmp$ mkdir drupal
~/tmp$ cd drupal
~/tmp/drupal$ getdrupal 6-10
...

and then, without changing directories, do:

~/tmp/drupal$ getmod cck 2-2
cvs checkout: Updating sites/all/modules/contrib/cck
U sites/all/modules/contrib/cck/CHANGELOG.txt
U sites/all/modules/contrib/cck/DEVELOPER.txt
U sites/all/modules/contrib/cck/README.txt
U sites/all/modules/contrib/cck/UPGRADE.txt
...

Here is my modified function getmod() which specifies the "sensible" contributed modules directory that I always use these days. Notice that CVS creates all the intervening directories and inserts CVS meta data in each one.

function getmod () {
MODULE="${1}";
VERSION="${2}";
cvs -z6 -d:pserver:anonymous:anonymous@cvs.drupal.org:/cvs/drupal-contrib co -r DRUPAL-6--$VERSION -d sites/all/modules/contrib/$MODULE contributions/modules/$MODULE;
cd sites/all/modules/contrib/$MODULE;
getrevs $MODULE.module
}

The complete listing as I use it in .bashrc:


function getmod () {
MODULE="${1}";
VERSION="${2}";
cvs -z6 -d:pserver:anonymous:anonymous@cvs.drupal.org:/cvs/drupal-contrib co -r DRUPAL-6--$VERSION -d sites/all/modules/contrib/$MODULE contributions/modules/$MODULE;
cd sites/all/modules/contrib/$MODULE;
getrevs $MODULE.module
}
function getrevs() {
cvs log "${1}" | egrep 'DRUPAL-.*:' | sort
}
function updatemod(){
cvs update -r DRUPAL-6--"${1}" -dP
}
function getdrupal(){
FOLDER=${PWD##*/};
cd ..;
cvs -z6 -d:pserver:anonymous:anonymous@cvs.drupal.org:/cvs/drupal checkout -d $FOLDER -r DRUPAL-"${1}" drupal;
cd $FOLDER
}
function updatedrupal(){
cvs update -r DRUPAL-"${1}" -dP
}

Chapter 3 - Getting Organized

Enhancing the initial development environment with a Subversion source and assets repository, plus a Trac instance.

Growing the on-line literary workshop website application prototype with Drupal itself... implementing business objects... deployment from development workstation to test site... getting feedback from the client.

Important Errata p. 81

The entries in the third column of the table (Primary menu) should not be capitalized! They should be "blog", "forum", "book", "node/add/application".


Another Errata p. 111

Thanks also to drupalista Alan Page, who spotted another typo in one of the commands during the discussion of the .cvsignore file on page 111. The command listed to create the file uses the cat command:

$ cat sites/default/settings.php > .cvsignore

Should be the ls command instead:

$ ls sites/default/settings.php > .cvsignore

Problem of teams needing to merge databases

There is an interesting thread at the Wrox P2P Forum on how to merge developoment, configuration and content efforts among teams working with multiple database for the same site.

See http://p2p.wrox.com/book-leveraging-drupal-getting-your-site-done-right-...

The two questions posed are:

Currently, I'm in chapter 3, and I have a couple of questions about the way you've handled storing a dump of the database in the repository.

1. What happens if you have multiple developers working on a project and they each install different modules? Won't there be a conflict between their database dumps?

2. What happens when you've deployed the site and the end user is happily adding content, but you continue to develop the site? Later down the road, your database will have new module configurations, but the production database will have lots of new content. How do you merge the configuration part of the development into a live site without loosing data?

Excellent questions.

My answer:

You hit the nail on the head

First of all, thanks so much for your praise. Basically, in writing the book I have tried to achieve just that, the possibility of folks not having to go through a steep learning curve by putting everything you need in a single place.

Your two questions point to a huge difficulty in Drupal architecture, namely that of the configuration, both in terms of things like blocks and module installation, being mixed with content in the database in such a way that it is no trivial task to "merge" similar databases in the way you describe.

On the one hand, there are possible solutions, that need to be digested and adapted to your process. On the other, there are integration and build strategies you must set up in your team.

Regarding possible solutions, every now and then in the Drupal community, solutions have been promised, but there has never been a straightforward, practical solution that just works.

Reading and ways forward on this topic:

1. http://ceardach.com/blog/2009/01/lea...and-deployment
2. Also see Kathleen's Database scripts, published on Drupal, which may very well be the answer to your question, but it needs to be digested and adapted to your process: http://drupal.org/project/dbscripts;
3. Greg's deploy module: http://drupal.org/project/deploy
4. The Aegir group of modules, with hosting, provisioning and site install/deploy scripts: http://groups.drupal.org/aegir-hosting-system
5. Other solutions that have been suggested over the years, like incrementing the node counter by odd or even increments on development and production machines, and the like.
6. Installing content remotely via protocols such as REST and XML-RPC (blog api and the services module) allow some separation of configuration and content. See my own article: http://awebfactory.com.ar/node/297

So, the other side of the coin is that a team strategy has to be developed, and it involves an "integration moment". That is, individual developers, who may use the SVN repository to branch, must each be invited to merge.

This may not be as painful a moment as might be imagined. By branching the code, that is, the Drupal instance, a developer may be able to abstract their work into a module, including code-based views as .inc files (dealt with in Chapter 14), where the database configuration can be dealt with as an install script that accompanies the module. Installation profile resources (also dealt with in Chapter 14) are another method, especially taking into account modules that export database configuration to scripts.

This is as realistic an answer as I can give you at this point in Drupal history.

On a practical level, you may simply need more than ever to work on a test site, figure out what you want to do, then repeat that work on the production site where the content is held.

We can explore this in more detail, as need be, and you are invited also to join in the Leveraging Drupal Workshop Central, by leaving comments, which is just getting underway (there is a separate child page for each chapter): http://awebfactory.com.ar/node/348#workshop

I will place the gist of this discussion under Chapter 3.

Thanks again for your feedback.

Victor Kane

Chapter 4 - Getting up to Speed

Finishing the user stories with the client... allocating user stories to project iterations... working on the architectural baseline... thinking about your team...

Where are all the user stories mentioned in the chapter?

You can find all the user stories written up in the code for Chapter 11.

Just install it and click on the left-hand side user stories link (logged in as dev/dev).

Chapter 5 - Finishing up the Elaboration Phase

The Elaboration Phase is a series of iterations with certain characteristics, having to do with going beyond the prototype (Inception) phase.

Here we are concerned with enhancing the development environment (how to easily update Drupal releases, especially security releases, and how to update modules easily and cleanly. And we move right along with the implementation of the functionality required by the on-line Literary Workshop.

11 July 2009 - Important note on drush -- please read:

Drush is now a standalone script, you install it once and use it on all the Drupal installations you have on the system

See the following:

Use it!

Chapter 6 - Pushing the Envelope

In Construction Phase, it's mainly about implementation... but also, how do we use Drupal itself to manage our project using the agile approach (user stories, tasks, acceptance tests...) with the external help of SVN and Trac, of course...

So we make a separate private group, and the on-line literary workshop becomes self-documenting...

And we are in full throttle with CCK and Views, and the Organic Groups modules.

Chapter 7 - Becoming an Expert Drupal Themer

But at the end of this chapter, the on-line Literary Workshop still looks ugly as sin (and will until Chapter 11)... what are you talking about, Victor Kane?

Yep, even though the site still looks ugly as sin, we can still learn what it takes to be a theming expert with this chapter... find out why...

Actually, find out what Drupal theming is all about...

Chapter 8 - Upgrading to Drupal 6

Join me as I upgrade this very blog - http://awebfactoyr.com.ar - to Drupal 6 back in August of 2008.

Chapter 9 - Ugrading to Drupal 6 Revisited

We examine all the steps involved, glean best practices from just about everyone, and then plunge in to update our on-line literary workshop website application to Drupal 6, complete with Views, CCK, Organic Groups and what have you.

Chapter 10 - Installing and Using Drupal 6 Fresh out of the Box

Let's take a breather from the on-line literary workshop... and develop a Drupal 6.x multi-lingual (l10n, i18n) translators studio online, where clients can post work, translators can accept work, do it, upload it, a real online translation factory, all in a single chapter.

Chapter 11 - Full Swing Agile Approach to Drupal Development

Honing our agile tools right from within the self-managing and self-documenting website application project!

Implementing the user stories, one by one (affinity groups, online publications, on-line blogs, and... adapting an XHTML/CSS theme to Drupal 6.x!

Chapter 12 - The JQuery Chapter

Enough said.

How did you find it?

Chapter 13 - From Drupal 6 to Drupal 7 and Beyond

What's going on with Drupal?

What was new in Drupal 6.x and why you should definitely move on from Drupal 5.x in your projects.

An in-depth look towards what's up with Drupal 7.

I even tried to make a minimalistic version of the on-line literary workshop in Drupal 7, but when the book was written, CCK in core was not yet usable.

Let's do the full port right here in the workshop!!!

Volunteers?

Chapter 14 - Deploying your Application as a Drupal Installation Profile

First we leverage the fantastic advanced help module (marvelously abstracted out of Views 2 by Earl Miles) for our own purposes. Then we see about using the Drupal installation Profile as a method for deployment.

Then, we abstract out the Views into the litworkshop Module code itself, as .inc files! Cool!

Here in the workshop, as Earl MilesĀ“ new CTools a.k.a the Chaos tool suite develops, including the Exportables tools, this will only shine brighter and brighter... real drupalistas prototype with the Views UI, then code!

I expect we will hone all of this greatly here in the workshop, as the method keeps on getting better and better for Drupal 6!

Chapter 15 - Acquia Drupal as an Enterprise-Ready Alternative

Trying out Acquia Drupal, installation, updates, the new Acquia Marina Theme, Mollom.

And about becoming a part of the Acquia Partner Program.

As time goes on (and by the time the workshop gets to this last chapter in the book), I expect there will be a lot of new developoments along these lines...