There are a lot of unresolved forum posts about this … if you’re using an older video card, you just need to uninstall the new Nvidia drivers and re-install the legacy drivers:

apt-get remove --purge nvidia-current
apt-get install nvidia-173

Lucid ships with php 5.3, the newest version of php. Unfortunately a number of popular open source content management systems do not yet run on 5.3. This was also an issue when I upgraded my mac to Snow Leopard. In both cases, php 5.3 must be downgraded to 5.2.10 (you could choose to run both versions too, but I’m not going to address that here).

Setting up a Lucid server on SliceHost

on Slicehost, if you’re willing to wipe out everything, you can rebuild a clean slice for Lucid, which is what I chose.

1. upgrade your kernel to 2.6.32-16.25
2. rebuild slice with Lucid

As soon as the build is complete, you need to ssh in, and

1. change the root password

   passwd

2. create a sudoers group and create and add a user to it

   groupadd sudoers
   visudo

   add

   %sudoers ALL=(ALL) ALL

   create user with sudo privileges

   useradd yourUserName
   usermod -a -G sudoers yourUserName

3. edit ssh configuration to disallow root login

    vi /etc/ssh/sshd_config 

   find

    PermitRootLogin yes

   and set it to

   PermitRootLogin no

   add

   AllowUsers yourusername

4. setup IPtables
   I used http://articles.slicehost.com/2010/4/30/ubuntu-lucid-setup-part-1
5. reload ssh

   /etc/init.d/ssh reload

6. Add additional sources to sources.list. See sources list generator for more sources lists.

   vi /etc/apt/sources.list

7. Set to use bash

   dpkg-reconfigure dash

   select “no”

8. synchronize date

   apt-get install ntp ntpdate

9. Install tools for compiling

   apt-get install build-essential

10. if you’re a vi user, install vim-nox (hint: I used to install vim-full, but that’s now deprecated in Lucid)

    apt-get install vim-nox

Set up LAMP server

1. If you don’t need to compile from source, you can use one of Ubuntu’s 1-liners for installing a LAMP server.

   tasksel1
   or
   1apt-get install lamp-server^

   I used the latter. This will return the following:

   apache2 apache2-mpm-prefork apache2-utils apache2.2-bin apache2.2-common libapache2-mod-php5 libapr1 libaprutil1 libaprutil1-dbd-sqlite3 libaprutil1-ldap libdbd-mysql-perl libdbi-perl libexpat1 libhtml-template-perl libmysqlclient16 libnet-daemon-perl libplrpc-perl mysql-client-5.1 mysql-client-core-5.1 mysql-common mysql-server mysql-server-5.1 mysql-server-core-5.1 php5-common php5-mysql psmisc
   Suggested packages:
   www-browser apache2-doc apache2-suexec apache2-suexec-custom ufw php-pear zbishell  libipc-sharedcache-perl tinyca mailx php5-suhosin
   The following NEW packages will be installed:
   apache2 apache2-mpm-prefork apache2-utils apache2.2-bin apache2.2-common libapache2-mod-php5 libapr1 libaprutil1 libaprutil1-dbd-sqlite3 libaprutil1-ldap libdbd-mysql-perl libdbi-perl libexpat1 libhtml-template-perl libmysqlclient16 libnet-daemon-perl libplrpc-perl mysql-client-5.1 mysql-client-core-5.1 mysql-common mysql-server mysql-server-5.1 mysql-server-core-5.1 php5-common php5-mysql psmisc

2. test apache
   http://www.domainname.com/
   should return “It Works!”
3. test php

   vi /var/www/test.php
   <php phpinfo(); ?>

4. reload apache

   /etc/init.d/apache2 restart

   navigate in browser to http://www.domainname.com/test.php

5. check that mysql is bound to localhost

   cat /etc/mysql/my.cnf | grep bind-address

   should return

   bind-address	=127.0.0.1

   If it doesn't, edit /etc/mysql/my.cnf with correct address

Ok! everything works as planned. Two small tasks remaining: We need to downgrade php from 5.3 to 5.2.10 and install more php libraries.

Downgrading Lucid from PHP 5.3 to 5.2

There are several methods floating around out there, which all use the same idea.

• MrKandy wrote a good script
• Nck Veenhof
• the jibe blog. I used the bash script provided in the comments on this blog post because it was the most succinct.

For future use, I created this bash script

You can either execute that script as a bash script or execute it within the shell. I initially executed it within the shell, then later decided to save it as an executable script for future use.
To execute as a script

1. create a directory somewhere you'd like to store scripts. It could be ~/home/username/scripts. Copy the bash script above to your scripts directory, name it someName.sh, and make it executable

   chmod ugo+x

2. execute the script

   sudo ./someName.sh

3. sudo apt-get update

4. sudo apt-get install $php_packages

Essentially, the way it works is it finds all the php 5.3 packages and removes them, then uses sed to replace "lucid" with "karmic" in the sources list and saves that in /etc/apt/sources/preferences.d which is iterated over first during update. It then pins the karmic sources for php packages so that the php version isn't inadvertently updated during update/upgrade.

Next step for me was to install some more php packages since the initial install was so minimal and I wiped it out anyway.

apt-get install php5-cli php5-curl php5-imagick php-pear php5-sqlite php5-xmlrpc php5-xsl

and finally, I like to use phpmyadmin for managing multiple databases:

apt-get install libapache2-mod-auth-mysql phpmyadmin

select [*] apache 2 by hitting the space bar
If this is a fresh install, select yes when asked to configure dbconfig-common

enter the mysql root password created earlier
enter a password for the phpmyadmin admin account

navigate in a browser to http://www.domainname/phpmyadmin

Why didn't I just not install 5.3 and build 5.2 in? Because I think it's actually faster this way. For me at least. I install defaults, test that apache and mysql work as expected, downgrade to 5.2, install remaining php packages. Fairly straight-forward. If only it had been so easy on my mac.

Will I do this on my production servers? Probably not for php applications. Ubuntu 8.04 is supported for another year and 9.04 is supported for 2 years. There's no hurry. I will, however, probably go ahead and use 10.04 for Rails, Django, and Plone installations.

Update: Rolling Back

3/17/2011
To roll this back, do the following:

1. list out the php modules installed to compare

   dpkg -l | grep php

2. archive outside of sources directory or comment out /etc/apt/sources.list.d/karmic.list
3. archive /etc/apt/preferences.d/php outside of the sources directory

4. apt-get update
   apt-get upgrade

5. I manually reinstalled php5 and modules even though apt-get should update should update them:

   apt-get install libapache2-mod-php5  php5-common php5-mysql php-pear php5-suhosin libapache2-mod-php5 php5-curl php5-imagick php-pear php5-sqlite php5-xmlrpc php5-xsl

6. I had to remove the "#" mark from imagick.conf and cli.conf - apparently that's been deprecated as a comment symbol
7. if you're running WordPress, don't forget to edit php.ini with the date/time zone. Initially I got an internal server error after restoring php5. Fixing the date/time seemed to do the trick.

Why?

I was installing an application for a friend and needed to install ffmpeg packages. Unfortunately, when I ran apt-get update, I received the below error:

APT::Cache limit ...
.... Dynamic MMap out of room! ...
Problem with MergeList /var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_dapper_main_binary-i386_Packages
E: The package lists or status file could not be parsed or opened.

The default Apt Cache limit was too low for all the sources (Lucid and Karmic). I stepped through the troubleshooting procedure for packages, upping my apt cache to 200000000 which is quite high and the error disappeared. In the end I decided to just keep it tidy and rollback. All of the php applications I'm running work fine on 5.3.

In the apache error log, the first error to confront me was this:

 WordPress database error Illegal mix of collations
(latin1_swedish_ci,IMPLICIT) and (utf8_general_ci,COERCIBLE) for operation '=' for query SELECT comment_ID FROM wp_comments WHERE comment_post_ID = '2933' AND comment_approved != 'trash' AND  .... etc

Collation conflict …. I looked at the database structure, sure enough, there’s a mix of

 utf8_general_ci and latin1_swedish_ci 

. Most of the default WordPress database tables are Latin1 and it looks like the newer tables and all of my plugins are UTF 8.

Looking at WordPress installs that I haven’t upgraded yet, all of them are set to UTF 8. But these are newer installs. The WordPress installation that has this problem has been in place since version 1.* and WordPress used to default to Latin 1 encoding. The UTF 8 tables are newer.

So, in other words, if you’re maintaining a WordPress install that’s been around for a while, you probably also have this issue in your database.

Now, to get this particular database fixed up …
One suggestion I came across was to export the schema and data separately then recreate the database with UTF 8 charset and reimport the data. However, that will likely break the existing content because the content has Latin 1 characters which will cause the newly minted UTF tables to choke a little.

… When converting the character sets, all TEXT (and similar) fields are converted to UTF-8, but that conversion will BREAK existing TEXT because the conversion expects the data to be in latin1, but WordPress may have stored unicode characters in a latin1 database, and as a result, data could end up as garbage after a conversion!
ref

The better way is to run an ALTER tables query. The steps are:

1. backup everything up
2. ALTER all TEXT and related fields to their binary counterparts using the SQL statements generated below
3. alter the character set
4. change the binary data type fields back to TEXT
5. Add DB_CHARSET and DB_COLLATE definitions to wp-config.php

Using MySQL’s information_schema to generate the actual ALTER tables statements needed, I wanted to pipe out the statements to a reusable text document so I could do a test run on a copy of the problem database first. Generating the ALTER tables statements using information_schema will then include whatever extra tables or rows have been added by plugins.

The following 4 queries are lifted directly from Haidong Ji via the WordPress Codex. I added the pipe out to text file. Note that the path to mysql executable is on Ubuntu. Your path might be different, for example, another common location might be /usr/local/mysql/bin.

1. ALTER all TEXT and related string field types to binary field types counterparts using the SQL statements generated below. The list of conversions being made is as follows:

CHAR -> BINARY

VARCHAR -> VARBINARY

TINYTEXT -> TINYBLOB

TEXT -> BLOB

MEDIUMTEXT -> MEDIUMBLOB

LONGTEXT -> LONGBLOB

• echo "SELECT CONCAT('ALTER TABLE ', table_name, ' MODIFY ', column_name, ' ', REPLACE(column_type, 'char', 'binary'),';') FROM columns WHERE table_schema = 'testblog' and data_type LIKE '%char%';" | /usr/bin/mysql -u<em>user</em> -p<em>password</em> information_schema > /home/<em>userdir</em>/char2binary.txt

• echo "SELECT CONCAT('ALTER TABLE ', table_name, ' MODIFY ', column_name, ' ', REPLACE(column_type, 'text', 'blob'),';') FROM columns WHERE table_schema = 'testblog' and data_type LIKE '%text%';" | /usr/bin/mysql -u<em>user</em> -p<em>password</em> information_schema > /home/<em>userdir</em>/text2blob.txt

• echo "SELECT CONCAT('ALTER TABLE ', table_name, ' MODIFY ', column_name, ' ', column_type, 'CHARACTER SET utf8;') FROM columns WHERE table_schema = 'testblog' and data_type LIKE '%char%';" | /usr/bin/mysql -u<em>user</em> -p<em>password</em> information_schema > /home/<em>userdir</em>/char2utf.txt[/code]</li>
  	<li>1echo "SELECT CONCAT('ALTER TABLE ', table_name, ' MODIFY ', column_name, ' ', column_type, 'CHARACTER SET utf8;') FROM columns WHERE table_schema = 'testblog' and data_type LIKE '%text%';" | /usr/bin/mysql -u<em>user</em> -p<em>password</em> information_schema > /home/<em>userdir</em>/text2utf.txt

2. Change the default character set of the database from Latin 1 to UTF 8. This will ensure all new tables created are UTF 8, but doesn't affect existing tables (which is fine since we've already changed those above, right?).
This can be done either using phpmyadmin (select database -> click the Operations tab -> select UTF 8 from dropdown menu at bottom) or in your shell

ALTER DATABASE MyDb CHARACTER SET utf8;

3. change the binary data type fields back to TEXT
This can get sticky if your original structure had datatypes that ... were supposed to be binary or blobs. I looked through my database before altering it and did not find any blobs or binaries. But this could vary depending upon plugins used, etc. Changing the binaries back to TEXT essentially involves running the above sql scripts in reverse.

• echo "SELECT CONCAT('ALTER TABLE ', table_name, ' MODIFY ', column_name, ' ', REPLACE(column_type, 'binary', 'char'), ';') FROM columns WHERE table_schema = 'testblog' and data_type LIKE '%binary%';" | /usr/bin/mysql -u<em>user</em> -p<em>password</em> information_schema > /home/<em>userdir</em>/rev-binary2char.txt

• echo "SELECT CONCAT('ALTER TABLE ', table_name, ' MODIFY ', column_name, ' ', REPLACE(column_type, 'blob', 'text'), ';') FROM columns WHERE table_schema = 'testblog' and data_type LIKE '%blob%';" | /usr/bin/mysql -u<em>user</em> -p<em>password</em> information_schema > /home/<em>userdir</em>/rev-blob2text.txt

4. Add DB_CHARSET and DB_COLLATE definitions to wp-config.php

references

wordpress article on changing collation

article by alex king

1. migrate from it’s ancient server to a more modern server
2. upgrade to Plone 4

I’m migrating away from a 32 bit Red Hat 7(!) server to a 64 bit Ubuntu Lucid. The reasons should be obvious from that statement alone, but in short: Red Hat 7 reached its end-of-life years ago. It’s stable (so far) but … for how long?

The server environment I’m migrating away from is:
- Server: Zope/(Zope 2.7.3-0, python 2.3.3)
- ZServer/1.1
- Plone/2.0.3

I’ll also be upgrading from Apache 1.3 to Apache 2 and considering a switch from Squid to Varnish for caching

Getting started: Migrating from Red Hat 7.3 to Ubuntu 10

Zope is an “in-place” installation, which means it runs from the directory that you put it in. This is nice and simple, makes it easier to run different versions of Zope on the same server with associated different versions of Python and Plone and, theoretically it should be easy to move to another system altogether.

My first pass, I rsynced the Zope directory (which contained sources for Zope, Plone, and Python) over from the old server to the new one and tried to compile Python and Zope to see what would happen. Unfortunately, when I compiled Python, I got a bunch of errors that indicated my version of Python was optimized for a 32 bit system and … the new server is 64 bit. This error looks like this:

 ' Parser/tokenizer_pgen.o' is incompatible with i386:x86-64 output

Python 2.3.7 was the oldest version that I could get running on the new VM that didn’t throw 32 bit errors. I briefly considered re-compiling Python 2.3.3 with -m32 option to GCC and I could have spent some time getting 2.3.3 running but 2.3.7 was a bugfix release and the most stable of the 2.3 releases and should work fine so I chose to save time by going with it. We’ll see if I regret that later. I was also reminded in the course of this process to clean up my mess with

$ sudo make clean

When I compiled Python, the following error occurred: Can't locate Tcl/Tk libs and/or headers, so I had to install the tcl/tk libraries (added to setup below)

Plone/2.0.3 was hard to find, although I was able to locate a copy of Plone/2.0.5. I don’t have a development server to upgrade the website to 2.0.5 so if I have to use it, I may have to keep my fingers crossed. Since Plone is installed as a Zope Product and doesn’t require compilation, I will first try to copy over the Products directory of the website and see if it’ll just work.

I will also need to install PIL and PyXML

When I compiled Zope the first time, I got error messages that zlib was missing. It turns out that Ubuntu ships with zlib, but not zlib-dev installed. Also, when researching that error, I came across a comment that I would need to install system support (libjpeg62) for PIL so those 2 lines have been added to the setup below.

Initial Server Setup

update and install basics

sudo apt-get install update
sudo apt-get upgrade
make sure gcc is up to date
sudo apt-get install build-essential
sudo apt-get install zlib1g zlib1g-dev libjpeg62 libjpeg62-dev tcl tcl-dev tk tk-dev vim-nox lynx

#note on Ubuntu 10.10 vim-full has been replaced by vim-nox. I also installed libfreetype6-dev (libfreetype6 was already installed, but install it if it isn't)

update the locate db so you can find stuff

$sudo updatedb

change root password
$sudo passwd

create a user with sudo privileges:
$ useradd -m -c "real name" -s /bin/bash chris

The flags create a home directory with skel profile defaults and this account is linked to the defaults for bash shell, which is important if you like syntax coloring and tab completion.
If you haven’t created an admin group with sudo privileges, you can add sudo privileges to the individual user

$passwd chris
$sudo visudo

Find the line that says #User Privilege specification and add

chris ALL=(ALL) ALL
shift ZZ to save and exit

or just add the new user to the admin group
sudo usermod -G admin chris

create zope user
$useradd -m -c “zope” -s /bin/bash zope
$passwd zope

Do not add zope user to the sudoers list.

Disable root login
$sudo vi /etc/ssh/sshd_config

Change “PermitRootLogin yes” to “PermitRootLogin no”
Reload ssh
$sudo /etc/init.d/ssh reload

Create Directory Structure

The Library directory in the below structure may look a little over-organized at first pass. However, my next project after completing this migration will be to upgrade from Plone 2.0.3 to Plone 3 which will require several interim upgrades. This version of the website is running on Zope 2.7.3. Each interim upgrade will be installed to it’s respective directory within Library/Software keeping everything well organized and easy to roll back to if necessary.

in /usr/local

/usr/local/Zope
…./Downloads: where downloads will be stored
…./src: where software will be unarchived
…./Library: where compiled software will be compiled to
………./Software
……………/Zope273
………………../Python
………………./Zope
…/Sites: where each website to be hosted lives
………/instancename1
……../instancename2

# cd /usr/local/Zope/Downloads
# wget http://www.python.org/ftp/python/2.3.7/Python-2.3.7.tgz
# wget http://www.zope.org/Products/Zope/2.7.0/Zope-2.7.0.tgz
# wget http://dist.plone.org:5021/archive/Plone-2.0.5.tar.gz
# wget http://sourceforge.net/projects/pyxml/files/pyxml/0.8.4/PyXML-0.8.4.tar.gz
# wget http://effbot.org/downloads/Imaging-1.1.6.tar.gz
#cp * /usr/local/Zope/src/

Install Python 2.3.7

Build Python 2.3 from source with the following steps:

cd /usr/local/Zope/src/
tar -xvzf Python-2.3.7.tgz
cd Python-2.3.7

./configure BASECFLAGS=-U_FORTIFY_SOURCE --prefix=/usr/local/Zope/Library/Software/Zope273/Python
# Note: on Ubuntu 10.10 I had to add BASECFLAGS. The configure script for older versions of Ubuntu was:
./configure --prefix=/usr/local/Zope/Library/Software/Zope273/Python
make
 make install

Install PIL

More info

At the time of this writing, PIL 1.1.6 is compatible with Python 1.5.2 and up and fixes some 64 bit compatibility errors in Python 2.5 (which we’re not using yet for this Zope installation)

On Ubuntu, you have to edit setup.py before installing PIL
#sudo vi /usr/local/Zope/src/Imaging-1.1.6/setup.py

find the line
#TCL_ROOT = None

and replace it with
TCL_ROOT = "/usr/include/tk"

#/usr/local/Zope/Library/Software/Zope273/Python/bin/python2.3 setup.py build
#/usr/local/Zope/Library/Software/Zope273/Python/bin/python2.3 setup.py install

Install PyXML

cd /usr/local/Zope/src/PyXML-0.8.4/

#/usr/local/Zope/Library/Software/Zope273/Python/bin/python2.3 setup.py build
#/usr/local/Zope/Library/Software/Zope273/Python/bin/python2.3 setup.py install

Other recommended packages

elementree has been recommended, but it’s not installed on the old system so I’m not going to worry about it atm.

Also, the http://plone.org/documentation/kb/setup-from-source plone article recommends installing DocFinder as an invaluable development tool, but again, I don’t need this to get the website running so I’m going to come back to it.

install zope 2.7.3

Build Zope 2.7 from source with the following steps:

cd /usr/local/Zope/src
tar -xvzf Zope-2.7.3.tgz
cd Zope-2.7.3
./configure --with-python=/usr/local/Zope/Library/Software/Zope273/Python
bin/python2.3 --prefix=/usr/local/Zope/Library/Software/Zope273/Zope
make
make install

create zope instance(s)

The zope user must have write access to create the directory. After the instance is created, edit #”effective-user zope” into the etc/zope.conf file, so if you start it as root later it should #su itself to the non-root user. Again: make install should be run as root, #mkzopeinstance.py should not.

Initially, when I tried to create an instance, it failed with the error:
# /usr/local/Zope/Library/Software/Zope273/Zope/bin/mkzopeinstance.py: /usr/local/Zope/bin/python: bad interpreter: No such file or directory

I think this was a result of not cleaning up a bad compilation. To fix this edit the first line in mkzopeinstance.py from
#!/usr/local/Zope/bin/python
To
#!/usr/local/Zope/LIbrary/Software/Zope273/Zope/bin/python

When Zope is compiled, one of the last things it does is create a symbolic link from where you told Zope the python interpreter you wanted it to use lives to Zope/bin/python. mkzopeinstance.py is looking for that symbolic link, and not looking for python itself. This provides further separation so that we could theoretically upgrade python without touching Zope and then that symbolic link would only need to be changed.

The instance needs to be created with the zope user, NOT root

# su zope
# /usr/local/Zope/Library/Software/Zope273/Zope/bin/mkzopeinstance.py
# when prompted for the path to your instance, use:
# /usr/local/Zope/Sites/instance1  .... and so forth for each site.

test Zope

For the first instance, you can test by running /usr/local/Zope/Sites/instance1/bin/runzope. OR running /usr/local/Zope/Sites/instance1/bin/zopectl start

Once loaded, this will make Zope accessible on http://localhost:8080 (unless you changed the port), with the Zope Management Interface available on http://localhost:8080/manage (obviously if you’re accessing a remote server, then localhost might not work and you need to use the IP address)

shut down zone by either hitting ctrl+c if runzope was used or by running /usr/local/Zope/Sites/instance1/bin/zopectl stop if zopectl was used to start zone.

Install the website

cp originalSite/Products/* /usr/local/Zope/Sites/instance1/Products/
cp originalSite/var/Data.fs /usr/local/Zope/Sites/instance1/var/

/usr/local/Zope/Sites/instance1/bin/zopectl start

navigate to the ZMI localhost:8080/manage
(Note that the admin login credentials will match the credentials of the site migrated, not the credentials you compiled Zope with)
The website I migrated over, even though I physically placed the files in the instance1 directory within /usr/local/Zope/Sites/instance1 are in the ZMI under “Plone”

click the Root Folder of the ZMI
check the box beside “Plone”
click the “rename” button and rename it to your instance1 site
(at this point I discovered a dependency on lynx which I’ve added to the apt-get list above)

navigate to localhost:8080/instance1 to view the website

On first pass, everything worked!

Users

make sure that all the Zope files belong to the zope user
cd /usr/local
chown zope -R Zope
chgrp zope -R Zope
//////

start Zope

Start Zope with the following command:
su zope
/usr/local/Zope/Sites/dev/bin/zopectl start

check it out

Go to the http://mydomain.com:8080/manage, log in, create a new Plone site object.

resources/references

• setup from source
• plone installation

resolving tcl/tk error when installing PIL on Ubuntu Hardy

Next up:

• VirtualHostMonster and Apache with mod_rewrite & mod_proxy
• caching: Cachefu, Pound, Squid, Varnish

Bash (Terminal) File and Directory Colors

I do a lot of work within Terminal and not having directory and file coloring drives me nuts after a while. To add colors

cd ~/
vi .bash_profile

add

export CLICOLOR=1
export LSCOLORS=ExFxCxDxBxegedabagacad

reload it by typing (in Terminal)
source ~/.bash_profile

close out Terminal then start a new Terminal session. Do an ls and directories and files should now have color. A more in-depth explanation is listed below in “references”

References

adding file and directory colors to Terminal

The problem with setting up an environment piecemeal is when you sit down to do something, you have to figure out what rabbit hole you’d gone down when you left off. In this case, I remembered installing mysql, but usually the last thing I do when I install mysql is install phpmyadmin because it’s such a handy productivity tool. Well, there was no phpmyadmin on my system, but that could be because Snow Leopard came with PHP 5.3 out of the box which is incompatible with most of the popular PHP web applications out there.

Anyhoo … so I started by checking my $PATH variable and was surprised by the output.

echo $PATH gave me duplicates of /usr/local and /opt/local et. al. and had stuff in it that wasn’t in my ~/.profile

my ~/.profile had the following in it:

export PATH=/usr/local/sbin:/usr/local/mysql/bin:$PATH
##
# Your previous /Users/username/.profile file was backed up as /Users/username/.profile.macports-saved_2009-12-29_at_11:21:58
##
# MacPorts Installer addition on 2009-12-29_at_11:21:58: adding an appropriate PATH variable for use with MacPorts.
export PATH=/opt/local/bin:/opt/local/sbin:$PATH
# Finished adapting your PATH environment variable for use with MacPorts.

and the second entry made by MacPorts was overriding the first entry.

Pretty funky.

TheTao of Path Variables

1. /etc/profile is the default startup script for Bash, which is what I’m using. (If you’re using a different shell, then you may have a different startup script). /etc/profile calls /usr/libexec/path_helper
2. path_helper first calls /etc/path and /etc/manpath which contain the initial path environment variables. /etc/path contains system-wide defaults:

   /usr/bin
   /bin
   /usr/sbin
   /sbin
   /usr/local/bin

3. path_helper then looks for files in the directories: /etc/paths.d and /etc/manpaths.d and appends the paths found there. On my system, /etc/paths.d and /etc/manpaths.d contain a file named X11 which simply contains the paths for X11.
4. After /etc/profile has called path_helper, it then looks for /etc/.bashrc. I have a bashrc (no “.”), but my bashrc only has stuff in it specifying the bash shell prompt (name-of-my-computer:directory username$)
5. Bash next looks for ~/.bash_profile. This is the file where you’ll set file and directory colors and could also be where you place your Path environment variables. Obviously, ~/ represents your user directory, so your settings will only be valid for your user.
6. Next, bash looks for ~/.bash_login. I don’t have this on my system so it’s ignored in my case
7. next, bash looks for ~/.profile which I did have on my system and is the file that XCode wrote to.

Another file that can contain path variables is ~/.MacOSX/environment.plist. This sets environment variables, including paths, for gui applications. I’m not using it on my system so don’t have anything to say about it.

Recap

So, what that all means is that instead of exporting PATH environment variables to a .profile or .bash_profile in a user account directory, you (or your application) can, instead, make PATHs global by adding text files to the /etc/paths.d and /etc/manpaths.d directories.

If you need to control the order of a path, then try this:
Add a line PATH=”" before the call to path_helper like this in /etc/profile:

if [ -x /usr/libexec/path_helper ]; then
        PATH=""
        eval `/usr/libexec/path_helper -s`
fi

All that said and done … I’ll continue using ~/.bash_profile because it’s got that warm fuzzy familiarity. Personal preference, as always.

Update

I wanted more control over my Rails environment and wanted it to run from /usr/local/bin. I got partway setting up my cucumber tests when I realized that my application was running off of the default mac Rails environment in /usr/bin/. The solution is to get /usr/local/bin in front of /usr/bin. To do so, edit your ~/.profile or ~/.bashrc (wherever you decided to put your PATH variables)

References

man page for path_helper
making use of paths.d
mastering the path_helper

Delete files from the target directory that have been deleted in the source directory
rsync -avz -e ssh --delete remoteuser@remotehost:/remote/dir/ /this/dir/

Note: the ending {/} is important if you don’t want to hose a directory.

Rsync with a custom ssh port

place the ssh command in quotes; for example:
rsync -avz 'ssh -p1234'

Backup a website on a linux box to your windows machine

rsync reads ":" in a filepath as a remote directory, so if you're trying to rsync to "c://" rsync will be confused. Instead of using windows syntax for filepaths, use the cygwin directory structure.

1. Install cygwin to windows
2. test rsync and ssh are installed by typing from within the cygwin terminal:
rsync --version
ssh -l username somedomain
3. create a directory in c://cygwin named backups or whatever you want to call it.
4. rsync -avz -e ssh --delete remoteuser@remotehost:/remote/dir/ /cygdrive/c/directoryname Change the directory drive to whatever it should be

references

rsync over ssh