semioticpixels scratch pad

I’ve been slowly migrating from my desktop to my laptop so I’ve been setting it up piecemeal.

Bash (Terminal) File and Directory Colors

I do a lot of work within Terminal and not having directory and file coloring drives me nuts after a while. To add colors

cd ~/
vi .bash_profile

add

export CLICOLOR=1
export LSCOLORS=ExFxCxDxBxegedabagacad


reload it by typing (in Terminal)
source ~/.bash_profile

close out Terminal then start a new Terminal session. Do an ls and directories and files should now have color. A more in-depth explanation is listed below in “references”

References

adding file and directory colors to Terminal

Managing Paths

The problem with setting up an environment piecemeal is when you sit down to do something, you have to figure out what rabbit hole you’d gone down when you left off. In this case, I remembered installing mysql, but usually the last thing I do when I install mysql is install phpmyadmin because it’s such a handy productivity tool. Well, there was no phpmyadmin on my system, but that could be because Snow Leopard came with PHP 5.3 out of the box which is incompatible with most of the popular PHP web applications out there.

Anyhoo … so I started by checking my $PATH variable and was surprised by the output.

echo $PATH gave me duplicates of /usr/local and /opt/local et. al. and had stuff in it that wasn’t in my ~/.profile

my ~/.profile had the following in it:

export PATH=/usr/local/sbin:/usr/local/mysql/bin:$PATH
##
# Your previous /Users/username/.profile file was backed up as /Users/username/.profile.macports-saved_2009-12-29_at_11:21:58
##
# MacPorts Installer addition on 2009-12-29_at_11:21:58: adding an appropriate PATH variable for use with MacPorts.
export PATH=/opt/local/bin:/opt/local/sbin:$PATH
# Finished adapting your PATH environment variable for use with MacPorts.


and the second entry made by MacPorts was overriding the first entry.

Pretty funky.

TheTao of Path Variables

1. /etc/profile is the default startup script for Bash, which is what I’m using. (If you’re using a different shell, then you may have a different startup script). /etc/profile calls /usr/libexec/path_helper
2. path_helper first calls /etc/path and /etc/manpath which contain the initial path environment variables. /etc/path contains system-wide defaults:
   /usr/bin
/bin
/usr/sbin
/sbin
/usr/local/bin


3. path_helper then looks for files in the directories: /etc/paths.d and /etc/manpaths.d and appends the paths found there. On my system, /etc/paths.d and /etc/manpaths.d contain a file named X11 which simply contains the paths for X11.
4. After /etc/profile has called path_helper, it then looks for /etc/.bashrc. I have a bashrc (no “.”), but my bashrc only has stuff in it specifying the bash shell prompt (name-of-my-computer:directory username$)
5. Bash next looks for ~/.bash_profile. This is the file where you’ll set file and directory colors and could also be where you place your Path environment variables. Obviously, ~/ represents your user directory, so your settings will only be valid for your user.
6. Next, bash looks for ~/.bash_login. I don’t have this on my system so it’s ignored in my case
7. next, bash looks for ~/.profile which I did have on my system and is the file that XCode wrote to.

Another file that can contain path variables is ~/.MacOSX/environment.plist. This sets environment variables, including paths, for gui applications. I’m not using it on my system so don’t have anything to say about it.

Recap

So, what that all means is that instead of exporting PATH environment variables to a .profile or .bash_profile in a user account directory, you (or your application) can, instead, make PATHs global by adding text files to the /etc/paths.d and /etc/manpaths.d directories.

If you need to control the order of a path, then try this:
Add a line PATH=”" before the call to path_helper like this in /etc/profile:

if [ -x /usr/libexec/path_helper ]; then
PATH=""
eval `/usr/libexec/path_helper -s`
fi


All that said and done … I’ll continue using ~/.bash_profile because it’s got that warm fuzzy familiarity. Personal preference, as always.

References

man page for path_helper
making use of paths.d
mastering the path_helper

Rysnc from 1 linux box to another and keep the file structure identical

Delete files from the target directory that have been deleted in the source directory
rsync -avz -e ssh --delete remoteuser@remotehost:/remote/dir/ /this/dir/

Note: the ending {/} is important if you don’t want to hose a directory.

Backup a website on a linux box to your windows machine

rsync reads “:” in a filepath as a remote directory, so if you’re trying to rsync to “c://” rsync will be confused. Instead of using windows syntax for filepaths, use the cygwin directory structure.

1. Install cygwin to windows
2. test rsync and ssh are installed by typing from within the cygwin terminal:
   rsync --version
ssh -l username somedomain
3. create a directory in c://cygwin named backups or whatever you want to call it.
4. rsync -avz -e ssh --delete remoteuser@remotehost:/remote/dir/ /cygdrive/c/directoryname Change the directory drive to whatever it should be

references

rsync over ssh

I inherited a number of web servers that had been installed and configured by different contractors and am now in the process of evaluating what needs to happen to bring them all up to snuff as well as figuring out ways to streamline regular updates. Some of those web servers are still on Red 7.3 (!), the last free open source version of Red Hat. They’re quite old in server years. Additionally I have a few other servers I set up a year ago that are more up-to-date running Ubuntu 8 LTS, but the Apache version is 2.2.11 and 2.2.8. Our security scanning service notifies me regularly that SSL v 2 must be upgraded to SSL v3, which requires an upgrade to Apache 2.2.13

Environment:

Apache 1.3.29

References:

How to Disable SSL v 2 support in Apache

On Aug 10, 2009, Apache released an upgrade that addresses a DOS vulnerability.

server:

/usr/local/apache2/bin/httpd -v
# Server version: Apache/2.2.11 (Unix)
# Server built:   Jan 15 2009 13:39:20

dev server:

/usr/sbin/apache2 -v

Issues

1. caused 401 error for all http requests (worked correctly for https connections) source
2. Seem to be some issues with mac version – It’s unclear whether this is an issue with a pre-compiled mac version or a generic self-compiled (which should be identical to the linux version) source

Dev Server Tests

Ran apt-get update and apt-get upgrade on dev server

ran apt-get upgrade apache2 and message returned says that 2.2.8 is the current version. Which means my dev and my production servers are out of sync. I was surprised to learn that the last contractor from whom I took over server management had installed from source, removing Apache from package management. Not a big deal really, but it was undocumented.

In any case, there doesn’t appear to be a package release for Apache 2.2.13 yet in Ubuntu. Only one of the 4 bugfixes has a security bulletin attached, so I’ve decided to wait a few weeks to see if anything new transpires in the security bulletins. In general, I prefer to wait on updating production servers until a new release has been out long enough for bugfixes to be released.

http://www.techcrunch.com/2009/08/15/the-35-best-iphone-apps-of-the-year-so-far/

You can recover MySQL database server password with following five steps.

Step # 1: Stop the MySQL server process.

Step # 2: Start the MySQL (mysqld) server/daemon process with the –skip-grant-tables option so that it will not prompt for password

Step # 3: Connect to mysql server as the root user

Step # 4: Setup new root password

Step # 5: Exit and restart MySQL server

Here are commands you need to type for each step (login as the root user):

Step # 1 : Stop mysql service

# /etc/init.d/mysql stop
Output:

Stopping MySQL database server: mysqld.

Step # 2: Start MySQL server w/o password:

# mysqld_safe --skip-grant-tables &
Output:

[1] 5988
Starting mysqld daemon with databases from /var/lib/mysql
mysqld_safe[6025]: started

Step # 3: Connect to mysql server using mysql client:

# mysql -u root
Output:

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1 to server version: 4.1.15-Debian_1-log

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql>

Step # 4: Setup new MySQL root user password

mysql> use mysql;
mysql> update user set password=PASSWORD("NEW-ROOT-PASSWORD") where User='root';
mysql> flush privileges;
mysql> quit

Step # 5: Stop MySQL Server:

# /etc/init.d/mysql stop
Output:

Stopping MySQL database server: mysqld
STOPPING server from pid file /var/run/mysqld/mysqld.pid
mysqld_safe[6186]: ended

[1]+  Done                    mysqld_safe --skip-grant-tables

Step # 6: Start MySQL server and test it

# /etc/init.d/mysql start
# mysql -u root -p

reference

1. Add rotation script to logrotate

sudo vi /etc/logrotate.conf

add

# system-specific logs may be also be configured here.
/usr/local/Zope/Sites/SiteName/log/Z2.log {
rotate 5
weekly
compress
size=100k
sharedscripts
postrotate
#close and re-open all Zope log files (z2.log, event.log) The common idiom after rotating Zope log files
/bin/kill -s SIGUSR2 `cat /usr/local/Zope/Sites/SiteName/var/Z2.pid`
endscript
}

/usr/local/Zope/Sites/SiteName/log/event.log {
rotate 5
weekly
compress
size=100k

}

2. Test log rotation

Do a test run of the rotation without actually rotating anything:

/usr/sbin/logrotate -d /etc/logrotate.conf

if the test run completes without any erros, force a rotation:
/usr/sbin/logrotate -f /etc/logrotate.conf

3. Automate with crontab

On RedHat, crontab may be set up with runparts e.g.

# run-parts
01 * * * * root run-parts /etc/cron.hourly
02 4 * * * root run-parts /etc/cron.daily
22 4 * * 0 root run-parts /etc/cron.weekly
42 4 1 * * root run-parts /etc/cron.monthly


In this case, place a script in the directory where it should run regularly. I’m going to rotate weekly for now, so I’m placing a script in /etc/cron.weekly and naming it zope.cron (you can name it whatever you want. Any script in this directory will run weekely


sudo vi /etc/cron.weekly/zope.cron

Setting up the Slicehost Account

Update everything
sudo apt-get update
sudo apt-get upgrade

install vim full so you can edit files
apt-get install vim-full

update the “locate” db
sudo updatedb

Change the root password
sudo passwd

Set up Users
useradd zope (I like useradd because there’s no reason to have a home directory. If you want your user to have a home directory then use adduser or useradd -D zope to create ~/home/zope)
passwd zope
useradd -m -c "real name" -s /bin/bash auserwithsudoers The flags create a home directory with skel profile defaults and this account is linked to the defaults for bash shell, which is important if you like syntax coloring and tab completion.
passwd auserwithsudoers
sudo visudo
Find the line that says #User Privilege specification and add
auserwithsudoers ALL=(ALL) ALL
shift ZZ to save and exit

Now log out as root and login as your new admin user and test sudo
su auserwithsudoers
sudo bash

Disable root login
su root
sudo vim /etc/ssh/sshd_config
Change “PermitRootLogin yes” to “PermitRootLogin no”

Reload the ssh config
sudo /etc/init.d/ssh reload

Ok! Now we’re ready to think about plone

1. Follow the plone install instructions here
2. install some products (see plone documentation on using buildout). Here’s what I did to install a Press Release product
   1. download the product. If there’s a choice of files, choose the one with the naming convention Products.package.tar.gz. This is a python egg.
   2. su zope
vi /usr/local/Plone/zeocluster/buildout.cfg
      add the following:
      [buildout]
...
eggs =
Products.PressRelease

      and save (ctrl + ZZ)
   3. sudo /usr/local/Plone/zeocluster/bin/buildout
   4. sudo /usr/local/Plone/zeocluster/bin/plonectl restart
   5. In each Plone site where you want to install the Product, go Site Setup>Add/Remove Products and install the Product.

When reviewing the analytics for this site, I often guess how much of that traffic is myself. After some consideration, I decided to use Justin Cutroni’s cookie solution. He wrote the solution in 2006 for the old analytics. I’m testing it on the new GA code and it seems to be working just fine.

The basic idea is put a form on your website that lets you name a cookie. The form will set the cookie on your computer. This lets you set a cookie for any number of computers. The only downside is if you clear your cookies, you’ll have to reset it.

The next step is to create a custom filter in your Google Analytics account that will exclude traffic from computers with that cookie on them.

See Justin’s article at Analytics Talk

I know there are plenty of articles and posts for setting up a development server. Yet I still find the need to write up the exact commands and process for my later self to refer to, especially since I haven’t found a clone function in VMware Fusion like the linux version has. A pet “when-I-have-time” project would be to turn these steps into a bash script.

Again, the environment is Ubuntu Gutsy Gibbon

Setting up a new LAMP server

File Permissions

If you’re going to access your web directory from another computer, you’ll need to change the file permissions on the web directory so you can put files. For example, my development server is a Ubuntu virtual machine but I do some development in a Windows vm and some development on my mac. In both cases, I connect with my development server over an sftp client that’s integrated into my editor environment.

sudo chown yourusername /var/www
alternatively, you might create a new user like “web” or “www”. If you’ll occasionally make your server public, you might want to create a user that does not have sudo access for an extra layer of security.¹

install Apache 2

sudo apt-get install apache2
sudo /etc/init.d/apache2 start
test apache by navigating in browser to localhost.
place an index.html page into /var/www to test that’s all set up.

I like to stop apache while installing other things, but you can always restart it so it doesn’t really matter
sudo /etc/init.d/apache2 stop (or restart)

Install php 5

sudo apt-get install php5 libapache2-mod-php5
restart apache so it registers with php
sudo /etc/init.d/apache2 restart
test that it worked by placing a phpinfo file into /var/www
create the file
vi /var/www/phpinfo.php
then type
<?php phpinfo(); >

Navigate in browser to localhost/phpinfo.php to see that it’s working

install mysql server

sudo apt-get install mysql-server
sudo apt-get install libapache2-mod-auth-mysql php5-mysql mysql-client

Before you start configuring files, you need to install the full version of vi. By default, ubuntu only comes with vim-tiny. You’ll know it’s vim-tiny when you try to edit a file with vi and the console reads out characters like ^B. So,
sudo apt-get install vim-full

Now, tell php about mysql
sudo vi /etc/php5/apache2/php.ini

It doesn’t matter where you put it, but I like to put it under the line that says “Directory in which the loadable extensions (modules) reside”.
extension=mysql.so
restart apache again.
sudo /etc/init.d/apache2 restart

Install phpmyadmin

This can be done the Ubuntu way or it can be done manually by installing phpmyadmin to /var/www like any other web application.

The Ubuntu way

the benefit to doing it this way is that phpmyadmin is installed as a managed package which means it will be updated by our lovely package manager, apt-get which saves us a little time. Since I use phpmyadmin on multiple virtual machines, this makes a cumulative difference.

sudo apt-get install phpmyadmin

tell apache where phpmyadmin actually lives (which is in /etc/phpmyadmin)
sudo vi /etc/apache2/apache2.conf
You can place the following directive anywhere, but I prefer to place it under the line:
# Include all the user configurations
add the line
Include /etc/phpmyadmin/apache.conf

If you didn’t want to edit your apache.conf, you could always use a symlink instead.

navigate to localhost/phpmyadmin to check that it’s working

Some cleanup/maintenance/utility stuff

If this is a fresh install, you probably need to run the locate database updater so you can use it.
sudo updatedb

Done with the LAMP setup! Next up … subversion and trac

Footnotes

1. for more on managing user accounts see this O’Reilly article or for an in-depth treatise that addresses best practices – check out my favorite reference book “Essential System Administration”. [↩]