a fresh debian 4.0 etch, lighttpd, php, mysql, mod_geoip installation with maxmind database and visitors webstats, monitored by runit

its fuckin' big i know. but its cool.
we start from a complete plain installed debian etch 4.0.
one line is one command.

apt-get update
if this error shows up:
W: GPG error: http://ftp.de.debian.org etch Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY A70DAF536070D3A1 NO_PUBKEY B5D0C804ADB11277
... try this:
apt-get install debian-archive-keyring
now we want to update the system with the following, do it one after another:
apt-get update
apt-get upgrade
apt-get update
apt-get upgrade
apt-get dist-upgrade

now do a reboot
reboot
to make sure we didnt missed anything:
apt-get update
apt-get upgrade

install mysql and set the root user password:
apt-get install mysql-server mysql-client
mysqladmin -u root password yourpassword

install php:
apt-get install php5-cgi php5-cli php5-gd php5-mcrypt php5-mysql php5-pspell
if you dont need pspell, gd or mcrypt remove it from the list.

lets get to lighttpd:
cd /usr/local/src/
wget http://www.lighttpd.net/download/lighttpd-1.5.0-r1992.tar.gz
tar -xzf lighttpd-1.5.0-r1992.tar.gz
cd lighttpd-1.5.0/src
wget -O mod_geoip.c http://trac.lighttpd.net/trac/attachment/wiki/Docs/ModGeoip/mod_geoip.4.c?format=raw
nano Makefile.am

now nano should be open with the Makefile.am. we have to patch the mod_geip.c into it.
try to SEARCH for this:
lib_LTLIBRARIES += mod_accesslog.la
mod_accesslog_la_SOURCES = mod_accesslog.c
mod_accesslog_la_LDFLAGS = -module -export-dynamic -avoid-version -no-undefined
mod_accesslog_la_LIBADD = $(common_libadd)

... and ADD a line under it:
lib_LTLIBRARIES += mod_geoip.la
mod_geoip_la_SOURCES = mod_geoip.c
mod_geoip_la_LDFLAGS = -module -export-dynamic -avoid-version -no-undefined
mod_geoip_la_LIBADD = $(common_libadd) -lGeoIP

before we compile we install the make tools because we need to remake the makefiles:
cd ..
apt-get install autotools-dev autoconf automake pkg-config libtool binutils gcc make swig psmisc
automake -a

this probably throws this warning:
doc/Makefile.am:83: `%'-style pattern rules are a GNU make extension
just ignore it.

before we compile lighttpd we also have to install a few libs and set the LUA lib because... you just have to:
apt-get install libaio-dev liblua5.1-0-dev libgeoip-dev libglib2.0-dev libpcre3-dev zlib1g-dev

pkg-config --cflags lua5.1
export LUA_CFLAGS=-I/usr/include/lua5.1
pkg-config --libs lua5.1
export LUA_LIBS=-llua5.1

lets start compiling
./configure --with-lua --with-linux-aio
make
make install

if this is done we need to set up the config files:
mkdir /etc/lighttpd
cp /usr/local/src/lighttpd-1.5.0/doc/lighttpd.conf /etc/lighttpd/
nano /etc/lighttpd/lighttpd.conf

the lighttpd.conf should be open, here we add mod_geoip into the modules list and remove some #'s for modules we probably need.
it should now look like this:
server.modules = (
"mod_geoip",
"mod_rewrite",
# "mod_redirect",
# "mod_alias",
"mod_access",
# "mod_cml",
# "mod_trigger_b4_dl",
"mod_auth",
# "mod_status",
# "mod_setenv",
"mod_proxy_core",
# "mod_proxy_backend_http",
"mod_proxy_backend_fastcgi",
# "mod_proxy_backend_scgi",
# "mod_proxy_backend_ajp13",
# "mod_simple_vhost",
# "mod_evhost",
# "mod_userdir",
# "mod_cgi",
"mod_compress",
# "mod_ssi",
# "mod_usertrack",
# "mod_expire",
# "mod_secdownload",
# "mod_rrdtool",
"mod_accesslog" )

after the modules list add this two lines:
geoip.db-filename = "/www/GeoLiteCity.dat"
geoip.memory-cache = "enable"

and change following, you will find those vars in the config:

server.document-root = "/www/pages/nirvana/"
server.errorlog = "/www/logs/default.error.log"
accesslog.filename = "/www/logs/default.access.log"

server.username = "www-data"
server.groupname = "www-data"

compress.cache-dir = "/tmp/lighttpd/cache/compress/"
compress.filetype = ("text/plain", "text/html")

because we want PHP you have to find this:
#### fastcgi module
## read fastcgi.txt for more info
#$HTTP["url"] =~ "\.php$" {
# proxy-core.balancer = "round-robin"
# proxy-core.allow-x-sendfile = "enable"
# proxy-core.check-local = "enable"
# proxy-core.protocol = "fastcgi"
# proxy-core.backends = ( "unix:/tmp/php-fastcgi.sock" )
# proxy-core.max-pool-size = 16
#}

... and change it so it looks like this:
$PHYSICAL["existing-path"] =~ "\.php$" {
proxy-core.balancer = "round-robin"
proxy-core.allow-x-sendfile = "enable"
proxy-core.protocol = "fastcgi"
proxy-core.backends = ( "unix:/tmp/php-fastcgi.sock" )
proxy-core.max-pool-size = 16
}

(by the time you download the tarball, it is possible that the first wrong $PHYSICAL line is already correct and the check-local var is removed.)

add a global password protection for the /stats/ folder with adding this at the end of the file:
auth.backend = "htpasswd"
auth.backend.htpasswd.userfile = "/www/stats/lighttpd.htpasswd"
auth.require = ("/stats/" => (
"method" => "basic",
"realm" => "stats",
"require" => "valid-user"
))

also add a test vhost at after the auth redirect:
$HTTP["host"] =~ "^(|www\.)yourdomain\.com$" {
server.document-root = "/www/pages/yourdomain.com/htdocs/"
alias.url = ( "/stats/" => "/www/stats/htdocs/www.yourdomain.com/" )
accesslog.filename = "/www/logs/access_www_yourdomain_com.log"
url.rewrite-once = (
"^/stats/.*" => "$0",
"^/(.*)/$" => "/index.php"
)
}

close the editor now (you do this with ctrl+X and Y for saving).

we need to create several directories:
mkdir -p /tmp/lighttpd/cache/compress
mkdir -p /www/logs
mkdir -p /www/pages/nirvana
mkdir -p /www/pages/yourdomain.com/htdocs
mkdir -p /www/stats/htdocs
chown www-data:www-data /www/logs /tmp/lighttpd/cache/compress

whats missing at the moment is the maxmind database we added in the lighttpd.conf. just download the files and extract them:
wget -O /www/GeoLiteCity.dat.gz http://www.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
wget -O /www/GeoIP.dat.gz http://www.maxmind.com/download/geoip/database/GeoIP.dat.gz
cd /www/
gunzip *.dat.gz

as you can see we download two databases. one is filled only with the countries and the other also include cities. you have to choose for yourself what you need and change it in the lighttpd.conf.

we need "htpasswd" to create a htpasswd file so we need to install the apache2-utils, weird but useful.
it wont install apache2:
apt-get install apache2-utils

create htpasswd for protecting your /stats/ folder - "admin" is the username:
htpasswd -c /www/stats/lighttpd.htpasswd admin

if you want to add another user just remove the -c "create" command:
htpasswd /www/stats/lighttpd.htpasswd admin2

the stats have to be created somehow so we take care about this now:
apt-get install visitors graphviz
nano /www/stats/create.php

save this code into the create.php:
<?php

# create stats

$c['folder']['logs'] = '/home/www/logs/';
$c['folder']['stats'] = '/home/www/stats/htdocs/';
$c['visitors']['bin'] = '/usr/bin/visitors';
$c['visitors']['options'] = '-A -m 100 --screen-info -Z -H -M -X -Y -S -r 200 -g 100 --max-adsensed 100 --trails --prefix';

foreach( glob( $c['folder']['logs'].'access_*.log' ) as $logfile ) {
$logfile = explode( '/', $logfile );
$logfile = $logfile[count($logfile)-1];
$site = str_replace( array( 'access_', '.log', '_' ), array('','','.'), $logfile );
echo $logfile."\n";
echo $site."\n";
if( !is_dir( $c['folder']['stats'].$site ) ) {
mkdir( $c['folder']['stats'].$site );
}
exec( $c['visitors']['bin'].' --output html '.$c['visitors']['options'].' http://'.$site.' '.$c['folder']['logs'].$logfile.'* -o html > '.$c['folder']['stats'].$site.'/index.html' );
exec( $c['visitors']['bin'].' '.$c['folder']['logs'].$logfile.'* --prefix http://'.$site.' --graphviz > '.$c['folder']['stats'].$site.'/graph.dot' );
@exec( 'dot '.$c['folder']['stats'].$site.'/graph.dot -Tpng > '.$c['folder']['stats'].$site.'/graph.png' );
}

?>

we want the stats created every day so we set a cronjob for this:
crontab -e
and install the cronjob with adding this line and saving the file:
0 4 * * * php /www/stats/create.php 1>/dev/null 2>/dev/null
this will execute the script every day at 4am.

init.d script are so oldschool and we're cool so we install runit and set up the run scripts:
apt-get install runit
mkdir -p /etc/sv/lighttpd/log/
nano /etc/sv/lighttpd/run

copy & paste:
#!/bin/sh
exec lighttpd -D -f /etc/lighttpd/lighttpd.conf 2>&1

nano /etc/sv/lighttpd/log/run
copy & paste:
#!/bin/bash
exec chpst -u nobody svlogd -tt /var/log/sv/lighttpd

now we set up the permissions and other stuff:
chmod +x /etc/sv/lighttpd/run /etc/sv/lighttpd/log/run
mkdir -p /var/log/sv
chown nobody /var/log/sv
ln -s /etc/sv/lighttpd/ /var/service/

because we want no RC scripts we run this:
update-rc.d -f lighttpd remove

we do the same for php:
mkdir -p /etc/sv/php-fcgi/log
nano /etc/sv/php-fcgi/run

copy & paste:
#!/bin/bash
exec /usr/local/bin/spawn-fcgi -n -s /tmp/php-fastcgi.sock -P /var/run/spawn-fcgi.pid -C 8 -u www-data -g www-data -f /usr/bin/php5-cgi 2>&1

nano /etc/sv/php-fcgi/log/run
copy & paste:
#!/bin/bash
exec chpst -u nobody svlogd -tt /var/log/sv/php-fcgi

again the permissions and the link:
chmod +x /etc/sv/php-fcgi/run /etc/sv/php-fcgi/log/run
ln -s /etc/sv/php-fcgi/ /var/service/

it's done. start php and lighttpd with this commands:
sv start php-fcgi
sv start lighttpd

for restarting use the same commands just instead of "start" use "stop" and after it "start" again.

if it works you can access your server on http://yourip or http://yourdomain.com and also access /stats/ to see your webstats.

Activity

One comment, leave your comment or trackback.
  1. What a freaking outstanding post! I really appreciate the detail you cover, can’t believe you don’t have any posts here. I’m limping along on Etch with Lighty 4.1.3 - it’s memory overflows every two hours and monit kills/restarts it. So it’s covered, but it’s not optimal - I’m going to try the latest 1.5 now thanks to you.


Leave a Reply


Search

The archives run deep. Feel free to search older content using topic keywords.