Bits and pieces from the gibbed.me closure (incl. useful scripts)

[RandumKiwi]

Hi everyone,

Long time, no see - down to business (please keep this thread on-topic).

I came here to share the resources and data from my Linux "gibbed.me" servers (which have been shutdown). Whether you're new to running an AssaultCube server or quite experienced, I figure that some of it might be useful and I feel obliged to offer it to the masses to be used as you see fit and hope that it's found useful. So here goes...

DISCLAIMERS:
- I'm not a scripting/coding expert and often do things the long-hand/simpler way.
- Use any of this at your own risk.
- Feel free to ask for further directions from others (don't expect me to reply personally).


USING "SCREEN" TO RUN MULTIPLE SERVERS
 1) In the "bin_unix" folder make several copies of the "linux_server" file, using different names for each and every server you'll be running.

[SELECT ALL] Code:

cp bin_unix/linux_server bin_unix/server_1 && cp bin_unix/linux_server bin_unix/server_2 && cp bin_unix/linux_server bin_unix/server_3

 2) Setup and run new script files in the base directory (not in "bin_unix") for each server, using the following format:

[SELECT ALL] Code:

#!/bin/sh
cd /path/to/assaultcube-base-directory
screen -d -m -S server_1 bin_unix/server_1 \
-n"EXAMPLE TITLE: COME PLAY AT SERVER1" \
-o"HOW TO DO MOTD MULTI-LINED
INSIDE THE SCRIPT FILE"

 3) For ease, you'll also want to setup a seperate script with "kill" instances for ALL of the servers.

[SELECT ALL] Code:

#!/bin/sh
kill -9 $(pidof server_1) && kill -9 $(pidof server_2) && kill -9 $(pidof server_3)

I recommend the above method, as it allows you to easily identify the servers in a process list (e.g. when using "htop").

For development servers, I suggest having the revision automatically placed in its title, such as:

[SELECT ALL] Code:

GITREVISION=`cd /home/GIT && git rev-parse master | cut -c 1-7`
# ...
-n"\f8GIT REV#"$GITREVISION" - \f8LURCH\f3 @ \f8gibbed.me" \

RECOMMENDED GENERAL CODING FIXES
If you know (or can learn) how to compile, here's some easy to fix issues for your servers:
- https://github.com/assaultcube/AC/commit...ccb104a4fc
- https://github.com/assaultcube/AC/commit...378ada7dbb
- You can turn-on full IP addresses for ALL users to see by going to line 2749 of server.cpp and deleting everything after "&&", then add a closed-bracket ")".
- To get a server that has custom maps, that other servers won't have (while keeping within the same base directory), simply change where the maps are stored at the top of "serverfiles.h", then compile.


WANT ANY OLD DATA FROM MY WESBITE?
Probably not useful, but try https://web.archive.org/web/201707082235...gibbed.me/ if you want it. Includes things like the nickname whitelist and deban passwords.


RECOMMENDED VPS HOSTS?
I would seriously recommend http://www.linode.com. I chose them, because their Fremont datacentre had the lowest ping from USA to NZ + the smallest hops in a traceroute (helps decrease packetjump likelyhood). Basically, it's because Linode have direct access to an impressive backbone called "Hurricane Electric" with no bullshit inbetween.


ASSAULTCUBE "SERVER MONITOR"
This detects if an AssaultCube server crashes, then reboots the server if/when it does (there are existing Linux apps that can do this, however they're more complex than required).

[SELECT ALL] Code:

#!/bin/sh

# Checks if your AssaultCube servers are up, and if not, attempts to
# reboot them and sends you an e-mail to let you know whether that
# reboot failed or not. Please ensure you've install mailutils:
#
# sudo apt-get install mailutils
#
# Change all of the below aliases for this to work!

###
# E-mail subject:
ESUBJECT="Your AC server had a bit of a wobble..."
###
# E-mail address:
EMAILADDR="[email protected]"
###

###
# The AC directory (no trailing slash):
THEACDIR="/home/cubeservers"
###
# Name of the server executable in bin_unix directory:
NAMEEXEC1="server_lurch"
###
# Name of the shell-file to run that restarts your AC server:
ACSHEXEC1="server_lurch.sh"
###

###
# Where to place the files for counting fails, which will prevent excessive server restarts.
# Filenames are dated, so files can be arranged by name (no folders per-server required).
# It goes without saying, ensure the directory exists and read/write-permission is available.
#
FILECOUNTDIR="/home/www/logs/ac_monitor"
###
# FYI: This essentially allows you to log when your server fails, so I suggest not to delete them.
#      however, if you do want to delete them, please uncomment the following line and they'll be
#      deleted once 24 hours old.
#DELETECOUNTFILES="1"
###
###    This script has no log-rotate facility, your server shouldn't be crashing that much!
###
# Name of logfiles (i.e. your servername):
NAMESERV1="Lurch"
###

# What to write in your e-mail if your reboot was successful:
IFSUC1="Lurch bumped his head!\nLuckily, he's got a thick skull and he's now answering the door at "$(date +%F-%T)"."
###
# What to write in your e-mail if your reboot was *NOT* successful:
IFFAIL1="Lurch walked into quicksand.\nHe couldn't be found at "$(date +%F-%T)"."
###
# What to write in your e-mail after 3 failed restarts in 12 hours:
IFDONE1="Lurch has died 3 times in 12 hours. We won't resuccitate anymore, something is very wrong!"
###


if [ "$(date +%H%M)" = 0142 ] || [ "$(date +%H%M)" = 0143 ] || [ "$(date +%H%M)" = 0144 ]; then
 # Typical update of GIT takes less than a minute, so this just gives it 2 more, just in case.
 exit
fi

if [ -z "$(pidof $NAMEEXEC1)" ]; then
 touch "$FILECOUNTDIR"/"$NAMESERV1"-at-"$(date +%F-%H.%M.%S)".fail && sleep 1  # Long version of time, for Windows compatiblity.
 HOWMANYFAILS=`cd "$FILECOUNTDIR" && find ./"$NAMESERV1"* -mmin -720 | wc -l`
 if [ "$HOWMANYFAILS" -le "3" ]; then
   sh "$THEACDIR"/"$ACSHEXEC1"
   sleep 5
   if [ -n "$(pidof $NAMEEXEC1)" ]; then
     echo "$IFSUC1" | mail -s "$ESUBJECT" "$EMAILADDR"
   else
     echo "$IFFAIL1" | mail -s "$ESUBJECT" "$EMAILADDR"
   fi
 elif [ "$HOWMANYFAILS" = "4" ]; then
   echo "$IFDONE1" | mail -s "$ESUBJECT" "$EMAILADDR"
 fi
 if [ "$DELETECOUNTFILES" = "1" ]; then
   cd "$FILECOUNTDIR" && find ./"$NAMESERV1"* -mmin +1440 | xargs rm -f
 fi
fi

ALLOW ADMINS TO REBOOT A DEVELOPMENT SERVER
I don't think this would be useful for non-development servers (instead, use the server monitor above). It's quite useful for development servers (i.e. commit a change, then have the server update itself).

 1) Search "server.cpp" for "driAn::writelog" and understand that this function allows an admin to write ANYTHING into the log.
 2) Setup a script that checks the log for something written into it by the above function, then initiates a script that updates/reboots the server.
    CARE: Ensure that what you write/search for can't be manipulated by normal users.

Example from /etc/rsyslog.cfg to seperate this "writelog" out:

[SELECT ALL] Code:

if $syslogtag contains 'Lurch6677' and $msg contains 'writes to log: you rang?' then /home/www/logs/Admin-Log/Admin.log

Example script that then checks for the writelogs, and recompiles/reboots:

[SELECT ALL] Code:

# Find how many times "you rang?" appears in admin log (password for restarting GIT),
# if it's once or more, then restart/recompile GIT if it is a new entry.
ADMFIND=`find /home/www/logs/Admin-Log/Admin.log -type f -mmin -2`
if [ -n "$ADMFIND" ]; then
 HWMNYF=`grep " writes to log: you rang?" /home/www/logs/Admin-Log/Admin.log | wc -l`
 NMBR=`cat /home/randum/scripts/.rgittst`
 if [ "$NMBR" != "$HWMNYF" ]; then
   echo "$HWMNYF" > /home/randum/scripts/.rgittst
   sleep 1
   if [ "$NMBR" != "0" ]; then
     sh /home/cubeservers/UpdateGIT.sh
   fi
 fi
fi

SCRIPT FOR UPDATING THE DEVELOPMENT SERVER

[SELECT ALL] Code:

#! /bin/sh

# This script should be run daily, via a cron-tab!
#
# Make sure that you have installed GIT and mailutils prior to setting up this script:
# sudo apt-get install git mailutils
#
# Make sure, you've enabled SSH access to github, so that ACAC can be downloaded.
# Ensure there's no passphrase protecting your SSH key:
# https://help.github.com/articles/generating-ssh-keys
#
# Before setting up this script, GIT will need to have previously been checked out. Using
# the following command ensures you don't clone *ALL* of AC's history (just the latest):
# git clone --depth=1 https://github.com/assaultcube/AC.git ./
# git clone --depth=1 [email protected]:assaultcube/ACAC.git ./
#
# Also, ensure you have all the build dependencies:
# sudo apt-get install clang make automake libsdl1.2-dev libogg-dev libsdl-image1.2-dev \
# libcurl4-openssl-dev libopenal-dev libvorbis-dev -y
#
###
# Path to AC's GIT and ACAC's GIT (no trailing slashes):
PATHTOACDIR="/home/GIT"
PATHTOACACDIR="/home/GIT_ACAC"
###
# Path to log everything into, please note how by default, it's currently set-up to
# date all of the logfiles. If you don't want logs, make the path /dev/null
LOGTOPATH="/home/www/logs/Daily-GIT-updates/GIT-update-$(date +%F).log"
###
# Path to the shell-file that starts your AC server. Please note, this shell file needs
# to run the executable as "git_server", not native_server/linux_64_server/linux_server/etc.
PATHTOACSHELLFILE="/home/cubeservers/server_lurch.sh"
###
# What to rename your server executable (should mirror the name in your shell file that
# starts the AC server):
NAMEOFEXEC="server_lurch"
###
# Path to save the compiled executables. Highly suggested this is in a web-facing
# directory, assist with locating difficult to trace bugs, by enabling you to
# quickly download/test older revisions.
EXECSAVEPATHC="/home/www/dev/linux_64_client/linux_64_client_$(date +%F_%H.%M.%S)"
EXECSAVEPATHS="/home/www/dev/linux_64_server/linux_64_server_$(date +%F_%H.%M.%S)"
###
# E-mail subject:
MAILTITLE1="There was a problem compiling GIT"
###
# E-mail address:
EMAILADDR="[email protected]"
###
###
CURRENTREV=`cd "$PATHTOACDIR" && git rev-parse master | cut -c 1-7`


echo "**Running update at: "$(date +%F_%H.%M.%S)"**" >> "$LOGTOPATH"

# Step 1: Clean *ONLY* the GIT source directories completely and fully. Then,
#         update them. This way, your the AssaultCube server keeps running:
cd "$PATHTOACDIR" && git fetch https://github.com/assaultcube/AC.git master >> "$LOGTOPATH" 2>&1
cd source && git clean -fdx ./ && git checkout -f master ./ >> "$LOGTOPATH" 2>&1
cd "$PATHTOACACDIR" && git clean -fdx && git reset --hard && git pull [email protected]:assaultcube/ACAC.git master >> "$LOGTOPATH" 2>&1
echo "\n***AC + ACAC SOURCES CLEANED AND UPDATED***\n" >> "$LOGTOPATH"

# Step 2: Compile new source code:
cd "$PATHTOACDIR"/source/src && cp "$PATHTOACACDIR"/src/* ./ && make >> "$LOGTOPATH" 2>&1

# Step 3: Check if compile was successful. If not, then STOP!
if [ -r "$PATHTOACDIR"/source/src/ac_server ] &&  [ -r "$PATHTOACDIR"/source/src/ac_client ]; then
 echo "\n***COMPILE SUCCESSFUL, CONTINUING...***\n" >> "$LOGTOPATH"
else
 echo "\n***COMPILE FAILED. PLEASE DEBUG!***" >> "$LOGTOPATH"
 echo "Your server is still running on GIT revision# "$CURRENTREV"." >> "$LOGTOPATH"
 cat "$LOGTOPATH" | mail -s "$MAILTITLE1" "$EMAILADDR"
 exit
fi

# Step 4: Move successful executables to the download spot:
cd "$PATHTOACDIR"/source/src
mv ac_server "$EXECSAVEPATHS"
mv ac_client "$EXECSAVEPATHC"

# Step 5: As compile was successful, clean-out/update the entire repository (not just ./source).
#         This means uploaded maps are deleted too:
kill -9 $(pidof "$NAMEOFEXEC")
cd "$PATHTOACDIR" && git clean -fdx && git reset --hard && git pull https://github.com/assaultcube/AC.git master >> "$LOGTOPATH" 2>&1
echo "\n***ALL FILES NOW WIPED/UPDATED***\n" >> "$LOGTOPATH"

# Step 6: Reboot the server to the new version:
cd "$PATHTOACDIR" && cp "$EXECSAVEPATHS" ./bin_unix/"$NAMEOFEXEC"
sh "$PATHTOACSHELLFILE"

# Step 7: Check that server ran OK. If not, warn user.
sleep 5
if [ -z $(pidof "$NAMEOFEXEC") ]; then
 echo "\n***OH DEAR, JUST CHECKED SOMETHING...***" >> "$LOGTOPATH"
 echo "YOUR SERVER DOESN'T SEEM TO BE RUNNING AT "$(date +%F_%H.%M.%S)". PLEASE DEBUG!" >> "$LOGTOPATH"
 cat "$LOGTOPATH" | mail -s "$MAILTITLE1" "$EMAILADDR"
else
 echo "***SERVER UPDATED/RUNNING OK!***" >> "$LOGTOPATH"
 echo "***UPDATE SCRIPT FINISHED! AT: "$(date +%F_%H.%M.%S)"***\n" >> "$LOGTOPATH"
fi

# Step 8: Check for and clean-up any dead screen sessions.
# Don't bother logging this!
screen -wipe

There were some other scripts, but nothing important... I think.