Category Archives: Unix

New Rootkit infects Linux Web servers

A previously unknown rootkit is infecting Linux web servers and injecting malicious code into web pages served by infected servers. The rootkit was discovered by a user of security mailing list Full Disclosure, who has posted his observations, including the suspicious kernel module, to the mailing list. The malware adds an iframe to every web page served by the infected system via the nginxproxy – including error pages.

Anyone who visits a web page on the server is then attacked by a specially crafted web page which is loaded in an iframe. Criminals typically use exploit kits such as BlackHole to examine the system of the victim to establish which one of a number of vulnerabilities in Flash, Java and other applications can be exploited. Once an exploitable hole is identified, it is used to install malware on the visitor’s system. The web server is ultimately being used to redirect users to another web server which can then infect their system, such as poorly maintained Windows systems, with malware.

Anti-virus software company Kaspersky Lab has analysed the malware. According to them, the rootkit, which it has dubbed Rootkit.Linux.Snakso.a, is designed to target 64-bit systems and has been compiled for kernel version 2.6.32-5, used in Debian Squeeze. The rootkit adds the line insmod /lib/modules/2.6.32 5-amd64/kernel/sound/module_init.ko to the /etc/rc.local script, ensuring that the malicious module is executed each time the system boots.

After booting, it determines the memory address of a number of kernel functions, which it then hooks into. This allows it both to hide itself from the user and to manipulate the server’s network traffic. The rootkit obtains deployment instructions from a command and control server. According to Kaspersky, the rootkit may still be under development, as it has been compiled with debug information in situ.

Security expert Georg Wicherski has also analysed the rootkit, and suggests that it was developed by an advanced beginner who does not yet have a great deal of experience with the kernel. According to Wicherski, the attacker who deployed the rootkit is probably based in Russia.

Attacking TrueCrypt Containers

TCHead decrypts and verifies the information in a TrueCrypt container’s header. Containers can range from files to copied system volumes from fully-encrypted hard drives. For this, you do of course need the password. If you don’t have the right one, TCHead can run through a word list. Unlike cracking tool John the Ripper, however, it is not able to systematically vary these details by, for example, converting lower case letters to upper case or converting letters to leetspeak.

Still faster than doing it by hand – TCHead tests out different passwords. TCHead also carries out this kind of dictionary attack very, very slooooooowly. In tests on a fairly fast computer, the tool required about a minute to run through 1,000 candidate passwords. By comparison, password crackers usually measure their speed in millions of attempts per second. This poor performance is largely due to the fact that TrueCrypt saves keys for testing internally using Password-Based Key Derivation Function 2 (PBKDF2), which is specifically designed to slow down these types of brute-force attacks.

TCHead is able to deal with standard encryption algorithms such as AES, Serpent and Twofish. TrueCrypt, however, also offers the option of using cascaded algorithms such as a combination of AES and Serpent. Our attempts to deploy TCHead against a combination container failed, with no error message, even though the target password was in the word list. The tool does not currently support mixed encryption algorithms, but that this is on their to-do list.

A statically linked Linux binary and source code, which we were able to compile under Ubuntu 12.04 LTS with a little tinkering, are available to download from the project’s site. There is also a script for building a Windows version using the g++ compiler, though we have not tested this.

In summary, TCHead is a useful addition to any forensic IT specialist’s collection and is one of the few available options for tackling encrypted TrueCrypt containers. But don’t expect too much: if the targeted TrueCrypt user followed even basic password rules, you don’t stand a chance.

Narwhal’s Guide to Command Line | Processes

2.1 Listing and PIDs

Each process has a unique number, the PID. A list of all running process is retrieved with ps.

# ps -auxefw                         # Extensive list of all running process

However more typical usage is with a pipe or with pgrep

# ps axww | grep cron
586  ??  Is     0:01.48 /usr/sbin/cron -s
# ps axjf                            # All processes in a tree format
# ps aux | grep ‘ss[h]’              # Find all ssh pids without the grep pid
# pgrep -l sshd                      # Find the PIDs of processes by (part of) name
# echo $$                            # The PID of your shell
# fuser -va 22/tcp                   # List processes using port 22
# pmap PID                           # Memory map of process (hunt memory leaks)
# fuser -va /home                    # List processes accessing the /home partition
# strace df                          # Trace system calls and signals

2.2 Background/Foreground

When started from a shell, processes can be brought in the background and back to the foreground with [Ctrl]-[Z] (^Z), bg and fg. List the processes with jobs. When needed detach from the terminal with disown.

# ping > ping.log
^Z                                   # ping is suspended (stopped) with [Ctrl]-[Z]
# bg                                 # put in background and continues running
# jobs -l                            # List processes in background
[1]  – 36232 Running                       ping > ping.log
[2]  + 36233 Suspended (tty output)        top
# fg %2                              # Bring process 2 back in foregroun

# make                               # start a long compile job but need to leave the terminal
^Z                                   # suspended (stopped) with [Ctrl]-[Z]
# bg                                 # put in background and continues running
# disown -h %1                       # detatch process from terminal, won’t be killed at logout

No straight forward way to re-attach the process to a new terminal, try reptyr (Linux).
Use nohup to start a process which has to keep running when the shell is closed (immune to hangups).

2.3 Top

The program top displays running information of processes. See also the program htop from (a more powerful version of top) which runs on Linux. While top is running press the key h for a help overview. Useful keys are:

  • u [user name] To display only the processes belonging to the user. Use + or blank to see all users
  • k [pid] Kill the process with pid.
  • 1 To display all processors statistics (Linux only)
  • R Toggle normal/reverse sort.

2.4 Signals/Kill

Terminate or send a signal with kill or killall.

# ping -i 60 > ping.log &
[1] 4712
# kill -s TERM 4712                  # same as kill -15 4712
# killall -1 httpd                   # Kill HUP processes by exact name
# pkill -9 http                      # Kill TERM processes by (part of) name
# pkill -TERM -u www                 # Kill TERM processes owned by www
# fuser -k -TERM -m /home            # Kill every process accessing /home (to umount)

Important signals are:

1       HUP (hang up)
2       INT (interrupt)
3       QUIT (quit)
9       KILL (non-catchable, non-ignorable kill)
15     TERM (software termination signal)

Next up in Narwhal’s Toolbox: File System

Narwhal’s Guide To Command Line | System

I’ve decided to write up a list of commands for Unix/Linux that are useful for IT work or for advanced users. This is a guide with concise explanations, however you are supposed to have some knowledge of Unix/Linux in order to use this guide to it’s full extent.

1.1 – Basic System Information

Running kernel and system information

# uname -a                           # Get the kernel version
# lsb_release -a                     # Full release info of any LSB distribution
# cat /etc/SuSE-release              # Get SuSE version
# cat /etc/debian_version            # Get Debian version

# uptime                             # Show how long the system has been running + load
# hostname                           # system’s host name
# hostname -i                        # Display the IP address of the host. (Linux only)
# man hier                           # Description of the file system hierarchy
# last reboot                        # Show system reboot history

1.2 – Hardware Information

Kernel Detected Hardware

# cat /proc/cpuinfo                  # CPU model
# cat /proc/meminfo                  # Hardware memory
# grep MemTotal /proc/meminfo        # Display the physical memory
# watch -n1 ‘cat /proc/interrupts’   # Watch changeable interrupts continuously
# free -m                            # Used and free memory (-m for MB)
# cat /proc/devices                  # Configured devices
# lspci -tv                          # Show PCI devices
# lsusb -tv                          # Show USB devices
# lshal                              # Show a list of all devices with their properties
# dmidecode                          # Show DMI/SMBIOS: hw info from the BIOS

1.3 – Load, statistics and messages

The following commands are useful to find out what is going on on the system.

# top                                # display and update the top cpu processes
# mpstat 1                           # display processors related statistics
# vmstat 2                           # display virtual memory statistics
# iostat 2                           # display I/O statistics (2 s intervals)
# tail -n 500 /var/log/messages      # Last 500 kernel/syslog messages
# tail /var/log/warn                 # System warnings messages see syslog.conf

1.4 – Users

# id                                 # Show the active user id with login and group
# last                               # Show last logins on the system
# who                                # Show who is logged on the system
# groupadd admin                     # Add group “admin” and user colin (Linux/Solaris)
# useradd -c “Colin Barschel” -g admin -m colin
# usermod -a -G <group> <user>       # Add existing user to group (Debian)
# groupmod -A <user> <group>         # Add existing user to group (SuSE)
# userdel colin                      # Delete user colin (Linux/Solaris)

Encrypted passwords are stored in /etc/shadow for Linux and Solaris. If the master.passwd is modified manually (say to delete a password), run # pwd_mkdb -p master.passwd to rebuild the database. (This has large implications especially for a Unix with an encrypted partition you could in theory modify the owners password in master.passwd giving you read/write access to the encrypted partition. Correct me if I’m wrong.)

1.5 – Limits

Some application require higher limits on open files and sockets (like a proxy
web server, database). The default limits are usually too low.

Per Shell/Script

# ulimit -n 10240                    # This is only valid within the shell

Per User/Process

# cat /etc/security/limits.conf
*   hard    nproc   250              # Limit user processes
asterisk hard nofile 409600          # Limit application open files

System Wide

# sysctl -a                          # View all system limits
# sysctl fs.file-max                 # View max open files limit
# sysctl fs.file-max=102400          # Change max open files limit
# echo “1024 50000” > /proc/sys/net/ipv4/ip_local_port_range  # port range
# cat /etc/sysctl.conf
fs.file-max=102400                   # Permanent entry in sysctl.conf
# cat /proc/sys/fs/file-nr           # How many file descriptors are in use

1.6 – Compile the Kernel

# cd /usr/src/linux
# make mrproper                      # Clean everything, including config files
# make oldconfig                     # Reuse the old .config if existent
# make menuconfig                    # or xconfig (Qt) or gconfig (GTK)
# make                               # Create a compressed kernel image
# make modules                       # Compile the modules
# make modules_install               # Install the modules
# make install                       # Install the kernel
# reboot

1.7 – Repair Grub

So you broke grub? Boot from a live cd, [find your linux partition under /dev and use fdisk to find the linux partion] mount the linux partition, add /proc and /dev and use grub-install /dev/xyz. Suppose linux lies on /dev/sda6:

# mount /dev/sda6 /mnt               # mount the linux partition on /mnt
# mount –bind /proc /mnt/proc       # mount the proc subsystem into /mnt
# mount –bind /dev /mnt/dev         # mount the devices into /mnt
# chroot /mnt                        # change root to the linux partition
# grub-install /dev/sda              # reinstall grub with your old settings

1.8 – Reset Root Password

At the boot loader (lilo or grub), enter the following boot option:


The kernel will mount the root partition and init will start the bourne shell
instead of rc and then a runlevel. Use the command passwd at the prompt to change the password and then reboot. Forget the single user mode as you need the password for that.

If, after booting, the root partition is mounted read only, remount it rw:

# mount -o remount,rw /
# passwd                             # or delete the root password (/etc/shadow)
# sync; mount -o remount,ro /        # sync before to remount read only
# reboot

Next up in Narwhal’s Toolbox: Process