Category Archives: Alienvault OSSIM

Firesight integration with OSSIM

Firesight, which is now Cisco, was originally developed by the same guys who wrote snort. The software runs the professional VRT snort feed underneath. The output is a slightly different format but we can tweak OSSIM to read in the syslog alerts.

1. In your Firesight intrusion policy click on Advanced Settings -> Syslog Alerting.

2. Type in the IP of your OSSIM server and assign a priority etc.

3. Save and push the policy to your Sourcefire nodes.

4. Now on the OSSIM box connect over SSH and select Jailbreak from the menu.

5. Now we need to send the alerts into the alerts file. Create a new file /etc/rsyslog.d/zzzzz_snort_syslog.conf and add in this text:

if $msg contains 'SFIMS' then -/var/log/snort/alert
& ~
if $syslogtag contains 'SFIMS' then -/var/log/snort/alert
#Stop
& ~

6. Now you need to edit the snort-syslog config file so that it can understand the Firesight format. Edit /etc/ossim/agent/plugins/snort_syslog.cfg and at the bottom add:

[05_snort-syslog-sourcefire-format]
event_type=event
regexp=(\w+\s+\d{1,2}\s+\d\d:\d\d:\d\d)\s+([a-zA-Z0-9\-]+)\s+[SFIMS:]{1,6}\s+\[([a-zA-Z0-9_\s]+)\s+\(([0-9a-z\-]+)\)\]\[(.+)\]\[(([0-9]+)\:([0-9]+)\:[0-9]+)\]\s+\"(.+)\"\s+\[Classification\:\s+(.+)\]\s+User\:\s+(.+)\,\s+Application\:\s+(.+)\,\s+Client:\s+(.+)\,\s+App Protocol\:\s+(.+)\,\s+Interface Ingress\:\s+([a-zA-Z\-\_0-9]+)\,\s+Interface Egress\:\s+([a-zA-Z\-\_0-9]+)\,\s+Security Zone Ingress\:\s+([a-zA-Z\-\_0-9]+)\,\s+Security Zone Egress\:\s+([a-zA-Z\-\_0-9]+)\,\s+Context\:\s+([a-zA-Z\-\_0-9]+)\,\s+\[Priority\:\s+([0-9]+)\]\s+\{([A-Z]+)\}\s+([0-9.]+):([0-9]+)\s->\s([0-9.]+):([0-9]+)
date={normalize_date($1)}
device={resolv($2)}
plugin_id=1001
plugin_sid={$8}
protocol={$21}
src_ip={$22}
src_port={$23}
dst_ip={$24}
dst_port={$25}
userdata1={$5}
userdata2={$4}
userdata3={$9}
userdata4={$15}
userdata5={$16}
userdata6={$17}
userdata7={$18}
userdata8={$20}

7. Save the file.

8. Now enable the collector. Type ossim-setup to load the ossim curses gui, choose Configure Sensor -> Configure Data Source Plugins -> Select snort-syslog then click OK -> Back -> Apply all Changes. This will start the snort-syslog collector.

9. Generate some dummy alerts in Sourcefire and then come back and check in the OSSIM GUI to see they have been processed.

Advertisements
Tagged , , ,

Keeping OSSIM DB tables in check

In every OSSIM install I have done I’ve found the built in system for keeping the database size to manageable level doesn’t work very well. Eventually several tables gobble up all the disk space and fill the mysql partition. To make matters worse the default install uses a single ibdata file which doesn’t get released when you delete entries from the table.

So, I wrote this script to periodically check several problem tables and remove the oldest entries when they get to a certain size. What this size is depends on your environment. The problem tables I found to be were:

  • alienvault.extra_data
  • alienvault_siem.extra_data
  • alienvault.host_qualification
  • alienvault_siem.acid_event

I’ve just copied and pasted the same code 4 times as I was feeling lazy. You can find the mysql root password in /etc/ossim/ossim_setup.conf. Input that into the MYSQL_PASS variable. By default to script looks for when the number of rows in the table is greated than the THRESHOLD_ROWS value of 5000000. It will remove the number of rows that are more than this up to 5000000 in one go, so you should probably set this to run frequently, say hourly in a busy system.


#!/bin/bash
MYSQL_USER="root"
MYSQL_PASS="123456789"
THRESHOLD_ROWS=5000000
TARGET_DB="alienvault"
TARGET_TABLE="extra_data"
TARGET_COL_ORDER="event_id"
ROWS_PER_RUN=5000000

echo "Getting row count from table $TARGET_DB.$TARGET_TABLE"
numrows=`mysql -u $MYSQL_USER --password=$MYSQL_PASS -e "select count($TARGET_COL_ORDER) from $TARGET_DB.$TARGET_TABLE" | grep -v count | grep -v "-"`

echo "Got $numrows rows from table $TARGET_DB.$TARGET_TABLE"

if [ $numrows -gt $THRESHOLD_ROWS ]
then
        let "rowstodelete = $numrows - $THRESHOLD_ROWS"
        if [ $rowstodelete -gt $ROWS_PER_RUN ]
        then
                echo "Deleting $ROWS_PER_RUN from table $TARGET_DB.$TARGET_TABLE"
                result=`mysql -u $MYSQL_USER --password=$MYSQL_PASS -e "delete from $TARGET_DB.$TARGET_TABLE order by $TARGET_COL_ORDER ASC LIMIT $ROWS_PER_RUN"`
        else
                echo "Deleting  $rowstodelete from table $TARGET_DB.$TARGET_TABLE"
                result=`mysql -u $MYSQL_USER --password=$MYSQL_PASS -e "delete from $TARGET_DB.$TARGET_TABLE order by $TARGET_COL_ORDER ASC LIMIT $rowstodelete"`
        fi
        echo "Result=$result (blank is good)"
else
        echo "Number of rows = $numrows which not more than $THRESHOLD_ROWS so nothing to do."
fi

TARGET_DB="alienvault_siem"
TARGET_TABLE="extra_data"
TARGET_COL_ORDER="event_id"

echo "Getting row count from table $TARGET_DB.$TARGET_TABLE"
numrows=`mysql -u $MYSQL_USER --password=$MYSQL_PASS -e "select count($TARGET_COL_ORDER) from $TARGET_DB.$TARGET_TABLE" | grep -v count | grep -v "-"`

echo "Got $numrows rows from table $TARGET_DB.$TARGET_TABLE"

if [ $numrows -gt $THRESHOLD_ROWS ]
then
        let "rowstodelete = $numrows - $THRESHOLD_ROWS"
        if [ $rowstodelete -gt $ROWS_PER_RUN ]
        then
                echo "Deleting $ROWS_PER_RUN from table $TARGET_DB.$TARGET_TABLE"
                result=`mysql -u $MYSQL_USER --password=$MYSQL_PASS -e "delete from $TARGET_DB.$TARGET_TABLE order by $TARGET_COL_ORDER ASC LIMIT $ROWS_PER_RUN"`
        else
                echo "Deleting  $rowstodelete from table $TARGET_DB.$TARGET_TABLE"
                result=`mysql -u $MYSQL_USER --password=$MYSQL_PASS -e "delete from $TARGET_DB.$TARGET_TABLE order by $TARGET_COL_ORDER ASC LIMIT $rowstodelete"`
        fi
        echo "Result=$result (blank is good)"
else
        echo "Number of rows = $numrows which not more than $THRESHOLD_ROWS so nothing to do."
fi

TARGET_DB="alienvault"
TARGET_TABLE="host_qualification"
TARGET_COL_ORDER="hex(host_id)"

echo "Getting row count from table $TARGET_DB.$TARGET_TABLE"
numrows=`mysql -u $MYSQL_USER --password=$MYSQL_PASS -e "select count(host_id) from $TARGET_DB.$TARGET_TABLE" | grep -v count | grep -v "-"`

echo "Got $numrows rows from table $TARGET_DB.$TARGET_TABLE"

if [ $numrows -gt $THRESHOLD_ROWS ]
then
        let "rowstodelete = $numrows - $THRESHOLD_ROWS"
        if [ $rowstodelete -gt $ROWS_PER_RUN ]
        then
                echo "Deleting $ROWS_PER_RUN from table $TARGET_DB.$TARGET_TABLE"
                result=`mysql -u $MYSQL_USER --password=$MYSQL_PASS -e "delete from $TARGET_DB.$TARGET_TABLE order by $TARGET_COL_ORDER ASC LIMIT $ROWS_PER_RUN"`
        else
                echo "Deleting  $rowstodelete from table $TARGET_DB.$TARGET_TABLE"
                result=`mysql -u $MYSQL_USER --password=$MYSQL_PASS -e "delete from $TARGET_DB.$TARGET_TABLE order by $TARGET_COL_ORDER ASC LIMIT $rowstodelete"`
        fi
        echo "Result=$result (blank is good)"
else
        echo "Number of rows = $numrows which not more than $THRESHOLD_ROWS so nothing to do."
fi

TARGET_DB="alienvault_siem"
TARGET_TABLE="acid_event"
TARGET_COL_ORDER="timestamp"

echo "Getting row count from table $TARGET_DB.$TARGET_TABLE"
numrows=`mysql -u $MYSQL_USER --password=$MYSQL_PASS -e "select count($TARGET_COL_ORDER) from $TARGET_DB.$TARGET_TABLE" | grep -v count | grep -v "-"`

echo "Got $numrows rows from table $TARGET_DB.$TARGET_TABLE"

if [ $numrows -gt $THRESHOLD_ROWS ]
then
        let "rowstodelete = $numrows - $THRESHOLD_ROWS"
        if [ $rowstodelete -gt $ROWS_PER_RUN ]
        then
                echo "Deleting $ROWS_PER_RUN from table $TARGET_DB.$TARGET_TABLE"
                result=`mysql -u $MYSQL_USER --password=$MYSQL_PASS -e "delete from $TARGET_DB.$TARGET_TABLE order by $TARGET_COL_ORDER ASC LIMIT $ROWS_PER_RUN"`
        else
                echo "Deleting  $rowstodelete from table $TARGET_DB.$TARGET_TABLE"
                result=`mysql -u $MYSQL_USER --password=$MYSQL_PASS -e "delete from $TARGET_DB.$TARGET_TABLE order by $TARGET_COL_ORDER ASC LIMIT $rowstodelete"`
        fi
        echo "Result=$result (blank is good)"
else
        echo "Number of rows = $numrows which not more than $THRESHOLD_ROWS so nothing to do."
fi

You can also download the script here.

Tagged ,

OSSIM directive taxonomy settings do not update / save

When you try to edit the Taxonomy settings for a user generated directive in OSSIM the changes do not save. Instead the webpage updates and shows the old settings.

This happened for me when I upgraded to 4.3.4.

To fix you can clear out the taxonomy values in the alarm_taxonomy table and then re-enter them using the webGUI. The problem seems to be that OSSIM adds a second entry to the table rather than updating the existing one.

1. SSH to the OSSIM box holding the mysql database
2. Backup your database before editing the tables
3. Then type

ossim-db
select * from alarm_taxonomy WHERE sid like '5000%';

This should list the taxonomy for your generated directives (since they’re all in the 50000 range. For the exact sids check the /etc/ossim/server/<GUID>/user.xml file.

Now to clear the problem directive that won’t update (for example sid number 500010)

delete from alarm_taxonomy WHERE sid='500010';

Now open the web interface and the taxonomy for that directive should have cleared. Now edit it and set it correctly and restart the ossim-server by clicking on the button at the top.

Your taxonomy settings should have updated OK.

Tagged ,

Alienvault OSSIM: Asset page broken after upgrading to 4.4

After upgrading OSSIM to 4.4.0 (or 4.4.1) the Asset section may show the error:

Operation was not completed due to an database error

If you then check the status of the table on the CLI you’ll find the table is missing!

alienvault:~# ossim-db
mysql> select * from asset limit 1;
ERROR 1146 (42S02): Table 'alienvault.asset' doesn't exist
mysql> quit

To resolve re-run the SQL upgrade script which should recreate your table (albeit empty):

cd /usr/share/ossim/include/upgrades
gunzip 4.4.0_mysql.sql.gz
gunzip 4.4.1_mysql.sql.gz
ossim-db < 4.4.0_mysql.sql
ossim-db < 4.4.1_mysql.sql

Then reload the Assets page and it should work.

Tagged , , ,

How to run Alienvault OSSIM 4.2 in (custom) text mode

This is also a fix for

  1. GUI installer hanging on “Configure network” when you try and enter the IP address
  2. Configuring disk setup
  3. Selecting which components to install

These options were available in 4.1 but were removed from the boot menu of the installer in 4.2.

The options are still there though. To run the custom text installer do the following:

  1. Boot from the OSSIM 4.2 CD
  2. At the installer menu highlight USM 4.2 (the top one)
  3. Hit the TAB button
  4. Edit the kernel boot line so it shows as (all one line)
/install.amd/vmlinux preseed/file=/cdrom/preseed debian/priority=low preseed/interactive=true vga=normal initrd=/install.amd/initrd.gz quiet ALLinONEauto --

5. Then hit enter to boot into custom text mode.

For the lazy out there you can also:

  1. Put the 4.1 installer CD in the CDROM and boot to the menu.
  2. Swap the CD over and put in the 4.2 CD
  3. Select custom text mode from the menu

It’ll then boot.

Q.E.D?

Tagged , , ,

Updating to OSSIM 4.1.3 causes ossim-agent not to start

On updating OSSIM via the update the ossim-agent starts and then stops. No logs are parsed and both /var/log/ossim/agent.log and /var/log/ossim/agent_error.log are empty or contain old information. Listing the processes shows that the agent is not running.

When the agent is started manually using

/usr/bin/ossim-agent -v 

the following error is logged:

OSError: [Errno 2] No such file or directory: '/etc/ossim/agent/host_cache_pro.dic

Looking in the /etc/ossim/agent directory there is no host_cache_pro.dic file but there is a host_cache.dic.

To fix, rename the host_cache.dic to host_cache.dic.old and restart the ossim-agent.

cd /etc/ossim/agent
mv host_cache.dic host_cache.dic.old
/etc/init.d/ossim-agent restart

The agent should now start and write to the agent.log and start processing.

PHP-IDS warning when submitting rule on Alienvault OSSIM 4.x

When building a new correlation rule in Alienvault OSSIM 4.x you may get an error like:

"Sorry, operation not completed due to security reasons. An attack attempt has been logged to the system"

PHP-IDS-CreateRuleOSSIM

This is caused by the PHP-IDS implementation within OSSIM and can be fixed by adding an exemption rule:

  1. In the error note the “Variable” that caused the error. In this example it was Get.product_list
  2. SSH to your OSSIM server
  3. Open the fileĀ /usr/share/ossim/include/php-ids.ini in your favourite editor.
  4. In the [General] section are a list of exceptions. Scroll to the bottom of the exceptions list and add a new entry:
exceptions[] = GET.product_list

5. Restart ossim-framework and try submitting the rule again.

service ossim-framework restart
Tagged ,

Transferring user-created correlation directives between servers on OSSIM 4

It takes a while to create correlation directives through the GUI so if you have to do this several times on different OSSIM servers it can get a bit tiresome. Here is how to transfer the directives you’ve created from one server to another:

1. Create your directive(s) on an OSSIM server
2. SSH to the server and cd to /etc/ossim/server/uid/, where uid is a generated string. This is specific to the install and will be different on the source and destination server.
3. In the folder is a file named user.xml. Open it using your favourite editor (vim?). This file contains the configuration for your directives in xml format. Copy and paste the contents into notepad ready to be transferred to the new OSSIM server.

The file starts with the xml header at the top:

<?xml version="1.0" encoding="UTF-8"?>

Each directive is enclosed in a <directive> tag, for example:

<directive id="500001" name="My example directive" priority="3">

Make a note of the ids of each directive, they’ll need to be distinct on the new server.

4. SSH to the new server and cd to /etc/ossim/server/uid/, where uid is a generated string.
5. Open the user.xml and if there are any rules created find the directive with the highest id. Go back to the directives you copied from the source server in notepad and amend the directive ids so they start on an id not in use. For instance if you highest id on the destination server is 50007 then start using 50008:

<directive id="500008" name="My example directive #1" priority="3">
<directive id="500009" name="My example directive #2" priority="3">
<directive id="500010" name="My example directive #3" priority="3">

6. Open the user.xml file on the destination server and scroll to the bottom. Paste the edited directives into the file.

7. Save the file and then restart ossim-server (service ossim-server restart).

8. Logon to OSSIM throught the GUI and check to see if your copied directives are in the list.

Tagged ,

Offline update of Alienvault OSSIM

Alienvault OSSIM has a built in upgrade mechanism for updates. However, not all installs exist in locations with an active internet connection. To get around this you can either

  1. Mirror the update repository locally down from Alienvault and hack the update script
  2. Download the CD/DVD and hack the update script!

This is a description of the latter method.

Start by downloading the CD from the Alienvault OSSIM website and mount the iso on the server to be updated.

When the alienvault-update runs it tried to download an update script. You can grab this from the website. For v4 this script is located atĀ http://data.alienvault.com/RELEASES/alienvault4_update-script. Download the script and write a copy to the OSSIM server to run manually.

The script uses apt-get to get the updates from alienvault but those packages are also on the CD we downloaded. Add in the CD as a source by typing:

apt-cdrom add

This adds in the CD as a source into /etc/apt/sources.list, for example:

deb cdrom:[Debian GNU/Linux 6.0.6 _Squeeze_ - Unofficial amd64 DVD Binary-1 20121002-12:02]/ squeeze main non-free

Comment out the other lines that refer to debian in this file by putting a hash (#) in front of them:

#deb http://ftp.us.debian.org/debian/ squeeze main contrib
#deb-src http://ftp.us.debian.org/debian/ squeeze main contrib
#deb http://security.debian.org/ squeeze/updates main contrib
#deb-src http://security.debian.org/ squeeze/updates main contrib

Save the file and open the downloaded update script. Find the part of the script that says “download-only” and remove that option. For instance:

apt-get dist-upgrade --download-only -y --force-yes

change to

apt-get dist-upgrade -y --force-yes

When you’ve changed all the lines with “download-only” in them you’re ready to run the update script.

As root run the script, for example:

sh ./alienvault4_update-script

and watch as the packages are updated. Once done reboot and verify the system has been updated by browsing to Configuration -> Sensors or by using the command

ossim-server -v
Tagged ,

Enabling Open Threat Exchange (AV-OTX) in Alienvault

The Alienvault website has several posts about Open Threat Exchange but I wasn’t able to find instructions on how to enable it. Eventually I found the option hidden away in the advanced menu.

Here’s how to set it up:

1. Open the OSSIM web interface and click on the Configuration menu then Main

2. Select the Advanced tab and then select Open Threat Exchange

3. Select Yes from the dropdown to contribute to OTX

4. Click on the activation link. This takes you to the Alienvault website.

5. Fill out the form and submit it.

6. You’ll then get an email from Alienvault with a confirmation link. Click it and you will get a page saying that you’ve activated OTX.

7. Now go back to the OSSIM page and click on the Send Now button. This evaluates the threats your system has picked up and it will then show you a page like this one:

8. Click Send Now to send the details to Alienvault so they can be distributed to others.

Tagged , , , , ,