Logstash: Received an event that has a different character encoding


When using logstash you may see an error like this one:

Received an event that has a different character encoding than you configured. {:text=>"1.2.3.4\\t\\\"www.google.com\\\"\\t-\\t-\\t[01/Feb/2014:11:45:56 +0000]\\t\\\"-\\\"\\t\\\"GET /index.html\\xA0 HTTP/1.1\\\"\\t404\\t14015\\t\\\"80778000924267169,0:1:1\\\"\\tN\\t0.041725\\t0.040730\\t0.000695", :expected_charset=>"UTF-8", :level=>:warn}

This is because the default charset is UTF-8 and the incoming message contained a character not in the UTF-8 set, for example special characters:

\xA0                 non-breaking space
\xA3                 £

To fix this set the charset in the input section using codec and the correct charset. For example for

file {
                path => "var/log/http/access_log"
                type => apache_access_log
                codec => plain {
                        charset => "ISO-8859-1"
                }
                stat_interval => 60
}

For a full list of charset options you can use check out the website.

Advertisements
Tagged

One thought on “Logstash: Received an event that has a different character encoding

  1. FranzB says:

    Thank you Jordan this was very useful

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: