January 10, 2015

Kibana, ElasticSearch, logstash issue

Kibana is ElasticSearch's logs and time-stamped data visualization tool.

For development environment, I have installed XAMPP on Win7.  I tried Kibana 3.1.2 uner XAMPP's Apache, but somehow it didn't work -- just a blank page.  I switched to Kibana 4 beta 3 (standalone).

Environment: Win7 64bit, Java 1.8.0_25-b18 64bit, ElasticSearch 1.4.2, Kibana 4 beta 3, logstash 1.4.2.

Installing Kibana 4 beta 3 is straight forward, just uncompress it and run .bat file.  But Kibana shows this error message (http://localhost:5601) - Kibana: This version of Kibana requires Elasticsearch 1.4.0 or higher on all nodes. I found the following incompatible nodes in your cluster :

After some googling, found this:

In summary:
You have a logstash that uses ES version 1.1.1, and it reports as a node via node status of ES API, and therefore Kibana gives this error message.

Check your nodes, http://localhost:9200/_nodes -- it'll show logstash and ES version 1.1.1.

Shut down the node (using ES Head plugin, or issue this command), and refresh Kibana page.  It doesn't give the error message any more:

To use logstash, get ES (1.4 or later) jar file, and replace it in logstash -- Sidr found this workaround:
  1. Stop logstash.
  2. Get ElasticSearch zip, http://www.elasticsearch.org/overview/elkdownloads/
  3. Uncompress it and put it in <logstash home>\vendor\jar\ directory.
  4. Remove elasticsearch-1.1.1 directory from logstash.  
  5. Start logstash.
 After step #3, it should look like this:

And Kibana can be configured as a Windows service using nssm:

Now all seems to be good:

January 9, 2015

logstash on Windows

It's easier for me to use Linux for development, but often times I find myself in the situation to run popular tools on Windows.

logstash is a popular tool for logs and event managing. 

To install logstash on Windows, modify and save this configuration file.  Below is just an example to be used with a local elasticsearch.  Make sure it's in ANSI encoding, not UTF-8.  I saved it as C:\opt\logstash-1.4.2\logstash.conf  Please note that below configuration is just an example.  Please read logstash documents:

input { stdin { } }
output {
  elasticsearch { host => localhost }
  stdout { codec => rubydebug }

For Windows event logs, use this:

input {
  eventlog {
    type => 'Win32-EventLog'
    logfile => 'System'

Run it:

C:\> cd \opt\logstash-1.4.2\
C:\opt\logstash-1.4.2>bin\logstash.bat agent -f logstash.conf

To make this as a Windows service, there are some utilities, mainly I found these are the easiest to use:
I like nssm's approach is more Windows-like and simpler.  See NSSM usage page: http://nssm.cc/usage  Run it by issuing this command: nssm install
GUI window will appear, and configure it as following:


Just configure Application and Environment tabs.  Check services:


IE11 and JSON page (e.g. ElasticSearch)

I'm more familar with Linux with FireFox, but trying to use more of Windows and its native programs, such as IE.  I also use three different browsers at the same time -- FF(FireFox), IE, Opera, for example, for testing and other reasons.

When I tried open ElasticSearch page (http://localhost:9200) using IE11, I was surprised to find that IE tries to download JSON, instead of showing it like FireFox. 


Environment: Win7 64-bit.

Tried to view http://localhost:9200 on IE11 and got this:

Three solutions to this:
  1. Use JSON viewer, http://jsonviewer.codeplex.com/
  2. Registry change, http://www.codeproject.com/Tips/216175/View-JSON-in-Internet-Explorer
  3. Just click "Open", and select a text editor.  And next time, it'll open it with the editor instead.
I went with #3 solution - it's the simplest, and I don't like changing registry.  #1 is nice, but don't need it for now.