The elasticsearch servers that run logstash.wikimedia.org will soon be upgraded from 1.7 to 2.3. As part of this upgrade we also need to upgrade kibana (the web interface that runs on logstash.wikimedia.org). The current version uses elasticsearch features that have been long deprecated (since 1.4) and as of 2.0 have been removed. I've setup a test environment in the beta cluster running this new version of elasticsearch along with the new version of kibana.
Be warned, kibana 4 is a complete rewrite. It needs to be evaluated for our purposes and we need to make sure it does what we need. I've recreated most of the dashboards that were linked from the homepage of logstash-beta.wmflabs.org to help with the evaluation. The first few took a short while, but due to the way kibana re-uses visualizations it became a good bit faster/easier after that.
The main feature i've noticed missing from our old dashboards is the trends panel. This is the part that told you how the volume of messages compared to the previous hour/day/week/etc. There is a long standing github issue about replacing this functionality but it has been stalled out. I don't personally use that part of the visualization much, but perhaps you do?
Please try it out, and give some feedback. https://kibana.wmflabs.org/
Erik B
That url should have been: https://kibana4.wmflabs.org/
On Thu, Jun 23, 2016 at 6:26 PM, Erik Bernhardson < ebernhardson@wikimedia.org> wrote:
The elasticsearch servers that run logstash.wikimedia.org will soon be upgraded from 1.7 to 2.3. As part of this upgrade we also need to upgrade kibana (the web interface that runs on logstash.wikimedia.org). The current version uses elasticsearch features that have been long deprecated (since 1.4) and as of 2.0 have been removed. I've setup a test environment in the beta cluster running this new version of elasticsearch along with the new version of kibana.
Be warned, kibana 4 is a complete rewrite. It needs to be evaluated for our purposes and we need to make sure it does what we need. I've recreated most of the dashboards that were linked from the homepage of logstash-beta.wmflabs.org to help with the evaluation. The first few took a short while, but due to the way kibana re-uses visualizations it became a good bit faster/easier after that.
The main feature i've noticed missing from our old dashboards is the trends panel. This is the part that told you how the volume of messages compared to the previous hour/day/week/etc. There is a long standing github issue about replacing this functionality but it has been stalled out. I don't personally use that part of the visualization much, but perhaps you do?
Please try it out, and give some feedback. https://kibana.wmflabs.org/
Erik B
On Thu, Jun 23, 2016 at 7:47 PM, Erik Bernhardson ebernhardson@wikimedia.org wrote:
That url should have been: https://kibana4.wmflabs.org/
On Thu, Jun 23, 2016 at 6:26 PM, Erik Bernhardson ebernhardson@wikimedia.org wrote:
The elasticsearch servers that run logstash.wikimedia.org will soon be upgraded from 1.7 to 2.3. As part of this upgrade we also need to upgrade kibana (the web interface that runs on logstash.wikimedia.org). The current version uses elasticsearch features that have been long deprecated (since 1.4) and as of 2.0 have been removed. I've setup a test environment in the beta cluster running this new version of elasticsearch along with the new version of kibana.
Be warned, kibana 4 is a complete rewrite. It needs to be evaluated for our purposes and we need to make sure it does what we need. I've recreated most of the dashboards that were linked from the homepage of logstash-beta.wmflabs.org to help with the evaluation. The first few took a short while, but due to the way kibana re-uses visualizations it became a good bit faster/easier after that.
The main feature i've noticed missing from our old dashboards is the trends panel. This is the part that told you how the volume of messages compared to the previous hour/day/week/etc. There is a long standing github issue about replacing this functionality but it has been stalled out. I don't personally use that part of the visualization much, but perhaps you do?
Please try it out, and give some feedback. https://kibana.wmflabs.org/
Thanks for working on this Erik!
Kibana4 is a complete rewrite from Kibana3 (and I think the 5th or 6th ground up rewrite of Kibana in its history). Elastic.co seems to have high hopes for this iteration providing a platform for better visualizations and performance. As a rewrite there are some feature regressions that haven't been worked out upstream yet. We (well I) didn't upgrade us to Kibana4 for over a year due to a lack of support for pinning the dates shown in the UI to a particular timezone [0]. Thankfully that regression has been fixed in the version that Erik is demoing in the beta cluster. I haven't found any other regressions that are worthy of blocking yet, and that's good because we really do have to upgrade in order to have a sane state of services in production.
There are a few issues that I have noticed that others may find as well:
* No support for "markers" on a time histogram [1]. These were used in Kibana3 to show when scap changed files on the servers. The markers weren't highly reliable so its not the end of the world that this is missing for now.
* Date picker works in local browser TZ instead of UTC [2]. Honestly I think this bug or a variant of it exists in our Kibana3 install as well. Working with dates is a pain in just about every language. :/
There are some neat new features too, like permalinks to individual log events and quite a few new visualization types [3].
[0]: https://github.com/elastic/kibana/issues/1600 [1]: https://github.com/elastic/kibana/issues/2706 [2]: https://github.com/elastic/kibana/issues/5370 [3]: https://www.elastic.co/guide/en/kibana/4.0/whats-new.html
Bryan
On Fri, Jun 24, 2016 at 12:01 PM, Bryan Davis bd808@wikimedia.org wrote:
- Date picker works in local browser TZ instead of UTC [2]. Honestly I
think this bug or a variant of it exists in our Kibana3 install as well.
It does, I always have to convert UTC to local time when I'm trying to get Kibana to tell me the log events around some error; then Kibana displays the results in UTC again in the timeline and so on. :/
wikitech-l@lists.wikimedia.org