20-21 November 2014

15Hatfields, London, UK

Supercharging Cyanite with Clojure's core.async

Session Title

Supercharging Cyanite with Clojure's core.async

Session Type Experience Report
Duration 60 minutes
Session Description

Join me for a rapid journey through making metrics move rapidly. The medium in question for these metrics being Cyanite, an open source Clojure project that replaces Graphites default storage engine, Carbon, with a combination of Cassandra and ElasticSearch.

We'll look at the overall data flow, before looking at how it got dialled up to 11 using Clojure's core.async, custom channels, changing database clients and modifying libraries to suit our needs. Expect code, graphs, words and, in the end, lots of metrics!

The session will walk through the performance enhancements added to the Cyanite project over the last few weeks. The initial conversation can be found on github: https://github.com/pyr/cyanite/issues/40

The final pull request and sorting out can be seen here: https://github.com/pyr/cyanite/pull/41

That main focus of this experience report will be putting Clojure's core.async library to work. The following points will form the basis of the session:

  • Showing the impact of different go channel creation styles. Keep them to a minimum.
  • The custom timer buffer I developed, with a look at the initial version that contained a subtle memory leak. Highlight the ease with which you can add new implementations.
  • Visualising the multi core utilisation of using go blocks in this fashion.
  • The importance of making all external calls asynchronous.

The last point will led into the work done to the database clients:

  • Modifying ClojureWerks Elastisch client using http-kit instead of the standard http-client.
  • Performance findings from using the Cassandra client Alia with the Hayt library and the simple solution to those problems.

Underlying all these points will be the methodology of performance tuning and measuring instead of guessing about performance.

This session will aim to provide some concrete tips for this kind of work and to show, rather than tell, what the impact of using these non-blocking techniques can be.

Speaker(s)

Tom Coupland (MixRadio)

Tom Coupland is a developer at MixRadio Bristol. Having spent years working with transaction processing monoliths, producing one of his own in the process, he’s now enjoying the world of SOA and the freedoms it can bring. After subjective eons working with Java, he set out to find more productive and elegant tools.

Currently he can be found basking in the joy that is Clojure during the day, while keeping an eye out for other suitors in the evening. When not trying to make his bit of the world of software a better place, he can be found out in the real one, running around, preferably in the mud, wind and rain.

Support FP Days 2014

In return for promoting FP Days 2014 to members of your user group, we offer a 10% discount on all tickets to group members.

We'll also help to promote your group by featuring it here on the FP Days website.

If you run a user group and you'd like more information on supporting FP Days 2014, please get in touch!


Contact Cara

Cara Hanman
×

FP Days 2014 Session Types

Need help planning which sessions to attend? We've provided a breakdown of our various session types below.


Case Study/Experience Report

A presentation and discussion of real-life (not theoretical) experiences of the application (or mis-application) of service design techniques. Case studies and experience reports include some discussion of lessons learned and an indication of how novel the work is.

Hands-On

Participants learn a new approach, tool or technology through using it to solve one or more practical exercises. Any software/hardware requirements are disclosed in the session description.

Tutorial

A session focused around some specific tool, technique or issue. Primarily led by the speaker, tutorials usually include some elements of interactivity or individual / group exercise.

Workshop

An in-depth working session on a specific topic. May include paper presentations.

×