The previous episode demonstrated the caching of HTTP API responses. But, what if we want our cache to be shared among multiple Ruby processes? What if we want to use an external data store for it, such as memcached, redis, or our SQL database?
In this episode, you’ll learn about an extraordinarily useful Ruby library which enables you to easily drop in any of these data stores—and many others besides—to serve as a cache backend. You’ll also see how this library makes it simple to impose cache expiration policies as well.
Episode Script
In the last episode we added a simple caching layer to a class that hit an HTTP API. We used Ruby’s Hash as a model for the caching API, and in that episode we only used Hashes as the cache objects. Of course, as caching implementations go a Hash is pretty limited. It isn’t persistent, so the cache won’t survive past the life of the current process. It can’t be shared among multiple processes. And there’s no way to expire old cache entries.
require 'open-uri'
require 'json'
class Weather
Report = Struct.new(:temperature)
def initialize(options={})
@cache = options.fetch(:cache){ {} }
end
def report(query)
key = ENV['WUNDERGROUND_KEY']
url = "http://api.wunderground.com/api/#{key}/conditions/q/#{query}.json"
body = @cache.fetch(query) {
@cache[query] = open(url).read
}
data = JSON.parse(body)
Report.new(data['current_observation']['temp_f'])
end
end
To address all of these issues, we’ll use a gem called Moneta. Moneta provides a simple abstraction layer over many different key-value stores. No matter what storage back-end is chosen, a Moneta object supports the same core interface: a superset of the Ruby Hash interface.
Just to demonstrate what I mean by this, let’s instantiate a Moneta object backed by a YAML store. Then we’ll store some values in it. We do this exactly the same way we’d store values in a Hash.
require 'moneta'
store = Moneta.new(:YAML, file: "store.yaml")
store['question'] = "Life, the universe, and everything"
store['answer'] = 42
When we examine the store.yaml file, we can see the values have been stored in YAML form.
---
question: Life, the universe, and everything
answer: 42
Now let’s switch the Moneta object to be backed by a directory instead of a YAML file.
require 'moneta'
store = Moneta.new(:File, dir: "store")
store['question'] = "Life, the universe, and everything"
store['answer'] = 42
After executing this code, we find a new directory has been created. Inside it there is a file for each key we set.
tree store
store ├── answer └── question 0 directories, 2 files
I’ve shown off the YAML and File backends because they are easy to demonstrate with little setup. But Moneta has backends for a vast array of data stores, including Memcache, Redis, various SQL databases, and many others.
Now that we know how to use a Moneta object, let’s use one in our Weather class. We’ll replace the default Hash cache with a YAML-backed Moneta object, with an option setting the filename to “weather.yaml”. Then we’ll ask for the current weather:
require 'moneta'
cache = Moneta.new(:YAML, file: "weather.yaml")
puts Weather.new(cache: cache).report('17361')
After the request is made, we can see that the “weather.yaml” file has been created, and it contains data fetched from the weather API:
ls *.yaml
cat weather.yaml
067-moneta.yaml
store.yaml
weather3.yaml
weather_expires.yaml
weather.yaml
---
'17361': ! "\n{\n\t\"response\": {\n\t\t\"version\": \"0.1\"\n\t\t,\"termsofService\":
\"http://www.wunderground.com/weather/api/d/terms.html\"\n\t\t,\"features\": {\n\t\t\"conditions\":
1\n\t\t}\n\t}\n\t\t,\t\"current_observation\": {\n\t\t\"image\": {\n\t\t\"url\":\"http://icons-ak.wxug.com/graphics/wu2/logo_130x80.png\",\n\t\t\"title\":\"Weather
Underground\",\n\t\t\"link\":\"http://www.wunderground.com\"\n\t\t},\n\t\t\"display_location\":
{\n\t\t\"full\":\"Shrewsbury, PA\",\n\t\t\"city\":\"Shrewsbury\",\n\t\t\"state\":\"PA\",\n\t\t\"state_name\":\"Pennsylvania\",\n\t\t\"country\":\"US\",\n\t\t\"country_iso3166\":\"US\",\n\t\t\"zip\":\"17361\",\n\t\t\"latitude\":\"39.76274109\",\n\t\t\"longitude\":\"-76.67727661\",\n\t\t\"elevation\":\"293.00000000\"\n\t\t},\n\t\t\"observation_location\":
{\n\t\t\"full\":\"Freedom Hills, New Freedom, Pennsylvania\",\n\t\t\"city\":\"Freedom
Hills, New Freedom\",\n\t\t\"state\":\"Pennsylvania\",\n\t\t\"country\":\"US\",\n\t\t\"country_iso3166\":\"US\",\n\t\t\"latitude\":\"39.751175\",\n\t\t\"longitude\":\"-76.696236\",\n\t\t\"elevation\":\"912
ft\"\n\t\t},\n\t\t\"estimated\": {\n\t\t},\n\t\t\"station_id\":\"KPANEWFE2\",\n\t\t\"observation_time\":\"Last
Updated on January 31, 5:41 PM EST\",\n\t\t\"observation_time_rfc822\":\"Thu, 31
Jan 2013 17:41:19 -0500\",\n\t\t\"observation_epoch\":\"1359672079\",\n\t\t\"local_time_rfc822\":\"Thu,
31 Jan 2013 17:41:20 -0500\",\n\t\t\"local_epoch\":\"1359672080\",\n\t\t\"local_tz_short\":\"EST\",\n\t\t\"local_tz_long\":\"America/New_York\",\n\t\t\"local_tz_offset\":\"-0500\",\n\t\t\"weather\":\"Mostly
Cloudy\",\n\t\t\"temperature_string\":\"32.0 F (0.0 C)\",\n\t\t\"temp_f\":32.0,\n\t\t\"temp_c\":0.0,\n\t\t\"relative_humidity\":\"61%\",\n\t\t\"wind_string\":\"From
the SSE at 1.0 MPH\",\n\t\t\"wind_dir\":\"SSE\",\n\t\t\"wind_degrees\":167,\n\t\t\"wind_mph\":1.0,\n\t\t\"wind_gust_mph\":0,\n\t\t\"wind_kph\":1.6,\n\t\t\"wind_gust_kph\":0,\n\t\t\"pressure_mb\":\"1007\",\n\t\t\"pressure_in\":\"29.75\",\n\t\t\"pressure_trend\":\"+\",\n\t\t\"dewpoint_string\":\"20
F (-7 C)\",\n\t\t\"dewpoint_f\":20,\n\t\t\"dewpoint_c\":-7,\n\t\t\"heat_index_string\":\"NA\",\n\t\t\"heat_index_f\":\"NA\",\n\t\t\"heat_index_c\":\"NA\",\n\t\t\"windchill_string\":\"32
F (0 C)\",\n\t\t\"windchill_f\":\"32\",\n\t\t\"windchill_c\":\"0\",\n\t\t\"feelslike_string\":\"32
F (0 C)\",\n\t\t\"feelslike_f\":\"32\",\n\t\t\"feelslike_c\":\"0\",\n\t\t\"visibility_mi\":\"10.0\",\n\t\t\"visibility_km\":\"16.1\",\n\t\t\"solarradiation\":\"1782\",\n\t\t\"UV\":\"0\",\n\t\t\"precip_1hr_string\":\"0.00
in ( 0 mm)\",\n\t\t\"precip_1hr_in\":\"0.00\",\n\t\t\"precip_1hr_metric\":\" 0\",\n\t\t\"precip_today_string\":\"0.02
in (1 mm)\",\n\t\t\"precip_today_in\":\"0.02\",\n\t\t\"precip_today_metric\":\"1\",\n\t\t\"icon\":\"mostlycloudy\",\n\t\t\"icon_url\":\"http://icons-ak.wxug.com/i/c/k/nt_mostlycloudy.gif\",\n\t\t\"forecast_url\":\"http://www.wunderground.com/US/PA/Shrewsbury.html\",\n\t\t\"history_url\":\"http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KPANEWFE2\",\n\t\t\"ob_url\":\"http://www.wunderground.com/cgi-bin/findweather/getForecast?query=39.751175,-76.696236\"\n\t}\n}\n"
Now, the code as it is written right now will never expire cache entries. Optimistically assuming that the temperature in my town will not stay at 32 degrees Fahrenheit until the end of time, we should probably cause old cached data to be thrown out after a certain amount of time.
The simplest way to do this is to pass the expires option to the Moneta initializer. We can pass a number of seconds as the value, which will become the default time-out for cache entries. Here we’re making entries expire after 5 minutes.
Moneta.new(:YAML, file: "weather3.yaml", expires: 300)
That’s all there is to it: from now on, if a value older than 5 minutes is fetched from the cache, the value will be discarded and our Weather class will re-fetch the data.
So, by using a Hash as the model for our cache interface, we were able to drop in a Moneta store with zero code changes to the weather class. This means we can easily switch to caching using Memcache, Redis, an SQL database table, or pretty much any other store we might want to use.
There are many more features worth exploring in Moneta. For instance, it comes with a large selection of middlewares for more advanced configurations, such as logging cache access, stacking multiple backend stores, using an alternative serialization format, or compressing the stored data. Check out the Moneta page on Github for more information.
That’s all I have for today. Happy hacking!