testing stuff

Gemfile

+source "http://rubygems.org"
+gem "sinatra", "1.1.2"
+gem "sinatra-reloader", "0.5.0"
+gem "ohm", "0.1.3"
+gem "ohm-contrib", "0.1.0"

Gemfile.lock

+GEM
+  remote: http://rubygems.org/
+  specs:
+    backports (1.18.2)
+    monkey-lib (0.5.4)
+      backports
+    nest (1.1.0)
+      redis (~> 2.1)
+    ohm (0.1.3)
+      nest (~> 1.0)
+    ohm-contrib (0.1.0)
+      ohm
+    rack (1.2.1)
+    redis (2.1.1)
+    sinatra (1.1.2)
+      rack (~> 1.1)
+      tilt (~> 1.2)
+    sinatra-advanced-routes (0.5.1)
+      monkey-lib (~> 0.5.0)
+      sinatra (~> 1.0)
+      sinatra-sugar (~> 0.5.0)
+    sinatra-reloader (0.5.0)
+      sinatra (~> 1.0)
+      sinatra-advanced-routes (~> 0.5.0)
+    sinatra-sugar (0.5.0)
+      monkey-lib (~> 0.5.0)
+      sinatra (~> 1.0)
+    tilt (1.2.1)
+
+PLATFORMS
+  ruby
+
+DEPENDENCIES
+  ohm (= 0.1.3)
+  ohm-contrib (= 0.1.0)
+  sinatra (= 1.1.2)
+  sinatra-reloader (= 0.5.0)

README.md

-_(looking for the real code? See the section "Where's the code?")_
+# Noah testing quickstart
+(make sure redis is running)
 
-# Noah
-Noah is a lightweight registry based on the concepts in the Apache Zookeeper project.
+## Setup
+Edit setup.rb and config/db.rb if you want to point to a different db on your local redis install
 
-## Impetus
-Apache Zookeeper is an amazing peice of software. From the project's own description:
+_ruby setup.rb_
 
-> ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. All of these kinds of services are used in some form or another by distributed applications.
+	Creating Host entry for 'localhost'
+	Create Service entry for redis
+	Create Service entry for noah
+	Creating Application entry for 'noah'
+	Creating Configuration entry for 'noah'
+	Setup successful!
 
-The problem is that zookeeper is *BIG*. It has quite a bit of overhead associated with it and it's not easily accessible from anything other than C or Java. You could argue that you need to be big when this service is the crux of your infrastructure but that's confusing big with highly available and highly scalable.
+## Run it
+_rackup config.ru_
 
-_(yes I'm aware that you can run Zookeeper in single server mode from a single JAR file)_
+	[2011-01-17 08:00:30] INFO  WEBrick 1.3.1
+	[2011-01-17 08:00:30] INFO  ruby 1.9.2 (2010-12-25) [x86_64-linux]
+	[2011-01-17 08:00:30] INFO  WEBrick::HTTPServer#start: pid=15349 port=9292
 
-## So why reinvent the wheel?
-It's something I've wanted to do for a while. Everytime I've needed something like Zookeeper, Zookeeper has always been too bulky and had too many moving parts. I've also always needed to interact with it from more than just Java or C. Sometimes it's been Ruby and sometimes it's been Python.
 
-In the end, we reinvent the wheel ANYWAY. Maybe we do something like have our CM tool write our application config files with a list of memcached hosts. Maybe we write our own logic around (shudder) RMI to do some chatty broadcasting around the network for finding local nodes of type X. We always reinvent the wheel in some way.
-
-## Design Goals
-I have a few basic design goals:
-- The system must support RESTful interaction for operations where REST maps properly
-- The system must support basic concepts of hosts, services and configuration parameters
-- The system must support horizontal scaling
-- The system must support traditional load balancing methodologies
-- The system must support ephemeral clients and stateless operation where applicable
-
-In essence, the system must be resemble your typical web application.
-Here are some "would really like to have" things:
-- The system should support watches similar to ZooKeeper
-- The system should support pluggable callback methods (more below)
-- The system should support being a client of itself
-
-## Opinionated stack
-I've found that being opinionated has its benefits. Swappable storage backends are nice but those have to evolve naturally over time. To this end I've tried to keep the stack as lightweight and easy to distribute as possible. I'm also a big fan of the Unix philosophy of doing one thing and doing it very well. To this end, I've chosen the followng initial stack:
-- Ruby
-- EventMachine/Sinatra/Ohm
-- Redis
-The main reason for choosing Redis is that much of the functionality I need/that I want to implement from ZooKeeper is already available in Redis semantics. Essentially Noah becomes a custom API for Redis - a way to provide disconnected client operations for it. By using Ohm, I get an easy and unobtrusive orm that only does what I tell it to do. Sinatra was originally going to be the only inteface but as I thought about how best to handle the Watch scenario and callbacks, I realized I needed something a bit more evented - hence EventMachine.
-
-## Pretty pictures
-This is the original Mindmap I did when thinking about Noah. It's not complete and was just a dump of what was in my head at the time:
-
-[Noah MindMap](https://github.com/lusis/Noah/raw/master/doc/noah-mindmap-original.png)
-
-## Where's the code?
-I'm still working on the codebase in a private local branch. Sorry. I'm going to be flailing around a bit trying a few things and don't want people to fork yet. I'll still commit regularly to that branch so you'll see said flailing in the history when I merge. What I've done is create a simple sinatra app that you can get a feel for where I'm headed with this. It's under the branch called "testing".
+## All configs
+curl http://localhost:9292/c/
+## All services
+curl http://localhost:9292/s/
+## All hosts
+curl http://localhost:9292/h/
+## All applications
+curl http://localhost:9292/a/

app.rb

+require 'sinatra/reloader' if development?
+require 'ohm'
+require 'json'
+
+require File.join(File.dirname(__FILE__), 'config/db')
+require File.join(File.dirname(__FILE__), 'models')
+
+# Read operations
+## Hosts
+### Still not sold on this pathing structure
+get '/h/:hostname/s/:servicename/?' do
+  host = Host.find(:name => params[:hostname]).first
+  service = Service.find(:name => params[:servicename], :host_id => host.id).first
+  "#{service.to_json}"
+end
+
+get '/h/:hostname/s/?' do
+  host = Host.find(:name => params[:hostname]).first
+  services = []
+  Service.find(:host_id => host.id).sort.each {|s| services << s.to_hash}
+  "#{services.to_json}"
+end
+
+
+get '/h/:hostname/?' do
+  host = Host.find(:name => "#{params[:hostname]}").first
+  "#{host.to_json}"
+end
+
+# Catchall
+get '/h/?' do
+  hosts = []
+  Host.all.sort.each {|h| hosts << h.to_hash}
+  "#{hosts.to_json}"
+end
+
+## Services
+get '/s/:servicename/h/:hostname/?' do
+  s = Service.find(:name => "#{params[:servicename]}").first
+  h = Host.find(:name => "#{params[:hostname]}", :service_id => s.id).first
+  "#{h.to_json}"
+end
+
+get '/s/:servicename/?' do
+  services = []
+  Service.find(:name => "#{params[:servicename]}").sort.each {|s| services << s.to_hash}
+  "#{services.to_json}"
+end
+
+get '/s/?' do
+  services = []
+  Service.all.sort.each {|s| services << s.to_hash}
+  "#{services.to_json}"  
+end
+
+# Applications
+get '/a/:appname/:config/?' do
+  app = Application.find(:name => params[:appname]).first
+  c = Configuration.find(:name => params[:config], :application_id => app.id).first
+  "#{c.to_json}"
+end
+
+get '/a/:appname/?' do
+  app = Application.find(:name => params[:appname]).first
+  "#{app.to_json}"
+end
+
+get '/a/?' do
+  apps = []
+  Application.all.sort.each {|a| apps << a.to_hash}
+  "#{apps.to_json}"
+end
+
+## Configs
+get '/c/:appname/:element/?' do
+  a = Application.find(:name => params[:appname]).first
+  c = Configuration.find(:name => params[:element], :application_id => a.id).first
+  "#{c.to_json}"
+end
+
+get '/c/:appname/?' do
+  config = []
+  a = Application.find(:name => params[:appname]).first
+  Configuration.find(:application_id => a.id).sort.each {|c| config << c.to_hash}
+  "#{config.to_json}"
+end
+
+get '/c/?' do
+  configs = []
+  Configuration.all.sort.each {|c| configs << c.to_hash}
+  "#{configs.to_json}"
+end
+
+
+# A route for adding a new host
+put '/h/:hostname' do
+  host = params[:hostname]
+  data = JSON.parse(request.body.read)
+  begin
+    h = Host.find(:name => host).first
+    h.state = 1
+  rescue
+    # If we have to register the host, we mark it as inactive
+    # Feels like the right thing to do in case of accidental creation
+    # from mispelling
+    h = Host.create(:name => params[:hostname], :state => 0)
+  end
+  if h.save
+    data['services'].each {|x| h.services << Service.create(:name=> x['name'], :state => x['state'], :host => h)} if data['services']
+    {"msg" => "success"}.to_json
+  end
+end

config.ru

+require 'rubygems'
+require 'sinatra'
+
+require File.join(File.dirname(__FILE__), 'app')
+
+set :env, :development
+set :root, File.dirname(__FILE__)
+set :server, %[thin mongrel webrick]
+disable :run
+
+set :raise_errors, true
+run Sinatra::Application

config/db.rb

+Ohm.connect(:url => 'redis://localhost:6379/0')

models.rb

+require 'ohm/contrib'
+
+class Host < Ohm::Model
+  include Ohm::Typecast
+  include Ohm::Timestamping
+  include Ohm::Callbacks
+
+  attribute :name
+  attribute :state, Boolean
+  collection :services, Service
+
+  index :name
+  index :state
+
+  def validate
+    assert_present :name, :state
+    assert_unique :name
+  end
+
+  def to_hash
+    arr = []
+    services.sort.each {|s| arr << s.to_hash}
+    super.merge(:name => name, :state => state, :updated_at => updated_at, :services => arr)
+  end
+
+end
+
+class Service < Ohm::Model
+  include Ohm::Typecast
+  include Ohm::Timestamping
+  include Ohm::Callbacks
+
+  attribute :name
+  attribute :state, Boolean
+  reference :host, Host
+
+  index :name
+  index :state
+
+  def validate
+    assert_present :name, :state
+    assert_unique [:name, :host_id]
+  end
+
+  def to_hash
+    super.merge(:name => name, :state => state, :updated_at => updated_at, :host => Host[host_id].name)
+  end
+end
+
+class Configuration < Ohm::Model
+  include Ohm::Typecast
+  include Ohm::Timestamping
+  include Ohm::Callbacks
+
+  attribute :name
+  attribute :format
+  attribute :body
+  reference :application, Application
+
+  index :name
+
+  def validate
+    assert_present :name
+    assert_present :format
+    assert_present :body
+    assert_unique [:name, :application_id]
+  end
+
+  def to_hash
+    super.merge(:name => name, :format => format, :body => body, :update_at => updated_at, :application => Application[application_id].name)
+  end
+end
+
+class Application < Ohm::Model
+  include Ohm::Typecast
+  include Ohm::Timestamping
+  include Ohm::Callbacks
+
+  attribute :name
+  collection :configurations, Configuration
+
+  index :name
+
+  def validate
+    assert_present :name
+    assert_unique :name
+  end
+
+  def to_hash
+    super.merge(:name => name, :updated_at => updated_at)
+  end
+end
+
+class Watcher < Ohm::Model #NYI
+  include Ohm::Typecast
+  include Ohm::Timestamping
+  include Ohm::Callbacks
+
+  attribute :client
+  attribute :endpoint
+  attribute :event
+  attribute :action 
+
+  index :client
+  index :event
+
+  def validate
+    assert_present :client, :endpoint, :event, :action
+    assert_unique [:client, :endpoint, :event, :action]
+  end
+end

setup.rb

+require 'ohm'
+require 'json'
+
+require File.join(File.dirname(__FILE__), 'config/db')
+require File.join(File.dirname(__FILE__), 'models')
+
+# Add an entry for my localhost
+puts "Creating Host entry for 'localhost'"
+h = Host.create(:name => 'localhost', :state => 1)
+if h.save
+  %w[redis noah].each do |service|
+    puts "Create Service entry for #{service}"
+    s = Service.create(:name => service, :state => 1, :host => h)
+    h.services << s
+  end
+end
+
+puts "Creating Application entry for 'noah'"
+a = Application.create(:name => 'noah')
+if a.save
+  puts "Creating Configuration entry for 'noah'"
+  c = Configuration.create(:name => 'db', :format => 'string', :body => 'redis://127.0.0.1:6379/0', :application => a)
+  a.configurations << c
+end
+
+puts "Setup successful!"