Thursday, December 23, 2010

Creating a Kerberized client/server app in Ruby with GSSAPI

It's been awhile since I posted anything so I thought I'd write a little bit about the GSSAPI Ruby gem that I have been working on recently. GSSAPI and Kerberos are probably the biggest things I've longed for in Ruby for some time now. I work in an Enterprise environment for the State of North Dakota and we use Active Directory authentication for most everything. Since AD has Kerberos built-in, it opens a lot of options for authentication and encryption. I wanted to bring this functionality into the Ruby world so I wrote a gem that utilizes FFI and the underlying GSSAPI C libraries to do exactly that. Since the MIT Kerberos source comes with a sample client and server application I thought I would create a simple duplication of that in Ruby.

The Client

GSSAPI/Kerberos relies on a Service Principal Name(SPN) on the server. We specify it in the client code so that is why I mention it here. If you were creating an actual server for production you would want to create a separate SPN for it, but since this is a demo I will be using the 'host' SPN which should exist by the mere fact that the server is part of the Kerberos Realm. To run the client you will also need to have credentials of your own via 'kinit'.

Require the appropriate gems.
require 'gssapi'
require 'base64'
require 'socket'

Specify the server and the service name. Together these become the SPN.
host = 'example.org'
service = 'host'

Open up the connection to the server
sock = TCPSocket.new(host, 8082)

Initialize the GSSAPI security context and Base64 encode it so it is ready for transport. The Base64 encoding is not really necessary in this case, but since GSSAPI is often used in HTTP communication and is Base64 encoded there I am using it in this example. The GSSAPI::Simple wrapper is doing a lot in the background for you. It's importing the SPN for use in GSSAPI string types, managing memory and all of that good stuff. If you need to do something a bit out of the ordinary with GSSAPI and need to call the gss_* methods directly, check out the Simple wrapper for examples.
cli = GSSAPI::Simple.new(host, service)
tok = cli.init_context
stok = Base64.strict_encode64(tok)

Send the initial token, get back a response token and complete the security context. Once this step is done the set-up is complete and we are ready to start passing messages.
sock.write("#{stok}\n") # send initial token
stok = sock.gets.chomp # get back continuation token
ctx = cli.init_context(Base64.strict_decode64(stok.chomp)) # complete security context
puts "Connection #{(ctx ? 'succeeded' : 'failed')}"

Now just prompt for text from STDIN, encrypt the message and send it to the server. Do this until "exit" is typed on the command line.
begin
print "> "
msg = STDIN.gets.chomp
emsg = cli.wrap_message(msg)
sock.write("#{Base64.strict_encode64(emsg)}\n")
end while msg != 'exit'

sock.close

The Server


Set up is the same.
require 'gssapi'
require 'base64'
require 'socket'

The host and service should be the same as the client as well.
host = 'example.org'
service = 'host'

You won't always want to run as root so the system keytab may not be available. You can specify an alternate keytab as long as it contains the correct SPN.
keytab = "#{ENV['HOME']}/.gssapi/krb5.keytab" # this is optional, but probably required if not running as root

Start listening....
tcpsrv = TCPServer.new(host, 8082)

Initialize the security context for the server and acquire the credentials from the keytab.
srv = GSSAPI::Simple.new(host, service, keytab)
srv.acquire_credentials

Start the main server loop. This will listen for incoming requests, accept the incoming token, and send back a token so the client can complete the security context. After that it just listens for messages, decrypts them and prints them out. It will exit when it receives the "exit" message.
loop do
Thread.start(tcpsrv.accept) do |s|
print(s, "Accepted Connection\n")
stok = s.gets.chomp
print(s, "Received string#{stok}\n")
otok = srv.accept_context(Base64.strict_decode64(stok.chomp))
s.write("#{Base64.strict_encode64(otok)}\n")

begin
emsg = s.gets.chomp
msg = srv.unwrap_message(Base64.strict_decode64(emsg.chomp))
puts "Received: #{msg}"
end while msg != 'exit'

print(s, "Closing Socket\n")
s.close
end
end

That's really it and it's much shorter and easier to understand than the C counter-part. If you have issues running it make sure you have your credentials on the client (kinit) and the server has access to the appropriate keytab. These are the two big follies that will mess things up. Other than that have fun with the gssapi gem. If you use it and want to share, add your app to the gssapi gem wiki:
https://github.com/zenchild/gssapi/wiki/Users-of-GSSAPI

Cheers and Happy Solstice!

Friday, September 24, 2010

Connecting to Windows from Capistrano with WinRM

After developing the WinRM library for Ruby I thought about what uses it may have. One thing I had longed to put to use in the enterprise is a tool like Capistrano. It has long worked well in the UNIX world where every system really should be running SSH, but installing SSH on every MS Windows system enterprise-wide really isn't realistic in many cases. That's when I began to experiment in integrating WinRM into Capistrano. The goals were simple:

  1. Do not require the user to patch Capistrano.
  2. Make the usage of WinRM in the Capfile as seamless as possible.

To accomplish what I wanted I had to override some methods within Capistrano to inject WinRM functionality. The end result adds a winrm command that should work almost identical to Capistrano's own run command. It should also allow your current Capfile to run unmodified if you do not have any winrm tasks.

To define a WinRM task you can simply do this:



winrm also supports using blocks just like run so you can format your output a bit better like so:



There you have it. As long is WinRM is configured on the remote Windows systems you can now manage them with your Capistrano infrastructure. There is a gem up on Rubygems that adds in the functionality so just do a:
gem install -r capistrano_winrm

The source code for this extension is also up on Github:
github.com/zenchild/capistrano_winrm

Thursday, August 19, 2010

Managing Windows with Ruby (Part 1) or...

How I learned to stop worrying and manage Windows from Linux




Intro to Windows Remote Management

I have long been a UNIX guy and have always struggled living in a mixed OS environment. Traditionally Windows based systems have made it very hard to do things remotely from systems other than other Windows boxes. This is where Microsoft's implementation of the WS-Management Protocol comes into play. The newer versions of the OS (2003 R2, Vista, 2008, and Win7) all allow you to turn on Windows Remote Management (WinRm). This is a major step forward for heterogenious envirnments because WinRM communicates over HTTP and SOAP. For more information as well as instructions on how to configure WinRM, please visit Microsoft's WinRM site. If you would like to follow along and read the WinRM docs later you can simply open up an Admin Powershell and issue the following:

winrm quickconfig
This will create an HTTP listener on port 5985 by default. I still recommend that you read the docs for more information.

Once WinRM is configured and listening the hard part is really over. Now all we need to do is pass SOAP packets back and forth in order to tell the remote Windows box what we want it to do. I have created a Ruby library called 'winrm' that aids in the building and parsing of the SOAP packets so all you have to do is call your Powershell or run your WQL and go.


WinRM for Ruby Set-up

Before diving into the examples WinRM for Ruby needs a little set-up. First and foremost the gem needs to be installed. It is up on Rubygems.org so you can just do a:

gem install -r winrm
Every program that uses WinRM will have to set-up the following:
WinRM::WinRM.endpoint = 'http://mywinhost:5985/wsman'
WinRM::WinRM.set_auth('user','pass')
winrm = WinRM::WinRM.instance
The 'winrm' instance created on the last line will be used in the following examples.

Running Powershell scripts

Powershell has a nice little option called '-encodedCommand' that lets you run a Base64 encoded script. Because of this it is relatively simple to write Powershell scripts locally and run them remotely with WinRM. To do this with the Ruby library you can simply issue

winrm.powershell(script_file)
This will return an array in the form [stdout, stderr].

Just to clarify, these scripts are local to the machine running the WinRM for Ruby library. That means you can store them on your Linux desktop or wherever you store your scripts and run them from there.


Running WMI Queries (WQL)

WMI is invaluable when enumerating system information from a Windows system. One way at getting values out of WMI is via WMI Query Language (WQL). WQL is SQL-like in syntax and very easy to use. The WinRM Ruby library implements this feature with the WinRM::WinRM#wql method. You can issue WQL queries like this:

winrm.wql 'select * from Win32_Service'
This method will return an Array of Hashes. Each Hash will contain the name/value pairs retrieved from WMI. For instance, if your WQL was
'select Name,Status from Win32_Service'
your return would look something like this:
[
  {'Name' => 'Appinfo', 'Status' => 'OK'},
  {'Name' => 'Browser','Status' => 'OK},
...
]


Give it a try...

That's a quick overview of how to use WinRM. It's a very new library so enhancements will be coming and I hope to have a Part 2 to this post that discusses the actual SOAP protocol that WinRM uses. In the meantime, give it a try and let me know how you're using it in the comments. You can also clone the source on github or open an issue on Github if you run into any issues.


Github: http://github.com/zenchild/WinRM

Cheers!

Monday, August 2, 2010

Viewpoint for Exchange Web Services - Next Generation BETA

I would like to announce the next generation, BETA release of Viewpoint, my Exchange Web Services client library for Ruby. This release has some major changes from the previous one which I'll list below. It was also developed in Ruby 1.9.x and the RSPEC tests have also completed in Ruby 1.8.7. There is much work that needs to be done adding features to the Model, but the SOAP guts have been hashed out fairly well in my mind and adding features from here on out should be fairly incremental. So without further ado, here are some of the major changes found in this BETA release.

Check out the latest code from the Viewpoint Github repository
...or install the gem version 0.1.0 "gem install viewpoint"


New SOAP backend

Viewpoint now uses Handsoap as its back-end instead of soap4r. While soap4r does some very nice things for you automatically, it ends up making your code base fairly large and it complicates any customizations that you might want to make. One example is adding custom headers. Soap4r required you to create a subclass to use as a sort of hook. I can do the same thing in Handsoap with a one-liner in the services #on_create_document method.


Models are completely rewritten

The models are completely new and not backward compatible with the old version of Viewpoint. Some of the methods still exist, but don't count on them. I've tried to make this version much more extensible than the last.

Configuration loader removed

After much thought I determined it really wasn't the place of a client library to provide a configuration loader for credentials. To that end I removed the need for a .viewpointrc file. If this is something you liked it would be relatively easy to write it into your application. Now authentication configuration information is set via class methods:
Viewpoint::EWS::EWS.endpoint = 'https://mysite.com/ews/exchange.asmx'
Viewpoint::EWS::EWS.set_auth('myuser','mypass')

Delegate access is supported

One thing that was often asked for, but missing from the previous version was delegate access to mailboxes and calendars. This is now supported via the 'act_as' parameter to the GenericFolder::get_folder method. For example:

ofolder = Folder.get_folder(:inbox,'otheruser@test.com')

If your user has delegate access to the Inbox for otheruser@test.com this operation will retrieve their inbox and allow you to manipulate it as you would with your own Inbox. There is also some support for manipulation of delegate access itself via the methods MailboxUser#add_delegate!, MailboxUser#update_delegate!, and MailboxUser#get_delegate_info.

Some Exchange 2010 Stubs


Some of the new features of Exchange 2010 are being stubbed in. I do not have access to an Exchange 2010 server to test them on so I cannot verify if they work or not.

Misc other changes

Since it's a complete rewrite there are tons of other changes that you'll notice. I've tried to keep the code comments coming so stay tuned to the API docs for library information. I'll also be posting more examples to my blog.

So at a glance there you have it. If there are any features you would like to see added or comments please add it to the bottom of this post or add an issue on the Github page. Also, if you're interested in sponsoring an Exchange 2010 account for me please let me know. I would love to try out some of the new advanced query stuff that 2010 has to offer.



Cheers!

GITHUB: http://bit.ly/viewpoint_ews
RubyGems: http://rubygems.org/gems/viewpoint
RDOC: http://rdoc.info/projects/zenchild/Viewpoint

Tuesday, June 29, 2010

Ruby and OpenSSL with SSH keys

One of the challenges of dealing with encryption is the management of keys. We have a SSH key, a PGP key (probably multiple), maybe a OpenSSL cert, perhaps another cert somewhere for S/MIME and maybe a ton more for various other purposes. This post is basically an addendum to my last File Encryption in Ruby with OpenSSL, but I will step through the code using an SSH key instead. This will minimize the amount of keys you have to manage and it will utilize a key that many people have readily available to them without the need to create yet another encryption key.

SSH Keys


On the chance that you do not have an ssh key (you really should you know) or you are using DSA keys you will need to create an RSA key with ssh-keygen.

home$ ssh-keygen -t rsa -b 2048 -N 'mypass'

Loading the SSH key in Ruby


Loading the SSH/RSA key is relatively straight forward. Instead of pointing to our generated RSA key from the last posting we will simply point it to the path of our SSH/RSA key and load it with the password we supplied during the ssh-keygen step. I will demonstrate in an irb session.

# Require the OpenSSL library
irb> require 'openssl'
# Load the SSH key from the default location (Linux/UNIX)
irb> key = OpenSSL::PKey::RSA.new(File.read("#{ENV['HOME']}/.ssh/id_rsa"),'mypass')

The generated public SSH/RSA key is not stored in PEM format so you cannot consume it with OpenSSL, but you can generate it from the private key and store it alongside the SSH formatted one.

# Write the PEM encoded public key alongside the SSH public key
irb> File.open("#{ENV['HOME']}/.ssh/id_rsa.pub.pem") do |f|
irb>     f.write(key.public_key.to_pem)
irb> end


So there you go. You can now load your SSH key for encryption/decryption purposes with the OpenSSL library and you won't have to worry about managing keys. All else remains the same and you can do everything that was done in my previous OpenSSL/Ruby posting.

Cheers!

Wednesday, June 23, 2010

File Encryption in Ruby with OpenSSL

Disclaimer: I am far from an expert on cryptography so there are probably a million things wrong with this. Any constructive criticism is always welcome... but be gentle ;-)
REQUIRED: Ruby 1.9 *because of new hash syntax.

For Starters....

I am often torn when programming between using libraries and following the DIY approach. On one hand you can accomplish a lot in a short time by including libraries that have the functionality you need and just bridging the gaps, but I am often left with the feeling of some loss-of-control and bloat. Obviously some library usage is inevitable, but I do like to take a minimalist approach to including external libraries because the management burden of updating libraries, the effort to make various libraries play together and because moving library-heavy applications between platforms can be... interesting. Most of all I like the learning that takes place when building it yourself. It was for this reason that I decided to see if I could effectively utilize the OpenSSL library that is typically part of a Ruby installation to create my own higher-level encryption/decryption class. Before we dive in I would like to layout the details of the application.

Password Management App


I have for years maintained a PGP encrypted file for my account passwords. I had the desire to centralize this and make it more web accessible. In order to do this I wanted assurance that the data was not only encrypted on the wire via SSL, but also at rest. I looked at various Ruby PGP/GPG libraries, but all of the ones I surveyed were wrappers around the GPG binary itself which didn't make it very web host friendly ( I planned on hosting it on Heroku with a Couchdb backend on Cloudant). That was when I decided to look at OpenSSL for file encryption since it is typically part of a Ruby installation and was available on Heroku. I also had the need for public-key encryption so a password store could be shared between two or more people (I share some accounts with my partner).

Take One... RSA

I thought I could simply use RSA encryption to meet all my needs and at first it worked quite well. Creating keys was straight forward and encryption/decryption worked well. One caveat is that the string passed to the FileEncryptor#encrypt_string method cannot be larger than the key size + PKCS padding size (11 bytes). So for a 128 byte (1024 bit) key the string should only be 117 bytes (see String#size).



This seemed to work quite well until I remembered that I needed to have the encrypted data be readable by multiple parties. I then had to change my approach and decided to use a symmetric AES key that is then shared between the multiple parties and each person encrypts the key with their RSA public key.

Take Two... RSA + AES

In order to accomplish secure use of the AES key we start out the same way as the previous solution by creating a RSA key. It will only be used to secure AES keys that we use to protect certain data stores. So if I have a data store I want to share with my brother I encrypt the data store with an AES key then I encrypt that key with my public RSA key and my brother's public RSA key. Now both he and I can access anything in that store and the AES key is still secure from prying eyes. Here is an overview of the steps taken before I post the code.
  1. Create a RSA key-pair
  2. Create an AES key to encrypt the data store
  3. Encrypt the AES key with your RSA public key
  4. To give access to the data store encrypt the AES key with the person's public RSA key send them back the cipher-text.

And here is the class I wrote to make this work:



Below is a demonstration of the use of this class to accomplish the needs I had for my password protector application. I am aware that this class needs some refactoring and it's a bit annoying to have to pass the AES key file to the methods, but logically it is working the way I had hoped.



fe = Encryptor.new('mysecret')

aesfile = 'aeskey.sec'

fe.gen_aes_key(aesfile)

etxt = fe.aes_encrypt('this is a test', aesfile)

txt = fe.aes_decrypt(etxt, aesfile)

pub_key = < Assume I got someone's public RSA Key somehow >

fe.give_aes_key(aesfile, pub_key)

Like I mentioned, this code needs to be cleaned up a bit, but it gives me all of the functionality I need to build my application and it doesn't require any external libraries except OpenSSL and YAML which are typically standard in any Ruby installation.

Hopefully someone finds this post useful and if you have any additions, corrections, criticisms please post them below. Feedback is always welcome.

Cheers!

Wednesday, April 7, 2010

Using Ruby with Zenoss - Part 2

This is the second installment of Using Ruby with Zenoss.  This posting will mainly focus on using the library.  If you have not read Part 1, I would strongly recommend going there first in order to make sure your environment is set up correctly.

In creating the Zenoss library for Ruby I have tried to keep the interface to the programmer fairly similar to the Python API.  It is somewhat simplified because the depth of modules and classes don't map one-for-one to a REST client world.  There is still much to be in its implementation, but here are a few examples of what you can do with the library today.



Requiring the gem

!!! You need to have at least version 0.0.4 of the gem installed. !!!

Make sure you add this to any of the following examples:
require 'rubygems' # if you're using gems
require 'zenoss'

Getting Started

You need to tell zenoss_client where to point and how to authenticate. To do that use the following code:

# Set the base uri that we talk to Zenoss with. Your host will
# be unique but the port will probably be 8080 unless you proxy
# it through 80.
Zenoss.uri 'https://:/zport/dmd/'

# Add the appropriate credentials
Zenoss.set_auth('user','pass')

Getting a list of devices

Now that we're set up we can get down to it. One of the primary things that I find myself doing is managing devices through the REST interface. You need a starting point so let's get the base Device class ( /zport/dmd/Devices ):

devices = Zenoss.devices

You can think of the 'Zenoss' object as a slimmed down equivalent of 'dmd' in the zendmd Python interface.

If you want to start at a DeviceClass deeper in the tree you can do so by passing the path to DeviceClass:

linux = Zenoss::Model::DeviceClass.new('/Devices/Server/Linux/')

Once you have your DeviceClass you can list the devices beneath it like so:

subdevs = linux.get_sub_devices

Device Info

Using the Device Array from the last step, we'll take the first device and get some information from it.

mydev = subdevs.first

# Get the status id
mydev.get_status

# Get the icon associated with this status
mydev.get_status_img_src(mydev.get_status)

# Get uptime
mydev.sys_uptime

# Fetch some RRD Values
dps = mydev.get_rrd_data_points
dsnames = []
dsnames << dps[1].name
dsnames << dps[2].name
mydev.get_rrd_values(dsnames)
# => {"laLoadInt1_laLoadInt1"=>"252.27000000000001", "laLoadInt15_laLoadInt15"=>"255.86000000000001"}

Systems

The library has basic support for Systems now. It's mainly just the ability to create and delete subsystems. More support is forthcoming.

# Fetch the base '/Systems' System
systems = Zenoss.systems

# Create a new subsystem
ltst = systems.add_subsystem('linuxtst')

# delete the newly created System
ltst.delete!

That's it, go play

That's a quick introduction to the Zenoss Ruby library. I'm adding new functionality all the time so check for updates often. The code is up on github at http://github.com/zenchild/zenoss_client so if you want to fork it and help out with development you're welcome to do so. If you're not interested in developing it but would like to see a feature added please comment below or open up a ticket on github and I'll get to it as time allows. I hope you find the library useful.

Addendum: The RDocs are available at http://rdoc.info/projects/zenchild/zenoss_client

Cheers,

Dan

Thursday, April 1, 2010

Using Ruby with Zenoss - Part 1

I have been using Zenoss now for over a year and for the most part have been quite happy with it. It's quite customizable and you can find a way to pretty much monitor anything if you put some thought into it. However, my one point of contention has been with the REST interface. Most things work as you would expect, but somethings blow up and don't give you a very good reason except for a nice Zope stack-trace.  Prior to using Zenoss, I had very little experience with Zope so some of my issues were simply just lack of knowledge of how REST methods were called from within it.  I wrongly thought that there was a translation layer that sat between the web user and the back-end to marshal and unmarshal datatypes.  This does not seem to be what is going on.  So if I can sum up all of the issues that I struggled with it's this:
  •  If the Python method you're calling on the back-end is expecting something other than a string you're probably going to have issues.
So, coupled with my poor knowledge of Zope and my frustration on how to call REST methods in a consistent way, I decided to create a library for Zenoss in Ruby.  My goal was to create that middle layer to correctly marshal and unmarshal data-types.
I chose Ruby because I find it a fun language to program in and well, I guess that pretty much sums it up ;-)
 In order for the library to work there is one thing that you need to do on the Zenoss/Zope side.  To marshal data-types into Python you most certainly need to do it in Python so we have to add a custom script to Zope.
  1. Go to the Zope Management page:
    • http://zenoss:8080/zport/dmd/manage
  2.  Select "Script (Python)" from the upper-right drop-down
    • ID = callZenossMethod
      • "Add and Edit"
    • Title = "Work around for unsupported Zenoss methods"
    • Parameter List = methodName
    • Clear out the code contents from the text box and cut-and-paste the code found here:
    • "Save Changes"
Once the custom Python is in place simply install the ruby gem:
  • gem install -r zenoss_client
That's it, you should be ready to roll.  Check back here soon for the second part of this post for example usage of the zenoss_client library.



addendum: Link to zenoss_client source code: zenoss_client

Tuesday, February 2, 2010

Closing Javascript Alerts in Selenium Tests

I have been plagued some time now with a problem in my Selenium testing, the dreaded modal Javascript alert generated from the "onload" event (See the Selenium FAQ for more info). OK, maybe dreaded is a bit strong of a word, but it has been quite frustrating in trying to automate vendor provided code that we cannot change. Finally there is a fix.

Jason Huggins, the creator of Selenium, presented at JSConf 2009 and demonstrated a way around Javascript alerts that Selenium cannot handle. After speaking with Jason, he provided me with the original code snippet from Aaron Boodman and I have created a Firefox extension for general use that can be downloaded from my Github site.

The magic happens in a function called alertClose. If all you need to do is close an alert it should work unmodified for you. If you need to do something a little more exotic you may need to edit the subject.location which specifies what types of windows it closes. By default it points to chrome://global/content/commonDialog.xul, which is the chrome type for alert boxes. There is a Mozillazine article that has a listing of the various types of chrome windows if you need to customize.



So there you have it. A very simple way to get around Javascript alerts. Hopefully this solution is only temporary and the new WebDriver code being integrated into Selenium 2 will work around it in a more elegant way.