It also comes with the puppet-module tool that allows you to build modules for, manage and install modules from the forge.  You can install puppet-module via a gem:

$ sudo gem install puppet-module

Both the site and tool are in public beta right now so hammer away at it and tell us what you think!

Let’s start with our type:

Puppet::Type.newtype(:shells) do
    @doc = "Manage the contents of /etc/shells
 
            shells { "/bin/newshell":
                ensure => present,
            }"
 
    ensurable
 
    newparam(:shell) do
        desc "The shell to manage"
 
        isnamevar
 
    end
 
    newproperty(:target) do
        desc "Location of the shells file"
 
        defaultto {
            if
                @resource.class.defaultprovider.ancestors.include? (Puppet::Provider::ParsedFile)
                @resource.class.defaultprovider.default_target
            else
                nil
            end
        }
    end
end

So – pretty simple. We create a block Puppet::Type.newtype(:shells) do that creates a new type, which we’ve called shells. Inside the block we’ve got a @doc string. This is the documentation for the type. Add whatever level of detail and examples in here that is required.

We’ve also got the ensurable statement. Ensurable provides some “automagic” that creates a basic ensure property. Puppet types use the ensure property to determine the state of a configuration item. In our previous example, ensurable resulted in three methods in the provider: create, destroy, and exists?. In a ParsedFile provider we don’t use these methods at all as we’ll see shortly but rather specify how to handle each record in the file.

We’ve defined a new parameter – this one called shell.

newparam(:shell) do
        desc "The shell to manage"
 
        isnamevar
end

The shell parameter is the shell we’re going to manage in the /etc/shells file. We’ve also used another piece of Puppet automagic, isnamevar, to make this parameter the “name” variable for this type. In Puppet-speak, the value of this parameter is used as the name of the resource.

Lastly in our type we’ve specified an optional parameter, target, that allows us to override the default location of the shells file, usually /etc/shells.

newproperty(:target) do
        desc "Location of the shells file"
 
        defaultto {
            if
                @resource.class.defaultprovider.ancestors.include? (Puppet::Provider::ParsedFile)
                @resource.class.defaultprovider.default_target
            else
                nil
            end
        }
end

The target parameter is optional and would only be specified if the shells file wasn’t located in the /etc/ directory. It uses the defaultto structure to specify that the default value for the parameter is the value of default_target variable in the provider.

The provider for our type is also very simple:

require 'puppet/provider/parsedfile'
shells = "/etc/shells"
 
Puppet::Type.type(:shells).provide(:parsed, :parent => Puppet::Provider::ParsedFile, :default_target => shells, :filetype => :flat) do
 
    desc "The shells provider that uses the ParsedFile class"
 
    text_line :comment, :match => /^#/;
    text_line :blank, :match => /^\s*$/;
 
    record_line :parsed, :fields => %w{name}
end

The shells provider is stored in a file called parsed.rb in a directory named for the provider in the provider directory, for example:

/usr/lib/ruby/site_ruby/1.8/puppet/type/shells.rb
/usr/lib/ruby/site_ruby/1.8/puppet/provider/shells/parsed.rb

The file needs to be named parsed.rb to allow Puppet to load the ParsedFile support.

We first include the ParsedFile provider code at the top of our provider, require 'puppet/provider/parsedfile' and set a variable called shells to the location of the /etc/shells file. We’re going to use this variable a bit later.

Then we tell Puppet that this is a provider called shells. We specify a :parent value that tells Puppet that this provider should inherit the ParsedFile provider and make its functions available. We then specify the :default_target value to the shells variable we’ve just created. This tells the provider, that unless it is overridden by the target attribute, that the file to act upon is /etc/shells.

Then we use a desc method that allows us to add some documentation to our provider.

The next lines are the core of the provider. They tell the Puppet how to manipulate the target file to add or remove the required shell. The first two lines, both text_lines, tell Puppet how to match comments and blank lines respectively.

    text_line :comment, :match => /^#/;
    text_line :blank, :match => /^\s*$/;

We specify these to let Puppet know to ignore these lines as unimportant.

The next line performs the actual parsing of the relevant line in the /etc/shells file:

    record_line :parsed, :fields => %w{name}

The record_line parses each line and divides it into fields, in our case we only have one field: name. The name in this case is the shell we want to manage. So if we specify:

shells { "/bin/newshell":
    ensure => present,

Then Puppet would use the provider to add the /bin/newshell by parsing each line of the /etc/shells file and checking if the newshell is present. If it is, then Puppet will do nothing. If not, then Puppet will add newshell to the file. If we changed the ensure attribute to absent then Puppet would go through the file and remove the newshell if it is present.

It is important to remember that ParsedFile providers do have some limitations, they aren’t good at managing complex files such as configuration files with multi-line options, they are best for simple files that contain single line lists of entries such as the cron file entries or the /etc/hosts and /etc/shells files.

You can see the complete code for this type and its providers at my Puppet repository on GitHub. Quite a lot of the existing Puppet types and providers use ParsedFile providers (the cron type for example) and you can use these as examples of how to create your own providers. You can also find further documentation (in a lot more detail!) on creating your own types and providers at the Puppet wiki.

There are a lot of types, in fact I think Puppet covers a pretty good spectrum of configuration items that need to be managed.  I don’t know of anything in particular that is missing that I can’t live without.  But there are little gaps that are annoying, I’d like network and firewall types for example, but creating both these types in a generic enough way to support multiple platforms would be, IMHO, a non-trivial problem. 

Another gap is VCS/DVCS management. A lot of people use source code in repositories to do things with (including install stuff from you bad people – package things … it’s healthier). Puppet currently relies on creating and removing these repositories with the exec type (which executes scripts or binaries), for example:

exec { "svn co http://core.svn.wordpress.org/trunk/ /var/www/wp":
    creates => "/var/www/wp",
}

This is a bit ugly and it’d be a lot easier to write a Puppet type to manage repositories. But Puppet types and providers are written in Ruby and really, really complex and hard to develop. Right? Right?

No. No, they are not… and I’m going to create a simple type and provider to show you.

Here’s a very (very!) simple Puppet type, called repo, for managing repositories. I’ve created providers for SVN and Git as examples also. The first part of the repo type is the type itself – these are usually stored in lib/puppet/type or distributed via modules (see the PluginsInModules page in the Puppet wiki). I’ll create a file called repo.rb.

$ touch repo.rb

And then populate the file:

Puppet::Type.newtype(:repo) do
    @doc = "Manage repos"
 
    ensurable
 
    newparam(:source) do
        desc "The repo source"
 
        validate do |value|
            if value =~ /^git/
                resource[:provider] = :git
            else
                resource[:provider] = :svn
            end
        end
 
        isnamevar
 
    end
 
    newparam(:path) do
        desc "Destination path"
 
        validate do |value|
            unless value =~ /^\/[a-z0-9]+/
                raise ArgumentError , "%s is not a valid file path" % value
            end
        end
    end
end

So – pretty simple. We create a block Puppet::Type.newtype(:repo) do that creates a new type, which we’ve called repo.

Inside the block we’ve got a @doc string. This is the documentation for the type. Add whatever level of detail and examples in here that is required.

We’ve also got the ensurable statement. Ensurable provides some “automagic” that creates a basic ensure property. Puppet types use the ensure property to determine the state of a configuration item.

service { "sshd":
    ensure => present,
}

The ensurable statement tells Puppet to expect three methods: create, destroy and exists? in our provider. These methods, allow, respectively:

• A command to create the resource
• A command to delete the resource, and
• A command to check for the existence of the resource

All we then need to do is specify these methods and their contents and Puppet creates the supporting infrastructure around them but more on this when we look at our providers.

Next, we’ve defined a new parameter – this one called source.

    newparam(:source) do
        desc "The repo source"
 
        validate do |value|
            if value =~ /^git/
                resource[:provider] = :git
            else
                resource[:provider] = :svn
            end
        end
 
        isnamevar
    end

The source parameter will tell the repo type where to go to retrieve/clone/checkout our source repository.

In this parameter we’re also using a hook called validate. Normally used to check the value for appropriateness here we’re using it to take a guess at what provider to use. Our code says, if the source parameter starts with git then use the Git provider, if not default to the Subversion provider. This is obviously fairly crude as a default and we can override this by defining the provider attribute in our resources:

provider => git,

We’ve also used another piece of Puppet automagic, isnamevar, to make this parameter the “name” variable for this type. In Puppet-speak, the value of this parameter is used as the name of the resource.

(Types have two kinds of values – properties and parameters. Properties “do things”. They tell us HOW the provider works. We’ve only defined one property, ensure, by using the ensurable statement. Parameters are more like variables, they contain information relevant to configuring the resource the type manages rather than “doing things”.)

Finally, we’ve defined another parameter, path.

    newparam(:path) do
        desc "Destination path"
 
        validate do |value|
            unless value =~ /^\/[a-z0-9]+/
                raise ArgumentError , "%s is not a valid file path" % value
            end
        end
    end

This is a variable value that specifies where the repo type should put the cloned/checked-out repository. In this parameter we’ve again used the validate hook to create a block that checks the value for appropriateness. Here we’re just checking, very crudely, to make sure it looks like the destination path is a valid fully-qualified file path. We could also use this validation for the source parameter to confirm a valid source URL/location was being provided.

(You can also use another hook called munge to adjust the value of the parameter rather than validating it before passing it to the provider.)

And that is it for the type.

Next, we need to create a provider for our type. Let’s start with a Subversion provider like so:

require 'fileutils'
 
Puppet::Type.type(:repo).provide(:svn) do
    desc "SVN Support"
 
    commands :svncmd => "svn"
    commands :svnadmin => "svnadmin"
 
    def create
        svncmd "checkout", resource[:name], resource[:path]
    end
 
    def destroy
        FileUtils.rm_rf resource[:path]
    end
 
    def exists?
        File.directory? resource[:path]
    end
end

Up front we’ve required the fileutils library, which we’re going to use a method from. Next, we’ve defined the provider as a block:

Puppet::Type.type(:repo).provide(:svn) do

We tell Puppet that this is a provider called svn for the type called repo.

Then we use a desc method that allows us to add some documentation to our provider.

Next, we define the commands that this provider will use, here the svn and svnadmin binaries, to manipulate our resource’s configuration.

    commands :svncmd => "svn"
    commands :svnadmin => "svnadmin"

Puppet uses these commands to determine if the provider is appropriate to use on a client, if Puppet can’t find these commands in the local path then it will disable the provider.

Next, we’ve defined three methods – create, destroy and exists?. Sounds familiar? Yep, these are the methods that the ensurable statement expects to find in the provider:

The create method ensures our resource is created. It uses the svn command to create a repository with a source of resource[:name] (remember the source parameter in our type is also the name variable of the type – we could also specify resource[:source] here too) and a destination of resource[:path] (the value of the path attribute).

The delete method ensures the deletion of the resource. In this case, it deletes the directory and files specified in the resource[:path] parameter.

Lastly, the exists? method checks to see if the resource exists. Its operation is pretty simple and closely linked with the value of the ensure attribute in the resource:

• If exists? is false and ensure is present, then create method will be called.
• If exists? is true and ensure is set to absent, then the destroy method will be called.

In this case the exists? method checks if there is already a directory at the location specified in the resource[:path] parameter.

So, let’s put all this together and create a resource with our new type. I’ve assumed you’ve already distributed your type and providers to Puppet. We can then create a resource like:

repo { "wp":
    source => "http://core.svn.wordpress.org/trunk/",
    path => "/var/www/wp",
    ensure => present,
}

Simple eh? We specify a repo resource, the source we wish to check out or clone from, the destination path and the ensure attribute (present or absent) and that’s it.

You can see the complete code for this type and its providers at my Puppet repository on GitHub. It’s obviously very basic but should be easy to extend to provide additional capabilities (and currently has no tests – my bad). You can find further documentation (in a lot more detail!) on creating your own types and providers at the Puppet wiki.

1. The “Book” content type. I am a writer and I am thinking my next book will be authored between a bunch of us online. Drupal offers the ability to do that and export the resulting content as DocBook. I had a look to see if EE could do this but I don’t have the time to develop something.

2. My preference is generally open source software – I sit on the executive council of Linux Australia – and I don’t see EE as truly FOSS. Drupal makes me feel intellectually more comfortable.

3. The ease of theme application – I don’t have to design or tweak HTML/CSS anymore. (well only a little)

Now I get the annoying feeling though that I will end up tweaking PHP in a few places but *shrugs* it’s all good learning.

I haven’t been to a Lucksmith’s gig in .. well a very long time … I am thinking the last time I saw them play was at the Punters but that part of my life is a little blurry. It was very amusing watching the crowd – still mostly full of nice middle-class private school girls and their cardigan wearing floppy-haired, thick black rimmed square glasses wearing boyfriends. Of course, these days there is a much broader cross-section of ages now – some of those private school girls are now early thirty somethings and some of the floppy hair is looking a little thin.

I don’t know how much Ruth liked it – might have been a little happy, cheery, hopeful pop for her – but I had a good time. The support was good, barring Kirsty Stegwazi having some serious guitar problems early in in her set and their drummer being “in jail”, and the Smallgoods are very late Brian Wilson. The Lucksmith’s did a good set – I don’t know a lot of the later stuff – but they played some early stuff – though not enough of the early, early stuff which is my favourite. But I guess bands evolve away from their older stuff (except the Rolling Stones who really need to retire already) and I’ll just have to burn some CDs onto my iPod to get that all time Lucksmith’s sound.

Finally, the Corner is a little juxtaposed these days – Ruth took me upstairs and that is a weird crowd up there – at least compared to the scruffy types like me downstairs. And the no smoking thing in the band room is good but does mean the smell of BO is bloody strong. I am of two minds about which smell I hate more…

P.S. Girlfriend also bought me Lego Mindstorm! Hmmmmm robots… Very happy.

This, however, does mean I am not on IM nor Skype. If you want to chat – call my mobile.

P.S. Oh yes – house good, lots of little things that need to be fixed, cats come home this weekend, etc, etc.

Ah well. Interviews have always been my weakest point – I get stomach cramps, panic and my normally reasonably articulate self goes totally to shit. I did have a vague plan of having a nerve-steadying drink beforehand. I was all sensible and decided not to. Am now rather regretting that decision. At least I would have gone down with a higher-than-zero blood alcohol level….

But one thing I discovered when setting up Dovecot is that there is very limited documentation on using CRAM-MD5 authentication with Dovecot. As a result I am going to quickly document the process I used to get this up and running.

Firstly you need to enable the mechanism and specify a passwd database file in Dovecot. The mechanism and passdb file are specified in the dovecot.conf configuration file, on my system this is located in the /usr/local/etc/ directory.

auth default {
# Space separated list of wanted authentication mechanisms:
# plain login digest-md5 cram-md5 ntlm rpa apop anonymous gssapi
mechanisms = plain login cram-md5

}

I’ve added the cram-md5 mechanism to the mechanisms statement and then added a passdb file, /etc/cram-md5.pwd.

Next, you need to create this passdb file and set appropriate permissions.

# touch /etc/cram-md5.pwd
# chmod 0600 /etc/cram-md5.pwd

After creating the file you need to add your users and hashed passwords to the passdb file. The users and passwords are added in the format:

user:passwordhash

Dovecot has a utility that allows you to convert passwords to the appropriate hashes. This utility is called dovecotpw and is installed into the /usr/local/sbin directory or is available in the source package in the src/util directory. You can run dovecotpw like so:

# dovecotpw
Enter new password:
Retype new password:
{HMAC-MD5}26b633ec8bf9dd526293c5897400bddeef9299fad

Enter the user’s password when prompted and it will be converted and outputted as a hash. The default hashed output is in the HMAC-MD5 scheme (which is appropriate for CRAM-MD5). You can change the scheme of the outputted hashes using the -s command line switch. Now add the generated password to the passdb file, /etc/cram-md5.pwd.

kartar:{HMAC-MD5}26b633ec8bf9dd526293c5897400bddeef9299fad

Finally, restart Dovecot and test authentication by enabling the appropriate mechanism in your email client. For example, to enable CRAM-MD5 authentication in Thunderbird you need to check the “Use secure authentication” checkbox in the Account Settings page.

Obviously I recommend that you use TLS/SSL to encrypt the authentication process as well.