Composite namevars in Puppet

An advanced look at everything you never wanted to know about composite namevars in Puppet resource types

In this post I’m going to look at some of the more advanced concepts around Puppet resource types known as composite namevars. I’m going to assume you have a reasonable understanding of types and providers at this point, if not, you should probably go and read Fun with providers part 1 and Seriously, what is this provider doing, two excellent blog posts from Gary Larizza.

A composite what-now?… let’s start with the namevar

Maybe I’m getting ahead of myself, before we delve in it might be worth a quick refresh of the basics, let’s start with a fairly primitive concept of any Puppet resource type, the namevar. Take this simple example;

It’s important to get our terminology right here. The above piece of code is a Puppet resource declaration, the resource type is a package, there is a resource title of mysql and we have declared an attribute. When we run this code, Puppet will elect a provider to configure the managed entity. When we talk about the managed entity in this example, we are referring to the actual mysql package on the node (deb, rpm…etc). When we refer to the package resource we are talking about the resource within the Puppet catalog.

Puppet needs a way to map the resource declaration to the actual managed entity that it needs to configure, this is where the namevar comes in. Each resource type has a namevar that the provider will use to uniquely identify the managed entity. The namevar attribute is normally, and sensibly, called “name” – although this should never be assumed as we will find out later. What this means is I can use the namevar in a resource declaration like this…

In this example, we’ve changed our resource title to “mysql database” and added the name attribute with a value of “mysql”. The title of the resource in the world of Puppet is “mysql database”, and we would refer to the resource within Puppet as Package['mysql database']. But the provider will identify the managed entity to configure using the namevar, the name attribute. That is to say, whilst the resource is called “mysql database”, the actual thing that the provider will try and manage is a package called mysql.

So, you might be asking yourself, what’s the deal with the first example? We didn’t specify a namevar, so what happened there? The short answer is that in the absence of a namevar being specified in the resource declaration, Puppet will use the title of the resource as the namevar, or more correctly, the value of the name parameter will magically be set to the resource title.

Not all resources have name as their namevar. To find out which attribute is a namevar you can use the puppet describe command to view the documentation of resource attributes to see which attribute is namevar. For example, if we look at the file resource type;

Note that puppet describe will only tell you which attribute is the namevar if it isn’t name, which is confusing.

So, for the file resource, both of the following examples do the same thing, they both manage a file called /etc/foo.conf

Puppet is a stickler for uniqueness

You’ve probably already figured out that things in Puppet must be unique, that is to say, only one resource type can define the desired state of a managed entity. We already know that you can’t declare multiple resources with the same title, so what happens with the above code, the two resource declarations have different titles, let’s see…

Puppet is smart enough to figure out that although the resource titles are different, the namevar is resolving to the same value, which means you are trying to configure the same managed entity twice, which is firmly against the law of Puppet.

So, that was a quick recap of namevars and what they are. Pheew that was easy, are we done? not by a long shot!

When a name is not enough

Sometimes you can’t quantify a managed entity just from a single name alone. A classic example of this is actually the package example used earlier. Let’s revisit that, it sounded simple right? If we want to manage the mysql package we just declare:

This is all well and dandy, if you’re only using one packaging system, but sadly modern day systems come with a whole host of packaging systems, and we can use other providers of the package resource type to install different types of packages, like if I want to install a rubygem I can declare something like

But now consider what happens if you want to manage the yum package “mysql” and the rubygem that is also called “mysql”. We clearly can’t do this;

This will obviously fail to compile, we have two resources declared in Puppet with the same title, and that’s breaking one of the number one rules of Puppet that resources must be unique. So what if we change the titles to be different and use the namevar stuff we’ve just talked about;

Of course, as we’ve already dicussed, name is the package resources namevar and we’ve defined it twice, and as we saw with the file resource, Puppet is so smart if we try and run this code it is obviously going to fail like this;

WTF now? why did that work? Surely we’ve just broken one of Puppet’s golden rules? In general circumstances yes, but this bring us to the crux of this post, Puppet has the concept of composite namevars to solve this very issue. That is, a resource type can actually have more than one namevar, and rather than uniqueness being based soley on one parameter, Puppet can evaluate uniqueness based on a combination of different resource attributes.

Confused?? me too, let’s write some code then….

We’re going to craft a fairly basic type and provider to manage a fictional config file /etc/conf.ini. We want to be able to manage the value of configuration settings within a particular section of the INI file, something like:

Sounds easy enough! We have three basic elements to our managed entitiy, the section, the setting and the value. So let’s create a module called “config” and start with a very primitive resource type.

Now let’s add a provider so our resource type actually does something, for the sake of trying to keep the code to a minimum and focus on the relevant topics I’m going to re-use the ini_file library from the module puppetlabs/inifile… so if you want to follow along from home you’ll need to install that module so puppet/util/ini_file is in your ruby load path. Heres the provider we’ll use;

Whilst not the most comprehensive provider ever written, this should do the job, all we need to do now is write some Puppet code to implement it, let’s do that now.

Let’s just recap on that resource declaration statement, in our resource type we have said that the config resource type has three attributes, the section and setting parameters, and the value property. We don’t have an attribute called name, but we have made the parameter setting the namevar. Like in our file example at the beginning, this means that if we don’t explicitly give the setting parameter in the resource declaration, then the title of the resource, in this case hostname will automatically be used. Now to make sure everything is working…

Now it works, lets break it.

So far so good, but now consider that want to manage settings in other sections within the config file, that may have the same name, for example;

Both sections have a setting called server, so how can we express this in our Puppet manifest? We can’t declare both resources with the same title, and neither can we declare them with different titles but with the setting set to foo.com because, as we saw with the file example earlier, Puppet will fail to compile because the namevars must be unique as well as the resource titles and server is our namevar.

To solve this, we must tell Puppet that a config setting is not only uniquly identifiable by it’s setting name, but rather by the combination of the setting name and the section together. To do this, we need to make the section attribute of our type a namevar. That’s not rocket science, we just add a :namevar argument to our section parameter, like this;

What about the title? introducing title_params!

We’re not done yet! Remember in our earlier example using the package resource we discussed how in the absence of the namevar the resource title will be allocated as the namevar. Ths is still the case, but now we have two namevars we need to give Puppet a hand in deciding what to do with the resource title. We do this by creating a method within our type called self.title_patterns, and it goes something like this;

The self.title_patterns method returns a simple array of arrays of arrays containing arrays, or something like that. This particular nugget of insanity is provided in the Puppet::Type class, with a comment saying # The entire construct is somewhat strange…. No shit. If we delve into the Puppet core code in lib/puppet/type.rb we see the output of this method should return this mad data structure;

What we are saying with our above method is that any resource title that matched /(.*)/, which matches anything, will be assigned to the setting attribute if we have not declared it, meaning that we can still run our original Puppet code and get the same behaviour;

Now back to the problem at hand, now we have composite namevars, if we need to manage the setting hostname in both the [server] and [client] setion of our INI file, this is now possible by using our compisite namevars with differing titles.

Let’s do more with title_patterns

You’re probably wondering why the title_patterns method is so complicated (even by Puppet internals standards) for something that does so little, actually, it’s a rather powerful, albeit cryptic, beast. Our current method assigns any title to the setting attribute, we can make this smarter. We can enhance this method to also look for patterns matching section/setting and assign the relevant parts of the title to the right attributes. So let’s change our original regexp and add another element to the main array.

No, I haven’t broken my keyboard and you’re not going blind (yet!). We now have a title pattern with two matches. If we do not specify a title pattern containing / then the type will behave as before and the title will be assigned to the setting attribute, however, if it matches a string with a / then it will parse this as section/setting and assign the section and setting attributes from the title. This means, aswell as using the puppet declarations above, we could also shorten these and write;

The above shortened version has the same behaviour and the provider see’s the attributes in the same way.

How cool was that!

Here’s a recap of what our final type looks like

Final thoughts

Hopefully this has given you a basic understanding of what we mean by composite namevars, and maybe you didn’t break your brain reading this. I think I’ve only touched on the surface of what you could possibly do with title_vars, theres some very scary patterns involving procs out there! But I’ll leave off here. So there you have it, composite namevars and title_patterns, clear as mud, right?

facebooktwittergoogle_plusredditpinterestlinkedinmail