Code Generation in Ruby

The automatic creation of source code (i.e. Code Generation) is an underused, and often misunderstood, time saving technique that can have positive impact on both your productivity and the quality of your code.

This article will go into some depth on what code generation is, what the benefits are and how to use it. In addition to providing a practical example that details code generation for database access.

Caveat Emptor

Code generation is a technique which covers a lot of ground. I am not presenting the only method for building or using a code generator, or the only target for which the technique can be applied. Code generation is a powerful technique that can be used in a wide variety of different ways to solve a number of different problems.

I've written the example code in Ruby because I enjoy the language and think that it is suited to not only the task of code generation, but also database access. Is it the only language that can do this? Of course not, but the examples have to be written in something, and Ruby is generally easy to understand.

You might also want to spend a little time looking around the web to see if someone has already built your code generator for you. A quick search on Google on for "EJB Code Generator" found pages full of links to a number of different Code Generators all for EJB database access.

What is a code generator?

'Code Generator' is a generic term for a wide variety of tools. In this article it applies to a command line application that turns a definition file into one or more implementation files in some target language.

The 'definition file' is a high-level description of the code that is to be output by the code generator. It is the responsibility of the code generator to turn the definition file into 'target code', which is code that can then be used by an application to get work done. In this article the code generator application will use a set of text templates to build the 'target code'.

From this brief description you can start to see two of the key benefits of code generation. The first is the seperation between the syntax of the 'definition file' and the 'target code' output. It is very possible that the language of the 'target code' could change with no alteration to the 'definition file'. For example, while the 'target code' in this article is Ruby, it could easily by Java, C#, TCL, Perl or Python.

Second is the reusability of the 'definition file'. While it can be used to generate code, it could also be used to generate test cases, documentation, IDLs, SOAP WSDL definitions, or any number of resources that your project requires. When to generate code

When should I use code generator?

There are a couple of clues that you might want to use a code generator. One is that feeling that you get when you have done all of the necessary prototyping to make sure it can be done. Then you get that sinking feeling that all that needs to be done is a lot of typing and then you will be finished.

Another is when you find yourself doing the same thing over and over again, you should be thinking about a code generator. Computers are better at doing repetitive tasks than humans. Use the computer to your advantage and let it do your grunt work.

Both of these should give you a hint that you might want to write a small code generator to get the job done for you. Then it becomes a decision about if you can get it done faster with a generator.

Another common scenario is the support for many languages problem, your company wants to deliver an API for Java, C#, Perl, Python, Ruby, etc. While you know these languages you do not have the time to maintain an API in every one. This a great opportunity to use a code generator. Have a single set of files that describe the API at a high level, and use a code generator to build support for every language.

What can I use a code generator for?

Two common uses for code generators are database access systems and building user interfaces. Database access systems are ideal for code generation since the code for these systems tends to be redundant and somewhat complex, so you have to write lots of it and it's easy to mess up. The problem to solve for database access, bridging between the object world and the relational world, also tends to be fairly simple and straightforward (depending on your solution.)

User interface code also tends to be be redundant and fairly complex code, with a fairly simple purpose; getting the data out of a dialog management system into data structures and validating it. A code generator keeps the level of the detail that the programmer needs to work at high, simply connecting user interface elements to screen design, then doing all the grunt work for them. In addition a code generator can generate screens or markup on a variety of platforms.

Code generation can also be used for building unit tests, building glue code, and many other systems to solve a wide variety of problems. Using some of the heuristics mentioned above can give you a clue about when a code generator could be useful to you.

What does the architecture of the code generator look like?

The code generator presented here is a fairly simple black box. Coming in the input side is a definition file, and coming out the output side are the 'target code' files. If you look inside the box you will see it has a few small implementation files, and then template files. The implmentation files chew through the input, and invoke the templates to create text blocks that are dumped into the output files.

How to write templates

Templates are at the core of an template-driven code generator. The better the templating language easier it is to build more intelligent templates. In fact, for one job I built templates using HTML::Mason, an extremely powerful Perl markup language.

There is no one tried and true process to create the templates, but here is a process that works well for me.

1. Put together a design for the system you are going to build (specifically the portion you are considering generating). At this point you should not be thinking about how to optomize for code generation, but about what a solid design is that you will want to use and that will have the characteristics you want.
2. Stand back from the design and figure out if you want to generate it all, or just parts of it, or none at all.
3. Analyze your design for the system you are going to generate, and decide where the points of variety are. For example, in our database access system the points of variety are; reading an array of structures, reading a single structure, reading a single field of a single structure, reading an array of single fields from multiple structures, updating a structure, adding a structure, and deleting a structure.
4. Code an example or two of each of the different points of variety, including test cases. At this point you can take some time writing this code to get it exactly the way you want it. Your test cases should beat it up, and you should check it for memory leaks, unintended side effects and edge cases. Ideally you should rewrite each example once or twice to make sure that you are using the language the way it was intended, or the APIs the way that they were meant to be used.
5. Design your code generator. Specifically you may want to spend some time deciding when the generator is to be used. Is it on demand? Is it on check-out? Is it on compile? You will also want to think about where the code generator and it's templates will fit into the source tree, how it will be invoked, etc.
6. From here on out the actual implementation technique of the code generator varies depending on the project and what is required. For me it usually involves building the generator framework to handle each variety case, then compare those against the hand written examples.
7. Run the generated code against the test cases that you built for the examples to make sure that the results are the same.

On to CodeGenerationWithRuby2

Make comments on CodeGenerationWithRubyComments

Download the code

All content and code is copyright (c) 2002, Jack D Herrington