Automated Deployments for Clarify
I had an inquiry this week asking about our experience with automatic deployments for Clarify – specifically around deploying ClearBasic code and schema changes.
So I jotted down a few rambling thoughts that might be useful to others. Most of this stuff is pretty standard for software development shops, but not as common in enterprise internal IT organizations. (But it should be!)
At Dovetail, we have lots of experience in automated deployment. Let me lay out some of our experience and tools and scenarios.
Obviously, one of the big benefits of automation is that repeatable work can be moved from a manual to an automated strategy, and by reducing the manual components, we can eliminate the human error factors, while also improving productivity and increasing employee happiness.
Customization Replicator
Years ago, we created a product called Customization Replicator (CR).
It was a cbbatch script that allowed you to define a set of “customizations”, and then in one step, deploy those customizations to a database. This made it easy to deploy the same set of customizations to a development database, test databases, and then finally to production.
When we said “customizations”, this included:
- applying schema changes
- creation of resource configs
- importing of user forms
- save new versions of existing forms
- put forms into resource configs
- importing DAT files
- creating and editing lists (application, user-defined)
- creating numbering schemes
- compiling ClearBasic code
- executing external programs (registering components, stored procs, data loads, etc.)
A number of our customers used this product, and we used it a lot internally. For example, if a customer called us and had a support question about one of our Clarify add-on products, we could simply run Customization Replicator, which would install our product into a database so we could test it. Customers could also use it to create their own set of “customizations” that could be easily deployed. It all worked pretty well. CR was a cbbatch script. It knew how to make schema changes, and it shelled out to run dataex, cbex, and other applications as part of its process. This product isn’t in use anymore, but the concepts that it used around automation are still relevant.
As we’ve progressed and matured, our usage of CR and cbbatch as a build tool has declined come to a grinding halt, and we’ve started using more contemporary tools.
Build Scripts and Tools
Now days, we tend to use nant or rake as drivers for a lot of our automation. (There are also build script tools such aspsake for Powershell and ant for Java) We also use tools like nunit for automating our test suite runs. We couple these with many of our Clarify tools, including the Dovetail SDK, ArchiveManager, and SchemaEditor.
Like our customers, we have applications that are in our production environment that use the Clarify database. When we want to deploy them, we simply run the build script, which may do any number of things.
The script may:
- apply schema changes
- import DAT files
- copy files to a remote server
- configure web servers
- set permissions
- register DLLs
- use the Dovetail SDK to do something in the Clarify database (such as install list items, create numbering schemes, or configure privileges)
- etc.
By simply running one build script, our dev team can easily deploy to a local development environment, to our test environment, and then when we’re ready – to our production environment.
Continuous Integration
We also use Continuous Integration (CI) servers (such as CruiseControl, Hudson, or TeamCity) for even more automation. What this means is that whenever someone checks in code to our source-code repository (Subversion orGithub), the CI server automatically does a full checkout of code, builds the application, and runs the full test suite. We have notifications, and a large monitor in our team room that shows the status of our build and test processes. So if a build should fail for some reason, then everyone immediately knows. This ensures that our applications stay “green” – meaning all tests pass.
Continuous Deployment
This can also lead to Continuous Deployment. This is a popular topic today, especially around web-based applications (such as flickr or IMVU). What this means is that code gets checked in, tests are run via CI, and if everything succeeds, then the changes are automatically deployed to production. It’s not uncommon to hear organizations that push code to production 20 or even 50 times a day! There’s some work to get to that level, but sophisticated development organizations can reap a lot of benefits from this. This tends to be a process only for mature development organizations that have a very good handle on the rest of their development and testing processes.
For a cool view of this, scroll to the bottom of code.flickr.com, and you’ll see recent deployment statistics:
Automating Clarify changes
So where does all this lead us?
Doing automatic deployments of Clarify changes is pretty straightforward. If I was starting off, I would use a contemporary build script tool such as nant or rake, and simply have it call out to your schema editing tool (Dovetail SchemaEditor, or ddcomp) to perform schema changes, use an import tool (such as Dovetail ArchiveManager, or dataex) for importing forms, and then compile your Clearbasic code using CBEX, or just import it as a DAT file using Dovetail ArchiveManager. File copying, registering components, configuring web servers, and setting permissions are all common tasks that are available in these build tools as well.
Should you also need to do additional stuff in your database, some code that uses the Dovetail SDK can be useful.
You could tie this in with a CI server if you wish, so when new code is checked in, the CI server could automatically push it to a test database.
Postlude
One of the nice things about using common tools (like nant or rake or nunit) is that you can leverage the collective knowledge of the large user base of these tools. If I don’t know how to do something in nant, a simple Google search almost always gives me the answer I was looking for – either in the product docs, a forum, someone’s blog post, or example scripts that I can steal learn from.
Hopefully this outlines some of the experience and knowledge that we have around automated testing and deployment practices, and gives you some ideas around ways to make this happen in your environment.
Need some more information, or perhaps even some example scripts? Hit me up, either here in the comments or viaemail. If there’s interest, I could share some of our automation scripts.
Rock on.
