Updater packages are database upgrade scripts which have been bundled (or packaged) together for use by the Updater application when upgrading xTuple ERP databases from one version to the next.
What is the Updater?
The Updater for xTuple ERP is an application that helps you update your xTuple databases from one release to the next, to upgrade from one xTuple ERP edition to another, such as from Manufacturing to Enterprise Edition, and to load extensions developed by third parties into your xTuple database. The Updater reads and processes upgrade scripts or packages, which are collections of files bundled together into .gz files. This page the basics of creating update packages. Additional details are provided on subsequent pages for creating extensions and core updates.
Here are some other related documents:
If you want to know how to use the Updater to upgrade a database or install an add-on, look at UpdaterDoc
What is a Package?
A Package is a compressed collection of files which may contain either a) custom software designed to extend the functionality of xTuple ERP without touching the core xTuple ERP source code or b) upgrade bundles designed to migrate xTuple ERP databases from one version to the next. xTuple creates packages to upgrade databases from one release to the next and to upgrade from one edition to another. Using Packages, third-party software developers can add features without needing C++ programming skills. These features can be integrated seamlessly into xTuple ERP independent of the release cycles and source code requirements defined by xTuple. In short, Packages are flexible. They make it easy for software developers to add new features—or even just simple tweaks—to customize and enhance the xTuple ERP experience.
The structure of an Updater Package
At the most basic level, Updater Packages are compressed tar files. This format has been adopted as one of several standards for bundling files for distribution over the Internet.
The contents of the tar file have a fairly uniform structure. At the top level is a single directory. Inside this directory there must be a file called package.xml. This file describes to the Updater how to load the rest of the package. The other files included in the package are either SQL scripts to run to modify the database or any of the following items to load into the database:
- user interface forms
- application scripts
- report definitions
- MetaSQL statements
These may all be directly in the top level directory of the package or arranged in subdirectories.
Getting started building an Updater Package
The first thing you need to do when building an Updater package is to create a directory:
$ mkdir sampledir
Note: All of the command line examples in this document assume that you are working in a UNIX-like environment. That could be a Terminal window on the Macintosh, a Shell or Console window on Linux, or a Cygwin window on Windows. Things that the user types will always be preceded by '$ ', indicating a shell prompt. Lines without this prompt are sample output lines.
Everything that you want to include in this package should eventually be put into this directory. The first thing you should put in this directory is a package.xml file. Building a package is an iterative process of adding files to the directory, modifying package.xml, bundling everything together, and testing. So don't worry yet about having all of the pieces together.
The package.xml file contains the information required to orchestrate a database upgrade or package addition to an xTuple database. The syntax is described in full later in this article. The file is, as its name suggests, an XML file. The root element must be <package>. Attributes of the <package> element name the package, provide a consistency check with the package's top-level directory, and let the creator specify a minimum Updater version to use when loading the package:
<package id = "sampledir"
name = "samplepackage"
version = "1.0alpha"
developer = "xTuple"
descrip = "Sample described on the Create Updater Packages page."
updater = "2.2.0beta2">
This is an example for the page describing the
packaging functionality in xTuple ERP's Updater utility.
In order of appearance, here's what this all means:
The id attribute says that the contents of this package all come from a directory named sampledir.
By providing a name, the author is telling the Updater that this is an add-on package, rather than an update to the core xTuple ERP application. The package will be stored with the name samplepackage.
The version indicates this is an alpha release of the first version of this package.
This package was developed by the company xTuple.
If this is an add-on package, the descrip will be displayed on the Packages and Package windows in xTuple ERP.
This package cannot be loaded by any version of the Updater before 2.2.0Beta2. This means that this package cannot be loaded by Updater 1.0, 1.1, or even 2.0.0beta. It can be loaded by Updater 2.2.0, 2.2.1ALPHA, or 3.2.7 (not that versions 2.2.1ALPHA or 3.2.7 exist yet:-).
The <pkgnotes> are stored as comments for add-on packages and are visible on the Package window in the xTuple ERP application.
The id and updater attributes of the <package> element allow for simplistic consistency checking: Does this package contain the expected directory and are you running a version of the Updater that can load this package. This is rarely sufficient for loading a package of any real complexity.
The Updater lets you provide database queries to make more interesting checks. xTuple uses these Query prerequisites extensively in upgrade packages to confirm that the starting database version for the upgrade matches expectations and to check for data that need to be fixed before the upgrade will be successful. The sole criterion for writing a correct prerequisite query is that it return an SQL boolean TRUE value if it is safe for the Updater to load this package.
Here is an example:
<prerequisite type="Query" name="Checking if xtbatch is installed" >
<query>SELECT COUNT(*) = 1
WHERE pkghead_name = 'xtbatch';</query>
This package must be applied to an xTuple ERP database with the Batch Manager package.
The type attributes says that this <prerequisite> element is a database query.
Before the Updater runs the statement in the <query> element it writes the value of the name attribute, "Checking if xtbatch is installed", to the Updater's message window.
The <query> element then gets run, checking to see if this is an xTuple database with the xtbatch package. If the database does not have a pkghead table (therefore not an xTuple ERP database), then the prerequisite check will fail.
If the query fails then the contents of the <message> element will be written to the Updater's message window, probably in red type.
All of the prerequisites are checked before the Updater returns from this phase of processing. This lets the user learn about all of the problems that s/he must fix before trying to run the Updater again to load this package.
Note: This also imposes an obligation on the package developer: Create prerequisite checks for any database modifications that might lead to upgrade or installation failures.
There is also a license prerequisite, which allows the package developer to present the user with an end-user license agreement. The user must click an Acceptbutton before the Updater will allow the package to be installed. This license is not written anywhere by the Updater. The license text in the <message> element may contain rich text, as defined by Trolltech:
<prerequisite type="license" name="sampleLicense" >
<p>This is the content of a <i>rich text</i> license.
You must either accept or not.</p>
Notice the < in several places. Because this is an XML file, the < character used to mark the beginnings of rich text tags must be escaped so they are not treated as part of the <package> element tree. xTuple has tested the Updater using < substitution for <. Using XML CDATA sections should also work but xTuple has not tested this.
Most of the work is described in external files
Now that the preliminaries have been dispensed with, how do we actually load things into the database? For the most part by creating small files that do (or contain) just one thing.
You can create arbitrary database scripts. If you have several changes that you want to make to the database that are all related, they should go in a single script file. If there are several unrelated changes, create several different files. Put these script files into the package directory and add a line to package.xml for each:
<script file="runFirst.sql" />
The Updater has several different elements for specific kinds of database object. These can be used for either update or add-on packages; the syntax was specifically designed to support the bookkeeping necessary for add-on packages but may also be used in database upgrade or xTuple ERP edition upgrade packages. Put a script file for each of the following sample lines into the package directory:
<createtable file = "pkgtest.sql" name = "pkgtest" />
<createfunction file = "myfunction.sql" name = "myfunction" />
<createtrigger file = "pkgtesttrigger.sql" name = "pkgtestbefore" />
<createview file = "pkgtestview.sql" name = "pkgtestview" />
Run the SQL script in the file called pkgtest.sql and, if this is an add-on package, record that a table named pkgtest is part of this package.
Run the SQL script in myfunction.sql and, if this is an add-on package, record that one or more functions named myfunction are part of this package.
Run the SQL script in pkgtesttrigger.sql and, if this is an add-on package, record that there is a pkgtestbefore trigger in this package.
Run the SQL script in pkgtestview.sql and, if this is an add-on package, record that there is a view pkgtestview in this package.
The package developer has to decide whether to use the specific <createstuff> elements or <script>. If the package is a non-core feature contribution outside the application core, then the decision is easy to make: Use <createstuff> because these elements automatically do the proper bookkeeping for storing the package as a unit in the database and reporting the package contents accurately. When building upgrades to the core, however, the choice is less clear. One of the properties of the <createstuff> elements is that the Updater processes them in a particular order to minimize conflict. If this default order does not work for this particular upgrade, however, the package creator must use <script>, <initscript>, and <finalscript> elements to force a sequence that will work.
The Updater also has different elements for xTuple ERP objects. These are stored in special tables used by the xTuple ERP database. The files used for the following elements are not database scripts. Instead they hold the actual objects themselves.
<loadmetasql file="sampleQuery.mql" />
<loadreport file="sampleReport.xml" />
<loadappscript file="sampleScreen.script" name="sampleScreen" />
<loadappui file="sampleScreen.ui" />
<loadimage file="sampleImage.png" name="aPicture" />
Read a MetaSQL statement from the file sampleQuery.mql and save it in the database. See the Special notes on MetaSQL definitions for more information.
Read an OpenRPT report definition from the file sampleReport.xml and save it in the database.
Read an application script from sampleScreen.script and save it in the database with the name sampleScreen. This name allows it to work nicely with the next item in the package.
Read an application user interface definition from sampleScreen.ui and save it in the database.
Read an image from the file sampleImage.png and save it in the database as an image with the application-level name aPicture.
If you have a complex package with more than a handful of files, you'll probably want to create a small directory structure to organize the files. If you do this, each file attribute will need to hold the directory path to the file, not just the simple file name. For example, if you put all of your report definitions in a reports subdirectory, the loadreport line above would have to set the file to "reports/sampleReport.xml".
A few things are described in the file itself
You could create new privileges and custom commands using <script> elements. However, these things are so simple and so common that the Updater has direct support for creating them without SQL scripts.
<loadpriv name="ViewSampleScreen" module="System">
Allowed to view the sample screen
This creates a privilege named ViewSampleScreen. It can be assigned to users or groups of users by selecting System from the Module combo box on the User Information or Group windows.
<loadcmd name="SampleCMD" title="Sample Screen"
Sample Screen Custom Command Description - view mode
<arg value="uiform=SampleScreen" />
<arg value="mode=view" />
This creates a new custom command called SampleCMD, which appears in the menu system as System -> Custom Commands -> Sample Screen. When the user selects this menu item, the application opens the User Interface form Sample Screen, which was loaded earlier, and passes it the argument mode=view.
Creating the Package itself
After you have copied all of the scripts, .ui files, .mql files, OpenRPT .xml files, etc. to your package directory, you can bundle all of the pieces together with a single command:
tar czf packagedir.gz packagedir
You might have problems with Apple's version of the tar utility. The .gz file will be created just fine but the file format might not be recognized by the Updater utility. If this happens, then try the following command instead:
# try the gnutar command on Macs
gnutar czf packagedir.gz --format gnu packagedir