tools/README

The tool directory contains several tools that may be essential or useful in E-Maj development:
- regress.sh is a shell script that selects and executes regression tests,
- copy2Expected.sh is a shell script that copys the result files of regression tests environment to the reference directory,
- create_sql_install_script.pl is a perl script that generates the psql install script from the extension install script,
- check_code.pl is a perl script that performs various checks on the script that creates the emaj extension,
- check_error_messages.pl is a perl script that analyzes how the non regression tests cover the error or warning messages coded in the E-Maj source file,
- sync_fct_in_upgrade_script.pl is a perl script that generates the functions part of the upgrade scripts,
- create_cluster.sh is a shell script that ... creates a postgres cluster,
- create_version.sh helps in creating a new E-Maj version.

It also contains some parameter files loaded by various tools:
- emaj_tools.env-dist is a template used to create the emaj_tools.env file that contains all vars you should customize for your E-Maj tools environment,
- emaj_tools.profile contains variables and functions, it is loaded in some scripts,

emaj_tools.env-dist (and emaj_tools.env)
----------------------------------------
It contains vars about yours E-Maj tools environments.
The variable containing the list of versions that you will use in your regression tests is:
  EMAJ_USER_PGVER
The vars for one version of PostgreSQL are:
  PG<version_pg>_DIR
  PG<version_pg>_BIN
  PG<version_pg>_SHARE
... and for a cluster (only one cluster for a major version):
  PG<version_pg>_USER
  PG<version_pg>_DATA
  PG<version_pg>_HOST
  PG<version_pg>_PORT
This file is read by emaj_postgresql.profile.

emaj_tools.profile
------------------
It contains some functions that checks and gets variables about the E-Maj tools environment (for example: PostgreSQL's binaries &  datas).
This file is loaded by regress.sh, create_cluster.sh and all_cluster.sh.

regress.sh
----------
The regress.sh script is the key tool to perform regression tests. It executes scenarios located into the test directory. These scenarios use sql scripts located into the test/sql directory. The output files are stored into the test/<pg_version>/results directory and compared with reference files located into the test/<pg_version>/expected directory.
Several types of tests are available:
- the standard set of tests starting with the emaj installation,
- a dump of an emaj environment from a cluster in one postgres version and a restore in another cluster running another postgres version,
- the standard set of tests starting with the installation of the prior emaj version followed by the migration script,
- some specific tests including an emaj version migration in the middle of the operations.
To complete the standart tests scenarios, a test of the parallel rollback client is run.

Customisation:
First, some symbolic links must be created so that the unique test/sql directory can be used for all postgres version test environment. To do this, just issue for each postgres version the following command:
    ln -s test/sql test/<pg_version>/sql
Similarly, a symbolic link has to be created to reach the emaj.sql script:
    ln -s sql/emaj.sql test/sql/emaj.sql
Then the regress.sh script may be adapted.
At the end of the script, a menu is displayed to propose all available tests. You can adapt it to your environment by modifing values in the arrays EMAJ_REGTEST_*_PGVER.

copy2Expected.sh
----------------
The copy2Expected.sh shell script copies the result files of regression tests environment to the reference directory. Once result regression test files are considered as representing a new reference, you can simply execute the script by typing:
    sh copy2Expected.sh

Customisation:
At the beginning of the script (near line 7), the variable EMAJ_HOME is initialized with the emaj project root directory. You must adapt its value to fit your installation.
Then for each available major postgres version on the environment, 2 commands copy the ouput test files from the /results to the /expected directories. You must adapt the script to your installed major postgres versions.

create_sql_install_script.pl
----------------------------
The create_sql_install_script.pl perl script generates the psql install script from the extension install script. It reads the sql/emaj--devel.sql file and writes the sql/emaj-devel.sql file. The tool is called by regress.sh at every execution, so that both install scripts are maintained in-sync. It performs about 10 changes in the source code. To do that, it recognizes some specific pieces of code and performs the changes. A final check verifies that all expected patterns have been found.

Customisation:
No customization is required.

check_code.pl
-------------
The check_code.pl perl script performs various checks on the script that creates the emaj extension. It detects unused variables in plpgsql functions. It checks the format of lines and reports tab characters or trailing spaces.

Customisation:
Line 29 and 30, two variables set the pathname of the source script to analyze.


check_error_messages.pl
-----------------------
The check_error_messages.pl perl script analyzes how the non regression tests cover the error or warning messages coded in the E-Maj source file. It first reads and extracts error or warning messeges from the source file. It then reads and extracts error or warning messages from the regression tests output files and finaly displays the messages from the source file that have not been found in the test files. Some messages are excluded from the list as they are known as not been covered by tests.

Customisation:
Line 12 to 14, three variables set the pathname of the source script to analyze and the regression tests output results files.


sync_fct_in_upgrade_script.pl
-----------------------------
The sync_fct_in_upgrade_script.pl perl script generates the functions part of the upgrade scripts. It compares the installation scripts of the previous and the current E-Maj versions and deducts the functions to drop and the functions to create in the related upgrade script. In the upgrade script, it keeps the sections related to other objetcs (tables, types, grants,...), two predefined patterns delimiting the functions section.

Customisation:
Line 19 to 21, three variables set the absolute pathname of the scripts to process: the script of the current version (variable $ficCurrSrc), the script of the previous version ($ficPrevSrc) and the already existing upgrade script (variable $ficUpgrade). The script has to be adjusted at each upgrade.


create_cluster.sh
-----------------
The create_cluster.sh shell script ... creates a postgres cluster. It drops a previous installation if it exists. Then, it performs the initdb, customizes the parameters, starts the cluster and executes a psql session to setup the needed objects - tablespaces, roles, extensions.

Customization:
No additional customization is required.
But some SQL commands or postgresql.conf parameters may need to be adjusted.


create_version.sh
-----------------
The create_version.sh shell script materializes a new E-Maj version. It takes a snapshot of the main emaj directory, by creating a parallel ne emaj-<version> directory that will be used to create the tar file of the version. The script has to be run with a bash command from the directory holding the main emaj directory.
A single argument has to be supplied: the name of the new version to create (for instance: 3.1.0). In the new version directory, all components mentioning the <devel> string are adapted to report the proper version identifier.

Customization:
No customization is needed for this script.
