Perfect Links

The append process involves taking data from the appended dataset and inserting it into an existing master dataset. The append settings include column links, which are instructions on how to map the appended columns (the data columns in the appended dataset) to the master columns (the data columns in the master dataset). A column link might say, for example, that data from AppendedColumnX should be inserted into MasterColumnY.

The column links are 'perfect' if they:

• give instructions on what to do with every appended data column
• do not give instructions on any nonexistent appended data column

In other words, perfect links align perfectly with the appended dataset. They define exactly what to do with every appended data column, with no extraneous instructions. Imperfect links do no align perfectly with the appended dataset; they contain either too much or too little information.

When you require perfect links, you are controlling the append process in the most cautious way. You are effectively telling Windographer to interrupt the append process whenever it detects any mismatch between the append instruction and the appended dataset.

Example 1: Perfect Links

Consider a simple example where the appended and master datasets each contain four data columns. The four column links below identify what to do with each of the four appended data columns, so they are perfect:

Appended Columns Master Columns Column1 ColumnA Column2 ColumnB Column3 ColumnC Column4 ColumnD

Column Links Appended Column1 → Master ColumnA Appended Column2 → Master ColumnB Appended Column3 → Master ColumnC Appended Column4 → Master ColumnD

Example 2: Perfect Links

Now let's say the master dataset contains additional data columns that the links do not mention, as shown below. These links are still perfect because they clearly identify what to do with every appended data column. Some master data columns will not get added to, but that presents no problem. The point of the column links is to identify what to do with every appended data column, and these column links do that perfectly.

Appended Columns Master Columns Column1 ColumnA Column2 ColumnB Column3 ColumnC Column4 ColumnD ColumnE ColumnF ColumnG

Column Links Appended Column1 → Master ColumnA Appended Column2 → Master ColumnB Appended Column3 → Master ColumnC Appended Column4 → Master ColumnD

Example 3: Perfect Links That Include Ignoring

A link that instructs the append process to ignore an appended data column is still a valid link. The column links below say to ignore two of the four appended data columns. These links are perfect since they give instructions for every appended data column, with no extraneous instructions.

Appended Columns Master Columns Column1 ColumnA Column2 ColumnB Column3 ColumnC Column4 ColumnD

Column Links Appended Column1 → Appended Column2 → Master ColumnD Appended Column3 → Master ColumnB Appended Column4 →

Example 4: Perfect Links That Include Adding

If a column link leads to the addition of a new master data column, that too is a valid link. The column links below say to ignore the first appended data column, map the second and third to new master data columns, and map the fourth to an existing master data column. Once again, these links are perfect because they give instructions for every appended data column, with no extraneous instructions.

Appended Columns Master Columns Column1 ColumnA Column2 ColumnB Column3 ColumnC Column4 ColumnD

Column Links Appended Column1 → Appended Column2 → Master Column2 Appended Column3 → Master Column3 Appended Column4 → Master ColumnB

Example 5: Imperfection Due To Insufficient Instructions

Column links that contain insufficient instructions are imperfect. Imagine the case of a met tower that writes one data file per day, where a new data channel appears on October 13, maybe due to the installation of a new sensor. If you append the 31 data files covering October in a single append process, and you define the column links based on the October 1st file, then when the append process reaches the October 13 file those column links will become imperfect because they don't specify what to do with the new data channel.

In the below illustration of such a scenario, the four column links only instruct what to do with four of the five appended data columns. They are silent on the fifth appended data column, meaning they contain insufficient instructions and are therefore imperfect. Windographer can make a decision as to what to do with Column5, and it will do so if you have allowed imperfect links, but if you have required perfect links then the append process will stop until you provide further instructions. (You provide those instructions via the Append Settings window when using Windographer in the normal way, or via a WAS file when appending automatically using Windographer Monitor.)

Appended Columns Master Columns Column1 ColumnA Column2 ColumnB Column3 ColumnC Column4 ColumnD Column5

Column Links Appended Column1 → Master ColumnA Appended Column2 → Master ColumnB Appended Column3 → Master ColumnC Appended Column4 → Master ColumnD

Example 6: Imperfection Due To Extraneous Instructions

Column links that contain extraneous instructions are also imperfect, because they align imperfectly with the appended dataset. In the example below, the fourth column link refers to an appended data column that does not exist. This is an extraneous instruction, one that Windographer cannot use. It's easy for Windographer simply to ignore this column link, and it will do exactly that if you allow imperfect links, but if you have required perfect links then this situation will cause the append process to stop until you provide further instructions. Essentially, the existing instructions (the column links) appear to apply to an older state of the appended dataset, or maybe to an entirely different dataset, so they are suspect, and the most cautious course of action is to stop the append process and require additional instructions from the user.

Appended Columns Master Columns Column1 ColumnA Column2 ColumnB Column3 ColumnC ColumnD

Column Links Appended Column1 → Master ColumnA Appended Column2 → Master ColumnB Appended Column3 → Master ColumnC Appended Column4 → Master ColumnD

The Effect of Requiring Perfect Links

When you use the Append Settings window to append a single data file, the perfect links option has no real effect because the window itself makes the links perfect. However, it has an important effect when you append multiple files at once. If you append 200 files at once, for example, and you choose to require perfect data links when the Append Settings window appears for the first of those files, then if any of the next 199 files contain a different set of data columns than the first (e.g. a new data channel appears, or an existing data channel disappears), the Append Settings window will reappear so you can provide further instructions.

The perfect links option also matters when you save your settings to a WAS file and use it to direct the automated appending that Windographer Monitor performs.

See also

Append Settings window

WAS file

Written by: Tom Lambert
Contact: windographer.support@ul.com
Last modified: October 13, 2022