Title: Windows Live Agents: XML Normalization
Slug: windows-live-agents-xml-normalization
Date: 2008-03-24 15:44:00
Author: Kartones
Lang: en
Tags: Development, XML, Windows Live Agents
Description: XML normalization using Buddyscript to handle complex XML structures.

 <p align="justify"><br>Buddyscript is capable of managing xmls with ease, and accessing external databases for example with a web service. </p> <p align="justify">But sometimes we may need to have non-trivial xmls locally, and those xmls may have complex structures not so easy to handle.</p> <p> </p> <p>This is the graphical representation of a sample xml that may not be easy to read:</p> <p><img src="https://images.kartones.net/posts/screenshots/wla_xml_normalization_001.png"> </p> <p>This is a possible xml content:</p> <p><font size="2"><font face="Courier New"><font color="blue">&lt;</font><font color="maroon">?xml</font><font color="red"> version</font><font color="blue">="1.0"</font><font color="red"> encoding</font><font color="blue">="utf-8"?&gt;</font></font></font><font color="black"><br></font><font size="2"><font face="Courier New"><font color="blue">&lt;</font><font color="maroon">fields</font><font color="blue">&gt;</font></font></font><font color="black"><br><font face="Courier New" size="2">  </font></font><font size="2"><font face="Courier New"><font color="blue">&lt;</font><font color="maroon">field</font><font color="blue">&gt;</font></font></font><font color="black"><br><font face="Courier New" size="2">    </font></font><font size="2"><font face="Courier New"><font color="blue">&lt;</font><font color="maroon">field1</font><font color="blue">&gt;</font><font color="black">1</font><font color="blue">&lt;/</font><font color="maroon">field1</font><font color="blue">&gt;</font></font></font><font color="black"><br><font face="Courier New" size="2">    </font></font><font size="2"><font face="Courier New"><font color="blue">&lt;</font><font color="maroon">field2</font><font color="blue">&gt;</font></font></font><font color="black"><br><font face="Courier New" size="2">      </font></font><font size="2"><font face="Courier New"><font color="blue">&lt;</font><font color="maroon">value</font><font color="blue">&gt;</font><font color="black">121</font><font color="blue">&lt;/</font><font color="maroon">value</font><font color="blue">&gt;</font></font></font><font color="black"><br><font face="Courier New" size="2">      </font></font><font size="2"><font face="Courier New"><font color="blue">&lt;</font><font color="maroon">value</font><font color="blue">&gt;</font><font color="black">122</font><font color="blue">&lt;/</font><font color="maroon">value</font><font color="blue">&gt;</font></font></font><font color="black"><br><font face="Courier New" size="2">      </font></font><font size="2"><font face="Courier New"><font color="blue">&lt;</font><font color="maroon">value</font><font color="blue">&gt;</font><font color="black">123</font><font color="blue">&lt;/</font><font color="maroon">value</font><font color="blue">&gt;</font></font></font><font color="black"><br><font face="Courier New" size="2">    </font></font><font size="2"><font face="Courier New"><font color="blue">&lt;/</font><font color="maroon">field2</font><font color="blue">&gt;</font></font></font><font color="black"><br><font face="Courier New" size="2">    </font></font><font size="2"><font face="Courier New"><font color="blue">&lt;</font><font color="maroon">field3</font><font color="blue">&gt;</font><font color="black">13</font><font color="blue">&lt;/</font><font color="maroon">field3</font><font color="blue">&gt;</font></font></font><font color="black"><br><font face="Courier New" size="2">  </font></font><font size="2"><font face="Courier New"><font color="blue">&lt;/</font><font color="maroon">field</font><font color="blue">&gt;</font></font></font><font color="black"><br><font face="Courier New" size="2">  </font></font><font size="2"><font face="Courier New"><font color="blue">&lt;</font><font color="maroon">field</font><font color="blue">&gt;</font></font></font><font color="black"><br><font face="Courier New" size="2">    </font></font><font size="2"><font face="Courier New"><font color="blue">&lt;</font><font color="maroon">field1</font><font color="blue">&gt;</font><font color="black">2</font><font color="blue">&lt;/</font><font color="maroon">field1</font><font color="blue">&gt;</font></font></font><font color="black"><br><font face="Courier New" size="2">    </font></font><font size="2"><font face="Courier New"><font color="blue">&lt;</font><font color="maroon">field2</font><font color="blue">&gt;</font></font></font><font color="black"><br><font face="Courier New" size="2">      </font></font><font size="2"><font face="Courier New"><font color="blue">&lt;</font><font color="maroon">value</font><font color="blue">&gt;</font><font color="black">221</font><font color="blue">&lt;/</font><font color="maroon">value</font><font color="blue">&gt;</font></font></font><font color="black"><br><font face="Courier New" size="2">      </font></font><font size="2"><font face="Courier New"><font color="blue">&lt;</font><font color="maroon">value</font><font color="blue">&gt;</font><font color="black">222</font><font color="blue">&lt;/</font><font color="maroon">value</font><font color="blue">&gt;</font></font></font><font color="black"><br><font face="Courier New" size="2">      </font></font><font size="2"><font face="Courier New"><font color="blue">&lt;</font><font color="maroon">value</font><font color="blue">&gt;</font><font color="black">223</font><font color="blue">&lt;/</font><font color="maroon">value</font><font color="blue">&gt;</font></font></font><font color="black"><br><font face="Courier New" size="2">    </font></font><font size="2"><font face="Courier New"><font color="blue">&lt;/</font><font color="maroon">field2</font><font color="blue">&gt;</font></font></font><font color="black"><br><font face="Courier New" size="2">    </font></font><font size="2"><font face="Courier New"><font color="blue">&lt;</font><font color="maroon">field3</font><font color="blue">&gt;</font><font color="black">23</font><font color="blue">&lt;/</font><font color="maroon">field3</font><font color="blue">&gt;</font></font></font><font color="black"><br><font face="Courier New" size="2">  </font></font><font size="2"><font face="Courier New"><font color="blue">&lt;/</font><font color="maroon">field</font><font color="blue">&gt;</font></font></font><font color="black"><br></font><font size="2"><font face="Courier New"><font color="blue">&lt;/</font><font color="maroon">fields</font><font color="blue">&gt;</font></font></font><font color="black"></font></p> <p> </p> <p>The buddyscript code to read this xml you could incorrectly think that it might be something like this:</p> <p><font face="Courier New" size="2">datasource Table1XMLFileLoader() =&gt; Field1, Field2, Field3<br>  file<br>    table1.xml<br>  simple xml<br>      fields <br>     <b>   field {loop=content} <br></b>          field1<br>          field2<br>            <b>value {loop=content}<br></b>          field3</font>  </p><p>The problem is that if you execute this code, the agent can incorrectly loop the values, because it doesn't supports multiple anidated loops. This gets worse with three or more "looping fields". The XML should only have one looping tag/element.</p> <p> </p> <p>So, how to fix this limitation? Well, I do it applying what I call <i>xml normalization</i>: Like with a <a href="http://en.wikipedia.org/wiki/Database_normalization">DB normalization</a>, re-estructure your xml in multiple files, splitted so that each sub-table now only has as simple 1-1 relations (but they can be repeated as many times as wanted, that'sd the trick ;)</p> <p>This is the graphical representation of the normalized xmls:</p> <p><img src="https://images.kartones.net/posts/screenshots/wla_xml_normalization_002.png"> </p> <p>This are our normalized xmls:</p> <p><u>Table1_1.xml</u></p> <p><font color="blue">&lt;</font><font color="maroon">?xml</font><font color="red"> version</font><font color="blue">="1.0"</font><font color="red"> encoding</font><font color="blue">="utf-8"?&gt;</font><font color="black"><br></font><font color="blue">&lt;</font><font color="maroon">fields</font><font color="blue">&gt;</font><font color="black"><br>  </font><font color="blue">&lt;</font><font color="maroon">field</font><font color="blue">&gt;</font><font color="black"><br>    </font><font color="blue">&lt;</font><font color="maroon">field1</font><font color="blue">&gt;</font><font color="black">1</font><font color="blue">&lt;/</font><font color="maroon">field1</font><font color="blue">&gt;</font><font color="black"><br>    </font><font color="blue">&lt;</font><font color="maroon">field2</font><font color="blue">&gt;</font><font color="black">121</font><font color="blue">&lt;/</font><font color="maroon">field2</font><font color="blue">&gt;</font><font color="black"><br>  </font><font color="blue">&lt;/</font><font color="maroon">field</font><font color="blue">&gt;</font><font color="black"><br>  </font><font color="blue">&lt;</font><font color="maroon">field</font><font color="blue">&gt;</font><font color="black"><br>    </font><font color="blue">&lt;</font><font color="maroon">field1</font><font color="blue">&gt;</font><font color="black">1</font><font color="blue">&lt;/</font><font color="maroon">field1</font><font color="blue">&gt;</font><font color="black"><br>    </font><font color="blue">&lt;</font><font color="maroon">field2</font><font color="blue">&gt;</font><font color="black">122</font><font color="blue">&lt;/</font><font color="maroon">field2</font><font color="blue">&gt;</font><font color="black"><br>  </font><font color="blue">&lt;/</font><font color="maroon">field</font><font color="blue">&gt;</font><font color="black"><br>  </font><font color="blue">&lt;</font><font color="maroon">field</font><font color="blue">&gt;</font><font color="black"><br>    </font><font color="blue">&lt;</font><font color="maroon">field1</font><font color="blue">&gt;</font><font color="black">1</font><font color="blue">&lt;/</font><font color="maroon">field1</font><font color="blue">&gt;</font><font color="black"><br>    </font><font color="blue">&lt;</font><font color="maroon">field2</font><font color="blue">&gt;</font><font color="black">123</font><font color="blue">&lt;/</font><font color="maroon">field2</font><font color="blue">&gt;</font><font color="black"><br>  </font><font color="blue">&lt;/</font><font color="maroon">field</font><font color="blue">&gt;</font><font color="black"><br>  </font><font color="blue">&lt;</font><font color="maroon">field</font><font color="blue">&gt;</font><font color="black"><br>    </font><font color="blue">&lt;</font><font color="maroon">field1</font><font color="blue">&gt;</font><font color="black">2</font><font color="blue">&lt;/</font><font color="maroon">field1</font><font color="blue">&gt;</font><font color="black"><br>    </font><font color="blue">&lt;</font><font color="maroon">field2</font><font color="blue">&gt;</font><font color="black">221</font><font color="blue">&lt;/</font><font color="maroon">field2</font><font color="blue">&gt;</font><font color="black"><br>  </font><font color="blue">&lt;/</font><font color="maroon">field</font><font color="blue">&gt;</font><font color="black"><br>  </font><font color="blue">&lt;</font><font color="maroon">field</font><font color="blue">&gt;</font><font color="black"><br>    </font><font color="blue">&lt;</font><font color="maroon">field1</font><font color="blue">&gt;</font><font color="black">2</font><font color="blue">&lt;/</font><font color="maroon">field1</font><font color="blue">&gt;</font><font color="black"><br>    </font><font color="blue">&lt;</font><font color="maroon">field2</font><font color="blue">&gt;</font><font color="black">222</font><font color="blue">&lt;/</font><font color="maroon">field2</font><font color="blue">&gt;</font><font color="black"><br>  </font><font color="blue">&lt;/</font><font color="maroon">field</font><font color="blue">&gt;</font><font color="black"><br>  </font><font color="blue">&lt;</font><font color="maroon">field</font><font color="blue">&gt;</font><font color="black"><br>    </font><font color="blue">&lt;</font><font color="maroon">field1</font><font color="blue">&gt;</font><font color="black">2</font><font color="blue">&lt;/</font><font color="maroon">field1</font><font color="blue">&gt;</font><font color="black"><br>    </font><font color="blue">&lt;</font><font color="maroon">field2</font><font color="blue">&gt;</font><font color="black">223</font><font color="blue">&lt;/</font><font color="maroon">field2</font><font color="blue">&gt;</font><font color="black"><br>  </font><font color="blue">&lt;/</font><font color="maroon">field</font><font color="blue">&gt;</font><font color="black"><br></font><font color="blue">&lt;/</font><font color="maroon">fields</font><font color="blue">&gt;</font><font color="black"></font></p> <p><u>Table2_2.xml</u></p> <p><font color="blue">&lt;</font><font color="maroon">?xml</font><font color="red"> version</font><font color="blue">="1.0"</font><font color="red"> encoding</font><font color="blue">="utf-8"?&gt;</font><font color="black"><br></font><font color="blue">&lt;</font><font color="maroon">fields</font><font color="blue">&gt;</font><font color="black"><br>  </font><font color="blue">&lt;</font><font color="maroon">field</font><font color="blue">&gt;</font><font color="black"><br>    </font><font color="blue">&lt;</font><font color="maroon">field1</font><font color="blue">&gt;</font><font color="black">1</font><font color="blue">&lt;/</font><font color="maroon">field1</font><font color="blue">&gt;</font><font color="black"><br>    </font><font color="blue">&lt;</font><font color="maroon">field3</font><font color="blue">&gt;</font><font color="black">13</font><font color="blue">&lt;/</font><font color="maroon">field3</font><font color="blue">&gt;</font><font color="black"><br>  </font><font color="blue">&lt;/</font><font color="maroon">field</font><font color="blue">&gt;</font><font color="black"><br>  </font><font color="blue">&lt;</font><font color="maroon">field</font><font color="blue">&gt;</font><font color="black"><br>    </font><font color="blue">&lt;</font><font color="maroon">field1</font><font color="blue">&gt;</font><font color="black">2</font><font color="blue">&lt;/</font><font color="maroon">field1</font><font color="blue">&gt;</font><font color="black"><br>    </font><font color="blue">&lt;</font><font color="maroon">field3</font><font color="blue">&gt;</font><font color="black">23</font><font color="blue">&lt;/</font><font color="maroon">field3</font><font color="blue">&gt;</font><font color="black"><br>  </font><font color="blue">&lt;/</font><font color="maroon">field</font><font color="blue">&gt;</font><font color="black"><br></font><font color="blue">&lt;/</font><font color="maroon">fields</font><font color="blue">&gt;</font><font color="black"></font></p> <p> </p> <p>And finally the buddyscript code to read them, this time correctly:</p> <p><font face="Courier New" size="2">datasource Table1_1XMLFileLoader() =&gt; Field1, Field2<br>  file<br>    table1.xml<br>  simple xml<br>      fields <br>        field {loop=content} <br>          field1<br>          field2 </font> </p><p><font face="Courier New" size="2">datasource Table1_2XMLFileLoader() =&gt; Field1, Field3<br>  file<br>    table1.xml<br>  simple xml<br>      fields <br>        field {loop=content} <br>          field1<br>          field3</font>  </p><p>  </p><p>We can now do simple querys against each of the sub-tables and cross the results (with <font color="blue">&lt;</font><font color="maroon">field1</font><font color="blue">&gt;</font>) to obtain one or more registers (original Table1 <font color="blue">&lt;</font><font color="maroon">field</font><font color="blue">&gt;</font>) with multiple subfields (origintal Table1 <font color="blue">&lt;</font><font color="maroon">field2</font><font color="blue">&gt;</font>).  </p>
