Liquid Studio Documentation
Data Mapping / Data Mapper Functions / By Category / Set / Distinct
In This Topic
    Distinct
    In This Topic

    Function Name
    Distinct
    Category
    Set
    Icon
    Distinct
    Description
    Map only unique values
    Inputs
    Context Context in which the values should be unique
    Value Values which must be unique within the context in order to be mapped
    Outputs
    Distinct Values Those values that are unique within the context.
    The output dynamically takes its type from the input 'Value'.
    Properties
    None -

    Usage

    The Data Mapper Set Distinct function allows you to ensure only items that are unique within a certain context are mapped. Often, data sources contain duplicate values, sometimes intentionally and sometimes due to data corruption. If your target application requires only unique values of a particular item within a specific context, you can use the Distinct function. To apply the Set Distinct function, use the following process:

    Create a new Data Mapper file, dragging your XML data source and targets into the editor area. For this example we are using clubs_source.xsd, an XML Schema Definition inferred from the following source:

    Source XML
    Copy Code
    <sport_clubs>
     <club>
      <name>New Town Football Club</name>
      <member>
       <active>Yes</active>
       <member_name>Chris Black</member_name>
       <member_since>2005</member_since>
       <won>Cup 2007</won>
       <won>League 2009</won>
       <won>Cup 2007</won>
      </member>
      <member>
       <active>Yes</active>
       <member_name>Jim Thomson</member_name>
       <member_since>2003</member_since>
       <won>League 2010</won>
      </member>
      <member>
       <active>No</active>
       <member_name>Barry Jones</member_name>
       <member_since>1996</member_since>
       <won>League 1999</won>
       <won>Cup 1999</won>
       <won>League 2000</won>
      </member>
     </club>
     <club>
      <name>City Netball Team</name>
      <member>
       <active>No</active>
       <member_name>Jacqueline Johnson</member_name>
       <member_since>1999</member_since>
       <won>National Championship 2001</won>
       <won>National Championship 2001</won>
       <won>Regional Championship 2002</won>
       <won>National Championship 2002</won>
      </member>
      <member>
       <active>Yes</active>
       <member_name>Joanne Smith</member_name>
       <member_since>2005</member_since>
       <won>Regional Championship 2007</won>
       <won>National championship 2007</won>
      </member>
      <member>
       <active>Yes</active>
       <member_name>Mary Owens</member_name>
       <member_since>2007</member_since>
       <won>National Championship 2008</won>
      </member>
     </club>
    </sport_clubs>
    

    For our target schema we will be using clubs_target.xsd:

    The data models sporting clubs and their members. Some of the elements can appear multiple times. Notice that, in the source data, a couple of the "won" elements have duplicate values within their parent "member" elements. Let's assume that we want to eliminate this duplication when we map the data. Here is the Data Mapper with Source and Target imported:

    Data Mapper Source and Target

    Drag the Set Distinct function from the Component Palette into the mapping area:

    Set Distinct Component

    Distinct Transform Added

    The Set Distinct function takes two inputs, representing the value and the context in which it should be unique. The Value input should be connected to the item you only want to map unique instances of. The Context item should be connected to the item you want the values to be unique within. In this case, we want the "won" items to be unique within each "member" element. This means that the same "won" value can appear more than once within a single "club" element or within the document as a whole, but not within a single "member" element - we only want to list each title the member has won once for their entry in the data. The Context input is often connected to the parent of the unique value as in this case. The output should be connected to the target item, whether an item in the XML Writer or another component in the Mapper.

    Connect the Context input of the Distinct function to the "member" output in the XML Reader and the Value input to the "won" output:

    Reader to Distinct

    Now we can connect the Distinct output to the "title" input in the XML Writer. We connect it to the "title" input because that is the value we are mapping only unique values of.

    Reader and Writer to Distinct

    Now items will only be mapped if they are not duplicates. Finally, connect the remaining inputs and outputs. Remember to map "member" to "player". Although we have used the item as a reference for the Distinct function, we have not yet mapped it.

    Inputs and Outputs Connected

    We can now execute the transform by pressing Shift-F5 or the Execute button (Execute). The transform is applied and the file we selected as output opens in the editor:

    Output XML
    Copy Code
    <clubs>
     <team t_name="New Town Football Club">
      <player p_name="Chris Black" is_active="Yes">
       <joined>2005</joined>
       <title>Cup 2007</title>
       <title>League 2009</title>
      </player>
      <player p_name="Jim Thomson" is_active="Yes">
       <joined>2003</joined>
       <title>League 2010</title>
      </player>
      <player p_name="Barry Jones" is_active="No">
       <joined>1996</joined>
       <title>League 1999</title>
       <title>Cup 1999</title>
       <title>League 2000</title>
      </player>
     </team>
     <team t_name="City Netball Team">
      <player p_name="Jacqueline Johnson" is_active="No">
       <joined>1999</joined>
       <title>National Championship 2001</title>
       <title>Regional Championship 2002</title>
       <title>National Championship 2002</title>
      </player>
      <player p_name="Joanne Smith" is_active="Yes">
       <joined>2005</joined>
       <title>Regional Championship 2007</title>
       <title>National championship 2007</title>
      </player>
      <player p_name="Mary Owens" is_active="Yes">
       <joined>2007</joined>
       <title>National Championship 2008</title>
      </player>
     </team>
    </clubs>
    

    The "player" elements only contain unique instances of the "title" element, as the duplicates have not been mapped.

    You may find the Distinct function particularly useful in conjunction with other Set and Aggregate functions such as Sort.
    See Also