Dataload Utility

Data can be loaded into WCS tables using data load utility.

WCS Data Load utility performs the following functions in a single operation:

1.             Reads the data from the input source file.

2.             Transforms the source data to Web Sphere Commerce business objects.

3.             Allocates and resolves Web Sphere Commerce business objects to physical data.

4.             Loads the physical data into the database.

There are three configurations files and a input source file required to complete the data loader setup. WCS supports only CSV File Reader for other input source you have to write your own Reader Class. This means that if you are using OOB CSV Reader then your input file must be a CSV file. The three configuration files are listed below.

1.             wc-dataload-env.xml

2.             wc-dataload-loader.xml

3.             wc-dataload.xml

wc-dataload-env.xml holds the environment settings such as database name, encrypted password, database type, user name, schema name.

wc-dataload-loader.xml holds the mapping between your input CSV file and the WCS tables.

wc-dataload.xml is the main xml that is passed as parameter to the data load utility, this file contains information about the environment file, loader file and input CSV file for the loader.

Now let us see a simple example to understand the data load utility.

First let us create our environment file, it should look like the one shown below.

<?xml version="1.0" encoding="UTF-8" ?>
<_config:DataLoadEnvConfiguration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.ibm.com/xmlns/prod/commerce/foundation/config ../../../../xml/config/xsd/wc-dataload-env.xsd"
xmlns:_config="http://www.ibm.com/xmlns/prod/commerce/foundation/config">
<_config:BusinessContext storeIdentifier="Aurora" catalogIdentifier="Aurora" languageId="-1" currency="USD" />
<_config:Database type="derby" name="..\..\db\mall" />
<_config:DataWriter className="com.ibm.commerce.foundation.dataload.datawriter.JDBCDataWriter" />
</_config:DataLoadEnvConfiguration>

In <_config:BusinessContext> tag, storeIdentifier attribute contains the store information to which the data should get loaded. catalogIdentifier attribute contains the catalog identifier of the store (This attribute is optional), languageId(optional attribute) is the default language ID that should be used if language information is not specified in the input CSV file. currency(optional attribute) is the default currency used for price.

In <_config:Database> tag, type attribute holds the database type, type can be Derby, Oracle or DB2. Name attribute holds the database name. We are using Derby database, no password is required to connect to Derby. If you want to connect to Oracle or DB2 the syntax would be the one shown below.

For Oracle,

<_config:Database type="Oracle" name="database name" user="user" password="encrypted password" port="1521" schema="schema name" server="server" driverType="thin" />

For DB2,

<_config:Database type="db2" name="database name" user="user" password="encrypted password" server="server" port="50000" schema="schema name" />

<_config:DataWriter> tag specifies the Writer that should be used to perform insert,update and delete operations. If you do not sepcify the DataWriter then by default JDBCDataWriter is used. JDBCDataWriter can be used for ORACLE,DERBY and DB2. If you want to use database native data writer for Oracle and DB2, it can be used using the below syntax

<_config:DataWriter className="com.ibm.commerce.foundation.dataload.datawriter.NativeDBDataWriter" />

Note : NativeDBDataWriter only supports insert operation.

Now let us switch to the loader file that contains the mapping information between input CSV file and Business Objects. This loader file creates a new Catalog for the store mentioned in the environment file.

<_config:DataMapping>
<_config:mapping xpath="CatalogGroupIdentifier/UniqueID" value="GroupUniqueId" />
<_config:mapping xpath="CatalogGroupIdentifier/ExternalIdentifier/GroupIdentifier" value="GroupIdentifier" />
<_config:mapping xpath="ParentCatalogGroupIdentifier/UniqueID" value="ParentGroupUniqueId" />
<_config:mapping xpath="CatalogGroupIdentifier/ExternalIdentifier/StoreIdentifier/ExternalIdentifier/NameIdentifier" value="StoreIdentifier" valueFrom="InputData" />
<_config:mapping xpath="ParentCatalogGroupIdentifier/ExternalIdentifier/GroupIdentifier" value="ParentGroupIdentifier" />
<_config:mapping xpath="ParentCatalogGroupIdentifier/ExternalIdentifier/StoreIdentifier/ExternalIdentifier/NameIdentifier" value="ParentStoreIdentifier" valueFrom="InputData" />
<_config:mapping xpath="topCatalogGroup" value="TopGroup" />
<_config:mapping xpath="displaySequence" value="Sequence" />
<_config:mapping xpath="Description[0]/Name" value="Name" />
<_config:mapping xpath="Description[0]/ShortDescription" value="ShortDescription" />
<_config:mapping xpath="Description[0]/LongDescription" value="LongDescription" />
<_config:mapping xpath="Description[0]/Thumbnail" value="Thumbnail" />
<_config:mapping xpath="Description[0]/FullImage" value="FullImage" />
<_config:mapping xpath="Description[0]/Attributes/published" value="Published" />
<_config:mapping xpath="Description[0]/Keyword" value="Keyword" />
<_config:mapping xpath="Description[0]/Attributes/note" value="Note" />
<_config:mapping xpath="Attributes/field1" value="Field1"/>
<_config:mapping xpath="Attributes/field2" value="Field2"/>
<_config:mapping xpath="" value="Delete" valueFrom="InputData" deleteValue="1"/>
</_config:DataMapping>

Our Dataload file would like the one shown below.
<?xml version="1.0" encoding="UTF-8" ?>
<_config:DataLoadConfiguration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.ibm.com/xmlns/prod/commerce/foundation/config ../../../../xml/config/xsd/wc-dataload.xsd"
xmlns:_config="http://www.ibm.com/xmlns/prod/commerce/foundation/config">
<_config:DataLoadEnvironment configFile="wc-dataload-env.xml" />
<_config:LoadOrder commitCount="100" batchSize="1" dataLoadMode="Replace">
<_config:LoadItem name="CatalogGroup" businessObjectConfigFile="wc-loader-catalog-group.xml">
<_config:DataSourceLocation location="CatalogGroups.csv" />
</_config:LoadItem>
<_config:LoadItem name="CatalogEntry" businessObjectConfigFile="wc-loader-catalog-entry.xml">
<_config:DataSourceLocation location="CatalogEntries.csv" />
</_config:LoadItem>
</_config:LoadOrder>
</_config:DataLoadConfiguration>

Running the Data load Utility
To run  the dataload utility, just go to command prompt and navigate to WCS installation folder and then open bin folder, by default it would be the one shown below.

C:\IBM\WCDE_INSTALL70\bin

Type dataload.bat <path of dataload file> for e.g dataload 

C:/IBM/WCDE_INSTALL70/samples/DataLoad/Catalog/wc-dataload.xml