Linked Data Cubes Explorer (LDCX)

On this page, we present the Linked Data Cubes Explorer (LDCX), an application of olap4ld.

General info

• Most current demo website: http://ldcx.linked-data-cubes.org/projects/ldcx/
• Old link to demo: http://www.ldcx.linked-data-cubes.org:8000/ldcx-trunk/ldcx/ld-cubes-explorer.html

Architecture

LDCX consists of several parts: an HTML webpage using the JavaScript library xmla4js to issue XMLA queries; olap4j-xmlaserver that can answer XMLA queries via olap4ld; olap4ld to answer metadata and olap queries on QB datasets.

Architecture of LDCX.

Three-step user interface

1. a user selects one or more comma-separated URIs of of qb:DataSets. With ``Explore Dataset..., metadata queries are issued to populate the user interface;
2. the user selects measures to be displayed in the pivot table cells;
3. the user selects dimensions to add member combinations to rows and columns of the pivot table and clicks ``Update Table\ldots. Note LDCX automatically queries every dimension on the most granular level since multi-level hierarchies are rarely used and users can still slice dimensions to view datasets on a higher aggregation level.

Linked Data Cubes Engine: EmbeddedSesameEngine

• EmbeddedSesameEngine

Client Requirements

• LDCX has been tested on Firefox and Chrome (however, only on a Linux PC).

Server Requirements

• olap4j
• olap4ld
• Sesame
• Olap4j-xmlaserver
• xmla4js
• LDCX interface
• Tomcat
  • sudo apt-get install tomcat6 tomcat6-admin
  • nano /var/lib/tomcat6/conf/tomcat-users.xml - added users
  • Added new folder in webapps directory: /var/lib/tomcat6/webapps/xmla4js

Evaluation

• LDCX Evaluation for ESWC 2014 Demo
• OLAP4LD Demo at ESWC 2014

Roadmap

• Displaying of background information useful for interpretation
• NON EMPTY Clause in MDX

How to connect all systems?

• I used eclipse
• I manually created a dynamic web project
• In there, I copied all files from olap4j-xmlaserver master
• Then I manually added libraries to WEB-INF/lib that xmlaserver relied upon:

commons-collections.jar
commons-dbcp.jar
commons-pool.jar
log4j.jar
nxparser-1.2.3.jar
olap4j-xmlaserver-0.0.1-SNAPSHOT.jar
saxon-9.2.jar
servlet-api.jar
xercesImpl.jar

• Then I used the web.xml from Rossi:

<?xml version="1.0" encoding="ISO-8859-1"?>
<web-app>
	<display-name>XMLA Server</display-name>
	
	<servlet>
		<servlet-name>xmla</servlet-name>
		<servlet-class>mondrian.xmla.impl.Olap4jXmlaServlet</servlet-class>
		
		<init-param>
			<param-name>OlapDriverUsePreConfiguredDiscoverDatasourcesResponse</param-name>
			<param-value>true</param-value>
		</init-param>

		<init-param>
			<param-name>OlapDriverDiscoverDatasources.dataSourceName</param-name>
			<param-value>@dataSourceName@</param-value>
		</init-param>
		
		<init-param>
			<param-name>OlapDriverDiscoverDatasources.dataSourceInfo</param-name>
			<param-value>@dataSourceInfo@</param-value>
		</init-param>
		<init-param>
			<param-name>OlapDriverDiscoverDatasources.url</param-name>
			<param-value>@datasourceUrl@</param-value>
		</init-param>
		<init-param>
			<param-name>OlapDriverDiscoverDatasources.providerType</param-name>
			<param-value>TDP</param-value>
		</init-param>
		
		<init-param>
			<param-name>OlapDriverDiscoverDatasources.authenticationMode</param-name>
			<param-value>Unauthenticated</param-value>
		</init-param>

		<init-param>
			<param-name>OlapDriverMaxNumConnectionsPerUser</param-name>
			<param-value>15</param-value>
		</init-param>
				
		<init-param>
			<param-name>OlapDriverClassName</param-name>
			<param-value>org.olap4j.driver.ld.LdOlap4jDriver</param-value>
		</init-param>

		<init-param>
			<param-name>OlapDriverConnectionString</param-name>
			<param-value>jdbc:ld://olap4ld;Catalog=LdCatalog;JdbcDrivers=com.mysql.jdbc.Driver;Server=http://public.b-kaempgen.de:8890/sparql;Database=OPENVIRTUOSO;Datastructuredefinitions=http://fios:saiku,http://fios:saikumanually;Datasets=http://fios:saiku,http://fios:saikumanually</param-value>
		</init-param>
		
		<init-param>
			<param-name>OlapDriverConnectionProperty.Prefix</param-name>
			<param-value>@connectionPrefix@</param-value>
		</init-param>

		<init-param>
			<param-name>OlapDriverConnectionProperty.FetchLevelProperties</param-name>
			<param-value>false</param-value>
		</init-param>

		<init-param>
			<param-name>OlapDriverConnectionProperty.FetchMemberProperties</param-name>
			<param-value>false</param-value>
		</init-param>

		<init-param>
			<param-name>OlapDriverConnectionProperty.FetchMembers</param-name>
			<param-value>true</param-value>
		</init-param>

		<init-param>
			<param-name>OlapDriverConnectionProperty.QueryTimeoutMs</param-name>
			<param-value>@queryTimeout@</param-value>
		</init-param>
		
		<init-param>
			<param-name>OlapDriverConnectionProperty.AuthenticationProvider</param-name>
			<param-value>com.someotherclass.XmlaAuthenticationProvider</param-value>
		</init-param>

		<init-param>
			<param-name>OlapDriverConnectionProperty.ConnectionTimeoutMs</param-name>
			<param-value>@connectionTimeout@</param-value>
		</init-param>
		
		<init-param>
			<param-name>OlapDriverConnectionProperty.TotalCellLimit</param-name>
			<param-value>@totalCellLimit@</param-value>
		</init-param>
		
		<init-param>
			<param-name>OlapDriverConnectionProperty.PerPivotCellLimit</param-name>
			<param-value>@perPivotCellLimit@</param-value>
		</init-param>
    
		<init-param>
			<param-name>OlapDriverConnectionProperty.Source</param-name>
			<param-value>Excel</param-value>
		</init-param>
        
        <init-param>
            <param-name>IdleConnectionsTimeoutMinutes</param-name>
            <param-value>@idleConnectionCleanupTimeout@</param-value>
        </init-param>
        
        <init-param>
            <param-name>EnableConnectionReuse</param-name>
            <param-value>@enableConnectionsReuse@</param-value>
        </init-param>
        
        <init-param>
            <param-name>EnableCubesMetadataPrefetch</param-name>
            <param-value>@enableCubesMetadataPrefetch@</param-value>
        </init-param>
	</servlet>

	<servlet-mapping>
		<servlet-name>xmla</servlet-name>
		<url-pattern>/xmla/*</url-pattern>
	</servlet-mapping>
</web-app>

• Then, I copied over into WebContent Xmla4j (xmla4js) in order to avoid CORS.
• The question is, why so far only asynchronous communication works:

Problems

• Had to insert "[LdCatalog]" instead of @dataSourceInfo@ <= watch out [] was needed (damn naming!)
• Apparently, XMLA does not have "Schema" as an own Had to change "execute-asynchronous.html" and its "executeMDX()" to have properties:

DataSourceInfo: select_datasources.options[select_datasources.selectedIndex].value,
Catalog: select_catalogs.options[select_catalogs.selectedIndex].value

• Had to increase timeout in Xmla.js:

requestTimeout: 300000,

• Apparently, olap4ld has not implemented cell.getPropertyValue, correctly:

The following propertiess are mandatory for cells:

BACK_COLOR
CELL_EVALUATION_LIST
CELL_ORDINAL
FORE_COLOR
FONT_NAME
FONT_SIZE
FONT_FLAGS
FORMAT_STRING
FORMATTED_VALUE
NON_EMPTY_BEHAVIOR
SOLVE_ORDER
VALUE

• I have added formatted_value, format_string and value to CellSet.getCell (from NewValues.map).

CORS

• Make sure that either the xmlaserver is setup for CORS or have both LDCX interface and xmlaserver put on same machine.

Logging

• Possibly, the server cannot create the log file. To help here, we print on console output the global location of the logging file.

Status

2013-07-04

• Example URL: http://localhost:8080/xmlaserver-manually/Xmla4j/samples/execute-asynchronous.html
• Address: http://localhost:8080/xmlaserver-manually/xmla
• adapted defaultXmlaUrl.js (cache reload)
• changed @datasourceUrl@ zu jdbc:ld://olap4ld;Catalog=LdCatalog;JdbcDrivers=com.mysql.jdbc.Driver;Server=http://;Database=EMBEDDEDSESAME;Datastructuredefinitions=;Datasets=;
  • Not needed, sufficient: replace "url: datasource.URL? datasource.URL : url.value,"
• Have Catalog and Schema named same: LdCSXXX.
• Problem: I cannot ask for specific cubes if I do not know them for retrieval.

Query:

SELECT
{[httpXXX3AXXX2FXXX2FlocalhostXXX2Ffios_xmla4jsXXX2FexampleYYYttlXXX23customer_1]} ON COLUMNS,
{[httpXXX3AXXX2FXXX2FlocalhostXXX2Ffios_xmla4jsXXX2FexampleYYYttlXXX23lo_revenue]} ON ROWS
FROM [httpXXX3AXXX2FXXX2FlocalhostXXX2Ffios_xmla4jsXXX2FexampleYYYttlXXX23ds]

Logical OLAP Operator Tree:

Rollup (
	Slice (
		Dice (
			Projection (
				BaseCube ([httpXXX3AXXX2FXXX2FlocalhostXXX2Ffios_xmla4jsXXX2FexampleYYYttlXXX23ds]), 
			[httpXXX3AXXX2FXXX2FlocalhostXXX2Ffios_xmla4jsXXX2FexampleYYYttlXXX23lo_revenue]),
		 ),
	 [httpXXX3AXXX2FXXX2FlocalhostXXX2Ffios_xmla4jsXXX2FexampleYYYttlXXX23lo_custkey], [httpXXX3AXXX2FXXX2FlocalhostXXX2Ffios_xmla4jsXXX2FexampleYYYttlXXX23lo_orderdate], 		[httpXXX3AXXX2FXXX2FlocalhostXXX2Ffios_xmla4jsXXX2FexampleYYYttlXXX23lo_partkey], [httpXXX3AXXX2FXXX2FlocalhostXXX2Ffios_xmla4jsXXX2FexampleYYYttlXXX23lo_suppkey], 		Measures), 
[httpXXX3AXXX2FXXX2FlocalhostXXX2Ffios_xmla4jsXXX2FexampleYYYttlXXX23lo_custkey] : [httpXXX3AXXX2FXXX2FlocalhostXXX2Ffios_xmla4jsXXX2FexampleYYYttlXXX23lo_custkeyCodeList])

• Had to change xmla-server to take into account restrictions of cube so that olap4ld would retrieve cubes.

Problem: 2013-08-01: ld-cubes LDCX does not work anymore

• Problem:
  • If I run it normally, then when stopping at "xmla.executeMultiDimensional({" (ld-cubes-explorer.html, 809), I enable breakpoint at "function initAjaxLoader() { -> events : Xmla.EVENT_GENERAL...". When I go on, I come to Xmla.js.
  • Ajax call
  • Xmla.js 2016: return this.response returns null
  • Actually, ajaxOptions states that when complete: function xmla._requestSuccess(options);
  • Maybe this._fireEvent(Xmla.EVENT_REQUEST) == false OR this._fireEvent(Xmla.EVENT_EXECUTE, options) == false, so that myXhr = ajax(ajaxOptions) is never called? - No.
  • Or maybe: myXhr->complete is never called?

• One option would be to enable synchronous call so that at Xmla.js->_ajax (207), handler.call(xhr) is called.

• Apparently, for asynchronous calls, the return value should be undefined, see Xmla.js 2016.

• Synchronous calls did not work, either, because the entire code of ld-cubes-ldxc were adapted to it.

• => Finally, it was simply because we had not simply reused the way "select empty cells", "render table etc" were used originally