Linked Data Cubes Explorer (LDCX)

From www.b-kaempgen.de
Revision as of 14:38, 17 November 2014 by Benedikt Kämpgen (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

On this page, we present the Linked Data Cubes Explorer (LDCX), an application of olap4ld.

General info

Architecture

LDCX consists of several parts: an HTML webpage using the JavaScript library xmla4js to issue XMLA queries; olap4j-xmlaserver that can answer XMLA queries via olap4ld; olap4ld to answer metadata and olap queries on QB datasets.

Ld-cubes-explorer overall architecture.png Architecture of LDCX.

Three-step user interface

  1. a user selects one or more comma-separated URIs of of qb:DataSets. With ``Explore Dataset..., metadata queries are issued to populate the user interface;
  2. the user selects measures to be displayed in the pivot table cells;
  3. the user selects dimensions to add member combinations to rows and columns of the pivot table and clicks ``Update Table\ldots. Note LDCX automatically queries every dimension on the most granular level since multi-level hierarchies are rarely used and users can still slice dimensions to view datasets on a higher aggregation level.

Linked Data Cubes Engine: EmbeddedSesameEngine

Client Requirements

  • LDCX has been tested on Firefox and Chrome (however, only on a Linux PC).

Server Requirements

Evaluation

Roadmap

How to connect all systems?

  • I used eclipse
  • I manually created a dynamic web project
  • In there, I copied all files from olap4j-xmlaserver master
  • Then I manually added libraries to WEB-INF/lib that xmlaserver relied upon:
commons-collections.jar
commons-dbcp.jar
commons-pool.jar
log4j.jar
nxparser-1.2.3.jar
olap4j-xmlaserver-0.0.1-SNAPSHOT.jar
saxon-9.2.jar
servlet-api.jar
xercesImpl.jar
  • Then I used the web.xml from Rossi:
<?xml version="1.0" encoding="ISO-8859-1"?>
<web-app>
	<display-name>XMLA Server</display-name>
	
	<servlet>
		<servlet-name>xmla</servlet-name>
		<servlet-class>mondrian.xmla.impl.Olap4jXmlaServlet</servlet-class>
		
		<init-param>
			<param-name>OlapDriverUsePreConfiguredDiscoverDatasourcesResponse</param-name>
			<param-value>true</param-value>
		</init-param>

		<init-param>
			<param-name>OlapDriverDiscoverDatasources.dataSourceName</param-name>
			<param-value>@dataSourceName@</param-value>
		</init-param>
		
		<init-param>
			<param-name>OlapDriverDiscoverDatasources.dataSourceInfo</param-name>
			<param-value>@dataSourceInfo@</param-value>
		</init-param>
		<init-param>
			<param-name>OlapDriverDiscoverDatasources.url</param-name>
			<param-value>@datasourceUrl@</param-value>
		</init-param>
		<init-param>
			<param-name>OlapDriverDiscoverDatasources.providerType</param-name>
			<param-value>TDP</param-value>
		</init-param>
		
		<init-param>
			<param-name>OlapDriverDiscoverDatasources.authenticationMode</param-name>
			<param-value>Unauthenticated</param-value>
		</init-param>

		<init-param>
			<param-name>OlapDriverMaxNumConnectionsPerUser</param-name>
			<param-value>15</param-value>
		</init-param>
				
		<init-param>
			<param-name>OlapDriverClassName</param-name>
			<param-value>org.olap4j.driver.ld.LdOlap4jDriver</param-value>
		</init-param>

		<init-param>
			<param-name>OlapDriverConnectionString</param-name>
			<param-value>jdbc:ld://olap4ld;Catalog=LdCatalog;JdbcDrivers=com.mysql.jdbc.Driver;Server=http://public.b-kaempgen.de:8890/sparql;Database=OPENVIRTUOSO;Datastructuredefinitions=http://fios:saiku,http://fios:saikumanually;Datasets=http://fios:saiku,http://fios:saikumanually</param-value>
		</init-param>
		
		<init-param>
			<param-name>OlapDriverConnectionProperty.Prefix</param-name>
			<param-value>@connectionPrefix@</param-value>
		</init-param>

		<init-param>
			<param-name>OlapDriverConnectionProperty.FetchLevelProperties</param-name>
			<param-value>false</param-value>
		</init-param>

		<init-param>
			<param-name>OlapDriverConnectionProperty.FetchMemberProperties</param-name>
			<param-value>false</param-value>
		</init-param>

		<init-param>
			<param-name>OlapDriverConnectionProperty.FetchMembers</param-name>
			<param-value>true</param-value>
		</init-param>

		<init-param>
			<param-name>OlapDriverConnectionProperty.QueryTimeoutMs</param-name>
			<param-value>@queryTimeout@</param-value>
		</init-param>
		
		<init-param>
			<param-name>OlapDriverConnectionProperty.AuthenticationProvider</param-name>
			<param-value>com.someotherclass.XmlaAuthenticationProvider</param-value>
		</init-param>

		<init-param>
			<param-name>OlapDriverConnectionProperty.ConnectionTimeoutMs</param-name>
			<param-value>@connectionTimeout@</param-value>
		</init-param>
		
		<init-param>
			<param-name>OlapDriverConnectionProperty.TotalCellLimit</param-name>
			<param-value>@totalCellLimit@</param-value>
		</init-param>
		
		<init-param>
			<param-name>OlapDriverConnectionProperty.PerPivotCellLimit</param-name>
			<param-value>@perPivotCellLimit@</param-value>
		</init-param>
    
		<init-param>
			<param-name>OlapDriverConnectionProperty.Source</param-name>
			<param-value>Excel</param-value>
		</init-param>
        
        <init-param>
            <param-name>IdleConnectionsTimeoutMinutes</param-name>
            <param-value>@idleConnectionCleanupTimeout@</param-value>
        </init-param>
        
        <init-param>
            <param-name>EnableConnectionReuse</param-name>
            <param-value>@enableConnectionsReuse@</param-value>
        </init-param>
        
        <init-param>
            <param-name>EnableCubesMetadataPrefetch</param-name>
            <param-value>@enableCubesMetadataPrefetch@</param-value>
        </init-param>
	</servlet>

	<servlet-mapping>
		<servlet-name>xmla</servlet-name>
		<url-pattern>/xmla/*</url-pattern>
	</servlet-mapping>
</web-app>
  • Then, I copied over into WebContent Xmla4j (xmla4js) in order to avoid CORS.
  • The question is, why so far only asynchronous communication works:

Problems

  • Had to insert "[LdCatalog]" instead of @dataSourceInfo@ <= watch out [] was needed (damn naming!)
  • Apparently, XMLA does not have "Schema" as an own Had to change "execute-asynchronous.html" and its "executeMDX()" to have properties:
DataSourceInfo: select_datasources.options[select_datasources.selectedIndex].value,
Catalog: select_catalogs.options[select_catalogs.selectedIndex].value
  • Had to increase timeout in Xmla.js:
requestTimeout: 300000,
  • Apparently, olap4ld has not implemented cell.getPropertyValue, correctly:
The following propertiess are mandatory for cells:

BACK_COLOR
CELL_EVALUATION_LIST
CELL_ORDINAL
FORE_COLOR
FONT_NAME
FONT_SIZE
FONT_FLAGS
FORMAT_STRING
FORMATTED_VALUE
NON_EMPTY_BEHAVIOR
SOLVE_ORDER
VALUE
  • I have added formatted_value, format_string and value to CellSet.getCell (from NewValues.map).

CORS

  • Make sure that either the xmlaserver is setup for CORS or have both LDCX interface and xmlaserver put on same machine.

Logging

  • Possibly, the server cannot create the log file. To help here, we print on console output the global location of the logging file.

Status

2013-07-04

Query:

SELECT
{[httpXXX3AXXX2FXXX2FlocalhostXXX2Ffios_xmla4jsXXX2FexampleYYYttlXXX23customer_1]} ON COLUMNS,
{[httpXXX3AXXX2FXXX2FlocalhostXXX2Ffios_xmla4jsXXX2FexampleYYYttlXXX23lo_revenue]} ON ROWS
FROM [httpXXX3AXXX2FXXX2FlocalhostXXX2Ffios_xmla4jsXXX2FexampleYYYttlXXX23ds]

Logical OLAP Operator Tree:

Rollup (
	Slice (
		Dice (
			Projection (
				BaseCube ([httpXXX3AXXX2FXXX2FlocalhostXXX2Ffios_xmla4jsXXX2FexampleYYYttlXXX23ds]), 
			[httpXXX3AXXX2FXXX2FlocalhostXXX2Ffios_xmla4jsXXX2FexampleYYYttlXXX23lo_revenue]),
		 ),
	 [httpXXX3AXXX2FXXX2FlocalhostXXX2Ffios_xmla4jsXXX2FexampleYYYttlXXX23lo_custkey], [httpXXX3AXXX2FXXX2FlocalhostXXX2Ffios_xmla4jsXXX2FexampleYYYttlXXX23lo_orderdate], 		[httpXXX3AXXX2FXXX2FlocalhostXXX2Ffios_xmla4jsXXX2FexampleYYYttlXXX23lo_partkey], [httpXXX3AXXX2FXXX2FlocalhostXXX2Ffios_xmla4jsXXX2FexampleYYYttlXXX23lo_suppkey], 		Measures), 
[httpXXX3AXXX2FXXX2FlocalhostXXX2Ffios_xmla4jsXXX2FexampleYYYttlXXX23lo_custkey] : [httpXXX3AXXX2FXXX2FlocalhostXXX2Ffios_xmla4jsXXX2FexampleYYYttlXXX23lo_custkeyCodeList])
  • Had to change xmla-server to take into account restrictions of cube so that olap4ld would retrieve cubes.


Problem: 2013-08-01: ld-cubes LDCX does not work anymore

  • Problem:
    • If I run it normally, then when stopping at "xmla.executeMultiDimensional({" (ld-cubes-explorer.html, 809), I enable breakpoint at "function initAjaxLoader() { -> events : Xmla.EVENT_GENERAL...". When I go on, I come to Xmla.js.
    • Ajax call
    • Xmla.js 2016: return this.response returns null
    • Actually, ajaxOptions states that when complete: function xmla._requestSuccess(options);
    • Maybe this._fireEvent(Xmla.EVENT_REQUEST) == false OR this._fireEvent(Xmla.EVENT_EXECUTE, options) == false, so that myXhr = ajax(ajaxOptions) is never called? - No.
    • Or maybe: myXhr->complete is never called?
  • One option would be to enable synchronous call so that at Xmla.js->_ajax (207), handler.call(xhr) is called.
  • Apparently, for asynchronous calls, the return value should be undefined, see Xmla.js 2016.
  • Synchronous calls did not work, either, because the entire code of ld-cubes-ldxc were adapted to it.
  • => Finally, it was simply because we had not simply reused the way "select empty cells", "render table etc" were used originally