1. University of Toronto
School of Continuing Studies
SCS 2942 – 002
Foundations of Enterprise Data Analytics
- Concepts and Controls
Instructors
Larry Simon, MBA, CMC Brad Brown, MBA, MSc.
larry.simon@utoronto.ca bradbbrown@gmail.com
Big Data, Investment Management
and Wavelets
Prepared by:
Serene Zawaydeh, MBA, B.Sc. EE
serene.zawaydeh@gmail.com
March 24, 2014

2. Serene Zawaydeh
Big Data, Investment Management and Wavelets March 2014
Page 2 of 24
Table of Contents
Big Data in Investment Management...........................................................................................................3
“Acute Big Data Challenges Facing Asset Managers”...................................................................................4
Capital Market Firms using Unstructured Data ............................................................................................4
Big Data Applications in Investment Industry and Big Data Technologies used by Capital Market Firms...5
Examples of Big Data Systems in Investment Management.........................................................................6
Visual Representation of Stock Market Co-Movement using Wavelets.......................................................8
Building Wavelet Histograms on Large Data in MapReduce ........................................................................9
Data Mining.................................................................................................................................................10
K-Nearest Neighbor – Machine Learning....................................................................................................10
Visualization of Unstructured Text .............................................................................................................11
Sentiment Analysis......................................................................................................................................11
Data Mining in Investment Management...................................................................................................12
Theoretical Big Data Model ........................................................................................................................13
Implementation of Big Data Projects..........................................................................................................15
Wavelets and Non Stationary Signals .........................................................................................................16
19 Level Filter Bank System ........................................................................................................................17
Bibliography ................................................................................................................................................20
Table of Figures
Figure 1: Big Data Applications in Investment Industry................................................................................5
Figure 2: Big Data Technologies used by Capital Market Firms....................................................................5
Figure 3: Comovement between stock returns ............................................................................................9
Figure 4: MapReduce....................................................................................................................................9
Figure 5: Time Series Classification.............................................................................................................10
Figure 6: Theoretical Big Data Analytics Model incorporating Applications of Wavelets ..........................14
Figure 7: Changing value of Beta by changing the period around Beta is calculated.................................17
Figure 8: 19 Level Filter Bank System for Analyzing ECG Signals using Wavelet Packets...........................18
Figure 9: Spectrum of the Successive Filters using Daubechies 4...............................................................19
Figure 10: Wave Packet Tiling of the Time-Frequency Plane
Figure 11: Scaling Function (Low Pass Filter) and Wavelet Mother (Band Pass Filter)...............................19

3. Serene Zawaydeh
Big Data, Investment Management and Wavelets March 2014
Page 3 of 24
Big Data in Investment Management
Volume, velocity, and variety characterize big data in general, and these characteristics apply to data in
the investment industry. The fourth V according to McKinsey is Value: Big data has high commercial
value and will be a major source of competitive advantage for firms, allowing them to better understand
their customers and their own business. (Manyika & al, 2011)
Companies are heavily investing in big data. 15% of 1,217 surveyed companies by TCS (TCS) spent at
least $100 million each on Big Data initiatives in 2012, and 7% invested at least $500 million. 643
companies undertook Big Data initiatives in 2012. 24% spent less than $2.5 million each. Industries
spending the most are telecommunications, travel-related, high tech, and banking. The value of
implementing big data systems was proven in the healthcare industry, which has seen a 20% decrease in
patient mortality by analyzing streaming patient data, while the Telco industry has seen a 92% increase
in processing time by analyzing network and call data.
Diversifying investment portfolio across markets increases the amount of data that needs to be
processed when making investment decisions. Global markets are interdependent. Political changes in
one country affect neighboring markets, and a financial crisis starting in one economy can have a global
impact. Understanding interdependence between stock markets provides valuable insights to
diversification strategies. Negative news on one company can be positively reflected on a competitor’s
stock price. Organizational changes, layoffs, earnings and dividend announcements, bankruptcies, as
well as currency changes and inflation (GetSmarterAboutMoney.ca) affect stock markets. With the
efficient market hypothesis and according to the random walk, stocks’ past performance is not an
indicator of the future performance. Historical data, publicly available data, and confidential data are all
integrated into the stock prices. Emerging events affect stock markets, and therefore a system that
integrates real time market changes, extending to tweets about stocks and companies, would be of
value.
Big data solutions are being implemented in the investment industry, allowing processing of a large
volume of variables including real time changes. Traditional technical analysis that looks into the stock
price movement does not look at the company’s financial statements. Meanwhile, valuation models
such as the discounted cash flow models, including fundamental analysis of companies’ historical
financial statements and projecting future cash flows, do not have room for real time changes that
affect the market. Big data visualizations and integration of structured and unstructured data from
different sources, and the ability to integrate real time changes into the system make it a suitable
solution for investment management.
In addition to highlighting current applications of big data in the investment industry, this paper
identified applications of Wavelets in finance and Big Data. Academic studies proved the benefits of
using Wavelets for forecasting stock prices and data mining among other applications.

4. Serene Zawaydeh
Big Data, Investment Management and Wavelets March 2014
Page 4 of 24
“Acute Big Data Challenges Facing Asset Managers”
A survey of 400 asset managers and owners conducted by Economic Intelligence Unit on behalf of State
Street (Chris, 2013) found out that Data accuracy; lack of data integration facilities; high pricing; and
lack of timeliness in external data are key challenges.
• Nine out of 10 institutional investors view data and analytics as a key strategic priority.
• Investments in data technologies and platforms increased, yet less than a third currently gain
any competitive advantage from their data and analytics capabilities.
• The ability to aggregate, analyze and transform data is key to institutional investors’ ability to
compete.
Investment in Big Data is rising, and targeted at tools to support decision making in the front office, and
solutions to manage risk and regulatory compliance more efficiently. Data and analytics are a primary
strategic priority, and are a source of competitive advantage.
Data leaders need to improve risk tools with multi-asset class capabilities; Develop better tools to
manage regulation in multiple jurisdictions; improve the ability to manage and extract insight from
multiple data sources; and optimize electronic trading platforms developing a scalable data architecture
that will grow with the business. (State Street, 2013)
Capital Market Firms using Unstructured Data
Risk analytics, regulation, and trading analytics are three of the five areas identified by Infosys in which
Capital Market firms are using big unstructured data and the State Street report (State Street, 2013).
The two other areas are financial data management and reference data management and data tagging.
•••• Financial data management and reference data mangement: Data storage for historical trading,
internal data management challenge, and overall control on reference data (on-demand data mining
to dig into meta-data to deconstruct/ reconstruct data models, etc.). It can be very tough in
maintaining (storing, handling, and processing) data from various asset classes coming from various
vendors.
•••• Regulation: Includes preparation for regulations like Dodd Frank, Solvenc II, EMIR, audits etc.
•••• Risk analytics: Includes fraud minitgation, anti-money laundering (AML), Know Your Customer (KYC),
rogue trading, on-demand enterprise risk management, etc.
•••• Trading analytics: Includes Analytics for High Frequency Trading, Predictive Analytics, Pre-trade
decision-support analytics, including sentiment measurement and temporal / bi-temporal analytics
etc.
•••• Data tagging: In enterprise-level monitoring and report, it is often hard to match and reconcile
trades from various systems built on different symbology standards – usually resulting in invalid,
duplicated and missed trades. Data tagging can easily identify trades and events such as corporate
actions and enable regulators to detect stress signs early.

5. Serene Zawaydeh
Big Data, Investment Management and Wavelets March 2014
Page 5 of 24
Big Data Applications in Investment Industry and Big Data Technologies used by Capital Market Firms
Figure 1: Big Data Applications in Investment Industry
Investment Bank Big Data Use Cases Areas
Investment Bank
An investment firm with assets of over $1 trillion and operations in approximately 50
countries uses big data technology to deliver Reference Data to the Murex trading
platform and other downstream operations
Reference Data
Management
Investment Bank
An investment firm with assets of over $1 trillion and operations in approximately 50
countries uses big data to manage risk exposure through real-time communication
across bond, futures and credits trading. Risk Analytics
US Investment
Bank
An investment bank shifted to the risk management and P&L towards a real-time
environment. Big Data technologies were leveraged to help the firm to gather all
relevant data into one place. Regulation
European
Investment Bank
An investment bank used Big Data analytics to track performance monitoring, risk
analysis and reporting
Risk Analytics
and Regulation
Asian Investment
Bank
An investment bank used Big Data technologies to generate on-demand performance
metrics for risk measures across multiple global trading businesses.
Trading
Analytics
European
Investment Bank
An investment manager used Big Data technologies to gather relevant details so as to
respond as a witness to litigation action against a prime broker Compliance
US Investment
Manager
Investment manager used Big Data technology to centralize data and applications to
apply governance policies and mitigate risk of damages from litigation discovery
Risk Analytics
and Regulation
Global Exchange
A major global exchange used Big Data technology to provide global market
participants with on-demand access to data and data-mining tools for trading,
analytics and risk management in a cloud-based / hosted environment
Trading
Analytics & Risk
Analytics
US Regulator
A US regulator used Big Data technology to create a searchable library of research,
econometric and other information generated by the regulator's activities. Regulation
Buy Side firm
A major buy-side firm uses Big Data technologies for market surveillance, an activity
requiring processing of vast quantities of market information Regulation
Asset Manager
Fiduciary management - a new area of interest in which asset managers outsource
management of their portfolios to third-party administrators in order to benefit from
economies of scale
Fiduciary
Management-
Emerging Area
Regulatory
compliance and
advanced
analytics
An investment bank uses big-data techniques to handle and manage petabytes of
regulatory compliance and advanced analytics. The bank used technology from
Hadoop, open source framework that supports data-intensive distributed computing,
which allows data to be crunched over a distributed network of computers
Regulation &
Risk Analytics
http://www.infosys.com/industries/financial-services/white-papers/Documents/big-data-analytics.pdf
Figure 2: Big Data Technologies used by Capital Market Firms
Data Grids Use distributed catching to manage large volumes ofdata across a network ofservers.
Compute grids Offer a way of parellizing processes across multiplesservers, handling capacity / failure issues
and orchestrating tasks across the grid
Massively parallel processors Involves the coordinated processing of a programme between multiple independent
computers, each with its own operating system and memory
In-memory databases Databases that store data in the main memory rather than a disk, as is the case with traditional
databases
NoSQL Shell relational database management systems that don’t use Structured Query Language, are
more simple than traditional databases and whose tables are compatible with a wide range of
external platforms
Specialized databases Contain the necessary architecture to store the unstructured data. Ex: IBM Viper DB2 database,
EMC Greenplum, Greeplum
Hadoop A tool used to query the unstructured data which is a major part of big data analytics. Ex. EMC
Map Reduce, IBM Netezza
http://www.infosys.com/industries/financial-services/white-papers/Documents/big-data-analytics.pdf

6. Serene Zawaydeh
Big Data, Investment Management and Wavelets March 2014
Page 6 of 24
Examples of Big Data Systems in Investment Management
Regulatory Compliance Systems
Regulations in the investment industry require safeguarding insider information, and ensuring that no
improper trading occurs. Privacy of investors and security of data needs to be maintained in traditional
trading and investment environment and also in Big Data.
Merwin et al’s Patent Application US 20130290218 A1 introduces a system and method for regulatory
compliance management (Merwin, Higgins, Hardy, & Marks, 2013). For an investment portfolio, the
method tags and searches regulatory and other documents using a query module. It includes receiving
the documents by an analysis module. The system determines trading errors; transactions in different
accounts in the investment portfolio; long and short positions occurring within a same security in the
investment portfolio; daily trading volume for a security in the investment portfolio; value of
transactions; and commissions.
AIMCO www.aimco.alberta.ca
Alberta Investment Management Corporation (AIMCo), established in 2008, manages approximately
C$70 billion on behalf of 60 pension, endowment, government reserve clients in the Alberta. There was
a need for clean and timely data not just through spreadsheets, a practice that is not uncommon even
for investment organization managing billions of dollars.
AIMCO introduced a new architecture, with a centralized data warehouse where information can be
shared across the firm. This reduced the redundant systems, reduced reconciliation efforts required to
keep all information in synch. It aimed at storing more detailed, granular data that would support in-
depth queries in real time, allowing investment professionals to select various views of portfolios to gain
unique insights. The data infrastructure supports strong internal audit and compliance processes for
data, while allowing AIMCo to be competitive through enriched analytics that cannot be purchased from
outside vendors.
State Street Global Exchange www.statestreetglobalexchange.com
State Street Global Exchange established a big data division devoted to portfolio modelling, investment
analytics, data management and data projections. The division aims at enabling institutional investors
to better analyse client data to identify risks and monitor the efficiency of portfolios. Insurance
companies are also working on projects to better understand their customers’ profiles. (McGrath, 2014).
Portfolio Analytics
StatPro North America www.statepro.com is a global provider of portfolio analytics for investment.
StatPro Revolution a sophisticated portfolio analysis platform based in the cloud. It provides instant
access to information on portfolio performance, risk, attribution and allocation analysis. Portfolio
analysis is shared with the clients. What If analysis allows viewing the effects of investment decisions.
(StatPro, 2012).

7. Serene Zawaydeh
Big Data, Investment Management and Wavelets March 2014
Page 7 of 24
Angoss http://www.angoss.com/
Angoss provides a predictive analytics solution which is being implemented in the financial industry,
insurance, and mutual funds. It maintains data integrity, security. Angoss business intelligence software
and predictive analytics solutions provide more than 300 companies worldwide. Its clients include
Microsoft, Bank of America, and mutual funds such as Vanguard, Fidelity, Dynamic Funds.
Apama: Real Time Analytics
The Progress Apama platform implements a patented CEP architecture that can monitor market data
(both market feeds and related information, like electronic news). The sub-millisecond responsiveness
of Apama is sustainable when market data volumes reach the tens of thousands of events per second
and when concurrent strategies number in the thousands.
Apama strategies execute within the Apama Correlator, which employs a patented, multi-dimensional
filtering architecture to detect patterns and identify appropriate actions, in under a millisecond. The
Apama Correlator offers tremendous scalability with flexible configuration options that support various
designs for load balancing and fault tolerance. Unlike single-purposed CEP engines, the Apama
Correlator can support thousands of discrete strategies executing simultaneously with no performance
degradation. Additionally, the Apama high-availability (HA) architecture incorporates a “cluster” model
that enables recovery of failed nodes such that they can re-synchronize with redundant nodes and
commence operation with no impact on performance.
Wavelets in Finance and Big Data
Multidimentional filter banks and Discrete Wavelet Analysis were the topic of my digital signal
processing research. Hardoon et al used a tree structure for predictions (Hardoon & Shmueli, 2013).
Associating that to Wavelets which also have a tree structure, I investigated whether there are other
applications for Wavelets in Big Data concepts learned throughout the course.
Wavelets are suitable for the analysis of stock data and time series that have a non stationary
distribution. Several applications were identified for Wavelets in finance in addition to different
components of the Big Data system. Scientific papers on using Wavelets to calculate beta, forecast time
series, Map Reduce, conduct sentiment analysis, text visualization, financial data mining, studying
comovement of stock markets, and clustering techniques. Since several of these topics were discussed
during the course, following is a review of identified research papers.
Ramsey reviewed the contribution of Wavelets to the analysis of economic and financial data (Ramsey,
1996). The author provided suggestions about improving understanding and evaluation of forecasts
using a wavelet approach.
Wavelets were among the discussion points listed in Proceedings of the Fourth World Congress on
Engineering Asset Management (WCEAM) 2009. (Kiritsis, Emmanouilidis, Koronios, & Mathew, 2010).
Sun et al introduced “A new wavelet-based denoising algorithm for high-frequency financial data

8. Serene Zawaydeh
Big Data, Investment Management and Wavelets March 2014
Page 8 of 24
mining” (Sun & Meinl, 2012). Meanwhile, Giovanis used MATLAB applications for Trading Rules using
Wavelets (Giovanis, 2009).
While investment managers diversify asset allocation across geographies and industries, the global
financial crisis in 2008 started in USA, and its effect spread across stock markets. Interdependence and
correlation between global markets was researched. Sahu et al studied the co-integration of stock
markets using Wavelets and Data mining (R.Sahu). The authors state that “real data for the stock
indices, using tick-by-tick observations obtained, are no longer accepted to be stationary.” Leonel et al
also used Wavelets, and observed the behavior of stock market with its links and correlations using
network and graph theory (Leonel & Yoneyama). Zhao et al developed simulation of a Wavelet neural
network that can forecast stock market returns (Zhao, Zhang, & Qi, 2008). Wavelets were also used to
predict oil prices (Shahriar Yousefi, 2004).
A Wavelet-Based beta estimation of China Stock market was used by Xiong et al (Xiong, Zhang, Zhang, &
Li, 2005) and was linked to behavioral finance. Empirical results showed that the predictions of the
CAPM model are more relevant at short time horizons as compared to long.
Visual Representation of Stock Market Co-Movement using Wavelets
Having a visual representation of the interdependence between the stock markets is useful when
determining the diversification strategy. Basdas investigated the integration of emerging stock markets
over different time horizons using daily data over 1992-2011 (Basdas, 2012). The links among major
Middle East and North African (MENA) stock exchange markets were considered by adopting wavelet
comovement analysis (Rua, 2010). The results indicated that MENA stock markets are partially
integrated and the degree of interdependence increased significantly after 2008 Crisis.

9. Serene Zawaydeh
Big Data, Investment Management and Wavelets March 2014
Page 9 of 24
Figure 3: Comovement between stock returns
Source: (Basdas, Interaction between MENA Stock Markets: A Comovement Wavelet Analysis, 2012)
Building Wavelet Histograms on Large Data in MapReduce
MapReduce is becoming the de facto framework for storing and processing massive data, due to its
excellent scalability, reliability, and elasticity. In many MapReduce applications, obtaining a compact
accurate summary of data is essential. Among various data summarization tools, histograms have
proven to be particularly important and useful for summarizing data, and the wavelet histogram is one
of the most widely used histograms. Researchers investigated the problem of building wavelet
histograms efficiently on large datasets in MapReduce.
Figure 4: MapReduce
Two-level sampling at mapper Two-level sampling at reducer
A wavelet-based measure
through a contour plot,
representing comovement
across 7 stock exchanges.
The horizontal axis refers
to time (in years) while the
vertical axis refers to
frequency (time units, in
years). In the plots,
deepening red color
corresponds to an
increasing value of
interdependence and
mimics the height in a
surface plot. Changes in
comovement can be
followed over time and
different frequencies.
Red
Red
Red
Red
Red Red

10. Serene Zawaydeh
Big Data, Investment Management and Wavelets March 2014
Page 10 of 24
Time Series Classification
Time series data are widely seen in analytics (Zhao Y. , 2011), including stock indexes/ prices. However,
classification and clustering of time series data are not readily supported by existing R functions or
packages. Time series classification builds a classification model based on labelled time series and then
uses the model to predict the label of unlabelled time series. The way for time series classification with R
is to extract and build features from time series data first, and then apply existing classification
techniques, such as SVM, k-NN, neural networks, regression and decision trees, to the feature set.
Discrete Wavelet Transform (DWT) provides a multi-resolution representation using wavelets. The
author provided the following example for time series classification using Wavelets.
Figure 5: Time Series Classification
http://i0.wp.com/rdatamining.files.wordpress.com/2011/08/image0131.png
Clustering
Ray and Mallick (Ray & Mallick, 2004) proposed a nonparametric Bayes wavelet model for clustering of
functional data. The wavelet-based methodology is aimed at the resolution of generic global and local
features during clustering and is suitable for clustering high dimensional data.
Data Mining
Pentaho Data Mining Tools and Techniques provide Discrete Wavelet Transform among unsupervised
attribute-based learning techniques used in Weka.
K-Nearest Neighbor – Machine Learning
Knn is a non parametric method used for classification and regression. K-NN is a type of instance-based
learning, or lazy learning, where the function is only approximated locally and all computation is
deferred until classification. The K-NN algorithm is the simplest form of machine learning algorithm.

11. Serene Zawaydeh
Big Data, Investment Management and Wavelets March 2014
Page 11 of 24
(Qiao, Lu, & Sun, 2006) reviewed two fast algorithms based on wavelet transform. References search the
k closest vectors in the wavelet domain. The motivation is that the time for performing the wavelet
transform is low and the energy of the vector is compacted on a few coefficients. The algorithm speeds
up searching k nearest neighbors, which is confirmed with the experimental results.
Knowledge Discovery in Financial Investment
Li & Kuo (Li & Kuo, 2008) used wavelet-based Self Organizing Map (SOM) networks for knowledge
discovery in financial investment for forecasting and trading strategy. The authors proposed a hybrid
approach on the basis of the knowledge discovery methodology by integrating K-chart technical analysis
for feature representation of stock price movements, discrete wavelet transform for feature extraction
to overcome the multi-resolution obstacle, and a novel two-level self-organizing map network for the
underlying forecasting model. A visual trajectory analysis was conducted to reveal the relationship of
movements between primary bull and bear markets and help determine appropriate trading strategies
for short-term investors and trend followers. The resultant intelligent investment model can help
investors, fund managers and investment decision-makers of national stabilization funds make
profitable decisions.
Visualization of Unstructured Text
Miller et al (Miller, Wong, Brewster, & Foote) developed TOPIC-O-GRAPHYTM
Technology to provide a
visualization of unstructured text using Wavelets. The patent was granted in 2000.
Patent US 6070133 (Brewster & Miller, 2000) provides an information retrieval system using Wavelet
Transform. The method is for automatically partitioning an unstructured electronically formatted
natural language document into its sub-topic structure. The document is converted to an electronic
signal and a wavelet transform is then performed on the signal. The resultant signal may then be used to
graphically display and interact with the sub-topic structure of the document.
Sentiment Analysis
Patent Application number WO 2011123378 A1 uses Wavelets for sentiment analysis (O'neil, 2011). A
document can be processed to provide sentiment values for phrases in the document. The sequence of
sentiment values associated with the sequence of phrases in a document can be handled as if they were
a sampled discrete time signal. For phrases which have been identified as entities, a filtering operation
can be applied to the sequence of sentiment values around each entity to determine a sentiment value
for the entity.

12. Serene Zawaydeh
Big Data, Investment Management and Wavelets March 2014
Page 12 of 24
Data Mining in Investment Management
Data Mining – Customer Behavior
Investors’ behaviour and investment strategies can be “learned”, to propose which stocks, and sectors
would be suitable to invest in. In fact, Mak et al developed an intelligent financial data mining model to
extract customer behavior in the financial industry, to increase customer satisfaction. The model
investigated customization of investment portfolio to the customers using clusters. The financial model
clustered the customers into several sectors, then found the correlation between the sectors. This
improves the workflow of a financial company, and deepens the understanding of investment
behaviour. The company can customize the most suitable products and services for customers on the
basis of the rules extracted. (Mak, Ho, & Ting, 2011)
Data Mining and Algorithmic Asset Management
Giovanni and Parrella used data mining for algorithmic asset management using an ensemble learning
approach. Algorithmic asset management refers to the use of expert systems that enter trading orders
without any intervention. Market-neutral systems aim at generating positive returns regardless of
underlying market conditions. The algorithm developed learns the fair price of the security under
management, using the most recent market information acquired by means of streaming financial data.
The difficult issue of learning in non-stationary environments was addressed by adopting an ensemble
learning strategy, where a meta-algorithm strategically combines the opinion of a pool of experts.
Experimental results based on nearly seven years of historical data for the iShare S&P500 ETF
demonstrate that satisfactory risk-adjusted returns can be achieved by the temporal data mining system
after transaction costs. (Giovanni & Parrella)

13. Serene Zawaydeh
Big Data, Investment Management and Wavelets March 2014
Page 13 of 24
Theoretical Big Data Model
Big data solutions would provide insights to investment managers to quickly make decisions through an
easy to navigate dashboard and data visualization tools can be acceptable by the finance community.
The suggested model would be one that makes use of different applications of Wavelets in finance and
big data: Forecasting time series; Beta estimation; studying interdependence between stock markets to
help with diversification strategies; MapReduce algorithms with Wavelet histograms; data mining with
multi dimensional filtering method; and K-nearest neighbors Wavelet algorithm that speeds up the
search for K nearest neighbor; trading rules; unstructured text visualization; sentiment analysis.
Other components of the system would include: Twitter based trading strategies; and real time
Sentiment Analysis. Stock Sonar provides sentiment analysis for the stock market.
Data visualization technology providers: Some of the companies that provide data visualization tools
are: Streambase, SAS, IntraLinks, Panopticon Software, and Aqumin. (Rodier, 2012)
Compliance with the rules and regulations can be implemented using the patent pending compliance
system US 20130290218 A1. Data about the investors needs to be protected, and security of
transactions needs to be maintained.

14. Serene Zawaydeh
Big Data, Investment Management and Wavelets March 2014
Page 14 of 24
Figure 6: Theoretical Big Data Analytics Model incorporating Applications of Wavelets
From Data to Dash
Time series forecasting
(Stock Prices, currencies,
economy)
Wavelets
Sentiment Analysis
Real time market data on individual
companies, local and regional stock
markets, local and international
economy, political and organizational
changes
Wavelets
HADOOP
Function of All
the Data
MapReduce
processor
Wavelet
Historgram
Wavelets to calculate Beta
Real time decision on suitability
of investment
Equip portfolio managers with
tools to help in the decision of
assets to buy or sell to which
investor based on investment
strategy
Correlation
Interdependence between
Stock Markets
Wavelets
Data Mining
Algorithmic asset management
Customize portfolio to clients
Data Visualization
Visualization of Unstructured text
Wavelets (Patented)
Trading Rules – Wavelets
Twitter –based trading strategy
Knowledge Discovery –
Wavelet SOM network
Regulatory Compliance
system (Patent Application)

15. Serene Zawaydeh
Big Data, Investment Management and Wavelets March 2014
Page 15 of 24
Implementation of Big Data Projects
Numerous scientific papers provide results that can be of added value in the investment decision making
process, and can result in higher returns. Turning scientific research into commercialized products that
enable portfolio managers to make decisions based on real time data is needed. Wavelets can help with
forecasting stock prices, and have applications in different stages of the Big Data system, such as
MapReduce, financial data dining to increase customer satisfaction, sentiment analysis, unstructured
text visualization, and trading strategies, while the comovement of stock markets can be used in
diversification strategies.
The implementation of a big data system that incorporates findings from scientific research produced by
researchers across the globe could be the challenge. There is a need for collaborative research. As an
example, while one part of the big data system needs digital signal processing researchers, the final
product needs to provide visualization tools, which might need input from graphic designers. Such a
product will be used by portfolio managers, who come from a finance background. It needs to be on a
secure IT infrastructure that protects investors’ privacy, and therefore needs IT support. It needs to
adhere to the regulatory requirements in the investment industry, and therefore the product need to be
built on a legal framework. Developing a comprehensive system would require bridging the divide
between different domains. It can be costly, as some intellectual property could be patented or could be
patent pending. Developing a big data solution in house would be a challenge.
Statpro North America, a provider of Portfolio Analytics for the Investment community, emphasizes the
need for alliances in big data. Alliances and data aggregators are enablers for “Turning Big Data into a
Dashboard for Investment Managers”. Such alliances would enable portfolio managers and their clients
to get a complete view of a portfolio’s performance, and have total confidence in the completeness of
the information. (Peddar, 2012)
Zitek wonders how long it will be before securities analysts become data scientists (Zitek, 2014), and
describes primary fundamental research as fading away. Data discovery and visualization tools will
replace spreadsheets for identifying dependencies, patterns and trends, valuation analysis, and
investment decision making.

16. Serene Zawaydeh
Big Data, Investment Management and Wavelets March 2014
Page 16 of 24
Wavelets and Non Stationary Signals
I had used Wavelets for the analysis of biomedical non stationary signals in my digital signal processing –
Electrical Engineering thesis in 1997. I used Discrete Wavelet Analysis for the analysis of non stationary
biomedical ECG (heart beats) signals and heart sounds (PCG). (Zawaydeh, Discrete Wavelet Analysis and
Applications to ECG and PCG Signals, 1997). There is a clear distinction between stationary and non
stationary signals which can be quickly captured when looking at a distribution in the time domain.
Stationary signals have one frequency, and look like sine waves. Meanwhile, non stationary signals, have
different ups and downs, and consequently have more than one frequency.
In 2008, I used Event Study methodology in my MBA thesis to investigate value creation for acquisitions
and divestiture operations for two telecom operators with an internationalization strategy (Zawaydeh,
Etude d’événements sur des operations financières et d'acquisitions et application aux cas de deux
opérateurs de telecom Arabes, 2008). This followed four years of professional research experience on
telecom markets in the Middle East and North Africa (between 2003 and 2004).
I investigated abnormal stock return. Positive abnormal return was an indicator of value creation, and
negative abnormal return indicated destruction of value. At least 40% of the acquisitions created value.
I used the Capital Asset Pricing Model to calculate Beta, the systemic risk coefficient. Beta is calculated
using the Covariance of the stock return, and the market return, divided by the Variance of the market
return.
Beta is used in portfolio management to determine the risk of the asset, and whether it has higher or
lower risk than the market. Negative Beta means lower risk than the market, and Beta higher than 1 is
higher risk than the market. Beta is one of the indicators used when providing information on equities
(stocks).
Changing the period around which Beta is calculated, changes the value of Beta. The theory of the
Capital Asset Pricing Model showed that it is based on the hypothesis of “stationary” returns. (Fama,
1976). Based on my previous research on the applications of Wavelets for the analysis of non stationary
signals, I concluded that the Wavelet Transform could be used to analyze stock data, which has non
uniform ups and downs. I was able to find papers on the applications of Wavelets in Finance.
Los et al used Wavelets in a “Multi-Fractal Spectral Analysis of the 1987 Stock Market Crash”. The
authors state that the most striking result, was that the multifractal spectra of stock market returns are
not stationary. (Los & Yamalova, 2004)

17. Serene Zawaydeh
Big Data, Investment Management and Wavelets March 2014
Page 17 of 24
Ramazan Gencay (Ramazan Gençay, 2001) authored a book on using Filter Banks in Finance. NAG
highlights the applications of Wavelets in Finance (Tong), and the increasing demand for wavelet
analysis. Gençay et al proposed an alternative multiscale estimator for the systemic risk or beta of an
asset using Wavelets. (Gençay, Selçuk, & Whitcher, 2004)
Figure 7: Changing value of Beta by changing the period around Beta is calculated
Source: (Zawaydeh, Etude d’événements sur des operations financières et d'acquisitions et application
aux cas de deux opérateurs de telecom Arabes, 2008).
19 Level Filter Bank System
Using MATLAB, I designed the following 19 level Filter Bank system. The incoming signal was analyzed
into low frequencies, and high frequencies. The low frequencies were subsequently divided again into
low frequencies and high frequencies, and so on.
The Wavelet Transform can be seen as zooming into the signal to see the details that cannot be
captured by looking at the original non stationary signal in the time domain.

18. Serene Zawaydeh
Big Data, Investment Management and Wavelets March 2014
Page 18 of 24
Figure 8: 19 Level Filter Bank System for Analyzing ECG Signals using Wavelet Packets
Source: “Discrete Wavelet Analysis and Applications to ECG and PCG Signals”, Serene Zawaydeh, 1997
Input
Signal

19. Serene Zawaydeh
Big Data, Investment Management and Wavelets March 2014
Page 19 of 24
Figure 9: Spectrum of the Successive Filters using Daubechies 4
Source: “Discrete Wavelet Analysis and Applications to ECG and PCG Signals”, Serene Zawaydeh, 1997
Figure 10: Wave Packet Tiling of the Time-Frequency Plane Figure 11: Scaling Function (Low Pass Filter) and Wavelet
Mother (Band Pass Filter)
Source: “Discrete Wavelet Analysis and Applications to ECG and PCG Signals”, Serene Zawaydeh, 1997
Gain

20. Serene Zawaydeh
Big Data, Investment Management and Wavelets March 2014
Page 20 of 24
Bibliography
Basdas, U. (2012, August 1). Interaction between MENA Stock Markets: A Comovement Wavelet
Analysis. Retrieved from http://ssrn.com/abstract=2333774
Basdas, U. (2012, August 1). Interaction between MENA Stock Markets: A Comovement Wavelet
Analysis. Retrieved from SSRN: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2333774
Brewster, M., & Miller, N. (2000, May 30). Patent No. 6070133. US.
Buffett , M., & Clark , D. (2002). The New Buffettology: The Proven Techniques for Investing Successfully
in Changing Markets That Have Made Warren Buffett the World's Most Famous Investor. New
York: Rawson Associates.
Cavoukian, A., & Jonas, J. (2012, June 8). Privacy by Design in the Age of Big Data. Retrieved February 4,
2014, from Privacy by Design: http://privacybydesign.ca/content/uploads/2012/06/pbd-
big_data.pdf
Chris, F. (2013, November 13). Acute ‘big data’ challenges facing asset managers. Retrieved March 1,
2014, from http://www.ft.com/cms/s/0/5c6a8e34-4880-11e3-8237-
00144feabdc0.html#ixzz2ulV04ZFG
Daruvala, T. (2013, April). How advanced analytics are redefining banking. Retrieved February 21, 2014,
from McKinsey & Company - Insights and Publications:
http://www.mckinsey.com/insights/business_technology/how_advanced_analytics_are_redefin
ing_banking
de Bever, L., Bachher, J. S., Chuyan, R., & Monk, A. (2013, July 3). Towards the Next Generation of
Performance Attribution for Institutional Investment Management. Retrieved February 21, 2014,
from SSRN: http://ssrn.com/abstract=2289562 or http://dx.doi.org/10.2139/ssrn.2289562
Fama, E. (1976). The Behavior of Stock Market Returns.
Gençay, R., Selçuk, F., & Whitcher, B. (2004). Multiscale systematic risk. Retrieved March 23, 2014, from
http://www.sfu.ca/~rgencay/jarticles/jimf-capm.pdf
GetSmarterAboutMoney.ca. (n.d.). Factors that can affect stock prices . Retrieved March 12, 2014, from
GetSmarterAboutMoney.ca: http://www.getsmarteraboutmoney.ca/en/managing-your-
money/investing/stocks/Pages/Factors-that-can-affect-stock-prices.aspx
Giovanis, E. (2009). MATLAB Applications of Trading Rules and GARCH with Wavelets Analysis.
Giovanni , M., & Parrella, F. (n.d.). Data mining for algorithmic asset management:an ensemble learning
approach. Retrieved March 23, 2014, from Academia.edu:

21. Serene Zawaydeh
Big Data, Investment Management and Wavelets March 2014
Page 21 of 24
http://www.academia.edu/1054897/Data_mining_for_algorithmic_asset_management_an_ens
emble_learning_approach
Hardoon, D. R., & Shmueli, G. (2013). Getting Started with Business Analytics. Insightful Decision-
Making. FL: CRC Press.
How long before Securities Analyst become Scientists. (n.d.). Retrieved March 22, 2014, from
http://disruptivetechnologyinvestments.com/big-data/how-long-before-securities-analysts-
become-scientists/
Kiritsis, D., Emmanouilidis, C., Koronios, A., & Mathew, J. (2010). Engineering Asset Management.
Retrieved 3 1, 2014, from
http://www.springer.com/engineering/production+engineering/book/978-0-85729-320-6
Kristoufek, L. (2013, October ). Fractal Markets Hypothesis and the Global Financial Crisis: Wavelet
Power Evidence. Retrieved January 29, 2014, from nature.com:
http://www.nature.com/srep/2013/131004/srep02857/full/srep02857.html
Lahmiri, S. (2012). Wavelet Transform, Neural Networks, and the prediction of S&P Price Index: A
comparative study of back propagation numerical algorithms. Retrieved March 23, 2014, from
saycocorporativo.com:
http://www.saycocorporativo.com/saycoUK/BIJ/journal/Vol5No2/Article_4.pdf
Leonel, M. A., & Yoneyama, T. (n.d.). Forecasting short term abrupt changes in the stock market with
wavelet decomposition and graph theory. Retrieved February 2, 2014, from Forecasters.com:
http://www.forecasters.org/proceedings12/CAETANOMARCOISF2012.pdf
Li, S.-T., & Kuo, S.-C. (2008, February). Knowledge discovery in financial investment for forecasting and
trading strategy through wavelet-based SOM networks. Retrieved March 23, 2014, from Science
Direct: http://www.sciencedirect.com/science/article/pii/S0957417406003551
Los, C., & Yamalova, R. (2004). Multi-Fractal Spectral Analysis of the 1987 Stock Market Crash. Retrieved
from http://ssrn.com/abstract=588823
Mak, M. K., Ho, G. T., & Ting, S. (2011, July 23). A Financial Data Mining Model for Extracting Customer
Behaviour. Retrieved March 23, 2014, from http://cdn.intechopen.com/pdfs-wm/17982.pdf
Manyika, J., & al, e. (2011, May). Big data: The next frontier for innovation, competition, and
productivity. Retrieved March 22, 2014, from McKinsey & Company:
http://www.mckinsey.com/insights/business_technology/big_data_the_next_frontier_for_inno
vation

22. Serene Zawaydeh
Big Data, Investment Management and Wavelets March 2014
Page 22 of 24
Marathe, A., & Shawky, H. (n.d.). Categorizing Mutual Funds using Clusters. Retrieved March 23, 2014,
from citeseerx.ist.psu.edu:
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.202.9593&rep=rep1&type=pdf
McGrath, J. (2014, January 13). Why big data is big business for asset managers. Retrieved March 1,
2014, from http://www.efinancialnews.com/story/2014-01-13/why-big-data-is-big-business-for-
asset-managers?ea9c8a2de0ee111045601ab04d673622
Merwin, J., Higgins, J., Hardy, C., & Marks, K. (2013). Patent No. 13/840,191. US.
Miller, N., Wong, P. C., Brewster, M., & Foote, H. (n.d.). TOPIC ISLANDS – A Wavelet-Based Text
Visualization System. Retrieved March 23, 2014, from http://www-hagen.informatik.uni-
kl.de/vis98/archive/tp/papers/vis98paper.pdf
Millman, R. (2013, April 26). Google big data sets forecast stock market movements, researchers claim.
Retrieved from IT Pro. IT Analysis. Business Insight: http://www.itpro.co.uk/asset-
management/19710/google-big-data-sets-forecast-stock-market-movements-researchers-claim
O'neil, J. (2011, October 6). Patent No. 2011123378 A1. WO.
Peddar, A. (2012, April 19). Turning Big Data into a Dashboard for Investment Managers. Retrieved
February 21, 2014, from http://www.wallstreetandtech.com/data-management/turning-big-
data-into-a-dashboard-for-in/232900574?pgno=2
Peddar, A. (2012, April 19). Turning Big Data into a Dashboard for Investment Managers. Retrieved from
http://www.wallstreetandtech.com/data-management/turning-big-data-into-a-dashboard-for-
in/232900574?pgno=2
Prabakaran, S., Verma, S., & Sahu, R. (2006). A Clustering and Selection Method using Wavelet Power
Spectrum. Retrieved March 23, 2014, from
http://www.iaeng.org/IJCS/issues_v32/issue_4/IJCS_32_4_6.pdf
Qiao, Y.-L., Lu, Z.-M., & Sun, S.-H. (2006). Fast K Nearest Neighbors Search Algorithm Based on Wavelet
Transform. Retrieved March 09, 2014, from
http://www.google.ca/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CC4QFjAA&url=htt
p%3A%2F%2Fwww.paper.edu.cn%2Fselfs%2Fdownpaper%2Fqiaoyulong315734-self-200809-
8&ei=fxkdU6b-
AojmrQGYmoGIBA&usg=AFQjCNGTexkvIB7_zDlQYX2btcS3Xy7Ftg&sig2=PCz7B2RFUnh5wURreO
8R8Q
R.Sahu, P. (n.d.). Co-integration of Stock Markets using Wavelet Theory and Data Mining. Retrieved
February 2, 2014, from Finance Innovation: http://www.finance-
innovation.org/risk08/files/1588125.pdf

23. Serene Zawaydeh
Big Data, Investment Management and Wavelets March 2014
Page 23 of 24
Ramazan Gençay, F. S. (2001). An Introduction to Wavelets and Other Filtering Methods in Finance and
Economics. Academic Press.
Ramsey, J. (1996). The Contribution of Wavelets to the Analysis of Economic and Financial Data. New
York University, Department of Economics. New York: Royal Society Transcript.
Rao, S. (2013, December 12). Ukraine tests Templeton star Hasenstab's contrarian style. Retrieved
March 1, 2014, from http://www.reuters.com/article/2013/12/12/ukraine-templeton-
hasenstab-idUSL6N0JR2JB20131212
Ray, S., & Mallick, B. (2004, April). Functional clustering by Bayesian wavelet methods. Retrieved March
23, 2014, from people.ee.duke.edu:
http://people.ee.duke.edu/~lcarin/BayesianWaveletsDP.pdf
Rodier, M. (2012). Capital Markets Outlook 2012: Data Visualization. Retrieved March 23, 2014, from
Wallstreet and Technology: http://www.wallstreetandtech.com/2012-outlook/data-
visualization
Shahriar Yousefi, I. W. (2004). Wavelet-based prediction of oil prices. Elsevier, 265-275.
State Street. (2013). Leader or Laggard? How data drives competitive advantage in the Investment
Community. Retrieved March 16, 2014, from
http://www.statestreetglobalexchange.com/downloads/DataFullReport.pdf
StatPro. (2012, August 14). StatPro Revolution. Retrieved March 22, 2014, from
https://www.youtube.com/watch?v=aBUo7oZ7Nzo
Stephan Schlüte, C. D. (2010). Using Wavelets for Time Series Forecasting –Does it Pay Off? Nuremberg:
Department of Statistics and Econometrics, University of Erlangen.
Sun, E., & Meinl, T. (2012, March 16). A new wavelet-based denoising algorithm for high-frequency
financial data mining. Retrieved from
http://www.sciencedirect.com/science/article/pii/S0377221711009027
Tan, C. (2009, May 31). Financial Time Series Forecasting Using Improved Wavelet Neural Network.
Retrieved from http://users-birc.au.dk/cstorm/students/Chong_Jul2009.pdf
TCS. (n.d.). The Emerging Big Returns on Big Data. Retrieved March 16, 2014, from
http://www.tcs.com/SiteCollectionDocuments/Trends_Study/TCS-Big-Data-Global-Trend-Study-
2013.pdf
The 3vs That Define Big Data. (n.d.). Retrieved from Data Science Central:
http://www.datasciencecentral.com/forum/topics/the-3vs-that-define-big-data

24. Serene Zawaydeh
Big Data, Investment Management and Wavelets March 2014
Page 24 of 24
Tong, R. (n.d.). Wavelet Analysis for Financial Market Data. Retrieved February 2, 2014, from NAG:
http://www.nag.com/IndustryArticles/Wavelet_Analysis_for_Financial_Market_Data.asp
Wang , W., Lu, D., Zhou, X., Zhang, B., & Mu, J. (2013). Statistical wavelet-based anomaly detection in big
data with compressive sensing. Retrieved 2 10, 2014, from
http://jwcn.eurasipjournals.com/content/pdf/1687-1499-2013-269.pdf
Wang, X. (2013). Patent No. 8548574 B2. US.
Witten, I., Frank, E., & Hall, M. (2011). Data Mining. Practical Machine Learning Tools and Techniques.
3rd Ed. Elsevier Inc.
Wood, P. (n.d.). How to tackle big data from a security point of view. Retrieved February 4, 2014, from
Computer Weekly: http://www.computerweekly.com/feature/How-to-tackle-big-data-from-a-
security-point-of-view
Xiong, X., Zhang, X.-T., Zhang, W., & Li, C.-Y. (2005). Wavelet-based beta estimation of China stock
market. Machine Learning and Cybernetics. Proceedings of 2005 International Conference on .
Zawaydeh, S. (1997, December 31). Discrete Wavelet Analysis and Applications to ECG and PCG Signals.
Irbid, Jordan: Jordan University of Science and Technology.
Zawaydeh, S. (2008). Etude d’événements sur des operations financières et d'acquisitions et application
aux cas de deux opérateurs de telecom Arabes. Beirut, Lebanon: Ecole Supérieure des Affaires
(ESA).
Zhao, Y. (2011, 08 23). Time Series Analysis and Mining with R. Retrieved March 23, 2014, from R-
Bloggers: http://www.r-bloggers.com/time-series-analysis-and-mining-with-r/
Zhao, Y., Zhang, Y., & Qi, C. (2008). Prediction Model of Stock Market Returns Based on Wavelet Neural
Network.
Zitek, J. (2014, Jaunary 08). How long before Securities Analyst become Scientists. Retrieved March 22,
2014, from http://disruptivetechnologyinvestments.com/big-data/how-long-before-securities-
analysts-become-scientists/