Magnet community identification on social networks

Guan Wang, Yuchen Zhao,
Xiaoxiao Shi, Philip S. Yu
Department of Computer Science
University of Illinois at ChicagoKDD 2012

Magnet Community
 More Examples
 The ones those attract people’s interests more than
their peers
myspace facebook
vs
Magnet school Magnet Conference Magnet Company

Attention Flow among Communities
http://blog.topprospect.com/2011/06/the-biggest-talent-losers-and-winners/

What makes a community magnet?
 Attention : Its in-flow should be
larger than the out-flow
Attention : The in-flow comes from
other communities with high attractiveness
levels
Attention : Its first two
properties should be persistent

How to Identify Magnet Community
in Social Networks
 Rule # 1:
PageRank does work……
 Problem
 The ranking screwed
towards large communities
 Normalizing ranking
scores by community size
does not work either
More Challenges
ble 1 lists the results after normalization. The top
become tiny start-ups with about 100 employees
are the companies recognized as “ideal employe
survey result from Universumglobal 2
. It aligns b
common sense about IT industry. As we have se
dom walk schemes cannot accurately measure th
communities. Although the survey result can c
mance, it takes a lot of efforts and manual work
infeasible for large-scale identification tasks.
Rank PageRank Normalized PR
1 Hewlett Packard Zuora
2 IBM Silver Peak Systems
3 Oracle Kony Solutions
4 Microsoft Palo Alto Networks
5 Cisco Systems Quickoffice
Table 1: Top Ranked IT Compa
Therefore, the magnet community identificatio
lenging than it appears to be. First, there is no s
we could rely on to determine the attractivenes
“Top Ranked” IT Companies

More Challenges
: Information from
heterogeneous sources, such as node, edge…
:
Modeling the heterogeneity coherently with
attractiveness properties, including attention
flow, quality, and persistency
: The overall results must make
sense, although small portion of people prefer non-
magnet communities to magnet ones

Framework
 Preprocessing
 Graph Compression: treat each community as a
single node, and edges among them as meta-edges

Framework
 Preliminary Definitions
 is a vector of
magnetism, or attractiveness values, for each
community.
is the estimated
is the standalone feature vector for every
community
 is the dependency feature vector

Framework
 Attractiveness Computation
?

Framework
 Attractiveness Features
 Standalone Features
 Dependency Features
▪ Where D is the attention transition matrix ( is how
much attention transfers from i to j)
▪ is the normalization matrix
▪ is the probabilistic transitional matrix,
(FE(i,j) is the fraction of attention that j draws
from i) It is the dependency feature
comes inactive in community i. Respectively, joining
erson becomes active in j . Note that what we assume
very person could only be active in one community at
vector A = (ai )k∗1 = D · ebe the attention vector,
k-by-1 unit vector. Thus, ai is the total number of
epart from community i. Let A be the element-wise
or of A, where A = (a− 1
i )k∗1. We have dependency
mmunities as FE = A ◦ DT
, which is the Hadamard
and D.
dency matrix, or edge feature, FE is a probabilistic
atrix 3
. Each column of FE is the distribution of peo-
entions are migrating to other communities.
crete formula of magnet community
king framework
be c
of in
cont
A
of th
i ov
cons
lowe
(i ,j
T
tured
ther. This unique relation is modeled as an attention
matrix D = (di j )k ∗ k , where di j is the actual number
who depart from community i and join j . Departing from
he person becomes inactive in community i. Respectivel
means the person becomes active in j . Note that what w
ere is that every person could only be active in one com
ne time. Let vector A = (ai )k ∗ 1 = D · e be the attentio
where e is a k-by-1 unit vector. Thus, ai is the total n
eople who depart from community i. Let A be the elem
nverted vector of A, where A = (a− 1
i )k ∗ 1. We have de
eatures of communities as FE = A ◦ D T
, which is the H
roduct of A and D .
The dependency matrix, or edge feature, FE is a pro
ansitional matrix 3
. Each column of FE is the distributio
le whose attentions are migrating to other communities.
ion away, so that they become active in somewhere
communities draw people’s attention among each
que relation is modeled as an attention migrating
i j )k ∗k , where di j is the actual number of people
m community i and join j . Departing from i means
mes inactive in community i. Respectively, joining
on becomes active in j . Note that what we assume
y person could only be active in one community at
ctor A = (ai )k ∗1 = D · e be the attention vector,
by-1 unit vector. Thus, ai is the total number of
art from community i. Let A be the element-wise
of A, where A = (a− 1
i )k ∗ 1. We have dependency
munities as FE = A ◦ DT
, which is the Hadamard
d D.
ncy matrix, or edge feature, FE is a probabilistic
ix 3
. Each column of FE is the distribution of peo-
be contr
of in-ﬂo
contribu
Altho
of the ab
i over j
constrain
lower bo

Framework
 Constraints
 mi > mj when:
1. i’s attention flow is higher than j’s
2. i’s standalone feature is better than j’s
 An example could be the employee
transferring case:
 one jumps from companyA to company B,
either because B is promising or B provides
better salary
?

Framework
 Optimization with constraints
and in (0,1) are weighting params
is the lower bound of the constraint
M = αFEM + (1 − α)FV , 0 ≤ α ≤ 1 (4)
where α is a weighting parameter. With that formula, we can
ewrite the objective function as
min ||M ∗
− M ||2
F (5)
= min ||αFE M + (1 − α)FV − M ||2
F (6)
= min ||(αFE − I )M + (1 − α)FV ||2
F (7)
Now let us focus on the constraint for the above objective function.
When we say one community is more magnetic than the other, at
east one of the following two conditions are very likely to happen.
First, this community has better standalone features. Second, it
raws people’s attention out of other similar communities. On the
ontrast, it is unlikely for a community to be more magnetic than
thers if it is inferior on both conditions. Formally, when i is more
magnetic than j , i.e., mi − mj > 0, we want at least one of the
ollowing conditions hold.
• f i > f j
where ni
i
ni
ou t is n
meaning
We org
Here, M
Now w
THEO
following
eople
means
ining
sume
ity at
ector,
er of
-wise
dency
mard
listic
peo-
nity
node
ctive-
other
Figure 3: Contribution imbalance
be contributed significantly than smaller ones with the same size
of in-flow (see Figure 3). Therefore, we call the second condition
contribution imbalance.
Although it is possible for people to move from i to j if only one
of the above conditions is true, it is very unlikely for them to prefer
i over j if none of the two condition is true. Thus, we make our
constraint as follows, where µ is a weighting parameter and ζ is a
lower bound.
(i ,j )
(mi − mj ) ∗ (µ(
dj i
Si
−
di j
Sj
) + (1 − µ)(f i − f j )) ≥ ζ (8)
Therefore, the three properties of magnet communities are cap-
tured into Eq. 7 and Eq. 8 in a subtle way. Eq. 7 states that a com-
munity would have better chance to be a magnet one if it attracts
attentions from other high magnet communities, which implies the
second property. Eq. 8 constraints the magnet computation results
must consistent with the first and third properties in Definition 2.3,
eature, FE is a probabilistic
F E is the distribution of peo-
other communities.
magnet community
s of a community, i.e., a node
estart. A node’s attractive-
of it being visited from other
ability that people’s attention
. Upon combining heteroge-
n GC, we have
)F V , 0 ≤ α ≤ 1 (4)
With that formula, we can
lower bound.
( i ,j )
(mi − mj ) ∗ (µ(
Therefore, the three p
tured into Eq. 7 and Eq.
munity would have bett
attentions from other hig
second property. Eq. 8 c
must consistent with the
which are reflected by th
We rewrite the constra
n
i = 1 u ∈ n i
i n
ontribution imbalance
than smaller ones with the same size
herefore, we call the second condition
people to move from i to j if only one
ue, it is very unlikely for them to prefer
condition is true. Thus, we make our
µ is a weighting parameter and ζ is a
−
di j
Sj
) + (1 − µ)(f i − f j )) ≥ ζ (8)
erties of magnet communities are cap-
ting parameter and ζ is a
− µ)(f i − f j )) ≥ ζ (8)
net communities are cap-
y. Eq. 7 states that a com-
magnet one if it attracts
unities, which implies the
agnet computation results
operties in Definition 2.3,
.
ng like terms as
v i mi ≥ ζ (9)
s in GC and

Framework
 Equivalency to the following canonical
quadratic programming forms:
 Q is positive definitive in this
case, which guarantees the solution only
costs polynomial time
ΦM ≥ ζ (10)
e, M is the vector of { mi } 1∗n and Φ is its coefﬁcient vector.
ow we discuss how to solve the optimization framework.
HEOREM 1. Our optimization framework is equivalent to the
owing canonical quadratic programming form:
min M T
QM − 2uT
M (11)
s.t., H M ≤ ξ (12)
ROOF. The objective function of Eq. 7 can be rewritten as
||(αFE − I )M + (1 − α)FV ||2
F
(M T
(αFE
T
− I )(αFE − I )M + (1− α)M T
(αFE
T
− I )FV
+ (1 − α)FV
T
(αFE − I )M + (1 − α)2
FV
T
FV )

Experiments:
Data Sets
(1 − α)(αFE
T
− I )FV ,
ur optimization framework
olution of our optimization,
whether the global minimal
optimization.
ramework is positive deﬁ-
is the eigenvalue of FE .
atrix, |FE | = 0. We have
= αFE X − X = (αλ −
E − I ) is αλ − 1. Thus,
αFE − I )T
(αFE − I ), Q
(a) Facebook employee
ﬂow
(
p
Figure 4: Employee Migra
 Standalone features (reuters.com,
linkedin.com)
Size
Location
Industry growth
Age
P/E ratio
 Dependency features (linkedin.com)

Experiments:
Compared Methods
Personalized PageRank
The restart probability depends on standalone features
Ideal Employer Ranking by Universumglobal
Most Admired Company by Fortune

Rank PageRank MIM Ideal Employer Admired Company
1 IBM Google Google Apple
2 Hewlett Packard Amazon.com Microsoft Google
3 Oracle Apple Apple Amazon.com
4 Microsoft Microsoft Facebook IBM
5 Cisco Systems Facebook IBM Qualcomm
6 Google Salesforce.com Electronics Arts Intel
7 Tata Consult. Services Cisco Systems Amazon Texas Instruments
8 Cognizant Tech. Solu. Juniper Networks Cisco Systems Cisco Systems
9 Dell Yahoo! Intel Adobe Systems
10 EMC Linkedin Sony Oracle
Table 3: Top 10 IT Companies
1 J.P. Morgan Chase J.P. Morgan Chase Goldman Sachs US Bank
k PageRank MIM Ideal Employer Admired Company
IBM Google Google Apple
Hewlett Packard Amazon.com Microsoft Google
Oracle Apple Apple Amazon.com
Microsoft Microsoft Facebook IBM
Cisco Systems Facebook IBM Qualcomm
Google Salesforce.com Electronics Arts Intel
Tata Consult. Services Cisco Systems Amazon Texas Instruments
Cognizant Tech. Solu. Juniper Networks Cisco Systems Cisco Systems
Dell Yahoo! Intel Adobe Systems
EMC Linkedin Sony Oracle

2 Citigroup Goldman Sachs J.P. Morgan Chase Goldman Sachs
3 HSBC Morgan Stanley Boston Consult. Grp. J.P. Morgan Chase
4 PWC Citigroup Deloitte Merrill Lynch
5 Merrill Lynch Merrill Lynch Merrill Lynch Northern Trust Corp.
6 Ernst & Young CB Richard Ellis Ernst & Young Credit Suisse
7 Deutsche Bank Wells Fargo Morgan Stanley CB Richard Eills
8 Credit Suisse PWC PWC HSBC
9 Barclays Capital Jones Lang LaSalle American Express Barclays
10 Goldman Sachs Blackrock Bain & Company Jones Lang LaSalle
Table 4: Top 10 Finance Companies
chs are relatively unscathed by the recent company data in IT industry. As it shows
2 Citigroup Goldman Sachs J.P. Morgan Chase Goldman Sachs
3 HSBC Morgan Stanley Boston Consult. Grp. J.P. Morgan Chase
4 PWC Citigroup Deloitte Merrill Lynch
5 Merrill Lynch Merrill Lynch Merrill Lynch Northern Trust Corp.
6 Ernst & Young CB Richard Ellis Ernst & Young Credit Suisse
7 Deutsche Bank Wells Fargo Morgan Stanley CB Richard Eills
8 Credit Suisse PWC PWC HSBC
9 Barclays Capital Jones Lang LaSalle American Express Barclays
10 Goldman Sachs Blackrock Bain & Company Jones Lang LaSalle
Table 4: Top 10 Finance Companies

Discount Cumulative Gain: (bigger the better)
 Widely used in IR to evaluate search engines
 A measure on how reasonable a ranking is
 Its value is higher when an entity is ranked higher
if it should be ranked higher
Average Weighted Distance: (smaller the better)
 A measure on how far a ranking is from the ground truth
 Its value is smaller when an entity is ranked higher
if it should be ranked higher
 It cares more on the top ranked entities

(a) DCG on IT Ideal Employers (b) DCG on IT Admired Companies (c) Av
Figure 5: Performance on IT Indus
(a) DCG on IT Ideal Employers (b) DCG on IT Admired Companies (c) Av
Figure 5: Performance on IT Indus
(a) DCG on IT Ideal Employers (b) DCG on IT Admired Companies (c) Averag
Figure 5: Performance on IT Industry
(a) DCG on Finance Ideal Employers (b) DCG on Finance Admired Corp. (c) Avg W
(a) DCG on IT Ideal Employers (b) DCG on IT Admired Companies (c) Ave
Figure 5: Performance on IT Industr
(a) DCG on Finance Ideal Employers (b) DCG on Finance Admired Corp. (c) Av
Discount Cumulative Gain: (bigger the better)

es (c) Average Weighted Distance on IT
n IT Industry
(b) DCG on IT Admired Companies (c) Average Weighted Distance on
ers (b) DCG on Finance Admired Corp. (c) Avg Weighted Dist. on Financ
Figure 6: Performance on Finance Industry
nies (c) Average Weighted Distance on IT
on IT Industry
yers (b) DCG on IT Admired Companies (c) Average Weighted Distance on IT
ployers (b) DCG on Finance Admired Corp. (c) Avg Weighted Dist. on Finance
Figure 6: Performance on Finance Industry
• Average weighted distance: (smaller the better)

• Average Precision at cut-off K
e on high ranked entities, in addition to weighted distance,
measure the model’s performance on average precision
n
k = 1 P(k)∆ R(k), where P(k) is the precision at cut-
∆ R(k) is the change of recall from position k − 1 to k.
er normalize wDist using nwD ist = 1
Z
wDist, where
ormalization factor to make it align in the same scale as
Figure 7 we can see that our model performs consistently
ent α values. The fluctuations are in a small range. We
rve that the best performances are achieved at α = 0.6.
are achieved simul
and 0.5 to α and µ
to them for the sam
4. RELATE
Network comm
for a long time. Ho
dynamic communi
lution. To our bes
related to magnet c
Initially, people
the structural prop
connection densiti
tection has been d
addressing dynami
random walks to id
could also rank com
their method also
captured the chang
(a) EP and nwDist on α
the two parameters. Due to space limitation, we only show the
results on IT industry data and using admired company list as com-
parison. (Financial industry data give similar results.) Since we
care more on high ranked entities, in addition to weighted distance,
we also measure the model’s performance on average precision
EP = n
k= 1 P(k)∆ R(k), where P(k) is the precision at cut-
off k and ∆ R(k) is the change of recall from position k − 1 to k.
We further normalize wDist using nwDist = 1
Z
wDist, where
Z is a normalization factor to make it align in the same scale as
EP. In Figure 7 we can see that our model performs consistently
on different α values. The fluctuations are in a small range. We
also observe that the best performances are achieved at α = 0.6.
• Normalized Weighted Distance
o space limitation, we only show the
nd using admired company list as com-
y data give similar results.) Since we
tities, in addition to weighted distance,
l’s performance on average precision
, where P(k) is the precision at cut-
nge of recall from position k − 1 to k.
st using nwDist = 1
Z
wDist, where
to make it align in the same scale as
e that our model performs consistently
fluctuations are in a small range. We
rformances are achieved at α = 0.6.
Figure 7 also shows that µ
performance varies on differe
small range. The highest ave
are achieved simultaneously a
and 0.5 to α and µ to generate
to them for the same reason fo
4. RELATED WOR
Network community analy
for a long time. However, prev
dynamic community detectio
lution. To our best knowledg
related to magnet community
Initially, people paid great a
the structural properties of c

Conclusions
 Magnet community identification is a new
direction
 many application cases on social networks
 The optimization framework is more suitable for the
problem than PageRank variations
 It is also adaptable to different applications due to the
flexibility of defining constraints
Thank you!
 Future works
 other data and applications
▪ Magnet community with time evolving

Magnet community identification on social networks

Recommandé

Recommandé

Contenu connexe

En vedette

En vedette (14)

Similaire à Magnet community identification on social networks

Similaire à Magnet community identification on social networks (20)

Dernier

Dernier (20)

Magnet community identification on social networks

Notes de l'éditeur